summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/ScheduleDAGInstrs.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Fix a bunch more layering of CodeGen headers that are in TargetDavid Blaikie2017-11-171-2/+2
| | | | | | | | All these headers already depend on CodeGen headers so moving them into CodeGen fixes the layering (since CodeGen depends on Target, not the other way around). llvm-svn: 318490
* Reverting r315590; it did not include changes for llvm-tblgen, which is ↵Aaron Ballman2017-10-151-3/+3
| | | | | | | | causing link errors for several people. Error LNK2019 unresolved external symbol "public: void __cdecl `anonymous namespace'::MatchableInfo::dump(void)const " (?dump@MatchableInfo@?A0xf4f1c304@@QEBAXXZ) referenced in function "public: void __cdecl `anonymous namespace'::AsmMatcherEmitter::run(class llvm::raw_ostream &)" (?run@AsmMatcherEmitter@?A0xf4f1c304@@QEAAXAEAVraw_ostream@llvm@@@Z) llvm-tblgen D:\llvm\2017\utils\TableGen\AsmMatcherEmitter.obj 1 llvm-svn: 315854
* [dump] Remove NDEBUG from test to enable dump methods [NFC]Don Hinton2017-10-121-3/+3
| | | | | | | | | | | | | | | Summary: Add LLVM_FORCE_ENABLE_DUMP cmake option, and use it along with LLVM_ENABLE_ASSERTIONS to set LLVM_ENABLE_DUMP. Remove NDEBUG and only use LLVM_ENABLE_DUMP to enable dump methods. Move definition of LLVM_ENABLE_DUMP from config.h to llvm-config.h so it'll be picked up by public headers. Differential Revision: https://reviews.llvm.org/D38406 llvm-svn: 315590
* [ScheduleDAGInstrs] fix behavior of getUnderlyingObjectsForCodeGen when no ↵Hiroshi Inoue2017-10-121-10/+17
| | | | | | | | | | | | | | | identifiable object found This patch fixes the bug introduced in https://reviews.llvm.org/D35907; the bug is reported by http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20171002/491452.html. Before D35907, when GetUnderlyingObjects fails to find an identifiable object, allMMOsOkay lambda in getUnderlyingObjectsForInstr returns false and Objects vector is cleared. This behavior is unintentionally changed by D35907. This patch makes the behavior for such case same as the previous behavior. Since D35907 introduced a wrapper function getUnderlyingObjectsForCodeGen around GetUnderlyingObjects, getUnderlyingObjectsForCodeGen is modified to return a boolean value to ask the caller to clear the Objects vector. Differential Revision: https://reviews.llvm.org/D38735 llvm-svn: 315565
* Fix typo.Eric Christopher2017-08-031-1/+1
| | | | llvm-svn: 309997
* [StackColoring] Update AliasAnalysis information in stack coloring passHiroshi Inoue2017-08-011-61/+2
| | | | | | | | | | | | | | | | | Stack coloring pass need to maintain AliasAnalysis information when merging stack slots of different types. Actually, there is a FIXME comment in StackColoring.cpp // FIXME: In order to enable the use of TBAA when using AA in CodeGen, // we'll also need to update the TBAA nodes in MMOs with values // derived from the merged allocas. But, TBAA has been already enabled in CodeGen without fixing this pass. The incorrect TBAA metadata results in recent failures in bootstrap test on ppc64le (PR33928) by allowing unsafe instruction scheduling. Although we observed the problem on ppc64le, this is a platform neutral issue. This patch makes the stack coloring pass maintains AliasAnalysis information when merging multiple stack slots. llvm-svn: 309651
* [CodeGen] Rename DEBUG_TYPE to match passnamesEvandro Menezes2017-07-111-1/+1
| | | | | | | | | Rename missing DEBUG_TYPE "machine-scheduler" from backend files, which were absent from https://reviews.llvm.org/rL303921. Differential revision: https://reviews.llvm.org/D35231 llvm-svn: 307719
* ScheduleDAGInstrs: Fix fixupKills() adding too many kill flags.Matthias Braun2017-06-271-1/+1
| | | | | | | | | Remove invalid shortcut in fixupKills(): A register needs to be marked live even when we are not adding a kill flag. This is because a partially live register must not get a kill flags, but it still needs to be fully marked live when walking backwards. llvm-svn: 306352
* Sort the remaining #include lines in include/... and lib/....Chandler Carruth2017-06-061-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | I did this a long time ago with a janky python script, but now clang-format has built-in support for this. I fed clang-format every line with a #include and let it re-sort things according to the precise LLVM rules for include ordering baked into clang-format these days. I've reverted a number of files where the results of sorting includes isn't healthy. Either places where we have legacy code relying on particular include ordering (where possible, I'll fix these separately) or where we have particular formatting around #include lines that I didn't want to disturb in this patch. This patch is *entirely* mechanical. If you get merge conflicts or anything, just ignore the changes in this patch and run clang-format over your #include lines in the files. Sorry for any noise here, but it is important to keep these things stable. I was seeing an increasing number of patches with irrelevant re-ordering of #include lines because clang-format was used. This patch at least isolates that churn, makes it easy to skip when resolving conflicts, and gets us to a clean baseline (again). llvm-svn: 304787
* [CodeGen] Fix some Clang-tidy modernize-use-using and Include What You Use ↵Eugene Zelenko2017-06-011-27/+53
| | | | | | warnings; other minor fixes (NFC). llvm-svn: 304495
* ScheduleDAGInstrs: Fix fixupKills()Matthias Braun2017-05-271-157/+49
| | | | | | | | | | | | Rewrite fixupKills() to use the LivePhysRegs class. Simplifies the code and fixes a bug where the CSR registers in return blocks where missed leading to invalid kill flags. Also remove the unnecessary rule that we wouldn't set kill flags on tied operands. No tests as I have an upcoming commit improving MachineVerifier checks to catch these cases in multiple existing lit tests. llvm-svn: 304055
* Refactor alias check from MISched into common helper. NFC.Eli Friedman2017-03-091-61/+1
| | | | | | Differential Revision: https://reviews.llvm.org/D30598 llvm-svn: 297421
* [MISched] Remove unused arguments. NFC.Eli Friedman2017-03-041-4/+2
| | | | llvm-svn: 296934
* Cleanup dump() functions.Matthias Braun2017-01-281-4/+5
| | | | | | | | | | | | | | | | | | We had various variants of defining dump() functions in LLVM. Normalize them (this should just consistently implement the things discussed in http://lists.llvm.org/pipermail/cfe-dev/2014-January/034323.html For reference: - Public headers should just declare the dump() method but not use LLVM_DUMP_METHOD or #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP) - The definition of a dump method should look like this: #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP) LLVM_DUMP_METHOD void MyClass::dump() { // print stuff to dbgs()... } #endif llvm-svn: 293359
* ScheduleDAGInstrs: Do not try to toggle kill flags on debug usesMatthias Braun2017-01-271-0/+3
| | | | | | | | Preparation for upcoming changes. No testcase as none of the public targets bundles early enough and has a post machine scheduler enabled at the same time. The error is also easily catched by asserts. llvm-svn: 293324
* ScheduleDAGInstrs: Cleanup toggleKillFlag(); NFCMatthias Braun2017-01-271-11/+10
| | | | llvm-svn: 293323
* ScheduleDAGInstrs: Cleanup; NFCMatthias Braun2017-01-271-69/+45
| | | | | | Comment, doxygen and a bit of whitespace cleanup. llvm-svn: 293322
* Implement LaneBitmask::any(), use it to replace !none(), NFCIKrzysztof Parzyszek2016-12-161-4/+4
| | | | llvm-svn: 289974
* Extract LaneBitmask into a separate typeKrzysztof Parzyszek2016-12-151-12/+13
| | | | | | | | | | | | Specifically avoid implicit conversions from/to integral types to avoid potential errors when changing the underlying type. For example, a typical initialization of a "full" mask was "LaneMask = ~0u", which would result in a value of 0x00000000FFFFFFFF if the type was extended to uint64_t. Differential Revision: https://reviews.llvm.org/D27454 llvm-svn: 289820
* ScheduleDAGInstrs: Move VRegUses to ScheduleDAGMILive; NFCIMatthias Braun2016-11-111-43/+0
| | | | | | | | | | | | | Push VRegUses/collectVRegUses() down the class hierarchy towards its only user ScheduleDAGMILive. NFCI: The initialization of the map happens at a later point but that should not matter. This is in preparation to allow DAG mutators to merge nodes, which relies on this map getting computed later. llvm-svn: 286654
* ScheduleDAGInstrs: Add condjump deps to addSchedBarrierDeps()Matthias Braun2016-11-111-7/+4
| | | | | | | | | | | | | | | addSchedBarrierDeps() is supposed to add use operands to the ExitSU node. The current implementation adds uses for calls/barrier instruction and the MBB live-outs in all other cases. The use operands of conditional jump instructions were missed. Also added code to macrofusion to set the latencies between nodes to zero to avoid problems with the fusing nodes lingering around in the pending list now. Differential Revision: https://reviews.llvm.org/D25140 llvm-svn: 286544
* ScheduleDAGInstrs: Ignore dependencies of constant physregsMatthias Braun2016-11-101-0/+3
| | | | | | | | | There is no need to track dependencies for constant physregs, as they don't change their value no matter in what order you read/write to them. Differential Revision: https://reviews.llvm.org/D26221 llvm-svn: 286526
* ScheduleDAGInstrs: Slightly simplify code; NFCMatthias Braun2016-11-101-20/+12
| | | | llvm-svn: 286510
* MachineInstrBundle: Pass iterators to getBundle(Start|End); NFCMatthias Braun2016-10-251-2/+2
| | | | | | | | This is a function to go backwards in a block to find the first instruction in a bundle, so iterator is a more natural choice for parameter/return rather than a reference to a MachineInstruction. llvm-svn: 285051
* Fix machine operand traversal in ScheduleDAGInstrs::fixupKillsKrzysztof Parzyszek2016-10-051-2/+7
| | | | llvm-svn: 283315
* ScheduleDAGInstrs: Cleanup, use range based for; NFCMatthias Braun2016-09-301-61/+45
| | | | llvm-svn: 282979
* [CodeGen] Rename MachineInstr::isInvariantLoad to ↵Justin Lebar2016-09-101-2/+3
| | | | | | | | | | | | | | | | | | | | isDereferenceableInvariantLoad. NFC Summary: I want to separate out the notions of invariance and dereferenceability at the MI level, so that they correspond to the equivalent concepts at the IR level. (Currently an MI load is MI-invariant iff it's IR-invariant and IR-dereferenceable.) First step is renaming this function. Reviewers: chandlerc Subscribers: MatzeB, jfb, llvm-commits Differential Revision: https://reviews.llvm.org/D23370 llvm-svn: 281125
* MachineFunction: Return reference for getFrameInfo(); NFCMatthias Braun2016-07-281-5/+5
| | | | | | | getFrameInfo() never returns nullptr so we should use a reference instead of a pointer. llvm-svn: 277017
* Untabify.NAKAMURA Takumi2016-07-251-1/+1
| | | | llvm-svn: 276597
* CodeGen: Use MachineInstr& in ScheduleDAGIntrs, NFCDuncan P. N. Exon Smith2016-07-011-43/+43
| | | | | | | | | | | | | Use MachineInstr& to avoid implicit conversions from MachineBasicBlock::iterator to MachineInstr*. In one case, this could use a range-based for loop, but the other loops iterated in reverse order. One of the reverse-loops checked the MachineInstr* for nullptr, a condition that is provably unreachable. (And even if my proof has a flaw, UBSan would catch the bug.) llvm-svn: 274360
* CodeGen: Use MachineInstr& in TargetInstrInfo, NFCDuncan P. N. Exon Smith2016-06-301-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is mostly a mechanical change to make TargetInstrInfo API take MachineInstr& (instead of MachineInstr* or MachineBasicBlock::iterator) when the argument is expected to be a valid MachineInstr. This is a general API improvement. Although it would be possible to do this one function at a time, that would demand a quadratic amount of churn since many of these functions call each other. Instead I've done everything as a block and just updated what was necessary. This is mostly mechanical fixes: adding and removing `*` and `&` operators. The only non-mechanical change is to split ARMBaseInstrInfo::getOperandLatencyImpl out from ARMBaseInstrInfo::getOperandLatency. Previously, the latter took a `MachineInstr*` which it updated to the instruction bundle leader; now, the latter calls the former either with the same `MachineInstr&` or the bundle leader. As a side effect, this removes a bunch of MachineInstr* to MachineBasicBlock::iterator implicit conversions, a necessary step toward fixing PR26753. Note: I updated WebAssembly, Lanai, and AVR (despite being off-by-default) since it turned out to be easy. I couldn't run tests for AVR since llc doesn't link with it turned on. llvm-svn: 274189
* ScheduleDAGInstrs: Fix memory corruptionMatthias Braun2016-05-251-2/+2
| | | | | | | | We have to modify V2SU before inserting new elements into the CurrentVRegDefs set because that may move V2SU in memory invalidating the reference. llvm-svn: 270644
* ScheduleDAGInstrs: Comment on why subreg defs are not seen as uses; NFCMatthias Braun2016-05-101-0/+4
| | | | | | | | Usually subregister definitions are consider uses of the remaining lanes that did not get defined. Add a comment why the code in ScheduleDAGInstrs does not add use dependencies regardless. llvm-svn: 269107
* Fix PR26655: Bail out if all regs of an inst BUNDLE have the correct kill flagMandeep Singh Grang2016-05-101-27/+11
| | | | | | | | | | | | | | | | | | | | Summary: While setting kill flags on instructions inside a BUNDLE, we bail out as soon as we set kill flag on a register. But we are missing a check when all the registers already have the correct kill flag set. We need to bail out in that case as well. This patch refactors the old code and simply makes use of the addRegisterKilled function in MachineInstr.cpp in order to determine whether to set/remove kill on an instruction. Reviewers: apazos, t.p.northover, pete, MatzeB Subscribers: MatzeB, davide, llvm-commits Differential Revision: http://reviews.llvm.org/D17356 llvm-svn: 269092
* [ScheduleDAG] Make sure to process all def operands before any use operandsKrzysztof Parzyszek2016-05-101-8/+25
| | | | | | | | | | | | | | | | | An example from Hexagon where things went wrong: %R0<def> = L2_loadrigp <ga:@fp04> ; load function address J2_callr %R0<kill>, ..., %R0<imp-def> ; call *R0, return value in R0 ScheduleDAGInstrs::buildSchedGraph would visit all instructions going backwards, and in each instruction it would visit all operands in their order on the operand list. In the case of this call, it visited the use of R0 first, then removed it from the set Uses after it visited the def. This caused the DAG to be missing the data dependence edge on R0 between the load and the call. Differential Revision: http://reviews.llvm.org/D20102 llvm-svn: 269076
* ScheduleDAGInstrs.cpp: Don't peel the iterator when it points the end. This ↵NAKAMURA Takumi2016-05-021-1/+1
| | | | | | will fix the crash in r268143. llvm-svn: 268257
* [NFC] Header cleanupMehdi Amini2016-04-181-1/+0
| | | | | | | | | | | | | | Removed some unused headers, replaced some headers with forward class declarations. Found using simple scripts like this one: clear && ack --cpp -l '#include "llvm/ADT/IndexedMap.h"' | xargs grep -L 'IndexedMap[<]' | xargs grep -n --color=auto 'IndexedMap' Patch by Eugene Kosov <claprix@yandex.ru> Differential Revision: http://reviews.llvm.org/D19219 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 266595
* Do not modify a cl::opt programmatically, global mutable state is evil.Mehdi Amini2016-04-161-9/+13
| | | | | | | Found by TSAN on ThinLTO. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 266514
* [ScheduleDAGInstrs] Re-factor for based on review feedback. NFC.Geoff Berry2016-04-141-52/+45
| | | | | | | | | | | | | | Summary: Re-factor some code to improve clarity and style based on review comments from http://reviews.llvm.org/D18093. Reviewers: MatzeB, mcrosier Subscribers: MatzeB, mcrosier, llvm-commits Differential Revision: http://reviews.llvm.org/D19128 llvm-svn: 266372
* [ScheduleDAGInstrs] Handle instructions with multiple MMOsGeoff Berry2016-04-121-30/+41
| | | | | | | | | | | | | | | | | | | | | Summary: In getUnderlyingObjectsForInstr(): Don't give up on instructions with multiple MMOs, instead look through all the MMOs and if they all meet the conservative criteria previously used for single MMO instructions, then return all of the underlying objects derived from the MMOs. The change to ScheduleDAGInstrs::buildSchedGraph() is needed to avoid the case where multiple underlying objects are present and are related in such a way that successive iterations of the loop end up adding a dependency from an instruction to itself. Reviewers: atrick, hfinkel Subscribers: MatzeB, mcrosier, llvm-commits Differential Revision: http://reviews.llvm.org/D18093 llvm-svn: 266084
* Fix PR26940 where compiles times regressed massively.Chandler Carruth2016-03-311-33/+30
| | | | | | | | | | | | | | | Patch by Jonas Paulsson. Original description: Bugfix in buildSchedGraph() to make -dag-maps-huge-region work properly I found that the reduction of the maps did in fact never happen in this test case. This was because *all* the stores / loads were made with addresses from arguments and they thus became "unknown" stores / loads. Fixed by removing continue statements and making sure that the test for reduction always takes place. Differential Revision: http://reviews.llvm.org/D18673 llvm-svn: 265063
* WIP: CodeGen: Use MachineInstr& in MachineInstrBundle.h, NFCDuncan P. N. Exon Smith2016-02-271-2/+2
| | | | | | | | Update APIs in MachineInstrBundle.h to take and return MachineInstr& instead of MachineInstr* when the instruction cannot be null. Besides being a nice cleanup, this is tacking toward a fix for PR26753. llvm-svn: 262141
* CodeGen: Take MachineInstr& in SlotIndexes and LiveIntervals, NFCDuncan P. N. Exon Smith2016-02-271-1/+1
| | | | | | | | | | | | | | Take MachineInstr by reference instead of by pointer in SlotIndexes and the SlotIndex wrappers in LiveIntervals. The MachineInstrs here are never null, so this cleans up the API a bit. It also incidentally removes a few implicit conversions from MachineInstrBundleIterator to MachineInstr* (see PR26753). At a couple of call sites it was convenient to convert to a range-based for loop over MachineBasicBlock::instr_begin/instr_end, so I added MachineBasicBlock::instrs. llvm-svn: 262115
* Revert "CodeGen: MachineInstr::getIterator() => getInstrIterator(), NFC"Duncan P. N. Exon Smith2016-02-221-4/+4
| | | | | | | | | | This reverts commit r261504, since it's not obvious the new name is better: http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20160222/334298.html I'll recommit if we get consensus that it's the right direction. llvm-svn: 261567
* CodeGen: MachineInstr::getIterator() => getInstrIterator(), NFCDuncan P. N. Exon Smith2016-02-211-4/+4
| | | | | | | | | | | | | | | | | | | | | | | Delete MachineInstr::getIterator(), since the term "iterator" is overloaded when talking about MachineInstr. - Downcast to ilist_node in iplist::getNextNode() and getPrevNode() so that ilist_node::getIterator() is still available. - Add it back as MachineInstr::getInstrIterator(). This matches the naming in MachineBasicBlock. - Add MachineInstr::getBundleIterator(). This is explicitly called "bundle" (not matching MachineBasicBlock) to disintinguish it clearly from ilist_node::getIterator(). - Update all calls. Some of these I switched to `auto` to remove boiler-plate, since the new name is clear about the type. There was one call I updated that looked fishy, but it wasn't clear what the right answer was. This was in X86FrameLowering::inlineStackProbe(), added in r252578 in lib/Target/X86/X86FrameLowering.cpp. I opted to leave the behaviour unchanged, but I'll reply to the original commit on the list in a moment. llvm-svn: 261504
* [ScheduleDAGInstrs] isUnsafeMemoryObject() removedJonas Paulsson2016-02-151-36/+5
| | | | | | | | | | | This function was basically useless, since volatile memacesses or MIs with unmodelled sideffects become global memory objects, and the other little checks are also done elsewhere. Reviewed by Andy Trick http://reviews.llvm.org/D16881 llvm-svn: 260899
* [ScheduleDagInstrs] Improved commentsJonas Paulsson2016-02-041-9/+9
| | | | llvm-svn: 259783
* [ScheduleDAGInstrs::buildSchedGraph()] Handling of memory dependecies rewritten.Jonas Paulsson2016-02-031-347/+362
| | | | | | | | | | | | | | | | | | | Recommited, after some fixing with test cases. Updated test cases: test/CodeGen/AArch64/arm64-misched-memdep-bug.ll test/CodeGen/AArch64/tailcall_misched_graph.ll Temporarily disabled test cases: test/CodeGen/AMDGPU/split-vector-memoperand-offsets.ll test/CodeGen/PowerPC/ppc64-fastcc.ll (partially updated) test/CodeGen/PowerPC/vsx-fma-m.ll test/CodeGen/PowerPC/vsx-fma-sp.ll http://reviews.llvm.org/D8705 Reviewers: Hal Finkel, Andy Trick. llvm-svn: 259673
* Temporarily revert "[ScheduleDAGInstrs::buildSchedGraph()] Handling of ↵Jonas Paulsson2016-01-291-376/+347
| | | | | | | | memory dependecies rewritten." Some buildbot failures needs to be debugged. llvm-svn: 259213
* [ScheduleDAGInstrs::buildSchedGraph()] Handling of memory dependecies rewritten.Jonas Paulsson2016-01-291-347/+376
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The buildSchedGraph() was in need of reworking as the AA features had been added on top of earlier code. It was very difficult to understand, and buggy. There had been found cases where scheduling dependencies had actually been missed (see r228686). AliasChain, RejectMemNodes, adjustChainDeps() and iterateChainSucc() have been removed. There are instead now just the four maps from Value to SUs, which have been renamed to Stores, Loads, NonAliasStores and NonAliasLoads. An unknown store used to become the AliasChain, but now becomes a store mapped to 'unknownValue' (in Stores). What used to be PendingLoads is instead the list of SUs mapped to 'unknownValue' in Loads. RejectMemNodes and adjustChainDeps() used to be a safety-net for everything. The SU maps were sometimes cleared and SUs were put in RejectMemNodes, where adjustChainDeps() would look. Instead of this, a more straight forward approach is used in maintaining the SU maps without clearing them and simply letting them grow over time. Instead of the cutt-off in adjustChainDeps() search, a reduction of maps will be done if needed (see below). Each SUnit either becomes the BarrierChain, or is put into one of the maps. For each SUnit encountered, all the information about previous ones are still available until a new BarrierChain is set, at which point the maps are cleared. For huge regions, the algorithm becomes slow, therefore the maps will get reduced at a threshold (current default is 1000 nodes), by a fraction (default 1/2). These values can be tuned by use of CL options in case some test case shows that they need to be changed (-dag-maps-huge-region and -dag-maps-reduction-size). There has not been any considerable change observed in output quality or compile time. There may now be more DAG edges inserted than before (i.e. if A->B->C, then A->C is not needed). However, in a comparison run there were fewer total calls to AA, and a somewhat improved compile time, which means this seems to be not a problem. http://reviews.llvm.org/D8705 Reviewers: Hal Finkel, Andy Trick. llvm-svn: 259201
OpenPOWER on IntegriCloud