summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/ScheduleDAGInstrs.cpp
Commit message (Collapse)AuthorAgeFilesLines
...
* [ScheduleDAGInstrs] Simplify logic to improve readability. NFC.Chad Rosier2016-01-261-2/+1
| | | | | | The call to isInvariantLoad() already returns false for non-load instructions. llvm-svn: 258841
* MachineScheduler: Allow independent scheduling of sub register defsMatthias Braun2016-01-201-0/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Note that this is disabled by default and still requires a patch to handleMove() which is not upstreamed yet. If the TrackLaneMasks policy/strategy is enabled the MachineScheduler will build a schedule graph where definitions of independent subregisters are no longer serialised. Implementation comments: - Without lane mask tracking a sub register def also counts as a use (except for the first one with the read-undef flag set), with lane mask tracking enabled this is no longer the case. - Pressure Diffs where previously maintained per definition of a vreg with the help of the SSA information contained in the LiveIntervals. With lanemask tracking enabled we cannot do this anymore and instead change the pressure diffs for all uses of the vreg as it becomes live/dead. For this changed style to work correctly we ignore uses of instructions that define the same register again: They won't affect register pressure. - With lanemask tracking we remove all read-undef flags from sub register defs when building the graph and re-add them later when all vreg lanes have become dead. Differential Revision: http://reviews.llvm.org/D14969 llvm-svn: 258259
* RegisterPressure: Make liveness tracking subregister awareMatthias Braun2016-01-201-1/+1
| | | | | | Differential Revision: http://reviews.llvm.org/D14968 llvm-svn: 258258
* RegisterPressure: Expose RegisterOperands APIMatthias Braun2016-01-121-4/+9
| | | | | | | | | | | | | | | | | | | | | | Previously the RegisterOperands have only been used internally in RegisterPressure.cpp. However this datastructure can be useful for other tasks as well and allows refactoring of PDiff initialisation out of RPTracker::recede(). This patch: - Exposes RegisterOperands as public API - Splits RPTracker::recede() into a part that skips DebugValues and maintains the region borders, and the core that changes register pressure when given a set of RegisterOperands. - This allows to move the PDiff initialisation out recede() into a method of the PressureDiffs class. - The upcoming subregister scheduling code will also use RegisterOperands to avoid pushing more unrelated functionality into recede()/advance(). Differential Revision: http://reviews.llvm.org/D15473 llvm-svn: 257535
* ScheduleDAGInstrs: Bug fix for missed memory dependency.Geoff Berry2016-01-061-3/+12
| | | | | | | | | | | | | | | | | | | | Summary: In buildSchedGraph(), when adding memory dependencies for loads, move the call to adjustChainDeps() after the call to addChainDependency(AliasChain) to handle the case where addChainDependency(AliasChain) ends up not adding a dependency and instead putting the SU on the RejectMemNodes list. The call to adjustChainDeps() must be done after the call to addChainDependency() in order to process the SU added to the RejectMemNodes list to create memory dependencies for it. Reviewers: hfinkel, atrick, jonpa, resistor Subscribers: mcrosier, llvm-commits Differential Revision: http://reviews.llvm.org/D15927 llvm-svn: 256950
* ScheduleDAGInstrs: Move LiveIntervals field to ScheduleDAGMIMatthias Braun2015-12-041-2/+1
| | | | | | | Now that ScheduleDAGInstrs doesn't need it anymore we can move the field down the class hierarcy to ScheduleDAGMI. llvm-svn: 254759
* ScheduleDAGInstrs: Rework schedule graph builder.Matthias Braun2015-12-041-66/+161
| | | | | | | | | | | | | | | | | | Re-comitting with a change that avoids undefined uses getting put into the VRegUses list. The new algorithm remembers the uses encountered while walking backwards until a matching def is found. Contrary to the previous version this: - Works without LiveIntervals being available - Allows to increase the precision to subregisters/lanemasks (not used for now) The changes in the AMDGPU tests are necessary because the R600 scheduler is not stable with respect to the order of nodes in the ready queues. Differential Revision: http://reviews.llvm.org/D9068 llvm-svn: 254683
* Revert "ScheduleDAGInstrs: Rework schedule graph builder."Matthias Braun2015-12-031-159/+66
| | | | | | | | | | This works mostly fine but breaks some stage 1 builders when compiling compiler-rt on i386. Revert for further investigation as I can't see an obvious cause/fix. This reverts commit r254577. llvm-svn: 254586
* ScheduleDAGInstrs: Rework schedule graph builder.Matthias Braun2015-12-031-66/+159
| | | | | | | | | | | | | | | The new algorithm remembers the uses encountered while walking backwards until a matching def is found. Contrary to the previous version this: - Works without LiveIntervals being available - Allows to increase the precision to subregisters/lanemasks (not used for now) The changes in the AMDGPU tests are necessary because the R600 scheduler is not stable with respect to the order of nodes in the ready queues. Differential Revision: http://reviews.llvm.org/D9068 llvm-svn: 254577
* ScheduleDAGInstrs: Remove IsPostRA flag; NFCMatthias Braun2015-11-031-14/+7
| | | | | | | | | | | | | | | | | ScheduleDAGInstrs doesn't behave differently before or after register allocation. It was only used in a method of MachineSchedulerBase which behaved differently in MachineScheduler/PostMachineScheduler. Change this to let MachineScheduler/PostMachineScheduler just pass in a parameter to that function. The order of the LiveIntervals* and bool RemoveKillFlags paramters have been switched to make out-of-tree code fail instead of unintentionally passing a value intended for the IsPostRA flag to the (previously following and default initialized) RemoveKillFlags. Differential Revision: http://reviews.llvm.org/D14245 llvm-svn: 251883
* Revert "ScheduleDAGInstrs: Remove IsPostRA flag"Matthias Braun2015-10-291-5/+12
| | | | | | | | It broke 3 arm testcases. This reverts commit r251608. llvm-svn: 251615
* ScheduleDAGInstrs: Remove IsPostRA flagMatthias Braun2015-10-291-12/+5
| | | | | | | | This was a layering violation in ScheduleDAGInstrs (and MachineSchedulerBase) they both shouldn't know directly whether they are used by the PostMachineScheduler or the MachineScheduler. llvm-svn: 251608
* Refactor: Simplify boolean conditional return statements in lib/CodeGen.Rafael Espindola2015-10-241-5/+3
| | | | | | Patch by Richard. llvm-svn: 251213
* CodeGen: Avoid more ilist iterator implicit conversions, NFCDuncan P. N. Exon Smith2015-10-091-2/+2
| | | | llvm-svn: 249903
* Save LaneMask with livein registersMatthias Braun2015-09-091-5/+5
| | | | | | | | | | | | | | | | | With subregister liveness enabled we can detect the case where only parts of a register are live in, this is expressed as a 32bit lanemask. The current code only keeps registers in the live-in list and therefore enumerated all subregisters affected by the lanemask. This turned out to be too conservative as the subregister may also cover additional parts of the lanemask which are not live. Expressing a given lanemask by enumerating a minimum set of subregisters is computationally expensive so the best solution is to simply change the live-in list to store the lanemasks as well. This will reduce memory usage for targets using subregister liveness and slightly increase it for other targets Differential Revision: http://reviews.llvm.org/D12442 llvm-svn: 247171
* MachineBasicBlock: Add liveins() method returning an iterator_rangeMatthias Braun2015-08-241-9/+5
| | | | llvm-svn: 245895
* Fix some comment typos.Benjamin Kramer2015-08-081-4/+4
| | | | llvm-svn: 244402
* Move most user of TargetMachine::getDataLayout to the Module oneMehdi Amini2015-07-161-17/+17
| | | | | | | | | | | | | | | | | | | | | Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. This patch is quite boring overall, except for some uglyness in ASMPrinter which has a getDataLayout function but has some clients that use it without a Module (llmv-dsymutil, llvm-dwarfdump), so some methods are taking a DataLayout as parameter. Reviewers: echristo Subscribers: yaron.keren, rafael, llvm-commits, jholewinski Differential Revision: http://reviews.llvm.org/D11090 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 242386
* Revert r240137 (Fixed/added namespace ending comments using clang-tidy. NFC)Alexander Kornienko2015-06-231-1/+1
| | | | | | Apparently, the style needs to be agreed upon first. llvm-svn: 240390
* [PM/AA] Hoist the AliasResult enum out of the AliasAnalysis class.Chandler Carruth2015-06-221-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | This will allow classes to implement the AA interface without deriving from the class or referencing an internal enum of some other class as their return types. Also, to a pretty fundamental extent, concepts such as 'NoAlias', 'MayAlias', and 'MustAlias' are first class concepts in LLVM and we aren't saving anything by scoping them heavily. My mild preference would have been to use a scoped enum, but that feature is essentially completely broken AFAICT. I'm extremely disappointed. For example, we cannot through any reasonable[1] means construct an enum class (or analog) which has scoped names but converts to a boolean in order to test for the possibility of aliasing. [1]: Richard Smith came up with a "solution", but it requires class templates, and lots of boilerplate setting up the enumeration multiple times. Something like Boost.PP could potentially bundle this up, but even that would be quite painful and it doesn't seem realistically worth it. The enum class solution would probably work without the need for a bool conversion. Differential Revision: http://reviews.llvm.org/D10495 llvm-svn: 240255
* Fixed/added namespace ending comments using clang-tidy. NFCAlexander Kornienko2015-06-191-1/+1
| | | | | | | | | | | | | The patch is generated using this command: tools/clang/tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \ -checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \ llvm/lib/ Thanks to Eugene Kosov for the original patch! llvm-svn: 240137
* [PM/AA] Remove the Location typedef from the AliasAnalysis class nowChandler Carruth2015-06-171-5/+5
| | | | | | | | | | | | that it is its own entity in the form of MemoryLocation, and update all the callers. This is an entirely mechanical change. References to "Location" within AA subclases become "MemoryLocation", and elsewhere "AliasAnalysis::Location" becomes "MemoryLocation". Hope that helps out-of-tree folks update. llvm-svn: 239885
* CodeGen: Use mop_iterator instead of MIOperands/ConstMIOperandsMatthias Braun2015-05-291-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | MIOperands/ConstMIOperands are classes iterating over the MachineOperand of a MachineInstr, however MachineInstr::mop_iterator does the same thing. I assume these two iterators exist to have a uniform interface to iterate over the operands of a machine instruction bundle and a single machine instruction. However in practice I find it more confusing to have 2 different iterator classes, so this patch transforms (nearly all) the code to use mop_iterators. The only exception being MIOperands::anlayzePhysReg() and MIOperands::analyzeVirtReg() still needing an equivalent, I leave that as an exercise for the next patch. Differential Revision: http://reviews.llvm.org/D9932 This version is slightly modified from the proposed revision in that it introduces MachineInstr::getOperandNo to avoid the extra counting variable in the few loops that previously used MIOperands::getOperandNo. llvm-svn: 238539
* use range-based for-loopSanjay Patel2015-05-211-3/+2
| | | | llvm-svn: 237914
* use range-based for-loop; NFCISanjay Patel2015-05-211-4/+1
| | | | llvm-svn: 237908
* Remove MCInstrItineraries includes in parts that don't use them anymoreMatthias Braun2015-05-141-1/+0
| | | | llvm-svn: 237375
* CodeGen: ignore DEBUG_VALUE nodes in KILL taggingSaleem Abdulrasool2015-05-121-0/+6
| | | | | | | DEBUG_VALUE nodes do not take part in code generation. Ignore them when performing KILL updates. Addresses PR23486. llvm-svn: 237211
* ScheduleDAGInstrs: In functions with tail calls PseudoSourceValues are not ↵Arnold Schwaighofer2015-05-081-0/+8
| | | | | | | | | | | | | | | | | | | | non-aliasing distinct objects The code that builds the dependence graph assumes that two PseudoSourceValues don't alias. In a tail calling function two FixedStackObjects might refer to the same location. Worse 'immutable' fixed stack objects like function arguments are not immutable and will be clobbered. Change this so that a load from a FixedStackObject is not invariant in a tail calling function and don't return a PseudoSourceValue for an instruction in tail calling functions when building the dependence graph so that we handle function arguments conservatively. Fix for PR23459. rdar://20740035 llvm-svn: 236916
* ScheduleDAGInstrs should toggle kill flags on bundled instrs.Pete Cooper2015-05-041-1/+46
| | | | | | | | | | | | | | | | | | | | | | | | ScheduleDAGInstrs wasn't setting or clearing the kill flags on instructions inside bundles. This led to code such as this %R3<def> = t2ANDrr %R0 BUNDLE %ITSTATE<imp-def,dead>, %R0<imp-use,kill> t2IT 1, 24, %ITSTATE<imp-def> R6<def,tied6> = t2ORRrr %R0<kill>, ... being transformed to BUNDLE %ITSTATE<imp-def,dead>, %R0<imp-use> t2IT 1, 24, %ITSTATE<imp-def> R6<def,tied6> = t2ORRrr %R0<kill>, ... %R3<def> = t2ANDrr %R0<kill> where the kill flag was removed from the BUNDLE instruction, but not the t2ORRrr inside it. The verifier then thought that R0 was undefined when read by the AND. This change make the toggleKillFlags method also check for bundles and toggle flags on bundled instructions. Setting the kill flag is special cased as we only want to set the kill flag on the last instruction in the bundle. llvm-svn: 236428
* DataLayout is mandatory, update the API to reflect it with references.Mehdi Amini2015-03-101-47/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Now that the DataLayout is a mandatory part of the module, let's start cleaning the codebase. This patch is a first attempt at doing that. This patch is not exactly NFC as for instance some places were passing a nullptr instead of the DataLayout, possibly just because there was a default value on the DataLayout argument to many functions in the API. Even though it is not purely NFC, there is no change in the validation. I turned as many pointer to DataLayout to references, this helped figuring out all the places where a nullptr could come up. I had initially a local version of this patch broken into over 30 independant, commits but some later commit were cleaning the API and touching part of the code modified in the previous commits, so it seemed cleaner without the intermediate state. Test Plan: Reviewers: echristo Subscribers: llvm-commits From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 231740
* Rewrite MachineOperand::print and MachineInstr::print to avoidEric Christopher2015-02-271-1/+1
| | | | | | | | | | | | uses of TM->getSubtargetImpl and propagate to all calls. This could be a debugging regression in places where we had a TargetMachine and/or MachineFunction but don't have it as part of the MachineInstr. Fixing this would require passing a MachineFunction/Function down through the print operator, but none of the existing uses in tree seem to do this. llvm-svn: 230710
* Bugfix for missed dependency from store to load in buildSchedGraph().Jonas Paulsson2015-02-101-4/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Background: When handling underlying objects for a store, the vector of previous mem uses, mapped to the same Value, is afterwards cleared (regardless of ThisMayAlias). This means that during handling of the next store using the same Value, adjustChainDeps() must be called, otherwise a dependency might be missed. For example, three spill/reload (NonAliasing) memory accesses using the same Value 'a', with different offsets: SU(2): store @a SU(1): store @a, Offset:1 SU(0): load @a In this case we have: * SU(1) does not need a dep against SU(0). Therefore,SU(0) ends up in RejectMemNodes and is removed from the mem-uses list (AliasMemUses or NonAliasMemUses), as this list is cleared. * SU(2) needs a dep against SU(0). Therefore, SU(2) must check RejectMemNodes by calling adjustChainDeps(). Previously, for store SUs, adjustChainDeps() was only called if MayAlias was true, missing the S(2) to S(0) dependency in the case above. The fix is to always call adjustChainDeps(), regardless of MayAlias, since this applies both for AliasMemUses and NonAliasMemUses. No testcase found for any in-tree target. llvm-svn: 228686
* Update a few calls to getSubtarget<> to either be getSubtargetImplEric Christopher2015-01-271-9/+8
| | | | | | | when we didn't need the cast to the base class or the cached version off of the subtarget. llvm-svn: 227176
* New method SDep::isNormalMemoryOrBarrier() in ScheduleDAGInstrs.cpp.Jonas Paulsson2015-01-071-5/+6
| | | | | | | | | | | | | | | | | | | | Used to iterate over previously added memory dependencies in adjustChainDeps() and iterateChainSucc(). SDep::isCtrl() was previously used in these places, that also gave anti and output edges. The code may be worse if these are followed, because MisNeedChainEdge() will conservatively return true since a non-memory instruction has no memory operands, and a false chain dep will be added. It is also unnecessary since all memory accesses of interest will be reached by memory dependencies, and there is a budget limit for the number of edges traversed. This problem was found on an out-of-tree target with enabled alias analysis. No test case for an in-tree target has been found. Reviewed by Hal Finkel. llvm-svn: 225351
* Fix typos in comment and option help texts.Jonas Paulsson2015-01-071-3/+3
| | | | | | For -enable-aa-sched-mi and -use-tbaa-in-sched-mi. llvm-svn: 225350
* MISched: Fix moving stores across barriersTom Stellard2014-12-081-6/+7
| | | | | | | | This fixes an issue with ScheduleDAGInstrs::buildSchedGraph where stores without an underlying object would not be added as a predecessor to the current BarrierChain. llvm-svn: 223717
* Revert "r223440 - Consider subregs when calling MI::registerDefIsDead for ↵Hal Finkel2014-12-051-7/+1
| | | | | | | | | phys deps" Reverting this because, while it fixes the problem in the reduced test case, it does not fix the problem in the full test case from the bug report. llvm-svn: 223442
* Consider subregs when calling MI::registerDefIsDead for phys depsHal Finkel2014-12-051-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The scheduling dependency graph is built bottom-up within each scheduling region, and ScheduleDAGInstrs::addPhysRegDeps is called to add output/anti dependencies, based on physical registers, to the SUs for instructions based on those that come before them. In the test case, we start before post-RA scheduling with a block that looks like this: ... INLINEASM <... andc $0,$0,$2 stdcx. $0,0,$3 bne- 1b > [sideeffect] [mayload] [maystore] [attdialect], $0:[regdef-ec:G8RC], %X6<earlyclobber,def,dead>, $1:[mem], %X3<kill>, $2:[reguse:G8RC], %X5<kill>, $3:[reguse:G8RC], %X3, $4:[mem], %X3, $5:[clobber], %CC<earlyclobber,imp-def,dead>, <<badref>> ... %X4<def,dead> = ANDIo8 %X4<kill>, 1, %CR0<imp-def,dead>, %CR0GT<imp-def> ... %R29<def> = ISEL %R3<undef>, %R4<kill>, %CR0GT<kill> where it is relevant that %CC is an alias to %CR0, and that %CR0GT is a subregister of %CR0. However, for post-RA scheduling, no dependency was added to prevent the INLINEASM from being scheduled in between the ANDIo8 and the ISEL (which communicate via the %CR0GT register). In ScheduleDAGInstrs::addPhysRegDeps, when called for the %CC operand, we'd iterate over all of its aliases (which include %CC itself and also %CR0), and look for previously-encountered defs of those registers. We'd find the ANDIo8, but decide not to add a dependency between the INLINEASM and the ANDIo8 because both the INLINEASM's def of %CC is dead, and also the ANDIo8 def of %CR0 is dead. This ignores, however, that ANDIo8 has a non-dead def of %CR0GT, a subregister of %CR0, and thus a dependency still must exist. To fix this problem, when calling registerDefIsDead on the SU with the def, we also check all subregisters for possible non-dead defs, and add the dependency if any are found. Fixes PR21742. llvm-svn: 223440
* Update SetVector to rely on the underlying set's insert to return a ↵David Blaikie2014-11-191-2/+2
| | | | | | | | | | | | | pair<iterator, bool> This is to be consistent with StringSet and ultimately with the standard library's associative container insert function. This lead to updating SmallSet::insert to return pair<iterator, bool>, and then to update SmallPtrSet::insert to return pair<iterator, bool>, and then to update all the existing users of those functions... llvm-svn: 222334
* Remove an unnecessary restriction. MIsNeedChainEdge() should be checked ↵Owen Anderson2014-09-121-1/+1
| | | | | | | | | even when scheduler AliasAnalysis is not enabled. A good chunk of the MIsNeedChainEdge() is logic that is valid and should be applied even for targets that are not using for alias analysis. llvm-svn: 217706
* [AArch64] Improve AA to remove unneeded edges in the AA MI scheduling graph.Chad Rosier2014-09-081-0/+9
| | | | | | | Patch by Sanjin Sijaric <ssijaric@codeaurora.org>! Phabricator Review: http://reviews.llvm.org/D5103 llvm-svn: 217371
* Change MCSchedModel to be a struct of statically initialized data.Pete Cooper2014-09-021-1/+1
| | | | | | | | This removes static initializers from the backends which generate this data, and also makes this struct match the other Tablegen generated structs in behaviour Reviewed by Andy Trick and Chandler C llvm-svn: 216919
* Repace SmallPtrSet with SmallPtrSetImpl in function arguments to avoid ↵Craig Topper2014-08-211-1/+1
| | | | | | needing to mention the size. llvm-svn: 216158
* Cleanup: Delete seemingly unused reference to MachineDominatorTree from ↵Alexey Samsonov2014-08-201-2/+1
| | | | | | ScheduleDAGInstrs. llvm-svn: 216124
* Fix null reference creation in ScheduleDAGInstrs constructor call.Alexey Samsonov2014-08-201-2/+2
| | | | | | | | | Both MachineLoopInfo and MachineDominatorTree may be null in ScheduleDAGMI constructor call. It is undefined behavior to take references to these values. This bug is reported by UBSan. llvm-svn: 216118
* Revert "Repace SmallPtrSet with SmallPtrSetImpl in function arguments to ↵Craig Topper2014-08-181-1/+1
| | | | | | | | avoid needing to mention the size." Getting a weird buildbot failure that I need to investigate. llvm-svn: 215870
* Repace SmallPtrSet with SmallPtrSetImpl in function arguments to avoid ↵Craig Topper2014-08-171-1/+1
| | | | | | needing to mention the size. llvm-svn: 215868
* AA metadata refactoring (introduce AAMDNodes)Hal Finkel2014-07-241-2/+2
| | | | | | | | | | | | | | | | | | | | In order to enable the preservation of noalias function parameter information after inlining, and the representation of block-level __restrict__ pointer information (etc.), additional kinds of aliasing metadata will be introduced. This metadata needs to be carried around in AliasAnalysis::Location objects (and MMOs at the SDAG level), and so we need to generalize the current scheme (which is hard-coded to just one TBAA MDNode*). This commit introduces only the necessary refactoring to allow for the introduction of other aliasing metadata types, but does not actually introduce any (that will come in a follow-up commit). What it does introduce is a new AAMDNodes structure to hold all of the aliasing metadata nodes associated with a particular memory-accessing instruction, and uses that structure instead of the raw MDNode* in AliasAnalysis::Location, etc. No functionality change intended. llvm-svn: 213859
* Move remaining LLVM_ENABLE_DUMP conditionals out of the headersAlp Toker2014-07-011-2/+3
| | | | | | | | | This macro is sometimes defined manually but isn't (and doesn't need to be) in llvm-config.h so shouldn't appear in the headers, likewise NDEBUG. Instead switch them over to LLVM_DUMP_METHOD on the definitions. llvm-svn: 212130
* Revert "Introduce a string_ostream string builder facilty"Alp Toker2014-06-261-1/+2
| | | | | | Temporarily back out commits r211749, r211752 and r211754. llvm-svn: 211814
OpenPOWER on IntegriCloud