summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/ScheduleDAGInstrs.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Implement LaneBitmask::any(), use it to replace !none(), NFCIKrzysztof Parzyszek2016-12-161-4/+4
| | | | llvm-svn: 289974
* Extract LaneBitmask into a separate typeKrzysztof Parzyszek2016-12-151-12/+13
| | | | | | | | | | | | Specifically avoid implicit conversions from/to integral types to avoid potential errors when changing the underlying type. For example, a typical initialization of a "full" mask was "LaneMask = ~0u", which would result in a value of 0x00000000FFFFFFFF if the type was extended to uint64_t. Differential Revision: https://reviews.llvm.org/D27454 llvm-svn: 289820
* ScheduleDAGInstrs: Move VRegUses to ScheduleDAGMILive; NFCIMatthias Braun2016-11-111-43/+0
| | | | | | | | | | | | | Push VRegUses/collectVRegUses() down the class hierarchy towards its only user ScheduleDAGMILive. NFCI: The initialization of the map happens at a later point but that should not matter. This is in preparation to allow DAG mutators to merge nodes, which relies on this map getting computed later. llvm-svn: 286654
* ScheduleDAGInstrs: Add condjump deps to addSchedBarrierDeps()Matthias Braun2016-11-111-7/+4
| | | | | | | | | | | | | | | addSchedBarrierDeps() is supposed to add use operands to the ExitSU node. The current implementation adds uses for calls/barrier instruction and the MBB live-outs in all other cases. The use operands of conditional jump instructions were missed. Also added code to macrofusion to set the latencies between nodes to zero to avoid problems with the fusing nodes lingering around in the pending list now. Differential Revision: https://reviews.llvm.org/D25140 llvm-svn: 286544
* ScheduleDAGInstrs: Ignore dependencies of constant physregsMatthias Braun2016-11-101-0/+3
| | | | | | | | | There is no need to track dependencies for constant physregs, as they don't change their value no matter in what order you read/write to them. Differential Revision: https://reviews.llvm.org/D26221 llvm-svn: 286526
* ScheduleDAGInstrs: Slightly simplify code; NFCMatthias Braun2016-11-101-20/+12
| | | | llvm-svn: 286510
* MachineInstrBundle: Pass iterators to getBundle(Start|End); NFCMatthias Braun2016-10-251-2/+2
| | | | | | | | This is a function to go backwards in a block to find the first instruction in a bundle, so iterator is a more natural choice for parameter/return rather than a reference to a MachineInstruction. llvm-svn: 285051
* Fix machine operand traversal in ScheduleDAGInstrs::fixupKillsKrzysztof Parzyszek2016-10-051-2/+7
| | | | llvm-svn: 283315
* ScheduleDAGInstrs: Cleanup, use range based for; NFCMatthias Braun2016-09-301-61/+45
| | | | llvm-svn: 282979
* [CodeGen] Rename MachineInstr::isInvariantLoad to ↵Justin Lebar2016-09-101-2/+3
| | | | | | | | | | | | | | | | | | | | isDereferenceableInvariantLoad. NFC Summary: I want to separate out the notions of invariance and dereferenceability at the MI level, so that they correspond to the equivalent concepts at the IR level. (Currently an MI load is MI-invariant iff it's IR-invariant and IR-dereferenceable.) First step is renaming this function. Reviewers: chandlerc Subscribers: MatzeB, jfb, llvm-commits Differential Revision: https://reviews.llvm.org/D23370 llvm-svn: 281125
* MachineFunction: Return reference for getFrameInfo(); NFCMatthias Braun2016-07-281-5/+5
| | | | | | | getFrameInfo() never returns nullptr so we should use a reference instead of a pointer. llvm-svn: 277017
* Untabify.NAKAMURA Takumi2016-07-251-1/+1
| | | | llvm-svn: 276597
* CodeGen: Use MachineInstr& in ScheduleDAGIntrs, NFCDuncan P. N. Exon Smith2016-07-011-43/+43
| | | | | | | | | | | | | Use MachineInstr& to avoid implicit conversions from MachineBasicBlock::iterator to MachineInstr*. In one case, this could use a range-based for loop, but the other loops iterated in reverse order. One of the reverse-loops checked the MachineInstr* for nullptr, a condition that is provably unreachable. (And even if my proof has a flaw, UBSan would catch the bug.) llvm-svn: 274360
* CodeGen: Use MachineInstr& in TargetInstrInfo, NFCDuncan P. N. Exon Smith2016-06-301-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is mostly a mechanical change to make TargetInstrInfo API take MachineInstr& (instead of MachineInstr* or MachineBasicBlock::iterator) when the argument is expected to be a valid MachineInstr. This is a general API improvement. Although it would be possible to do this one function at a time, that would demand a quadratic amount of churn since many of these functions call each other. Instead I've done everything as a block and just updated what was necessary. This is mostly mechanical fixes: adding and removing `*` and `&` operators. The only non-mechanical change is to split ARMBaseInstrInfo::getOperandLatencyImpl out from ARMBaseInstrInfo::getOperandLatency. Previously, the latter took a `MachineInstr*` which it updated to the instruction bundle leader; now, the latter calls the former either with the same `MachineInstr&` or the bundle leader. As a side effect, this removes a bunch of MachineInstr* to MachineBasicBlock::iterator implicit conversions, a necessary step toward fixing PR26753. Note: I updated WebAssembly, Lanai, and AVR (despite being off-by-default) since it turned out to be easy. I couldn't run tests for AVR since llc doesn't link with it turned on. llvm-svn: 274189
* ScheduleDAGInstrs: Fix memory corruptionMatthias Braun2016-05-251-2/+2
| | | | | | | | We have to modify V2SU before inserting new elements into the CurrentVRegDefs set because that may move V2SU in memory invalidating the reference. llvm-svn: 270644
* ScheduleDAGInstrs: Comment on why subreg defs are not seen as uses; NFCMatthias Braun2016-05-101-0/+4
| | | | | | | | Usually subregister definitions are consider uses of the remaining lanes that did not get defined. Add a comment why the code in ScheduleDAGInstrs does not add use dependencies regardless. llvm-svn: 269107
* Fix PR26655: Bail out if all regs of an inst BUNDLE have the correct kill flagMandeep Singh Grang2016-05-101-27/+11
| | | | | | | | | | | | | | | | | | | | Summary: While setting kill flags on instructions inside a BUNDLE, we bail out as soon as we set kill flag on a register. But we are missing a check when all the registers already have the correct kill flag set. We need to bail out in that case as well. This patch refactors the old code and simply makes use of the addRegisterKilled function in MachineInstr.cpp in order to determine whether to set/remove kill on an instruction. Reviewers: apazos, t.p.northover, pete, MatzeB Subscribers: MatzeB, davide, llvm-commits Differential Revision: http://reviews.llvm.org/D17356 llvm-svn: 269092
* [ScheduleDAG] Make sure to process all def operands before any use operandsKrzysztof Parzyszek2016-05-101-8/+25
| | | | | | | | | | | | | | | | | An example from Hexagon where things went wrong: %R0<def> = L2_loadrigp <ga:@fp04> ; load function address J2_callr %R0<kill>, ..., %R0<imp-def> ; call *R0, return value in R0 ScheduleDAGInstrs::buildSchedGraph would visit all instructions going backwards, and in each instruction it would visit all operands in their order on the operand list. In the case of this call, it visited the use of R0 first, then removed it from the set Uses after it visited the def. This caused the DAG to be missing the data dependence edge on R0 between the load and the call. Differential Revision: http://reviews.llvm.org/D20102 llvm-svn: 269076
* ScheduleDAGInstrs.cpp: Don't peel the iterator when it points the end. This ↵NAKAMURA Takumi2016-05-021-1/+1
| | | | | | will fix the crash in r268143. llvm-svn: 268257
* [NFC] Header cleanupMehdi Amini2016-04-181-1/+0
| | | | | | | | | | | | | | Removed some unused headers, replaced some headers with forward class declarations. Found using simple scripts like this one: clear && ack --cpp -l '#include "llvm/ADT/IndexedMap.h"' | xargs grep -L 'IndexedMap[<]' | xargs grep -n --color=auto 'IndexedMap' Patch by Eugene Kosov <claprix@yandex.ru> Differential Revision: http://reviews.llvm.org/D19219 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 266595
* Do not modify a cl::opt programmatically, global mutable state is evil.Mehdi Amini2016-04-161-9/+13
| | | | | | | Found by TSAN on ThinLTO. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 266514
* [ScheduleDAGInstrs] Re-factor for based on review feedback. NFC.Geoff Berry2016-04-141-52/+45
| | | | | | | | | | | | | | Summary: Re-factor some code to improve clarity and style based on review comments from http://reviews.llvm.org/D18093. Reviewers: MatzeB, mcrosier Subscribers: MatzeB, mcrosier, llvm-commits Differential Revision: http://reviews.llvm.org/D19128 llvm-svn: 266372
* [ScheduleDAGInstrs] Handle instructions with multiple MMOsGeoff Berry2016-04-121-30/+41
| | | | | | | | | | | | | | | | | | | | | Summary: In getUnderlyingObjectsForInstr(): Don't give up on instructions with multiple MMOs, instead look through all the MMOs and if they all meet the conservative criteria previously used for single MMO instructions, then return all of the underlying objects derived from the MMOs. The change to ScheduleDAGInstrs::buildSchedGraph() is needed to avoid the case where multiple underlying objects are present and are related in such a way that successive iterations of the loop end up adding a dependency from an instruction to itself. Reviewers: atrick, hfinkel Subscribers: MatzeB, mcrosier, llvm-commits Differential Revision: http://reviews.llvm.org/D18093 llvm-svn: 266084
* Fix PR26940 where compiles times regressed massively.Chandler Carruth2016-03-311-33/+30
| | | | | | | | | | | | | | | Patch by Jonas Paulsson. Original description: Bugfix in buildSchedGraph() to make -dag-maps-huge-region work properly I found that the reduction of the maps did in fact never happen in this test case. This was because *all* the stores / loads were made with addresses from arguments and they thus became "unknown" stores / loads. Fixed by removing continue statements and making sure that the test for reduction always takes place. Differential Revision: http://reviews.llvm.org/D18673 llvm-svn: 265063
* WIP: CodeGen: Use MachineInstr& in MachineInstrBundle.h, NFCDuncan P. N. Exon Smith2016-02-271-2/+2
| | | | | | | | Update APIs in MachineInstrBundle.h to take and return MachineInstr& instead of MachineInstr* when the instruction cannot be null. Besides being a nice cleanup, this is tacking toward a fix for PR26753. llvm-svn: 262141
* CodeGen: Take MachineInstr& in SlotIndexes and LiveIntervals, NFCDuncan P. N. Exon Smith2016-02-271-1/+1
| | | | | | | | | | | | | | Take MachineInstr by reference instead of by pointer in SlotIndexes and the SlotIndex wrappers in LiveIntervals. The MachineInstrs here are never null, so this cleans up the API a bit. It also incidentally removes a few implicit conversions from MachineInstrBundleIterator to MachineInstr* (see PR26753). At a couple of call sites it was convenient to convert to a range-based for loop over MachineBasicBlock::instr_begin/instr_end, so I added MachineBasicBlock::instrs. llvm-svn: 262115
* Revert "CodeGen: MachineInstr::getIterator() => getInstrIterator(), NFC"Duncan P. N. Exon Smith2016-02-221-4/+4
| | | | | | | | | | This reverts commit r261504, since it's not obvious the new name is better: http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20160222/334298.html I'll recommit if we get consensus that it's the right direction. llvm-svn: 261567
* CodeGen: MachineInstr::getIterator() => getInstrIterator(), NFCDuncan P. N. Exon Smith2016-02-211-4/+4
| | | | | | | | | | | | | | | | | | | | | | | Delete MachineInstr::getIterator(), since the term "iterator" is overloaded when talking about MachineInstr. - Downcast to ilist_node in iplist::getNextNode() and getPrevNode() so that ilist_node::getIterator() is still available. - Add it back as MachineInstr::getInstrIterator(). This matches the naming in MachineBasicBlock. - Add MachineInstr::getBundleIterator(). This is explicitly called "bundle" (not matching MachineBasicBlock) to disintinguish it clearly from ilist_node::getIterator(). - Update all calls. Some of these I switched to `auto` to remove boiler-plate, since the new name is clear about the type. There was one call I updated that looked fishy, but it wasn't clear what the right answer was. This was in X86FrameLowering::inlineStackProbe(), added in r252578 in lib/Target/X86/X86FrameLowering.cpp. I opted to leave the behaviour unchanged, but I'll reply to the original commit on the list in a moment. llvm-svn: 261504
* [ScheduleDAGInstrs] isUnsafeMemoryObject() removedJonas Paulsson2016-02-151-36/+5
| | | | | | | | | | | This function was basically useless, since volatile memacesses or MIs with unmodelled sideffects become global memory objects, and the other little checks are also done elsewhere. Reviewed by Andy Trick http://reviews.llvm.org/D16881 llvm-svn: 260899
* [ScheduleDagInstrs] Improved commentsJonas Paulsson2016-02-041-9/+9
| | | | llvm-svn: 259783
* [ScheduleDAGInstrs::buildSchedGraph()] Handling of memory dependecies rewritten.Jonas Paulsson2016-02-031-347/+362
| | | | | | | | | | | | | | | | | | | Recommited, after some fixing with test cases. Updated test cases: test/CodeGen/AArch64/arm64-misched-memdep-bug.ll test/CodeGen/AArch64/tailcall_misched_graph.ll Temporarily disabled test cases: test/CodeGen/AMDGPU/split-vector-memoperand-offsets.ll test/CodeGen/PowerPC/ppc64-fastcc.ll (partially updated) test/CodeGen/PowerPC/vsx-fma-m.ll test/CodeGen/PowerPC/vsx-fma-sp.ll http://reviews.llvm.org/D8705 Reviewers: Hal Finkel, Andy Trick. llvm-svn: 259673
* Temporarily revert "[ScheduleDAGInstrs::buildSchedGraph()] Handling of ↵Jonas Paulsson2016-01-291-376/+347
| | | | | | | | memory dependecies rewritten." Some buildbot failures needs to be debugged. llvm-svn: 259213
* [ScheduleDAGInstrs::buildSchedGraph()] Handling of memory dependecies rewritten.Jonas Paulsson2016-01-291-347/+376
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The buildSchedGraph() was in need of reworking as the AA features had been added on top of earlier code. It was very difficult to understand, and buggy. There had been found cases where scheduling dependencies had actually been missed (see r228686). AliasChain, RejectMemNodes, adjustChainDeps() and iterateChainSucc() have been removed. There are instead now just the four maps from Value to SUs, which have been renamed to Stores, Loads, NonAliasStores and NonAliasLoads. An unknown store used to become the AliasChain, but now becomes a store mapped to 'unknownValue' (in Stores). What used to be PendingLoads is instead the list of SUs mapped to 'unknownValue' in Loads. RejectMemNodes and adjustChainDeps() used to be a safety-net for everything. The SU maps were sometimes cleared and SUs were put in RejectMemNodes, where adjustChainDeps() would look. Instead of this, a more straight forward approach is used in maintaining the SU maps without clearing them and simply letting them grow over time. Instead of the cutt-off in adjustChainDeps() search, a reduction of maps will be done if needed (see below). Each SUnit either becomes the BarrierChain, or is put into one of the maps. For each SUnit encountered, all the information about previous ones are still available until a new BarrierChain is set, at which point the maps are cleared. For huge regions, the algorithm becomes slow, therefore the maps will get reduced at a threshold (current default is 1000 nodes), by a fraction (default 1/2). These values can be tuned by use of CL options in case some test case shows that they need to be changed (-dag-maps-huge-region and -dag-maps-reduction-size). There has not been any considerable change observed in output quality or compile time. There may now be more DAG edges inserted than before (i.e. if A->B->C, then A->C is not needed). However, in a comparison run there were fewer total calls to AA, and a somewhat improved compile time, which means this seems to be not a problem. http://reviews.llvm.org/D8705 Reviewers: Hal Finkel, Andy Trick. llvm-svn: 259201
* [ScheduleDAGInstrs] Simplify logic to improve readability. NFC.Chad Rosier2016-01-261-2/+1
| | | | | | The call to isInvariantLoad() already returns false for non-load instructions. llvm-svn: 258841
* MachineScheduler: Allow independent scheduling of sub register defsMatthias Braun2016-01-201-0/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Note that this is disabled by default and still requires a patch to handleMove() which is not upstreamed yet. If the TrackLaneMasks policy/strategy is enabled the MachineScheduler will build a schedule graph where definitions of independent subregisters are no longer serialised. Implementation comments: - Without lane mask tracking a sub register def also counts as a use (except for the first one with the read-undef flag set), with lane mask tracking enabled this is no longer the case. - Pressure Diffs where previously maintained per definition of a vreg with the help of the SSA information contained in the LiveIntervals. With lanemask tracking enabled we cannot do this anymore and instead change the pressure diffs for all uses of the vreg as it becomes live/dead. For this changed style to work correctly we ignore uses of instructions that define the same register again: They won't affect register pressure. - With lanemask tracking we remove all read-undef flags from sub register defs when building the graph and re-add them later when all vreg lanes have become dead. Differential Revision: http://reviews.llvm.org/D14969 llvm-svn: 258259
* RegisterPressure: Make liveness tracking subregister awareMatthias Braun2016-01-201-1/+1
| | | | | | Differential Revision: http://reviews.llvm.org/D14968 llvm-svn: 258258
* RegisterPressure: Expose RegisterOperands APIMatthias Braun2016-01-121-4/+9
| | | | | | | | | | | | | | | | | | | | | | Previously the RegisterOperands have only been used internally in RegisterPressure.cpp. However this datastructure can be useful for other tasks as well and allows refactoring of PDiff initialisation out of RPTracker::recede(). This patch: - Exposes RegisterOperands as public API - Splits RPTracker::recede() into a part that skips DebugValues and maintains the region borders, and the core that changes register pressure when given a set of RegisterOperands. - This allows to move the PDiff initialisation out recede() into a method of the PressureDiffs class. - The upcoming subregister scheduling code will also use RegisterOperands to avoid pushing more unrelated functionality into recede()/advance(). Differential Revision: http://reviews.llvm.org/D15473 llvm-svn: 257535
* ScheduleDAGInstrs: Bug fix for missed memory dependency.Geoff Berry2016-01-061-3/+12
| | | | | | | | | | | | | | | | | | | | Summary: In buildSchedGraph(), when adding memory dependencies for loads, move the call to adjustChainDeps() after the call to addChainDependency(AliasChain) to handle the case where addChainDependency(AliasChain) ends up not adding a dependency and instead putting the SU on the RejectMemNodes list. The call to adjustChainDeps() must be done after the call to addChainDependency() in order to process the SU added to the RejectMemNodes list to create memory dependencies for it. Reviewers: hfinkel, atrick, jonpa, resistor Subscribers: mcrosier, llvm-commits Differential Revision: http://reviews.llvm.org/D15927 llvm-svn: 256950
* ScheduleDAGInstrs: Move LiveIntervals field to ScheduleDAGMIMatthias Braun2015-12-041-2/+1
| | | | | | | Now that ScheduleDAGInstrs doesn't need it anymore we can move the field down the class hierarcy to ScheduleDAGMI. llvm-svn: 254759
* ScheduleDAGInstrs: Rework schedule graph builder.Matthias Braun2015-12-041-66/+161
| | | | | | | | | | | | | | | | | | Re-comitting with a change that avoids undefined uses getting put into the VRegUses list. The new algorithm remembers the uses encountered while walking backwards until a matching def is found. Contrary to the previous version this: - Works without LiveIntervals being available - Allows to increase the precision to subregisters/lanemasks (not used for now) The changes in the AMDGPU tests are necessary because the R600 scheduler is not stable with respect to the order of nodes in the ready queues. Differential Revision: http://reviews.llvm.org/D9068 llvm-svn: 254683
* Revert "ScheduleDAGInstrs: Rework schedule graph builder."Matthias Braun2015-12-031-159/+66
| | | | | | | | | | This works mostly fine but breaks some stage 1 builders when compiling compiler-rt on i386. Revert for further investigation as I can't see an obvious cause/fix. This reverts commit r254577. llvm-svn: 254586
* ScheduleDAGInstrs: Rework schedule graph builder.Matthias Braun2015-12-031-66/+159
| | | | | | | | | | | | | | | The new algorithm remembers the uses encountered while walking backwards until a matching def is found. Contrary to the previous version this: - Works without LiveIntervals being available - Allows to increase the precision to subregisters/lanemasks (not used for now) The changes in the AMDGPU tests are necessary because the R600 scheduler is not stable with respect to the order of nodes in the ready queues. Differential Revision: http://reviews.llvm.org/D9068 llvm-svn: 254577
* ScheduleDAGInstrs: Remove IsPostRA flag; NFCMatthias Braun2015-11-031-14/+7
| | | | | | | | | | | | | | | | | ScheduleDAGInstrs doesn't behave differently before or after register allocation. It was only used in a method of MachineSchedulerBase which behaved differently in MachineScheduler/PostMachineScheduler. Change this to let MachineScheduler/PostMachineScheduler just pass in a parameter to that function. The order of the LiveIntervals* and bool RemoveKillFlags paramters have been switched to make out-of-tree code fail instead of unintentionally passing a value intended for the IsPostRA flag to the (previously following and default initialized) RemoveKillFlags. Differential Revision: http://reviews.llvm.org/D14245 llvm-svn: 251883
* Revert "ScheduleDAGInstrs: Remove IsPostRA flag"Matthias Braun2015-10-291-5/+12
| | | | | | | | It broke 3 arm testcases. This reverts commit r251608. llvm-svn: 251615
* ScheduleDAGInstrs: Remove IsPostRA flagMatthias Braun2015-10-291-12/+5
| | | | | | | | This was a layering violation in ScheduleDAGInstrs (and MachineSchedulerBase) they both shouldn't know directly whether they are used by the PostMachineScheduler or the MachineScheduler. llvm-svn: 251608
* Refactor: Simplify boolean conditional return statements in lib/CodeGen.Rafael Espindola2015-10-241-5/+3
| | | | | | Patch by Richard. llvm-svn: 251213
* CodeGen: Avoid more ilist iterator implicit conversions, NFCDuncan P. N. Exon Smith2015-10-091-2/+2
| | | | llvm-svn: 249903
* Save LaneMask with livein registersMatthias Braun2015-09-091-5/+5
| | | | | | | | | | | | | | | | | With subregister liveness enabled we can detect the case where only parts of a register are live in, this is expressed as a 32bit lanemask. The current code only keeps registers in the live-in list and therefore enumerated all subregisters affected by the lanemask. This turned out to be too conservative as the subregister may also cover additional parts of the lanemask which are not live. Expressing a given lanemask by enumerating a minimum set of subregisters is computationally expensive so the best solution is to simply change the live-in list to store the lanemasks as well. This will reduce memory usage for targets using subregister liveness and slightly increase it for other targets Differential Revision: http://reviews.llvm.org/D12442 llvm-svn: 247171
* MachineBasicBlock: Add liveins() method returning an iterator_rangeMatthias Braun2015-08-241-9/+5
| | | | llvm-svn: 245895
* Fix some comment typos.Benjamin Kramer2015-08-081-4/+4
| | | | llvm-svn: 244402
OpenPOWER on IntegriCloud