summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/Scalar/RewriteStatepointsForGC.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Replace some callers of setTailCall with setTailCallKindDavid Majnemer2016-11-251-1/+1
| | | | | | | We were a little sloppy with adding tailcall markers. Be more consistent by using setTailCallKind instead of setTailCall. llvm-svn: 287955
* [RS4GC] Fix comment to show TODO. NFCAnna Thomas2016-10-061-1/+1
| | | | llvm-svn: 283449
* [RS4GC] Handle ShuffleVector instruction in findBasePointerAnna Thomas2016-10-041-10/+32
| | | | | | | | | | | | | | | Summary: This patch modifies the findBasePointer to handle the shufflevector instruction. Tests run: RS4GC tests, local downstream tests. Reviewers: reames, sanjoy Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D25197 llvm-svn: 283219
* [RS4GC] Remat in presence of phi and use live valueAnna Thomas2016-09-221-8/+21
| | | | | | | | | | Summary: Reviewers: Subscribers: llvm-svn: 282150
* [RS4GC] Refactor code for Rematerializing in presence of phi. NFCAnna Thomas2016-09-201-59/+68
| | | | | | | | | | | | | | | | | | | | | | | | Summary: This is an NFC refactoring change as a precursor to the actual fix for rematerializing in presence of phi. https://reviews.llvm.org/D24399 Pasted from review: findRematerializableChainToBasePointer changed to return the root of the chain. instead of true or false. move the PHI matching logic into the caller by inspecting the root return value. This includes an assertion that the alternate root is in the liveset for the call. Tested with current RS4GC tests. Reviewers: reames, sanjoy Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D24780 llvm-svn: 282023
* [statepoints][experimental] Add support for live-in semantics of values in ↵Philip Reames2016-08-311-0/+26
| | | | | | | | | | | | | | | | deopt bundles This is a first step towards supporting deopt value lowering and reporting entirely with the register allocator. I hope to build on this in the near future to support live-on-return semantics, but I have a use case which allows me to test and investigate code quality with just the live-in semantics so I've chosen to start there. For those curious, my use cases is our implementation of the "__llvm_deoptimize" function we bind to @llvm.deoptimize. I'm choosing not to hard code that fact in the patch and instead make it configurable via function attributes. The basic approach here is modelled on what is done for the "Live In" values on stackmaps and patchpoints. (A secondary goal here is to remove one of the last barriers to merging the pseudo instructions.) We start by adding the operands directly to the STATEPOINT SDNode. Once we've lowered to MI, we extend the remat logic used by the register allocator to fold virtual register uses into StackMap::Indirect entries as needed. This does rely on the fact that the register allocator rematerializes. If it didn't along some code path, we could end up with more vregs than physical registers and fail to allocate. Today, we *only* fold in the register allocator. This can create some weird effects when combined with arguments passed on the stack because we don't fold them appropriately. I have an idea how to fix that, but it needs this patch in place to work on that effectively. (There's some weird interaction with the scheduler as well, more investigation needed.) My near term plan is to land this patch off-by-default, experiment in my local tree to identify any correctness issues and then start fixing codegen problems one by one as I find them. Once I have the live-in lowering fully working (both correctness and code quality), I'm hoping to move on to the live-on-return semantics. Note: I don't have any *known* miscompiles with this patch enabled, but I'm pretty sure I'll find at least a couple. Thus, the "experimental" tag and the fact it's off by default. Differential Revision: https://reviews.llvm.org/D24000 llvm-svn: 280250
* [RewriteStatepointsForGC] Update comment for same PHI node check. NFCAnna Thomas2016-08-301-1/+2
| | | | llvm-svn: 280052
* ADT: Give ilist<T>::reverse_iterator a handle to the current nodeDuncan P. N. Exon Smith2016-08-301-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Reverse iterators to doubly-linked lists can be simpler (and cheaper) than std::reverse_iterator. Make it so. In particular, change ilist<T>::reverse_iterator so that it is *never* invalidated unless the node it references is deleted. This matches the guarantees of ilist<T>::iterator. (Note: MachineBasicBlock::iterator is *not* an ilist iterator, but a MachineInstrBundleIterator<MachineInstr>. This commit does not change MachineBasicBlock::reverse_iterator, but it does update MachineBasicBlock::reverse_instr_iterator. See note at end of commit message for details on bundle iterators.) Given the list (with the Sentinel showing twice for simplicity): [Sentinel] <-> A <-> B <-> [Sentinel] the following is now true: 1. begin() represents A. 2. begin() holds the pointer for A. 3. end() represents [Sentinel]. 4. end() holds the poitner for [Sentinel]. 5. rbegin() represents B. 6. rbegin() holds the pointer for B. 7. rend() represents [Sentinel]. 8. rend() holds the pointer for [Sentinel]. The changes are #6 and #8. Here are some properties from the old scheme (which used std::reverse_iterator): - rbegin() held the pointer for [Sentinel] and rend() held the pointer for A; - operator*() cost two dereferences instead of one; - converting from a valid iterator to its valid reverse_iterator involved a confusing increment; and - "RI++->erase()" left RI invalid. The unintuitive replacement was "RI->erase(), RE = end()". With vector-like data structures these properties are hard to avoid (since past-the-beginning is not a valid pointer), and don't impose a real cost (since there's still only one dereference, and all iterators are invalidated on erase). But with lists, this was a poor design. Specifically, the following code (which obviously works with normal iterators) now works with ilist::reverse_iterator as well: for (auto RI = L.rbegin(), RE = L.rend(); RI != RE;) fooThatMightRemoveArgFromList(*RI++); Converting between iterator and reverse_iterator for the same node uses the getReverse() function. reverse_iterator iterator::getReverse(); iterator reverse_iterator::getReverse(); Why doesn't iterator <=> reverse_iterator conversion use constructors? In order to catch and update old code, reverse_iterator does not even have an explicit conversion from iterator. It wouldn't be safe because there would be no reasonable way to catch all the bugs from the changed semantic (see the changes at call sites that are part of this patch). Old code used this API: std::reverse_iterator::reverse_iterator(iterator); iterator std::reverse_iterator::base(); Here's how to update from old code to new (that incorporates the semantic change), assuming I is an ilist<>::iterator and RI is an ilist<>::reverse_iterator: [Old] ==> [New] reverse_iterator(I) (--I).getReverse() reverse_iterator(I) ++I.getReverse() --reverse_iterator(I) I.getReverse() reverse_iterator(++I) I.getReverse() RI.base() (--RI).getReverse() RI.base() ++RI.getReverse() --RI.base() RI.getReverse() (++RI).base() RI.getReverse() delete &*RI, RE = end() delete &*RI++ RI->erase(), RE = end() RI++->erase() ======================================= Note: bundle iterators are out of scope ======================================= MachineBasicBlock::iterator, also known as MachineInstrBundleIterator<MachineInstr>, is a wrapper to represent MachineInstr bundles. The idea is that each operator++ takes you to the beginning of the next bundle. Implementing a sane reverse iterator for this is harder than ilist. Here are the options: - Use std::reverse_iterator<MBB::i>. Store a handle to the beginning of the next bundle. A call to operator*() runs a loop (usually operator--() will be called 1 time, for unbundled instructions). Increment/decrement just works. This is the status quo. - Store a handle to the final node in the bundle. A call to operator*() still runs a loop, but it iterates one time fewer (usually operator--() will be called 0 times, for unbundled instructions). Increment/decrement just works. - Make the ilist_sentinel<MachineInstr> *always* store that it's the sentinel (instead of just in asserts mode). Then the bundle iterator can sniff the sentinel bit in operator++(). I initially tried implementing the end() option as part of this commit, but updating iterator/reverse_iterator conversion call sites was error-prone. I have a WIP series of patches that implements the final option. llvm-svn: 280032
* [StatepointsForGC] Rematerialize in the presence of PHIsAnna Thomas2016-08-291-0/+35
| | | | | | | | | | | | | | | | Summary: While walking the use chain for identifying rematerializable values in RS4GC, add the case where the current value and base value are the same PHI nodes. This will aid rematerialization of geps and casts instead of relocating. Reviewers: sanjoy, reames, igor Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D23920 llvm-svn: 279975
* Use the range variant of remove_if instead of unpacking begin/endDavid Majnemer2016-08-121-3/+2
| | | | | | No functionality change is intended. llvm-svn: 278475
* Use the range variant of find instead of unpacking begin/endDavid Majnemer2016-08-111-3/+2
| | | | | | | | | If the result of the find is only used to compare against end(), just use is_contained instead. No functionality change is intended. llvm-svn: 278433
* Use range algorithms instead of unpacking begin/endDavid Majnemer2016-08-111-2/+1
| | | | | | No functionality change is intended. llvm-svn: 278417
* Apply clang-tidy's modernize-loop-convert to most of lib/Transforms.Benjamin Kramer2016-06-261-3/+2
| | | | | | Only minor manual fixes. No functionality change intended. llvm-svn: 273808
* [RSForGC] Appease MSVCSanjoy Das2016-06-261-2/+4
| | | | llvm-svn: 273805
* [RSForGC] Bring the BDVState struct up to code; NFCSanjoy Das2016-06-261-25/+33
| | | | llvm-svn: 273800
* [RSForGC] Bring computeLiveInValues up to code; NFCSanjoy Das2016-06-261-8/+5
| | | | llvm-svn: 273799
* [RSForGC] Bring computeLiveOutSeed up to code; NFCSanjoy Das2016-06-261-7/+7
| | | | llvm-svn: 273798
* [RSForGC] Bring computeLiveInValues up to code; NFCSanjoy Das2016-06-261-19/+8
| | | | llvm-svn: 273797
* [RSForGC] Bring recomputeLiveInValues up to code; NFCSanjoy Das2016-06-261-9/+9
| | | | llvm-svn: 273796
* [RSForGC] Bring containsGCPtrType, isGCPointerType up to code; NFCSanjoy Das2016-06-261-3/+2
| | | | llvm-svn: 273795
* [RSForGC] Bring analyzeParsePointLiveness up to code; NFCSanjoy Das2016-06-261-7/+7
| | | | llvm-svn: 273794
* [RSForGC] Bring meetBDVStateImpl up to code; NFCSanjoy Das2016-06-261-14/+13
| | | | llvm-svn: 273793
* [RSForGC] Get rid of the unnecessary MeetBDVStates struct; NFCSanjoy Das2016-06-261-58/+36
| | | | | | All of its implementation is in just one function. llvm-svn: 273792
* [RSForGC] Bring findBasePointer up to code; NFCSanjoy Das2016-06-261-110/+92
| | | | | | | Name-casing and minor style changes to bring the function up to the LLVM coding style. llvm-svn: 273791
* Fix unused variable warning by folding the temporary into the debug statement.Eric Christopher2016-06-231-2/+2
| | | | llvm-svn: 273523
* [RS4GC] Use StringRef; NFCSanjoy Das2016-06-221-4/+3
| | | | | | Spotted during random inspection. llvm-svn: 273512
* Avoid duplicated map lookups. No functionality change intended.Benjamin Kramer2016-06-171-2/+1
| | | | llvm-svn: 273030
* [RS4GC] Pass CallSite by value instead of const ref; NFCSanjoy Das2016-06-171-11/+10
| | | | | | That's the idiomatic LLVM pattern. llvm-svn: 272981
* [RewriteStatepointsForGC] All constant should have null base pointerIgor Laevsky2016-05-271-8/+16
| | | | | | | | | | | | | | | | | | Currently we consider that each constant has itself as a base value. I.e "base(const) = const". This introduces couple of problems when we are trying to avoid reporting constants in statepoint live sets: 1. When querying "base( phi(const1, const2) )" we will get "phi(const1, const2)" as a base pointer. Since it's not a constant we will record it in a stack map. However on practice we don't want this to happen (constant are never relocated). 2. base( phi(const, gc ptr) ) = phi( const, base(gc ptr) ). This particular case imposes challenge on our runtime - we don't expect to see constant base pointers other than null. This problems can be avoided by treating all constant as if they were derived from null pointer base. I.e in a first case we will not include constant pointer in a stack map at all. In a second case we will get "phi(null, base(gc ptr))" as a base pointer which is a lot more convenient. Differential Revision: http://reviews.llvm.org/D20584 llvm-svn: 270993
* [RewriteStatepointsForGC] Remove obsolete assertionIgor Laevsky2016-05-171-8/+0
| | | | | | | | | | This is assertion is no longer necessary since we never record constants in the live set anyway. (They are never recorded in the initial live set, and constant bases are removed near line 2119) Differential Revision: http://reviews.llvm.org/D20293 llvm-svn: 269764
* [PM] RewriterStatepointForGC: add missing dependency.Davide Italiano2016-05-161-0/+1
| | | | llvm-svn: 269624
* [RS4GC] Fix typo in commentSanjoy Das2016-05-061-1/+1
| | | | llvm-svn: 268790
* [RS4GC] Use SetVector/MapVector instead of DenseSet/DenseMap to guarantee ↵Igor Laevsky2016-05-041-99/+34
| | | | | | | | | | | | | | | stable ordering Goal of this change is to guarantee stable ordering of the statepoint arguments and other newly inserted values such as gc.relocates. Previously we had explicit sorting in a couple of places. However for unnamed values ordering was partial and overall we didn't have any strong invariant regarding it. This change switches all data structures to use SetVector's and MapVector's which provide possibility for deterministic iteration over them. Explicit sorting is now redundant and was removed. Differential Revision: http://reviews.llvm.org/D19669 llvm-svn: 268502
* Unify XDEBUG and EXPENSIVE_CHECKS (into the latter), and add an option to ↵Filipe Cabecinhas2016-04-291-1/+1
| | | | | | | | | | | | | | | | | | | the cmake build to enable them. Summary: Historically, we had a switch in the Makefiles for turning on "expensive checks". This has never been ported to the cmake build, but the (dead-ish) code is still around. This will also make it easier to turn it on in buildbots. Reviewers: chandlerc Subscribers: jyknight, mzolotukhin, RKSimon, gberry, llvm-commits Differential Revision: http://reviews.llvm.org/D19723 llvm-svn: 268050
* Add parentheses to silence warning.Richard Trieu2016-04-061-1/+2
| | | | llvm-svn: 265516
* [RS4GC] Add a commentSanjoy Das2016-04-061-0/+4
| | | | llvm-svn: 265503
* [RS4GC] NFC cleanup of the DeferredReplacement classSanjoy Das2016-04-051-5/+18
| | | | | | Instead of constructors use clearly named factory methods. llvm-svn: 265486
* [RS4GC] Better codegen for deoptimize callsSanjoy Das2016-04-051-16/+52
| | | | | | | | | Don't emit a gc.result for a statepoint lowered from @llvm.experimental.deoptimize since the call into __llvm_deoptimize is effectively noreturn. Instead follow the corresponding gc.statepoint with an "unreachable". llvm-svn: 265485
* [RS4GC] Lower calls to @llvm.experimental.deoptimizeSanjoy Das2016-03-251-1/+21
| | | | | | | | | | | | | | This changes RS4GC to lower calls to ``@llvm.experimental.deoptimize`` to gc.statepoints wrapping ``__llvm_deoptimize``, and changes ``callsGCLeafFunction`` to recognize ``@llvm.experimental.deoptimize`` as a non GC leaf function. I've had to hard code the ``"__llvm_deoptimize"`` name in RewriteStatepointsForGC; since ``TargetLibraryInfo`` is available only during codegen. This isn't without precedent in the codebase, so I'm not overtly concerned. llvm-svn: 264456
* [Statepoints] Export a magic constant into a header; NFCSanjoy Das2016-03-171-1/+1
| | | | llvm-svn: 263733
* [Statepoints] Separate out logic for statepoint directives; NFCSanjoy Das2016-03-171-12/+8
| | | | | | | | | | | This splits out the logic that maps the `"statepoint-id"` attribute into the actual statepoint ID, and the `"statepoint-num-patch-bytes"` attribute into the number of patchable bytes the statpeoint is lowered into. The new home of this logic is in IR/Statepoint.cpp, and this refactoring will support similar functionality when lowering calls with deopt operand bundles in the future. llvm-svn: 263685
* [RS4GC] "Constant fold" the rs4gc-split-vector-values flagPhilip Reames2016-02-221-156/+0
| | | | | | This flag was part of a migration to a new means of handling vectors-of-points which was described in the llvm-dev thread "FYI: Relocating vector of pointers". The old code path has been off by default for a while without complaints, so time to cleanup. llvm-svn: 261569
* [RS4GC] Revert optimization attempt due to memory corruptionPhilip Reames2016-02-221-63/+3
| | | | | | | | This change reverts "246133 [RewriteStatepointsForGC] Reduce the number of new instructions for base pointers" and a follow on bugfix 12575. As pointed out in pr25846, this code suffers from a memory corruption bug. Since I'm (empirically) not going to get back to this any time soon, simply reverting the problematic change is the right answer. llvm-svn: 261565
* [RS4GC] Pass DenseMap by reference, NFCJoseph Tremoulet2016-02-051-5/+4
| | | | | | | | | | | | | | | Summary: Passing the rematerialized values map to insertRematerializationStores by value looks to be a simple oversight; update it to pass by reference. Reviewers: reames, sanjoy Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D16911 llvm-svn: 259867
* Avoid overly large SmallPtrSet/SmallSetMatthias Braun2016-01-301-1/+1
| | | | | | | These sets perform linear searching in small mode so it is never a good idea to use SmallSize/N bigger than 32. llvm-svn: 259283
* Fix the buildDavid Majnemer2016-01-291-1/+1
| | | | llvm-svn: 259215
* [RS4GC] Address post-commit review on r259208 from DavidSanjoy Das2016-01-291-8/+5
| | | | | | NFC llvm-svn: 259211
* [RS4GC] Remove unnecessary const_cast; NFCSanjoy Das2016-01-291-1/+1
| | | | | | | GCRelocateInst::getDerivedPtr already returns a non-const llvm::Value pointer. llvm-svn: 259209
* [RS4GC] Minor local cleanup to StabilizeOrder; NFCSanjoy Das2016-01-291-21/+20
| | | | | | | | | - Locally declare struct, and call it BaseDerivedPair - Use a lambda to compare, instead of a singleton with uninitialized fields - Add a constructor to BaseDerivedPair and use SmallVector::emplace_back llvm-svn: 259208
* [RS4GC] Minor cleanups enabled by the previous change; NFCSanjoy Das2016-01-291-14/+6
| | | | llvm-svn: 259133
OpenPOWER on IntegriCloud