summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/Scalar/RewriteStatepointsForGC.cpp
Commit message (Collapse)AuthorAgeFilesLines
* [RS4GC] Look through vector bitcasts when looking for base pointerDaniel Neilson2017-10-131-0/+5
| | | | | | | | | | | | | | | | | | Summary: In RS4GC it is possible that a base pointer is contained in a vector that has undergone a bitcast from one element-pointertype to another. We teach RS4GC how to look through bitcasts of vector types when looking for a base pointer. Reviewers: anna Reviewed By: anna Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D38849 llvm-svn: 315694
* [Analysis, Transforms] Fix some Clang-tidy modernize and Include What You ↵Eugene Zelenko2017-09-011-34/+70
| | | | | | Use warnings; other minor fixes (NFC). llvm-svn: 312383
* All libcalls should be considered to be GC-leaf functions.Daniel Neilson2017-07-271-2/+6
| | | | | | | | | | | | | | | | | | Summary: It is possible for some passes to materialize a call to a libcall (ex: ldexp, exp2, etc), but these passes will not mark the call as a gc-leaf-function. All libcalls are actually gc-leaf-functions, so we change llvm::callsGCLeafFunction() to tell us that available libcalls are equivalent to gc-leaf-function calls. Reviewers: sanjoy, anna, reames Reviewed By: anna Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D35840 llvm-svn: 309291
* fix trivial typos; NFCHiroshi Inoue2017-07-021-1/+1
| | | | | | suport -> support llvm-svn: 306968
* fix trivial typos, NFCHiroshi Inoue2017-07-011-1/+1
| | | | llvm-svn: 306952
* [RS4GC] Drop invalid metadata after pointers are relocatedAnna Thomas2017-06-121-17/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: After RS4GC, we should drop metadata that is no longer valid. These metadata is used by optimizations scheduled after RS4GC, and can cause a miscompile. One such metadata is invariant.load which is used by LICM sinking transform. After rewriting statepoints, the address of a load maybe relocated. With invariant.load metadata on a load instruction, LICM sinking assumes the loaded value (from a dererenceable address) to be invariant, and rematerializes the load operand and the load at the exit block. This transforms the IR to have an unrelocated use of the address after a statepoint, which is incorrect. Other metadata we conservatively remove are related to dereferenceability and noalias metadata. This patch drops such metadata on store and load instructions after rewriting statepoints. Reviewers: reames, sanjoy, apilipenko Reviewed by: reames Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D33756 llvm-svn: 305234
* Sort the remaining #include lines in include/... and lib/....Chandler Carruth2017-06-061-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | | I did this a long time ago with a janky python script, but now clang-format has built-in support for this. I fed clang-format every line with a #include and let it re-sort things according to the precise LLVM rules for include ordering baked into clang-format these days. I've reverted a number of files where the results of sorting includes isn't healthy. Either places where we have legacy code relying on particular include ordering (where possible, I'll fix these separately) or where we have particular formatting around #include lines that I didn't want to disturb in this patch. This patch is *entirely* mechanical. If you get merge conflicts or anything, just ignore the changes in this patch and run clang-format over your #include lines in the files. Sorry for any noise here, but it is important to keep these things stable. I was seeing an increasing number of patches with irrelevant re-ordering of #include lines because clang-format was used. This patch at least isolates that churn, makes it easy to skip when resolving conflicts, and gets us to a clean baseline (again). llvm-svn: 304787
* [Statepoint] Be consistent about using deopt naming [NFCI]Philip Reames2017-06-021-3/+3
| | | | | | We'd called this "vm state" in the early days, but have long since standardized on calling it "deopt" in line with the operand bundle tag. Fix a few cases we'd missed. llvm-svn: 304607
* [RS4GC] Comment clarificationPhilip Reames2017-06-021-2/+2
| | | | llvm-svn: 304514
* [IR] Abstract away ArgNo+1 attribute indexing as much as possibleReid Kleckner2017-05-031-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Do three things to help with that: - Add AttributeList::FirstArgIndex, which is an enumerator currently set to 1. It allows us to change the indexing scheme with fewer changes. - Add addParamAttr/removeParamAttr. This just shortens addAttribute call sites that would otherwise need to spell out FirstArgIndex. - Remove some attribute-specific getters and setters from Function that take attribute list indices. Most of these were only used from BuildLibCalls, and doesNotAlias was only used to test or set if the return value is malloc-like. I'm happy to split the patch, but I think they are probably easier to review when taken together. This patch should be NFC, but it sets the stage to change the indexing scheme to this, which is more convenient when indexing into an array: 0: func attrs 1: retattrs 2...: arg attrs Reviewers: chandlerc, pete, javed.absar Subscribers: david2050, llvm-commits Differential Revision: https://reviews.llvm.org/D32811 llvm-svn: 302060
* Re-land r301697 "[IR] Make add/remove Attributes use AttrBuilder instead of ↵Reid Kleckner2017-05-021-2/+1
| | | | | | | | | | AttributeList" This time, I fixed, built, and tested clang. This reverts r301712. llvm-svn: 301981
* Revert r301697 "[IR] Make add/remove Attributes use AttrBuilder instead of ↵Hans Wennborg2017-04-281-1/+2
| | | | | | | | | | | | | | | | | | AttributeList" This broke the Clang build. (Clang-side patch missing?) Original commit message: > [IR] Make add/remove Attributes use AttrBuilder instead of > AttributeList > > This change cleans up call sites and avoids creating temporary > AttributeList objects. > > NFC llvm-svn: 301712
* [IR] Make add/remove Attributes use AttrBuilder instead of AttributeListReid Kleckner2017-04-281-2/+1
| | | | | | | | | This change cleans up call sites and avoids creating temporary AttributeList objects. NFC llvm-svn: 301697
* [RS4GC] Simplify attribute handling code NFCReid Kleckner2017-04-281-47/+25
| | | | | | | Avoids use of AttributeList::getNumSlots, making it easier to change the underlying implementation. llvm-svn: 301671
* [SystemZ] TargetTransformInfo cost functions implemented.Jonas Paulsson2017-04-121-1/+1
| | | | | | | | | | | | | | | | getArithmeticInstrCost(), getShuffleCost(), getCastInstrCost(), getCmpSelInstrCost(), getVectorInstrCost(), getMemoryOpCost(), getInterleavedMemoryOpCost() implemented. Interleaved access vectorization enabled. BasicTTIImpl::getCastInstrCost() improved to check for legal extending loads, in which case the cost of the z/sext instruction becomes 0. Review: Ulrich Weigand, Renato Golin. https://reviews.llvm.org/D29631 llvm-svn: 300052
* Reland "[IR] Make AttributeSetNode public, avoid temporary AttributeList copies"Reid Kleckner2017-04-101-6/+9
| | | | | | | | | | | | | | | | | | | | | | | | | This re-lands r299875. I introduced a bug in Clang code responsible for replacing K&R, no prototype declarations with a real function definition with a prototype. The bug was here: // Collect any return attributes from the call. - if (oldAttrs.hasAttributes(llvm::AttributeList::ReturnIndex)) - newAttrs.push_back(llvm::AttributeList::get(newFn->getContext(), - oldAttrs.getRetAttributes())); + newAttrs.push_back(oldAttrs.getRetAttributes()); Previously getRetAttributes() carried AttributeList::ReturnIndex in its AttributeList. Now that we return the AttributeSetNode* directly, it no longer carries that index, and we call this overload with a single node: AttributeList::get(LLVMContext&, ArrayRef<AttributeSetNode*>) That aborted with an assertion on x86_32 targets. I added an explicit triple to the test and added CHECKs to help find issues like this in the future sooner. llvm-svn: 299899
* Allow DataLayout to specify addrspace for allocas.Matt Arsenault2017-04-101-1/+3
| | | | | | | | | | | | | | | | | | | | | | | LLVM makes several assumptions about address space 0. However, alloca is presently constrained to always return this address space. There's no real way to avoid using alloca, so without this there is no way to opt out of these assumptions. The problematic assumptions include: - That the pointer size used for the stack is the same size as the code size pointer, which is also the maximum sized pointer. - That 0 is an invalid, non-dereferencable pointer value. These are problems for AMDGPU because alloca is used to implement the private address space, which uses a 32-bit index as the pointer value. Other pointers are 64-bit and behave more like LLVM's notion of generic address space. By changing the address space used for allocas, we can change our generic pointer type to be LLVM's generic pointer type which does have similar properties. llvm-svn: 299888
* Revert "[IR] Make AttributeSetNode public, avoid temporary AttributeList copies"Reid Kleckner2017-04-101-9/+6
| | | | | | | This reverts r299875. A Linux bot came back with a test failure: http://bb.pgr.jp/builders/test-clang-i686-linux-RA/builds/741/steps/test_clang/logs/Clang%20%3A%3A%20CodeGen__2006-05-19-SingleEltReturn.c llvm-svn: 299878
* [IR] Make AttributeSetNode public, avoid temporary AttributeList copiesReid Kleckner2017-04-101-6/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: AttributeList::get(Fn|Ret|Param)Attributes no longer creates a temporary AttributeList just to hide the AttributeSetNode type. I've also added a factory method to create AttributeLists from a parallel array of AttributeSetNodes. I think this simplifies construction of AttributeLists when rewriting function prototypes. Previously we would test if a particular index had attributes, and conditionally add a temporary attribute list to a vector. Now the attribute set vector is parallel to the argument vector already that these passes already construct. My long term vision is to wrap AttributeSetNode* inside an AttributeSet type that holds the enum attributes, but that will come in a follow up change. I haven't done any performance measurements for this change because profiling hasn't shown that any of the affected code is hot. Reviewers: pete, chandlerc, sanjoy, hfinkel Reviewed By: pete Subscribers: jfb, llvm-commits Differential Revision: https://reviews.llvm.org/D31198 llvm-svn: 299875
* Rename AttributeSet to AttributeListReid Kleckner2017-03-211-17/+16
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: This class is a list of AttributeSetNodes corresponding the function prototype of a call or function declaration. This class used to be called ParamAttrListPtr, then AttrListPtr, then AttributeSet. It is typically accessed by parameter and return value index, so "AttributeList" seems like a more intuitive name. Rename AttributeSetImpl to AttributeListImpl to follow suit. It's useful to rename this class so that we can rename AttributeSetNode to AttributeSet later. AttributeSet is the set of attributes that apply to a single function, argument, or return value. Reviewers: sanjoy, javed.absar, chandlerc, pete Reviewed By: pete Subscribers: pete, jholewinski, arsenm, dschuff, mehdi_amini, jfb, nhaehnle, sbc100, void, llvm-commits Differential Revision: https://reviews.llvm.org/D31102 llvm-svn: 298393
* [RSForGC] Handle vector GEPsSanjoy Das2017-03-171-0/+5
| | | | | | | | | We were not handling getelemenptr instructions of vector type before. Since getelemenptr instructions for vector types follow the same rule as getelementptr instructions for non-vector types, we can just handle them in the same way. llvm-svn: 298028
* Apply clang-tidy's performance-unnecessary-value-param to LLVM.Benjamin Kramer2017-01-131-1/+1
| | | | | | | With some minor manual fixes for using function_ref instead of std::function. No functional change intended. llvm-svn: 291904
* Revert @llvm.assume with operator bundles (r289755-r289757)Daniel Jasper2016-12-191-1/+0
| | | | | | | This creates non-linear behavior in the inliner (see more details in r289755's commit thread). llvm-svn: 290086
* Remove the AssumptionCacheHal Finkel2016-12-151-0/+1
| | | | | | | | | After r289755, the AssumptionCache is no longer needed. Variables affected by assumptions are now found by using the new operand-bundle-based scheme. This new scheme is more computationally efficient, and also we need much less code... llvm-svn: 289756
* Replace some callers of setTailCall with setTailCallKindDavid Majnemer2016-11-251-1/+1
| | | | | | | We were a little sloppy with adding tailcall markers. Be more consistent by using setTailCallKind instead of setTailCall. llvm-svn: 287955
* [RS4GC] Fix comment to show TODO. NFCAnna Thomas2016-10-061-1/+1
| | | | llvm-svn: 283449
* [RS4GC] Handle ShuffleVector instruction in findBasePointerAnna Thomas2016-10-041-10/+32
| | | | | | | | | | | | | | | Summary: This patch modifies the findBasePointer to handle the shufflevector instruction. Tests run: RS4GC tests, local downstream tests. Reviewers: reames, sanjoy Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D25197 llvm-svn: 283219
* [RS4GC] Remat in presence of phi and use live valueAnna Thomas2016-09-221-8/+21
| | | | | | | | | | Summary: Reviewers: Subscribers: llvm-svn: 282150
* [RS4GC] Refactor code for Rematerializing in presence of phi. NFCAnna Thomas2016-09-201-59/+68
| | | | | | | | | | | | | | | | | | | | | | | | Summary: This is an NFC refactoring change as a precursor to the actual fix for rematerializing in presence of phi. https://reviews.llvm.org/D24399 Pasted from review: findRematerializableChainToBasePointer changed to return the root of the chain. instead of true or false. move the PHI matching logic into the caller by inspecting the root return value. This includes an assertion that the alternate root is in the liveset for the call. Tested with current RS4GC tests. Reviewers: reames, sanjoy Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D24780 llvm-svn: 282023
* [statepoints][experimental] Add support for live-in semantics of values in ↵Philip Reames2016-08-311-0/+26
| | | | | | | | | | | | | | | | deopt bundles This is a first step towards supporting deopt value lowering and reporting entirely with the register allocator. I hope to build on this in the near future to support live-on-return semantics, but I have a use case which allows me to test and investigate code quality with just the live-in semantics so I've chosen to start there. For those curious, my use cases is our implementation of the "__llvm_deoptimize" function we bind to @llvm.deoptimize. I'm choosing not to hard code that fact in the patch and instead make it configurable via function attributes. The basic approach here is modelled on what is done for the "Live In" values on stackmaps and patchpoints. (A secondary goal here is to remove one of the last barriers to merging the pseudo instructions.) We start by adding the operands directly to the STATEPOINT SDNode. Once we've lowered to MI, we extend the remat logic used by the register allocator to fold virtual register uses into StackMap::Indirect entries as needed. This does rely on the fact that the register allocator rematerializes. If it didn't along some code path, we could end up with more vregs than physical registers and fail to allocate. Today, we *only* fold in the register allocator. This can create some weird effects when combined with arguments passed on the stack because we don't fold them appropriately. I have an idea how to fix that, but it needs this patch in place to work on that effectively. (There's some weird interaction with the scheduler as well, more investigation needed.) My near term plan is to land this patch off-by-default, experiment in my local tree to identify any correctness issues and then start fixing codegen problems one by one as I find them. Once I have the live-in lowering fully working (both correctness and code quality), I'm hoping to move on to the live-on-return semantics. Note: I don't have any *known* miscompiles with this patch enabled, but I'm pretty sure I'll find at least a couple. Thus, the "experimental" tag and the fact it's off by default. Differential Revision: https://reviews.llvm.org/D24000 llvm-svn: 280250
* [RewriteStatepointsForGC] Update comment for same PHI node check. NFCAnna Thomas2016-08-301-1/+2
| | | | llvm-svn: 280052
* ADT: Give ilist<T>::reverse_iterator a handle to the current nodeDuncan P. N. Exon Smith2016-08-301-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Reverse iterators to doubly-linked lists can be simpler (and cheaper) than std::reverse_iterator. Make it so. In particular, change ilist<T>::reverse_iterator so that it is *never* invalidated unless the node it references is deleted. This matches the guarantees of ilist<T>::iterator. (Note: MachineBasicBlock::iterator is *not* an ilist iterator, but a MachineInstrBundleIterator<MachineInstr>. This commit does not change MachineBasicBlock::reverse_iterator, but it does update MachineBasicBlock::reverse_instr_iterator. See note at end of commit message for details on bundle iterators.) Given the list (with the Sentinel showing twice for simplicity): [Sentinel] <-> A <-> B <-> [Sentinel] the following is now true: 1. begin() represents A. 2. begin() holds the pointer for A. 3. end() represents [Sentinel]. 4. end() holds the poitner for [Sentinel]. 5. rbegin() represents B. 6. rbegin() holds the pointer for B. 7. rend() represents [Sentinel]. 8. rend() holds the pointer for [Sentinel]. The changes are #6 and #8. Here are some properties from the old scheme (which used std::reverse_iterator): - rbegin() held the pointer for [Sentinel] and rend() held the pointer for A; - operator*() cost two dereferences instead of one; - converting from a valid iterator to its valid reverse_iterator involved a confusing increment; and - "RI++->erase()" left RI invalid. The unintuitive replacement was "RI->erase(), RE = end()". With vector-like data structures these properties are hard to avoid (since past-the-beginning is not a valid pointer), and don't impose a real cost (since there's still only one dereference, and all iterators are invalidated on erase). But with lists, this was a poor design. Specifically, the following code (which obviously works with normal iterators) now works with ilist::reverse_iterator as well: for (auto RI = L.rbegin(), RE = L.rend(); RI != RE;) fooThatMightRemoveArgFromList(*RI++); Converting between iterator and reverse_iterator for the same node uses the getReverse() function. reverse_iterator iterator::getReverse(); iterator reverse_iterator::getReverse(); Why doesn't iterator <=> reverse_iterator conversion use constructors? In order to catch and update old code, reverse_iterator does not even have an explicit conversion from iterator. It wouldn't be safe because there would be no reasonable way to catch all the bugs from the changed semantic (see the changes at call sites that are part of this patch). Old code used this API: std::reverse_iterator::reverse_iterator(iterator); iterator std::reverse_iterator::base(); Here's how to update from old code to new (that incorporates the semantic change), assuming I is an ilist<>::iterator and RI is an ilist<>::reverse_iterator: [Old] ==> [New] reverse_iterator(I) (--I).getReverse() reverse_iterator(I) ++I.getReverse() --reverse_iterator(I) I.getReverse() reverse_iterator(++I) I.getReverse() RI.base() (--RI).getReverse() RI.base() ++RI.getReverse() --RI.base() RI.getReverse() (++RI).base() RI.getReverse() delete &*RI, RE = end() delete &*RI++ RI->erase(), RE = end() RI++->erase() ======================================= Note: bundle iterators are out of scope ======================================= MachineBasicBlock::iterator, also known as MachineInstrBundleIterator<MachineInstr>, is a wrapper to represent MachineInstr bundles. The idea is that each operator++ takes you to the beginning of the next bundle. Implementing a sane reverse iterator for this is harder than ilist. Here are the options: - Use std::reverse_iterator<MBB::i>. Store a handle to the beginning of the next bundle. A call to operator*() runs a loop (usually operator--() will be called 1 time, for unbundled instructions). Increment/decrement just works. This is the status quo. - Store a handle to the final node in the bundle. A call to operator*() still runs a loop, but it iterates one time fewer (usually operator--() will be called 0 times, for unbundled instructions). Increment/decrement just works. - Make the ilist_sentinel<MachineInstr> *always* store that it's the sentinel (instead of just in asserts mode). Then the bundle iterator can sniff the sentinel bit in operator++(). I initially tried implementing the end() option as part of this commit, but updating iterator/reverse_iterator conversion call sites was error-prone. I have a WIP series of patches that implements the final option. llvm-svn: 280032
* [StatepointsForGC] Rematerialize in the presence of PHIsAnna Thomas2016-08-291-0/+35
| | | | | | | | | | | | | | | | Summary: While walking the use chain for identifying rematerializable values in RS4GC, add the case where the current value and base value are the same PHI nodes. This will aid rematerialization of geps and casts instead of relocating. Reviewers: sanjoy, reames, igor Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D23920 llvm-svn: 279975
* Use the range variant of remove_if instead of unpacking begin/endDavid Majnemer2016-08-121-3/+2
| | | | | | No functionality change is intended. llvm-svn: 278475
* Use the range variant of find instead of unpacking begin/endDavid Majnemer2016-08-111-3/+2
| | | | | | | | | If the result of the find is only used to compare against end(), just use is_contained instead. No functionality change is intended. llvm-svn: 278433
* Use range algorithms instead of unpacking begin/endDavid Majnemer2016-08-111-2/+1
| | | | | | No functionality change is intended. llvm-svn: 278417
* Apply clang-tidy's modernize-loop-convert to most of lib/Transforms.Benjamin Kramer2016-06-261-3/+2
| | | | | | Only minor manual fixes. No functionality change intended. llvm-svn: 273808
* [RSForGC] Appease MSVCSanjoy Das2016-06-261-2/+4
| | | | llvm-svn: 273805
* [RSForGC] Bring the BDVState struct up to code; NFCSanjoy Das2016-06-261-25/+33
| | | | llvm-svn: 273800
* [RSForGC] Bring computeLiveInValues up to code; NFCSanjoy Das2016-06-261-8/+5
| | | | llvm-svn: 273799
* [RSForGC] Bring computeLiveOutSeed up to code; NFCSanjoy Das2016-06-261-7/+7
| | | | llvm-svn: 273798
* [RSForGC] Bring computeLiveInValues up to code; NFCSanjoy Das2016-06-261-19/+8
| | | | llvm-svn: 273797
* [RSForGC] Bring recomputeLiveInValues up to code; NFCSanjoy Das2016-06-261-9/+9
| | | | llvm-svn: 273796
* [RSForGC] Bring containsGCPtrType, isGCPointerType up to code; NFCSanjoy Das2016-06-261-3/+2
| | | | llvm-svn: 273795
* [RSForGC] Bring analyzeParsePointLiveness up to code; NFCSanjoy Das2016-06-261-7/+7
| | | | llvm-svn: 273794
* [RSForGC] Bring meetBDVStateImpl up to code; NFCSanjoy Das2016-06-261-14/+13
| | | | llvm-svn: 273793
* [RSForGC] Get rid of the unnecessary MeetBDVStates struct; NFCSanjoy Das2016-06-261-58/+36
| | | | | | All of its implementation is in just one function. llvm-svn: 273792
* [RSForGC] Bring findBasePointer up to code; NFCSanjoy Das2016-06-261-110/+92
| | | | | | | Name-casing and minor style changes to bring the function up to the LLVM coding style. llvm-svn: 273791
* Fix unused variable warning by folding the temporary into the debug statement.Eric Christopher2016-06-231-2/+2
| | | | llvm-svn: 273523
* [RS4GC] Use StringRef; NFCSanjoy Das2016-06-221-4/+3
| | | | | | Spotted during random inspection. llvm-svn: 273512
OpenPOWER on IntegriCloud