summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/Scalar/RewriteStatepointsForGC.cpp
Commit message (Collapse)AuthorAgeFilesLines
* [RewriteStatepointsForGC] Avoid using unrelocated pointers after safepointsPhilip Reames2015-08-121-0/+31
| | | | | | | | | | | | | | | | To be clear: this is an *optimization* not a correctness change. CodeGenPrep likes to duplicate icmps feeding branch instructions to take advantage of x86's ability to fuze many comparison/branch patterns into a single micro-op and to reduce the need for materializing i1s into general registers. PlaceSafepoints likes to place safepoint polls right at the end of basic blocks (immediately before terminators) when inserting entry and backedge safepoints. These two heuristics interact in a somewhat unfortunate way where the branch terminating the original block will be controlled by a condition driven by unrelocated pointers. This forces the register allocator to keep both the relocated and unrelocated values of the pointers feeding the icmp alive over the safepoint poll. One simple fix would have been to just adjust PlaceSafepoints to move one back in the basic block, but you can reach similar cases as a result of LICM or other hoisting passes. As a result, doing a post insertion fixup seems to be more robust. I considered doing this in CodeGenPrep itself, but having to update the live sets of already rewritten safepoints gets complicated fast. In particular, you can't just use def/use information because by moving the icmp, we're extending the live range of it's inputs potentially. Instead, this patch teaches RewriteStatepointsForGC to make the required adjustments before making the relocations explicit in the IR. This change really highlights the fact that RSForGC is a CodeGenPrep-like pass which is performing target specific lowering. In the long run, we may even want to combine the two though this would require a lot more smarts to be integrated into RSForGC first. We currently rely on being able to run a set of cleanup passes post rewriting because the IR RSForGC generates is pretty damn ugly. Differential Revision: http://reviews.llvm.org/D11819 llvm-svn: 244821
* [RewriteStatepointsForGC] Handle extractelement fully in the base pointer ↵Philip Reames2015-08-121-61/+96
| | | | | | | | | | algorithm When rewriting the IR such that base pointers are available for every live pointer, we potentially need to duplicate instructions to propagate the base. The original code had only handled PHI and Select under the belief those were the only instructions which would need duplicated. When I added support for vector instructions, I'd added a collection of hacks for ExtractElement which caught most of the common cases. Of course, I then found the one test case my hacks couldn't cover. :) This change removes all of the early hacks for extract element. By defining extractelement as a BDV (rather than trying to look through it), we can extend the rewriting algorithm to duplicate the extract as needed. Note that a couple of peephole optimizations were left in for the moment, because while we now handle extractelement as a first class citizen, we're not yet handling insertelement. That change will follow in the near future. llvm-svn: 244808
* Fix some comment typos.Benjamin Kramer2015-08-081-19/+19
| | | | llvm-svn: 244402
* Rename inst_range() to instructions() for consistency. NFCNico Rieck2015-08-061-2/+2
| | | | llvm-svn: 244248
* De-constify pointers to Type since they can't be modified. NFCCraig Topper2015-08-011-2/+2
| | | | | | This was already done in most places a while ago. This just fixes the ones that crept in over time. llvm-svn: 243842
* [RewriteStatepointsForGC] Adjust naming scheme to be more stablePhilip Reames2015-07-241-3/+7
| | | | | | The names for instructions inserted were previous dependent on iteration order. By deriving the names from the original instructions, we can avoid instability in tests without resorting to ordered traversals. It also makes the IR mildly easier to read at large scale. llvm-svn: 243140
* [RewriteStatepointsForGC] Fix release build warningPhilip Reames2015-07-241-0/+2
| | | | llvm-svn: 243076
* [RewriteStatepointsForGC] Use a worklist algorithm for first part of base ↵Philip Reames2015-07-241-36/+39
| | | | | | | | | | pointer algorithm [NFC] The new code should hopefully be equivalent to the old code; it just uses a worklist to track instructions which need to visited rather than iterating over all instructions visited each time. This should be faster, but the primary benefit is that the purpose should be more clear and the diff of adding another instruction type (forthcoming) much more obvious. Differential Revision: http://reviews.llvm.org/D11480 llvm-svn: 243071
* [RewriteStatepointsForGC] Rename PhiState to reflect that it's associated ↵Philip Reames2015-07-231-41/+43
| | | | | | | | w/more than just PHIs Today, Select instructions also have associated PhiStates. In the near future, so will ExtractElement and SuffleVector. llvm-svn: 243056
* [RewriteStatepointsForGC] Use idomatic mechanisms for debug tracing [NFC]Philip Reames2015-07-231-21/+22
| | | | | | Deleting much of the code using trace-rewrite-statepoints and use idiomatic DEBUG statements instead. This includes adding operator<< to a helper class. llvm-svn: 243054
* [RewriteStatepointsForGC] Simplify code around meet of PhiStates [NFC]Philip Reames2015-07-231-33/+34
| | | | | | We don't need to pass in the map from BDV to PhiStates; we can instead handle that externally and let the MeetPhiStates helper class just meet PhiStates. llvm-svn: 243045
* [PM/AA] Remove all of the dead AliasAnalysis pointers being threadedChandler Carruth2015-07-221-1/+1
| | | | | | | | | | through APIs that are no longer necessary now that the update API has been removed. This will make changes to the AA interfaces significantly less disruptive (I hope). Either way, it seems like a really nice cleanup. llvm-svn: 242882
* [RewriteStatepointsForGC] minor style cleanupPhilip Reames2015-07-211-26/+25
| | | | | | Use a named lambda for readability, common some code, remove a stale comments, and use llvm style variable names. llvm-svn: 242827
* [RewriteStatepointsForGC] Hoist some code out of a loopPhilip Reames2015-07-211-12/+13
| | | | llvm-svn: 242808
* [RewriteStatepointsForGC] Delete trivial codePhilip Reames2015-07-211-13/+5
| | | | | | A bit more code cleanup: delete some a trivial true assertion and supporting code, remove a redundant cast, and use count in assertions where feasible. llvm-svn: 242805
* [RewriteStatepointsForGC] Minor code cleanup [NFC]Philip Reames2015-07-211-21/+5
| | | | | | We can use builders to simplify part of the code and we only check for the existance of the metadata value; this enables us to delete some redundant code. llvm-svn: 242751
* [RewriteStatepointsForGC] Generalized vector phi/select handling for base ↵Philip Reames2015-06-261-55/+124
| | | | | | | | | | | | | | | pointers This change extends the detection of base pointers for vector constructs to handle arbitrary phi and select nodes. The existing non-vector code already handles those, so this is basically just extending the vector special case to be less special cased. It still isn't generalized vector handling since we can't handle arbitrary vector instructions (e.g. shufflevectors), but it's a lot closer. The general structure of the change is as follows: * Extend the base defining value relation over a subset of vector instructions and vector typed phi & select instructions. * Move scalarization from before base pointer rewriting to after base pointer rewriting. The extension of the BDV relation is sufficient to find vector base phis for vector inputs. * Preserve the existing special case logic for when the base of a vector element is locally obvious. This general idea could be extended to the scalar case as well. Differential Revision: http://reviews.llvm.org/D10461#inline-84275 llvm-svn: 240850
* Revert r240137 (Fixed/added namespace ending comments using clang-tidy. NFC)Alexander Kornienko2015-06-231-2/+2
| | | | | | Apparently, the style needs to be agreed upon first. llvm-svn: 240390
* Fixed/added namespace ending comments using clang-tidy. NFCAlexander Kornienko2015-06-191-2/+2
| | | | | | | | | | | | | The patch is generated using this command: tools/clang/tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \ -checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \ llvm/lib/ Thanks to Eugene Kosov for the original patch! llvm-svn: 240137
* [Statepoints] Skip a vector copy when uniquing values.Benjamin Kramer2015-06-131-11/+4
| | | | | | No functionality change intended. llvm-svn: 239688
* Remove stray semicolon. NFC.Vasileios Kalintiris2015-06-031-1/+1
| | | | llvm-svn: 238908
* [RewriteStatepointsForGC] Strip deref info after rewriting.Sanjoy Das2015-06-021-0/+102
| | | | | | | | | | | | | | | | | | | | | Summary: Once a gc.statepoint has been rewritten to relocate live references, the SSA values represent physical pointers instead of logical references. Logical dereferencability does not imply physical dereferencability and after RewriteStatepointsForGC has run any attributes that imply dereferencability of the logical references need to be stripped. This current approach is conservative, and can be made more precise later if needed. For starters, we need to strip dereferencable attributes only from pointers that live in the GC address space. Reviewers: reames, pgavlin Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D10105 llvm-svn: 238883
* [NFCI] Change RewriteStatepointsForGC to a ModulePass.Sanjoy Das2015-06-021-13/+16
| | | | | | | | | | | | | | Summary: A later change that has RewriteStatepointsForGC change function attributes throughout the module depends on this. Reviewers: reames, pgavlin Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D10104 llvm-svn: 238882
* Reformat.NAKAMURA Takumi2015-05-251-3/+2
| | | | llvm-svn: 238126
* Prune CRLFs.NAKAMURA Takumi2015-05-251-9/+9
| | | | llvm-svn: 238125
* [RewriteStatepointsForGC] Fix debug assertion during derivable pointer ↵Igor Laevsky2015-05-211-6/+6
| | | | | | | | | | rematerialization Correct assertion would be that there is no other uses from chain we are currently cloning. It is ok to have other uses of values not from this chain. Differential Revision: http://reviews.llvm.org/D9882 llvm-svn: 237899
* Silencing a -Wsign-compare warning; NFC.Aaron Ballman2015-05-201-1/+1
| | | | llvm-svn: 237794
* Add a GCStrategy for CoreCLRSwaroop Sridhar2015-05-201-3/+8
| | | | | | | | | | | | | | This change adds a new GC strategy for supporting the CoreCLR runtime. This strategy is currently identical to Statepoint-example GC, but is necessary for several upcoming changes specific to CoreCLR, such as: 1. Base-pointers not explicitly reported for interior pointers 2. Different format for stack-map encoding 3. Location of Safe-point polls: polls are only needed before loop-back edges and before tail-calls (not needed at function-entry) 4. Runtime specific handshake between calls to managed/unmanaged functions. llvm-svn: 237753
* [RewriteStatepointsForGC] Fix up naming in "relocationViaAlloca" and run it ↵Igor Laevsky2015-05-191-56/+56
| | | | | | | | through clang-format. Differential Revision: http://reviews.llvm.org/D9774 llvm-svn: 237703
* [RewriteStatepointsForGC] For some values (like gep's and bitcasts) it's ↵Igor Laevsky2015-05-191-6/+278
| | | | | | | | cheaper to clone them after statepoint than to emit proper relocates for them. This change implements this logic. There is alredy similar optimization in CodeGenPrepare, but doing so during RewriteStatepointsForGC allows to capture more opprtunities such as relocates in loops and longer instruction chains. Differential Revision: http://reviews.llvm.org/D9774 llvm-svn: 237701
* Simplify IRBuilder::CreateCall* by using ArrayRef+initializer_list/braced ↵David Blaikie2015-05-181-2/+2
| | | | | | init only llvm-svn: 237624
* [RewriteStatepointsForGC] Extend base pointer to handle more cases w/vectorsPhilip Reames2015-05-121-9/+36
| | | | | | | | | | When relocating a pointer, we need to determine a base pointer for the derived pointer being relocated. We have limited support for handling a pointer extracted from a vector; the current code only handled the case where the entire vector was known to contain base pointers. This patch extends the reasoning to handle chains of insertelements where the indices are constants. This case turns out to be fairly common in vectorized code. We can now handle vectors which contains mixtures of base and derived pointers provided the insertelements use constant indices. Note that this doesn't solve the general problem. To handle variable indexed insertelements, we'd need to scalarize and introduce conditional branching based on the index. Alternatively, we could eagerly scalarize, but the code structure doesn't currently make either fix easy. The patch also doesn't handle shufflevector or other vector manipulation for much the same reasons. I plan to defer this work until I have a motivating test case. Differential Revision: http://reviews.llvm.org/D9676 llvm-svn: 237200
* Rename variables in gc_relocate related functions to follow LLVM's naming ↵Sanjoy Das2015-05-111-39/+39
| | | | | | | | | | | | | | | | | | | | conventions. Summary: This patch is to rename some variables to CamelCase in gc_relocate related functions. There is no functionality change. Patch by Chen Li! Reviewers: reames, AndyAyers, sanjoy Reviewed By: sanjoy Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D9681 llvm-svn: 237069
* [RewriteStatepointsForGC] Fix a bug on creating gc_relocate for pointer to ↵Sanjoy Das2015-05-111-3/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | vector of pointers Summary: In RewriteStatepointsForGC pass, we create a gc_relocate intrinsic for each relocated pointer, and the gc_relocate has the same type with the pointer. During the creation of gc_relocate intrinsic, llvm requires to mangle its type. However, llvm does not support mangling of all possible types. RewriteStatepointsForGC will hit an assertion failure when it tries to create a gc_relocate for pointer to vector of pointers because mangling for vector of pointers is not supported. This patch changes the way RewriteStatepointsForGC pass creates gc_relocate. For each relocated pointer, we erase the type of pointers and create an unified gc_relocate of type i8 addrspace(1)*. Then a bitcast is inserted to convert the gc_relocate to the correct type. In this way, gc_relocate does not need to deal with different types of pointers and the unsupported type mangling is no longer a problem. This change would also ease further merge when LLVM erases types of pointers and introduces an unified pointer type. Some minor changes are also introduced to gc_relocate related part in InstCombineCalls, CodeGenPrepare, and Verifier accordingly. Patch by Chen Li! Reviewers: reames, AndyAyers, sanjoy Reviewed By: sanjoy Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D9592 llvm-svn: 237009
* [Statepoint] Clean up Statepoint.h: accessor names.Sanjoy Das2015-05-061-1/+2
| | | | | | Use getFoo() as accessors consistently and some other naming changes. llvm-svn: 236564
* [RewriteStatepointsForGC] Exclude constant values from being considered live ↵Philip Reames2015-04-261-14/+13
| | | | | | | | | | | | at a safepoint There can be various constant pointers in the IR which do not get relocated at a safepoint. One example is the address of a global variable. Another example is a pointer created via inttoptr. Note that the optimizer itself likes to create such inttoptrs when locally propagating constants through dynamically dead code. To deal with this, we need to exclude uses of constants from contributing to the liveness of a safepoint which might reach that use. At some later date, it might be worth exploring what could be done to support the relocation of various special types of "constants", but that's future work. Differential Revision: http://reviews.llvm.org/D9236 llvm-svn: 235821
* [RewriteStatepointsForGC] Delete dead code [NFC]Philip Reames2015-04-141-26/+4
| | | | | | | | | Before we had real liveness, we needed to track every value that base pointer insertion code created because these now might be live. We now just rerun the data flow liveness algorithm (which is actually faster!) and no longer need the associated code. llvm-svn: 234827
* [RwriteStatepointsForGC] Minor indentation and naming [NFC]Philip Reames2015-04-131-39/+30
| | | | | | Use early-return style that's preferred in LLVM and updating the naming in places I touched with other changes in the last few days. Hopefully, NFC. llvm-svn: 234785
* [RewriteStatepointsForGC] Avoid inserting empty holderPhilip Reames2015-04-131-0/+4
| | | | | | We use dummy calls to adjust the liveness of values over statepoints in the midst of the insertion. If there are no values which need held live, there's no point in actually inserting the holder. llvm-svn: 234779
* [RewriteStatepointsForGC] Fix a latent bug in normalization for invoke ↵Philip Reames2015-04-131-37/+40
| | | | | | | | | | statepoint [NFC] Since we're restructuring the CFG, we also need to make sure to update the analsis passes. While I'm touching the code, I dedicided to restructure it a bit. The code involved here was very confusing. This change moves the normalization to essentially being a pre-pass before the main insertion work and updates a few comments to actually say what is happening and *why*. The restructuring should be covered by existing tests. I couldn't easily see how to create a test for the invalidation bug. Suggestions welcome. llvm-svn: 234769
* [RewriteStatepointsForGC] Strengthen assertions around livenessPhilip Reames2015-04-131-0/+18
| | | | | | This is related to the issues addressed in 234651. These assertions check the properties ensured by that change at the place of use. Note that a similiar property is checked in checkBasicSSA, but without the reachability constraint. Technically, the liveness would be correct to include unreachable values, but this would be problematic for actual relocation. llvm-svn: 234766
* [RewriteStatepointsForGC] Move an expensive debugging check to XDEBUGPhilip Reames2015-04-131-33/+44
| | | | | | | | | | The check in question is attempting to help find cases where we haven't relocated a pointer at a safepoint we should have. It does this by coercing the value to null at any safepoint which doesn't relocate it. Unfortunately, this turns out to be rather expensive in terms of memory usage and time. The number of stores inserted can grow with O(number of values x number of statepoints). On at least one example I looked at, over half of peak memory usage was coming from this check. With this change, the check is no longer enabled by default in Asserts builds. It is enabled for expensive asserts builds and has a command line option to enable it in both Asserts and non-Asserts builds. llvm-svn: 234761
* [Statepoints] Fix a release only build failurePhilip Reames2015-04-111-2/+2
| | | | | | A function which is used only in Asserts builds needs to be defined only in Asserts builds. llvm-svn: 234667
* [RewriteStatepointsForGC] Use a SetVector for a worklist [NFC]Philip Reames2015-04-101-6/+4
| | | | | | Using a SetVector to replace equivelent but more verbose functionality. llvm-svn: 234662
* [RewriteStatepointsForGC] Use an actual liveness algorithmPhilip Reames2015-04-101-204/+294
| | | | | | | | When rewriting statepoints to make relocations explicit, we need to have a conservative but consistent notion of where a particular pointer is live at a particular site. The old code just used dominance, which is correct, but decidedly more conservative then it needed to be. This patch implements a simple dataflow algorithm that's run one per function (well, twice counting fixup after base pointer insertion). There's still lots of room to make this faster, but it's fast enough for all practical purposes today. Differential Revision: http://reviews.llvm.org/D8674 llvm-svn: 234657
* [RewriteStatepointsForGC] clang-format filePhilip Reames2015-04-101-58/+57
| | | | | | Format the entire file to reduce diff of change to follow. llvm-svn: 234656
* [RewriteStatepointsForGC] Missed review comment from 234651 & build fixPhilip Reames2015-04-101-3/+4
| | | | | | After submitting 234651, I noticed I hadn't responded to a review comment by mjacob. This patch addresses that comment and fixes a Release only build problem due to an unused variable. llvm-svn: 234653
* [RewriteStatepointsForGC] Preprocess the IR to remove unreachable blocks and ↵Philip Reames2015-04-101-6/+34
| | | | | | | | | | | | | | | single entry phis Two related small changes: Various dominance based queries about liveness can get confused if we're talking about unreachable blocks. To avoid reasoning about such cases, just remove them before rewriting statepoints. Remove single entry phis (likely left behind by LCSSA) to reduce the number of live values. Both of these are motivated by http://reviews.llvm.org/D8674 which will be submitted shortly. Differential Revision: http://reviews.llvm.org/D8675 llvm-svn: 234651
* [RewriteStatepointsForGC] Limited support for vectors of pointersPhilip Reames2015-04-101-25/+224
| | | | | | | | | | | | This patch adds limited support for inserting explicit relocations when there's a vector of pointers live over the statepoint. This doesn't handle the case where the vector contains a mix of base and non-base pointers; that's future work. The current implementation just scalarizes the vector over the gc.statepoint before doing the explicit rewrite. An alternate approach would be to plumb the vector all the way though the backend lowering, but doing that appears challenging. In particular, the size of the indirect spill slot is currently assumed to be sizeof(pointer) throughout the backend. In practice, this is enough to allow running the SLP and Loop vectorizers before RewriteStatepointsForGC. Differential Revision: http://reviews.llvm.org/D8671 llvm-svn: 234647
* Code cleanup [NFC]Philip Reames2015-03-271-14/+12
| | | | | | The assertion here was more expensive then it needed to be. We're only inserting allocas in the entry block, so we only need to consider ones in the entry block. llvm-svn: 233362
OpenPOWER on IntegriCloud