summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/Scalar
Commit message (Collapse)AuthorAgeFilesLines
...
* Revert [SROA] Enhance SROA to handle `addrspacecast`ed allocasJordan Rupprecht2019-06-181-6/+0
| | | | | | | | | | This reverts r363711 (git commit 76a149ef8187310a60fd20481fdb2a10c8ba968e) This causes stage2 build failures, e.g.: http://lab.llvm.org:8011/builders/clang-x64-windows-msvc/builds/132/steps/stage%202%20build/logs/stdio http://lab.llvm.org:8011/builders/ppc64le-lld-multistage-test/builds/87/steps/build-stage2-unified-tree/logs/stdio llvm-svn: 363718
* [SROA] Enhance SROA to handle `addrspacecast`ed allocasMichael Liao2019-06-181-0/+6
| | | | | | | | | | | | | | | | | | Summary: - After `addrspacecast` is allowed to be eliminated in SROA, the adjusting of storage pointer (from `alloca) needs to handle the potential different address spaces between the storage pointer (from alloca) and the pointer being used. Reviewers: arsenm Subscribers: wdng, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D63501 llvm-svn: 363711
* Teach getSCEVAtScope how to handle loop phis w/invariant operands in loops ↵Philip Reames2019-06-171-2/+5
| | | | | | | | | | | | w/taken backedges This patch really contains two pieces: Teach SCEV how to fold a phi in the header of a loop to the value on the backedge when a) the backedge is known to execute at least once, and b) the value is safe to use globally within the scope dominated by the original phi. Teach IndVarSimplify's rewriteLoopExitValues to allow loop invariant expressions which already exist (and thus don't need new computation inserted) even in loops where we can't optimize away other uses. Differential Revision: https://reviews.llvm.org/D63224 llvm-svn: 363619
* Fix a bug w/inbounds invalidation in LFTR (recommit)Philip Reames2019-06-171-11/+85
| | | | | | | | | | | | | | | | | | Recommit r363289 with a bug fix for crash identified in pr42279. Issue was that a loop exit test does not have to be an icmp, leading to a null dereference crash when new logic was exercised for that case. Test case previously committed in r363601. Original commit comment follows: This contains fixes for two cases where we might invalidate inbounds and leave it stale in the IR (a miscompile). Case 1 is when switching to an IV with no dynamically live uses, and case 2 is when doing pre-to-post conversion on the same pointer type IV. The basic scheme used is to prove that using the given IV (pre or post increment forms) would have to already trigger UB on the path to the test we're modifying. As such, our potential UB triggering use does not change the semantics of the original program. As was pointed out in the review thread by Nikita, this is defending against a separate issue from the hasConcreteDef case. This is about poison, that's about undef. Unfortunately, the two are different, see Nikita's comment for a fuller explanation, he explains it well. (Note: I'm going to address Nikita's last style comment in a separate commit just to minimize chance of subtle bugs being introduced due to typos.) Differential Revision: https://reviews.llvm.org/D62939 llvm-svn: 363613
* [EarlyCSE] Fix hashing of self-comparesJoseph Tremoulet2019-06-171-2/+7
| | | | | | | | | | | | | | | | | | | | | | | Summary: Update compare normalization in SimpleValue hashing to break ties (when the same value is being compared to itself) by switching to the swapped predicate if it has a lower numerical value. This brings the hashing in line with isEqual, which already recognizes the self-compares with swapped predicates as equal. Fixes PR 42280. Reviewers: spatel, efriedma, nikic, fhahn, uabelho Reviewed By: nikic Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D63349 llvm-svn: 363598
* PHINode: introduce setIncomingValueForBlock() function, and use it.Whitney Tsang2019-06-174-12/+6
| | | | | | | | | | | | | | | | Summary: There is PHINode::getBasicBlockIndex() and PHINode::setIncomingValue() but no function to replace incoming value for a specified BasicBlock* predecessor. Clearly, there are a lot of places that could use that functionality. Reviewer: craig.topper, lebedev.ri, Meinersbur, kbarton, fhahn Reviewed By: Meinersbur, fhahn Subscribers: fhahn, hiraditya, zzheng, jsji, llvm-commits Tag: LLVM Differential Revision: https://reviews.llvm.org/D63338 llvm-svn: 363566
* InferAddressSpaces: Fix cloning original addrspacecastMatt Arsenault2019-06-171-2/+6
| | | | | | | | If an addrspacecast needed to be inserted again, this was creating a clone of the original cast for each user. Just use the original, which also saves losing the value name. llvm-svn: 363562
* SROA: Allow eliminating addrspacecasted allocasMatt Arsenault2019-06-141-8/+43
| | | | | | | | | | | | | | | | | | | There is a circular dependency between SROA and InferAddressSpaces today that requires running both multiple times in order to be able to eliminate all simple allocas and addrspacecasts. InferAddressSpaces can't remove addrspacecasts when written to memory, and SROA helps move pointers out of memory. This should avoid inserting new commuting addrspacecasts with GEPs, since there are unresolved questions about pointer wrapping between different address spaces. For now, don't replace volatile operations that don't match the alloca addrspace, as it would change the address space of the access. It may be still OK to insert an addrspacecast from the new alloca, but be more conservative for now. llvm-svn: 363462
* Revert Fix a bug w/inbounds invalidation in LFTRFlorian Hahn2019-06-141-83/+11
| | | | | | | | | Reverting because it breaks a green dragon build: http://green.lab.llvm.org/green/job/clang-stage2-Rthinlto/18208 This reverts r363289 (git commit eb88badff96dacef8fce3f003dec34c2ef6900bf) llvm-svn: 363427
* Revert [LFTR] Stylistic cleanup as suggested in last review comment of ↵Florian Hahn2019-06-141-9/+9
| | | | | | | | | | | D62939 [NFC] Reverting because it depends on r363289, which breaks a green dragon build: http://green.lab.llvm.org/green/job/clang-stage2-Rthinlto/18208 This reverts r363292 (git commit 42a3fc133d3544b5c0c032fe99c6e8a469a836c2) llvm-svn: 363426
* Revert [LFTR] Rename variable to minimize confusion [NFC]Florian Hahn2019-06-141-15/+18
| | | | | | | | | | Reverting because it depends on r363289, which breaks a green dragon build: http://green.lab.llvm.org/green/job/clang-stage2-Rthinlto/18208 This reverts r363293 (git commit c37be29634214fb1cb4c823840bffc31e5ebfe40) llvm-svn: 363425
* [LFTR] Rename variable to minimize confusion [NFC]Philip Reames2019-06-131-18/+15
| | | | | | As pointed out by Nikita in D62625, BackedgeTakenCount is generally used to refer to the backedge taken count of the loop. A conditional backedge taken count - one which only applies if a particular exit is taken - is called a ExitCount in SCEV code, so be consistent here. llvm-svn: 363293
* [LFTR] Stylistic cleanup as suggested in last review comment of D62939 [NFC]Philip Reames2019-06-131-9/+9
| | | | llvm-svn: 363292
* Fix a bug w/inbounds invalidation in LFTRPhilip Reames2019-06-131-11/+83
| | | | | | | | | | | | | | This contains fixes for two cases where we might invalidate inbounds and leave it stale in the IR (a miscompile). Case 1 is when switching to an IV with no dynamically live uses, and case 2 is when doing pre-to-post conversion on the same pointer type IV. The basic scheme used is to prove that using the given IV (pre or post increment forms) would have to already trigger UB on the path to the test we're modifying. As such, our potential UB triggering use does not change the semantics of the original program. As was pointed out in the review thread by Nikita, this is defending against a separate issue from the hasConcreteDef case. This is about poison, that's about undef. Unfortunately, the two are different, see Nikita's comment for a fuller explanation, he explains it well. (Note: I'm going to address Nikita's last style comment in a separate commit just to minimize chance of subtle bugs being introduced due to typos.) Differential Revision: https://reviews.llvm.org/D62939 llvm-svn: 363289
* [EarlyCSE] Ensure equal keys have the same hash valueJoseph Tremoulet2019-06-131-64/+110
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The logic in EarlyCSE that looks through 'not' operations in the predicate recognizes e.g. that `select (not (cmp sgt X, Y)), X, Y` is equivalent to `select (cmp sgt X, Y), Y, X`. Without this change, however, only the latter is recognized as a form of `smin X, Y`, so the two expressions receive different hash codes. This leads to missed optimization opportunities when the quadratic probing for the two hashes doesn't happen to collide, and assertion failures when probing doesn't collide on insertion but does collide on a subsequent table grow operation. This change inverts the order of some of the pattern matching, checking first for the optional `not` and then for the min/max/abs patterns, so that e.g. both expressions above are recognized as a form of `smin X, Y`. It also adds an assertion to isEqual verifying that it implies equal hash codes; this fires when there's a collision during insertion, not just grow, and so will make it easier to notice if these functions fall out of sync again. A new flag --earlycse-debug-hash is added which can be used when changing the hash function; it forces hash collisions so that any pair of values inserted which compare as equal but hash differently will be caught by the isEqual assertion. Reviewers: spatel, nikic Reviewed By: spatel, nikic Subscribers: lebedev.ri, arsenm, craig.topper, efriedma, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62644 llvm-svn: 363274
* [IndVars] Extend diagnostic -replexitval flag w/ability to bypass hard use ↵Philip Reames2019-06-121-2/+5
| | | | | | | hueristic Note: This does mean that "always" is now more powerful than it was. llvm-svn: 363196
* LoopLoadElim: Respect convergentMatt Arsenault2019-06-121-0/+6
| | | | llvm-svn: 363162
* LoopDistribute/LAA: Respect convergentMatt Arsenault2019-06-121-1/+14
| | | | | | | | | | | | | | | | | | This case is slightly tricky, because loop distribution should be allowed in some cases, and not others. As long as runtime dependency checks don't need to be introduced, this should be OK. This is further complicated by the fact that LoopDistribute partially ignores if LAA says that vectorization is safe, and then does its own runtime pointer legality checks. Note this pass still does not handle noduplicate correctly, as this should always be forbidden with it. I'm not going to bother trying to fix it, as it would require more effort and I think noduplicate should be removed. https://reviews.llvm.org/D62607 llvm-svn: 363160
* Only passes that preserve MemorySSA must mark it as preserved.Alina Sbirlea2019-06-115-3/+17
| | | | | | | | | | | | | | | | | | | | Summary: The method `getLoopPassPreservedAnalyses` should not mark MemorySSA as preserved, because it's being called in a lot of passes that do not preserve MemorySSA. Instead, mark the MemorySSA analysis as preserved by each pass that does preserve it. These changes only affect the new pass mananger. Reviewers: chandlerc Subscribers: mehdi_amini, jlebar, Prazek, george.burgess.iv, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62536 llvm-svn: 363091
* [LFTR] Use recomputed BE countPhilip Reames2019-06-101-9/+1
| | | | | | | | | | | | This was discussed as part of D62880. The basic thought is that computing BE taken count after widening should produce (on average) an equally good backedge taken count as the one before widening. Since there's only one test in the suite which is impacted by this change, and it's essentially equivelent codegen, that seems to be a reasonable assertion. This change was separated from r362971 so that if this turns out to be problematic, the triggering piece is obvious and easily revertable. For the nestedIV example from elim-extend.ll, we end up with the following BE counts: BEFORE: (-2 + (-1 * %innercount) + %limit) AFTER: (-1 + (sext i32 (-1 + %limit) to i64) + (-1 * (sext i32 %innercount to i64))<nsw>) Note that before is an i32 type, and the after is an i64. Truncating the i64 produces the i32. llvm-svn: 362975
* Prepare for multi-exit LFTR [NFC]Philip Reames2019-06-101-65/+77
| | | | | | | | | | | | | | | | This change does the plumbing to wire an ExitingBB parameter through the LFTR implementation, and reorganizes the code to work in terms of a set of individual loop exits. Most of it is fairly obvious, but there's one key complexity which makes it worthy of consideration. The actual multi-exit LFTR patch is in D62625 for context. Specifically, it turns out the existing code uses the backedge taken count from before a IV is widened. Oddly, we can end up with a different (more expensive, but semantically equivelent) BE count for the loop when requerying after widening. For the nestedIV example from elim-extend, we end up with the following BE counts: BEFORE: (-2 + (-1 * %innercount) + %limit) AFTER: (-1 + (sext i32 (-1 + %limit) to i64) + (-1 * (sext i32 %innercount to i64))<nsw>) This is the only test in tree which seems sensitive to this difference. The actual result of using the wider BETC on this example is that we actually produce slightly better code. :) In review, we decided to accept that test change. This patch is structured to preserve the old behavior, but a separate change will immediate follow with the behavior change. (I wanted it separate for problem attribution purposes.) Differential Revision: https://reviews.llvm.org/D62880 llvm-svn: 362971
* [GVN] non-functional code movementKeno Fischer2019-06-071-12/+8
| | | | | | | | | | | | Summary: Move some code around, in preparation for later fixes to the non-integral addrspace handling (D59661) Patch By Jameson Nash <jameson@juliacomputing.com> Reviewed By: reames, loladiro Differential Revision: https://reviews.llvm.org/D59729 llvm-svn: 362853
* [LoopPred] Fix a bug in unconditional latch bailout introduced in r362284Philip Reames2019-06-061-2/+2
| | | | | | This is a really silly bug that even a simple test w/an unconditional latch would have caught. I tried to guard against the case, but put it in the wrong if check. Oops. llvm-svn: 362727
* NewGVN: Handle addrspacecastMatt Arsenault2019-06-051-2/+3
| | | | | | | | | | The AllConstant check needs to be moved out of the if/else if chain to avoid a test regression. The "there is no SimplifyZExt" comment puzzles me, since there is SimplifyCastInst. Additionally, the Simplify* calls seem to not see the operand as constant, so this needs to be tried if the simplify failed. llvm-svn: 362653
* Resubmit "[CorrelatedValuePropagation] Fix prof branch_weights metadata ↵Yevgeny Rouban2019-06-051-56/+61
| | | | | | | | | | | | | | | | | handling for SwitchInst" This reverts commit 5b32f60ec31ce136edac6f693538aeb6039f4ad0. The fix is in commit 4f9e68148bd0dada2d6997625432385918ac2e2c. This patch fixes the CorrelatedValuePropagation pass to keep prof branch_weights metadata of SwitchInst consistent. It makes use of SwitchInstProfUpdateWrapper. New tests are added. Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D62126 llvm-svn: 362583
* [Scalarizer] Add UnaryOperator visitor to scalarization passCameron McInally2019-06-041-0/+38
| | | | | | Differential Revision: https://reviews.llvm.org/D62858 llvm-svn: 362558
* [SCCP] Add UnaryOperator visitor to SCCP for unary FNegCameron McInally2019-06-031-0/+26
| | | | | | Differential Revision: https://reviews.llvm.org/D62819 llvm-svn: 362449
* [LoopPred] Convert a second member function to a static helper [NFC]Philip Reames2019-06-031-14/+15
| | | | | | (And remember to actually mark the first one static.) llvm-svn: 362415
* [LoopPred] Convert member function to free helper function [NFC]Philip Reames2019-06-031-43/+47
| | | | llvm-svn: 362411
* [IndVarSimplify] Fixup nowrap flags during LFTR (PR31181)Nikita Popov2019-06-011-0/+21
| | | | | | | | | | | | | | | | | | | | | | | | Fix for https://bugs.llvm.org/show_bug.cgi?id=31181 and partial fix for LFTR poison handling issues in general. When LFTR moves a condition from pre-inc to post-inc, it may now depend on value that is poison due to nowrap flags. To avoid this, we clear any nowrap flag that SCEV cannot prove for the post-inc addrec. Additionally, LFTR may switch to a different IV that is dynamically dead and as such may be arbitrarily poison. This patch will correct nowrap flags in some but not all cases where this happens. This is related to the adoption of IR nowrap flags for the pre-inc addrec. (See some of the switch_to_different_iv tests, where flags are not dropped or insufficiently dropped.) Finally, there are likely similar issues with the handling of GEP inbounds, but we don't have a test case for this yet. Differential Revision: https://reviews.llvm.org/D60935 llvm-svn: 362292
* Inline variable into assert to fix unused variable warning.Richard Trieu2019-06-011-3/+3
| | | | llvm-svn: 362285
* [LoopPred] Eliminate a redundant/confusing cover function [NFC]Philip Reames2019-06-011-19/+20
| | | | llvm-svn: 362284
* [LoopPred] Handle a subset of NE comparison based latchesPhilip Reames2019-06-011-21/+33
| | | | | | | | | | | | | | | | | | | | | At the moment, LoopPredication completely bails out if it sees a latch of the form: %cmp = icmp ne %iv, %N br i1 %cmp, label %loop, label %exit OR %cmp = icmp ne %iv.next, %NPlus1 br i1 %cmp, label %loop, label %exit This is unfortunate since this is exactly the form that LFTR likes to produce. So, go ahead and recognize simple cases where we can. For pre-increment loops, we leverage the fact that LFTR likes canonical counters (i.e. those starting at zero) and a (presumed) range fact on RHS to discharge the check trivially. For post-increment forms, the key insight is in remembering that LFTR had to insert a (N+1) for the RHS. CVP can hopefully prove that add nsw/nuw (if there's appropriate range on N to start with). This leaves us both with the post-inc IV and the RHS involving an nsw/nuw add, and SCEV can discharge that with no problem. This does still need to be extended to handle non-one steps, or other harder patterns of variable (but range restricted) starting values. That'll come later. Differential Revision: https://reviews.llvm.org/D62748 llvm-svn: 362282
* Reapply [CVP] Simplify non-overflowing saturating add/subNikita Popov2019-05-311-1/+24
| | | | | | | | | | | | | If we can determine that a saturating add/sub will not overflow based on range analysis, convert it into a simple binary operation. This is a sibling transform to the existing with.overflow handling. Reapplying this with an additional check that the saturating intrinsic has integer type, as LVI currently does not support vector types. Differential Revision: https://reviews.llvm.org/D62703 llvm-svn: 362263
* [CVP] Fix assertion failure on vector with.overflowNikita Popov2019-05-311-1/+1
| | | | | | | | Noticed on D62703. LVI only handles plain integers, not vectors of integers. This was previously not an issue, because vector support for with.overflow is only a relatively recent addition. llvm-svn: 362261
* Revert "[CVP] Simplify non-overflowing saturating add/sub"Nikita Popov2019-05-311-24/+1
| | | | | | | | This reverts commit 1e692d1777ae34dcb93524b5798651a29defae09. Causes assertion failure in builtins-wasm.c clang test. llvm-svn: 362254
* [CVP] Simplify non-overflowing saturating add/subNikita Popov2019-05-311-1/+24
| | | | | | | | | | If we can determine that a saturating add/sub will not overflow based on range analysis, convert it into a simple binary operation. This is a sibling transform to the existing with.overflow handling. Differential Revision: https://reviews.llvm.org/D62703 llvm-svn: 362242
* [CVP] Generalize willNotOverflow(); NFCNikita Popov2019-05-301-5/+5
| | | | | | | Change argument from WithOverflowInst to BinaryOpIntrinsic, so this function can also be used for saturating math intrinsics. llvm-svn: 362152
* [LoopIdiom] Basic OptimizationRemarkEmitter handlingRoman Lebedev2019-05-301-4/+40
| | | | | | | | | | | | | | | | | | Summary: I'm adding ORE to memset/memcpy formation, with tests, but mainly this is split off from D61144. Reviewers: reames, anemet, thegameg, craig.topper Reviewed By: thegameg Subscribers: llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62631 llvm-svn: 362092
* [LoopIdiomRecognize][NFC] Sort includesRoman Lebedev2019-05-301-2/+2
| | | | | | Split off from D61144 llvm-svn: 362091
* LoopVersioningLICM: Respect convergent and noduplicateMatt Arsenault2019-05-291-1/+9
| | | | llvm-svn: 362031
* [LoopIdiomRecognize][NFC] Use DEBUG_TYPE, add LLVM_DEBUG() to ↵Roman Lebedev2019-05-291-2/+8
| | | | | | | | runOnNoncountableLoop() Split off from D61144 llvm-svn: 362022
* CallSiteSplitting: Respect convergent and noduplicateMatt Arsenault2019-05-291-0/+3
| | | | llvm-svn: 361990
* SpeculateAroundPHIs: Respect convergentMatt Arsenault2019-05-291-0/+8
| | | | llvm-svn: 361957
* Revert "[CorrelatedValuePropagation] Fix prof branch_weights metadata ↵Nikita Popov2019-05-281-61/+56
| | | | | | | | | | | | handling for SwitchInst" This reverts commit 53f2f3286572cb879b3861d7c15480e4d830dd3b. As reported on D62126, this causes assertion failures if the switch has incorrect branch_weights metadata, which may happen as a result of other transforms not handling it correctly yet. llvm-svn: 361881
* [CorrelatedValuePropagation] Fix prof branch_weights metadata handling for ↵Yevgeny Rouban2019-05-281-56/+61
| | | | | | | | | | | | | | SwitchInst This patch fixes the CorrelatedValuePropagation pass to keep prof branch_weights metadata of SwitchInst consistent. It makes use of SwitchInstProfUpdateWrapper. New tests are added. Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D62126 llvm-svn: 361808
* [LoopInterchange] Fix handling of LCSSA nodes defined in headers and latches.Florian Hahn2019-05-261-22/+64
| | | | | | | | | | | | | | | | | | The code to preserve LCSSA PHIs currently only properly supports reduction PHIs and PHIs for values defined outside the latches. This patch improves the LCSSA PHI handling to cover PHIs for values defined in the latches. Fixes PR41725. Reviewers: efriedma, mcrosier, davide, jdoerfert Reviewed By: jdoerfert Differential Revision: https://reviews.llvm.org/D61576 llvm-svn: 361743
* [CVP] Remove unnecessary checks for empty GNWR; NFCNikita Popov2019-05-251-27/+11
| | | | | | | | | | The guaranteed no-wrap region is never empty, it always contains at least zero, so these optimizations don't ever apply. To make this more obviously true, replace the conversative return in makeGNWR with an assertion. llvm-svn: 361698
* Use the DataLayout::typeSizeEqualsStoreSize helper. NFCBjorn Pettersson2019-05-242-6/+3
| | | | | | | | | Just a minor refactoring to use the new helper method DataLayout::typeSizeEqualsStoreSize(). This is done when checking if getTypeSizeInBits is equal/non-equal to getTypeStoreSizeInBits. llvm-svn: 361613
* StructurizeCFG: Relax uniformity checks.Neil Henning2019-05-241-3/+30
| | | | | | | | | | | | | | | This change relaxes the checks for hasOnlyUniformBranches such that our region is uniform if: 1. All conditional branches that are direct children are uniform. 2. And either: a. All sub-regions are uniform. b. There is one or less conditional branches among the direct children. Differential Revision: https://reviews.llvm.org/D62198 llvm-svn: 361610
OpenPOWER on IntegriCloud