summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
* [LoopInterchange] Allow some loops with PHI nodes in the exit block.Florian Hahn2018-04-271-23/+48
| | | | | | | | | | | | | | | | | | | | We currently support LCSSA PHI nodes in the outer loop exit, if their incoming values do not come from the outer loop latch or if the outer loop latch has a single predecessor. In that case, the outer loop latch will be executed only if the inner loop gets executed. If we have multiple predecessors for the outer loop latch, it may be executed even if the inner loop does not get executed. This is a first step to support the case described in https://bugs.llvm.org/show_bug.cgi?id=30472 Reviewers: efriedma, karthikthecool, mcrosier Reviewed By: efriedma Differential Revision: https://reviews.llvm.org/D43237 llvm-svn: 331037
* Revert "[SimplifyLibcalls] Replace locked IO with unlocked IO"Matt Morehouse2018-04-272-216/+19
| | | | | | This reverts r331002 due to sanitizer bot breakage. llvm-svn: 331011
* [LowerTypeTests] Mark .cfi.jumptable nounwind.Eli Friedman2018-04-271-0/+2
| | | | | | | | | It doesn't unwind, and the wrong marking leads to the creation of an .eh_frame section when it isn't necessary. Differential Revision: https://reviews.llvm.org/D46082 llvm-svn: 331008
* [SimplifyLibcalls] Replace locked IO with unlocked IODavid Bolvansky2018-04-262-19/+216
| | | | | | | | | | | | Summary: If file stream arg is not captured and source is fopen, we could replace IO calls by unlocked IO ("_unlocked" function variants) to gain better speed, Reviewers: efriedma, RKSimon, spatel, sanjoy, hfinkel, majnemer Subscribers: lebedev.ri, llvm-commits Differential Revision: https://reviews.llvm.org/D45736 llvm-svn: 331002
* [InstCombine] Simplify Add with remainder expressions as operands.Sanjoy Das2018-04-262-0/+116
| | | | | | | | | | | | | | | | | | | | | Summary: Simplify integer add expression X % C0 + (( X / C0 ) % C1) * C0 to X % (C0 * C1). This is a common pattern seen in code generated by the XLA GPU backend. Add test cases for this new optimization. Patch by Bixia Zheng! Reviewers: sanjoy Reviewed By: sanjoy Subscribers: efriedma, craig.topper, lebedev.ri, llvm-commits, jlebar Differential Revision: https://reviews.llvm.org/D45976 llvm-svn: 330992
* Revert "Enable EliminateAvailableExternally pass for -O1"Vlad Tsyrklevich2018-04-261-1/+1
| | | | | | This reverts commit r330961 because it breaks a handful of clang tests. llvm-svn: 330964
* Enable EliminateAvailableExternally pass for -O1Vlad Tsyrklevich2018-04-261-1/+1
| | | | | | | | | | | | | | | | | | Summary: Follow-up to D43690, the EliminateAvailableExternally pass currently runs under -O0 and -O2 and up. Under -O1 we would still want to drop available_externally symbols to reduce space without inlining having run. Reviewers: tejohnson Reviewed By: tejohnson Subscribers: mehdi_amini, llvm-commits, kcc Differential Revision: https://reviews.llvm.org/D46093 llvm-svn: 330961
* [LoopInterchange] Ignore debug intrinsics during legality checks.Florian Hahn2018-04-261-3/+3
| | | | | | | | | | | | Reviewers: aprantl, mcrosier, karthikthecool Reviewed By: aprantl Subscribers: mattd, vsk, #debug-info, llvm-commits Differential Revision: https://reviews.llvm.org/D45379 llvm-svn: 330931
* [SimplifyLibcalls] Atoi, strtol replacementsDavid Bolvansky2018-04-251-0/+55
| | | | | | | | | | | | Reviewers: spatel, lebedev.ri, xbolva00, efriedma Reviewed By: xbolva00, efriedma Subscribers: efriedma, llvm-commits Differential Revision: https://reviews.llvm.org/D45418 llvm-svn: 330860
* [ICP] Do not attempt type matching for variable length arguments.Taewook Oh2018-04-251-5/+7
| | | | | | | | | | | | | | | | | Summary: When performing indirect call promotion, current implementation inspects "all" parameters of the callsite and attemps to match with the formal argument type of the callee function. However, it is not possible to find the type for variable length arguments, and the compiler crashes when it attemps to match the type for variable lenght argument. It seems that the bug is introduced with D40658. Prior to that, the type matching is performed only for the parameters whose ID is less than callee->getFunctionNumParams(). The attached test case will crash without the patch. Reviewers: mssimpso, davidxl, davide Reviewed By: mssimpso Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D46026 llvm-svn: 330844
* [InstCombine] clean up foldSelectICmpAnd(); NFCSanjay Patel2018-04-251-46/+39
| | | | | | | | | | As discussed in D45862, we want to delete parts of this code because it can create more instructions than it removes. But we also want to preserve some folds that are winners, so tidy up what's here to make splitting the good from bad a bit easier. llvm-svn: 330841
* [LoopInterchange] Use getExitBlock()/getExitingBlock instead of manual impl.Florian Hahn2018-04-251-33/+27
| | | | | | | | | | | | | | | This also means we have to check if the latch is the exiting block now, as `transform` expects the latches to be the exiting blocks too. https://bugs.llvm.org/show_bug.cgi?id=36586 Reviewers: efriedma, davide, karthikthecool Reviewed By: efriedma Differential Revision: https://reviews.llvm.org/D45279 llvm-svn: 330806
* [DebugInfo] Invalidate debug info in ReassociatePass::RewriteExprTreeBjorn Pettersson2018-04-251-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: When Reassociate is rewriting an expression tree it may reuse old binary expression nodes, for new expressions. Whenever an expression node is reused, but with a non-trivial change in the result, we need to invalidate any debug info that is associated with the node. If for example rewriting x = mul a, b y = mul c, x into x = mul c, b y = mul a, x we still get the same result for 'y', but 'x' is a new expression. All debug info referring to 'x' must be invalidated (marked as optimized out) since we no longer calculate the expected value. As a side-effect this patch avoid (at least some) problems where reassociate could end up creating IR with debug-use before def. Earlier the dbg.value nodes where left untouched in the IR, while the reused binary nodes where sinked to just before the root node of the rewritten expression tree. See PR27273 for more info about such problems. Reviewers: dblaikie, aprantl, dexonsmith Reviewed By: aprantl Subscribers: JDevlieghere, llvm-commits Tags: #debug-info Differential Revision: https://reviews.llvm.org/D45975 llvm-svn: 330804
* Merging r46043:David Bolvansky2018-04-251-3/+3
| | | | | | ------------------------------------------------------------------------ llvm-svn: 330799
* [DivRemPairs] Fix non-determinism in use list order.Geoff Berry2018-04-251-3/+8
| | | | | | | | | | | | | | | | | Summary: Use a MapVector instead of a DenseMap for RemMap since it is iteratated over and the order of iteration can effect the order that new instructions are created. This can in turn effect the use list order of div/rem input values if multiple new instructions are created that share any input values. Reviewers: spatel Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D45858 llvm-svn: 330792
* [PM/LoopUnswitch] Begin teaching SimpleLoopUnswitch to use the newChandler Carruth2018-04-251-70/+78
| | | | | | | | | | | | | | | | | | | | | | | update API for dominators rather than doing manual, hacky updates. This is just the first step, but in some ways the most important as it moves the non-trivial unswitching to update the domtree rather than fully recalculating it each time. Subsequent patches should remove the custom update logic used by the trivial unswitch and replace it with uses of the update API. This also fixes a number of bugs I was seeing when testing non-trivial unswitch due to it querying the quasi-correct dominator tree. Now the tree is 100% correct and safe to query. That said, there are still more bugs I can see with non-trivial unswitch just running over the test suite, so more bugfix patches are needed as well. Thanks to both Sanjoy and Fedor for reviews and testing! Differential Revision: https://reviews.llvm.org/D45943 llvm-svn: 330787
* [LV][VPlan] Detect outer loops for explicit vectorization.Diego Caballero2018-04-242-29/+394
| | | | | | | | | | | | | | | | | Patch #2 from VPlan Outer Loop Vectorization Patch Series #1 (RFC: http://lists.llvm.org/pipermail/llvm-dev/2017-December/119523.html). This patch introduces the basic infrastructure to detect, legality check and process outer loops annotated with hints for explicit vectorization. All these changes are protected under the feature flag -enable-vplan-native-path. This should make this patch NFC for the existing inner loop vectorizer. Reviewers: hfinkel, mkuper, rengolin, fhahn, aemerson, mssimpso. Differential Revision: https://reviews.llvm.org/D42447 llvm-svn: 330739
* [LoopInterchange] Make isProfitableForVectorization slightly more conservative.Florian Hahn2018-04-241-1/+2
| | | | | | | | | | | | | | | | After D43236, we started interchanging loops with empty dependence matrices. In isProfitableForVectorization, we try to determine if interchanging makes the loop dependences more friendly to the vectorizer. If there are no dependences, we should not interchange, based on that heuristic. Reviewers: efriedma, mcrosier, karthikthecool, blitz.opensource Reviewed By: mcrosier Differential Revision: https://reviews.llvm.org/D45208 llvm-svn: 330738
* Fix some layering in AggressiveInstCombine (avoiding inclusion of Scalar.h)David Blaikie2018-04-243-5/+6
| | | | llvm-svn: 330726
* [LoadStoreVectorize] Ignore interleaved invariant loads.Benjamin Kramer2018-04-241-5/+13
| | | | | | | | | The memory location an invariant load is using can never be clobbered by any store, so it's safe to move the load ahead of the store. Differential Revision: https://reviews.llvm.org/D46011 llvm-svn: 330725
* [PM/LoopUnswitch] Fix a bug in the loop block set formation of the newChandler Carruth2018-04-241-4/+10
| | | | | | | | | | | | | | | | | | loop unswitch. This code incorrectly added the header to the loop block set early. As a consequence we would incorrectly conclude that a nested loop body had already been visited when the header of the outer loop was the preheader of the nested loop. In retrospect, adding the header eagerly doesn't really make sense. It seems nicer to let the cycle be formed naturally. This will catch crazy bugs in the CFG reconstruction where we can't correctly form the cycle earlier rather than later, and makes the rest of the logic just fall out. I've also added various asserts that make these issues *much* easier to debug. llvm-svn: 330707
* [NFC] Remove recently added SE verification because it may be false-positiveMax Kazantsev2018-04-242-12/+0
| | | | llvm-svn: 330699
* [NFC] Use forgetTopmostLoop instead of logic duplicationMax Kazantsev2018-04-241-6/+2
| | | | llvm-svn: 330683
* [PM/LoopUnswitch] Remove another over-aggressive assert.Chandler Carruth2018-04-241-4/+1
| | | | | | | | | | This code path can very clearly be called in a context where we have baselined all the cloned blocks to a particular loop and are trying to handle nested subloops. There is no harm in this, so just relax the assert. I've added a test case that will make sure we actually exercise this code path. llvm-svn: 330680
* [NFC] Add clarification commentMax Kazantsev2018-04-241-2/+4
| | | | llvm-svn: 330677
* InstCombine: Fix layering by not including Scalar.h in InstCombineDavid Blaikie2018-04-243-5/+7
| | | | | | | | (notionally Scalar.h is part of libLLVMScalarOpts, so it shouldn't be included by InstCombine which doesn't/shouldn't need to depend on ScalarOpts) llvm-svn: 330669
* [AggressiveInstCombine] Add aggressive inst combiner to the LLVM C API.Craig Topper2018-04-242-0/+8
| | | | | | I just tried to copy what was done for regular InstCombine. Hopefully I didn't miss anything. llvm-svn: 330668
* [HWASan] Use dynamic shadow memory on Android only (LLVM)Alex Shlyapnikov2018-04-241-4/+2
| | | | | | | | There're issues with IFUNC support on other platforms. DIfferential Revision: https://reviews.llvm.org/D45840 llvm-svn: 330665
* [AggressiveInstCombine] Add library initializer routine for ↵Craig Topper2018-04-241-0/+5
| | | | | | | | AggressiveInstCombine library. Use it in bugpoint and llvm-opt-fuzzer to match regular InstCombine. This should make aggressive instcombine usable with these tools. llvm-svn: 330663
* [LoopInterchange] Do not change LI for BBs in child loops.Florian Hahn2018-04-231-0/+3
| | | | | | | | | | | | | | If a loop with child loops becomes our new inner loop after interchanging, we only need to update LoopInfo for the blocks defined in the old outer loop. BBs in child loops will stay there. Reviewers: efriedma, karthikthecool, mcrosier Reviewed By: efriedma Differential Revision: https://reviews.llvm.org/D45970 llvm-svn: 330653
* [CallSiteSplit] Make sure we remove nonnull if the parameter turns out to be ↵Xin Tong2018-04-231-1/+5
| | | | | | | | | | | | | | a constant. Summary: We do not need nonull attribute if we know an argument is going to be constant. Reviewers: junbuml, davide, fhahn Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45608 llvm-svn: 330641
* [MemCpyOpt] Skip optimizing basic blocks not reachable from entryBjorn Pettersson2018-04-231-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Skip basic blocks not reachable from the entry node in MemCpyOptPass::iterateOnFunction. Code that is unreachable may have properties that do not exist for reachable code (an instruction in a basic block can for example be dominated by a later instruction in the same basic block, for example if there is a single block loop). MemCpyOptPass::processStore is only safe to use for reachable basic blocks, since it may iterate past the basic block beginning when used for unreachable blocks. By simply skipping to optimize unreachable basic blocks we can avoid asserts such as "Assertion `!NodePtr->isKnownSentinel()' failed." in MemCpyOptPass::processStore. The problem was detected by fuzz tests. Reviewers: eli.friedman, dneilson, efriedma Reviewed By: efriedma Subscribers: efriedma, llvm-commits Differential Revision: https://reviews.llvm.org/D45889 llvm-svn: 330635
* [DSE] Teach the pass that atomic memory intrinsics are stores.Daniel Neilson2018-04-231-5/+12
| | | | | | | | | | | | | | | | | | | Summary: This change teaches DSE that the atomic memory intrinsics are stores that can be eliminated, and can allow other stores to be eliminated. This change specifically does not teach DSE that these intrinsics can be partially eliminated (i.e. length reduced, and dest/src changed); that will be handled in another change. Reviewers: mkazantsev, skatkov, apilipenko, efriedma, rsmith Reviewed By: efriedma Subscribers: dmgreen, llvm-commits Differential Revision: https://reviews.llvm.org/D45535 llvm-svn: 330629
* [HWASan] Switch back to fixed shadow mapping for x86-64Alex Shlyapnikov2018-04-231-1/+1
| | | | | | | | | | For now switch back to fixed shadow mapping for x86-64 due to the issues with IFUNC linking on older binutils. More details will be added to https://bugs.chromium.org/p/chromium/issues/detail?id=835864 Differential Revision: https://reviews.llvm.org/D45840 llvm-svn: 330623
* [LoopRotate] Fix incorrect SCEV invalidation in loop rotationMax Kazantsev2018-04-231-2/+9
| | | | | | | | | | | LoopRotate only invalidates innermost loops while the changes that it makes may also affert any of this parents. With patch rL329047, SCEV becomes much smarter about calculation of exit counts for outer loops, so we cannot assume that they are not affected. Differential Revision: https://reviews.llvm.org/D45945 llvm-svn: 330582
* [LoopUnroll] Fix potentially incorrect SCEV invalidation in UnrollRuntimeMax Kazantsev2018-04-231-4/+3
| | | | | | | | | | | | | | | | | Current runtime unrolling invalidates parent loop saying that it might have changed after the inner loop has changed, but it doesn't bother to do the same to its parents. With patch rL329047, SCEV becomes much smarter about calculation of exit counts for outer loops. We might need to invalidate not only the immediate parent, but also any of its parents as well. There is no clear evidence that there is some miscompile happening because of this (at least I don't have such test), but the common sense says that the current code is wrong. Differential Revision: https://reviews.llvm.org/D45940 Reviewed By: chandlerc llvm-svn: 330577
* [LoopSimplify] Fix incorrect SCEV invalidationMax Kazantsev2018-04-231-11/+12
| | | | | | | | | | | | | | | | | | | | | | | | | In the function `simplifyOneLoop` we optimistically assume that changes in the inner loop only affect this very loop and have no impact on its parents. In fact, after rL329047 has been merged, we can now calculate exit counts for outer loops which may depend on inner loops. Thus, we need to invalidate all parents when we do something to a loop. There is an evidence of incorrect behavior of `simplifyOneLoop`: when we insert `SE->verify()` check in the end of this funciton, it fails on a bunch of existing test, in particular: LLVM :: Transforms/LoopUnroll/peel-loop-not-forced.ll LLVM :: Transforms/LoopUnroll/peel-loop-pgo.ll LLVM :: Transforms/LoopUnroll/peel-loop.ll LLVM :: Transforms/LoopUnroll/peel-loop2.ll Note that previously we have fixed issues of this variety, see rL328483. This patch makes this function invalidate the outermost loop properly. Differential Revision: https://reviews.llvm.org/D45937 Reviewed By: chandlerc llvm-svn: 330576
* [PM/LoopUnswitch] Remove a buggy assert in the new loop unswitch.Chandler Carruth2018-04-231-6/+5
| | | | | | | | The condition this was asserting doesn't actually hold. I've added comments to explain why, removed the assert, and added a fun test case reduced out of 403.gcc. llvm-svn: 330564
* [PM/LoopUnswitch] Fix comment typo. NFC.Chandler Carruth2018-04-231-1/+1
| | | | llvm-svn: 330560
* [PatternMatch] allow undef elements when matching a vector zeroSanjay Patel2018-04-221-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | This is the last step in getting constant pattern matchers to allow undef elements in constant vectors. I'm adding a dedicated m_ZeroInt() function and building m_Zero() from that. In most cases, calling code can be updated to use m_ZeroInt() directly when there's no need to match pointers, but I'm leaving that efficiency optimization as a follow-up step because it's not always clear when that's ok. There are just enough icmp folds in InstSimplify that can be used for integer or pointer types, that we probably still want a generic m_Zero() for those cases. Otherwise, we could eliminate it (and possibly add a m_NullPtr() as an alias for isa<ConstantPointerNull>()). We're conservatively returning a full zero vector (zeroinitializer) in InstSimplify/InstCombine on some of these folds (see diffs in InstSimplify), but I'm not sure if that's actually necessary in all cases. We may be able to propagate an undef lane instead. One test where this happens is marked with 'TODO'. llvm-svn: 330550
* [ObjCARC] Take BlockColors by const reference. NFCShoaib Meenai2018-04-202-9/+8
| | | | llvm-svn: 330489
* [ObjCARC] Account for funclet token in storeStrong transformShoaib Meenai2018-04-201-16/+27
| | | | | | | | | | | | | | When creating a call to storeStrong in ObjCARCContract, ensure the call gets the correct funclet token, otherwise WinEHPrepare will turn the call (and all subsequent instructions) into unreachable. We already have logic to do this for the ARC autorelease elision marker; factor that out into a common function that's used for both. These are the only two places in this transform that create call instructions. Differential Revision: https://reviews.llvm.org/D45857 llvm-svn: 330487
* [HWASan] Introduce non-zero based and dynamic shadow memory (LLVM).Alex Shlyapnikov2018-04-201-28/+128
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Support the dynamic shadow memory offset (the default case for user space now) and static non-zero shadow memory offset (-hwasan-mapping-offset option). Keeping the the latter case around for functionality and performance comparison tests (and mostly for -hwasan-mapping-offset=0 case). The implementation is stripped down ASan one, picking only the relevant parts in the following assumptions: shadow scale is fixed, the shadow memory is dynamic, it is accessed via ifunc global, shadow memory address rematerialization is suppressed. Keep zero-based shadow memory for kernel (-hwasan-kernel option) and calls instreumented case (-hwasan-instrument-with-calls option), which essentially means that the generated code is not changed in these cases. Reviewers: eugenis Subscribers: srhines, llvm-commits Differential Revision: https://reviews.llvm.org/D45840 llvm-svn: 330475
* [PartialInlining] Fix Crash from holding a reference to a destructed ORE.Sean Fertile2018-04-201-37/+20
| | | | | | | | | | | | | The callback used to create an ORE for the legacy PI pass caches the allocated object in a unique_ptr in the runOnModule function, and returns a reference to that object. Under certian circumstances we can end up holding onto that reference after the OREs destruction. Rather then allowing the new and legacy passes to create ORE object in diffrent ways, create the ORE at the point of use. Differential Revision: https://reviews.llvm.org/D43219 llvm-svn: 330473
* Revert r330431.Michael Zolotukhin2018-04-201-30/+26
| | | | | | There are still stage3/stage4 miscompares :( llvm-svn: 330446
* [NewGVN] Split OpPHI detection and creation.Florian Hahn2018-04-201-103/+117
| | | | | | | | | | | | | It also adds a check making sure PHIs for operands are all in the same block. Patch by Daniel Berlin <dberlin@dberlin.org> Reviewers: dberlin, davide Differential Revision: https://reviews.llvm.org/D43865 llvm-svn: 330444
* Revert "Revert r330403 and r330413."Michael Zolotukhin2018-04-202-39/+42
| | | | | | | | | | | Reapply the patches with a fix. Thanks Ilya and Hans for the reproducer! This reverts commit r330416. The issue was that removing predecessors invalidated uses that we stored for rewrite. The fix is to finish manipulating with CFG before we select uses for rewrite. llvm-svn: 330431
* Revert r330403 and r330413.Ilya Biryukov2018-04-202-32/+29
| | | | | | | | | | | | | | | | Revert r330413: "[SSAUpdaterBulk] Use SmallVector instead of DenseMap for storing rewrites." Revert r330403 "Reapply "[PR16756] Use SSAUpdaterBulk in JumpThreading." one more time." r330403 commit seems to crash clang during our integrate while doing PGO build with the following stacktrace: #2 llvm::SSAUpdaterBulk::RewriteAllUses(llvm::DominatorTree*, llvm::SmallVectorImpl<llvm::PHINode*>*) #3 llvm::JumpThreadingPass::ThreadEdge(llvm::BasicBlock*, llvm::SmallVectorImpl<llvm::BasicBlock*> const&, llvm::BasicBlock*) #4 llvm::JumpThreadingPass::ProcessThreadableEdges(llvm::Value*, llvm::BasicBlock*, llvm::jumpthreading::ConstantPreference, llvm::Instruction*) #5 llvm::JumpThreadingPass::ProcessBlock(llvm::BasicBlock*) The crash happens while compiling 'lib/Analysis/CallGraph.cpp'. r3340413 is reverted due to conflicting changes. llvm-svn: 330416
* [SSAUpdaterBulk] Use SmallVector instead of DenseMap for storing rewrites.Michael Zolotukhin2018-04-202-16/+13
| | | | llvm-svn: 330413
* Reapply "[PR16756] Use SSAUpdaterBulk in JumpThreading." one more time.Michael Zolotukhin2018-04-201-16/+22
| | | | | | | | | Hopefully, changing set to vector removes nondeterminism detected by some bots, or the new assert will catch something. This reverts commit r330180. llvm-svn: 330403
OpenPOWER on IntegriCloud