summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
* [InstCombine] use m_APInt to allow icmp eq (op X, Y), C folds for splat ↵Sanjay Patel2016-08-041-11/+0
| | | | | | | | | constant vectors I'm removing a misplaced pair of more specific folds from InstCombine in this patch as well, so we know where those folds are happening in InstSimplify. llvm-svn: 277738
* LoadStoreVectorizer: Remove TargetBaseAlign. Keep alignment for stack ↵Alina Sbirlea2016-08-041-11/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | adjustments. Summary: TargetBaseAlign is no longer required since LSV checks if target allows misaligned accesses. A constant defining a base alignment is still needed for stack accesses where alignment can be adjusted. Previous patch (D22936) was reverted because tests were failing. This patch also fixes the cause of those failures: - x86 failing tests either did not have the right target, or the right alignment. - NVPTX failing tests did not have the right alignment. - AMDGPU failing test (merge-stores) should allow vectorization with the given alignment but the target info considers <3xi32> a non-standard type and gives up early. This patch removes the condition and only checks for a maximum size allowed and relies on the next condition checking for %4 for correctness. This should be revisited to include 3xi32 as a MVT type (on arsenm's non-immediate todo list). Note that checking the sizeInBits for a MVT is undefined (leads to an assertion failure), so we need to create an EVT, hence the interface change in allowsMisaligned to include the Context. Reviewers: arsenm, jlebar, tstellarAMD Subscribers: jholewinski, arsenm, mzolotukhin, llvm-commits Differential Revision: https://reviews.llvm.org/D23068 llvm-svn: 277735
* [InstCombine] use m_APInt to allow icmp eq (sub C1, X), C2 folds for splat ↵Sanjay Patel2016-08-041-4/+4
| | | | | | constant vectors llvm-svn: 277731
* Add popcount(n) == bitsize(n) -> n == -1 transformation.Amaury Sechet2016-08-041-4/+10
| | | | | | | | | | | | Summary: As per title. Reviewers: majnemer, spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D23139 llvm-svn: 277694
* Forgot the dyn_cast_or_null intended for r277691.David Majnemer2016-08-041-1/+1
| | | | llvm-svn: 277693
* Reinstate "[CloneFunction] Don't remove side effecting calls"David Majnemer2016-08-041-2/+33
| | | | | | | This reinstates r277611 + r277614 and reverts r277642. A cast_or_null should have been a dyn_cast_or_null. llvm-svn: 277691
* Revert "GVN-hoist: enable by default" & "Make GVN Hoisting obey optnone/bisect."Bruno Cardoso Lopes2016-08-041-2/+2
| | | | | | | | This reverts commits r277685 & r277688. r277685 broke compiler-rt compilation http://lab.llvm.org:8080/green/job/clang-stage1-configure-RA_build/23335 and r277685 is a followup from it. llvm-svn: 277690
* GVN-hoist: enable by defaultSebastian Pop2016-08-041-2/+2
| | | | | | | | | | As we addressed all compilation time problems with GVN-hoist https://llvm.org/bugs/show_bug.cgi?id=28670 this patch turns GVN-hoist back by default. Differential Revision: https://reviews.llvm.org/D23136 llvm-svn: 277685
* [InstCombine] use m_APInt to allow icmp eq (add X, C1), C2 folds for splat ↵Sanjay Patel2016-08-031-6/+8
| | | | | | constant vectors llvm-svn: 277659
* [MSSA] Fix a bug in MemorySSA's move ctor.George Burgess IV2016-08-031-0/+2
| | | | | | | Not a correctness issue, but it would be nice if we didn't have to recompute our block numbering (worst-case) every time we move MSSA. llvm-svn: 277652
* GVN-hoist: limit the length of dependent instructionsSebastian Pop2016-08-031-0/+10
| | | | | | | | | | Limit the number of times the while(1) loop is executed. With this restriction the number of hoisted instructions does not change in a significant way on the test-suite. Differential Revision: https://reviews.llvm.org/D23028 llvm-svn: 277651
* GVN-hoist: compute DFS numbers onceSebastian Pop2016-08-031-12/+22
| | | | | | | | | With this patch we compute the DFS numbers of instructions only once and update them during the code generation when an instruction gets hoisted. Differential Revision: https://reviews.llvm.org/D23021 llvm-svn: 277650
* GVN-hoist: compute MSSA once per function (PR28670)Sebastian Pop2016-08-031-12/+49
| | | | | | | | With this patch we compute the MemorySSA once and update it in the code generator. Differential Revision: https://reviews.llvm.org/D22966 llvm-svn: 277649
* Revert "[CloneFunction] Don't remove side effecting calls"Reid Kleckner2016-08-031-33/+2
| | | | | | | | | This reverts commit r277611 and the followup r277614. Bootstrap builds and chromium builds are crashing during inlining after this change. llvm-svn: 277642
* [MSSA] clang-format. NFC.George Burgess IV2016-08-031-8/+4
| | | | | | | Didn't want to fold this in with r277640, since it touches bits that aren't entirely related to r277640. llvm-svn: 277641
* [MSSA] Add special handling for invariant/constant loads.George Burgess IV2016-08-031-0/+23
| | | | | | | This is a follow-up to r277637. It teaches MemorySSA that invariant loads (and loads of provably constant memory) are always liveOnEntry. llvm-svn: 277640
* [InstCombine] use m_APInt to allow icmp eq (srem X, C1), C2 folds for splat ↵Sanjay Patel2016-08-031-4/+3
| | | | | | constant vectors llvm-svn: 277638
* [MSSA] Add logic for special handling of atomics/volatiles.George Burgess IV2016-08-031-0/+58
| | | | | | | | | | | | | | | This patch makes MemorySSA recognize atomic/volatile loads, and makes MSSA treat said loads specially. This allows us to be a bit more aggressive in some cases. Administrative note: Revision was LGTM'ed by reames in person. Additionally, this doesn't include the `invariant.load` recognition in the differential revision, because I feel it's better to commit that separately. Will commit soon. Differential Revision: https://reviews.llvm.org/D16875 llvm-svn: 277637
* [InstCombine] Refactor optimization of zext(or(icmp, icmp)) to enable more ↵Tobias Grosser2016-08-032-6/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | aggressive cast-folding Summary: InstCombine unfolds expressions of the form `zext(or(icmp, icmp))` to `or(zext(icmp), zext(icmp))` such that in a later iteration of InstCombine the exposed `zext(icmp)` instructions can be optimized. We now combine this unfolding and the subsequent `zext(icmp)` optimization to be performed together. Since the unfolding doesn't happen separately anymore, we also again enable the folding of `logic(cast(icmp), cast(icmp))` expressions to `cast(logic(icmp, icmp))` which had been disabled due to its interference with the unfolding transformation. Tested via `make check` and `lnt`. Background ========== For a better understanding on how it came to this change we subsequently summarize its history. In commit r275989 we've already tried to enable the folding of `logic(cast(icmp), cast(icmp))` to `cast(logic(icmp, icmp))` which had to be reverted in r276106 because it could lead to an endless loop in InstCombine (also see http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20160718/374347.html). The root of this problem is that in `visitZExt()` in InstCombineCasts.cpp there also exists a reverse of the above folding transformation, that unfolds `zext(or(icmp, icmp))` to `or(zext(icmp), zext(icmp))` in order to expose `zext(icmp)` operations which would then possibly be eliminated by subsequent iterations of InstCombine. However, before these `zext(icmp)` would be eliminated the folding from r275989 could kick in and cause InstCombine to endlessly switch back and forth between the folding and the unfolding transformation. This is the reason why we now combine the `zext`-unfolding and the elimination of the exposed `zext(icmp)` to happen at one go because this enables us to still allow the cast-folding in `logic(cast(icmp), cast(icmp))` without entering an endless loop again. Details on the submitted changes ================================ - In `visitZExt()` we combine the unfolding and optimization of `zext` instructions. - In `transformZExtICmp()` we have to use `Builder->CreateIntCast()` instead of `CastInst::CreateIntegerCast()` to make sure that the new `CastInst` is inserted in a `BasicBlock`. The new calls to `transformZExtICmp()` that we introduce in `visitZExt()` would otherwise cause according assertions to be triggered (in our case this happend, for example, with lnt for the MultiSource/Applications/sqlite3 and SingleSource/Regression/C++/EH/recursive-throw tests). The subsequent usage of `replaceInstUsesWith()` is necessary to ensure that the new `CastInst` replaces the `ZExtInst` accordingly. - In InstCombineAndOrXor.cpp we again allow the folding of casts on `icmp` instructions. - The instruction order in the optimized IR for the zext-or-icmp.ll test case is different with the introduced changes. - The test cases in zext.ll have been adopted from the reverted commits r275989 and r276105. Reviewers: grosser, majnemer, spatel Subscribers: eli.friedman, majnemer, llvm-commits Differential Revision: https://reviews.llvm.org/D22864 Contributed-by: Matthias Reisinger <d412vv1n@gmail.com> llvm-svn: 277635
* [InstCombine] use m_APInt to allow icmp (binop X, Y), C folds with constant ↵Sanjay Patel2016-08-031-14/+20
| | | | | | | | | splat vectors This removes the restriction for the icmp constant, but as noted by the FIXME comments, we still need to change individual checks for binop operand constants. llvm-svn: 277629
* [CloneFunction] Don't crash if the value map doesn't hold somethingDavid Majnemer2016-08-031-1/+1
| | | | | | | | | It is possible for the value map to not have an entry for some value that has already been removed. I don't have a testcase, this is fall-out from a buildbot. llvm-svn: 277614
* use local variables; NFCSanjay Patel2016-08-031-29/+23
| | | | llvm-svn: 277612
* [CloneFunction] Don't remove side effecting callsDavid Majnemer2016-08-031-2/+33
| | | | | | | | | | | We were able to figure out that the result of a call is some constant. While propagating that fact, we added the constant to the value map. This is problematic because it results in us losing the call site when processing the value map. This fixes PR28802. llvm-svn: 277611
* Revert "Teach CorrelatedValuePropagation to mark adds as no wrap"Renato Golin2016-08-031-57/+0
| | | | | | This reverts commit r277592, trying to fix the AArch64 42VMA buildbot. llvm-svn: 277607
* [Loop Vectorizer] Move store-predication into its own function, remove ↵Gil Rapaport2016-08-031-12/+18
| | | | | | | | obsolete comment (NFC) Differential Revision: https://reviews.llvm.org/D23013 llvm-svn: 277595
* Teach CorrelatedValuePropagation to mark adds as no wrapArtur Pilipenko2016-08-031-0/+57
| | | | | | | | | | Use LVI to prove that adds do not wrap. The change is motivated by https://llvm.org/bugs/show_bug.cgi?id=28620 bug and it's the first step to fix that problem. Reviewed By: sanjoy Differential Revision: http://reviews.llvm.org/D23059 llvm-svn: 277592
* [ADCE] Refactor anticipating new functionality (NFC)David Callahan2016-08-031-17/+34
| | | | | | | | | | | | | | | | | Summary: This is the first refactoring before adding new functionality. Add a class wrapper for the functions and container for state associated with the transformation. No functional change Reviewers: majnemer, nadav, mehdi_amini Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D23065 llvm-svn: 277565
* [MSSA] Fix a caching bug.George Burgess IV2016-08-031-8/+8
| | | | | | | | | | | | | | | This fixes a bug where we'd sometimes cache overly-conservative results with our walker. This bug was made more obvious by r277480, which makes our cache far more spotty than it was. Test case is llvm-unit, because we're likely going to use CachingWalker only for def optimization in the future. The bug stems from that there was a place where the walker assumed that `DefNode.Last` was a valid target to cache to when failing to optimize phis. This is sometimes incorrect if we have a cache hit. The fix is to use the thing we *can* assume is a valid target to cache to. :) llvm-svn: 277559
* [Inliner] clang-format various parts of the inliner prior to changesChandler Carruth2016-08-033-71/+71
| | | | | | here. NFC. llvm-svn: 277557
* Add -lowertypetests-bitsets-level to control bitsets generation.Ivan Krasin2016-08-031-2/+9
| | | | | | | | | | | | | | | Summary: Sometimes, bitsets could get really large (>300k entries) and we might want to drop a check, as it would have a too much cost. Adding a flag to control how much penalty are we willing to pay for bitsets. Reviewers: kcc Differential Revision: https://reviews.llvm.org/D23088 llvm-svn: 277556
* Support for lifetime begin/end markers in the MemorySSA use optimizerDaniel Berlin2016-08-031-1/+38
| | | | | | | | | | | | Summary: Depends on D23072 Reviewers: george.burgess.iv Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D23076 llvm-svn: 277553
* [InstCombine] replace dyn_casts with matches; NFCISanjay Patel2016-08-022-15/+15
| | | | | | Clean-up before changing this to allow folds for vectors. llvm-svn: 277538
* Imported statistics types changesPiotr Padlewski2016-08-021-23/+25
| | | | | | | | | | Reviewers: tejohnson, eraman Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D22980 llvm-svn: 277534
* Move to having a single real instructionClobbersQueryDaniel Berlin2016-08-021-88/+94
| | | | | | | | | | | | Summary: We really want to move towards MemoryLocOrCall (or fix AA) everywhere, but for now, this lets us have a single instructionClobbersQuery. Reviewers: george.burgess.iv Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D23072 llvm-svn: 277530
* [LoopUnroll] Switch the default value of -unroll-runtime-epilog back to its ↵Michael Zolotukhin2016-08-021-1/+1
| | | | | | | | | | original value. As agreed in post-commit review of r265388, I'm switching the flag to its original value until the 90% runtime performance regression on SingleSource/Benchmarks/Stanford/Bubblesort is addressed. llvm-svn: 277524
* [LoopVectorize] Change comment for isOutOfScope in collectLoopUniforms, NFCWei Mi2016-08-021-3/+4
| | | | | | | | | Update comment for isOutOfScope and add a testcase for uniform value being used out of scope. Differential Revision: https://reviews.llvm.org/D23073 llvm-svn: 277515
* Fixes for post-commit review comments on r277480Daniel Berlin2016-08-021-12/+10
| | | | llvm-svn: 277510
* [IRCE] Rename variable; NFCSanjoy Das2016-08-021-6/+6
| | | | | | There is nothing "Original" about "OriginalLoopInfo". llvm-svn: 277506
* [IRCE] Preserve DomTree and LCSSASanjoy Das2016-08-021-5/+11
| | | | | | | This changes IRCE to "preserve" LCSSA and DomTree by recomputing them. It still does not preserve LoopSimplify. llvm-svn: 277505
* [LoopUnroll] Ensure we create prolog loops in simplified form.Michael Zolotukhin2016-08-021-0/+12
| | | | llvm-svn: 277502
* MSVC 2013 does not implement C++11 unions properly, so remove the anoymous ↵Daniel Berlin2016-08-021-2/+2
| | | | | | | | union for now, and leave a FIXME. llvm-svn: 277485
* Rewrite the use optimizer to be less memory intensive and 50% faster.Daniel Berlin2016-08-021-31/+311
| | | | | | | | | | | | | | | | | | | | | | | | Fixes PR28670 Summary: Rewrite the use optimizer to be less memory intensive and 50% faster. Fixes PR28670 The new use optimizer works like a standard SSA renaming pass, storing all possible versions a MemorySSA use could get in a stack, and just tracking indexes into the stack. This uses much less memory than caching N^2 alias query results. It's also a lot faster. The current version defers phi node walking to the normal walker. Reviewers: george.burgess.iv Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D23032 llvm-svn: 277480
* [LV] Generate both scalar and vector integer induction variablesMatthew Simpson2016-08-021-34/+66
| | | | | | | | | | | | | | | | This patch enables the vectorizer to generate both scalar and vector versions of an integer induction variable for a given loop. Previously, we only generated a scalar induction variable if we knew all its users were going to be scalar. Otherwise, we generated a vector induction variable. In the case of a loop with both scalar and vector users of the induction variable, we would generate the vector induction variable and extract scalar values from it for the scalar users. With this patch, we now generate both versions of the induction variable when there are both scalar and vector users and select which version to use based on whether the user is scalar or vector. Differential Revision: https://reviews.llvm.org/D22869 llvm-svn: 277474
* [LV] Untangle the concepts of uniform and scalarMatthew Simpson2016-08-021-63/+107
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch refactors the logic in collectLoopUniforms and collectValuesToIgnore, untangling the concepts of "uniform" and "scalar". It adds isScalarAfterVectorization along side isUniformAfterVectorization to distinguish the two. Known scalar values include those that are uniform, getelementptr instructions that won't be vectorized, and induction variables and induction variable update instructions whose users are all known to be scalar. This patch includes the following functional changes: - In collectLoopUniforms, we mark uniform the pointer operands of interleaved accesses. Although non-consecutive, these pointers are treated like consecutive pointers during vectorization. - In collectValuesToIgnore, we insert a value into VecValuesToIgnore if it isScalarAfterVectorization rather than isUniformAfterVectorization. This differs from the previous functionaly in that we now add getelementptr instructions that will not be vectorized into VecValuesToIgnore. This patch also removes the ValuesNotWidened set used for induction variable scalarization since, after the above changes, it is now equivalent to isScalarAfterVectorization. Differential Revision: https://reviews.llvm.org/D22867 llvm-svn: 277460
* [LoadStoreVectorizer] Don't use a linear walk for an existence check in a ↵Benjamin Kramer2016-08-021-1/+1
| | | | | | | | SmallPtrSet No functionality change intended. llvm-svn: 277436
* Minor code cleanups. NFC.Junmo Park2016-08-021-2/+2
| | | | llvm-svn: 277415
* CodeExtractor : Add ability to preserve profile data.Sean Silva2016-08-022-19/+122
| | | | | | | | | | | Added ability to estimate the entry count of the extracted function and the branch probabilities of the exit branches. Patch by River Riddle! Differential Revision: https://reviews.llvm.org/D22744 llvm-svn: 277411
* [ADT] NFC: Generalize GraphTraits requirement of "NodeType *" in interfaces ↵Tim Shen2016-08-011-0/+1
| | | | | | | | | | | | | | to "NodeRef", and migrate SCCIterator.h to use NodeRef Summary: By generalize the interface, users are able to inject more flexible Node token into the algorithm, for example, a pair of vector<Node>* and index integer. Currently I only migrated SCCIterator to use NodeRef, but more is coming. It's a NFC. Reviewers: dblaikie, chandlerc Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D22937 llvm-svn: 277399
* [WebAssembly] Support CFI for WebAssembly targetDerek Schuff2016-08-011-25/+68
| | | | | | | | | | | | | | | Summary: This patch implements CFI for WebAssembly. It modifies the LowerTypeTest pass to pre-assign table indexes to functions that are called indirectly, and lowers type checks to test against the appropriate table indexes. It also modifies the WebAssembly backend to support a special ".indidx" assembly directive that propagates the table index assignments out to the linker. Patch by Dominic Chen Differential Revision: https://reviews.llvm.org/D21768 llvm-svn: 277398
* [PM] Port SpeculativeExecution to the new PMMichael Kuperstein2016-08-012-40/+58
| | | | | | Differential Revision: https://reviews.llvm.org/D23033 llvm-svn: 277393
OpenPOWER on IntegriCloud