summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/Scalar
Commit message (Collapse)AuthorAgeFilesLines
* [LIR] Push check into helper function. NFC.Chad Rosier2015-12-011-4/+4
| | | | llvm-svn: 254416
* Remove an intermediate lambda. NFCCraig Topper2015-11-291-3/+2
| | | | llvm-svn: 254246
* Use range-based for loops. NFCCraig Topper2015-11-281-37/+20
| | | | llvm-svn: 254222
* [SCCP] More informative message if we don't know how to handle a terminator.Davide Italiano2015-11-251-1/+1
| | | | llvm-svn: 254093
* use convenience function for copying IR flags; NFCISanjay Patel2015-11-241-12/+2
| | | | llvm-svn: 253996
* [LIR] Put includes in correct order. NFC.Chad Rosier2015-11-231-1/+1
| | | | llvm-svn: 253915
* [WinEH] Fix a case where GVN could incorrectly PRE a load into an EH pad.Andrew Kaylor2015-11-231-0/+10
| | | | | | Differential Revision: http://reviews.llvm.org/D14842 llvm-svn: 253908
* [LoopStrengthReduce] Mark dump() definitions as LLVM_DUMP_METHOD.Davide Italiano2015-11-231-14/+7
| | | | llvm-svn: 253841
* Use modulo operator instead of multiplying result of a divide and ↵Craig Topper2015-11-211-1/+1
| | | | | | subtracting from the original dividend. NFC. llvm-svn: 253792
* Fix a pair of issues that caused an infinite loop in reassociate.Owen Anderson2015-11-201-1/+1
| | | | | | | | Terrifyingly, one of them is a mishandling of floating point vectors in Constant::isZero(). How exactly this issue survived this long is beyond me. llvm-svn: 253655
* Use range-based for loops. NFCCraig Topper2015-11-201-12/+8
| | | | llvm-svn: 253652
* [LIR] Update some comments. NFC.Chad Rosier2015-11-191-5/+4
| | | | llvm-svn: 253603
* [LIR] Fix 80-column from previous commit.Chad Rosier2015-11-191-1/+2
| | | | llvm-svn: 253586
* [LIR] Sink checks into function to enable future refactoring. NFC.Chad Rosier2015-11-191-15/+17
| | | | | | | The purpose of this change is help delineate the memset and memcpy optimizations with the overall goal of resolving PR25520. llvm-svn: 253585
* [LIR] Use the more appropriate method. NFC.Chad Rosier2015-11-191-1/+1
| | | | llvm-svn: 253578
* Revert "Change memcpy/memset/memmove to have dest and source alignments."Pete Cooper2015-11-196-67/+46
| | | | | | | | | | This reverts commit r253511. This likely broke the bots in http://lab.llvm.org:8011/builders/clang-ppc64-elf-linux2/builds/20202 http://bb.pgr.jp/builders/clang-3stage-i686-linux/builds/3787 llvm-svn: 253543
* Fix bug 25440: GVN assertion after coercing loadsWeiming Zhao2015-11-191-1/+12
| | | | | | | | | | | | | | | | Optimizations like LoadPRE in GVN will insert new instructions. If the insertion point is in a already processed BB, they should get a value number explicitly. If the insertion point is after current instruction, then just leave it. However, current GVN framework has no support for it. In this patch, we just bail out if a VN can't be found. Dfferential Revision: http://reviews.llvm.org/D14670 A test/Transforms/GVN/pr25440.ll M lib/Transforms/Scalar/GVN.cpp llvm-svn: 253536
* Fix returned value for GVN: could return "false" even after modifying the IRMehdi Amini2015-11-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | This bug would manifest in some very specific cases where all the following conditions are fullfilled: - GVN didn't remove block - The regular GVN iteration didn't change the IR - PRE is enabled - PRE will not split critical edge - The last instruction processed by PRE didn't change the IR Because the CallGraph PassManager relies on this returned value to decide if it needs to recompute a node after the execution of Function passes, not returning the right value can lead to unexpected results. Fix for: https://llvm.org/bugs/show_bug.cgi?id=24715 Patch by Wenxiang Qiu <vincentqiuuu@gmail.com> From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 253518
* Change memcpy/memset/memmove to have dest and source alignments.Pete Cooper2015-11-186-46/+67
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Note, this was reviewed (and more details are in) http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20151109/312083.html These intrinsics currently have an explicit alignment argument which is required to be a constant integer. It represents the alignment of the source and dest, and so must be the minimum of those. This change allows source and dest to each have their own alignments by using the alignment attribute on their arguments. The alignment argument itself is removed. There are a few places in the code for which the code needs to be checked by an expert as to whether using only src/dest alignment is safe. For those places, they currently take the minimum of src/dest alignments which matches the current behaviour. For example, code which used to read: call void @llvm.memcpy.p0i8.p0i8.i32(i8* %dest, i8* %src, i32 500, i32 8, i1 false) will now read: call void @llvm.memcpy.p0i8.p0i8.i32(i8* align 8 %dest, i8* align 8 %src, i32 500, i1 false) For out of tree owners, I was able to strip alignment from calls using sed by replacing: (call.*llvm\.memset.*)i32\ [0-9]*\,\ i1 false\) with: $1i1 false) and similarly for memmove and memcpy. I then added back in alignment to test cases which needed it. A similar commit will be made to clang which actually has many differences in alignment as now IRBuilder can generate different source/dest alignments on calls. In IRBuilder itself, a new argument was added. Instead of calling: CreateMemCpy(Dst, Src, getInt64(Size), DstAlign, /* isVolatile */ false) you now call CreateMemCpy(Dst, Src, getInt64(Size), DstAlign, SrcAlign, /* isVolatile */ false) There is a temporary class (IntegerAlignment) which takes the source alignment and rejects implicit conversion from bool. This is to prevent isVolatile here from passing its default parameter to the source alignment. Note, changes in future can now be made to codegen. I didn't change anything here, but this change should enable better memcpy code sequences. Reviewed by Hal Finkel. llvm-svn: 253511
* Disable gvn non-local speculative loads under asan.Mike Aizatsky2015-11-181-0/+4
| | | | | | | | Summary: Fix for https://llvm.org/bugs/show_bug.cgi?id=25550 Differential Revision: http://reviews.llvm.org/D14763 llvm-svn: 253498
* Revert "Revert "Strip metadata when speculatively hoisting instructions ↵Igor Laevsky2015-11-181-0/+4
| | | | | | | | (r252604)" Failing clang test is now fixed by the r253458. llvm-svn: 253459
* Replace dyn_cast with isa in places that weren't using the returned value ↵Craig Topper2015-11-181-3/+3
| | | | | | for more than a boolean check. NFC. llvm-svn: 253441
* [PRE] Preserve !invariant.load metadataPhilip Reames2015-11-171-0/+2
| | | | | | Spoted via inspection. Test case included. llvm-svn: 253275
* Add intermediate subtract instructions to reassociation worklist.Owen Anderson2015-11-161-9/+39
| | | | | | | | | | We sometimes create intermediate subtract instructions during reassociation. Adding these to the worklist to revisit exposes many additional reassociation opportunities. Patch by Aditya Nandakumar. llvm-svn: 253240
* [LoopStrengthReduce] Don't increment iterator past the end of the BBDavid Majnemer2015-11-161-1/+1
| | | | | | | | | | | We tried to move the insertion point beyond instructions like landingpad and cleanuppad. However, we *also* tried to move past catchpad. This is problematic because catchpad is also a terminator. This fixes PR25541. llvm-svn: 253238
* [Sink] Don't move landingpadsKeno Fischer2015-11-161-1/+1
| | | | | | | | | | | | | | | Summary: Moving landingpads into successor basic blocks makes the verifier sad. Teach Sink that much like PHI nodes and terminator instructions, landingpads (and cleanuppads, etc.) may not be moved between basic blocks. Reviewers: majnemer Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D14475 llvm-svn: 253182
* [LIR] Add support for creating memcpys from loops with a negative stride.Chad Rosier2015-11-131-12/+21
| | | | | | | | | | | | | | | This allows us to transform the below loop into a memcpy. void test(unsigned *__restrict__ a, unsigned *__restrict__ b) { for (int i = 2047; i >= 0; --i) { a[i] = b[i]; } } This is the memcpy version of r251518, which added support for memset with negative strided loops. llvm-svn: 253091
* Add a comment that should have made my last commit.Chad Rosier2015-11-131-0/+1
| | | | llvm-svn: 253063
* [LIR] Factor out the code to compute base ptr for negative strided loops.Chad Rosier2015-11-131-10/+15
| | | | | | This will allow for the code to be reused in the memcpy optimization. llvm-svn: 253061
* Revert "Fix bug 25440: GVN assertion after coercing loads"Tobias Grosser2015-11-121-41/+13
| | | | | | This reverts 252919 which broke LNT: MultiSource/Applications/SPASS llvm-svn: 252936
* [LIR] Minor refactoring. NFCI.Chad Rosier2015-11-121-21/+44
| | | | | | | This change prevents uninteresting stores from being inserted into the list of candidate stores for memset/memcpy conversion. llvm-svn: 252926
* Fix bug 25440: GVN assertion after coercing loadsWeiming Zhao2015-11-121-13/+41
| | | | | | | | | | | | | | | | Summary: when coercing loads, it inserts some instructions, which have no GV assigned. https://llvm.org/bugs/show_bug.cgi?id=25440 Reviewers: hfinkel, dberlin Subscribers: dberlin, llvm-commits Differential Revision: http://reviews.llvm.org/D14479 llvm-svn: 252919
* [LIR] General refactor to improve compile-time and simplify code.Chad Rosier2015-11-111-16/+26
| | | | | | | | | | | First create a list of candidates, then transform. This simplifies the code in that you have don't have to worry that you may be using an invalidated iterator. Previously, each time we created a memset/memcpy we would reevaluate the entire loop potentially resulting in lots of redundant work for large basic blocks. llvm-svn: 252817
* Revert "Strip metadata when speculatively hoisting instructions"Renato Golin2015-11-101-4/+0
| | | | | | | This reverts commit r252604, as it broke all ARM and AArch64 buildbots, as well as some x86, et al. llvm-svn: 252623
* Strip metadata when speculatively hoisting instructionsIgor Laevsky2015-11-101-0/+4
| | | | | | | | | | | | | | | | This is fix for PR24059. When we are hoisting instruction above some condition it may turn out that metadata on this instruction was control dependant on the condition. This metadata becomes invalid and we need to drop it. This patch should cover most obvious places of speculative execution (which I have found by greping isSafeToSpeculativelyExecute). I think there are more cases but at least this change covers the severe ones. Differential Revision: http://reviews.llvm.org/D14398 llvm-svn: 252604
* Simplify. NFC.Chad Rosier2015-11-091-4/+1
| | | | llvm-svn: 252491
* Allow LLE/LD and the loop versioning infrastructure to use SCEV predicatesSilviu Baranga2015-11-092-6/+37
| | | | | | | | | | | | | | | | | | | Summary: LAA currently generates a set of SCEV predicates that must be checked by users. In the case of Loop Distribute/Loop Load Elimination, no such predicates could have been emitted, since we don't allow stride versioning. However, in the future there could be SCEV predicates that will need to be checked. This change adds support for SCEV predicate versioning in the Loop Distribute, Loop Load Eliminate and the loop versioning infrastructure. Reviewers: anemet Subscribers: mssimpso, sanjoy, llvm-commits Differential Revision: http://reviews.llvm.org/D14240 llvm-svn: 252467
* [LoopStrengthReduce] Don't bother fixing up PHIs from EH Pad predsDavid Majnemer2015-11-081-0/+3
| | | | | | | | We cannot really insert fixup code into a PHI's predecessor. This fixes PR25445. llvm-svn: 252416
* ADT: Remove last implicit ilist iterator conversions, NFCDuncan P. N. Exon Smith2015-11-071-1/+1
| | | | | | | | | | Some implicit ilist iterator conversions have crept back into Analysis, Transforms, Hexagon, and llvm-stress. This removes them. I'll commit a patch immediately after this to disallow them (in a separate patch so that it's easy to revert if necessary). llvm-svn: 252371
* Add 'notail' marker for call instructions.Akira Hatanaka2015-11-061-2/+4
| | | | | | | | | | | | This marker prevents optimization passes from adding 'tail' or 'musttail' markers to a call. Is is used to prevent tail call optimization from being performed on the call. rdar://problem/22667622 Differential Revision: http://reviews.llvm.org/D12923 llvm-svn: 252368
* [ValueTracking] Add parameters to isImpliedCondition; NFCSanjoy Das2015-11-061-1/+3
| | | | | | | | | | | | | | | | Summary: This change makes the `isImpliedCondition` interface similar to the rest of the functions in ValueTracking (in that it takes a DataLayout, AssumptionCache etc.). This is an NFC, intended to make a later diff less noisy. Depends on D14369 Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D14391 llvm-svn: 252333
* [LIR] Simplify code by making DataLayout globally accessible. NFC.Chad Rosier2015-11-061-11/+10
| | | | llvm-svn: 252317
* Fix some Clang-tidy modernize warnings, other minor fixes.Eugene Zelenko2015-11-041-2/+2
| | | | | | | | Fixed warnings are: modernize-use-override, modernize-use-nullptr and modernize-redundant-void-arg. Differential revision: http://reviews.llvm.org/D14312 llvm-svn: 252087
* [CVP] Fold return values if possiblePhilip Reames2015-11-041-0/+51
| | | | | | | | | | | | In my previous change to CVP (251606), I made CVP much more aggressive about trying to constant fold comparisons. This patch is a reversal in direction. Rather than being agressive about every compare, we restore the non-block local restriction for most, and then try hard for compares feeding returns. The motivation for this is two fold: * The more I thought about it, the less comfortable I got with the possible compile time impact of the other approach. There have been no reported issues, but after talking to a couple of folks, I've come to the conclusion the time probably isn't justified. * It turns out we need to know the context to leverage the full power of LVI. In particular, asking about something at the end of it's block (the use of a compare in a return) will frequently get more precise results than something in the middle of a block. This is an implementation detail, but it's also hard to get around since mid-block queries have to reason about possible throwing instructions and don't get to use most of LVI's block focused infrastructure. This will become particular important when combined with http://reviews.llvm.org/D14263. Differential Revision: http://reviews.llvm.org/D14271 llvm-svn: 252032
* Fix unused variable warning from r252017Adam Nemet2015-11-041-3/+2
| | | | llvm-svn: 252019
* LLE 6/6: Add LoopLoadElimination passAdam Nemet2015-11-033-0/+556
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The goal of this pass is to perform store-to-load forwarding across the backedge of a loop. E.g.: for (i) A[i + 1] = A[i] + B[i] => T = A[0] for (i) T = T + B[i] A[i + 1] = T The pass relies on loop dependence analysis via LoopAccessAnalisys to find opportunities of loop-carried dependences with a distance of one between a store and a load. Since it's using LoopAccessAnalysis, it was easy to also add support for versioning away may-aliasing intervening stores that would otherwise prevent this transformation. This optimization is also performed by Load-PRE in GVN without the option of multi-versioning. As was discussed with Daniel Berlin in http://reviews.llvm.org/D9548, this is inferior to a more loop-aware solution applied here. Hopefully, we will be able to remove some complexity from GVN/MemorySSA as a consequence. In the long run, we may want to extend this pass (or create a new one if there is little overlap) to also eliminate loop-indepedent redundant loads and store that *require* versioning due to may-aliasing intervening stores/loads. I have some motivating cases for store elimination. My plan right now is to wait for MemorySSA to come online first rather than using memdep for this. The main motiviation for this pass is the 456.hmmer loop in SPECint2006 where after distributing the original loop and vectorizing the top part, we are left with the critical path exposed in the bottom loop. Being able to promote the memory dependence into a register depedence (even though the HW does perform store-to-load fowarding as well) results in a major gain (~20%). This gain also transfers over to x86: it's around 8-10%. Right now the pass is off by default and can be enabled with -enable-loop-load-elim. On the LNT testsuite, there are two performance changes (negative number -> improvement): 1. -28% in Polybench/linear-algebra/solvers/dynprog: the length of the critical paths is reduced 2. +2% in Polybench/stencils/adi: Unfortunately, I couldn't reproduce this outside of LNT The pass is scheduled after the loop vectorizer (which is after loop distribution). The rational is to try to reuse LAA state, rather than recomputing it. The order between LV and LLE is not critical because normally LV does not touch scalar st->ld forwarding cases where vectorizing would inhibit the CPU's st->ld forwarding to kick in. LoopLoadElimination requires LAA to provide the full set of dependences (including forward dependences). LAA is known to omit loop-independent dependences in certain situations. The big comment before removeDependencesFromMultipleStores explains why this should not occur for the cases that we're interested in. Reviewers: dberlin, hfinkel Subscribers: junbuml, dberlin, mssimpso, rengolin, sanjoy, llvm-commits Differential Revision: http://reviews.llvm.org/D13259 llvm-svn: 252017
* [LAA] LLE 3/6: Rename InterestingDependence to Dependences, NFCAdam Nemet2015-11-031-6/+5
| | | | | | | | | | | | | | Summary: We now collect all types of dependences including lexically forward deps not just "interesting" ones. Reviewers: hfinkel Subscribers: rengolin, llvm-commits Differential Revision: http://reviews.llvm.org/D13256 llvm-svn: 251985
* Revert "[IndVarSimplify] Rewrite loop exit values with their initial values ↵Tobias Grosser2015-11-031-73/+0
| | | | | | | | | | | | | | | | | | | from loop preheader" Commit 251839 triggers miscompiles on some bots: http://lab.llvm.org:8011/builders/perf-x86_64-penryn-O3-polly-fast/builds/13723 (The commit is listed in 13722, but due to an existing failure introduced in 13721 and reverted in 13723 the failure is only visible in 13723) To verify r251839 is indeed the only change that triggered the buildbot failures and to ensure the buildbots remain green while investigating I temporarily revert this commit. At the current state it is unclear if this commit introduced some miscompile or if it only exposed code to Polly that is subsequently miscompiled by Polly. llvm-svn: 251901
* [IndVarSimplify] Rewrite loop exit values with their initial values from ↵Chen Li2015-11-021-0/+73
| | | | | | | | | | | | | | | | | loop preheader Summary: This patch adds support to check if a loop has loop invariant conditions which lead to loop exits. If so, we know that if the exit path is taken, it is at the first loop iteration. If there is an induction variable used in that exit path whose value has not been updated, it will keep its initial value passing from loop preheader. We can therefore rewrite the exit value with its initial value. This will help remove phis created by LCSSA and enable other optimizations like loop unswitch. Reviewers: sanjoy Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D13974 llvm-svn: 251839
* [PM] Port ADCE to the new pass managerJustin Bogner2015-10-302-27/+34
| | | | llvm-svn: 251725
OpenPOWER on IntegriCloud