summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
* Changed sign of LastCallToStaticBounsPiotr Padlewski2016-08-101-1/+1
| | | | | | | | | | | | | | | | | Summary: I think it is much better this way. When I firstly saw line: Cost += InlineConstants::LastCallToStaticBonus; I though that this is a bug, because everywhere where the cost is being reduced it is usuing -=. Reviewers: eraman, tejohnson, mehdi_amini Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D23222 llvm-svn: 278290
* [IndVarSimplify] Eliminate zext of a signed IV when the IV is known to be ↵Andrew Kaylor2016-08-101-2/+7
| | | | | | | | | | non-negative Patch by Li Huang Differential Revision: https://reviews.llvm.org/D18867 llvm-svn: 278269
* Fix LCSSA increased compile timeRong Xu2016-08-101-7/+7
| | | | | | | | | | | | | | | | | | | | | We are seeing r276077 drastically increasing compiler time for our larger benchmarks in PGO profile generation build (both clang based and IR based mode) -- it can be 20x slower than without the patch (like from 30 secs to 780 secs) The increased time are all in pass LCSSA. The problematic code is about PostProcessPHIs after use-rewrite. Note that the InsertedPhis from ssa_updater is accumulating (never been cleared). Since the inserted PHIs are added to the candidate for each rewrite, The earlier ones will be repeatedly added. Later when adding the new PHIs to the work-list, we don't check the duplication either. This can result in extremely long work-list that containing tons of duplicated PHIs. This patch fixes the issue by hoisting the code out of the loop. Differential Revision: http://reviews.llvm.org/D23344 llvm-svn: 278250
* [Coroutines] Part 6: Elide dynamic allocation of a coroutine frame when possibleGor Nishanov2016-08-104-28/+213
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: A particular coroutine usage pattern, where a coroutine is created, manipulated and destroyed by the same calling function, is common for coroutines implementing RAII idiom and is suitable for allocation elision optimization which avoid dynamic allocation by storing the coroutine frame as a static `alloca` in its caller. coro.free and coro.alloc intrinsics are used to indicate which code needs to be suppressed when dynamic allocation elision happens: ``` entry: %elide = call i8* @llvm.coro.alloc() %need.dyn.alloc = icmp ne i8* %elide, null br i1 %need.dyn.alloc, label %coro.begin, label %dyn.alloc dyn.alloc: %alloc = call i8* @CustomAlloc(i32 4) br label %coro.begin coro.begin: %phi = phi i8* [ %elide, %entry ], [ %alloc, %dyn.alloc ] %hdl = call i8* @llvm.coro.begin(i8* %phi, i32 0, i8* null, i8* bitcast ([2 x void (%f.frame*)*]* @f.resumers to i8*)) ``` and ``` %mem = call i8* @llvm.coro.free(i8* %hdl) %need.dyn.free = icmp ne i8* %mem, null br i1 %need.dyn.free, label %dyn.free, label %if.end dyn.free: call void @CustomFree(i8* %mem) br label %if.end if.end: ... ``` If heap allocation elision is performed, we replace coro.alloc with a static alloca on the caller frame and coro.free with null constant. Also, we need to make sure that if there are any tail calls referencing the coroutine frame, we need to remote tail call attribute, since now coroutine frame lives on the stack. Documentation and overview is here: http://llvm.org/docs/Coroutines.html. Upstreaming sequence (rough plan) 1.Add documentation. (https://reviews.llvm.org/D22603) 2.Add coroutine intrinsics. (https://reviews.llvm.org/D22659) 3.Add empty coroutine passes. (https://reviews.llvm.org/D22847) 4.Add coroutine devirtualization + tests. ab) Lower coro.resume and coro.destroy (https://reviews.llvm.org/D22998) c) Do devirtualization (https://reviews.llvm.org/D23229) 5.Add CGSCC restart trigger + tests. (https://reviews.llvm.org/D23234) 6.Add coroutine heap elision + tests. <= we are here 7.Add the rest of the logic (split into more patches) Reviewers: mehdi_amini, majnemer Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D23245 llvm-svn: 278242
* Teach CorrelatedValuePropagation to mark adds as no wrapArtur Pilipenko2016-08-101-0/+57
| | | | | | | | | | | | This is a resubmission of previously reverted r277592. It was hitting overly strong assertion in getConstantRange which was relaxed in r278217. Use LVI to prove that adds do not wrap. The change is motivated by https://llvm.org/bugs/show_bug.cgi?id=28620 bug and it's the first step to fix that problem. Reviewed By: sanjoy Differential Revision: http://reviews.llvm.org/D23059 llvm-svn: 278220
* [SimplifyLibCalls] Restore the old behaviour, emit a libcall.Davide Italiano2016-08-101-3/+5
| | | | | | | | Hal pointed out that the semantic of our intrinsic and the libc call are slightly different. Add a comment while I'm here to explain why we can't emit an intrinsic. Thanks Hal! llvm-svn: 278200
* Do not directly use inline threshold cl options in cost analysis.Easwaran Raman2016-08-101-14/+8
| | | | | | | | This adds an InlineParams struct which is populated from the command line options by getInlineParams and passed to getInlineCost for the call analyzer to use. Differential revision: https://reviews.llvm.org/D22120 llvm-svn: 278189
* [Inliner,OptDiag] Add hotness attribute to opt diagnosticsAdam Nemet2016-08-101-35/+40
| | | | | | | | | | | | | | | | | | | | Summary: The inliner not being a function pass requires the work-around of generating the OptimizationRemarkEmitter and in turn BFI on demand. This will go away after the new PM is ready. BFI is only computed inside ORE if the user has requested hotness information for optimization diagnostitics (-pass-remark-with-hotness at the 'opt' level). Thus there is no additional overhead without the flag. Reviewers: hfinkel, davidxl, eraman Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D22694 llvm-svn: 278185
* [LoopSimplify] Rebuild LCSSA for the inner loop after separating nested loops.Michael Zolotukhin2016-08-091-32/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This hopefully fixes PR28825. The problem now was that a value from the original loop was used in a subloop, which became a sibling after separation. While a subloop doesn't need an lcssa phi node, a sibling does, and that's where we broke LCSSA. The most natural way to fix this now is to simply call formLCSSA on the original loop: it'll do what we've been doing before plus it'll cover situations described above. I think we don't need to run formLCSSARecursively here, and we have an assert to verify this (I've tried testing it on LLVM testsuite + SPECs). I'd be happy to be corrected here though. I also changed a run line in the test from '-lcssa -loop-unroll' to '-lcssa -loop-simplify -indvars', because it exercises LCSSA preservation to the same extent, but also makes less unrelated transformation on the CFG, which makes it easier to verify. Reviewers: chandlerc, sanjoy, silvas Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D23288 llvm-svn: 278173
* Fix the runtime error caused by "Use ValueOffsetPair to enhance value reuse ↵Wei Mi2016-08-091-1/+1
| | | | | | | | | | | | | | during SCEV expansion". The patch is to fix the bug in PR28705. It was caused by setting wrong return value for SCEVExpander::findExistingExpansion. The return values of findExistingExpansion have different meanings when the function is used in different ways so it is easy to make mistake. The fix creates two new interfaces to replace SCEVExpander::findExistingExpansion, and specifies where each interface is expected to be used. Differential Revision: https://reviews.llvm.org/D22942 llvm-svn: 278161
* [EarlyCSE] Teach about CSE'ing over invariant.start intrinsicsAnna Thomas2016-08-091-0/+13
| | | | | | | | | | | | | | Summary: Teach EarlyCSE about invariant.start intrinsic. Specifically, we can perform store-load, load-load forwarding over this call. Reviewers: majnemer, reames, dberlin, sanjoy Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D23268 llvm-svn: 278153
* [Profile] turn off verbose warnings by defaultXinliang David Li2016-08-091-1/+1
| | | | | | | | | no prof data for func warning is turned off by default due to its high verbosity and minimal usefulness. Differential Revision: http://reviews.llvm.org/D23295 llvm-svn: 278127
* Consistently use LoopAnalysisManagerSean Silva2016-08-099-9/+9
| | | | | | | | | | | | | | | | | One exception here is LoopInfo which must forward-declare it (because the typedef is in LoopPassManager.h which depends on LoopInfo). Also, some includes for LoopPassManager.h were needed since that file provides the typedef. Besides a general consistently benefit, the extra layer of indirection allows the mechanical part of https://reviews.llvm.org/D23256 that requires touching every transformation and analysis to be factored out cleanly. Thanks to David for the suggestion. llvm-svn: 278079
* Consistently use ModuleAnalysisManagerSean Silva2016-08-0913-14/+14
| | | | | | | | | | | Besides a general consistently benefit, the extra layer of indirection allows the mechanical part of https://reviews.llvm.org/D23256 that requires touching every transformation and analysis to be factored out cleanly. Thanks to David for the suggestion. llvm-svn: 278078
* Consistently use FunctionAnalysisManagerSean Silva2016-08-0919-19/+19
| | | | | | | | | | | Besides a general consistently benefit, the extra layer of indirection allows the mechanical part of https://reviews.llvm.org/D23256 that requires touching every transformation and analysis to be factored out cleanly. Thanks to David for the suggestion. llvm-svn: 278077
* [WebAssembly] Fix CFI index to account for padding nullptr functionDerek Schuff2016-08-081-1/+2
| | | | | | | | | | | | | The WebAssembly linker now creates a dummy function at index 0 to prevent miscomparisons with the NULL pointer, see https://github.com/WebAssembly/binaryen/pull/658. Thanks to pcc for pointing out this problem! Patch by Dominic Chen Differential Revision: https://reviews.llvm.org/D23137 llvm-svn: 278073
* InstCombine: Remove a redundant #ifdef NDEBUG. NFCJustin Bogner2016-08-081-2/+0
| | | | | | The DEBUG() macro already does this. llvm-svn: 278049
* [LoopUnroll] Simplify loops created by unrolling.Michael Zolotukhin2016-08-081-0/+19
| | | | | | | | | | | | | | Summary: Currently loop-unrolling doesn't preserve loop-simplified form. This patch fixes it by resimplifying affected loops. Reviewers: chandlerc, sanjoy, hfinkel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D23148 llvm-svn: 278038
* [MemorySSA] Fix windows build breakage caused by r278028Geoff Berry2016-08-081-4/+4
| | | | | r278028: [MemorySSA] Ensure address stability of MemorySSA object. llvm-svn: 278035
* [MemorySSA] Ensure address stability of MemorySSA object.Geoff Berry2016-08-081-17/+13
| | | | | | | | | | | | | | | | | Summary: Ensure that the MemorySSA object never changes address when using the new pass manager since the walkers contained by MemorySSA cache pointers to it at construction time. This is achieved by wrapping the MemorySSAAnalysis result in a unique_ptr. Also add some asserts that check for this bug. Reviewers: george.burgess.iv, dberlin Subscribers: mcrosier, hfinkel, chandlerc, silvas, llvm-commits Differential Revision: https://reviews.llvm.org/D23171 llvm-svn: 278028
* GVN-hoist: enable by defaultSebastian Pop2016-08-081-2/+2
| | | | llvm-svn: 278010
* Add some comments linking back to PR28400.Sean Silva2016-08-084-0/+12
| | | | | | Thanks to Mehdi for the suggestion! llvm-svn: 277984
* [PM] More workaround for PR28400Sean Silva2016-08-083-0/+4
| | | | llvm-svn: 277982
* [PM] Invalidate CallGraphAnalysis because it holds AssertingVHSean Silva2016-08-081-0/+5
| | | | | | | This is essentially PR28400. The fix here is similar to that implemented in r274656. llvm-svn: 277980
* [MSSA] Fix PR28880 by fixing use optimizer's lower bound tracking behavior.Daniel Berlin2016-08-081-4/+16
| | | | | | | | | | | | | | | | Summary: In the use optimizer, we need to keep of whether the lower bound still dominates us or else we may decide a lower bound is still valid when it is not due to intervening pushes/pops. Fixes PR28880 (and probably a bunch of other things). Reviewers: george.burgess.iv Subscribers: MatzeB, llvm-commits, sebpop Differential Revision: https://reviews.llvm.org/D23237 llvm-svn: 277978
* [JumpThreading] Fix handling of aliasing metadata.Eli Friedman2016-08-083-8/+28
| | | | | | | | | | | | | | | | | | Summary: The correctness fix here is that when we CSE a load with another load, we need to combine the metadata on the two loads. This matches the behavior of other passes, like instcombine and GVN. There's also a minor optimization improvement here: for load PRE, the aliasing metadata on the inserted load should be the same as the metadata on the original load. Not sure why the old code was throwing it away. Issue found by inspection. Differential Revision: http://reviews.llvm.org/D21460 llvm-svn: 277977
* [SimplifyLibCalls] Emit sqrt intrinsic instead of a libcall.Davide Italiano2016-08-081-2/+3
| | | | llvm-svn: 277972
* [SROA] Fix crash with lifetime intrinsic partially covering alloca.Eli Friedman2016-08-081-1/+7
| | | | | | | | | | | | | Summary: PromoteMemToReg looks specifically for the pattern bitcast+lifetime.start (or a bitcast-equivalent GEP); any offset will lead to an assertion failure. Fixes https://llvm.org/bugs/show_bug.cgi?id=27999 . Differential Revision: https://reviews.llvm.org/D22737 llvm-svn: 277969
* [SLC] Emit an intrinsic instead of a libcall for pow.Davide Italiano2016-08-071-9/+13
| | | | | | Differential Revision: https://reviews.llvm.org/D22104 llvm-svn: 277963
* [InstCombine] Infer inbounds on geps of allocasDavid Majnemer2016-08-071-0/+19
| | | | llvm-svn: 277950
* Revert "Revert "[LoopSimplify] Fix updating LCSSA after separating nested ↵Michael Zolotukhin2016-08-071-0/+15
| | | | | | | | | loops."" This reverts commit r277901. Reaaply the commit as it looks like it has nothing to do with the bots failures. llvm-svn: 277946
* CoroSplit: Squash unused variable FnTrigger warning in NDEBUGGor Nishanov2016-08-061-0/+2
| | | | llvm-svn: 277938
* [Coroutines] Part 5: Add CGSCC restart triggerGor Nishanov2016-08-065-8/+154
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: CoroSplit pass processes the coroutine twice. First, it lets it go through complete IPO optimization pipeline as a single function. It forces restart of the pipeline by inserting an indirect call to an empty function "coro.devirt.trigger" which is devirtualized by CoroElide pass that triggers a restart of the pipeline by CGPassManager. (In later patches, when CoroSplit pass sees the same coroutine the second time, it splits it up, adds coroutine subfunctions to the SCC to be processed by IPO pipeline.) Documentation and overview is here: http://llvm.org/docs/Coroutines.html. Upstreaming sequence (rough plan) 1.Add documentation. (https://reviews.llvm.org/D22603) 2.Add coroutine intrinsics. (https://reviews.llvm.org/D22659) 3.Add empty coroutine passes. (https://reviews.llvm.org/D22847) 4.Add coroutine devirtualization + tests. ab) Lower coro.resume and coro.destroy (https://reviews.llvm.org/D22998) c) Do devirtualization (https://reviews.llvm.org/D23229) 5.Add CGSCC restart trigger + tests. <= we are here 6.Add coroutine heap elision + tests. 7.Add the rest of the logic (split into more patches) Reviewers: mehdi_amini, majnemer Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D23234 llvm-svn: 277936
* [Inliner] Use function_ref for functors which are never taken ownership of.Benjamin Kramer2016-08-061-5/+5
| | | | llvm-svn: 277922
* [LoadCombine] Simplify code with a brace init. NFC.Benjamin Kramer2016-08-061-4/+1
| | | | llvm-svn: 277921
* Move helpers into anonymous namespaces. NFC.Benjamin Kramer2016-08-063-1/+6
| | | | llvm-svn: 277916
* [InstCombine] Don't coerce non-integral pointers to integersSanjoy Das2016-08-061-1/+2
| | | | | | | | | | Reviewers: majnemer Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D23231 llvm-svn: 277910
* Revert "(refs/bisect/bad) GVN-hoist: enable by default"Matthias Braun2016-08-061-2/+2
| | | | | | | | | | | GVN-Hoist appears to miscompile llvm-testsuite SingleSource/Benchmarks/Misc/fbench.c at the moment. I filed http://llvm.org/PR28880 This reverts commit r277786. llvm-svn: 277909
* Part 4c: Coroutine Devirtualization: Devirtualize coro.resume and coro.destroy.Gor Nishanov2016-08-065-35/+161
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This is the 4c patch of the coroutine series. CoroElide pass now checks if PostSplit coro.begin is referenced by coro.subfn.addr intrinsics. If so replace coro.subfn.addrs with an appropriate coroutine subfunction associated with that coro.begin. Documentation and overview is here: http://llvm.org/docs/Coroutines.html. Upstreaming sequence (rough plan) 1.Add documentation. (https://reviews.llvm.org/D22603) 2.Add coroutine intrinsics. (https://reviews.llvm.org/D22659) 3.Add empty coroutine passes. (https://reviews.llvm.org/D22847) 4.Add coroutine devirtualization + tests. ab) Lower coro.resume and coro.destroy (https://reviews.llvm.org/D22998) c) Do devirtualization <= we are here 5.Add CGSCC restart trigger + tests. 6.Add coroutine heap elision + tests. 7.Add the rest of the logic (split into more patches) Reviewers: majnemer Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D23229 llvm-svn: 277908
* Revert "[LoopSimplify] Fix updating LCSSA after separating nested loops."Michael Zolotukhin2016-08-061-15/+0
| | | | | | | This reverts commit r277877. Try to appease clang-x64-ninja-win7 buildbot. llvm-svn: 277901
* [IRCE] Remove unused headers; NFCSanjoy Das2016-08-061-7/+0
| | | | llvm-svn: 277892
* [IRCE] Preserve loop-simplify formSanjoy Das2016-08-061-0/+2
| | | | | | | | Fixes PR28764. Right now there is no way to test this, but (as mentioned on the PR) with Michael Zolotukhin's yet to be checked in LoopSimplify verfier, 8 of the llvm-lit tests for IRCE crash. llvm-svn: 277891
* [InstCombine] refactor ctlz/cttz folds (NFCI)Sanjay Patel2016-08-051-34/+33
| | | | | | | | | | Note that this fold really belongs in InstSimplify. Refactoring here anyway as an intermediate step because there's a planned addition to this function in D23134. Differential Revision: https://reviews.llvm.org/D23223 llvm-svn: 277883
* [MSSA] Use depth first iterator instead of custom version.Daniel Berlin2016-08-051-19/+3
| | | | | | | | | | | | | | | | | Summary: Originally the plan was to use the custom worklist to do some block popping, and because we don't actually need a visited set. The custom one we have here is slightly broken, and it's not worth fixing vs using depth_first_iterator since we aren't going to go the route we originally were. Fixes PR28874 Reviewers: george.burgess.iv Subscribers: llvm-commits, gberry Differential Revision: https://reviews.llvm.org/D23187 llvm-svn: 277880
* [LoopSimplify] Fix updating LCSSA after separating nested loops.Michael Zolotukhin2016-08-051-0/+15
| | | | | | | | | This fixes PR28825. The problem was that we only checked if a value from a created inner loop is used in the outer loop, and fixed LCSSA for them. But we missed to fixup LCSSA for values used in exits of the outer loop. llvm-svn: 277877
* [MSSA] Match assert vs llvm_unreachable style in verification functions.Daniel Berlin2016-08-051-11/+12
| | | | llvm-svn: 277873
* Rewrite domination verifier to handle local domination as well.Daniel Berlin2016-08-051-38/+19
| | | | | | | | | | | | | | Summary: Rewrite domination verifier to handle local domination as well. This catches a bug Geoff Berry noticed. Reviewers: george.burgess.iv Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D23184 llvm-svn: 277872
* [FlattenCFG] Simplify + remove unused variable. NFCI.Davide Italiano2016-08-051-7/+2
| | | | llvm-svn: 277864
* WholeProgramDevirt: print remarks with devirtualized method names.Ivan Krasin2016-08-051-2/+18
| | | | | | | | | | | | | | | | | | Summary: Chrome on Linux uses WholeProgramDevirt for speed ups, and it's important to detect regressions on both sides: the toolchain, if fewer methods get devirtualized after an update, and Chrome, if an innocently looking change caused many hot methods become virtual again. The need to track devirtualized methods is not Chrome-specific, but it's probably the only user of the pass at this time. Reviewers: kcc Differential Revision: https://reviews.llvm.org/D23219 llvm-svn: 277856
* [ADCE] Refactoring for new functionality (NFC)David Callahan2016-08-051-46/+84
| | | | | | | | | | | | | | Summary: This is another refactoring to break up the one function into three logical components functions. Another non-functional change before we start added in features. Reviewers: nadav, mehdi_amini, majnemer Subscribers: twoh, freik, llvm-commits Differential Revision: https://reviews.llvm.org/D23102 llvm-svn: 277855
OpenPOWER on IntegriCloud