summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/Scalar/Scalar.cpp
Commit message (Collapse)AuthorAgeFilesLines
...
* Sort the remaining #include lines in include/... and lib/....Chandler Carruth2017-06-061-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | I did this a long time ago with a janky python script, but now clang-format has built-in support for this. I fed clang-format every line with a #include and let it re-sort things according to the precise LLVM rules for include ordering baked into clang-format these days. I've reverted a number of files where the results of sorting includes isn't healthy. Either places where we have legacy code relying on particular include ordering (where possible, I'll fix these separately) or where we have particular formatting around #include lines that I didn't want to disturb in this patch. This patch is *entirely* mechanical. If you get merge conflicts or anything, just ignore the changes in this patch and run clang-format over your #include lines in the files. Sorry for any noise here, but it is important to keep these things stable. I was seeing an increasing number of patches with irrelevant re-ordering of #include lines because clang-format was used. This patch at least isolates that churn, makes it easy to skip when resolving conflicts, and gets us to a clean baseline (again). llvm-svn: 304787
* [GVNSink] GVNSink passJames Molloy2017-05-251-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch provides an initial prototype for a pass that sinks instructions based on GVN information, similar to GVNHoist. It is not yet ready for commiting but I've uploaded it to gather some initial thoughts. This pass attempts to sink instructions into successors, reducing static instruction count and enabling if-conversion. We use a variant of global value numbering to decide what can be sunk. Consider: [ %a1 = add i32 %b, 1 ] [ %c1 = add i32 %d, 1 ] [ %a2 = xor i32 %a1, 1 ] [ %c2 = xor i32 %c1, 1 ] \ / [ %e = phi i32 %a2, %c2 ] [ add i32 %e, 4 ] GVN would number %a1 and %c1 differently because they compute different results - the VN of an instruction is a function of its opcode and the transitive closure of its operands. This is the key property for hoisting and CSE. What we want when sinking however is for a numbering that is a function of the *uses* of an instruction, which allows us to answer the question "if I replace %a1 with %c1, will it contribute in an equivalent way to all successive instructions?". The (new) PostValueTable class in GVN provides this mapping. This pass has some shown really impressive improvements especially for codesize already on internal benchmarks, so I have high hopes it can replace all the sinking logic in SimplifyCFG. Differential revision: https://reviews.llvm.org/D24805 llvm-svn: 303850
* [PM/LoopUnswitch] Introduce a new, simpler loop unswitch pass.Chandler Carruth2017-04-271-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, this pass only focuses on *trivial* loop unswitching. At that reduced problem it remains significantly better than the current loop unswitch: - Old pass is worse than cubic complexity. New pass is (I think) linear. - New pass is much simpler in its design by focusing on full unswitching. (See below for details on this). - New pass doesn't carry state for thresholds between pass iterations. - New pass doesn't carry state for correctness (both miscompile and infloop) between pass iterations. - New pass produces substantially better code after unswitching. - New pass can handle more trivial unswitch cases. - New pass doesn't recompute the dominator tree for the entire function and instead incrementally updates it. I've ported all of the trivial unswitching test cases from the old pass to the new one to make sure that major functionality isn't lost in the process. For several of the test cases I've worked to improve the precision and rigor of the CHECKs, but for many I've just updated them to handle the new IR produced. My initial motivation was the fact that the old pass carried state in very unreliable ways between pass iterations, and these mechansims were incompatible with the new pass manager. However, I discovered many more improvements to make along the way. This pass makes two very significant assumptions that enable most of these improvements: 1) Focus on *full* unswitching -- that is, completely removing whatever control flow construct is being unswitched from the loop. In the case of trivial unswitching, this means removing the trivial (exiting) edge. In non-trivial unswitching, this means removing the branch or switch itself. This is in opposition to *partial* unswitching where some part of the unswitched control flow remains in the loop. Partial unswitching only really applies to switches and to folded branches. These are very similar to full unrolling and partial unrolling. The full form is an effective canonicalization, the partial form needs a complex cost model, cannot be iterated, isn't canonicalizing, and should be a separate pass that runs very late (much like unrolling). 2) Leverage LLVM's Loop machinery to the fullest. The original unswitch dates from a time when a great deal of LLVM's loop infrastructure was missing, ineffective, and/or unreliable. As a consequence, a lot of complexity was added which we no longer need. With these two overarching principles, I think we can build a fast and effective unswitcher that fits in well in the new PM and in the canonicalization pipeline. Some of the remaining functionality around partial unswitching may not be relevant today (not many test cases or benchmarks I can find) but if they are I'd like to add support for them as a separate layer that runs very late in the pipeline. Purely to make reviewing and introducing this code more manageable, I've split this into first a trivial-unswitch-only pass and in the next patch I'll add support for full non-trivial unswitching against a *fixed* threshold, exactly like full unrolling. I even plan to re-use the unrolling thresholds, as these are incredibly similar cost tradeoffs: we're cloning a loop body in order to end up with simplified control flow. We should only do that when the total growth is reasonably small. One of the biggest changes with this pass compared to the previous one is that previously, each individual trivial exiting edge from a switch was unswitched separately as a branch. Now, we unswitch the entire switch at once, with cases going to the various destinations. This lets us unswitch multiple exiting edges in a single operation and also avoids numerous extremely bad behaviors, where we would introduce 1000s of branches to test for thousands of possible values, all of which would take the exact same exit path bypassing the loop. Now we will use a switch with 1000s of cases that can be efficiently lowered into a jumptable. This avoids relying on somehow forming a switch out of the branches or getting horrible code if that fails for any reason. Another significant change is that this pass actively updates the CFG based on unswitching. For trivial unswitching, this is actually very easy because of the definition of loop simplified form. Doing this makes the code coming out of loop unswitch dramatically more friendly. We still should run loop-simplifycfg (at the least) after this to clean up, but it will have to do a lot less work. Finally, this pass makes much fewer attempts to simplify instructions based on the unswitch. Something like loop-instsimplify, instcombine, or GVN can be used to do increasingly powerful simplifications based on the now dominating predicate. The old simplifications are things that something like loop-instsimplify should get today or a very, very basic loop-instcombine could get. Keeping that logic separate is a big simplifying technique. Most of the code in this pass that isn't in the old one has to do with achieving specific goals: - Updating the dominator tree as we go - Unswitching all cases in a switch in a single step. I think it is still shorter than just the trivial unswitching code in the old pass despite having this functionality. Differential Revision: https://reviews.llvm.org/D32409 llvm-svn: 301576
* Split the SimplifyCFG pass into two variants.Joerg Sonnenberger2017-03-261-0/+5
| | | | | | | | | | | | | | | | | | | | | | | The first variant contains all current transformations except transforming switches into lookup tables. The second variant contains all current transformations. The switch-to-lookup-table conversion results in code that is more difficult to analyze and optimize by other passes. Most importantly, it can inhibit Dead Code Elimination. As such it is often beneficial to only apply this transformation very late. A common example is inlining, which can often result in range restrictions for the switch expression. Changes in execution time according to LNT: SingleSource/Benchmarks/Misc/fp-convert +3.03% MultiSource/Benchmarks/ASC_Sequoia/CrystalMk/CrystalMk -11.20% MultiSource/Benchmarks/Olden/perimeter/perimeter -10.43% and a couple of smaller changes. For perimeter it also results 2.6% a smaller binary. Differential Revision: https://reviews.llvm.org/D30333 llvm-svn: 298799
* Split NewGVN class into a legacy pass and an impl, instead of a merged class.Daniel Berlin2017-03-121-1/+1
| | | | llvm-svn: 297576
* NVPTX: Move InferAddressSpaces to generic codeMatt Arsenault2017-01-311-0/+1
| | | | llvm-svn: 293579
* [Guards] Introduce loop-predication passArtur Pilipenko2017-01-251-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch introduces guard based loop predication optimization. The new LoopPredication pass tries to convert loop variant range checks to loop invariant by widening checks across loop iterations. For example, it will convert for (i = 0; i < n; i++) { guard(i < len); ... } to for (i = 0; i < n; i++) { guard(n - 1 < len); ... } After this transformation the condition of the guard is loop invariant, so loop-unswitch can later unswitch the loop by this condition which basically predicates the loop by the widened condition: if (n - 1 < len) for (i = 0; i < n; i++) { ... } else deoptimize This patch relies on an NFC change to make ScalarEvolution::isMonotonicPredicate public (revision 293062). Reviewed By: sanjoy Differential Revision: https://reviews.llvm.org/D29034 llvm-svn: 293064
* [GVN] Initial check-in of a new global value numbering algorithm.Davide Italiano2016-12-221-0/+5
| | | | | | | | | | | | | | | | | | | | | The code have been developed by Daniel Berlin over the years, and the new implementation goal is that of addressing shortcomings of the current GVN infrastructure, i.e. long compile time for large testcases, lack of phi predication, no load/store value numbering etc... The current code just implements the "core" GVN algorithm, although other pieces (load coercion, phi handling, predicate system) are already implemented in a branch out of tree. Once the core is stable, we'll start adding pieces on top of the base framework. The test currently living in test/Transform/NewGVN are a copy of the ones in GVN, with proper `XFAIL` (missing features in NewGVN). A flag will be added in a future commit to enable NewGVN, so that interested parties can exercise this code easily. Differential Revision: https://reviews.llvm.org/D26224 llvm-svn: 290346
* Add Loop Sink pass to reverse the LICM based of basic block frequency.Dehao Chen2016-10-271-0/+5
| | | | | | | | | | | | Summary: LICM may hoist instructions to preheader speculatively. Before code generation, we need to sink down the hoisted instructions inside to loop if it's beneficial. This pass is a reverse of LICM: looking at instructions in preheader and sinks the instruction to basic blocks inside the loop body if basic block frequency is smaller than the preheader frequency. Reviewers: hfinkel, davidxl, chandlerc Subscribers: anna, modocache, mgorny, beanz, reames, dberlin, chandlerc, mcrosier, junbuml, sanjoy, mzolotukhin, llvm-commits Differential Revision: https://reviews.llvm.org/D22778 llvm-svn: 285308
* [EarlyCSE] Change C API pass interface for EarlyCSE w/ MemorySSAGeoff Berry2016-09-011-2/+6
| | | | | | | | | | Previous change broke the C API for creating an EarlyCSE pass w/ MemorySSA by adding a bool parameter to control whether MemorySSA was used or not. This broke the OCaml bindings. Instead, change the old C API entry point back and add a new one to request an EarlyCSE pass with MemorySSA. llvm-svn: 280379
* [EarlyCSE] Optionally use MemorySSA. NFC.Geoff Berry2016-08-311-2/+3
| | | | | | | | | | | | | | | | | Summary: Use MemorySSA, if requested, to do less conservative memory dependency checking. This change doesn't enable the MemorySSA enhanced EarlyCSE in the default pipelines, so should be NFC. Reviewers: dberlin, sanjoy, reames, majnemer Subscribers: mcrosier, llvm-commits Differential Revision: http://reviews.llvm.org/D19821 llvm-svn: 280279
* [PM] Port LoopDataPrefetch to new pass managerTeresa Johnson2016-08-131-1/+1
| | | | | | | | | | | | | | | | Summary: Refactor the existing support into a LoopDataPrefetch implementation class and a LoopDataPrefetchLegacyPass class that invokes it. Add a new LoopDataPrefetchPass for the new pass manager that utilizes the LoopDataPrefetch implementation class. Reviewers: mehdi_amini Subscribers: sanjoy, mzolotukhin, nemanjai, llvm-commits Differential Revision: https://reviews.llvm.org/D23483 llvm-svn: 278591
* [PM] Port SpeculativeExecution to the new PMMichael Kuperstein2016-08-011-1/+1
| | | | | | Differential Revision: https://reviews.llvm.org/D23033 llvm-svn: 277393
* [PM] Port LowerGuardIntrinsic to the new PM.Michael Kuperstein2016-07-281-1/+1
| | | | llvm-svn: 277057
* [PM] Port NaryReassociate to the new PMWei Mi2016-07-211-1/+1
| | | | | | Differential Revision: https://reviews.llvm.org/D22648 llvm-svn: 276349
* [LoopDist] Port to new PMAdam Nemet2016-07-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | Summary: The direct motivation for the port is to ensure that the OptRemarkEmitter tests work with the new PM. This remains a function pass because we not only create multiple loops but could also version the original loop. In the test I need to invoke opt with -passes='require<aa>,loop-distribute'. LoopDistribute does not directly depend on AA however LAA does. LAA uses getCachedResult so I *think* we need manually pull in 'aa'. Reviewers: davidxl, silvas Subscribers: sanjoy, llvm-commits, mzolotukhin Differential Revision: https://reviews.llvm.org/D22437 llvm-svn: 275811
* [PM] Convert LoopInstSimplify Pass to new PMDehao Chen2016-07-151-1/+1
| | | | | | | | | | | | Summary: Convert LoopInstSimplify to new PM. Unfortunately there is no exisiting unittest for this pass. Reviewers: davidxl, silvas Subscribers: silvas, llvm-commits, mzolotukhin Differential Revision: https://reviews.llvm.org/D22280 llvm-svn: 275576
* code hoisting pass based on GVNSebastian Pop2016-07-151-0/+5
| | | | | | | | | | | | | This pass hoists duplicated computations in the program. The primary goal of gvn-hoist is to reduce the size of functions before inline heuristics to reduce the total cost of function inlining. Pass written by Sebastian Pop, Aditya Kumar, Xiaoyu Hu, and Brian Rzycki. Important algorithmic contributions by Daniel Berlin under the form of reviews. Differential Revision: http://reviews.llvm.org/D19338 llvm-svn: 275561
* [PM] Port Dead Loop Deletion Pass to the new PMJun Bum Lim2016-07-141-1/+1
| | | | | | | | | | | | Summary: Port Dead Loop Deletion Pass to the new pass manager. Reviewers: silvas, davide Subscribers: llvm-commits, sanjoy, mcrosier Differential Revision: https://reviews.llvm.org/D21483 llvm-svn: 275453
* Revert r275401, it caused PR28551.Nico Weber2016-07-141-5/+0
| | | | llvm-svn: 275420
* code hoisting pass based on GVNSebastian Pop2016-07-141-0/+5
| | | | | | | | | | | | | This pass hoists duplicated computations in the program. The primary goal of gvn-hoist is to reduce the size of functions before inline heuristics to reduce the total cost of function inlining. Pass written by Sebastian Pop, Aditya Kumar, Xiaoyu Hu, and Brian Rzycki. Important algorithmic contributions by Daniel Berlin under the form of reviews. Differential Revision: http://reviews.llvm.org/D19338 llvm-svn: 275401
* New pass manager for LICM.Dehao Chen2016-07-121-1/+1
| | | | | | | | | | | | Summary: Port LICM to the new pass manager. Reviewers: davidxl, silvas Subscribers: krasin, vitalybuka, silvas, davide, sanjoy, llvm-commits, mehdi_amini Differential Revision: http://reviews.llvm.org/D21772 llvm-svn: 275222
* [PM] Port LoopIdiomRecognize Pass to new PMDehao Chen2016-07-121-1/+1
| | | | | | | | | | | | Summary: Port LoopIdiomRecognize Pass to new PM Reviewers: davidxl Subscribers: davide, sanjoy, mzolotukhin, llvm-commits Differential Revision: http://reviews.llvm.org/D22250 llvm-svn: 275202
* Revert "New pass manager for LICM."Vitaly Buka2016-07-121-1/+1
| | | | | | | | | | Summary: This reverts commit r275118. Subscribers: sanjoy, mehdi_amini Differential Revision: http://reviews.llvm.org/D22259 llvm-svn: 275156
* New pass manager for LICM.Dehao Chen2016-07-111-1/+1
| | | | | | | | | | | | Summary: Port LICM to the new pass manager. Reviewers: davidxl, silvas Subscribers: silvas, davide, sanjoy, llvm-commits, mehdi_amini Differential Revision: http://reviews.llvm.org/D21772 llvm-svn: 275118
* Rename LoopAccessAnalysis to LoopAccessLegacyAnalysis /NFCXinliang David Li2016-07-081-1/+1
| | | | llvm-svn: 274927
* [PM] Port ConstantHoisting to the new Pass ManagerMichael Kuperstein2016-07-021-1/+1
| | | | | | Differential Revision: http://reviews.llvm.org/D21945 llvm-svn: 274411
* Revert "code hoisting pass based on GVN"Duncan P. N. Exon Smith2016-07-011-5/+0
| | | | | | | | | | | | This reverts commit r274305, since it breaks self-hosting: http://lab.llvm.org:8080/green/job/clang-stage1-configure-RA_build/22349/ http://lab.llvm.org:8011/builders/clang-x86_64-linux-selfhost-modules/builds/17232 Note that the blamelist on lab.llvm.org:8011 is incorrect. The previous build was r274299, but somehow r274305 wasn't included in the blamelist: http://lab.llvm.org:8011/builders/clang-x86_64-linux-selfhost-modules llvm-svn: 274320
* code hoisting pass based on GVNSebastian Pop2016-07-011-0/+5
| | | | | | | | | | | | | This pass hoists duplicated computations in the program. The primary goal of gvn-hoist is to reduce the size of functions before inline heuristics to reduce the total cost of function inlining. Pass written by Sebastian Pop, Aditya Kumar, Xiaoyu Hu, and Brian Rzycki. Important algorithmic contributions by Daniel Berlin under the form of reviews. Differential Revision: http://reviews.llvm.org/D19338 llvm-svn: 274305
* [PM] Port float2int to the new pass managerMichael Kuperstein2016-06-241-1/+1
| | | | | | Differential Revision: http://reviews.llvm.org/D21704 llvm-svn: 273747
* [PM] Port MergedLoadStoreMotion to the new pass manager, take two.Davide Italiano2016-06-171-1/+1
| | | | | | | | | This is indeed a much cleaner approach (thanks to Daniel Berlin for pointing out), and also David/Sean for review. Differential Revision: http://reviews.llvm.org/D21454 llvm-svn: 273032
* [PM] Revert the port of MergeLoadStoreMotion to the new pass manager.Davide Italiano2016-06-161-1/+1
| | | | | | | | Daniel Berlin expressed some real concerns about the port and proposed and alternative approach. I'll revert this for now while working on a new patch, which I hope to put up for review shortly. Sorry for the churn. llvm-svn: 272925
* Remove the ScalarReplAggregates passDavid Majnemer2016-06-151-5/+3
| | | | | | | | | | Nearly all the changes to this pass have been done while maintaining and updating other parts of LLVM. LLVM has had another pass, SROA, which has superseded ScalarReplAggregates for quite some time. Differential Revision: http://reviews.llvm.org/D21316 llvm-svn: 272737
* [PM] Port MemCpyOpt to the new PM.Sean Silva2016-06-141-1/+1
| | | | | | | | | The need for all these Lookup* functions is just because of calls to getAnalysis inside methods (i.e. not at the top level) of the runOnFunction method. They should be straightforward to clean up when the old PM is gone. llvm-svn: 272615
* [PM] Port MergedLoadStoreMotion to the new pass manager.Davide Italiano2016-06-141-1/+1
| | | | llvm-svn: 272606
* [IndVarSimplify] Extract the logic of `-indvars` out into a class; NFCSanjoy Das2016-05-291-1/+1
| | | | | | This will be used later to port IndVarSimplify to the new pass manager. llvm-svn: 271190
* [PM] Port PartiallyInlineLibCalls to the new pass manager.Davide Italiano2016-05-251-1/+1
| | | | llvm-svn: 270798
* [PM] Port BDCE to the new pass manager.Davide Italiano2016-05-251-1/+1
| | | | llvm-svn: 270647
* New pass: guard wideningSanjoy Das2016-05-181-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Implement guard widening in LLVM. Description from GuardWidening.cpp: The semantics of the `@llvm.experimental.guard` intrinsic lets LLVM transform it so that it fails more often that it did before the transform. This optimization is called "widening" and can be used hoist and common runtime checks in situations like these: ``` %cmp0 = 7 u< Length call @llvm.experimental.guard(i1 %cmp0) [ "deopt"(...) ] call @unknown_side_effects() %cmp1 = 9 u< Length call @llvm.experimental.guard(i1 %cmp1) [ "deopt"(...) ] ... ``` to ``` %cmp0 = 9 u< Length call @llvm.experimental.guard(i1 %cmp0) [ "deopt"(...) ] call @unknown_side_effects() ... ``` If `%cmp0` is false, `@llvm.experimental.guard` will "deoptimize" back to a generic implementation of the same function, which will have the correct semantics from that point onward. It is always _legal_ to deoptimize (so replacing `%cmp0` with false is "correct"), though it may not always be profitable to do so. NB! This pass is a work in progress. It hasn't been tuned to be "production ready" yet. It is known to have quadriatic running time and will not scale to large numbers of guards Reviewers: reames, atrick, bogner, apilipenko, nlewycky Subscribers: mcrosier, llvm-commits Differential Revision: http://reviews.llvm.org/D20143 llvm-svn: 269997
* [PM] Port per-function SCCP to the new pass manager.Davide Italiano2016-05-181-1/+1
| | | | llvm-svn: 269937
* [PM] Port DSE to the new pass managerJustin Bogner2016-05-171-1/+1
| | | | | | Patch by JakeVanAdrighem. Thanks! llvm-svn: 269847
* [PM] Port LowerAtomic to the new pass manager.Davide Italiano2016-05-131-1/+1
| | | | llvm-svn: 269511
* [PM] Port Interprocedural SCCP to the new pass manager.Davide Italiano2016-05-051-1/+1
| | | | llvm-svn: 268684
* PM: Port LoopRotation to the new loop pass managerJustin Bogner2016-05-031-1/+1
| | | | llvm-svn: 268452
* PM: Port LoopSimplifyCFG to the new pass managerJustin Bogner2016-05-031-1/+1
| | | | llvm-svn: 268446
* PM: Port Reassociate to the new pass managerJustin Bogner2016-04-261-1/+1
| | | | llvm-svn: 267631
* PM: Port SinkingPass to the new pass managerJustin Bogner2016-04-221-1/+1
| | | | llvm-svn: 267199
* PM: Port DCE to the new pass managerJustin Bogner2016-04-221-1/+1
| | | | | | | Also add a very basic test, since apparently there aren't any tests for DCE whatsoever to add the new pass version to. llvm-svn: 267196
* Introduce a @llvm.experimental.guard intrinsicSanjoy Das2016-03-311-0/+1
| | | | | | | | | | | | | | | | | | | | | | | Summary: As discussed on llvm-dev[1]. This change adds the basic boilerplate code around having this intrinsic in LLVM: - Changes in Intrinsics.td, and the IR Verifier - A lowering pass to lower @llvm.experimental.guard to normal control flow - Inliner support [1]: http://lists.llvm.org/pipermail/llvm-dev/2016-February/095523.html Reviewers: reames, atrick, chandlerc, rnk, JosephTremoulet, echristo Subscribers: mcrosier, llvm-commits Differential Revision: http://reviews.llvm.org/D18527 llvm-svn: 264976
* [PM] Port GVN to the new pass manager, wire it up, and teach a couple ofChandler Carruth2016-03-111-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | tests to run GVN in both modes. This is mostly the boring refactoring just like SROA and other complex transformation passes. There is some trickiness in that GVN's ValueNumber class requires hand holding to get to compile cleanly. I'm open to suggestions about a better pattern there, but I tried several before settling on this. I was trying to balance my desire to sink as much implementation detail into the source file as possible without introducing overly many layers of abstraction. Much like with SROA, the design of this system is made somewhat more cumbersome by the need to support both pass managers without duplicating the significant state and logic of the pass. The same compromise is struck here. I've also left a FIXME in a doxygen comment as the GVN pass seems to have pretty woeful documentation within it. I'd like to submit this with the FIXME and let those more deeply familiar backfill the information here now that we have a nice place in an interface to put that kind of documentaiton. Differential Revision: http://reviews.llvm.org/D18019 llvm-svn: 263208
OpenPOWER on IntegriCloud