summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/Scalar
Commit message (Collapse)AuthorAgeFilesLines
...
* Refactor LICM pass in preparation for LoopSink pass.Dehao Chen2016-10-031-8/+17
| | | | | | | | | | | | Summary: LoopSink pass uses some common function in LICM. This patch refactor the LICM code to make it usable by LoopSink pass (https://reviews.llvm.org/D22778). Reviewers: davidxl, danielcdh, hfinkel, chandlerc Subscribers: hfinkel, llvm-commits Differential Revision: https://reviews.llvm.org/D24168 llvm-svn: 283134
* Jump threading: avoid trying to split edge into landingpad block (PR27840)Hans Wennborg2016-10-031-0/+4
| | | | | | | | | Splitting the edge is nontrivial because of the landing pad, and we would currently assert trying to do it. Differential Revision: https://reviews.llvm.org/D24680 llvm-svn: 283129
* Use StringRef in Pass/PassManager APIs (NFC)Mehdi Amini2016-10-016-8/+6
| | | | llvm-svn: 283004
* Update loop unroller cost model to make sure debug info does not affect ↵Dehao Chen2016-09-301-0/+3
| | | | | | | | | | | | | | optimization decisions. Summary: Debug info should *not* affect optimization decisions. This patch updates loop unroller cost model to make it not affected by debug info. Reviewers: davidxl, mzolotukhin Subscribers: haicheng, llvm-commits, mzolotukhin Differential Revision: https://reviews.llvm.org/D25098 llvm-svn: 282894
* CVP. Turn marking adds as no wrap on by default (was turned off by 279082)Artur Pilipenko2016-09-301-1/+1
| | | | | | With 282650 in tree extra no wrap on adds doesn't cause regressions anymore. Reenable the optimzation. llvm-svn: 282872
* [LDist] Port to new streaming API for opt remarksAdam Nemet2016-09-301-17/+26
| | | | llvm-svn: 282838
* [LoopUnroll] Port to the new streaming interface for opt remarks.Adam Nemet2016-09-301-21/+28
| | | | llvm-svn: 282834
* [LoopDataPrefetch] Port to new streaming API for opt remarksAdam Nemet2016-09-301-1/+2
| | | | llvm-svn: 282826
* Wisely choose sext or zext when widening IV.Evgeny Stupachenko2016-09-281-37/+88
| | | | | | | | | | | | Summary: The patch fixes regression caused by two earlier patches D18777 and D18867. Reviewers: reames, sanjoy Differential Revision: http://reviews.llvm.org/D24280 From: Li Huang llvm-svn: 282650
* [SystemZ] Implementation of getUnrollingPreferences().Jonas Paulsson2016-09-281-6/+3
| | | | | | | | | | | | | | This commit enables more unrolling for SystemZ by implementing the SystemZTargetTransformInfo::getUnrollingPreferences() method. It has been found that it is better to only unroll moderately, so the DefaultUnrollRuntimeCount has been moved into UnrollingPreferences in order to set this to a lower value for SystemZ (4). Reviewers: Evgeny Stupachenko, Ulrich Weigand. https://reviews.llvm.org/D24451 llvm-svn: 282570
* Shorten DiagnosticInfoOptimizationRemark* to OptimizationRemark*. NFCAdam Nemet2016-09-271-3/+2
| | | | | | | With the new streaming interface, these class names need to be typed a lot and it's way too looong. llvm-svn: 282544
* Scalar: Ignore ConstantData in processAssumptionDuncan P. N. Exon Smith2016-09-241-0/+5
| | | | | | | | | | | | Assumptions on UndefValue and ConstantPointerNull aren't relevant to other users. Ignore them entirely to avoid wasting cycles walking through their (possibly extremely extensive (cross-module)) use-lists. It wasn't clear how to add a specific test for this, and it'll be covered anyway by an eventual patch that asserts when trying to access the use-list of an instance of ConstantData. llvm-svn: 282334
* Revert r282168 "GVN-hoist: fix store past load dependence analysis (PR30216)"Hans Wennborg2016-09-221-35/+28
| | | | | | | | and also the dependent r282175 "GVN-hoist: do not dereference null pointers" It's causing compiler crashes building Harfbuzz (PR30499). llvm-svn: 282199
* GVN-hoist: do not dereference null pointersSebastian Pop2016-09-221-0/+3
| | | | | | | there may be basic blocks without memory accesses, in which case the list of accesses is a null pointer. llvm-svn: 282175
* GVN-hoist: fix store past load dependence analysis (PR30216)Sebastian Pop2016-09-221-29/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | | To hoist stores past loads, we used to search for potential conflicting loads on the hoisting path by following a MemorySSA def-def link from the store to be hoisted to the previous defining memory access, and from there we followed the def-use chains to all the uses that occur on the hoisting path. The problem is that the def-def link may point to a store that does not alias with the store to be hoisted, and so the loads that are walked may not alias with the store to be hoisted, and even as in the testcase of PR30216, the loads that may alias with the store to be hoisted are not visited. The current patch visits all loads on the path from the store to be hoisted to the hoisting position and uses the alias analysis to ask whether the store may alias the load. I was not able to use the MemorySSA functionality to ask for whether load and store are clobbered: I'm not sure which function to call, so I used a call to AA->isNoAlias(). Store past store is still working as before using a MemorySSA query: I added an extra test to pr30216.ll to make sure store past store does not regress. Differential Revision: https://reviews.llvm.org/D24517 llvm-svn: 282168
* GVN-hoist: fix typoSebastian Pop2016-09-221-1/+1
| | | | llvm-svn: 282165
* GVN-hoist: only hoist relevant scalar instructionsSebastian Pop2016-09-221-0/+4
| | | | | | | | | | | | | | | | | | | Without this patch, GVN-hoist would think that a branch instruction is a scalar instruction and would try to value number it. The patch filters out all such kind of irrelevant instructions. A bit frustrating is that there is no easy way to discard all those very infrequent instructions, a bit like isa<TerminatorInst> that stands for a large family of instructions. I'm thinking that checking for those very infrequent other instructions would cost us more in compilation time than just letting those instructions getting numbered, so I'm still thinking that a simpler check: if (isa<TerminatorInst>(I)) return false; is better than listing all the other less frequent instructions. Differential Revision: https://reviews.llvm.org/D23929 llvm-svn: 282160
* [RS4GC] Remat in presence of phi and use live valueAnna Thomas2016-09-221-8/+21
| | | | | | | | | | Summary: Reviewers: Subscribers: llvm-svn: 282150
* Fix revision 281960Dorit Nuzman2016-09-221-2/+4
| | | | llvm-svn: 282139
* [LoopInterchange] Track all dependencies, not just anti dependencies.Chad Rosier2016-09-211-50/+47
| | | | | | | | | | | | | | | | | Currently, we give up on loop interchange if we encounter a flow dependency anywhere in the loop list. Worse yet, we don't even track output dependencies. This patch updates the dependency matrix computation to track flow and output dependencies in the same way we track anti dependencies. This improves an internal workload by 2.2x. Note the loop interchange pass is off by default and it can be enabled with '-mllvm -enable-loopinterchange' Differential Revision: https://reviews.llvm.org/D24564 llvm-svn: 282101
* [LoopInterchange] Various cleanup. NFC.Chad Rosier2016-09-211-23/+19
| | | | llvm-svn: 282071
* [RS4GC] Refactor code for Rematerializing in presence of phi. NFCAnna Thomas2016-09-201-59/+68
| | | | | | | | | | | | | | | | | | | | | | | | Summary: This is an NFC refactoring change as a precursor to the actual fix for rematerializing in presence of phi. https://reviews.llvm.org/D24399 Pasted from review: findRematerializableChainToBasePointer changed to return the root of the chain. instead of true or false. move the PHI matching logic into the caller by inspecting the root return value. This includes an assertion that the alternate root is in the liveset for the call. Tested with current RS4GC tests. Reviewers: reames, sanjoy Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D24780 llvm-svn: 282023
* Reverting revision 281960 due to test failures.Dorit Nuzman2016-09-201-10/+2
| | | | llvm-svn: 281961
* [SROA] Preserve llvm.mem.parallel_loop_access metadata.Dorit Nuzman2016-09-201-2/+10
| | | | | | | | | | | | SROA doesn't preserve the llvm.mem.parallel_loop_access metadata when it transforms loads/stores. This patch fixes a couple occurences of this issue. (Partially addresses PR28981). Differential Revision: https://reviews.llvm.org/D23549 llvm-svn: 281960
* Merge branch 'ADCE5'David Callahan2016-09-191-2/+39
| | | | llvm-svn: 281947
* LoopDistribute should preserve GlobalsAA.Eli Friedman2016-09-161-0/+2
| | | | | | Differential Revision: https://reviews.llvm.org/D24204 llvm-svn: 281758
* LoopLoadElimination should preserve GlobalsAA.Eli Friedman2016-09-161-0/+2
| | | | | | | | Avoids losing GlobalsAA in the standard pass pipeline. Differential Revision: https://reviews.llvm.org/D24094 llvm-svn: 281757
* [LoopInterchange] Typo. NFC.Chad Rosier2016-09-141-4/+4
| | | | llvm-svn: 281501
* [LoopInterchange] Add CL option to override cost threshold.Chad Rosier2016-09-141-3/+6
| | | | | | Mostly useful for getting consistent lit testing. llvm-svn: 281500
* [LoopInterchange] Cleanup debug whitespace. NFC.Chad Rosier2016-09-141-4/+4
| | | | llvm-svn: 281497
* [LoopInterchange] Minor refactor. NFC.Chad Rosier2016-09-131-12/+11
| | | | llvm-svn: 281334
* Don't use else if after return. Tidy comments. NFC.Chad Rosier2016-09-131-5/+3
| | | | llvm-svn: 281331
* Typo. NFC.Chad Rosier2016-09-131-3/+3
| | | | llvm-svn: 281330
* [LoopInterchange] Tidy up and remove unnecessary dyn_casts. NFC.Chad Rosier2016-09-131-13/+12
| | | | llvm-svn: 281328
* [LoopInterchange] Improve debug output. NFC.Chad Rosier2016-09-121-2/+2
| | | | llvm-svn: 281212
* ScalarOpts: Use std::list for Candidates, NFCDuncan P. N. Exon Smith2016-09-111-2/+3
| | | | | | | There is nothing intrusive about the Candidate list; use std::list over llvm::ilist for simplicity. llvm-svn: 281177
* ScalarOpts: Sort includes, NFCDuncan P. N. Exon Smith2016-09-111-2/+1
| | | | llvm-svn: 281176
* [LoopDataPrefetch] Use range based for loop; NFCIBalaram Makam2016-09-081-17/+12
| | | | | | | Switch to range based for loop. No functional change, but more readable code. llvm-svn: 280966
* revert r280427Dehao Chen2016-09-081-2/+2
| | | | | | | Refactor replaceDominatedUsesWith to have a flag to control whether to replace uses in BB itself. Summary: This is in preparation for LoopSink pass which calls replaceDominatedUsesWith to update after sinking. llvm-svn: 280949
* IR: Remove Value::intersectOptionalDataWith, replace all calls with calls to ↵Peter Collingbourne2016-09-072-4/+4
| | | | | | | | | | Instruction::andIRFlags. The two functions are functionally equivalent. Differential Revision: https://reviews.llvm.org/D22830 llvm-svn: 280884
* [LoopUnroll] Correct a debug message. NFC.Haicheng Wu2016-09-071-1/+1
| | | | | | Differential Revision: https://reviews.llvm.org/D24299 llvm-svn: 280865
* Typo. NFC.Chad Rosier2016-09-071-1/+1
| | | | llvm-svn: 280834
* [LoopInterchange] Improve debug output. NFC.Chad Rosier2016-09-071-6/+6
| | | | llvm-svn: 280820
* [LoopInterchange] Improve debug output. NFC.Chad Rosier2016-09-071-4/+6
| | | | llvm-svn: 280819
* Fix typo in comment, NFCNick Lewycky2016-09-071-1/+1
| | | | llvm-svn: 280774
* [JumpThreading] Only write back branch-weight MDs for blocks that originally ↵Adam Nemet2016-09-061-1/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | had PGO info Currently the pass updates branch weights in the IR if the function has any PGO info (entry frequency is set). However we could still have regions of the CFG that does not have branch weights collected (e.g. a cold region). In this case we'd use static estimates. Since static estimates for branches are determined independently, they are inconsistent. Updating them can "randomly" inflate block frequencies. I've run into this in a completely cold loop of h264ref from SPEC. -Rpass-with-hotness showed the loop to be completely cold during inlining (before JT) but completely hot during vectorization (after JT). The new testcase demonstrate the problem. We check array elements against 1, 2 and 3 in a loop. The check against 3 is the loop-exiting check. The block names should be self-explanatory. In this example, jump threading incorrectly updates the weight of the loop-exiting branch to 0, drastically inflating the frequency of the loop (in the range of billions). There is no run-time profile info for edges inside the loop, so branch probabilities are estimated. These are the resulting branch and block frequencies for the loop body: check_1 (16) (8) / | eq_1 | (8) \ | check_2 (16) (8) / | eq_2 | (8) \ | check_3 (16) (1) / | (loop exit) | (15) | (back edge) First we thread eq_1 -> check_2 to check_3. Frequencies are updated to remove the frequency of eq_1 from check_2 and then from the false edge leaving check_2. Changed frequencies are highlighted with * *: check_1 (16) (8) / | eq_1~ | (8) / | / check_2 (*8*) / (8) / | \ eq_2 | (*0*) \ \ | ` --- check_3 (16) (1) / | (loop exit) | (15) | (back edge) Next we thread eq_1 -> check_3 and eq_2 -> check_3 to check_1 as new back edges. Frequencies are updated to remove the frequency of eq_1 and eq_3 from check_3 and then the false edge leaving check_3 (changed frequencies are highlighted with * *): check_1 (16) (8) / | eq_1~ | (8) / | / check_2 (*8*) / (8) / | /-- eq_2~ | (*0*) (back edge) | check_3 (*0*) (*0*) / | (loop exit) | (*0*) | (back edge) As a result, the loop exit edge ends up with 0 frequency which in turn makes the loop header to have maximum frequency. There are a few potential problems here: 1. The profile data seems odd. There is a single profile sample of the loop being entered. On the other hand, there are no weights inside the loop. 2. Based on static estimation we shouldn't set edges to "extreme" values, i.e. extremely likely or unlikely. 3. We shouldn't create profile metadata that is calculated from static estimation. I am not sure what policy is but it seems to make sense to treat profile metadata as something that is known to originate from profiling. Estimated probabilities should only be reflected in BPI/BFI. Any one of these would probably fix the immediate problem. I went for 3 because I think it's a good policy to have and added a FIXME about 2. Differential Revision: https://reviews.llvm.org/D24118 llvm-svn: 280713
* Cleanup : Use metadata preserving API for branch creationXinliang David Li2016-09-031-9/+4
| | | | | | | Use the wrapper API in IRBuilder that does meta data copy to create new branch in LoopUnswitch. llvm-svn: 280602
* [Profile] handle select instruction in 'expect' loweringXinliang David Li2016-09-021-11/+25
| | | | | | | | | Builtin expect lowering currently ignores select. This patch fixes the issue Differential Revision: http://reviews.llvm.org/D24166 llvm-svn: 280547
* revert r280429 and r280425:Dehao Chen2016-09-021-22/+24
| | | | | | | | | | | | | | | r280425 | dehao | 2016-09-01 16:15:50 -0700 (Thu, 01 Sep 2016) | 9 lines Refactor LICM pass in preparation for LoopSink pass. Summary: LoopSink pass uses some common function in LICM. This patch refactor the LICM code to make it usable by LoopSink pass (https://reviews.llvm.org/D22778). r280429 | dehao | 2016-09-01 16:31:25 -0700 (Thu, 01 Sep 2016) | 9 lines Refactor LICM to expose canSinkOrHoistInst to LoopSink pass. Summary: LoopSink pass shares the same canSinkOrHoistInst functionality with LICM pass. This patch exposes this function in preparation of https://reviews.llvm.org/D22778 llvm-svn: 280453
* Refactor LICM to expose canSinkOrHoistInst to LoopSink pass.Dehao Chen2016-09-011-7/+3
| | | | | | | | | | | | Summary: LoopSink pass shares the same canSinkOrHoistInst functionality with LICM pass. This patch exposes this function in preparation of https://reviews.llvm.org/D22778 Reviewers: chandlerc, davidxl, danielcdh Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D24171 llvm-svn: 280429
OpenPOWER on IntegriCloud