summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/Scalar
Commit message (Collapse)AuthorAgeFilesLines
* [LoopUnswitch] Add an else clause to IsTrivialUnswitchCondition() when ↵Chen Li2015-07-151-1/+2
| | | | | | | | | | | | | | | | | checking HeaderTerm instruction type Summary: This is a trivial code change with no functionality effect. When LoopUnswitch determines trivial unswitch condition, it checks whether the loop header's terminator instruction is a branch instruction or switch instruction since trivial unswitch condition can only apply to these two instruction types. The current code does not fail the check directly on other instruction types, but check the nullness of LoopExitBB variable instead. The added else clause makes the check fail immediately on other instruction types and makes the code more obvious. Reviewers: reames Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D11239 llvm-svn: 242345
* [LoopUnrolling] Handle cast instructions.Michael Zolotukhin2015-07-151-0/+15
| | | | | | | | | During estimation of unrolling effect we should be able to propagate constants through casts. Differential Revision: http://reviews.llvm.org/D10207 llvm-svn: 242257
* [LAA] Introduce RuntimePointerChecking::PointerInfo, NFCAdam Nemet2015-07-141-2/+2
| | | | | | | Turn this structure-of-arrays (i.e. the various pointer attributes) into array-of-structures. llvm-svn: 242219
* [LAA] Lift RuntimePointerCheck out of LoopAccessInfo, NFCAdam Nemet2015-07-141-3/+2
| | | | | | | | | I am planning to add more nested classes inside RuntimePointerCheck so all these triple-nesting would be hard to follow. Also rename it to RuntimePointerChecking (i.e. append 'ing'). llvm-svn: 242218
* GVN: use a static array instead of regenerating it each time. NFC.Tim Northover2015-07-141-1/+1
| | | | llvm-svn: 242202
* GVN: tolerate an instruction being replaced without existing in the leaderboardTim Northover2015-07-141-1/+4
| | | | | | | | | | | | | | Sometimes an incidentally created instruction can duplicate a Value used elsewhere. It then often doesn't end up in the leader table. If it's later removed, we attempt to remove it from the leader table and segfault. Instead we should just ignore the removal request, which won't cause any problems. The reverse situation, where the original instruction is replaced by the new one (which you might think could leave the leader table empty) cannot occur, because the incidental instruction will never be found in the first place. llvm-svn: 242199
* [SROA] Don't de-atomic volatile loads and storesDavid Majnemer2015-07-141-6/+15
| | | | | | | | | | | Volatile loads and stores are made visible in global state regardless of what memory is involved. It is not correct to disregard the ordering and synchronization scope because it is possible to synchronize with memory operations performed by hardware. This partially addresses PR23737. llvm-svn: 242126
* Loop idiom recognizer was replacing too many uses of popcount.Pete Cooper2015-07-131-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | When spotting that a loop can use ctpop, we were incorrectly replacing all uses of a value with a value derived from ctpop. The bug here was exposed because we were replacing a use prior to the ctpop with the ctpop value and so we have a use before def, i.e., we changed %tobool.5 = icmp ne i32 %num, 0 store i1 %tobool.5, i1* %ptr br i1 %tobool.5, label %for.body.lr.ph, label %for.end to store i1 %1, i1* %ptr %0 = call i32 @llvm.ctpop.i32(i32 %num) %1 = icmp ne i32 %0, 0 br i1 %1, label %for.body.lr.ph, label %for.end Even if we inserted the ctpop so that it dominates the store here, that would still be incorrect. The store doesn’t want the result of ctpop. The fix is very simple, and involves replacing only the branch condition with the ctpop instead of all uses. Reviewed by Hal Finkel. llvm-svn: 242068
* Enable runtime unrolling with unroll pragma metadataMark Heffernan2015-07-131-2/+4
| | | | | | | | | | Enable runtime unrolling for loops with unroll count metadata ("#pragma unroll N") and a runtime trip count. Also, do not unroll loops with unroll full metadata if the loop has a runtime loop count. Previously, such loops would be unrolled with a very large threshold (pragma-unroll-threshold) if runtime unrolled happened to be enabled resulting in a very large (and likely unwise) unroll factor. llvm-svn: 242047
* Avoid using Loop::getSubLoopsVector.Benjamin Kramer2015-07-131-6/+6
| | | | | | | Passes should never modify it, just use the const version. While there reduce copying in LoopInterchange. No functional change intended. llvm-svn: 242041
* [LICM] Don't try to sink values out of loops without any exitsDavid Majnemer2015-07-121-1/+12
| | | | | | | | | | | | | There is no suitable basic block to sink instructions in loops without exits. The only way an instruction in a loop without exits can be used is as an incoming value to a PHI. In such cases, the incoming block for the corresponding value is unreachable. This fixes PR24013. Differential Revision: http://reviews.llvm.org/D10903 llvm-svn: 241987
* [PM/AA] Completely remove the AliasAnalysis::copyValue interface.Chandler Carruth2015-07-112-13/+1
| | | | | | | | | | | | | | | | | | | | | No in-tree alias analysis used this facility, and it was not called in any particularly rigorous way, so it seems unlikely to be correct. Note that one of the only stateful AA implementations in-tree, GlobalsModRef is completely broken currently (and any AA passes like it are equally broken) because Module AA passes are not effectively invalidated when a function pass that fails to update the AA stack runs. Ultimately, it doesn't seem like we know how we want to build stateful AA, and until then trying to support and maintain correctness for an untested API is essentially impossible. To that end, I'm planning to rip out all of the update API. It can return if and when we need it and know how to build it on top of the new pass manager and as part of *tested* stateful AA implementations in the tree. Differential Revision: http://reviews.llvm.org/D10889 llvm-svn: 241975
* [LoopDist/LoopVer] Move LoopVersioning to a new module, NFCAdam Nemet2015-07-101-115/+1
| | | | | | | | | | | | | | | | | | | Summary: The class will obviously need improvement down the road. For one, there is no reason that addPHINodes would have to be exposed like that. I will make this and other improvements in follow-up patches. The main goal is to be able to share this functionality. The LoopLoadElimination pass I am working on needs it too. Later we can move other clients as well (LV and Ashutosh's LICMVer). Reviewers: hfinkel, ashutosh.nema Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D10577 llvm-svn: 241932
* [LoopDist] Move loop-versioning helper functions to Cloning, NFCAdam Nemet2015-07-101-66/+4
| | | | | | | | | | | | | | Summary: This makes them available to the LoopVersioning class as that is moved to its own module in the next patch. Reviewers: ashutosh.nema, hfinkel Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D10576 llvm-svn: 241931
* Revert the new EH instructionsDavid Majnemer2015-07-104-11/+10
| | | | | | This reverts commits r241888-r241891, I didn't mean to commit them. llvm-svn: 241893
* New EH representation for MSVC compatibilityDavid Majnemer2015-07-104-10/+11
| | | | | | | | | | | | | | | Summary: This introduces new instructions neccessary to implement MSVC-compatible exception handling support. Most of the middle-end and none of the back-end haven't been audited or updated to take them into account. Reviewers: rnk, JosephTremoulet, reames, nlewycky, rjmccall Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D11041 llvm-svn: 241888
* [IndVars] Try to use existing values in RewriteLoopExitValues.Sanjoy Das2015-07-091-2/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: In RewriteLoopExitValues, before expanding out an SCEV expression using SCEVExpander, try to see if an existing LLVM IR expression already computes the value we're interested in. If so use that existing expression. Apart from reducing IndVars' reliance on the rest of the compilation pipeline, this also prevents IndVars from concluding some expressions as "high cost" when they're not. For instance, `InductiveRangeCheckElimination` often emits code of the following form: ``` len = umin(len_A, len_B) loop: ... if (i++ < len) goto loop outside_loop: use(i) ``` `SCEVExpander` refuses to rewrite the use of `i` in `outside_loop`, since it thinks the value of `i` on loop exit, `len`, is a high cost expansion since it contains an `umax` in it. With this change, `IndVars` can see that it can re-use `len` instead of creating a new expression to compute `umin(len_A, len_B)`. I considered putting this cleverness in `SCEVExpander`, but I was worried that it may then have a deterimental effect on other passes that use it. So I decided it was better to just do this in the one place where it seems like an obviously good idea, with the intent of generalizing later if needed. Reviewers: atrick, reames Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D10782 llvm-svn: 241838
* Rename llvm.frameescape and llvm.framerecover to localescape and localrecoverReid Kleckner2015-07-071-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Initially, these intrinsics seemed like part of a family of "frame" related intrinsics, but now I think that's more confusing than helpful. Initially, the LangRef specified that this would create a new kind of allocation that would be allocated at a fixed offset from the frame pointer (EBP/RBP). We ended up dropping that design, and leaving the stack frame layout alone. These intrinsics are really about sharing local stack allocations, not frame pointers. I intend to go further and add an `llvm.localaddress()` intrinsic that returns whatever register (EBP, ESI, ESP, RBX) is being used to address locals, which should not be confused with the frame pointer. Naming suggestions at this point are welcome, I'm happy to re-run sed. Reviewers: majnemer, nicholas Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D11011 llvm-svn: 241633
* Revert "Revert r241570, it caused PR24053"David Majnemer2015-07-071-2/+1
| | | | | | | | This reverts commit r241602. We had a latent bug in SCCP where we would make a basic block empty and then proceed to ask questions about it's terminator. llvm-svn: 241616
* [SCCP] Turn loads of null into undef instead of zero initialized valuesDavid Majnemer2015-07-011-1/+1
| | | | | | | | | | Surprisingly, this is a correctness issue: the mmx type exists for calling convention purposes, LLVM doesn't have a zero representation for them. This partially fixes PR23999. llvm-svn: 241142
* [NaryReassociate] enhances nsw by leveraging @llvm.assumeJingyue Wu2015-07-011-14/+55
| | | | | | | | | | | | | | | | | | | | | Summary: nsw are flaky and can often be removed by optimizations. This patch enhances nsw by leveraging @llvm.assume in the IR. Specifically, NaryReassociate now understands that assume(a + b >= 0) && assume(a >= 0) ==> a +nsw b As a result, it can split more sext(a + b) into sext(a) + sext(b) for CSE. Test Plan: nary-gep.ll Reviewers: broune, meheff Subscribers: jholewinski, llvm-commits Differential Revision: http://reviews.llvm.org/D10822 llvm-svn: 241139
* [SLSR] S's basis must have the same type as SJingyue Wu2015-06-281-2/+4
| | | | llvm-svn: 240910
* [RewriteStatepointsForGC] Generalized vector phi/select handling for base ↵Philip Reames2015-06-261-55/+124
| | | | | | | | | | | | | | | pointers This change extends the detection of base pointers for vector constructs to handle arbitrary phi and select nodes. The existing non-vector code already handles those, so this is basically just extending the vector special case to be less special cased. It still isn't generalized vector handling since we can't handle arbitrary vector instructions (e.g. shufflevectors), but it's a lot closer. The general structure of the change is as follows: * Extend the base defining value relation over a subset of vector instructions and vector typed phi & select instructions. * Move scalarization from before base pointer rewriting to after base pointer rewriting. The extension of the BDV relation is sufficient to find vector base phis for vector inputs. * Preserve the existing special case logic for when the base of a vector element is locally obvious. This general idea could be extended to the scalar case as well. Differential Revision: http://reviews.llvm.org/D10461#inline-84275 llvm-svn: 240850
* GVN: If a branch has two identical successors, we cannot declare either dead.Peter Collingbourne2015-06-251-0/+4
| | | | | | | | | This previously caused miscompilations as a result of phi nodes receiving undef incoming values from blocks dominated by such successors. Differential Revision: http://reviews.llvm.org/D10726 llvm-svn: 240670
* Add simplify_type<const WeakVH>; simplify IndVarSimplifyDuncan P. N. Exon Smith2015-06-241-4/+3
| | | | | | | | r240214 fixed some UB in IndVarSimplify, and it needed a temporary `WeakVH` to do it. Add `simplify_type<const WeakVH>` so that this temporary isn't necessary. llvm-svn: 240599
* [GVN] Intersect the IR flags when CSE'ing two instructionsDavid Majnemer2015-06-241-7/+3
| | | | | | | | | We performed a simple, but incomplete, intersection when it came time to CSE instructions. It didn't handle, for example, the 'exact' flag. This fixes PR23922. llvm-svn: 240595
* [Reassociate] Don't propogate flags when creating negationsDavid Majnemer2015-06-241-0/+10
| | | | | | | | | Reassociate mutated existing instructions in order to form negations which would create additional reassociate opportunities. This fixes PR23926. llvm-svn: 240593
* fix typos; NFCSanjay Patel2015-06-241-3/+2
| | | | llvm-svn: 240592
* don't repeat function names in comments; NFCSanjay Patel2015-06-241-3/+3
| | | | llvm-svn: 240591
* This change fixes three bugs in loop unswitching. This change causes an 81% ↵Mark Heffernan2015-06-231-40/+65
| | | | | | | | | | | | | | | speed-up on a benchmark that is based on EigenConvolutionKernel2D from Eigen3, where the lack of loop unswitching blocks hoisting of loads out of a nested loop (see bug 23816 for how loop unswitching and load hoisting are related). Change 1: Unswitching on trivial conditions should always happen regardless of the computed unswitching cost, as really the cost is zero. While there is code to make that happen, the logic that checks the unswitching cost against a threshold was moved to an earlier point (revision 147935) than the point where trivial unswitching is detected, so trivial unswitching is currently blocked by the cost threshold. This change fixes that. Change 2: Before revision 147935 (from 2012-01-11), the threshold parameter was a per-loop threshold. So an unswitching happened only if the cost of the unswitching was less than the threshold. In an indirect way (and I believe unintentionally), the logic for this since then has been that the threshold is an over-all budget across all loops for all loop unswitching done by a given LoopUnswitch loop pass object. So if an unswitching with cost 100 happens in one function, that in effect reduces the threshold from 100 to 0 for the loops even in another function. This persists for the lifetime of that loop pass object. This makes no difference for most small examples but it is important for large examples. This revision fixes that. Change 3: The cost is currently calculated as std::min(NumInstructions, 5 * NumBlocks). So a loop with 2 blocks and a million instructions will have an unswitching cost of 10. I changed this to just NumInstructions, as it were before revision 147935, though I'm open to e.g. instead replacing std::min with std::max. I've tried to make the change minimally invasive while staying with what I think was the original intent of the code. Submitted on behalf of broune@. llvm-svn: 240438
* Revert r240137 (Fixed/added namespace ending comments using clang-tidy. NFC)Alexander Kornienko2015-06-2339-72/+72
| | | | | | Apparently, the style needs to be agreed upon first. llvm-svn: 240390
* Fix PR13851: Preserve metadata for the unswitched branchWeiming Zhao2015-06-231-20/+67
| | | | | | | This patch copies the metadata of the unswitched branch to the newly crreated branch in loop unswitch pass. llvm-svn: 240378
* [LoopDist] Improve variable names and comments in LoopVersioning class, NFCAdam Nemet2015-06-221-25/+29
| | | | | | | | As with the previous patch, the goal is to turn the class into a general loop-versioning class. This patch removes any references to loop distribution. llvm-svn: 240352
* IndVarSimplify: Avoid UB from binding a reference to a null pointerJustin Bogner2015-06-201-3/+4
| | | | | | | | | | Calling operator* on a WeakVH whose Value is null hits undefined behaviour, since we bind the value to a reference. Instead, go through `operator Value*` so that we work with the pointer itself. Found by ubsan. llvm-svn: 240214
* [LoopDist] Rename RuntimeCheckEmitter to LoopVersioning, NFCAdam Nemet2015-06-191-8/+8
| | | | llvm-svn: 240165
* [LoopDist] Move pointer-to-partition computation out of RuntimeCheckEmitter, NFCAdam Nemet2015-06-191-18/+12
| | | | | | | This starts preparing the class to become a (more) general LoopVersioning utility class. llvm-svn: 240164
* Fixed/added namespace ending comments using clang-tidy. NFCAlexander Kornienko2015-06-1939-72/+72
| | | | | | | | | | | | | The patch is generated using this command: tools/clang/tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \ -checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \ llvm/lib/ Thanks to Eugene Kosov for the original patch! llvm-svn: 240137
* Fix "the the" in comments.Eric Christopher2015-06-192-2/+2
| | | | llvm-svn: 240112
* [NFC] more comments in SLSRJingyue Wu2015-06-181-0/+9
| | | | llvm-svn: 239984
* [PM/AA] Remove the UnknownSize static member from AliasAnalysis.Chandler Carruth2015-06-172-7/+7
| | | | | | | | This is now living in MemoryLocation, which is what it pertains to. It is also an enum there rather than a static data member which is left never defined. llvm-svn: 239886
* [PM/AA] Remove the Location typedef from the AliasAnalysis class nowChandler Carruth2015-06-175-43/+40
| | | | | | | | | | | | that it is its own entity in the form of MemoryLocation, and update all the callers. This is an entirely mechanical change. References to "Location" within AA subclases become "MemoryLocation", and elsewhere "AliasAnalysis::Location" becomes "MemoryLocation". Hope that helps out-of-tree folks update. llvm-svn: 239885
* Rename Reduction variables/structures to Recurrence.Tyler Nowicki2015-06-161-4/+4
| | | | | | | | A reduction is a special kind of recurrence. In the loop vectorizer we currently identify basic reductions. Future patches will extend this to identifying basic recurrences. llvm-svn: 239835
* Move logic from JumpThreading into LazyValue info to simplify caller. Philip Reames2015-06-161-51/+17
| | | | | | | | This change is hopefully NFC. The only tricky part is that I changed the context instruction being used to the branch rather than the comparison. I believe both to be correct, but the branch is strictly more powerful. With the moved code, using the branch instruction is required for the basic block comparison test to return the same result. The previous code was able to directly access both the branch and the comparison where the revised code is not. Differential Revision: http://reviews.llvm.org/D9652 llvm-svn: 239797
* [Statepoints] Skip a vector copy when uniquing values.Benjamin Kramer2015-06-131-11/+4
| | | | | | No functionality change intended. llvm-svn: 239688
* Revert 239644.Matt Wala2015-06-131-3/+1
| | | | llvm-svn: 239650
* [Scalarizer] Fix potential for stale data in Scattered across invocationsMatt Wala2015-06-121-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Scalarizer has two data structures that hold information about changes to the function, Gathered and Scattered. These are cleared in finish() at the end of runOnFunction() if finish() detects any changes to the function. However, finish() was checking for changes by only checking if Gathered was non-empty. The function visitStore() only modifies Scattered without touching Gathered. As a result, Scattered could have ended up having stale data if Scalarizer only scalarized store instructions. Since the data in Scattered is used during the execution of the pass, this introduced dangling pointer errors. The fix is to check whether both Scattered and Gathered are empty before deciding what to do in finish(). Reviewers: srhines Reviewed By: srhines Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D10422 llvm-svn: 239644
* Fix a typo in a comment in MemCpyOpt (test commit)Matt Wala2015-06-121-1/+1
| | | | llvm-svn: 239628
* [GVN] Use a simpler form of IRBuilder constructor.Alexey Samsonov2015-06-121-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: A side effect of this change is that it IRBuilder now automatically created debug info locations for new instructions, which is the same as debug location of insertion point. This is fine for the functions in questions (GetStoreValueForLoad and GetMemInstValueForLoad), as they are used in two situations: * GVN::processLoad, which tries to eliminate a load. In this case new instructions would have the same debug location as the load they eventually replace; * MaterializeAdjustedValue, which adds new instructions to the end of the basic blocks, which could later be used to replace the load definition. In this case we don't yet know the way the load would be eventually replaced (either by assembling the precomputed values via PHI, or by using them directly), so just using the basic block strategy seems to be reasonable. There is also a special case in the code that *would* adjust the location of the last instruction replacing the load definition to the location of the load. Test Plan: regression test suite Reviewers: echristo, dberlin, dblaikie Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D10405 llvm-svn: 239585
* [GVN] Use IRBuilder more actively instead of creating instructions manually.Alexey Samsonov2015-06-121-20/+21
| | | | llvm-svn: 239584
* Update stale comment before analyzeLoopUnrollCost. NFC.Michael Zolotukhin2015-06-111-7/+9
| | | | llvm-svn: 239565
OpenPOWER on IntegriCloud