summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/Scalar
Commit message (Collapse)AuthorAgeFilesLines
...
* OpaquePtr: use load instruction directly for type. NFC.Tim Northover2019-07-111-2/+1
| | | | llvm-svn: 365768
* NFC: Pass DataLayout into isBytewiseValueVitaly Buka2019-07-102-7/+8
| | | | | | | | | | | | | | | | | | Summary: We will need to handle IntToPtr which I will submit in a separate patch as it's not going to be NFC. Reviewers: eugenis, pcc Reviewed By: eugenis Subscribers: hiraditya, cfe-commits, llvm-commits Tags: #clang, #llvm Differential Revision: https://reviews.llvm.org/D63940 llvm-svn: 365709
* [SimpleLoopUnswitch] Don't consider unswitching `switch` insructions with ↵Serguei Katkov2019-07-101-1/+1
| | | | | | | | | | | | | | | one unique successor Only instructions with two or more unique successors should be considered for unswitching. Patch Author: Daniil Suchkov. Reviewers: reames, asbirlea, skatkov Reviewed By: skatkov Subscribers: hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D64404 llvm-svn: 365611
* OpaquePtr: add Type parameter to Loads analysis API.Tim Northover2019-07-094-9/+15
| | | | | | | | | | | | | This makes the functions in Loads.h require a type to be specified independently of the pointer Value so that when pointers have no structure other than address-space, it can still do its job. Most callers had an obvious memory operation handy to provide this type, but a SROA and ArgumentPromotion were doing more complicated analysis. They get updated to merge the properties of the various instructions they were considering. llvm-svn: 365468
* [LoopInfo] Update getExitEdges to accept vector of pairs for non const ↵Serguei Katkov2019-07-091-1/+1
| | | | | | | | | | | | | | | | | | | BasicBlock D63921 requires getExitEdges fills a vector of Edge pairs where BasicBlocks are not constant. The rest Loop API mostly returns non-const BasicBlocks, so to be more consistent with other Loop API getExitEdges is modified to return non-const BasicBlocks as well. This is an alternative solution to D64060. Reviewers: reames, fhahn Reviewed By: reames, fhahn Subscribers: hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D64309 llvm-svn: 365437
* [LoopPred] Stylistic improvement to recently added NE/EQ normalization [NFC]Philip Reames2019-07-091-9/+5
| | | | llvm-svn: 365425
* [LoopPred] Extend LFTR normalization to the inverse EQ casePhilip Reames2019-07-091-0/+5
| | | | | | A while back, I added support for NE latches formed by LFTR. I didn't think that quite through, as LFTR will also produce the inverse EQ form for some loops and I hadn't handled that. This change just adds handling for that case as well. llvm-svn: 365419
* [Float2Int] Add support for unary FNeg to Float2IntCameron McInally2019-07-081-0/+14
| | | | | | Differential Revision: https://reviews.llvm.org/D63941 llvm-svn: 365324
* [IRBuilder] Introduce helpers for and/or of multiple values at oncePhilip Reames2019-07-062-22/+7
| | | | | | | | We had versions of this code scattered around, so consolidate into one location. Not strictly NFC since the order of intermediate results may change in some places, but since these operations are associatives, should not change results. llvm-svn: 365259
* [PowerPC] Hardware Loop branch instruction's condition may not be icmp.Chen Zheng2019-07-041-1/+1
| | | | | | | This fixes pr42492. Differential Revision: https://reviews.llvm.org/D64124 llvm-svn: 365104
* [JumpThreading] Fix threading with unusual PHI nodes.Eli Friedman2019-07-031-3/+16
| | | | | | | | | | | | | | | | | | | | If the block being cloned contains a PHI node, in general, we need to clone that PHI node, even though it's trivial. If the operand of the PHI is an instruction in the block being cloned, the correct value for the operand doesn't exist until SSAUpdater constructs it. We usually don't hit this issue because we try to avoid threading across loop headers, but it's possible to hit this in some cases involving irreducible CFGs. I added a flag to allow threading across loop headers to make the testcase easier to understand. Thanks to Brian Rzycki for reducing the testcase. Fixes https://bugs.llvm.org/show_bug.cgi?id=42085. Differential Revision: https://reviews.llvm.org/D63913 llvm-svn: 365094
* [LFTR] Use SCEVExpander for the pointer limit case instead of manual IR genPhilip Reames2019-07-031-10/+5
| | | | | | As noted in the test change, this is not trivially NFC, but all of the changes in output are cases where the SCEVExpander form is more canonical/optimal than the hand generation. llvm-svn: 365075
* [LFTR] Remove a stray variable shadow *of the same value* [NFC]Philip Reames2019-07-031-1/+0
| | | | llvm-svn: 365072
* [LFTR] Style and comment changes to clarify the narrow vs wide bitwidth ↵Philip Reames2019-07-031-17/+18
| | | | | | evaluation behavior [NFC] llvm-svn: 365071
* [LFTR] Sink the decision not use truncate scheme for constants into ↵Philip Reames2019-07-031-46/+43
| | | | | | | | genLoopLimit [NFC] We might as well just evaluate the constants using SCEV, and having the cases grouped makes the logic slightly easier to read anyway. llvm-svn: 365070
* [LFTR] Remove falsely generalized (dead) code [NFC]Philip Reames2019-07-031-5/+2
| | | | llvm-svn: 365067
* [LFTR] Hoist extend expressions outside of loops w/o waiting for LICMPhilip Reames2019-07-031-1/+4
| | | | | | | | The motivation for this is two fold: 1) Make the output (and thus tests) a bit more readable to a human trying to understand the result of the transform 2) Reduce spurious diffs in a potential future change to restructure all of this logic to use SCEVExpander (which hoists by default) llvm-svn: 365066
* [PowerPC] exclude ICmpZero in LSR if icmp can be replaced in later hardware ↵Chen Zheng2019-07-031-7/+26
| | | | | | | | | loop. Differential Revision: https://reviews.llvm.org/D63477 llvm-svn: 364993
* [SimpleLoopUnswitch] Implement handling of prof branch_weights metadata for ↵Yevgeny Rouban2019-07-011-17/+39
| | | | | | | | SwitchInst Differential Revision: https://reviews.llvm.org/D60606 llvm-svn: 364734
* Cleanup: llvm::bsearch -> llvm::partition_point after r364719Fangrui Song2019-06-301-2/+2
| | | | llvm-svn: 364720
* [LFTR] Rephrase getLoopTest into "based-on" check; NFCINikita Popov2019-06-291-23/+23
| | | | | | | | | What we want to know here is whether we're already using this value for the loop condition, so make the query about that. We can extend this to a more general "based-on" relationship, rather than a direct icmp use later. llvm-svn: 364715
* [LFTR] Remove unnecessary latch check; NFCINikita Popov2019-06-291-14/+9
| | | | | | | | | | | The whole indvars pass works on loops in simplified form, so there is always a unique latch. Convert the condition into an assertion in needsLFTR (though we also assert this in later LFTR functions). Additionally update the comment on getLoopTest() now that we are dealing with multiple exits. llvm-svn: 364713
* [LFTR] Fix post-inc pointer IV with truncated exit count (PR41998)Nikita Popov2019-06-291-40/+37
| | | | | | | | | | | | | | | | | | | Fixes https://bugs.llvm.org/show_bug.cgi?id=41998. Usually when we have a truncated exit count we'll truncate the IV when comparing against the limit, in which case exit count overflow in post-inc form doesn't matter. However, for pointer IVs we don't do that, so we have to be careful about incrementing the IV in the wide type. I'm fixing this by removing the IVCount variable (which was ExitCount or ExitCount+1) and replacing it with a UsePostInc flag, and then moving the actual limit adjustment to the individual cases (which are: pointer IV where we add to the wide type, integer IV where we add to the narrow type, and constant integer IV where we add to the wide type). Differential Revision: https://reviews.llvm.org/D63686 llvm-svn: 364709
* [IndVars] Remove a bit of manual constant folding [NFC]Philip Reames2019-06-291-12/+7
| | | | | | SCEV is more than capable of folding (add x, trunc(0)) to x. llvm-svn: 364693
* [NewGVN] Add unary FNeg support to NewGVN passCameron McInally2019-06-281-0/+1
| | | | | | Differential Revision: https://reviews.llvm.org/D63933 llvm-svn: 364680
* [GVNSink] Add unary FNeg support to GVNSink passCameron McInally2019-06-281-0/+1
| | | | | | Differential Revision: https://reviews.llvm.org/D63900 llvm-svn: 364678
* [GVN] Add support for unary FNeg to GVN passCameron McInally2019-06-271-0/+1
| | | | | | Differential Revision: https://reviews.llvm.org/D63896 llvm-svn: 364592
* [SCCP] Fix non-deterministic uselists of return values (DenseMap -> MapVector)Gerolf Hoflehner2019-06-261-6/+7
| | | | llvm-svn: 364482
* [IndVars] Kill a redundant bit of debug outputPhilip Reames2019-06-261-2/+0
| | | | llvm-svn: 364449
* Revert "r364412 [ExpandMemCmp][MergeICmps] Move passes out of CodeGen into ↵Clement Courbet2019-06-264-898/+1
| | | | | | | | | | | | | | opt pipeline." Breaks sanitizers: libFuzzer :: cxxstring.test libFuzzer :: memcmp.test libFuzzer :: recommended-dictionary.test libFuzzer :: strcmp.test libFuzzer :: value-profile-mem.test libFuzzer :: value-profile-strcmp.test llvm-svn: 364416
* [ExpandMemCmp][MergeICmps] Move passes out of CodeGen into opt pipeline.Clement Courbet2019-06-264-1/+898
| | | | | | | | | This allows later passes (in particular InstCombine) to optimize more cases. One that's important to us is `memcmp(p, q, constant) < 0` and memcmp(p, q, constant) > 0. llvm-svn: 364412
* [LFTR] Adjust debug output to include extensions (if any)Philip Reames2019-06-251-7/+8
| | | | llvm-svn: 364346
* [ExpandMemCmp] Move all options to TargetTransformInfo.Clement Courbet2019-06-251-1/+1
| | | | | | Split off from D60318. llvm-svn: 364281
* [CVP] Reenable nowrap flag inferenceNikita Popov2019-06-241-1/+1
| | | | | | | | | | | Inference of nowrap flags in CVP has been disabled, because it triggered a bug in LFTR (https://bugs.llvm.org/show_bug.cgi?id=31181). This issue has been fixed in D60935, so we should be able to reenable nowrap flag inference now. Differential Revision: https://reviews.llvm.org/D62776 llvm-svn: 364228
* Fix typo in comment; NFCSanjoy Das2019-06-231-1/+1
| | | | llvm-svn: 364159
* [IndVars] Remove dead instructions after folding trivial loop exitPhilip Reames2019-06-231-3/+5
| | | | | | In rL364135, I taught IndVars to fold exiting branches in loops with a zero backedge taken count (i.e. loops that only run one iteration). This extends that to eliminate the dead comparison left around. llvm-svn: 364155
* Exploit a zero LoopExit count to eliminate loop exitsPhilip Reames2019-06-221-2/+14
| | | | | | | | | | This turned out to be surprisingly effective. I was originally doing this just for completeness sake, but it seems like there are a lot of cases where SCEV's exit count reasoning is stronger than it's isKnownPredicate reasoning. Once this is in, I'm thinking about trying to build on the same infrastructure to eliminate provably untaken checks. There may be something generally interesting here. Differential Revision: https://reviews.llvm.org/D63618 llvm-svn: 364135
* [NewGVN] Fix copy/paste mistake in castNikita Popov2019-06-221-1/+1
| | | | llvm-svn: 364130
* [NewGVN] Remove dead SwitchEdges variable; NFCNikita Popov2019-06-221-4/+0
| | | | llvm-svn: 364129
* [GVNSink] prevent crashing on mismatched instructions (PR42346)Sanjay Patel2019-06-211-0/+9
| | | | | | | Patch based on suggestion by James Molloy (@jmolloy) in: https://bugs.llvm.org/show_bug.cgi?id=42346 llvm-svn: 364062
* [Scalarizer] Propagate IR flagsJay Foad2019-06-211-4/+6
| | | | | | | | | | | | | | | | | | Summary: The motivation for this was to propagate fast-math flags like nnan and ninf on vector floating point operations to the corresponding scalar operations to take advantage of follow-on optimizations. But I think the same argument applies to all of our IR flags: if they apply to the vector operation then they also apply to all the individual scalar operations, and they might enable follow-on optimizations. Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D63593 llvm-svn: 364051
* Simplify std::lower_bound with llvm::{bsearch,lower_bound}. NFCFangrui Song2019-06-213-5/+4
| | | | llvm-svn: 364006
* [Reassociate] Remove bogus assert reported in PR42349.Cameron McInally2019-06-201-5/+1
| | | | | | | | Also, add a FIXME for the unsafe transform on a unary FNeg. A unary FNeg can only be transformed to a FMul by -1.0 when the nnan flag is present. The unary FNeg project is a WIP, so the unsafe transformation is acceptable until that work is complete. The bogus assert with introduced in D63445. llvm-svn: 363998
* [LICM & MSSA] Limit unsafe sinking and hoisting.Alina Sbirlea2019-06-201-10/+50
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The getClobberingMemoryAccess API checks for clobbering accesses in a loop by walking the backedge. This may check if a memory access is being clobbered by the loop in a previous iteration, depending how smart AA got over the course of the updates in MemorySSA (it does not occur when built from scratch). If no clobbering access is found inside the loop, it will optimize to an access outside the loop. This however does not mean that access is safe to sink. Given: ``` for i load a[i] store a[i] ``` The access corresponding to the load can be optimized to outside the loop, and the load can be hoisted. But it is incorrect to sink it. In order to sink the load, we'd need to check no Def clobbers the Use in the same iteration. With this patch we currently restrict sinking to either Defs not existing in the loop, or Defs preceding the load in the same block. An easy extension is to ensure the load (Use) post-dominates all Defs. Caught by PR42294. This issue also shed light on the converse problem: hoisting stores in this same scenario would be illegal. With this patch we restrict hoisting of stores to the case when their corresponding Defs are dominating all Uses in the loop. Reviewers: george.burgess.iv Subscribers: jlebar, Prazek, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D63582 llvm-svn: 363982
* [LFTR] Fix a (latent?) bug related to nested loopsPhilip Reames2019-06-201-0/+6
| | | | | | | | I can't actually come up with a test case this triggers on without an out of tree change, but in theory, it's a bug in the recently added multiple exit LFTR support. The root issue is that an exiting block common to two loops can (in theory) have computable exit counts for both loops. Rewriting the exit of an inner loop in terms of the outer loops IV would cause the inner loop to either a) run forever, or b) terminate on the first iteration. In practice, we appear to get lucky and not have the exit count computable for the outer loop, except when it's trivially zero. Given we bail on zero exit counts, we don't appear to ever trigger this. But I can't come up with a reason we *can't* compute an exit count for the outer loop on the common exiting block, so this may very well be triggering in some cases. llvm-svn: 363964
* LFTR for multiple exit loopsPhilip Reames2019-06-191-17/+16
| | | | | | | | | | | | | | Teach IndVarSimply's LinearFunctionTestReplace transform to handle multiple exit loops. LFTR does two key things 1) it rewrites (all) exit tests in terms of a common IV potentially eliminating one in the process and 2) it moves any offset/indexing/f(i) style logic out of the loop. This turns out to actually be pretty easy to implement. SCEV already has all the information we need to know what the backedge taken count is for each individual exit. (We use that when computing the BE taken count for the loop as a whole.) We basically just need to iterate through the exiting blocks and apply the existing logic with the exit specific BE taken count. (The previously landed NFC makes this super obvious.) I chose to go ahead and apply this to all loop exits instead of only latch exits as originally proposed. After reviewing other passes, the only case I could find where LFTR form was harmful was LoopPredication. I've fixed the latch case, and guards aren't LFTRed anyways. We'll have some more work to do on the way towards widenable_conditions, but that's easily deferred. I do want to note that I added one bit after the review. When running tests, I saw a new failure (no idea why didn't see previously) which pointed out LFTR can rewrite a constant condition back to a loop varying one. This was theoretically possible with a single exit, but the zero case covered it in practice. With multiple exits, we saw this happening in practice for the eliminate-comparison.ll test case because we'd compute a ExitCount for one of the exits which was guaranteed to never actually be reached. Since LFTR ran after simplifyAndExtend, we'd immediately turn around and undo the simplication work we'd just done. The solution seemed obvious, so I didn't bother with another round of review. Differential Revision: https://reviews.llvm.org/D62625 llvm-svn: 363883
* [LFTR] Stylistic cleanup as suggested in last review comment of D62939 [NFC]Philip Reames2019-06-191-9/+9
| | | | | | (Resumbit of r363292 which was reverted along w/an earlier patch) llvm-svn: 363877
* [LFTR] Rename variable to minimize confusion [NFC]Philip Reames2019-06-191-15/+15
| | | | | | | | (Recommit of r363293 which was reverted when a dependent patch was.) As pointed out by Nikita in D62625, BackedgeTakenCount is generally used to refer to the backedge taken count of the loop. A conditional backedge taken count - one which only applies if a particular exit is taken - is called a ExitCount in SCEV code, so be consistent here. llvm-svn: 363875
* [Reassociate] Handle unary FNeg in the Reassociate passCameron McInally2019-06-191-15/+28
| | | | | | Differential Revision: https://reviews.llvm.org/D63445 llvm-svn: 363813
* Recommit [SROA] Enhance SROA to handle `addrspacecast`ed allocasMichael Liao2019-06-181-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | [SROA] Enhance SROA to handle `addrspacecast`ed allocas - Fix typo in original change - Add additional handling to ensure all return pointers are properly casted. Summary: - After `addrspacecast` is allowed to be eliminated in SROA, the adjusting of storage pointer (from `alloca) needs to handle the potential different address spaces between the storage pointer (from alloca) and the pointer being used. Reviewers: arsenm Subscribers: wdng, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D63501 llvm-svn: 363743
OpenPOWER on IntegriCloud