summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
...
* [SLP] Fix for PR32164: Improve vectorization of reverse order of extract ↵Alexey Bataev2018-01-161-10/+17
| | | | | | | | | | | | | | operations. Summary: Sometimes vectorization of insertelement instructions with extractelement operands may produce an extra shuffle operation, if these operands are in the reverse order. Patch tries to improve this situation by the reordering of the operands to remove this extra shuffle operation. Reviewers: mkuper, hfinkel, RKSimon, spatel Subscribers: mzolotukhin, llvm-commits Differential Revision: https://reviews.llvm.org/D33954 llvm-svn: 322579
* [SROA] fix assetion failureHiroshi Inoue2018-01-161-9/+4
| | | | | | | | | | | | | | This patch fixes the assertion failure in SROA reported in PR35657. PR35657 reports the assertion failure due to r319522 (splitting for non-whole-alloca slices), but this problem can happen even without r319522. The problem exists in a check for reusing an existing alloca when rewriting partitions. As the original comment said, we can reuse the existing alloca if the new alloca has the same type and offset with the existing one. But the code checks only type of the alloca and then check the offset using an assert. In a corner case with out-of-bounds access (e.g. @PR35657 function added in unit test), it is possible that the two allocas have the same type but different offsets. This patch makes the check of the offset in the if condition, and re-enables the splitting for non-whole-alloca slices. Differential Revision: https://reviews.llvm.org/D41981 llvm-svn: 322533
* [LV] Don't call recordVectorLoopValueForInductionCast for newly-created IV ↵Andrei Elovikov2018-01-151-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | from a trunc. Summary: This method is supposed to be called for IVs that have casts in their use-def chains that are completely ignored after vectorization under PSE. However, for truncates of such IVs the same InductionDescriptor is used during creation/widening of both original IV based on PHINode and new IV based on TruncInst. This leads to unintended second call to recordVectorLoopValueForInductionCast with a VectorLoopVal set to the newly created IV for a trunc and causes an assert due to attempt to store new information for already existing entry in the map. This is wrong and should not be done. Fixes PR35773. Reviewers: dorit, Ayal, mssimpso Reviewed By: dorit Subscribers: RKSimon, dim, dcaballe, hsaito, llvm-commits, hiraditya Differential Revision: https://reviews.llvm.org/D41913 llvm-svn: 322473
* [NFC] Fix comment to adjust to realityMax Kazantsev2018-01-151-1/+1
| | | | llvm-svn: 322468
* [hwasan] An LLVM flag to disable stack tag randomization.Evgeniy Stepanov2018-01-131-15/+53
| | | | | | | | | | | | Summary: Necessary to achieve consistent test results. Reviewers: kcc, alekseyshl Subscribers: kubamracek, llvm-commits, hiraditya Differential Revision: https://reviews.llvm.org/D42023 llvm-svn: 322429
* [NFC] Change MemIntrinsicInst::setAlignment() to take an unsigned instead of ↵Daniel Neilson2018-01-124-14/+8
| | | | | | | | | | | a Constant Summary: In preparation for https://reviews.llvm.org/D41675 this NFC changes this prototype of MemIntrinsicInst::setAlignment() to accept an unsigned instead of a Constant. llvm-svn: 322403
* [JumpThreading] Preservation of DT and LVI across the passBrian M. Rzycki2018-01-124-89/+314
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: See D37528 for a previous (non-deferred) version of this patch and its description. Preserves dominance in a deferred manner using a new class DeferredDominance. This reduces the performance impact of updating the DominatorTree at every edge insertion and deletion. A user may call DDT->flush() within JumpThreading for an up-to-date DT. This patch currently has one flush() at the end of runImpl() to ensure DT is preserved across the pass. LVI is also preserved to help subsequent passes such as CorrelatedValuePropagation. LVI is simpler to maintain and is done immediately (not deferred). The code to perform the preversation was minimally altered and simply marked as preserved for the PassManager to be informed. This extends the analysis available to JumpThreading for future enhancements such as threading across loop headers. Reviewers: dberlin, kuhar, sebpop Reviewed By: kuhar, sebpop Subscribers: mgorny, dmgreen, kuba, rnk, rsmith, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D40146 llvm-svn: 322401
* [IRCE][NFC] Make range check's End a non-null SCEVMax Kazantsev2018-01-121-16/+18
| | | | | | | | | | | | | Currently, IRC contains `Begin` and `Step` as SCEVs and `End` as value. Aside from that, `End` can also be `nullptr` which can be later conditionally converted into a non-null SCEV. To make this logic more transparent, this patch makes `End` a SCEV and calculates it early, so that it is never a null. Differential Revision: https://reviews.llvm.org/D39590 llvm-svn: 322364
* [LoopDeletion] Handle users in unreachable blockSerguei Katkov2018-01-121-0/+26
| | | | | | | | | | | | | | This is a fix for PR35884. When we want to delete dead loop we must clean uses in unreachable blocks otherwise we'll get an assert during deletion of instructions from the loop. Reviewers: anna, davide Reviewed By: anna Subscribers: llvm-commits, lebedev.ri Differential Revision: https://reviews.llvm.org/D41943 llvm-svn: 322357
* [hwasan] Stack instrumentation.Evgeniy Stepanov2018-01-111-3/+168
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Very basic stack instrumentation using tagged pointers. Tag for N'th alloca in a function is built as XOR of: * base tag for the function, which is just some bits of SP (poor man's random) * small constant which is a function of N. Allocas are aligned to 16 bytes. On every ReturnInst allocas are re-tagged to catch use-after-return. This implementation has a bunch of issues that will be taken care of later: 1. lifetime intrinsics referring to tagged pointers are not recognized in SDAG. This effectively disables stack coloring. 2. Generated code is quite inefficient. There is one extra instruction at each memory access that adds the base tag to the untagged alloca address. It would be better to keep tagged SP in a callee-saved register and address allocas as an offset of that XOR retag, but that needs better coordination between hwasan instrumentation pass and prologue/epilogue insertion. 3. Lifetime instrinsics are ignored and use-after-scope is not implemented. This would be harder to do than in ASan, because we need to use a differently tagged pointer depending on which lifetime.start / lifetime.end the current instruction is dominated / post-dominated. Reviewers: kcc, alekseyshl Subscribers: srhines, kubamracek, javed.absar, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D41602 llvm-svn: 322324
* Make internal/private GVs implicitly dso_local.Rafael Espindola2018-01-112-0/+2
| | | | | | | | | | | | | | | | While updating clang tests for having clang set dso_local I noticed that: - There are *a lot* of tests to update. - Many of the updates are redundant. They are redundant because a GV is "obviously dso_local". This patch starts formalizing that a bit by requiring that internal and private GVs be dso_local too. Since they all are, we don't have to print dso_local to the textual representation, making it a bit more compact and easier to read. llvm-svn: 322317
* [Sink] Really really fix predicate in legality checkFiona Glaser2018-01-111-1/+1
| | | | | | | | | | | LoadInst isn't enough; we need to include intrinsics that perform loads too. All side-effecting intrinsics and such are already covered by the isSafe check, so we just need to care about things that read from memory. D41960, originally from D33179. llvm-svn: 322311
* [InstCombine] Apply the fix from r322284 for sin / cos -> tan tooBenjamin Kramer2018-01-111-2/+3
| | | | llvm-svn: 322285
* [InstCombine] For cos/sin -> tan copy attributes from cos instead of theBenjamin Kramer2018-01-111-2/+3
| | | | | | | | | | | parent function Ideally we should merge the attributes from the functions somehow, but this is obviously an improvement over taking random attributes from the caller which will trip up the verifier if they're nonsensical for an unary intrinsic call. llvm-svn: 322284
* [InstCombine] Missed optimization in math expression: sin(x) / cos(x) => tan(x)Dmitry Venikov2018-01-113-15/+48
| | | | | | | | | | | | | | Summary: This patch enables folding sin(x) / cos(x) -> tan(x), cos(x) / sin(x) -> 1 / tan(x) under -ffast-math flag Reviewers: hfinkel, spatel Reviewed By: spatel Subscribers: andrew.w.kaylor, efriedma, scanon, llvm-commits Differential Revision: https://reviews.llvm.org/D41286 llvm-svn: 322255
* [NFC] Commit to mention that r322248 is actually made by AndrewScheideckerMarcello Maggioni2018-01-111-1/+1
| | | | llvm-svn: 322249
* [SimplifyCFG] Add cut-off for InitializeUniqueCases.Marcello Maggioni2018-01-111-13/+25
| | | | | | | | | | | | | The function can take a significant amount of time on some complicated test cases, but for the currently only use of the function we can stop the initialization much earlier when we find out we are going to discard the result anyway in the caller of the function. Adding configurable cut-off points so that we avoid wasting time. NFCI. llvm-svn: 322248
* Add explanatory comment to LoadStoreVectorizer.Justin Lebar2018-01-101-0/+32
| | | | | | | | | | Reviewers: arsenm Subscribers: rengolin, sanjoy, wdng, hiraditya, asbirlea Differential Revision: https://reviews.llvm.org/D41890 llvm-svn: 322157
* LowerTypeTests: Add limited support for aliasesVlad Tsyrklevich2018-01-102-0/+83
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: LowerTypeTests moves some function definitions from individual object files to the merged module, leaving a stub to be called in the merged module's jump table. If an alias was pointing to such a function definition LowerTypeTests would fail because the alias would be left without a definition to point to. This change 1) emits information about aliases to the ThinLTO summary, 2) replaces aliases pointing to function definitions that are moved to the merged module with function declarations, and 3) re-emits those aliases in the merged module pointing to the correct function definitions. The patch does not correctly fix all possible mis-uses of aliases in LowerTypeTests. For example, it does not handle aliases with a different type from the pointed to function. The addition of alias data increases the size of Chrome build artifacts by less than 1%. Reviewers: pcc Reviewed By: pcc Subscribers: mehdi_amini, eraman, mgrang, llvm-commits, eugenis, kcc Differential Revision: https://reviews.llvm.org/D41741 llvm-svn: 322139
* [LoopRotate] Detect loops with indirect branches better (we're giving up on ↵Michael Zolotukhin2018-01-091-1/+1
| | | | | | them). llvm-svn: 322137
* [IPSCCP] Remove calls without side effectsChris Bieneman2018-01-091-1/+1
| | | | | | | | | | | | | | | Summary: When performing constant propagation for call instructions we have historically replaced all uses of the return from a call, but not removed the call itself. This is required for correctness if the calls have side effects, however the compiler should be able to safely remove calls that don't have side effects. This allows the compiler to completely fold away calls to functions that have no side effects if the inputs are constant and the output can be determined at compile time. Reviewers: davide, sanjoy, bruno, dberlin Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D38856 llvm-svn: 322125
* NewGVN: Fix PR/33367, which was causing us to delete non-copy intrinsics ↵Daniel Berlin2018-01-091-2/+5
| | | | | | accidentally in some rare cases llvm-svn: 322115
* Add a pass to generate synthetic function entry counts.Easwaran Raman2018-01-093-1/+129
| | | | | | | | | | | | | | | | | | | | | | | | Summary: This pass synthesizes function entry counts by traversing the callgraph and using the relative block frequencies of the callsites. The intended use of these counts is in inlining to determine hot/cold callsites in the absence of profile information. The pass is split into two files with the code that propagates the counts in a callgraph in a Utils file. I plan to add support for propagation in the thinlto link phase and the propagation code will be shared and hence this split. I did not add support to the old PM since hot callsite determination in inlining is not possible in old PM (although we could use hot callee heuristic with synthetic counts in the old PM it is not worth the effort tuning it) Reviewers: davidxl, silvas Subscribers: mgorny, mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D41604 llvm-svn: 322110
* [InstCombine] weaken assertions for icmp folds (PR35846)Sanjay Patel2018-01-091-10/+4
| | | | | | | | | Because of potential UB (known bits conflicts with an llvm.assume), we have to check rather than assert here because InstSimplify doesn't kill the compare: https://bugs.llvm.org/show_bug.cgi?id=35846 llvm-svn: 322104
* [EarlyCSE] Salvage debug info during DCEPetar Jovanovic2018-01-091-0/+1
| | | | | | | | | | | EarlyCSE did not try to salvage debug info during erasing of instructions. This change fixes it. Patch by Djordje Todorovic. Differential Revision: https://reviews.llvm.org/D41496 llvm-svn: 322083
* [InstCombine] Check for out of range ashr values using APInt before calling ↵Simon Pilgrim2018-01-091-3/+5
| | | | | | | | getZExtValue Reduced from oss-fuzz #5032 test case llvm-svn: 322078
* AlwaysInliner: Alow setting InsertLifetime in the new-style passJustin Bogner2018-01-081-1/+2
| | | | llvm-svn: 322033
* ArgPromotion: Allow setting MaxElements in the new-style passJustin Bogner2018-01-081-1/+1
| | | | llvm-svn: 322025
* [CVP] Replace incoming values from unreachable blocks with undef.Davide Italiano2018-01-081-4/+24
| | | | | | | | | | | | | | | | | | | | | | This is an attempt of fixing PR35807. Due to the non-standard definition of dominance in LLVM, where uses in unreachable blocks are dominated by anything, you can have, in an unreachable block: %patatino = OP1 %patatino, CONSTANT When `SimplifyInstruction` receives a PHI where an incoming value is of the aforementioned form, in some cases, loops indefinitely. What I propose here instead is keeping track of the incoming values from unreachable blocks, and replacing them with undef. It fixes this case, and it seems to be good regardless (even if we can't prove that the value is constant, as it's coming from an unreachable block, we can ignore it). Differential Revision: https://reviews.llvm.org/D41812 llvm-svn: 322006
* [InstCombine] fold min/max tree with common operand (PR35717)Sanjay Patel2018-01-081-0/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is precedence for factorization transforms in instcombine for FP ops with fast-math. We also have similar logic in foldSPFofSPF(). It would take more work to add this to reassociate because that's specialized for binops, and min/max are not binops (or even single instructions). Also, I don't have evidence that larger min/max trees than this exist in real code, but if we find that's true, we might want to reorganize where/how we do this optimization. In the motivating example from https://bugs.llvm.org/show_bug.cgi?id=35717 , we have: int test(int xc, int xm, int xy) { int xk; if (xc < xm) xk = xc < xy ? xc : xy; else xk = xm < xy ? xm : xy; return xk; } This patch solves that problem because we recognize more min/max patterns after rL321672 https://rise4fun.com/Alive/Qjne https://rise4fun.com/Alive/3yg Differential Revision: https://reviews.llvm.org/D41603 llvm-svn: 321998
* [SLP] Fix PR35777: Incorrect handling of aggregate values.Alexey Bataev2018-01-081-50/+10
| | | | | | | | | | | | | | | | Summary: Fixes the bug with incorrect handling of InsertValue|InsertElement instrucions in SLP vectorizer. Currently, we may use incorrect ExtractElement instructions as the operands of the original InsertValue|InsertElement instructions. Reviewers: mkuper, hfinkel, RKSimon, spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41767 llvm-svn: 321994
* [SLP] Fix PR35628: Count external uses on extra reduction arguments.Alexey Bataev2018-01-081-1/+0
| | | | | | | | | | | | | | Summary: If the vectorized value is marked as extra reduction argument, its users are not considered as external users. Patch fixes this. Reviewers: mkuper, hfinkel, RKSimon, spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41786 llvm-svn: 321993
* Revert "[SCCP] Manually fold branches on undef."Davide Italiano2018-01-071-26/+3
| | | | | | | I thought this was responsible for PR35723, but I was wrong, the issue lies elsewhere. Revert while I debug. llvm-svn: 321975
* [SLPVectorizer] Reintroduce std::stable_sort(properlyDominates()).Davide Italiano2018-01-071-9/+23
| | | | | | | | The approach was never discussed, I wasn't able to reproduce this non-determinism, and the original author went AWOL. After a discussion on the ML, Philip suggested to revert this. llvm-svn: 321974
* [LV][VPlan] NFC patch to move LoopVectorizationPlanner class out of ↵Hal Finkel2018-01-074-271/+269
| | | | | | | | | | | | | | | | | | | | | | | | | LoopVectorize.cpp Another small step forward to move VPlan stuff outside of LoopVectorize.cpp. VPlanBuilder.h is renamed to LoopVectorizationPlanner.h LoopVectorizationPlanner class is moved from LoopVectorize.cpp to LoopVectorizationPlanner.h LoopVectorizationCostModel::VectorizationFactor class is moved to LoopVectorizationPlanner.h (used by the planner class) --- this needs further streamlining work in later patches and thus all I did was take it out of the CostModel class and moved to the header file. The callback function had to stay inside LoopVectorize.cpp since it calls an InnerLoopVectorizer member function declared in it. Next Steps: Make InnerLoopVectorizer, LoopVectorizationCostModel, and other classes more modular and more aligned with VPlan direction, in small increments. Previous step was: r320900 (https://reviews.llvm.org/D41045) Patch by Hideki Saito, thanks! Differential Revision: https://reviews.llvm.org/D41420 llvm-svn: 321962
* [CodeExtractor] Use subset of function attributes for extracted function.Florian Hahn2018-01-071-4/+74
| | | | | | | | | | | | | | | | | | | | | | | | In addition to target-dependent attributes, we can also preserve a white-listed subset of target independent function attributes. The white-list excludes problematic attributes, most prominently: * attributes related to memory accesses, as alloca instructions could be moved in/out of the extracted block * control-flow dependent attributes, like no_return or thunk, as the relerelevant instructions might or might not get extracted. Thanks @efriedma and @aemerson for providing a set of attributes that cannot be propagated. Reviewers: efriedma, davidxl, davide, silvas Reviewed By: efriedma Differential Revision: https://reviews.llvm.org/D41334 llvm-svn: 321961
* [InlineFunction] Preserve calling convention when forwarding VarArgs.Florian Hahn2018-01-061-0/+1
| | | | | | | | | | Reviewers: efriedma, rnk, davide Reviewed By: rnk, davide Differential Revision: https://reviews.llvm.org/D41556 llvm-svn: 321943
* [InlineFunction] Preserve attributes when forwarding VarArgs.Florian Hahn2018-01-061-4/+23
| | | | | | | | | | Reviewers: rnk, efriedma Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D41555 llvm-svn: 321942
* [InlineFunction] Inline vararg functions that do not access varargs.Florian Hahn2018-01-061-17/+20
| | | | | | | | | | | | | If the varargs are not accessed by a function, we can inline the function. Reviewers: dblaikie, chandlerc, davide, efriedma, rnk, hfinkel Reviewed By: efriedma Differential Revision: https://reviews.llvm.org/D41335 llvm-svn: 321940
* [InstCombine] relax use constraint for min/max (~a, ~b) --> ~min/max(a, b)Sanjay Patel2018-01-061-2/+2
| | | | | | | | | | | In the minimal case, this won't remove instructions, but it still improves uses of existing values. In the motivating example from PR35834, it does remove instructions, and sets that case up to be optimized by something like D41603: https://reviews.llvm.org/D41603 llvm-svn: 321936
* [Utils] Simplify salvageDebugInfo, NFCIVedant Kumar2018-01-051-34/+30
| | | | | | | | | Having a single call to findDbgUsers() allows salvageDebugInfo() to return earlier. Differential Revision: https://reviews.llvm.org/D41787 llvm-svn: 321915
* [InstCombine] add folds for min(~a, b) --> ~max(a, b)Sanjay Patel2018-01-051-22/+12
| | | | | | | | | | | | | | | | | Besides the bug of omitting the inverse transform of max(~a, ~b) --> ~min(a, b), the use checking and operand creation were off. We were potentially creating repeated identical instructions of existing values. This led to infinite looping after I added the extra folds. By using the simpler m_Not matcher and not creating new 'not' ops for a and b, we avoid that problem. It's possible that not using IsFreeToInvert() here is more limiting than the simpler matcher, but there are no tests for anything more exotic. It's also possible that we should relax the use checking further to handle a case like PR35834: https://bugs.llvm.org/show_bug.cgi?id=35834 ...but we can make that a follow-up if it is needed. llvm-svn: 321882
* WholeProgramDevirt: Simplify ORE getter mechanism for old PM. NFCI.Peter Collingbourne2018-01-051-34/+17
| | | | llvm-svn: 321841
* Revert "[JumpThreading] Preservation of DT and LVI across the pass"Reid Kleckner2018-01-044-310/+89
| | | | | | | This reverts r321825, it causes crashes in Chromium. Reproducer forthcoming. llvm-svn: 321832
* [JumpThreading] Preservation of DT and LVI across the passBrian M. Rzycki2018-01-044-89/+310
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: See D37528 for a previous (non-deferred) version of this patch and its description. Preserves dominance in a deferred manner using a new class DeferredDominance. This reduces the performance impact of updating the DominatorTree at every edge insertion and deletion. A user may call DDT->flush() within JumpThreading for an up-to-date DT. This patch currently has one flush() at the end of runImpl() to ensure DT is preserved across the pass. LVI is also preserved to help subsequent passes such as CorrelatedValuePropagation. LVI is simpler to maintain and is done immediately (not deferred). The code to perfom the preversation was minimally altered and was simply marked as preserved for the PassManager to be informed. This extends the analysis available to JumpThreading for future enhancements. One example is loop boundary threading. Reviewers: dberlin, kuhar, sebpop Reviewed By: kuhar, sebpop Subscribers: hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D40146 llvm-svn: 321825
* Add assertion on DT availability during LI update in UpdateAnalysisInformationAnna Thomas2018-01-041-0/+1
| | | | | | | | | | | | | This came up during discussions in llvm-commits for rL321653: Check for unreachable preds before updating LI in UpdateAnalysisInformation The assert provides hints to passes to require both DT and LI if we plan on updating LI through this function. Tests run: make check llvm-svn: 321805
* [InstCombine] safely create a constant of the right type (PR35794)Sanjay Patel2018-01-041-4/+4
| | | | llvm-svn: 321801
* [GVNHoist] Fix: PR35222 gvn-hoist incorrectly erases load in case of a loopAditya Kumar2018-01-041-1/+1
| | | | | | | | | | | Reviewers: dberlin sebpop eli.friedman Differential Revision: https://reviews.llvm.org/D41453 llvm-svn: 321789
* StructurizeCFG: Fix broken backedge detectionMatt Arsenault2018-01-031-82/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The work order was changed in r228186 from SCC order to RPO with an arbitrary sorting function. The sorting function attempted to move inner loop nodes earlier. This was was apparently relying on an assumption that every block in a given loop / the same loop depth would be seen before visiting another loop. In the broken testcase, a block outside of the loop was encountered before moving onto another block in the same loop. The testcase would then structurize such that one blocks unconditional successor could never be reached. Revert to plain RPO for the analysis phase. This fixes detecting edges as backedges that aren't really. The processing phase does use another visited set, and I'm unclear on whether the order there is as important. An arbitrary order doesn't work, and triggers some infinite loops. The reversed RPO list seems to work and is closer to the order that was used before, minus the arbitary custom sorting. A few of the changed tests now produce smaller code, and a few are slightly worse looking. llvm-svn: 321751
* [InstCombine] Check for out of range shift values using APInt before calling ↵Simon Pilgrim2018-01-031-4/+4
| | | | | | | | getZExtValue Reduced from oss-fuzz #4871 test case llvm-svn: 321748
OpenPOWER on IntegriCloud