summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Analysis
Commit message (Collapse)AuthorAgeFilesLines
...
* [Inliner] Restrict soft-float inlining penalty.Eli Friedman2017-12-221-11/+23
| | | | | | | | | | | | | | The penalty is currently getting applied in a bunch of places where it doesn't make sense, like bitcasts (which are free) and calls (which were getting the call penalty applied twice). Instead, just apply the penalty to binary operators and floating-point casts. While I'm here, also fix getFPOpCost() to do the right thing in more cases, so we don't have to dig into function attributes. Differential Revision: https://reviews.llvm.org/D41522 llvm-svn: 321332
* Add hasProfileData() to check if a function has profile data. NFC.Easwaran Raman2017-12-222-2/+2
| | | | | | | | | | | | | | | | | | | Summary: This replaces calls to getEntryCount().hasValue() with hasProfileData that does the same thing. This refactoring is useful to do before adding synthetic function entry counts but also a useful cleanup IMO even otherwise. I have used hasProfileData instead of hasRealProfileData as David had earlier suggested since I think profile implies "real" and I use the phrase "synthetic entry count" and not "synthetic profile count" but I am fine calling it hasRealProfileData if you prefer. Reviewers: davidxl, silvas Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41461 llvm-svn: 321331
* [ModRefInfo] Add must alias info to ModRefInfo.Alina Sbirlea2017-12-215-27/+166
| | | | | | | | | | | | | | | | | | | | | | Summary: Add an additional bit to ModRefInfo, ModRefInfo::Must, to be cleared for known must aliases. Shift existing Mod/Ref/ModRef values to include an additional most significant bit. Update wrappers that modify ModRefInfo values to reflect the change. Notes: * ModRefInfo::Must is almost entirely cleared in the AAResults methods, the remaining changes are trying to preserve it. * Only some small changes to make custom AA passes set ModRefInfo::Must (BasicAA). * GlobalsModRef already declares a bit, who's meaning overlaps with the most significant bit in ModRefInfo (MayReadAnyGlobal). No changes to shift the value of MayReadAnyGlobal (see AlignedMap). FunctionInfo.getModRef() ajusts most significant bit so correctness is preserved, but the Must info is lost. * There are cases where the ModRefInfo::Must is not set, e.g. 2 calls that only read will return ModRefInfo::NoModRef, though they may read from exactly the same location. Reviewers: dberlin, hfinkel, george.burgess.iv Subscribers: llvm-commits, sanjoy Differential Revision: https://reviews.llvm.org/D38862 llvm-svn: 321309
* [PGO] Function section hotness prefix should look at all blocksTeresa Johnson2017-12-201-19/+39
| | | | | | | | | | | | | | | | | | | | Summary: The function section prefix for PGO based layout (e.g. hot/unlikely) should look at the hotness of all blocks not just the entry BB. A function with a cold entry but a very hot loop should be placed in the hot section, for example, so that it is located close to other hot functions it may call. For SamplePGO it was already looking at the branch weights on calls, and I made that code conditional on whether this is SamplePGO since it was essentially a noop for instrumentation PGO anyway. Reviewers: davidxl Subscribers: eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D41395 llvm-svn: 321197
* Revert r320548:[SLP] Vectorize jumbled memory loadsMohammad Shahid2017-12-201-71/+0
| | | | llvm-svn: 321181
* [memcpyopt] Teach memcpyopt to optimize across basic blocksDan Gohman2017-12-201-9/+11
| | | | | | | | | | | | | | | | | This teaches memcpyopt to make a non-local memdep query when a local query indicates that the dependency is non-local. This notably allows it to eliminate many more llvm.memcpy calls in common Rust code, often by 20-30%. This is r319482 and r319483, along with fixes for PR35519: fix the optimization that merges stores into memsets to preserve cached memdep info, and fix memdep's non-local caching strategy to not assume that larger queries are always more conservative than smaller ones. Fixes PR28958 and PR35519. Differential Revision: https://reviews.llvm.org/D40802 llvm-svn: 321138
* Silence a bunch of implicit fallthrough warningsAdrian Prantl2017-12-191-0/+1
| | | | llvm-svn: 321114
* [CFGVPrinter] Fix -dot-cfg-onlyFrancis Visoiu Mistrih2017-12-191-4/+4
| | | | | | The refactoring in r281640 made -dot-cfg-only ignore the "-only" part. llvm-svn: 321079
* [InlineCost] Skip volatile loads when looking for repeated loadsHaicheng Wu2017-12-191-1/+2
| | | | | | This is a follow-up fix of r320814. A test case is also added. llvm-svn: 321075
* [Analysis] Generate more precise TBAA tags when one access encloses the otherIvan A. Kosarev2017-12-181-6/+17
| | | | | | | | | | | | | | There are cases when two tags with different base types denote accesses to the same direct or indirect member of a structure type. Currently, merging of such tags results in a tag that represents an access to an object that has the type of that member. This patch changes this so that if one of the accesses encloses the other, then the generic tag is the one of the enclosed access. Differential Revision: https://reviews.llvm.org/D39557 llvm-svn: 321019
* [Memcpy Loop Lowering] Remove the fixed int8 lowering.Sean Fertile2017-12-181-9/+0
| | | | | | | | Switch over to the lowering that uses target supplied operand types. Differential Revision: https://reviews.llvm.org/D41201 llvm-svn: 320989
* [LVI] Support for ashr in LVIMax Kazantsev2017-12-181-0/+1
| | | | | | | | | | Enhance LVI to analyze the ‘ashr’ binary operation. This leverages the infrastructure in ConstantRange for the ashr operation. Patch by Surya Kumari Jangala! Differential Revision: https://reviews.llvm.org/D40886 llvm-svn: 320983
* [TargetLibraryInfo] Discard library functions with incorrectly sized integersIgor Laevsky2017-12-181-3/+3
| | | | | | Differential Revision: https://reviews.llvm.org/D41184 llvm-svn: 320964
* [SimplifyLibCalls] Inline calls to cabs when it's safe to do soHal Finkel2017-12-161-0/+22
| | | | | | | | | | | | When unsafe algerbra is allowed calls to cabs(r) can be replaced by: sqrt(creal(r)*creal(r) + cimag(r)*cimag(r)) Patch by Paul Walker, thanks! Differential Revision: https://reviews.llvm.org/D40069 llvm-svn: 320901
* [TargetLibraryInfo] fix documentation comment; NFCSanjay Patel2017-12-151-3/+3
| | | | llvm-svn: 320842
* [InlineCost] Find repeated loads in the calleeHaicheng Wu2017-12-152-6/+52
| | | | | | | | | | | SROA analysis of InlineCost can figure out that some stores can be removed after inlining and then the repeated loads clobbered by these stores are also free. This patch finds these clobbered loads and adjust the inline cost accordingly. Differential Revision: https://reviews.llvm.org/D33946 llvm-svn: 320814
* [SCEV] Fix the movement of insertion point in expander. PR35406.Serguei Katkov2017-12-151-1/+19
| | | | | | | | | | | | We cannot move the insertion point to header if SCEV contains div/rem operations due to they may go over check for zero denominator. Reviewers: sanjoy, mkazantsev, sebpop Reviewed By: sebpop Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41229 llvm-svn: 320789
* Fix many -Wsign-compare and -Wtautological-constant-compare warnings.Zachary Turner2017-12-141-1/+1
| | | | | | | | | | | | Most of the -Wsign-compare warnings are due to the fact that enums are signed by default in the MS ABI, while the tautological comparison warnings trigger on x86 builds where sizeof(size_t) is 4 bytes, so N > numeric_limits<unsigned>::max() is always false. Differential Revision: https://reviews.llvm.org/D41256 llvm-svn: 320750
* [ScalarEvolution] Fix base condition in isNormalAddRecPHI.Bjorn Pettersson2017-12-141-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The function is meant to recurse until it comes upon the phi it's looking for. However, with the current condition, it will recurse until it finds anything _but_ the phi. The function will even fail for simple cases like: %i = phi i32 [ %inc, %loop ], ... ... %inc = add i32 %i, 1 because the base condition will not happen when the phi is recursed to, and the recursion will end with a 'false' result since the previous instruction is a phi. Reviewers: sanjoy, atrick Reviewed By: sanjoy Subscribers: Ka-Ka, bjope, llvm-commits Committing on behalf of: Bevin Hansson (bevinh) Differential Revision: https://reviews.llvm.org/D40946 llvm-svn: 320700
* [InlineCost] Tracking Values through PHI NodesHaicheng Wu2017-12-141-6/+138
| | | | | | | | | | | | This patch fix this FIXME in visitPHI() FIXME: We should potentially be tracking values through phi nodes, especially when they collapse to a single value due to deleted CFG edges during inlining. Differential Revision: https://reviews.llvm.org/D38594 llvm-svn: 320699
* [LV] Support efficient vectorization of an induction with redundant castsDorit Nuzman2017-12-141-6/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | D30041 extended SCEVPredicateRewriter to improve handling of Phi nodes whose update chain involves casts; PSCEV can now build an AddRecurrence for some forms of such phi nodes, under the proper runtime overflow test. This means that we can identify such phi nodes as an induction, and the loop-vectorizer can now vectorize such inductions, however inefficiently. The vectorizer doesn't know that it can ignore the casts, and so it vectorizes them. This patch records the casts in the InductionDescriptor, so that they could be marked to be ignored for cost calculation (we use VecValuesToIgnore for that) and ignored for vectorization/widening/scalarization (i.e. treated as TriviallyDead). In addition to marking all these casts to be ignored, we also need to make sure that each cast is mapped to the right vector value in the vector loop body (be it a widened, vectorized, or scalarized induction). So whenever an induction phi is mapped to a vector value (during vectorization/widening/ scalarization), we also map the respective cast instruction (if exists) to that vector value. (If the phi-update sequence of an induction involves more than one cast, then the above mapping to vector value is relevant only for the last cast of the sequence as we allow only the "last cast" to be used outside the induction update chain itself). This is the last step in addressing PR30654. llvm-svn: 320672
* Remove redundant includes from lib/Analysis.Michael Zolotukhin2017-12-1312-14/+0
| | | | llvm-svn: 320617
* Reintroduce r320049, r320014 and r319894.Igor Laevsky2017-12-131-0/+28
| | | | | | OpenGL issues should be fixed by now. llvm-svn: 320568
* [SLP] Vectorize jumbled memory loads.Mohammad Shahid2017-12-131-0/+71
| | | | | | | | | | | | | | | | | | | | | | Summary: This patch tries to vectorize loads of consecutive memory accesses, accessed in non-consecutive or jumbled way. An earlier attempt was made with patch D26905 which was reverted back due to some basic issue with representing the 'use mask' of jumbled accesses. This patch fixes the mask representation by recording the 'use mask' in the usertree entry. Change-Id: I9fe7f5045f065d84c126fa307ef6ebe0787296df Reviewers: mkuper, loladiro, Ayal, zvi, danielcdh Reviewed By: Ayal Subscribers: mgrang, dcaballe, hans, mzolotukhin Differential Revision: https://reviews.llvm.org/D36130 llvm-svn: 320548
* Revert r320049, r320014 and r319894Igor Laevsky2017-12-121-28/+0
| | | | | | | They were causing failures of the piglit OpenGL tests with AMD GPUs using the Mesa radeonsi driver. llvm-svn: 320466
* [SCEV] Fix wrong Equal predicate created in getAddRecForPhiWithCastsDorit Nuzman2017-12-101-6/+9
| | | | | | | | | | | | | | | | | | | CreateAddRecFromPHIWithCastsImpl() adds an IncrementNUSW overflow predicate which allows the PSCEV rewriter to rewrite this scev expression: (zext i8 {0, + , (trunc i32 step to i8)} to i32) into {0, +, (sext i8 (trunc i32 step to i8) to i32)} But then it adds the wrong Equal predicate: %step == (zext i8 (trunc i32 %step to i8) to i32). instead of: %step == (sext i8 (trunc i32 %step to i8) to i32) This is fixed here. Differential Revision: https://reviews.llvm.org/D40641 llvm-svn: 320298
* Infer lowest bits of an integer Multiply when the low bits of the operands ↵Simon Dardis2017-12-091-9/+66
| | | | | | | | | | | | | | | are known When the lowest bits of the operands to an integer multiply are known, the low bits of the result are deducible. Code to deduce known-zero bottom bits already existed, but this change improves on that by deducing known-ones. Patch by: Pedro Ferreira Reviewers: craig.topper, sanjoy, efriedma Differential Revision: https://reviews.llvm.org/D34029 llvm-svn: 320269
* Hardware-assisted AddressSanitizer (llvm part).Evgeniy Stepanov2017-12-092-3/+6
| | | | | | | | | | | | | | | | | | | | | Summary: This is LLVM instrumentation for the new HWASan tool. It is basically a stripped down copy of ASan at this point, w/o stack or global support. Instrumenation adds a global constructor + runtime callbacks for every load and store. HWASan comes with its own IR attribute. A brief design document can be found in clang/docs/HardwareAssistedAddressSanitizerDesign.rst (submitted earlier). Reviewers: kcc, pcc, alekseyshl Subscribers: srhines, mehdi_amini, mgorny, javed.absar, eraman, llvm-commits, hiraditya Differential Revision: https://reviews.llvm.org/D40932 llvm-svn: 320217
* Revert r320104: infinite loop profiling bug fixXinliang David Li2017-12-081-4/+0
| | | | | | | | | | | Causes unexpected memory issue with New PM this time. The new PM invalidates BPI but not BFI, leaving the reference to BPI from BFI invalid. Abandon this patch. There is a more general solution which also handles runtime infinite loop (but not statically). llvm-svn: 320180
* [NFC] Rename variable from Cond to Pred to make it more soundMax Kazantsev2017-12-081-12/+12
| | | | llvm-svn: 320144
* [SCEV] Fix predicate usage in computeExitLimitFromICmpMax Kazantsev2017-12-081-1/+2
| | | | | | | | | | | | | | | | | | | In this method, we invoke `SimplifyICmpOperands` which takes the `Cond` predicate by reference and may change it along with `LHS` and `RHS` SCEVs. But then we invoke `computeShiftCompareExitLimit` with Values from which the SCEVs have been derived, these Values have not been modified while `Cond` could be. One of possible outcomes of this is that we may falsely prove that an infinite loop ends within some finite number of iterations. In this patch, we save the original `Cond` and pass it along with original operands. This logic may be removed in future once `computeShiftCompareExitLimit` works with SCEVs instead of value operands. Reviewed By: sanjoy Differential Revision: https://reviews.llvm.org/D40953 llvm-svn: 320142
* [ModRefInfo] Make enum ModRefInfo an enum class [NFC].Alina Sbirlea2017-12-078-115/+119
| | | | | | | | | | | | | | | Summary: Make enum ModRefInfo an enum class. Changes to ModRefInfo values should be done using inline wrappers. This should prevent future bit-wise opearations from being added, which can be more error-prone. Reviewers: sanjoy, dberlin, hfinkel, george.burgess.iv Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40933 llvm-svn: 320107
* [PGO] detect infinite loop and form MST properlyXinliang David Li2017-12-071-0/+4
| | | | | | Differential Revision: http://reviews.llvm.org/D40873 llvm-svn: 320104
* [ModRefInfo] Replace remaining bit-wise operations with wrappers.Alina Sbirlea2017-12-073-5/+5
| | | | llvm-svn: 319993
* [ModRefInfo] Do not use ModRefInfo result in if conditions as this makesAlina Sbirlea2017-12-061-1/+1
| | | | | | | assumptions about the values in the enum. Replace with wrapper returning bool [NFC]. llvm-svn: 319949
* [ModRefInfo] Use createModRefInfo wrapper to create a ModRefInfo from ↵Alina Sbirlea2017-12-061-4/+4
| | | | | | FunctionModRefBehavior. llvm-svn: 319941
* InstructionSimplify: 'extractelement' with an undef index is undefZvi Rackover2017-12-061-0/+5
| | | | | | | | | | | | | | | | | | | Summary: An undef extract index can be arbitrarily chosen to be an out-of-range index value, which would result in the instruction being undef. This change closes a gap identified while working on lowering vector permute intrinsics with variable index vectors to pure LLVM IR. Reviewers: arsenm, spatel, majnemer Reviewed By: arsenm, spatel Subscribers: fhahn, nhaehnle, wdng, llvm-commits Differential Revision: https://reviews.llvm.org/D40231 llvm-svn: 319910
* [InstSimplify] Fold insertelement into undef if index is out of boundsIgor Laevsky2017-12-061-0/+28
| | | | | | Differential Revision: https://reviews.llvm.org/D40650 llvm-svn: 319894
* [SCEV][NFC] Check NoWrap flags before lexicographical comparison of SCEVsMax Kazantsev2017-12-061-0/+8
| | | | | | | | | | | | | | | | Lexicographical comparison of SCEV trees is potentially expensive for big expression trees. We can define ordering between them for AddRecs and N-ary operations by SCEV NoWrap flags to make non-equality check cheaper. This change does not prevent grouping eqivalent SCEVs together and is not supposed to have any meaningful impact on behavior of any transforms. Reviewed By: sanjoy Differential Revision: https://reviews.llvm.org/D40645 llvm-svn: 319889
* [SCEV][NFC] Share value cache between SCEVs in GroupByComplexityMax Kazantsev2017-12-061-22/+26
| | | | | | | | | | | | | Current implementation of `compareSCEVComplexity` is being unreasonable with `SCEVUnknown`s: every time it sees one, it creates a new value cache and tries to prove equality of two values using it. This cache reallocates and gets lost from SCEV to SCEV. This patch changes this behavior: now we create one cache for all values and share it between SCEVs. Reviewed By: sanjoy Differential Revision: https://reviews.llvm.org/D40597 llvm-svn: 319880
* Revert r319482 and r319483 "[memcpyopt] Teach memcpyopt to optimize across ↵Hans Wennborg2017-12-061-8/+0
| | | | | | | | | | | | | | | | | | | | | | | | basic blocks" This caused PR35519. > [memcpyopt] Teach memcpyopt to optimize across basic blocks > > This teaches memcpyopt to make a non-local memdep query when a local query > indicates that the dependency is non-local. This notably allows it to > eliminate many more llvm.memcpy calls in common Rust code, often by 20-30%. > > Fixes PR28958. > > Differential Revision: https://reviews.llvm.org/D38374 > > [memcpyopt] Commit file missed in r319482. > > This change was meant to be included with r319482 but was accidentally > omitted. llvm-svn: 319873
* [ModRefInfo] Initialize ArgMask to MRI_NoModRef.Alina Sbirlea2017-12-051-1/+1
| | | | llvm-svn: 319831
* Modify ModRefInfo values using static inline method abstractions [NFC].Alina Sbirlea2017-12-057-80/+85
| | | | | | | | | | | | | | | | | Summary: The aim is to make ModRefInfo checks and changes more intuitive and less error prone using inline methods that abstract the bit operations. Ideally ModRefInfo would become an enum class, but that change will require a wider set of changes into FunctionModRefBehavior. Reviewers: sanjoy, george.burgess.iv, dberlin, hfinkel Subscribers: nlopes, llvm-commits Differential Revision: https://reviews.llvm.org/D40749 llvm-svn: 319821
* [InstCombine] Don't crash on out of bounds shiftsIgor Laevsky2017-12-051-13/+17
| | | | | | Differential Revision: https://reviews.llvm.org/D40649 llvm-svn: 319761
* [SCEV] Use a "Discovered" set instead of a "Visited" set; NFCSanjoy Das2017-12-041-4/+3
| | | | | | Suggested by Max Kazantsev in https://reviews.llvm.org/D39361 llvm-svn: 319679
* [SCEV] A different fix for PR33494Sanjoy Das2017-12-041-29/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: I don't think rL309080 is the right fix for PR33494 -- caching ExitLimit only hides the problem[0]. The real issue is that because of how we forget SCEV expressions ScalarEvolution::getBackedgeTakenInfo, in the test case for PR33494 computing the backedge for any loop invalidates the trip count for every other loop. This effectively makes the SCEV cache useless. I've instead made the SCEV expression invalidation in ScalarEvolution::getBackedgeTakenInfo less aggressive to fix this issue. [0]: One way to think about this is that rL309080 essentially augmented the backedge-taken-count cache with another equivalent exit-limit cache. The bug went away because we were explicitly not clearing the exit-limit cache in getBackedgeTakenInfo. But instead of doing all of that, we can just avoid clearing the backedge-taken-count cache. Reviewers: mkazantsev, mzolotukhin Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D39361 llvm-svn: 319678
* Revert "[ValueTracking] Pass only a single lambda to ↵Sam McCall2017-12-041-29/+37
| | | | | | | | | computeKnownBitsFromShiftOperator by using KnownBits struct instead of separate APInts. NFCI" This reverts commit r319624, which seems to cause a miscompile (breaks the multistage PPC buildbots) llvm-svn: 319652
* [ValueTracking] Pass only a single lambda to ↵Craig Topper2017-12-021-37/+29
| | | | | | computeKnownBitsFromShiftOperator by using KnownBits struct instead of separate APInts. NFCI llvm-svn: 319624
* [opt-remarks] If hotness threshold is set, ignore remarks without hotnessAdam Nemet2017-12-011-5/+4
| | | | | | | | | | | | | | These are blocks that haven't not been executed during training. For large projects this could make a significant difference. For the project, I was looking at, I got an order of magnitude decrease in the size of the total YAML files with this and r319235. Differential Revision: https://reviews.llvm.org/D40678 Re-commit after fixing the failing testcase in rL319576, rL319577 and rL319578. llvm-svn: 319581
* IR printing improvement for loop passes - handle -print-module-scopeFedor Sergeev2017-12-011-0/+12
| | | | | | | | | | | | | | | | | Summary: Adding support for -print-module-scope similar to how it is being done for function passes. This option causes loop-pass printer to emit a whole-module IR instead of just a loop itself. Reviewers: sanjoy, silvas, weimingz Reviewed By: sanjoy Subscribers: apilipenko, skatkov, llvm-commits Differential Revision: https://reviews.llvm.org/D40247 llvm-svn: 319566
OpenPOWER on IntegriCloud