summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
* [InstCombine] use m_Neg() in dyn_castNegVal() to match vectors with undef eltsSanjay Patel2018-10-191-2/+3
| | | | llvm-svn: 344793
* Use llvm::{all,any,none}_of instead std::{all,any,none}_of. NFCFangrui Song2018-10-192-17/+13
| | | | llvm-svn: 344774
* [LV] Fold tail by masking to vectorize loops of arbitrary trip count under ↵Ayal Zaks2018-10-184-38/+188
| | | | | | | | | | | | | | | | | | | | | | | | opt for size When optimizing for size, a loop is vectorized only if the resulting vector loop completely replaces the original scalar loop. This holds if no runtime guards are needed, if the original trip-count TC does not overflow, and if TC is a known constant that is a multiple of the VF. The last two TC-related conditions can be overcome by 1. rounding the trip-count of the vector loop up from TC to a multiple of VF; 2. masking the vector body under a newly introduced "if (i <= TC-1)" condition. The patch allows loops with arbitrary trip counts to be vectorized under -Os, subject to the existing cost model considerations. It also applies to loops with small trip counts (under -O2) which are currently handled as if under -Os. The patch does not handle loops with reductions, live-outs, or w/o a primary induction variable, and disallows interleave groups. (Third, final and main part of -) Differential Revision: https://reviews.llvm.org/D50480 llvm-svn: 344743
* Add a emitUnaryFloatFnCall version that fetches the function name from TLIMikael Holmen2018-10-183-11/+65
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: In several places in the code we use the following pattern: if (hasUnaryFloatFn(&TLI, Ty, LibFunc_tan, LibFunc_tanf, LibFunc_tanl)) { [...] Value *Res = emitUnaryFloatFnCall(X, TLI.getName(LibFunc_tan), B, Attrs); [...] } In short, we check if there is a lib-function for a certain type, and then we _always_ fetch the name of the "double" version of the lib function and construct a call to the appropriate function, that we just checked exists, using that "double" name as a basis. This is of course a problem in cases where the target doesn't support the "double" version, but e.g. only the "float" version. In that case TLI.getName(LibFunc_tan) returns "", and emitUnaryFloatFnCall happily appends an "f" to "", and we erroneously end up with a call to a function called "f". To solve this, the above pattern is changed to if (hasUnaryFloatFn(&TLI, Ty, LibFunc_tan, LibFunc_tanf, LibFunc_tanl)) { [...] Value *Res = emitUnaryFloatFnCall(X, &TLI, LibFunc_tan, LibFunc_tanf, LibFunc_tanl, B, Attrs); [...] } I.e instead of first fetching the name of the "double" version and then letting emitUnaryFloatFnCall() add the final "f" or "l", we let emitUnaryFloatFnCall() fetch the right name from TLI. Reviewers: eli.friedman, efriedma Reviewed By: efriedma Subscribers: efriedma, bjope, llvm-commits Differential Revision: https://reviews.llvm.org/D53370 llvm-svn: 344725
* [TI removal] Switch simple loop unswitch to `Instruction`.Chandler Carruth2018-10-181-5/+5
| | | | llvm-svn: 344719
* [TI removal] Switch NewGVN to directly use `Instruction`.Chandler Carruth2018-10-181-3/+3
| | | | llvm-svn: 344718
* [TI removal] Use `Instruction` instead of `TerminatorInst` forChandler Carruth2018-10-181-2/+2
| | | | | | a variable's type. llvm-svn: 344717
* [TI removal] Update CodeExtractor to use Instruction directly.Chandler Carruth2018-10-181-4/+4
| | | | llvm-svn: 344716
* [TI removal] Switch ObjCARC code to directly use the nice range-basedChandler Carruth2018-10-182-16/+9
| | | | | | | successors API or directly build the iterators out of the terminator instruction and avoid requiring a TerminatorInst variable. llvm-svn: 344715
* [TI removal] Switch MergeFunctions to directly use Instruction API.Chandler Carruth2018-10-181-1/+1
| | | | llvm-svn: 344714
* StructurizeCFG: Simplify inserted PHI nodesNicolai Haehnle2018-10-171-1/+23
| | | | | | | | | | | | | | | Summary: This improves subsequent divergence analysis in some cases. Change-Id: I5e95e7ec7fd3fa80d414d1a53a02fea23e3d67d3 Reviewers: arsenm, rampitec Subscribers: jvesely, wdng, llvm-commits Differential Revision: https://reviews.llvm.org/D53316 llvm-svn: 344697
* [LoopPredication] add some simple statsFedor Sergeev2018-10-171-0/+8
| | | | | | | Just adding some useful statistics to LoopPredication pass which was lacking any of these. llvm-svn: 344681
* [Sanitizer][PassManager] Fix for failing ASan tests on arm-linux-gnueabihfLeonard Chan2018-10-171-1/+3
| | | | | | | | Forgot to initialize the legacy pass in it's constructor. Differential Revision: https://reviews.llvm.org/D53350 llvm-svn: 344659
* [ThinLTO] Add importing stats to thin linkTeresa Johnson2018-10-161-5/+27
| | | | | | | | | | | | | | | Summary: Previously we could only get the number of imported functions and variables from the backend. This adds stats to the thin link where the importing is decided. Reviewers: wmi Subscribers: inglorion, dexonsmith, llvm-commits Differential Revision: https://reviews.llvm.org/D53337 llvm-svn: 344658
* [SanitizerCoverage] Don't duplicate code to get section pointersJonathan Metzman2018-10-161-33/+15
| | | | | | | | | | | | | | | | | Summary: Merge code used to get section start and section end pointers for SanitizerCoverage constructors. This includes code that handles getting the start pointers when targeting MSVC. Reviewers: kcc, morehouse Reviewed By: morehouse Subscribers: kcc, hiraditya Differential Revision: https://reviews.llvm.org/D53211 llvm-svn: 344657
* [InstCombine] Cleanup libfunc attribute inferringDavid Bolvansky2018-10-164-56/+74
| | | | | | | | | | | | Reviewers: efriedma Reviewed By: efriedma Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D53338 llvm-svn: 344645
* [LV] Teach vectorizer about variant value store into uniform addressAnna Thomas2018-10-162-4/+4
| | | | | | | | | | | | | | | | | | | | Summary: Teach vectorizer about vectorizing variant value stores to uniform address. Similar to rL343028, we do not allow vectorization if we have multiple stores to the same uniform address. Cost model already has the change for considering the extract instruction cost for a variant value store. See added test cases for how vectorization is done. The patch also contains changes to the ORE messages. Reviewers: Ayal, mkuper, anemet, hsaito Subscribers: rkruppe, llvm-commits Differential Revision: https://reviews.llvm.org/D52656 llvm-svn: 344613
* revert rL344609: [InstCombine] try harder to form select from logic opsSanjay Patel2018-10-162-41/+29
| | | | | | | | I noticed a missing check and added it at rL344610, but there actually are codegen tests that will fail without that, so I'll edit those and submit a fixed patch with more tests. llvm-svn: 344612
* [InstCombine] make sure type is integer before calling ComputeNumSignBitsSanjay Patel2018-10-161-1/+2
| | | | llvm-svn: 344610
* [InstCombine] try harder to form select from logic opsSanjay Patel2018-10-162-29/+40
| | | | | | | | | | | | | | | This is part of solving PR37549: https://bugs.llvm.org/show_bug.cgi?id=37549 The patterns shown here are a special case of something that we already convert to select. Using ComputeNumSignBits() catches that case (but not the more complicated motivating patterns yet). The backend has hooks/logic to convert back to logic ops if that's better for the target. llvm-svn: 344609
* [NFC] Make LoopSafetyInfo abstract to allow alternative implementationsMax Kazantsev2018-10-164-4/+4
| | | | llvm-svn: 344592
* [NFC] Encapsulate work with BlockColors in LoopSafetyInfoMax Kazantsev2018-10-161-9/+6
| | | | llvm-svn: 344590
* [DebugInfo][LCSSA] Rewrite pre-existing debug values outside loopDavid Stenberg2018-10-162-0/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Extend LCSSA so that debug values outside loops are rewritten to use the PHI nodes that the pass creates. This fixes PR39019. In that case, we ran LCSSA on a loop that was later on vectorized, which left us with something like this: for.cond.cleanup: %add.lcssa = phi i32 [ %add, %for.body ], [ %34, %middle.block ] call void @llvm.dbg.value(metadata i32 %add, ret i32 %add.lcssa for.body: %add = [...] br i1 %exitcond, label %for.cond.cleanup, label %for.body which later resulted in the debug.value becoming undef when removing the scalar loop (and the location would have probably been wrong for the vectorized case otherwise). As we now may need to query the AvailableVals cache more than once for a basic block, FindAvailableVals() in SSAUpdaterImpl is changed so that it updates the cache for blocks that we do not create a PHI node for, regardless of the block's number of predecessors. The debug value in the attached IR reproducer would not be properly rewritten without this. Debug values residing in blocks where we have not inserted any PHI nodes are currently left as-is by this patch. I'm not sure what should be done with those uses. Reviewers: mattd, aprantl, vsk, probinson Reviewed By: mattd, aprantl Subscribers: jmorse, gbedwell, JDevlieghere, llvm-commits Differential Revision: https://reviews.llvm.org/D53130 llvm-svn: 344589
* [NFC] Turn isGuaranteedToExecute into a methodMax Kazantsev2018-10-162-4/+4
| | | | llvm-svn: 344587
* Change a TerminatorInst* to an Instruction* in HotColdSplitting.cpp.Lang Hames2018-10-151-1/+1
| | | | | | | | | | | r344558 added an assignment to a TerminatorInst* from BasicBlock::getTerminatorInst(), but BasicBlock::getTerminatorInst() returns an Instruction* rather than a TerminatorInst* since r344504 so this fails to compile. Changing the variable to an Instruction* should get the bots building again. llvm-svn: 344566
* [hot-cold-split] fix static analysis of cold regionsSebastian Pop2018-10-151-7/+41
| | | | | | | | | | | | | | | | | | | | | | Make the code of blockEndsInUnreachable to match the function blockEndsInUnreachable in CodeGen/BranchFolding.cpp. I also have added a note to make sure the code of this function will not be modified unless the back-end version is also modified. An early return before outlining has been added to avoid outlining the full function body when the first block in the function is marked cold. The static analysis of cold code has been amended to avoid marking the whole function as cold by back-propagation because the back-propagation would mark blocks with return statements as cold. The patch adds debug statements to help discover these problems. Differential Revision: https://reviews.llvm.org/D52904 llvm-svn: 344558
* [CodeExtractor] Erase debug intrinsics in outlined thunks (fix PR22900)Vedant Kumar2018-10-151-0/+13
| | | | | | | | | | | | | | | Variable updates within the outlined function are invisible to debuggers. This could be improved by defining a DISubprogram for the new function. For the moment, simply erase the debug intrinsics instead. This fixes verifier failures about function-local metadata being used in the wrong function, seen while testing the hot/cold splitting pass. rdar://45142482 Differential Revision: https://reviews.llvm.org/D53267 llvm-svn: 344545
* [TI removal] Make `getTerminator()` return a generic `Instruction`.Chandler Carruth2018-10-153-11/+11
| | | | | | | | | | | | | | | | | | | | | | | | This removes the primary remaining API producing `TerminatorInst` which will reduce the rate at which code is introduced trying to use it and generally make it much easier to remove the remaining APIs across the codebase. Also clean up some of the stragglers that the previous mechanical update of variables missed. Users of LLVM and out-of-tree code generally will need to update any explicit variable types to handle this. Replacing `TerminatorInst` with `Instruction` (or `auto`) almost always works. Most of these edits were made in prior commits using the perl one-liner: ``` perl -i -ple 's/TerminatorInst(\b.* = .*getTerminator\(\))/Instruction\1/g' ``` This also my break some rare use cases where people overload for both `Instruction` and `TerminatorInst`, but these should be easily fixed by removing the `TerminatorInst` overload. llvm-svn: 344504
* [TI removal] Rework `InstVisitor` to support visiting instructions thatChandler Carruth2018-10-151-8/+8
| | | | | | | | | are terminators without relying on the specific `TerminatorInst` type. This required cleaning up two users of `InstVisitor`s usage of `TerminatorInst` as well. llvm-svn: 344503
* [TI removal] Make variables declared as `TerminatorInst` and initializedChandler Carruth2018-10-1537-116/+115
| | | | | | | | | | | | | by `getTerminator()` calls instead be declared as `Instruction`. This is the biggest remaining chunk of the usage of `getTerminator()` that insists on the narrow type and so is an easy batch of updates. Several files saw more extensive updates where this would cascade to requiring API updates within the file to use `Instruction` instead of `TerminatorInst`. All of these were trivial in nature (pervasively using `Instruction` instead just worked). llvm-svn: 344502
* [TI removal] Remove `TerminatorInst` from GVN.h and GVN.cpp.Chandler Carruth2018-10-151-1/+1
| | | | | | | | | | This is the last interesting usage in all of LLVM's headers. The remaining usages in headers are the core typesystem bits (Core.h, instruction types, and InstVisitor) and as the return of `BasicBlock::getTerminator`. The latter is the big remaining API point that I'll remove after mass updates to user code. llvm-svn: 344501
* [TI removal] Remove `TerminatorInst` from BasicBlockUtils.hChandler Carruth2018-10-158-32/+34
| | | | | | | | | This requires updating a number of .cpp files to adapt to the new API. I've just systematically updated all uses of `TerminatorInst` within these files te `Instruction` so thta I won't have to touch them again in the future. llvm-svn: 344498
* [TI removal] Remove TerminatorInst as an input parameter from all publicChandler Carruth2018-10-151-1/+1
| | | | | | | | | LLVM APIs. There weren't very many. We still have the instruction visitor, and APIs with TerminatorInst as a return type or an output parameter. llvm-svn: 344494
* [LV] Fix comments reported when not vectorizing single iteration loops; NFCAyal Zaks2018-10-141-1/+8
| | | | | | | | Landing this as a separate part of https://reviews.llvm.org/D50480, being a seemingly unrelated change ([LV] Vectorizing loops of arbitrary trip count without remainder under opt for size). llvm-svn: 344483
* [InstCombine] combine a shuffle and an extract subvector shuffle Sanjay Patel2018-10-141-0/+38
| | | | | | | | | | | | This is part of the missing IR-level folding noted in D52912. This should be ok as a canonicalization because the new shuffle mask can't be any more complicated than the existing shuffle mask. If there's some target where the shorter vector shuffle is not legal, it should just end up expanding to something like the pair of shuffles that we're starting with here. Differential Revision: https://reviews.llvm.org/D53037 llvm-svn: 344476
* recommit 344472 after fixing build failure on ARM and PPC.Dorit Nuzman2018-10-143-21/+116
| | | | llvm-svn: 344475
* revert 344472 due to failures.Dorit Nuzman2018-10-143-116/+21
| | | | llvm-svn: 344473
* [IAI,LV] Add support for vectorizing predicated strided accesses using maskedDorit Nuzman2018-10-143-21/+116
| | | | | | | | | | | | | | | | | | | | | | | interleave-group The vectorizer currently does not attempt to create interleave-groups that contain predicated loads/stores; predicated strided accesses can currently be vectorized only using masked gather/scatter or scalarization. This patch makes predicated loads/stores candidates for forming interleave-groups during the Loop-Vectorizer's analysis, and adds the proper support for masked-interleave- groups to the Loop-Vectorizer's planning and transformation stages. The patch also extends the TTI API to allow querying the cost of masked interleave groups (which each target can control); Targets that support masked vector loads/ stores may choose to enable this feature and allow vectorizing predicated strided loads/stores using masked wide loads/stores and shuffles. Reviewers: Ayal, hsaito, dcaballe, fhahn, javed.absar Reviewed By: Ayal Differential Revision: https://reviews.llvm.org/D53011 llvm-svn: 344472
* Move some helpers from the global namespace into anonymous ones.Benjamin Kramer2018-10-131-3/+4
| | | | llvm-svn: 344468
* [InstCombine] fix complexity canonicalization with fake unary vector opsSanjay Patel2018-10-131-2/+2
| | | | | | | This is a preliminary step to avoid regressions when we add an actual 'fneg' instruction to IR. See D52934 and D53205. llvm-svn: 344458
* [InstCombine] Fixed crash with aliased functionsDavid Bolvansky2018-10-134-24/+28
| | | | | | | | | | | | | | Summary: Fixes PR39177 Reviewers: spatel, jbuening Reviewed By: jbuening Subscribers: jbuening, llvm-commits Differential Revision: https://reviews.llvm.org/D53129 llvm-svn: 344454
* move GetOrCreateFunctionComdat to Instrumentation.cpp/Instrumentation.hKostya Serebryany2018-10-122-18/+19
| | | | | | | | | | | | | | | | | | Summary: GetOrCreateFunctionComdat is currently used in SanitizerCoverage, where it's defined. I'm planing to use it in HWASAN as well, so moving it into a common location. NFC Reviewers: morehouse Reviewed By: morehouse Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D53218 llvm-svn: 344433
* [SanitizerCoverage] Prevent /OPT:REF from stripping constructorsJonathan Metzman2018-10-121-0/+21
| | | | | | | | | | | | | | | | | | | | | Summary: Linking with the /OPT:REF linker flag when building COFF files causes the linker to strip SanitizerCoverage's constructors. Prevent this by giving the constructors WeakODR linkage and by passing the linker a directive to include sancov.module_ctor. Include a test in compiler-rt to verify libFuzzer can be linked using /OPT:REF Reviewers: morehouse, rnk Reviewed By: morehouse, rnk Subscribers: rnk, morehouse, hiraditya Differential Revision: https://reviews.llvm.org/D52119 llvm-svn: 344391
* [SanitizerCoverage] Make Inline8bit and TracePC counters dead stripping ↵Max Moroz2018-10-121-3/+4
| | | | | | | | | | | | | | | | | | | | resistant. Summary: Otherwise, at least on Mac, the linker eliminates unused symbols which causes libFuzzer to error out due to a mismatch of the sizes of coverage tables. Issue in Chromium: https://bugs.chromium.org/p/chromium/issues/detail?id=892167 Reviewers: morehouse, kcc, george.karpenkov Reviewed By: morehouse Subscribers: kubamracek, llvm-commits Differential Revision: https://reviews.llvm.org/D53113 llvm-svn: 344345
* SCCP: avoid caching DenseMap entry that might be invalidated.Tim Northover2018-10-121-3/+5
| | | | | | | Later calls to getValueState might insert entries into the ValueState map and cause reallocation, invalidating a reference. llvm-svn: 344327
* [ThinLTO] Don't import GV which contains blockaddressEugene Leviant2018-10-121-2/+1
| | | | | | Differential revision: https://reviews.llvm.org/D53139 llvm-svn: 344325
* merge two near-identical functions createPrivateGlobalForString into oneKostya Serebryany2018-10-113-33/+21
| | | | | | | | | | | | | | | | Summary: We have two copies of createPrivateGlobalForString (in asan and in esan). This change merges them into one. NFC Reviewers: vitalybuka Reviewed By: vitalybuka Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D53178 llvm-svn: 344314
* [PassManager/Sanitizer] Port of AddresSanitizer pass from legacy to new ↵Leonard Chan2018-10-112-65/+117
| | | | | | | | | | | | | | | | PassManager This patch ports the legacy pass manager to the new one to take advantage of the benefits of the new PM. This involved moving a lot of the declarations for `AddressSantizer` to a header so that it can be publicly used via PassRegistry.def which I believe contains all the passes managed by the new PM. This patch essentially decouples the instrumentation from the legacy PM such hat it can be used by both legacy and new PM infrastructure. Differential Revision: https://reviews.llvm.org/D52739 llvm-svn: 344274
* [InstCombine] Fix SimplifyLibCalls erasing an instruction while IC still had ↵Amara Emerson2018-10-112-11/+19
| | | | | | | | | | | | | references to it. InstCombine keeps a worklist and assumes that optimizations don't eraseFromParent() the instruction, which SimplifyLibCalls violates. This change adds a new callback to SimplifyLibCalls to let clients specify their own hander for erasing actions. Differential Revision: https://reviews.llvm.org/D52729 llvm-svn: 344251
* [InstCombine] Demand bits of UMinDavid Green2018-10-111-0/+10
| | | | | | | | | | This is the umin alternative to the umax code from rL344237. We use DeMorgans law on the umax case to bring us to the same thing on umin, but using countLeadingOnes, not countLeadingZeros. Differential Revision: https://reviews.llvm.org/D53036 llvm-svn: 344239
OpenPOWER on IntegriCloud