summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
...
* Recommit "[ThinLTO] Add correctness check for RO/WO variable import"evgeny2019-11-153-17/+46
| | | | | | ValueInfo has user-defined 'operator bool' which allows incorrect implicit conversion to GlobalValue::GUID (which is unsigned long). This causes bugs which are hard to track and should be removed in future.
* [Scalarizer] Treat values from unreachable blocks as undefMikael Holmen2019-11-151-5/+27
| | | | | | | | | | | | | | | | | | | | | | | | Summary: When scalarizing PHI nodes we might try to examine/rewrite InsertElement nodes in predecessors. If those predecessors are unreachable from entry, then the IR in those blocks could have unexpected properties resulting in infinite loops in Scatterer::operator[]. By simply treating values originating from instructions in unreachable blocks as undef we do not need to analyse them further. This fixes PR41723. Reviewers: bjope Reviewed By: bjope Subscribers: bjope, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70171
* [InstCombine] Don't use getFirstNonPHI in FoldIntegerTypedPHIFrancis Visoiu Mistrih2019-11-141-4/+5
| | | | | | | | | | | | | | | | | | | getFirstNonPHI iterates over all the instructions in a block until it finds a non-PHI. Then, the loop starts from the beginning of the block and goes through all the instructions until it reaches the instruction found by getFirstNonPHI. Instead of doing that, just stop when a non-PHI is found. This reduces the compile-time of a test case discussed in https://reviews.llvm.org/D47023 by 13x. Not entirely sure how to come up with a test case for this since it's a compile time issue that would significantly slow down running the tests. Differential Revision: https://reviews.llvm.org/D70016
* Add missing includes needed to prune LLVMContext.h include, NFCReid Kleckner2019-11-1428-5/+33
| | | | | These are a pre-requisite to removing #include "llvm/Support/Options.h" from LLVMContext.h: https://reviews.llvm.org/D70280
* Revert "Temporarily Revert:"Alexey Bataev2019-11-141-98/+178
| | | | | | | | | | | | This reverts commit e511c4b0dff1692c267addf17dce3cebe8f97faa: Temporarily Revert: "[SLP] Generalization of stores vectorization." "[SLP] Fix -Wunused-variable. NFC" "[SLP] Vectorize jumbled stores." after fixing the problem with compile time.
* [InstCombine] remove duplicate code for simplifying a shuffle; NFCISanjay Patel2019-11-141-7/+0
| | | | | The transform is already handled by InstSimplify or earlier in InstCombine, so trying to do it again is not necessary.
* Revert "[ThinLTO] Add correctness check for RO/WO variable import"Benjamin Kramer2019-11-143-48/+21
| | | | | This reverts commit a2292cc537b561416c21e8d4017715d652c144cc. Breaks clang selfhost w/ThinLTO.
* GCOVProfiling - fix uninitialized variable warnings + make getFuncChecksum() ↵Simon Pilgrim2019-11-141-3/+3
| | | | const. NFCI.
* WholeProgramDevirt - fix uninitialized variable warnings. NFCI.Simon Pilgrim2019-11-141-2/+2
|
* Fix uninitialized variable warning. NFCI.Simon Pilgrim2019-11-141-2/+2
|
* SROA - fix uninitialized variable warnings. NFCI.Simon Pilgrim2019-11-141-5/+5
|
* [LV] PreferPredicateOverEpilog respecting predicate loop hintSjoerd Meijer2019-11-142-3/+5
| | | | | | | | | | | The vectoriser queries TTI->preferPredicateOverEpilogue to determine if tail-folding is preferred for a loop, but it was not respecting loop hint 'predicate' that can disable this, which has now been added. This showed that we were incorrectly initialising loop hint 'vectorize.predicate.enable' with 0 (i.e. FK_Disabled) but this should have been FK_Undefined, which has been fixed. Differential Revision: https://reviews.llvm.org/D70125
* Revert "[InstCombine] Fold PHIs with equal incoming pointers"Daniil Suchkov2019-11-142-70/+0
| | | | | This reverts commit a2f6ae9abffcba260c22bb235879f0576bf3b783. It is reverted due to clang-cmake-armv7-selfhost buildbot failure.
* [InstCombine] Fold PHIs with equal incoming pointersDaniil Suchkov2019-11-142-0/+70
| | | | | | | | | | | | | | | | | | | | | | | | This is a resubmission of bbb29738b58aaf6f6518269abdcf8f64131665a9 that was reverted due to clang tests failures. It includes the fix and additional IR tests for the missed case. Summary: In case when all incoming values of a PHI are equal pointers, this transformation inserts a definition of such a pointer right after definition of the base pointer and replaces with this value both PHI and all it's incoming pointers. Primary goal of this transformation is canonicalization of this pattern in order to enable optimizations that can't handle PHIs. Non-inbounds pointers aren't currently supported. Reviewers: spatel, RKSimon, lebedev.ri, apilipenko Reviewed By: apilipenko Tags: #llvm Subscribers: hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D68128
* [ThinLTO] Add correctness check for RO/WO variable importevgeny2019-11-143-21/+48
| | | | | | | | | | | This patch adds an assertion check for exported read/write-only variables to be also in import list for module. If they aren't we may face linker errors, because read/write-only variables are internalized in their source modules. The patch also changes export lists to store ValueInfo instead of GUID for performance considerations. Differential revision: https://reviews.llvm.org/D70128
* Check result of emitStrLen before passing it to CreateGEPDimitry Andric2019-11-141-2/+2
| | | | | | | | | | | | | | | | | | Summary: This fixes PR43081, where the transformation of `strchr(p, 0) -> p + strlen(p)` can cause a segfault, if `-fno-builtin-strlen` is used. In that case, `emitStrLen` returns nullptr, which CreateGEP is not designed to handle. Also add the minimized code from the PR as a test case. Reviewers: xbolva00, spatel, jdoerfert, efriedma Reviewed By: efriedma Subscribers: lebedev.ri, hiraditya, cfe-commits, llvm-commits Tags: #clang, #llvm Differential Revision: https://reviews.llvm.org/D70143
* Sink all InitializePasses.h includesReid Kleckner2019-11-13147-41/+196
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This file lists every pass in LLVM, and is included by Pass.h, which is very popular. Every time we add, remove, or rename a pass in LLVM, it caused lots of recompilation. I found this fact by looking at this table, which is sorted by the number of times a file was changed over the last 100,000 git commits multiplied by the number of object files that depend on it in the current checkout: recompiles touches affected_files header 342380 95 3604 llvm/include/llvm/ADT/STLExtras.h 314730 234 1345 llvm/include/llvm/InitializePasses.h 307036 118 2602 llvm/include/llvm/ADT/APInt.h 213049 59 3611 llvm/include/llvm/Support/MathExtras.h 170422 47 3626 llvm/include/llvm/Support/Compiler.h 162225 45 3605 llvm/include/llvm/ADT/Optional.h 158319 63 2513 llvm/include/llvm/ADT/Triple.h 140322 39 3598 llvm/include/llvm/ADT/StringRef.h 137647 59 2333 llvm/include/llvm/Support/Error.h 131619 73 1803 llvm/include/llvm/Support/FileSystem.h Before this change, touching InitializePasses.h would cause 1345 files to recompile. After this change, touching it only causes 550 compiles in an incremental rebuild. Reviewers: bkramer, asbirlea, bollu, jdoerfert Differential Revision: https://reviews.llvm.org/D70211
* [PGO][PGSO] Temporarily disable the large working set size behavior.Hiroshi Yamauchi2019-11-131-0/+5
| | | | | | | | | | | | | | Summary: This temporarily disables the large working set size behavior in profile guided size optimization due to internal benchmark regressions. Reviewers: davidxl Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70207
* [SLP] fix miscompile on min/max reductions with extra uses (PR43948)Sanjay Patel2019-11-131-1/+14
| | | | | | | | | | | | | | | | | The bug manifests as replacing a reduction operand with an undef value. The problem appears to be limited to cases where a min/max reduction has extra uses of the compare operand to the select. In the general case, we are tracking "ExternallyUsedValues" and an "IgnoreList" of the reduction operations, but those may not apply to the final compare+select in a min/max reduction. For that, we use replaceAllUsesWith (RAUW) to ensure that the new vectorized reduction values are transferred to all subsequent users. Differential Revision: https://reviews.llvm.org/D70148
* [SLP] reduce code duplication for min/max vs. other reductions; NFCISanjay Patel2019-11-131-77/+31
|
* [InstCombine] propagate fast-math-flags (FMF) to select when inverting ↵Sanjay Patel2019-11-131-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | fcmp+select As noted by the FIXME comment, this is not correct based on our current FMF semantics. We should be propagating FMF from the final value in a sequence (in this case the 'select'). So the behavior even without this patch is wrong, but we did not allow FMF on 'select' until recently. But if we do the correct thing right now in this patch, we'll inevitably introduce regressions because we have not wired up FMF propagation for 'phi' and 'select' in other passes (like SimplifyCFG) or other places in InstCombine. I'm not seeing a better incremental way to make progress. That said, the potential extra damage over the existing wrong behavior from this patch is very limited. AFAIK, the only way to have different FMF on IR in the same function is if we have LTO inlined IR from 2 modules that were compiled using different fast-math settings. As seen in the tests, we may actually see some improvements with this patch because adding the FMF to the 'select' allows matching to min/max intrinsics that were previously missed (in the common case, the 'fcmp' and 'select' should have identical FMF to begin with). Next steps in the transition: Make similar changes in instcombine as needed. Enable phi-to-select FMF propagation in SimplifyCFG. Remove dependencies on fcmp with FMF. Deprecate FMF on fcmp. Differential Revision: https://reviews.llvm.org/D69720
* SLPVectorizer - make comparison operators + isInSchedulingRegion constSimon Pilgrim2019-11-131-3/+3
| | | | Fixes cppcheck warnings.
* [InstCombine] Avoid moving ops that do restrict undef across shuffles.Florian Hahn2019-11-131-1/+3
| | | | | | | | | | | | | | | | | | | I think we have to be a bit more careful when it comes to moving ops across shuffles, if the op does restrict undef. For example, without this patch, we would move 'and %v, <0, 0, -1, -1>' over a 'shufflevector %a, undef, <undef, undef, 1, 2>'. As a result, the first 2 lanes of the result are undef after the combine, but they really should be 0, unless I am missing something. For ops that do fold to undef on undef operands, the current behavior should be fine. I've add conservative check OpDoesRestrictUndef, maybe there's a better existing utility? Reviewers: spatel, RKSimon, lebedev.ri Reviewed By: spatel Differential Revision: https://reviews.llvm.org/D70093
* Temporarily revert "[InstCombine] Fold PHIs with equal incoming pointers"Daniil Suchkov2019-11-132-65/+0
| | | | | | Revert due to sanitizer-windows buildbot failure. This reverts commit bbb29738b58aaf6f6518269abdcf8f64131665a9.
* [InstCombine] Fold PHIs with equal incoming pointersDaniil Suchkov2019-11-132-0/+65
| | | | | | | | | | | | | | | | | | | In case when all incoming values of a PHI are equal pointers, this transformation inserts a definition of such a pointer right after definition of the base pointer and replaces with this value both PHI and all it's incoming pointers. Primary goal of this transformation is canonicalization of this pattern in order to enable optimizations that can't handle PHIs. Non-inbounds pointers aren't currently supported. Reviewers: spatel, RKSimon, lebedev.ri, apilipenko Reviewed By: apilipenko Tags: #llvm Subscribers: hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D68128
* [GVNHoist] Preserve AAResults.Alina Sbirlea2019-11-121-0/+1
| | | | Resolves PR38906, PR40898.
* [DBG][OPT] Attempt to salvage or undef debug info when removing trivially ↵Tom Weaver2019-11-121-0/+2
| | | | | | | | deletable instructions in the Reassociate Expression pass. Reviewed By: aprantl, vsk Differential revision: https://reviews.llvm.org/D69943
* [InstCombine] Skip scalable vectors in combineLoadToOperationTypeDiana Picus2019-11-121-0/+1
| | | | | | | | | | | | | | | | Don't try to canonicalize loads to scalable vector types to loads of integers. This removes one assertion when trying to use a TypeSize as a parameter to DataLayout::isLegalInteger. It does not handle the second part of the function (which looks at bitcasts). This patch also contains a NFC fix for Load Analysis, where a variable initialization that would cause the same assertion is moved closer to its use. This allows us to run the new test for InstCombine without having to teach LocationSize to play nicely with scalable vectors. Differential Revision: https://reviews.llvm.org/D70075
* [LoopInterchange] Only skip PHIs with incoming values from the inner loop.Florian Hahn2019-11-121-7/+7
| | | | | | | | | | | | | | Currently we have limited support for outer loops with multiple basic blocks after the inner loop exit. But the current checks for creating PHIs for loop exit values only assumes the header and latches of the outer loop. It is better to just skip incoming values defined in the original inner loops. Those are handled earlier. Reviewers: efriedma, mcrosier Reviewed By: efriedma Differential Revision: https://reviews.llvm.org/D70059
* [Attributor] Use must-be-executed-context in align deductionHideto Ueno2019-11-121-20/+59
| | | | | | | | | | | | | | | Summary: This patch introduces align attribute deduction for callsite argument, function argument, function returned and floating value based on must-be-executed-context. Reviewers: jdoerfert, sstefan1 Reviewed By: jdoerfert Subscribers: hiraditya, jfb, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69797
* [SLP] Look-ahead operand reordering heuristic.Vasileios Porpodas2019-11-111-46/+260
| | | | | | | | | | | | | | Summary: This patch introduces a new heuristic for guiding operand reordering. The new "look-ahead" heuristic can look beyond the immediate predecessors. This helps break ties when the immediate predecessors have identical opcodes (see lit test for examples). Reviewers: RKSimon, ABataev, dtemirbulatov, Ayal, hfinkel, rnk Reviewed By: RKSimon, dtemirbulatov Subscribers: xbolva00, Carrot, hiraditya, phosek, rnk, rcorcs, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60897
* [VFABI] Read/Write functions for the VFABI attribute.Francesco Petrogalli2019-11-121-0/+29
| | | | | | | | | | | | | | The attribute is stored at the `FunctionIndex` attribute set, with the name "vector-function-abi-variant". The get/set methods of the attribute have assertion to verify that: 1. Each name in the attribute is a valid VFABI mangled name. 2. Each name in the attribute correspond to a function declared in the module. Differential Revision: https://reviews.llvm.org/D69976
* Add InstCombine/InstructionSimplify support for Freeze Instructionaqjune2019-11-122-0/+10
| | | | | | | | | | | | | | | Summary: - Add llvm::SimplifyFreezeInst - Add InstCombiner::visitFreeze - Add llvm tests Reviewers: majnemer, sanjoy, reames, lebedev.ri, spatel Reviewed By: reames, lebedev.ri Subscribers: reames, lebedev.ri, filcab, regehr, trentxintong, llvm-commits Differential Revision: https://reviews.llvm.org/D29013
* Revert "[InstCombine] avoid crash from deleting an instruction that still ↵Sanjay Patel2019-11-111-18/+4
| | | | | | | | has uses (PR43723) (3rd try)" This reverts commit 3db8a3ef86e7b3331ab466a78c10a62be9e69829. This caused a different memory-sanitizer failure than earlier attempts, but it's still not right.
* [InstCombine] avoid crash from deleting an instruction that still has uses ↵Sanjay Patel2019-11-111-4/+18
| | | | | | | | | | | | | | | | | | | | | (PR43723) (3rd try) Re-try because earlier attempts were reverted due to use-after-free. Hopefully, diagnosed correctly this time - we replace/remove the invariant.start first rather than the invariant.end to avoid angering worklist-based iteration. We gather a set of white-listed instructions in isAllocSiteRemovable() and then replace/erase them. But we don't know in general if the instructions in the set have uses amongst themselves, so order of deletion makes a difference. There's already a special-case for the llvm.objectsize intrinsic, so add another for llvm.invariant.start. Should fix: https://bugs.llvm.org/show_bug.cgi?id=43723 Differential Revision: https://reviews.llvm.org/D69977
* Revert "[DBG][OPT] Attempt to salvage or undef debug info when removing ↵Tom Weaver2019-11-111-2506/+0
| | | | | | trivially deletable instructions in the Reassociate Expression pass." This reverts commit 1984a27db58e9053371ab6d6dc288c81c8a071ac.
* [DBG][OPT] Attempt to salvage or undef debug info when removing trivially ↵Tom Weaver2019-11-111-0/+2506
| | | | | | | | deletable instructions in the Reassociate Expression pass. Reviewed By: aprantl, vsk Differential revision: https://reviews.llvm.org/D69943
* [NFC][TEST_COMMIT] Add fullstop to comment.Tom Weaver2019-11-111-1/+1
|
* [InstCombine] Simplify binary op when only one operand is a selectJay Foad2019-11-111-0/+10
| | | | | | | | | | | | | | | | | | | | | | Summary: SimplifySelectsFeedingBinaryOp simplified binary ops when both operands were selects with the same condition. This patch extends it to handle these cases where only one operand is a select: X op (C ? P : Q) -> C ? (X op P) : (X op Q) // if X op P and X op Q both simplify (C ? P : Q) op Y -> C ? (P op Y) : (Q op Y) // if P op Y and Q op Y both simplify For example: X *fast (C ? 1.0 : 0.0) -> C ? X : 0.0 Reviewers: mcberg2017, majnemer, craig.topper, qcolombet, mcrosier Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D64713
* [InstCombine] Turn (extractelement <1 x i64/double> (bitcast (x86_mmx))) ↵Craig Topper2019-11-101-0/+7
| | | | | | | | | | | | | into a single bitcast from x86_mmx to i64/double. The _m64 type is represented in IR as <1 x i64>. The x86-64 ABI on Linux passes <1 x i64> as a double. MMX intrinsics use x86_mmx type in IR.These things result in a lot of bitcasts in mmx code. There's another instcombine that tries to turn bitcast <1 x i64> to double into extractelement and a bitcast. The combine here tries to reverse this extractelement conversion if we see an mmx type.
* Revert "[InstCombine] avoid crash from deleting an instruction that still ↵Sanjay Patel2019-11-101-20/+4
| | | | | | | has uses (PR43723) (2nd try)" This reverts commit 56b2aee1875a1ee47ddf859a6584f8728787fb7b. Still causes a use-after-free on sanitizer bots.
* [InstCombine] avoid crash from deleting an instruction that still has uses ↵Sanjay Patel2019-11-101-4/+20
| | | | | | | | | | | | | | | | | | | (PR43723) (2nd try) Re-try rGef02831f0a4e (reverted due to use-after-free), but bail out completely if we encounter an unexpected llvm.invariant.start. We gather a set of white-listed instructions in isAllocSiteRemovable() and then replace/erase them. But we don't know in general if the instructions in the set have uses amongst themselves, so order of deletion makes a difference. There's already a special-case for the llvm.objectsize intrinsic, so add another for llvm.invariant.end. Should fix: https://bugs.llvm.org/show_bug.cgi?id=43723 Differential Revision: https://reviews.llvm.org/D69977
* getArgOperandNo helper function.Stefan Stipanovic2019-11-101-2/+2
| | | | | | | | | | | | Summary: A helper function to get argument number of a arg operand Use. Reviewers: jdoerfert, uenoku Subscribers: hiraditya, lebedev.ri, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D66844
* Revert "[InstCombine] avoid crash from deleting an instruction that still ↵Sanjay Patel2019-11-101-17/+4
| | | | | | | has uses (PR43723)" This reverts commit ef02831f0a4e3b3ccaa45a5589e4cabecbf527ab. Sanitizer bots fail with this change.
* [InstCombine] avoid crash from deleting an instruction that still has uses ↵Sanjay Patel2019-11-101-4/+17
| | | | | | | | | | | | | | | | (PR43723) We gather a set of white-listed instructions in isAllocSiteRemovable() and then replace/erase them. But we don't know in general if the instructions in the set have uses amongst themselves, so order of deletion makes a difference. There's already a special-case for the llvm.objectsize intrinsic, so add another for llvm.invariant.end. Should fix: https://bugs.llvm.org/show_bug.cgi?id=43723 Differential Revision: https://reviews.llvm.org/D69977
* Remove unused variable (which allows us to remove vector include). NFC.Simon Pilgrim2019-11-101-3/+0
|
* [LV] Apply sink-after & interleave-groups as VPlan transformations (NFCI)Gil Rapaport2019-11-095-128/+175
| | | | | | | | | This recommits 11ed1c0239fd51fd2f064311dc7725277ed0a994 (reverted in 9f08ce0d2197d4f163dfa4633eae2347ce8fc881 for failing an assert) with a fix: tryToWidenMemory() now first checks if the widening decision is to interleave, thus maintaining previous behavior where tryToInterleaveMemory() was called first, giving priority to interleave decisions over widening/scalarization. This commit adds the test case that exposed this bug as a LIT.
* Refactor SimplifySelectsFeedingBinaryOp for D64713. NFC.Jay Foad2019-11-091-25/+32
|
* ThinLTO : Import always_inline functions irrespective of the thresholdTeresa Johnson2019-11-081-2/+4
| | | | | | | | | | | | | | | | Summary: A user can force a function to be inlined by specifying the always_inline attribute. Currently, thinlto implementation is not aware of always_inline functions and does not guarantee import of such functions, which in turn can prevent inlining of such functions. Patch by Bharathi Seshadri <bseshadr@cisco.com> Reviewers: tejohnson Reviewed By: tejohnson Subscribers: mehdi_amini, inglorion, hiraditya, steven_wu, dexonsmith, arphaman, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70014
* Revert "[LV] Apply sink-after & interleave-groups as VPlan transformations ↵Gil Rapaport2019-11-085-170/+125
| | | | | | (NFCI)" This reverts commit 11ed1c0239fd51fd2f064311dc7725277ed0a994 - causes an assert failure.
OpenPOWER on IntegriCloud