summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
...
* GCOVProfiling - fix uninitialized variable warnings + make getFuncChecksum() ↵Simon Pilgrim2019-11-141-3/+3
| | | | const. NFCI.
* WholeProgramDevirt - fix uninitialized variable warnings. NFCI.Simon Pilgrim2019-11-141-2/+2
|
* Fix uninitialized variable warning. NFCI.Simon Pilgrim2019-11-141-2/+2
|
* SROA - fix uninitialized variable warnings. NFCI.Simon Pilgrim2019-11-141-5/+5
|
* [LV] PreferPredicateOverEpilog respecting predicate loop hintSjoerd Meijer2019-11-142-3/+5
| | | | | | | | | | | The vectoriser queries TTI->preferPredicateOverEpilogue to determine if tail-folding is preferred for a loop, but it was not respecting loop hint 'predicate' that can disable this, which has now been added. This showed that we were incorrectly initialising loop hint 'vectorize.predicate.enable' with 0 (i.e. FK_Disabled) but this should have been FK_Undefined, which has been fixed. Differential Revision: https://reviews.llvm.org/D70125
* Revert "[InstCombine] Fold PHIs with equal incoming pointers"Daniil Suchkov2019-11-142-70/+0
| | | | | This reverts commit a2f6ae9abffcba260c22bb235879f0576bf3b783. It is reverted due to clang-cmake-armv7-selfhost buildbot failure.
* [InstCombine] Fold PHIs with equal incoming pointersDaniil Suchkov2019-11-142-0/+70
| | | | | | | | | | | | | | | | | | | | | | | | This is a resubmission of bbb29738b58aaf6f6518269abdcf8f64131665a9 that was reverted due to clang tests failures. It includes the fix and additional IR tests for the missed case. Summary: In case when all incoming values of a PHI are equal pointers, this transformation inserts a definition of such a pointer right after definition of the base pointer and replaces with this value both PHI and all it's incoming pointers. Primary goal of this transformation is canonicalization of this pattern in order to enable optimizations that can't handle PHIs. Non-inbounds pointers aren't currently supported. Reviewers: spatel, RKSimon, lebedev.ri, apilipenko Reviewed By: apilipenko Tags: #llvm Subscribers: hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D68128
* [ThinLTO] Add correctness check for RO/WO variable importevgeny2019-11-143-21/+48
| | | | | | | | | | | This patch adds an assertion check for exported read/write-only variables to be also in import list for module. If they aren't we may face linker errors, because read/write-only variables are internalized in their source modules. The patch also changes export lists to store ValueInfo instead of GUID for performance considerations. Differential revision: https://reviews.llvm.org/D70128
* Check result of emitStrLen before passing it to CreateGEPDimitry Andric2019-11-141-2/+2
| | | | | | | | | | | | | | | | | | Summary: This fixes PR43081, where the transformation of `strchr(p, 0) -> p + strlen(p)` can cause a segfault, if `-fno-builtin-strlen` is used. In that case, `emitStrLen` returns nullptr, which CreateGEP is not designed to handle. Also add the minimized code from the PR as a test case. Reviewers: xbolva00, spatel, jdoerfert, efriedma Reviewed By: efriedma Subscribers: lebedev.ri, hiraditya, cfe-commits, llvm-commits Tags: #clang, #llvm Differential Revision: https://reviews.llvm.org/D70143
* Sink all InitializePasses.h includesReid Kleckner2019-11-13147-41/+196
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This file lists every pass in LLVM, and is included by Pass.h, which is very popular. Every time we add, remove, or rename a pass in LLVM, it caused lots of recompilation. I found this fact by looking at this table, which is sorted by the number of times a file was changed over the last 100,000 git commits multiplied by the number of object files that depend on it in the current checkout: recompiles touches affected_files header 342380 95 3604 llvm/include/llvm/ADT/STLExtras.h 314730 234 1345 llvm/include/llvm/InitializePasses.h 307036 118 2602 llvm/include/llvm/ADT/APInt.h 213049 59 3611 llvm/include/llvm/Support/MathExtras.h 170422 47 3626 llvm/include/llvm/Support/Compiler.h 162225 45 3605 llvm/include/llvm/ADT/Optional.h 158319 63 2513 llvm/include/llvm/ADT/Triple.h 140322 39 3598 llvm/include/llvm/ADT/StringRef.h 137647 59 2333 llvm/include/llvm/Support/Error.h 131619 73 1803 llvm/include/llvm/Support/FileSystem.h Before this change, touching InitializePasses.h would cause 1345 files to recompile. After this change, touching it only causes 550 compiles in an incremental rebuild. Reviewers: bkramer, asbirlea, bollu, jdoerfert Differential Revision: https://reviews.llvm.org/D70211
* [PGO][PGSO] Temporarily disable the large working set size behavior.Hiroshi Yamauchi2019-11-131-0/+5
| | | | | | | | | | | | | | Summary: This temporarily disables the large working set size behavior in profile guided size optimization due to internal benchmark regressions. Reviewers: davidxl Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70207
* [SLP] fix miscompile on min/max reductions with extra uses (PR43948)Sanjay Patel2019-11-131-1/+14
| | | | | | | | | | | | | | | | | The bug manifests as replacing a reduction operand with an undef value. The problem appears to be limited to cases where a min/max reduction has extra uses of the compare operand to the select. In the general case, we are tracking "ExternallyUsedValues" and an "IgnoreList" of the reduction operations, but those may not apply to the final compare+select in a min/max reduction. For that, we use replaceAllUsesWith (RAUW) to ensure that the new vectorized reduction values are transferred to all subsequent users. Differential Revision: https://reviews.llvm.org/D70148
* [SLP] reduce code duplication for min/max vs. other reductions; NFCISanjay Patel2019-11-131-77/+31
|
* [InstCombine] propagate fast-math-flags (FMF) to select when inverting ↵Sanjay Patel2019-11-131-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | fcmp+select As noted by the FIXME comment, this is not correct based on our current FMF semantics. We should be propagating FMF from the final value in a sequence (in this case the 'select'). So the behavior even without this patch is wrong, but we did not allow FMF on 'select' until recently. But if we do the correct thing right now in this patch, we'll inevitably introduce regressions because we have not wired up FMF propagation for 'phi' and 'select' in other passes (like SimplifyCFG) or other places in InstCombine. I'm not seeing a better incremental way to make progress. That said, the potential extra damage over the existing wrong behavior from this patch is very limited. AFAIK, the only way to have different FMF on IR in the same function is if we have LTO inlined IR from 2 modules that were compiled using different fast-math settings. As seen in the tests, we may actually see some improvements with this patch because adding the FMF to the 'select' allows matching to min/max intrinsics that were previously missed (in the common case, the 'fcmp' and 'select' should have identical FMF to begin with). Next steps in the transition: Make similar changes in instcombine as needed. Enable phi-to-select FMF propagation in SimplifyCFG. Remove dependencies on fcmp with FMF. Deprecate FMF on fcmp. Differential Revision: https://reviews.llvm.org/D69720
* SLPVectorizer - make comparison operators + isInSchedulingRegion constSimon Pilgrim2019-11-131-3/+3
| | | | Fixes cppcheck warnings.
* [InstCombine] Avoid moving ops that do restrict undef across shuffles.Florian Hahn2019-11-131-1/+3
| | | | | | | | | | | | | | | | | | | I think we have to be a bit more careful when it comes to moving ops across shuffles, if the op does restrict undef. For example, without this patch, we would move 'and %v, <0, 0, -1, -1>' over a 'shufflevector %a, undef, <undef, undef, 1, 2>'. As a result, the first 2 lanes of the result are undef after the combine, but they really should be 0, unless I am missing something. For ops that do fold to undef on undef operands, the current behavior should be fine. I've add conservative check OpDoesRestrictUndef, maybe there's a better existing utility? Reviewers: spatel, RKSimon, lebedev.ri Reviewed By: spatel Differential Revision: https://reviews.llvm.org/D70093
* Temporarily revert "[InstCombine] Fold PHIs with equal incoming pointers"Daniil Suchkov2019-11-132-65/+0
| | | | | | Revert due to sanitizer-windows buildbot failure. This reverts commit bbb29738b58aaf6f6518269abdcf8f64131665a9.
* [InstCombine] Fold PHIs with equal incoming pointersDaniil Suchkov2019-11-132-0/+65
| | | | | | | | | | | | | | | | | | | In case when all incoming values of a PHI are equal pointers, this transformation inserts a definition of such a pointer right after definition of the base pointer and replaces with this value both PHI and all it's incoming pointers. Primary goal of this transformation is canonicalization of this pattern in order to enable optimizations that can't handle PHIs. Non-inbounds pointers aren't currently supported. Reviewers: spatel, RKSimon, lebedev.ri, apilipenko Reviewed By: apilipenko Tags: #llvm Subscribers: hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D68128
* [GVNHoist] Preserve AAResults.Alina Sbirlea2019-11-121-0/+1
| | | | Resolves PR38906, PR40898.
* [DBG][OPT] Attempt to salvage or undef debug info when removing trivially ↵Tom Weaver2019-11-121-0/+2
| | | | | | | | deletable instructions in the Reassociate Expression pass. Reviewed By: aprantl, vsk Differential revision: https://reviews.llvm.org/D69943
* [InstCombine] Skip scalable vectors in combineLoadToOperationTypeDiana Picus2019-11-121-0/+1
| | | | | | | | | | | | | | | | Don't try to canonicalize loads to scalable vector types to loads of integers. This removes one assertion when trying to use a TypeSize as a parameter to DataLayout::isLegalInteger. It does not handle the second part of the function (which looks at bitcasts). This patch also contains a NFC fix for Load Analysis, where a variable initialization that would cause the same assertion is moved closer to its use. This allows us to run the new test for InstCombine without having to teach LocationSize to play nicely with scalable vectors. Differential Revision: https://reviews.llvm.org/D70075
* [LoopInterchange] Only skip PHIs with incoming values from the inner loop.Florian Hahn2019-11-121-7/+7
| | | | | | | | | | | | | | Currently we have limited support for outer loops with multiple basic blocks after the inner loop exit. But the current checks for creating PHIs for loop exit values only assumes the header and latches of the outer loop. It is better to just skip incoming values defined in the original inner loops. Those are handled earlier. Reviewers: efriedma, mcrosier Reviewed By: efriedma Differential Revision: https://reviews.llvm.org/D70059
* [Attributor] Use must-be-executed-context in align deductionHideto Ueno2019-11-121-20/+59
| | | | | | | | | | | | | | | Summary: This patch introduces align attribute deduction for callsite argument, function argument, function returned and floating value based on must-be-executed-context. Reviewers: jdoerfert, sstefan1 Reviewed By: jdoerfert Subscribers: hiraditya, jfb, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69797
* [SLP] Look-ahead operand reordering heuristic.Vasileios Porpodas2019-11-111-46/+260
| | | | | | | | | | | | | | Summary: This patch introduces a new heuristic for guiding operand reordering. The new "look-ahead" heuristic can look beyond the immediate predecessors. This helps break ties when the immediate predecessors have identical opcodes (see lit test for examples). Reviewers: RKSimon, ABataev, dtemirbulatov, Ayal, hfinkel, rnk Reviewed By: RKSimon, dtemirbulatov Subscribers: xbolva00, Carrot, hiraditya, phosek, rnk, rcorcs, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60897
* [VFABI] Read/Write functions for the VFABI attribute.Francesco Petrogalli2019-11-121-0/+29
| | | | | | | | | | | | | | The attribute is stored at the `FunctionIndex` attribute set, with the name "vector-function-abi-variant". The get/set methods of the attribute have assertion to verify that: 1. Each name in the attribute is a valid VFABI mangled name. 2. Each name in the attribute correspond to a function declared in the module. Differential Revision: https://reviews.llvm.org/D69976
* Add InstCombine/InstructionSimplify support for Freeze Instructionaqjune2019-11-122-0/+10
| | | | | | | | | | | | | | | Summary: - Add llvm::SimplifyFreezeInst - Add InstCombiner::visitFreeze - Add llvm tests Reviewers: majnemer, sanjoy, reames, lebedev.ri, spatel Reviewed By: reames, lebedev.ri Subscribers: reames, lebedev.ri, filcab, regehr, trentxintong, llvm-commits Differential Revision: https://reviews.llvm.org/D29013
* Revert "[InstCombine] avoid crash from deleting an instruction that still ↵Sanjay Patel2019-11-111-18/+4
| | | | | | | | has uses (PR43723) (3rd try)" This reverts commit 3db8a3ef86e7b3331ab466a78c10a62be9e69829. This caused a different memory-sanitizer failure than earlier attempts, but it's still not right.
* [InstCombine] avoid crash from deleting an instruction that still has uses ↵Sanjay Patel2019-11-111-4/+18
| | | | | | | | | | | | | | | | | | | | | (PR43723) (3rd try) Re-try because earlier attempts were reverted due to use-after-free. Hopefully, diagnosed correctly this time - we replace/remove the invariant.start first rather than the invariant.end to avoid angering worklist-based iteration. We gather a set of white-listed instructions in isAllocSiteRemovable() and then replace/erase them. But we don't know in general if the instructions in the set have uses amongst themselves, so order of deletion makes a difference. There's already a special-case for the llvm.objectsize intrinsic, so add another for llvm.invariant.start. Should fix: https://bugs.llvm.org/show_bug.cgi?id=43723 Differential Revision: https://reviews.llvm.org/D69977
* Revert "[DBG][OPT] Attempt to salvage or undef debug info when removing ↵Tom Weaver2019-11-111-2506/+0
| | | | | | trivially deletable instructions in the Reassociate Expression pass." This reverts commit 1984a27db58e9053371ab6d6dc288c81c8a071ac.
* [DBG][OPT] Attempt to salvage or undef debug info when removing trivially ↵Tom Weaver2019-11-111-0/+2506
| | | | | | | | deletable instructions in the Reassociate Expression pass. Reviewed By: aprantl, vsk Differential revision: https://reviews.llvm.org/D69943
* [NFC][TEST_COMMIT] Add fullstop to comment.Tom Weaver2019-11-111-1/+1
|
* [InstCombine] Simplify binary op when only one operand is a selectJay Foad2019-11-111-0/+10
| | | | | | | | | | | | | | | | | | | | | | Summary: SimplifySelectsFeedingBinaryOp simplified binary ops when both operands were selects with the same condition. This patch extends it to handle these cases where only one operand is a select: X op (C ? P : Q) -> C ? (X op P) : (X op Q) // if X op P and X op Q both simplify (C ? P : Q) op Y -> C ? (P op Y) : (Q op Y) // if P op Y and Q op Y both simplify For example: X *fast (C ? 1.0 : 0.0) -> C ? X : 0.0 Reviewers: mcberg2017, majnemer, craig.topper, qcolombet, mcrosier Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D64713
* [InstCombine] Turn (extractelement <1 x i64/double> (bitcast (x86_mmx))) ↵Craig Topper2019-11-101-0/+7
| | | | | | | | | | | | | into a single bitcast from x86_mmx to i64/double. The _m64 type is represented in IR as <1 x i64>. The x86-64 ABI on Linux passes <1 x i64> as a double. MMX intrinsics use x86_mmx type in IR.These things result in a lot of bitcasts in mmx code. There's another instcombine that tries to turn bitcast <1 x i64> to double into extractelement and a bitcast. The combine here tries to reverse this extractelement conversion if we see an mmx type.
* Revert "[InstCombine] avoid crash from deleting an instruction that still ↵Sanjay Patel2019-11-101-20/+4
| | | | | | | has uses (PR43723) (2nd try)" This reverts commit 56b2aee1875a1ee47ddf859a6584f8728787fb7b. Still causes a use-after-free on sanitizer bots.
* [InstCombine] avoid crash from deleting an instruction that still has uses ↵Sanjay Patel2019-11-101-4/+20
| | | | | | | | | | | | | | | | | | | (PR43723) (2nd try) Re-try rGef02831f0a4e (reverted due to use-after-free), but bail out completely if we encounter an unexpected llvm.invariant.start. We gather a set of white-listed instructions in isAllocSiteRemovable() and then replace/erase them. But we don't know in general if the instructions in the set have uses amongst themselves, so order of deletion makes a difference. There's already a special-case for the llvm.objectsize intrinsic, so add another for llvm.invariant.end. Should fix: https://bugs.llvm.org/show_bug.cgi?id=43723 Differential Revision: https://reviews.llvm.org/D69977
* getArgOperandNo helper function.Stefan Stipanovic2019-11-101-2/+2
| | | | | | | | | | | | Summary: A helper function to get argument number of a arg operand Use. Reviewers: jdoerfert, uenoku Subscribers: hiraditya, lebedev.ri, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D66844
* Revert "[InstCombine] avoid crash from deleting an instruction that still ↵Sanjay Patel2019-11-101-17/+4
| | | | | | | has uses (PR43723)" This reverts commit ef02831f0a4e3b3ccaa45a5589e4cabecbf527ab. Sanitizer bots fail with this change.
* [InstCombine] avoid crash from deleting an instruction that still has uses ↵Sanjay Patel2019-11-101-4/+17
| | | | | | | | | | | | | | | | (PR43723) We gather a set of white-listed instructions in isAllocSiteRemovable() and then replace/erase them. But we don't know in general if the instructions in the set have uses amongst themselves, so order of deletion makes a difference. There's already a special-case for the llvm.objectsize intrinsic, so add another for llvm.invariant.end. Should fix: https://bugs.llvm.org/show_bug.cgi?id=43723 Differential Revision: https://reviews.llvm.org/D69977
* Remove unused variable (which allows us to remove vector include). NFC.Simon Pilgrim2019-11-101-3/+0
|
* [LV] Apply sink-after & interleave-groups as VPlan transformations (NFCI)Gil Rapaport2019-11-095-128/+175
| | | | | | | | | This recommits 11ed1c0239fd51fd2f064311dc7725277ed0a994 (reverted in 9f08ce0d2197d4f163dfa4633eae2347ce8fc881 for failing an assert) with a fix: tryToWidenMemory() now first checks if the widening decision is to interleave, thus maintaining previous behavior where tryToInterleaveMemory() was called first, giving priority to interleave decisions over widening/scalarization. This commit adds the test case that exposed this bug as a LIT.
* Refactor SimplifySelectsFeedingBinaryOp for D64713. NFC.Jay Foad2019-11-091-25/+32
|
* ThinLTO : Import always_inline functions irrespective of the thresholdTeresa Johnson2019-11-081-2/+4
| | | | | | | | | | | | | | | | Summary: A user can force a function to be inlined by specifying the always_inline attribute. Currently, thinlto implementation is not aware of always_inline functions and does not guarantee import of such functions, which in turn can prevent inlining of such functions. Patch by Bharathi Seshadri <bseshadr@cisco.com> Reviewers: tejohnson Reviewed By: tejohnson Subscribers: mehdi_amini, inglorion, hiraditya, steven_wu, dexonsmith, arphaman, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70014
* Revert "[LV] Apply sink-after & interleave-groups as VPlan transformations ↵Gil Rapaport2019-11-085-170/+125
| | | | | | (NFCI)" This reverts commit 11ed1c0239fd51fd2f064311dc7725277ed0a994 - causes an assert failure.
* [ThinLTO] Fix bug when importing writeonly variablesevgeny2019-11-082-7/+22
| | | | | | | | Patch enables import of write-only variables with non-trivial initializers to fix linker errors. Initializers of imported variables are converted to 'zeroinitializer' to avoid promotion of referenced objects. Differential revision: https://reviews.llvm.org/D70006
* [JumpThreading] Fix a comment typo (NFC)Kazu Hirata2019-11-081-1/+1
| | | | | | | | | | Reviewers: kazu Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70013
* [LICM] Support hosting of dynamic allocas out of loopsPhilip Reames2019-11-081-0/+45
| | | | | | | | | | | | This patch implements a correct, but not terribly useful, transform. In particular, if we have a dynamic alloca in a loop which is guaranteed to execute, and provably not captured, we hoist the alloca out of the loop. The capture tracking is needed so that we can prove that each previous stack region dies before the next one is allocated. The transform decreases the amount of stack allocation needed by a linear factor (e.g. the iteration count of the loop). Now, I really hope no one is actually using dynamic allocas. As such, why this patch? Well, the actual problem I'm hoping to make progress on is allocation hoisting. There's a large draft patch out for review (https://reviews.llvm.org/D60056), and this patch was the smallest chunk of testable functionality I could come up with which takes a step vaguely in that direction. Once this is in, it makes motivating the changes to capture tracking mentioned in TODOs testable. After that, I hope to extend this to trivial malloc free regions (i.e. free dominating all loop exits) and allocation functions for GCed languages. Differential Revision: https://reviews.llvm.org/D69227
* [LICM] Hoisting of widenable conditions out of loopsPhilip Reames2019-11-081-0/+4
| | | | | | The change itself is straight forward and obvious, but ... there's an existing test checking for exactly the opposite. Both I and Artur think this is simply conservatism in the initial implementation. If anyone bisects a problem to this, a counter example will be very interesting. Differential Revision: https://reviews.llvm.org/D69907
* [LV] Apply sink-after & interleave-groups as VPlan transformations (NFCI)Gil Rapaport2019-11-085-125/+170
| | | | | | | | This recommits 100e797adb433724a17c9b42b6533cd634cb796b (reverted in 009e032634b3bd7fc32071ac2344b12142286477 for failing an assert). While the root cause was independently reverted in eaff3004019f97c64c88ab76da6b25106b659b30, this commit includes a LIT to make sure IVDescriptor's SinkAfter logic does not try to sink branch instructions.
* [NFC][IndVarS] Adjust a commentDaniil Suchkov2019-11-081-1/+1
| | | | (test commit)
* [InstCombine] Don't transform bitcasts between x86_mmx and v1i64 into ↵Craig Topper2019-11-071-2/+3
| | | | | | | | | | insertelement/extractelement x86_mmx is conceptually a vector already. Don't introduce an extra conversion between it and scalar i64. I'm using VectorType::isValidElementType which checks for floating point, integer, and pointers to hopefully make this more readable than just blacklisting x86_mmx. Differential Revision: https://reviews.llvm.org/D69964
OpenPOWER on IntegriCloud