summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
* NewGVN: Remove DebugUnknownExprs, just mark the instructions as unusedDaniel Berlin2017-03-061-7/+3
| | | | llvm-svn: 297047
* NewGVN: Only call isInstructionTrivially dead once per instruction.Daniel Berlin2017-03-061-9/+10
| | | | llvm-svn: 297046
* Remove the sample pgo annotation heuristic that uses call count to annotate ↵Dehao Chen2017-03-061-5/+3
| | | | | | | | | | | | | | | | basic block count. Summary: We do not need that special handling because the debug info is more accurate now. Performance testing shows no regression on google internal benchmarks. Reviewers: davidxl, aprantl Reviewed By: aprantl Subscribers: llvm-commits, aprantl Differential Revision: https://reviews.llvm.org/D30658 llvm-svn: 297038
* [BasicBlockUtils] Check for nullptr before updating LoopInfo.Michael Kruse2017-03-061-3/+4
| | | | | | | | | | | | | | | | LoopInfo::getLoopFor returns nullptr if a BB is not in a loop and only then can the loop be updated to contain the newly created BBs. Add the missing nullptr check to SplitBlockAndInsertIfThen. Within LLVM, the only user of this function that also passes a LoopInfo to be updated is InnerLoopVectorizer::predicateInstructions(). As the method's name implies, the BB operataten on will always be within a loop, but out-of-tree users may also use it differently (here: Polly). All other uses of LoopInfo::getLoopFor in the file properly check its return value for nullptr. llvm-svn: 297016
* [SimplifyCFG] Use APInt::operator| instead of APInt::Or. NFCCraig Topper2017-03-051-1/+1
| | | | | | I'm looking to improve operator| to support rvalue references and may remove APInt::Or. llvm-svn: 296982
* Set option enabling LSR alternative way to resolve complex solution to false.Evgeny Stupachenko2017-03-041-1/+1
| | | | | | | Differential Revision: http://reviews.llvm.org/D29862 From: Evgeny Stupachenko <evstupac@gmail.com> llvm-svn: 296959
* Fix build.Peter Collingbourne2017-03-041-1/+1
| | | | llvm-svn: 296949
* WholeProgramDevirt: Implement exporting for uniform ret val opt.Peter Collingbourne2017-03-041-6/+19
| | | | | | Differential Revision: https://reviews.llvm.org/D29846 llvm-svn: 296948
* WholeProgramDevirt: Implement exporting for single-impl devirtualization.Peter Collingbourne2017-03-041-6/+54
| | | | | | Differential Revision: https://reviews.llvm.org/D29811 llvm-svn: 296945
* WholeProgramDevirt: Add any unsuccessful llvm.type.checked.load ↵Peter Collingbourne2017-03-041-12/+88
| | | | | | | | | | | | | devirtualizations to the list of llvm.type.test users. Any unsuccessful llvm.type.checked.load devirtualizations will be translated into uses of llvm.type.test, so we need to add the resulting llvm.type.test intrinsics to the function summaries so that the LowerTypeTests pass will export them. Differential Revision: https://reviews.llvm.org/D29808 llvm-svn: 296939
* NewGVN: Be consistent in what order we compare operands for swapping.Daniel Berlin2017-03-041-2/+2
| | | | | | NFC. llvm-svn: 296935
* Fix a compiler warningSanjoy Das2017-03-031-1/+2
| | | | llvm-svn: 296903
* [LoopUnrolling] Peel loops with invariant backedge Phi inputSanjoy Das2017-03-031-0/+25
| | | | | | | | | | | | | | | | | | | | | Summary: If a loop contains a Phi node which has an invariant input from back edge, it is profitable to peel such loops (rather than unroll them) to use the advantage that this Phi is always invariant starting from 2nd iteration. After the 1st iteration is peeled, other optimizations can potentially simplify calculations with this invariant. Patch by Max Kazantsev! Reviewers: sanjoy, apilipenko, igor-laevsky, anna, mkuper, reames Reviewed By: mkuper Subscribers: mkuper, mzolotukhin, llvm-commits Differential Revision: https://reviews.llvm.org/D30161 llvm-svn: 296898
* [LoopUnrolling] Re-prioritize Peeling and Partial unrollingSanjoy Das2017-03-032-10/+16
| | | | | | | | | | | | | | | | | | | | | | | Summary: In current implementation the loop peeling happens after trip-count based partial unrolling and may sometimes not happen at all due to it (for example, if trip count is known, but UP.Partial = false). This is generally bad, the more than there are some situations where peeling is profitable even if the partial unrolling is disabled. This patch is a NFC which reorders peeling and partial unrolling application and prepares the code for implementation of the said optimizations. Patch by Max Kazantsev! Reviewers: sanjoy, anna, reames, apilipenko, igor-laevsky, mkuper Reviewed By: mkuper Subscribers: mkuper, llvm-commits, mzolotukhin Differential Revision: https://reviews.llvm.org/D30243 llvm-svn: 296897
* Use APInt::getLowBitsSet instead of APInt::getBitsSet for lower bit mask ↵Simon Pilgrim2017-03-031-1/+1
| | | | | | creation llvm-svn: 296882
* Revert "Re-apply "[GVNHoist] Move GVNHoist to function simplification part ↵Benjamin Kramer2017-03-031-2/+2
| | | | | | | | of pipeline."" This reverts commit r296759. Miscompiles bash. llvm-svn: 296872
* [SLP] Fixes the bug due to absence of in order uses of scalars which needs ↵Mohammad Shahid2017-03-031-72/+79
| | | | | | | | | | | | | | | | to be available for VectorizeTree() API.This API uses it for proper mask computation to be used in shufflevector IR. The fix is to compute the mask for out of order memory accesses while building the vectorizable tree instead of actual vectorization of vectorizable tree.It also needs to recompute the proper Lane for external use of vectorizable scalars based on shuffle mask. Reviewers: mkuper Differential Revision: https://reviews.llvm.org/D30159 Change-Id: Ide8773ce0ad3562f3cf4d1a0ad0f487e2f60ce5d llvm-svn: 296863
* [msan] Handle x86_sse_stmxcsr and x86_sse_ldmxcsr.Evgeniy Stepanov2017-03-031-4/+46
| | | | llvm-svn: 296848
* [msan] Remove stale comments.Evgeniy Stepanov2017-03-031-2/+0
| | | | | | ClStoreCleanOrigin flag was removed back in 2014. llvm-svn: 296844
* ThinLTOBitcodeWriter: Do not follow operand edges of type GlobalValue when ↵Peter Collingbourne2017-03-021-0/+2
| | | | | | | | | | | | looking for virtual functions. Such edges may otherwise result in infinite recursion if a pointer to a vtable is reachable from the vtable itself. This can happen in practice if a TU defines the ABI types used to implement RTTI, and is itself compiled with RTTI. Fixes PR32121. llvm-svn: 296839
* Move defClobbersUseOrDef to being a protected member of a class since we ↵Daniel Berlin2017-03-022-4/+4
| | | | | | don't want anyone else using it llvm-svn: 296838
* [BypassSlowDivision] Use ValueTracking to simplify run-time checksNikolai Bozhenov2017-03-021-29/+108
| | | | | | | | | | | | | | | | | | | | | | | | ValueTracking is used for more thorough analysis of operands. Based on the analysis, either run-time checks can be simplified (e.g. check only one operand instead of two) or the transformation can be avoided. For example, it is quite often the case that a divisor is promoted from a shorter type and run-time checks for it are redundant. With additional compile-time analysis of values, two special cases naturally arise and are addressed by the patch: 1) Both operands are known to be short enough. Then, the long division can be simply replaced with a short one without CFG modification. 2) If a division is unsigned and the dividend is known to be short then the long division is not needed at all. Because if the divisor is too big for short division then the quotient is obviously zero (and the remainder is equal to the dividend). Actually, the division is not needed when (divisor > dividend). Differential Revision: https://reviews.llvm.org/D29897 llvm-svn: 296832
* [BypassSlowDivision] Refactor fast division insertion logic (NFC)Nikolai Bozhenov2017-03-021-160/+218
| | | | | | | | | | The most important goal of the patch is to break large insertFastDiv function into separate pieces, so that later a different fast insertion logic can be implemented using some of these pieces. Differential Revision: https://reviews.llvm.org/D29896 llvm-svn: 296828
* Revert "Fix PR 24415 (at least), by making our post-dominator tree behavior ↵Tobias Grosser2017-03-021-14/+37
| | | | | | | | | | | | | | | sane." and also "clang-format GenericDomTreeConstruction.h, since the current formatting makes it look like their is a bug in the loop indentation, and there is not" This reverts commit r296535. There are still some open design questions which I would like to discuss. I revert this for Daniel (who gave the OK), as he is on vacation. llvm-svn: 296812
* The patch turns on epilogue unroll for loops with constant recurency start.Evgeny Stupachenko2017-03-021-1/+44
| | | | | | | | | | | | | | | | | Summary: Set unroll remainder to epilog if a loop contains a phi with constant parameter: loop: pn = phi [Const, PreHeader], [pn.next, Latch] ... Reviewer: hfinkel Differential Revision: http://reviews.llvm.org/D27004 From: Evgeny Stupachenko <evstupac@gmail.com> llvm-svn: 296770
* Re-apply "[GVNHoist] Move GVNHoist to function simplification part of pipeline."Geoff Berry2017-03-021-2/+2
| | | | | | | | | This re-applies r289696, which caused TSan perf regression, which has since been addressed in separate changes (see PR for details). See PR31382. llvm-svn: 296759
* [InstCombine] Avoid faulty combines of select-cmp-brBjorn Pettersson2017-03-021-3/+5
| | | | | | | | | | | | | | | | | | | | | | Summary: When InstCombine is optimizing certain select-cmp-br patterns it replaces the result of the select in uses outside of the basic block containing the select. This is only legal if the path from the select to the outside use is disjoint from all other paths out from the originating basic block. The problem found was that InstCombiner::replacedSelectWithOperand did not consider the case when both edges out from the br pointed to the same label. In that case the paths aren't disjoint and the transformation is illegal. This patch avoids the faulty rewrites by verifying that there is a single flow to the successor where we want to replace uses. Reviewers: llvm-commits, spatel, majnemer Differential Revision: https://reviews.llvm.org/D30455 llvm-svn: 296752
* [LV] Considier non-consecutive but vectorizable accesses for VF selectionMatthew Simpson2017-03-021-3/+10
| | | | | | | | | | | | | | | | When computing the smallest and largest types for selecting the maximum vectorization factor, we currently ignore loads and stores of pointer types if the memory access is non-consecutive. We do this because such accesses must be scalarized regardless of vectorization factor, and thus shouldn't be considered when determining the factor. This patch makes this check less aggressive by also considering non-consecutive accesses that may be vectorized, such as interleaved accesses. Because we don't know at the time of the check if an accesses will certainly be vectorized (this is a cost model decision given a particular VF), we consider all accesses that can potentially be vectorized. Differential Revision: https://reviews.llvm.org/D30305 llvm-svn: 296747
* Fix typo. NFCIXin Tong2017-03-021-1/+1
| | | | llvm-svn: 296735
* [Constant Hoisting] Avoid inserting instructions before EH padsReid Kleckner2017-03-011-2/+10
| | | | | | | | | | | | | Now that terminators can be EH pads, this code needs to iterate over the immediate dominators of the EH pad to find a valid insertion point. Fix for PR32107 Patch by Robert Olliff! Differential Revision: https://reviews.llvm.org/D30511 llvm-svn: 296698
* NewGVN: Add debug counter for value numberingDaniel Berlin2017-03-011-5/+15
| | | | llvm-svn: 296665
* Revert r296575 "[SLP] Fixes the bug due to absence of in order uses of ↵Hans Wennborg2017-03-011-65/+70
| | | | | | | | scalars which needs to be available" It caused miscompiles, e.g. in Chromium (PR32109). llvm-svn: 296654
* [GVNHoist] Don't hoist unsafe scalars at -Oz (PR31729)Hans Wennborg2017-03-011-22/+8
| | | | | | | | Based on Aditya Kumar's patch: Differential Revision: https://reviews.llvm.org/D29092 llvm-svn: 296642
* [DeadStoreElimination] Check function modref behavior before considering ↵Igor Laevsky2017-03-011-1/+1
| | | | | | | | memory clobbered Differential Revision: https://reviews.llvm.org/D29996 llvm-svn: 296625
* [SLP] Preserve IR flags when vectorizing horizontal reductions.Alexey Bataev2017-03-011-2/+5
| | | | | | | | | | | | | | | | | | Summary: The SLP vectorizer should propagate IR-level optimization hints/flags (nsw, nuw, exact, fast-math) when converting scalar horizontal reductions instructions into vectors, just like for other vectorized instructions. It doe not include IR propagation for extra arguments, we need to handle original scalar operations for extra args to propagate correct flags. Reviewers: mkuper, mzolotukhin, hfinkel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D30418 llvm-svn: 296614
* [SLP] Preserve IR flags for extra args.Alexey Bataev2017-03-011-9/+11
| | | | | | | | | | | | | | | Summary: We should preserve IR flags for extra args. These IR flags should be taken from original scalar operations, not from the reduction operations. Reviewers: mkuper, mzolotukhin, hfinkel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D30447 llvm-svn: 296613
* [SLP] Fix for PR32038: extra add of PHI node when it is not required.Alexey Bataev2017-03-011-17/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: If horizontal reduction tree starts from the binary operation that is used in PHI node, but this PHI is not used in horizontal reduction, we may end up with extra addition of this PHI node after vectorization. Here is an example: ``` %phi = phi i32 [ %tmp, %end], ... ... %tmp = add i32 %tmp1, %tmp2 end: ``` after vectorization we always have something like: ``` %phi = phi i32 [ %tmp, %end], ... ... %red = extractelement <8 x 32> %vec.red, 0 %tmp = add i32 %red, %phi end: ``` even if `%phi` is not used in reduction tree. Patch considers these PHI nodes as extra arguments and considers them in the final result iff they really used in reduction. Reviewers: mkuper, hfinkel, mzolotukhin Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D30409 llvm-svn: 296606
* Remove sometimes faulty rewrite of memcpy in instcombine.Mikael Holmen2017-03-011-55/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Solves PR 31990. The bad rewrite could replace a memcpy of one word with store i4 -1 while it should actually be store i8 -1 Hopefully opt and llc has improved enough so the original optimization done by the code isn't needed anymore. One already existing testcase is affected. It originally tested that the memcpy was replaced with load double but since we now remove that rewrite it will be load i64 instead. Patch suggestion by Eli Friedman. Reviewers: eli.friedman, majnemer, efriedma Reviewed By: efriedma Subscribers: efriedma, llvm-commits Differential Revision: https://reviews.llvm.org/D30254 llvm-svn: 296585
* [LV] These remark should have been missed remarksAdam Nemet2017-03-011-2/+2
| | | | | | | | The practice in LV is that we emit analysis remarks and then finally report either a missed or applied remark on the final decision whether vectorization is taking place. On this code path, we were closing with an analysis remark. llvm-svn: 296578
* [SLP] Fixes the bug due to absence of in order uses of scalars which needs ↵Mohammad Shahid2017-03-011-70/+65
| | | | | | | | | | | | | | | to be available for VectorizeTree() API.This API uses it for proper mask computation to be used in shufflevector IR. The fix is to compute the mask for out of order memory accesses while building the vectorizable tree instead of actual vectorization of vectorizable tree. Reviewers: mkuper Differential Revision: https://reviews.llvm.org/D30159 Change-Id: Id1e287f073fa4959713ba545fa4254db5da8b40d llvm-svn: 296575
* Revert "(HEAD, origin/master, origin/HEAD, master) [LV] These should missed ↵Adam Nemet2017-02-281-2/+2
| | | | | | | | | | remarks" This reverts commit r296544. This got committed by accident. llvm-svn: 296546
* [LV] These should missed remarksAdam Nemet2017-02-281-2/+2
| | | | llvm-svn: 296544
* Fix PR 24415 (at least), by making our post-dominator tree behavior sane.Daniel Berlin2017-02-281-37/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Currently, our post-dom tree tries to ignore and remove the effects of infinite loops. It fails miserably at this, because it tries to do it ahead of time, and thus can only detect self-loops, and any other type of infinite loop, it pretends doesn't exist at all. This can, in a bunch of cases, lead to wrong answers and a completely empty post-dom tree. Wrong answer: ``` declare void foo() define internal void @f() { entry: br i1 undef, label %bb35, label %bb3.i bb3.i: call void @foo() br label %bb3.i bb35.loopexit3: br label %bb35 bb35: ret void } ``` We get: ``` Inorder PostDominator Tree: [1] <<exit node>> {0,7} [2] %bb35 {1,6} [3] %bb35.loopexit3 {2,3} [3] %entry {4,5} ``` This is a trivial modification of the testcase for PR 6047 Note that we pretend bb3.i doesn't exist. We also pretend that bb35 post-dominates entry. While it's true that it does not exit in a theoretical sense, it's not really helpful to try to ignore the effect and pretend that bb35 post-dominates entry. Worse, we pretend the infinite loop does nothing (it's usually considered a side-effect), and doesn't even exist, even when it calls a function. Sadly, this makes it impossible to use when you are trying to move code safely. All compilers also create virtual or real single exit nodes (including us), and connect infinite loops there (which this patch does). In fact, others have worked around our behavior here, to the point of building their own post-dom trees: https://zneak.github.io/fcd/2016/02/17/structuring.html and pointing out the region infrastructure is near-useless for them with postdom in this state :( Completely empty post-dom tree: ``` define void @spam() #0 { bb: br label %bb1 bb1: ; preds = %bb1, %bb br label %bb1 bb2: ; No predecessors! ret void } ``` Printing analysis 'Post-Dominator Tree Construction' for function 'foo': =============================-------------------------------- Inorder PostDominator Tree: [1] <<exit node>> {0,1} :( (note that even if you ignore the effects of infinite loops, bb2 should be present as an exit node that post-dominates nothing). This patch changes post-dom to properly handle infinite loops and does root finding during calculation to prevent empty tress in such cases. We match gcc's (and the canonical theoretical) behavior for infinite loops (find the backedge, connect it to the exit block). Testcases coming as soon as i finish running this on a ton of random graphs :) Reviewers: chandlerc, davide Subscribers: bryant, llvm-commits Differential Revision: https://reviews.llvm.org/D29705 llvm-svn: 296535
* Add function importing info from samplepgo profile to the module summary.Dehao Chen2017-02-281-8/+19
| | | | | | | | | | | | | | Summary: For SamplePGO, the profile may contain cross-module inline stacks. As we need to make sure the profile annotation happens when all the hot inline stacks are expanded, we need to pass this info to the module importer so that it can import proper functions if necessary. This patch implemented this feature by emitting cross-module targets as part of function entry metadata. In the module-summary phase, the metadata is used to build call edges that points to functions need to be imported. Reviewers: mehdi_amini, tejohnson Reviewed By: tejohnson Subscribers: davidxl, llvm-commits Differential Revision: https://reviews.llvm.org/D30053 llvm-svn: 296498
* Strip debug info when inlining into a nodebug function.Adrian Prantl2017-02-281-12/+30
| | | | | | | | | | | | | The LLVM backend cannot produce any debug info for an llvm::Function without a DISubprogram attachment. When inlining a debug-info-carrying function into a nodebug function, there is therefore no reason to keep any debug info intrinsic calls or debug locations on the instructions. This fixes a problem discovered in PR32042. rdar://problem/30679307 llvm-svn: 296488
* Empty line. NFCIXin Tong2017-02-281-1/+0
| | | | llvm-svn: 296438
* [LoopUnswitch] Common pushing LIC's user to worklist.Xin Tong2017-02-281-4/+2
| | | | llvm-svn: 296432
* AMDGPU: Basic folds for fmed3 intrinsicMatt Arsenault2017-02-271-0/+76
| | | | | | | Constant fold, canonicalize constants to RHS, reduce to minnum/maxnum when inputs are nan/undef. llvm-svn: 296409
* [AddressSanitizer] Put shadow at 0 for FuchsiaPetr Hosek2017-02-271-1/+6
| | | | | | | | | | The Fuchsia ASan runtime reserves the low part of the address space. Patch by Roland McGrath Differential Revision: https://reviews.llvm.org/D30426 llvm-svn: 296405
* Revert r296366 "[InlineFunction] add nonnull assumptions based on argument ↵Hans Wennborg2017-02-271-36/+22
| | | | | | | | attributes" It causes miscompiles e.g. during self-host of Clang (PR32082). llvm-svn: 296398
OpenPOWER on IntegriCloud