summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
...
* [SLP] Fix regression in broadcasts caused by operand reordering patch D59973.Dinar Temirbulatov2019-06-051-5/+35
| | | | | | | | | | | | This patch fixes a regression caused by the operand reordering refactoring patch https://reviews.llvm.org/D59973 . The fix changes the strategy to Splat instead of Opcode, if broadcast opportunities are found. Please see the lit test for some examples. Committed on behalf of @vporpo (Vasileios Porpodas) Differential Revision: https://reviews.llvm.org/D62427 llvm-svn: 362613
* [LoopUtils][SLPVectorizer] clean up management of fast-math-flagsSanjay Patel2019-06-052-34/+30
| | | | | | | | | | | | | | | | Instead of passing around fast-math-flags as a parameter, we can set those using an IRBuilder guard object. This is no-functional-change-intended. The motivation is to eventually fix the vectorizers to use and set the correct fast-math-flags for reductions. Examples of that not behaving as expected are: https://bugs.llvm.org/show_bug.cgi?id=23116 (should be able to reduce with less than 'fast') https://bugs.llvm.org/show_bug.cgi?id=35538 (possible miscompile for -0.0) D61802 (should be able to reduce with IR-level FMF) Differential Revision: https://reviews.llvm.org/D62272 llvm-svn: 362612
* [IPO] Disabled 'default only' switch statements to fix MSVC warnings.Simon Pilgrim2019-06-051-8/+8
| | | | | | @jdoerfert Looks like these are placeholders for incoming abstract attributes patches so I've just commented the code out, even though this is usually frowned upon. llvm-svn: 362592
* Resubmit "[CorrelatedValuePropagation] Fix prof branch_weights metadata ↵Yevgeny Rouban2019-06-051-56/+61
| | | | | | | | | | | | | | | | | handling for SwitchInst" This reverts commit 5b32f60ec31ce136edac6f693538aeb6039f4ad0. The fix is in commit 4f9e68148bd0dada2d6997625432385918ac2e2c. This patch fixes the CorrelatedValuePropagation pass to keep prof branch_weights metadata of SwitchInst consistent. It makes use of SwitchInstProfUpdateWrapper. New tests are added. Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D62126 llvm-svn: 362583
* Suppress false-positive GCC -Wreturn-type warning.Michael Liao2019-06-051-0/+1
| | | | llvm-svn: 362582
* [Attributor] Pass infrastructure and fixpoint frameworkJohannes Doerfert2019-06-054-1/+541
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | NOTE: Note that no attributes are derived yet. This patch will not go in alone but only with others that derive attributes. The framework is split for review purposes. This commit introduces the Attributor pass infrastructure and fixpoint iteration framework. Further patches will introduce abstract attributes into this framework. In a nutshell, the Attributor will update instances of abstract arguments until a fixpoint, or a "timeout", is reached. Communication between the Attributor and the abstract attributes that are derived is restricted to the AbstractState and AbstractAttribute interfaces. Please see the file comment in Attributor.h for detailed information including design decisions and typical use case. Also consider the class documentation for Attributor, AbstractState, and AbstractAttribute. Reviewers: chandlerc, homerdin, hfinkel, fedor.sergeev, sanjoy, spatel, nlopes, nicholas, reames Subscribers: mehdi_amini, mgorny, hiraditya, bollu, steven_wu, dexonsmith, dang, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D59918 llvm-svn: 362578
* [Scalarizer] Add UnaryOperator visitor to scalarization passCameron McInally2019-06-041-0/+38
| | | | | | Differential Revision: https://reviews.llvm.org/D62858 llvm-svn: 362558
* [Utils] Clean another duplicated util method.Alina Sbirlea2019-06-043-62/+13
| | | | | | | | | | | | | | | | | Summary: Following the cleanup in D48202, method foldBlockIntoPredecessor has the same behavior. Replace its uses with MergeBlockIntoPredecessor. Remove foldBlockIntoPredecessor. Reviewers: chandlerc, dmgreen Subscribers: jlebar, javed.absar, zzheng, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62751 llvm-svn: 362538
* [SCCP] Add UnaryOperator visitor to SCCP for unary FNegCameron McInally2019-06-031-0/+26
| | | | | | Differential Revision: https://reviews.llvm.org/D62819 llvm-svn: 362449
* Fix a crash when the default of a switch is removedAndrew Kaylor2019-06-031-0/+5
| | | | | | | | This patch fixes a problem that occurs in LowerSwitch when a switch statement has a PHI node as its condition, and the PHI node only has two incoming blocks, and one of those incoming blocks is through an unreachable default in the switch statement. When this condition occurs, LowerSwitch holds a pointer to the condition value, but removes the switch block as a predecessor of the PHI block, causing the PHI node to be replaced. LowerSwitch then tries to use its stale pointer to the original condition value, causing a crash. Differential Revision: https://reviews.llvm.org/D62560 llvm-svn: 362427
* [LoopPred] Convert a second member function to a static helper [NFC]Philip Reames2019-06-031-14/+15
| | | | | | (And remember to actually mark the first one static.) llvm-svn: 362415
* [LoopPred] Convert member function to free helper function [NFC]Philip Reames2019-06-031-43/+47
| | | | llvm-svn: 362411
* [SimplifyIndVar] Refactor overflow check elimination code; NFCNikita Popov2019-06-011-97/+43
| | | | | | | | | | | Extract a willNotOverflow() helper function that is shared between eliminateOverflowIntrinsic() and strengthenOverflowingOperation(). Use WithOverflowInst for the former. We'll be able to reuse the same code for saturating intrinsics as well. llvm-svn: 362305
* [IndVarSimplify] Fixup nowrap flags during LFTR (PR31181)Nikita Popov2019-06-011-0/+21
| | | | | | | | | | | | | | | | | | | | | | | | Fix for https://bugs.llvm.org/show_bug.cgi?id=31181 and partial fix for LFTR poison handling issues in general. When LFTR moves a condition from pre-inc to post-inc, it may now depend on value that is poison due to nowrap flags. To avoid this, we clear any nowrap flag that SCEV cannot prove for the post-inc addrec. Additionally, LFTR may switch to a different IV that is dynamically dead and as such may be arbitrarily poison. This patch will correct nowrap flags in some but not all cases where this happens. This is related to the adoption of IR nowrap flags for the pre-inc addrec. (See some of the switch_to_different_iv tests, where flags are not dropped or insufficiently dropped.) Finally, there are likely similar issues with the handling of GEP inbounds, but we don't have a test case for this yet. Differential Revision: https://reviews.llvm.org/D60935 llvm-svn: 362292
* Inline variable into assert to fix unused variable warning.Richard Trieu2019-06-011-3/+3
| | | | llvm-svn: 362285
* [LoopPred] Eliminate a redundant/confusing cover function [NFC]Philip Reames2019-06-011-19/+20
| | | | llvm-svn: 362284
* [LoopPred] Handle a subset of NE comparison based latchesPhilip Reames2019-06-011-21/+33
| | | | | | | | | | | | | | | | | | | | | At the moment, LoopPredication completely bails out if it sees a latch of the form: %cmp = icmp ne %iv, %N br i1 %cmp, label %loop, label %exit OR %cmp = icmp ne %iv.next, %NPlus1 br i1 %cmp, label %loop, label %exit This is unfortunate since this is exactly the form that LFTR likes to produce. So, go ahead and recognize simple cases where we can. For pre-increment loops, we leverage the fact that LFTR likes canonical counters (i.e. those starting at zero) and a (presumed) range fact on RHS to discharge the check trivially. For post-increment forms, the key insight is in remembering that LFTR had to insert a (N+1) for the RHS. CVP can hopefully prove that add nsw/nuw (if there's appropriate range on N to start with). This leaves us both with the post-inc IV and the RHS involving an nsw/nuw add, and SCEV can discharge that with no problem. This does still need to be extended to handle non-one steps, or other harder patterns of variable (but range restricted) starting values. That'll come later. Differential Revision: https://reviews.llvm.org/D62748 llvm-svn: 362282
* [SimplifyLibCalls] Fold more fortified functions into non-fortified variantsErik Pilkington2019-05-312-15/+203
| | | | | | | | | | | | When the object size argument is -1, no checking can be done, so calling the _chk variant is unnecessary. We already did this for a bunch of these functions. rdar://50797197 Differential revision: https://reviews.llvm.org/D62358 llvm-svn: 362272
* NFC: Pull out a function to reduce some duplicationErik Pilkington2019-05-312-119/+70
| | | | | | Part of https://reviews.llvm.org/D62358 llvm-svn: 362271
* Reapply [CVP] Simplify non-overflowing saturating add/subNikita Popov2019-05-311-1/+24
| | | | | | | | | | | | | If we can determine that a saturating add/sub will not overflow based on range analysis, convert it into a simple binary operation. This is a sibling transform to the existing with.overflow handling. Reapplying this with an additional check that the saturating intrinsic has integer type, as LVI currently does not support vector types. Differential Revision: https://reviews.llvm.org/D62703 llvm-svn: 362263
* [CVP] Fix assertion failure on vector with.overflowNikita Popov2019-05-311-1/+1
| | | | | | | | Noticed on D62703. LVI only handles plain integers, not vectors of integers. This was previously not an issue, because vector support for with.overflow is only a relatively recent addition. llvm-svn: 362261
* Revert "[CVP] Simplify non-overflowing saturating add/sub"Nikita Popov2019-05-311-24/+1
| | | | | | | | This reverts commit 1e692d1777ae34dcb93524b5798651a29defae09. Causes assertion failure in builtins-wasm.c clang test. llvm-svn: 362254
* [CVP] Simplify non-overflowing saturating add/subNikita Popov2019-05-311-1/+24
| | | | | | | | | | If we can determine that a saturating add/sub will not overflow based on range analysis, convert it into a simple binary operation. This is a sibling transform to the existing with.overflow handling. Differential Revision: https://reviews.llvm.org/D62703 llvm-svn: 362242
* [InstCombine] 'C-(C2-X) --> X+(C-C2)' constant-foldRoman Lebedev2019-05-311-1/+6
| | | | | | | | | | It looks this fold was already partially happening, indirectly via some other folds, but with one-use limitation. No other fold here has that restriction. https://rise4fun.com/Alive/ftR llvm-svn: 362217
* [InstCombine] 'add (sub C1, X), C2 --> sub (add C1, C2), X' constant-foldRoman Lebedev2019-05-311-1/+8
| | | | | | https://rise4fun.com/Alive/qJQ llvm-svn: 362216
* [CVP] Generalize willNotOverflow(); NFCNikita Popov2019-05-301-5/+5
| | | | | | | Change argument from WithOverflowInst to BinaryOpIntrinsic, so this function can also be used for saturating math intrinsics. llvm-svn: 362152
* [InstCombine] Avoid use after free in DenseMap, when built with GCCMartin Storsjo2019-05-301-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, this used a statement like this: Map[A] = Map[B]; This is equivalent to the following: const auto &Src = Map[B]; auto &Dest = Map[A]; Dest = Src; The second statement, "auto &Dest = Map[A];" can insert a new element into the DenseMap, which can potentially grow and reallocate the DenseMap's internal storage, which will invalidate the existing reference to the source. When doing the actual assignment, the Src reference is dereferenced, accessing memory that was freed when the DenseMap grew. This issue hasn't shown up when LLVM was built with Clang, because the right hand side ended up dereferenced before evaulating the left hand side. (If the value type is a larger data type, Clang doesn't do this but behaves like GCC.) With GCC, a cast to Value* isn't enough to make it dereference the right hand side reference before invoking operator[] (while that is enough to make Clang/LLVM do the right thing for larger types), but storing it in an intermediate variable in a separate statement works. This fixes PR42065. Differential Revision: https://reviews.llvm.org/D62624 llvm-svn: 362150
* Reapply: IR: add optional type to 'byval' function parametersTim Northover2019-05-301-0/+15
| | | | | | | | | | | | | | | | | When we switch to opaque pointer types we will need some way to describe how many bytes a 'byval' parameter should occupy on the stack. This adds a (for now) optional extra type parameter. If present, the type must match the pointee type of the argument. The original commit did not remap byval types when linking modules, which broke LTO. This version fixes that. Note to front-end maintainers: if this causes test failures, it's probably because the "byval" attribute is printed after attributes without any parameter after this change. llvm-svn: 362128
* [LV] Remove the redundant using LoopVectorizationPlanner:VPlanPtrFlorian Hahn2019-05-302-7/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | VPlan.h already contains the declaration of VPlanPtr type alias: using VPlanPtr = std::unique_ptr<VPlan>; The LoopVectorizationPlanner class also contains the same declaration of VPlanPtr and therefore LoopVectorize requires a long wording when its methods return VPlanPtr: LoopVectorizationPlanner::VPlanPtr LoopVectorizationPlanner::buildVPlanWithVPRecipes(...) but LoopVectorize.cpp includes VPlan.h (via LoopVectorizationPlanner.h) and can use VPlanPtr from that header. Patch by Pavel Samolysov. Reviewers: hsaito, rengolin, fhahn Reviewed By: fhahn Differential Revision: https://reviews.llvm.org/D62576 llvm-svn: 362126
* [LoopVectorize] Add FNeg instruction supportCraig Topper2019-05-301-9/+20
| | | | | | Differential Revision: https://reviews.llvm.org/D62510 llvm-svn: 362124
* [LoopIdiom] Basic OptimizationRemarkEmitter handlingRoman Lebedev2019-05-301-4/+40
| | | | | | | | | | | | | | | | | | Summary: I'm adding ORE to memset/memcpy formation, with tests, but mainly this is split off from D61144. Reviewers: reames, anemet, thegameg, craig.topper Reviewed By: thegameg Subscribers: llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62631 llvm-svn: 362092
* [LoopIdiomRecognize][NFC] Sort includesRoman Lebedev2019-05-301-2/+2
| | | | | | Split off from D61144 llvm-svn: 362091
* [LV] Inform about exactly reason of loop illegalityFlorian Hahn2019-05-301-2/+10
| | | | | | | | | | | | | | | | | | | | | | | Currently, only the following information is provided by LoopVectorizer in the case when the CF of the loop is not legal for vectorization: LV: Can't vectorize the instructions or CFG LV: Not vectorizing: Cannot prove legality. But this information is not enough for the root cause analysis; what is exactly wrong with the loop should also be printed: LV: Not vectorizing: The exiting block is not the loop latch. Patch by Pavel Samolysov. Reviewers: mkuper, hsaito, rengolin, fhahn Reviewed By: fhahn Differential Revision: https://reviews.llvm.org/D62311 llvm-svn: 362056
* LoopVersioningLICM: Respect convergent and noduplicateMatt Arsenault2019-05-291-1/+9
| | | | llvm-svn: 362031
* [LoopIdiomRecognize][NFC] Use DEBUG_TYPE, add LLVM_DEBUG() to ↵Roman Lebedev2019-05-291-2/+8
| | | | | | | | runOnNoncountableLoop() Split off from D61144 llvm-svn: 362022
* [InstCombine] Optimize always overflowing signed saturating add/subNikita Popov2019-05-291-8/+12
| | | | | | | | | Based on the overflow direction information added in D62463, we can now fold always overflowing signed saturating add/sub to signed min/max. Differential Revision: https://reviews.llvm.org/D62544 llvm-svn: 362006
* CallSiteSplitting: Respect convergent and noduplicateMatt Arsenault2019-05-291-0/+3
| | | | llvm-svn: 361990
* [ThinLTO] Use original alias visibility when importingTeresa Johnson2019-05-291-2/+3
| | | | | | | | | | | | | | | | | | | | | Summary: When we import an alias, we do so by making a clone of the aliasee. Just as this clone uses the original alias name and linkage, it should also use the same visibility (not the aliasee's visibility). Otherwise, linker behavior is affected (e.g. if the aliasee was hidden, but the alias is not, the resulting imported clone should not be hidden, otherwise the linker will make the final symbol hidden which is incorrect). Reviewers: wmi Subscribers: mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62535 llvm-svn: 361989
* SpeculateAroundPHIs: Respect convergentMatt Arsenault2019-05-291-0/+8
| | | | llvm-svn: 361957
* [PGO] Handle cases of failing to split critical edgesRong Xu2019-05-281-44/+56
| | | | | | | | | | | Fix PR41279 where critical edges to EHPad are not split. The fix is to not instrument those critical edges. We used to be able to know the size of counters right after MST is computed. With this, we have to pre-collect the instrument BBs to know the size, and then instrument them. Differential Revision: https://reviews.llvm.org/D62439 llvm-svn: 361882
* Revert "[CorrelatedValuePropagation] Fix prof branch_weights metadata ↵Nikita Popov2019-05-281-61/+56
| | | | | | | | | | | | handling for SwitchInst" This reverts commit 53f2f3286572cb879b3861d7c15480e4d830dd3b. As reported on D62126, this causes assertion failures if the switch has incorrect branch_weights metadata, which may happen as a result of other transforms not handling it correctly yet. llvm-svn: 361881
* [InstCombine] Clean up saturing math overflow optimizations; NFCNikita Popov2019-05-281-29/+20
| | | | | | | Reduce duplication and make it easier to handle signed always-overflows conditions in the future. llvm-svn: 361863
* [ValueTracking][ConstantRange] Distinguish low/high always overflowNikita Popov2019-05-282-3/+4
| | | | | | | | | | | | In order to fold an always overflowing signed saturating add/sub, we need to know in which direction the always overflow occurs. This patch splits up AlwaysOverflows into AlwaysOverflowsLow and AlwaysOverflowsHigh to pass through this information (but it is not used yet). Differential Revision: https://reviews.llvm.org/D62463 llvm-svn: 361858
* Re-commit r357452 (take 2): "SimplifyCFG SinkCommonCodeFromPredecessors: ↵Hans Wennborg2019-05-281-14/+15
| | | | | | | | | | | | | | | | | | | | Also sink function calls without used results (PR41259)" This was reverted in r360086 as it was supected of causing mysterious test failures internally. However, it was never concluded that this patch was the root cause. > The code was previously checking that candidates for sinking had exactly > one use or were a store instruction (which can't have uses). This meant > we could sink call instructions only if they had a use. > > That limitation seemed a bit arbitrary, so this patch changes it to > "instruction has zero or one use" which seems more natural and removes > the need to special-case stores. > > Differential revision: https://reviews.llvm.org/D59936 llvm-svn: 361811
* [CorrelatedValuePropagation] Fix prof branch_weights metadata handling for ↵Yevgeny Rouban2019-05-281-56/+61
| | | | | | | | | | | | | | SwitchInst This patch fixes the CorrelatedValuePropagation pass to keep prof branch_weights metadata of SwitchInst consistent. It makes use of SwitchInstProfUpdateWrapper. New tests are added. Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D62126 llvm-svn: 361808
* [LoopInterchange] Fix handling of LCSSA nodes defined in headers and latches.Florian Hahn2019-05-261-22/+64
| | | | | | | | | | | | | | | | | | The code to preserve LCSSA PHIs currently only properly supports reduction PHIs and PHIs for values defined outside the latches. This patch improves the LCSSA PHI handling to cover PHIs for values defined in the latches. Fixes PR41725. Reviewers: efriedma, mcrosier, davide, jdoerfert Reviewed By: jdoerfert Differential Revision: https://reviews.llvm.org/D61576 llvm-svn: 361743
* [SimplifyCFG] back out all SwitchInst commitsShawn Landden2019-05-261-90/+71
| | | | | | | | They caused the sanitizer builds to fail. My suspicion is the change the countLeadingZeros(). llvm-svn: 361736
* [InstCombine] prevent crashing with invalid extractelement indexSanjay Patel2019-05-261-2/+3
| | | | | | | This was found/reduced from a fuzzer report: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=14956 llvm-svn: 361729
* [SimplifyCFG] ReduceSwitchRange: Improve on the case where the SubThreshold ↵Shawn Landden2019-05-261-14/+24
| | | | | | doesn't trigger llvm-svn: 361728
* [SimplifyCFG] Run ReduceSwitchRange unconditionally, generalizeShawn Landden2019-05-261-56/+66
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Rather than gating on "isSwitchDense" (resulting in necessesarily sparse lookup tables even when they were generated), always run this quite cheap transform. This transform is useful not just for generating tables. LowerSwitch also wants this: read LowerSwitch.cpp:257. Be careful to not generate worse code, by introducing a SubThreshold heuristic. Instead of just sorting by signed, generalize the finding of the best base. And now that it is run unconditionally, do not replicate its functionality in SwitchToLookupTable (which could use a Sub when having a hole is smaller, hence the SubThreshold heuristic located in a single place). This simplifies SwitchToLookupTable, and fixes some ugly corner cases due to the use of signed numbers, such as a table containing i16 32768 and 32769, of which 32769 would be interpreted as -32768, and now the code thinks the table is size 65536. (We still use unconditional subtraction when building a single-register mask, but I think this whole block should go when the more general sparse map is added, which doesn't leave empty holes in the table.) And the reason test4 and test5 did not trigger was documented wrong: it was because they were not considered sufficiently "dense". Also, fix generation of invalid LLVM-IR: shl by bit-width. llvm-svn: 361727
OpenPOWER on IntegriCloud