summaryrefslogtreecommitdiffstats
path: root/llvm/test/Transforms
Commit message (Collapse)AuthorAgeFilesLines
* [InstCombine] adjust rotate direction in tests; NFCSanjay Patel2018-11-151-12/+12
| | | | | | Copy/paste errors - all of the changed tests rotated left before. llvm-svn: 346982
* [InstCombine] add tests for funnel shift (rotate) canonicalization; NFCSanjay Patel2018-11-151-1/+342
| | | | llvm-svn: 346975
* [TTI] Reduction costs only need to include a single extract element costSimon Pilgrim2018-11-153-282/+133
| | | | | | | | | | | | We were adding the entire scalarization extraction cost for reductions, which returns the total cost of extracting every element of a vector type. For reductions we don't need to do this - we just need to extract the 0'th element after the reduction pattern has completed. Fixes PR37731 Differential Revision: https://reviews.llvm.org/D54585 llvm-svn: 346970
* [InstCombine] fix rotate narrowing bug for non-pow-2 typesSanjay Patel2018-11-151-16/+16
| | | | llvm-svn: 346968
* [InstCombine] add rotate narrowing tests with odd types; NFCSanjay Patel2018-11-151-0/+46
| | | | | | | | | There's a potential miscompile here. It's unlikely in the real world because this transform is guarded with shouldChangeType(), but this test file doesn't include a standard data-layout for some reason (despite including a custom 1), so we can see the bug. llvm-svn: 346966
* [SLPVectorizer][X86] Regenerate reduction minmax tests and cleanup check ↵Simon Pilgrim2018-11-151-659/+272
| | | | | | prefixes llvm-svn: 346965
* [SLPVectorizer][X86] Regenerate reduction tests and add PR37731 testSimon Pilgrim2018-11-151-210/+196
| | | | | | Cleanup check prefixes llvm-svn: 346964
* [InstSimplify] delete shift-of-zero guard ops around funnel shiftsSanjay Patel2018-11-151-36/+34
| | | | | | | | | | | | | | | | | | | | | | | | This is a problem seen in common rotate idioms as noted in: https://bugs.llvm.org/show_bug.cgi?id=34924 Note that we are not canonicalizing standard IR (shifts and logic) to the intrinsics yet. (Although I've written this before...) I think this is the last step before we enable that transform. Ie, we could regress code by doing that transform without this simplification in place. In PR34924, I questioned whether this is a valid transform for target-independent IR, but I convinced myself this is ok. If we're speculating a funnel shift by turning cmp+br into select, then SimplifyCFG has already determined that the transform is justified. It's possible that SimplifyCFG is not taking into account profile or other metadata, but if that's true, then it's a bug independent of funnel shifts. Also, we do have CGP code to restore a guard like this around an intrinsic if it can't be lowered cheaply. But that isn't necessary for funnel shift because the default expansion in SelectionDAGBuilder includes this same cmp+select. Differential Revision: https://reviews.llvm.org/D54552 llvm-svn: 346960
* [InstSimplify] add more tests for funnel shift with select; NFCSanjay Patel2018-11-141-13/+78
| | | | | | | The cases are just different enough that we should have complete tests to avoid bugs from typos in the code. llvm-svn: 346902
* Mark @llvm.trap coldVedant Kumar2018-11-141-1/+1
| | | | | | | | | | | | | | | | A call to @llvm.trap can be expected to be cold (i.e. unlikely to be reached in a normal program execution). Outlining paths which unconditionally trap is an important memory saving. As the hot/cold splitting pass (imho) should not treat all noreturn calls as cold, explicitly mark @llvm.trap cold so that it can be outlined. Split out of https://reviews.llvm.org/D54244. Differential Revision: https://reviews.llvm.org/D54329 llvm-svn: 346885
* [ThinLTO] Update handling of vararg functions to match inlinerTeresa Johnson2018-11-142-9/+29
| | | | | | | | | | | | | | | | | Summary: Previously we marked all vararg functions as non-inlinable in the function summary, which prevented their importing. However, the corresponding inliner restriction was loosened in r321940/r342675 to only apply to functions calling va_start. Adjust the summary flag computation to match. Reviewers: davidxl Subscribers: mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, llvm-commits Differential Revision: https://reviews.llvm.org/D54270 llvm-svn: 346883
* [InstSimplify] add tests for funnel shift with select; NFCSanjay Patel2018-11-141-0/+65
| | | | llvm-svn: 346881
* [InstCombine] Remove a couple of asserts based on incorrect assumptionsMandeep Singh Grang2018-11-141-0/+18
| | | | | | | | | | | | | | | | Summary: These asserts are based on the assumption that the order of true/false operands in a select and those in the compare would always be the same. This fixes PR39595. Reviewers: craig.topper, spatel, dmgreen Reviewed By: craig.topper Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D54359 llvm-svn: 346874
* [SimplifyCFG] Regenerate preserve-branchweights.ll test. NFCJohn Brawn2018-11-141-114/+262
| | | | | | | Regenerate this test using update_test_checks.py in preparation for an upcomming commit, to make it not depend on the names of instructions. llvm-svn: 346869
* Recommit r346483: [CallSiteSplitting] Only record conditions up to the ↵Florian Hahn2018-11-141-10/+35
| | | | | | | | | IDom(call site). The underlying problem causing the expensive-check failure was fixed in rL346769. llvm-svn: 346843
* Revert r346810 "Preserve loop metadata when splitting exit blocks"Reid Kleckner2018-11-141-48/+0
| | | | | | | It broke the Windows self-host: http://lab.llvm.org:8011/builders/clang-x64-windows-msvc/builds/1457 llvm-svn: 346823
* [InstCombine] fold funnel shift amount based on demanded bitsSanjay Patel2018-11-131-14/+10
| | | | | | | | | | | | | | The shift amount of a funnel shift is modulo the scalar bitwidth: http://llvm.org/docs/LangRef.html#llvm-fshl-intrinsic ...so we can use demanded bits analysis on that operand to simplify it when we have a power-of-2 bitwidth. This is another step towards canonicalizing {shift/shift/or} to the intrinsics in IR. Differential Revision: https://reviews.llvm.org/D54478 llvm-svn: 346814
* Preserve loop metadata when splitting exit blocksCraig Topper2018-11-131-0/+48
| | | | | | | | | | LoopUtils.cpp contains a utility that splits an loop exit block, so that the new block contains only edges coming from the loop. In the case of nested loops, the exit path for the inner loop might also be the back-edge of the outer loop. The new block which is inserted on this path, is now a latch for the outer loop, and it needs to hold the loop metadata for the outer loop. (The test case gives a more concrete view of the situation.) Patch by Chang Lin (clin1) Differential Revision: https://reviews.llvm.org/D53876 llvm-svn: 346810
* [InstCombine] canonicalize rotate patterns with cmp/selectSanjay Patel2018-11-131-37/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The cmp+branch variant of this pattern is shown in: https://bugs.llvm.org/show_bug.cgi?id=34924 ...and as discussed there, we probably can't transform that without a rotate intrinsic. We do have that now via funnel shift, but we're not quite ready to canonicalize IR to that form yet. The case with 'select' should already be transformed though, so that's this patch. The sequence with negation followed by masking is what we use in the backend and partly in clang (though that part should be updated). https://rise4fun.com/Alive/TplC %cmp = icmp eq i32 %shamt, 0 %sub = sub i32 32, %shamt %shr = lshr i32 %x, %shamt %shl = shl i32 %x, %sub %or = or i32 %shr, %shl %r = select i1 %cmp, i32 %x, i32 %or => %neg = sub i32 0, %shamt %masked = and i32 %shamt, 31 %maskedneg = and i32 %neg, 31 %shl2 = lshr i32 %x, %masked %shr2 = shl i32 %x, %maskedneg %r = or i32 %shl2, %shr2 llvm-svn: 346807
* [IR] Add a dedicated FNeg IR InstructionCameron McInally2018-11-131-8/+8
| | | | | | | | | | | The IEEE-754 Standard makes it clear that fneg(x) and fsub(-0.0, x) are two different operations. The former is a bitwise operation, while the latter is an arithmetic operation. This patch creates a dedicated FNeg IR Instruction to model that behavior. Differential Revision: https://reviews.llvm.org/D53877 llvm-svn: 346774
* [CSP, Cloning] Update DuplicateInstructionsInSplitBetween to use DomTreeUpdater.Florian Hahn2018-11-131-1/+1
| | | | | | | | | | | | | | | | | | | | | This patch updates DuplicateInstructionsInSplitBetween to update a DTU instead of applying updates to the DT directly. Given that there only are 2 users, also updated them in this patch to avoid churn. I slightly moved the code in CallSiteSplitting around to reduce the places where we have to pass in DTU. If necessary, I could split those changes in a separate patch. This fixes missing DT updates when dealing with musttail calls in CallSiteSplitting, by using DTU->deleteBB. Reviewers: junbuml, kuhar, NutshellySima, indutny, brzycki Reviewed By: NutshellySima llvm-svn: 346769
* [InstCombine] add tests for funnel shift demanded bits; NFCSanjay Patel2018-11-131-0/+147
| | | | llvm-svn: 346762
* [InstCombine] add rotate variants that include select; NFCSanjay Patel2018-11-121-0/+84
| | | | llvm-svn: 346719
* [InstCombine] narrow width of rotate patterns, part 3Sanjay Patel2018-11-121-44/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a longer variant for the pattern handled in rL346713 This one includes zexts. Eventually, we should canonicalize all rotate patterns to the funnel shift intrinsics, but we need a bit more infrastructure to make sure the vectorizers handle those intrinsics as well as the shift+logic ops. https://rise4fun.com/Alive/FMn Name: narrow rotateright %neg = sub i8 0, %shamt %rshamt = and i8 %shamt, 7 %rshamtconv = zext i8 %rshamt to i32 %lshamt = and i8 %neg, 7 %lshamtconv = zext i8 %lshamt to i32 %conv = zext i8 %x to i32 %shr = lshr i32 %conv, %rshamtconv %shl = shl i32 %conv, %lshamtconv %or = or i32 %shl, %shr %r = trunc i32 %or to i8 => %maskedShAmt2 = and i8 %shamt, 7 %negShAmt2 = sub i8 0, %shamt %maskedNegShAmt2 = and i8 %negShAmt2, 7 %shl2 = lshr i8 %x, %maskedShAmt2 %shr2 = shl i8 %x, %maskedNegShAmt2 %r = or i8 %shl2, %shr2 llvm-svn: 346716
* [InstCombine] narrow width of rotate patterns, part 2 (PR39624)Sanjay Patel2018-11-121-32/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The sub-pattern for the shift amount in a rotate can take on several different forms, and there's apparently no way to canonicalize those without seeing the entire rotate sequence. This is the form noted in: https://bugs.llvm.org/show_bug.cgi?id=39624 https://rise4fun.com/Alive/qnT %zx = zext i8 %x to i32 %maskedShAmt = and i32 %shAmt, 7 %shl = shl i32 %zx, %maskedShAmt %negShAmt = sub i32 0, %shAmt %maskedNegShAmt = and i32 %negShAmt, 7 %shr = lshr i32 %zx, %maskedNegShAmt %rot = or i32 %shl, %shr %r = trunc i32 %rot to i8 => %truncShAmt = trunc i32 %shAmt to i8 %maskedShAmt2 = and i8 %truncShAmt, 7 %shl2 = shl i8 %x, %maskedShAmt2 %negShAmt2 = sub i8 0, %truncShAmt %maskedNegShAmt2 = and i8 %negShAmt2, 7 %shr2 = lshr i8 %x, %maskedNegShAmt2 %r = or i8 %shl2, %shr2 llvm-svn: 346713
* [InstCombine] add more tests for rotate narrowing; NFCSanjay Patel2018-11-121-43/+144
| | | | llvm-svn: 346703
* [InstCombine] regenerate checks; NFCSanjay Patel2018-11-121-15/+16
| | | | llvm-svn: 346689
* [CostModel] Add more realistic SK_InsertSubvector generic costs.Simon Pilgrim2018-11-121-55/+58
| | | | | | Instead of defaulting to a cost = 1, expand to element extract/insert like we do for other shuffles. llvm-svn: 346662
* [VectorUtils] add funnel-shifts to the list of vectorizable intrinsicsSanjay Patel2018-11-121-2/+2
| | | | | | | | | | | | | | | | This just identifies the intrinsics as candidates for vectorization. It does not mean we will attempt to vectorize under normal conditions (the test file is forcing vectorization). The cost model must be fixed to show that the transform is profitable in general. Allowing vectorization with these intrinsics is required to avoid potential regressions from canonicalizing to the intrinsics from generic IR: https://bugs.llvm.org/show_bug.cgi?id=37417 llvm-svn: 346661
* [LoopVectorize] add tests for funnel shifts; NFCSanjay Patel2018-11-121-0/+52
| | | | llvm-svn: 346658
* [LICM] Hoist guards from non-header blocksMax Kazantsev2018-11-121-2/+150
| | | | | | | | | | | This patch relaxes overconservative checks on whether or not we could write memory before we execute an instruction. This allows us to hoist guards out of loops even if they are not in the header block. Differential Revision: https://reviews.llvm.org/D50891 Reviewed By: fedor.sergeev llvm-svn: 346643
* [IPSCCP,PM] Preserve PDT in the new pass manager.Florian Hahn2018-11-111-16/+9
| | | | | | | | | | Reviewers: kuhar, chandlerc, NutshellySima, brzycki Reviewed By: NutshellySima, brzycki Differential Revision: https://reviews.llvm.org/D54317 llvm-svn: 346618
* [InstCombine] auto-generate full checks; NFCSanjay Patel2018-11-102-206/+304
| | | | llvm-svn: 346594
* [JumpThreading] Fix exponential time algorithm computing known values.Eli Friedman2018-11-091-1/+61
| | | | | | | | | | | | | | | | | | | ComputeValueKnownInPredecessors has a "visited" set to prevent infinite loops, since a value can be visited more than once. However, the implementation didn't prevent the algorithm from taking exponential time. Instead of removing elements from the RecursionSet one at a time, we should keep around the whole set until ComputeValueKnownInPredecessors finishes, then discard it. The testcase is synthetic because I was having trouble effectively reducing the original. But it's basically the same idea. Instead of failing, we could theoretically cache the result instead. But I don't think it would help substantially in practice. Differential Revision: https://reviews.llvm.org/D54239 llvm-svn: 346562
* [CostModel][X86] SK_ExtractSubvector is free if the subvector is at the ↵Simon Pilgrim2018-11-091-1/+1
| | | | | | start of the source vector llvm-svn: 346538
* Revert r346483: [CallSiteSplitting] Only record conditions up to the ↵Florian Hahn2018-11-091-33/+8
| | | | | | | | IDom(call site). This cause a failure with EXPENSIVE_CHECKS llvm-svn: 346492
* [IPSCCP,PM] Preserve DT in the new pass manager.Florian Hahn2018-11-091-0/+63
| | | | | | | | | | | | | | After D45330, Dominators are required for IPSCCP and can be preserved. This patch preserves DominatorTreeAnalysis in the new pass manager. AFAIK the legacy pass manager cannot preserve function analysis required by a module analysis. Reviewers: davide, dberlin, chandlerc, efriedma, kuhar, NutshellySima Reviewed By: chandlerc, kuhar, NutshellySima Differential Revision: https://reviews.llvm.org/D47259 llvm-svn: 346486
* [CallSiteSplitting] Only record conditions up to the IDom(call site).Florian Hahn2018-11-091-8/+33
| | | | | | | | | | | | | | | | | | | | | | | We can stop recording conditions once we reached the immediate dominator for the block containing the call site. Conditions in predecessors of the that node will be the same for all paths to the call site and splitting is not beneficial. This patch makes CallSiteSplitting dependent on the DT anlysis. because the immediate dominators seem to be the easiest way of finding the node to stop at. I had to update some exiting tests, because they were checking for conditions that were true/false on all paths to the call site. Those should now be handled by instcombine/ipsccp. Reviewers: davide, junbuml Reviewed By: junbuml Differential Revision: https://reviews.llvm.org/D44627 llvm-svn: 346483
* [LoopInterchange] Support reductions across inner and outer loop.Florian Hahn2018-11-084-3/+205
| | | | | | | | | | | | | | | | | | | | | This patch adds logic to detect reductions across the inner and outer loop by following the incoming values of PHI nodes in the outer loop. If the incoming values take part in a reduction in the inner loop or come from outside the outer loop, we found a reduction spanning across inner and outer loop. With this change, ~10% more loops are interchanged in the LLVM test-suite + SPEC2006. Fixes https://bugs.llvm.org/show_bug.cgi?id=30472 Reviewers: mcrosier, efriedma, karthikthecool, davide, hfinkel, dmgreen Reviewed By: efriedma Differential Revision: https://reviews.llvm.org/D43245 llvm-svn: 346438
* [LTO] Drop non-prevailing definitions only if linkage is not local or appendingPirama Arumuga Nainar2018-11-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This fixes PR 37422 In ELF, non-weak symbols can also be non-prevailing. In this particular PR, the __llvm_profile_* symbols are non-prevailing but weren't getting dropped - causing multiply-defined errors with lld. Also add a test, strong_non_prevailing.ll, to ensure that multiple copies of a strong symbol are dropped. To fix the test regressions exposed by this fix, - do not mark prevailing copies for symbols with 'appending' linkage. There's no one prevailing copy for such symbols. - fix the prevailing version in dead-strip-fulllto.ll - explicitly pass exported symbols to llvm-lto in fumcimport.ll and funcimport_var.ll Reviewers: tejohnson, pcc Subscribers: mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, dang, srhines, llvm-commits Differential Revision: https://reviews.llvm.org/D54125 llvm-svn: 346436
* InstCombine: Avoid introducing poison values when lowering llvm.amdgcn.[us]bfeTom Stellard2018-11-081-23/+7
| | | | | | | | | | | | | | | | | | | Summary: When the 3rd argument to these intrinsics is zero, lowering them to shift instructions produces poison values, since we end up with shift amounts equal to the number of bits in the shifted value. This means we can only lower these intrinsics if we can prove that the 3rd argument is not zero. Reviewers: arsenm Reviewed By: arsenm Subscribers: bnieuwenhuizen, jvesely, wdng, nhaehnle, llvm-commits Differential Revision: https://reviews.llvm.org/D53739 llvm-svn: 346422
* [CodeExtractor] Mark functions noreturn when applicableVedant Kumar2018-11-081-0/+3
| | | | | | | | | | This eliminates the outlining penalty for llvm.trap/unreachable, because callers no longer have to emit cleanup/ret instructions after calling an outlined `noreturn` function. rdar://45523626 llvm-svn: 346421
* Return "[IndVars] Smart hard uses detection"Max Kazantsev2018-11-082-0/+89
| | | | | | | | | | The patch has been reverted because it ended up prohibiting propagation of a constant to exit value. For such values, we should skip all checks related to hard uses because propagating a constant is always profitable. Differential Revision: https://reviews.llvm.org/D53691 llvm-svn: 346397
* [LSR] Combine unfolded offset into invariant registerGil Rapaport2018-11-082-55/+75
| | | | | | | | | | | | | | | LSR reassociates constants as unfolded offsets when the constants fit as immediate add operands, which currently prevents such constants from being combined later with loop invariant registers. This patch modifies GenerateCombinations() to generate a second formula which includes the unfolded offset in the combined loop-invariant register. This commit fixes a bug in the original patch (committed at r345114, reverted at r345123). Differential Revision: https://reviews.llvm.org/D51861 llvm-svn: 346390
* [MergeFuncs] Improve ordering of equal functionswhitequark2018-11-081-0/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: MergeFunctions currently tries to process strong functions before weak functions, because weak functions can simply call strong functions, while a strong/weak function cannot call a weak function (a backing strong function is needed). This patch additionally tries to process external functions before local functions, because we definitely have to keep the external function, but may be able to drop the local one (and definitely can if it is also unnamed_addr). Unfortunately, this exposes an existing bug in the implementation: The FnTree and FNodesInTree structures can currently go out of sync in the case where two weak functions are merged, because the function in FnTree/FNodesInTree is RAUWed. This leaves it behind in FnTree (this is intended, as it is the strong backing function which should be used for further merges), while it is replaced in FNodesInTree (this is not intended). This is fixed by switching FNodesInTree from using a ValueMap to using a DenseMap of AssertingVH. This exposes another minor issue: Currently FNodesInTree is not cleared after MergeFunctions finishes running. Currently, this is potentially dangerous (e.g. if something else wants to RAUW a function with a non-function), but at the very least it is unnecessary/inefficient. After the change to use AssertingVH it becomes more problematic, because there are certainly passes that remove functions. This issue is fixed by clearing FNodesInTree at the end of the pass. Reviewers: jfb, whitequark Reviewed By: whitequark Subscribers: rkruppe, llvm-commits Differential Revision: https://reviews.llvm.org/D53271 llvm-svn: 346386
* [MergeFuncs] Call removeUsers() prior to unnamed_addr RAUWwhitequark2018-11-081-0/+35
| | | | | | | | | | | | | | | | | Summary: For unnamed_addr functions we RAUW instead of only replacing direct callers. However, functions in which replacements were performed currently are not added back to the worklist, resulting in missed merging opportunities. Fix this by calling removeUsers() prior to RAUW. Reviewers: jfb, whitequark Reviewed By: whitequark Subscribers: rkruppe, llvm-commits Differential Revision: https://reviews.llvm.org/D53262 llvm-svn: 346385
* [PGO] Exit early if all count values are zeroRong Xu2018-11-072-0/+46
| | | | | | | | | If all the edge counts for a function are zero, skip count population and annotation, as nothing will happen. This can save some compile time. Differential Revision: https://reviews.llvm.org/D54212 llvm-svn: 346370
* [SimpleLoopUnswitch] partial unswitch needs to be careful when replacing ↵Fedor Sergeev2018-11-071-11/+98
| | | | | | | | | | | | | | | | | | | | | | | | | invariants with constants When partial unswitch operates on multiple conditions at once, .e.g: if (Cond1 || Cond2 || NonInv) ... it should infer (and replace) values for individual conditions only on one side of unswitch and not another. More precisely only these derivations hold true: (Cond1 || Cond2) == false => Cond1 == Cond2 == false (Cond1 && Cond2) == true => Cond1 == Cond2 == true By the way we organize unswitching it means only replacing on "continue" blocks and never on "unswitched" ones. Since trivial unswitch does not have "unswitched" blocks it does not have this problem. Fixes PR 39568. Reviewers: chandlerc, asbirlea Differential Revision: https://reviews.llvm.org/D54211 llvm-svn: 346350
* [LoopSink] Do not sink instructions into non-cold blocksMandeep Singh Grang2018-11-071-0/+112
| | | | | | | | | | | | | | Summary: This fixes PR39570. Reviewers: danielcdh, rnk, bkramer Reviewed By: rnk Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D54181 llvm-svn: 346337
* [NewGVN] Make sure we do not add a user to itself.Florian Hahn2018-11-071-0/+27
| | | | | | | | | | | | | | | | If we simplify an instruction to itself, we do not need to add a user to itself. For congruence classes with a defining expression, we already use a similar logic. Fixes PR38259. Reviewers: davide, efriedma, mcrosier Reviewed By: davide Differential Revision: https://reviews.llvm.org/D51168 llvm-svn: 346335
OpenPOWER on IntegriCloud