summaryrefslogtreecommitdiffstats
path: root/llvm/test/Transforms/SLPVectorizer/X86
Commit message (Collapse)AuthorAgeFilesLines
...
* [SLPVectorizer][X86] Add SSE common check prefix to let us merge SSE2+SLM checksSimon Pilgrim2019-09-262-150/+4
| | | | llvm-svn: 372955
* [CostModel][X86] Fix SLM <2 x i64> icmp costsSimon Pilgrim2019-09-262-32/+64
| | | | | | | | SLM is 2 x slower for <2 x i64> comparison ops than other vector types, we should account for this like we do for SLM <2 x i64> add/sub/mul costs. This should remove some of the SLM codegen diffs in D43582 llvm-svn: 372954
* [SLP] Fix for PR31847: Assertion failed: (isLoopInvariant(Operands[i], L) && ↵Alexey Bataev2019-09-2317-1175/+266
| | | | | | | | | | | | | | | | | | | | | "SCEVAddRecExpr operand is not loop-invariant!") Summary: Initially SLP vectorizer replaced all going-to-be-vectorized instructions with Undef values. It may break ScalarEvaluation and may cause a crash. Reworked SLP vectorizer so that it does not replace vectorized instructions by UndefValue anymore. Instead vectorized instructions are marked for deletion inside if BoUpSLP class and deleted upon class destruction. Reviewers: mzolotukhin, mkuper, hfinkel, RKSimon, davide, spatel Subscribers: RKSimon, Gerolf, anemet, hans, majnemer, llvm-commits, sanjoy Differential Revision: https://reviews.llvm.org/D29641 llvm-svn: 372626
* [Cost][X86] Add v2i64 truncation costsSimon Pilgrim2019-09-221-16/+64
| | | | | | | | We are missing costs for a lot of truncation cases, I'm hoping to address all the 'zero cost' cases in trunc.ll I thought this was a vector widening side effect, but even before this we had some interesting LV decisions (notably over indvars) being made due to these zero costs. llvm-svn: 372498
* [SLPVectorizer] add tests for bogus reductions; NFCSanjay Patel2019-09-201-0/+334
| | | | | | | https://bugs.llvm.org/show_bug.cgi?id=42708 https://bugs.llvm.org/show_bug.cgi?id=43146 llvm-svn: 372393
* [SLP] limit vectorization of Constant subclasses (PR33958)Sanjay Patel2019-09-151-6/+9
| | | | | | | | | | | | | | This is a fix for: https://bugs.llvm.org/show_bug.cgi?id=33958 It seems universally true that we would not want to transform this kind of sequence on any target, but if that's not correct, then we could view this as a target-specific cost model problem. We could also white-list ConstantInt, ConstantFP, etc. rather than blacklist Global and ConstantExpr. Differential Revision: https://reviews.llvm.org/D67362 llvm-svn: 371931
* [SLP] add test for vectorization of constant expressions; NFCSanjay Patel2019-09-131-0/+13
| | | | | | Goes with D67362. llvm-svn: 371879
* [SLP] add test for over-vectorization (PR33958); NFCSanjay Patel2019-09-091-0/+16
| | | | llvm-svn: 371426
* [X86] Add prefer-128-bit subtarget feature.Craig Topper2019-09-074-9/+17
| | | | | | | | | | | | | | | | | | | Summary: Similar to the previous prefer-256-bit flag. We might want to enable this by default some CPUs. This just starts the initial work to implement and prove that it effects TTI's vector width. Reviewers: RKSimon, echristo, spatel, atdt Reviewed By: RKSimon Subscribers: lebedev.ri, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D67311 llvm-svn: 371319
* [X86] Replace -mcpu with -mattr on some tests.Craig Topper2019-09-061-4/+3
| | | | llvm-svn: 371260
* [SLP] add test that requires shuffle of scalars; NFCSanjay Patel2019-08-191-0/+26
| | | | llvm-svn: 369255
* [SLP] add tests for PR16739; NFCSanjay Patel2019-08-161-0/+68
| | | | llvm-svn: 369127
* [X86][CostModel] Adjust the costs of ZERO_EXTEND/SIGN_EXTEND with less than ↵Craig Topper2019-08-143-208/+262
| | | | | | | | | | | | 128-bit inputs Now that we legalize by widening, the element types here won't change. Previously these were modeled as the elements being widened and then the instruction might become an AND or SHL/ASHR pair. But now they'll become something like a ZERO_EXTEND_VECTOR_INREG/SIGN_EXTEND_VECTOR_INREG. For AVX2, when the destination type is legal its clear the cost should be 1 since we have extend instructions that can produce 256 bit vectors from less than 128 bit vectors. I'm a little less sure about AVX1 costs, but I think the ones I changed were definitely too high, but they might still be too high. Differential Revision: https://reviews.llvm.org/D66169 llvm-svn: 368858
* Recommit r367901 "[X86] Enable ↵Craig Topper2019-08-076-147/+246
| | | | | | | | | | | | | | | | | | | | | | | | | | | -x86-experimental-vector-widening-legalization by default." The assert that caused this to be reverted should be fixed now. Original commit message: This patch changes our defualt legalization behavior for 16, 32, and 64 bit vectors with i8/i16/i32/i64 scalar types from promotion to widening. For example, v8i8 will now be widened to v16i8 instead of promoted to v8i16. This keeps the elements widths the same and pads with undef elements. We believe this is a better legalization strategy. But it carries some issues due to the fragmented vector ISA. For example, i8 shifts and multiplies get widened and then later have to be promoted/split into vXi16 vectors. This has the potential to cause regressions so we wanted to get it in early in the 10.0 cycle so we have plenty of time to address them. Next steps will be to merge tests that explicitly test the command line option. And then we can remove the option and its associated code. llvm-svn: 368183
* Revert "[X86] Enable -x86-experimental-vector-widening-legalization by default."Mitch Phillips2019-08-066-246/+147
| | | | | | | | | This reverts commit 3de33245d2c992c9e0af60372043540b60f3a810. This commit broke the MSan buildbots. See https://reviews.llvm.org/rL367901 for more information. llvm-svn: 368107
* [X86] Enable -x86-experimental-vector-widening-legalization by default.Craig Topper2019-08-056-147/+246
| | | | | | | | | | | | | | | | | | | | | This patch changes our defualt legalization behavior for 16, 32, and 64 bit vectors with i8/i16/i32/i64 scalar types from promotion to widening. For example, v8i8 will now be widened to v16i8 instead of promoted to v8i16. This keeps the elements widths the same and pads with undef elements. We believe this is a better legalization strategy. But it carries some issues due to the fragmented vector ISA. For example, i8 shifts and multiplies get widened and then later have to be promoted/split into vXi16 vectors. This has the potential to cause regressions so we wanted to get it in early in the 10.0 cycle so we have plenty of time to address them. Next steps will be to merge tests that explicitly test the command line option. And then we can remove the option and its associated code. llvm-svn: 367901
* [FunctionAttrs] Annotate "willreturn" for intrinsicsHideto Ueno2019-07-281-1/+1
| | | | | | | | | | | | | | | | | | | Summary: In D62801, new function attribute `willreturn` was introduced. In short, a function with `willreturn` is guaranteed to come back to the call site(more precise definition is in LangRef). In this patch, willreturn is annotated for LLVM intrinsics. Reviewers: jdoerfert Reviewed By: jdoerfert Subscribers: jvesely, nhaehnle, sstefan1, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D64904 llvm-svn: 367184
* Temporarily Revert "[SLP] Recommit: Look-ahead operand reordering heuristic."Eric Christopher2019-07-151-175/+33
| | | | | | | | | As there are some reported miscompiles with AVX512 and performance regressions in Eigen. Verified with the original committer and testcases will be forthcoming. This reverts commit r364964. llvm-svn: 366154
* [SLP] add tests for bitcasted vector pointer load; NFCSanjay Patel2019-07-031-0/+102
| | | | | | | | I'm not sure if this falls within the scope of SLP, but we could create vector loads for some of these patterns. llvm-svn: 365055
* [SLP] Recommit: Look-ahead operand reordering heuristic.Vasileios Porpodas2019-07-021-33/+175
| | | | | | | | | | | | | | | | Summary: This patch introduces a new heuristic for guiding operand reordering. The new "look-ahead" heuristic can look beyond the immediate predecessors. This helps break ties when the immediate predecessors have identical opcodes (see lit test for an example). Reviewers: RKSimon, ABataev, dtemirbulatov, Ayal, hfinkel, rnk Reviewed By: RKSimon, dtemirbulatov Subscribers: hiraditya, phosek, rnk, rcorcs, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60897 llvm-svn: 364964
* Revert [SLP] Look-ahead operand reordering heuristic.Jordan Rupprecht2019-07-011-87/+47
| | | | | | | | This reverts r364478 (git commit 574cb0eb3a7ac95e62d223a60bef891171dfe321) The patch is causing compilation timeouts. llvm-svn: 364846
* [SLP] Look-ahead operand reordering heuristic.Vasileios Porpodas2019-06-261-47/+87
| | | | | | | | | | | | | | | | Summary: This patch introduces a new heuristic for guiding operand reordering. The new "look-ahead" heuristic can look beyond the immediate predecessors. This helps break ties when the immediate predecessors have identical opcodes (see lit test for an example). Reviewers: RKSimon, ABataev, dtemirbulatov, Ayal, hfinkel, rnk Reviewed By: RKSimon, dtemirbulatov Subscribers: rnk, rcorcs, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60897 llvm-svn: 364478
* [SLPVectorizer] Precommit of supernode.ll test for D63661Simon Pilgrim2019-06-251-0/+327
| | | | | | | | | | This is a pre-commit of the tests introduced by the SuperNode SLP patch D63661. Committed on behalf of @vporpo (Vasileios Porpodas) Differential Revision: https://reviews.llvm.org/D63664 llvm-svn: 364320
* [SLP] Support unary FNeg vectorizationCameron McInally2019-06-242-22/+10
| | | | | | Differential Revision: https://reviews.llvm.org/D63609 llvm-svn: 364219
* Revert [SLP] Look-ahead operand reordering heuristic.Reid Kleckner2019-06-211-44/+47
| | | | | | | | | This reverts r364084 (git commit 5698921be2d567f6abf925479ac9f5a376d6d74f) It caused crashes while compiling a file in Chrome. Reduction forthcoming. llvm-svn: 364111
* [SLP] Look-ahead operand reordering heuristic.Simon Pilgrim2019-06-211-47/+44
| | | | | | | | | | This patch introduces a new heuristic for guiding operand reordering. The new "look-ahead" heuristic can look beyond the immediate predecessors. This helps break ties when the immediate predecessors have identical opcodes (see lit test for an example). Committed on behalf of @vporpo (Vasileios Porpodas) Differential Revision: https://reviews.llvm.org/D60897 llvm-svn: 364084
* [NFC][SLP] Pre-commit unary FNeg test to X86/propagate_ir_flags.llCameron McInally2019-06-201-0/+78
| | | | llvm-svn: 363978
* [NFC][SLP] Pre-commit unary FNeg test to X86/phi3.llCameron McInally2019-06-201-0/+41
| | | | llvm-svn: 363937
* [SLP][X86] Add lookahead reordering tests from D60897Simon Pilgrim2019-06-201-3/+235
| | | | llvm-svn: 363925
* [lit] Delete empty lines at the end of lit.local.cfg NFCFangrui Song2019-06-171-1/+0
| | | | llvm-svn: 363538
* [SLP] Update propagate_ir_flags.ll test to check that we do retain the ↵Dinar Temirbulatov2019-06-131-0/+36
| | | | | | common subset, NFC. llvm-svn: 363218
* [SLP] Fix regression in broadcasts caused by operand reordering patch D59973.Dinar Temirbulatov2019-06-051-29/+29
| | | | | | | | | | | | This patch fixes a regression caused by the operand reordering refactoring patch https://reviews.llvm.org/D59973 . The fix changes the strategy to Splat instead of Opcode, if broadcast opportunities are found. Please see the lit test for some examples. Committed on behalf of @vporpo (Vasileios Porpodas) Differential Revision: https://reviews.llvm.org/D62427 llvm-svn: 362613
* [SLPVectorizer][X86] Add other tests described in PR28474Simon Pilgrim2019-06-011-0/+256
| | | | llvm-svn: 362297
* [SLPVectorizer][X86] This test was from PR28474Simon Pilgrim2019-06-011-1/+1
| | | | llvm-svn: 362296
* [SLPVectorizer][X86] Add broadcast test case from D62427Simon Pilgrim2019-05-281-0/+124
| | | | llvm-svn: 361805
* [CostModel][X86] Add min/max reduction costs for all SSE targetsSimon Pilgrim2019-05-111-36/+54
| | | | | | | | The original costs stopped at SSE42, I've added conservative estimates for everything down to SSE1/SSE2 and moved some of the SSE42 costs to SSE41 (really only the addition of PCMPGT makes any difference). I've also added missing vXi8 costs (we use PHMINPOSUW for i8/i16 for scarily quick results) and 256-bit vector costs for AVX1. llvm-svn: 360528
* [SLP] Lit test that cannot get vectorized due to lack of look-ahead operand ↵Simon Pilgrim2019-04-301-0/+74
| | | | | | | | | | | | | reordering heuristic. The code in this test is not vectorized by SLP because its operand reordering cannot look beyond the immediate predecessors. This will get fixed in a follow-up patch that introduces the look-ahead operand reordering heuristic. Committed on behalf of @vporpo (Vasileios Porpodas) Differential Revision: https://reviews.llvm.org/D61283 llvm-svn: 359553
* [SLP] Fix crash after r358519, by V. Porpodas.Alexey Bataev2019-04-241-0/+47
| | | | | | | | | | | | | | | | Summary: The code did not check if operand was undef before casting it to Instruction. Reviewers: RKSimon, ABataev, dtemirbulatov Reviewed By: ABataev Subscribers: uabelho Tags: #llvm Differential Revision: https://reviews.llvm.org/D61024 llvm-svn: 359136
* Revert "Temporarily Revert "Add basic loop fusion pass.""Eric Christopher2019-04-17172-0/+54466
| | | | | | | | The reversion apparently deleted the test/Transforms directory. Will be re-reverting again. llvm-svn: 358552
* Temporarily Revert "Add basic loop fusion pass."Eric Christopher2019-04-17172-54466/+0
| | | | | | | | As it's causing some bot failures (and per request from kbarton). This reverts commit r358543/ab70da07286e618016e78247e4a24fcb84077fda. llvm-svn: 358546
* [SLP] Refactoring of the operand reordering code.Simon Pilgrim2019-04-163-55/+44
| | | | | | | | | | | | | | This is a refactoring patch which should have all the functionality of the current code. Its goal is twofold: i. Cleanup and simplify the reordering code, and ii. Generalize reordering so that it will work for an arbitrary number of operands, not just 2. This is the second patch in a series of patches that will enable operand reordering across chains of operations. An example of this was presented in EuroLLVM'18 https://www.youtube.com/watch?v=gIEn34LvyNo . Committed on behalf of @vporpo (Vasileios Porpodas) Differential Revision: https://reviews.llvm.org/D59973 llvm-svn: 358519
* [SLP][X86] Regenerate operandorder tests with arguments on same line. NFCI.Simon Pilgrim2019-04-041-25/+8
| | | | | | Stops update_test_checks.py from splitting the later arguments after the CHECKs. llvm-svn: 357679
* [InstCombine] canonicalize select shuffles by commutingSanjay Patel2019-03-312-7/+7
| | | | | | | | | | | | | | | | | | | | In PR41304: https://bugs.llvm.org/show_bug.cgi?id=41304 ...we have a case where we want to fold a binop of select-shuffle (blended) values. Rather than try to match commuted variants of the pattern, we can canonicalize the shuffles and check for mask equality with commuted operands. We don't produce arbitrary shuffle masks in instcombine, but select-shuffles are a special case that the backend is required to handle because we already canonicalize vector select to this shuffle form. So there should be no codegen difference from this change. It's possible that this improves CSE in IR though. Differential Revision: https://reviews.llvm.org/D60016 llvm-svn: 357366
* [SLP] Add support for commutative icmp/fcmp predicatesSimon Pilgrim2019-03-291-80/+16
| | | | | | | | | | For the cases where the icmp/fcmp predicate is commutative, use reorderInputsAccordingToOpcode to collect and commute the operands. This requires a helper to recognise commutativity in both general Instruction and CmpInstr types - the CmpInst::isCommutative doesn't overload the Instruction::isCommutative method for reasons I'm not clear on (maybe because its based on predicate not opcode?!?). Differential Revision: https://reviews.llvm.org/D59992 llvm-svn: 357266
* [SLP] Add support for swapping icmp/fcmp predicates to permit vectorizationSimon Pilgrim2019-03-291-60/+12
| | | | | | | | We should be able to match elements with the swapped predicate as well - as long as we commute the source operands. Differential Revision: https://reviews.llvm.org/D59956 llvm-svn: 357243
* [SLP][X86] Add tests showing failure to commute icmp/fcmp by swapping predicateSimon Pilgrim2019-03-281-0/+196
| | | | | | By swapping icmp/fcmp predicates we can commute their operands to improve vectorization llvm-svn: 357204
* [SLP][X86] Add tests showing failure to commute icmp/fcmp operandsSimon Pilgrim2019-03-281-0/+199
| | | | | | Some predicates are fully commutative - we should be able to easily commute their operands to improve vectorization llvm-svn: 357202
* [SLPVectorizer] Merge reorderAltShuffleOperands into ↵Simon Pilgrim2019-03-251-2/+2
| | | | | | | | | | reorderInputsAccordingToOpcode As discussed on D59738, this generalizes reorderInputsAccordingToOpcode to handle multiple + non-commutative instructions so we can get rid of reorderAltShuffleOperands and make use of the extra canonicalizations that reorderInputsAccordingToOpcode brings. Differential Revision: https://reviews.llvm.org/D59784 llvm-svn: 356939
* [SLPVectorizer] Update file missed in rL356913Simon Pilgrim2019-03-251-5/+5
| | | | | | Differential Revision: https://reviews.llvm.org/D59738 llvm-svn: 356915
* [SLPVectorizer] reorderInputsAccordingToOpcode - remove non-Instruction ↵Simon Pilgrim2019-03-2539-134/+134
| | | | | | | | | | | | | | canonicalization Remove attempts to commute non-Instructions to the LHS - the codegen changes appear to rely on chance more than anything else and also have a tendency to fight existing instcombine canonicalization which moves constants to the RHS of commutable binary ops. This is prep work towards: (a) reusing reorderInputsAccordingToOpcode for alt-shuffles and removing the similar reorderAltShuffleOperands (b) improving reordering to optimized cases with commutable and non-commutable instructions to still find splat/consecutive ops. Differential Revision: https://reviews.llvm.org/D59738 llvm-svn: 356913
OpenPOWER on IntegriCloud