summaryrefslogtreecommitdiffstats
path: root/llvm/test/Transforms/SLPVectorizer
Commit message (Collapse)AuthorAgeFilesLines
...
* Preserve IR flags (nsw, nuw, exact, fast-math) in SLP vectorizer (PR20802).Sanjay Patel2014-09-0312-24/+374
| | | | | | | | | | | | | | | | | The SLP vectorizer should propagate IR-level optimization hints/flags (nsw, nuw, exact, fast-math) when converting scalar instructions into vectors. But this isn't a simple copy - we need to take the intersection (the logical 'and') of the sets of flags on the scalars. The solution is further complicated because we can have non-uniform (non-SIMD) vector ops after: http://reviews.llvm.org/D4015 http://llvm.org/viewvc/llvm-project?view=revision&revision=211339 The vast majority of changed files are existing tests that were not propagating IR flags, but I've also added a new test file for focused testing of IR flag possibilities. Differential Revision: http://reviews.llvm.org/D5172 llvm-svn: 217051
* Generate extract for in-tree uses if the use is scalar operand in vectorized ↵Yi Jiang2014-09-021-0/+70
| | | | | | instruction. radar://18144665 llvm-svn: 216946
* Fix: SLPVectorizer tried to move an instruction which was replaced by a ↵Erik Eckstein2014-08-281-0/+41
| | | | | | | | | | vector instruction. For a detailed description of the problem see the comment in the test file. The problematic moveBefore() calls are not required anymore because the new scheduling algorithm ensures a correct ordering anyway. llvm-svn: 216656
* [SLP] Re-enable vectorization of GEP expressions (re-apply r210342 with a fix).Michael Zolotukhin2014-08-271-0/+41
| | | | llvm-svn: 216549
* Revert r210342 and r210343, add test case for the crasher.Joerg Sonnenberger2014-08-262-41/+19
| | | | | | PR 20642. llvm-svn: 216475
* Allow vectorization of division by uniform power of 2.Karthik Bhat2014-08-251-0/+43
| | | | | | | | This patch adds support to recognize division by uniform power of 2 and modifies the cost table to vectorize division by uniform power of 2 whenever possible. Updates Cost model for Loop and SLP Vectorizer.The cost table is currently only updated for X86 backend. Thanks to Hal, Andrea, Sanjay for the review. (http://reviews.llvm.org/D4971) llvm-svn: 216371
* fix: SLPVectorizer crashes for unreachable blocks containing not schedulable ↵Erik Eckstein2014-08-221-0/+40
| | | | | | | | | | | | instructions. In unreachable blocks it's legal to have instructions like "%x = op %x". Such instuctions are not schedulable. Therefore the SLPVectorizer has to check for unreachable blocks and ignore them. Fixes bug 20646. llvm-svn: 216256
* SLPVectorizer: Use the type of the value loaded/stored to get the ABI alignmentArnold Schwaighofer2014-08-071-1/+29
| | | | | | We were using the pointer type which is incorrect. llvm-svn: 215162
* Teach the SLP Vectorizer that keeping some values live over a callsite can ↵James Molloy2014-08-051-0/+46
| | | | | | | | have a cost. Some types, such as 128-bit vector types on AArch64, don't have any callee-saved registers. So if a value needs to stay live over a callsite, it must be spilled and refilled. This cost is now taken into account. llvm-svn: 214859
* fix bug 20513 - Crash in SLP VectorizerErik Eckstein2014-08-021-0/+47
| | | | llvm-svn: 214638
* SLPVectorizer: improved scheduling algorithm.Erik Eckstein2014-08-013-4/+85
| | | | llvm-svn: 214494
* SLP Vectorizer: Canonicalize tree operands of commutitive binary operands.Chad Rosier2014-07-301-0/+75
| | | | llvm-svn: 214338
* Add Support to Recognize and Vectorize NON SIMD instructions in SLPVectorizer.Karthik Bhat2014-06-201-0/+181
| | | | | | | | | This patch adds support to recognize patterns such as fadd,fsub,fadd,fsub.../add,sub,add,sub... and vectorizes them as vector shuffles if they are profitable. These patterns of vector shuffle can later be converted to instructions such as addsubpd etc on X86. Thanks to Arnold and Hal for the reviews. http://reviews.llvm.org/D4015 llvm-svn: 211339
* Reduce verbiage of lit.local.cfg filesAlp Toker2014-06-095-10/+5
| | | | | | We can just split targets_to_build in one place and make it immutable. llvm-svn: 210496
* Fix typo in a test from r210342.Michael Zolotukhin2014-06-061-1/+1
| | | | llvm-svn: 210343
* [SLP] Enable vectorization of GEP expressions.Michael Zolotukhin2014-06-061-0/+41
| | | | | | | | The use cases look like the following: x->a = y->a + 10 x->b = y->b + 12 llvm-svn: 210342
* Fix PR19657 (scalar loads not combined into vector load)Karthik Bhat2014-06-061-0/+73
| | | | | | | | If we have common uses on separate paths in the tree; process the one with greater common depth first. This makes sure that we do not assume we need to extract a load when it is actually going to be part of a vectorized tree. Review: http://reviews.llvm.org/D3800 llvm-svn: 210310
* Allow vectorization of intrinsics such as powi,cttz and ctlz in Loop and SLP ↵Karthik Bhat2014-05-301-0/+267
| | | | | | | | | | Vectorizer. This patch adds support to vectorize intrinsics such as powi, cttz and ctlz in Vectorizer. These intrinsics are different from other intrinsics as second argument to these function must be same in order to vectorize them and it should be represented as a scalar. Review: http://reviews.llvm.org/D3851#inline-32769 and http://reviews.llvm.org/D3937#inline-32857 llvm-svn: 209873
* AArch64/ARM64: move ARM64 into AArch64's placeTim Northover2014-05-242-1/+1
| | | | | | | | | | | | | | | This commit starts with a "git mv ARM64 AArch64" and continues out from there, renaming the C++ classes, intrinsics, and other target-local objects for consistency. "ARM64" test directories are also moved, and tests that began their life in ARM64 use an arm64 triple, those from AArch64 use an aarch64 triple. Both should be equivalent though. This finishes the AArch64 merge, and everyone should feel free to continue committing as normal now. llvm-svn: 209577
* Implement sext(C1 + C2*X) --> sext(C1) + sext(C2*X) andMichael Zolotukhin2014-05-241-0/+175
| | | | | | | | | | | sext{C1,+,C2} --> sext(C1) + sext{0,+,C2} transformation in Scalar Evolution. That helps SLP-vectorizer to recognize consecutive loads/stores. <rdar://problem/14860614> llvm-svn: 209568
* Fix broken FileCheck prefixesNico Rieck2014-05-231-1/+1
| | | | llvm-svn: 209538
* Flip on vectorization of bswap intrinsics.Benjamin Kramer2014-05-191-0/+44
| | | | | | | | | The cost model conservatively assumes that it will always get scalarized and that's about as good as we can get with the generic TTI; reasoning whether a shuffle with an efficient lowering is available is hard. We can override that conservative estimate for some targets in the future. llvm-svn: 209125
* SLPVectorizer: When sorting by domination for CSE don't assert on ↵Benjamin Kramer2014-05-091-0/+30
| | | | | | | | | | | | unreachable code. There is no total ordering if the CFG is disconnected. We don't care if we catch all CSE opportunities in dead code either so just exclude ignore them in the assert. PR19646 llvm-svn: 208461
* Always set alignment of vectorized LD/ST in SLP-Vectorizer. ↵Yi Jiang2014-05-051-0/+27
| | | | | | <rdar://problem/16812145> llvm-svn: 207983
* SLPVectorizer: Bring back the insertelement patch (r205965) with fixesArnold Schwaighofer2014-05-042-0/+140
| | | | | | | | | | | | | | | | When can't assume a vectorized tree is rooted in an instruction. The IRBuilder could have constant folded it. When we rebuild the build_vector (the series of InsertElement instructions) use the last original InsertElement instruction. The vectorized tree root is guaranteed to be before it. Also, we can't assume that the n-th InsertElement inserts the n-th element into a vector. This reverts r207746 which reverted the revert of the revert of r205018 or so. Fixes the test case in PR19621. llvm-svn: 207939
* Vectorize intrinsic math function calls in SLPVectorizer.Karthik Bhat2014-05-031-0/+128
| | | | | | | This patch adds support to recognize and vectorize intrinsic math functions in SLPVectorizer. Review: http://reviews.llvm.org/D3560 and http://reviews.llvm.org/D3559 llvm-svn: 207901
* Revert r205965, which essentially reverts r205018 for the second time.Chandler Carruth2014-05-011-24/+0
| | | | | | | | | | | | | | | | =[ Turns out that this was the root cause of PR19621. We found a crasher only recently (likely due to improvements elsewhere in the SLP vectorizer) but the reduced test case failed all the way back to here. I've confirmed that reverting this patch both fixes the reduced test case in PR19621 and the actual source file that led to it, so it seems to really be rooted here. I've replied to the commit thread with discussion of my (feeble) attempts to debug this. Didn't make it very far, so reverting now that we have a good test case so that things can get back to healthy while the debugging carries on. llvm-svn: 207746
* Continue slp vectorization even the BB already has vectorized store ↵Yi Jiang2014-04-291-0/+31
| | | | | | radar://16641956 llvm-svn: 207572
* Reapply "SLPVectorizer: Ignore users that are insertelements we can ↵Arnold Schwaighofer2014-04-101-0/+24
| | | | | | | | | reschedule them" This commit reapplies 205018. After 205855 we should correctly vectorize intrinsics. llvm-svn: 205965
* SLPVectorizer: Only vectorize intrinsics whose operands are widened equallyArnold Schwaighofer2014-04-091-0/+36
| | | | | | | | | The vectorizer only knows how to vectorize intrinics by widening all operands by the same factor. Patch by Tyler Nowicki! llvm-svn: 205855
* SLPVectorizer: compare entire intrinsic for SLP compatibility.Tim Northover2014-04-022-0/+21
| | | | | | | | | Some Intrinsics are overloaded to the extent that return type equality (all that's been checked up to now) does not guarantee that the arguments are the same. In these cases SLP vectorizer should not recurse into the operands, which can be achieved by comparing them as "Function *" rather than simply the ID. llvm-svn: 205424
* Revert "SLPVectorizer: Ignore users that are insertelements we can ↵Arnold Schwaighofer2014-03-311-24/+0
| | | | | | | | | | | | | | reschedule them" This reverts commit r205018. Conflicts: lib/Transforms/Vectorize/SLPVectorizer.cpp test/Transforms/SLPVectorizer/X86/insert-element-build-vector.ll This is breaking libclc build. llvm-svn: 205260
* SLPVectorizer: Take credit for free extractelement instructionsArnold Schwaighofer2014-03-281-0/+25
| | | | | | | | | Extract element instructions that will be removed when vectorzing lower the cost. Patch by Arch D. Robison! llvm-svn: 205020
* SLPVectorizer: Ignore users that are insertelements we can reschedule themArnold Schwaighofer2014-03-281-0/+24
| | | | | | Patch by Arch D. Robison! llvm-svn: 205018
* Resubmit "[SLPV] Recognize vectorizable intrinsics during SLP vectorization ..."Raul E. Silvera2014-03-121-0/+75
| | | | | | | This reverts commit 86cb795388643710dab34941ddcb5a9470ac39d8. The problems previously found have been resolved through other CLs. llvm-svn: 203707
* Trivial test commit.Raul E. Silvera2014-03-051-0/+1
| | | | llvm-svn: 202924
* SLPVectorizer: Try vectorizing 'splat' storesArnold Schwaighofer2014-02-241-0/+15
| | | | | | | | | Vectorize sequential stores of a broadcasted value. 5% on eon. radar://16124699 llvm-svn: 202067
* fix for null VectorizedValue assertion in the SLP Vectorizer (in function ↵Gerolf Hoflehner2014-02-171-0/+65
| | | | | | vectorizeTree()). radar://16064178 llvm-svn: 201501
* Revert "[SLPV] Recognize vectorizable intrinsics during SLP vectorization ..."Reid Kleckner2014-02-011-75/+0
| | | | | | | This reverts commit r200576. It broke 32-bit self-host builds by vectorizing two calls to @llvm.bswap.i64, which we then fail to expand. llvm-svn: 200602
* [SLPV] Recognize vectorizable intrinsics during SLP vectorization andChandler Carruth2014-01-311-0/+75
| | | | | | | | | | transform accordingly. Based on similar code from Loop vectorization. Subsequent commits will include vectorization of function calls to vector intrinsics and form function calls to vector library calls. Patch by Raul Silvera! (Much delayed due to my not running dcommit) llvm-svn: 200576
* Fix broken CHECK lines.Benjamin Kramer2014-01-111-1/+1
| | | | llvm-svn: 199016
* SLPVectorizer: An in-tree vectorized entry cannot also be a scalar external useArnold Schwaighofer2013-12-051-0/+35
| | | | | | | | | | | | | | | We were creating external uses for scalar values in MustGather entries that also had a ScalarToTreeEntry (they also are present in a vectorized tuple). This meant we would keep a value 'alive' as a scalar and vectorized causing havoc. This is not necessary because when we create a MustGather vector we explicitly create external uses entries for the insertelement instructions of the MustGather vector elements. Fixes PR18129. radar://15582184 llvm-svn: 196508
* opt: Mirror vectorization presets of clangArnold Schwaighofer2013-12-031-0/+30
| | | | | | | | | | clang enables vectorization at optimization levels > 1 and size level < 2. opt should behave similarily. Loop vectorization and SLP vectorization can be disabled with the flags -disable-(loop/slp)-vectorization. llvm-svn: 196294
* PR1860 - We can't save a list of ExtractElement instructions to CSE because ↵Nadav Rotem2013-11-261-1/+1
| | | | | | | | some of these instructions may be removed and optimized in future iterations. Instead we save a list of basic blocks that we need to CSE. llvm-svn: 195791
* PR18060 - When we RAUW values with ExtractElement instructions in some casesNadav Rotem2013-11-261-0/+47
| | | | | | | | we generate PHI nodes with multiple entries from the same basic block but with different values. Enabling CSE on ExtractElement instructions make sure that all of the RAUWed instructions are the same. llvm-svn: 195773
* Add the test case that I missed when committing r195528. Doh!Chandler Carruth2013-11-251-0/+60
| | | | llvm-svn: 195691
* Debug Info: update testing cases to specify the debug info version number.Manman Ren2013-11-221-1/+2
| | | | | | | | We are going to drop debug info without a version number or with a different version number, to make sure we don't crash when we see bitcode files with different debug info metadata format. llvm-svn: 195504
* SLP Vectorizer: Extract cost will only be added once even if the scalar has ↵Yi Jiang2013-11-221-0/+30
| | | | | | multiple external uses. llvm-svn: 195406
* Add llvm/test/Transforms/SLPVectorizer/ARM/lit.local.cfg. Tests there ↵NAKAMURA Takumi2013-10-291-0/+3
| | | | | | require ARM in targets. llvm-svn: 193580
* ARM cost model: Unaligned vectorized double stores are expensiveArnold Schwaighofer2013-10-291-0/+20
| | | | | | | | | Updated a test case that assumed that <2 x double> would vectorize to use <4 x float>. radar://15338229 llvm-svn: 193574
OpenPOWER on IntegriCloud