summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/Vectorize
Commit message (Collapse)AuthorAgeFilesLines
* LoopVectorize: Handle loops with multiple forward inductionsArnold Schwaighofer2013-05-141-17/+40
| | | | | | | | | | | | We used to give up if we saw two integer inductions. After this patch, we base further induction variables on the chosen one like we do in the reverse induction and pointer induction case. Fixes PR15720. radar://13851975 llvm-svn: 181746
* Suppress GCC compiler warnings in release builds about variables that are onlyDuncan Sands2013-05-131-0/+1
| | | | | | read in asserts. llvm-svn: 181689
* SLPVectorizer: Swap LHS and RHS. No functionality change.Nadav Rotem2013-05-131-4/+4
| | | | llvm-svn: 181684
* SLPVectorizer: Fix a bug in the code that generates extracts for values with ↵Nadav Rotem2013-05-121-7/+27
| | | | | | | | multiple users. The external user does not have to be in lane #0. We have to save the lane for each scalar so that we know which vector lane to extract. llvm-svn: 181674
* SLPVectorizer: Clear the map that maps between scalars to vectors after each ↵Nadav Rotem2013-05-121-0/+1
| | | | | | | | round of vectorization. Testcase in the next commit. llvm-svn: 181673
* LoopVectorize: Use the widest induction variable typeArnold Schwaighofer2013-05-111-21/+69
| | | | | | | | | | | | | | | | | | | | | | Use the widest induction type encountered for the cannonical induction variable. We used to turn the following loop into an empty loop because we used i8 as induction variable type and truncated 1024 to 0 as trip count. int a[1024]; void fail() { int reverse_induction = 1023; unsigned char forward_induction = 0; while ((reverse_induction) >= 0) { forward_induction++; a[reverse_induction] = forward_induction; --reverse_induction; } } radar://13862901 llvm-svn: 181667
* LoopVectorize: Use variable instead of repeated function callArnold Schwaighofer2013-05-111-3/+4
| | | | | | No functionality change intended. llvm-svn: 181666
* LoopVectorize: Use IRBuilder interface in more placesArnold Schwaighofer2013-05-111-25/+13
| | | | | | No functionality change intended. llvm-svn: 181665
* SLPVectorizer: Add support for trees with external users.Nadav Rotem2013-05-102-9/+55
| | | | | | | | | | | | | For example: bar() { int a = A[i]; int b = A[i+1]; B[i] = a; B[i+1] = b; foo(a); <--- a is used outside the vectorized expression. } llvm-svn: 181648
* Add a debug printNadav Rotem2013-05-101-0/+2
| | | | llvm-svn: 181647
* LoopVectorizer: Don't assert on the absence of induction variablesArnold Schwaighofer2013-05-091-1/+2
| | | | | | | | | A computable loop exit count does not imply the presence of an induction variable. Scalar evolution can return a value for an infinite loop. Fixes PR15926. llvm-svn: 181495
* LoopVectorizer: Improve reduction variable identificationArnold Schwaighofer2013-05-071-84/+132
| | | | | | | The two nested loops were confusing and also conservative in identifying reduction variables. This patch replaces them by a worklist based approach. llvm-svn: 181369
* LoopVectorize: getConsecutiveVector must respect signed arithmeticArnold Schwaighofer2013-05-071-5/+6
| | | | | | | | | | We were passing an i32 to ConstantInt::get where an i64 was needed and we must also pass the sign if we pass negatives numbers. The start index passed to getConsecutiveVector must also be signed. Should fix PR15882. llvm-svn: 181286
* Update the comment to mention that we use TTI.Nadav Rotem2013-05-061-3/+3
| | | | llvm-svn: 181178
* LoopVectorize: Print values instead of pointers in debug output.Benjamin Kramer2013-05-051-4/+4
| | | | llvm-svn: 181157
* LoopVectorize: Add support for floating point min/max reductionsArnold Schwaighofer2013-05-051-22/+69
| | | | | | | | | | Add support for min/max reductions when "no-nans-float-math" is enabled. This allows us to assume we have ordered floating point math and treat ordered and unordered predicates equally. radar://13723044 llvm-svn: 181144
* LoopVectorizer: Cleanup of miminimum/maximum pattern match codeArnold Schwaighofer2013-05-051-2/+2
| | | | | | | | | No need for setting the operands. The pointers are going to be bound by the matcher. radar://13723044 llvm-svn: 181142
* LoopVectorize: We don't need an identity element for min/max reductionsArnold Schwaighofer2013-05-051-32/+19
| | | | | | | | | | We can just use the initial element that feeds the reduction. max(max(x, y), z) == max(max(x,y), max(x,z)) radar://13723044 llvm-svn: 181141
* Add ArrayRef constructor from None, and do the cleanups that this ↵Dmitri Gribenko2013-05-051-1/+1
| | | | | | | | constructor enables Patch by Robert Wilhelm. llvm-svn: 181138
* LoopVectorizer: Add support for if-conversion of PHINodes with 3+ incoming ↵Nadav Rotem2013-05-031-20/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | values. By supporting the vectorization of PHINodes with more than two incoming values we can increase the complexity of nested if statements. We can now vectorize this loop: int foo(int *A, int *B, int n) { for (int i=0; i < n; i++) { int x = 9; if (A[i] > B[i]) { if (A[i] > 19) { x = 3; } else if (B[i] < 4 ) { x = 4; } else { x = 5; } } A[i] = x; } } llvm-svn: 181037
* This patch breaks up Wrap.h so that it does not have to include all of Filip Pizlo2013-05-011-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | the things, and renames it to CBindingWrapping.h. I also moved CBindingWrapping.h into Support/. This new file just contains the macros for defining different wrap/unwrap methods. The calls to those macros, as well as any custom wrap/unwrap definitions (like for array of Values for example), are put into corresponding C++ headers. Doing this required some #include surgery, since some .cpp files relied on the fact that including Wrap.h implicitly caused the inclusion of a bunch of other things. This also now means that the C++ headers will include their corresponding C API headers; for example Value.h must include llvm-c/Core.h. I think this is harmless, since the C API headers contain just external function declarations and some C types, so I don't believe there should be any nasty dependency issues here. llvm-svn: 180881
* Fix a typoNadav Rotem2013-04-301-1/+1
| | | | llvm-svn: 180806
* LoopVectorizer: Calculate the number of pointers to disambiguate at runtime ↵Nadav Rotem2013-04-261-4/+11
| | | | | | based on the numbers of reads and writes. llvm-svn: 180593
* LoopVectorizer: No need to generate pointer disambiguation checks between ↵Nadav Rotem2013-04-251-4/+12
| | | | | | readonly pointers. llvm-svn: 180570
* LoopVectorizer: Change variable name Stride to ConsecutiveStrideArnold Schwaighofer2013-04-241-6/+6
| | | | | | | | This makes it easier to read the code. No functionality change. llvm-svn: 180197
* LoopVectorize: Scalarize padded typesArnold Schwaighofer2013-04-241-1/+9
| | | | | | | | | | | | | | | | | | This patch disables memory-instruction vectorization for types that need padding bytes, e.g., x86_fp80 has 10 bytes store size with 6 bytes padding in darwin on x86_64. Because the load/store vectorization is performed by the bit casting to a packed vector, which has incompatible memory layout due to the lack of padding bytes, the present vectorizer produces inconsistent result for memory instructions of those types. This patch checks an equality of the AllocSize of a scalar type and allocated size for each vector element, to ensure that there is no padding bytes and the array can be read/written using vector operations. Patch by Daisuke Takahashi! Fixes PR15758. llvm-svn: 180196
* LoopVectorizer: Bail out if we don't have datalayout we need itArnold Schwaighofer2013-04-241-0/+5
| | | | llvm-svn: 180195
* LoopVectorizer: Fix 15830. When scalarizing and unrolling stores make sure ↵Nadav Rotem2013-04-231-4/+4
| | | | | | | | that the order in which the elements are scalarized is the same as the original order. This fixes a miscompilation in FreeBSD's regex library. llvm-svn: 180121
* Call the potentially costly isAnnotatedParallel() only once. Pekka Jaaskelainen2013-04-231-3/+5
| | | | | | Made the uniform write test's checks a bit stricter. llvm-svn: 180119
* Refuse to (even try to) vectorize loops which have uniform writes,Pekka Jaaskelainen2013-04-231-9/+9
| | | | | | | | | even if erroneously annotated with the parallel loop metadata. Fixes Bug 15794: "Loop Vectorizer: Crashes with the use of llvm.loop.parallel metadata" llvm-svn: 180081
* Move C++ code out of the C headers and into either C++ headersEric Christopher2013-04-221-0/+1
| | | | | | | or the C++ files themselves. This enables people to use just a C compiler to interoperate with LLVM. llvm-svn: 180063
* SLPVectorize: Add support for vectorization of casts.Nadav Rotem2013-04-211-0/+69
| | | | llvm-svn: 179975
* SLPVectorizer: Fix a bug in the code that scans the tree in search of nodes ↵Nadav Rotem2013-04-211-0/+1
| | | | | | | | with multiple users. We did not terminate the switch case and we executed the search routine twice. llvm-svn: 179974
* Fix PR15800. Do not try to vectorize vectors and structs.Nadav Rotem2013-04-201-1/+10
| | | | llvm-svn: 179960
* VecUtils: Clean up uses of dyn_cast.Benjamin Kramer2013-04-201-4/+4
| | | | llvm-svn: 179936
* SLPVectorizer: Strength reduce SmallVectors to ArrayRefs.Benjamin Kramer2013-04-203-30/+28
| | | | | | Avoids a couple of copies and allows more flexibility in the clients. llvm-svn: 179935
* SLPVectorizer: Reduce the compile time by eliminating the search for some of ↵Nadav Rotem2013-04-201-1/+1
| | | | | | the more expensive patterns. After this change will only check basic arithmetic trees that start at cmpinstr. llvm-svn: 179933
* refactor tryToVectorizePair to a new method that supports vectorization of ↵Nadav Rotem2013-04-201-0/+8
| | | | | | lists. llvm-svn: 179932
* Fix an unused variable warning.Nadav Rotem2013-04-201-0/+1
| | | | llvm-svn: 179931
* SLPVectorizer: Improve the cost model for loop invariant broadcast values.Nadav Rotem2013-04-203-11/+28
| | | | llvm-svn: 179930
* Report the number of stores that were found in the debug message.Nadav Rotem2013-04-201-6/+8
| | | | llvm-svn: 179929
* Fix the header comment.Nadav Rotem2013-04-202-2/+2
| | | | llvm-svn: 179928
* Use 64bit arithmetic for calculating distance between pointers.Nadav Rotem2013-04-201-2/+2
| | | | llvm-svn: 179927
* LoopVectorizer: Use matcher from PatternMatch.h for the min/max patternsArnold Schwaighofer2013-04-191-104/+102
| | | | | | | | | Also make some static function class functions to avoid having to mention the class namespace for enums all the time. No functionality change intended. llvm-svn: 179886
* Fix a -Wdocumentation warningDmitri Gribenko2013-04-181-1/+1
| | | | llvm-svn: 179789
* LoopVectorizer: Recognize min/max reductionsArnold Schwaighofer2013-04-181-34/+209
| | | | | | | | | | | | A min/max operation is represented by a select(cmp(lt/le/gt/ge, X, Y), X, Y) sequence in LLVM. If we see such a sequence we can treat it just as any other commutative binary instruction and reduce it. This appears to help bzip2 by about 1.5% on an imac12,2. radar://12960601 llvm-svn: 179773
* LoopVectorize: Use a set to avoid longer cycles in the reduction chain too.Benjamin Kramer2013-04-181-8/+6
| | | | | | Fixes PR15748. llvm-svn: 179757
* SLPVectorizer: Make it a function pass and add code for hoisting the ↵Nadav Rotem2013-04-153-159/+254
| | | | | | vector-gather sequence out of loops. llvm-svn: 179562
* SLPVectorizer: Add support for vectorizing trees that start at compare ↵Nadav Rotem2013-04-151-21/+40
| | | | | | instructions. llvm-svn: 179504
* Miscellaneous cleanups for VecUtils.hBenjamin Kramer2013-04-141-9/+6
| | | | llvm-svn: 179483
OpenPOWER on IntegriCloud