summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/Vectorize
Commit message (Collapse)AuthorAgeFilesLines
...
* [SLP] Arguments should be camel case, and start with an upper case letter. NFC.Chad Rosier2016-08-311-2/+2
| | | | llvm-svn: 280248
* [LoadStoreVectorizer] Change VectorSet to Vector to match head and tail ↵Alina Sbirlea2016-08-301-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | positions. Resolves PR29148. Summary: LSV was using two vector sets (heads and tails) to track pairs of adjiacent position to vectorize. A recent optimization is trying to obtain the longest chain to vectorize and assumes the positions in heads(H) and tails(T) match, which is not the case is there are multiple tails for the same head. e.g.: i1: store a[0] i2: store a[1] i3: store a[1] Leads to: H: i1 T: i2 i3 Instead of: H: i1 i1 T: i2 i3 So the positions for instructions that follow i3 will have different indexes in H/T. This patch resolves PR29148. This issue also surfaced the fact that if the chain is too long, and TLI returns a "not-fast" answer, the whole chain will be abandoned for vectorization, even though a smaller one would be beneficial. Added a testcase and FIXME for this. Reviewers: tstellarAMD, arsenm, jlebar Subscribers: mzolotukhin, wdng, llvm-commits Differential Revision: https://reviews.llvm.org/D24057 llvm-svn: 280179
* [LoopVectorizer] Predicate instructions in blocks with several incoming edgesMichael Kuperstein2016-08-301-12/+4
| | | | | | | | | | We don't need to limit predication to blocks that have a single incoming edge, we just need to use the right mask. This fixes PR30172. Differential Revision: https://reviews.llvm.org/D24009 llvm-svn: 280148
* ADT: Give ilist<T>::reverse_iterator a handle to the current nodeDuncan P. N. Exon Smith2016-08-301-5/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Reverse iterators to doubly-linked lists can be simpler (and cheaper) than std::reverse_iterator. Make it so. In particular, change ilist<T>::reverse_iterator so that it is *never* invalidated unless the node it references is deleted. This matches the guarantees of ilist<T>::iterator. (Note: MachineBasicBlock::iterator is *not* an ilist iterator, but a MachineInstrBundleIterator<MachineInstr>. This commit does not change MachineBasicBlock::reverse_iterator, but it does update MachineBasicBlock::reverse_instr_iterator. See note at end of commit message for details on bundle iterators.) Given the list (with the Sentinel showing twice for simplicity): [Sentinel] <-> A <-> B <-> [Sentinel] the following is now true: 1. begin() represents A. 2. begin() holds the pointer for A. 3. end() represents [Sentinel]. 4. end() holds the poitner for [Sentinel]. 5. rbegin() represents B. 6. rbegin() holds the pointer for B. 7. rend() represents [Sentinel]. 8. rend() holds the pointer for [Sentinel]. The changes are #6 and #8. Here are some properties from the old scheme (which used std::reverse_iterator): - rbegin() held the pointer for [Sentinel] and rend() held the pointer for A; - operator*() cost two dereferences instead of one; - converting from a valid iterator to its valid reverse_iterator involved a confusing increment; and - "RI++->erase()" left RI invalid. The unintuitive replacement was "RI->erase(), RE = end()". With vector-like data structures these properties are hard to avoid (since past-the-beginning is not a valid pointer), and don't impose a real cost (since there's still only one dereference, and all iterators are invalidated on erase). But with lists, this was a poor design. Specifically, the following code (which obviously works with normal iterators) now works with ilist::reverse_iterator as well: for (auto RI = L.rbegin(), RE = L.rend(); RI != RE;) fooThatMightRemoveArgFromList(*RI++); Converting between iterator and reverse_iterator for the same node uses the getReverse() function. reverse_iterator iterator::getReverse(); iterator reverse_iterator::getReverse(); Why doesn't iterator <=> reverse_iterator conversion use constructors? In order to catch and update old code, reverse_iterator does not even have an explicit conversion from iterator. It wouldn't be safe because there would be no reasonable way to catch all the bugs from the changed semantic (see the changes at call sites that are part of this patch). Old code used this API: std::reverse_iterator::reverse_iterator(iterator); iterator std::reverse_iterator::base(); Here's how to update from old code to new (that incorporates the semantic change), assuming I is an ilist<>::iterator and RI is an ilist<>::reverse_iterator: [Old] ==> [New] reverse_iterator(I) (--I).getReverse() reverse_iterator(I) ++I.getReverse() --reverse_iterator(I) I.getReverse() reverse_iterator(++I) I.getReverse() RI.base() (--RI).getReverse() RI.base() ++RI.getReverse() --RI.base() RI.getReverse() (++RI).base() RI.getReverse() delete &*RI, RE = end() delete &*RI++ RI->erase(), RE = end() RI++->erase() ======================================= Note: bundle iterators are out of scope ======================================= MachineBasicBlock::iterator, also known as MachineInstrBundleIterator<MachineInstr>, is a wrapper to represent MachineInstr bundles. The idea is that each operator++ takes you to the beginning of the next bundle. Implementing a sane reverse iterator for this is harder than ilist. Here are the options: - Use std::reverse_iterator<MBB::i>. Store a handle to the beginning of the next bundle. A call to operator*() runs a loop (usually operator--() will be called 1 time, for unbundled instructions). Increment/decrement just works. This is the status quo. - Store a handle to the final node in the bundle. A call to operator*() still runs a loop, but it iterates one time fewer (usually operator--() will be called 0 times, for unbundled instructions). Increment/decrement just works. - Make the ilist_sentinel<MachineInstr> *always* store that it's the sentinel (instead of just in asserts mode). Then the bundle iterator can sniff the sentinel bit in operator++(). I initially tried implementing the end() option as part of this commit, but updating iterator/reverse_iterator conversion call sites was error-prone. I have a WIP series of patches that implements the final option. llvm-svn: 280032
* [SLP] Return a boolean value for these static helpers. NFC.Chad Rosier2016-08-291-17/+16
| | | | | | Differential Revision: https://reviews.llvm.org/D24008 llvm-svn: 280020
* [LV] Move insertelement sequence after scalar definitionsMatthew Simpson2016-08-291-0/+11
| | | | | | | | | | | | | | After r279649 when getting a vector value from VectorLoopValueMap, we create an insertelement sequence on-demand if the value has been scalarized instead of vectorized. We previously inserted this insertelement sequence before the value's first vector user. However, this insert location is problematic if that user is the phi node of a first-order recurrence. With this patch, we move the insertelement sequence after the last scalar instruction we created when scalarizing the value. Thus, the value's vector definition in the new loop will immediately follow its scalar definitions. This should fix PR30183. Reference: https://llvm.org/bugs/show_bug.cgi?id=30183 llvm-svn: 280001
* [LV] Unify vector and scalar mapsMatthew Simpson2016-08-241-207/+284
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch unifies the data structures we use for mapping instructions from the original loop to their corresponding instructions in the new loop. Previously, we maintained two distinct maps for this purpose: WidenMap and ScalarIVMap. WidenMap maintained the vector values each instruction from the old loop was represented with, and ScalarIVMap maintained the scalar values each scalarized induction variable was represented with. With this patch, all values created for the new loop are maintained in VectorLoopValueMap. The change allows for several simplifications. Previously, when an instruction was scalarized, we had to insert the scalar values into vectors in order to maintain the mapping in WidenMap. Then, if a user of the scalarized value was also scalar, we had to extract the scalar values from the temporary vector we created. We now aovid these unnecessary scalar-to-vector-to-scalar conversions. If a scalarized value is used by a scalar instruction, the scalar value is used directly. However, if the scalarized value is needed by a vector instruction, we generate the needed insertelement instructions on-demand. A common idiom in several locations in the code (including the scalarization code), is to first get the vector values an instruction from the original loop maps to, and then extract a particular scalar value. This patch adds getScalarValue for this purpose along side getVectorValue as an interface into VectorLoopValueMap. These functions work together to return the requested values if they're available or to produce them if they're not. The mapping has also be made less permissive. Entries can be added to VectorLoopValue map with the new initVector and initScalar functions. getVectorValue has been modified to return a constant reference to the mapped entries. There's no real functional change with this patch; however, in some cases we will generate slightly different code. For example, instead of an insertelement sequence following the definition of an instruction, it will now precede the first use of that instruction. This can be seen in the test case changes. Differential Revision: https://reviews.llvm.org/D23169 llvm-svn: 279649
* [Loop Vectorizer] Support predication of div/remGil Rapaport2016-08-241-73/+234
| | | | | | | | | | div/rem instructions in basic blocks that require predication currently prevent vectorization. This patch extends the existing mechanism for predicating stores to handle other instructions and leverages it to predicate divs and rems. Differential Revision: https://reviews.llvm.org/D22918 llvm-svn: 279620
* [SLP] Avoid signed integer overflowMatthew Simpson2016-08-231-9/+35
| | | | | | | | | | | | | | | | | | | The test case included with r279125 exposed an existing signed integer overflow. Since getTreeCost can return INT_MAX, we can't sum this cost together with other costs, such as getReductionCost. This patch removes the possibility of assigning a cost of INT_MAX. Since we were previously using INT_MAX as an indicator for "should not vectorize", we now explicitly check this condition with "isTreeTinyAndNotFullyVectorizable" before computing a cost. This patch adds a run-line to the test case used for r279125 that ensures we don't vectorize. Previously, this line would vectorize the test case by chance due to undefined behavior in the cost calculation. Differential Revision: https://reviews.llvm.org/D23723 llvm-svn: 279562
* Reapply "[SLP] Initialize VectorizedValue when gathering"Matthew Simpson2016-08-201-8/+66
| | | | | | | | | | | The test case included in r279125 exposed existing undefined behavior in the SLP vectorizer that it did not introduce. This patch reapplies the original patch, but modifies the test case to avoid hitting the undefined behavior. This allows us to close PR28330 while keeping the UBSan bot happy. The undefined behavior the original test uncovered will be addressed in a follow-on patch. Reference: https://llvm.org/bugs/show_bug.cgi?id=28330 llvm-svn: 279370
* [SLP] Add command line option for minimum tree size (NFC)Matthew Simpson2016-08-201-1/+5
| | | | llvm-svn: 279369
* Revert "[SLP] Initialize VectorizedValue when gathering" to fix ubsan bot.Vitaly Buka2016-08-201-66/+8
| | | | | | | | This reverts commit r279125. https://reviews.llvm.org/D23410 llvm-svn: 279363
* [LoopVectorize] Don't copy std::vector in for-range loop.Benjamin Kramer2016-08-191-1/+1
| | | | llvm-svn: 279233
* [SLP] Initialize VectorizedValue when gatheringMatthew Simpson2016-08-181-8/+66
| | | | | | | | | | | | | | | | | We abort building vectorizable trees in some cases (e.g., if the maximum recursion depth is reached, if the region size is too large, etc.). If this happens for a reduction, we can be left with a root entry that needs to be gathered. For these cases, we need make sure we actually set VectorizedValue to the resulting vector. This patch ensures we properly set VectorizedValue, and it also ensures the insertelement sequence generated for the gathers is inserted at the correct location. Reference: https://llvm.org/bugs/show_bug.cgi?id=28330 Differential Revison: https://reviews.llvm.org/D23410 llvm-svn: 279125
* [LV] Move LoopBodyTraits to a better place, and add comment for simplifying ↵Tim Shen2016-08-171-57/+0
| | | | | | | | | | | | | | LoopBlocksTraversal. NFC. Summary: I later (after r278573) found that LoopIterator.h has some overlapping with LoopBodyTraits. It's good to use LoopBodyTraits because a *Traits struct is algorithm independent. Reviewers: anemet, nadav, mkuper Subscribers: mzolotukhin, llvm-commits Differential Revision: https://reviews.llvm.org/D23529 llvm-svn: 278996
* [LSV] Use a set rather than an ArraySlice at the end of ↵Justin Lebar2016-08-131-6/+5
| | | | | | | | | | | | | | getVectorizablePrefix. NFC Summary: This avoids a small O(n^2) loop. Reviewers: asbirlea Subscribers: mzolotukhin, llvm-commits, arsenm Differential Revision: https://reviews.llvm.org/D23473 llvm-svn: 278581
* [LSV] Use OrderedBasicBlock instead of rolling it ourselves. NFCJustin Lebar2016-08-131-23/+21
| | | | | | | | | | | | | | | | | | | Summary: In getVectorizablePrefix, this is less efficient (because we have to iterate over the BB twice), but boy is it simpler. Given how much trouble we've had here, I think the simplicity gain is worthwhile. In reorder(), this is actually more efficient, as DominatorTree::dominates iterates over the BB from the beginning when the two instructions are in the same BB. Reviewers: asbirlea Subscribers: arsenm, llvm-commits, mzolotukhin Differential Revision: https://reviews.llvm.org/D23472 llvm-svn: 278580
* [LoopVectorize] Detect loops in the innermost loop before creating ↵Tim Shen2016-08-121-6/+87
| | | | | | | | | | | | | InnerLoopVectorizer InnerLoopVectorizer shouldn't handle a loop with cycles inside the loop body, even if that cycle isn't a natural loop. Fixes PR28541. Differential Revision: https://reviews.llvm.org/D22952 llvm-svn: 278573
* Use the range variant of find/find_if instead of unpacking begin/endDavid Majnemer2016-08-122-6/+3
| | | | | | | | | If the result of the find is only used to compare against end(), just use is_contained instead. No functionality change is intended. llvm-svn: 278469
* Use the range variant of find instead of unpacking begin/endDavid Majnemer2016-08-111-5/+3
| | | | | | | | | If the result of the find is only used to compare against end(), just use is_contained instead. No functionality change is intended. llvm-svn: 278433
* Use range algorithms instead of unpacking begin/endDavid Majnemer2016-08-111-1/+1
| | | | | | No functionality change is intended. llvm-svn: 278417
* [SLP] Make RecursionMaxDepth a command line option (NFC)Matthew Simpson2016-08-111-2/+3
| | | | llvm-svn: 278343
* Move helpers into anonymous namespaces. NFC.Benjamin Kramer2016-08-061-1/+2
| | | | llvm-svn: 277916
* [LV, X86] Be more optimistic about vectorizing shifts.Michael Kuperstein2016-08-041-1/+3
| | | | | | | | | | | | | | | Shifts with a uniform but non-constant count were considered very expensive to vectorize, because the splat of the uniform count and the shift would tend to appear in different blocks. That made the splat invisible to ISel, and we'd scalarize the shift at codegen time. Since r201655, CodeGenPrepare sinks those splats to be next to their use, and we are able to select the appropriate vector shifts. This updates the cost model to to take this into account by making shifts by a uniform cheap again. Differential Revision: https://reviews.llvm.org/D23049 llvm-svn: 277782
* LoadStoreVectorizer: Remove TargetBaseAlign. Keep alignment for stack ↵Alina Sbirlea2016-08-041-11/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | adjustments. Summary: TargetBaseAlign is no longer required since LSV checks if target allows misaligned accesses. A constant defining a base alignment is still needed for stack accesses where alignment can be adjusted. Previous patch (D22936) was reverted because tests were failing. This patch also fixes the cause of those failures: - x86 failing tests either did not have the right target, or the right alignment. - NVPTX failing tests did not have the right alignment. - AMDGPU failing test (merge-stores) should allow vectorization with the given alignment but the target info considers <3xi32> a non-standard type and gives up early. This patch removes the condition and only checks for a maximum size allowed and relies on the next condition checking for %4 for correctness. This should be revisited to include 3xi32 as a MVT type (on arsenm's non-immediate todo list). Note that checking the sizeInBits for a MVT is undefined (leads to an assertion failure), so we need to create an EVT, hence the interface change in allowsMisaligned to include the Context. Reviewers: arsenm, jlebar, tstellarAMD Subscribers: jholewinski, arsenm, mzolotukhin, llvm-commits Differential Revision: https://reviews.llvm.org/D23068 llvm-svn: 277735
* [Loop Vectorizer] Move store-predication into its own function, remove ↵Gil Rapaport2016-08-031-12/+18
| | | | | | | | obsolete comment (NFC) Differential Revision: https://reviews.llvm.org/D23013 llvm-svn: 277595
* [LoopVectorize] Change comment for isOutOfScope in collectLoopUniforms, NFCWei Mi2016-08-021-3/+4
| | | | | | | | | Update comment for isOutOfScope and add a testcase for uniform value being used out of scope. Differential Revision: https://reviews.llvm.org/D23073 llvm-svn: 277515
* [LV] Generate both scalar and vector integer induction variablesMatthew Simpson2016-08-021-34/+66
| | | | | | | | | | | | | | | | This patch enables the vectorizer to generate both scalar and vector versions of an integer induction variable for a given loop. Previously, we only generated a scalar induction variable if we knew all its users were going to be scalar. Otherwise, we generated a vector induction variable. In the case of a loop with both scalar and vector users of the induction variable, we would generate the vector induction variable and extract scalar values from it for the scalar users. With this patch, we now generate both versions of the induction variable when there are both scalar and vector users and select which version to use based on whether the user is scalar or vector. Differential Revision: https://reviews.llvm.org/D22869 llvm-svn: 277474
* [LV] Untangle the concepts of uniform and scalarMatthew Simpson2016-08-021-63/+107
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch refactors the logic in collectLoopUniforms and collectValuesToIgnore, untangling the concepts of "uniform" and "scalar". It adds isScalarAfterVectorization along side isUniformAfterVectorization to distinguish the two. Known scalar values include those that are uniform, getelementptr instructions that won't be vectorized, and induction variables and induction variable update instructions whose users are all known to be scalar. This patch includes the following functional changes: - In collectLoopUniforms, we mark uniform the pointer operands of interleaved accesses. Although non-consecutive, these pointers are treated like consecutive pointers during vectorization. - In collectValuesToIgnore, we insert a value into VecValuesToIgnore if it isScalarAfterVectorization rather than isUniformAfterVectorization. This differs from the previous functionaly in that we now add getelementptr instructions that will not be vectorized into VecValuesToIgnore. This patch also removes the ValuesNotWidened set used for induction variable scalarization since, after the above changes, it is now equivalent to isScalarAfterVectorization. Differential Revision: https://reviews.llvm.org/D22867 llvm-svn: 277460
* [LoadStoreVectorizer] Don't use a linear walk for an existence check in a ↵Benjamin Kramer2016-08-021-1/+1
| | | | | | | | SmallPtrSet No functionality change intended. llvm-svn: 277436
* [LV] Move isGatherOrScatterLegal into LoopVectorizationLegality (NFC)Matthew Simpson2016-08-011-15/+13
| | | | llvm-svn: 277376
* [LV] Use getPointerOperand helper where appropriate (NFC)Matthew Simpson2016-08-011-27/+21
| | | | llvm-svn: 277375
* Revert r277038 until clearing why tests fail.Alina Sbirlea2016-07-281-9/+10
| | | | llvm-svn: 277039
* Remove TargetBaseAlign. Keep alignment for stack adjustments.Alina Sbirlea2016-07-281-10/+9
| | | | | | | | | | | | | | Summary: TargetBaseAlign is no longer required since LSV checks if target allows misaligned accesses. A constant defining a base alignment is still needed for stack accesses where alignment can be adjusted. Reviewers: llvm-commits, jlebar Subscribers: mzolotukhin, arsenm Differential Revision: https://reviews.llvm.org/D22936 llvm-svn: 277038
* Fix the assertion error in collectLoopUniforms caused by empty Worklist ↵Wei Mi2016-07-271-2/+2
| | | | | | | | | | before expanding. Contributed-by: David Callahan Differential Revision: https://reviews.llvm.org/D22886 llvm-svn: 276943
* [LSV] Use Instruction*s rather than Value*s where possible.Justin Lebar2016-07-271-87/+94
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: Given the crash in D22878, this patch converts the load/store vectorizer to use explicit Instruction*s wherever possible. This is an overall simplification and should be an improvement in safety, as we have fewer naked cast<>s, and now where we use Value*, we really mean something different from Instruction*. This patch also gets rid of some cast<>s around Value*s returned by Builder. Given that Builder constant-folds everything, we can't assume much about what we get out of it. One downside of this patch is that we have to copy our chain before calling propagateMetadata. But I don't think this is a big deal, as our chains are very small (usually 2 or 4 elems). Reviewers: asbirlea Subscribers: mzolotukhin, llvm-commits, arsenm Differential Revision: https://reviews.llvm.org/D22887 llvm-svn: 276938
* [LSV] Don't assume that bitcast ops are Instructions.Justin Lebar2016-07-271-9/+6
| | | | | | | | | | | | | | Summary: When we ask the builder to create a bitcast on a constant, we get back a constant, not an instruction. Reviewers: asbirlea Subscribers: jholewinski, mzolotukhin, llvm-commits, arsenm Differential Revision: https://reviews.llvm.org/D22878 llvm-svn: 276922
* [Loop Vectorizer] Handling loops FP induction variables.Elena Demikhovsky2016-07-241-32/+103
| | | | | | | | | | | | | | | | Allowed loop vectorization with secondary FP IVs. Like this: float *A; float x = init; for (int i=0; i < N; ++i) { A[i] = x; x -= fp_inc; } The auto-vectorization is possible when the induction binary operator is "fast" or the function has "unsafe" attribute. Differential Revision: https://reviews.llvm.org/D21330 llvm-svn: 276554
* [SLPVectorizer] Vectorize reverse-order loads in horizontal reductionsMichael Kuperstein2016-07-221-16/+53
| | | | | | | | | | | | | | | | | | When vectorizing a tree rooted at a store bundle, we currently try to sort the stores before building the tree, so that the stores can be vectorized. For other trees, the order of the root bundle - which determines the order of all other bundles - is arbitrary. That is bad, since if a leaf bundle of consecutive loads happens to appear in the wrong order, we will not vectorize it. This is partially mitigated when the root is a binary operator, by trying to build a "reversed" tree when that's considered profitable. This patch extends the workaround we have for binops to trees rooted in a horizontal reduction. This fixes PR28474. Differential Revision: https://reviews.llvm.org/D22554 llvm-svn: 276477
* [LV] Move vector int induction update to end of latchMatthew Simpson2016-07-211-3/+12
| | | | | | | | | | | This patch moves the update instruction for vectorized integer induction phi nodes to the end of the latch block. This ensures consistent placement of all induction updates across all the kinds of int inductions we create (scalar, splat vector, or vector phi). Differential Revision: https://reviews.llvm.org/D22416 llvm-svn: 276339
* [OptDiag,LV] Add hotness attribute to applied-optimization remarksAdam Nemet2016-07-211-7/+7
| | | | | | | Test coverage is provided by modifying the function in the FP-math testcase that we are allowed to vectorize. llvm-svn: 276223
* [OptDiag,LV] Add hotness attribute to the derived analysis remarksAdam Nemet2016-07-201-7/+11
| | | | | | | | This includes FPCompute and Aliasing. Testcase is based on no_fpmath.ll. llvm-svn: 276211
* [OptDiag,LV] Add hotness attribute to analysis remarksAdam Nemet2016-07-201-47/+53
| | | | | | | | The earlier change added hotness attribute to missed-optimization remarks. This follows up with the analysis remarks (the ones explaining the reason for the missed optimization). llvm-svn: 276192
* [LSV] Don't move stores across may-load instrs, and loosen restrictions on ↵Justin Lebar2016-07-201-2/+18
| | | | | | | | | | | | | | | | | | | | | moving loads. Summary: Previously we wouldn't move loads/stores across instructions that had side-effects, where that was defined as may-write or may-throw. But this is not sufficiently restrictive: Stores can't safely be moved across instructions that may load. This patch also adds a DEBUG check that all instructions in our chain are either loads or stores. Reviewers: asbirlea Subscribers: llvm-commits, jholewinski, arsenm, mzolotukhin Differential Revision: https://reviews.llvm.org/D22547 llvm-svn: 276171
* [LSV] Vectorize up to side-effecting instructions.Justin Lebar2016-07-201-7/+3
| | | | | | | | | | | | | | | Summary: Previously if we had a chain that contained a side-effecting instruction, we wouldn't vectorize it at all. Now we'll vectorize everything that comes before the side-effecting instruction. Reviewers: asbirlea Subscribers: arsenm, jholewinski, llvm-commits, mzolotukhin Differential Revision: https://reviews.llvm.org/D22536 llvm-svn: 276170
* [LV] Add hotness attribute to missed-optimization remarksAdam Nemet2016-07-201-11/+17
| | | | | | | The new OptimizationRemarkEmitter analysis pass is hooked up to both new and old PM passes. llvm-svn: 276080
* [LSV] Don't assume that loads/stores appear in address order in the BB.Justin Lebar2016-07-201-20/+40
| | | | | | | | | | | | | | | | Summary: getVectorizablePrefix previously didn't work properly in the face of aliasing loads/stores. It unwittingly assumed that the loads/stores appeared in the BB in address order. If they didn't, it would do the wrong thing. Reviewers: asbirlea, tstellarAMD Subscribers: arsenm, llvm-commits, mzolotukhin Differential Revision: https://reviews.llvm.org/D22535 llvm-svn: 276072
* [LSV] Insert stores at the right point.Justin Lebar2016-07-191-30/+28
| | | | | | | | | | | | | | | | | | | | | Summary: Previously, the insertion point for stores was the last instruction in Chain *before calling getVectorizablePrefixEndIdx*. Thus if getVectorizablePrefixEndIdx didn't return Chain.size(), we still would insert at the last instruction in Chain. This patch changes our internal API a bit in an attempt to make it less prone to this sort of error. As a result, we end up recalculating the Chain's boundary instructions, but I think worrying about the speed hit of this is a premature optimization right now. Reviewers: asbirlea, tstellarAMD Subscribers: mzolotukhin, arsenm, llvm-commits Differential Revision: https://reviews.llvm.org/D22534 llvm-svn: 276056
* [LSV] Use make_range, and reformat a DEBUG message. NFCJustin Lebar2016-07-191-12/+15
| | | | | | | | | | | | | | | Summary: The DEBUG message was hard to read because two Values were being printed on the same line with only the delimiter "aliases". This change makes us print each Value on its own line. Reviewers: asbirlea Subscribers: llvm-commits, arsenm, mzolotukhin Differential Revision: https://reviews.llvm.org/D22533 llvm-svn: 276055
* [LSV] Nix two global (ish) variables in the LoadStoreVectorizer. NFCJustin Lebar2016-07-191-10/+12
| | | | | | | | | | Reviewers: asbirlea Subscribers: mzolotukhin, llvm-commits, arsenm Differential Revision: https://reviews.llvm.org/D22532 llvm-svn: 276054
OpenPOWER on IntegriCloud