summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
Commit message (Collapse)AuthorAgeFilesLines
...
* [DAGCombine] Fix a bug in MergeConsecutiveStores.Akira Hatanaka2015-04-081-20/+21
| | | | | | | | | | | | | | | | | | | | | The bug manifests when there are two loads and two stores chained as follows in a DAG, (ld v3f32) -> (st f32) -> (ld v3f32) -> (st f32) and the stores' values are extracted from the preceding vector loads. MergeConsecutiveStores would replace the first store in the chain with the merged vector store, which would create a cycle between the merged store node and the last load node that appears in the chain. This commits fixes the bug by replacing the last store in the chain instead. rdar://problem/20275084 Differential Revision: http://reviews.llvm.org/D8849 llvm-svn: 234430
* [DAGCombiner] Add support for FCEIL, FFLOOR and FTRUNC vector constant foldingSimon Pilgrim2015-04-061-6/+3
| | | | | | Differential Revision: http://reviews.llvm.org/D8715 llvm-svn: 234179
* [DAGCombiner] Merge FMUL Scalar and Vector constant canonicalization to RHS. ↵Simon Pilgrim2015-04-051-8/+2
| | | | | | NFCI. llvm-svn: 234118
* [DAGCombiner] Canonicalize vector constants for ADD/MUL/AND/OR/XOR ↵Simon Pilgrim2015-04-041-6/+11
| | | | | | | | re-association Scalar integers are commuted to move constants to the RHS for re-association - this ensures vectors do the same. llvm-svn: 234092
* [DAGCombiner] Combine shuffles of BUILD_VECTOR and SCALAR_TO_VECTORSimon Pilgrim2015-04-031-0/+37
| | | | | | | | This patch attempts to fold the shuffling of 'scalar source' inputs - BUILD_VECTOR and SCALAR_TO_VECTOR nodes - if the shuffle node is the only user. This folds away a lot of unnecessary shuffle nodes, and allows quite a bit of constant folding that was being missed. Differential Revision: http://reviews.llvm.org/D8516 llvm-svn: 234004
* Fix PR23065. Avoid optimizing bitcast of build_vector with constant input to ↵Jiangning Liu2015-04-011-5/+0
| | | | | | scalar_to_vector. llvm-svn: 233778
* typos; NFCSanjay Patel2015-03-311-5/+5
| | | | llvm-svn: 233701
* Use SDValue bool check to tidyup some possible vector folding ops. NFC.Simon Pilgrim2015-03-291-40/+35
| | | | llvm-svn: 233498
* Use SDValue bool check to tidyup some possible ReassociateOps. NFC.Simon Pilgrim2015-03-291-10/+5
| | | | llvm-svn: 233495
* [DAGCombiner] Fixed incorrect test for buildvector of constant integers.Simon Pilgrim2015-03-281-14/+8
| | | | | | DAGCombiner::ReassociateOps was correctly testing for an constant integer scalar but failed to correctly test for constant integer vectors (it was testing for any constant vector). llvm-svn: 233482
* revert inadvertent changeSanjay Patel2015-03-261-2/+0
| | | | llvm-svn: 233294
* comment cleanup; NFCSanjay Patel2015-03-261-0/+2
| | | | llvm-svn: 233293
* fix indent; NFCSanjay Patel2015-03-261-1/+1
| | | | llvm-svn: 233288
* [DAGCombiner] Add support for TRUNCATE + FP_EXTEND vector constant foldingSimon Pilgrim2015-03-251-53/+23
| | | | | | | | | | This patch adds supports for the vector constant folding of TRUNCATE and FP_EXTEND instructions and tidies up the SINT_TO_FP and UINT_TO_FP instructions to match. It also moves the vector constant folding for the FNEG and FABS instructions to use the DAG.getNode() functionality like the other unary instructions. Differential Revision: http://reviews.llvm.org/D8593 llvm-svn: 233224
* 'optnone' should not disable DAG combiner.Paul Robinson2015-03-251-5/+0
| | | | | | | | | Reverts the code change from r221168 and the relevant test. It was a mistake to disable the combiner, and based on the ultimate definition of 'optnone' we shouldn't have considered the test case as failing in the first place. llvm-svn: 233153
* Move private classes into anonymous namespacesBenjamin Kramer2015-03-231-0/+2
| | | | | | NFC. llvm-svn: 232944
* Fix a nasty bug in DAGCombine of STORE nodes.Owen Anderson2015-03-191-3/+8
| | | | | | | | | | | | | | | This is very related to the bug fixed in r174431. The problem is that SelectionDAG does not include alignment in the uniquing of loads and stores. When an otherwise no-op DAGCombine would increase the alignment of a load or store, the original node would be returned (with the alignment increased), which would cause the node not to be processed by any further DAGCombines. I don't have a direct testcase for this that manifests on an in-tree target, but I did see some noise in the tests for other targets and have updated them for it. llvm-svn: 232780
* DAGCombiner: fold (xor (shl 1, x), -1) -> (rotl ~1, x)David Majnemer2015-03-181-0/+26
| | | | | | | | | | Targets which provide a rotate make it possible to replace a sequence of (XOR (SHL 1, x), -1) with (ROTL ~1, x). This saves an instruction on architectures like X86 and POWER(64). Differential Revision: http://reviews.llvm.org/D8350 llvm-svn: 232572
* XformToShuffleWithZero - Added clearer early outs and general tidy up. NFCISimon Pilgrim2015-03-171-31/+38
| | | | llvm-svn: 232557
* [DAGCombiner] Add a shuffle mask commutation helper function. NFCI.Simon Pilgrim2015-03-071-19/+2
| | | | | | | | | | We have an increasing number of cases where we are creating commuted shuffle masks - all implementing nearly the same code. This patch adds a static helper function - ShuffleVectorSDNode::commuteMask() and replaces a number of cases to use it. Differential Revision: http://reviews.llvm.org/D8139 llvm-svn: 231581
* Use SDValue bool check to tidyup some possible combines. NFC.Simon Pilgrim2015-03-071-6/+5
| | | | llvm-svn: 231569
* [DAGCombiner] Fix wrong folding of AND dag nodes.Andrea Di Biagio2015-03-071-3/+7
| | | | | | | | | | | | | | | | | | | | | | | This patch fixes the logic in the DAGCombiner that folds an AND node according to rule: (and (X (load V)), C) -> (X (load V)) An AND between a vector load 'X' and a constant build_vector 'C' can be folded into the load itself only if we can prove that the AND operation is redundant. The algorithm implemented by 'visitAND' firstly computes the splat value 'S' from C, and then checks if S has the lower 'B' bits set (where B is the size in bits of the vector element type). The algorithm takes into account also the 'undef' bits in the splat mask. Unfortunately, the algorithm only worked under the assumption that the size of S is a multiple of the vector element type. With this patch, we conservatively avoid folding the AND if the splat bits are not compatible with the vector element type. Added X86 test and-load-fold.ll Differential Revision: http://reviews.llvm.org/D8085 llvm-svn: 231563
* [DAGCombiner] SCALAR_TO_VECTOR(EXTRACT_VECTOR_ELT(V,C)) -> VECTOR_SHUFFLESimon Pilgrim2015-03-071-0/+30
| | | | | | | | | | | | This patch attempts to convert a SCALAR_TO_VECTOR using an operand from an EXTRACT_VECTOR_ELT into a VECTOR_SHUFFLE. This prevents many cases of spilling scalar data between the gpr + simd registers. At present the optimization only accepts cases where there is no TRUNC of the scalar type (i.e. all types must match). Differential Revision: http://reviews.llvm.org/D8132 llvm-svn: 231554
* DAGCombiner: Canonicalize select(and/or,x,y) depending on target.Matthias Braun2015-03-061-0/+63
| | | | | | | | | | | | | | | This is based on the following equivalences: select(C0 & C1, X, Y) <=> select(C0, select(C1, X, Y), Y) select(C0 | C1, X, Y) <=> select(C0, X, select(C1, X, Y)) Many target cannot perform and/or on the CPU flags and therefore the right side should be choosen to avoid materializign the i1 flags in an integer register. If the target can perform this operation efficiently we normalize to the left form. Differential Revision: http://reviews.llvm.org/D7622 llvm-svn: 231507
* DAGCombiner: Factor out some and/or combines.Matthias Braun2015-03-061-225/+252
| | | | | | | | | | | | | | | This is in preparation for changing visitSELECT to normalize towards select(Cond0, select(Cond1, X, Y), Y); select(Cond0, X, select(Cond1, X, Y)) which perfom an implicit and/or of the conditions. The factored function contains all DAGCombine rules which reduce two values combined by an And/Or operation to a single value. This does not include rules involving constants as visitSELECT already handles that case. Differential Revision: http://reviews.llvm.org/D8026 llvm-svn: 231506
* [DagCombiner] Allow shuffles to merge through bitcastsSimon Pilgrim2015-03-051-0/+83
| | | | | | | | | | | | Currently shuffles may only be combined if they are of the same type, despite the fact that bitcasts are often introduced in between shuffle nodes (e.g. x86 shuffle type widening). This patch allows a single input shuffle to peek through bitcasts and if the input is another shuffle will merge them, shuffling using the smallest sized type, and re-applying the bitcasts at the inputs and output instead. Dropped old ShuffleToZext test - this patch removes the use of the zext and vector-zext.ll covers these anyhow. Differential Revision: http://reviews.llvm.org/D7939 llvm-svn: 231380
* [DAGCombine] Fix a bug in a BUILD_VECTOR combineMichael Kuperstein2015-03-041-2/+3
| | | | | | | | | | When trying to convert a BUILD_VECTOR into a shuffle, we try to split a single source vector that is twice as wide as the destination vector. We can not do this when we also need the zero vector to create a blend. This fixes PR22774. Differential Revision: http://reviews.llvm.org/D8040 llvm-svn: 231219
* DAGCombiner::LoadedSlice: Remove explicit copy ctor in favor of the Rule of ZeroDavid Blaikie2015-03-031-3/+0
| | | | | | | This way, the copy assignment operator can be used without hitting the deprecated case in C++11. llvm-svn: 231144
* Revert "Remove the explicit SDNodeIterator::operator= in favor of the ↵David Blaikie2015-03-031-0/+3
| | | | | | | | | | | implicit default" Accidentally committed a few more of these cleanup changes than intended. Still breaking these out & tidying them up. This reverts commit r231135. llvm-svn: 231136
* Remove the explicit SDNodeIterator::operator= in favor of the implicit defaultDavid Blaikie2015-03-031-3/+0
| | | | | | | | | | There doesn't seem to be any need to assert that iterator assignment is between iterators over the same node - if you want to reuse an iterator variable to iterate another node, that's perfectly acceptable. Just don't mix comparisons between iterators into disjoint sequences, as usual. llvm-svn: 231135
* avoid infinite looping when folding vector multiplies of constants (PR22698)Sanjay Patel2015-03-011-5/+14
| | | | | | | | | | | | | | | We were missing a check for the following fold in DAGCombiner: // fold (fmul (fmul x, c1), c2) -> (fmul x, (fmul c1, c2)) If 'x' is also a constant, then we shouldn't do anything. Otherwise, we could end up swapping the operands back and forth forever. This should fix: http://llvm.org/bugs/show_bug.cgi?id=22698 Differential Revision: http://reviews.llvm.org/D7917 llvm-svn: 230884
* Convert push_back loops into append calls.Benjamin Kramer2015-02-281-4/+2
| | | | | | No functionality change intended. llvm-svn: 230849
* When the source has a series of assignments, users reasonably want toPaul Robinson2015-02-261-0/+3
| | | | | | | | | | | | have the debugger step through each one individually. Turn off the combine for adjacent stores at -O0 so we get this behavior. Possibly, DAGCombine shouldn't run at all at -O0, but that's for another day; see PR22346. Differential Revision: http://reviews.llvm.org/D7181 llvm-svn: 230659
* Reapplied D7816 & rL230177 & rL230278 - with an additional fix toensure that ↵Simon Pilgrim2015-02-241-45/+65
| | | | | | the smallest build vector input scalar type is always used. Additional (crash) test cases already committed. llvm-svn: 230388
* Revert:Eric Christopher2015-02-241-39/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Author: Simon Pilgrim <llvm-dev@redking.me.uk> Date: Mon Feb 23 23:04:28 2015 +0000 Fix based on post-commit comment on D7816 & rL230177 - BUILD_VECTOR operand truncation was using the the BV's output scalar type instead of the input type. and Author: Simon Pilgrim <llvm-dev@redking.me.uk> Date: Sun Feb 22 18:17:28 2015 +0000 [DagCombiner] Generalized BuildVector Vector Concatenation The CONCAT_VECTORS combiner pass can transform the concat of two BUILD_VECTOR nodes into a single BUILD_VECTOR node. This patch generalises this to support any number of BUILD_VECTOR nodes, and also permits UNDEF nodes to be included as well. This was noticed as AVX vec128 -> vec256 canonicalization sometimes creates a CONCAT_VECTOR with a real vec128 lower and an vec128 UNDEF upper. Differential Revision: http://reviews.llvm.org/D7816 as the root cause of PR22678 which is causing an assertion inside the DAG combiner. I'll follow up to the main thread as well. llvm-svn: 230358
* DAGCombiner: Move variable definitions closer to use; NFCMatthias Braun2015-02-241-38/+38
| | | | llvm-svn: 230354
* DAGCombiner: Move variable declaration closer to definiion; NFCMatthias Braun2015-02-241-3/+3
| | | | llvm-svn: 230353
* Fix based on post-commit comment on D7816 & rL230177 - BUILD_VECTOR operand ↵Simon Pilgrim2015-02-231-4/+5
| | | | | | truncation was using the the BV's output scalar type instead of the input type. llvm-svn: 230278
* [DagCombiner] Generalized BuildVector Vector ConcatenationSimon Pilgrim2015-02-221-23/+38
| | | | | | | | | | | | The CONCAT_VECTORS combiner pass can transform the concat of two BUILD_VECTOR nodes into a single BUILD_VECTOR node. This patch generalises this to support any number of BUILD_VECTOR nodes, and also permits UNDEF nodes to be included as well. This was noticed as AVX vec128 -> vec256 canonicalization sometimes creates a CONCAT_VECTOR with a real vec128 lower and an vec128 UNDEF upper. Differential Revision: http://reviews.llvm.org/D7816 llvm-svn: 230177
* [DAGCombine] Don't assume integer-type legailty in ↵Hal Finkel2015-02-221-0/+5
| | | | | | | | | | | | | | | | | | reduceBuildVecConvertToConvertBuildVec DAGCombine will rewrite an BUILD_VECTOR where all non-undef inputs some from [US]INT_TO_FP, as a BUILD_VECTOR of integers with the conversion applied as a vector operation. We check operation legality of the conversion, but fail to check legality of the integer vector type itself. Because targets don't normally override operation legality defaults for illegal types, we need to check this also. This came up in the context of the QPX vector entensions for PowerPC (which can have legal floating-point vector types without corresponding legal integer vector types). No in-tree test case for this yes, but one can be added once the QPX support has been committed. llvm-svn: 230176
* Add generic fmad DAG node.Matt Arsenault2015-02-201-95/+159
| | | | | | | | | | | This allows sharing of FMA forming combines to work with instructions that have the same semantics as a separate multiply and add. This is expand by default, and only formed post legalization so it shouldn't have much impact on targets that do not want it. llvm-svn: 230070
* [CodeGen] Use ArrayRef instead of std::vector&. NFC.Ahmed Bougacha2015-02-191-1/+1
| | | | | | The former lets us use SmallVectors. Do so in ARM and AArch64. llvm-svn: 229925
* [x86,sdag] Two interrelated changes to the x86 and sdag code.Chandler Carruth2015-02-191-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | First, don't combine bit masking into vector shuffles (even ones the target can handle) once operation legalization has taken place. Custom legalization of vector shuffles may exist for these patterns (making the predicate return true) but that custom legalization may in some cases produce the exact bit math this matches. We only really want to handle this prior to operation legalization. However, the x86 backend, in a fit of awesome, relied on this. What it would do is mark VSELECTs as expand, which would turn them into arithmetic, which this would then match back into vector shuffles, which we would then lower properly. Amazing. Instead, the second change is to teach the x86 backend to directly form vector shuffles from VSELECT nodes with constant conditions, and to mark all of the vector types we support lowering blends as shuffles as custom VSELECT lowering. We still mark the forms which actually support variable blends as *legal* so that the custom lowering is bypassed, and the legal lowering can even be used by the vector shuffle legalization (yes, i know, this is confusing. but that's how the patterns are written). This makes the VSELECT lowering much more sensible, and in fact should fix a bunch of bugs with it. However, as you'll see in the test cases, right now what it does is point out the *hilarious* deficiency of the new vector shuffle lowering when it comes to blends. Fortunately, my very next patch fixes that. I can't submit it yet, because that patch, somewhat obviously, forms the exact and/or pattern that the DAG combine is matching here! Without this patch, teaching the vector shuffle lowering to produce the right code infloops in the DAG combiner. With this patch alone, we produce terrible code but at least lower through the right paths. With both patches, all the regressions here should be fixed, and a bunch of the improvements (like using 2 shufps with no memory loads instead of 2 andps with memory loads and an orps) will stay. Win! There is one other change worth noting here. We had hilariously wrong vectorization cost estimates for vselect because we fell through to the code path that assumed all "expand" vector operations are scalarized. However, the "expand" lowering of VSELECT is vector bit math, most definitely not scalarized. So now we go back to the correct if horribly naive cost of "1" for "not scalarized". If anyone wants to add actual modeling of shuffle costs, that would be cool, but this seems an improvement on its own. Note the removal of 16 and 32 "costs" for doing a blend. Even in SSE2 we can blend in fewer than 16 instructions. ;] Of course, we don't right now because of OMG bad code, but I'm going to fix that. Next patch. I promise. llvm-svn: 229835
* Canonicalize splats as build_vectors (PR22283)Sanjay Patel2015-02-171-13/+10
| | | | | | | | | | | | | | | | | | | | | | This is a follow-on patch to: http://reviews.llvm.org/D7093 That patch canonicalized constant splats as build_vectors, and this patch removes the constant check so we can canonicalize all splats as build_vectors. This fixes the 2nd test case in PR22283: http://llvm.org/bugs/show_bug.cgi?id=22283 The unfortunate code duplication between SelectionDAG and DAGCombiner is discussed in the earlier patch review. At least this patch is just removing code... This improves an existing x86 AVX test and changes codegen in an ARM test. Differential Revision: http://reviews.llvm.org/D7389 llvm-svn: 229511
* Prefer SmallVector::append/insert over push_back loops.Benjamin Kramer2015-02-171-6/+2
| | | | | | Same functionality, but hoists the vector growth out of the loop. llvm-svn: 229500
* SelectionDAG: fold (fp_to_u/sint (s/uint_to_fp)) here tooMehdi Amini2015-02-161-2/+46
| | | | | | | Update SPARC tests to match. From: Fiona Glaser <fglaser@apple.com> llvm-svn: 229438
* [x86] Fix PR22377, a regression with the new vector shuffle legalityChandler Carruth2015-02-151-2/+3
| | | | | | | | | | | test. This was just a matter of the DAG combine for vector shuffles being too aggressive. This is a bit of a grey area, but I think generally if we can re-use intermediate shuffles, we should. Certainly, given the test cases I have available, this seems like the right call. llvm-svn: 229285
* CodeGen: Canonicalize access to function attributes, NFCDuncan P. N. Exon Smith2015-02-141-12/+7
| | | | | | | | | | | | | | | | | Canonicalize access to function attributes to use the simpler API. getAttributes().getAttribute(AttributeSet::FunctionIndex, Kind) => getFnAttribute(Kind) getAttributes().hasAttribute(AttributeSet::FunctionIndex, Kind) => hasFnAttribute(Kind) Also, add `Function::getFnStackAlignment()`, and canonicalize: getAttributes().getStackAlignment(AttributeSet::FunctionIndex) => getFnStackAlignment() llvm-svn: 229208
* MathExtras: Bring Count(Trailing|Leading)Ones and CountPopulation in line ↵Benjamin Kramer2015-02-121-1/+1
| | | | | | | | with countTrailingZeros Update all callers. llvm-svn: 228930
* [CodeGen] Don't blindly combine (fp_round (fp_round x)) to (fp_round x).Ahmed Bougacha2015-02-121-5/+10
| | | | | | | | | | | | We used to do this DAG combine, but it's not always correct: If the first fp_round isn't a value preserving truncation, it might introduce a tie in the second fp_round, that wouldn't occur in the single-step fp_round we want to fold to. In other words, double rounding isn't the same as rounding. Differential Revision: http://reviews.llvm.org/D7571 llvm-svn: 228911
OpenPOWER on IntegriCloud