summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
Commit message (Collapse)AuthorAgeFilesLines
...
* [DAGCombiner] Remove (xor (xor x, c1), c2) -> (xor x, (xor c1, c2)) fold. NFCI.Simon Pilgrim2017-12-211-15/+0
| | | | | | More general cases are already handled by constant canonicalization and then the ReassociateOps call at line 5327 llvm-svn: 321280
* [DAGCombiner] Generalize (or (and X, c1), c2) -> (and (or X, c2), c1|c2) ↵Simon Pilgrim2017-12-211-10/+10
| | | | | | | | combine to work on non-splat vectors The knownbits_mask_or_shuffle_uitofp change is interesting - shuffle combines manage to kick in, removing the AND constant mask load. For targets with fast-variable-shuffle this should reduce further to VPOR+VPSHUFB+VCVTDQ2PS. llvm-svn: 321279
* [DAGCombiner] Generalize (and (or x, C), D) -> D iff (C & D) == D combine to ↵Simon Pilgrim2017-12-211-4/+6
| | | | | | work on non-splat vectors llvm-svn: 321275
* [DAGCombine] Improve ReduceLoadWidth for SRLSam Parker2017-12-211-0/+26
| | | | | | | | | | | If the SRL node is only used by an AND, we may be able to set the ExtVT to the width of the mask, making the AND redundant. To support this, another check has been added in isLegalNarrowLoad which queries whether the load is valid. Differential Revision: https://reviews.llvm.org/D41350 llvm-svn: 321259
* [DAG] Fix condition on overlapping store check.Nirav Dave2017-12-201-2/+2
| | | | | | | Prevent overlapping store elision when overlapping store is pre-inc/dec as analysis is wrong in these cases. llvm-svn: 321204
* Silence a bunch of implicit fallthrough warningsAdrian Prantl2017-12-191-1/+1
| | | | llvm-svn: 321114
* [DAG] Elide overlapping storeNirav Dave2017-12-191-21/+21
| | | | | | | | | | | | | | | | | Summary: Extend overlapping store elision to handle overwrites of stores by larger stores. Nontemporal tests have been modified to add memory dependencies to prevent store elision. Reviewers: craig.topper, rnk, t.p.northover Subscribers: javed.absar, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D40969 llvm-svn: 321089
* [DAGCombine] Move AND nodes to multiple load leavesSam Parker2017-12-181-0/+144
| | | | | | | | | | | | | Search from AND nodes to find whether they can be propagated back to loads, so that the AND and load can be combined into a narrow load. We search through OR, XOR and other AND nodes and all bar one of the leaves are required to be loads or constants. The exception node then needs to be masked off meaning that the 'and' isn't removed, but the loads(s) are narrowed still. Differential Revision: https://reviews.llvm.org/D41177 llvm-svn: 320962
* MachineFunction: Return reference from getFunction(); NFCMatthias Braun2017-12-151-7/+7
| | | | | | The Function can never be nullptr so we can return a reference. llvm-svn: 320884
* Revert "[DAGCombine] Move AND nodes to multiple load leaves"Benjamin Kramer2017-12-141-124/+0
| | | | | | This reverts commit r320679. Causes miscompiles. llvm-svn: 320698
* [DAGCombine] Move AND nodes to multiple load leavesSam Parker2017-12-141-0/+124
| | | | | | | | | | | | | | | | | | | | | | | | Recommitting rL319773, which was reverted due to a recursive issue causing timeouts. This happened because I failed to check whether the discovered loads could be narrowed further. In the case of a tree with one or more narrow loads, that could not be further narrowed, as well as a node that would need masking, an AND could be introduced which could then be visited and recombined again with the same load. This could again create the masking load, with would be combined again... We now check that the load can be narrowed so that this process stops. Original commit message: Search from AND nodes to find whether they can be propagated back to loads, so that the AND and load can be combined into a narrow load. We search through OR, XOR and other AND nodes and all bar one of the leaves are required to be loads or constants. The exception node then needs to be masked off meaning that the 'and' isn't removed, but the loads(s) are narrowed still. Differential Revision: https://reviews.llvm.org/D41177 llvm-svn: 320679
* [DAGCombiner] protect against an infinite loop between shl <--> mul (PR35579)Sanjay Patel2017-12-111-1/+2
| | | | | | | | | | | | | | | | | | | | | | | At first, I tried to thread the x86 needle and use a target hook (isVectorShiftByScalarCheap()) to disable the transform only for non-splat pow-of-2 constants, but not AVX2, but only some element types, but...it's difficult. Here we just avoid the loop with the x86 vector transform that conflicts with the general DAG combine and preserve all of the existing behavior AFAICT otherwise. Some tests that will probably fail if someone does try to restrict this in a more targeted way for x86-only may be found in: test/CodeGen/X86/combine-mul.ll test/CodeGen/X86/vector-mul.ll test/CodeGen/X86/widen_arith-5.ll This should prevent the infinite looping seen with: https://bugs.llvm.org/show_bug.cgi?id=35579 Differential Revision: https://reviews.llvm.org/D41040 llvm-svn: 320374
* [DAGCombiner] Add combined indexed load to the work listNemanja Ivanovic2017-12-111-0/+1
| | | | | | | | | | This commit is the first part of https://reviews.llvm.org/D40348. In order to allow target combines to be performed on newly combined indexed loads, add them back to the worklist. The remainder of the above patch will be committed in subsequent revisions and will use this. Test cases will be included with those follow-up commits. llvm-svn: 320365
* [ARM] Use ADDCARRY / SUBCARRYRoger Ferrer Ibanez2017-12-111-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a preparatory step for D34515. This change: - makes nodes ISD::ADDCARRY and ISD::SUBCARRY legal for i32 - lowering is done by first converting the boolean value into the carry flag using (_, C) ← (ARMISD::ADDC R, -1) and converted back to an integer value using (R, _) ← (ARMISD::ADDE 0, 0, C). An ARMISD::ADDE between the two operations does the actual addition. - for subtraction, given that ISD::SUBCARRY second result is actually a borrow, we need to invert the value of the second operand and result before and after using ARMISD::SUBE. We need to invert the carry result of ARMISD::SUBE to preserve the semantics. - given that the generic combiner may lower ISD::ADDCARRY and ISD::SUBCARRYinto ISD::UADDO and ISD::USUBO we need to update their lowering as well otherwise i64 operations now would require branches. This implies updating the corresponding test for unsigned. - add new combiner to remove the redundant conversions from/to carry flags to/from boolean values (ARMISD::ADDC (ARMISD::ADDE 0, 0, C), -1) → C - fixes PR34045 - fixes PR34564 - fixes PR35103 Differential Revision: https://reviews.llvm.org/D35192 llvm-svn: 320355
* [DAGCombiner] Support folding (mulhs/u X, 0)->0 for vectors.Craig Topper2017-12-111-0/+16
| | | | | | We should probably also fold (mulhs/u X, 1) for vectors, but that's harder. llvm-svn: 320344
* [DAGCombiner] Reuse existing SDLoc variable instead of creating a new one. NFCCraig Topper2017-12-111-4/+3
| | | | llvm-svn: 320343
* [DAGCombiner] eliminate shuffle of insert elementSanjay Patel2017-12-071-0/+81
| | | | | | | | | | | | | | | | | | | I noticed this pattern in D38316 / D38388. We failed to combine a shuffle that is either repeating a scalar insertion at the same position in a vector or translated to a different element index. Like the earlier patch, this could be an instcombine too, but since we opted to make this a DAG transform earlier, I've made this one a DAG patch too. We do not need any legality checking because the new insert is identical to the existing insert except that it may have a different constant insertion operand. The constant insertion test in test/CodeGen/X86/vector-shuffle-combining.ll was the motivation for D38756. Differential Revision: https://reviews.llvm.org/D40209 llvm-svn: 320050
* [ARM][AArch64][DAG] Reenable post-legalize store mergeNirav Dave2017-12-061-12/+19
| | | | | | | | | | | | | | | | | | | | Reenable post-legalize stores with constant merging computation and corresponding test case. * Properly truncate store merge constants * Disable merging of truncated stores floating points * Ensure merges of constant stores into a single vector are constructed from legal elements. Reviewers: eastig, efriedma Reviewed By: eastig Subscribers: spatel, rengolin, aemerson, javed.absar, kristof.beyls, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D40701 llvm-svn: 319899
* Revert "[DAGCombine] Move AND nodes to multiple load leaves"Vlad Tsyrklevich2017-12-061-123/+0
| | | | | | | This reverts commit r319773. It was causing some buildbots to hang, e.g. http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-android/builds/5589 llvm-svn: 319867
* [DAGCombine] Move AND nodes to multiple load leavesSam Parker2017-12-051-0/+123
| | | | | | | | | | | | | Search from AND nodes to find whether they can be propagated back to loads, so that the AND and load can be combined into a narrow load. We search through OR, XOR and other AND nodes and all bar one of the leaves are required to be loads or constants. The exception node then needs to be masked off meaning that the 'and' isn't removed, but the loads(s) are narrowed still. Differential Revision: https://reviews.llvm.org/D39604 llvm-svn: 319773
* [DAGCombine] Handle big endian correctly in CombineConsecutiveLoadsBjorn Pettersson2017-12-051-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Found out, at code inspection, that there was a fault in DAGCombiner::CombineConsecutiveLoads for big-endian targets. A BUILD_PAIR is always having the least significant bits of the composite value in element 0. So when we are doing the checks for consecutive loads, for big endian targets, we should check if the load to elt 1 is at the lower address and the load to elt 0 is at the higher address. Normally this bug only resulted in missed oppurtunities for doing the load combine. I guess that in some rare situation it could lead to faulty combines, but I've not seen that happen. Note that this patch actually will trigger load combine for some big endian regression tests. One example is test/CodeGen/PowerPC/anon_aggr.ll where we now get t76: i64,ch = load<LD8[FixedStack-9] instead of t37: i32,ch = load<LD4[FixedStack-10]> t35: i32,ch = load<LD4[FixedStack-9]> t41: i64 = build_pair t37, t35 before legalization. Then the legalization will split the LD8 into two loads, so the end result is the same. That should verify that the transfomation is correct now. Reviewers: niravd, hfinkel Reviewed By: niravd Subscribers: nemanjai, llvm-commits Differential Revision: https://reviews.llvm.org/D40444 llvm-svn: 319771
* [DAGCombine] isLegalNarrowLoad function (NFC)Sam Parker2017-12-051-42/+60
| | | | | | | | | Pull the checks upon the load out from ReduceLoadWidth into their own function. Differential Revision: https://reviews.llvm.org/D40833 llvm-svn: 319766
* DAG: Follow-up to r319692 check the truncates inputs have the same typeHans Wennborg2017-12-041-1/+2
| | | | | | | | | MatchRotate assumes the types of the types of LHS and RHS are equal, which is always the case then they come from an OR node, but here we're getting them from two different TRUNC nodes, so we have to check the types. llvm-svn: 319695
* DAG: Match truncated rotation (PR35487)Hans Wennborg2017-12-041-0/+9
| | | | | | | | | If the truncation has been pushed past the or-node, look through it and truncate afterwards. Differential revision: https://reviews.llvm.org/D40792 llvm-svn: 319692
* [DAGCombine] Remove isAndLoadExtLoad argumentsSam Parker2017-12-041-14/+6
| | | | | | | | | Both LoadedVT and NarrowLoad are passed as references and neither of them are used by any of its callers. Differential Revision: https://reviews.llvm.org/D40713 llvm-svn: 319645
* [DAG][ARM] Revert "Reenable post-legalize store merge"Nirav Dave2017-12-011-11/+5
| | | | | | due to failures in AArch and ARM code gen. llvm-svn: 319587
* [DAGCombine] Simplify ISD::AND handling in ReduceLoadWidthEli Friedman2017-12-011-20/+5
| | | | | | | | Followup to D39595. Removes a bunch of redundant checks. Differential Revision: https://reviews.llvm.org/D40667 llvm-svn: 319573
* [ARM][DAG] Reenable post-legalize store mergeNirav Dave2017-12-011-5/+11
| | | | | | | | | | | | Summary: Reenable post-legalize stores with constant merging computation and cofrresponding test case. Reviewers: eastig, efriedma Subscribers: aemerson, javed.absar, kristof.beyls, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D40701 llvm-svn: 319547
* [DAGCombine] Refactor ReduceLoadWidthSam Parker2017-11-301-50/+33
| | | | | | | | | | visitAND attempts to narrow the width of extending loads that are then masked off. ReduceLoadWidth already exists for a similar purpose and handles shifts, so I've moved the code to handle AND nodes there. Differential Revision: https://reviews.llvm.org/D39595 llvm-svn: 319421
* Use getStoreSize() in various places instead of 'BitSize >> 3'.Jonas Paulsson2017-11-281-8/+8
| | | | | | | | | | | | | | | | | | This is needed for cases when the memory access is not as big as the width of the data type. For instance, storing i1 (1 bit) would be done in a byte (8 bits). Using 'BitSize >> 3' (or '/ 8') would e.g. give the memory access of an i1 a size of 0, which for instance makes alias analysis return NoAlias even when it shouldn't. There are no tests as this was done as a follow-up to the bugfix for the case where this was discovered (r318824). This handles more similar cases. Review: Björn Petterson https://reviews.llvm.org/D40339 llvm-svn: 319173
* [DAGCombine] Disable finding better chains for stores at O0Simon Dardis2017-11-281-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | Unoptimized IR can have linear sequences of stores to an array, where the initial GEP for the first store is formed from the pointer to the array, and the GEP for each store after the first is formed from the previous GEP with some offset in an inductive fashion. The (large) resulting DAG when analyzed by DAGCombine undergoes an excessive number of combines as each store node is examined every time its' offset node is combined with any child of the offset. One of the transformations is findBetterNeighborChains which assists MergeConsecutiveStores. The former relies on repeated chain walking to do its' work, however MergeConsecutiveStores is disabled at O0 which makes the transformation redundant. Any optimization level other than O0 would invoke InstCombine which would resolve the chain of GEPs into flat base + offset GEP for each store which does not exhibit the repeated examination of each store to the array. Disabling this optimization fixes an excessive compile time issue (30~ minutes for the test case provided) at O0. Reviewers: niravd, craig.topper, t.p.northover Differential Revision: https://reviews.llvm.org/D40193 llvm-svn: 319142
* [DAGCombiner] Don't combine aext(setcc) if the setcc is already using the ↵Craig Topper2017-11-271-8/+11
| | | | | | | | | | target's preferred result type. With AVX512 vXi1 types are legal so we shouldn't be extending them. This change is similar to existing code in the zext(setcc) combine. llvm-svn: 319120
* [DAGCombiner] Use EVT::changeVectorElementTypeToInteger() instead of ↵Craig Topper2017-11-271-4/+1
| | | | | | implementing manually. llvm-svn: 319119
* [DAGCombiner] Bugfix in isAlias().Jonas Paulsson2017-11-221-2/+2
| | | | | | | | | | | | | | | | | Since i1 is a legal type, this: NumBytes = Op1->getMemoryVT().getSizeInBits() >> 3; is wrong and should be instead NumBytes = Op0->getMemoryVT().getStoreSize(); There seems to be more places where this should be fixed outside DAGCombiner. Review: Hal Finkel https://bugs.llvm.org/show_bug.cgi?id=35366 llvm-svn: 318824
* Fix a bunch more layering of CodeGen headers that are in TargetDavid Blaikie2017-11-171-3/+3
| | | | | | | | All these headers already depend on CodeGen headers so moving them into CodeGen fixes the layering (since CodeGen depends on Target, not the other way around). llvm-svn: 318490
* [DAGCombiner] Use cast instead of an unchecked dyn_cast.Craig Topper2017-11-161-1/+1
| | | | llvm-svn: 318450
* [DAGCombine] Enable more srl -> load combinesSam Parker2017-11-161-5/+16
| | | | | | | | | | | | Change the calculation for the desired ValueType for non-sign extending loads, as in those cases we don't care about the higher bits. This creates a smaller ExtVT and allows for such combinations as: (srl (zextload i16, [addr]), 8) -> (zextload i8, [addr + 1]) Differential Revision: https://reviews.llvm.org/D40034 llvm-svn: 318390
* [DAGcombine] Do not replace truncate node by itself when doing constant ↵Amaury Sechet2017-11-101-3/+9
| | | | | | folding, this trigger needless extra rounds of combine for nothing. NFC llvm-svn: 317926
* Preserve debug info when DAG-combinging (zext (truncate x)) -> (and x, mask).Adrian Prantl2017-11-091-1/+5
| | | | | | rdar://problem/27139077 llvm-svn: 317825
* [DAGCombiner] Fix typos in comments. NFCCraig Topper2017-11-011-2/+2
| | | | llvm-svn: 317072
* [DAGCombine] Don't combine sext with extload if sextload is not supported ↵Guozhi Wei2017-10-271-1/+5
| | | | | | | | | | | | | | | and extload has multi users In function DAGCombiner::visitSIGN_EXTEND_INREG, sext can be combined with extload even if sextload is not supported by target, then if sext is the only user of extload, there is no big difference, no harm no benefit. if extload has more than one user, the combined sextload may block extload from combining with other zext, causes extra zext instructions generated. As demonstrated by the attached test case. This patch add the constraint that when sextload is not supported by target, sext can only be combined with extload if it is the only user of extload. Differential Revision: https://reviews.llvm.org/D39108 llvm-svn: 316802
* DAG: Fold fma (fneg x), K, y -> fma x, -K, yMatt Arsenault2017-10-271-0/+8
| | | | llvm-svn: 316753
* [DAGCombine] Permit combining of shuffles of equivalent splat BUILD_VECTORsSimon Pilgrim2017-10-231-5/+15
| | | | | | | | | | combineShuffleOfScalars is very conservative about shuffled BUILD_VECTORs that can be combined together. This patch adds one additional case - if both BUILD_VECTORs represent splats of the same scalar value but with different UNDEF elements, then we should create a single splat BUILD_VECTOR, sharing only the UNDEF elements defined by the shuffle mask. Differential Revision: https://reviews.llvm.org/D38696 llvm-svn: 316331
* [DAGCombine] Add SCALAR_TO_VECTOR undef handling to simplifyShuffleMask.Simon Pilgrim2017-10-171-2/+6
| | | | | | | | This allows us to simplify later visitVECTOR_SHUFFLE optimizations such as combineShuffleOfScalars. Noticed whilst working on D38696 llvm-svn: 316017
* DAG: Add opcode and source type to isFPExtFreeMatt Arsenault2017-10-131-235/+253
| | | | | | | | This is only currently used for mad/fma transforms. This is the only case where it should be used for AMDGPU, so add an opcode to be sure. llvm-svn: 315740
* Revert r307036 because of PR34919.Wei Mi2017-10-121-92/+0
| | | | llvm-svn: 315540
* [DAGCombiner] convert insertelement of bitcasted vector into shuffleSanjay Patel2017-10-111-3/+62
| | | | | | | | | | | | | | | | Eg: insert v4i32 V, (v2i16 X), 2 --> shuffle v8i16 V', X', {0,1,2,3,8,9,6,7} This is a generalization of the IR fold in D38316 to handle insertion into a non-undef vector. We may want to abandon that one if we can't find value in squashing the more specific pattern sooner. We're using the existing legal shuffle target hook to avoid AVX512 horror with vXi1 shuffles. There may be room for improvement in the shuffle lowering here, but that would be follow-up work. Differential Revision: https://reviews.llvm.org/D38388 llvm-svn: 315460
* [DAGCombine] Fix for shuffle to vector extend for non power 2 vectorsDavid Stuttard2017-10-101-0/+3
| | | | | | | | | | | | | | | | | | | | | Summary: See https://llvm.org/PR33743 for more details It seems that for non-power of 2 vector sizes, the algorithm can produce non-matching sizes for input and result causing an assert. This usually isn't a problem as the isAnyExtend check will weed these out, but in some cases (most often with lots of undefined values for the mask indices) it can pass this check for non power of 2 vectors. Adding in an extra check that ensures that bit size will match for the result and input (as required) Subscribers: nhaehnle Differential Revision: https://reviews.llvm.org/D35241 llvm-svn: 315307
* [DAG] combine assertsexts around a truncSanjay Patel2017-10-091-10/+10
| | | | | | | This was a suggested follow-up to: D37017 / https://reviews.llvm.org/rL313577 llvm-svn: 315206
* Eliminate ftrunc if source is know to be roundedStanislav Mekhanoshin2017-10-021-0/+13
| | | | | | Differential Revision: https://reviews.llvm.org/D38421 llvm-svn: 314688
OpenPOWER on IntegriCloud