summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Fix a bunch more layering of CodeGen headers that are in TargetDavid Blaikie2017-11-171-3/+3
| | | | | | | | All these headers already depend on CodeGen headers so moving them into CodeGen fixes the layering (since CodeGen depends on Target, not the other way around). llvm-svn: 318490
* [DAGCombiner] Use cast instead of an unchecked dyn_cast.Craig Topper2017-11-161-1/+1
| | | | llvm-svn: 318450
* [DAGCombine] Enable more srl -> load combinesSam Parker2017-11-161-5/+16
| | | | | | | | | | | | Change the calculation for the desired ValueType for non-sign extending loads, as in those cases we don't care about the higher bits. This creates a smaller ExtVT and allows for such combinations as: (srl (zextload i16, [addr]), 8) -> (zextload i8, [addr + 1]) Differential Revision: https://reviews.llvm.org/D40034 llvm-svn: 318390
* [DAGcombine] Do not replace truncate node by itself when doing constant ↵Amaury Sechet2017-11-101-3/+9
| | | | | | folding, this trigger needless extra rounds of combine for nothing. NFC llvm-svn: 317926
* Preserve debug info when DAG-combinging (zext (truncate x)) -> (and x, mask).Adrian Prantl2017-11-091-1/+5
| | | | | | rdar://problem/27139077 llvm-svn: 317825
* [DAGCombiner] Fix typos in comments. NFCCraig Topper2017-11-011-2/+2
| | | | llvm-svn: 317072
* [DAGCombine] Don't combine sext with extload if sextload is not supported ↵Guozhi Wei2017-10-271-1/+5
| | | | | | | | | | | | | | | and extload has multi users In function DAGCombiner::visitSIGN_EXTEND_INREG, sext can be combined with extload even if sextload is not supported by target, then if sext is the only user of extload, there is no big difference, no harm no benefit. if extload has more than one user, the combined sextload may block extload from combining with other zext, causes extra zext instructions generated. As demonstrated by the attached test case. This patch add the constraint that when sextload is not supported by target, sext can only be combined with extload if it is the only user of extload. Differential Revision: https://reviews.llvm.org/D39108 llvm-svn: 316802
* DAG: Fold fma (fneg x), K, y -> fma x, -K, yMatt Arsenault2017-10-271-0/+8
| | | | llvm-svn: 316753
* [DAGCombine] Permit combining of shuffles of equivalent splat BUILD_VECTORsSimon Pilgrim2017-10-231-5/+15
| | | | | | | | | | combineShuffleOfScalars is very conservative about shuffled BUILD_VECTORs that can be combined together. This patch adds one additional case - if both BUILD_VECTORs represent splats of the same scalar value but with different UNDEF elements, then we should create a single splat BUILD_VECTOR, sharing only the UNDEF elements defined by the shuffle mask. Differential Revision: https://reviews.llvm.org/D38696 llvm-svn: 316331
* [DAGCombine] Add SCALAR_TO_VECTOR undef handling to simplifyShuffleMask.Simon Pilgrim2017-10-171-2/+6
| | | | | | | | This allows us to simplify later visitVECTOR_SHUFFLE optimizations such as combineShuffleOfScalars. Noticed whilst working on D38696 llvm-svn: 316017
* DAG: Add opcode and source type to isFPExtFreeMatt Arsenault2017-10-131-235/+253
| | | | | | | | This is only currently used for mad/fma transforms. This is the only case where it should be used for AMDGPU, so add an opcode to be sure. llvm-svn: 315740
* Revert r307036 because of PR34919.Wei Mi2017-10-121-92/+0
| | | | llvm-svn: 315540
* [DAGCombiner] convert insertelement of bitcasted vector into shuffleSanjay Patel2017-10-111-3/+62
| | | | | | | | | | | | | | | | Eg: insert v4i32 V, (v2i16 X), 2 --> shuffle v8i16 V', X', {0,1,2,3,8,9,6,7} This is a generalization of the IR fold in D38316 to handle insertion into a non-undef vector. We may want to abandon that one if we can't find value in squashing the more specific pattern sooner. We're using the existing legal shuffle target hook to avoid AVX512 horror with vXi1 shuffles. There may be room for improvement in the shuffle lowering here, but that would be follow-up work. Differential Revision: https://reviews.llvm.org/D38388 llvm-svn: 315460
* [DAGCombine] Fix for shuffle to vector extend for non power 2 vectorsDavid Stuttard2017-10-101-0/+3
| | | | | | | | | | | | | | | | | | | | | Summary: See https://llvm.org/PR33743 for more details It seems that for non-power of 2 vector sizes, the algorithm can produce non-matching sizes for input and result causing an assert. This usually isn't a problem as the isAnyExtend check will weed these out, but in some cases (most often with lots of undefined values for the mask indices) it can pass this check for non power of 2 vectors. Adding in an extra check that ensures that bit size will match for the result and input (as required) Subscribers: nhaehnle Differential Revision: https://reviews.llvm.org/D35241 llvm-svn: 315307
* [DAG] combine assertsexts around a truncSanjay Patel2017-10-091-10/+10
| | | | | | | This was a suggested follow-up to: D37017 / https://reviews.llvm.org/rL313577 llvm-svn: 315206
* Eliminate ftrunc if source is know to be roundedStanislav Mekhanoshin2017-10-021-0/+13
| | | | | | Differential Revision: https://reviews.llvm.org/D38421 llvm-svn: 314688
* [DAGCombiner] Fix an off-by-one error in vector logicGeorge Burgess IV2017-09-281-2/+2
| | | | | | | | | Without this, we could end up trying to get the Nth (0-indexed) element from a subvector of size N. Differential Revision: https://reviews.llvm.org/D37880 llvm-svn: 314380
* [CodeGen] Fix some Clang-tidy modernize-use-bool-literals and Include What ↵Eugene Zelenko2017-09-211-99/+114
| | | | | | You Use warnings; other minor fixes (NFC). llvm-svn: 313941
* [DAGCombiner] Slightly simplify some code by using APInt::isMask() and ↵Craig Topper2017-09-211-3/+3
| | | | | | | | countTrailingOnes instead of getting active bits and checking if all the bits below that make a mask. At least for the 64-bit and less case, we should be able to determine if we even have a mask without counting any bits. This also removes the need to explicitly check for 0 active bits, isMask will return false for 0. llvm-svn: 313908
* [DAGCombiner] Remove duplicate code from visitZERO_EXTENDCraig Topper2017-09-211-14/+0
| | | | | | | | This exact block of code exists right below. Differential Revision: https://reviews.llvm.org/D38122 llvm-svn: 313891
* [DAGCombiner] fold assertzexts separated by truncSanjay Patel2017-09-181-2/+25
| | | | | | | | | | | | | If we have an AssertZext of a truncated value that has already been AssertZext'ed, we can assert on the wider source op to improve the zext-y knowledge: assert (trunc (assert X, i8) to iN), i1 --> trunc (assert X, i1) to iN This moves a fold from being Mips-specific to general combining, and x86 shows improvements. Differential Revision: https://reviews.llvm.org/D37017 llvm-svn: 313577
* [DAG, x86] allow store merging before and after legalization (PR34217)Sanjay Patel2017-09-181-4/+4
| | | | | | | | | | | | | | | | | rL310710 allowed store merging to occur after legalization to catch stores that are created late, but this exposes a logic hole seen in PR34217: https://bugs.llvm.org/show_bug.cgi?id=34217 We will miss merging stores if the target lowers vector extracts into target-specific operations. This patch allows store merging to occur both before and after legalization if the target chooses to get maximum merging. I don't think the potential regressions in the other tests are relevant. The tests are for correctness of weird IR constructs rather than perf tests, and I think those are still correct. Differential Revision: https://reviews.llvm.org/D37987 llvm-svn: 313564
* [DAGCombine] (shl (or x, c1), c2) -> (or (shl x, c2), c1 << c2)Simon Pilgrim2017-09-141-2/+4
| | | | | | | | | | We already have a combine for this pattern when the input to shl is add, so we just need to enable the transformation when the input is or. Original patch by @tstellar Differential Revision: https://reviews.llvm.org/D19325 llvm-svn: 313251
* DAG: Allow creating extract_vector_elt post-legalizeMatt Arsenault2017-09-071-1/+4
| | | | | | | | | | | | | | | | Fixes some combine issues for AMDGPU where we weren't getting the many extract_vector_elt combines expected in a future patch. This should really be checking isOperationLegalOrCustom on the extract. That improves a number of x86 lit tests, but a few get stuck in an infinite loop from one place where a similar looking extract is created. I have a different workaround in the backend for that which keeps many of those improvements, but also adds a few regressions. llvm-svn: 312730
* [DAGCombiner] When combining EXTRACT_SUBVECTOR of a BUILD_VECTOR, make sure ↵Craig Topper2017-09-061-2/+3
| | | | | | we don't create a BUILD_VECTOR with an illegal type after type legalization. llvm-svn: 312621
* [X86] Fix crash on assert of non-simple type after type-legalizationAyman Musa2017-09-031-5/+7
| | | | | | | | | | The function combineShuffleToVectorExtend in DAGCombine might generate an illegal typed node after "legalize types" phase, causing assertion on non-simple type to fail afterwards. Adding a type check in case the combine is running after the type legalize pass. Differential Revision: https://reviews.llvm.org/D37330 llvm-svn: 312438
* [DAGCombiner] Do a better job of ensuring we don't split elements when ↵Craig Topper2017-08-311-4/+7
| | | | | | combining an extract_subvector of a bitcasted build_vector. llvm-svn: 312253
* [DAG] Bound loop dependence check in merge optimization.Hans Wennborg2017-08-291-5/+17
| | | | | | | | | | | | | The loop dependence check looks for dependencies between store merge candidates not captured by the chain sub-DAG doing a check of predecessors which may be very large. Conservatively bound number of nodes checked for compilation time. (Resolves PR34326). Landing on behalf of Nirav Dave to unblock the 5.0.0 release. Differential Revision: https://reviews.llvm.org/D37220 llvm-svn: 312022
* [DAGCombiner] Teach visitEXTRACT_SUBVECTOR to turn extracts of BUILD_VECTOR ↵Craig Topper2017-08-281-0/+23
| | | | | | | | | | into smaller BUILD_VECTORs Only do this before operations are legalized of BUILD_VECTOR is Legal for the target. Differential Revision: https://reviews.llvm.org/D37186 llvm-svn: 311892
* [DAGCombiner] allow undef shuffle operands when eliminating bitcasts (PR34111)Sanjay Patel2017-08-271-1/+4
| | | | | | | | As noted in the FIXME, this could be improved more, but this is the smallest fix that helps: https://bugs.llvm.org/show_bug.cgi?id=34111 llvm-svn: 311853
* [DAGCombiner] Extending pattern detection for vector shuffle.Jatin Bhateja2017-08-261-3/+53
| | | | | | | | | | | | | | | | | | Summary: If all the operands of a BUILD_VECTOR extract elements from same vector then split the vector efficiently based on the maximum vector access index. This will also fix PR 33784 Reviewers: zvi, delena, RKSimon, thakis Reviewed By: RKSimon Subscribers: chandlerc, eladcohen, llvm-commits Differential Revision: https://reviews.llvm.org/D35788 llvm-svn: 311833
* Revert rL311247 : To rectify commit message.Jatin Bhateja2017-08-261-53/+3
| | | | | | | | Summary: This reverts commit rL311247. Differential Revision: https://reviews.llvm.org/D36927 llvm-svn: 311832
* [DAG] convert vector select-of-constants to logic/mathSanjay Patel2017-08-241-2/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | This goes back to a discussion about IR canonicalization. We'd like to preserve and convert more IR to 'select' than we currently do because that's likely the best choice in IR: http://lists.llvm.org/pipermail/llvm-dev/2016-September/105335.html ...but that's often not true for codegen, so we need to account for this pattern coming in to the backend and transform it to better DAG ops. Steps in this patch: 1. Add an EVT param to the existing convertSelectOfConstantsToMath() TLI hook to more finely enable this transform. Other targets will probably want that anyway to distinguish scalars from vectors. We're using that here to exclude AVX512 targets, but it may not be necessary. 2. Convert a vselect to ext+add. This eliminates a constant load/materialization, and the vector ext is often free. Implementing a more general fold using xor+and can be a follow-up for targets that don't have a legal vselect. It's also possible that we can remove the TLI hook for the special case fold implemented here because we're eliminating a constant, but it needs to be tested on other targets. Differential Revision: https://reviews.llvm.org/D36840 llvm-svn: 311731
* [DAG] Fix Node Replacement in PromoteIntBinOpHans Wennborg2017-08-241-10/+18
| | | | | | | | | | | | | | When one operand is a user of another in a promoted binary operation we may replace and delete the returned value before returning triggering an assertion. Reorder node replacements to prevent this. Fixes PR34137. Landing on behalf of Nirav. Differential Revision: https://reviews.llvm.org/D36581 llvm-svn: 311623
* [SelectionDAG] Make ISD::isConstantSplatVector always return an element ↵Craig Topper2017-08-221-7/+8
| | | | | | | | | | | | sized APInt. This partially reverts r311429 in favor of making ISD::isConstantSplatVector do something not confusing. Turns out the only other user of it was also having to deal with the weird property of it returning a smaller size. So rather than continue to deal with this quirk everywhere, just make the interface do something sane. Differential Revision: https://reviews.llvm.org/D37039 llvm-svn: 311510
* [DAGCombiner] Extending pattern detection for vector shuffle.Jatin Bhateja2017-08-191-3/+53
| | | | | | | | | | | | | | | | Summary: If all the operands of a BUILD_VECTOR extract elements from same vector then split the vector efficiently based on the maximum vector access index. Reviewers: zvi, delena, RKSimon, thakis Reviewed By: RKSimon Subscribers: chandlerc, eladcohen, llvm-commits Differential Revision: https://reviews.llvm.org/D35788 llvm-svn: 311255
* Revert rL311247 : To rectify commit message.Jatin Bhateja2017-08-191-53/+3
| | | | | | | | Summary: This reverts commit rL311247. Differential Revision: https://reviews.llvm.org/D36927 llvm-svn: 311252
* Merge branch 'arcpatch-D35788'Jatin Bhateja2017-08-191-3/+53
| | | | llvm-svn: 311247
* Revert rL311242 "Extension of shuffle vector pattern detection, updating ↵Jatin Bhateja2017-08-191-46/+3
| | | | | | | | | | | | post rebase." Summary: This reverts commit rL311242. Differential Revision: https://reviews.llvm.org/D36924 llvm-svn: 311246
* Extension of shuffle vector pattern detection, updating post rebase.Jatin Bhateja2017-08-191-3/+46
| | | | llvm-svn: 311242
* [DAGCombiner] Fix bad comment that had immediate values swapped from the ↵Craig Topper2017-08-181-1/+1
| | | | | | code and what they need to be to make sense. NFC llvm-svn: 311144
* [DAGCombiner] Add support for non-uniform constant vectors to (mul x, (1 << ↵Simon Pilgrim2017-08-171-5/+9
| | | | | | c)) -> x << c llvm-svn: 311083
* [DAGCombine] Do not try to deduplicate commutative operations if both ↵Amaury Sechet2017-08-141-3/+3
| | | | | | | | | | | | | | operand are the same. Summary: It is creating useless work as the commuted nodes is the same as the node we are working on in that case. Reviewers: jyknight, nemanjai, mkuper, spatel, RKSimon, zvi, bkramer Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D33840 llvm-svn: 310832
* Revert "[DAGCombiner] Extending pattern detection for vector shuffle ↵Elad Cohen2017-08-141-47/+2
| | | | | | | | (REAPPLIED)" This reverts commit r310782. llvm-svn: 310822
* [X86][ARM][TargetLowering] Add SrcVT to isExtractSubvectorCheapCraig Topper2017-08-131-1/+1
| | | | | | | | | | | | | | | | | Summary: Without the SrcVT its hard to know what is really being asked for. For example if your target has 128, 256, and 512 bit vectors. Maybe extracting 128 from 256 is cheap, but maybe extracting 128 from 512 is not. For x86 we do support extracting a quarter of a 512-bit register. But for i1 vectors we don't have isel patterns for extracting arbitrary pieces. So we need this to have a correct implementation of isExtractSubvectorCheap for mask vectors. Reviewers: RKSimon, zvi, efriedma Reviewed By: RKSimon Subscribers: aemerson, javed.absar, kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D36649 llvm-svn: 310793
* [DAGCombiner] Extending pattern detection for vector shuffle (REAPPLIED)Simon Pilgrim2017-08-121-2/+47
| | | | | | | | | | | | If all the operands of a BUILD_VECTOR extract elements from same vector then split the vector efficiently based on the maximum vector access index. Reapplied with fix to only work with simple value types. Committed on behalf of @jbhateja (Jatin Bhateja) Differential Revision: https://reviews.llvm.org/D35788 llvm-svn: 310782
* [x86] use more shift or LEA for select-of-constants (2nd try)Sanjay Patel2017-08-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The previous rev (r310208) failed to account for overflow when subtracting the constants to see if they're suitable for shift/lea. This version add a check for that and more test were added in r310490. We can convert any select-of-constants to math ops: http://rise4fun.com/Alive/d7d For this patch, I'm enhancing an existing x86 transform that uses fake multiplies (they always become shl/lea) to avoid cmov or branching. The current code misses cases where we have a negative constant and a positive constant, so this is just trying to plug that hole. The DAGCombiner diff prevents us from hitting a terrible inefficiency: we can start with a select in IR, create a select DAG node, convert it into a sext, convert it back into a select, and then lower it to sext machine code. Some notes about the test diffs: 1. 2010-08-04-MaskedSignedCompare.ll - We were creating control flow that didn't exist in the IR. 2. memcmp.ll - Choose -1 or 1 is the case that got me looking at this again. We could avoid the push/pop in some cases if we used 'movzbl %al' instead of an xor on a different reg? That's a post-DAG problem though. 3. mul-constant-result.ll - The trade-off between sbb+not vs. setne+neg could be addressed if that's a regression, but those would always be nearly equivalent. 4. pr22338.ll and sext-i1.ll - These tests have undef operands, so we don't actually care about these diffs. 5. sbb.ll - This shows a win for what is likely a common case: choose -1 or 0. 6. select.ll - There's another borderline case here: cmp+sbb+or vs. test+set+lea? Also, sbb+not vs. setae+neg shows up again. 7. select_const.ll - These are motivating cases for the enhancement; replace cmov with cheaper ops. Assembly differences between movzbl and xor to avoid a partial reg stall are caused later by the X86 Fixup SetCC pass. Differential Revision: https://reviews.llvm.org/D35340 llvm-svn: 310717
* Improve handling of insert_subvector of bitcast valuesNirav Dave2017-08-111-0/+35
| | | | | | | | | | | | Fix insert_subvector / extract_subvector merges of bitcast values. Reviewers: efriedma, craig.topper, RKSimon Subscribers: RKSimon, llvm-commits Differential Revision: https://reviews.llvm.org/D34571 llvm-svn: 310711
* [DAGCombiner] Remove shuffle support from simplifyShuffleMaskSimon Pilgrim2017-08-111-2/+0
| | | | | | | | rL310372 enabled simplifyShuffleMask to support undef shuffle mask inputs, but its causing hangs. Removing support until I can triage the problem llvm-svn: 310699
* Revert "[DAG] Cleanup unused nodes after store merge. NFCI."Nirav Dave2017-08-101-11/+1
| | | | | | This reverts commit r310648 which causes an unexpected assertion failure llvm-svn: 310659
OpenPOWER on IntegriCloud