summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
Commit message (Collapse)AuthorAgeFilesLines
...
* [DAGCombine] Prune unnused nodes.Nirav Dave2019-03-291-15/+48
| | | | | | | | | | | | | | | | | | | Summary: Nodes that have no uses are eventually pruned when they are selected from the worklist. Record nodes newly added to the worklist or DAG and perform pruning after every combine attempt. Reviewers: efriedma, RKSimon, craig.topper, spatel, jyknight Reviewed By: jyknight Subscribers: jdoerfert, jyknight, nemanjai, jvesely, nhaehnle, javed.absar, hiraditya, jsji, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D58070 llvm-svn: 357283
* [DAG] Set up infrastructure to avoid smart constructor-based dangling nodesNirav Dave2019-03-291-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: Various SelectionDAG non-combine operations (e.g. the getNode smart constructor and legalization) may leave dangling nodes by applying optimizations without fully pruning unused result values. This results in nodes that are never added to the worklist and therefore can not be pruned. Add a node inserter for the combiner to make sure such nodes have the chance of being pruned. This allows a number of additional peephole optimizations. Reviewers: efriedma, RKSimon, craig.topper, jyknight Reviewed By: jyknight Subscribers: msearles, jyknight, sdardis, nemanjai, javed.absar, hiraditya, jrtc27, atanasyan, jsji, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D58068 llvm-svn: 357279
* [DAGCombiner] simplify shuffle of shuffleSanjay Patel2019-03-291-0/+33
| | | | | | | | | | | | | | | | | | | | | | | | | After investigating the examples from D59777 targeting an SSE4.1 machine, it looks like a very different problem due to how we map illegal types (256-bit in these cases). We're missing a shuffle simplification that maps elements of a vector back to a shuffled operand. We have a more general version of this transform in DAGCombiner::visitVECTOR_SHUFFLE(), but that generality means it is limited to patterns with a one-use constraint, and the examples here have 2 uses. We don't need any uses or legality limitations for a simplification (no new value is created). It looks like we miss this pattern in IR too. In one of the zext examples here, we have shuffle masks like this: Shuf0 = vector_shuffle<0,u,3,7,0,u,3,7> Shuf = vector_shuffle<4,u,6,7,u,u,u,u> ...so that's moving the high half of the 1st vector into the low half. But the high half of the 1st vector is already identical to the low half. Differential Revision: https://reviews.llvm.org/D59961 llvm-svn: 357258
* [DAGCombine] Improve Lifetime node chains.Nirav Dave2019-03-291-0/+30
| | | | | | | | | | | | | | | | Improve both start and end lifetime nodes chain dependencies. Reviewers: courbet Reviewed By: courbet Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D59795 llvm-svn: 357256
* [DAGCombiner] fold sext into decrementSanjay Patel2019-03-291-0/+9
| | | | | | | | | | | | | | | | | | | | | | | This is a sibling to rL357178 that I noticed we'd hit if we chose an alternate transform in D59818. %z = zext i8 %x to i32 %dec = add i32 %z, -1 %r = sext i32 %dec to i64 => %z2 = zext i8 %x to i64 %r = add i64 %z2, -1 https://rise4fun.com/Alive/kPP The x86 vector diffs show a slight regression, so there's a chance that we should limit this and the previous transform to scalars. But given that we allowed vectors before, I'm matching that behavior here. We should change both transforms together if that's the right thing to do. llvm-svn: 357254
* [DAGCombiner] fold sext into negationSanjay Patel2019-03-281-0/+10
| | | | | | | | | | | | | | As noted in D59818: %z = zext i8 %x to i32 %neg = sub i32 0, %z %r = sext i32 %neg to i64 => %z2 = zext i8 %x to i64 %r = sub i64 0, %z2 https://rise4fun.com/Alive/KzSR llvm-svn: 357178
* [DAGCombiner] Fold truncate(build_vector(x,y)) -> ↵Simon Pilgrim2019-03-281-1/+15
| | | | | | | | | | | | build_vector(truncate(x),truncate(y)) If scalar truncates are free, attempt to pre-truncate build_vectors source operands. Only attempt to do this before legalization as we often end up with truncations/extensions during build_vector lowering. Differential Revision: https://reviews.llvm.org/D59654 llvm-svn: 357161
* [DAGCombiner] Teach TokenFactor pruning to peek through lifetime nodesNirav Dave2019-03-271-0/+2
| | | | | | | | | | | | | | Summary: Lifetime nodes were inhibiting TokenFactor simplification inhibiting chain-based optimizations. Reviewers: courbet, jyknight Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D59897 llvm-svn: 357121
* Revert r356996 "[DAG] Avoid smart constructor-based dangling nodes."Nirav Dave2019-03-271-12/+0
| | | | | | | This patch appears to trigger very large compile time increases in halide builds. llvm-svn: 357116
* [DAGCombiner] Unify Lifetime and memory Op aliasing.Nirav Dave2019-03-271-69/+91
| | | | | | | | | | | | | | | | | | | Rework BaseIndexOffset and isAlias to fully work with lifetime nodes and fold in lifetime alias analysis. This is mostly NFC. Reviewers: courbet Reviewed By: courbet Subscribers: hiraditya, jdoerfert, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D59794 llvm-svn: 357070
* [DAGCombine] Refactor GatherAllAliases. NFCI.Nirav Dave2019-03-271-65/+66
| | | | llvm-svn: 357069
* [DAGCombiner] Don't allow addcarry if the carry producer is illegal.Jonas Paulsson2019-03-271-0/+4
| | | | | | | | | | | | | | | getAsCarry() checks that the input argument is a carry-producing node before allowing a transformation to addcarry. This patch adds a check to make sure that the carry-producing node is legal. If it is not, it may not remain in a form that is manageable by the target backend. The test case caused a compilation failure during instruction selection for this reason on SystemZ. Patch by Ulrich Weigand. Review: Sanjay Patel https://reviews.llvm.org/D59822 llvm-svn: 357052
* [DAG] Avoid smart constructor-based dangling nodes.Nirav Dave2019-03-261-0/+12
| | | | | | | | | | | | | | | Various SelectionDAG non-combine operations (e.g. the getNode smart constructor and legalization) may leave dangling nodes by applying optimizations or not fully pruning unused result values. This can result in nodes that are never added to the worklist and therefore can not be pruned. Add a node inserter as the current node deleter to make sure such nodes have the chance of being pruned. Many minor changes, mostly positive. llvm-svn: 356996
* [TargetLowering] Add SimplifyDemandedBits support for ISD::INSERT_VECTOR_ELTSimon Pilgrim2019-03-261-3/+7
| | | | | | | | | | | | This helps us relax the extension of a lot of scalar elements before they are inserted into a vector. Its exposes an issue in DAGCombiner::convertBuildVecZextToZext as some/all the zero-extensions may be relaxed to ANY_EXTEND, so we need to handle that case to avoid a couple of AVX2 VPMOVZX test regressions. Once this is in it should be easier to fix a number of remaining failures to fold loads into VBROADCAST nodes. Differential Revision: https://reviews.llvm.org/D59484 llvm-svn: 356989
* [DAGCombiner] Use getTokenFactor in a few more cases.Florian Hahn2019-03-211-4/+4
| | | | | | | | | | | | | | | SDNodes can only have 64k operands and for some inputs (e.g. large number of stores), we can reach this limit when creating TokenFactor nodes. This patch is a follow up to D56740 and updates a few more places that potentially can create TokenFactors with too many operands. Reviewers: efriedma, craig.topper, aemerson, RKSimon Reviewed By: RKSimon Differential Revision: https://reviews.llvm.org/D59156 llvm-svn: 356668
* [DAGCombine] SimplifySelectCC - call FoldSetCC with the setcc result typeSimon Pilgrim2019-03-211-2/+3
| | | | | | | | We were calling FoldSetCC with the compare operand type instead of the result type. Found by OSS-Fuzz #13838 (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=13838) llvm-svn: 356667
* Remove out of date comment. NFCI.Simon Pilgrim2019-03-201-1/+0
| | | | | | DAGCombiner::convertBuildVecZextToZext just requires the extractions to be sequential, they don't have to start from 0'th index. llvm-svn: 356552
* [DAGCombine] Fix a miscompile when reducing BUILD_VECTORs to a shuffleJustin Bogner2019-03-191-11/+9
| | | | | | | | | | | | | | | | | | In r311255 we added a case where we split vectors whose elements are all derived from the same input vector so that we could shuffle it more efficiently. In doing so, createBuildVecShuffle was taught to adjust for the fact that all indices would be based off of the first vector when this happens, but it's possible for the code that checked that to fire incorrectly if we happen to have a BUILD_VECTOR of extracts from subvectors and don't hit this new optimization. Instead of trying to detect if we've split the vector by checking if we have extracts from the same base vector, we can just pass that information into createBuildVecShuffle, avoiding the miscompile. Differential Revision: https://reviews.llvm.org/D59507 llvm-svn: 356476
* [SelectionDAG] Handle unary SelectPatternFlavor for ABS case in ↵Simon Pilgrim2019-03-191-30/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | SelectionDAGBuilder::visitSelect These changes are related to PR37743 and include: SelectionDAGBuilder::visitSelect handles the unary SelectPatternFlavor::SPF_ABS case to build ABS node. Delete the redundant recognizer of the integer ABS pattern from the DAGCombiner. Add promoting the integer ABS node in the LegalizeIntegerType. Expand-based legalization of integer result for the ABS nodes. Expand-based legalization of ABS vector operations. Add some integer abs testcases for different typesizes for Thumb arch Add the custom ABS expanding and change the SAD pattern recognizer for X86 arch: The i64 result of the ABS is expanded to: tmp = (SRA, Hi, 31) Lo = (UADDO tmp, Lo) Hi = (XOR tmp, (ADDCARRY tmp, hi, Lo:1)) Lo = (XOR tmp, Lo) The "detectZextAbsDiff" function is changed for the recognition of pattern with the ABS node. Given a ABS node, detect the following pattern: (ABS (SUB (ZERO_EXTEND a), (ZERO_EXTEND b))). Change integer abs testcases for codegen with the ABS node support for AArch64. Indicate that the ABS is legal for the i64 type when the NEON is supported. Change the integer abs testcases to show changing of codegen. Add combine and legalization of ABS nodes for Thumb arch. Extend 'matchSelectPattern' to recognize the ABS patterns with ICMP_SGE condition. For discussion, see https://bugs.llvm.org/show_bug.cgi?id=37743 Patch by: @ikulagin (Ivan Kulagin) Differential Revision: https://reviews.llvm.org/D49837 llvm-svn: 356468
* [TargetLowering] Add code size information on isFPImmLegal. NFCAdhemerval Zanella2019-03-181-38/+61
| | | | | | | | | | | This allows better code size for aarch64 floating point materialization in a future patch. Reviewers: evandro Differential Revision: https://reviews.llvm.org/D58690 llvm-svn: 356389
* [DAG] Cleanup unused node in SimplifySelectCC.Nirav Dave2019-03-181-8/+7
| | | | | | | | | | | | | | | | Delete temporarily constructed node uses for analysis after it's use, holding onto original input nodes. Ideally this would be rewritten without making nodes, but this appears relatively complex. Reviewers: spatel, RKSimon, craig.topper Subscribers: jdoerfert, hiraditya, deadalnix, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D57921 llvm-svn: 356382
* [DAGCombine] Fold (x & ~y) | y patternsNikita Popov2019-03-171-0/+22
| | | | | | | | | | | | | Fold (x & ~y) | y and it's four commuted variants to x | y. This pattern can in particular appear when a vselect c, x, -1 is expanded to (x & ~c) | (-1 & c) and combined to (x & ~c) | c. This change has some overlap with D59066, which avoids creating a vselect of this form in the first place during uaddsat expansion. Differential Revision: https://reviews.llvm.org/D59174 llvm-svn: 356333
* [DAGCombine] combineShuffleOfScalars - handle non-zero SCALAR_TO_VECTOR ↵Simon Pilgrim2019-03-161-2/+2
| | | | | | | | indices (PR41097) rL356292 reduces the size of scalar_to_vector if we know the upper bits are undef - which means that shuffles may find they are suddenly referencing scalar_to_vector elements other than zero - so make sure we handle this as undef. llvm-svn: 356327
* [DAGCombiner] Fix Comment. NFC.Nirav Dave2019-03-131-1/+1
| | | | llvm-svn: 356069
* [DAGCombiner] If a TokenFactor would be merged into its user, consider the ↵Nirav Dave2019-03-131-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | user later. Summary: A number of optimizations are inhibited by single-use TokenFactors not being merged into the TokenFactor using it. This makes we consider if we can do the merge immediately. Most tests changes here are due to the change in visitation causing minor reorderings and associated reassociation of paired memory operations. CodeGen tests with non-reordering changes: X86/aligned-variadic.ll -- memory-based add folded into stored leaq value. X86/constant-combiners.ll -- Optimizes out overlap between stores. X86/pr40631_deadstore_elision -- folds constant byte store into preceding quad word constant store. Reviewers: RKSimon, craig.topper, spatel, efriedma, courbet Reviewed By: courbet Subscribers: dylanmckay, sdardis, nemanjai, jvesely, nhaehnle, javed.absar, eraman, hiraditya, kbarton, jrtc27, atanasyan, jsji, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D59260 llvm-svn: 356068
* Re-land r354244 "[DAGCombiner] Eliminate dead stores to stack."Clement Courbet2019-03-131-0/+89
| | | | | | Always check candidates for hasOtherUses(), not only stores. llvm-svn: 356050
* [DAGCombine] Pull out repeated demanded bitmask generation. NFCI.Simon Pilgrim2019-03-121-10/+9
| | | | llvm-svn: 355932
* [SDAG][AArch64] Legalize VECREDUCENikita Popov2019-03-111-0/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes https://bugs.llvm.org/show_bug.cgi?id=36796. Implement basic legalizations (PromoteIntRes, PromoteIntOp, ExpandIntRes, ScalarizeVecOp, WidenVecOp) for VECREDUCE opcodes. There are more legalizations missing (esp float legalizations), but there's no way to test them right now, so I'm not adding them. This also includes a few more changes to make this work somewhat reasonably: * Add support for expanding VECREDUCE in SDAG. Usually experimental.vector.reduce is expanded prior to codegen, but if the target does have native vector reduce, it may of course still be necessary to expand due to legalization issues. This uses a shuffle reduction if possible, followed by a naive scalar reduction. * Allow the result type of integer VECREDUCE to be larger than the vector element type. For example we need to be able to reduce a v8i8 into an (nominally) i32 result type on AArch64. * Use the vector operand type rather than the scalar result type to determine the action, so we can control exactly which vector types are supported. Also change the legalize vector op code to handle operations that only have vector operands, but no vector results, as is the case for VECREDUCE. * Default VECREDUCE to Expand. On AArch64 (only target using VECREDUCE), explicitly specify for which vector types the reductions are supported. This does not handle anything related to VECREDUCE_STRICT_*. Differential Revision: https://reviews.llvm.org/D58015 llvm-svn: 355860
* Remove redundant extractBooleanFlip argument. NFCAmaury Sechet2019-03-111-3/+5
| | | | llvm-svn: 355794
* Refactor isBooleanFlip into extractBooleanFlip so that users do not depend ↵Amaury Sechet2019-03-091-19/+28
| | | | | | on the patern matched. NFC llvm-svn: 355769
* [DAGCombiner] fold (add (add (xor a, -1), b), 1) -> (sub b, a)Amaury Sechet2019-03-081-4/+24
| | | | | | | | | | | | | | Summary: This pattern is sometime created after legalization. Reviewers: efriedma, spatel, RKSimon, zvi, bkramer Subscribers: llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D58874 llvm-svn: 355716
* [DAGCombine] Merge visitSMULO+visitUMULO into visitMULO. NFCI.Simon Pilgrim2019-03-081-17/+8
| | | | llvm-svn: 355690
* [DAGCombine] Merge visitSADDO+visitUADDO into visitADDO. NFCI.Simon Pilgrim2019-03-081-48/+24
| | | | llvm-svn: 355689
* [DAGCombine] Merge visitSSUBO+visitUSUBO into visitSUBO. NFCI.Simon Pilgrim2019-03-081-33/+8
| | | | llvm-svn: 355688
* [DAGCombine] Improve select (not Cond), N1, N2 -> select Cond, N2, N1 foldSimon Pilgrim2019-03-061-9/+15
| | | | | | | | Move the x86 combine from D58974 into the DAGCombine VSELECT code and update the SELECT version to use the isBooleanFlip helper as well. Requested by @spatel on D59006 llvm-svn: 355533
* [DAGCombiner] Enable UADDO/USUBO vector combine supportSimon Pilgrim2019-03-061-11/+8
| | | | | | Differential Revision: https://reviews.llvm.org/D58965 llvm-svn: 355517
* [DAGCombiner] Add SADDO/SSUBO combine supportSimon Pilgrim2019-03-061-0/+54
| | | | | | | | Basic constant handling folds, for both scalars and vectors Differential Revision: https://reviews.llvm.org/D58967 llvm-svn: 355506
* [DAGCombiner] Enable SMULO/UMULO vector combine support (PR40442)Simon Pilgrim2019-03-061-2/+2
| | | | | | Differential Revision: https://reviews.llvm.org/D58968 llvm-svn: 355495
* [DAGCombiner][X86][SystemZ][AArch64] Combine some cases of (bitcast ↵Craig Topper2019-03-041-6/+9
| | | | | | | | | | (build_vector constants)) between legalize types and legalize dag. This patch enables combining integer bitcasts of integer build vectors when the new scalar type is legal. I've avoided floating point because the implementation bitcasts float to int along the way and we would need to check the intermediate types for legality Differential Revision: https://reviews.llvm.org/D58884 llvm-svn: 355324
* [DAG] Fix constant store folding to handle non-byte sizes.Nirav Dave2019-02-261-8/+8
| | | | | | | | | | | | | | | | Avoid crashes from zero-byte values due to sub-byte store sizes. Reviewers: uabelho, courbet, rnk Reviewed By: courbet Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D58626 llvm-svn: 354884
* Fix a sign compare warning breaking the -Werror build.Andrea Di Biagio2019-02-251-1/+1
| | | | | | The warning was introduced at r354793. llvm-svn: 354810
* [DAGCombine] Add undef shuffle elt support to partitionShuffleOfConcatsSimon Pilgrim2019-02-251-28/+29
| | | | | | | | Support undef shuffle mask indices in the shuffle(concat_vectors, concat_vectors) -> concat_vectors fold Differential Revision: https://reviews.llvm.org/D58585 llvm-svn: 354793
* [NFC] Fix typos: preceeding -> precedingJordan Rupprecht2019-02-231-4/+4
| | | | llvm-svn: 354715
* Disable big-endian constant store merges from rL354676.Nirav Dave2019-02-221-10/+11
| | | | llvm-svn: 354677
* [DAGCombine] Fold overlapping constant storesNirav Dave2019-02-221-0/+26
| | | | | | | | | | | | | | | Fold a smaller constant store into larger constant stores immediately preceeding it. Reviewers: rnk, courbet Subscribers: javed.absar, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D58468 llvm-svn: 354676
* [DAGCombiner] prevent infinite looping by truncating 'and' (PR40793)Sanjay Patel2019-02-211-2/+3
| | | | | | | | | | | | | | This fold can occur during legalization, so it can fight with promotion to the larger type. It apparently takes a special sequence and subtarget to avoid more basic simplifications that would hide the problem. But there's a bigger question raised here: why does distributeTruncateThroughAnd() even exist? It duplicates functionality from a more minimal pattern that we already have. But getting rid of this function requires some preliminary steps. https://bugs.llvm.org/show_bug.cgi?id=40793 llvm-svn: 354594
* [DAGCombine] Generalize Dead Store to overlapping stores.Nirav Dave2019-02-201-14/+17
| | | | | | | | | | | | | | | | | | Summary: Remove stores that are immediately overwritten by larger stores. Reviewers: courbet, rnk Reviewed By: rnk Subscribers: javed.absar, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D58467 llvm-svn: 354518
* Re-land the refactoring part of r354244 "[DAGCombiner] Eliminate dead stores ↵Clement Courbet2019-02-201-35/+5
| | | | | | | | to stack." This is an NFC. llvm-svn: 354476
* Revert r354244 "[DAGCombiner] Eliminate dead stores to stack."Clement Courbet2019-02-181-90/+35
| | | | | | Breaks some bots. llvm-svn: 354245
* [DAGCombiner] Eliminate dead stores to stack.Clement Courbet2019-02-181-35/+90
| | | | | | | | | | | | | | | Summary: A store to an object whose lifetime is about to end can be removed. See PR40550 for motivation. Reviewers: niravd Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D57541 llvm-svn: 354244
OpenPOWER on IntegriCloud