summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
Commit message (Collapse)AuthorAgeFilesLines
...
* [DAGCombiner] Add a peekThroughBitcast to MergeStoresOfConstantsOrVecElts to ↵Craig Topper2018-03-041-0/+1
| | | | | | | | fix a crash if we are storing a bitcast of a constant. Loading a constant into a k-register in AVX512 requires a bitcast from a scalar constant. In the test case here we have a k-register store that gets split into multiple parts of KNL. MergeConsecutiveStores sees each of these pieces as a consecutive store and looks through the bitcast to find the underly scalar constant. But when we went to create the combined store we didn't look through the same bitcast. llvm-svn: 326677
* [DAGCombiner] When combining zero_extend of a truncate, only mask before ↵Craig Topper2018-03-011-1/+1
| | | | | | | | | | extending for vectors. Masking first, prevents the extend from being combine with loads. Its also interfering with some vXi1 extraction code. Differential Revision: https://reviews.llvm.org/D42679 llvm-svn: 326500
* [DAGCOmbine] Ensure that (brcond (setcc ...)) is handled in a canonical manner.Amaury Sechet2018-02-231-75/+67
| | | | | | | | | | | | | | | Summary: There are transformation that change setcc into other constructs, and transform that try to reconstruct a setcc from the brcond condition. Depending on what order these transform are done, the end result differs. Most of the time, it is preferable to get a setcc as a brcond argument (and this is why brcond try to recreate the setcc in the first place) so we ensure this is done every time by also doing it at the setcc level when the only user is a brcond. Reviewers: spatel, hfinkel, niravd, craig.topper Subscribers: nhaehnle, llvm-commits Differential Revision: https://reviews.llvm.org/D41235 llvm-svn: 325892
* [SelectionDAG] Move matchUnaryPredicate/matchBinaryPredicate into ↵Simon Pilgrim2018-02-221-62/+12
| | | | | | | | | | SelectionDAGNodes.h This allows us to improve vector constant matching in more DAG code (backends, TargetLowering etc.). Differential Revision: https://reviews.llvm.org/D43466 llvm-svn: 325815
* [DAGCombiner] Add two calls to isVector before making calls to ↵Craig Topper2018-02-221-4/+5
| | | | | | | | | | getVectorElementType/getVectorNumElements to avoid an assert. We looked through a BITCAST, but the bitcast might be a from a scalar type rather than a vector. I don't have a test case. I stumbled onto it while prototyping another change that isn't ready yet. llvm-svn: 325750
* [SelectionDAG] Add LegalTypes flag to getShiftAmountTy. Use it to unify and ↵Craig Topper2018-02-201-5/+1
| | | | | | | | | | | | | | | | | | simplify DAGCombiner and simplifySetCC code and fix a bug. DAGCombiner and SimplifySetCC both use getPointerTy for shift amounts pre-legalization. DAGCombiner uses a single helper function to hide this. SimplifySetCC does it in multiple places. This patch adds a defaulted parameter to getShiftAmountTy that can make it return getPointerTy for scalar types. Use this parameter to simplify the SimplifySetCC and DAGCombiner. Additionally, there were two places in SimplifySetCC that were creating shifts using the target's preferred shift amount pre-legalization. If the target uses a narrow type and the type is illegal, this can cause SimplfiySetCC to create a shift with an amount that can't represent all possible shift values for the type. To fix this we should use pointer type there too. Alternatively we could make getScalarShiftAmountTy for each target return a safe value for large types as proposed in D43445. And maybe we should still do that, but fixing the SimplifySetCC code keeps other targets from tripping over this in the future. Fixes PR36250. Differential Revision: https://reviews.llvm.org/D43449 llvm-svn: 325602
* [DAGCombiner] Remove simplifyShuffleMask - now handled more generally by ↵Simon Pilgrim2018-02-171-35/+0
| | | | | | SimplifyDemandedVectorElts. llvm-svn: 325429
* [DAGCombiner] Call ExtendUsesToFormExtLoad in (zext (and (load)))->(and ↵Craig Topper2018-02-151-13/+14
| | | | | | | | | | | | | | (zextload)) even when the and does not have multiple uses Same for the sign extend case. Currently we check for multiple uses on the binop. Then we call ExtendUsesToFormExtLoad to capture SetCCs that use the load. So we only end up finding any setccs when the and has additional uses and the load is used by a setcc. I don't think the and having multiple uses is relevant here. I think we should only be checking for the load having multiple uses. This changes an NVPTX test because we now find that the load has a second use by a truncate, but ExtendUsesToFormExtLoad only looks at setccs it can extend. All other operations just check isTruncateFree. Maybe we should allow widening of an existing truncate even if its not free? Differential Revision: https://reviews.llvm.org/D43063 llvm-svn: 325289
* [SelectionDAG] Add initial implementation of ↵Simon Pilgrim2018-02-151-91/+35
| | | | | | | | | | | | TargetLowering::SimplifyDemandedVectorElts This is mainly a move of simplifyShuffleOperands from DAGCombiner::visitVECTOR_SHUFFLE to create a more general purpose TargetLowering::SimplifyDemandedVectorElts implementation. Further features can be moved/added in future patches. Differential Revision: https://reviews.llvm.org/D42896 llvm-svn: 325232
* [SelectionDAG][X86] Fix incorrect offset generated for VMASKMOVAlexander Ivchenko2018-02-141-9/+10
| | | | | | | | When creating high MachineMemOperand for MSTORE/MLOAD we supply it with the original PointerInfo, while the pointer itself had been incremented. The patch adds the proper offset to the PointerInfo. llvm-svn: 325135
* [DAGCombiner] Add one use check to fold (not (and x, y)) -> (or (not x), ↵Craig Topper2018-02-131-2/+2
| | | | | | | | | | | | | | | | | | | (not y)) Summary: If the and has an additional use we shouldn't invert it. That creates an additional instruction. While there add a one use check to the transform above that looked similar. Reviewers: spatel, RKSimon Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D43225 llvm-svn: 325019
* [X86][SSE] Enable SMIN/SMAX/UMIN/UMAX custom lowering for all legal typesSimon Pilgrim2018-02-111-0/+19
| | | | | | | | This allows us to recognise more saturation patterns and also simplify some MINMAX codegen that was failing to combine CMPGE comparisons to a legal CMPGT. Differential Revision: https://reviews.llvm.org/D43014 llvm-svn: 324837
* [SelectionDAG] Remove TargetLowering::getConstTrueVal. Use ↵Craig Topper2018-02-111-2/+3
| | | | | | | | SelectionDAG::getBoolConstant in the one place it was used. SelectionDAG::getBoolConstant was recently introduced. At the time I didn't know getConstTrueVal existed, but I think getBoolConstant is better as it will use the source VT to make sure it can properly detect floating point if it is configured differently. llvm-svn: 324832
* Revert "WIP: [DAGCombiner] Assert that debug info is preserved"Vedant Kumar2018-02-081-31/+4
| | | | | | This reverts commit r324648. It was committed accidentally. llvm-svn: 324650
* WIP: [DAGCombiner] Assert that debug info is preservedVedant Kumar2018-02-081-4/+31
| | | | llvm-svn: 324648
* [DAGCombiner] Fix a couple mistakes from r324311 by really passing the ↵Craig Topper2018-02-081-2/+4
| | | | | | | | | | original load to ExtendSetCCUses. We're passing the binary op that uses the load instead of the load. Noticed by inspection. Not sure how to test this because this just prevents the introduction of an extend that will later be truncated and will probably be combined out. llvm-svn: 324568
* [DAGCombiner] Don't create truncate nodes in (aext (zextload x)) -> ↵Craig Topper2018-02-081-15/+5
| | | | | | | | (zextload x) and similar folds. NFCI The truncate is being used to replace other users of of the load, but we checked that the load only has one use so there are no other uses to replace. llvm-svn: 324567
* [DAGCombiner] Avoid creating truncate nodes in (zext (and (load)))->(and ↵Craig Topper2018-02-081-22/+25
| | | | | | | | (zextload)) fold until we know for sure we're going to need it. NFCI The truncate is only needed if the load has additional users. It used to get passed to extendSetCCUses so was created early, but that's no longer the case. llvm-svn: 324562
* [DAGCombiner] Rename variable to be slightly better. NFCCraig Topper2018-02-081-21/+21
| | | | | | We were calling a load LN0 but it came from N0.getOperand(0) so its really more like LN00 if we follow the name used in other places. llvm-svn: 324561
* [DAGCombiner][AMDGPU][X86] Turn cttz/ctlz into ↵Craig Topper2018-02-061-0/+14
| | | | | | | | | | | | cttz_zero_undef/ctlz_zero_undef if we can prove the input is never zero X86 currently has a late DAG combine after cttz/ctlz are turned into BSR+BSF+CMOV to detect this and remove the CMOV. But we should be able to do this much earlier and avoid creating the cmov all together. For the changed AMDGPU test case it appears that previously the i8 cttz was type legalized to i16 which introduced an OR with 256 in order to limit the result to 8 on the widened type. At this point the result is known to never be zero, but nothing checked that. Then operation legalization is told to promote all i16 cttz to i32. This introduces an extend and a truncate and another OR with 65536 to limit the result to 16. With the DAG combiner change we are able to prevent the creation of the second OR since the opcode will have been changed to cttz_zero_undef after the first OR. I the lack of the OR caused the instruction to change to v_ffbl_b32_sdwa Differential Revision: https://reviews.llvm.org/D42985 llvm-svn: 324427
* [DAGCombiner] Pass the original load to ExtendSetCCUses not the turncate.Craig Topper2018-02-061-11/+12
| | | | | | | | | | | | | | | | | | | | | | | Summary: This method is trying to use the truncate node to find which SETCC operand should be replaced directly with the extended load. This used to work correctly because all uses of the original load were replaced by the truncate before this function was called. So this was used to effectively bypass the truncate and find the load under it. All but one of the callers now call this before the truncate has replaced the laod so the setcc doesn't yet use the truncate. To account for this we should pass the original load instead. I changed the order of that one caller to make this work there too. I don't have a test case because this is probably hidden by later DAG combines causing the extend and truncate to cancel out. I assume this way is a little more efficient and matches what was originally intended. Reviewers: RKSimon, spatel, niravd Reviewed By: niravd Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D42878 llvm-svn: 324311
* [DAGCombiner] When folding fold (sext/zext (and/or/xor (sextload/zextload ↵Craig Topper2018-02-031-4/+6
| | | | | | | | | | | | | | | | | | | x), cst)) -> (and/or/xor (sextload/zextload x), (sext/zext cst)) make sure we check the legality of the full extended load. Summary: If the load is already an extended load we should be using the memory VT for the legality check, not just the VT of the current extension. I don't have a test case, just noticed it while investigating some load extension improvements. Reviewers: RKSimon, spatel, niravd Reviewed By: niravd Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D42783 llvm-svn: 324181
* [DAGCombiner] When folding (insert_subvector undef, (bitcast ↵Craig Topper2018-02-011-1/+3
| | | | | | | | | | | | (extract_subvector N1, Idx)), Idx) -> (bitcast N1) make sure that N1 has the same total size as the original output We were only checking the element count, but not the total width. This could cause illegal bitcasts to be created if for example the output was 512-bits, but N1 is 256 bits, and the extraction size was 128-bits. Fixes PR36199 Differential Revision: https://reviews.llvm.org/D42809 llvm-svn: 324002
* [DAGCombiner] filter out denorm inputs when calculating sqrt estimate (PR34994)Sanjay Patel2018-02-011-10/+25
| | | | | | | | | | | | | | | | | | | | | | | As shown in the example in PR34994: https://bugs.llvm.org/show_bug.cgi?id=34994 ...we can return a very wrong answer (inf instead of 0.0) for square root when using a reciprocal square root estimate instruction. Here, I've conditionalized the filtering out of denorms based on the function having "denormal-fp-math"="ieee" in its attributes. The other options for this attribute are 'preserve-sign' and 'positive-zero'. So we don't generate this extra code by default with just '-ffast-math' (because then there's no denormal attribute string at all), but it works if you specify '-ffast-math -fdenormal-fp-math=ieee' from clang. As noted in the review, there may be other problems in clang that affect the results depending on platform (Linux x86 at least), but this should allow creating the desired codegen. Differential Revision: https://reviews.llvm.org/D42323 llvm-svn: 323981
* [X86] Make foldLogicOfSetCCs work better for vectors pre legal types/operationsCraig Topper2018-01-291-1/+1
| | | | | | | | | | | | | | | | | Summary: There's a check in the code to only check getSetCCResultType after LegalOperations or if the type is MVT::i1. But the i1 check is only allowing scalar types through. I think it should check that the scalar type is MVT::i1 so that it will work for vectors. The changed test already does this combine with AVX512VL where getSetCCResultType returns vXi1. But with avx512f and no VLX getSetCCResultType returns a type matching the width of the input type. Reviewers: spatel, RKSimon Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D42619 llvm-svn: 323631
* [DAGCombine] reduceBuildVecToShuffle - ensure EXTRACT_VECTOR_ELT index is in ↵Simon Pilgrim2018-01-261-1/+5
| | | | | | | | range From OSS Fuzz Test Case #5688 llvm-svn: 323535
* [DAGCombiner] Bail out if vector size is not a multipleSven van Haastregt2018-01-241-0/+4
| | | | | | | | | | | | For the included test case, the DAG transformation concat_vectors(scalar, undef) -> scalar_to_vector(sclr) would attempt to create a v2i32 vector for a v9i8 concat_vector. Bail out to avoid creating a bitcast with mismatching sizes later on. Differential Revision: https://reviews.llvm.org/D42379 llvm-svn: 323312
* [DAGCombiner] Add a DAG combine to turn a splat build_vector where the splat ↵Craig Topper2018-01-181-0/+23
| | | | | | | | | | elemnt is a bitcast from a vector type into a concat_vector For example, a build_vector of i64 bitcasted from v2i32 can be turned into a concat_vectors of the v2i32 vectors with a bitcast to a vXi64 type Differential Revision: https://reviews.llvm.org/D42090 llvm-svn: 322811
* Revert "[DAG] Elide overlapping stores"Benjamin Kramer2018-01-151-20/+21
| | | | | | | This reverts commit r322085. Internal PPC testing is still showing the same symptoms as when this patch landed the last time. llvm-svn: 322474
* dag-combine: Transfer debug information when folding (zext (truncate x))Adrian Prantl2018-01-111-1/+4
| | | | | | | | | | | -> (zext (truncate x)) This patch adds debug info support to the dagcombine rule (zext (truncate x)) -> (zext (truncate x)). Differential Revision: https://reviews.llvm.org/D41924 llvm-svn: 322304
* DAGCombine: Let truncates negate extension through extract-subvectorZvi Rackover2018-01-111-0/+16
| | | | | | | | | | | | | | | | | | | | | | Summary: Fold cases such as: (v8i8 truncate (v8i32 extract_subvector (v16i32 sext (v16i8 V), Idx))) -> (v8i8 extract_subvector (v16i8 V), Idx) This can be generalized to cases where the truncate and extend do not fully cancel each other out, but it may require querying the target about profitability. Reviewers: RKSimon, craig.topper, spatel, efriedma Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41927 llvm-svn: 322300
* [SelectionDAG][X86] Explicitly store the scale in the gather/scatter ISD nodesCraig Topper2018-01-101-6/+8
| | | | | | | | | | Currently we infer the scale at isel time by analyzing whether the base is a constant 0 or not. If it is we assume scale is 1, else we take it from the element size of the pass thru or stored value. This seems a little weird and I think it makes more sense to make it explicit in the DAG rather than doing tricky things in the backend. Most of this patch is just making sure we copy the scale around everywhere. Differential Revision: https://reviews.llvm.org/D40055 llvm-svn: 322210
* [DAG] Elide overlapping storesNirav Dave2018-01-091-21/+20
| | | | | | | | | | | | | | | | Relanding after fixing handling of pre-indexed memory operations in BaseIndexOffset analysis (r322003). Extend overlapping store elision to handle overwrites of stores by larger stores. Reviewers: craig.topper, rnk, t.p.northover Subscribers: javed.absar, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D40969 llvm-svn: 322085
* [DAG] Teach BaseIndexOffset to correctly handle with indexed operationsNirav Dave2018-01-081-45/+47
| | | | | | | | | | | | | | BaseIndexOffset address analysis incorrectly ignores offsets folded into indexed memory operations causing potential errors in alias analysis of pre-indexed operations. Reviewers: efriedma, RKSimon, hfinkel, jyknight Subscribers: hiraditya, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D41701 llvm-svn: 322003
* [DAGCombine] Fix for PR35761Sam Parker2018-01-081-4/+10
| | | | | | | | | | | I had falsely assumed that constant operands would be operand(1) of the bin ops that may need their constant operand to be masked. Bugzilla: https://bugs.llvm.org/show_bug.cgi?id=35761 Differential Revision: https://reviews.llvm.org/D41667 llvm-svn: 321991
* [DAGCombine] Fix for PR37563Sam Parker2018-01-051-2/+12
| | | | | | | | | | | | While searching for loads to be narrowed, equal sized loads were not added to the list, resulting in anyext loads not being converted to zext loads. https://bugs.llvm.org/show_bug.cgi?id=35763 Differential Revision: https://reviews.llvm.org/D41628 llvm-svn: 321862
* [DAGCombine] Ensure SDNode use iterator is incremented properly.Amara Emerson2018-01-041-2/+2
| | | | | | Fixes an ASAN bug found by oss-fuzz. llvm-svn: 321813
* [DAGCombine] Handle out of range EXTRACT_VECTOR_ELT indices Simon Pilgrim2018-01-031-0/+4
| | | | | | | | Handle this in DAGCombiner::visitEXTRACT_VECTOR_ELT the same as we already do in SelectionDAG::getNode and use APInt instead of getZExtValue. This should also fix oss-fuzz #4910 llvm-svn: 321767
* Revert r321089: "[DAG] Elide overlapping store" (and subsequent fix in r321204)Daniel Jasper2018-01-021-21/+21
| | | | | | | Our internal testing has revealed has discovered bugs in PPC builds. I have forward reproduction instructions to the original author (Nirav). llvm-svn: 321649
* [DAGCombine] Fix for PR35765Sam Parker2018-01-021-1/+0
| | | | | | | | | | | Remove the acceptance of ANY_EXTEND nodes while trying to move and nodes back to loads. Bugzilla: https://bugs.llvm.org/show_bug.cgi?id=35765 Differential Revision: https://reviews.llvm.org/D41625 llvm-svn: 321641
* [DAGCombine] foldBinOpIntoSelect can fail to constant fold in some cases.Simon Pilgrim2017-12-271-6/+8
| | | | | | | | For example, float operations may fail to constant fold under certain circumstances (inf/nan/denormal creation etc.) Reduced from oss-fuzz #4802 test case llvm-svn: 321488
* [DAGCombine] visitANDLike - ensure APInt is is in range for ↵Simon Pilgrim2017-12-261-4/+7
| | | | | | | | getSExtValue/getZExtValue Reduced from oss-fuzz #4782 test case llvm-svn: 321464
* [DAGCombine] Don't combine (and (setne X, 0), (setne X, -1)) --> (setuge ↵Simon Pilgrim2017-12-261-1/+2
| | | | | | | | (add X, 1), 2) for i1 Reduced from oss-fuzz #4773 test case llvm-svn: 321455
* [DAGCombiners] Don't turn ANDs to shuffles with zero so early. Give some ↵Craig Topper2017-12-241-7/+8
| | | | | | | | other combines a chance to run. This moves the combine for turning ANDs into shuffle with zero out of SimplifyVBinOps and places it only in visitAND below the reassociate handling. This fixes the specific case I noticed where we failed to combine two ands with constants. llvm-svn: 321417
* [SelectionDAG][X86] Don't use ->getValueType(0) after a call to getOperand ↵Craig Topper2017-12-231-3/+3
| | | | | | | | | | to get the type of the operand. getOperand returns an SDValue that contains the node and the result number. There is no guarantee that the result number if 0. By using the -> operator we are calling SDNode::getValueType rather than SDValue::getValueType. This requires supplying a result number and we shouldn't assume it was 0. I don't have a test case. Just noticed while cleaning up some other code and saw that it occurred in other places. llvm-svn: 321397
* [DAG] Add missing case check from findbaseoffset merge from r321389.Nirav Dave2017-12-221-2/+4
| | | | llvm-svn: 321391
* Integrate findBaseOffset address analyses to BaseIndexOffset. NFCI.Nirav Dave2017-12-221-70/+10
| | | | | | | | | BaseIndexOffset supercedes findBaseOffset analysis save only Constant Pool addresses. Migrate analysis to BaseIndexOffset. Relanding after correcting base address matching check. llvm-svn: 321389
* Revert "[DAG] Integrate findBaseOffset address analyses to BaseIndexOffset. ↵Nirav Dave2017-12-221-10/+70
| | | | | | | | | | NFCI." which was causing miscompilations in for some test-suite components. This reverts commit 3e9de9ff0f3162920a2a3cba51c7dc14b54b4d16. llvm-svn: 321380
* [DAG] Integrate findBaseOffset address analyses to BaseIndexOffset. NFCI.Nirav Dave2017-12-221-70/+10
| | | | | | | BaseIndexOffset supercedes findBaseOffset analysis save only Constant Pool addresses. Migrate analysis to BaseIndexOffset. llvm-svn: 321364
* [DAGCombine] Revert r321259Sam Parker2017-12-221-26/+0
| | | | | | | Improve ReduceLoadWidth for SRL Patch is causing an issue on the PPC64 BE santizer. llvm-svn: 321349
OpenPOWER on IntegriCloud