summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Allow FP types for atomicrmw xchgMatt Arsenault2019-01-171-0/+18
| | | | llvm-svn: 351427
* [LegalizeDAG] Remove 'NeedInvert' code from expansion of BR_CC. Replace with ↵Craig Topper2019-01-131-4/+1
| | | | | | | | | | | | an assert. I accidentally triggered this code while doing some experiments and it doesn't look lke it could possibly work. It calls 'getNOT' on a node that should be a CondCode. I think to do this right we would need to swap the branch target and the fallthrough target. But that's not easy to do. Or we could create an explicit SetCC and feed that into a new BR_CC? llvm-svn: 351022
* [X86] Rename overly verbose method; NFCNikita Popov2019-01-131-5/+3
| | | | | | As suggested on D56636. llvm-svn: 351021
* [X86][AARCH64] Improve ISD::ABS supportSimon Pilgrim2019-01-121-0/+4
| | | | | | | | This patch takes some of the code from D49837 to allow us to enable ISD::ABS support for all SSE vector types. Differential Revision: https://reviews.llvm.org/D56544 llvm-svn: 350998
* [TargetLowering] Add ISD::ROTL/ROTR vector expansionSimon Pilgrim2018-12-131-40/+5
| | | | | | | | | | Move existing rotation expansion code into TargetLowering and set it up for vectors as well. Ideally this would share more of the funnel shift expansion, but we handle the shift amount modulo quite differently at the moment. Begun removing x86 vector rotate custom lowering to use the expansion. llvm-svn: 349025
* [Intrinsic] Signed Fixed Point Multiplication IntrinsicLeonard Chan2018-12-121-0/+10
| | | | | | | | | | | | Add an intrinsic that takes 2 signed integers with the scale of them provided as the third argument and performs fixed point multiplication on them. This is a part of implementing fixed point arithmetic in clang where some of the more complex operations will be implemented as intrinsics. Differential Revision: https://reviews.llvm.org/D54719 llvm-svn: 348912
* [SelectionDAG] Initial support for FSHL/FSHR funnel shift opcodes (PR39467)Simon Pilgrim2018-12-051-0/+7
| | | | | | | | | | This is an initial patch to add a minimum level of support for funnel shifts to the SelectionDAG and to begin wiring it up to the X86 SHLD/SHRD instructions. Some partial legalization code has been added to handle the case for 'SlowSHLD' where we want to expand instead and I've added a few DAG combines so we don't get regressions from the existing DAG builder expansion code. Differential Revision: https://reviews.llvm.org/D54698 llvm-svn: 348353
* [FPEnv] Add constrained CEIL/FLOOR/ROUND/TRUNC intrinsicsCameron McInally2018-11-051-0/+8
| | | | | | Differential Revision: https://reviews.llvm.org/D53411 llvm-svn: 346141
* [COFF, ARM64] Implement Intrinsic.sponentry for AArch64Mandeep Singh Grang2018-11-011-0/+1
| | | | | | | | | | | | | | | | Summary: This patch adds Intrinsic.sponentry. This intrinsic is required to correctly support setjmp for AArch64 Windows platform. Patch by: Yin Ma (yinma@codeaurora.org) Reviewers: mgrang, ssijaric, eli.friedman, TomTan, mstorsjo, rnk, compnerd, efriedma Reviewed By: efriedma Subscribers: efriedma, javed.absar, kristof.beyls, chrib, llvm-commits Differential Revision: https://reviews.llvm.org/D53996 llvm-svn: 345909
* Revert "[COFF, ARM64] Implement Intrinsic.sponentry for AArch64"Mandeep Singh Grang2018-11-011-1/+0
| | | | | | This reverts commit 585b6667b4712e3c7f32401e929855b3313b4ff2. llvm-svn: 345863
* [COFF, ARM64] Implement Intrinsic.sponentry for AArch64Mandeep Singh Grang2018-10-311-0/+1
| | | | | | | | | | | | | | Summary: This patch adds Intrinsic.sponentry. This intrinsic is required to correctly support setjmp for AArch64 Windows platform. Reviewers: mgrang, TomTan, rnk, compnerd, mstorsjo, efriedma Reviewed By: efriedma Subscribers: majnemer, chrib, javed.absar, kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D53673 llvm-svn: 345791
* Fix comment typo. NFCI.Simon Pilgrim2018-10-311-1/+1
| | | | llvm-svn: 345758
* [SelectionDAG] SelectionDAGLegalize::ExpandBITREVERSE - ensure we use ShiftTySimon Pilgrim2018-10-311-6/+6
| | | | | | We should be using the getShiftAmountTy value type for shift amounts. llvm-svn: 345756
* [FPEnv] [FPEnv] Add constrained intrinsics for MAXNUM and MINNUMCameron McInally2018-10-301-0/+4
| | | | | | Differential Revision: https://reviews.llvm.org/D53216 llvm-svn: 345650
* [Intrinsic] Signed and Unsigned Saturation Subtraction IntirnsicsLeonard Chan2018-10-291-3/+7
| | | | | | | | | | | | Add an intrinsic that takes 2 integers and perform saturation subtraction on them. This is a part of implementing fixed point arithmetic in clang where some of the more complex operations will be implemented as intrinsics. Differential Revision: https://reviews.llvm.org/D53783 llvm-svn: 345512
* [TargetLowering] Move i64/vXi64 to f32/vXf32 UINT_TO_FP handling to ↵Simon Pilgrim2018-10-281-27/+0
| | | | | | TargetLowering::expandUINT_TO_FP. llvm-svn: 345478
* [TargetLowering] Move LegalizeDAG FP_TO_UINT handling to ↵Simon Pilgrim2018-10-271-22/+3
| | | | | | | | TargetLowering::expandFP_TO_UINT. NFCI. First step towards fixing PR17686 and adding vector support. llvm-svn: 345452
* [LegalizeDAG] Remove dead SINT_TO_FP legalization codeSimon Pilgrim2018-10-251-54/+19
| | | | | | | | As noticed on D52965, the SINT_TO_FP i64 to f32 legalization code has been dead for years - protected by an assert. Differential Revision: https://reviews.llvm.org/D53703 llvm-svn: 345290
* Missing semicolon.Simon Pilgrim2018-10-251-1/+1
| | | | llvm-svn: 345257
* [TargetLowering] Improve vXi64 UINT_TO_FP vXf64 support (P38226)Simon Pilgrim2018-10-251-25/+6
| | | | | | | | | | | | As suggested on D52965, this patch moves the i64 to f64 UINT_TO_FP expansion code from LegalizeDAG into TargetLowering and makes it available to LegalizeVectorOps as well. Not only does this help perform X86 lowering as a true vectorization instead of (partially vectorized) scalar conversions, it avoids the HADDPD op from the scalar code which can be slow on most targets. The AVX512F does have the vcvtusi2sdq scalar operation but we don't unroll to use it as it seems to only help for the v2f64 case - otherwise the unrolling cost will certainly be too high. My feeling is that we should leave it to the vectorizers - and if it generates the vector UINT_TO_FP we should use it. Differential Revision: https://reviews.llvm.org/D53649 llvm-svn: 345256
* [LegalizeDAG] ExpandLegalINT_TO_FP - cleanup UINT_TO_FP i64 -> f32 expansion.Simon Pilgrim2018-10-241-11/+12
| | | | | | | | Use SrcVT/DestVT types and correct shift type. Part of prep work for D52965 llvm-svn: 345158
* [LegalizeDAG] Share Vector/Scalar CTPOP ExpansionSimon Pilgrim2018-10-231-58/+2
| | | | | | | | As suggested on D53258, this patch move the CTPOP expansion code from SelectionDAGLegalize to TargetLowering to allow it to be reused by the VectorLegalizer. Proper vector support will be added by D53258. llvm-svn: 345066
* [LegalizeDAG] Share Vector/Scalar CTLZ ExpansionSimon Pilgrim2018-10-231-32/+5
| | | | | | | | As suggested on D53258, this patch shares common CTLZ expansion code between VectorLegalizer and SelectionDAGLegalize by putting it in TargetLowering. Extension to D53474 llvm-svn: 345060
* [LegalizeDAG] Share Vector/Scalar CTTZ ExpansionSimon Pilgrim2018-10-231-31/+5
| | | | | | | | | | As suggested on D53258, this patch demonstrates sharing common CTTZ expansion code between VectorLegalizer and SelectionDAGLegalize by putting it in TargetLowering. I intend to move CTLZ and (scalar) CTPOP over as well and then update D53258 accordingly. Differential Revision: https://reviews.llvm.org/D53474 llvm-svn: 345039
* [Intrinsic] Unigned Saturation Addition IntrinsicLeonard Chan2018-10-221-3/+5
| | | | | | | | | | | | Add an intrinsic that takes 2 integers and perform unsigned saturation addition on them. This is a part of implementing fixed point arithmetic in clang where some of the more complex operations will be implemented as intrinsics. Differential Revision: https://reviews.llvm.org/D53340 llvm-svn: 344971
* DAG: Change behavior of fminnum/fmaxnum nodesMatt Arsenault2018-10-221-1/+6
| | | | | | | | | | | Introduce new versions that follow the IEEE semantics to help with legalization that may need quieted inputs. There are some regressions from inserting unnecessary canonicalizes when these are matched from fast math fcmp + select which should be fixed in a future commit. llvm-svn: 344914
* [Intrinsic] Signed Saturation Addition IntrinsicLeonard Chan2018-10-161-0/+8
| | | | | | | | | | | Add an intrinsic that takes 2 integers and perform saturation addition on them. This is a part of implementing fixed point arithmetic in clang where some of the more complex operations will be implemented as intrinsics. Differential Revision: https://reviews.llvm.org/D53053 llvm-svn: 344629
* [LegalizeDAG] ExpandLegalINT_TO_FP - cleanup UINT_TO_FP i64 -> f64 expansion.Simon Pilgrim2018-10-161-20/+17
| | | | | | | | Use SrcVT/DestVT types, correct shift type and AND instead of ZERO_EXTEND_IN_REG. Part of prep work for D52965 llvm-svn: 344602
* [LegalizeDAG] Don't bother with final MUL+SRL stage for byte CTPOP. Simon Pilgrim2018-10-141-3/+4
| | | | | | | | The final stage of CTPOP expansion (v = (v * 0x01010101...) >> (Len - 8)) is completely pointless for the byte (Len = 8) case as it reduces to (v = (v * 0x01...) >> 0), but annoyingly this doesn't always get optimized away. Found while investigating generic vector CTPOP expansion (PR32655). llvm-svn: 344477
* Pull out repeated variables from SelectionDAGLegalize::ExpandBitCount.Simon Pilgrim2018-10-131-8/+2
| | | | | | The CTPOP case has been changed from VT.getSizeInBits to VT.getScalarSizeInBits - but this fits in with future work for vector support (PR32655) and doesn't affect any current (scalar) uses. llvm-svn: 344461
* [X86][SSE] Remove most of vector CTTZ custom lowering and use LegalizeDAG ↵Simon Pilgrim2018-10-131-2/+2
| | | | | | | | | | instead. There is one remnant - AVX1 custom splitting of 256-bit vectors - which is due to a regression where the X86ISD::ANDNP is still performed as a YMM. I've also tightened the CTLZ or CTPOP lowering in SelectionDAGLegalize::ExpandBitCount to require a legal CTLZ - it doesn't affect existing users and fixes an issue with AVX512 codegen. llvm-svn: 344457
* [X86][SSE] Begin removing vector CTTZ custom lowering and use LegalizeDAG ↵Simon Pilgrim2018-10-131-1/+1
| | | | | | | | instead. Adds CTTZ vector legalization support and begins the removal of the X86/SSE custom lowering. llvm-svn: 344453
* Pull out repeated value types. NFCI.Simon Pilgrim2018-10-121-6/+5
| | | | llvm-svn: 344355
* [SelectionDAG] Move VectorLegalizer::ExpandCTLZ codegen into ↵Simon Pilgrim2018-10-121-1/+1
| | | | | | | | SelectionDAGLegalize Generalize SelectionDAGLegalize's CTLZ expansion to handle vectors - lets VectorLegalizer::ExpandCTLZ to just pass the expansion on instead of repeating the same codegen. llvm-svn: 344349
* Revert r343948 "[LegalizeDAG] Make one of the ReplaceNode signatures take an ↵Craig Topper2018-10-081-8/+6
| | | | | | | | ArrayRef instead a pointer to an array. Add assert on size of array. NFC" The assert is failing some asan tests on the bots. llvm-svn: 343950
* [LegalizeDAG] Make one of the ReplaceNode signatures take an ArrayRef ↵Craig Topper2018-10-081-6/+8
| | | | | | instead a pointer to an array. Add assert on size of array. NFC llvm-svn: 343948
* [LegalizeDAG] Move legalization of scatter and masked store from ↵Craig Topper2018-10-081-0/+8
| | | | | | | | | | LegalizeVectorOps to LegalizeDAG. This is where we legalize gather and masked load so this is consistent. Since these ops are always on vectors I've chosen to go with LegalizeDAG since that's what we do for other vector only ops like BUILD_VECTOR, VECTOR_SHUFFLE, etc. The ScalarizeMaskedMemIntrinsic pass should take care of scalarizing these before SelectionDAG so hopefully we don't need to worry about illegally typed scalar ops being emitted in the legalizing. If we did we would need to do this in LegalizeVectorOps so we could get the second type legalization that runs between LegalizeVectorOps and LegalizeDAG. llvm-svn: 343947
* [DAG] SelectionDAGLegalize::ExpandLegalINT_TO_FP - use getFPExtendOrRound ↵Simon Pilgrim2018-09-261-11/+1
| | | | | | | | helper. NFCI. Handles SrcVT == DstVT as well. llvm-svn: 343121
* [DAG] ExpandLegalINT_TO_FP - pull out repeated getValueType() call. NFCI.Simon Pilgrim2018-09-261-9/+9
| | | | llvm-svn: 343101
* [LegalizeDAG] Prune Predecessor check in ↵Nirav Dave2018-09-251-0/+1
| | | | | | ExpandExtractFromVectorThroughStack. NFCI. llvm-svn: 342985
* [DAGCombiner] try to convert pow(x, 1/3) to cbrt(x)Sanjay Patel2018-09-161-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | This is a follow-up suggested in D51630 and originally proposed as an IR transform in D49040. Copying the motivational statement by @evandro from that patch: "This transformation helps some benchmarks in SPEC CPU2000 and CPU2006, such as 188.ammp, 447.dealII, 453.povray, and especially 300.twolf, as well as some proprietary benchmarks. Otherwise, no regressions on x86-64 or A64." I'm proposing to add only the minimum support for a DAG node here. Since we don't have an LLVM IR intrinsic for cbrt, and there are no other DAG ways to create a FCBRT node yet, I don't think we need to worry about DAG builder, legalization, a strict variant, etc. We should be able to expand as needed when adding more functionality/transforms. For reference, these are transform suggestions currently listed in SimplifyLibCalls.cpp: // * cbrt(expN(X)) -> expN(x/3) // * cbrt(sqrt(x)) -> pow(x,1/6) // * cbrt(cbrt(x)) -> pow(x,1/9) Also, given that we bail out on long double for now, there should not be any logical differences between platforms (unless there's some platform out there that has pow() but not cbrt()). Differential Revision: https://reviews.llvm.org/D51753 llvm-svn: 342348
* fix typosAdrian Prantl2018-09-141-1/+1
| | | | llvm-svn: 342241
* [ARM] Lower llvm.ctlz.i32 to a libcall when clz is not available.Eli Friedman2018-08-221-0/+15
| | | | | | | | | | The inline sequence is very long (about 70 bytes on Thumb1), so it's not really a good idea to inline it, especially when optimizing for size. Differential Revision: https://reviews.llvm.org/D47917 llvm-svn: 340458
* [FPEnv] Support constrained FREM intrinsicCameron McInally2018-08-201-0/+2
| | | | | | Differential Revision: https://reviews.llvm.org/D50975 llvm-svn: 340201
* Fix FCOPYSIGN expansionLei Liu2018-08-021-16/+12
| | | | | | | | | | | | In expansion of FCOPYSIGN, the shift node is missing when the two operands of FCOPYSIGN are of the same size. We should always generate shift node (if the required shift bit is not zero) to put the sign bit into the right position, regardless of the size of underlying types. Differential Revision: https://reviews.llvm.org/D49973 llvm-svn: 338665
* [CodeGen] Fix inconsistent declaration parameter nameFangrui Song2018-07-161-1/+1
| | | | llvm-svn: 337200
* [FPEnv] Expand constrained FP POWICameron McInally2018-06-151-0/+4
| | | | | | | | | | | | Modify ExpandStrictFPOp(...) to handle nodes that have scalar operands. Also, add a Strict FMA test and do some other light cleanup in the Strict FP code. Differential Revision: https://reviews.llvm.org/D48149 llvm-svn: 334863
* [SelectionDAG] Provide default expansion for rotatesKrzysztof Parzyszek2018-06-121-0/+40
| | | | | | | | | | | | | Implement default legalization of rotates: either in terms of the rotation in the opposite direction (if legal), or in terms of shifts and ors. Implement generating of rotate instructions for Hexagon. Hexagon only supports rotates by an immediate value, so implement custom lowering of ROTL/ROTR on Hexagon. If a rotate is not legal, use the default expansion. Differential Revision: https://reviews.llvm.org/D47725 llvm-svn: 334497
* Get rid of SETCCEAmaury Sechet2018-06-041-2/+1
| | | | | | | | | | | | Summary: It has been deprecated in favor of SETCCCARRY for a year now and isn't used by any in tree backend. Reviewers: efriedma, craig.topper, dblaikie, bkramer Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D47685 llvm-svn: 333939
* [SelectionDAG] Expand UADDO/USUBO into ADD/SUBCARRY if legal for targetKrzysztof Parzyszek2018-06-011-5/+15
| | | | | | | | | Additionally, implement handling of ADD/SUBCARRY on Hexagon, utilizing the UADDO/USUBO expansion. Differential Revision: https://reviews.llvm.org/D47559 llvm-svn: 333751
OpenPOWER on IntegriCloud