summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp
Commit message (Collapse)AuthorAgeFilesLines
...
* Reapply: IR: add optional type to 'byval' function parametersTim Northover2019-05-301-0/+1
| | | | | | | | | | | | | | | | | When we switch to opaque pointer types we will need some way to describe how many bytes a 'byval' parameter should occupy on the stack. This adds a (for now) optional extra type parameter. If present, the type must match the pointee type of the argument. The original commit did not remap byval types when linking modules, which broke LTO. This version fixes that. Note to front-end maintainers: if this causes test failures, it's probably because the "byval" attribute is printed after attributes without any parameter after this change. llvm-svn: 362128
* Revert "IR: add optional type to 'byval' function parameters"Tim Northover2019-05-291-1/+0
| | | | | | | The IRLinker doesn't delve into the new byval attribute when mapping types, and this breaks LTO. llvm-svn: 362029
* IR: add optional type to 'byval' function parametersTim Northover2019-05-291-0/+1
| | | | | | | | | | | | | | When we switch to opaque pointer types we will need some way to describe how many bytes a 'byval' parameter should occupy on the stack. This adds a (for now) optional extra type parameter. If present, the type must match the pointee type of the argument. Note to front-end maintainers: if this causes test failures, it's probably because the "byval" attribute is printed after attributes without any parameter after this change. llvm-svn: 362012
* [SelectionDAG] computeKnownBits - support constant pool values from targetSimon Pilgrim2019-05-241-0/+12
| | | | | | | | | | | | | | | | This patch adds the overridable TargetLowering::getTargetConstantFromLoad function which allows targets to return any constant value loaded by a LoadSDNode node - only X86 makes use of this so far but everything should be in place for other targets. computeKnownBits then uses this function to improve codegen, notably vector code after legalization. A future commit will do the same for ComputeNumSignBits but computeKnownBits sees the bigger benefit. This required a couple of fixes: * SimplifyDemandedBits must early-out for getTargetConstantFromLoad cases to prevent infinite loops of constant regeneration (similar to what we already do for BUILD_VECTOR). * Fix a DAGCombiner::visitTRUNCATE issue as we had trunc(shl(v8i32),v8i16) <-> shl(trunc(v8i16),v8i32) infinite loops after legalization on AVX512 targets. Differential Revision: https://reviews.llvm.org/D61887 llvm-svn: 361620
* [TargetLowering] Extend bool args to inline-asm according to getBooleanTypeKees Cook2019-05-221-1/+10
| | | | | | | | | | | | | | | | | Summary: This extends Krzysztof Parzyszek's X86-specific solution (https://reviews.llvm.org/D60208) to the generic code pointed out by James Y Knight. Reviewers: kparzysz, craig.topper, nickdesaulniers Subscribers: efriedma, sdardis, nemanjai, javed.absar, eraman, fedor.sergeev, asb, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, jrtc27, zzheng, edward-jones, atanasyan, rogfer01, MartinMosbeck, brucehoult, the_o, PkmX, jocewei, jsji, llvm-commits, srhines, void, nickdesaulniers, jyknight Tags: #llvm Differential Revision: https://reviews.llvm.org/D60224 llvm-svn: 361404
* [TargetLowering] Add blank line (test commit)Kees Cook2019-05-221-0/+1
| | | | llvm-svn: 361403
* [Intrinsic] Signed Fixed Point Saturation Multiplication IntrinsicLeonard Chan2019-05-211-11/+45
| | | | | | | | | | | | | | Add an intrinsic that takes 2 signed integers with the scale of them provided as the third argument and performs fixed point multiplication on them. The result is saturated and clamped between the largest and smallest representable values of the first 2 operands. This is a part of implementing fixed point arithmetic in clang where some of the more complex operations will be implemented as intrinsics. Differential Revision: https://reviews.llvm.org/D55720 llvm-svn: 361289
* Add TargetLoweringInfo hook for explicitly setting the ABI calling ↵Dylan McKay2019-05-211-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | convention endianess Summary: The endianess used in the calling convention does not always match the endianess of the target on all architectures, namely AVR. When an argument is too large to be legalised by the architecture and is split for the ABI, a new hook TargetLoweringInfo::shouldSplitFunctionArgumentsAsLittleEndian is queried to find the endianess that function arguments must be laid out in. This approach was recommended by Eli Friedman. Originally reported in https://github.com/avr-rust/rust/issues/129. Patch by Carl Peto. Reviewers: bogner, t.p.northover, RKSimon, niravd, efriedma Reviewed By: efriedma Subscribers: JDevlieghere, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62003 llvm-svn: 361222
* [SDAG] Vector op legalization for overflow opsNikita Popov2019-05-201-0/+74
| | | | | | | | | | | | | | | | | | Fixes issue reported by aemerson on D57348. Vector op legalization support is added for uaddo, usubo, saddo and ssubo (umulo and smulo were already supported). As usual, by extracting TargetLowering methods and calling them from vector op legalization. Vector op legalization doesn't really deal with multiple result nodes, so I'm explicitly performing a recursive legalization call on the result value that is not being legalized. There are some existing test changes because expansion happens earlier, so we don't get a DAG combiner run in between anymore. Differential Revision: https://reviews.llvm.org/D61692 llvm-svn: 361166
* [SDAG] fix unused variable warning and unneeded indirection; NFCSanjay Patel2019-05-141-2/+2
| | | | llvm-svn: 360640
* [SDAG, x86] allow targets to override test for binop opcodesSanjay Patel2019-05-141-1/+2
| | | | | | | | This follows the pattern of the existing isCommutativeBinOp(). x86 shows improvements from vector narrowing for the min/max opcodes. llvm-svn: 360639
* [TargetLowering] Handle multi depth GEPs w/ inline asm constraintsNick Desaulniers2019-05-131-38/+33
| | | | | | | | | | | | | | | | | | | | | | | Summary: X86TargetLowering::LowerAsmOperandForConstraint had better support than TargetLowering::LowerAsmOperandForConstraint for arbitrary depth getelementpointers for "i", "n", and "s" extended inline assembly constraints. Hoist its support from the derived class into the base class. Link: https://github.com/ClangBuiltLinux/linux/issues/469 Reviewers: echristo, t.p.northover Reviewed By: t.p.northover Subscribers: t.p.northover, E5ten, kees, jyknight, nemanjai, javed.absar, eraman, hiraditya, jsji, llvm-commits, void, craig.topper, nathanchance, srhines Tags: #llvm Differential Revision: https://reviews.llvm.org/D61560 llvm-svn: 360604
* [TargetLowering] Add SimplifyDemandedBits support for ZERO_EXTEND_VECTOR_INREGSimon Pilgrim2019-05-131-0/+24
| | | | | | More work for PR39709. llvm-svn: 360592
* TargetLowering::SimplifyDemandedBits - early-out for UNDEF ops. NFCI.Simon Pilgrim2019-05-131-3/+5
| | | | llvm-svn: 360579
* Recommit r358887 "[TargetLowering][AMDGPU][X86] Improve SimplifyDemandedBits ↵Craig Topper2019-05-131-1/+25
| | | | | | | | | | | | | | | | | | | | bitcast handling" I've included a new fix in X86RegisterInfo to prevent PR41619 without reintroducing r359392. We might be able to improve that in the base class implementation of shouldRewriteCopySrc somehow. But this hopefully enables forward progress on SimplifyDemandedBits improvements for now. Original commit message: This patch adds support for BigBitWidth -> SmallBitWidth bitcasts, splitting the DemandedBits/Elts accordingly. The AMDGPU backend needed an extra (srl (and x, c1 << c2), c2) -> (and (srl(x, c2), c1) combine to encourage BFE creation, I investigated putting this in DAGComb but it caused a lot of noise on other targets - some improvements, some regressions. The X86 changes are all definite wins. llvm-svn: 360552
* [DAG] Add SimplifyDemandedBits support for BITREVERSESimon Pilgrim2019-05-111-0/+10
| | | | | | Pulled out of D58017 while I continue to investigate the BSWAP regression on PPC llvm-svn: 360534
* Revert r359392 and r358887Craig Topper2019-05-061-25/+1
| | | | | | | | | | | | | | | | | | | | Reverts "[X86] Remove (V)MOV64toSDrr/m and (V)MOVDI2SSrr/m. Use 128-bit result MOVD/MOVQ and COPY_TO_REGCLASS instead" Reverts "[TargetLowering][AMDGPU][X86] Improve SimplifyDemandedBits bitcast handling" Eric Christopher and Jorge Gorbe Moya reported some issues with these patches to me off list. Removing the CodeGenOnly instructions has changed how fneg is handled during fast-isel with sse/sse2. We're now emitting fsub -0.0, x instead moving to the integer domain(in a GPR), xoring the sign bit, and then moving back to xmm. This is because the fast isel table no longer contains an entry for (f32/f64 bitcast (i32/i64)) so the target independent fneg code fails. The use of fsub changes the behavior of nan with respect to -O2 codegen which will always use a pxor. NOTE: We still have a difference with double with -m32 since the move to GPR doesn't work there. I'll file a separate PR for that and add test cases. Since removing the CodeGenOnly instructions was fixing PR41619, I'm reverting r358887 which exposed that PR. Though I wouldn't be surprised if that bug can still be hit independent of that. This should hopefully get Google back to green. I'll work with Simon and other X86 folks to figure out how to move forward again. llvm-svn: 360066
* [TargetLowering] SimplifySetCC - remove repeated variable. NFCI.Simon Pilgrim2019-05-031-2/+1
| | | | | | Also reduce scope of Temp variable. llvm-svn: 359911
* [TargetLowering] ShrinkDemandedConstant - reduce scope of TLO.DAG variable. ↵Simon Pilgrim2019-05-031-3/+2
| | | | | | | | NFCI. Only ever used in one block llvm-svn: 359890
* [TargetLowering] expandUnalignedStore - cleanup EVT variables. NFCI.Simon Pilgrim2019-05-031-23/+18
| | | | | | Avoid duplicated EVTs and rename Store/Load VTs to avoid -Wshadow warnings. llvm-svn: 359877
* [TargetLowering] findOptimalMemOpLowering. NFCI.Sjoerd Meijer2019-04-301-0/+101
| | | | | | | | | | This was a local static funtion in SelectionDAG, which I've promoted to TargetLowering so that I can reuse it to estimate the cost of a memory operation in D59787. Differential Revision: https://reviews.llvm.org/D59766 llvm-svn: 359543
* [TargetLowering][AMDGPU][X86] Improve SimplifyDemandedBits bitcast handlingSimon Pilgrim2019-04-221-1/+25
| | | | | | | | | | | | This patch adds support for BigBitWidth -> SmallBitWidth bitcasts, splitting the DemandedBits/Elts accordingly. The AMDGPU backend needed an extra (srl (and x, c1 << c2), c2) -> (and (srl(x, c2), c1) combine to encourage BFE creation, I investigated putting this in DAGCombine but it caused a lot of noise on other targets - some improvements, some regressions. The X86 changes are all definite wins. Differential Revision: https://reviews.llvm.org/D60462 llvm-svn: 358887
* [TargetLowering][X86] Teach SimplifyDemandedBits to use ShrinkDemandedOp on ↵Craig Topper2019-04-121-0/+6
| | | | | | | | | | ISD::SHL nodes. If the upper bits of the SHL result aren't used, we might be able to use a narrower shift. For example, on X86 this can turn a 64-bit into 32-bit enabling a smaller encoding. Differential Revision: https://reviews.llvm.org/D60358 llvm-svn: 358257
* [TargetLowering] SimplifyDemandedBits - add ISD::INSERT_SUBVECTOR supportSimon Pilgrim2019-04-091-0/+39
| | | | llvm-svn: 358019
* [TargetLowering] SimplifyDemandedBits - Remove GetDemandedSrcMask lambda. NFCI.Simon Pilgrim2019-04-091-28/+21
| | | | | | An older version of this could return false but now that this always succeeds we can just inline and simplify it. llvm-svn: 357999
* [TargetLowering] SimplifyDemandedBits - call SimplifyDemandedBits in bitcast ↵Simon Pilgrim2019-04-091-6/+16
| | | | | | | | handling When bitcasting from a source op to a larger bitwidth op, split the demanded bits and OR them on top of one another and demand those merged bits in the SimplifyDemandedBits call on the source op. llvm-svn: 357992
* [TargetLowering] SimplifyDemandedBits - use DemandedElts in bitcast handlingSimon Pilgrim2019-04-081-12/+13
| | | | | | Be more selective in the SimplifyDemandedBits -> SimplifyDemandedVectorElts bitcast call based on the demanded elts. llvm-svn: 357942
* [DAG] Pull out ComputeNumSignBits call to make debugging easier. NFCI.Simon Pilgrim2019-04-071-2/+2
| | | | llvm-svn: 357861
* [TargetLowering] Add SimplifyDemandedBits support for ISD::INSERT_VECTOR_ELTSimon Pilgrim2019-03-261-0/+38
| | | | | | | | | | | | This helps us relax the extension of a lot of scalar elements before they are inserted into a vector. Its exposes an issue in DAGCombiner::convertBuildVecZextToZext as some/all the zero-extensions may be relaxed to ANY_EXTEND, so we need to handle that case to avoid a couple of AVX2 VPMOVZX test regressions. Once this is in it should be easier to fix a number of remaining failures to fold loads into VBROADCAST nodes. Differential Revision: https://reviews.llvm.org/D59484 llvm-svn: 356989
* [TargetLowering] SimplifyDemandedBits trunc(srl(x, C1)) - early out for out ↵Simon Pilgrim2019-03-221-19/+19
| | | | | | of range C1. NFCI. llvm-svn: 356810
* [TargetLowering] improve the default expansion of uaddsat/usubsatSanjay Patel2019-03-171-0/+11
| | | | | | | | | | | | | | | This is a subset of what was proposed in: D59006 ...and may overlap with test changes from: D59174 ...but it seems like a good general optimization to turn selects into bitwise-logic when possible because we never know exactly what can happen at this stage of DAG combining depending on how the target has defined things. Differential Revision: https://reviews.llvm.org/D59066 llvm-svn: 356332
* [SelectionDAG] Add SimplifyDemandedBits handling for ISD::SCALAR_TO_VECTORSimon Pilgrim2019-03-151-0/+13
| | | | | | Fixes a lot of constant folding mismatches between i686 and x86_64 llvm-svn: 356273
* [DAG] Move integer setcc %x, %x folding into FoldSetCCSimon Pilgrim2019-03-131-5/+2
| | | | | | | | | | First step towards PR40800 - I intend to move the float case in a separate future patch. I had to tweak the (overly reduced) thumb2 test and the x86 widening test change is annoying (no longer rematerializable) but we should address this separately. Differential Revision: https://reviews.llvm.org/D59244 llvm-svn: 356040
* [SDAG] Expand pow2 mulo using shiftsNikita Popov2019-03-121-4/+23
| | | | | | | | | | | Expand MULO with constant power of two operand into a shift. The overflow is checked with (x << shift) >> shift == x, where the right shift will be logical for umulo and arithmetic for smulo (with exception for multiplications by signed_min). Differential Revision: https://reviews.llvm.org/D59041 llvm-svn: 355937
* [SDAG][AArch64] Legalize VECREDUCENikita Popov2019-03-111-0/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes https://bugs.llvm.org/show_bug.cgi?id=36796. Implement basic legalizations (PromoteIntRes, PromoteIntOp, ExpandIntRes, ScalarizeVecOp, WidenVecOp) for VECREDUCE opcodes. There are more legalizations missing (esp float legalizations), but there's no way to test them right now, so I'm not adding them. This also includes a few more changes to make this work somewhat reasonably: * Add support for expanding VECREDUCE in SDAG. Usually experimental.vector.reduce is expanded prior to codegen, but if the target does have native vector reduce, it may of course still be necessary to expand due to legalization issues. This uses a shuffle reduction if possible, followed by a naive scalar reduction. * Allow the result type of integer VECREDUCE to be larger than the vector element type. For example we need to be able to reduce a v8i8 into an (nominally) i32 result type on AArch64. * Use the vector operand type rather than the scalar result type to determine the action, so we can control exactly which vector types are supported. Also change the legalize vector op code to handle operations that only have vector operands, but no vector results, as is the case for VECREDUCE. * Default VECREDUCE to Expand. On AArch64 (only target using VECREDUCE), explicitly specify for which vector types the reductions are supported. This does not handle anything related to VECREDUCE_STRICT_*. Differential Revision: https://reviews.llvm.org/D58015 llvm-svn: 355860
* [DAG] Move SetCC NaN handling into FoldSetCCSimon Pilgrim2019-03-111-15/+1
| | | | llvm-svn: 355845
* [DAG] TargetLowering::SimplifySetCC - call FoldSetCC early to handle ↵Simon Pilgrim2019-03-111-13/+6
| | | | | | | | constant/commute folds. Noticed while looking at PR40800 (and also D57921) llvm-svn: 355828
* [TargetLowering] simplify code for uaddsat/usubsat expansion; NFCSanjay Patel2019-03-061-17/+13
| | | | | | We had 2 local variable names for the same type. llvm-svn: 355516
* [TargetLowering] simplify code for uaddsat/usubsat expansion; NFCSanjay Patel2019-03-061-8/+5
| | | | llvm-svn: 355508
* Use SDValue::getConstantOperandAPInt helper where possible. NFCI.Simon Pilgrim2019-03-021-5/+3
| | | | llvm-svn: 355267
* Add support for computing "zext of value" in KnownBits. NFCIBjorn Pettersson2019-02-281-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The description of KnownBits::zext() and KnownBits::zextOrTrunc() has confusingly been telling that the operation is equivalent to zero extending the value we're tracking. That has not been true, instead the user has been forced to explicitly set the extended bits as known zero afterwards. This patch adds a second argument to KnownBits::zext() and KnownBits::zextOrTrunc() to control if the extended bits should be considered as known zero or as unknown. Reviewers: craig.topper, RKSimon Reviewed By: RKSimon Subscribers: javed.absar, hiraditya, jdoerfert, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D58650 llvm-svn: 355099
* [SDAG] Support vector UMULO/SMULONikita Popov2019-02-201-16/+24
| | | | | | | | | | | | | | | Second part of https://bugs.llvm.org/show_bug.cgi?id=40442. This adds an extra UnrollVectorOverflowOp() method to SDAG, because the general UnrollOverflowOp() method can't deal with multiple results. Additionally we need to expand UMULO/SMULO during vector op legalization, as it may result in unrolling, which may need additional type legalization. Differential Revision: https://reviews.llvm.org/D57997 llvm-svn: 354513
* [SelectionDAG] Extract [US]MULO expansion into TL method; NFCNikita Popov2019-02-171-1/+121
| | | | | | | | | | | | In preparation for supporting vector expansion. Add an isPostTypeLegalization flag to makeLibCall(), because this expansion relies on the legalized form using MERGE_VALUES. Drop the corresponding variant of ExpandLibCall, which is no longer used. Differential Revision: https://reviews.llvm.org/D58006 llvm-svn: 354226
* [X86] Fix LowerAsmOutputForConstraint.Nirav Dave2019-02-151-1/+1
| | | | | | | | | | | | | | | | | Summary: Update Flag when generating cc output. Fixes PR40737. Reviewers: rnk, nickdesaulniers, craig.topper, spatel Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D58283 llvm-svn: 354163
* Fix 80-column limit in SimplifyDemandedBits/SimplifyDemandedVectorElts. NFCI.Simon Pilgrim2019-02-151-70/+78
| | | | llvm-svn: 354152
* [CallSite removal] Migrate the statepoint GC infrastructure to use theChandler Carruth2019-02-111-12/+12
| | | | | | | | | | | | | | | `CallBase` class rather than `CallSite` wrappers. I pushed this change down through most of the statepoint infrastructure, completely removing the use of CallSite where I could reasonably do so. I ended up making a couple of cut-points: generic call handling (instcombine, TLI, SDAG). As soon as it hit truly generic handling with users outside the immediate code, I simply transitioned into or out of a `CallSite` to make this a reasonable sized chunk. Differential Revision: https://reviews.llvm.org/D56122 llvm-svn: 353660
* [CodeGen][X86] Don't scalarize vector saturating add/subNikita Popov2019-02-101-15/+6
| | | | | | | | | | | Now that we have vector support for [US](ADD|SUB)O we no longer need to scalarize when expanding [US](ADD|SUB)SAT. This matches what the cost model already does. Differential Revision: https://reviews.llvm.org/D57348 llvm-svn: 353651
* [TargetLowering] refactor setcc folds to fix another miscompile (PR40657)Sanjay Patel2019-02-101-55/+55
| | | | | | | | | | SimplifySetCC still has much room for improvement, but this should fix the remaining problem examples from: https://bugs.llvm.org/show_bug.cgi?id=40657 The initial fix for this problem was rL353615. llvm-svn: 353639
* [TargetLowering] add tests to show effect of setcc sub->shift; NFCSanjay Patel2019-02-091-1/+0
| | | | | | | | | There's effectively no difference for the cases with variables. We just trade a sub for an add on those. But the case with a subtract from constant would require an extra move instruction on x86, so this looks like a reasonable generic combine. llvm-svn: 353619
* [TargetLowering] avoid miscompile in setcc transform (PR40657)Sanjay Patel2019-02-091-1/+3
| | | | llvm-svn: 353615
OpenPOWER on IntegriCloud