summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Fix uninitialized variable warning. NFCI.Simon Pilgrim2019-11-021-1/+1
|
* [AArch64][SVE] Implement masked load intrinsicsKerry McLaughlin2019-10-281-1/+3
| | | | | | | | | | | | | | | | Summary: Adds support for codegen of masked loads, with non-extending, zero-extending and sign-extending variants. Reviewers: huntergr, rovka, greened, dmgreen Reviewed By: dmgreen Subscribers: dmgreen, samparker, tschuett, kristof.beyls, hiraditya, rkruppe, psnobl, cfe-commits, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68877
* [SDAG] fold insert_vector_elt with undef indexSanjay Patel2019-10-271-0/+9
| | | | | | | | | | | | Similar to: rG4c47617627fb This makes the DAG behavior consistent with IR's insertelement. https://bugs.llvm.org/show_bug.cgi?id=42689 I've tried to maintain test intent for AArch64 and WebAssembly by replacing undef index operands with something else.
* [SDAG] fold extract_vector_elt with undef indexSanjay Patel2019-10-251-2/+2
| | | | | | | | | | | This makes the DAG behavior consistent with IR's extractelement after: rGb32e4664a715 https://bugs.llvm.org/show_bug.cgi?id=42689 I've tried to maintain test intent for WebAssembly. The AMDGPU test is trying to test for crashing or other bad behavior, but I'm not sure if that's possible after this change.
* [FPEnv] Add constrained intrinsics for lrint and lroundKevin P. Neal2019-10-071-0/+4
| | | | | | | | | | | Earlier in the year intrinsics for lrint, llrint, lround and llround were added to llvm. The constrained versions are now implemented here. Reviewed by: andrew.w.kaylor, craig.topper, cameron.mcinally Approved by: craig.topper Differential Revision: https://reviews.llvm.org/D64746 llvm-svn: 373900
* Support MemoryLocation::UnknownSize in TargetLowering::IntrinsicInfoTamas Berghammer2019-09-301-1/+1
| | | | | | | | | | | | | | | | Summary: Previously IntrinsicInfo::size was an unsigned what can't represent the 64 bit value used by MemoryLocation::UnknownSize. Reviewers: jmolloy Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68219 llvm-svn: 373214
* [Alignment][NFC] Remove unneeded llvm:: scoping on Align typesGuillaume Chatelet2019-09-271-15/+14
| | | | llvm-svn: 373081
* [DAG] Pull out minimum shift value calc into a helper function. NFCI.Simon Pilgrim2019-09-251-20/+28
| | | | llvm-svn: 372856
* [SelectionDAG] computeKnownBits/ComputeNumSignBits - cleanup ↵Simon Pilgrim2019-09-221-7/+6
| | | | | | | | demanded/unknown paths. NFCI. Merge the calls, just adjust the demandedelts if we have a valid extract_subvector constant index, else demand all elts. llvm-svn: 372521
* [DAG] Add SelectionDAG::MaxRecursionDepth constantSimon Pilgrim2019-09-191-3/+3
| | | | | | | | | | As commented on D67557 we have a lot of uses of depth checks all using magic numbers. This patch adds the SelectionDAG::MaxRecursionDepth constant and moves over some general cases to use this explicitly. Differential Revision: https://reviews.llvm.org/D67711 llvm-svn: 372315
* [SDAG] Update generic code to conservatively check for isAtomic in addition ↵Philip Reames2019-09-121-1/+4
| | | | | | | | | | to isVolatile This is the first sweep of generic code to add isAtomic bailouts where appropriate. The intention here is to have the switch from AtomicSDNode to LoadSDNode/StoreSDNode be close to NFC; that is, I'm not looking to allow additional optimizations at this time. That will come later. See D66309 for context. Differential Revision: https://reviews.llvm.org/D66318 llvm-svn: 371786
* [Alignment] Use Align for TargetLowering::MinStackArgumentAlignmentGuillaume Chatelet2019-09-101-7/+6
| | | | | | | | | | | | | | | | | Summary: This is patch is part of a series to introduce an Alignment type. See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html See this patch for the introduction of the type: https://reviews.llvm.org/D64790 Reviewers: courbet Subscribers: sdardis, nemanjai, hiraditya, kbarton, jrtc27, MaskRay, atanasyan, jsji, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D67288 llvm-svn: 371498
* [SelectionDAG] Remove ISD::FP_ROUND_INREGCraig Topper2019-09-091-19/+1
| | | | | | | | | | | | I don't think anything in tree creates this node. So all of this code appears to be dead. Code coverage agrees http://lab.llvm.org:8080/coverage/coverage-reports/llvm/coverage/Users/buildslave/jenkins/workspace/clang-stage2-coverage-R/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp.html Differential Revision: https://reviews.llvm.org/D67312 llvm-svn: 371431
* [FPEnv] Add fptosi and fptoui constrained intrinsics.Kevin P. Neal2019-08-281-0/+2
| | | | | | | | | | | | | | | | | This implements constrained floating point intrinsics for FP to signed and unsigned integers. Quoting from D32319: The purpose of the constrained intrinsics is to force the optimizer to respect the restrictions that will be necessary to support things like the STDC FENV_ACCESS ON pragma without interfering with optimizations when these restrictions are not needed. Reviewed by: Andrew Kaylor, Craig Topper, Hal Finkel, Cameron McInally, Roman Lebedev, Kit Barton Approved by: Craig Topper Differential Revision: http://reviews.llvm.org/D63782 llvm-svn: 370228
* DAG: computeNumSignBits for MULMatt Arsenault2019-08-271-0/+12
| | | | | | | | | | Copied directly from the IR version. Most of the testcases I've added for this are somewhat problematic because they really end up testing the yet to be implemented version for MUL_I24/MUL_U24. llvm-svn: 370099
* [SelectionDAG] Add a node creation debug message to getMachineNode.Craig Topper2019-08-181-0/+1
| | | | llvm-svn: 369204
* [SDAG][x86] check for relaxed math when matching an FP reductionSanjay Patel2019-08-151-2/+15
| | | | | | | | | | | | | | | | If the last step in an FP add reduction allows reassociation and doesn't care about -0.0, then we are free to recognize that computation as a reduction that may reorder the intermediate steps. This is requested directly by PR42705: https://bugs.llvm.org/show_bug.cgi?id=42705 and solves PR42947 (if horizontal math instructions are actually faster than the alternative): https://bugs.llvm.org/show_bug.cgi?id=42947 Differential Revision: https://reviews.llvm.org/D66236 llvm-svn: 368995
* [SDAG] move variable closer to use; NFCSanjay Patel2019-08-141-1/+1
| | | | llvm-svn: 368905
* Recommit "[MS] Emit S_HEAPALLOCSITE debug info in Selection DAG"Amy Huang2019-08-071-0/+1
| | | | | | with a fix to clear the SDNode map when SelectionDAG is cleared. llvm-svn: 368230
* [SelectionDAG] Extend base addressing modes supported by MGATHER/MSCATTERCullen Rhodes2019-08-061-6/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Before this patch MGATHER/MSCATTER is capable of representing all common addressing modes, but only when illegal types are used. This patch adds an IndexType property so more representations are available when using legal types only. Original modes: vector of bases base + vector of signed scaled offsets New modes: base + vector of signed unscaled offsets base + vector of unsigned scaled offsets base + vector of unsigned unscaled offsets The current behaviour of addressing modes for gather/scatter remains unchanged. Patch by Paul Walker. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D65636 llvm-svn: 368008
* [LLVM][Alignment] Introduce Alignment Type in DataLayoutGuillaume Chatelet2019-08-051-1/+1
| | | | | | | | | | | | | | | | | | | Summary: This is patch is part of a serie to introduce an Alignment type. See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html See this patch for the introduction of the type: https://reviews.llvm.org/D64790 Reviewers: courbet, jfb, jakehehrlich Subscribers: hiraditya, dexonsmith, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D65521 Make getFunctionPtrAlign() return MaybeAlign llvm-svn: 367817
* [SelectionDAG] Add node creation debug message to getMemIntrinsicNode.Craig Topper2019-08-041-1/+3
| | | | llvm-svn: 367771
* [SelectionDAG] Use APInt::isSubsetOf/intersects to simplify some code.Craig Topper2019-08-011-2/+2
| | | | | | Also use KnownBits::isNegative/isNonNegative to further simplify. llvm-svn: 367518
* Migrate some more fadd and fsub cases away from UnsafeFPMath control to ↵Michael Berg2019-07-311-1/+1
| | | | | | | | | | | | | | | | utilize NoSignedZerosFPMath options control Summary: Honoring no signed zeroes is also available as a user control through clang separately regardless of fastmath or UnsafeFPMath context, DAG guards should reflect this context. Reviewers: spatel, arsenm, hfinkel, wristow, craig.topper Reviewed By: spatel Subscribers: rampitec, foad, nhaehnle, wuzish, nemanjai, jvesely, wdng, javed.absar, MaskRay, jsji Differential Revision: https://reviews.llvm.org/D65170 llvm-svn: 367486
* SelectionDAG, MI, AArch64: Widen target flags fields/arguments from unsigned ↵Peter Collingbourne2019-07-311-15/+12
| | | | | | | | | | | | | char to unsigned. This makes the field wider than MachineOperand::SubReg_TargetFlags so that we don't end up silently truncating any higher bits. We should still catch any bits truncated from the MachineOperand field as a consequence of the assertion in MachineOperand::setTargetFlags(). Differential Revision: https://reviews.llvm.org/D65465 llvm-svn: 367474
* [SelectionDAG] Check for any recursion depth greater than or equal to limit ↵Simon Pilgrim2019-07-271-3/+3
| | | | | | | | | | | | instead of just equal the limit. If anything called the recursive isKnownNeverNaN/computeKnownBits/ComputeNumSignBits/SimplifyDemandedBits/SimplifyMultipleUseDemandedBits with an incorrect depth then we could continue to recurse if we'd already exceeded the depth limit. This replaces the limit check (Depth == 6) with a (Depth >= 6) to make sure that we don't circumvent it. This causes a couple of regressions as a mixture of calls (SimplifyMultipleUseDemandedBits + combineX86ShufflesRecursively) were calling with depths that were already over the limit. I've fixed SimplifyMultipleUseDemandedBits to not do this. combineX86ShufflesRecursively is trickier as we get a lot of regressions if we reduce its own limit from 8 to 6 (it also starts at Depth == 1 instead of Depth == 0 like the others....) - I'll see what I can do in future patches. llvm-svn: 367171
* [SelectionDAG] GetDemandedBits - update SIGN_EXTEND_INREG op to just call ↵Simon Pilgrim2019-07-261-9/+1
| | | | | | SimplifyMultipleUseDemandedBits. llvm-svn: 367098
* [SelectionDAG] GetDemandedBits - update OR/XOR ops to just call ↵Simon Pilgrim2019-07-261-6/+2
| | | | | | | | SimplifyMultipleUseDemandedBits. Eventually all of these will be moved over, but we create nodes in GetDemandedBits recursion at the moment which causes regressions when we try to remove them all. llvm-svn: 367092
* Fix signed/unsigned comparison warning. NFCI.Simon Pilgrim2019-07-241-1/+1
| | | | llvm-svn: 366935
* [DAGCombine] matchBinOpReduction - add partial reduction matchingSimon Pilgrim2019-07-241-7/+32
| | | | | | | | | | | | | | | | | | | | This patch adds support for recognizing cases where a larger vector type is being used to reduce just the elements in the lower subvector: e.g. <8 x i32> reduction pattern in a <16 x i32> vector: <4,5,6,7,u,u,u,u,u,u,u,u,u,u,u,u> <2,3,u,u,u,u,u,u,u,u,u,u,u,u,u,u> <1,u,u,u,u,u,u,u,u,u,u,u,u,u,u,u> matchBinOpReduction returns the lower extracted subvector in such cases, assuming isExtractSubvectorCheap accepts the extraction. I've only enabled it for X86 reduction sums so far. I intend to enable it for the bitop/minmax cases in future patches, and eventually I think its worth turning it on all the time. This is mainly just a case of ensuring calls to matchBinOpReduction don't make assumptions on the vector width based on the original vector extraction. Fixes the x86 partial reduction sum cases in PR33758 and PR42023. Differential Revision: https://reviews.llvm.org/D65047 llvm-svn: 366933
* [SelectionDAG] makeEquivalentMemoryOrdering - early out for equal chains ↵Simon Pilgrim2019-07-241-1/+1
| | | | | | | | | | (PR42727) If we are already using the same chain for the old/new memory ops then just return. Fixes PR42727 which had getLoad() reusing an existing node. llvm-svn: 366922
* Fixing @llvm.memcpy not honoring volatile.Guillaume Chatelet2019-07-091-19/+24
| | | | | | | | | | | | | | | | This is explicitly not addressing target-specific code, or calls to memcpy. Summary: https://bugs.llvm.org/show_bug.cgi?id=42254 Reviewers: courbet Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D63215 llvm-svn: 365449
* [SelectionDAG] Propagate alias metadata to target intrinsic nodesJames Molloy2019-07-031-2/+2
| | | | | | | | When a target intrinsic has been determined to touch memory, we construct a MachineMemOperand during SDAG construction. In this case, we should propagate AAMDNodes metadata to the MachineMemOperand where available. Differential revision: https://reviews.llvm.org/D64131 llvm-svn: 365043
* [SelectionDAG] Use the memory VT instead of result VT for FoldingSet ↵Craig Topper2019-06-301-3/+2
| | | | | | | | | profiling in getMaskedLoad/getMaskedStore. This matches what is done by the Profile function. Otherwise CSE won't work properly. llvm-svn: 364717
* [X86] X86DAGToDAGISel::matchBitExtract(): pattern b: truncation awarenessRoman Lebedev2019-06-261-0/+6
| | | | | | | | | | | | | | | | | | Summary: (Not so) boringly identical to pattern a (D62786) Not yet sure how do deal with the last pattern c. Reviewers: RKSimon, craig.topper, spatel Reviewed By: RKSimon Subscribers: llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62793 llvm-svn: 364418
* [DAGCombiner] Support (shl (ext (shl x, c1)), c2) -> 0 non-uniform folds.Simon Pilgrim2019-06-191-4/+4
| | | | | | | | Use matchBinaryPredicate instead of isConstOrConstSplat to let us handle non-uniform shift cases. This requires us to tweak matchBinaryPredicate to allow it to (optionally) handle constants with different type widths. llvm-svn: 363792
* [SelectionDAG] Fold insert_subvector(undef, extract_subvector(v, c), c) -> v ↵Simon Pilgrim2019-06-171-0/+6
| | | | | | | | in getNode This is already done in DAGCombiner::visitINSERT_SUBVECTOR, but this helps a number of shuffles across different vector widths recognise when they come from the same source. llvm-svn: 363542
* [SelectionDAG] ComputeNumSignBits - support constant pool values from targetSimon Pilgrim2019-06-041-0/+30
| | | | | | | | | | | | As I mentioned on D61887 we don't get many hits on ComputeNumSignBits as we did on computeKnownBits. The case we do get is interesting though - it allows us to use the 'ConditionalNegate' combine in combineLogicBlendIntoPBLENDV to remove a select. It comes too late for SSE41 (BLENDV) cases, but SSE2 tests can hit it now. We should probably try to make use of this for SSE41+ targets as well - avoiding variable blends is usually a good idea. I'll investigate as a followup. Differential Revision: https://reviews.llvm.org/D62777 llvm-svn: 362486
* [SelectionDAG] ComputeNumSignBits - clang-format + improve *EXTLOAD ↵Simon Pilgrim2019-06-041-7/+7
| | | | | | | | comments. NFCI. Pre-commit requested for D62777. llvm-svn: 362485
* [SelectionDAG] Add fpto[us]i(undef) --> undef constant foldSimon Pilgrim2019-06-041-0/+5
| | | | | | | | Follow up to D62807. Differential Revision: https://reviews.llvm.org/D62811 llvm-svn: 362483
* [SelectionDAG] Add [us]itofp(undef) --> 0 constant fold (PR39205)Simon Pilgrim2019-06-031-0/+6
| | | | | | | | We were missing this fold in the DAG, which I've copied directly from llvm::ConstantFoldCastInstruction Differential Revision: https://reviews.llvm.org/D62807 llvm-svn: 362397
* [DAG] isBitwiseNot / isConstOrConstSplat - add support for build vector ↵Simon Pilgrim2019-06-021-13/+28
| | | | | | | | | | | | undefs + truncation (PR41020) Add (opt-in) support for implicit truncation to isConstOrConstSplat, which allows us to match truncated 'all ones' cases in isBitwiseNot. PR41020 compares against using ISD::isBuildVectorAllOnes() instead, but that predicate silently accepts any UNDEF elements in the build vector which might not be what we want in isBitwiseNot - so I've added an opt-in 'AllowUndefs' flag that is set to false by default but will allow us to enable it on individual cases where its safe. Differential Revision: https://reviews.llvm.org/D62783 llvm-svn: 362323
* [SelectionDAG] Make the code in mutateStrictFPToFP less aware of how many ↵Craig Topper2019-05-311-55/+34
| | | | | | | | | | | | | operands each node has. NFCI Just copy all of the operands except the chain and call MorphNode on that. This removes the IsUnary and IsTernary flags. Also always get the result type from the result type of the original nodes. Previously we got it from the operand except for two nodes where that didn't work. llvm-svn: 362269
* [SelectionDAG] fold concat of extract subvectorsSanjay Patel2019-05-271-0/+25
| | | | | | | | | | | | | | This is derived from the related fold for build vectors. We also have a version of this in DAGCombiner. The benefit of having this fold at node creation time is (1) efficiency and (2) preventing infinite looping from creating patterns that should not exist in the first place. Currently, the inf-loop could happen with MergeConsecutiveStores() because it naively creates concat of extracts when forming a wider vector store. That could fight with target-specific store narrowing. llvm-svn: 361780
* [SelectionDAG] fix formatting and redundant comments; NFCSanjay Patel2019-05-271-7/+6
| | | | | | | | | There's a possible missing fold here for extracting from the same source vector. It's similar to a check that we use to squash a build vector with all extracted elements from the same source vector. llvm-svn: 361778
* [SelectionDAG] GetDemandedBits - add demanded elements wrapper implementationSimon Pilgrim2019-05-271-1/+15
| | | | | | The DemandedElts variable is pretty much inert at the moment - the original GetDemandedBits implementation calls it with an 'all ones' DemandedElts value so the function is active and behaves exactly as it used to. llvm-svn: 361773
* [SelectionDAG] GetDemandedBits - cleanup to more closely match ↵Simon Pilgrim2019-05-261-16/+21
| | | | | | | | SimplifyDemandedBits. NFCI. Prep work before adding demanded elts support. llvm-svn: 361739
* [SelectionDAG] MaskedValueIsZero - add demanded elements implementationSimon Pilgrim2019-05-261-2/+15
| | | | | | Will be used in an upcoming patch but I've updated the original implementation to call this to ensure test coverage. llvm-svn: 361738
* [SelectionDAG] computeKnownBits - support constant pool values from targetSimon Pilgrim2019-05-241-2/+53
| | | | | | | | | | | | | | | | This patch adds the overridable TargetLowering::getTargetConstantFromLoad function which allows targets to return any constant value loaded by a LoadSDNode node - only X86 makes use of this so far but everything should be in place for other targets. computeKnownBits then uses this function to improve codegen, notably vector code after legalization. A future commit will do the same for ComputeNumSignBits but computeKnownBits sees the bigger benefit. This required a couple of fixes: * SimplifyDemandedBits must early-out for getTargetConstantFromLoad cases to prevent infinite loops of constant regeneration (similar to what we already do for BUILD_VECTOR). * Fix a DAGCombiner::visitTRUNCATE issue as we had trunc(shl(v8i32),v8i16) <-> shl(trunc(v8i16),v8i32) infinite loops after legalization on AVX512 targets. Differential Revision: https://reviews.llvm.org/D61887 llvm-svn: 361620
* [SelectionDAG] fold insert subvector of undef into undefSanjay Patel2019-05-211-0/+3
| | | | | | | | | | | | DAGCombiner simplifies this more liberally as: // If inserting an UNDEF, just return the original vector. if (N1.isUndef()) return N0; So there's no way to make this visible in output AFAIK, but doing this at node creation time should be slightly more efficient. llvm-svn: 361287
OpenPOWER on IntegriCloud