summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG
Commit message (Collapse)AuthorAgeFilesLines
...
* [SelectionDAG] move splat util functions up from x86 loweringSanjay Patel2019-04-221-0/+52
| | | | | | | | | | This was supposed to be NFC, but the change in SDLoc definitions causes instruction scheduling changes. There's nothing x86-specific in this code, and it can likely be used from DAGCombiner's simplifyVBinOp(). llvm-svn: 358930
* [TargetLowering][AMDGPU][X86] Improve SimplifyDemandedBits bitcast handlingSimon Pilgrim2019-04-221-1/+25
| | | | | | | | | | | | This patch adds support for BigBitWidth -> SmallBitWidth bitcasts, splitting the DemandedBits/Elts accordingly. The AMDGPU backend needed an extra (srl (and x, c1 << c2), c2) -> (and (srl(x, c2), c1) combine to encourage BFE creation, I investigated putting this in DAGCombine but it caused a lot of noise on other targets - some improvements, some regressions. The X86 changes are all definite wins. Differential Revision: https://reviews.llvm.org/D60462 llvm-svn: 358887
* [DAGCombiner] make variable name less ambiguous; NFCSanjay Patel2019-04-221-4/+4
| | | | llvm-svn: 358886
* [DAGCombiner] prepare shuffle-of-splat to handle more patterns; NFCSanjay Patel2019-04-221-11/+16
| | | | llvm-svn: 358884
* [MS] Emit S_HEAPALLOCSITE debug infoAmy Huang2019-04-191-0/+6
| | | | | | | | | | | | | | | | | Summary: This emits labels around heapallocsite calls and S_HEAPALLOCSITE debug info in codeview. Currently only changes FastISel, so emitting labels still needs to be implemented in SelectionDAG. Reviewers: hans, rnk Subscribers: aprantl, hiraditya, cfe-commits, llvm-commits Tags: #clang, #llvm Differential Revision: https://reviews.llvm.org/D60800 llvm-svn: 358783
* [SelectionDAG] soften splat mask assert/unreachable (PR41535)Sanjay Patel2019-04-191-1/+4
| | | | | | | | These are general queries, so they should not die when given a degenerate input like an all undef mask. Callers should be able to deal with an op that will eventually be simplified away. llvm-svn: 358761
* [DAGCombine] Add SimplifyDemandedBits helper that handles demanded elts mask ↵Simon Pilgrim2019-04-171-4/+13
| | | | | | | | as well The other SimplifyDemandedBits helpers become wrappers to this new demanded elts variant. llvm-svn: 358585
* [ScheduleDAGRRList] Recompute topological ordering on demand.Florian Hahn2019-04-171-24/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | Currently there is a single point in ScheduleDAGRRList, where we actually query the topological order (besides init code). Currently we are recomputing the order after adding a node (which does not have predecessors) and then we add predecessors edge-by-edge. We can avoid adding edges one-by-one after we added a new node. In that case, we can just rebuild the order from scratch after adding the edges to the DAG and avoid all the updates to the ordering. Also, we can delay updating the DAG until we query the DAG, if we keep a list of added edges. Depending on the number of updates, we can either apply them when needed or recompute the order from scratch. This brings down the geomean compile time for of CTMark with -O1 down 0.3% on X86, with no regressions. Reviewers: MatzeB, atrick, efriedma, niravd, paquette Reviewed By: efriedma Differential Revision: https://reviews.llvm.org/D60125 llvm-svn: 358583
* [TargetLowering] Rename preferShiftsToClearExtremeBits and ↵Simon Pilgrim2019-04-161-2/+2
| | | | | | | | | | | | shouldFoldShiftPairToMask (PR41359) As discussed on PR41359, this patch renames the pair of shift-mask target feature functions to make their purposes more obvious. shouldFoldShiftPairToMask -> shouldFoldConstantShiftPairToMask preferShiftsToClearExtremeBits -> shouldFoldMaskToVariableShiftPair llvm-svn: 358526
* [DAGCombiner] Add missing flag to addressing mode checkLuis Marques2019-04-161-0/+2
| | | | | | | | | | | | The checks in `canFoldInAddressingMode` tested for addressing modes that have a base register but didn't set the `HasBaseReg` flag to true (it's false by default). This patch fixes that. Although the omission of the flag was technically incorrect it had no known observable impact, so no tests were changed by this patch. Differential Revision: https://reviews.llvm.org/D60314 llvm-svn: 358502
* DAG: propagate ConsecutiveRegs flags to returns too.Tim Northover2019-04-151-0/+18
| | | | | | | | | | Arguments already have a flag to inform backends when they have been split up. The AArch64 arm64_32 ABI makes use of these on return types too, so that code emitted for armv7k can be ABI-compliant. There should be no CodeGen changes yet, just making more information available. llvm-svn: 358399
* DAG: propagate whether an arg is a pointer for CallingConv decisions.Tim Northover2019-04-152-5/+30
| | | | | | | | | | | | | | | The arm64_32 ABI specifies that pointers (despite being 32-bits) should be zero-extended to 64-bits when passed in registers for efficiency reasons. This means that the SelectionDAG needs to be able to tell the backend that an argument was originally a pointer, which is implmented here. Additionally, some memory intrinsics need to be declared as taking an i8* instead of an iPTR. There should be no CodeGen change yet, but it will be triggered when AArch64 backend support for ILP32 is added. llvm-svn: 358398
* [SelectionDAG] Use KnownBits::computeForAddSub/computeForAddCarryBjorn Pettersson2019-04-151-58/+21
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: Use KnownBits::computeForAddSub/computeForAddCarry in SelectionDAG::computeKnownBits when doing value tracking for addition/subtraction. This should improve the precision of the known bits, as we only used to make a simple estimate of known zeroes. The KnownBits support functions are also able to deduce bits that are known to be one in the result. Reviewers: spatel, RKSimon, nikic, lebedev.ri Reviewed By: nikic Subscribers: nikic, javed.absar, lebedev.ri, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60460 llvm-svn: 358372
* [DAGCombiner] narrow shuffle of concatenated vectorsSanjay Patel2019-04-121-0/+50
| | | | | | | | | | | | | | | | | | // shuffle (concat X, undef), (concat Y, undef), Mask --> // concat (shuffle X, Y, Mask0), (shuffle X, Y, Mask1) The ARM changes with 'vtrn' and narrowed 'vuzp' are improvements. The x86 changes look neutral or better. There's one test with an extra instruction, but that could be reversed for a subtarget with the right attributes. But by default, we want to avoid the 256-bit op when possible (in my motivating benchmark, a handful of ymm ops sprinkled into a sequence of xmm ops are triggering frequency throttling on Haswell resulting in significantly worse perf). Differential Revision: https://reviews.llvm.org/D60545 llvm-svn: 358291
* [TargetLowering][X86] Teach SimplifyDemandedBits to use ShrinkDemandedOp on ↵Craig Topper2019-04-121-0/+6
| | | | | | | | | | ISD::SHL nodes. If the upper bits of the SHL result aren't used, we might be able to use a narrower shift. For example, on X86 this can turn a 64-bit into 32-bit enabling a smaller encoding. Differential Revision: https://reviews.llvm.org/D60358 llvm-svn: 358257
* [DAGCombiner] refactor narrowing of extracted vector binop; NFCSanjay Patel2019-04-111-20/+19
| | | | | | | There's a TODO comment about handling patterns with insert_subvector, and we do want to match that. llvm-svn: 358187
* [DAGCombiner][x86] scalarize inserted vector FP opsSanjay Patel2019-04-111-0/+58
| | | | | | | | | | | | | | | | | | | | | | | // bo (build_vec ...undef, x, undef...), (build_vec ...undef, y, undef...) --> // build_vec ...undef, (bo x, y), undef... The lifetime of the nodes in these examples is different for variables versus constants, but they are all build vectors briefly, so I'm proposing to catch them in this form to handle all of the leading examples in the motivating test file. Before we have build vectors, we might have insert_vector_element. After that, we might have scalar_to_vector and constant pool loads. It's going to take more work to ensure that FP vector operands are getting simplified with undef elements, so this transform can apply more widely. In a non-loose FP environment, we are likely simplifying FP elements to NaN values rather than undefs. We also need to allow more opcodes down this path. Eg, we don't handle FP min/max flavors yet. Differential Revision: https://reviews.llvm.org/D60514 llvm-svn: 358172
* Revert rL357745: [SelectionDAG] Compute known bits of CopyFromRegDavid Green2019-04-101-20/+0
| | | | | | | | | | Certain optimisations from ConstantHoisting and CGP rely on Selection DAG not seeing through to the constant in other blocks. Revert this patch while we come up with a better way to handle that. I will try to follow this up with some better tests. llvm-svn: 358113
* [DAGCombiner][X86][SystemZ] Canonicalize SSUBO with immediate RHS to SADDO ↵Craig Topper2019-04-091-0/+8
| | | | | | | | | | by negating the immediate. This lines up with what we do for regular subtract and it matches up better with X86 assumptions in isel patterns that add with immediate is more canonical than sub with immediate. Differential Revision: https://reviews.llvm.org/D60020 llvm-svn: 358027
* [TargetLowering] SimplifyDemandedBits - add ISD::INSERT_SUBVECTOR supportSimon Pilgrim2019-04-091-0/+39
| | | | llvm-svn: 358019
* [TargetLowering] SimplifyDemandedBits - Remove GetDemandedSrcMask lambda. NFCI.Simon Pilgrim2019-04-091-28/+21
| | | | | | An older version of this could return false but now that this always succeeds we can just inline and simplify it. llvm-svn: 357999
* [TargetLowering] SimplifyDemandedBits - call SimplifyDemandedBits in bitcast ↵Simon Pilgrim2019-04-091-6/+16
| | | | | | | | handling When bitcasting from a source op to a larger bitwidth op, split the demanded bits and OR them on top of one another and demand those merged bits in the SimplifyDemandedBits call on the source op. llvm-svn: 357992
* [TargetLowering] SimplifyDemandedBits - use DemandedElts in bitcast handlingSimon Pilgrim2019-04-081-12/+13
| | | | | | Be more selective in the SimplifyDemandedBits -> SimplifyDemandedVectorElts bitcast call based on the demanded elts. llvm-svn: 357942
* [DAG] Pull out ComputeNumSignBits call to make debugging easier. NFCI.Simon Pilgrim2019-04-071-2/+2
| | | | llvm-svn: 357861
* [SelectionDAG] Add fcmp UNDEF handling to SelectionDAG::FoldSetCCSimon Pilgrim2019-04-051-3/+8
| | | | | | | | | | Second half of PR40800, this patch adds DAG undef handling to fcmp instructions to match the behavior in llvm::ConstantFoldCompareInstruction, this permits constant folding of vector comparisons where some elements had been reduced to UNDEF (by SimplifyDemandedVectorElts etc.). This involves a lot of tweaking to reduced tests as bugpoint loves to reduce fcmp arguments to undef........ Differential Revision: https://reviews.llvm.org/D60006 llvm-svn: 357765
* [DAGCombiner][x86] scalarize splatted vector FP opsSanjay Patel2019-04-051-2/+19
| | | | | | | | | | | | | | | There are a variety of vector patterns that may be profitably reduced to a scalar op when scalar ops are performed using a subset (typically, the first lane) of the vector register file. For x86, this is true for float/double ops and element 0 because insert/extract is just a sub-register rename. Other targets should likely enable the hook in a similar way. Differential Revision: https://reviews.llvm.org/D60150 llvm-svn: 357760
* [SelectionDAG] Compute known bits of CopyFromRegPiotr Sobczak2019-04-051-0/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Teach SelectionDAG how to compute known bits of ISD::CopyFromReg if the virtual reg used has one def only. This can be particularly useful when calling isBaseWithConstantOffset() with the ISD::CopyFromReg argument, as more optimizations may get enabled in the result. Also add a missing truncation on X86, found by testing of this patch. Change-Id: Id1c9fceec862d118c54a5b53adf72ada5d6daefa Reviewers: bogner, craig.topper, RKSimon Reviewed By: RKSimon Subscribers: lebedev.ri, nemanjai, jvesely, nhaehnle, javed.absar, jsji, jdoerfert, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D59535 llvm-svn: 357745
* [FastISel] Fix crash for gc.relocate lowringSerguei Katkov2019-04-051-1/+6
| | | | | | | | | | | | | | Lowering safepoint checks that all gc.relocaes observed in safepoint must be lowered. However Fast-Isel is able to skip dead gc.relocate. To resolve this issue we just ignore dead gc.relocate in the check. Reviewers: reames Reviewed By: reames Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D60184 llvm-svn: 357742
* [IR] Refactor attribute methods in Function class (NFC)Evandro Menezes2019-04-044-11/+11
| | | | | | | | Rename the functions that query the optimization kind attributes. Differential revision: https://reviews.llvm.org/D60287 llvm-svn: 357731
* [FastISel] Fix the crash in gc.result loweringSerguei Katkov2019-04-041-1/+2
| | | | | | | | | | | | | | | | | | | | | | | The Fast ISel has a fallback to SelectionDAGISel in case it cannot handle the instruction. This works as follows: Using reverse order, try to select instruction using Fast ISel, if it cannot handle instruction it fallbacks to SelectionDAGISel for these instructions if it is a call and continue fast instruction selections. However if unhandled instruction is not a call or statepoint related instruction it fallbacks to SelectionDAGISel for all remaining instructions in basic block. However gc.result instruction is missed and as a result it is possible that gc.result is processed earlier than statepoint causing breakage invariant the gc.results should be handled after statepoint. Test is updated because in the current form fast-isel cannot handle ret instruction (due to i1 ret type without explicit ext) and as a result test does not check fast-isel at all. Reviewers: reames Reviewed By: reames Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D60182 llvm-svn: 357672
* [DAGCombiner] Rename variables Demanded -> DemandedBits/DemandedElts. NFCI.Simon Pilgrim2019-04-031-9/+10
| | | | | | Use consistent variable names down the SimplifyDemanded* call stack so debugging isn't such a annoyance. llvm-svn: 357602
* [DAGCombiner] loosen restrictions for moving shuffles after vector binopSanjay Patel2019-04-031-16/+19
| | | | | | | | | | | | There are 3 changes to make this correspond to the same transform in instcombine: 1. Remove the legality check - we can't create anything less legal than we started with. 2. Ease the use restriction, so we only bail out if both operands have >1 use. 3. Ease the use restriction for binops with a repeated operand (eg, mul x, x). As discussed in D60150, there's a scalarization opportunity that will be made easier by allowing this transform more generally. llvm-svn: 357580
* [DAGCombine] Don't use getZExtValue() until we know the constant is in range.Simon Pilgrim2019-04-031-2/+2
| | | | | | Noticed during prep for a patch for PR40758. llvm-svn: 357571
* Revert r357256 "[DAGCombine] Improve Lifetime node chains."Hans Wennborg2019-04-031-31/+0
| | | | | | | | | | | | | | | | | | | | | | | | As it caused a pathological compile-time regressionin V8, see PR41352. > Improve both start and end lifetime nodes chain dependencies. > > Reviewers: courbet > > Reviewed By: courbet > > Subscribers: hiraditya, llvm-commits > > Tags: #llvm > > Differential Revision: https://reviews.llvm.org/D59795 This also reverts the follow-up r357309: > [DAGCombiner] Rewrite ImproveLifetimeNodeChain to avoid DAG loop. > > Avoid EXPENSIVE_CHECK failure. NFCI. llvm-svn: 357563
* [DAGCombiner] reduce code duplication; NFCSanjay Patel2019-04-021-8/+8
| | | | llvm-svn: 357498
* [DAGCombiner] Rewrite ImproveLifetimeNodeChain to avoid DAG loop.Nirav Dave2019-03-291-8/+9
| | | | | | Avoid EXPENSIVE_CHECK failure. NFCI. llvm-svn: 357309
* [DAG] Avoid redundancy in StoreMerge TokenFactor generation.Nirav Dave2019-03-291-2/+2
| | | | | | | Avoid generating redundant TokenFactor when all merged stores have the same chain. llvm-svn: 357299
* [DAGCombine] Prune unnused nodes.Nirav Dave2019-03-291-15/+48
| | | | | | | | | | | | | | | | | | | Summary: Nodes that have no uses are eventually pruned when they are selected from the worklist. Record nodes newly added to the worklist or DAG and perform pruning after every combine attempt. Reviewers: efriedma, RKSimon, craig.topper, spatel, jyknight Reviewed By: jyknight Subscribers: jdoerfert, jyknight, nemanjai, jvesely, nhaehnle, javed.absar, hiraditya, jsji, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D58070 llvm-svn: 357283
* [DAG] Set up infrastructure to avoid smart constructor-based dangling nodesNirav Dave2019-03-292-0/+16
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: Various SelectionDAG non-combine operations (e.g. the getNode smart constructor and legalization) may leave dangling nodes by applying optimizations without fully pruning unused result values. This results in nodes that are never added to the worklist and therefore can not be pruned. Add a node inserter for the combiner to make sure such nodes have the chance of being pruned. This allows a number of additional peephole optimizations. Reviewers: efriedma, RKSimon, craig.topper, jyknight Reviewed By: jyknight Subscribers: msearles, jyknight, sdardis, nemanjai, javed.absar, hiraditya, jrtc27, atanasyan, jsji, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D58068 llvm-svn: 357279
* [DAGCombiner] simplify shuffle of shuffleSanjay Patel2019-03-291-0/+33
| | | | | | | | | | | | | | | | | | | | | | | | | After investigating the examples from D59777 targeting an SSE4.1 machine, it looks like a very different problem due to how we map illegal types (256-bit in these cases). We're missing a shuffle simplification that maps elements of a vector back to a shuffled operand. We have a more general version of this transform in DAGCombiner::visitVECTOR_SHUFFLE(), but that generality means it is limited to patterns with a one-use constraint, and the examples here have 2 uses. We don't need any uses or legality limitations for a simplification (no new value is created). It looks like we miss this pattern in IR too. In one of the zext examples here, we have shuffle masks like this: Shuf0 = vector_shuffle<0,u,3,7,0,u,3,7> Shuf = vector_shuffle<4,u,6,7,u,u,u,u> ...so that's moving the high half of the 1st vector into the low half. But the high half of the 1st vector is already identical to the low half. Differential Revision: https://reviews.llvm.org/D59961 llvm-svn: 357258
* [DAGCombine] Improve Lifetime node chains.Nirav Dave2019-03-291-0/+30
| | | | | | | | | | | | | | | | Improve both start and end lifetime nodes chain dependencies. Reviewers: courbet Reviewed By: courbet Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D59795 llvm-svn: 357256
* [DAGCombiner] fold sext into decrementSanjay Patel2019-03-291-0/+9
| | | | | | | | | | | | | | | | | | | | | | | This is a sibling to rL357178 that I noticed we'd hit if we chose an alternate transform in D59818. %z = zext i8 %x to i32 %dec = add i32 %z, -1 %r = sext i32 %dec to i64 => %z2 = zext i8 %x to i64 %r = add i64 %z2, -1 https://rise4fun.com/Alive/kPP The x86 vector diffs show a slight regression, so there's a chance that we should limit this and the previous transform to scalars. But given that we allowed vectors before, I'm matching that behavior here. We should change both transforms together if that's the right thing to do. llvm-svn: 357254
* Switch lowering: exploit unreachable fall-through when lowering case range ↵Hans Wennborg2019-03-292-3/+23
| | | | | | | | | | | | | | | | | | | | cluster In the example below, we would previously emit two range checks, one for cases 1--3 and one for 4--6. This patch makes us exploit the fact that the fall-through is unreachable and only one range check is necessary. switch i32 %i, label %default [ i32 1, label %bb1 i32 2, label %bb1 i32 3, label %bb1 i32 4, label %bb2 i32 5, label %bb2 i32 6, label %bb2 ] default: unreachable llvm-svn: 357252
* [SelectionDAGBuilder] Fix 80 column violation. NFCCraig Topper2019-03-281-1/+2
| | | | llvm-svn: 357213
* [DAG] Fix Lifetime Node ID hashing.Nirav Dave2019-03-281-0/+7
| | | | llvm-svn: 357179
* [DAGCombiner] fold sext into negationSanjay Patel2019-03-281-0/+10
| | | | | | | | | | | | | | As noted in D59818: %z = zext i8 %x to i32 %neg = sub i32 0, %z %r = sext i32 %neg to i64 => %z2 = zext i8 %x to i64 %r = sub i64 0, %z2 https://rise4fun.com/Alive/KzSR llvm-svn: 357178
* [DAGCombiner] Fold truncate(build_vector(x,y)) -> ↵Simon Pilgrim2019-03-281-1/+15
| | | | | | | | | | | | build_vector(truncate(x),truncate(y)) If scalar truncates are free, attempt to pre-truncate build_vectors source operands. Only attempt to do this before legalization as we often end up with truncations/extensions during build_vector lowering. Differential Revision: https://reviews.llvm.org/D59654 llvm-svn: 357161
* [DAGCombiner] Teach TokenFactor pruning to peek through lifetime nodesNirav Dave2019-03-271-0/+2
| | | | | | | | | | | | | | Summary: Lifetime nodes were inhibiting TokenFactor simplification inhibiting chain-based optimizations. Reviewers: courbet, jyknight Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D59897 llvm-svn: 357121
* [LegalizeVectorTypes] Allow single loads and stores for more short vectorsJustin Bogner2019-03-271-1/+6
| | | | | | | | | | | | | | | | | | | When lowering a load or store for TypeWidenVector, the type legalizer would use a single load or store if the associated integer type was legal or promoted. E.g. it loads a v4i8 as an i32 if i32 is legal/promotable. (See https://reviews.llvm.org/rL236528 for reference.) This applies that behaviour to vector types. If the vector type is TypePromoteInteger, the element type is going to be TypePromoteInteger as well, which will lead to have a single promoting load rather than N individual promoting loads. For instance, if we have a v3i1, we would now have a load of v4i1 instead of 3 loads of i1. Patch by Guillaume Marques. Thanks! Differential Revision: https://reviews.llvm.org/D56201 llvm-svn: 357120
* Revert r356996 "[DAG] Avoid smart constructor-based dangling nodes."Nirav Dave2019-03-272-15/+0
| | | | | | | This patch appears to trigger very large compile time increases in halide builds. llvm-svn: 357116
OpenPOWER on IntegriCloud