summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG
Commit message (Collapse)AuthorAgeFilesLines
...
* [TargetLowering] Remove redundant term in two ifs in SimplifySetCC. NFCCraig Topper2018-03-091-2/+2
| | | | | | If we get into the right hand side of the OR, we know that isOpaque is true since the left hand size is just a check that it's false. llvm-svn: 327179
* Improve Dependency analysis when doing multi-node Instruction SelectionNirav Dave2018-03-091-215/+76
| | | | | | | | | | | | | | | | | | | | Relanding after fixing NodeId Invariant. Cleanup cycle/validity checks in ISel (IsLegalToFold, HandleMergeInputChains) and X86 (isFusableLoadOpStore). Now do a full search for cycles / dependencies pruning the search when topological property of NodeId allows. As part of this propogate the NodeId-based cutoffs to narrow hasPreprocessorHelper searches. Reviewers: craig.topper, bogner Subscribers: llvm-commits, hiraditya Differential Revision: https://reviews.llvm.org/D41293 llvm-svn: 327171
* [DAG] Enforce stricter NodeId invariant during Instruction selectionNirav Dave2018-03-091-13/+93
| | | | | | | | | | | | | | | | | | | | | | | | | | | Instruction Selection makes use of the topological ordering of nodes by node id (a node's operands have smaller node id than it) when doing cycle detection. During selection we may violate this property as a selection of multiple nodes may induce a use dependence (and thus a node id restriction) between two unrelated nodes. If a selected node has an unselected successor this may allow us to miss a cycle in detection an invalid selection. This patch fixes this by marking all unselected successors of a selected node have negated node id. We avoid pruning on such negative ids but still can reconstruct the original id for pruning. In-tree targets have been updated to replace DAG-level replacements with ISel-level ones which enforce this property. This preemptively fixes PR36312 before triggering commit r324359 relands Reviewers: craig.topper, bogner, jyknight Subscribers: arsenm, nhaehnle, javed.absar, llvm-commits, hiraditya Differential Revision: https://reviews.llvm.org/D43198 llvm-svn: 327170
* Make early exit hasPredecessorHelper return true. NFCI.Nirav Dave2018-03-091-5/+1
| | | | | | | All uses conservatively assume in early exit case that it will be a predecessor. Changing default removes checking code in all uses. llvm-svn: 327169
* Use branch funnels for virtual calls when retpoline mitigation is enabled.Peter Collingbourne2018-03-091-0/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The retpoline mitigation for variant 2 of CVE-2017-5715 inhibits the branch predictor, and as a result it can lead to a measurable loss of performance. We can reduce the performance impact of retpolined virtual calls by replacing them with a special construct known as a branch funnel, which is an instruction sequence that implements virtual calls to a set of known targets using a binary tree of direct branches. This allows the processor to speculately execute valid implementations of the virtual function without allowing for speculative execution of of calls to arbitrary addresses. This patch extends the whole-program devirtualization pass to replace certain virtual calls with calls to branch funnels, which are represented using a new llvm.icall.jumptable intrinsic. It also extends the LowerTypeTests pass to recognize the new intrinsic, generate code for the branch funnels (x86_64 only for now) and lay out virtual tables as required for each branch funnel. The implementation supports full LTO as well as ThinLTO, and extends the ThinLTO summary format used for whole-program devirtualization to support branch funnels. For more details see RFC: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120672.html Differential Revision: https://reviews.llvm.org/D42453 llvm-svn: 327163
* [x86][aarch64] ask the backend whether it has a vector blend instructionSebastian Pop2018-03-091-24/+26
| | | | | | | | | | | | | The code to match and produce more x86 vector blends was enabled for all architectures even though the transform may pessimize the code for other architectures that do not provide a vector blend instruction. Added an aarch64 testcase to check that a VZIP instruction is generated instead of byte movs. Differential Revision: https://reviews.llvm.org/D44118 llvm-svn: 327132
* Propagate flags to SDValue in SplitVecOp_VECREDUCESameer AbuAsal2018-03-081-2/+2
| | | | | | | | | | | | This patch is a fix for PR36642. While legalizing long vector types, make sure the smaller types get the flags of the wider type. bugzilla link: https://bugs.llvm.org/show_bug.cgi?id=36642 Change-Id: I0c2829639f094c862c10a6b51b342d4c2563e1fa llvm-svn: 327079
* [TargetLowering] Remove redundant if condition in SimplifySetcc. NFCCraig Topper2018-03-081-1/+1
| | | | | | We were checking the condition code a second time when we were already in a block with this same condition code check. llvm-svn: 327069
* [TargetLowering] Add vector BITCAST support to SimplifyDemandedVectorEltsSimon Pilgrim2018-03-061-0/+68
| | | | | | | | Notably helps cleanup after legalization of vector types Differential Revision: https://reviews.llvm.org/D43674 llvm-svn: 326838
* Fix an unused variable warning introduced by rr326703. NFCEric Liu2018-03-051-1/+3
| | | | llvm-svn: 326732
* Pass Divergence Analysis data to Selection DAG to drive divergenceAlexander Timofeev2018-03-054-5/+135
| | | | | | | | dependent instruction selection. Differential revision: https://reviews.llvm.org/D35267 llvm-svn: 326703
* [DAGCombiner] Add a peekThroughBitcast to MergeStoresOfConstantsOrVecElts to ↵Craig Topper2018-03-041-0/+1
| | | | | | | | fix a crash if we are storing a bitcast of a constant. Loading a constant into a k-register in AVX512 requires a bitcast from a scalar constant. In the test case here we have a k-register store that gets split into multiple parts of KNL. MergeConsecutiveStores sees each of these pieces as a consecutive store and looks through the bitcast to find the underly scalar constant. But when we went to create the combined store we didn't look through the same bitcast. llvm-svn: 326677
* [LegalizeVectorTypes] When scalarizing the operand of a unary op like TRUNC, ↵Craig Topper2018-03-021-1/+1
| | | | | | | | | | | | use a SCALAR_TO_VECTOR rather than a single element BUILD_VECTOR to convert back to a vector type. X86 considers v1i1 a legal type under AVX512 and as such a truncate from a v1iX type to v1i1 can be turned into a scalar truncate plus a conversion to v1i1. We would much prefer a v1i1 SCALAR_TO_VECTOR over a one element BUILD_VECTOR. During lowering we were detecting the v1i1 BUILD_VECTOR as a splat BUILD_VECTOR like we try to do for v2i1/v4i1/etc. In this case we create (select i1 splat_elt, v1i1 all-ones, v1i1 all-zeroes). That goes through some more legalization and we end up with a CMOV choosing between 0 and 1 in scalar and a scalar_to_vector. Arguably we could detect the v1i1 BUILD_VECTOR and do this better in X86 target code. But just using a SCALAR_TO_VECTOR in legalization is much easier. llvm-svn: 326637
* Add DBG_VALUE support to the linear DAG schedulerAdrian Prantl2018-03-021-1/+14
| | | | | | | | | | | The fast/linear DAG scheduler doesn't lower DBG_VALUEs except for function entry nodes. Patch by Joshua Cranmer! Differential Revision: https://reviews.llvm.org/D43028 llvm-svn: 326631
* [DAGCombiner] When combining zero_extend of a truncate, only mask before ↵Craig Topper2018-03-011-1/+1
| | | | | | | | | | extending for vectors. Masking first, prevents the extend from being combine with loads. Its also interfering with some vXi1 extraction code. Differential Revision: https://reviews.llvm.org/D42679 llvm-svn: 326500
* [SelectionDAG] Support some SimplifySetCC cases for comparing against vector ↵Craig Topper2018-03-011-3/+6
| | | | | | | | | | | | | | splats of constants. This supports things like (setcc ugt X, 0) -> (setcc ne X, 0) I've restricted to only make changes to vectors before legalize ops because I doubt all targets have accurate condition code legality information for vectors given how little we did before. Differential Revision: https://reviews.llvm.org/D42948 llvm-svn: 326495
* [CodeGen] fix argument attribute in lowering statepoint/patchpointThan McIntosh2018-03-012-2/+2
| | | | | | | | | | | | | | | Summary: Use the correct loop index varaible, ArgI, to retrieve attributes. Reviewers: thanm, sanjoy, rnk Reviewed By: rnk Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D43832 llvm-svn: 326433
* [SelectionDAG] Remove code from PromoteIntRes_CONCAT_VECTORS that was added ↵Craig Topper2018-02-271-15/+0
| | | | | | | | | | in r320674 to help X86. AVX512 used to promote v32i1 to v32i8 during legalization when BWI was disabled. So this code was added to improve legalization of v32i1 concat_vectors of v16i1 by extending the v16i1 to v16i8 to avoid scalarization. X86 has since switched to legalizing v32i1 by splitting to v16i1 instead. This has rendered this code unnecessary and its no longer exercised. llvm-svn: 326153
* [TargetLowering] SimplifyDemandedVectorElts - pass demanded elts through ↵Simon Pilgrim2018-02-241-0/+13
| | | | | | ADD/SUB ops llvm-svn: 326044
* [TargetLowering] SimplifyDemandedVectorElts - pass demanded elts through ↵Simon Pilgrim2018-02-241-0/+5
| | | | | | TRUNCATE ops llvm-svn: 326043
* [DAGCOmbine] Ensure that (brcond (setcc ...)) is handled in a canonical manner.Amaury Sechet2018-02-231-75/+67
| | | | | | | | | | | | | | | Summary: There are transformation that change setcc into other constructs, and transform that try to reconstruct a setcc from the brcond condition. Depending on what order these transform are done, the end result differs. Most of the time, it is preferable to get a setcc as a brcond argument (and this is why brcond try to recreate the setcc in the first place) so we ensure this is done every time by also doing it at the setcc level when the only user is a brcond. Reviewers: spatel, hfinkel, niravd, craig.topper Subscribers: nhaehnle, llvm-commits Differential Revision: https://reviews.llvm.org/D41235 llvm-svn: 325892
* [TargetLowering] Rename isCondCodeLegal to isCondCodeLegalOrCustom. Add real ↵Craig Topper2018-02-222-13/+12
| | | | | | | | | | | | | | | | isCondCodeLegal. Update callers to use one or the other. isCondCodeLegal internally checked Legal or Custom which is misleading. Though no targets set any cond code action to Custom today. So I've renamed isCondCodeLegal to isCondCodeLegalOrCustom and added a real isCondCodeLegal that only checks Legal. I've changed legalization code to use isCondCodeLegalOrCustom and left things reachable via DAG combine as isCondCodeLegal. I've also changed some places that called getCondCodeAction and compared to Legal to just use isCondCodeLegal. I'm looking at trying to keep SETCC all the way to isel for the AVX512 integer comparisons and I suspect I'll need to make some condition codes Custom to stop DAG combine from changing things post LegalizeOps. Prior to this only Expand stopped DAG combine, but that causes LegalizeOps to try to swap operands or invert rather than calling our Custom handler. Differential Revision: https://reviews.llvm.org/D43607 llvm-svn: 325829
* Revert "[DebugInfo][FastISel] Fix dropping dbg.value()"Sander de Smalen2018-02-221-1/+1
| | | | | | | | | This patch reverts r325440 and r325438 because it triggers an assertion in SelectionDAGBuilder.cpp. Also having debug enabled may unintentionally affect code-gen. The patch is reverted until we find a better solution. llvm-svn: 325825
* [SelectionDAG] Move matchUnaryPredicate/matchBinaryPredicate into ↵Simon Pilgrim2018-02-222-62/+58
| | | | | | | | | | SelectionDAGNodes.h This allows us to improve vector constant matching in more DAG code (backends, TargetLowering etc.). Differential Revision: https://reviews.llvm.org/D43466 llvm-svn: 325815
* [DAGCombiner] Add two calls to isVector before making calls to ↵Craig Topper2018-02-221-4/+5
| | | | | | | | | | getVectorElementType/getVectorNumElements to avoid an assert. We looked through a BITCAST, but the bitcast might be a from a scalar type rather than a vector. I don't have a test case. I stumbled onto it while prototyping another change that isn't ready yet. llvm-svn: 325750
* [SelectionDAG] Support known true/false SimplifySetCC cases for comparing ↵Craig Topper2018-02-201-58/+87
| | | | | | | | | | | | | | against vector splats of constants. This is split off from D42948 and includes just the cases that constant fold to true or false. It also includes some refactoring to keep predicate checks together. This supports things like (setcc uge X, 0) -> true Differential Revision: https://reviews.llvm.org/D43489 llvm-svn: 325627
* [SelectionDAG] Add LegalTypes flag to getShiftAmountTy. Use it to unify and ↵Craig Topper2018-02-202-16/+11
| | | | | | | | | | | | | | | | | | simplify DAGCombiner and simplifySetCC code and fix a bug. DAGCombiner and SimplifySetCC both use getPointerTy for shift amounts pre-legalization. DAGCombiner uses a single helper function to hide this. SimplifySetCC does it in multiple places. This patch adds a defaulted parameter to getShiftAmountTy that can make it return getPointerTy for scalar types. Use this parameter to simplify the SimplifySetCC and DAGCombiner. Additionally, there were two places in SimplifySetCC that were creating shifts using the target's preferred shift amount pre-legalization. If the target uses a narrow type and the type is illegal, this can cause SimplfiySetCC to create a shift with an amount that can't represent all possible shift values for the type. To fix this we should use pointer type there too. Alternatively we could make getScalarShiftAmountTy for each target return a safe value for large types as proposed in D43445. And maybe we should still do that, but fixing the SimplifySetCC code keeps other targets from tripping over this in the future. Fixes PR36250. Differential Revision: https://reviews.llvm.org/D43449 llvm-svn: 325602
* [VectorLegalizer] Fix uint64_t typo in ExpandUINT_TO_FLOAT (PR36391)Simon Pilgrim2018-02-201-1/+1
| | | | | | | | ExpandUINT_TO_FLOAT can accept vXi32 or vXi64 inputs, so we need to use a uint64_t shift to generate the 2^(BW/2) constant. No test case unfortunately as no upstream target uses this, but its affecting a downstream target. llvm-svn: 325578
* [SelectionDAG] ComputeKnownBits - add support for SMIN+SMAX clamp patternsSimon Pilgrim2018-02-191-5/+32
| | | | | | | | | | If we have a clamp pattern, SMIN(SMAX(X, LO),HI) or SMAX(SMIN(X, HI),LO) then we can deduce that the number of signbits (zeros/ones) will be at least the minimum of the LO and HI constants. ComputeKnownBits equivalent of D43338. Differential Revision: https://reviews.llvm.org/D43463 llvm-svn: 325521
* Fix Wparentheses warning. NFCISimon Pilgrim2018-02-171-1/+1
| | | | llvm-svn: 325451
* [SelectionDAG] ComputeNumSignBits - add support for SMIN+SMAX clamp patternsSimon Pilgrim2018-02-171-1/+26
| | | | | | | | | | If we have a clamp pattern, SMIN(SMAX(X, LO),HI) or SMAX(SMIN(X, HI),LO) then we can deduce that the number of signbits will be at least the minimum of the LO and HI constants. I haven't bothered with the UMIN/UMAX equivalent as (1) we don't have any current use cases and (2) I wonder if we'd be better off immediately falling back for ComputeKnownBits for UMIN/UMAX which already has optimization patterns useful for unsigned cases. Differential Revision: https://reviews.llvm.org/D43338 llvm-svn: 325450
* [SelectionDAG] SimplifyDemandedVectorElts - add support for VECTOR_INSERT_ELTSimon Pilgrim2018-02-171-0/+34
| | | | | | Differential Revision: https://reviews.llvm.org/D43431 llvm-svn: 325449
* [DebugInfo][FastISel] Fix dropping dbg.value()Sander de Smalen2018-02-171-1/+1
| | | | | | | | | | | | | | | | | | | | | | | Summary: https://llvm.org/PR36263 shows that when compiling at -O0 a dbg.value() instruction (that remains from an original dbg.declare()) is dropped by FastISel. Since FastISel selects instructions by iterating a basic block backwards, it drops the dbg.value if one of its operands is not yet instantiated by a previously selected instruction. Instead of calling 'lookUpRegForValue()' we can call 'getRegForValue()' instead that will insert a placeholder for the operand to be filled in when continuing the instruction selection. Reviewers: aprantl, dblaikie, probinson Reviewed By: aprantl Subscribers: llvm-commits, dstenb, JDevlieghere Differential Revision: https://reviews.llvm.org/D43386 llvm-svn: 325438
* [DAGCombiner] Remove simplifyShuffleMask - now handled more generally by ↵Simon Pilgrim2018-02-171-35/+0
| | | | | | SimplifyDemandedVectorElts. llvm-svn: 325429
* [DAG, X86] Revert r324797, r324491, and r324359.Chandler Carruth2018-02-171-79/+215
| | | | | | | | | | | | Sadly, r324359 caused at least PR36312. There is a patch out for review but it seems to be taking a bit and we've already had these crashers in tree for too long. We're hitting this PR in real code now and are blocked on shipping new compilers as a consequence so I'm reverting us back to green. Sorry for the churn due to the stacked changes that I had to revert. =/ llvm-svn: 325420
* Fix signed/unsigned comparison warning. NFCI.Simon Pilgrim2018-02-161-1/+1
| | | | llvm-svn: 325363
* Fix signed/unsigned comparison warning. NFCI.Simon Pilgrim2018-02-161-3/+3
| | | | llvm-svn: 325359
* [SelectionDAG] Enable SimplifyDemandedVectorElts support for simplifying ↵Simon Pilgrim2018-02-161-0/+25
| | | | | | | | | | shuffle masks Based off the DemandedElts mask the and UNDEF elements returned from the SimplifyDemandedVectorElts calls to the shuffle operands, we can attempt to simplify the shuffle mask. I had to be very conservative here as accepting post-legalized shuffle masks could cause problems for targets that legalize UNDEF mask elements back to inrange values (PowerPC), similarly combining to identity shuffle masks could cause too much UNDEF information to disappear for later combines. llvm-svn: 325354
* [SelectionDAG] Add initial SimplifyDemandedVectorElts support for ↵Simon Pilgrim2018-02-161-0/+20
| | | | | | | | simplifying VSELECT operands This just adds a basic pass through - we can add constant selection mask handling in a future patch to fully match InstCombine. llvm-svn: 325338
* [LegalizeDAG] Fix legalization of SETCCMikhail Maltsev2018-02-161-1/+1
| | | | | | | | | | | | | | | | | | | Summary: Currently when expanding a SETCC node into a SELECT_CC, LLVM uses an incorrect type for determining BooleanContent of the result. This patch fixes the issue. Fixes PR36079. Reviewers: rogfer01, javed.absar, efriedma Reviewed By: efriedma Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D43282 llvm-svn: 325325
* [DAGCombiner] Call ExtendUsesToFormExtLoad in (zext (and (load)))->(and ↵Craig Topper2018-02-151-13/+14
| | | | | | | | | | | | | | (zextload)) even when the and does not have multiple uses Same for the sign extend case. Currently we check for multiple uses on the binop. Then we call ExtendUsesToFormExtLoad to capture SetCCs that use the load. So we only end up finding any setccs when the and has additional uses and the load is used by a setcc. I don't think the and having multiple uses is relevant here. I think we should only be checking for the load having multiple uses. This changes an NVPTX test because we now find that the load has a second use by a truncate, but ExtendUsesToFormExtLoad only looks at setccs it can extend. All other operations just check isTruncateFree. Maybe we should allow widening of an existing truncate even if its not free? Differential Revision: https://reviews.llvm.org/D43063 llvm-svn: 325289
* [SelectionDAG] Pull out repeated Op.getOpcode(). NFCI.Simon Pilgrim2018-02-151-6/+7
| | | | llvm-svn: 325253
* [SelectionDAG] Add initial implementation of ↵Simon Pilgrim2018-02-152-91/+238
| | | | | | | | | | | | TargetLowering::SimplifyDemandedVectorElts This is mainly a move of simplifyShuffleOperands from DAGCombiner::visitVECTOR_SHUFFLE to create a more general purpose TargetLowering::SimplifyDemandedVectorElts implementation. Further features can be moved/added in future patches. Differential Revision: https://reviews.llvm.org/D42896 llvm-svn: 325232
* [SelectionDAG][X86] Fix incorrect offset generated for VMASKMOVAlexander Ivchenko2018-02-142-18/+21
| | | | | | | | When creating high MachineMemOperand for MSTORE/MLOAD we supply it with the original PointerInfo, while the pointer itself had been incremented. The patch adds the proper offset to the PointerInfo. llvm-svn: 325135
* Adding a width of the GEP index to the Data Layout.Elena Demikhovsky2018-02-142-9/+8
| | | | | | | | | | | | | | | | | | Making a width of GEP Index, which is used for address calculation, to be one of the pointer properties in the Data Layout. p[address space]:size:memory_size:alignment:pref_alignment:index_size_in_bits. The index size parameter is optional, if not specified, it is equal to the pointer size. Till now, the InstCombiner normalized GEPs and extended the Index operand to the pointer width. It works fine if you can convert pointer to integer for address calculation and all registered targets do this. But some ISAs have very restricted instruction set for the pointer calculation. During discussions were desided to retrieve information for GEP index from the Data Layout. http://lists.llvm.org/pipermail/llvm-dev/2018-January/120416.html I added an interface to the Data Layout and I changed the InstCombiner and some other passes to take the Index width into account. This change does not affect any in-tree target. I added tests to cover data layouts with explicitly specified index size. Differential Revision: https://reviews.llvm.org/D42123 llvm-svn: 325102
* [SelectionDAG] Remove duplicate code from TargetLowering::SimplifySetCC.Craig Topper2018-02-141-4/+0
| | | | | | This exact code already exists a little further up. llvm-svn: 325101
* [DAGCombiner] Add one use check to fold (not (and x, y)) -> (or (not x), ↵Craig Topper2018-02-131-2/+2
| | | | | | | | | | | | | | | | | | | (not y)) Summary: If the and has an additional use we shouldn't invert it. That creates an additional instruction. While there add a one use check to the transform above that looked similar. Reviewers: spatel, RKSimon Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D43225 llvm-svn: 325019
* [DAG] fix type of undef returned by getNode()Sanjay Patel2018-02-131-2/+2
| | | | | | | | The bug has been lying dormant, but apparently was never exposed, until after rL324941 because we didn't return the correct result for shifts with undef operands. llvm-svn: 325010
* [DAG] make binops with undef operands consistent with IRSanjay Patel2018-02-121-20/+7
| | | | | | | | | | | | | | | | | | | | | This started by noticing that scalar and vector types were producing different results with div ops in PR36305: https://bugs.llvm.org/show_bug.cgi?id=36305 ...but the problem is bigger. I couldn't keep it straight without a table, so I'm attaching that as a PDF to the review. The x86 tests in undef-ops.ll correspond to that table. Green means that instsimplify and the DAG agree on the result for all types. Red means the DAG was returning undef when IR was not. Yellow means the DAG was returning a non-undef result when IR returned undef. This patch assumes that we're currently doing the right thing in IR. Note: I couldn't find any problems with lowering vector constants as the code comments were warning, but those comments were written long ago in rL36413 . Differential Revision: https://reviews.llvm.org/D43141 llvm-svn: 324941
* [AArch64] Improve v8.1-A code-gen for atomic load-andOliver Stannard2018-02-124-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | Armv8.1-A added an atomic load-clear instruction (which performs bitwise and with the complement of it's operand), but not a load-and instruction. Our current code-generation for atomic load-and always inserts an MVN instruction to invert its argument, even if it could be folded into a constant or another instruction. This adds lowering early in selection DAG to convert a load-and operation into an xor with -1 and a load-clear, allowing the normal DAG optimisations to work on it. To do this, I've had to add a new ISD opcode, ATOMIC_LOAD_CLR. I don't see any easy way to do this with an AArch64-specific ISD node, because the code-generation for atomic operations assumes the SDNodes are of type AtomicSDNode. I've left the old tablegen patterns in because they are still needed for global isel. Differential revision: https://reviews.llvm.org/D42478 llvm-svn: 324908
OpenPOWER on IntegriCloud