summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG
Commit message (Collapse)AuthorAgeFilesLines
* [DAG] Fix constant store folding to handle non-byte sizes.Nirav Dave2019-02-262-12/+14
| | | | | | | | | | | | | | | | Avoid crashes from zero-byte values due to sub-byte store sizes. Reviewers: uabelho, courbet, rnk Reviewed By: courbet Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D58626 llvm-svn: 354884
* [LegalizeDAG] Use APInt::getSplat helper to create bitreverse masks. NFCI.Simon Pilgrim2019-02-261-10/+6
| | | | llvm-svn: 354867
* [LegalizeDAG] Expand SADDO/SSUBO using SADDSAT/SSUBSAT (PR37763)Simon Pilgrim2019-02-261-5/+17
| | | | | | | | | | If SADDSAT/SSUBSAT are legal, then we can expand SADDO/SSUBO by performing a ADD/SUB and a SADDO/SSUBO and then compare the results. I looked at doing this for UADDO/USUBO as well but as we don't have to do as many range comparisons I didn't see any/much benefit. Differential Revision: https://reviews.llvm.org/D58637 llvm-svn: 354866
* Fix a sign compare warning breaking the -Werror build.Andrea Di Biagio2019-02-251-1/+1
| | | | | | The warning was introduced at r354793. llvm-svn: 354810
* [SelectionDAG] Add demanded elts variants to isConstOrConstSplat helpers. NFCI.Simon Pilgrim2019-02-251-37/+74
| | | | | | | | | | | | These helpers extend the existing isConstOrConstSplat helper checks to support DemandedElts masks as well. We already had a local version of this in SelectionDAG that computeKnownBits/ComputeNumSignBits made use of, but this adds the functionality directly to the BuildVectorSDNode node and extends isConstOrConstSplat etc. to use that. This will allow us to reuse the functionality in SimplifyDemandedVectorElts/SimplifyDemandedBits. Differential Revision: https://reviews.llvm.org/D58503 llvm-svn: 354797
* [DAGCombine] Add undef shuffle elt support to partitionShuffleOfConcatsSimon Pilgrim2019-02-251-28/+29
| | | | | | | | Support undef shuffle mask indices in the shuffle(concat_vectors, concat_vectors) -> concat_vectors fold Differential Revision: https://reviews.llvm.org/D58585 llvm-svn: 354793
* [SelectionDAG] Add a OPC_CheckChild2CondCode to SelectionDAGISel to remove a ↵Craig Topper2019-02-251-0/+14
| | | | | | | | | | MoveChild and MoveParent pair. OPC_CheckCondCode is always used as operand 2 of a setcc. And its always surrounded by a MoveChild2 and a MoveParent. By having a dedicated opcode for this case we can reduce the number of bytes needed for this pattern from 4 bytes to 2. This saves ~3000 bytes in the X86 table. llvm-svn: 354763
* [LegalizeTypes][AArch64][X86] Make type legalization of vector ↵Craig Topper2019-02-242-7/+17
| | | | | | | | | | | | | | | | | | | | | (S/U)ADD/SUB/MULO follow getSetCCResultType for the overflow bits. Make UnrollVectorOverflowOp properly convert from scalar boolean contents to vector boolean contents Summary: When promoting the over flow vector for these ops we should use the target's desired setcc result type. This way a v8i32 result type will use a v8i32 overflow vector instead of a v8i16 overflow vector. A v8i16 overflow vector will cause LegalizeDAG/LegalizeVectorOps to have to use v8i32 and truncate to v8i16 in its expansion. By doing this in type legalization instead, we get the truncate into the DAG earlier and give DAG combine more of a chance to optimize it. We also have to fix unrolling to use the scalar setcc result type for the scalarized operation, and convert it to the required vector element type after the scalar operation. We have to observe the vector boolean contents when doing this conversion. The previous code was just taking the scalar result and putting it in the vector. But for X86 and AArch64 that would have only put a the boolean value in bit 0 of the element and left all other bits in the element 0. We need to ensure all bits in the element are the same. I'm using a select with constants here because that's what setcc unrolling in LegalizeVectorOps used. Reviewers: spatel, RKSimon, nikic Reviewed By: nikic Subscribers: javed.absar, kristof.beyls, dmgreen, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D58567 llvm-svn: 354753
* Recommit r354647 and r354648 "[LegalizeTypes] When promoting the result of ↵Craig Topper2019-02-231-3/+7
| | | | | | | | | | EXTRACT_SUBVECTOR, also check if the input needs to be promoted. Use that to determine the element type to extract" r354648 was a follow up to fix a regression "[X86] Add a DAG combine for (aext_vector_inreg (aext_vector_inreg X)) -> (aext_vector_inreg X) to fix a regression from my previous commit." These were reverted in r354713 as their context depended on other patches that were reverted for a bug. llvm-svn: 354734
* [NFC] Fix typos: preceeding -> precedingJordan Rupprecht2019-02-231-4/+4
| | | | llvm-svn: 354715
* Revert r354363 & co "[X86][SSE] Generalize X86ISD::BLENDI support to more ↵Reid Kleckner2019-02-231-7/+3
| | | | | | | | | | | | | | | value types" r354363 caused https://crbug.com/934963#c1, which has a plain C reduced test case. I also had to revert some dependent changes: - r354648 - r354647 - r354640 - r354511 llvm-svn: 354713
* [LegalizeTypes] Use PromoteTargetBoolean in PromoteIntOp_ADDSUBCARRY instead ↵Craig Topper2019-02-231-13/+1
| | | | | | of reimplementing it. NFCI llvm-svn: 354710
* Disable big-endian constant store merges from rL354676.Nirav Dave2019-02-221-10/+11
| | | | llvm-svn: 354677
* [DAGCombine] Fold overlapping constant storesNirav Dave2019-02-222-3/+28
| | | | | | | | | | | | | | | Fold a smaller constant store into larger constant stores immediately preceeding it. Reviewers: rnk, courbet Subscribers: javed.absar, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D58468 llvm-svn: 354676
* [LegalizeVectorOps] Improve the placement of ANDs in the ExpandLoad path for ↵Craig Topper2019-02-221-6/+7
| | | | | | | | | | | | non-byte-sized loads. When we need to merge two adjacent loads the AND mask for the low piece was still sized for the full src element size. But we didn't have that many bits. The upper bits are already zero due to the SRL. So we can skip the AND if we're going to combine with the high bits. We do need an AND to clear out any bits from the high part. We were anding the high part before combining with the low part, but it looks like ANDing after the OR gets better results. So we can just emit the final AND after the optional concatentation is done. That will handling skipping before the OR and get rid of extra high bits after the OR. llvm-svn: 354655
* [LegalizeVectorOps] Simplify the non-byte sized load handling ↵Craig Topper2019-02-221-11/+8
| | | | | | | | VectorLegalizer::ExpandLoad. NFCI Remove an if that should always be true. Merge the body of another into the only block that could make the if true. llvm-svn: 354654
* DAG: Add helper for creating shifts with correct typeMatt Arsenault2019-02-222-1/+8
| | | | llvm-svn: 354649
* [LegalizeTypes] When promoting the result of EXTRACT_SUBVECTOR, also check ↵Craig Topper2019-02-221-3/+7
| | | | | | | | | | | | | | if the input needs to be promoted. Use that to determine the element type to extract. Otherwise we end up creating extract_vector_elts that then each need to have their input promoted. This can lead to truncates needing to be emitted for each of those. But we already emitted any_extends when we legalized the extract_subvector. So now we have pairs of any_extend+trunc that partially cancel. But depending on how DAGCombiner visits them we can get weird results. By promoting the input at the same time we can create only a single any_extend or truncate. There's one regression in the vector-narrow-binop.ll case, but that looks easy to fix with a follow up patch. llvm-svn: 354647
* [DAGCombiner] prevent infinite looping by truncating 'and' (PR40793)Sanjay Patel2019-02-211-2/+3
| | | | | | | | | | | | | | This fold can occur during legalization, so it can fight with promotion to the larger type. It apparently takes a special sequence and subtarget to avoid more basic simplifications that would hide the problem. But there's a bigger question raised here: why does distributeTruncateThroughAnd() even exist? It duplicates functionality from a more minimal pattern that we already have. But getting rid of this function requires some preliminary steps. https://bugs.llvm.org/show_bug.cgi?id=40793 llvm-svn: 354594
* Re-land part of r354244 "[DAGCombiner] Eliminate dead stores to stack."Clement Courbet2019-02-213-10/+44
| | | | | | This part introduces the lifetime node. llvm-svn: 354578
* [DAGCombine] Generalize Dead Store to overlapping stores.Nirav Dave2019-02-201-14/+17
| | | | | | | | | | | | | | | | | | Summary: Remove stores that are immediately overwritten by larger stores. Reviewers: courbet, rnk Reviewed By: rnk Subscribers: javed.absar, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D58467 llvm-svn: 354518
* [SelectionDAG] Teach GetDemandedBits to look at the known zeros of the LHS ↵Craig Topper2019-02-201-3/+7
| | | | | | | | | | | | when handling ISD::AND If the LHS has known zeros, then the RHS immediate mask might have been simplified to remove those bits. This patch adds a call to computeKnownBits to get the known zeroes to handle that possibility. I left an early out to skip the call if all of the demanded bits are set in the mask. Differential Revision: https://reviews.llvm.org/D58464 llvm-svn: 354514
* [SDAG] Support vector UMULO/SMULONikita Popov2019-02-204-19/+89
| | | | | | | | | | | | | | | Second part of https://bugs.llvm.org/show_bug.cgi?id=40442. This adds an extra UnrollVectorOverflowOp() method to SDAG, because the general UnrollOverflowOp() method can't deal with multiple results. Additionally we need to expand UMULO/SMULO during vector op legalization, as it may result in unrolling, which may need additional type legalization. Differential Revision: https://reviews.llvm.org/D57997 llvm-svn: 354513
* Revert r354498 "[X86] Add test case to show missed opportunity to remove an ↵Craig Topper2019-02-201-7/+3
| | | | | | | | explicit AND on the bit position from BT when it has known zeros." I accidentally committed more than just the test. llvm-svn: 354499
* [X86] Add test case to show missed opportunity to remove an explicit AND on ↵Craig Topper2019-02-201-3/+7
| | | | | | | | | | the bit position from BT when it has known zeros. If the bit position has known zeros in it, then the AND immediate will likely be optimized to remove bits. This can prevent GetDemandedBits from recognizing that the AND is unnecessary. llvm-svn: 354498
* Re-land the refactoring part of r354244 "[DAGCombiner] Eliminate dead stores ↵Clement Courbet2019-02-202-35/+80
| | | | | | | | to stack." This is an NFC. llvm-svn: 354476
* [NFC] add/modify wrapper function for findRegisterDefOperand().Chen Zheng2019-02-201-1/+2
| | | | llvm-svn: 354438
* [SDAG] Use shift amount type in MULO promotion; NFCNikita Popov2019-02-191-2/+4
| | | | | | | | | | Directly use the correct shift amount type if it is possible, and future-proof the code against vectors. The added test makes sure that bitwidths that do not fit into the shift amount type do not assert. Split out from D57997. llvm-svn: 354359
* Revert r354244 "[DAGCombiner] Eliminate dead stores to stack."Clement Courbet2019-02-185-209/+45
| | | | | | Breaks some bots. llvm-svn: 354245
* [DAGCombiner] Eliminate dead stores to stack.Clement Courbet2019-02-185-45/+209
| | | | | | | | | | | | | | | Summary: A store to an object whose lifetime is about to end can be removed. See PR40550 for motivation. Reviewers: niravd Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D57541 llvm-svn: 354244
* [SelectionDAG] Extract [US]MULO expansion into TL method; NFCNikita Popov2019-02-172-148/+124
| | | | | | | | | | | | In preparation for supporting vector expansion. Add an isPostTypeLegalization flag to makeLibCall(), because this expansion relies on the legalized form using MERGE_VALUES. Drop the corresponding variant of ExpandLibCall, which is no longer used. Differential Revision: https://reviews.llvm.org/D58006 llvm-svn: 354226
* [X86] Fix LowerAsmOutputForConstraint.Nirav Dave2019-02-152-3/+2
| | | | | | | | | | | | | | | | | Summary: Update Flag when generating cc output. Fixes PR40737. Reviewers: rnk, nickdesaulniers, craig.topper, spatel Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D58283 llvm-svn: 354163
* Fix 80-column limit in SimplifyDemandedBits/SimplifyDemandedVectorElts. NFCI.Simon Pilgrim2019-02-151-70/+78
| | | | llvm-svn: 354152
* Fix an accidentally flipped pair of arguments, NFCIJeremy Morse2019-02-141-1/+1
| | | | | | | | | While rebasing a refactor in r353950 I accidentally swapped two function arguments; one is SelectionDAGBuilders "current" DebugLoc, the other is the one from the "current" debug intrinsic. They're probably always identical, but I haven't proved that yet. llvm-svn: 354019
* [SelectionDAG] Inline a single use helper function, and remove last non-MMO ↵Philip Reames2019-02-132-33/+22
| | | | | | | | interface [NFC] For D57601, we need to know whether the instruction is volatile. We'd either have to pass yet another parameter, or just standardize on the MMO interface. I chose the second. llvm-svn: 353989
* [SelectionDAG] Kill last uses of getAtomic w/o a MMO operand [NFC]Philip Reames2019-02-132-44/+36
| | | | | | The helper function was used by only two callers, and largely ended up providing distinct functionality based on optional arguments and opcode. Inline and simply to make the functionality much more clear. llvm-svn: 353977
* [DebugInfo][DAG] Either salvage dangling debug info or emit Undef DBG_VALUEsJeremy Morse2019-02-133-19/+96
| | | | | | | | | | | | | | | | | | | | | | | | | In this patch SelectionDAG tries to salvage any dbg.values that are going to be dropped, in case they can be recovered from Values in the current BB. It also strengthens SelectionDAGs handling of dangling debug data, so that dbg.values are *always* emitted (as Undef or otherwise) instead of dangling forever. The motivation behind this patch exists in the new test case: a memory address (here a bitcast and GEP) exist in one basic block, and a dbg.value referring to the address is left in the 'next' block. The base pointer is live across all basic blocks. In current llvm trunk the dbg.value cannot be encoded, and it isn't even emitted as an Undef DBG_VALUE. The change is simply: if we're definitely going to drop a dbg.value, repeatedly apply salvageDebugInfo to its operand until either we find something that can be encoded, or we can't salvage any further in which case we produce an Undef DBG_VALUE. To know when we're "definitely going to drop a dbg.value", SelectionDAG signals SelectionDAGBuilder when all IR instructions have been encoded to force salvaging. This ensures that any dbg.value that's dangling after DAG creation will have a corresponding DBG_VALUE encoded. Differential Revision: https://reviews.llvm.org/D57694 llvm-svn: 353954
* [DebugInfo][DAG] Refactor dbg.value lowering into its own methodJeremy Morse2019-02-132-84/+100
| | | | | | | | | | This is a pure copy-and-paste job, moving the logic for lowering dbg.value intrinsics to SDDbgValues into its own function. This is ahead of adding some more users of this logic. Differential Revision: https://reviews.llvm.org/D57697 llvm-svn: 353950
* [DebugInfo][DAG] Limit special-casing of dbg.values for ArgumentsJeremy Morse2019-02-131-4/+10
| | | | | | | | | | | | | | | | | | | | | SelectionDAGBuilder has special handling for dbg.value intrinsics that are understood to define the location of function parameters on entry to the function. To enable this, we avoid recording a dbg.value as a virtual register reference if it might be such a parameter, so that it later hits EmitFuncArgumentDbgValue. This patch reduces the set of circumstances where we avoid recording a dbg.value as a virtual register reference, to allow more "normal" variables to be recorded that way. We now only bypass for potential parameters if: * The dbg.value operand is an Argument, * The Variable is a parameter, and * The Variable is not inlined. meaning it's very likely that the dbg.value is a function-entry parameter location. Differential Revision: https://reviews.llvm.org/D57584 llvm-svn: 353948
* [SelectionDAG] Clean up comments in SelectionDAGBuilder.h. NFCBjorn Pettersson2019-02-121-46/+42
| | | | | | | Remove redundant function/variable names from doxygen comments (as suggested in https://reviews.llvm.org/D57697). llvm-svn: 353886
* [DAGCombiner] convert logic-of-setcc into bit magic (PR40611)Sanjay Patel2019-02-121-0/+26
| | | | | | | | | | | | | | | | | | | | If we're comparing some value for equality against 2 constants and those constants have an absolute difference of just 1 bit, then we can offset and mask off that 1 bit and reduce to a single compare against zero: and/or (setcc X, C0, ne), (setcc X, C1, ne/eq) --> setcc ((add X, -C1), ~(C0 - C1)), 0, ne/eq https://rise4fun.com/Alive/XslKj This transform is disabled by default using a TLI hook ("convertSetCCLogicToBitwiseLogic()"). That should be overridden for AArch64, MIPS, Sparc and possibly others based on the asm shown in: https://bugs.llvm.org/show_bug.cgi?id=40611 llvm-svn: 353859
* [SelectionDAG] Fix return calling convention in expansion of ?MULOwhitequark2019-02-121-3/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The SMULO/UMULO DAG nodes, when not directly supported by the target, expand to a multiplication twice as wide. In case that the resulting type is not legal, the legalizer cannot directly call the intrinsic with the wide arguments; instead, it "pre-lowers" them by splitting them in halves. rL283203 made sure that on big endian targets, the legalizer passes the argument halves in the correct order. It did not do the same for the return value halves because the existing code used a hack; it put an illegal type into DAG and hoped that nothing would break and it would be correctly lowered elsewhere. rL307207 fixed this, handling return value halves similar to how argument handles are handled, but did not take big-endian targets into account. This commit fixes the expansion on big-endian targets, such as the out-of-tree OR1K target. Reviewers: eli.friedman, vadimcn Subscribers: george-hopkins, efriedma, llvm-commits Differential Revision: https://reviews.llvm.org/D45355 llvm-svn: 353854
* [Statepoint Lowering] Update misleading comments about chainsPhilip Reames2019-02-121-9/+7
| | | | llvm-svn: 353800
* [LegalizeTypes] Expand FNEG to bitwise op for IEEE FP typesAna Pazos2019-02-111-0/+9
| | | | | | | | | | | | | | | | | Summary: Except for custom floating point types x86_fp80 and ppc_fp128, expand Y = FNEG(X) to Y = X ^ sign mask to avoid library call. Using bitwise operation can improve code size and performance. Reviewers: efriedma Reviewed By: efriedma Subscribers: efriedma, kpn, arsenm, eli.friedman, javed.absar, rbar, johnrusso, simoncook, sabuasal, niosHD, zzheng, edward-jones, rogfer01, MartinMosbeck, brucehoult, the_o, PkmX, jocewei, asb, llvm-commits Differential Revision: https://reviews.llvm.org/D57875 llvm-svn: 353757
* [SelectionDAGBuilder] Add restrictions to EmitFuncArgumentDbgValueBjorn Pettersson2019-02-112-0/+72
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch fixes PR40587. When a dbg.value instrinsic is emitted to the DAG by using EmitFuncArgumentDbgValue the resulting DBG_VALUE is hoisted to the beginning of the entry block. I think the idea is to be able to locate a formal argument already from the start of the function. However, EmitFuncArgumentDbgValue only checked that the value that was used to describe a variable was originating from a function parameter, not that the variable itself actually was an argument to the function. So when for example assigning a local variable "local" the value from an argument "a", the assocated DBG_VALUE instruction would be hoisted to the beginning of the function, even if the scope for "local" started somewhere else (or if "local" was mapped to other values earlier in the function). This patch adds some logic to EmitFuncArgumentDbgValue to check that the variable being described actually is an argument to the function. And that the dbg.value being lowered already is in the entry block. Otherwise we bail out, and the dbg.value will be handled as an ordinary dbg.value (not as a "FuncArgumentDbgValue"). A tricky situation is when both the variable and the value is related to function arguments, but not neccessarily the same argument. We make sure that we do not describe the same argument more than once as a "FuncArgumentDbgValue". This solution works as long as opt has injected a "first" dbg.value that corresponds to the formal argument at the function entry. Reviewers: jmorse, aprantl Subscribers: jyknight, hiraditya, fedor.sergeev, dstenb, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D57702 llvm-svn: 353735
* Move some classes into anonymous namespaces. NFC.Benjamin Kramer2019-02-111-0/+2
| | | | llvm-svn: 353710
* [CallSite removal] Migrate the statepoint GC infrastructure to use theChandler Carruth2019-02-114-38/+37
| | | | | | | | | | | | | | | `CallBase` class rather than `CallSite` wrappers. I pushed this change down through most of the statepoint infrastructure, completely removing the use of CallSite where I could reasonably do so. I ended up making a couple of cut-points: generic call handling (instcombine, TLI, SDAG). As soon as it hit truly generic handling with users outside the immediate code, I simply transitioned into or out of a `CallSite` to make this a reasonable sized chunk. Differential Revision: https://reviews.llvm.org/D56122 llvm-svn: 353660
* [CodeGen][X86] Don't scalarize vector saturating add/subNikita Popov2019-02-101-15/+6
| | | | | | | | | | | Now that we have vector support for [US](ADD|SUB)O we no longer need to scalarize when expanding [US](ADD|SUB)SAT. This matches what the cost model already does. Differential Revision: https://reviews.llvm.org/D57348 llvm-svn: 353651
* [DAG] Add optional AllowUndefs to isNullOrNullSplatSimon Pilgrim2019-02-102-7/+3
| | | | | | No change in default behaviour (AllowUndefs = false) llvm-svn: 353646
* [DAGCombine] Simplify funnel shifts with undef/zero args to bitshiftsSimon Pilgrim2019-02-101-2/+41
| | | | | | | | Now that we have SimplifyDemandedBits support for funnel shifts (rL353539), we need to simplify funnel shifts back to bitshifts in cases where either argument has been folded to undef/zero. Differential Revision: https://reviews.llvm.org/D58009 llvm-svn: 353645
OpenPOWER on IntegriCloud