summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG
Commit message (Collapse)AuthorAgeFilesLines
* DAG: Add calling convention argument to calling convention funcsMatt Arsenault2018-07-285-104/+128
| | | | | | | | This seems like a pretty glaring omission, and AMDGPU wants to treat kernels differently from other calling conventions. llvm-svn: 338194
* [DAGCombiner] Teach DAG combiner that A-(B-C) can be folded to A+(C-B)Craig Topper2018-07-281-0/+6
| | | | | | | | This can be useful since addition is commutable, and subtraction is not. This matches a transform that is also done by InstCombine. llvm-svn: 338181
* [DAGCombiner] fold 'not' with signbit mathSanjay Patel2018-07-271-0/+45
| | | | | | | | | | | | | | | | | | | This is a follow-up suggested in D48970. Alive proofs: https://rise4fun.com/Alive/sII We can eliminate an instruction in the usual select-of-constants to bit hack transform by adjusting the add/sub with constant. This is always a win. There are more transforms that are likely wins, but they may need target hooks in case some targets do not benefit. This is another step towards making up for canonicalizing to select-of-constants in rL331486. llvm-svn: 338132
* DAG: Remove unnecessary .str()Matt Arsenault2018-07-271-1/+1
| | | | llvm-svn: 338112
* [SelectionDAGBuilder] Add masked loads to PendingLoads rather than calling ↵Craig Topper2018-07-261-4/+2
| | | | | | | | | | DAG.setRoot. Masked loads are calling DAG.getRoot rather than calling SelectionDAGBuilder::getRoot, which means the PendingLoads weren't emptied to update the root and create any needed TokenFactor. So it would be incorrect to call setRoot for the masked load. This patch instead adds the masked load to PendingLoads so that the root doesn't get update until a store or scatter or something happens.. Alternatively, we could call SelectionDAGBuilder::getRoot before it, but that would create unnecessary serialization. llvm-svn: 338085
* [SelectionDAG] Add MLOAD/MSTORE/MGATHER/MSCATTER to AddNodeIDCustom to ↵Craig Topper2018-07-261-0/+28
| | | | | | properly calculate their folding set ID to allow them to be CSEd. llvm-svn: 338080
* [DAGCombiner] Remove some calls to AddToWorklist that should be unnecessary.Craig Topper2018-07-261-3/+0
| | | | | | The DAGCombiner has a system for ensuring all nodes are visited. It doesn't require an AddToWorkList for every node that is created by a combine. llvm-svn: 338079
* [DebugInfo] LowerDbgDeclare: Add derefs when handling CallInst usersVedant Kumar2018-07-264-33/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | LowerDbgDeclare inserts a dbg.value before each use of an address described by a dbg.declare. When inserting a dbg.value before a CallInst use, however, it fails to append DW_OP_deref to the DIExpression. The DW_OP_deref is needed to reflect the fact that a dbg.value describes a source variable directly (as opposed to a dbg.declare, which relies on pointer indirection). This patch adds in the DW_OP_deref where needed. This results in the correct values being shown during a debug session for a program compiled with ASan and optimizations (see https://reviews.llvm.org/D49520). Note that ConvertDebugDeclareToDebugValue is already correct -- no changes there were needed. One complication is that SelectionDAG is unable to distinguish between direct and indirect frame-index (FRAMEIX) SDDbgValues. This patch also fixes this long-standing issue in order to not regress integration tests relying on the incorrect assumption that all frame-index SDDbgValues are indirect. This is a necessary fix: the newly-added DW_OP_derefs cannot be lowered properly otherwise. Basically the fix prevents a direct SDDbgValue with DIExpression(DW_OP_deref) from being dereferenced twice by a debugger. There were a handful of tests relying on this incorrect "FRAMEIX => indirect" assumption which actually had incorrect DW_AT_locations: these are all fixed up in this patch. Testing: - check-llvm, and an end-to-end test using lldb to debug an optimized program. - Existing unit tests for DIExpression::appendToStack fully cover the new DIExpression::append utility. - check-debuginfo (the debug info integration tests) Differential Revision: https://reviews.llvm.org/D49454 llvm-svn: 338069
* [DAGCombine] optimizeSetCCOfSignedTruncationCheck(): handle ule,ugt CondCodes.Roman Lebedev2018-07-261-9/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: A follow-up for D49266 / rL337166. At least one of these cases is more canonical, so we really do have to handle it. https://godbolt.org/g/pkzP3X https://rise4fun.com/Alive/pQyhZZ We won't get to these cases with I1 being -1, as that will be constant-folded to true or false. I'm also not sure we actually hit the 'ule' case, but i think the worst think that could happen is that being dead code. Reviewers: spatel, craig.topper, RKSimon, javed.absar, efriedma Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D49497 llvm-svn: 338044
* [SelectionDAG] try to convert funnel shift directly to rotate if legalSanjay Patel2018-07-251-1/+10
| | | | | | | | | | | If the DAGCombiner's rotate matching was working as expected, I don't think we'd see any test diffs here. This sidesteps the issue of custom lowering for rotates raised in PR38243: https://bugs.llvm.org/show_bug.cgi?id=38243 ...by only dealing with legal operations. llvm-svn: 337966
* Fix corruption of result number in LegalizeVectorOps.cppUlrich Weigand2018-07-251-1/+2
| | | | | | | | | | | | When VectorLegalizer::LegalizeOp creates a new SDValue after iterating over its arguments, we need to refer to the same result number of the new node that the original value used. Reviewed by: cameron.mcinally Differential Revision: https://reviews.llvm.org/D49805 llvm-svn: 337939
* Fix PR34170: Crash on inline asm with 64bit output in 32bit GPRThomas Preud'homme2018-07-251-20/+36
| | | | | | | | Add support for inline assembly with output operand that do not naturally go in the register class it is constrained to (eg. double in a 32-bit GPR as in the PR). llvm-svn: 337903
* [SelectionDAG] Reduce DanglingDebugInfo memory traffic, NFCVedant Kumar2018-07-231-19/+23
| | | | | | | This avoids approx. 2 x 10^5 DenseMap insertions in both non-debug and debug -O2 builds of the sqlite3 amalgamation. llvm-svn: 337751
* [Legalize] Elide MERGE_VALUES created by scalarizeVectorLoad.Nirav Dave2018-07-232-3/+10
| | | | | | | scalarizeVectorLoad creates MERGE_VALUES nodes which are immediately decomposed in expandLoad. Elide the node in these cases. llvm-svn: 337708
* [FPEnv] Legalize double width StrictFP vector operationsCameron McInally2018-07-232-0/+70
| | | | | | Differential Revision: https://reviews.llvm.org/D48809 llvm-svn: 337698
* [SelectionDAGBuilder] Use APInt::isZero instead of comparing ↵Craig Topper2018-07-221-1/+1
| | | | | | | | APInt::getZExtValue to 0 in a place where we can't be sure contents of the APInt fit in a uint64_t. This is used on an extract vector element index which is most cases is going to be an i32 or i64 and the element will be a valid element number. But it is possible to construct IR with a larger type and large out of range value. llvm-svn: 337652
* [SelectionDAGBuilder] Restrict vector reduction check to types with a power ↵Craig Topper2018-07-221-0/+4
| | | | | | | | of 2 number of elements. The check for the shuffles usages probably isn't correct for non power of 2 vectors. llvm-svn: 337651
* [DAG] Avoid Node Update assertion due to AND simplificationNirav Dave2018-07-201-3/+5
| | | | | | | | | | | | | | | | | Check for construction-time folding for incomplete AND nodes in BackwardsPropagateMask. Fixes PR38185. Reviewers: RKSimon, samparker Reviewed By: samparker Subscribers: llvm-commits, hiraditya Differential Revision: https://reviews.llvm.org/D49444 llvm-svn: 337563
* [DAG] Fix Memory ordering check in ReduceLoadOpStore.Nirav Dave2018-07-201-16/+18
| | | | | | | | | | | | | | | | | | When merging through a TokenFactor we need to check that the load may be ordered such that no other aliasing memory operations may happen. It is not sufficient to just check that the load is a member of the chain token factor as it there may be a indirect chain. Require the load's chain has only one use. This fixes PR37826. Reviewers: spatel, davide, efriedma, craig.topper, RKSimon Subscribers: hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D49388 llvm-svn: 337560
* [DAGCombiner] Fold X - (-Y *Z) -> X + (Y * Z)Craig Topper2018-07-201-0/+18
| | | | llvm-svn: 337518
* Skip out of SimplifyDemandedBits for BITCAST of f16 to i16Stephen Canon2018-07-191-0/+1
| | | | | | | | Mirrors the existing exit path for f128, avoiding a crash later on. Differential Revision: https://reviews.llvm.org/D49524 llvm-svn: 337506
* [DAGCombiner] Teach DAGCombiner that A-(-B) is A+B.Craig Topper2018-07-191-0/+5
| | | | | | We already knew A+(-B) is A-B in visitAdd. This does the opposite for visitSub. llvm-svn: 337502
* [ScheduleDAG] Fix unfolding of SUnits to already existent nodes.Nirav Dave2018-07-181-18/+30
| | | | | | | | | | | | | | | | | Summary: If unfolding an SUnit results in both load or the operation using it which already exist in the DAG, abort the unfold if they are already scheduled. If not, make sure we don't add duplicate dependencies. This fixes PR37916. Reviewers: davide, eli.friedman, fhahn, bogner Subscribers: MatzeB, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D48666 llvm-svn: 337409
* [DAGCombiner] Call SimplifyDemandedVectorElts from EXTRACT_VECTOR_ELTSimon Pilgrim2018-07-171-4/+23
| | | | | | | | If we are only extracting vector elements via EXTRACT_VECTOR_ELT(s) we may be able to use SimplifyDemandedVectorElts to avoid unnecessary vector ops. Differential Revision: https://reviews.llvm.org/D49262 llvm-svn: 337258
* [Intrinsics] define funnel shift IR intrinsics + DAG builder supportSanjay Patel2018-07-161-0/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | As discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-May/123292.html http://lists.llvm.org/pipermail/llvm-dev/2018-July/124400.html We want to add rotate intrinsics because the IR expansion of that pattern is 4+ instructions, and we can lose pieces of the pattern before it gets to the backend. Generalizing the operation by allowing 2 different input values (plus the 3rd shift/rotate amount) gives us a "funnel shift" operation which may also be a single hardware instruction. Initially, I thought we needed to define new DAG nodes for these ops, and I spent time working on that (much larger patch), but then I concluded that we don't need it. At least as a first step, we have all of the backend support necessary to match these ops...because it was required. And shepherding these through the IR optimizer is the primary concern, so the IR intrinsics are likely all that we'll ever need. There was also a question about converting the intrinsics to the existing ROTL/ROTR DAG nodes (along with improving the oversized shift documentation). Again, I don't think that's strictly necessary (as the test results here prove). That can be an efficiency improvement as a small follow-up patch. So all we're left with is documentation, definition of the IR intrinsics, and DAG builder support. Differential Revision: https://reviews.llvm.org/D49242 llvm-svn: 337221
* [CodeGen] Fix inconsistent declaration parameter nameFangrui Song2018-07-165-32/+32
| | | | llvm-svn: 337200
* [X86][AArch64][DAGCombine] Unfold 'check for [no] signed truncation' patternRoman Lebedev2018-07-161-0/+78
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: [[ https://bugs.llvm.org/show_bug.cgi?id=38149 | PR38149 ]] As discussed in https://reviews.llvm.org/D49179#1158957 and later, the IR for 'check for [no] signed truncation' pattern can be improved: https://rise4fun.com/Alive/gBf ^ that pattern will be produced by Implicit Integer Truncation sanitizer, https://reviews.llvm.org/D48958 https://bugs.llvm.org/show_bug.cgi?id=21530 in signed case, therefore it is probably a good idea to improve it. But the IR-optimal patter does not lower efficiently, so we want to undo it.. This handles the simple pattern. There is a second pattern with predicate and constants inverted. NOTE: we do not check uses here. we always do the transform. Reviewers: spatel, craig.topper, RKSimon, javed.absar Reviewed By: spatel Subscribers: kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D49266 llvm-svn: 337166
* Avoid losing Hi part when expanding VAARG nodes on big endian machinesDaniel Cederman2018-07-161-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | Summary: If the high part of the load is not used the offset to the next element will not be set correctly. For example, on Sparc V8, the following code will read val2 from offset 4 instead of 8. ``` int val = __builtin_va_arg(va, long long); int val2 = __builtin_va_arg(va, int); ``` Reviewers: jyknight Reviewed By: jyknight Subscribers: fedor.sergeev, jrtc27, llvm-commits Differential Revision: https://reviews.llvm.org/D48595 llvm-svn: 337161
* [DAGCombiner] fix typo in comment; NFCSanjay Patel2018-07-151-1/+1
| | | | llvm-svn: 337132
* [DAGCombiner] extend(ifpositive(X)) -> shift-right (not X)Sanjay Patel2018-07-151-0/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is almost the same as an existing IR canonicalization in instcombine, so I'm assuming this is a good early generic DAG combine too. The motivation comes from reduced bit-hacking for select-of-constants in IR after rL331486. We want to restore that functionality in the DAG as noted in the commit comments for that change and the llvm-dev discussion here: http://lists.llvm.org/pipermail/llvm-dev/2018-July/124433.html The PPC and AArch tests show that those targets are already doing something similar. x86 will be neutral in the minimal case and generally better when this pattern is extended with other ops as shown in the signbit-shift.ll tests. Note the asymmetry: we don't include the (extend (ifneg X)) transform because it already exists in SimplifySelectCC(), and that is verified in the later unchanged tests in the signbit-shift.ll files. Without the 'not' op, the general transform to use a shift is always a win because that's a single instruction. Alive proofs: https://rise4fun.com/Alive/ysli Name: if pos, get -1 %c = icmp sgt i16 %x, -1 %r = sext i1 %c to i16 => %n = xor i16 %x, -1 %r = ashr i16 %n, 15 Name: if pos, get 1 %c = icmp sgt i16 %x, -1 %r = zext i1 %c to i16 => %n = xor i16 %x, -1 %r = lshr i16 %n, 15 Differential Revision: https://reviews.llvm.org/D48970 llvm-svn: 337130
* CodeGen: Remove pipeline dependencies on StackProtector; NFCMatthias Braun2018-07-131-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | This re-applies r336929 with a fix to accomodate for the Mips target scheduling multiple SelectionDAG instances into the pass pipeline. PrologEpilogInserter and StackColoring depend on the StackProtector analysis being alive from the point it is run until PEI, which requires that they are all scheduled in the same FunctionPassManager. Inserting a (machine) ModulePass between StackProtector and PEI results in these passes being in separate FunctionPassManagers and the StackProtector is not available for PEI. PEI and StackColoring don't use much information from the StackProtector pass, so transfering the required information to MachineFrameInfo is cleaner than keeping the StackProtector pass around. This commit moves the SSP layout information to MFI instead of keeping it in the pass. This patch set (D37580, D37581, D37582, D37583, D37584, D37585, D37586, D37587) is a first draft of the pagerando implementation described in http://lists.llvm.org/pipermail/llvm-dev/2017-June/113794.html. Patch by Stephen Crane <sjc@immunant.com> Differential Revision: https://reviews.llvm.org/D49256 llvm-svn: 336964
* Revert "(HEAD -> master, origin/master, arcpatch-D37582) CodeGen: Remove ↵Matthias Braun2018-07-121-4/+2
| | | | | | | | | | pipeline dependencies on StackProtector; NFC" This was triggering pass scheduling failures. This reverts commit r336929. llvm-svn: 336934
* CodeGen: Remove pipeline dependencies on StackProtector; NFCMatthias Braun2018-07-121-2/+4
| | | | | | | | | | | | | | | | | | | | | | | PrologEpilogInserter and StackColoring depend on the StackProtector analysis being alive from the point it is run until PEI, which requires that they are all scheduled in the same FunctionPassManager. Inserting a (machine) ModulePass between StackProtector and PEI results in these passes being in separate FunctionPassManagers and the StackProtector is not available for PEI. PEI and StackColoring don't use much information from the StackProtector pass, so transfering the required information to MachineFrameInfo is cleaner than keeping the StackProtector pass around. This commit moves the SSP layout information to MFI instead of keeping it in the pass. This patch set (D37580, D37581, D37582, D37583, D37584, D37585, D37586, D37587) is a first draft of the pagerando implementation described in http://lists.llvm.org/pipermail/llvm-dev/2017-June/113794.html. Patch by Stephen Crane <sjc@immunant.com> Differential Revision: https://reviews.llvm.org/D49256 llvm-svn: 336929
* [CodeGen] Emit more precise AssertZext/AssertSext nodes.Eli Friedman2018-07-111-25/+8
| | | | | | | | | | | | This is marginally helpful for removing redundant extensions, and the code is easier to read, so it seems like an all-around win. In the new test i8-phi-ext.ll, we used to emit an AssertSext i8; now we emit an AssertZext i2, which allows the extension of the return value to be eliminated. Differential Revision: https://reviews.llvm.org/D49004 llvm-svn: 336868
* [NFC][InstCombine] Converts isLegalNarrowLoad into isLegalNarrowLdStDiogo N. Sampaio2018-07-111-41/+55
| | | | | | | | | | | Reuse this function as to test correctness and profitability of reducing width of either load or store operations. Reviewsers: samparker Differential Revision: https://reviews.llvm.org/D48624 llvm-svn: 336800
* [SelectionDAG] Add constant buildvector support to isKnownNeverZeroSimon Pilgrim2018-07-112-6/+9
| | | | | | This allows us to use SelectionDAG::isKnownNeverZero in DAGCombiner::visitREM (visitSDIVLike/visitUDIVLike handle the checking for constants). llvm-svn: 336779
* [DAGCombiner] Support non-uniform X%C -> X-(X/C)*C foldsSimon Pilgrim2018-07-111-1/+4
| | | | | | | | | | First stage in PR38057 - support non-uniform constant vectors in the combine to reuse the division-by-constant logic. We can definitely do better for srem pow2 remainders (and avoid that extra multiply....) but this at least helps keep everything on the vector unit. Differential Revision: https://reviews.llvm.org/D48975 llvm-svn: 336774
* [DAGCombiner] Add (urem X, -1) -> select(X == -1, 0, x) foldSimon Pilgrim2018-07-111-0/+6
| | | | llvm-svn: 336773
* [DAGCombiner] Add special case fast paths for udiv x,1 and udiv x,-1Simon Pilgrim2018-07-101-0/+9
| | | | | | udiv x,-1 was going down the (slow) BuildUDIV route resulting in unnecessary shifts. llvm-svn: 336701
* [DAGCombiner] visitREM - call visitSDIVLike/visitUDIVLike directly to avoid ↵Simon Pilgrim2018-07-101-12/+9
| | | | | | | | recursive combining. As suggested by @efriedma on D48975 use the visitSDIVLike/visitUDIVLike functions introduced at rL336656. llvm-svn: 336664
* [DAGCombiner] Split SDIV/UDIV optimization expansions from the rest of the ↵Simon Pilgrim2018-07-101-15/+44
| | | | | | | | combines. NFCI. As suggested by @efriedma on D48975, this patch separates the BuildDiv/Pow2 style optimizations from the rest of the visitSDIV/visitUDIV to make it easier to reuse the combines and will allow us to avoid some rather nasty node recursive combining in visitREM. llvm-svn: 336656
* [X86][TLI] DAGCombine: Unfold variable bit-clearing mask to two shifts.Roman Lebedev2018-07-091-0/+58
| | | | | | | | | | | | | | | | | | | | | Summary: This adds a reverse transform for the instcombine canonicalizations that were added in D47980, D47981. As discussed later, that was worse at least for the code size, and potentially for the performance, too. https://rise4fun.com/Alive/Zmpl Reviewers: craig.topper, RKSimon, spatel Reviewed By: spatel Subscribers: reames, llvm-commits Differential Revision: https://reviews.llvm.org/D48768 llvm-svn: 336585
* [SelectionDAG] Add VT consistency checks to the creation of ISD::FMA.Craig Topper2018-07-091-0/+3
| | | | | | This is similar to what is done for binops. I don't know if this would have helped us catch the bug fixed in r336566 earlier or not, but I figured it couldn't hurt. llvm-svn: 336576
* [SelectionDAG] Split float and integer isKnownNeverZero testsSimon Pilgrim2018-07-071-1/+11
| | | | | | | | | | Splits off isKnownNeverZeroFloat to handle +/- 0 float cases. This will make it easier to be more aggressive with the integer isKnownNeverZero tests (similar to ValueTracking), use computeKnownBits etc. Differential Revision: https://reviews.llvm.org/D48969 llvm-svn: 336492
* Use const APInt& to avoid extra copy. NFCI.Simon Pilgrim2018-07-071-1/+1
| | | | | | As discussed on D48825. llvm-svn: 336491
* [DAGCombiner] Add EXTRACT_SUBVECTOR to SimplifyDemandedVectorEltsSimon Pilgrim2018-07-072-0/+22
| | | | | | | | As discussed on PR37989, this patch adds EXTRACT_SUBVECTOR handling to TargetLowering::SimplifyDemandedVectorElts and calls it from DAGCombiner::visitEXTRACT_SUBVECTOR. Differential Revision: https://reviews.llvm.org/D48825 llvm-svn: 336490
* Revert 336426 (and follow-ups 428, 440), it very likely caused PR38084.Nico Weber2018-07-061-105/+0
| | | | llvm-svn: 336453
* Added missing semicolonDiogo N. Sampaio2018-07-061-2/+1
| | | | llvm-svn: 336428
* [SelectionDAG] https://reviews.llvm.org/D48278Diogo N. Sampaio2018-07-061-0/+106
| | | | | | | | | | | | | | | | D48278 Allow to reduce redundant shift masks. For example: x1 = x & 0xAB00 x2 = (x >> 8) & 0xAB can be reduced to: x1 = x & 0xAB00 x2 = x1 >> 8 It only allows folding when the masks and shift values are constants. llvm-svn: 336426
* Testing commit permisionDiogo N. Sampaio2018-07-051-1/+1
| | | | llvm-svn: 336384
OpenPOWER on IntegriCloud