summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
Commit message (Collapse)AuthorAgeFilesLines
* [TargetLowering] Rename isCondCodeLegal to isCondCodeLegalOrCustom. Add real ↵Craig Topper2018-02-221-10/+8
| | | | | | | | | | | | | | | | isCondCodeLegal. Update callers to use one or the other. isCondCodeLegal internally checked Legal or Custom which is misleading. Though no targets set any cond code action to Custom today. So I've renamed isCondCodeLegal to isCondCodeLegalOrCustom and added a real isCondCodeLegal that only checks Legal. I've changed legalization code to use isCondCodeLegalOrCustom and left things reachable via DAG combine as isCondCodeLegal. I've also changed some places that called getCondCodeAction and compared to Legal to just use isCondCodeLegal. I'm looking at trying to keep SETCC all the way to isel for the AVX512 integer comparisons and I suspect I'll need to make some condition codes Custom to stop DAG combine from changing things post LegalizeOps. Prior to this only Expand stopped DAG combine, but that causes LegalizeOps to try to swap operands or invert rather than calling our Custom handler. Differential Revision: https://reviews.llvm.org/D43607 llvm-svn: 325829
* [LegalizeDAG] Fix legalization of SETCCMikhail Maltsev2018-02-161-1/+1
| | | | | | | | | | | | | | | | | | | Summary: Currently when expanding a SETCC node into a SELECT_CC, LLVM uses an incorrect type for determining BooleanContent of the result. This patch fixes the issue. Fixes PR36079. Reviewers: rogfer01, javed.absar, efriedma Reviewed By: efriedma Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D43282 llvm-svn: 325325
* [AArch64] Improve v8.1-A code-gen for atomic load-andOliver Stannard2018-02-121-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | Armv8.1-A added an atomic load-clear instruction (which performs bitwise and with the complement of it's operand), but not a load-and instruction. Our current code-generation for atomic load-and always inserts an MVN instruction to invert its argument, even if it could be folded into a constant or another instruction. This adds lowering early in selection DAG to convert a load-and operation into an xor with -1 and a load-clear, allowing the normal DAG optimisations to work on it. To do this, I've had to add a new ISD opcode, ATOMIC_LOAD_CLR. I don't see any easy way to do this with an AArch64-specific ISD node, because the code-generation for atomic operations assumes the SDNodes are of type AtomicSDNode. I've left the old tablegen patterns in because they are still needed for global isel. Differential revision: https://reviews.llvm.org/D42478 llvm-svn: 324908
* [SDAG] Legalize all CondCodes by inverting them and/or swapping operandsKrzysztof Parzyszek2018-02-051-12/+19
| | | | | | Differential Revision: https://reviews.llvm.org/D42788 llvm-svn: 324274
* DAG: Fix not truncating when promoting bswap/bitreverseMatt Arsenault2018-01-311-1/+2
| | | | | | | These need to convert back to the original type, like any other promotion. llvm-svn: 323932
* [mips] Fix incorrect sign extension for fpowi libcallSimon Dardis2018-01-301-5/+7
| | | | | | | | | | | | | | | | | PR36061 showed that during the expansion of ISD::FPOWI, that there was an incorrect zero extension of the integer argument which for MIPS64 would then give incorrect results. Address this with the existing mechanism for correcting sign extensions. This resolves PR36061. Thanks to James Cowgill for reporting the issue! Reviewers: atanasyan, hfinkel Differential Revision: https://reviews.llvm.org/D42537 llvm-svn: 323781
* [LegalizeDAG] Fix ATOMIC_CMP_SWAP_WITH_SUCCESS legalization.Eli Friedman2018-01-171-2/+2
| | | | | | | | | | | | | The code wasn't zero-extending correctly, so the comparison could spuriously fail. Adds some AArch64 tests to cover this case. Inspired by D41791. Differential Revision: https://reviews.llvm.org/D41798 llvm-svn: 322767
* [SelectionDAG] lower math intrinsics to finite version of libcalls when ↵Sanjay Patel2018-01-091-18/+62
| | | | | | | | | | | | | | | | | | | | | | possible (PR35672) Ingredients in this patch: 1. Add HANDLE_LIBCALL defs for finite mathlib functions that correspond to LLVM intrinsics. 2. Plumbing to send TargetLibraryInfo down to SelectionDAGLegalize. 3. Relaxed math and library checking in SelectionDAGLegalize::ConvertNodeToLibcall() to choose finite libcalls. There was a bug about determining the availability of the finite calls that should be fixed with: rL322010 Not in this patch: This doesn't resolve the question/bug of clang creating the intrinsic IR in the first place. There's likely follow-up work needed to support the long double variants better. There's room for improvement to reduce the code duplication. Create finite calls that don't originate from a corresponding intrinsic or DAG node? Differential Revision: https://reviews.llvm.org/D41338 llvm-svn: 322087
* MachineFunction: Return reference from getFunction(); NFCMatthias Braun2017-12-151-2/+2
| | | | | | The Function can never be nullptr so we can return a reference. llvm-svn: 320884
* DAG: Legalize truncstores to illegal int typesMatt Arsenault2017-11-281-6/+16
| | | | | | | Truncate to a legal int type, and produce a new truncstore from a narrower type. llvm-svn: 319185
* Fix a bunch more layering of CodeGen headers that are in TargetDavid Blaikie2017-11-171-3/+3
| | | | | | | | All these headers already depend on CodeGen headers so moving them into CodeGen fixes the layering (since CodeGen depends on Target, not the other way around). llvm-svn: 318490
* Move TargetFrameLowering.h to CodeGen where it's implementedDavid Blaikie2017-11-031-1/+1
| | | | | | | | | | | This header already includes a CodeGen header and is implemented in lib/CodeGen, so move the header there to match. This fixes a link error with modular codegeneration builds - where a header and its implementation are circularly dependent and so need to be in the same library, not split between two like this. llvm-svn: 317379
* Don't crash when we see unallocatable registers in clobbersGeorge Burgess IV2017-10-231-2/+5
| | | | | | | | | | | | | | This fixes a bug where we'd crash given code like the test-case from https://bugs.llvm.org/show_bug.cgi?id=30792 . Instead, we let the offending clobber silently slide through. This doesn't fully fix said bug, since the assembler will still complain the moment it sees a crypto/fp/vector op, and we still don't diagnose calls that require vector regs. Differential Revision: https://reviews.llvm.org/D39030 llvm-svn: 316374
* Implement custom lowering for ISD::CTTZ_ZERO_UNDEF and ISD::CTTZ.Wei Ding2017-10-121-4/+15
| | | | | | Differential Revision: http://reviews.llvm.org/D37348 llvm-svn: 315610
* Remove unused variables. No functionality change.Benjamin Kramer2017-10-081-1/+0
| | | | llvm-svn: 315185
* ISel type legalization: add debug messages. NFCI.Sjoerd Meijer2017-10-031-168/+196
| | | | | | | | | | This adds some more debug messages to the type legalizer and functions like PromoteNode, ExpandNode, ExpandLibCall in an attempt to make the debug messages a little bit more informative and useful. Differential Revision: https://reviews.llvm.org/D38450 llvm-svn: 314773
* [CodeGen] Fix some Clang-tidy modernize-use-bool-literals and Include What ↵Eugene Zelenko2017-09-211-55/+59
| | | | | | You Use warnings; other minor fixes (NFC). llvm-svn: 313941
* [SelectionDAG] Remove a check for type being a vector type after calling ↵Craig Topper2017-09-111-2/+0
| | | | | | | | getShiftAmountTy. NFCI getShiftAmountTy already returns the vector type when called for vectors. llvm-svn: 312924
* Remove duplicate codeVictor Leschuk2017-08-241-4/+0
| | | | llvm-svn: 311675
* Add missing break in switchVictor Leschuk2017-08-241-0/+1
| | | | llvm-svn: 311673
* Add ‘llvm.experimental.constrained.fma‘ Intrinsic.Wei Ding2017-08-241-1/+7
| | | | | | Differential Revision: http://reviews.llvm.org/D36335 llvm-svn: 311629
* [ARM] Avoid creating duplicate ANDs in SelectionDAGRenato Golin2017-08-221-4/+9
| | | | | | | | | | | When expanding a BRCOND into a BR_CC, do not create an AND 1 if one already exists. Review: D36705 Patch by Joel Galenson <jgalenson@google.com> llvm-svn: 311447
* Fix libcall expansion creating DAG nodes with invalid type post type ↵Vadim Chugunov2017-07-051-12/+8
| | | | | | | | | | | | legalization. If we are lowering a libcall after legalization, we'll split the return type into a pair of legal values. Patch by Jatin Bhateja and Eli Friedman. Differential Revision: https://reviews.llvm.org/D34240 llvm-svn: 307207
* DAG: correctly legalize UMULO.Tim Northover2017-06-201-11/+18
| | | | | | | | | We were incorrectly sign extending into the high word (as you would for SMULO) when legalizing UMULO in terms of a wider full multiplication. Patch by James Duley. llvm-svn: 305800
* [SelectionDAG] Allow sin/cos -> sincos optimization on GNU triples w/ just ↵Geoff Berry2017-06-121-14/+1
| | | | | | | | | | | | | | | | | | | | | -fno-math-errno Summary: This change enables the sin(x) cos(x) -> sincos(x) optimization on GNU target triples. This optimization was being inhibited when -ffast-math wasn't set because sincos in GLibC does not set errno, while sin and cos do. However, this optimization will only run if the attributes on the sin/cos calls include readnone, which is how clang represents the fact that it doesn't care about the errno values set by these functions (via the -fno-math-errno flag). Reviewers: hfinkel, bogner Subscribers: mcrosier, javed.absar, llvm-commits, paul.redmond Differential Revision: https://reviews.llvm.org/D32921 llvm-svn: 305204
* SelectionDAG: Remove deleted nodes from legalized set to avoid clash with ↵Zvi Rackover2017-06-091-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | newly created nodes Summary: During DAG legalization loop in SelectionDAG::Legalize(), bookkeeping of the SDNodes that were already legalized is implemented with SmallPtrSet (LegalizedNodes). This kind of set stores only pointers to objects, not the objects themselves. Unfortunately, if SDNode is deleted during legalization for some reason, LegalizedNodes set is not informed about this fact. This wouldn’t be so bad, if SelectionDAG wouldn’t reuse space deallocated after deletion of unused nodes, for creation of new ones. Because of this, new nodes, created during legalization often can have pointers identical to ones that have been previously legalized, added to the LegalizedNodes set, and deleted afterwards. This in turn causes, that newly created nodes, sharing the same pointer as deleted old ones, are present in LegalizedNodes *already at the moment of creation*, so we never call Legalize on them. The fix facilitates the fact, that DAG notifies listeners about each modification. I have registered DAGNodeDeletedListener inside SelectionDAG::Legalize, with a callback function that removes any pointer of any deleted SDNode from the LegalizedNodes set. With this modification, LegalizeNodes set does not contain pointers to nodes that were deleted, so newly created nodes can always be inserted to it, even if they share pointers with old deleted nodes. Patch by pawel.szczerbuk@intel.com The issue this patch addresses causes failures in an out-of-tree target, and i was not able to create a reproducer for an in-tree target, hence there is no test-case. Reviewers: delena, spatel, RKSimon, hfinkel, davide, qcolombet Reviewed By: delena Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D33891 llvm-svn: 305084
* [SelectionDAG] Remove special case for ISD::FPOWI from the strict FP ↵Craig Topper2017-05-301-4/+0
| | | | | | | | intrinsic handling. This code was compensating for FPOWI defaulting to Legal and many targets not changing it to Expand. This was fixed in r304215 to default to Expand so this special handling should no longer be necessary. llvm-svn: 304221
* [SelectionDAG] Set ISD::FPOWI to Expand by defaultCraig Topper2017-05-301-1/+0
| | | | | | | | | | | | | | | | | Summary: Currently FPOWI defaults to Legal and LegalizeDAG.cpp turns Legal into Expand for this opcode because Legal is a "lie". This patch changes the default for this opcode to Expand and removes the hack from LegalizeDAG.cpp. It also removes all the code in the targets that set this opcode to Expand themselves since they can just rely on the default. Reviewers: spatel, RKSimon, efriedma Reviewed By: RKSimon Subscribers: jfb, dschuff, sbc100, jgravelle-google, nemanjai, javed.absar, andrew.w.kaylor, llvm-commits Differential Revision: https://reviews.llvm.org/D33530 llvm-svn: 304215
* Add constrained intrinsics for some libm-equivalent operationsAndrew Kaylor2017-05-251-0/+67
| | | | | | Differential revision: https://reviews.llvm.org/D32319 llvm-svn: 303922
* [MSP430] Generate EABI-compliant libcallsVadzim Dambrouski2017-05-111-0/+1
| | | | | | | | | | | | | Updates the MSP430 target to generate EABI-compatible libcall names. As a byproduct, adjusts the hardware multiplier options available in the MSP430 target, adds support for promotion of the ISD::MUL operation for 8-bit integers, and correctly marks R11 as used by call instructions. Patch by Andrew Wygle. Differential Revision: https://reviews.llvm.org/D32676 llvm-svn: 302820
* Add extra operand to CALLSEQ_START to keep frame part set up previouslySerge Pavlov2017-05-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Using arguments with attribute inalloca creates problems for verification of machine representation. This attribute instructs the backend that the argument is prepared in stack prior to CALLSEQ_START..CALLSEQ_END sequence (see http://llvm.org/docs/InAlloca.htm for details). Frame size stored in CALLSEQ_START in this case does not count the size of this argument. However CALLSEQ_END still keeps total frame size, as caller can be responsible for cleanup of entire frame. So CALLSEQ_START and CALLSEQ_END keep different frame size and the difference is treated by MachineVerifier as stack error. Currently there is no way to distinguish this case from actual errors. This patch adds additional argument to CALLSEQ_START and its target-specific counterparts to keep size of stack that is set up prior to the call frame sequence. This argument allows MachineVerifier to calculate actual frame size associated with frame setup instruction and correctly process the case of inalloca arguments. The changes made by the patch are: - Frame setup instructions get the second mandatory argument. It affects all targets that use frame pseudo instructions and touched many files although the changes are uniform. - Access to frame properties are implemented using special instructions rather than calls getOperand(N).getImm(). For X86 and ARM such replacement was made previously. - Changes that reflect appearance of additional argument of frame setup instruction. These involve proper instruction initialization and methods that access instruction arguments. - MachineVerifier retrieves frame size using method, which reports sum of frame parts initialized inside frame instruction pair and outside it. The patch implements approach proposed by Quentin Colombet in https://bugs.llvm.org/show_bug.cgi?id=27481#c1. It fixes 9 tests failed with machine verifier enabled and listed in PR27481. Differential Revision: https://reviews.llvm.org/D32394 llvm-svn: 302527
* Generalize the specialized flag-carrying SDNodes by moving flags into SDNode.Amara Emerson2017-05-011-1/+1
| | | | | | | | This removes BinaryWithFlagsSDNode, and flags are now all passed by value. Differential Revision: https://reviews.llvm.org/D32527 llvm-svn: 301803
* [APInt] Use inplace shift methods where possible. NFCICraig Topper2017-04-281-1/+1
| | | | llvm-svn: 301612
* Fix signed multiplication with overflow fallback.Ranjeet Singh2017-04-261-2/+2
| | | | | | | | | | | | | | | | For targets that don't have ISD::MULHS or ISD::SMUL_LOHI for the type and the double width type is illegal, then the two operands are sign extended to twice their size then multiplied to check for overflow. The extended upper halves were mismatched causing an incorrect result. This fixes the mismatch. A test was added for ARM V6-M where the bug was detected. Patch by James Duley. Differential Revision: https://reviews.llvm.org/D31807 llvm-svn: 301404
* Revert "[APInt] Fix a few places that use APInt::getRawData to operate ↵Renato Golin2017-04-231-1/+1
| | | | | | | | | | | | | | | | within the normal API." This reverts commit r301105, 4, 3 and 1, as a follow up of the previous revert, which broke even more bots. For reference: Revert "[APInt] Use operator<<= where possible. NFC" Revert "[APInt] Use operator<<= instead of shl where possible. NFC" Revert "[APInt] Use ashInPlace where possible." PR32754. llvm-svn: 301111
* [APInt] Use operator<<= instead of shl where possible. NFCCraig Topper2017-04-231-1/+1
| | | | llvm-svn: 301103
* [APInt] Rename getSignBit to getSignMaskCraig Topper2017-04-201-2/+2
| | | | | | | | getSignBit is a static function that creates an APInt with only the sign bit set. getSignMask seems like a better name to convey its functionality. In fact several places use it and then store in an APInt named SignMask. Differential Revision: https://reviews.llvm.org/D32108 llvm-svn: 300856
* Fix for PR#30562: Selection DAG error: Detected cycle in SelectionDAG.Andrew V. Tischenko2017-04-141-2/+5
| | | | | | Patch by Dinar Temirbulatov llvm-svn: 300314
* Make library calls sensitive to regparm module flag (Fixes PR3997).Nirav Dave2017-03-181-23/+37
| | | | | | | | | | Reviewers: mkuper, rnk Subscribers: mehdi_amini, jyknight, aemerson, llvm-commits, rengolin Differential Revision: https://reviews.llvm.org/D27050 llvm-svn: 298179
* Capitalize ArgListEntry fields. NFC.Nirav Dave2017-03-181-16/+16
| | | | llvm-svn: 298178
* [DAGCombine] Use APInt::operator|(uint64_t) instead of creating a temporary ↵Craig Topper2017-03-051-6/+6
| | | | | | | | APInt and calling APInt::Or. NFC This is more efficient by itself. But this is prep for a future patch that may remove APInt::Or while making operator| support rvalue references similar to add/sub. llvm-svn: 296981
* Use SelectionDAG::getBuildVector helper function where possible. NFCI.Simon Pilgrim2017-01-301-5/+4
| | | | llvm-svn: 293532
* DAG: Avoid OOB when legalizing vector indexingMatt Arsenault2017-01-101-30/+10
| | | | | | | | | If a vector index is out of bounds, the result is supposed to be undefined but is not undefined behavior. Change the legalization for indexing the vector on the stack so that an out of bounds index does not create an out of bounds memory access. llvm-svn: 291604
* [SelectionDAG] Add expansion and promotion of [US]MUL_LOHINicolai Haehnle2016-12-081-5/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Most targets set the action for these nodes to Expand even though there isn't actually any code for them in ExpandNode. Instead, targets simply relied on the fact that no code generates these nodes as long as the nodes aren't legal or custom. However, generating these nodes can be useful e.g. for divide-by-constant in wider integer types. Expand of [US]MUL_LOHI will use MULH[US] when legal or custom, and a sequence of half-width multiplications otherwise. Promote uses a wider multiply. This patch intends to not change the generated code, but indirect effects are possible since expansions/promotions that were previously done in DAGCombine may now be done in LegalizeDAG. See D24822 for a change that actually uses the new expansion. Reviewers: spatel, bkramer, venkatra, efriedma, hfinkel, ast, nadav, tstellarAMD Subscribers: arsenm, jyknight, nemanjai, wdng, nhaehnle, llvm-commits Differential Revision: https://reviews.llvm.org/D24956 llvm-svn: 289050
* [CodeGen] Fix result type for SMULO/UMULO legalizationEli Friedman2016-12-061-0/+9
| | | | | | | | | | | | | | | On some platforms (like MSP430) the second element of the result structure for SMULO/UMULO may have a shorter type than the one returned by SetCC. We need to truncate it to the right type, or else some incorrect code may be generated later on. This fixes issue https://github.com/rust-lang/rust/issues/37829 Patch by Vadzim Dambrouski! Differential Revision: https://reviews.llvm.org/D27154 llvm-svn: 288857
* Fix spelling mistakes in SelectionDAG comments. NFC.Simon Pilgrim2016-11-201-1/+1
| | | | | | Identified by Pedro Giffuni in PR27636. llvm-svn: 287487
* Introduce TLI predicative for base-relative Jump Tables.Joerg Sonnenberger2016-11-151-1/+1
| | | | | | | | | | | For 64bit ABIs it is common practice to use relative Jump Tables with potentially different relocation bases. As the logic for the jump table itself doesn't depend on the relocation base, make it easier for targets to use the generic logic. Start by dropping the now redundant MIPS logic. Differential Revision: https://reviews.llvm.org/D26578 llvm-svn: 286951
* Simplify.Joerg Sonnenberger2016-11-021-2/+2
| | | | llvm-svn: 285802
* LegalizeDAG: Support promoting [US]DIV and [US]REM operationsTom Stellard2016-10-261-1/+18
| | | | | | | | | | | | | | | | | | Summary: AMDGPU will need this one i16 is added as a legal type. This is tested by: test/CodeGen/AMDGPU/sdiv.ll test/CodeGen/AMDGPU/sdivrem24.ll test/CodeGen/AMDGPU/udiv.ll test/CodeGen/AMDGPU/udivrem24.ll Reviewers: bogner, efriedma Subscribers: efriedma, wdng, llvm-commits Differential Revision: https://reviews.llvm.org/D25699 llvm-svn: 285199
* [DAG] optimize negation of boolSanjay Patel2016-10-191-2/+19
| | | | | | | | | | | | | | | | Use mask and negate for legalization of i1 source type with SIGN_EXTEND_INREG. With the mask, this should be no worse than 2 shifts. The mask can be eliminated in some cases, so that should be better than 2 shifts. This change exposed some missing folds related to negation: https://reviews.llvm.org/rL284239 https://reviews.llvm.org/rL284395 There may be others, so please let me know if you see any regressions. Differential Revision: https://reviews.llvm.org/D25485 llvm-svn: 284611
OpenPOWER on IntegriCloud