summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
Commit message (Collapse)AuthorAgeFilesLines
* No longer generate calls to *_finiteserge-sans-paille2020-02-281-61/+12
| | | | | | | | | | | | | | | | | | | | | | | | | According to Joseph Myers, a libm maintainer > They were only ever an ABI (selected by use of -ffinite-math-only or > options implying it, which resulted in the headers using "asm" to redirect > calls to some libm functions), not an API. The change means that ABI has > turned into compat symbols (only available for existing binaries, not for > anything newly linked, not included in static libm at all, not included in > shared libm for future glibc ports such as RV32), so, yes, in any case > where tools generate direct calls to those functions (rather than just > following the "asm" annotations on function declarations in the headers), > they need to stop doing so. As a consequence, we should no longer assume these symbols are available on the target system. Still keep the TargetLibraryInfo for constant folding. Differential Revision: https://reviews.llvm.org/D74712 (cherry picked from commit 6d15c4deab51498b70825fb6cefbbfe8f3d9bdcf) For https://bugs.llvm.org/show_bug.cgi?id=45034
* [Intrinsic] Add fixed point division intrinsics.Bevin Hansson2020-01-081-1/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch adds intrinsics and ISelDAG nodes for signed and unsigned fixed-point division: llvm.sdiv.fix.* llvm.udiv.fix.* These intrinsics perform scaled division on two integers or vectors of integers. They are required for the implementation of the Embedded-C fixed-point arithmetic in Clang. Patch by: ebevhan Reviewers: bjope, leonardchan, efriedma, craig.topper Reviewed By: craig.topper Subscribers: Ka-Ka, ilya, hiraditya, jdoerfert, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70007
* [X86][FPEnv] Promote some float strictfp operations to double on ↵Craig Topper2019-12-261-0/+31
| | | | | | | | i686-pc-windows-msvc to match what we do for non-strict. The float libcalls are inlined in MSVC's math header where they just cast to double and use the double libcall. Do the same when we emit libcalls.
* [LegalizeDAG] Add return to the strict node handling in ↵Craig Topper2019-12-191-0/+1
| | | | PromoteLegalINT_TO_FP to prevent an invalid strict fp node from being created by falling into non-strict code path.
* [FPEnv][LegalizeTypes][LegalizeDAG][AArch64] Few fixes/improvements for ↵Craig Topper2019-12-171-1/+2
| | | | | | | | | | | | | | | | legalizing fp<->int conversion nodes. This started with adding a test to support get code coverage on ScalarizeVecOp_UnaryOp_StrictFP by copying an existing AArch64 test and using constrained sitofp/uitofp intrinsics. This found 3 separate issues: -ScalarizeVecOp_UnaryOp_StrictFP needs to do its own replacement because the caller can't handle replacing multiple results. -Missing integer promotion support for sitofp/uitofp -Chain result not always assigned in ExpandLegalINT_TO_FP. Committing them together so I can add the test case.
* This adds constrained intrinsics for the signed and unsigned conversionsKevin P. Neal2019-12-171-29/+91
| | | | | | | | | of integers to floating point. This includes some of Craig Topper's changes for promotion support from D71130. Differential Revision: https://reviews.llvm.org/D69275
* [NFC] Use SelectionDAG::getMemBasePlusOffset() instead of getNode(ISD::ADD)Alex Richardson2019-12-131-14/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: To find potential opportunities to use getMemBasePlusOffset() I looked at all ISD::ADD uses found with the regex getNode\(ISD::ADD,.+,.+Ptr in lib/CodeGen/SelectionDAG. If this patch is accepted I will convert the files in the individual backends too. The motivation for this change is our out-of-tree CHERI backend (https://github.com/CTSRD-CHERI/llvm-project). We use a separate register type to store pointers (128-bit capabilities, which are effectively unforgeable and monotonic fat pointers). These capabilities permit a reduced set of operations and therefore use a separate ValueType (iFATPTR). to represent pointers implemented as capabilities. Therefore, we need to avoid using ISD::ADD for our patterns that operate on pointers and need to use a function that chooses ISD::ADD or a new ISD::PTRADD opcode depending on the value type. We originally added a new DAG.getPointerAdd() function, but after this patch series we can modify the implementation of getMemBasePlusOffset() instead. Avoiding direct uses of ISD::ADD for pointer types will significantly reduce the amount of assertion/instruction selection failures for us in future upstream merges. Reviewers: spatel Reviewed By: spatel Subscribers: merge_guards_bot, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71207
* [NFC] Use EVT instead of bool for getSetCCInverse()Alex Richardson2019-12-131-3/+2
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: The use of a boolean isInteger flag (generally initialized using VT.isInteger()) caused errors in our out-of-tree CHERI backend (https://github.com/CTSRD-CHERI/llvm-project). In our backend, pointers use a separate ValueType (iFATPTR) and therefore .isInteger() returns false. This meant that getSetCCInverse() was using the floating-point variant and generated incorrect code for us: `(void *)0x12033091e < (void *)0xffffffffffffffff` would return false. Committing this change will significantly reduce our merge conflicts for each upstream merge. Reviewers: spatel, bogner Reviewed By: bogner Subscribers: wuzish, arsenm, sdardis, nemanjai, jvesely, nhaehnle, hiraditya, kbarton, jrtc27, atanasyan, jsji, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70917
* [FPEnv][X86] Constrained FCmp intrinsics enabling on X86Wang, Pengfei2019-12-111-20/+41
| | | | | | | | | | | | Summary: This is a follow up of D69281, it enables the X86 backend support for the FP comparision. Reviewers: uweigand, kpn, craig.topper, RKSimon, cameron.mcinally, andrew.w.kaylor Subscribers: hiraditya, llvm-commits, annita.zhang, LuoYuanke, LiuChen3 Tags: #llvm Differential Revision: https://reviews.llvm.org/D70582
* [FPEnv] Constrained FCmp intrinsicsUlrich Weigand2019-12-071-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds support for constrained floating-point comparison intrinsics. Specifically, we add: declare <ty2> @llvm.experimental.constrained.fcmp(<type> <op1>, <type> <op2>, metadata <condition code>, metadata <exception behavior>) declare <ty2> @llvm.experimental.constrained.fcmps(<type> <op1>, <type> <op2>, metadata <condition code>, metadata <exception behavior>) The first variant implements an IEEE "quiet" comparison (i.e. we only get an invalid FP exception if either argument is a SNaN), while the second variant implements an IEEE "signaling" comparison (i.e. we get an invalid FP exception if either argument is any NaN). The condition code is implemented as a metadata string. The same set of predicates as for the fcmp instruction is supported (except for the "true" and "false" predicates). These new intrinsics are mapped by SelectionDAG codegen onto two new ISD opcodes, ISD::STRICT_FSETCC and ISD::STRICT_FSETCCS, again representing quiet vs. signaling comparison operations. Otherwise those nodes look like SETCC nodes, with an additional chain argument and result as usual for strict FP nodes. The patch includes support for the common legalization operations for those nodes. The patch also includes full SystemZ back-end support for the new ISD nodes, mapping them to all available SystemZ instruction to fully implement strict semantics (scalar and vector). Differential Revision: https://reviews.llvm.org/D69281
* [LegalizeDAG] Return true from ExpandNode for some nodes that don't have ↵Craig Topper2019-12-021-1/+3
| | | | | | | | | | | | | | expand support. These nodes have a FIXME that they only get here because a Custom handler returned SDValue() instead of the original Op. Even though we aren't expanding them, we should return true here to prevent ConvertNodeToLibcall from also trying to process them until the FIXME has been addressed. I'm hoping to add checking to ConvertNodeToLibcall to make sure we don't give it nodes it doesn't have support for.
* [LegalizeDAG] When expanding vector SRA/SRL/SHL add the new BUILD_VECTOR to ↵Craig Topper2019-12-021-1/+1
| | | | | | | | | | | | | | | | the Results vector instead of just calling ReplaceNode The code that processes the Results vector also calls ReplaceNode and makes ExpandNode return true. If we don't add it to the Results node, we end up returning false from ExpandNode. This causes ConvertNodeToLibcall to be called next. But ConvertNodeToLibcall doesn't do anything for shifts so they just pass through unmodified. Except for printing a debug message. Ultimately, I'd like to add more checks to ExpandNode and ConvertNodeToLibcall to make sure we don't have nodes marked as Expand that don't have any Expand or libcall handling.
* [LegalizeDAG] Use getOperationAction instead of getStrictFPOperationAction ↵Craig Topper2019-11-261-2/+2
| | | | for STRICT_LRINT/LROUND/LLRINT/LLROUND.
* [TargetLowering] Merge ExpandChainLibCall with makeLibCallCraig Topper2019-11-251-6/+21
| | | | | | I need to be able to drop an operand for STRICT_FP_ROUND handling on X86. Merging these functions gives me the ArrayRef interface that passes the return type, operands, and debugloc instead of the Node. Differential Revision: https://reviews.llvm.org/D70503
* [FPEnv] Add an option to disable strict float node mutating to an normalPengfei Wang2019-11-211-5/+17
| | | | | | | | | | | | | float node This patch add an option 'disable-strictnode-mutation' to prevent strict node mutating to an normal node. So we can make sure that the patch which sets strict-node as legal works correctly. Patch by Chen Liu(LiuChen3) Differential Revision: https://reviews.llvm.org/D70226
* [LegalizeDAG][X86] Add support for turning STRICT_FADD/SUB/MUL/DIV into ↵Craig Topper2019-11-211-0/+4
| | | | | | | libcalls. Use it for fp128 on x86-64. This requires a minor hack for f32/f64 strict fadd/fsub to avoid turning those into libcalls.
* [LegalizeDAG][X86] Enable STRICT_FP_TO_SINT/UINT to be promotedCraig Topper2019-11-191-16/+26
| | | | Differential Revision: https://reviews.llvm.org/D70220
* [SelectionDAG] Merge the two identical ExpandChainLibCall methods from ↵Craig Topper2019-11-181-42/+6
| | | | | | | | | | | | | | LegalizeTypes and LegalizeDAG to one version in TaretLowering. Reviewers: RKSimon, efriedma, spatel Reviewed By: efriedma Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70354
* [LegalizeDAG] Convert strict fp nodes to libcalls without losing the chain.Craig Topper2019-11-181-153/+170
| | | | | | | | | | Previously we mutated the node and then converted it to a libcall. But this loses the chain information. This patch keeps the chain, but unfortunately breaks tail call optimization as the functions involved in deciding if a node is in tail call position can't handle the chain. But correct ordering seems more important to be right. Somehow the SystemZ tests improved. I looked at one of them and it seemed that we're handling the split vector elements in a different order and that made the copies work better. Differential Revision: https://reviews.llvm.org/D70334
* [AArch64][X86] Don't assume __powidf2 is available on Windows.Eli Friedman2019-11-081-1/+21
| | | | | | | | | | We had some code for this for 32-bit ARM, but this doesn't really need to be in target-specific code; generalize it. (I think this started showing up recently because we added an optimization that converts pow to powi.) Differential Revision: https://reviews.llvm.org/D69013
* Fix PR40644: miscompile indexed FP constant storeThomas Preud'homme2019-11-051-0/+3
| | | | | | | | | | | | | | | | | | | | | Summary: Functions replaceStoreOfFPConstant() and OptimizeFloatStore() both replace store of float by a store of an integer unconditionally. However this generates wrong code when the store that is replaced is an indexed or truncating store. This commit solves this issue by adding an early return in these functions when the store being considered is not a normal store. Bug was only observed on out of tree targets, hence the lack of testcase in this commit. Reviewers: efriedma Subscribers: hiraditya, arphaman, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68420
* [NFC] Move this set of STRICT_* cases to be next to the non-strict cases.Kevin P. Neal2019-10-301-10/+10
| | | | Requested by Cameron McInally in D69275.
* Prune Analysis includes from SelectionDAG.hReid Kleckner2019-10-191-0/+1
| | | | | | Only forward declarations are needed here. Follow-on to r375311. llvm-svn: 375319
* [AArch64][SVE] Add SPLAT_VECTOR ISD NodeGraham Hunter2019-10-181-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | Adds a new ISD node to replicate a scalar value across all elements of a vector. This is needed for scalable vectors, since BUILD_VECTOR cannot be used. Fixes up default type legalization for scalable vectors after the new MVT type ranges were introduced. At present I only use this node for scalable vectors. A DAGCombine has been added to transform a BUILD_VECTOR into a SPLAT_VECTOR if all elements are the same, but only if the default operation action of Expand has been overridden by the target. I've only added result promotion legalization for scalable vector i8/i16/i32/i64 types in AArch64 for now. Reviewers: t.p.northover, javed.absar, greened, cameron.mcinally, jmolloy Reviewed By: jmolloy Differential Revision: https://reviews.llvm.org/D47775 llvm-svn: 375222
* [FPEnv] Add constrained intrinsics for lrint and lroundKevin P. Neal2019-10-071-28/+60
| | | | | | | | | | | Earlier in the year intrinsics for lrint, llrint, lround and llround were added to llvm. The constrained versions are now implemented here. Reviewed by: andrew.w.kaylor, craig.topper, cameron.mcinally Approved by: craig.topper Differential Revision: https://reviews.llvm.org/D64746 llvm-svn: 373900
* [TargetLowering] Make allowsMemoryAccess methode virtual.Thomas Raoux2019-09-261-6/+6
| | | | | | | | | | | Rename old function to explicitly show that it cares only about alignment. The new allowsMemoryAccess call the function related to alignment by default and can be overridden by target to inform whether the memory access is legal or not. Differential Revision: https://reviews.llvm.org/D67121 llvm-svn: 372935
* [LegalizeTypes] Remove code for softening a float type to itself.Craig Topper2019-09-121-4/+2
| | | | | | | | | This was previously used to turn fp128 operations into libcalls on X86. This is now done through op legalization after r371672. This restores much of this code to before r254653. llvm-svn: 371709
* [SelectionDAG] Remove ISD::FP_ROUND_INREGCraig Topper2019-09-091-14/+0
| | | | | | | | | | | | I don't think anything in tree creates this node. So all of this code appears to be dead. Code coverage agrees http://lab.llvm.org:8080/coverage/coverage-reports/llvm/coverage/Users/buildslave/jenkins/workspace/clang-stage2-coverage-R/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp.html Differential Revision: https://reviews.llvm.org/D67312 llvm-svn: 371431
* [Intrinsic] Add the llvm.umul.fix.sat intrinsicBjorn Pettersson2019-09-071-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Add an intrinsic that takes 2 unsigned integers with the scale of them provided as the third argument and performs fixed point multiplication on them. The result is saturated and clamped between the largest and smallest representable values of the first 2 operands. This is a part of implementing fixed point arithmetic in clang where some of the more complex operations will be implemented as intrinsics. Patch by: leonardchan, bjope Reviewers: RKSimon, craig.topper, bevinh, leonardchan, lebedev.ri, spatel Reviewed By: leonardchan Subscribers: ychen, wuzish, nemanjai, MaskRay, jsji, jdoerfert, Ka-Ka, hiraditya, rjmccall, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D57836 llvm-svn: 371308
* [LegalizeDAG] Pass DAG to two calls to SDNode::dump in debug prints so that ↵Craig Topper2019-09-031-2/+2
| | | | | | | | | they will print target specific nodes correctly. The dump methods can only print target node names correctly if they can get access to the TLI object. llvm-svn: 370694
* Fix shadow variable warning. NFCI.Simon Pilgrim2019-08-291-4/+3
| | | | llvm-svn: 370332
* LegalizeSetCCCondCode - Reduce scope of NeedSwap to fix cppcheck warning. NFCI.Simon Pilgrim2019-08-291-1/+1
| | | | | | No need for this to be defined outside the only switch case its used in. llvm-svn: 370320
* [FPEnv] Add fptosi and fptoui constrained intrinsics.Kevin P. Neal2019-08-281-1/+28
| | | | | | | | | | | | | | | | | This implements constrained floating point intrinsics for FP to signed and unsigned integers. Quoting from D32319: The purpose of the constrained intrinsics is to force the optimizer to respect the restrictions that will be necessary to support things like the STDC FENV_ACCESS ON pragma without interfering with optimizations when these restrictions are not needed. Reviewed by: Andrew Kaylor, Craig Topper, Hal Finkel, Cameron McInally, Roman Lebedev, Kit Barton Approved by: Craig Topper Differential Revision: http://reviews.llvm.org/D63782 llvm-svn: 370228
* [Strict FP] Allow custom operation actionsUlrich Weigand2019-08-061-33/+24
| | | | | | | | | | | | | | | | | | | | | This patch changes the DAG legalizer to respect the operation actions set by the target for strict floating-point operations. (Currently, the legalizer will usually fall back to mutate to the non-strict action (which is assumed to be legal), and only skip mutation if the strict operation is marked legal.) With this patch, if whenever a strict operation is marked as Legal or Custom, it is passed to the target as usual. Only if it is marked as Expand will the legalizer attempt to mutate to the non-strict operation. Note that this will now fail if the non-strict operation is itself marked as Custom -- the target will have to provide a Custom definition for the strict operation then as well. Reviewed By: hfinkel Differential Revision: https://reviews.llvm.org/D65226 llvm-svn: 368012
* [SelectionDAG] Simplify some calls to getSetCCResultType. NFCBjorn Pettersson2019-07-091-2/+1
| | | | | | | | DAGTypeLegalizer and SelectionDAGLegalize has helper functions wrapping the call to TLI.getSetCCResultType(...). Use those helpers in more places. llvm-svn: 365456
* adding more fmf propagation for selects plus updated testsMichael Berg2019-06-151-0/+3
| | | | llvm-svn: 363484
* Revert "adding more fmf propagation for selects plus tests"Fangrui Song2019-06-151-3/+0
| | | | | | | | | | | This reverts rL363474. -debug-only=isel was added to some tests that don't specify `REQUIRES: asserts`. This causes failures on -DLLVM_ENABLE_ASSERTIONS=off builds. I chose to revert instead of fixing the tests because I'm not sure whether we should add `REQUIRES: asserts` to more tests. llvm-svn: 363482
* adding more fmf propagation for selects plus testsMichael Berg2019-06-141-0/+3
| | | | llvm-svn: 363474
* [TargetLowering] Add allowsMemoryAccess(MachineMemOperand) helper wrapper. NFCI.Simon Pilgrim2019-06-111-13/+9
| | | | | | As suggested by @arsenm on D63075 - this adds a TargetLowering::allowsMemoryAccess wrapper that takes a Load/Store node's MachineMemOperand to handle the AddressSpace/Alignment arguments and will also implicitly handle the MachineMemOperand::Flags change in D63075. llvm-svn: 363048
* Propagate fmf for setcc in SDAG for select foldsMichael Berg2019-06-031-4/+7
| | | | llvm-svn: 362448
* [CodeGen] Add lrint/llrint builtinsAdhemerval Zanella2019-05-281-0/+14
| | | | | | | | | | | | | | | | | This patch add the ISD::LRINT and ISD::LLRINT along with new intrinsics. The changes are straightforward as for other floating-point rounding functions, with just some adjustments required to handle the return value being an interger. The idea is to optimize lrint/llrint generation for AArch64 in a subsequent patch. Current semantic is just route it to libm symbol. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D62017 llvm-svn: 361875
* [X86] Custom lower CONCAT_VECTORS of v2i1Benjamin Kramer2019-05-281-0/+1
| | | | | | | The generic legalizer cannot handle this. Add an assert instead of silently miscompiling vectors with elements smaller than 8 bits. llvm-svn: 361814
* [Intrinsic] Signed Fixed Point Saturation Multiplication IntrinsicLeonard Chan2019-05-211-0/+2
| | | | | | | | | | | | | | Add an intrinsic that takes 2 signed integers with the scale of them provided as the third argument and performs fixed point multiplication on them. The result is saturated and clamped between the largest and smallest representable values of the first 2 operands. This is a part of implementing fixed point arithmetic in clang where some of the more complex operations will be implemented as intrinsics. Differential Revision: https://reviews.llvm.org/D55720 llvm-svn: 361289
* [SDAG] Vector op legalization for overflow opsNikita Popov2019-05-201-66/+8
| | | | | | | | | | | | | | | | | | Fixes issue reported by aemerson on D57348. Vector op legalization support is added for uaddo, usubo, saddo and ssubo (umulo and smulo were already supported). As usual, by extracting TargetLowering methods and calling them from vector op legalization. Vector op legalization doesn't really deal with multiple result nodes, so I'm explicitly performing a recursive legalization call on the result value that is not being legalized. There are some existing test changes because expansion happens earlier, so we don't get a DAG combiner run in between anymore. Differential Revision: https://reviews.llvm.org/D61692 llvm-svn: 361166
* [CodeGen] Add lround/llround builtinsAdhemerval Zanella2019-05-161-0/+39
| | | | | | | | | | | | | This patch add the ISD::LROUND and ISD::LLROUND along with new intrinsics. The changes are straightforward as for other floating-point rounding functions, with just some adjustments required to handle the return value being an interger. The idea is to optimize lround/llround generation for AArch64 in a subsequent patch. Current semantic is just route it to libm symbol. llvm-svn: 360889
* Add constrained fptrunc and fpext intrinsics.Kevin P. Neal2019-05-131-5/+30
| | | | | | | | | | | The new fptrunc and fpext intrinsics are constrained versions of the regular fptrunc and fpext instructions. Reviewed by: Andrew Kaylor, Craig Topper, Cameron McInally, Conner Abbot Approved by: Craig Topper Differential Revision: https://reviews.llvm.org/D55897 llvm-svn: 360581
* [SelectionDAG] Expand ADD/SUBCARRYLeonard Chan2019-05-091-0/+42
| | | | | | | | This patch allows for expansion of ADDCARRY and SUBCARRY when the target does not support it. Differential Revision: https://reviews.llvm.org/D61411 llvm-svn: 360303
* [LegalizeDAG] Assert non-power-of-2 load/store op splits are in range. NFCI.Simon Pilgrim2019-05-081-2/+6
| | | | | | Fixes static analyzer undefined/out-of-range shift warnings. llvm-svn: 360245
* [IR] Refactor attribute methods in Function class (NFC)Evandro Menezes2019-04-041-1/+1
| | | | | | | | Rename the functions that query the optimization kind attributes. Differential revision: https://reviews.llvm.org/D60287 llvm-svn: 357731
* [LegalizeDAG] Expand i16 bswap directly to a rotate by 8 instead of relying ↵Craig Topper2019-03-241-3/+2
| | | | | | | | | | | | | | | | on DAG combine. An i16 bswap can be implemented with an i16 rotate by 8. We previously emitted a shift and OR sequence that DAG combine should be able to turn back into rotate. But we might as well go there directly. If rotate isn't legal, LegalizeDAG should further legalize it to either the opposite rotate, or the shift and OR pattern. I don't know of any way to get the existing DAG combine reliance to fail. So I don't know any way to add new tests for this that wouldn't have worked previously. llvm-svn: 356860
OpenPOWER on IntegriCloud