summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG
Commit message (Collapse)AuthorAgeFilesLines
* [DAGCombiner] Ensure we use the correct CC result type in visitSDIV (REAPPLIED)Simon Pilgrim2018-06-281-5/+6
| | | | | | | | | | We could get away with it for constant folded cases, but not for rL335719. Thanks to Krzysztof Parzyszek for noticing. Reapply original commit rL335821 which was reverted at rL335871 due to a WebAssembly bug that was fixed at rL335884. llvm-svn: 335886
* SelectionDAGBuilder, mach-o: Skip trap after noreturn call (for Mach-O)Matthias Braun2018-06-281-3/+17
| | | | | | | | | | | | | Add NoTrapAfterNoreturn target option which skips emission of traps behind noreturn calls even if TrapUnreachable is enabled. Enable the feature on Mach-O to save code size; Comments suggest it is not possible to enable it for the other users of TrapUnreachable. rdar://41530228 DifferentialRevision: https://reviews.llvm.org/D48674 llvm-svn: 335877
* Revert "[DAGCombiner] Ensure we use the correct CC result type in visitSDIV"Haojian Wu2018-06-281-6/+5
| | | | | | | | This reverts commit r335821. This crashes the webassembly test, run "ninja check-llvm-codegen-webassembly" to reproduce. llvm-svn: 335871
* [DAGCombiner] Ensure we use the correct CC result type in visitSDIVSimon Pilgrim2018-06-281-5/+6
| | | | | | | | We could get away with it for constant folded cases, but not for rL335719. Thanks to Krzysztof Parzyszek for noticing. llvm-svn: 335821
* [DAGCombiner] Remove unused variable. NFCI.Simon Pilgrim2018-06-281-2/+0
| | | | | | Noticed in D45806 review. llvm-svn: 335817
* [DAGCombine] Disable TokenFactor simplifications when optnone.Nirav Dave2018-06-271-0/+4
| | | | llvm-svn: 335773
* [DAGCombiner] restrict (float)((int) f) --> ftrunc with no-signed-zerosSanjay Patel2018-06-271-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | As noted in the D44909 review, the transform from (fptosi+sitofp) to ftrunc can produce -0.0 where the original code does not: #include <stdio.h> int main(int argc) { float x; x = -0.8 * argc; printf("%f\n", (float)((int)x)); return 0; } $ clang -O0 -mavx fp.c ; ./a.out 0.000000 $ clang -O1 -mavx fp.c ; ./a.out -0.000000 Ideally, we'd use IR/node flags to predicate the transform, but the IR parser doesn't currently allow fast-math-flags on the cast instructions. So for now, just use the function attribute that corresponds to clang's "-fno-signed-zeros" option. Differential Revision: https://reviews.llvm.org/D48085 llvm-svn: 335761
* [DAGCombiner] visitSDIV - add special case handling for (sdiv X, 1) -> X in ↵Simon Pilgrim2018-06-271-11/+7
| | | | | | | | pow2 expansion For divisor = 1, perform a select of X - reduces scalarisation of simple SDIVs llvm-svn: 335727
* [DAGCombiner] visitSDIV - simplify pow2 handling. NFCI.Simon Pilgrim2018-06-271-29/+12
| | | | | | Use the builtin constant folding of getNode() etc. instead of doing it manually. llvm-svn: 335720
* [DAGCombiner] Fold SDIV(%X, MIN_SIGNED) -> SELECT(%X == MIN_SIGNED, 1, 0)Simon Pilgrim2018-06-271-0/+5
| | | | | | Fixes PR37569. llvm-svn: 335719
* [DAGCombiner] Don't accept signbit sdiv divisors in sdiv-by-pow2 vector ↵Simon Pilgrim2018-06-271-0/+2
| | | | | | expansion (PR37569) llvm-svn: 335717
* [DAGCombiner] use isBitwiseNot to simplify code; NFCSanjay Patel2018-06-261-8/+3
| | | | llvm-svn: 335652
* [DAGCombiner] Don't accept -1 sdiv divisors in sdiv-by-pow2 vector expansion ↵Simon Pilgrim2018-06-261-0/+2
| | | | | | | | (PR37119) Temporary fix until I've managed to get D45806 updated - both +1 and -1 special cases need to be properly supported. llvm-svn: 335637
* [DAGCombiner] Pull out VT bitwidth in visitSDIV. NFCI.Simon Pilgrim2018-06-261-4/+4
| | | | llvm-svn: 335617
* [SelectionDAG] Remove debug locations from ConstantSD(FP)NodesVedant Kumar2018-06-251-2/+2
| | | | | | | | | | | | | | | | | | This removes debug locations from ConstantSDNode and ConstantSDFPNode. When this kind of node is materialized we no longer create a line table entry which jumps back to the constant's first point of use. This makes single-stepping behavior smoother, and it matches the model used by IR, where Constants have no locations. See this thread for more context: http://lists.llvm.org/pipermail/llvm-dev/2018-June/124164.html I'd like to handle constant BuildVectorSDNodes and to try to eliminate passing SDLocs to SelectionDAG::getConstant*() in follow-up commits. Differential Revision: https://reviews.llvm.org/D48468 llvm-svn: 335497
* Fix -Wparentheses gcc warning. NFCI.Simon Pilgrim2018-06-251-1/+1
| | | | llvm-svn: 335451
* [DAGCombiner] eliminate setcc bool math when input is low-bit of some valueSanjay Patel2018-06-241-0/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch has the same motivating example as D48466: define void @foo(i64 %x, i32 %c.0282.in, i32 %d.0280, i32* %ptr0, i32* %ptr1) { %c.0282 = and i32 %c.0282.in, 268435455 %a16 = lshr i64 32508, %x %a17 = and i64 %a16, 1 %tobool = icmp eq i64 %a17, 0 %. = select i1 %tobool, i32 1, i32 2 %.286 = select i1 %tobool, i32 27, i32 26 %shr97 = lshr i32 %c.0282, %. %shl98 = shl i32 %c.0282.in, %.286 %or99 = or i32 %shr97, %shl98 %shr100 = lshr i32 %d.0280, %. %shl101 = shl i32 %d.0280, %.286 %or102 = or i32 %shr100, %shl101 store i32 %or99, i32* %ptr0 store i32 %or102, i32* %ptr1 ret void } ...but I'm trying to kill the setcc bool math sooner rather than later. By matching a larger pattern that includes both the low-bit mask and the trailing add/sub, we can create a universally good fold because we always eliminate the condition code intermediate value. Here are Alive proofs for these (currently instcombine folds the 'add' variants, but misses the 'sub' patterns): https://rise4fun.com/Alive/Gsyp Name: sub of zext cmp mask %a = and i8 %x, 1 %c = icmp eq i8 %a, 0 %z = zext i1 %c to i32 %r = sub i32 C1, %z => %optional_cast = zext i8 %a to i32 %r = add i32 %optional_cast, C1-1 Name: add of zext cmp mask %a = and i32 %x, 1 %c = icmp eq i32 %a, 0 %z = zext i1 %c to i8 %r = add i8 %z, C1 => %optional_cast = trunc i32 %a to i8 %r = sub i8 C1+1, %optional_cast All of the tests look like improvements or neutral to me. But it is possible that x86 test+set+bitop is better than what we now show here. I suspect we could do better by adding another fold for the 'sub' variants. We start with select-of-constant in IR in the larger motivating test, so that's why I included tests with selects. Proofs for those variants: https://rise4fun.com/Alive/Bx1 Name: true const is bigger Pre: C2 == (C1 + 1) %a = and i8 %x, 1 %c = icmp eq i8 %a, 0 %r = select i1 %c, i64 C2, i64 C1 => %z = zext i8 %a to i64 %r = sub i64 C2, %z Name: false const is bigger Pre: C2 == (C1 + 1) %a = and i8 %x, 1 %c = icmp eq i8 %a, 0 %r = select i1 %c, i64 C1, i64 C2 => %z = zext i8 %a to i64 %r = add i64 C1, %z Differential Revision: https://reviews.llvm.org/D48466 llvm-svn: 335433
* DAG combine "and|or (select c, -1, 0), x" -> "select c, x, 0|-1"Stanislav Mekhanoshin2018-06-211-3/+14
| | | | | | | | | | | | | | | Allowed folding for "and/or" binops with non-constant operand if arguments of select are 0/-1 values. Normally this code with "and" opcode does not get to a DAG combiner and simplified yet in the InstCombine. However AMDGPU produces it during lowering and InstCombine has no chance to optimize it out. In turn the same pattern with "or" opcode can reach DAG. Differential Revision: https://reviews.llvm.org/D48301 llvm-svn: 335250
* [DebugInfo] Make sure all DBG_VALUEs' reguse operands have IsDebug propertyMikael Holmen2018-06-212-25/+9
| | | | | | | | | | | | | | | | | | | | | | Summary: In some cases, these operands lacked the IsDebug property, which is meant to signal that they should not affect codegen. This patch adds a check for this property in the MachineVerifier and adds it where it was missing. This includes refactorings to use MachineInstrBuilder construction functions instead of manually setting up the intrinsic everywhere. Patch by: JesperAntonsson Reviewers: aprantl, rnk, echristo, javed.absar Reviewed By: aprantl Subscribers: qcolombet, sdardis, nemanjai, JDevlieghere, atanasyan, llvm-commits Differential Revision: https://reviews.llvm.org/D48319 llvm-svn: 335214
* [DAGCombine] Fix alignment for offset loads/storesDavid Green2018-06-211-6/+8
| | | | | | | | | | | | | | The alignment parameter to getExtLoad is treated as a base alignment, not the alignment of the load (base + offset). When we infer a better alignment for a Ptr we need to ensure that it applies to the base to prevent the alignment on the load from being wrong. This fixes a bug where the alignment could then be used to incorrectly prove noalias between a load and a store, leading to a miscompile. Differential Revision: https://reviews.llvm.org/D48029 llvm-svn: 335210
* Allow binop C1, (select cc, CF, CT) -> select foldingStanislav Mekhanoshin2018-06-201-10/+28
| | | | | | | | | | Previously this folding was done only if select is a first operand. However, for non-commutative operations constant may go before select. Differential Revision: https://reviews.llvm.org/D48223 llvm-svn: 335167
* [DAG] Don't map a TableId to itself in the ReplacedValues mapBjorn Pettersson2018-06-202-3/+7
| | | | | | | | | | | | | | | | | Summary: Found some regressions (infinite loop in DAGTypeLegalizer::RemapId) after r334880. This patch makes sure that we do map a TableId to itself. Reviewers: niravd Reviewed By: niravd Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D48364 llvm-svn: 335141
* [DAG] Fix and-mask folding when narrowing loads.Nirav Dave2018-06-201-2/+3
| | | | | | | | | | | | | | | | Summary: Check that and masks are strictly smaller than implicit mask from narrowed load. Fixes PR37820. Reviewers: samparker, RKSimon, nemanjai Subscribers: hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D48335 llvm-svn: 335137
* [DAGCombiner] Add some comments to some true/false arguments to make it ↵Craig Topper2018-06-201-2/+2
| | | | | | obvious what they are. NFC llvm-svn: 335095
* [SelectionDAG] Don't crash on inline assembly errors when the inline ↵Craig Topper2018-06-201-2/+11
| | | | | | | | | | | | | | | | | | | | | | | | | assembly return type is a struct. Summary: If we get an error building the SelectionDAG for inline assembly we try to continue and still build the DAG. But if the return type for the inline assembly is a struct we end up crashing because we try to create an UNDEF node with a struct type which isn't valid. Instead we need to create an UNDEF for each element of the struct and join them with merge_values. This patch relies on single operand merge_values being handled gracefully by getMergeValues. If the return type is void there will be no VTs returned by ComputeValueVTs and now we just return instead of calling setValue. Hopefully that's ok, I assumed nothing would need to look up the mapped value for void node. Fixes PR37359 Reviewers: rengolin, rovka, echristo, efriedma, bogner Reviewed By: efriedma Subscribers: craig.topper, llvm-commits Differential Revision: https://reviews.llvm.org/D46560 llvm-svn: 335093
* [WebAssembly] Add WasmEHFuncInfo for unwind destination informationHeejin Ahn2018-06-191-19/+42
| | | | | | | | | | | | | | | Summary: Add WasmEHFuncInfo and routines to calculate and fill in this struct to keep track of unwind destination information. This will be used in other EH related passes. Reviewers: dschuff Subscribers: sbc100, jgravelle-google, sunfish, chrib, llvm-commits Differential Revision: https://reviews.llvm.org/D48263 llvm-svn: 335005
* Utilize new SDNode flag functionality to expand current support for faddMichael Berg2018-06-182-29/+15
| | | | | | | | | | | | | | Summary: This patch originated from D46562 and is a proper subset, with some issues addressed. Reviewers: spatel, hfinkel, wristow, arsenm, javed.absar Reviewed By: spatel Subscribers: wdng, nhaehnle Differential Revision: https://reviews.llvm.org/D47909 llvm-svn: 334996
* refactor of visitFADD for AllowNewConst casesMichael Berg2018-06-181-17/+20
| | | | | | | | | | | | | | Summary: Refactoring for all constant cases which require AllowNewConst and some staging for future fmf usage. Reviewers: spatel, hfinkel, wristow Reviewed By: spatel Subscribers: nhaehnle Differential Revision: https://reviews.llvm.org/D48289 llvm-svn: 334984
* Avoid needing to walk out legalization tables. NFCI.Nirav Dave2018-06-162-193/+157
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Relanding after fixing expensive check from modifying tables. To avoid redundant work, during DAG legalization we keep tables mapping pre-legalized SDValues to post-legalized SDValues and a SDValue-to-SDValue map to enable fast node replacements. However, as the keys are nodes which may be reused it is possible that an entry in a table refers to a now deleted node N (that should have been renamed by the value replacement map) while a new node N' exists. If N' is then replaced that entry would be wrong. Previously we avoided this by when potentially violating this property, walking every table and updating all node pointers. This is very expensive but hopefully rare occurance. This patch assigns each instance of a SDValue used in legalization a unique id and uses these ids in the legalization tables. This avoids any such aliasing issue, avoiding the full table search and allowing more aggressive incremental table pruning. In some cases this is a 1000x speedup to compilation. Reviewers: jyknight, echristo, bogner, tra Reviewed By: bogner Subscribers: dberris, grandinj, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D47959 llvm-svn: 334880
* Utilize new SDNode flag functionality to expand current support for fmaMichael Berg2018-06-161-18/+14
| | | | | | | | | | | | | | Summary: This patch originated from D47388 and is a proper subset of the originating changes, containing only the fmf optimization guard extensions. Reviewers: spatel, hfinkel, wristow, arsenm, javed.absar, rampitec, nhaehnle, nemanjai Reviewed By: rampitec, nhaehnle Subscribers: tpr, nemanjai, wdng Differential Revision: https://reviews.llvm.org/D47918 llvm-svn: 334876
* Revert r334729 "[DAG] Avoid needing to walk out legalization tables. NFCI."Francis Visoiu Mistrih2018-06-152-155/+193
| | | | | | This reverts commit r334729. llvm-svn: 334869
* Revert r334731 "Avoid unused variable in non-assert builds."Francis Visoiu Mistrih2018-06-151-1/+0
| | | | | | | | This reverts commit r334731. It breaks EXPENSIVE_CHECKS bots. llvm-svn: 334868
* [FPEnv] Expand constrained FP POWICameron McInally2018-06-152-4/+22
| | | | | | | | | | | | Modify ExpandStrictFPOp(...) to handle nodes that have scalar operands. Also, add a Strict FMA test and do some other light cleanup in the Strict FP code. Differential Revision: https://reviews.llvm.org/D48149 llvm-svn: 334863
* Utilize new SDNode flag functionality to expand current support for fdivMichael Berg2018-06-151-1/+1
| | | | | | | | | | | | | | Summary: This patch originated from D46562 and is a proper subset, with some issues addressed. Reviewers: spatel, hfinkel, wristow, arsenm Reviewed By: spatel Subscribers: wdng, nhaehnle Differential Revision: https://reviews.llvm.org/D47954 llvm-svn: 334862
* DAG: Fix creating concat_vectors with illegal typeMatt Arsenault2018-06-151-3/+6
| | | | | | | Test passes as is, but fails with future patch to make v4i16/v4f16 legal. llvm-svn: 334823
* easing the constraint for isNegatibleForFree and GetNegatedExpressionMichael Berg2018-06-141-4/+0
| | | | | | | | | | | | | | | | | Summary: Here we relax the old constraint which utilized unsafe with the TargetOption flag HonorSignDependentRoundingFPMathOption, with the assertion that unsafe is no longer needed or never was required for correctness on FDIV/FMUL. Reviewers: spatel, hfinkel, wristow, arsenm, javed.absar Reviewed By: spatel Subscribers: efriedma, wdng, tpr Differential Revision: https://reviews.llvm.org/D48057 llvm-svn: 334769
* updating isNegatibleForFree and GetNegatedExpression with fmf for faddMichael Berg2018-06-141-4/+3
| | | | | | | | | | | | | | Summary: A FMF constraint is added to FADD with unsafe still available as the fallback Reviewers: spatel, wristow, arsenm, hfinkel Reviewed By: spatel Subscribers: wdng Differential Revision: https://reviews.llvm.org/D48180 llvm-svn: 334753
* Avoid unused variable in non-assert builds.Nirav Dave2018-06-141-0/+1
| | | | llvm-svn: 334731
* [DAG] Avoid needing to walk out legalization tables. NFCI.Nirav Dave2018-06-142-193/+155
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To avoid redundant work, during DAG legalization we keep tables mapping pre-legalized SDValues to post-legalized SDValues and a SDValue-to-SDValue map to enable fast node replacements. However, as the keys are nodes which may be reused it is possible that an entry in a table refers to a now deleted node N (that should have been renamed by the value replacement map) while a new node N' exists. If N' is then replaced that entry would be wrong. Previously we avoided this by when potentially violating this property, walking every table and updating all node pointers. This is very expensive but hopefully rare occurance. This patch assigns each instance of a SDValue used in legalization a unique id and uses these ids in the legalization tables. This avoids any such aliasing issue, avoiding the full table search and allowing more aggressive incremental table pruning. In some cases this is a 1000x speedup to compilation. Reviewers: jyknight, echristo, bogner, tra Reviewed By: bogner Subscribers: dberris, grandinj, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D47959 llvm-svn: 334729
* [DAGCombiner] remove hasOneUse() check from fadd constants transformSanjay Patel2018-06-131-7/+6
| | | | | | | | | | | We're constant folding here, so we shouldn't check uses. This matches the IR optimizer behavior. The x86 test shows the expected win. The AArch64 test shows something else. This only seems to happen if the "generic" AArch64 CPU model is used by MachineCombiner, so I'll file a bug report to follow-up. llvm-svn: 334608
* [FPEnv] Expand constrained FP operationsCameron McInally2018-06-131-8/+88
| | | | | | | | | | Add a helper function to expand constrained FP operations as needed. Note that the Strict POWI operation is not handled in this patch since the format is slightly different from the others. Differential Revision: https://reviews.llvm.org/D47491 llvm-svn: 334603
* [DAGCombiner] Recognize more patterns for ABSKrzysztof Parzyszek2018-06-121-7/+27
| | | | | | Differential Revision: https://reviews.llvm.org/D47831 llvm-svn: 334553
* Utilize new SDNode flag functionality to expand current support for fmulMichael Berg2018-06-121-2/+5
| | | | | | | | | | | | | | Summary: This patch originated from D46562 and is a proper subset, with some issues addressed for fmul. Reviewers: spatel, hfinkel, wristow, arsenm Reviewed By: spatel Subscribers: nhaehnle, wdng Differential Revision: https://reviews.llvm.org/D47911 llvm-svn: 334514
* [SelectionDAG] Provide default expansion for rotatesKrzysztof Parzyszek2018-06-122-2/+52
| | | | | | | | | | | | | Implement default legalization of rotates: either in terms of the rotation in the opposite direction (if legal), or in terms of shifts and ors. Implement generating of rotate instructions for Hexagon. Hexagon only supports rotates by an immediate value, so implement custom lowering of ROTL/ROTR on Hexagon. If a rotate is not legal, use the default expansion. Differential Revision: https://reviews.llvm.org/D47725 llvm-svn: 334497
* DAG: Fix extract_subvector combine for a single elementMatt Arsenault2018-06-111-3/+12
| | | | | | | | | | | | | | | | This would fail before because 1x vectors aren't legal, so instead just use the scalar type. Avoids regressions in a future AMDGPU commit to add v4i16/v4f16 as legal types. Test update is just the one test that this triggers on in tree now. It wasn't checking anything before. The result is completely changed since the selects are eliminated. Not sure if it's considered better or not. llvm-svn: 334440
* [DAGCombiner] match vector compare and select sizes with extload operand ↵Sanjay Patel2018-06-101-0/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | (PR37427) This patch started off much more general and ambitious, but it's been a nightmare seeing all the ways x86 vector codegen can go wrong. So the code is still structured to allow extending easily, but it's currently limited in several ways: 1. Only handle cases with an extending load. 2. Only handle cases with a zero constant compare. 3. Ignore setcc with vector bitmask (SetCCWidth != 1) - so AVX512 should be unaffected. The motivating case from PR37427: https://bugs.llvm.org/show_bug.cgi?id=37427 ...is the 1st test, and that shows the expected win - we eliminated the unnecessary intermediate cast. There's a clear regression in the last test (sgt_zero_fp_select) because we longer recognize a 'SHRUNKBLEND' opportunity. I think that general problem is also present in sgt_zero, so I'll try to fix that in a follow-up. We need to match a sign-bit setcc from a sign-extended operand and remove it. Differential Revision: https://reviews.llvm.org/D47330 llvm-svn: 334378
* Use SmallPtrSet instead of SmallSet in places where we iterate over the set.Craig Topper2018-06-091-1/+1
| | | | | | | | SmallSet forwards to SmallPtrSet for pointer types. SmallPtrSet supports iteration, but a normal SmallSet doesn't. So if it wasn't for the forwarding, this wouldn't work. These places were found by hiding the begin/end methods in the SmallSet forwarding llvm-svn: 334343
* [DAGCombiner] clean up comments; NFCSanjay Patel2018-06-081-8/+5
| | | | llvm-svn: 334312
* Utilize new SDNode flag functionality to expand current support for fsubMichael Berg2018-06-081-17/+23
| | | | | | | | | | | | | | Summary: This patch originated from D46562 and is a proper subset, with some issues addressed for fsub. Reviewers: spatel, hfinkel, wristow, arsenm Reviewed By: spatel Subscribers: wdng Differential Revision: https://reviews.llvm.org/D47910 llvm-svn: 334306
* [DAGCombine] Fix for PR37667Sam Parker2018-06-081-0/+16
| | | | | | | | | | While trying to propagate AND masks back to loads, we currently allow one non-load node to be included as a leaf in chain. This fix now limits that node to produce only a single data value. Differential Revision: https://reviews.llvm.org/D47878 llvm-svn: 334268
OpenPOWER on IntegriCloud