summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
Commit message (Collapse)AuthorAgeFilesLines
...
* [DAG] Remove Node csonstruction from BaseIndexOffset match. NFCI.Nirav Dave2017-06-211-52/+69
| | | | | | | | Move GlobalAddress Offset decomposition from initial match into comparision check and removing the possibility of constructing a new offseted global address when examining addresses. llvm-svn: 305917
* [DAGCombiner] Add another combine from build vector to shuffleGuy Blank2017-06-211-0/+5
| | | | | | | Add support for combining a build vector to a shuffle. When the build vector is of extracted elements from 2 vectors (vec1, vec2) where vec2 is 2 times smaller than vec1. llvm-svn: 305883
* [DAG] Simplify BaseIndexOffset. NFCI.Nirav Dave2017-06-201-59/+57
| | | | | | Remove tail calls and cleanup codeflow. llvm-svn: 305768
* Allow truncated and extend memory operations in Store Merge. NFCI.Nirav Dave2017-06-191-24/+46
| | | | | | | | | | As all store merges checks are based on the memory operation performed, allow use of truncated stores and extended loads as valid input candidates for merging. Relanding after fixing selection between truncated and normal store. llvm-svn: 305701
* [SelectionDAG] Use APInt::isSubsetOf. NFCCraig Topper2017-06-161-1/+1
| | | | llvm-svn: 305606
* [SelectionDAG] Use APInt::isNullValue/isOneValue. NFCCraig Topper2017-06-161-2/+2
| | | | llvm-svn: 305605
* Revert "[DAG] Allow truncated and extend memory operations in Store Merge. ↵Ahmed Bougacha2017-06-151-21/+10
| | | | | | | | NFCI." This reverts commit r305468, as it caused PR33475. llvm-svn: 305527
* [DAG] As StoreMerge now generates only legal nodes remove unecessary guard ↵Nirav Dave2017-06-151-4/+2
| | | | | | when run post-legalization NFCI. llvm-svn: 305477
* [DAG] Defer Pre/Post IndexStore merge to after mergestore. NFCI.Nirav Dave2017-06-151-4/+4
| | | | | | | | In preparation for doing storemerge post-legalization, reorder visitSTORE passes to move pre/post-index combining after store merge. Reordered passes other than store merge are unaffected. llvm-svn: 305473
* [DAG] Allow truncated and extend memory operations in Store Merge. NFCI.Nirav Dave2017-06-151-10/+21
| | | | | | | | As all store merges checks are based on the memory operation performed, allow use of truncated stores and extended loads as valid input candidates for merging. llvm-svn: 305468
* [DAG] Make MergeStores generate legalized stores. NFCI.Nirav Dave2017-06-151-4/+21
| | | | | | | Realized merged stores as truncstores if store will be realized as such by legalization. llvm-svn: 305467
* [DAG] Use correct size for truncated store merge of load. NFCI.Nirav Dave2017-06-151-2/+2
| | | | | | | Avoid non-legal memory ops by checking correct size when merging stores of loads into a extload-truncstore pair. llvm-svn: 305466
* [DAG] add helper to bind memop chains; NFCISanjay Patel2017-06-121-15/+1
| | | | | | | | | | This step is just intended to reduce code duplication rather than change any functionality. A follow-up would be to replace PPCTargetLowering::spliceIntoChain() usage with this new helper. Differential Revision: https://reviews.llvm.org/D33649 llvm-svn: 305192
* [DAGCombine] Make sure we check the ResNo from UADDO before combiningAmaury Sechet2017-06-111-1/+2
| | | | | | | | | | | | Summary: UADDO has 2 result, and one must check the result no before doing any kind of combine. Without it, the transform is invalid. Reviewers: joerg Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34088 llvm-svn: 305162
* [DAG] Improve Store Merge candidate pruning. NFC.Nirav Dave2017-06-071-3/+15
| | | | | | | | | When considering merging stores values are the results of loads only consider stores whose values come from loads from the same base. This fixes much of the longer compile times in PR33330. llvm-svn: 304934
* [DAG] Move SelectionDAG::isCommutativeBinOp to TargetLowering.Simon Pilgrim2017-06-071-1/+1
| | | | | | | | This will allow commutation of target-specific DAG nodes in future patches Differential Revision: https://reviews.llvm.org/D33882 llvm-svn: 304911
* [DAGCombine] Fix unchecked calls to DAGCombiner::*ExtPromoteOperandSanjay Patel2017-06-051-6/+6
| | | | | | | | | | | | | | | | | Other calls to DAGCombiner::*PromoteOperand check the result, but here it could cause an assertion in getNode. Falling back to any extend in this case instead of failing outright seems correct to me. No test case because: The failure was triggered by an out of tree backend. In order to trigger it, a backend would need to overload TargetLowering::IsDesirableToPromoteOp to return true for a type for which ISD::SIGN_EXTEND_INREG is marked illegal. In tree, only X86 overloads and sometimes returns true for MVT::i16 yet it marks setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::i16 , Legal);. Patch by Jacob Young! Differential Revision: https://reviews.llvm.org/D33633 llvm-svn: 304723
* [SDAG] Fix CombineTo ordering in visitZERO_EXTEND and visitSIGN_EXTENDNirav Dave2017-06-011-15/+8
| | | | | | | | | | | | Reorder CombineTo Calls to prevent references to stale/deleted SDNodes which caused undue assertions. Reviewers: dbabokin Subscribers: aemerson, rengolin, llvm-commits Differential Revision: https://reviews.llvm.org/D31625 llvm-svn: 304460
* DAG: Remove pointless type checkMatt Arsenault2017-06-011-1/+1
| | | | | | These are only integer operations. llvm-svn: 304417
* Only generate addcarry node when it is legal.Amaury Sechet2017-06-011-7/+11
| | | | | | | | | | | | | | | Summary: This is a problem uncovered by stage2 testing. ADDCARRY end up being generated on target that do not support it. The patch that introduced the problem has other patches layed on top of it, so we want to fix the issue rather than revert it to avoid creating a lor of churn. A regression test will be added shortly, but this is committed as this in order to get the build back to green promptly. Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D33770 llvm-svn: 304409
* Do not legalize large setcc with setcce, introduce setcccarry and do it with ↵Amaury Sechet2017-06-011-0/+15
| | | | | | | | | | | | | | | | | usubo/setcccarry. Summary: This is a continuation of the work started in D29872 . Passing the carry down as a value rather than as a glue allows for further optimizations. Introducing setcccarry makes the use of addc/subc unecessary and we can start the removal process. This patch only introduce the optimization strictly required to get the same level of optimization as was available before nothing more. Reviewers: jyknight, nemanjai, mkuper, spatel, RKSimon, zvi, bkramer Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D33374 llvm-svn: 304404
* [DAGCombine] Refactor common addcarry pattern.Amaury Sechet2017-06-011-0/+35
| | | | | | | | | | | | Summary: This pattern is no very useful per se, but it exposes optimization for toehr patterns that wouldn't kick in otherwize. It's very common and worth optimizing for. Reviewers: jyknight, nemanjai, mkuper, spatel, RKSimon, zvi, bkramer Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D32756 llvm-svn: 304402
* [DAGCombine] (add/uaddo X, Carry) -> (addcarry X, 0, Carry)Amaury Sechet2017-06-011-0/+49
| | | | | | | | | | | | | | | Summary: This enables further transforms. Depends on D32916 Reviewers: jyknight, nemanjai, mkuper, spatel, RKSimon, zvi, bkramer Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D32925 llvm-svn: 304401
* [DAG] Avoid use of stale store.Nirav Dave2017-05-311-2/+2
| | | | | | | | | | | | | | | | | | Correct references to alignment of store which may be deleted in a previous iteration of merge. Instead use first store that would be merged. Corrects pr33172's use-after-poison caught by ASan. Reviewers: spatel, hfinkel, RKSimon Reviewed By: RKSimon Subscribers: thegameg, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D33686 llvm-svn: 304299
* [DAGCombiner] fix load narrowing transform to exclude loads with extensionSanjay Patel2017-05-291-1/+2
| | | | | | | | | | The extending load possibility was missed in: https://reviews.llvm.org/rL304072 We might want to handle this cases as a follow-up, but bailing out for now to avoid miscompiling. llvm-svn: 304153
* [DAGCombiner] use narrow load to avoid vector extractSanjay Patel2017-05-271-0/+50
| | | | | | | | | | | | | | | | | | If we have (extract_subvector(load wide vector)) with no other users, that can just be (load narrow vector). This is intentionally conservative. Follow-ups may loosen the one-use constraint to account for the extract cost or just remove the one-use check. The memop chain updating is based on code that already exists multiple times in x86 lowering, so that should be pulled into a helper function as a follow-up. Background: this is a potential improvement noticed via regressions caused by making x86's peekThroughBitcasts() not loop on consecutive bitcasts (see comments in D33137). Differential Revision: https://reviews.llvm.org/D33578 llvm-svn: 304072
* Make helper functions static. NFC.Benjamin Kramer2017-05-261-5/+6
| | | | llvm-svn: 304029
* [DAGCombiner] use narrow vector ops to eliminate concat/extract (PR32790)Sanjay Patel2017-05-261-0/+96
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In the best case: extract (binop (concat X1, X2), (concat Y1, Y2)), N --> binop XN, YN ...we kill all of the extract/concat and just have narrow binops remaining. If only one of the binop operands is amenable, this transform is still worthwhile because we kill some of the extract/concat. Optional bitcasting makes the code more complicated, but there doesn't seem to be a way to avoid that. The TODO about extending to more than bitwise logic is there because we really will regress several x86 tests including madd, psad, and even a plain integer-multiply-by-2 or shift-left-by-1. I don't think there's anything fundamentally wrong with this patch that would cause those regressions; those folds are just missing or brittle. If we extend to more binops, I found that this patch will fire on at least one non-x86 regression test. There's an ARM NEON test in test/CodeGen/ARM/coalesce-subregs.ll with a pattern like: t5: v2f32 = vector_shuffle<0,3> t2, t4 t6: v1i64 = bitcast t5 t8: v1i64 = BUILD_VECTOR Constant:i64<0> t9: v2i64 = concat_vectors t6, t8 t10: v4f32 = bitcast t9 t12: v4f32 = fmul t11, t10 t13: v2i64 = bitcast t12 t16: v1i64 = extract_subvector t13, Constant:i32<0> There was no functional change in the codegen from this transform from what I could see though. For the x86 test changes: 1. PR32790() is the closest call. We don't reduce the AVX1 instruction count in that case, but we improve throughput. Also, on a core like Jaguar that double-pumps 256-bit ops, there's an unseen win because two 128-bit ops have the same cost as the wider 256-bit op. SSE/AVX2/AXV512 are not affected which is expected because only AVX1 has the extract/concat ops to match the pattern. 2. do_not_use_256bit_op() is the best case. Everyone wins by avoiding the concat/extract. Related bug for IR filed as: https://bugs.llvm.org/show_bug.cgi?id=33026 3. The SSE diffs in vector-trunc-math.ll are just scheduling/RA, so nothing real AFAICT. 4. The AVX1 diffs in vector-tzcnt-256.ll are all the same pattern: we reduced the instruction count by one in each case by eliminating two insert/extract while adding one narrower logic op. https://bugs.llvm.org/show_bug.cgi?id=32790 Differential Revision: https://reviews.llvm.org/D33137 llvm-svn: 303997
* [DAG] Move legal type checks in store merge to be checked onlyNirav Dave2017-05-261-2/+4
| | | | | | on non-legal cases. NFC. llvm-svn: 303994
* [DAG] Prevent crashes when merging constant stores with high-bit set. NFC.Nirav Dave2017-05-241-2/+2
| | | | llvm-svn: 303802
* [DAG] Add AddressSpace parameter to canMergeStoresTo. NFC.Nirav Dave2017-05-231-7/+10
| | | | llvm-svn: 303673
* [DAG] Add canMergeStoresTo predicate checks. NFCI.Nirav Dave2017-05-231-4/+6
| | | | | | Propagate canMergeStoresTo checks to missing cases in StoreMerge. llvm-svn: 303668
* [DAG] Rework store merge to loop on load candidates. NFCI.Nirav Dave2017-05-221-202/+225
| | | | | | | Continue to consider remaining candidate merges until all possible merges have been considered. llvm-svn: 303560
* [DAGCombine] (addcarry 0, 0, X) -> (ext/trunc X)Amaury Sechet2017-05-191-0/+11
| | | | | | | | | | | | | | | Summary: While this makes some case better and some case worse - so it's unclear if it is a worthy combine just by itself - this is a useful canonicalisation. As per discussion in D32756 . Reviewers: jyknight, nemanjai, mkuper, spatel, RKSimon, zvi, bkramer Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D32916 llvm-svn: 303441
* Elide stores which are overwritten without being observed.Nirav Dave2017-05-161-7/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: In SelectionDAG, when a store is immediately chained to another store to the same address, elide the first store as it has no observable effects. This is causes small improvements dealing with intrinsics lowered to stores. Test notes: * Many testcases overwrite store addresses multiple times and needed minor changes, mainly making stores volatile to prevent the optimization from optimizing the test away. * Many X86 test cases optimized out instructions associated with associated with va_start. * Note that test_splat in CodeGen/AArch64/misched-stp.ll no longer has dependencies to check and can probably be removed and potentially replaced with another test. Reviewers: rnk, john.brawn Subscribers: aemerson, rengolin, qcolombet, jyknight, nemanjai, nhaehnle, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D33206 llvm-svn: 303198
* [DAG] Prune deleted nodes in TokenFactorNirav Dave2017-05-161-8/+3
| | | | | | Fix visitTokenFactor to correctly remove deleted nodes. NFC. llvm-svn: 303181
* [PPC] Move the combine "a << (b % (sizeof(a) * 8)) -> (PPCshl a, b)" to the ↵Tim Shen2017-05-121-33/+0
| | | | | | | | | | | | | | | | | | | | | | backend. NFC. Summary: Eli pointed out that it's unsafe to combine the shifts to ISD::SHL etc., because those are not defined for b > sizeof(a) * 8, even after some of the combiners run. However, PPCISD::SHL defines that behavior (as the instructions themselves). Move the combination to the backend. The tests in shift_mask.ll still pass. Reviewers: echristo, hfinkel, efriedma, iteratee Subscribers: nemanjai, llvm-commits Differential Revision: https://reviews.llvm.org/D33076 llvm-svn: 302937
* [DAGCombine] Use SelectionDAG::getAnyExtOrTrunc helper. NFCI.Simon Pilgrim2017-05-121-8/+2
| | | | llvm-svn: 302907
* [DAGCombine] Use SelectionDAG::getZExtOrTrunc helper. NFCI.Simon Pilgrim2017-05-121-8/+2
| | | | llvm-svn: 302897
* [DAGCombine] Use SelectionDAG::getAnyExtOrTrunc helper. NFCI.Simon Pilgrim2017-05-111-18/+4
| | | | llvm-svn: 302808
* Revert "[SDAG] Relax conditions under stores of loaded values can be merged"David L. Jones2017-05-101-22/+10
| | | | | | | | | | | | | | | | | | | | | This reverts r302712. The change fails with ASAN enabled: ERROR: AddressSanitizer: use-after-poison on address ... at ... READ of size 2 at ... thread T0 #0 ... in llvm::SDNode::getNumValues() const <snip>/include/llvm/CodeGen/SelectionDAGNodes.h:855:42 #1 ... in llvm::SDNode::hasAnyUseOfValue(unsigned int) const <snip>/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:7270:3 #2 ... in llvm::SDValue::use_empty() const <snip> include/llvm/CodeGen/SelectionDAGNodes.h:1042:17 #3 ... in (anonymous namespace)::DAGCombiner::MergeConsecutiveStores(llvm::StoreSDNode*) <snip>/lib/CodeGen/SelectionDAG/DAGCombiner.cpp:12944:7 Reviewers: niravd Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D33081 llvm-svn: 302746
* [SDAG] Relax conditions under stores of loaded values can be mergedNirav Dave2017-05-101-10/+22
| | | | | | | | | | | | | | | | | | | | | | | | Summary: Allow consecutive stores whose values come from consecutive loads to merged in the presense of other uses of the loads. Previously this was disallowed as in general the merged load cannot be shared with the other uses. Merging N stores into 1 may cause as many as N redundant loads. However in the context of caching this should have neglible affect on memory pressure and reduce instruction count making it almost always a win. Fixes PR32086. Reviewers: spatel, jyknight, andreadb, hfinkel, efriedma Reviewed By: efriedma Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D30471 llvm-svn: 302712
* Small refactoring in DAGCombine. NFCAmaury Sechet2017-05-101-3/+3
| | | | llvm-svn: 302699
* [DAGCombiner] Dropped explicit (sra 0, x) -> 0 and (sra -1, x) -> 0 folds.Simon Pilgrim2017-05-101-6/+2
| | | | | | These are both handled (and tested) by the earlier ComputeNumSignBits == EltSizeInBits fold. llvm-svn: 302651
* [DAGCombiner] Add vector support to fold (shl/srl 0, x) -> 0Simon Pilgrim2017-05-101-2/+2
| | | | llvm-svn: 302641
* [CodeGen] Don't require AA in SDAGISel at -O0.Ahmed Bougacha2017-05-101-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | Before r247167, the pass manager builder controlled which AA implementations were used, exporting them all in the AliasAnalysis analysis group. Now, AAResultsWrapperPass always uses BasicAA, but still uses other AA implementations if made available in the pass pipeline. But regardless, SDAGISel is required at O0, and really doesn't need to be doing fancy optimizations based on useful AA results. Don't require AA at CodeGenOpt::None, and only use it otherwise. This does have a functional impact (and one testcase is pessimized because we can't reuse a load). But I think that's desirable no matter what. Note that this alone doesn't result in less DT computations: TwoAddress was previously able to reuse the DT we computed for SDAG. That will be fixed separately. Differential Revision: https://reviews.llvm.org/D32766 llvm-svn: 302611
* DAGCombine: Combine shuffles of splat-shufflesZvi Rackover2017-05-091-0/+54
| | | | | | | | | | | | | | Summary: Reapply r299047, but this time handle correctly splat-masks with undef elements. Reviewers: spatel, RKSimon, eli.friedman, andreadb Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D31961 llvm-svn: 302583
* [DAGCombiner] If ISD::ABS is legal/custom, use it directly instead of ↵Simon Pilgrim2017-05-061-0/+3
| | | | | | | | | | canonicalizing first. Remove an extra canonicalization step if ISD::ABS is going to be used anyway. Updated x86 abs combine to check that we are lowering from both canonicalizations. llvm-svn: 302337
* [DAGCombine] Transform (fadd A, (fmul B, -2.0)) -> (fsub A, (fadd B, B)).Chad Rosier2017-05-041-0/+18
| | | | | | Differential Revision: http://reviews.llvm.org/D32596 llvm-svn: 302153
* [DAGCombine] (addcarry (add|uaddo X, Y), 0, Carry) -> (addcarry X, Y, Carry)Amaury Sechet2017-05-031-0/+19
| | | | | | | | | | | | Summary: Do the transform when the carry isn't used. It's a pattern exposed when legalizing large integers. Reviewers: jyknight, nemanjai, mkuper, spatel, RKSimon, zvi, bkramer Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D32755 llvm-svn: 302047
OpenPOWER on IntegriCloud