summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
Commit message (Collapse)AuthorAgeFilesLines
...
* Revert "[DAG] Improve loads-from-store forwarding to handle TokenFactor"Nirav Dave2016-11-281-13/+2
| | | | | | This reverts commit r287773 which caused issues with ppc64le builds. llvm-svn: 288035
* Use SDValue helpers instead of explicitly going via SDValue::getNode(). NFCISimon Pilgrim2016-11-251-7/+7
| | | | llvm-svn: 287941
* [DAGCombine] Teach DAG combine that if both inputs of a vselect are the ↵Craig Topper2016-11-241-0/+4
| | | | | | | | same, then the condition doesn't matter and the vselect can be removed. Selects with scalar condition already handle this correctly. llvm-svn: 287904
* [DAG] Improve loads-from-store forwarding to handle TokenFactorNirav Dave2016-11-231-2/+13
| | | | | | | | | | | | | Forward store values to matching loads down through token factors. Factored from D14834. Reviewers: jyknight, hfinkel Subscribers: hfinkel, nemanjai, llvm-commits Differential Revision: https://reviews.llvm.org/D26080 llvm-svn: 287773
* [DAGCombiner] Fix infinite loop in vector mul/shl combiningJohn Brawn2016-11-231-6/+6
| | | | | | | | | | | | | | | | | | | | | | We have the following DAGCombiner transformations: (mul (shl X, c1), c2) -> (mul X, c2 << c1) (mul (shl X, C), Y) -> (shl (mul X, Y), C) (shl (mul x, c1), c2) -> (mul x, c1 << c2) Usually the constant shift is optimised by SelectionDAG::getNode when it is constructed, by SelectionDAG::FoldConstantArithmetic, but when we're dealing with vectors and one of those vector constants contains an undef element FoldConstantArithmetic does not fold and we enter an infinite loop. Fix this by making FoldConstantArithmetic use getNode to decide how to fold each vector element, the same as FoldConstantVectorArithmetic does, and rather than adding the constant shift to the work list instead only apply the transformation if it's already been folded into a constant, as if it's not we're going to loop endlessly. Additionally add missing NoOpaques to one of those transformations, which I noticed when writing the tests for this. Differential Revision: https://reviews.llvm.org/D26605 llvm-svn: 287766
* Type legalization for compressstore and expandload intrinsics. Elena Demikhovsky2016-11-231-16/+13
| | | | | | | | Implemented widening (v2f32) and splitting (v16f64). On splitting, I use "popcnt" to calculate memory increment. More type legalization work will come in the next patches. llvm-svn: 287761
* Fix spelling mistakes in SelectionDAG comments. NFC.Simon Pilgrim2016-11-201-2/+2
| | | | | | Identified by Pedro Giffuni in PR27636. llvm-svn: 287487
* DAGCombiner: fix combine of trunc and selectAsaf Badouh2016-11-151-1/+1
| | | | | | | | | | | | | bugzilla: https://llvm.org/bugs/show_bug.cgi?id=29002 pr29002 Differential Revision: https://reviews.llvm.org/D26449 llvm-svn: 286938
* [DAG Combiner] Fix the native computation of the Newton series for reciprocalsEvandro Menezes2016-11-101-28/+30
| | | | | | | | | | | | The generic infrastructure to compute the Newton series for reciprocal and reciprocal square root was conceived to allow a target to compute the series itself. However, the original code did not properly consider this condition if returned by a target. This patch addresses the issues to allow a target to compute the series on its own. Differential revision: https://reviews.llvm.org/D22975 llvm-svn: 286523
* Use common SDLoc. NFCI.Simon Pilgrim2016-11-101-3/+3
| | | | llvm-svn: 286473
* [DAGCombiner] Correctly extract the ConstOrConstSplat shift value for SHL nodesSimon Pilgrim2016-11-101-3/+2
| | | | | | | | We were failing to extract a constant splat shift value if the shifted value was being masked. The (shl (and (setcc) N01CV) N1CV) -> (and (setcc) N01CV<<N1CV) combine was unnecessarily preventing this. llvm-svn: 286454
* DAGCombiner: fix use-after-free when merging consecutive storesNicolai Haehnle2016-11-031-18/+22
| | | | | | | | | | | | | | | Summary: Have MergeConsecutiveStores explicitly return information about the stores that were merged, so that we can safely determine whether the starting node has been freed. Reviewers: chandlerc, bogner, niravd Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D25601 llvm-svn: 285916
* Expandload and Compressstore intrinsicsElena Demikhovsky2016-11-031-2/+2
| | | | | | | | 2 new intrinsics covering AVX-512 compress/expand functionality. This implementation includes syntax, DAG builder, operation lowering and tests. Does not include: handling of illegal data types, codegen prepare pass and the cost model. llvm-svn: 285876
* [DAG] x | x --> xSanjay Patel2016-10-301-0/+4
| | | | llvm-svn: 285522
* [DAG] x & x --> xSanjay Patel2016-10-301-0/+4
| | | | llvm-svn: 285521
* [DAGCombiner] Fix a crash visiting `AND` nodes.Davide Italiano2016-10-281-1/+6
| | | | | | | | | | Instead of asserting that the shift count is != 0 we just bail out as it's not profitable trying to optimize a node which will be removed anyway. Differential Revision: https://reviews.llvm.org/D26098 llvm-svn: 285480
* [DAGCombiner] Enable (urem x, (shl pow2, y)) -> (and x, (add (shl pow2, y), ↵Simon Pilgrim2016-10-251-3/+3
| | | | | | -1)) combine for splatted vectors llvm-svn: 285129
* [DAGCombiner] Enable srem(x.y) -> urem(x,y) combine for vectorsSimon Pilgrim2016-10-251-4/+2
| | | | | | SelectionDAG::SignBitIsZero (via SelectionDAG::computeKnownBits) has supported vectors since rL280927 llvm-svn: 285123
* [DAGCombiner] Enable sdiv(x.y) -> udiv(x,y) combine for vectorsSimon Pilgrim2016-10-251-4/+2
| | | | | | SelectionDAG::SignBitIsZero (via SelectionDAG::computeKnownBits) has supported vectors since rL280927 llvm-svn: 285118
* [DAGCombine] Preserve shuffles when one of the vector operands is constantZvi Rackover2016-10-251-34/+75
| | | | | | | | | | | | | | | | | | | | Summary: Do *not* perform combines such as: vector_shuffle<4,1,2,3>(build_vector(Ud, C0, C1 C2), scalar_to_vector(X)) -> build_vector(X, C0, C1, C2) Keeping the shuffle allows lowering the constant build_vector to a materialized constant vector (such as a vector-load from the constant-pool or some other idiom). Reviewers: delena, igorb, spatel, mkuper, andreadb, RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D25524 llvm-svn: 285063
* Use SDValue::getConstantOperandVal() helper. NFCI.Simon Pilgrim2016-10-231-4/+1
| | | | llvm-svn: 284949
* [DAG] fold negation of sign-bitSanjay Patel2016-10-211-11/+27
| | | | | | | | | | | | | | | | 0 - X --> 0, if the sub is NUW 0 - X --> 0, if X is 0 or the minimum signed value and the sub is NSW 0 - X --> X, if X is 0 or the minimum signed value This is the DAG equivalent of: https://reviews.llvm.org/rL284649 plus the fold for the NUW case which already existed in InstSimplify. Note that we miss a vector fold because of a deficiency in the DAG version of computeKnownBits(). llvm-svn: 284844
* [DAG] use SDNode flags 'nsz' to enable fadd/fsub with zero foldsSanjay Patel2016-10-211-16/+20
| | | | | | | | | | | | | | | | | | | | | As discussed in D24815, let's start the process of killing off the broken fast-math global state housed in TargetOptions and eliminate the need for function-level fast-math attributes. Here we enable two similar folds that are possible when we don't care about signed-zero: fadd nsz x, 0 --> x fsub nsz 0, x --> -x Note that although the test cases include a 'sin' function call, I'm side-stepping the FMF-on-calls question (and lack of support in the DAG) for now. It's not needed for these tests - isNegatibleForFree/GetNegatedExpression just look through a ISD::FSIN node. Also, when we create an FNEG node and propagate the Flags of the FSUB to it, this doesn't actually do anything today because Flags are silently dropped for any node that is not a binary operator. Differential Revision: https://reviews.llvm.org/D25297 llvm-svn: 284824
* [Target] remove TargetRecip class; 2nd trySanjay Patel2016-10-201-11/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a retry of r284495 which was reverted at r284513 due to use-after-scope bugs caused by faulty usage of StringRef. This version also renames a pair of functions: getRecipEstimateDivEnabled() getRecipEstimateSqrtEnabled() as suggested by Eric Christopher. original commit msg: [Target] remove TargetRecip class; move reciprocal estimate isel functionality to TargetLowering This is a follow-up to https://reviews.llvm.org/D24816 - where we changed reciprocal estimates to be function attributes rather than TargetOptions. This patch is intended to be a structural, but not functional change. By moving all of the TargetRecip functionality into TargetLowering, we can remove all of the reciprocal estimate state, shield the callers from the string format implementation, and simplify/localize the logic needed for a target to enable this. If a function has a "reciprocal-estimates" attribute, those settings may override the target's default reciprocal preferences for whatever operation and data type we're trying to optimize. If there's no attribute string or specific setting for the op/type pair, just use the target default settings. As noted earlier, a better solution would be to move the reciprocal estimate settings to IR instructions and SDNodes rather than function attributes, but that's a multi-step job that requires infrastructure improvements. I intend to work on that, but it's not clear how long it will take to get all the pieces in place. Differential Revision: https://reviews.llvm.org/D25440 llvm-svn: 284746
* [DAGCombiner] Add general constant vector support to (srl (shl x, c), c) -> ↵Simon Pilgrim2016-10-201-8/+8
| | | | | | | | (and x, cst2) We already supported scalar constant / splatted constant vector - now accepts any (non opaque) constant scalar / vector llvm-svn: 284717
* Merged nested ifs. NFCI.Simon Pilgrim2016-10-191-7/+6
| | | | llvm-svn: 284616
* [DAGCombiner] Add general constant vector support to (shl (add x, c1), c2) ↵Simon Pilgrim2016-10-191-4/+5
| | | | | | | | -> (add (shl x, c2), c1 << c2) We already supported scalar constant / splatted constant vector - now accepts any (non opaque) constant scalar / vector llvm-svn: 284613
* [DAGCombiner] Add general constant vector support to (shl (sra x, c1), c1) ↵Simon Pilgrim2016-10-191-7/+6
| | | | | | | | -> (and x, (shl -1, c1)) We already supported scalar constant / splatted constant vector - now accepts any (non opaque) constant scalar / vector llvm-svn: 284608
* [DAGCombiner] Add general constant vector support to (shl (mul x, c1), c2) ↵Simon Pilgrim2016-10-191-5/+6
| | | | | | | | -> (mul x, c1 << c2) We already supported scalar constant / splatted constant vector - now accepts any (non opaque) constant scalar / vector llvm-svn: 284607
* [DAGCombiner] Just call isConstOrConstSplat directly. NFCI.Simon Pilgrim2016-10-191-8/+4
| | | | | | This will get the same ConstantSDNode scalar or vector splat value as the current separate dyn_cast<ConstantSDNode> / isVector() approach. llvm-svn: 284578
* [DAGCombine] Generalize distributeTruncateThroughAnd to work with any ↵Simon Pilgrim2016-10-191-13/+9
| | | | | | non-opaque constant or constant vector llvm-svn: 284574
* revert r284495: [Target] remove TargetRecip classSanjay Patel2016-10-181-32/+9
| | | | | | There's something wrong with the StringRef usage while parsing the attribute string. llvm-svn: 284513
* [Target] remove TargetRecip class; move reciprocal estimate isel ↵Sanjay Patel2016-10-181-9/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | functionality to TargetLowering This is a follow-up to D24816 - where we changed reciprocal estimates to be function attributes rather than TargetOptions. This patch is intended to be a structural, but not functional change. By moving all of the TargetRecip functionality into TargetLowering, we can remove all of the reciprocal estimate state, shield the callers from the string format implementation, and simplify/localize the logic needed for a target to enable this. If a function has a "reciprocal-estimates" attribute, those settings may override the target's default reciprocal preferences for whatever operation and data type we're trying to optimize. If there's no attribute string or specific setting for the op/type pair, just use the target default settings. As noted earlier, a better solution would be to move the reciprocal estimate settings to IR instructions and SDNodes rather than function attributes, but that's a multi-step job that requires infrastructure improvements. I intend to work on that, but it's not clear how long it will take to get all the pieces in place. Differential Revision: https://reviews.llvm.org/D25440 llvm-svn: 284495
* [DAGCombiner] Add splatted vector support to (udiv x, (shl pow2, y)) -> x ↵Simon Pilgrim2016-10-181-2/+3
| | | | | | >>u (log2(pow2)+y) llvm-svn: 284491
* Strip trailing whitespace (NFCI)Simon Pilgrim2016-10-181-1/+1
| | | | llvm-svn: 284478
* [DAG] make isConstOrConstSplat and isConstOrConstSplatFP more accessible; NFCSanjay Patel2016-10-171-38/+0
| | | | | | | | | | | | As noted in: https://reviews.llvm.org/D25685 This is the next-to-smallest step needed to enable the ComputeNumSignBits fix in that patch. In a minor attempt to keep some structure, we're pulling the FP helper over along with its integer sibling, but clearly we can and should do more refactoring of the similar helper functions in DAGCombiner and SelectionDAG to simplify and not duplicate functionality. llvm-svn: 284421
* [DAG] optimize away an arithmetic-right-shift of a 0 or -1 valueSanjay Patel2016-10-171-0/+4
| | | | | | | | | This came up as part of: https://reviews.llvm.org/D25485 Note that the vector case is missed because ComputeNumSignBits() is deficient for vectors. llvm-svn: 284395
* [DAG] avoid creating illegal node when transforming negated shifted sign bitSanjay Patel2016-10-141-2/+3
| | | | | | | | Eli noted this potential bug in the post-commit thread for: https://reviews.llvm.org/rL284239 ...but I'm not sure how to trigger it, so there's no test case yet. llvm-svn: 284268
* [DAG] add folds for negated shifted sign bitSanjay Patel2016-10-141-0/+13
| | | | | | | | | The same folds exist in InstCombine already. This came up as part of: https://reviews.llvm.org/D25485 llvm-svn: 284239
* Fix use-after-freesNicolai Haehnle2016-10-141-2/+2
| | | | | | Extracted from D25313, as suggested by Justin Bogner. llvm-svn: 284220
* [DAGCombiner] Teach createBuildVecShuffle to handle cases where input ↵Craig Topper2016-10-141-5/+9
| | | | | | | | vectors are less than half of the output vector size. This will be needed by a future commit to support sign/zero extending from v8i8 to v8i64 which requires a sign/zero_extend_vector_inreg to be created which requires v8i8 to be concatenated upto v64i8 and goes through this code. llvm-svn: 284204
* [DAG] hoist DL(N) and fix formatting; NFCSanjay Patel2016-10-131-24/+31
| | | | llvm-svn: 284170
* Revert "In visitSTORE, always use FindBetterChain, rather than only when ↵Nirav Dave2016-10-131-120/+271
| | | | | | | | | UseAA is enabled." This reverts commit r284151 which appears to be triggering a LTO failures on Hexagon llvm-svn: 284157
* In visitSTORE, always use FindBetterChain, rather than only when UseAA is ↵Nirav Dave2016-10-131-271/+120
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | enabled. Retrying after upstream changes. Simplify Consecutive Merge Store Candidate Search Now that address aliasing is much less conservative, push through simplified store merging search which only checks for parallel stores through the chain subgraph. This is cleaner as the separation of non-interfering loads/stores from the store-merging logic. Whem merging stores, search up the chain through a single load, and finds all possible stores by looking down from through a load and a TokenFactor to all stores visited. This improves the quality of the output SelectionDAG and generally the output CodeGen (with some exceptions). Additional Minor Changes: 1. Finishes removing unused AliasLoad code 2. Unifies the the chain aggregation in the merged stores across code paths 3. Re-add the Store node to the worklist after calling SimplifyDemandedBits. 4. Increase GatherAllAliasesMaxDepth from 6 to 18. That number is arbitrary, but seemed sufficient to not cause regressions in tests. This finishes the change Matt Arsenault started in r246307 and jyknight's original patch. Many tests required some changes as memory operations are now reorderable. Some tests relying on the order were changed to use volatile memory operations Noteworthy tests: CodeGen/AArch64/argument-blocks.ll - It's not entirely clear what the test_varargs_stackalign test is supposed to be asserting, but the new code looks right. CodeGen/AArch64/arm64-memset-inline.lli - CodeGen/AArch64/arm64-stur.ll - CodeGen/ARM/memset-inline.ll - The backend now generates *worse* code due to store merging succeeding, as we do do a 16-byte constant-zero store efficiently. CodeGen/AArch64/merge-store.ll - Improved, but there still seems to be an extraneous vector insert from an element to itself? CodeGen/PowerPC/ppc64-align-long-double.ll - Worse code emitted in this case, due to the improved store->load forwarding. CodeGen/X86/dag-merge-fast-accesses.ll - CodeGen/X86/MergeConsecutiveStores.ll - CodeGen/X86/stores-merging.ll - CodeGen/Mips/load-store-left-right.ll - Restored correct merging of non-aligned stores CodeGen/AMDGPU/promote-alloca-stored-pointer-value.ll - Improved. Correctly merges buffer_store_dword calls CodeGen/AMDGPU/si-triv-disjoint-mem-access.ll - Improved. Sidesteps loading a stored value and merges two stores CodeGen/X86/pr18023.ll - This test has been removed, as it was asserting incorrect behavior. Non-volatile stores *CAN* be moved past volatile loads, and now are. CodeGen/X86/vector-idiv.ll - CodeGen/X86/vector-lzcnt-128.ll - It's basically impossible to tell what these tests are actually testing. But, looks like the code got better due to the memory operations being recognized as non-aliasing. CodeGen/X86/win32-eh.ll - Both loads of the securitycookie are now merged. CodeGen/AMDGPU/vgpr-spill-emergency-stack-slot-compute.ll - This test appears to work but no longer exhibits the spill behavior. Reviewers: arsenm, hfinkel, tstellarAMD, jyknight, nhaehnle Subscribers: wdng, nhaehnle, nemanjai, arsenm, weimingz, niravd, RKSimon, aemerson, qcolombet, dsanders, resistor, tstellarAMD, t.p.northover, spatel Differential Revision: https://reviews.llvm.org/D14834 llvm-svn: 284151
* [DAGCombiner] Add vector support to (mul (shl X, Y), Z) -> (shl (mul X, Z), ↵Simon Pilgrim2016-10-131-7/+6
| | | | | | Y) style combines llvm-svn: 284122
* [DAGCombiner] Add vector support to C2-(A+C1) -> (C2-C1)-A foldingSimon Pilgrim2016-10-131-5/+5
| | | | llvm-svn: 284117
* [DAGCombiner] Add vector support to (sub -1, x) -> (xor x, -1) canonicalizationSimon Pilgrim2016-10-131-1/+12
| | | | | | Improves commutation potential llvm-svn: 284113
* [DAGCombiner] Update most ADD combines to support general vector combinesSimon Pilgrim2016-10-121-12/+54
| | | | | | | | Add a number of helper functions to match scalar or vector equivalent constant/splat values to allow most of the combine patterns to be used by vectors. Differential Revision: https://reviews.llvm.org/D25374 llvm-svn: 284015
* [DAGCombiner] Do not remove the load of stored values when optimizations are ↵Konstantin Zhuravlyov2016-10-121-1/+2
| | | | | | | | | | | | | | | | | | | | disabled This combiner breaks debug experience and should not be run when optimizations are disabled. For example: int main() { int j = 0; j += 2; if (j == 2) return 0; return 5; } When debugging this code compiled in /O0, it should be valid to break at line "j+=2;" and edit the value of j. It should change the return value of the function. Differential Revision: https://reviews.llvm.org/D19268 llvm-svn: 284014
* [DAG] Fix crash in build_vector -> vector_shuffle combineMichael Kuperstein2016-10-111-0/+5
| | | | | | | | Fixes a crash in the build_vector -> vector_shuffle combine when the first vector input is twice as wide as the output, and the second input vector is even wider. llvm-svn: 283953
OpenPOWER on IntegriCloud