summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
Commit message (Collapse)AuthorAgeFilesLines
...
* [DAG] Relax type restriction for store mergeNirav Dave2017-08-101-24/+64
| | | | | | | | | | | | | | Summary: Allow stores of bitcastable types to be merged by peeking through BITCAST nodes and recasting stored values constant and vector extract nodes as necessary. Reviewers: jyknight, hfinkel, efriedma, RKSimon, spatel Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34569 llvm-svn: 310655
* [DAG] Cleanup unused nodes after store merge. NFCI.Nirav Dave2017-08-101-1/+11
| | | | llvm-svn: 310648
* [DAG] Rewrite expression. NFC.Nirav Dave2017-08-101-2/+2
| | | | llvm-svn: 310608
* [DAG] Explicitly cleanup merged load values during store merge. NFCI.Nirav Dave2017-08-091-2/+8
| | | | llvm-svn: 310474
* [DAG] Introduce peekThroughBitcast function. NFCI.Nirav Dave2017-08-081-23/+14
| | | | llvm-svn: 310405
* [DAG] Update comments. NFC.Nirav Dave2017-08-081-8/+9
| | | | llvm-svn: 310404
* [DAGCombiner] simplifyShuffleMask - handle UNDEF inputs from shuffles as ↵Simon Pilgrim2017-08-081-11/+10
| | | | | | | | well as BUILD_VECTOR Minor extension to D36393 llvm-svn: 310372
* [DAGCombiner] Simplify shuffle mask index if the referenced input element is ↵Simon Pilgrim2017-08-081-0/+36
| | | | | | | | | | UNDEF Fixes one of the cases in PR34041. Differential Revision: https://reviews.llvm.org/D36393 llvm-svn: 310344
* [x86] revert r310208 to investigate test-suite failures (PR34105 / PR34097) Sanjay Patel2017-08-071-1/+1
| | | | llvm-svn: 310264
* [DAG] Extend visitSCALAR_TO_VECTOR optimization to truncated vector.Nirav Dave2017-08-071-12/+35
| | | | | | | | | | | | | | | | Relanding after case to insert explicit truncation as necessary. Allow SCALAR_TO_VECTOR of EXTRACT_VECTOR_ELT to reduce to EXTRACT_SUBVECTOR of vector shuffle when output is smaller. Marginally improves vector shuffle computations. Reviewers: efriedma, RKSimon, spatel Subscribers: javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D35566 llvm-svn: 310256
* [x86] use more shift or LEA for select-of-constantsSanjay Patel2017-08-061-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We can convert any select-of-constants to math ops: http://rise4fun.com/Alive/d7d For this patch, I'm enhancing an existing x86 transform that uses fake multiplies (they always become shl/lea) to avoid cmov or branching. The current code misses cases where we have a negative constant and a positive constant, so this is just trying to plug that hole. The DAGCombiner diff prevents us from hitting a terrible inefficiency: we can start with a select in IR, create a select DAG node, convert it into a sext, convert it back into a select, and then lower it to sext machine code. Some notes about the test diffs: 1. 2010-08-04-MaskedSignedCompare.ll - We were creating control flow that didn't exist in the IR. 2. memcmp.ll - Choose -1 or 1 is the case that got me looking at this again. I think we could avoid the push/pop in some cases if we used 'movzbl %al' instead of an xor on a different reg? That's a post-DAG problem though. 3. mul-constant-result.ll - The trade-off between sbb+not vs. setne+neg could be addressed if that's a regression, but I think those would always be nearly equivalent. 4. pr22338.ll and sext-i1.ll - These tests have undef operands, so I don't think we actually care about these diffs. 5. sbb.ll - This shows a win for what I think is a common case: choose -1 or 0. 6. select.ll - There's another borderline case here: cmp+sbb+or vs. test+set+lea? Also, sbb+not vs. setae+neg shows up again. 7. select_const.ll - These are motivating cases for the enhancement; replace cmov with cheaper ops. Assembly differences between movzbl and xor to avoid a partial reg stall are caused later by the X86 Fixup SetCC pass. Differential Revision: https://reviews.llvm.org/D35340 llvm-svn: 310208
* Revert r310058, it caused PR34073.Nico Weber2017-08-041-47/+2
| | | | llvm-svn: 310118
* [DAGCombiner] Extending pattern detection for vector shuffle.Simon Pilgrim2017-08-041-2/+47
| | | | | | | | | | If all the operands of a BUILD_VECTOR extract elements from same vector then split the vector efficiently based on the maximum vector access index. Committed on behalf of @jbhateja (Jatin Bhateja) Differential Revision: https://reviews.llvm.org/D35788 llvm-svn: 310058
* [DAG] Allow merging of stores of vector loadsNirav Dave2017-08-031-6/+0
| | | | | | | | | | | | | Remove restriction disallowing merging of stores vector loads into larger store of larger vector load. Reviewers: RKSimon, efriedma, spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D36158 llvm-svn: 309951
* [DAG] Improve candidate pruning in store merge failure case. NFCINirav Dave2017-08-021-20/+61
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | During store merge we construct a sorted list of consecutive store candidates and consider subsequences for merging into a single store. For each subsequence we check if the stored value type is legal the merged store would have valid and fast and if the constructed value to be stored is valid. The only properties that affect this check between subsequences is the size of the subsequence, the alignment of the first store, the alignment of the stored load value (when merging stores-of-loads), and whether the merged value is a constant zero. If we do not find a viable mergeable subsequence starting from the first store of length N, we know that a subsequence starting at a later store of length N will also fail unless the new store's alignment, the new load's alignment (if we're merging store-of-loads), or we've dropped stores of nonzero value and could construct a merged stores of zero (for merging constants). As a result if we fail to find a valid subsequence starting from the first store we can safely skip considering subsequences that start with subsequent stores unless one of the above properties is true. This significantly (2x) improves compile time in some pathological cases. Reviewers: RKSimon, efriedma, zvi, spatel, waltl Subscribers: grandinj, llvm-commits Differential Revision: https://reviews.llvm.org/D35901 llvm-svn: 309830
* [DAG] Refactor store merge subexpressions. NFC.Nirav Dave2017-08-021-23/+28
| | | | | | Distribute various expressions across ifs. llvm-svn: 309777
* DAG: Undo and->or combine with FrameIndexesMatt Arsenault2017-08-021-0/+9
| | | | | | | | | | | | | | This pattern shows up when lowering byval copies on AMDGPU. The byval object access is split into 4-byte chunks, adding a constant offset to the FixedStack base. When some of the offsets turn into ors, this prevents combining the constant offsets. This makes it not apparent that the object is there when matching addressing modes, so it ends up using a scratch wave offset relative access and the lengthy frame index expansion for that. llvm-svn: 309775
* [DAG] Factor out common expressions. NFC.Nirav Dave2017-08-011-19/+21
| | | | llvm-svn: 309740
* Pull out VectorNumElements value. NFC.Nirav Dave2017-08-011-13/+9
| | | | llvm-svn: 309719
* Revert "[DAG] Extend visitSCALAR_TO_VECTOR optimization to truncated vector."Nirav Dave2017-08-011-26/+11
| | | | | | | This reverts commit r309680 which appears to be raising an assertion in the test-suite. llvm-svn: 309717
* [DAG] Convert extload check to equivalent type check. NFC.Nirav Dave2017-08-011-5/+10
| | | | | | Replace check with check that consuming store has the same type. llvm-svn: 309708
* [DAG] Move extload check in store merge. NFC.Nirav Dave2017-08-011-5/+3
| | | | | | Move candidate check from later check to initial candidate check. llvm-svn: 309698
* [DAG] Extend visitSCALAR_TO_VECTOR optimization to truncated vector.Nirav Dave2017-08-011-11/+26
| | | | | | | | | | | | | | | Summary: Allow SCALAR_TO_VECTOR of EXTRACT_VECTOR_ELT to reduce to EXTRACT_SUBVECTOR of vector shuffle when output is smaller. Marginally improves vector shuffle computations. Reviewers: efriedma, RKSimon, spatel Subscribers: javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D35566 llvm-svn: 309680
* DAGCombiner: Extend reduceBuildVecToTrunc to handle non-zero offsetZvi Rackover2017-07-261-12/+32
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: Adding support for combining power2-strided build_vector's where the first build_vectori's operand is extracted from a non-zero index. Example: v4i32 build_vector((extract_elt V, 1), (extract_elt V, 3), (extract_elt V, 5), (extract_elt V, 7)) --> v4i32 truncate (bitcast (shuffle<1,u,3,u,5,u,7,u> V, u) to v4i64) Reviewers: delena, RKSimon, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D35700 llvm-svn: 309108
* [DAG] Move DAGCombiner::GetDemandedBits to SelectionDAGSimon Pilgrim2017-07-251-59/+4
| | | | | | | | This patch moves the DAGCombiner::GetDemandedBits function to SelectionDAG::GetDemandedBits as a first step towards making it easier for targets to get to the source of any demanded bits without the limitations of SimplifyDemandedBits. Differential Revision: https://reviews.llvm.org/D35841 llvm-svn: 308983
* Fix endianness bug in DAGCombiner::visitTRUNCATE and visitEXTRACT_VECTOR_ELTFrancois Pichet2017-07-251-4/+7
| | | | | | | | | | | | | | | | Summary: Do not assume little endian architecture in DAGCombiner::visitTRUNCATE and DAGCombiner::visitEXTRACT_VECTOR_ELT. PR33682 Reviewers: hfinkel, sdardis, RKSimon Reviewed By: sdardis, RKSimon Subscribers: uabelho, RKSimon, sdardis, llvm-commits Differential Revision: https://reviews.llvm.org/D34990 llvm-svn: 308960
* [DAG] Fix typo preventing some stores merges to truncated stores.Nirav Dave2017-07-231-4/+4
| | | | | | | | | | | | | | | Check the actual memory type stored and not the extended value size when considering if truncated store merge is worthwhile. Reviewers: efriedma, RKSimon, spatel, jyknight Reviewed By: efriedma Subscribers: llvm-commits, nhaehnle Differential Revision: https://reviews.llvm.org/D35623 llvm-svn: 308833
* [DAGCombiner] Update comment. NFCXin Tong2017-07-211-1/+1
| | | | llvm-svn: 308772
* [DAG] Commit missed nit cleanup from r308617. NFC.Nirav Dave2017-07-201-1/+1
| | | | llvm-svn: 308645
* [DAG] Handle missing transform in fold of value extension case.Nirav Dave2017-07-201-0/+14
| | | | | | | | | | | | | | | Summary: When pushing an extension of a constant bitwise operator on a load into the load, change other uses of the load value if they exist to prevent the old load from persisting. Reviewers: spatel, RKSimon, efriedma Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D35030 llvm-svn: 308618
* [DAG] Optimize away degenerate INSERT_VECTOR_ELT nodes.Nirav Dave2017-07-201-0/+6
| | | | | | | | | | | | | | | | | Summary: Add missing vector write of vector read reduction, i.e.: (insert_vector_elt x (extract_vector_elt x idx) idx) to x Reviewers: spatel, RKSimon, efriedma Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D35563 llvm-svn: 308617
* [DAGCombiner] Match ISD::SRL non-uniform constant vectors patterns using ↵Simon Pilgrim2017-07-201-13/+26
| | | | | | | | predicates. Use predicate matchers introduced in D35492 to match more ISD::SRL constant folds llvm-svn: 308602
* Remove trailing whitespace. NFCI.Simon Pilgrim2017-07-201-1/+1
| | | | llvm-svn: 308601
* [DAGCombiner] Match ISD::SRA non-uniform constant vectors patterns using ↵Simon Pilgrim2017-07-201-13/+28
| | | | | | | | predicates. Use predicate matchers introduced in D35492 to match more ISD::SRA constant folds llvm-svn: 308600
* [DAGCombiner] Match non-uniform constant vectors using predicates.Simon Pilgrim2017-07-201-28/+81
| | | | | | | | | | | | Most combines currently recognise scalar and splat-vector constants, but not non-uniform vector constants. This patch introduces a matching mechanism that uses predicates to check against BUILD_VECTOR of ConstantSDNode, as well as scalar ConstantSDNode cases. I've changed a couple of predicates to demonstrate - the combine-shl changes add currently unsupported cases, while the MatchRotate replaces an existing mechanism. Differential Revision: https://reviews.llvm.org/D35492 llvm-svn: 308598
* {DAGCombine] Convert (Val & Mask) == Mask to Mask.isSubsetof(Val). NFCI.Simon Pilgrim2017-07-191-1/+1
| | | | llvm-svn: 308460
* [DAG] Improve Aliasing of operations to static allocaNirav Dave2017-07-181-6/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Re-recommiting after landing DAG extension-crash fix. Recommiting after adding check to avoid miscomputing alias information on addresses of the same base but different subindices. Memory accesses offset from frame indices may alias, e.g., we may merge write from function arguments passed on the stack when they are contiguous. As a result, when checking aliasing, we consider the underlying frame index's offset from the stack pointer. Static allocs are realized as stack objects in SelectionDAG, but its offset is not set until post-DAG causing DAGCombiner's alias check to consider access to static allocas to frequently alias. Modify isAlias to consider access between static allocas and access from other frame objects to be considered aliasing. Many test changes are included here. Most are fixes for tests which indirectly relied on our aliasing ability and needed to be modified to preserve their original intent. The remaining tests have minor improvements due to relaxed ordering. The exception is CodeGen/X86/2011-10-19-widen_vselect.ll which has a minor degradation dispite though the pre-legalized DAG is improved. Reviewers: rnk, mkuper, jonpa, hfinkel, uweigand Reviewed By: rnk Subscribers: sdardis, nemanjai, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D33345 llvm-svn: 308350
* [DAG] Reverse node replacement in extension operation. NFCI.Nirav Dave2017-07-181-12/+20
| | | | | | | | Reorder replacements to be user first in preparation for multi-level folding to premptively avoid inadvertantly deleting later nodes from sharing found from replacement. llvm-svn: 308348
* [DAG] Avoid deleting nodes before combining them.Nirav Dave2017-07-181-7/+26
| | | | | | | | | | | | | | | | | | When replacing a node and it's operand, replacing the operand node may cause the deletion of the original node leading to an assertion failure. Case around these replacements to avoid this without relying on inspecting the DELETED_NODE opcode in various extend dagcombiner cases. Fixes PR32515. Reviewers: dbabokin, RKSimon, davide, chandlerc Subscribers: chandlerc, llvm-commits Differential Revision: https://reviews.llvm.org/D34095 llvm-svn: 308330
* [DAG] Allow base element type of store merge type to also be a vector.Nirav Dave2017-07-181-1/+6
| | | | | | Correctly calculate merged vector size if MemVT is already a vector. llvm-svn: 308312
* [DAGCombine] Fix issue with out of bound constant rotation (PR33828)Simon Pilgrim2017-07-181-1/+10
| | | | | | Take the modulo of rotations by a constant greater than or equal to the bit-width llvm-svn: 308302
* Revert r308025 due to uncovering a crash in SelectionDAG. This is filedChandler Carruth2017-07-181-16/+6
| | | | | | | | | with a minimal test case in http://llvm.org/PR33833. Original commit message: Improve Aliasing of operations to static alloca llvm-svn: 308271
* [DAGCombiner] Recognise vector rotations with non-splat constantsAndrew Zhogin2017-07-161-13/+21
| | | | | | | | Fixes PR33691. Differential revision: https://reviews.llvm.org/D35381 llvm-svn: 308150
* Strip trailing whitespace. NFCISimon Pilgrim2017-07-151-1/+1
| | | | llvm-svn: 308108
* Improve Aliasing of operations to static allocaNirav Dave2017-07-141-6/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Recommiting after adding check to avoid miscomputing alias information on addresses of the same base but different subindices. Memory accesses offset from frame indices may alias, e.g., we may merge write from function arguments passed on the stack when they are contiguous. As a result, when checking aliasing, we consider the underlying frame index's offset from the stack pointer. Static allocs are realized as stack objects in SelectionDAG, but its offset is not set until post-DAG causing DAGCombiner's alias check to consider access to static allocas to frequently alias. Modify isAlias to consider access between static allocas and access from other frame objects to be considered aliasing. Many test changes are included here. Most are fixes for tests which indirectly relied on our aliasing ability and needed to be modified to preserve their original intent. The remaining tests have minor improvements due to relaxed ordering. The exception is CodeGen/X86/2011-10-19-widen_vselect.ll which has a minor degradation dispite though the pre-legalized DAG is improved. Reviewers: rnk, mkuper, jonpa, hfinkel, uweigand Reviewed By: rnk Subscribers: sdardis, nemanjai, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D33345 llvm-svn: 308025
* [DAGCombiner] Fix issue with rotate combines asserting if the constant value ↵Simon Pilgrim2017-07-131-15/+18
| | | | | | types differ from the result type. llvm-svn: 307900
* Use isNullConstantOrNullSplatConstant helper. NFCI.Simon Pilgrim2017-07-131-3/+2
| | | | llvm-svn: 307895
* Revert "[DAG] Improve Aliasing of operations to static alloca"Matthias Braun2017-07-101-14/+6
| | | | | | | | | Reverting as it breaks tramp3d-v4 in the llvm test-suite. I added some comments to https://reviews.llvm.org/D33345 about it. This reverts commit r307546. llvm-svn: 307589
* Add DAG argument to canMergeStoresTo NFC.Nirav Dave2017-07-101-7/+9
| | | | llvm-svn: 307583
* [DAG] Improve Aliasing of operations to static allocaNirav Dave2017-07-101-6/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Memory accesses offset from frame indices may alias, e.g., we may merge write from function arguments passed on the stack when they are contiguous. As a result, when checking aliasing, we consider the underlying frame index's offset from the stack pointer. Static allocs are realized as stack objects in SelectionDAG, but its offset is not set until post-DAG causing DAGCombiner's alias check to consider access to static allocas to frequently alias. Modify isAlias to consider access between static allocas and access from other frame objects to be considered aliasing. Many test changes are included here. Most are fixes for tests which indirectly relied on our aliasing ability and needed to be modified to preserve their original intent. The remaining tests have minor improvements due to relaxed ordering. The exception is CodeGen/X86/2011-10-19-widen_vselect.ll which has a minor degradation dispite though the pre-legalized DAG is improved. Reviewers: rnk, mkuper, jonpa, hfinkel, uweigand Reviewed By: rnk Subscribers: sdardis, nemanjai, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D33345 llvm-svn: 307546
OpenPOWER on IntegriCloud