summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG
Commit message (Collapse)AuthorAgeFilesLines
...
* [DebugInfo] Don't turn dbg.declare into DBG_VALUE for static allocasReid Kleckner2017-08-011-0/+7
| | | | | | | | | | | | | | | | Summary: We already have information about static alloca stack locations in our side table. Emitting instructions for them is inefficient, and it only happens when the address of the alloca has been materialized within the current block, which isn't often. Reviewers: aprantl, probinson, dblaikie Subscribers: jfb, dschuff, sbc100, jgravelle-google, hiraditya, llvm-commits, aheejin Differential Revision: https://reviews.llvm.org/D36117 llvm-svn: 309729
* Pull out VectorNumElements value. NFC.Nirav Dave2017-08-011-13/+9
| | | | llvm-svn: 309719
* Revert "[DAG] Extend visitSCALAR_TO_VECTOR optimization to truncated vector."Nirav Dave2017-08-011-26/+11
| | | | | | | This reverts commit r309680 which appears to be raising an assertion in the test-suite. llvm-svn: 309717
* [DAG] Convert extload check to equivalent type check. NFC.Nirav Dave2017-08-011-5/+10
| | | | | | Replace check with check that consuming store has the same type. llvm-svn: 309708
* [DAG] Move extload check in store merge. NFC.Nirav Dave2017-08-011-5/+3
| | | | | | Move candidate check from later check to initial candidate check. llvm-svn: 309698
* [DAG] Extend visitSCALAR_TO_VECTOR optimization to truncated vector.Nirav Dave2017-08-011-11/+26
| | | | | | | | | | | | | | | Summary: Allow SCALAR_TO_VECTOR of EXTRACT_VECTOR_ELT to reduce to EXTRACT_SUBVECTOR of vector shuffle when output is smaller. Marginally improves vector shuffle computations. Reviewers: efriedma, RKSimon, spatel Subscribers: javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D35566 llvm-svn: 309680
* [ScheduleDAG] Don't schedule node with physical register interferenceEli Friedman2017-08-011-25/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | https://reviews.llvm.org/D31536 didn't really solve the problem it was trying to solve; it got rid of the assertion failure, but we were still scheduling the DAG incorrectly (mixing together instructions from different calls), leading to a MachineVerifier failure. In order to schedule the DAG correctly, we have to make sure we don't schedule a node which should be blocked by an interference. Fix ScheduleDAGRRList::PickNodeToScheduleBottomUp so it doesn't pick a node like that. The added call to FindAvailableNode() is the key change here; this makes sure we don't try to schedule a call while we're in the middle of scheduling a different call. I'm not sure this is the right approach; in particular, I'm not sure how to prove we don't end up with an infinite loop of repeatedly backtracking. This also reverts the code change from D31536. It doesn't do anything useful: we should never schedule an ADJCALLSTACKDOWN unless we've already scheduled the corresponding ADJCALLSTACKUP. Differential Revision: https://reviews.llvm.org/D33818 llvm-svn: 309642
* [SelectionDAG][mips] Fix PR33883Simon Dardis2017-07-311-15/+24
| | | | | | | | | | | | | | | PR33883 shows that calls to intrinsic functions should not have their vector arguments or returns subject to ABI changes required by the target. This resolves PR33883. Thanks to Alex Crichton for reporting the issue! Reviewers: zoran.jovanovic, atanasyan Differential Revision: https://reviews.llvm.org/D35765 llvm-svn: 309561
* [SelectionDAG][X86] CombineBT - more aggressively determine demanded bitsSimon Pilgrim2017-07-291-0/+12
| | | | | | | | | | | | This patch is in 2 parts: 1 - replace combineBT's use of SimplifyDemandedBits (hasOneUse only) with SelectionDAG::GetDemandedBits to more aggressively determine the lower bits used by BT. 2 - update SelectionDAG::GetDemandedBits to support ANY_EXTEND - if the demanded bits are only in the non-extended portion, then peek through and demand from the source value and then ANY_EXTEND that if we found a match. Differential Revision: https://reviews.llvm.org/D35896 llvm-svn: 309486
* Remove the unused offset from DBG_VALUE (NFC)Adrian Prantl2017-07-283-6/+8
| | | | | | | Followup to r309426. rdar://problem/33580047 llvm-svn: 309450
* Remove the unused dbg.value offset from SelectionDAG (NFC)Adrian Prantl2017-07-285-63/+43
| | | | | | | Followup to r309426. rdar://problem/33580047 llvm-svn: 309436
* Remove the obsolete offset parameter from @llvm.dbg.valueAdrian Prantl2017-07-282-9/+9
| | | | | | | | | | | | There is no situation where this rarely-used argument cannot be substituted with a DIExpression and removing it allows us to simplify the DWARF backend. Note that this patch does not yet remove any of the newly dead code. rdar://problem/33580047 Differential Revision: https://reviews.llvm.org/D35951 llvm-svn: 309426
* [SelectionDAG] Improve DAGTypeLegalizer::convertMask assertion (PR33960)Simon Pilgrim2017-07-271-12/+9
| | | | | | Improve DAGTypeLegalizer::convertMask's isSETCCorConvertedSETCC assertion to properly check for any mixture of SETCC or BUILD_VECTOR of constants, or a logical mask op of them. llvm-svn: 309302
* [SelectionDAG] Avoid repeated calls to getNumOperands in for loop. NFCI.Simon Pilgrim2017-07-271-1/+1
| | | | llvm-svn: 309283
* [SelectionDAG] Tidyup mask creation. NFCI.Simon Pilgrim2017-07-271-6/+3
| | | | | | Assign all concat elements to UNDEF and then just replace the first element, instead of copying everything individually. llvm-svn: 309277
* Change CallLoweringInfo::CS to be an ImmutableCallSite instead of a pointer. ↵Peter Collingbourne2017-07-261-3/+2
| | | | | | | | NFCI. This was a use-after-free waiting to happen. llvm-svn: 309159
* DAGCombiner: Extend reduceBuildVecToTrunc to handle non-zero offsetZvi Rackover2017-07-261-12/+32
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: Adding support for combining power2-strided build_vector's where the first build_vectori's operand is extracted from a non-zero index. Example: v4i32 build_vector((extract_elt V, 1), (extract_elt V, 3), (extract_elt V, 5), (extract_elt V, 7)) --> v4i32 truncate (bitcast (shuffle<1,u,3,u,5,u,7,u> V, u) to v4i64) Reviewers: delena, RKSimon, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D35700 llvm-svn: 309108
* [DAG] Move DAGCombiner::GetDemandedBits to SelectionDAGSimon Pilgrim2017-07-252-62/+58
| | | | | | | | This patch moves the DAGCombiner::GetDemandedBits function to SelectionDAG::GetDemandedBits as a first step towards making it easier for targets to get to the source of any demanded bits without the limitations of SimplifyDemandedBits. Differential Revision: https://reviews.llvm.org/D35841 llvm-svn: 308983
* Fix endianness bug in DAGCombiner::visitTRUNCATE and visitEXTRACT_VECTOR_ELTFrancois Pichet2017-07-251-4/+7
| | | | | | | | | | | | | | | | Summary: Do not assume little endian architecture in DAGCombiner::visitTRUNCATE and DAGCombiner::visitEXTRACT_VECTOR_ELT. PR33682 Reviewers: hfinkel, sdardis, RKSimon Reviewed By: sdardis, RKSimon Subscribers: uabelho, RKSimon, sdardis, llvm-commits Differential Revision: https://reviews.llvm.org/D34990 llvm-svn: 308960
* [DAG] Fix typo preventing some stores merges to truncated stores.Nirav Dave2017-07-231-4/+4
| | | | | | | | | | | | | | | Check the actual memory type stored and not the extended value size when considering if truncated store merge is worthwhile. Reviewers: efriedma, RKSimon, spatel, jyknight Reviewed By: efriedma Subscribers: llvm-commits, nhaehnle Differential Revision: https://reviews.llvm.org/D35623 llvm-svn: 308833
* [DAGCombiner] Update comment. NFCXin Tong2017-07-211-1/+1
| | | | llvm-svn: 308772
* [DAG] Commit missed nit cleanup from r308617. NFC.Nirav Dave2017-07-201-1/+1
| | | | llvm-svn: 308645
* [DAG] Handle missing transform in fold of value extension case.Nirav Dave2017-07-201-0/+14
| | | | | | | | | | | | | | | Summary: When pushing an extension of a constant bitwise operator on a load into the load, change other uses of the load value if they exist to prevent the old load from persisting. Reviewers: spatel, RKSimon, efriedma Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D35030 llvm-svn: 308618
* [DAG] Optimize away degenerate INSERT_VECTOR_ELT nodes.Nirav Dave2017-07-201-0/+6
| | | | | | | | | | | | | | | | | Summary: Add missing vector write of vector read reduction, i.e.: (insert_vector_elt x (extract_vector_elt x idx) idx) to x Reviewers: spatel, RKSimon, efriedma Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D35563 llvm-svn: 308617
* [DAGCombiner] Match ISD::SRL non-uniform constant vectors patterns using ↵Simon Pilgrim2017-07-201-13/+26
| | | | | | | | predicates. Use predicate matchers introduced in D35492 to match more ISD::SRL constant folds llvm-svn: 308602
* Remove trailing whitespace. NFCI.Simon Pilgrim2017-07-201-1/+1
| | | | llvm-svn: 308601
* [DAGCombiner] Match ISD::SRA non-uniform constant vectors patterns using ↵Simon Pilgrim2017-07-201-13/+28
| | | | | | | | predicates. Use predicate matchers introduced in D35492 to match more ISD::SRA constant folds llvm-svn: 308600
* [DAGCombiner] Match non-uniform constant vectors using predicates.Simon Pilgrim2017-07-201-28/+81
| | | | | | | | | | | | Most combines currently recognise scalar and splat-vector constants, but not non-uniform vector constants. This patch introduces a matching mechanism that uses predicates to check against BUILD_VECTOR of ConstantSDNode, as well as scalar ConstantSDNode cases. I've changed a couple of predicates to demonstrate - the combine-shl changes add currently unsupported cases, while the MatchRotate replaces an existing mechanism. Differential Revision: https://reviews.llvm.org/D35492 llvm-svn: 308598
* {DAGCombine] Convert (Val & Mask) == Mask to Mask.isSubsetof(Val). NFCI.Simon Pilgrim2017-07-191-1/+1
| | | | llvm-svn: 308460
* [DAG] Improve Aliasing of operations to static allocaNirav Dave2017-07-181-6/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Re-recommiting after landing DAG extension-crash fix. Recommiting after adding check to avoid miscomputing alias information on addresses of the same base but different subindices. Memory accesses offset from frame indices may alias, e.g., we may merge write from function arguments passed on the stack when they are contiguous. As a result, when checking aliasing, we consider the underlying frame index's offset from the stack pointer. Static allocs are realized as stack objects in SelectionDAG, but its offset is not set until post-DAG causing DAGCombiner's alias check to consider access to static allocas to frequently alias. Modify isAlias to consider access between static allocas and access from other frame objects to be considered aliasing. Many test changes are included here. Most are fixes for tests which indirectly relied on our aliasing ability and needed to be modified to preserve their original intent. The remaining tests have minor improvements due to relaxed ordering. The exception is CodeGen/X86/2011-10-19-widen_vselect.ll which has a minor degradation dispite though the pre-legalized DAG is improved. Reviewers: rnk, mkuper, jonpa, hfinkel, uweigand Reviewed By: rnk Subscribers: sdardis, nemanjai, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D33345 llvm-svn: 308350
* [DAG] Reverse node replacement in extension operation. NFCI.Nirav Dave2017-07-181-12/+20
| | | | | | | | Reorder replacements to be user first in preparation for multi-level folding to premptively avoid inadvertantly deleting later nodes from sharing found from replacement. llvm-svn: 308348
* [DAG] Avoid deleting nodes before combining them.Nirav Dave2017-07-181-7/+26
| | | | | | | | | | | | | | | | | | When replacing a node and it's operand, replacing the operand node may cause the deletion of the original node leading to an assertion failure. Case around these replacements to avoid this without relying on inspecting the DELETED_NODE opcode in various extend dagcombiner cases. Fixes PR32515. Reviewers: dbabokin, RKSimon, davide, chandlerc Subscribers: chandlerc, llvm-commits Differential Revision: https://reviews.llvm.org/D34095 llvm-svn: 308330
* [DAG] Allow base element type of store merge type to also be a vector.Nirav Dave2017-07-181-1/+6
| | | | | | Correctly calculate merged vector size if MemVT is already a vector. llvm-svn: 308312
* [DAGCombine] Fix issue with out of bound constant rotation (PR33828)Simon Pilgrim2017-07-181-1/+10
| | | | | | Take the modulo of rotations by a constant greater than or equal to the bit-width llvm-svn: 308302
* Revert r308025 due to uncovering a crash in SelectionDAG. This is filedChandler Carruth2017-07-181-16/+6
| | | | | | | | | with a minimal test case in http://llvm.org/PR33833. Original commit message: Improve Aliasing of operations to static alloca llvm-svn: 308271
* [DAGCombiner] Recognise vector rotations with non-splat constantsAndrew Zhogin2017-07-161-13/+21
| | | | | | | | Fixes PR33691. Differential revision: https://reviews.llvm.org/D35381 llvm-svn: 308150
* Strip trailing whitespace. NFCISimon Pilgrim2017-07-151-1/+1
| | | | llvm-svn: 308108
* Improve Aliasing of operations to static allocaNirav Dave2017-07-141-6/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Recommiting after adding check to avoid miscomputing alias information on addresses of the same base but different subindices. Memory accesses offset from frame indices may alias, e.g., we may merge write from function arguments passed on the stack when they are contiguous. As a result, when checking aliasing, we consider the underlying frame index's offset from the stack pointer. Static allocs are realized as stack objects in SelectionDAG, but its offset is not set until post-DAG causing DAGCombiner's alias check to consider access to static allocas to frequently alias. Modify isAlias to consider access between static allocas and access from other frame objects to be considered aliasing. Many test changes are included here. Most are fixes for tests which indirectly relied on our aliasing ability and needed to be modified to preserve their original intent. The remaining tests have minor improvements due to relaxed ordering. The exception is CodeGen/X86/2011-10-19-widen_vselect.ll which has a minor degradation dispite though the pre-legalized DAG is improved. Reviewers: rnk, mkuper, jonpa, hfinkel, uweigand Reviewed By: rnk Subscribers: sdardis, nemanjai, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D33345 llvm-svn: 308025
* Reland "[mips] Fix multiprecision arithmetic."Simon Dardis2017-07-131-4/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | For multiprecision arithmetic on MIPS, rather than using ISD::ADDE / ISD::ADDC, get SelectionDAG to break down the operation into ISD::ADDs and ISD::SETCCs. For MIPS, only the DSP ASE has a carry flag, so in the general case it is not useful to directly support ISD::{ADDE, ADDC, SUBE, SUBC} nodes. Also improve the generation code in such cases for targets with TargetLoweringBase::ZeroOrOneBooleanContent by directly using the result of the comparison node rather than using it in selects. Similarly for ISD::SUBE / ISD::SUBC. Address optimization breakage by moving the generation of MIPS specific integer multiply-accumulate nodes to before legalization. This revolves PR32713 and PR33424. Thanks to Simonas Kazlauskas and Pirama Arumuga Nainar for reporting the issue! Reviewers: slthakur Differential Revision: https://reviews.llvm.org/D33494 The previous version of this patch was too aggressive in producing fused integer multiple-addition instructions. llvm-svn: 307906
* [DAGCombiner] Fix issue with rotate combines asserting if the constant value ↵Simon Pilgrim2017-07-131-15/+18
| | | | | | types differ from the result type. llvm-svn: 307900
* Use isNullConstantOrNullSplatConstant helper. NFCI.Simon Pilgrim2017-07-131-3/+2
| | | | llvm-svn: 307895
* [TargetLowering] Add hook for adding target MMO flags when doing ISel.Geoff Berry2017-07-131-0/+2
| | | | | | | | | | | | | Summary: Add TargetLowering hook getMMOFlags() to add target specific MMO flags to load/store instructions created by ISel. Reviewers: bogner, hfinkel, qcolombet, MatzeB Subscribers: mcrosier, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D34962 llvm-svn: 307879
* Add element atomic memset intrinsicDaniel Neilson2017-07-121-0/+39
| | | | | | | | | | | | | | Summary: Continuing the work from https://reviews.llvm.org/D33240, this change introduces an element unordered-atomic memset intrinsic. This intrinsic is essentially memset with the implementation requirement that all stores used for the assignment are done with unordered-atomic stores of a given element size. Reviewers: eli.friedman, reames, mkazantsev, skatkov Reviewed By: reames Subscribers: jfb, dschuff, sbc100, jgravelle-google, aheejin, efriedma, llvm-commits Differential Revision: https://reviews.llvm.org/D34885 llvm-svn: 307854
* Add element atomic memmove intrinsicDaniel Neilson2017-07-121-0/+38
| | | | | | | | | | | | | | Summary: Continuing the work from https://reviews.llvm.org/D33240, this change introduces an element unordered-atomic memmove intrinsic. This intrinsic is essentially memmove with the implementation requirement that all loads/stores used for the copy are done with unordered-atomic loads/stores of a given element size. Reviewers: eli.friedman, reames, mkazantsev, skatkov Reviewed By: reames Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34884 llvm-svn: 307796
* Enhance synchscope representationKonstantin Zhuravlyov2017-07-112-13/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | OpenCL 2.0 introduces the notion of memory scopes in atomic operations to global and local memory. These scopes restrict how synchronization is achieved, which can result in improved performance. This change extends existing notion of synchronization scopes in LLVM to support arbitrary scopes expressed as target-specific strings, in addition to the already defined scopes (single thread, system). The LLVM IR and MIR syntax for expressing synchronization scopes has changed to use *syncscope("<scope>")*, where <scope> can be "singlethread" (this replaces *singlethread* keyword), or a target-specific name. As before, if the scope is not specified, it defaults to CrossThread/System scope. Implementation details: - Mapping from synchronization scope name/string to synchronization scope id is stored in LLVM context; - CrossThread/System and SingleThread scopes are pre-defined to efficiently check for known scopes without comparing strings; - Synchronization scope names are stored in SYNC_SCOPE_NAMES_BLOCK in the bitcode. Differential Revision: https://reviews.llvm.org/D21723 llvm-svn: 307722
* Revert "[DAG] Improve Aliasing of operations to static alloca"Matthias Braun2017-07-101-14/+6
| | | | | | | | | Reverting as it breaks tramp3d-v4 in the llvm test-suite. I added some comments to https://reviews.llvm.org/D33345 about it. This reverts commit r307546. llvm-svn: 307589
* Add DAG argument to canMergeStoresTo NFC.Nirav Dave2017-07-101-7/+9
| | | | llvm-svn: 307583
* [DAG] Improve Aliasing of operations to static allocaNirav Dave2017-07-101-6/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Memory accesses offset from frame indices may alias, e.g., we may merge write from function arguments passed on the stack when they are contiguous. As a result, when checking aliasing, we consider the underlying frame index's offset from the stack pointer. Static allocs are realized as stack objects in SelectionDAG, but its offset is not set until post-DAG causing DAGCombiner's alias check to consider access to static allocas to frequently alias. Modify isAlias to consider access between static allocas and access from other frame objects to be considered aliasing. Many test changes are included here. Most are fixes for tests which indirectly relied on our aliasing ability and needed to be modified to preserve their original intent. The remaining tests have minor improvements due to relaxed ordering. The exception is CodeGen/X86/2011-10-19-widen_vselect.ll which has a minor degradation dispite though the pre-legalized DAG is improved. Reviewers: rnk, mkuper, jonpa, hfinkel, uweigand Reviewed By: rnk Subscribers: sdardis, nemanjai, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D33345 llvm-svn: 307546
* fix typos in comments and error messages; NFCHiroshi Inoue2017-07-102-2/+2
| | | | llvm-svn: 307533
* [X86] Relax an assertion when legalizing vector types.Davide Italiano2017-07-091-0/+4
| | | | | | | | | | | | | | WidenVSELECTAndMask can fold (and it folds in this case) so we get a BUILD_VECTOR of constants as mask. convertMask() seems to work fine when the input is a vector of constants, and we still need to call it to extend/add elements at the end. but the current code just asserts on anything but a SETCC or AND/OR/XOR of 2xSETCC. This change was discussed briefly with Simon Pilgrim, who also suggests we might consider dropping this assertion in the future. Fixes PR33715. llvm-svn: 307508
OpenPOWER on IntegriCloud