summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG
Commit message (Collapse)AuthorAgeFilesLines
* [DAGCombine] Move AND nodes to multiple load leavesSam Parker2017-12-141-0/+124
| | | | | | | | | | | | | | | | | | | | | | | | Recommitting rL319773, which was reverted due to a recursive issue causing timeouts. This happened because I failed to check whether the discovered loads could be narrowed further. In the case of a tree with one or more narrow loads, that could not be further narrowed, as well as a node that would need masking, an AND could be introduced which could then be visited and recombined again with the same load. This could again create the masking load, with would be combined again... We now check that the load can be narrowed so that this process stops. Original commit message: Search from AND nodes to find whether they can be propagated back to loads, so that the AND and load can be combined into a narrow load. We search through OR, XOR and other AND nodes and all bar one of the leaves are required to be loads or constants. The exception node then needs to be masked off meaning that the 'and' isn't removed, but the loads(s) are narrowed still. Differential Revision: https://reviews.llvm.org/D41177 llvm-svn: 320679
* [SelectionDAG][X86] Improve legalization of v32i1 CONCAT_VECTORS of v16i1 ↵Craig Topper2017-12-141-0/+15
| | | | | | | | | | for AVX512F. A v32i1 CONCAT_VECTORS of v16i1 uses promotion to v32i8 to legalize the v32i1. This results in a bunch of extract_vector_elts and a build_vector that ultimately gets scalarized. This patch checks to see if v16i8 is legal and inserts a any_extend to that so that we can concat v16i8 to v32i8 and avoid creating the extracts. llvm-svn: 320674
* [SelectionDAG] When legalizing the result type of CONCAT_VECTORS, take into ↵Craig Topper2017-12-141-3/+8
| | | | | | | | | | account whether the input type also needs to be promoted. If so go ahead and get the promoted input vector to extract from. Previously, we would create a bunch of any_extends of extract_vector_elts with illegal input type that needs to be promoted. The legalization of those extract_vector_elts would then potentially introduce a truncate. So now we have a bunch of any_extends of truncates. By legalizing both parts together we avoid creating these extra nodes. The test changes seem to be because we were previously combining the build_vector with the any_extend before the any_extend got combined with the truncate. llvm-svn: 320669
* Remove redundant includes from lib/CodeGen.Michael Zolotukhin2017-12-133-6/+0
| | | | llvm-svn: 320619
* [DAG] Promote ADDCARRY / SUBCARRYRoger Ferrer Ibanez2017-12-131-1/+24
| | | | | | | | Add missing case that was not implemented yet. Differential Revision: https://reviews.llvm.org/D38942 llvm-svn: 320567
* [DAGCombiner] protect against an infinite loop between shl <--> mul (PR35579)Sanjay Patel2017-12-111-1/+2
| | | | | | | | | | | | | | | | | | | | | | | At first, I tried to thread the x86 needle and use a target hook (isVectorShiftByScalarCheap()) to disable the transform only for non-splat pow-of-2 constants, but not AVX2, but only some element types, but...it's difficult. Here we just avoid the loop with the x86 vector transform that conflicts with the general DAG combine and preserve all of the existing behavior AFAICT otherwise. Some tests that will probably fail if someone does try to restrict this in a more targeted way for x86-only may be found in: test/CodeGen/X86/combine-mul.ll test/CodeGen/X86/vector-mul.ll test/CodeGen/X86/widen_arith-5.ll This should prevent the infinite looping seen with: https://bugs.llvm.org/show_bug.cgi?id=35579 Differential Revision: https://reviews.llvm.org/D41040 llvm-svn: 320374
* [DAGCombiner] Add combined indexed load to the work listNemanja Ivanovic2017-12-111-0/+1
| | | | | | | | | | This commit is the first part of https://reviews.llvm.org/D40348. In order to allow target combines to be performed on newly combined indexed loads, add them back to the worklist. The remainder of the above patch will be committed in subsequent revisions and will use this. Test cases will be included with those follow-up commits. llvm-svn: 320365
* [ARM] Use ADDCARRY / SUBCARRYRoger Ferrer Ibanez2017-12-111-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a preparatory step for D34515. This change: - makes nodes ISD::ADDCARRY and ISD::SUBCARRY legal for i32 - lowering is done by first converting the boolean value into the carry flag using (_, C) ← (ARMISD::ADDC R, -1) and converted back to an integer value using (R, _) ← (ARMISD::ADDE 0, 0, C). An ARMISD::ADDE between the two operations does the actual addition. - for subtraction, given that ISD::SUBCARRY second result is actually a borrow, we need to invert the value of the second operand and result before and after using ARMISD::SUBE. We need to invert the carry result of ARMISD::SUBE to preserve the semantics. - given that the generic combiner may lower ISD::ADDCARRY and ISD::SUBCARRYinto ISD::UADDO and ISD::USUBO we need to update their lowering as well otherwise i64 operations now would require branches. This implies updating the corresponding test for unsigned. - add new combiner to remove the redundant conversions from/to carry flags to/from boolean values (ARMISD::ADDC (ARMISD::ADDE 0, 0, C), -1) → C - fixes PR34045 - fixes PR34564 - fixes PR35103 Differential Revision: https://reviews.llvm.org/D35192 llvm-svn: 320355
* [RISCV] Support lowering FrameIndexAlex Bradbury2017-12-111-0/+19
| | | | | | | | | | | | | | | | Introduces the AddrFI "addressing mode", which is necessary simply because it's not possible to write a pattern that directly matches a frameindex. Ensure callee-saved registers are accessed relative to the stackpointer. This is necessary as callee-saved register spills are performed before the frame pointer is set. Move HexagonDAGToDAGISel::isOrEquivalentToAdd to SelectionDAGISel, so we can make use of it in the RISC-V backend. Differential Revision: https://reviews.llvm.org/D39848 llvm-svn: 320353
* [DAGCombiner] Support folding (mulhs/u X, 0)->0 for vectors.Craig Topper2017-12-111-0/+16
| | | | | | We should probably also fold (mulhs/u X, 1) for vectors, but that's harder. llvm-svn: 320344
* [DAGCombiner] Reuse existing SDLoc variable instead of creating a new one. NFCCraig Topper2017-12-111-4/+3
| | | | llvm-svn: 320343
* Relax unaligned access assertion when type is byte alignedDylan McKay2017-12-091-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | Summary: This relaxes an assertion inside SelectionDAGBuilder which is overly restrictive on targets which have no concept of alignment (such as AVR). In these architectures, all types are aligned to 8-bits. After this, LLVM will only assert that accesses are aligned on targets which actually require alignment. This patch follows from a discussion on llvm-dev a few months ago http://llvm.1065342.n5.nabble.com/llvm-dev-Unaligned-atomic-load-store-td112815.html Reviewers: bogner, nemanjai, joerg, efriedma Reviewed By: efriedma Subscribers: efriedma, cactus, llvm-commits Differential Revision: https://reviews.llvm.org/D39946 llvm-svn: 320243
* Generalize llvm::replaceDbgDeclare and actually support the use-case thatAdrian Prantl2017-12-081-0/+1
| | | | | | is mentioned in the documentation (inserting a deref before the plus_uconst). llvm-svn: 320203
* [DAGCombiner] eliminate shuffle of insert elementSanjay Patel2017-12-071-0/+81
| | | | | | | | | | | | | | | | | | | I noticed this pattern in D38316 / D38388. We failed to combine a shuffle that is either repeating a scalar insertion at the same position in a vector or translated to a different element index. Like the earlier patch, this could be an instcombine too, but since we opted to make this a DAG transform earlier, I've made this one a DAG patch too. We do not need any legality checking because the new insert is identical to the existing insert except that it may have a different constant insertion operand. The constant insertion test in test/CodeGen/X86/vector-shuffle-combining.ll was the motivation for D38756. Differential Revision: https://reviews.llvm.org/D40209 llvm-svn: 320050
* [SelectionDAG] In SplitVecOp_EXTRACT_VECTOR_ELT, simplify the code that ↵Craig Topper2017-12-071-11/+2
| | | | | | | | makes the type byte addressable. We can just extend the original vector to vXi1 and trust that the legalization process will revisit it. llvm-svn: 320013
* [SelectionDAG] Use TLI.getVectorIdxTy to determine type for an ↵Craig Topper2017-12-071-1/+2
| | | | | | EXTRACT_VECTOR_ELT index instead of hardcoding MVT::i8. llvm-svn: 320012
* [ARM][AArch64][DAG] Reenable post-legalize store mergeNirav Dave2017-12-061-12/+19
| | | | | | | | | | | | | | | | | | | | Reenable post-legalize stores with constant merging computation and corresponding test case. * Properly truncate store merge constants * Disable merging of truncated stores floating points * Ensure merges of constant stores into a single vector are constructed from legal elements. Reviewers: eastig, efriedma Reviewed By: eastig Subscribers: spatel, rengolin, aemerson, javed.absar, kristof.beyls, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D40701 llvm-svn: 319899
* Revert "[DAGCombine] Move AND nodes to multiple load leaves"Vlad Tsyrklevich2017-12-061-123/+0
| | | | | | | This reverts commit r319773. It was causing some buildbots to hang, e.g. http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-android/builds/5589 llvm-svn: 319867
* [SelectionDAG] Don't promote the condition operand of VSELECT when promoting ↵Craig Topper2017-12-051-2/+0
| | | | | | | | the result. The condition operand should be promoted during operand promotion. llvm-svn: 319853
* [SelectionDAG] Don't promote mask operand when widening mstore and mscatter.Craig Topper2017-12-054-37/+23
| | | | | | If the mask needs to be promoted that should occur by the legalizer detecting the mask operand needs to be promoted not as a side effect of another action. llvm-svn: 319852
* [SelectionDAG] Don't promote mask when splitting mstore.Craig Topper2017-12-051-3/+0
| | | | | | If the mask needs to be promoted it should be handled by operand promotion after the result is legalized. llvm-svn: 319851
* [SelectionDAG] Don't promote mask operands of MGATHER and MLOAD to setcc ↵Craig Topper2017-12-051-2/+10
| | | | | | | | result type while widening the result. Just widen the mask. The mask will be promoted if necessary when operands are promoted. It's possible the mask type is legal, but the setcc result type is a different. We shouldn't promote to the setcc result type unless the mask needs to be promoted. llvm-svn: 319850
* [SelectionDAG] Don't call GetWidenedVector for mask operands of MLOAD/MSTORE.Craig Topper2017-12-051-13/+6
| | | | | | GetWidenedVector does't guarantee the widened elements are zero which would break the intended behavior of the operation. llvm-svn: 319849
* Re-commit r319490 "XOR the frame pointer with the stack cookie when ↵Hans Wennborg2017-12-051-4/+9
| | | | | | | | | | | | | | | | | | protecting the stack" The patch originally broke Chromium (crbug.com/791714) due to its failing to specify that the new pseudo instructions clobber EFLAGS. This commit fixes that. > Summary: This strengthens the guard and matches MSVC. > > Reviewers: hans, etienneb > > Subscribers: hiraditya, JDevlieghere, vlad.tsyrklevich, llvm-commits > > Differential Revision: https://reviews.llvm.org/D40622 llvm-svn: 319824
* [SelectionDAG] Remove the code that handles SETCC with a scalar result type ↵Craig Topper2017-12-052-15/+1
| | | | | | | | | | from vector widening. There's no such thing as a setcc with vector operands and scalar result. And if we're trying to widen the result we would have to already be looking at a vector result type. So this patch renames the VSETCC function as the SETCC function and delete the original SETCC function. llvm-svn: 319799
* [SelectionDAG] Remove unused method declaration.Craig Topper2017-12-051-1/+0
| | | | | | The method implementation was removed in r318982. llvm-svn: 319798
* [DAGCombine] Move AND nodes to multiple load leavesSam Parker2017-12-051-0/+123
| | | | | | | | | | | | | Search from AND nodes to find whether they can be propagated back to loads, so that the AND and load can be combined into a narrow load. We search through OR, XOR and other AND nodes and all bar one of the leaves are required to be loads or constants. The exception node then needs to be masked off meaning that the 'and' isn't removed, but the loads(s) are narrowed still. Differential Revision: https://reviews.llvm.org/D39604 llvm-svn: 319773
* [DAGCombine] Handle big endian correctly in CombineConsecutiveLoadsBjorn Pettersson2017-12-051-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Found out, at code inspection, that there was a fault in DAGCombiner::CombineConsecutiveLoads for big-endian targets. A BUILD_PAIR is always having the least significant bits of the composite value in element 0. So when we are doing the checks for consecutive loads, for big endian targets, we should check if the load to elt 1 is at the lower address and the load to elt 0 is at the higher address. Normally this bug only resulted in missed oppurtunities for doing the load combine. I guess that in some rare situation it could lead to faulty combines, but I've not seen that happen. Note that this patch actually will trigger load combine for some big endian regression tests. One example is test/CodeGen/PowerPC/anon_aggr.ll where we now get t76: i64,ch = load<LD8[FixedStack-9] instead of t37: i32,ch = load<LD4[FixedStack-10]> t35: i32,ch = load<LD4[FixedStack-9]> t41: i64 = build_pair t37, t35 before legalization. Then the legalization will split the LD8 into two loads, so the end result is the same. That should verify that the transfomation is correct now. Reviewers: niravd, hfinkel Reviewed By: niravd Subscribers: nemanjai, llvm-commits Differential Revision: https://reviews.llvm.org/D40444 llvm-svn: 319771
* [DAGCombine] isLegalNarrowLoad function (NFC)Sam Parker2017-12-051-42/+60
| | | | | | | | | Pull the checks upon the load out from ReduceLoadWidth into their own function. Differential Revision: https://reviews.llvm.org/D40833 llvm-svn: 319766
* [SelectionDAG] Use WidenTargetBoolean in WidenVecRes_MLOAD and ↵Craig Topper2017-12-051-29/+2
| | | | | | | | WidenVecOp_MSTORE instead of implementing it manually and incorrectly. The CONCAT_VECTORS operand get its type from getSetCCResultType, but if the mask type and the setcc have different scalar sizes this creates an illegal CONCAT_VECTORS operation. The concat type should be 2x the mask type, and then an extend should be added if needed. llvm-svn: 319744
* Revert r319490 "XOR the frame pointer with the stack cookie when protecting ↵Hans Wennborg2017-12-041-9/+4
| | | | | | | | | | | | | | | | | | the stack" This broke the Chromium build (crbug.com/791714). Reverting while investigating. > Summary: This strengthens the guard and matches MSVC. > > Reviewers: hans, etienneb > > Subscribers: hiraditya, JDevlieghere, vlad.tsyrklevich, llvm-commits > > Differential Revision: https://reviews.llvm.org/D40622 > > git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@319490 91177308-0d34-0410-b5e6-96231b3b80d8 llvm-svn: 319706
* DAG: Follow-up to r319692 check the truncates inputs have the same typeHans Wennborg2017-12-041-1/+2
| | | | | | | | | MatchRotate assumes the types of the types of LHS and RHS are equal, which is always the case then they come from an OR node, but here we're getting them from two different TRUNC nodes, so we have to check the types. llvm-svn: 319695
* DAG: Match truncated rotation (PR35487)Hans Wennborg2017-12-041-0/+9
| | | | | | | | | If the truncation has been pushed past the or-node, look through it and truncate afterwards. Differential revision: https://reviews.llvm.org/D40792 llvm-svn: 319692
* [CodeGen] Unify MBB reference format in both MIR and debug outputFrancis Visoiu Mistrih2017-12-043-29/+46
| | | | | | | | | | | | | | | | As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
* [DAGCombine] Remove isAndLoadExtLoad argumentsSam Parker2017-12-041-14/+6
| | | | | | | | | Both LoadedVT and NarrowLoad are passed as references and neither of them are used by any of its callers. Differential Revision: https://reviews.llvm.org/D40713 llvm-svn: 319645
* [SelectionDAG] Teach computeKnownBits some improvements to ISD::SRL with a ↵Craig Topper2017-12-041-0/+19
| | | | | | | | | non-splat constant shift amount. If we have a non-splat constant shift amount, the minimum shift amount can be used to infer the number of zero upper bits of the result. There's probably a lot more that we can do here, but this fixes a case where I wanted to infer the sign bit as zero when all the shift amounts are non-zero. llvm-svn: 319639
* CodeGen: Fix SelectionDAGISel::LowerArguments for sret addr spaceYaxun Liu2017-12-031-7/+13
| | | | | | | | | | | SelectionDAGISel::LowerArguments assumes sret addr space is 0, which is not true for amdgcn---amdgiz target. This patch fixes that. Differential Revision: https://reviews.llvm.org/D40255 llvm-svn: 319630
* [SelectionDAG] Use the inlined APInt shift methods since we've already ↵Craig Topper2017-12-031-8/+11
| | | | | | | | bounds checked the shift. The version that takes APInt is out of line. The 'unsigned' version optimizes for the common case of single word APInts. llvm-svn: 319628
* CodeGen: Fix pointer info in ↵Yaxun Liu2017-12-023-21/+28
| | | | | | | | | | | | | | | | | | | | | SplitVecOp_EXTRACT_VECTOR_ELT/SplitVecRes_INSERT_VECTOR_ELT Two issues found when doing codegen for splitting vector with non-zero alloca addr space: DAGTypeLegalizer::SplitVecRes_INSERT_VECTOR_ELT/SplitVecOp_EXTRACT_VECTOR_ELT uses dummy pointer info for creating SDStore. Since one pointer operand contains multiply and add, InferPointerInfo is unable to infer the correct pointer info, which ends up with a dummy pointer info for the target to lower store and results in isel failure. The fix is to introduce MachinePointerInfo::getUnknownStack to represent MachinePointerInfo which is known in alloca address space but without other information. TargetLowering::getVectorElementPointer uses value type of pointer in addr space 0 for multiplication of index and then add it to the pointer. However the pointer may be in an addr space which has different size than addr space 0. The fix is to use the pointer value type for index multiplication. Differential Revision: https://reviews.llvm.org/D39758 llvm-svn: 319622
* Revert "[X86] Improvement in CodeGen instruction selection for LEAs."Matt Morehouse2017-12-011-11/+0
| | | | | | This reverts r319543, due to ASan bot breakage. llvm-svn: 319591
* [DAG][ARM] Revert "Reenable post-legalize store merge"Nirav Dave2017-12-011-11/+5
| | | | | | due to failures in AArch and ARM code gen. llvm-svn: 319587
* [DAGCombine] Simplify ISD::AND handling in ReduceLoadWidthEli Friedman2017-12-011-20/+5
| | | | | | | | Followup to D39595. Removes a bunch of redundant checks. Differential Revision: https://reviews.llvm.org/D40667 llvm-svn: 319573
* [ARM][DAG] Reenable post-legalize store mergeNirav Dave2017-12-011-5/+11
| | | | | | | | | | | | Summary: Reenable post-legalize stores with constant merging computation and cofrresponding test case. Reviewers: eastig, efriedma Subscribers: aemerson, javed.absar, kristof.beyls, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D40701 llvm-svn: 319547
* [X86] Improvement in CodeGen instruction selection for LEAs.Jatin Bhateja2017-12-011-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: 1/ Operand folding during complex pattern matching for LEAs has been extended, such that it promotes Scale to accommodate similar operand appearing in the DAG e.g. T1 = A + B T2 = T1 + 10 T3 = T2 + A For above DAG rooted at T3, X86AddressMode will now look like Base = B , Index = A , Scale = 2 , Disp = 10 2/ During OptimizeLEAPass down the pipeline factorization is now performed over LEAs so that if there is an opportunity then complex LEAs (having 3 operands) could be factored out e.g. leal 1(%rax,%rcx,1), %rdx leal 1(%rax,%rcx,2), %rcx will be factored as following leal 1(%rax,%rcx,1), %rdx leal (%rdx,%rcx) , %edx 3/ Aggressive operand folding for AM based selection for LEAs is sensitive to loops, thus avoiding creation of any complex LEAs within a loop. 4/ Simplify LEA converts (lea (BASE,1,INDEX,0) --> add (BASE, INDEX) which offers better through put. PR32755 will be taken care of by this pathc. Previous patch revisions : r313343 , r314886 Reviewers: lsaba, RKSimon, craig.topper, qcolombet, jmolloy, jbhateja Reviewed By: lsaba, RKSimon, jbhateja Subscribers: jmolloy, spatel, igorb, llvm-commits Differential Revision: https://reviews.llvm.org/D35014 llvm-svn: 319543
* [X86][SelectionDAG] Make sure we explicitly sign extend the index when type ↵Craig Topper2017-12-011-0/+6
| | | | | | | | promoting the index of scatter and gather. Type promotion makes no guarantee about the contents of the promoted bits. Since the gather/scatter instruction will use the bits to calculate addresses, we need to ensure they aren't garbage. llvm-svn: 319520
* Mark all library options as hidden.Zachary Turner2017-12-011-5/+5
| | | | | | | | | | | | | | | | | These command line options are not intended for public use, and often don't even make sense in the context of a particular tool anyway. About 90% of them are already hidden, but when people add new options they forget to hide them, so if you were to make a brand new tool today, link against one of LLVM's libraries, and run tool -help you would get a bunch of junk that doesn't make sense for the tool you're writing. This patch hides these options. The real solution is to not have libraries defining command line options, but that's a much larger effort and not something I'm prepared to take on. Differential Revision: https://reviews.llvm.org/D40674 llvm-svn: 319505
* XOR the frame pointer with the stack cookie when protecting the stackReid Kleckner2017-11-301-4/+9
| | | | | | | | | | | | Summary: This strengthens the guard and matches MSVC. Reviewers: hans, etienneb Subscribers: hiraditya, JDevlieghere, vlad.tsyrklevich, llvm-commits Differential Revision: https://reviews.llvm.org/D40622 llvm-svn: 319490
* [CodeGen] Always use `printReg` to print registers in both MIR and debugFrancis Visoiu Mistrih2017-11-301-4/+6
| | | | | | | | | | | | | | output As part of the unification of the debug format and the MIR format, always use `printReg` to print all kinds of registers. Updated the tests using '_' instead of '%noreg' until we decide which one we want to be the default one. Differential Revision: https://reviews.llvm.org/D40421 llvm-svn: 319445
* [DAGCombine] Refactor ReduceLoadWidthSam Parker2017-11-301-50/+33
| | | | | | | | | | visitAND attempts to narrow the width of extending loads that are then masked off. ReduceLoadWidth already exists for a similar purpose and handles shifts, so I've moved the code to handle AND nodes there. Differential Revision: https://reviews.llvm.org/D39595 llvm-svn: 319421
* [SelectionDAG][X86] Teach promotion legalization for fp_to_sint/fp_to_uint ↵Craig Topper2017-11-291-3/+11
| | | | | | | | | | | | to insert an assertsext/assertzext based on the original type If we put in an assertsext/zext here, we're able to generate better truncate code using pack on pre-avx512 targets. Similar is already done during type legalization. This is the equivalent for op legalization Differential Revision: https://reviews.llvm.org/D40591 llvm-svn: 319368
OpenPOWER on IntegriCloud