summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG
Commit message (Collapse)AuthorAgeFilesLines
...
* Don't look past volatile loads.Bill Wendling2013-11-251-0/+5
| | | | | | | A volatile load should block us from trying to coalesce stores. PR18023 llvm-svn: 195599
* Teach ISel not to optimize 'optnone' functions (revised).Paul Robinson2013-11-221-0/+44
| | | | | | | | | | | | | Improvements over r195317: - Set/restore EnableFastISel flag instead of just running FastISel within SelectAllBasicBlocks; the flag is checked in various places, and FastISel won't run properly if those places don't do the right thing. - Test looks for normal ISel versus FastISel behavior, and not something more subtle that doesn't work everywhere. Based on work by Andrea Di Biagio. llvm-svn: 195491
* patchpoint: factor SD builder code for live vars. Plain stackmap also ↵Andrew Trick2013-11-221-13/+19
| | | | | | optimizes Constant values now. llvm-svn: 195488
* patchpoint: eliminate hard coded operand indices.Andrew Trick2013-11-221-13/+19
| | | | llvm-svn: 195487
* SelectionDAG: Optimize expansion of vec_type = BITCAST scalar_typeTom Stellard2013-11-222-10/+41
| | | | | | | | | The legalizer can now do this type of expansion for more type combinations without loading and storing to and from the stack. NOTE: This is a candidate for the 3.4 branch. llvm-svn: 195398
* Split SETCC if VSELECT requires splitting too.Tom Stellard2013-11-222-21/+43
| | | | | | | | | | | | | This patch is a rewrite of the original patch commited in r194542. Instead of relying on the type legalizer to do the splitting for us, we now peform the splitting ourselves in the DAG combiner. This is necessary for the case where the vector mask is a legal type after promotion and still wouldn't require splitting. Patch by: Juergen Ributzka NOTE: This is a candidate for the 3.4 branch. llvm-svn: 195397
* Add support for legalizing SETNE/SETEQ by inverting the condition code and ↵Daniel Sanders2013-11-211-14/+54
| | | | | | | | | | | | | | | | | | | | | | | | | the result of the comparison. Summary: LegalizeSetCCCondCode can now legalize SETEQ and SETNE by returning the inverse condition and requesting that the caller invert the result of the condition. The caller of LegalizeSetCCCondCode must handle the inverted CC, and they do so as follows: SETCC, BR_CC: Invert the result of the SETCC with SelectionDAG::getNOT() SELECT_CC: Swap the true/false operands. This is necessary for MSA which lacks an integer SETNE instruction. Reviewers: resistor CC: llvm-commits Differential Revision: http://llvm-reviews.chandlerc.com/D2229 llvm-svn: 195355
* Revert r195317 (and r195333), "Teach ISel not to optimize 'optnone' functions."NAKAMURA Takumi2013-11-211-40/+1
| | | | | | | | It broke, at least, i686 target. It is reproducible with "llc -mtriple=i686-unknown". FYI, it didn't appear to add either "-O0" or "-fast-isel". llvm-svn: 195339
* Teach ISel not to optimize 'optnone' functions.Paul Robinson2013-11-211-1/+40
| | | | | | Based on work by Andrea Di Biagio. llvm-svn: 195317
* long line correctionJack Carter2013-11-201-1/+2
| | | | llvm-svn: 195179
* long lines and white space correctionJack Carter2013-11-191-29/+34
| | | | llvm-svn: 195170
* [DAG] Refactor vector splitting code in SelectionDAG. No functional change ↵Juergen Ributzka2013-11-196-114/+75
| | | | | | | | intended. Reviewed by Tom llvm-svn: 195156
* Fix patchpoint comments.Andrew Trick2013-11-191-1/+2
| | | | llvm-svn: 195103
* DAGCombiner: Partially revert r192795, getNOT was fixed not to create ↵Benjamin Kramer2013-11-171-1/+1
| | | | | | illegal constants. llvm-svn: 194959
* Use more getZExtOrTruncsMatt Arsenault2013-11-172-9/+2
| | | | llvm-svn: 194945
* Use getZExtOrTrunc instead of repeating the same logic.Matt Arsenault2013-11-171-5/+1
| | | | llvm-svn: 194944
* Use right address space pointer sizeMatt Arsenault2013-11-171-1/+2
| | | | llvm-svn: 194940
* Fix assert on unaligned access to global with different address space size.Matt Arsenault2013-11-161-1/+1
| | | | llvm-svn: 194934
* Fix codegen for null different sized pointer.Matt Arsenault2013-11-161-2/+4
| | | | llvm-svn: 194932
* Avoid illegal integer promotion in fastiselBob Wilson2013-11-151-0/+15
| | | | | | | | | | | | | | | | | Stop folding constant adds into GEP when the type size doesn't match. Otherwise, the adds' operands are effectively being promoted, changing the conditions of an overflow. Results are different when: sext(a) + sext(b) != sext(a + b) Problem originally found on x86-64, but also fixed issues with ARM and PPC, which used similar code. <rdar://problem/15292280> Patch by Duncan Exon Smith! llvm-svn: 194840
* Fix illegal DAG produced by SelectionDAG::getConstant() for v2i64 typeDaniel Sanders2013-11-152-1/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: When getConstant() is called for an expanded vector type, it is split into multiple scalar constants which are then combined using appropriate build_vector and bitcast operations. In addition to the usual big/little endian differences, the case where the element-order of the vector does not have the same endianness as the elements themselves is also accounted for. For example, for v4i32 on big-endian MIPS, the byte-order of the vector is <3210,7654,BA98,FEDC>. For little-endian, it is <0123,4567,89AB,CDEF>. Handling this case turns out to be a nop since getConstant() returns a splatted vector (so reversing the element order doesn't change the value) This fixes a number of cases in MIPS MSA where calling getConstant() during operation legalization introduces illegal types (e.g. to legalize v2i64 UNDEF into a v2i64 BUILD_VECTOR of illegal i64 zeros). It should also handle bigger differences between illegal and legal types such as legalizing v2i64 into v8i16. lowerMSASplatImm() in the MIPS backend no longer needs to avoid calling getConstant() so this function has been updated in the same patch. For the sake of transparency, the steps I've taken since the review are: * Added 'virtual' to isVectorEltOrderLittleEndian() as requested. This revealed that the MIPS tests were falsely passing because a polymorphic function was not actually polymorphic in the reviewed patch. * Fixed the tests that were now failing. This involved deleting the code to handle the MIPS MSA element-order (which was previously doing an byte-order swap instead of an element-order swap). This left isVectorEltOrderLittleEndian() unused and it was deleted. * Fixed build failures caused by rebasing beyond r194467-r194472. These build failures involved the bset, bneg, and bclr instructions added in these commits using lowerMSASplatImm() in a way that was no longer valid after this patch. Some of these were fixed by calling SelectionDAG::getConstant() instead, others were fixed by a new function getBuildVectorSplat() that provided the removed functionality of lowerMSASplatImm() in a more sensible way. Reviewers: bkramer Reviewed By: bkramer CC: llvm-commits Differential Revision: http://llvm-reviews.chandlerc.com/D1973 llvm-svn: 194811
* Add target hook to prevent folding some bitcasted loads.Matt Arsenault2013-11-151-1/+2
| | | | | | | | | | | | | This is to avoid this transformation in some cases: fold (conv (load x)) -> (load (conv*)x) On architectures that don't natively support some vector loads efficiently casting the load to a smaller vector of larger types and loading is more efficient. Patch by Micah Villmow. llvm-svn: 194783
* Add addrspacecast instruction.Matt Arsenault2013-11-154-0/+51
| | | | | | Patch by Michele Scandale! llvm-svn: 194760
* Minor extension to llvm.experimental.patchpoint: don't require a call.Andrew Trick2013-11-141-1/+1
| | | | | | | | If a null call target is provided, don't emit a dummy call. This allows the runtime to reserve as little nop space as it needs without the requirement of emitting a call. llvm-svn: 194676
* SelectionDAG: Teach the legalizer to split SETCC if VSELECT needs splitting too.Juergen Ributzka2013-11-132-8/+36
| | | | | | | | | | | | | | | | | | | | | | This patch reapplies r193676 with an additional fix for the Hexagon backend. The SystemZ backend has already been fixed by r194148. The Type Legalizer recognizes that VSELECT needs to be split, because the type is to wide for the given target. The same does not always apply to SETCC, because less space is required to encode the result of a comparison. As a result VSELECT is split and SETCC is unrolled into scalar comparisons. This commit fixes the issue by checking for VSELECT-SETCC patterns in the DAG Combiner. If a matching pattern is found, then the result mask of SETCC is promoted to the expected vector mask type for the given target. Now the type legalizer will split both VSELECT and SETCC. This allows the following X86 DAG Combine code to sucessfully detect the MIN/MAX pattern. This fixes PR16695, PR17002, and <rdar://problem/14594431>. Reviewed by Nadav llvm-svn: 194542
* Vector forms of SHL, SRA, and SRL can be constant folded using ↵Daniel Sanders2013-11-111-0/+18
| | | | | | | | | | | | | | SimplifyVBinOp too Reviewers: dsanders Reviewed By: dsanders CC: llvm-commits, nadav Differential Revision: http://llvm-reviews.chandlerc.com/D1958 llvm-svn: 194393
* [Stackmap] Materialize the jump address within the patchpoint noop slide.Juergen Ributzka2013-11-092-4/+15
| | | | | | | | | | | | | | | This patch moves the jump address materialization inside the noop slide. This enables patching of the materialization itself or its complete removal. This patch also adds the ability to define scratch registers that can be used safely by the code called from the patchpoint intrinsic. At least one scratch register is required, because that one is used for the materialization of the jump address. This patch depends on D2009. Differential Revision: http://llvm-reviews.chandlerc.com/D2074 Reviewed by Andy llvm-svn: 194306
* [Stackmap] Add AnyReg calling convention support for patchpoint intrinsic.Juergen Ributzka2013-11-083-35/+88
| | | | | | | | | | | | | | The idea of the AnyReg Calling Convention is to provide the call arguments in registers, but not to force them to be placed in a paticular order into a specified set of registers. Instead it is up tp the register allocator to assign any register as it sees fit. The same applies to the return value (if applicable). Differential Revision: http://llvm-reviews.chandlerc.com/D2009 Reviewed by Andy llvm-svn: 194293
* Slightly change the way stackmap and patchpoint intrinsics are lowered.Andrew Trick2013-11-051-9/+27
| | | | | | | | | | | | | | MorphNodeTo is not safe to call during DAG building. It eagerly deletes dependent DAG nodes which invalidates the NodeMap. We could expose a safe interface for morphing nodes, but I don't think it's worth it. Just create a new MachineNode and replaceAllUsesWith. My understaning of the SD design has been that we want to support early target opcode selection. That isn't very well supported, but generally works. It seems reasonable to rely on this feature even if it isn't widely used. llvm-svn: 194102
* [Stackmap] Remove erroneous assert.Juergen Ributzka2013-11-011-3/+0
| | | | llvm-svn: 193871
* Commenting out this assert because it is causing the build bots to fail. ↵Aaron Ballman2013-11-011-2/+2
| | | | | | This effectively reverts r193861, but needs to be fixed as part of r193769. llvm-svn: 193862
* Fixing an order of evaluation error in an assert.Aaron Ballman2013-11-011-1/+1
| | | | llvm-svn: 193861
* Add support for stack map generation in the X86 backend.Andrew Trick2013-10-311-0/+3
| | | | | | Originally implemented by Lang Hames. llvm-svn: 193811
* Lower stackmap intrinsics directly to their target opcode in the DAG builder.Andrew Trick2013-10-313-11/+216
| | | | llvm-svn: 193769
* whitespaceAndrew Trick2013-10-311-7/+7
| | | | llvm-svn: 193765
* Legalize: Improve legalization of long vector extends.Jim Grosbach2013-10-312-3/+63
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When an extend more than doubles the size of the elements (e.g., a zext from v16i8 to v16i32), the normal legalization method of splitting the vectors will run into problems as by the time the destination vector is legal, the source vector is illegal. The end result is the operation often becoming scalarized, with the typical horrible performance. For example, on x86_64, the simple input of: define void @bar(<16 x i8> %a, <16 x i32>* %p) nounwind { %tmp = zext <16 x i8> %a to <16 x i32> store <16 x i32> %tmp, <16 x i32>*%p ret void } Generates: .section __TEXT,__text,regular,pure_instructions .section __TEXT,__const .align 5 LCPI0_0: .long 255 ## 0xff .long 255 ## 0xff .long 255 ## 0xff .long 255 ## 0xff .long 255 ## 0xff .long 255 ## 0xff .long 255 ## 0xff .long 255 ## 0xff .section __TEXT,__text,regular,pure_instructions .globl _bar .align 4, 0x90 _bar: vpunpckhbw %xmm0, %xmm0, %xmm1 vpunpckhwd %xmm0, %xmm1, %xmm2 vpmovzxwd %xmm1, %xmm1 vinsertf128 $1, %xmm2, %ymm1, %ymm1 vmovaps LCPI0_0(%rip), %ymm2 vandps %ymm2, %ymm1, %ymm1 vpmovzxbw %xmm0, %xmm3 vpunpckhwd %xmm0, %xmm3, %xmm3 vpmovzxbd %xmm0, %xmm0 vinsertf128 $1, %xmm3, %ymm0, %ymm0 vandps %ymm2, %ymm0, %ymm0 vmovaps %ymm0, (%rdi) vmovaps %ymm1, 32(%rdi) vzeroupper ret So instead we can check if there are legal types that enable us to split more cleverly when the input vector is already legal such that we don't turn it into an illegal type. If the extend is such that it's more than doubling the size of the input we check if - the number of vector elements is even, - the source type is legal, - the type of a split source is illegal, - the type of an extended (by doubling element size) source is legal, and - the type of that extended source when split is legal. If the conditions are met, instead of just splitting both the destination and the source types, we create an extend that only goes up one "step" (doubling the element width), and the continue legalizing the rest of the operation normally. The result is that this operates as a new, more effecient, termination condition for the loop of "split the operation until the destination type is legal." With this change, the above example now compiles to: _bar: vpxor %xmm1, %xmm1, %xmm1 vpunpcklbw %xmm1, %xmm0, %xmm2 vpunpckhwd %xmm1, %xmm2, %xmm3 vpunpcklwd %xmm1, %xmm2, %xmm2 vinsertf128 $1, %xmm3, %ymm2, %ymm2 vpunpckhbw %xmm1, %xmm0, %xmm0 vpunpckhwd %xmm1, %xmm0, %xmm3 vpunpcklwd %xmm1, %xmm0, %xmm0 vinsertf128 $1, %xmm3, %ymm0, %ymm0 vmovaps %ymm0, 32(%rdi) vmovaps %ymm2, (%rdi) vzeroupper ret This generalizes a custom lowering that was added a while back to the ARM backend. That lowering is no longer necessary, and is removed. The testcases for it, however, provide excellent ARM tests for this change and so remain. rdar://14735100 llvm-svn: 193727
* Fix CodeGen for unaligned loads with address spacesMatt Arsenault2013-10-301-2/+5
| | | | llvm-svn: 193721
* Revert "SelectionDAG: Teach the legalizer to split SETCC if VSELECT needs ↵Juergen Ributzka2013-10-302-35/+8
| | | | | | | | splitting too." Now Hexagon and SystemZ are not happy with it :-( llvm-svn: 193677
* SelectionDAG: Teach the legalizer to split SETCC if VSELECT needs splitting too.Juergen Ributzka2013-10-302-8/+35
| | | | | | | | | | | | | | | | | | | | The Type Legalizer recognizes that VSELECT needs to be split, because the type is to wide for the given target. The same does not always apply to SETCC, because less space is required to encode the result of a comparison. As a result VSELECT is split and SETCC is unrolled into scalar comparisons. This commit fixes the issue by checking for VSELECT-SETCC patterns in the DAG Combiner. If a matching pattern is found, then the result mask of SETCC is promoted to the expected vector mask type for the given target. This mask has usually the same size as the VSELECT return type (except for Intel KNL). Now the type legalizer will split both VSELECT and SETCC. This allows the following X86 DAG Combine code to sucessfully detect the MIN/MAX pattern. This fixes PR16695, PR17002, and <rdar://problem/14594431>. Reviewed by Nadav llvm-svn: 193676
* Fix "existant" typosAlp Toker2013-10-291-1/+1
| | | | llvm-svn: 193579
* [DAGCombiner] Respect volatility when checking for aliasesRichard Sandiford2013-10-281-18/+25
| | | | | | | | Making useAA() default to true for SystemZ showed that the combiner alias analysis wasn't handling volatile accesses. This hit many of the SystemZ tests, but I arbitrarily picked one for the purpose of this patch. llvm-svn: 193518
* Keep TBAA info when rewriting SelectionDAG loads and storesRichard Sandiford2013-10-288-191/+181
| | | | | | | | | | | | | | | | | Most SelectionDAG code drops the TBAA info when creating a new form of a load and store (e.g. during legalization, or when converting a plain load to an extending one). This patch tries to catch all cases where the TBAA information can legitimately be carried over. The patch adds alternative forms of getLoad() and getExtLoad() that take a MachineMemOperand instead of individual fields. (The corresponding getTruncStore() already exists.) The idea is to use the MachineMemOperand forms when all fields are carried over (size, pointer info, isVolatile, isNonTemporal, alignment and TBAA info). If some adjustment is being made, e.g. to narrow the load, then we still pass the individual fields but also pass the TBAA info. llvm-svn: 193517
* LegalizeDAG: allow libcalls for max/min atomic operationsTim Northover2013-10-251-0/+40
| | | | | | | | | | | ARM processors without ldrex/strex need to be able to make libcalls for all atomic operations, including the newer min/max versions. The alternative would probably be expanding these operations in terms of cmpxchg (as x86 does always), but in the configurations where this matters code-size tends to be paramount so the libcall is more desirable. llvm-svn: 193398
* Optimize concat_vectors(X, undef) -> scalar_to_vector(X).Nadav Rotem2013-10-251-1/+28
| | | | | | | This optimization is not SSE specific so I am moving it to DAGco. The new scalar_to_vector dag node exposed a missing pattern in the AArch64 target that I needed to add. llvm-svn: 193393
* SelectionDAG: Pass along the original argument/element type in ISD::InputArgTom Stellard2013-10-231-5/+8
| | | | | | | | | | | | | | | | For some targets, it is useful to be able to look at the original type of an argument without having to dig through the original IR. This also fixes a bug in SelectionDAGBuilder where InputArg.PartOffset was not taking into account the offset of structure elements. Patch by: Justin Holewinski Tom Stellard: - Changed the type of ArgVT to EVT, so it can store non-simple types like v3i32. llvm-svn: 193214
* Using FoldingSet in SelectionDAG::getVTList.Wan Xiaofei2013-10-221-59/+64
| | | | | | | | | | VTList has a long life cycle through the module and getVTList is frequently called. In current getVTList, sequential search over a std::vector is used, this is inefficient in big module. This patch use FoldingSet to implement hashing mechanism when searching. Reviewer: Nadav Rotem Test : Pass unit tests & LNT test suite llvm-svn: 193150
* Fix CodeGen for different size address space GEPsMatt Arsenault2013-10-211-6/+4
| | | | llvm-svn: 193111
* Reuse variableMatt Arsenault2013-10-211-1/+1
| | | | llvm-svn: 193107
* [PATCH] Fix PR17168 (DAG scheduler inserts DBG_VALUE before PHI with fast-isel)Bill Schmidt2013-10-181-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | PR17168 describes a test case that fails when compiling for debug with fast-isel. Investigation showed that the test was failing because a DBG_VALUE machine instruction was placed prior to a PHI. For this problem to occur requires the following: * Compile for debug * Compile with fast-isel * In a block B, fast-isel must partially succeed before punting to DAG-isel * B must start with a PHI * The first unhandled node in the DAG must not generate a machine instruction * A debug value with an order less than that of that first node exists When all of these circumstances apply, the existing test that an instruction was not inserted won't fire. Currently it tests whether the block is empty, or whether the last instruction generated is a phi. When fast-isel has partially succeeded, the last instruction generated will not be a phi. Instead, we need to check whether the current insert position is immediately following a phi. This patch adds that check, and adds the test case from the PR as a regression test. llvm-svn: 192976
* CodeGen: Emit a libcall if the target doesn't support 16-byte wide atomicsDavid Majnemer2013-10-182-0/+16
| | | | | | | | | | There are targets that support i128 sized scalars but cannot emit instructions that modify them directly. The proper thing to do is to emit a libcall. This fixes PR17481. llvm-svn: 192957
OpenPOWER on IntegriCloud