summaryrefslogtreecommitdiffstats
path: root/llvm/test/CodeGen/X86
Commit message (Collapse)AuthorAgeFilesLines
* Add Constant Hoisting PassJuergen Ributzka2014-01-241-0/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This pass identifies expensive constants to hoist and coalesces them to better prepare it for SelectionDAG-based code generation. This works around the limitations of the basic-block-at-a-time approach. First it scans all instructions for integer constants and calculates its cost. If the constant can be folded into the instruction (the cost is TCC_Free) or the cost is just a simple operation (TCC_BASIC), then we don't consider it expensive and leave it alone. This is the default behavior and the default implementation of getIntImmCost will always return TCC_Free. If the cost is more than TCC_BASIC, then the integer constant can't be folded into the instruction and it might be beneficial to hoist the constant. Similar constants are coalesced to reduce register pressure and materialization code. When a constant is hoisted, it is also hidden behind a bitcast to force it to be live-out of the basic block. Otherwise the constant would be just duplicated and each basic block would have its own copy in the SelectionDAG. The SelectionDAG recognizes such constants as opaque and doesn't perform certain transformations on them, which would create a new expensive constant. This optimization is only applied to integer constants in instructions and simple (this means not nested) constant cast experessions. For example: %0 = load i64* inttoptr (i64 big_constant to i64*) Reviewed by Eric llvm-svn: 200022
* Fix known typosAlp Toker2014-01-244-7/+7
| | | | | | | Sweep the codebase for common typos. Includes some changes to visible function names that were misspelt. llvm-svn: 200018
* [X86] Prevent the creation of redundant ops for sadd and ssub with overflow.Juergen Ributzka2014-01-241-0/+34
| | | | | | | | | | | | | This commit teaches the X86 backend to create the same X86 instructions when it lowers an sadd/ssub with overflow intrinsic and a conditional branch that uses that overflow result. This allows SelectionDAG to recognize and remove one of the redundant operations. This fixes <rdar://problem/15874016> and <rdar://problem/15661073>. Reviewed by Nadav llvm-svn: 199976
* Replace vfmaddxx213 instructions with their 231-type equivalents in accumulatorLang Hames2014-01-231-0/+15
| | | | | | | loops. Writing back to the accumulator (231-type) allows the coalescer to eliminate an extra copy. llvm-svn: 199933
* AVX-512: added VPERM2D VPERM2Q VPERM2PS VPERM2PD instructions,Elena Demikhovsky2014-01-231-7/+7
| | | | | | they give better sequences than VPERMI llvm-svn: 199893
* Revert r162101 and replace it with a solution that works for targets where ↵Owen Anderson2014-01-221-1/+1
| | | | | | | | | | the pointer type is illegal. This is a horrible bit of code. We're calling a simplification routine *in the middle* of type legalization. We tell the simplification routine that it's running after legalization, but some of the types it will encounter will be illegal! The fix is only to invoke the simplification if the types in question were legal, so that none of its invariants will be violated. llvm-svn: 199847
* Add a testcase for r199430.Quentin Colombet2014-01-221-0/+4
| | | | llvm-svn: 199831
* AVX512: combining setcc and zext is wrong on AVX512Elena Demikhovsky2014-01-221-0/+11
| | | | | | because vector compare instruction puts result in mask register. llvm-svn: 199798
* MachineCopyPropagation has special logic for removing COPY instructions. It ↵James Molloy2014-01-221-1/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | will remove plain COPYs using eraseFromParent(), but if the COPY has imp-defs/imp-uses it will convert it to a KILL, to keep the imp-def around. This actually totally breaks and causes the machine verifier to cry in several cases, one of which being: %RAX<def> = COPY %RCX<kill> %ECX<def> = COPY %EAX<kill>, %RAX<imp-use,kill> These subregister copies are together identified as noops, so are both removed. However, the second one as it has an imp-use gets converted into a kill: %ECX<def> = KILL %EAX<kill>, %RAX<imp-use,kill> As the original COPY has been removed, the verifier goes into tears at the use of undefined EAX and RAX. There are several hacky solutions to this hacky problem (which is all to do with imp-use/def weirdnesses), but the least hacky I've come up with is to *always* remove COPYs by converting to KILLs. KILLs are no-ops to the code generator so the generated code doesn't change (which is why they were partially used in the first place), but using them also keeps the def/use and imp-def/imp-use chains alive: %RAX<def> = KILL %RCX<kill> %ECX<def> = KILL %EAX<kill>, %RAX<imp-use,kill> The patch passes all test cases including the ones that check the removal of MOVs in this circumstance, along with an extra test I added to check subregister behaviour (which made the machine verifier fall over before my patch). The patch also adds some DEBUG() statements because the file hadn't got any. llvm-svn: 199797
* Tweak the spelling of the asserts requirement a bit more. This makes itChandler Carruth2014-01-212-2/+2
| | | | | | | match the (reasonably prevelant) usage in Clang's test suite and so seems more "canonical". llvm-svn: 199767
* [X86] Teach how to combine a vselect into a movss/movsdAndrea Di Biagio2014-01-206-27/+320
| | | | | | | | | | | | | | | | | | | Add target specific rules for combining vselect dag nodes into movss/movsd when possible. If the vector type of the vselect dag node in input is either MVT::v4i13 or MVT::v4f32, then try to fold according to rules: 1) fold (vselect (build_vector (0, -1, -1, -1)), A, B) -> (movss A, B) 2) fold (vselect (build_vector (-1, 0, 0, 0)), A, B) -> (movss B, A) If the vector type of the vselect dag node in input is either MVT::v2i64 or MVT::v2f64 (and we have SSE2), then try to fold according to rules: 3) fold (vselect (build_vector (0, -1)), A, B) -> (movsd A, B) 4) fold (vselect (build_vector (-1, 0)), A, B) -> (movsd B, A) llvm-svn: 199683
* Fix misched-aa-colored.ll to require asserts (trying again)Hal Finkel2014-01-201-2/+1
| | | | | | Perhaps it needs to be in caps. llvm-svn: 199661
* Fix misched-aa-colored.ll to require asserts.Hal Finkel2014-01-201-0/+4
| | | | | | -misched=shuffle is NDEBUG only. Maybe we should change that. llvm-svn: 199659
* Update IR when merging slots in stack coloringHal Finkel2014-01-202-70/+186
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The way that stack coloring updated MMOs when merging stack slots, while correct, is suboptimal, and is incompatible with the use of AA during instruction scheduling. The solution, which involves the use of const_cast (and more importantly, updating the IR from within an MI-level pass), obviously requires some explanation: When the stack coloring pass was originally committed, the code in ScheduleDAGInstrs::buildSchedGraph tracked possible alias sets by using GetUnderlyingObject, and all load/store and store/store memory control dependencies where added between SUs at the object level (where only one object, that returned by GetUnderlyingObject, was used to identify the object associated with each MMO). When stack coloring merged stack slots, it would replace MMOs derived from the remapped alloca with the alloca with which the remapped alloca was being replaced. Because ScheduleDAGInstrs only used single objects, and tracked alias sets at the object level, this was a fine solution. In r169744, (Andy and) I updated the code in ScheduleDAGInstrs to use GetUnderlyingObjects, and track alias sets using, potentially, multiple underlying objects for each MMO. This was done, primarily, to provide the ability to look through PHIs, and provide better scheduling for induction-variable-dependent loads and stores inside loops. At this point, the MMO-updating code in stack coloring became suboptimal, because it would clear the MMOs for (i.e. completely pessimize) all instructions for which r169744 might help in scheduling. Updating the IR directly is the simplest fix for this (and the one with, by far, the least compile-time impact), but others are possible (we could give each MMO a small vector of potential values, or make use of a remapping table, constructed from MFI, inside ScheduleDAGInstrs). Unfortunately, replacing all MMO values derived from the remapped alloca with the base replacement alloca fundamentally breaks our ability to use AA during instruction scheduling (which is critical to performance on some targets). The reason is that the original MMO might have had an offset (either constant or dynamic) from the base remapped alloca, and that offset is not present in the updated MMO. One possible way around this would be to use GetPointerBaseWithConstantOffset, and update not only the MMO's value, but also its offset based on the original offset. Unfortunately, this solution would only handle constant offsets, and for safety (because AA is not completely restricted to deducing relationships with constant offsets), we would need to clear all MMOs without constant offsets over the entire function. This would be an even worse pessimization than the current single-object restriction. Any other solution would involve passing around a vector of remapped allocas, and teaching AA to use it, introducing additional complexity and overhead into AA. Instead, when remapping an alloca, we replace all IR uses of that alloca as well (optionally inserting a bitcast as necessary). This is even more efficient that the old MMO-updating code in the stack coloring pass (because it removes the need to call GetUnderlyingObject on all MMO values), removes the single-object pessimization in the default configuration, and enables the correct use of AA during instruction scheduling (all without any additional overhead). LLVM now no longer miscompiles itself on x86_64 when using -enable-misched -enable-aa-sched-mi -misched-bottomup=0 -misched-topdown=0 -misched=shuffle! Fixed PR18497. Because the alloca replacement is now done at the IR level, unless the MMO directly refers to the remapped alloca, the change cannot be seen at the MI level. As a result, there is no good way to fix test/CodeGen/X86/pr14090.ll. llvm-svn: 199658
* Add two new calling conventions for runtime callsJuergen Ributzka2014-01-172-0/+190
| | | | | | | | | | | | | | This patch adds two new target-independent calling conventions for runtime calls - PreserveMost and PreserveAll. The target-specific implementation for X86-64 is defined as following: - Arguments are passed as for the default C calling convention - The same applies for the return value(s) - PreserveMost preserves all GPRs - except R11 - PreserveAll preserves all GPRs and all XMMs/YMMs - except R11 Reviewed by Lang and Philip llvm-svn: 199508
* AVX-512: fixed a compare patternElena Demikhovsky2014-01-161-0/+9
| | | | llvm-svn: 199366
* Convert test to FileCheck.Rafael Espindola2014-01-161-4/+7
| | | | llvm-svn: 199355
* Update test/CodeGen/X86/vbinop-simplify-bug.ll.Andrea Di Biagio2014-01-151-1/+1
| | | | | | Redirect the output of llc to /dev/null. llvm-svn: 199329
* [DAGCombiner] Fix a wrong check in method SimplifyVBinOp.Andrea Di Biagio2014-01-151-0/+23
| | | | | | | | | | | | | | | | | | | This fixes a regression intruced by r199135. Revision 199135 tried to simplify part of the logic in method DAGCombiner::SimplifyVBinOp introducing calls to method BuildVectorSDNode::isConstant(). However, that revision wrongly changed the check performed by method SimplifyVBinOp to identify dag nodes that can be folded. Before revision 199135, that method only tried to simplify vector binary operations if both operands were build_vector of Constant/ConstantFP/Undef only. After revision 199135, method SimplifyVBinop tried to simplify also vector binary operations with only one constant operand. This fixes the problem restoring the old behavior of SimplifyVBinOp. llvm-svn: 199328
* Handle dllexport for global aliasesNico Rieck2014-01-142-0/+50
| | | | llvm-svn: 199219
* Decouple dllexport/dllimport from linkageNico Rieck2014-01-145-20/+280
| | | | | | | | | | | | | | | | | | | Representing dllexport/dllimport as distinct linkage types prevents using these attributes on templates and inline functions. Instead of introducing further mixed linkage types to include linkonce and weak ODR, the old import/export linkage types are replaced with a new separate visibility-like specifier: define available_externally dllimport void @f() {} @Var = dllexport global i32 1, align 4 Linkage for dllexported globals and functions is now equal to their linkage without dllexport. Imported globals and functions must be either declarations with external linkage, or definitions with AvailableExternallyLinkage. llvm-svn: 199218
* AVX-512: optimized scalar compare patternsElena Demikhovsky2014-01-141-1/+23
| | | | | | removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
* [X86] Fix assertion failure caused by a wrong folding of vector shifts by ↵Andrea Di Biagio2014-01-141-0/+36
| | | | | | | | | | | | | | | | | | immediate count. This fixes a regression intruced by r198113. Revision r198113 introduced an algorithm that tries to fold a vector shift by immediate count into a build_vector if the input vector is a known vector of constants. However the algorithm only worked under the assumption that the input vector type and the shift type are exactly the same. This patch disables the folding of vector shift by immediate count if the input vector type and the shift value type are not the same. llvm-svn: 199213
* Revert "Decouple dllexport/dllimport from linkage"Nico Rieck2014-01-145-280/+20
| | | | | | | | Revert this for now until I fix an issue in Clang with it. This reverts commit r199204. llvm-svn: 199207
* Revert "Handle dllexport for global aliases"Nico Rieck2014-01-142-50/+0
| | | | | | This reverts commit r199205. llvm-svn: 199206
* Handle dllexport for global aliasesNico Rieck2014-01-142-0/+50
| | | | llvm-svn: 199205
* Decouple dllexport/dllimport from linkageNico Rieck2014-01-145-20/+280
| | | | | | | | | | | | | | | | | | | Representing dllexport/dllimport as distinct linkage types prevents using these attributes on templates and inline functions. Instead of introducing further mixed linkage types to include linkonce and weak ODR, the old import/export linkage types are replaced with a new separate visibility-like specifier: define available_externally dllimport void @f() {} @Var = dllexport global i32 1, align 4 Linkage for dllexported globals and functions is now equal to their linkage without dllexport. Imported globals and functions must be either declarations with external linkage, or definitions with AvailableExternallyLinkage. llvm-svn: 199204
* Fix llc to not reuse spill slots in functions that invoke setjmp()Mark Seaborn2014-01-141-0/+141
| | | | | | | | | | | | | | | | | | We need to ensure that StackSlotColoring.cpp does not reuse stack spill slots in functions that call "returns_twice" functions such as setjmp(), otherwise this can lead to miscompiled code, because a stack slot would be clobbered when it's still live. This was already handled correctly for functions that call setjmp() (though this wasn't covered by a test), but not for functions that invoke setjmp(). We fix this by changing callsFunctionThatReturnsTwice() to check for invoke instructions. This fixes PR18244. llvm-svn: 199180
* [DAG] Teach DAG to also reassociate vector operationsJuergen Ributzka2014-01-131-11/+20
| | | | | | | | | | This commit teaches DAG to reassociate vector ops, which in turn enables constant folding of vector op chains that appear later on during custom lowering and DAG combine. Reviewed by Andrea Di Biagio llvm-svn: 199135
* AVX-512: Embedded Rounding Control - encoding and printingElena Demikhovsky2014-01-132-30/+47
| | | | | | Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
* Make test independent of schedulingNico Rieck2014-01-121-1/+1
| | | | llvm-svn: 199055
* llvm/test/CodeGen/X86/shl_undef.ll: Tweak to satisfy r199050.NAKAMURA Takumi2014-01-121-1/+3
| | | | | | Use intel syntax, or "shl" might hit "pushl". llvm-svn: 199051
* Fix non-deterministic SDNodeOrder-dependent codegenNico Rieck2014-01-126-9/+9
| | | | | | | Reset SelectionDAGBuilder's SDNodeOrder to ensure deterministic code generation. llvm-svn: 199050
* Fix broken CHECK lines.Benjamin Kramer2014-01-113-10/+10
| | | | llvm-svn: 199016
* llvm/test/CodeGen/X86/anyregcc.ll: Add explicit -mtriple=x86_64-unknown-unknown.NAKAMURA Takumi2014-01-111-1/+1
| | | | | | XMM(s) are really spilling for targeting Win64. llvm-svn: 198999
* [anyregcc] Fix callee-save mask for anyregccJuergen Ributzka2014-01-111-0/+103
| | | | | | | Use separate callee-save masks for XMM and YMM registers for anyregcc on X86 and select the proper mask depending on the target cpu we compile for. llvm-svn: 198985
* llvm.experimental.stackmap: fix encoding of large constants.Andrew Trick2014-01-091-4/+46
| | | | | | | | | | In the stackmap format we advertise the constant field as signed. However, we were determining whether to promote to a 64-bit constant pool based on an unsigned comparison. This fix allows -1 to be encoded as a small constant. llvm-svn: 198816
* Conservatively handle multiple MMOs in MIsNeedChainEdgeHal Finkel2014-01-081-0/+37
| | | | | | | | | | | | | | | | | | | | | | MIsNeedChainEdge, which is used by -enable-aa-sched-mi (AA in misched), had an llvm_unreachable when -enable-aa-sched-mi is enabled and we reach an instruction with multiple MMOs. Instead, return a conservative answer. This allows testing -enable-aa-sched-mi on x86. Also, this moves the check above the isUnsafeMemoryObject checks. isUnsafeMemoryObject is currently correct only for instructions with one MMO (as noted in the comment in isUnsafeMemoryObject): // We purposefully do no check for hasOneMemOperand() here // in hope to trigger an assert downstream in order to // finish implementation. The problem with this is that, had the candidate edge passed the "!MIa->mayStore() && !MIb->mayStore()" check, the hoped-for assert would never happen (which could, in theory, lead to incorrect behavior if one of these secondary MMOs was volatile, for example). llvm-svn: 198795
* Teach the DAGCombiner how to fold 'vselect' dag nodes accordingAndrea Di Biagio2014-01-083-11/+52
| | | | | | | | to the following two rules: 1) fold (vselect (build_vector AllOnes), A, B) -> A 2) fold (vselect (build_vector AllZeros), A, B) -> B llvm-svn: 198777
* [x86] Disambiguate RET[QL] and fix aliases for 16-bit modeDavid Woodhouse2014-01-086-24/+24
| | | | | | | | | | I couldn't see how to do this sanely without splitting RETQ from RETL. Eric says: "sad about the inability to roundtrip them now, but...". I have no idea what that means, but perhaps it wants preserving in the commit comment. llvm-svn: 198756
* AVX-512: Added more intrinsics for pmin/pmax, pabs, blend, pmuldq.Elena Demikhovsky2014-01-081-71/+56
| | | | llvm-svn: 198745
* [patch] Adjust behavior of FDE cross-section relocs for targets that don't ↵Iain Sandoe2014-01-081-1/+47
| | | | | | | | | | | | support abs-differences. Modern versions of OSX/Darwin's ld (ld64 > 97.17) have an optimisation present that allows the back end to omit relocations (and replace them with an absolute difference) for FDE some text section refs. This patch allows a backend to opt-in to this behaviour by setting "DwarfFDESymbolsUseAbsDiff". At present, this is only enabled for modern x86 OSX ports. test changes by David Fang. llvm-svn: 198744
* Don't assert with private type info variables.Rafael Espindola2014-01-071-0/+21
| | | | | | | With the gnu objc runtime private strings are used. Since we only need to produce a unique label, the fix is to just drop the asserts. llvm-svn: 198701
* Fix for PR18396: Assertion: MO->isDead "Cannot fold physreg def".Andrew Trick2014-01-071-0/+48
| | | | | | InlineSpiller::foldMemoryOperand needs to handle undef call operands. llvm-svn: 198679
* AVX-512: added intrinsic vcvtpd2ps (with rounding mode and without)Elena Demikhovsky2014-01-061-0/+7
| | | | llvm-svn: 198593
* AVX-512: changed property name from "neverHasSideEffects=1" to ↵Elena Demikhovsky2014-01-051-1/+9
| | | | | | | | "hasSideEffects=0", added this property to VMOVSS/VMOVSD; Optimized a truncate pattern. llvm-svn: 198562
* AVX-512: Added more intrinsics for convert and min/max.Elena Demikhovsky2014-01-051-1/+67
| | | | | | Removed vzeroupper from AVX-512 mode - our optimization gude does not recommend to insert vzeroupper at all. llvm-svn: 198557
* [RegAlloc] Make tryInstructionSplit less aggressive.Quentin Colombet2014-01-021-0/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The greedy register allocator tries to split a live-range around each instruction where it is used or defined to relax the constraints on the entire live-range (this is a last chance split before falling back to spill). The goal is to have a big live-range that is unconstrained (i.e., that can use the largest legal register class) and several small local live-range that carry the constraints implied by each instruction. E.g., Let csti be the constraints on operation i. V1= op1 V1(cst1) op2 V1(cst2) V1 live-range is constrained on the intersection of cst1 and cst2. tryInstructionSplit relaxes those constraints by aggressively splitting each def/use point: V1= V2 = V1 V3 = V2 op1 V3(cst1) V4 = V2 op2 V4(cst2) Because of how the coalescer infrastructure works, each new variable (V3, V4) that is alive at the same time as V1 (or its copy, here V2) interfere with V1. Thus, we end up with an uncoalescable copy for each split point. To make tryInstructionSplit less aggressive, we check if the split point actually relaxes the constraints on the whole live-range. If it does not, we do not insert it. Indeed, it will not help the global allocation problem: - V1 will have the same constraints. - V1 will have the same interference + possibly the newly added split variable VS. - VS will produce an uncoalesceable copy if alive at the same time as V1. <rdar://problem/15570057> llvm-svn: 198369
* AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmpElena Demikhovsky2014-01-011-11/+39
| | | | | | | Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
* llvm/test/CodeGen/X86/vselect.ll: Unbreak Windows x64 targets to add ↵NAKAMURA Takumi2013-12-281-1/+1
| | | | | | -mtriple=x86_64-unknown-unknown. llvm-svn: 198114
OpenPOWER on IntegriCloud