summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG
Commit message (Collapse)AuthorAgeFilesLines
* [Layering] Move DebugLoc.h into the IR library. The implementationChandler Carruth2014-03-052-2/+2
| | | | | | | | | | | already lived there and it is where it belongs -- this is the in-memory debug location representation. This is just cleanup -- Modules can actually cope with this, but that doesn't make it right. After chatting with folks that have out-of-tree stuff, going ahead and moving the rest of the headers seems preferable. llvm-svn: 202960
* Make stackmap machineinstrs clobber the scratch regs too.Andrew Trick2014-03-051-4/+10
| | | | | | | | | | | | | Patchpoints already did this. Doing it for stackmaps is a convenience for the runtime in the event that it needs to scratch register to patch or perform a runtime call thunk. Unlike patchpoints, we just assume the AnyRegCC calling convention. This is the only language and target independent calling convention specific to stackmaps so makes sense. Although the calling convention is not currently used to select the scratch registers. llvm-svn: 202943
* Fix unused variable in FunctionLoweringInfo.cppHans Wennborg2014-03-051-1/+1
| | | | llvm-svn: 202932
* Check for dynamic allocas and inline asm that clobbers sp before buildingHans Wennborg2014-03-053-9/+45
| | | | | | | | | | | | | | | | | | | selection dag (PR19012) In X86SelectionDagInfo::EmitTargetCodeForMemcpy we check with MachineFrameInfo to make sure that ESI isn't used as a base pointer register before we choose to emit rep movs (which clobbers esi). The problem is that MachineFrameInfo wouldn't know about dynamic allocas or inline asm that clobbers the stack pointer until SelectionDAGBuilder has encountered them. This patch fixes the problem by checking for such things when building the FunctionLoweringInfo. Differential Revision: http://llvm-reviews.chandlerc.com/D2954 llvm-svn: 202930
* [DAGCombiner] Factor out distributeTruncateThroughAndAdam Nemet2014-03-041-47/+42
| | | | | | | | | Currently this code is duplicated across visitSHL, visitSRA and visitSRL. The plan is to add rotates as clients to this new function. There is no functional change intended here. llvm-svn: 202908
* [Modules] Move CallSite into the IR library where it belogs. It isChandler Carruth2014-03-041-1/+1
| | | | | | | abstracting between a CallInst and an InvokeInst, both of which are IR concepts. llvm-svn: 202816
* [C++11] Replace llvm::tie with std::tie.Benjamin Kramer2014-03-023-32/+32
| | | | | | The old implementation is no longer needed in C++11. llvm-svn: 202644
* [C++11] Replace llvm::next and llvm::prior with std::next and std::prev.Benjamin Kramer2014-03-028-19/+19
| | | | | | Remove the old functions. llvm-svn: 202636
* Now that we have C++11, turn simple functors into lambdas and remove a ton ↵Benjamin Kramer2014-03-011-21/+10
| | | | | | | | of boilerplate. No intended functionality change. llvm-svn: 202588
* Fix visitTRUNCATE for legal i1 valuesHal Finkel2014-02-281-1/+1
| | | | | | | | | | | This extract-and-trunc vector optimization cannot work for i1 values as currently implemented, and so I'm disabling this for now for i1 values. In the future, this can be fixed properly. Soon I'll commit support for i1 CR bit tracking in the PowerPC backend, and this will be covered by one of the existing regression tests. llvm-svn: 202449
* Add missing constMatt Arsenault2014-02-241-1/+1
| | | | llvm-svn: 202074
* Trivial code simplificationMatt Arsenault2014-02-241-2/+1
| | | | llvm-svn: 202073
* [DAGCombiner] PCMP* sets its result to all ones or zeros so we can AND with theQuentin Colombet2014-02-211-0/+18
| | | | | | | | | | | | | | shifted mask rather than masking and shifting separately. The patch adds this transformation to the DAGCombiner:   (shl (and (setcc:i8v16 ...) N01C) N1C) -> (and (setcc:i8v16 ...) N01C<<N1C) <rdar://problem/16054492> Patch by Adam Nemet <anemet@apple.com> llvm-svn: 201906
* Rename a few more DataLayout variables from TD to DL.Rafael Espindola2014-02-212-15/+15
| | | | llvm-svn: 201870
* Rename a DebugLoc variable to DbgLoc and a DataLayout to DL.Rafael Espindola2014-02-181-73/+73
| | | | | | This is quiet a bit less confusing now that TargetData was renamed DataLayout. llvm-svn: 201606
* Rename some member variables from TD to DL.Rafael Espindola2014-02-181-1/+1
| | | | | | TargetData was renamed DataLayout back in r165242. llvm-svn: 201581
* [DAG] Fix the recognition of opaque constants in the SelectionDAGBuilder.Juergen Ributzka2014-02-131-3/+7
| | | | | | | | | | | This fix checks the original LLVM IR node to identify opaque constants by looking for the bitcast-constant pattern. Originally we looked at the generated SDNode, but this might lead to incorrect results. The SDNode could have been generated by an constant expression that was folded to a constant. This fixes <rdar://problem/16050719> llvm-svn: 201291
* [Stackmaps] Improve the stackmap lowering code in the SelectionDAGBuilder.Juergen Ributzka2014-02-121-33/+33
| | | | | | | | | | We are now no longer relying on the target-specific call lowering implementation to lower a stackmap intrinsic call. Instead we perform the call lowering in a target-independent way directly in the stackmap lowering code. This simplifies the code and removes the need to fixup the code after the target-specific call lowering. llvm-svn: 201263
* [Stackmaps] Fix the ID type to be i64 also for stackmaps (as we claim in the ↵Juergen Ributzka2014-02-121-6/+7
| | | | | | | | | | | documenation) The ID type for the stackmap and patchpoint intrinsics are in both cases i64. This fixes an zero extend in the SelectionDAGBuilder that still used i32. This also updates the target independent instructions STACKMAP and PATCHPOINT to use the correct type. llvm-svn: 201262
* Teach the DAGCombiner how to fold concat_vector nodes when the input is twoRobert Lougher2014-02-111-0/+20
| | | | | | | | | | | | | BUILD_VECTOR nodes, e.g.: (concat_vectors (BUILD_VECTOR a1, a2, a3, a4), (BUILD_VECTOR b1, b2, b3, b4)) -> (BUILD_VECTOR a1, a2, a3, a4, b1, b2, b3, b4) This fixes an issue with AVX, where a sequence was not recognized as a 256-bit vbroadcast due to the concat_vectors. llvm-svn: 201158
* [DAG] Don't pull the binary operation though the shift if the operands have ↵Juergen Ributzka2014-02-061-2/+9
| | | | | | | | | | | | opaque constants. During DAGCombine visitShiftByConstant assumes that certain binary operations with only constant operands can always be folded successfully. This is no longer true when the constant is opaque. This commit fixes visitShiftByConstant by not performing the optimization for opaque constants. Otherwise we would end up in an infinite DAGCombine loop. llvm-svn: 200900
* Pass address space to allowsUnalignedMemoryAccessesMatt Arsenault2014-02-053-11/+25
| | | | llvm-svn: 200888
* Add address space argument to allowsUnalignedMemoryAccess.Matt Arsenault2014-02-051-1/+1
| | | | | | | On R600, some address spaces have more strict alignment requirements than others. llvm-svn: 200887
* Add CheckChildInteger to ISelMatcher operations. Removes nearly 2000 bytes ↵Craig Topper2014-02-051-2/+23
| | | | | | from X86 matcher table. llvm-svn: 200821
* Expand vector bswap in LegalizeVectorOpsHal Finkel2014-02-031-0/+1
| | | | | | | ISD::BSWAP was missing from the list of node types that should be expanded element-wise. llvm-svn: 200705
* Implement inalloca codegen for x86 with the new inalloca designReid Kleckner2014-01-312-2/+25
| | | | | | | | | | | | | | | | Calls with inalloca are lowered by skipping all stores for arguments passed in memory and the initial stack adjustment to allocate argument memory. Now the frontend is responsible for the memory layout, and the backend doesn't have to do any work. As a result these changes are pretty minimal. Reviewers: echristo Differential Revision: http://llvm-reviews.chandlerc.com/D2637 llvm-svn: 200596
* Don't put non-static allocas in the static alloca mapReid Kleckner2014-01-311-1/+7
| | | | | | | | Allocas marked inalloca are never static, but we were trying to put them into the static alloca map if they were in the entry block. Also add an assertion in x86 fastisel. llvm-svn: 200593
* This patch teaches the DAGCombiner how to fold insert_subvector nodesManman Ren2014-01-311-0/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | when the input is a concat_vectors and the insert replaces one of the concat halves: Lower half: fold (insert_subvector (concat_vectors X, Y), Z) -> (concat_vectors Z, Y) Upper half: fold (insert_subvector (concat_vectors X, Y), Z) -> (concat_vectors X, Z) This can be seen with the following IR: define <8 x float> @lower_half(<4 x float> %v1, <4 x float> %v2, <4 x float> %v3) { %1 = shufflevector <4 x float> %v1, <4 x float> %v2, <8 x i32> <i32 0, i32 1, i32 2, i32 3, i32 4, i32 5, i32 6, i32 7> %2 = tail call <8 x float> @llvm.x86.avx.vinsertf128.ps.256(<8 x float> %1, <4 x float> %v3, i8 0) The vinsertf128 intrinsic is converted into an insert_subvector node in SelectionDAGBuilder.cpp. Using AVX, without the patch this generates two vinsertf128 instructions: vinsertf128 $1, %xmm1, %ymm0, %ymm0 vinsertf128 $0, %xmm2, %ymm0, %ymm0 With the patch this is optimized into: vinsertf128 $1, %xmm1, %ymm2, %ymm0 Patch by Robert Lougher. llvm-svn: 200506
* DAGCombine should not produce ISD::OR nodes after operation legalization if ↵Owen Anderson2014-01-311-2/+4
| | | | | | they're not legal. llvm-svn: 200503
* PGO branch weight: update edge weights in SelectionDAGBuilder.Manman Ren2014-01-312-12/+65
| | | | | | | | | | | | | | | | When converting from "or + br" to two branches, or converting from "and + br" to two branches, we correctly update the edge weights of the two branches. The previous attempt at r200431 was reverted at r200434 because of two testing case failures. I modified my patch a little, but forgot to re-run "make check-all". Testing case CodeGen/ARM/lsr-unfolded-offset.ll is updated because of the patch's impact on branch probability which causes changes in spill placement. llvm-svn: 200502
* Revert r200431 due to bot failures.Manman Ren2014-01-302-65/+12
| | | | llvm-svn: 200434
* PGO branch weight: update edge weights in SelectionDAGBuilder.Manman Ren2014-01-302-12/+65
| | | | | | | | When converting from "or + br" to two branches, or converting from "and + br" to two branches, we correctly update the edge weights of the two branches. llvm-svn: 200431
* [DAGCombiner] Avoid introducing an illegal build_vector when folding a ↵Andrea Di Biagio2014-01-281-9/+15
| | | | | | | | | | | | | | | | | sign_extend. Make sure that we don't introduce illegal build_vector dag nodes when trying to fold a sign_extend of a build_vector. This fixes a regression introduced by r200234. Added test CodeGen/X86/fold-vector-sext-crash.ll to verify that llc no longer crashes with an assertion failure due to an illegal build_vector of type MVT::v4i64. Thanks to Ilia Filippov for spotting this regression and for providing a reproducible test case. llvm-svn: 200313
* [TLI] Add a new hook to TargetLowering to query the target if a load of a ↵Juergen Ributzka2014-01-282-11/+5
| | | | | | | | | | | | | | | | | constant should be converted to simply the constant itself. Before this patch we used getIntImmCost from TargetTransformInfo to determine if a load of a constant should be converted to just a constant, but the threshold for this was set to an arbitrary value. This value works well for the two targets (X86 and ARM) that implement this target-hook, but it isn't target-independent at all. Now targets have the possibility to decide directly if this optimization should be performed. The default value is set to false to preserve the current behavior. The target hook has been moved to TargetLowering, which removed the last use and need of TargetTransformInfo in SelectionDAG. llvm-svn: 200271
* Fix sext(setcc) -> select_cc using wrong type for setcc.Matt Arsenault2014-01-271-10/+16
| | | | | | | | | | | | | | | | Also update the comment, since it actually produces a select (setcc) instead of select_cc. It was checking and using the setcc result type for the type of the sext, instead of the type of the compared items. In my problem case, the sext was to i32 and was used as the setcc type, but the expected type was i64. No test since I haven't been able to hit the problem with this on any in-tree targets. llvm-svn: 200249
* [DAGCombiner] Teach how to fold sext/aext/zext of constant build vectors.Andrea Di Biagio2014-01-271-9/+64
| | | | | | | | | | | | | This patch teaches the DAGCombiner how to fold a sext/aext/zext dag node when the operand in input is a build vector of constants (or UNDEFs). The inability to fold a sext/zext of a constant build_vector was the root cause of some pcg bugs affecting vselect expansion on x86-64 with AVX support. Before this change, the DAGCombiner only knew how to fold a sext/zext/aext of a ConstantSDNode. llvm-svn: 200234
* Fix for PR18102.Stepan Dyatkovskiy2014-01-271-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Issue outcomes from DAGCombiner::MergeConsequtiveStores, more precisely from mem-ops sequence sorting. Consider, how MergeConsequtiveStores works for next example: store i8 1, a[0] store i8 2, a[1] store i8 3, a[1] ; a[1] again. return ; DAG starts here 1. Method will collect all the 3 stores. 2. It sorts them by distance from the base pointer (farthest with highest index). 3. It takes first consecutive non-overlapping stores and (if possible) replaces them with a single store instruction. The point is, we can't determine here which 'store' instruction would be the second after sorting ('store 2' or 'store 3'). It happens that 'store 3' would be the second, and 'store 2' would be the third. So after merging we have the next result: store i16 (1 | 3 << 8), base ; is a[0] but bit-casted to i16 store i8 2, a[1] So actually we swapped 'store 3' and 'store 2' and got wrong contents in a[1]. Fix: In sort routine just also take into account mem-op sequence number. llvm-svn: 200201
* [AArch64 NEON] Fix pattern match failed on FP_ROUND from v1f128 to v1f64.Kevin Qin2014-01-262-0/+13
| | | | llvm-svn: 200109
* Disable the use of TBAA when using AA in CodeGenHal Finkel2014-01-251-2/+14
| | | | | | | | | | | | | | | | | There are currently two issues, of which I currently know, that prevent TBAA from being correctly usable in CodeGen: 1. Stack coloring does not update TBAA when merging allocas. This is easy enough to fix, but is not the largest problem. 2. CGP inserts ptrtoint/inttoptr pairs when sinking address computations. Because BasicAA does not handle inttoptr, we'll often miss basic type punning idioms that we need to catch so we don't miscompile real-world code (like LLVM). I don't yet have a small test case for this, but this fixes self hosting a non-asserts build of LLVM on PPC64 when using -enable-aa-sched-mi and -misched=shuffle. llvm-svn: 200093
* Add combiner-aa-only-func (debug only)Hal Finkel2014-01-251-0/+22
| | | | | | | | | This option (which is !NDEBUG only) allows restricting the use of alias analysis in DAGCombiner to a specific function. This has proved extremely valuable to isolating bugs related to this feature, and mirrors the misched-only-func option provided by the new instruction scheduler. llvm-svn: 200088
* Improve descriptions of combiner-alias-analysis and ↵Hal Finkel2014-01-251-2/+2
| | | | | | combiner-global-alias-analysis llvm-svn: 200087
* Revert "Revert "Add Constant Hoisting Pass" (r200034)"Juergen Ributzka2014-01-255-25/+56
| | | | | | | This reverts commit r200058 and adds the using directive for ARMTargetTransformInfo to silence two g++ overload warnings. llvm-svn: 200062
* Revert "Add Constant Hoisting Pass" (r200034)Hans Wennborg2014-01-255-56/+25
| | | | | | | | | | | | | | | This commit caused -Woverloaded-virtual warnings. The two new TargetTransformInfo::getIntImmCost functions were only added to the superclass, and to the X86 subclass. The other targets were not updated, and the warning highlighted this by pointing out that e.g. ARMTTI::getIntImmCost was hiding the two new getIntImmCost variants. We could pacify the warning by adding "using TargetTransformInfo::getIntImmCost" to the various subclasses, or turning it off, but I suspect that it's wrong to leave the functions unimplemnted in those targets. The default implementations return TCC_Free, which I don't think is right e.g. for ARM. llvm-svn: 200058
* Add Constant Hoisting PassJuergen Ributzka2014-01-245-25/+56
| | | | | | | | Retry commit r200022 with a fix for the build bot errors. Constant expressions have (unlike instructions) module scope use lists and therefore may have users in different functions. The fix is to simply ignore these out-of-function uses. llvm-svn: 200034
* Fix DAGCombiner::GatherAllAliases to account for non-chain dependenciesHal Finkel2014-01-241-1/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | DAGCombiner::GatherAllAliases, which is only used when AA used is enabled during DAGCombine, had a fundamentally incorrect assumption for which this change compensates. GatherAllAliases, which is used to find aliasing predecessor chain nodes (so that a better chain can be selected for a load or store to enable subsequent optimizations) assumed that walking up the chain would always catch all possibly-aliasing loads and stores. This is not true: To really find all aliases, we also need to search for aliases through the value operand of a store, etc. Consider the following situation: Token1 = ... L1 = load Token1, %52 S1 = store Token1, L1, %51 L2 = load Token1, %52+8 S2 = store Token1, L2, %51+8 Token2 = Token(S1, S2) L3 = load Token2, %53 S3 = store Token2, L3, %52 L4 = load Token2, %53+8 S4 = store Token2, L4, %52+8 If we search for aliases of S3 (which loads address %52), and we look only through the chain, then we'll miss the trivial dependence on L1 (which loads from %52). We then might change all loads and stores to use Token1 as their chain operand, which could result in copying %53 into %52 before copying %52 into %51 (which should happen first). The problem is, however, that searching for such data dependencies can become expensive, and the cost is not directly related to the chain depth. Instead, we'll rule out such configurations by insisting that we've visited all chain users (except for users of the original chain, which is not necessary). When doing this, we need to look through nodes we don't care about (otherwise, things like register copies will interfere with trivial use cases). Unfortunately, I don't have a small test case for this problem. Creating the underlying situation is not hard (a pair of memcpys will do it), but arranging for the default instruction schedule to be incorrect is very fragile. This unbreaks self hosting on PPC64 when using -mllvm -combiner-global-alias-analysis -mllvm -combiner-alias-analysis. llvm-svn: 200033
* Revert "Add Constant Hoisting Pass"Juergen Ributzka2014-01-245-56/+25
| | | | | | This reverts commit r200022 to unbreak the build bots. llvm-svn: 200024
* Restrict FindBetterChain DAG combines to unindexed nodesHal Finkel2014-01-241-2/+2
| | | | | | | | | | | | These transformations obviously won't work for indexed (pre/post-inc) loads and stores. In practice, I'm not sure there is any benefit to enabling them for indexed nodes because other transformations that these might enable likely also won't handle indexed nodes. I don't have an in-tree test case that hits this problem, but an upcoming bug fix will make it much more likely. llvm-svn: 200023
* Add Constant Hoisting PassJuergen Ributzka2014-01-245-25/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This pass identifies expensive constants to hoist and coalesces them to better prepare it for SelectionDAG-based code generation. This works around the limitations of the basic-block-at-a-time approach. First it scans all instructions for integer constants and calculates its cost. If the constant can be folded into the instruction (the cost is TCC_Free) or the cost is just a simple operation (TCC_BASIC), then we don't consider it expensive and leave it alone. This is the default behavior and the default implementation of getIntImmCost will always return TCC_Free. If the cost is more than TCC_BASIC, then the integer constant can't be folded into the instruction and it might be beneficial to hoist the constant. Similar constants are coalesced to reduce register pressure and materialization code. When a constant is hoisted, it is also hidden behind a bitcast to force it to be live-out of the basic block. Otherwise the constant would be just duplicated and each basic block would have its own copy in the SelectionDAG. The SelectionDAG recognizes such constants as opaque and doesn't perform certain transformations on them, which would create a new expensive constant. This optimization is only applied to integer constants in instructions and simple (this means not nested) constant cast experessions. For example: %0 = load i64* inttoptr (i64 big_constant to i64*) Reviewed by Eric llvm-svn: 200022
* Fix known typosAlp Toker2014-01-241-3/+3
| | | | | | | Sweep the codebase for common typos. Includes some changes to visible function names that were misspelt. llvm-svn: 200018
* Revert r162101 and replace it with a solution that works for targets where ↵Owen Anderson2014-01-222-7/+11
| | | | | | | | | | the pointer type is illegal. This is a horrible bit of code. We're calling a simplification routine *in the middle* of type legalization. We tell the simplification routine that it's running after legalization, but some of the types it will encounter will be illegal! The fix is only to invoke the simplification if the types in question were legal, so that none of its invariants will be violated. llvm-svn: 199847
OpenPOWER on IntegriCloud