summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG
Commit message (Collapse)AuthorAgeFilesLines
* Reduce double set lookups.Benjamin Kramer2015-02-271-2/+1
| | | | llvm-svn: 230798
* Change the fast-isel-abort option from bool to int to enable "levels"Mehdi Amini2015-02-271-24/+24
| | | | | | | | | | | | | | | | | | | | | | | Summary: Currently fast-isel-abort will only abort for regular instructions, and just warn for function calls, terminators, function arguments. There is already fast-isel-abort-args but nothing for calls and terminators. This change turns the fast-isel-abort options into an integer option, so that multiple levels of strictness can be defined. This will help no being surprised when the "abort" option indeed does not abort, and enables the possibility to write test that verifies that no intrinsics are forgotten by fast-isel. Reviewers: resistor, echristo Subscribers: jfb, llvm-commits Differential Revision: http://reviews.llvm.org/D7941 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 230775
* getRegForInlineAsmConstraint wants to use TargetRegisterInfo forEric Christopher2015-02-263-27/+30
| | | | | | | | | a lookup, pass that in rather than use a naked call to getSubtargetImpl. This involved passing down and around either a TargetMachine or TargetRegisterInfo. Update all callers/definitions around the targets and SelectionDAG. llvm-svn: 230699
* When the source has a series of assignments, users reasonably want toPaul Robinson2015-02-261-0/+3
| | | | | | | | | | | | have the debugger step through each one individually. Turn off the combine for adjacent stores at -O0 so we get this behavior. Possibly, DAGCombine shouldn't run at all at -O0, but that's for another day; see PR22346. Differential Revision: http://reviews.llvm.org/D7181 llvm-svn: 230659
* Reapplied D7816 & rL230177 & rL230278 - with an additional fix toensure that ↵Simon Pilgrim2015-02-241-45/+65
| | | | | | the smallest build vector input scalar type is always used. Additional (crash) test cases already committed. llvm-svn: 230388
* Revert:Eric Christopher2015-02-241-39/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Author: Simon Pilgrim <llvm-dev@redking.me.uk> Date: Mon Feb 23 23:04:28 2015 +0000 Fix based on post-commit comment on D7816 & rL230177 - BUILD_VECTOR operand truncation was using the the BV's output scalar type instead of the input type. and Author: Simon Pilgrim <llvm-dev@redking.me.uk> Date: Sun Feb 22 18:17:28 2015 +0000 [DagCombiner] Generalized BuildVector Vector Concatenation The CONCAT_VECTORS combiner pass can transform the concat of two BUILD_VECTOR nodes into a single BUILD_VECTOR node. This patch generalises this to support any number of BUILD_VECTOR nodes, and also permits UNDEF nodes to be included as well. This was noticed as AVX vec128 -> vec256 canonicalization sometimes creates a CONCAT_VECTOR with a real vec128 lower and an vec128 UNDEF upper. Differential Revision: http://reviews.llvm.org/D7816 as the root cause of PR22678 which is causing an assertion inside the DAG combiner. I'll follow up to the main thread as well. llvm-svn: 230358
* DAGCombiner: Move variable definitions closer to use; NFCMatthias Braun2015-02-241-38/+38
| | | | llvm-svn: 230354
* DAGCombiner: Move variable declaration closer to definiion; NFCMatthias Braun2015-02-241-3/+3
| | | | llvm-svn: 230353
* ARM: treat [N x i32] and [N x i64] as AAPCS composite typesTim Northover2015-02-241-8/+7
| | | | | | | | | | | The logic is almost there already, with our special homogeneous aggregate handling. Tweaking it like this allows front-ends to emit AAPCS compliant code without ever having to count registers or add discarded padding arguments. Only arrays of i32 and i64 are needed to model AAPCS rules, but I decided to apply the logic to all integer arrays for more consistency. llvm-svn: 230348
* [SDAG] Handle LowerOperation returning its input consistentlyHal Finkel2015-02-242-5/+9
| | | | | | | | | | | | | | | | | | | For almost all node types, if the target requested custom lowering, and LowerOperation returned its input, we'd treat the original node as legal. This did not work, however, for many loads and stores, because they follow slightly different code paths, and we did not account for the possibility of LowerOperation returning its input at those call sites. I think that we now handle this consistently everywhere. At the call sites in LegalizeDAG, we used to assert in this case, so there's no functional change for any existing code there. For the call sites in LegalizeVectorOps, this really only affects whether or not we set Changed = true, but I think makes the semantics clearer. No test case here, but it will be covered by an upcoming PowerPC commit adding QPX support. llvm-svn: 230332
* Fix based on post-commit comment on D7816 & rL230177 - BUILD_VECTOR operand ↵Simon Pilgrim2015-02-231-4/+5
| | | | | | truncation was using the the BV's output scalar type instead of the input type. llvm-svn: 230278
* [X86] Teach how to custom lower double-to-half conversions under fast-math.Andrea Di Biagio2015-02-231-0/+15
| | | | | | | | | | | | | This patch teaches the backend how to expand a double-half conversion into a double-float conversion immediately followed by a float-half conversion. We do this only under fast-math, and if float-half conversions are legal for the target. Added test CodeGen/X86/fastmath-float-half-conversion.ll Differential Revision: http://reviews.llvm.org/D7832 llvm-svn: 230276
* [DagCombiner] Generalized BuildVector Vector ConcatenationSimon Pilgrim2015-02-221-23/+38
| | | | | | | | | | | | The CONCAT_VECTORS combiner pass can transform the concat of two BUILD_VECTOR nodes into a single BUILD_VECTOR node. This patch generalises this to support any number of BUILD_VECTOR nodes, and also permits UNDEF nodes to be included as well. This was noticed as AVX vec128 -> vec256 canonicalization sometimes creates a CONCAT_VECTOR with a real vec128 lower and an vec128 UNDEF upper. Differential Revision: http://reviews.llvm.org/D7816 llvm-svn: 230177
* [DAGCombine] Don't assume integer-type legailty in ↵Hal Finkel2015-02-221-0/+5
| | | | | | | | | | | | | | | | | | reduceBuildVecConvertToConvertBuildVec DAGCombine will rewrite an BUILD_VECTOR where all non-undef inputs some from [US]INT_TO_FP, as a BUILD_VECTOR of integers with the conversion applied as a vector operation. We check operation legality of the conversion, but fail to check legality of the integer vector type itself. Because targets don't normally override operation legality defaults for illegal types, we need to check this also. This came up in the context of the QPX vector entensions for PowerPC (which can have legal floating-point vector types without corresponding legal integer vector types). No in-tree test case for this yes, but one can be added once the QPX support has been committed. llvm-svn: 230176
* [SDAG] Use correct alignments on expanded vector trunc-store/ext-loadsHal Finkel2015-02-221-4/+7
| | | | | | | | | | | | | When expanding a truncating store or extending load using vector extracts or inserts and scalar stores and loads, we were giving each of these scalar stores or loads the same alignment as the original vector operation. While this will often be right (most vector operations, especially those produced by autovectorization, have the alignment of the underlying scalar type), the vector operation could certainly have a larger alignment. No test case (yet); noticed by inspection. llvm-svn: 230175
* X86: Call __main using the SelectionDAGDavid Majnemer2015-02-211-1/+0
| | | | | | | | Synthesizing a call directly using the MI layer would confuse the frame lowering code. This is problematic as frame lowering is highly sensitive the particularities of calls, etc. llvm-svn: 230129
* Add generic fmad DAG node.Matt Arsenault2015-02-203-95/+163
| | | | | | | | | | | This allows sharing of FMA forming combines to work with instructions that have the same semantics as a separate multiply and add. This is expand by default, and only formed post legalization so it shouldn't have much impact on targets that do not want it. llvm-svn: 230070
* Generalize statepoint lowering to use ImmutableStatepoint. Move statepoint ↵Igor Laevsky2015-02-202-40/+41
| | | | | | | | lowering into a separate function 'LowerStatepoint' which uses ImmutableStatepoint instead of a CallInst. Also related utility functions are changed to receive ImmutableCallSite. Differential Revision: http://reviews.llvm.org/D7756 llvm-svn: 230017
* [CodeGen] Use ArrayRef instead of std::vector&. NFC.Ahmed Bougacha2015-02-191-1/+1
| | | | | | The former lets us use SmallVectors. Do so in ARM and AArch64. llvm-svn: 229925
* Demote vectors to arrays. No functionality change.Benjamin Kramer2015-02-191-7/+3
| | | | llvm-svn: 229861
* [x86,sdag] Two interrelated changes to the x86 and sdag code.Chandler Carruth2015-02-191-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | First, don't combine bit masking into vector shuffles (even ones the target can handle) once operation legalization has taken place. Custom legalization of vector shuffles may exist for these patterns (making the predicate return true) but that custom legalization may in some cases produce the exact bit math this matches. We only really want to handle this prior to operation legalization. However, the x86 backend, in a fit of awesome, relied on this. What it would do is mark VSELECTs as expand, which would turn them into arithmetic, which this would then match back into vector shuffles, which we would then lower properly. Amazing. Instead, the second change is to teach the x86 backend to directly form vector shuffles from VSELECT nodes with constant conditions, and to mark all of the vector types we support lowering blends as shuffles as custom VSELECT lowering. We still mark the forms which actually support variable blends as *legal* so that the custom lowering is bypassed, and the legal lowering can even be used by the vector shuffle legalization (yes, i know, this is confusing. but that's how the patterns are written). This makes the VSELECT lowering much more sensible, and in fact should fix a bunch of bugs with it. However, as you'll see in the test cases, right now what it does is point out the *hilarious* deficiency of the new vector shuffle lowering when it comes to blends. Fortunately, my very next patch fixes that. I can't submit it yet, because that patch, somewhat obviously, forms the exact and/or pattern that the DAG combine is matching here! Without this patch, teaching the vector shuffle lowering to produce the right code infloops in the DAG combiner. With this patch alone, we produce terrible code but at least lower through the right paths. With both patches, all the regressions here should be fixed, and a bunch of the improvements (like using 2 shufps with no memory loads instead of 2 andps with memory loads and an orps) will stay. Win! There is one other change worth noting here. We had hilariously wrong vectorization cost estimates for vselect because we fell through to the code path that assumed all "expand" vector operations are scalarized. However, the "expand" lowering of VSELECT is vector bit math, most definitely not scalarized. So now we go back to the correct if horribly naive cost of "1" for "not scalarized". If anyone wants to add actual modeling of shuffle costs, that would be cool, but this seems an improvement on its own. Note the removal of 16 and 32 "costs" for doing a blend. Even in SSE2 we can blend in fewer than 16 instructions. ;] Of course, we don't right now because of OMG bad code, but I'm going to fix that. Next patch. I promise. llvm-svn: 229835
* Fixes two issue in SimplifyDemandedBits of sext_in_reg:Michael Kuperstein2015-02-181-11/+18
| | | | | | | | | | | 1) We should not try to simplify if the sext has multiple uses 2) There is no need to simplify is the source value is already sign-extended. Patch by Gil Rapaport <gil.rapaport@intel.com> Differential Revision: http://reviews.llvm.org/D6949 llvm-svn: 229659
* Canonicalize splats as build_vectors (PR22283)Sanjay Patel2015-02-172-28/+22
| | | | | | | | | | | | | | | | | | | | | | This is a follow-on patch to: http://reviews.llvm.org/D7093 That patch canonicalized constant splats as build_vectors, and this patch removes the constant check so we can canonicalize all splats as build_vectors. This fixes the 2nd test case in PR22283: http://llvm.org/bugs/show_bug.cgi?id=22283 The unfortunate code duplication between SelectionDAG and DAGCombiner is discussed in the earlier patch review. At least this patch is just removing code... This improves an existing x86 AVX test and changes codegen in an ARM test. Differential Revision: http://reviews.llvm.org/D7389 llvm-svn: 229511
* Prefer SmallVector::append/insert over push_back loops.Benjamin Kramer2015-02-175-32/+11
| | | | | | Same functionality, but hoists the vector growth out of the loop. llvm-svn: 229500
* SelectionDAG: fold (fp_to_u/sint (s/uint_to_fp)) here tooMehdi Amini2015-02-161-2/+46
| | | | | | | Update SPARC tests to match. From: Fiona Glaser <fglaser@apple.com> llvm-svn: 229438
* AArch64: Safely handle the incoming sret call argument.Andrew Trick2015-02-161-1/+2
| | | | | | | | | | | | | | | | | | This adds a safe interface to the machine independent InputArg struct for accessing the index of the original (IR-level) argument. When a non-native return type is lowered, we generate the hidden machine-level sret argument on-the-fly. Before this fix, we were representing this argument as OrigArgIndex == 0, which is an outright lie. In particular this crashed in the AArch64 backend where we actually try to access the type of the original argument. Now we use a sentinel value for machine arguments that have no original argument index. AArch64, ARM, Mips, and PPC now check for this case before accessing the original argument. Fixes <rdar://19792160> Null pointer assertion in AArch64TargetLowering llvm-svn: 229413
* [SDAG] Teach the SelectionDAG to canonicalize vector shuffles of splatsChandler Carruth2015-02-151-0/+28
| | | | | | | | | | | | | | | directly into blends of the splats. These patterns show up even very late in the vector shuffle lowering where we don't have any chance for DAG combining to kick in, and blending is a tremendously simpler operation to model. By coercing the shuffle into a blend we can much more easily match and lower shuffles of splats. Immediately with this change there are significantly more blends being matched in the x86 vector shuffle lowering. llvm-svn: 229308
* [x86] Fix PR22377, a regression with the new vector shuffle legalityChandler Carruth2015-02-151-2/+3
| | | | | | | | | | | test. This was just a matter of the DAG combine for vector shuffles being too aggressive. This is a bit of a grey area, but I think generally if we can re-use intermediate shuffles, we should. Certainly, given the test cases I have available, this seems like the right call. llvm-svn: 229285
* CodeGen: Canonicalize access to function attributes, NFCDuncan P. N. Exon Smith2015-02-143-21/+11
| | | | | | | | | | | | | | | | | Canonicalize access to function attributes to use the simpler API. getAttributes().getAttribute(AttributeSet::FunctionIndex, Kind) => getFnAttribute(Kind) getAttributes().hasAttribute(AttributeSet::FunctionIndex, Kind) => hasFnAttribute(Kind) Also, add `Function::getFnStackAlignment()`, and canonicalize: getAttributes().getStackAlignment(AttributeSet::FunctionIndex) => getFnStackAlignment() llvm-svn: 229208
* Unify the two EH personality classification routines I wroteReid Kleckner2015-02-141-1/+1
| | | | | | We only need one. llvm-svn: 229193
* Re-sort #include lines using my handy dandy ./utils/sort_includes.pyChandler Carruth2015-02-132-2/+2
| | | | | | script. This is in preparation for changes to lots of include lines. llvm-svn: 229088
* [SDAG] Don't try to use FP_EXTEND/FP_ROUND for int<->fp promotionsHal Finkel2015-02-121-3/+5
| | | | | | | | | | The PowerPC backend has long promoted some floating-point vector operations (such as select) to integer vector operations. Unfortunately, this behavior was broken by r216555. When using FP_EXTEND/FP_ROUND for promotions, we must check that both the old and new types are floating-point types. Otherwise, we must use BITCAST as we did prior to r216555 for everything. llvm-svn: 228969
* MathExtras: Bring Count(Trailing|Leading)Ones and CountPopulation in line ↵Benjamin Kramer2015-02-122-4/+4
| | | | | | | | with countTrailingZeros Update all callers. llvm-svn: 228930
* [CodeGen] Don't blindly combine (fp_round (fp_round x)) to (fp_round x).Ahmed Bougacha2015-02-121-5/+10
| | | | | | | | | | | | We used to do this DAG combine, but it's not always correct: If the first fp_round isn't a value preserving truncation, it might introduce a tie in the second fp_round, that wouldn't occur in the single-step fp_round we want to fold to. In other words, double rounding isn't the same as rounding. Differential Revision: http://reviews.llvm.org/D7571 llvm-svn: 228911
* Fix SelectionDAG compile time issue with alias analysis.Jonas Paulsson2015-02-111-2/+5
| | | | | | | | | | | Add new token factor node and its users to worklist if alias analysis is turned on, in DAGCombiner::visitTokenFactor(). Alias analysis may cause a lot of new token factors to be inserted into the DAG, and they need to be optimized to avoid significant slow-downs. Reviewed by Hal Finkel. llvm-svn: 228841
* Fix makeLibCall argument (signed) in SoftenFloatRes_XINT_TO_FP functionPetar Jovanovic2015-02-101-1/+1
| | | | | | | | | | | | | | The isSigned argument of makeLibCall function was hard-coded to false (unsigned). This caused zero extension on MIPS64 soft float. As the result SingleSource/Benchmarks/Stanford/FloatMM test and SingleSource/UnitTests/2005-07-17-INT-To-FP test failed. The solution was to use the proper argument. Patch by Strahinja Petrovic. Differential Revision: http://reviews.llvm.org/D7292 llvm-svn: 228765
* Adding support for llvm.eh.begincatch and llvm.eh.endcatch intrinsics and ↵Andrew Kaylor2015-02-101-0/+3
| | | | | | | | beginning the documentation of native Windows exception handling. Differential Revision: http://reviews.llvm.org/D7398 llvm-svn: 228733
* Two comment typo fixes in lib/CodeGen/SelectionDAG/DAGCombiner.cpp.Jonas Paulsson2015-02-101-2/+2
| | | | llvm-svn: 228700
* [x86] Fix PR22524: the DAG combiner was incorrectly handling illegalChandler Carruth2015-02-101-13/+9
| | | | | | | | | | | | | | | | | | | | nodes when folding bitcasts of constants. We can't fold things and then check after-the-fact whether it was legal. Once we have formed the DAG node, arbitrary other nodes may have been collapsed to it. There is no easy way to go back. Instead, we need to test for the specific folding cases we're interested in and ensure those are legal first. This could in theory make this less powerful for bitcasting from an integer to some vector type, but AFAICT, that can't actually happen in the SDAG so its fine. Now, we *only* whitelist specific int->fp and fp->int bitcasts for post-legalization folding. I've added the test case from the PR. (Also as a note, this does not appear to be in 3.6, no backport needed) llvm-svn: 228656
* [CodeGen] Add hook/combine to form vector extloads, enabled on X86.Ahmed Bougacha2015-02-051-12/+121
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The combine that forms extloads used to be disabled on vector types, because "None of the supported targets knows how to perform load and sign extend on vectors in one instruction." That's not entirely true, since at least SSE4.1 X86 knows how to do those sextloads/zextloads (with PMOVS/ZX). But there are several aspects to getting this right. First, vector extloads are controlled by a profitability callback. For instance, on ARM, several instructions have folded extload forms, so it's not always beneficial to create an extload node (and trying to match extloads is a whole 'nother can of worms). The interesting optimization enables folding of s/zextloads to illegal (splittable) vector types, expanding them into smaller legal extloads. It's not ideal (it introduces some legalization-like behavior in the combine) but it's better than the obvious alternative: form illegal extloads, and later try to split them up. If you do that, you might generate extloads that can't be split up, but have a valid ext+load expansion. At vector-op legalization time, it's too late to generate this kind of code, so you end up forced to scalarize. It's better to just avoid creating egregiously illegal nodes. This optimization is enabled unconditionally on X86. Note that the splitting combine is happy with "custom" extloads. As is, this bypasses the actual custom lowering, and just unrolls the extload. But from what I've seen, this is still much better than the current custom lowering, which does some kind of unrolling at the end anyway (see for instance load_sext_4i8_to_4i64 on SSE2, and the added FIXME). Also note that the existing combine that forms extloads is now also enabled on legal vectors. This doesn't have a big effect on X86 (because sext+load is usually combined to sext_inreg+aextload). On ARM it fires on some rare occasions; that's for a separate commit. Differential Revision: http://reviews.llvm.org/D6904 llvm-svn: 228325
* Fixes a bug in vector load legalization that confused bits and bytes.Michael Kuperstein2015-02-041-3/+3
| | | | | | Differential Revision: http://reviews.llvm.org/D7400 llvm-svn: 228168
* Add nullptr checks for TargetSelectionDAGInfo in SelectionDAG.Manuel Jacob2015-01-281-13/+19
| | | | | | TSI is not guaranteed be non-null in SelectionDAG. llvm-svn: 227397
* Revert r227242 - Merge vector stores into wider vector stores (PR21711).Quentin Colombet2015-01-271-54/+30
| | | | | | | This commit creates infinite loop in DAG combine for in the LLVM test-suite for aarch64 with mcpu=cylcone (just having neon may be enough to expose this). llvm-svn: 227272
* Merge vector stores into wider vector stores (PR21711)Sanjay Patel2015-01-271-30/+54
| | | | | | | | | | | | | | | | | | | | This patch resolves part of PR21711 ( http://llvm.org/bugs/show_bug.cgi?id=21711 ). The 'f3' test case in that report presents a situation where we have two 128-bit stores extracted from a 256-bit source vector. Instead of producing this: vmovaps %xmm0, (%rdi) vextractf128 $1, %ymm0, 16(%rdi) This patch merges the 128-bit stores into a single 256-bit store: vmovups %ymm0, (%rdi) Differential Revision: http://reviews.llvm.org/D7208 llvm-svn: 227242
* Add a FIXME in SelectionDAGBuilder before an assert that is valid only on X86.Manuel Jacob2015-01-271-0/+3
| | | | | | | | | When lowering memcpy, memset or memmove, this assert checks whether the pointer operands are in an address space < 256 which means "user defined address space" on X86. However, this notion of "user defined address space" does not exist for other targets. llvm-svn: 227191
* Grab the TargetLowering info from the DAG rather than querying forEric Christopher2015-01-271-3/+2
| | | | | | a subtarget. llvm-svn: 227156
* [SelectionDAG] Fix assert message copypasta. NFC.Ahmed Bougacha2015-01-261-2/+2
| | | | llvm-svn: 227119
* Move DataLayout back to the TargetMachine from TargetSubtargetInfoEric Christopher2015-01-263-4/+4
| | | | | | | | | | | | | | | | | | | derived classes. Since global data alignment, layout, and mangling is often based on the DataLayout, move it to the TargetMachine. This ensures that global data is going to be layed out and mangled consistently if the subtarget changes on a per function basis. Prior to this all targets(*) have had subtarget dependent code moved out and onto the TargetMachine. *One target hasn't been migrated as part of this change: R600. The R600 port has, as a subtarget feature, the size of pointers and this affects global data layout. I've currently hacked in a FIXME to enable progress, but the port needs to be updated to either pass the 64-bitness to the TargetMachine, or fix the DataLayout to avoid subtarget dependent features. llvm-svn: 227113
* Revert GCStrategy ownership changesPhilip Reames2015-01-263-3/+3
| | | | | | | | | | | | This change reverts the interesting parts of 226311 (and 227046). This change introduced two problems, and I've been convinced that an alternate approach is preferrable anyways. The bugs were: - Registery appears to require all users be within the same linkage unit. After this change, asking for "statepoint-example" in Transform/ would sometimes get you nullptr, whereas asking the same question in CodeGen would return the right GCStrategy. The correct long term fix is to get rid of the utter hack which is Registry, but I don't have time for that right now. 227046 appears to have been an attempt to fix this, but I don't believe it does so completely. - GCMetadataPrinter::finishAssembly was being called more than once per GCStrategy. Each Strategy was being added to the GCModuleInfo multiple times. Once I get time again, I'm going to split GCModuleInfo into the gc.root specific part and a GCStrategy owning Analysis pass. I'm probably also going to kill off the Registry. Once that's done, I'll move the new GCStrategyAnalysis and all built in GCStrategies into Analysis. (As original suggested by Chandler.) This will accomplish my original goal of being able to access GCStrategy from Transform/ without adding all of the builtin GCs to IR/. llvm-svn: 227109
* [DAG] Fix wrong canonicalization performed on shuffle nodes.Andrea Di Biagio2015-01-241-7/+9
| | | | | | | | | | | This fixes a regression introduced by r226816. When replacing a splat shuffle node with a constant build_vector, make sure that the new build_vector has a valid number of elements. Thanks to Patrik Hagglund for reporting this problem and providing a small reproducible. llvm-svn: 227002
OpenPOWER on IntegriCloud