summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
Commit message (Collapse)AuthorAgeFilesLines
...
* [DAGCombiner] Added CTTZ vector constant folding support.Simon Pilgrim2015-06-081-0/+2
| | | | llvm-svn: 239293
* [DAGCombiner] Added CTPOP vector constant folding support.Simon Pilgrim2015-06-071-1/+2
| | | | | | Added tests to the existing SSE/AVX test files. llvm-svn: 239252
* [sdag] Add the helper I most want to the DAG -- building a bitcastChandler Carruth2015-05-301-0/+7
| | | | | | | | around a value using its existing SDLoc. Start using this in just one function to save omg lines of code. llvm-svn: 238638
* SDAG: Give SDDbgValues their own allocator (and reset it)Duncan P. N. Exon Smith2015-05-221-3/+4
| | | | | | | | | | | | Previously `SDDbgValue`s used the general allocator that lives for all of `SelectionDAG`. Instead, give them their own allocator, and reset it whenever `SDDbgInfo::clear()` is called, plugging a spiritual leak. This drops `SelectionDAGBuilder::visitIntrinsicCall()` off of my heap profile (was at around 2% of `llc` for codegen of `-flto -g`). Thanks to Pete Cooper for spotting the problem and suggesting the fix. llvm-svn: 237998
* [X86][SSE] Improve support for 128-bit vector sign extensionSimon Pilgrim2015-05-211-4/+27
| | | | | | | | | | This patch improves support for sign extension of the lower lanes of vectors of integers by making use of the SSE41 pmovsx* sign extension instructions where possible, and optimizing the sign extension by shifts on pre-SSE41 targets (avoiding the use of i64 arithmetic shifts which require scalarization). It converts SIGN_EXTEND nodes to SIGN_EXTEND_VECTOR_INREG where necessary, that more closely matches the pmovsx* instruction than the default approach of using SIGN_EXTEND_INREG which splits the operation (into an ANY_EXTEND lowered to a shuffle followed by shifts) making instruction matching difficult during lowering. Necessary support for SIGN_EXTEND_VECTOR_INREG has been added to the DAGCombiner. Differential Revision: http://reviews.llvm.org/D9848 llvm-svn: 237885
* SelectionDAG: Cleanup and simplify FoldConstantArithmeticMatthias Braun2015-05-191-100/+91
| | | | | | | | | | | This cleans up the FoldConstantArithmetic code by factoring out the case of two ConstantSDNodes into an own function. This avoids unnecessary complexity for many callers who already have ConstantSDNode arguments. This also avoids an intermeidate SmallVector datastructure and a loop over that datastructure. llvm-svn: 237651
* Revert r237046. See the testcase on the thread where r237046 was committed.Nick Lewycky2015-05-131-31/+41
| | | | llvm-svn: 237317
* [DebugInfo] Debug locations for constant SD nodesSergey Dmitrouk2015-05-131-41/+76
| | | | | | | | | | | | | | | | | | | | | Several updates for [DebugInfo] Add debug locations to constant SD nodes (r235989). Includes: * re-enabling the change (disabled recently); * missing change for FP constants; * resetting debug location of constant node if it's used more than at one place to prevent emission of wrong locations in case of coalesced constants; * a couple of additional tests. Now all look ups in CSEMap are wrapped by additional method. Comment in D9084 suggests that debug locations aren't useful for "target constants", so there might be one more change related to this API (namely, dropping debug locations for getTarget*Constant methods). Differential Revision: http://reviews.llvm.org/D9604 llvm-svn: 237237
* propagate IR-level fast-math-flags to DAG nodes; 2nd try; NFCSanjay Patel2015-05-111-41/+31
| | | | | | | | | | | | | | | | | | | | | | | This is a less ambitious version of: http://reviews.llvm.org/rL236546 because that was reverted in: http://reviews.llvm.org/rL236600 because it caused memory corruption that wasn't related to FMF but was actually due to making nodes with 2 operands derive from a plain SDNode rather than a BinarySDNode. This patch adds the minimum plumbing necessary to use IR-level fast-math-flags (FMF) in the backend without actually using them for anything yet. This is a follow-on to: http://reviews.llvm.org/rL235997 ...which split the existing nsw / nuw / exact flags and FMF into their own struct. llvm-svn: 237046
* [SelectionDAG] Fixed constant folding issue when legalised types are smaller ↵Simon Pilgrim2015-05-101-2/+3
| | | | | | | | then the folded type. Found when testing with llvm-stress on i686 targets. llvm-svn: 236954
* Reformat.NAKAMURA Takumi2015-05-061-4/+3
| | | | llvm-svn: 236601
* Revert r236546, "propagate IR-level fast-math-flags to DAG nodes (NFC)"NAKAMURA Takumi2015-05-061-39/+50
| | | | | | It caused undefined behavior. llvm-svn: 236600
* SelectionDAG: Handle out-of-bounds index in extract vector elementPawel Bylica2015-05-061-0/+4
| | | | | | | | | | | | | | | | | | Summary: This patch correctly handles undef case of EXTRACT_VECTOR_ELT node where the element index is constant and not less than vector size. Test Plan: CodeGen for X86 test included. Also one incorrect regression test fixed. Reviewers: qcolombet, chandlerc, hfinkel Reviewed By: hfinkel Subscribers: hfinkel, llvm-commits Differential Revision: http://reviews.llvm.org/D9250 llvm-svn: 236584
* propagate IR-level fast-math-flags to DAG nodes (NFC)Sanjay Patel2015-05-051-50/+39
| | | | | | | | | | | | | | | | | | | | | | | | This patch adds the minimum plumbing necessary to use IR-level fast-math-flags (FMF) in the backend without actually using them for anything yet. This is a follow-on to: http://reviews.llvm.org/rL235997 ...which split the existing nsw / nuw / exact flags and FMF into their own struct. There are 2 structural changes here: 1. The main diff is that we're preparing to extend the optimization flags to affect more than just binary SDNodes. Eg, IR intrinsics ( https://llvm.org/bugs/show_bug.cgi?id=21290 ) or non-binop nodes that don't even exist in IR such as FMA, FNEG, etc. 2. The other change is that we're actually copying the FP fast-math-flags from the IR instructions to SDNodes. Differential Revision: http://reviews.llvm.org/D8900 llvm-svn: 236546
* [DAGCombiner] Enabled vector float/double -> int constant foldingSimon Pilgrim2015-05-021-0/+2
| | | | llvm-svn: 236387
* [SelectionDAG] Unary vector constant folding integer legality fixesSimon Pilgrim2015-05-011-5/+25
| | | | | | | | | | | | This patch fixes issues with vector constant folding not correctly handling scalar input operands if they require implicit truncation - this was tested with llvm-stress as recommended by Patrik H Hagglund. The patch ensures that integer input scalars from a build vector are correctly truncated before folding, and that constant integer scalar results are promoted to a legal type before inclusion in the new folded build vector. I have added another crash test case and also a test for UINT_TO_FP / SINT_TO_FP using an non-truncated scalar input, which was failing before this patch. Differential Revision: http://reviews.llvm.org/D9282 llvm-svn: 236308
* IR: Give 'DI' prefix to debug info metadataDuncan P. N. Exon Smith2015-04-291-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Finish off PR23080 by renaming the debug info IR constructs from `MD*` to `DI*`. The last of the `DIDescriptor` classes were deleted in r235356, and the last of the related typedefs removed in r235413, so this has all baked for about a week. Note: If you have out-of-tree code (like a frontend), I recommend that you get everything compiling and tests passing with the *previous* commit before updating to this one. It'll be easier to keep track of what code is using the `DIDescriptor` hierarchy and what you've already updated, and I think you're extremely unlikely to insert bugs. YMMV of course. Back to *this* commit: I did this using the rename-md-di-nodes.sh upgrade script I've attached to PR23080 (both code and testcases) and filtered through clang-format-diff.py. I edited the tests for test/Assembler/invalid-generic-debug-node-*.ll by hand since the columns were off-by-three. It should work on your out-of-tree testcases (and code, if you've followed the advice in the previous paragraph). Some of the tests are in badly named files now (e.g., test/Assembler/invalid-mdcompositetype-missing-tag.ll should be 'dicompositetype'); I'll come back and move the files in a follow-up commit. llvm-svn: 236120
* move IR-level optimization flags into their own structSanjay Patel2015-04-281-5/+7
| | | | | | | | | | | | | | | | | | | | | | This is a preliminary step to using the IR-level floating-point fast-math-flags in the SDAG (D8900). In this patch, we introduce the optimization flags as their own struct. As noted in the TODO comment, we should eventually share this data between the IR passes and the backend. We also switch the existing nsw / nuw / exact bit functionality of the BinaryWithFlagsSDNode class to use the new struct. The tradeoff is that instead of using the free but limited space of SDNode's SubclassData, we add a data member to the subclass. This means we don't have to repeat all of the get/set methods per flag, but we're potentially adding size to all nodes of this subclassi type. In practice on 64-bit systems (measured on Linux and MacOS X), there is no size difference between an SDNode and BinaryWithFlagsSDNode after this change: they're both 80 bytes. This means that we had at least one free byte to play with due to struct alignment. Differential Revision: http://reviews.llvm.org/D9325 llvm-svn: 235997
* Reapply r235977 "[DebugInfo] Add debug locations to constant SD nodes"Sergey Dmitrouk2015-04-281-119/+127
| | | | | | | | | | | | | | | | | | | | | | | | | [DebugInfo] Add debug locations to constant SD nodes This adds debug location to constant nodes of Selection DAG and updates all places that create constants to pass debug locations (see PR13269). Can't guarantee that all locations are correct, but in a lot of cases choice is obvious, so most of them should be. At least all tests pass. Tests for these changes do not cover everything, instead just check it for SDNodes, ARM and AArch64 where it's easy to get incorrect locations on constants. This is not complete fix as FastISel contains workaround for wrong debug locations, which drops locations from instructions on processing constants, but there isn't currently a way to use debug locations from constants there as llvm::Constant doesn't cache it (yet). Although this is a bit different issue, not directly related to these changes. Differential Revision: http://reviews.llvm.org/D9084 llvm-svn: 235989
* Revert "[DebugInfo] Add debug locations to constant SD nodes"Daniel Jasper2015-04-281-127/+119
| | | | | | | This breaks a test: http://bb.pgr.jp/builders/cmake-llvm-x86_64-linux/builds/23870 llvm-svn: 235987
* [DebugInfo] Add debug locations to constant SD nodesSergey Dmitrouk2015-04-281-119/+127
| | | | | | | | | | | | | | | | | | | | | | | This adds debug location to constant nodes of Selection DAG and updates all places that create constants to pass debug locations (see PR13269). Can't guarantee that all locations are correct, but in a lot of cases choice is obvious, so most of them should be. At least all tests pass. Tests for these changes do not cover everything, instead just check it for SDNodes, ARM and AArch64 where it's easy to get incorrect locations on constants. This is not complete fix as FastISel contains workaround for wrong debug locations, which drops locations from instructions on processing constants, but there isn't currently a way to use debug locations from constants there as llvm::Constant doesn't cache it (yet). Although this is a bit different issue, not directly related to these changes. Differential Revision: http://reviews.llvm.org/D9084 llvm-svn: 235977
* Masked gather and scatter: Added code for SelectionDAG.Elena Demikhovsky2015-04-281-0/+49
| | | | | | | | All other patches, including tests will follow. http://reviews.llvm.org/D7665 llvm-svn: 235970
* CONCAT_VECTOR of BUILD_VECTOR - minor fixSimon Pilgrim2015-04-211-0/+12
| | | | | | | | Fixed issue with the combine of CONCAT_VECTOR of 2 BUILD_VECTOR nodes - the optimisation wasn't ensuring that the scalar operands of both nodes were the same type/size for implicit truncation. Test case spotted by Patrik Hagglund llvm-svn: 235371
* TRUNCATE constant folding - minor fix for rL233224Simon Pilgrim2015-04-161-1/+4
| | | | | | Fix for test case found by James Molloy - TRUNCATE of constant build vectors can be more simply achieved by simply replacing with a new build vector node with the truncated value type - no need to touch the scalar operands at all. llvm-svn: 235079
* Allow memory intrinsics to be tail callsKrzysztof Parzyszek2015-04-131-8/+11
| | | | llvm-svn: 234764
* CodeGen: Stop using DIDescriptor::is*() and auto-castingDuncan P. N. Exon Smith2015-04-061-3/+3
| | | | | | Same as r234255, but for lib/CodeGen and lib/Target. llvm-svn: 234258
* [DAGCombiner] Add support for FCEIL, FFLOOR and FTRUNC vector constant foldingSimon Pilgrim2015-04-061-0/+4
| | | | | | Differential Revision: http://reviews.llvm.org/D8715 llvm-svn: 234179
* less space; NFCSanjay Patel2015-04-041-1/+0
| | | | llvm-svn: 234106
* CodeGen: Assert that inlined-at locations agreeDuncan P. N. Exon Smith2015-04-031-0/+6
| | | | | | | | | | | | | | | | | As a follow-up to r234021, assert that a debug info intrinsic variable's `MDLocalVariable::getInlinedAt()` always matches the `MDLocation::getInlinedAt()` of its `!dbg` attachment. The goal here is to get rid of `MDLocalVariable::getInlinedAt()` entirely (PR22778), but I'll let these assertions bake for a while first. If you have an out-of-tree backend that just broke, you're probably attaching the wrong `DebugLoc` to a `DBG_VALUE` instruction. The one you want is the location that was attached to the corresponding `@llvm.dbg.declare` or `@llvm.dbg.value` call that you started with. llvm-svn: 234038
* [SDAG] Handle non-integer preferred memset types for non-constant valuesHal Finkel2015-03-311-2/+17
| | | | | | | | | | | | | | The existing code in getMemsetValue only handled integer-preferred types when the fill value was not a constant. Make this more robust in two ways: 1. If the preferred type is a floating-point value, do the mul-splat trick on the corresponding integer type and then bitcast. 2. If the preferred type is a vector, do the mul-splat trick on one vector element, and then build a vector out of them. Fixes PR22754 (although, we should also turn off use of vector types at -O0). llvm-svn: 233749
* CodeGen: Use the new DebugLoc API, NFCDuncan P. N. Exon Smith2015-03-301-2/+1
| | | | | | Update lib/CodeGen (and lib/Target) to use the new `DebugLoc` API. llvm-svn: 233582
* [DAGCombiner] Add support for TRUNCATE + FP_EXTEND vector constant foldingSimon Pilgrim2015-03-251-2/+27
| | | | | | | | | | This patch adds supports for the vector constant folding of TRUNCATE and FP_EXTEND instructions and tidies up the SINT_TO_FP and UINT_TO_FP instructions to match. It also moves the vector constant folding for the FNEG and FABS instructions to use the DAG.getNode() functionality like the other unary instructions. Differential Revision: http://reviews.llvm.org/D8593 llvm-svn: 233224
* [SelectionDAG] Fixed issue with uitofp vector constant folding being treated ↵Simon Pilgrim2015-03-231-4/+2
| | | | | | | | | | | | | | | | | | | | | | | as sitofp While the uitofp scalar constant folding treats an integer as an unsigned value (from lang ref): %X = sitofp i8 -1 to double ; yields double:-1.0 %Y = uitofp i8 -1 to double ; yields double:255.0 The vector constant folding was always using sitofp: %X = sitofp <2 x i8> <i8 -1, i8 -1> to <2 x double> ; yields <double -1.0, double -1.0> %Y = uitofp <2 x i8> <i8 -1, i8 -1> to <2 x double> ; yields <double -1.0, double -1.0> This patch fixes this so that the correct opcode is used for sitofp and uitofp. %X = sitofp <2 x i8> <i8 -1, i8 -1> to <2 x double> ; yields <double -1.0, double -1.0> %Y = uitofp <2 x i8> <i8 -1, i8 -1> to <2 x double> ; yields <double 255.0, double 255.0> Differential Revision: http://reviews.llvm.org/D8560 llvm-svn: 233033
* DataLayout is mandatory, update the API to reflect it with references.Mehdi Amini2015-03-101-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Now that the DataLayout is a mandatory part of the module, let's start cleaning the codebase. This patch is a first attempt at doing that. This patch is not exactly NFC as for instance some places were passing a nullptr instead of the DataLayout, possibly just because there was a default value on the DataLayout argument to many functions in the API. Even though it is not purely NFC, there is no change in the validation. I turned as many pointer to DataLayout to references, this helped figuring out all the places where a nullptr could come up. I had initially a local version of this patch broken into over 30 independant, commits but some later commit were cleaning the API and touching part of the code modified in the previous commits, so it seemed cleaner without the intermediate state. Test Plan: Reviewers: echristo Subscribers: llvm-commits From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 231740
* [DAGCombiner] Add a shuffle mask commutation helper function. NFCI.Simon Pilgrim2015-03-071-20/+3
| | | | | | | | | | We have an increasing number of cases where we are creating commuted shuffle masks - all implementing nearly the same code. This patch adds a static helper function - ShuffleVectorSDNode::commuteMask() and replaces a number of cases to use it. Differential Revision: http://reviews.llvm.org/D8139 llvm-svn: 231581
* Accidentaly inverted the condition again. Sorry.Benjamin Kramer2015-03-021-1/+1
| | | | llvm-svn: 230973
* Avoid assertion in MSVC 2013 debug builds.Benjamin Kramer2015-03-021-1/+1
| | | | llvm-svn: 230972
* Simplify code. NFC.Benjamin Kramer2015-03-021-11/+3
| | | | llvm-svn: 230948
* Canonicalize splats as build_vectors (PR22283)Sanjay Patel2015-02-171-15/+12
| | | | | | | | | | | | | | | | | | | | | | This is a follow-on patch to: http://reviews.llvm.org/D7093 That patch canonicalized constant splats as build_vectors, and this patch removes the constant check so we can canonicalize all splats as build_vectors. This fixes the 2nd test case in PR22283: http://llvm.org/bugs/show_bug.cgi?id=22283 The unfortunate code duplication between SelectionDAG and DAGCombiner is discussed in the earlier patch review. At least this patch is just removing code... This improves an existing x86 AVX test and changes codegen in an ARM test. Differential Revision: http://reviews.llvm.org/D7389 llvm-svn: 229511
* Prefer SmallVector::append/insert over push_back loops.Benjamin Kramer2015-02-171-3/+1
| | | | | | Same functionality, but hoists the vector growth out of the loop. llvm-svn: 229500
* [SDAG] Teach the SelectionDAG to canonicalize vector shuffles of splatsChandler Carruth2015-02-151-0/+28
| | | | | | | | | | | | | | | directly into blends of the splats. These patterns show up even very late in the vector shuffle lowering where we don't have any chance for DAG combining to kick in, and blending is a tremendously simpler operation to model. By coercing the shuffle into a blend we can much more easily match and lower shuffles of splats. Immediately with this change there are significantly more blends being matched in the x86 vector shuffle lowering. llvm-svn: 229308
* CodeGen: Canonicalize access to function attributes, NFCDuncan P. N. Exon Smith2015-02-141-7/+3
| | | | | | | | | | | | | | | | | Canonicalize access to function attributes to use the simpler API. getAttributes().getAttribute(AttributeSet::FunctionIndex, Kind) => getFnAttribute(Kind) getAttributes().hasAttribute(AttributeSet::FunctionIndex, Kind) => hasFnAttribute(Kind) Also, add `Function::getFnStackAlignment()`, and canonicalize: getAttributes().getStackAlignment(AttributeSet::FunctionIndex) => getFnStackAlignment() llvm-svn: 229208
* Add nullptr checks for TargetSelectionDAGInfo in SelectionDAG.Manuel Jacob2015-01-281-13/+19
| | | | | | TSI is not guaranteed be non-null in SelectionDAG. llvm-svn: 227397
* [SelectionDAG] Fix assert message copypasta. NFC.Ahmed Bougacha2015-01-261-2/+2
| | | | llvm-svn: 227119
* [DAG] Fix wrong canonicalization performed on shuffle nodes.Andrea Di Biagio2015-01-241-7/+9
| | | | | | | | | | | This fixes a regression introduced by r226816. When replacing a splat shuffle node with a constant build_vector, make sure that the new build_vector has a valid number of elements. Thanks to Patrik Hagglund for reporting this problem and providing a small reproducible. llvm-svn: 227002
* DAGCombine: always constant fold FMA when target disable FP exceptionsMehdi Amini2015-01-231-1/+1
| | | | | | | | | | | | | | | | | Summary: When trying to constant fold an FMA in the DAG, getNode() fails to fold the FMA if an operand is not finite. In this case this patch allows the constant folding if !TLI->hasFloatingPointExceptions() Reviewers: resistor Reviewed By: resistor Subscribers: hfinkel, llvm-commits Differential Revision: http://reviews.llvm.org/D6912 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 226901
* SelectionDAG: Add KnownBits and SignBits computation for EXTRACT_ELEMENTJan Vesely2015-01-221-0/+30
| | | | | | | | | | | | v2: use getZExtValue add missing break codestyle v3: add few more comments Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu> Reviewed-by: Matt Arsenault <Matthew.Arsenault@amd.com> llvm-svn: 226880
* [DAGCombine] Produce better code for constant splatsMichael Kuperstein2015-01-221-1/+22
| | | | | | | | | | | This solves PR22276. Splats of constants would sometimes produce redundant shuffles, sometimes ridiculously so (see the PR for details). Fold these shuffles into BUILD_VECTORs early on instead. Differential Revision: http://reviews.llvm.org/D7093 Fixed recommit of r226811. llvm-svn: 226816
* Revert r226811, MSVC accepts code sane compilers don't.Michael Kuperstein2015-01-221-22/+1
| | | | llvm-svn: 226814
* [DAGCombine] Produce better code for constant splatsMichael Kuperstein2015-01-221-1/+22
| | | | | | | | | This solves PR22276. Splats of constants would sometimes produce redundant shuffles, sometimes ridiculously so (see the PR for details). Fold these shuffles into BUILD_VECTORs early on instead. Differential Revision: http://reviews.llvm.org/D7093 llvm-svn: 226811
OpenPOWER on IntegriCloud