summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG
Commit message (Collapse)AuthorAgeFilesLines
* Recommit r232027 with PR22883 fixed: Add infrastructure for support of ↵Daniel Sanders2015-03-132-1/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | multiple memory constraints. The operand flag word for ISD::INLINEASM nodes now contains a 15-bit memory constraint ID when the operand kind is Kind_Mem. This constraint ID is a numeric equivalent to the constraint code string and is converted with a target specific hook in TargetLowering. This patch maps all memory constraints to InlineAsm::Constraint_m so there is no functional change at this point. It just proves that using these previously unused bits in the encoding of the flag word doesn't break anything. The next patch will make each target preserve the current mapping of everything to Constraint_m for itself while changing the target independent implementation of the hook to return Constraint_Unknown appropriately. Each target will then be adapted in separate patches to use appropriate Constraint_* values. PR22883 was caused the matching operands copying the whole of the operand flags for the matched operand. This included the constraint id which needed to be replaced with the operand number. This has been fixed with a conversion function. Following on from this, matching operands also used the operand number as the constraint id. This has been fixed by looking up the matched operand and taking it from there. llvm-svn: 232165
* [X86, AVX2] Replace inserti128 and extracti128 intrinsics with generic shufflesSanjay Patel2015-03-121-22/+0
| | | | | | | | | | | | | | | | | This should complete the job started in r231794 and continued in r232045: We want to replace as much custom x86 shuffling via intrinsics as possible because pushing the code down the generic shuffle optimization path allows for better codegen and less complexity in LLVM. AVX2 introduced proper integer variants of the hacked integer insert/extract C intrinsics that were created for this same functionality with AVX1. This should complete the removal of insert/extract128 intrinsics. The Clang precursor patch for this change was checked in at r232109. llvm-svn: 232120
* Revert "r232027 - Add infrastructure for support of multiple memory constraints"Hal Finkel2015-03-122-15/+1
| | | | | | | | | | | | | | | | | | | | | | | | This (r232027) has caused PR22883; so it seems those bits might be used by something else after all. Reverting until we can figure out what else to do. Original commit message: The operand flag word for ISD::INLINEASM nodes now contains a 15-bit memory constraint ID when the operand kind is Kind_Mem. This constraint ID is a numeric equivalent to the constraint code string and is converted with a target specific hook in TargetLowering. This patch maps all memory constraints to InlineAsm::Constraint_m so there is no functional change at this point. It just proves that using these previously unused bits in the encoding of the flag word doesn't break anything. The next patch will make each target preserve the current mapping of everything to Constraint_m for itself while changing the target independent implementation of the hook to return Constraint_Unknown appropriately. Each target will then be adapted in separate patches to use appropriate Constraint_* values. llvm-svn: 232093
* [X86, AVX] replace vextractf128 intrinsics with generic shufflesSanjay Patel2015-03-121-3/+0
| | | | | | | | | | | | | | | | | Now that we've replaced the vinsertf128 intrinsics, do the same for their extract twins. This is very much like D8086 (checked in at r231794): We want to replace as much custom x86 shuffling via intrinsics as possible because pushing the code down the generic shuffle optimization path allows for better codegen and less complexity in LLVM. This is also the LLVM sibling to the cfe D8275 patch. Differential Revision: http://reviews.llvm.org/D8276 llvm-svn: 232045
* Add infrastructure for support of multiple memory constraints.Daniel Sanders2015-03-122-1/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The operand flag word for ISD::INLINEASM nodes now contains a 15-bit memory constraint ID when the operand kind is Kind_Mem. This constraint ID is a numeric equivalent to the constraint code string and is converted with a target specific hook in TargetLowering. This patch maps all memory constraints to InlineAsm::Constraint_m so there is no functional change at this point. It just proves that using these previously unused bits in the encoding of the flag word doesn't break anything. The next patch will make each target preserve the current mapping of everything to Constraint_m for itself while changing the target independent implementation of the hook to return Constraint_Unknown appropriately. Each target will then be adapted in separate patches to use appropriate Constraint_* values. Reviewers: hfinkel Reviewed By: hfinkel Subscribers: hfinkel, jholewinski, llvm-commits Differential Revision: http://reviews.llvm.org/D8171 llvm-svn: 232027
* Handle big index in getelementptr instructionReid Kleckner2015-03-112-21/+12
| | | | | | | | | | | | | | | CodeGen incorrectly ignores (assert from APInt) constant index bigger than 2^64 in getelementptr instruction. This is a test and fix for that. Patch by Paweł Bylica! Reviewed By: rnk Subscribers: majnemer, rnk, mcrosier, resistor, llvm-commits Differential Revision: http://reviews.llvm.org/D8219 llvm-svn: 231984
* Remove useMachineScheduler and replace it with subtarget optionsEric Christopher2015-03-111-1/+2
| | | | | | | | | | | | | that control, individually, all of the disparate things it was controlling. At the same time move a FIXME in the Hexagon port to a new subtarget function that will enable a user of the machine scheduler to avoid using the source scheduler for pre-RA-scheduling. The FIXME would have this removed, but involves either testcase changes or adding -pre-RA-sched=source to a few testcases. llvm-svn: 231980
* Have getCallPreservedMask and getThisCallPreservedMask take aEric Christopher2015-03-111-1/+2
| | | | | | | MachineFunction argument so that we can grab subtarget specific features off of it. llvm-svn: 231979
* Teach lowering to correctly handle invoke statepoint and gc results tied to ↵Igor Laevsky2015-03-103-22/+92
| | | | | | | | | | | them. Note that we still can not lower gc.relocates for invoke statepoints. Also it extracts getCopyFromRegs helper function in SelectionDAGBuilder as we need to be able to customize type of the register exported from basic block during lowering of the gc.result. (Resubmitting this change after not being able to reproduce buildbot failure) Differential Revision: http://reviews.llvm.org/D7760 llvm-svn: 231800
* [X86, AVX] replace vinsertf128 intrinsics with generic shufflesSanjay Patel2015-03-101-3/+0
| | | | | | | | | | | | | | We want to replace as much custom x86 shuffling via intrinsics as possible because pushing the code down the generic shuffle optimization path allows for better codegen and less complexity in LLVM. This is the sibling patch for the Clang half of this change: http://reviews.llvm.org/D8088 Differential Revision: http://reviews.llvm.org/D8086 llvm-svn: 231794
* The operand flag word used in ISD::INLINEASM is an i32 not a pointer. NFC.Daniel Sanders2015-03-101-4/+2
| | | | | | | | | | | | | | | | | | | Summary: This is part of the work to support memory constraints that behave differently to 'm'. The subsequent patches will expand on the existing encoding (which is a 32-bit int) and as a result in some flag words will no longer fit into an i16. This problem only affected the MSP430 target which appears to have 16-bit pointers. Reviewers: hfinkel Reviewed By: hfinkel Subscribers: hfinkel, llvm-commits Differential Revision: http://reviews.llvm.org/D8168 llvm-svn: 231783
* DataLayout is mandatory, update the API to reflect it with references.Mehdi Amini2015-03-102-6/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Now that the DataLayout is a mandatory part of the module, let's start cleaning the codebase. This patch is a first attempt at doing that. This patch is not exactly NFC as for instance some places were passing a nullptr instead of the DataLayout, possibly just because there was a default value on the DataLayout argument to many functions in the API. Even though it is not purely NFC, there is no change in the validation. I turned as many pointer to DataLayout to references, this helped figuring out all the places where a nullptr could come up. I had initially a local version of this patch broken into over 30 independant, commits but some later commit were cleaning the API and touching part of the code modified in the previous commits, so it seemed cleaner without the intermediate state. Test Plan: Reviewers: echristo Subscribers: llvm-commits From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 231740
* [CodeGen] Replace the reused stores' chain for extractelt expansion.Ahmed Bougacha2015-03-091-6/+20
| | | | | | | | | | | | | | | | | | This fixes a subtle issue that was introduced in r205153. When reusing a store for the extractelement expansion (to load directly from it, inserting of going through the stack), later stores to the same location might have overwritten the data we were expecting to extract from. To fix that, we need to explicitly replace the chain going out of the reused store, so that later stores also have an explicit dependency on the generated element-extracting loads, and can't clobber them. rdar://20066785 Differential Revision: http://reviews.llvm.org/D8180 llvm-svn: 231721
* [DAGCombiner] Add a shuffle mask commutation helper function. NFCI.Simon Pilgrim2015-03-072-39/+5
| | | | | | | | | | We have an increasing number of cases where we are creating commuted shuffle masks - all implementing nearly the same code. This patch adds a static helper function - ShuffleVectorSDNode::commuteMask() and replaces a number of cases to use it. Differential Revision: http://reviews.llvm.org/D8139 llvm-svn: 231581
* Make constant arrays that are passed to functions as const.Benjamin Kramer2015-03-071-1/+1
| | | | | | | | In theory this allows the compiler to skip materializing the array on the stack. In practice clang often fails to do that, but that's a different story. NFC. llvm-svn: 231571
* Use SDValue bool check to tidyup some possible combines. NFC.Simon Pilgrim2015-03-071-6/+5
| | | | llvm-svn: 231569
* [DAGCombiner] Fix wrong folding of AND dag nodes.Andrea Di Biagio2015-03-071-3/+7
| | | | | | | | | | | | | | | | | | | | | | | This patch fixes the logic in the DAGCombiner that folds an AND node according to rule: (and (X (load V)), C) -> (X (load V)) An AND between a vector load 'X' and a constant build_vector 'C' can be folded into the load itself only if we can prove that the AND operation is redundant. The algorithm implemented by 'visitAND' firstly computes the splat value 'S' from C, and then checks if S has the lower 'B' bits set (where B is the size in bits of the vector element type). The algorithm takes into account also the 'undef' bits in the splat mask. Unfortunately, the algorithm only worked under the assumption that the size of S is a multiple of the vector element type. With this patch, we conservatively avoid folding the AND if the splat bits are not compatible with the vector element type. Added X86 test and-load-fold.ll Differential Revision: http://reviews.llvm.org/D8085 llvm-svn: 231563
* [DAGCombiner] SCALAR_TO_VECTOR(EXTRACT_VECTOR_ELT(V,C)) -> VECTOR_SHUFFLESimon Pilgrim2015-03-071-0/+30
| | | | | | | | | | | | This patch attempts to convert a SCALAR_TO_VECTOR using an operand from an EXTRACT_VECTOR_ELT into a VECTOR_SHUFFLE. This prevents many cases of spilling scalar data between the gpr + simd registers. At present the optimization only accepts cases where there is no TRUNC of the scalar type (i.e. all types must match). Differential Revision: http://reviews.llvm.org/D8132 llvm-svn: 231554
* DAGCombiner: Canonicalize select(and/or,x,y) depending on target.Matthias Braun2015-03-061-0/+63
| | | | | | | | | | | | | | | This is based on the following equivalences: select(C0 & C1, X, Y) <=> select(C0, select(C1, X, Y), Y) select(C0 | C1, X, Y) <=> select(C0, X, select(C1, X, Y)) Many target cannot perform and/or on the CPU flags and therefore the right side should be choosen to avoid materializign the i1 flags in an integer register. If the target can perform this operation efficiently we normalize to the left form. Differential Revision: http://reviews.llvm.org/D7622 llvm-svn: 231507
* DAGCombiner: Factor out some and/or combines.Matthias Braun2015-03-061-225/+252
| | | | | | | | | | | | | | | This is in preparation for changing visitSELECT to normalize towards select(Cond0, select(Cond1, X, Y), Y); select(Cond0, X, select(Cond1, X, Y)) which perfom an implicit and/or of the conditions. The factored function contains all DAGCombine rules which reduce two values combined by an And/Or operation to a single value. This does not include rules involving constants as visitSELECT already handles that case. Differential Revision: http://reviews.llvm.org/D8026 llvm-svn: 231506
* LegalizeTypes: Handle shift by 0 in ExpandShiftByConstant.Michael Zolotukhin2015-03-061-1/+8
| | | | | | | Though such shifts are usually optimized away by combiner, we still can encounter them after a vector shift is legalized. llvm-svn: 231443
* SelectionDAGBuilder: Merge 3 copies of the limited precision exp2 emission code.Benjamin Kramer2015-03-051-251/+93
| | | | | | NFC intended. llvm-svn: 231406
* SDAG: Merge the meat of two ExpandAtomic implementations.Benjamin Kramer2015-03-052-212/+4
| | | | | | | The copies already diverged, don't let them become any worse. Reduce redundancy in code with a little macro metaprogramming. llvm-svn: 231401
* X86: Optimize address mode matching for FRAME_ALLOC_RECOVER nodesDavid Majnemer2015-03-051-0/+1
| | | | | | | We know that the absolute symbol will be less than 2GB and thus will always fit. llvm-svn: 231389
* Replace llvm.frameallocate with llvm.frameescapeReid Kleckner2015-03-051-22/+24
| | | | | | | | | | Turns out it's pretty straightforward and simplifies the implementation. Reviewers: andrew.w.kaylor Differential Revision: http://reviews.llvm.org/D8051 llvm-svn: 231386
* [DagCombiner] Allow shuffles to merge through bitcastsSimon Pilgrim2015-03-051-0/+83
| | | | | | | | | | | | Currently shuffles may only be combined if they are of the same type, despite the fact that bitcasts are often introduced in between shuffle nodes (e.g. x86 shuffle type widening). This patch allows a single input shuffle to peek through bitcasts and if the input is another shuffle will merge them, shuffling using the smallest sized type, and re-applying the bitcasts at the inputs and output instead. Dropped old ShuffleToZext test - this patch removes the use of the zext and vector-zext.ll covers these anyhow. Differential Revision: http://reviews.llvm.org/D7939 llvm-svn: 231380
* Revert change r231366 as it broke clang-native-arm-cortex-a9 ↵Igor Laevsky2015-03-053-92/+22
| | | | | | Analysis/properties.m test. llvm-svn: 231374
* AVX-512, SKX: Enabled masked_load/store operations for this target.Elena Demikhovsky2015-03-051-2/+2
| | | | | | | Added lowering for ISD::CONCAT_VECTORS and ISD::INSERT_SUBVECTOR for i1 vectors, it is needed to pass all masked_memop.ll tests for SKX. llvm-svn: 231371
* Teach lowering to correctly handle invoke statepoint and gc results tied to ↵Igor Laevsky2015-03-053-22/+92
| | | | | | | | them. Note that we still can not lower gc.relocates for invoke statepoints. Also it extracts getCopyFromRegs helper function in SelectionDAGBuilder as we need to be able to customize type of the register exported from basic block during lowering of the gc.result. llvm-svn: 231366
* [DAGCombine] Fix a bug in a BUILD_VECTOR combineMichael Kuperstein2015-03-041-2/+3
| | | | | | | | | | When trying to convert a BUILD_VECTOR into a shuffle, we try to split a single source vector that is twice as wide as the destination vector. We can not do this when we also need the zero vector to create a blend. This fixes PR22774. Differential Revision: http://reviews.llvm.org/D8040 llvm-svn: 231219
* Use report_fatal_error instead of unreachable for -fast-isel-abortMehdi Amini2015-03-041-3/+3
| | | | | | | Suggestion by Andrea Di Biagio From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 231201
* DAGCombiner::LoadedSlice: Remove explicit copy ctor in favor of the Rule of ZeroDavid Blaikie2015-03-031-3/+0
| | | | | | | This way, the copy assignment operator can be used without hitting the deprecated case in C++11. llvm-svn: 231144
* Revert "Remove the explicit SDNodeIterator::operator= in favor of the ↵David Blaikie2015-03-031-0/+3
| | | | | | | | | | | implicit default" Accidentally committed a few more of these cleanup changes than intended. Still breaking these out & tidying them up. This reverts commit r231135. llvm-svn: 231136
* Remove the explicit SDNodeIterator::operator= in favor of the implicit defaultDavid Blaikie2015-03-031-3/+0
| | | | | | | | | | There doesn't seem to be any need to assert that iterator assignment is between iterators over the same node - if you want to reuse an iterator variable to iterate another node, that's perfectly acceptable. Just don't mix comparisons between iterators into disjoint sequences, as usual. llvm-svn: 231135
* unique_ptrify ResourcePriorityQueue::ResourceModelDavid Blaikie2015-03-031-1/+1
| | | | llvm-svn: 231127
* Remove ResourcePriorityQueue::dump as it relies on copying a non-copyable ↵David Blaikie2015-03-031-14/+0
| | | | | | type which would result in a double-delete llvm-svn: 231126
* Accidentaly inverted the condition again. Sorry.Benjamin Kramer2015-03-021-1/+1
| | | | llvm-svn: 230973
* Avoid assertion in MSVC 2013 debug builds.Benjamin Kramer2015-03-021-1/+1
| | | | llvm-svn: 230972
* Simplify code. NFC.Benjamin Kramer2015-03-021-11/+3
| | | | llvm-svn: 230948
* avoid infinite looping when folding vector multiplies of constants (PR22698)Sanjay Patel2015-03-011-5/+14
| | | | | | | | | | | | | | | We were missing a check for the following fold in DAGCombiner: // fold (fmul (fmul x, c1), c2) -> (fmul x, (fmul c1, c2)) If 'x' is also a constant, then we shouldn't do anything. Otherwise, we could end up swapping the operands back and forth forever. This should fix: http://llvm.org/bugs/show_bug.cgi?id=22698 Differential Revision: http://reviews.llvm.org/D7917 llvm-svn: 230884
* Fixup for recent -fast-isel-abort change: code didn't match descriptionMehdi Amini2015-02-281-1/+4
| | | | | | | | Level 1 should abort for all instructions but call/terminators/args. Instead it was aborting only if the level was > 2 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 230861
* Convert push_back loops into append calls.Benjamin Kramer2015-02-281-4/+2
| | | | | | No functionality change intended. llvm-svn: 230849
* Reduce double set lookups.Benjamin Kramer2015-02-271-2/+1
| | | | llvm-svn: 230798
* Change the fast-isel-abort option from bool to int to enable "levels"Mehdi Amini2015-02-271-24/+24
| | | | | | | | | | | | | | | | | | | | | | | Summary: Currently fast-isel-abort will only abort for regular instructions, and just warn for function calls, terminators, function arguments. There is already fast-isel-abort-args but nothing for calls and terminators. This change turns the fast-isel-abort options into an integer option, so that multiple levels of strictness can be defined. This will help no being surprised when the "abort" option indeed does not abort, and enables the possibility to write test that verifies that no intrinsics are forgotten by fast-isel. Reviewers: resistor, echristo Subscribers: jfb, llvm-commits Differential Revision: http://reviews.llvm.org/D7941 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 230775
* getRegForInlineAsmConstraint wants to use TargetRegisterInfo forEric Christopher2015-02-263-27/+30
| | | | | | | | | a lookup, pass that in rather than use a naked call to getSubtargetImpl. This involved passing down and around either a TargetMachine or TargetRegisterInfo. Update all callers/definitions around the targets and SelectionDAG. llvm-svn: 230699
* When the source has a series of assignments, users reasonably want toPaul Robinson2015-02-261-0/+3
| | | | | | | | | | | | have the debugger step through each one individually. Turn off the combine for adjacent stores at -O0 so we get this behavior. Possibly, DAGCombine shouldn't run at all at -O0, but that's for another day; see PR22346. Differential Revision: http://reviews.llvm.org/D7181 llvm-svn: 230659
* Reapplied D7816 & rL230177 & rL230278 - with an additional fix toensure that ↵Simon Pilgrim2015-02-241-45/+65
| | | | | | the smallest build vector input scalar type is always used. Additional (crash) test cases already committed. llvm-svn: 230388
* Revert:Eric Christopher2015-02-241-39/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Author: Simon Pilgrim <llvm-dev@redking.me.uk> Date: Mon Feb 23 23:04:28 2015 +0000 Fix based on post-commit comment on D7816 & rL230177 - BUILD_VECTOR operand truncation was using the the BV's output scalar type instead of the input type. and Author: Simon Pilgrim <llvm-dev@redking.me.uk> Date: Sun Feb 22 18:17:28 2015 +0000 [DagCombiner] Generalized BuildVector Vector Concatenation The CONCAT_VECTORS combiner pass can transform the concat of two BUILD_VECTOR nodes into a single BUILD_VECTOR node. This patch generalises this to support any number of BUILD_VECTOR nodes, and also permits UNDEF nodes to be included as well. This was noticed as AVX vec128 -> vec256 canonicalization sometimes creates a CONCAT_VECTOR with a real vec128 lower and an vec128 UNDEF upper. Differential Revision: http://reviews.llvm.org/D7816 as the root cause of PR22678 which is causing an assertion inside the DAG combiner. I'll follow up to the main thread as well. llvm-svn: 230358
* DAGCombiner: Move variable definitions closer to use; NFCMatthias Braun2015-02-241-38/+38
| | | | llvm-svn: 230354
* DAGCombiner: Move variable declaration closer to definiion; NFCMatthias Braun2015-02-241-3/+3
| | | | llvm-svn: 230353
OpenPOWER on IntegriCloud