summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG
Commit message (Collapse)AuthorAgeFilesLines
* [statepoints][experimental] Add support for live-in semantics of values in ↵Philip Reames2016-08-311-5/+35
| | | | | | | | | | | | | | | | deopt bundles This is a first step towards supporting deopt value lowering and reporting entirely with the register allocator. I hope to build on this in the near future to support live-on-return semantics, but I have a use case which allows me to test and investigate code quality with just the live-in semantics so I've chosen to start there. For those curious, my use cases is our implementation of the "__llvm_deoptimize" function we bind to @llvm.deoptimize. I'm choosing not to hard code that fact in the patch and instead make it configurable via function attributes. The basic approach here is modelled on what is done for the "Live In" values on stackmaps and patchpoints. (A secondary goal here is to remove one of the last barriers to merging the pseudo instructions.) We start by adding the operands directly to the STATEPOINT SDNode. Once we've lowered to MI, we extend the remat logic used by the register allocator to fold virtual register uses into StackMap::Indirect entries as needed. This does rely on the fact that the register allocator rematerializes. If it didn't along some code path, we could end up with more vregs than physical registers and fail to allocate. Today, we *only* fold in the register allocator. This can create some weird effects when combined with arguments passed on the stack because we don't fold them appropriately. I have an idea how to fix that, but it needs this patch in place to work on that effectively. (There's some weird interaction with the scheduler as well, more investigation needed.) My near term plan is to land this patch off-by-default, experiment in my local tree to identify any correctness issues and then start fixing codegen problems one by one as I find them. Once I have the live-in lowering fully working (both correctness and code quality), I'm hoping to move on to the live-on-return semantics. Note: I don't have any *known* miscompiles with this patch enabled, but I'm pretty sure I'll find at least a couple. Thus, the "experimental" tag and the fact it's off by default. Differential Revision: https://reviews.llvm.org/D24000 llvm-svn: 280250
* Propagate TBAA info in SelectionDAG::getIndexedLoadKrzysztof Parzyszek2016-08-291-1/+2
| | | | | | Patch by Pranav Bhandarkar. llvm-svn: 279998
* Fixed a bug in type legalizer for masked gather.Igor Breger2016-08-291-1/+9
| | | | | | | | | The problem occurs when the Node doesn't updated in place , UpdateNodeOperation() return the node that already exist. In this case assert fail in PromoteIntegerOperand() , N have 2 results ( val + chain). Differential Revision: http://reviews.llvm.org/D23756 llvm-svn: 279961
* [SelectionDAG] Do not run the ISel process on already selected code.Quentin Colombet2016-08-261-0/+4
| | | | | | | Right now, this cannot happen, but with the fall back path of GlobalISel it will show up eventually. llvm-svn: 279877
* Reuse an SDLoc throughout a function. NFC.Michael Kuperstein2016-08-251-18/+12
| | | | llvm-svn: 279767
* [SelectionDAG] Use a union of bitfield structs for SDNode::SubclassData.Justin Lebar2016-08-231-43/+18
| | | | | | | | | | | | | | | | | | Summary: This greatly simplifies our handling of SDNode::SubclassData. NFC, hopefully. :) See discussion in D23035 for discussion about the design API of these bitfields. Reviewers: chandlerc Subscribers: llvm-commits, rnk Differential Revision: https://reviews.llvm.org/D23036 llvm-svn: 279537
* Fix some more asserts after r279466.Pete Cooper2016-08-232-2/+2
| | | | | | | | | | | | | | | | That commit added a new version of Intrinsic::getName which should only be called when the intrinsic has no overloaded types. There are several debugging paths, such as SDNode::dump which are printing the name of the intrinsic but don't have the overloaded types. These paths should be ok to just print the name instead of crashing. The fix here is ultimately to just add a 'None' second argument as that calls the overload capable getName, which is less efficient, but this is a debugging path anyway, and not perf critical. Thanks to Björn Pettersson for pointing out that there were more crashes. llvm-svn: 279528
* Use SDValue::getOpcode() helper instead of via SDValue::getNode()Simon Pilgrim2016-08-201-2/+2
| | | | llvm-svn: 279381
* [CodeGen] Fix a trivial type conversion bug dating back to pre-2008James Molloy2016-08-191-1/+1
| | | | | | | | The heuristic above this code is incredibly suspect, but disregarding that it mutates the cast opcode so we need to check the *mutated* opcode later to see if we need to emit an AssertSext or AssertZext node. Fixes PR29041. llvm-svn: 279223
* Replace a few more "fall through" comments with LLVM_FALLTHROUGHJustin Bogner2016-08-176-15/+14
| | | | | | Follow up to r278902. I had missed "fall through", with a space. llvm-svn: 278970
* Fix bug in DAGBuilder for getelementptr with expanded vector.Ayman Musa2016-08-171-1/+2
| | | | | | | Replacing the usage of MVT with EVT in case the vector type is expanded. Differential Revision: https://reviews.llvm.org/D23306 llvm-svn: 278913
* First commit (test commit) - Adding empty line.Ayman Musa2016-08-171-0/+1
| | | | llvm-svn: 278910
* Replace "fallthrough" comments with LLVM_FALLTHROUGHJustin Bogner2016-08-173-4/+5
| | | | | | | This is a mechanical change of comments in switches like fallthrough, fall-through, or fall-thru to use the LLVM_FALLTHROUGH macro instead. llvm-svn: 278902
* [x86] Refactor a PowerPC specific ctlz/srl transformation (NFC).Pierre Gousseau2016-08-161-0/+25
| | | | | | | | Following the discussion on D22038, this refactors a PowerPC specific setcc -> srl(ctlz) transformation so it can be used by other targets. Differential Revision: https://reviews.llvm.org/D23445 llvm-svn: 278799
* Fix typo in lowering for fp128 ueq.Eli Friedman2016-08-151-1/+1
| | | | | | | | Regression from r259791. Differential Revision: https://reviews.llvm.org/D23374 llvm-svn: 278750
* Local variables whose address is taken and passed on to a call are described Wolfgang Pieb2016-08-152-4/+33
| | | | | | | | | | | in debug info using their stack slots instead of as an indirection of param reg + 0 offset. This is done by detecting FrameIndexSDNodes in SelectionDAG and generating FrameIndexDbgValues for them. This ultimately generates DBG_VALUEs with stack location operands. Differential Revision: http://reviews.llvm.org/D23283 llvm-svn: 278703
* ADT: Remove all ilist_iterator => pointer casts, NFCDuncan P. N. Exon Smith2016-08-121-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove all ilist_iterator to pointer casts. There were two reasons for casts: - Checking for an uninitialized (i.e., null) iterator. I added MachineInstrBundleIterator::isValid() to check for that case. - Comparing an iterator against the underlying pointer value while avoiding converting the pointer value to an iterator. This is occasionally necessary in MachineInstrBundleIterator, since there is an assertion in the constructors that the underlying MachineInstr is not bundled (but we don't care about that if we're just checking for pointer equality). To support the latter case, I rewrote the == and != operators for ilist_iterator and MachineInstrBundleIterator. - The implicit constructors now use enable_if to exclude const-iterator => non-const-iterator conversions from overload resolution (previously it was a compiler error on instantiation, now it's SFINAE). - The == and != operators are now global (friends), and are not templated. - MachineInstrBundleIterator has overloads to compare against both const_pointer and const_reference. This avoids the implicit conversions to MachineInstrBundleIterator that assert, instead just checking the address (and I added unit tests to confirm this). Notably, the only remaining uses of ilist_iterator::getNodePtrUnchecked are in ilist.h, and no code outside of ilist*.h directly relies on this UB end-iterator-to-pointer conversion anymore. It's still needed for ilist_*sentinel_traits, but I'll clean that up soon. llvm-svn: 278478
* Use the range variant of find instead of unpacking begin/endDavid Majnemer2016-08-115-11/+7
| | | | | | | | | If the result of the find is only used to compare against end(), just use is_contained instead. No functionality change is intended. llvm-svn: 278433
* Use range algorithms instead of unpacking begin/endDavid Majnemer2016-08-113-7/+6
| | | | | | No functionality change is intended. llvm-svn: 278417
* [DAGCombine] Avoid INSERT_SUBVECTOR reinsertions (PR28678)Simon Pilgrim2016-08-101-1/+10
| | | | | | | | | | | If the input vector to INSERT_SUBVECTOR is another INSERT_SUBVECTOR, and this inserted subvector replaces the last insertion, then insert into the common source vector. i.e. INSERT_SUBVECTOR( INSERT_SUBVECTOR( Vec, SubOld, Idx ), SubNew, Idx ) --> INSERT_SUBVECTOR( Vec, SubNew, Idx ) Differential Revision: https://reviews.llvm.org/D23330 llvm-svn: 278211
* [DAGCombiner] Better support for shifting large value type by constantsSimon Pilgrim2016-08-091-17/+42
| | | | | | | | | | As detailed on D22726, much of the shift combining code assume constant values will fit into a uint64_t value and calls ConstantSDNode::getZExtValue where it probably shouldn't (leading to asserts). Using APInt directly avoids this problem but we encounter other assertions if we attempt to compare/operate on 2 APInt of different bitwidths. This patch adds a helper function to ensure that 2 APInt values are zero extended as required so that they can be safely used together. I've only added an initial example use for this to the '(SHIFT (SHIFT x, c1), c2) --> (SHIFT x, (ADD c1, c2))' combines. Further cases can easily be added as required. Differential Revision: https://reviews.llvm.org/D23007 llvm-svn: 278141
* [SelectionDAG] Refactor visitInlineAsm a bit. NFCI.Diana Picus2016-08-081-151/+198
| | | | | | This shaves off ~100 lines from visitInlineAsm. llvm-svn: 277987
* [X86] Heuristic to selectively build Newton-Raphson SQRT estimationNikolai Bozhenov2016-08-041-2/+6
| | | | | | | | | | | | | | | | | | | | | On modern Intel processors hardware SQRT in many cases is faster than RSQRT followed by Newton-Raphson refinement. The patch introduces a simple heuristic to choose between hardware SQRT instruction and Newton-Raphson software estimation. The patch treats scalars and vectors differently. The heuristic is that for scalars the compiler should optimize for latency while for vectors it should optimize for throughput. It is based on the assumption that throughput bound code is likely to be vectorized. Basically, the patch disables scalar NR for big cores and disables NR completely for Skylake. Firstly, scalar SQRT has shorter latency than NR code in big cores. Secondly, vector SQRT has been greatly improved in Skylake and has better throughput compared to NR. Differential Revision: https://reviews.llvm.org/D21379 llvm-svn: 277725
* Typo fix in comment. NFCDiana Picus2016-08-041-1/+1
| | | | llvm-svn: 277704
* Disable shrinking of SNaN constantsElliot Colp2016-08-031-11/+17
| | | | | | | | | When expanding FP constants, we attempt to shrink doubles to floats and perform an extending load. However, on SystemZ, and possibly on other targets (I've only confirmed the problem on SystemZ), the FP extending load instruction may convert SNaN into QNaN, or may cause an exception. So in the general case, we would still like to shrink FP constants, but SNaNs should be left as doubles. Differential Revision: https://reviews.llvm.org/D22685 llvm-svn: 277602
* [DAGCombine] Make sext(setcc) combine respect getBooleanContentsMichael Kuperstein2016-08-012-9/+33
| | | | | | | | | | | We used to combine "sext(setcc x, y, cc) -> (select (setcc x, y, cc), -1, 0)" Instead, we should combine to (select (setcc x, y, cc), T, 0) where the value of T is 1 or -1, depending on the type of the setcc, and getBooleanContents() for the type if it is not i1. This fixes PR28504. llvm-svn: 277371
* DAG: avoid duplicated truncating for sign extended operandWeiming Zhao2016-07-291-8/+10
| | | | | | | | | | | | | | | Summary: When performing cmp for EQ/NE and the operand is sign extended, we can avoid the truncaton if the bits to be tested are no less than origianl bits. Reviewers: eli.friedman Subscribers: eli.friedman, aemerson, nemanjai, t.p.northover, llvm-commits Differential Revision: https://reviews.llvm.org/D22933 llvm-svn: 277252
* Recommitting r275284: add support to inline __builtin_mempcpyAndrew Kaylor2016-07-292-0/+48
| | | | | | | | Patch by Sunita Marathe Third try, now following fixes to MSan to handle mempcy in such a way that this commit won't break the MSan buildbots. (Thanks, Evegenii!) llvm-svn: 277189
* Cleanup TransferDbgValuesNirav Dave2016-07-291-2/+9
| | | | | | | | | | | | | | | [DAG] Check debug values for invalidation before transferring and mark old debug values invalid when transferring to another SDValue. This fixes PR28613. Reviewers: jyknight, hans, dblaikie, echristo Subscribers: yaron.keren, ismail, llvm-commits Differential Revision: https://reviews.llvm.org/D22858 llvm-svn: 277135
* Fix DbgValue handling in SelectionDAG.Nirav Dave2016-07-281-2/+3
| | | | | | | [DAG] Relocate TransferDbgValues in ReplaceAllUsesWith(SDValue, SDValue) to before we modify the CSE maps. llvm-svn: 277027
* MachineFunction: Return reference for getFrameInfo(); NFCMatthias Braun2016-07-288-60/+59
| | | | | | | getFrameInfo() never returns nullptr so we should use a reference instead of a pointer. llvm-svn: 277017
* [DAGCombiner] Use APInt directly to detect out of range shift constantsSimon Pilgrim2016-07-271-3/+3
| | | | | | | | Using getZExtValue() will assert if the value doesn't fit into uint64_t - SHL was already doing this, I've just updated ASHR/LSHR to match As mentioned on D22726 llvm-svn: 276855
* Reverting r276771 due to MSan failures.Andrew Kaylor2016-07-272-48/+0
| | | | llvm-svn: 276824
* Re-committing r275284: add support to inline __builtin_mempcpyAndrew Kaylor2016-07-262-0/+48
| | | | | | | | Patch by Sunita Marathe Differential Revision: http://reviews.llvm.org/D21920 llvm-svn: 276771
* [SelectionDAG] Optimization of BITREVERSE legalization for power-of-2 ↵Simon Pilgrim2016-07-221-3/+46
| | | | | | | | | | | | | | integer scalar/vector types An extension of D19978, this patch replaces the default BITREVERSE evaluation of individual bit masks+shifts with block mask+shifts when we have integer elements of power-of-2 bits in size. After calling BSWAP to reverse the order of the constituent bytes (which typically follows a similar approach), every neighbouring 4-bits, 2-bits and finally 1-bit pairs are masked off and swapped over with shifts. In doing so we can significantly reduce the number of operations required. Differential Revision: https://reviews.llvm.org/D21578 llvm-svn: 276432
* [FastISel] Ignore @llvm.assume.Ahmed Bougacha2016-07-221-0/+2
| | | | llvm-svn: 276410
* AVX-512: Fixed BT instruction selection.Elena Demikhovsky2016-07-191-0/+4
| | | | | | | | | | | The following condition expression ( a >> n) & 1 is converted to "bt a, n" instruction. It works on all intel targets. But on AVX-512 it was broken because the expression is modified to (truncate (a >>n) to i1). I added the new sequence (truncate (a >>n) to i1) to the BT pattern. Differential Revision: https://reviews.llvm.org/D22354 llvm-svn: 275950
* [X86] Accept SELECT op code for x86-64 fp128 typeChih-Hung Hsieh2016-07-181-0/+1
| | | | | | | | | | DAGTypeLegalizer::CanSkipSoftenFloatOperand should allow SELECT op code for x86_64 fp128 type for MME targets, so SoftenFloatOperand does not abort on SELECT op code. Differential Revision: http://reviews.llvm.org/D21758 llvm-svn: 275818
* [inlineasm] Propagate operand constraints to the backendSimon Dardis2016-07-181-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | When SelectionDAGISel transforms a node representing an inline asm block, memory constraint information is not preserved. This can cause constraints to be broken when a memory offset is of the form: offset + frame index when the frame is resolved. By propagating the constraints all the way to the backend, targets can enforce memory operands of inline assembly to conform to their constraints. For MIPSR6, some instructions had their offsets reduced to 9 bits from 16 bits such as ll/sc. This becomes problematic when using inline assembly to perform atomic operations, as an offset can generated that is too big to encode in the instruction. Reviewers: dsanders, vkalintris Differential Review: https://reviews.llvm.org/D21615 llvm-svn: 275786
* [SelectionDAG] Get rid of bool parameters in SelectionDAG::getLoad, ↵Justin Lebar2016-07-1512-607/+462
| | | | | | | | | | | | | | | | | | | | | | | getStore, and friends. Summary: Instead, we take a single flags arg (a bitset). Also add a default 0 alignment, and change the order of arguments so the alignment comes before the flags. This greatly simplifies many callsites, and fixes a bug in AMDGPUISelLowering, wherein the order of the args to getLoad was inverted. It also greatly simplifies the process of adding another flag to getLoad. Reviewers: chandlerc, tstellarAMD Subscribers: jholewinski, arsenm, jyknight, dsanders, nemanjai, llvm-commits Differential Revision: http://reviews.llvm.org/D22249 llvm-svn: 275592
* [CodeGen] Take a MachineMemOperand::Flags in ↵Justin Lebar2016-07-153-11/+9
| | | | | | | | | | | | | | | | | MachineFunction::getMachineMemOperand. Summary: Previously we took an unsigned. Hooray for type-safety. Reviewers: chandlerc Subscribers: dsanders, llvm-commits Differential Revision: http://reviews.llvm.org/D22282 llvm-svn: 275591
* Fix copy/paste bug in r275340.Michael Kuperstein2016-07-131-1/+1
| | | | llvm-svn: 275343
* [DAG] Correctly chain masked loadsMichael Kuperstein2016-07-131-9/+8
| | | | | | | | | If a masked loads is not added to the chain, it should not reset the chain's root. This fixes the remaining part of PR28515. llvm-svn: 275340
* Reverting r275284 due to platform-specific test failuresAndrew Kaylor2016-07-132-47/+0
| | | | llvm-svn: 275304
* Fix for Bug 26903, adds support to inline __builtin_mempcpyAndrew Kaylor2016-07-132-0/+47
| | | | | | | | Patch by Sunita Marathe Differential Revision: http://reviews.llvm.org/D21920 llvm-svn: 275284
* fix documentation comments; NFCSanjay Patel2016-07-111-42/+7
| | | | llvm-svn: 275101
* [DAG] make isConstantSplatVector() available to the rest of loweringSanjay Patel2016-07-102-32/+25
| | | | llvm-svn: 275025
* fix documentation comments; NFCSanjay Patel2016-07-101-11/+3
| | | | llvm-svn: 275021
* reformat, fix comments/names; NFCISanjay Patel2016-07-101-27/+22
| | | | llvm-svn: 275015
* Give helper classes/functions internal linkage. NFC.Benjamin Kramer2016-07-101-1/+1
| | | | llvm-svn: 275014
OpenPOWER on IntegriCloud