summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
Commit message (Collapse)AuthorAgeFilesLines
* [ARM] Fix lowering of misaligned memcpy/memsetJohn Brawn2017-05-261-12/+12
| | | | | | | | | | | | | | | | | Currently getOptimalMemOpType returns i32 for large enough sizes without checking for alignment, leading to poor code generation when misaligned accesses aren't permitted as we generate a word store then later split it up into byte stores. This means we inadvertantly go over the MaxStoresPerMemcpy limit and for memset we splat the memset value into a word then immediately split it up again. Fix this by leaving it up to FindOptimalMemOpLowering to figure out which type to use, but also fix a bug there where it wasn't correctly checking if misaligned memory accesses are allowed. Differential Revision: https://reviews.llvm.org/D33442 llvm-svn: 303990
* Add constrained intrinsics for some libm-equivalent operationsAndrew Kaylor2017-05-251-0/+57
| | | | | | Differential revision: https://reviews.llvm.org/D32319 llvm-svn: 303922
* SimplifyLibCalls: Optimize wcslenMatthias Braun2017-05-191-19/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | Refactor the strlen optimization code to work for both strlen and wcslen. This especially helps with programs in the wild where people pass L"string"s to const std::wstring& function parameters and the wstring constructor gets inlined. This also fixes a lingerind API problem/bug in getConstantStringInfo() where zeroinitializers would always give you an empty string (without a length) back regardless of the actual length of the initializer which did not work well in the TrimAtNul==false causing the PR mentioned below. Note that the fixed getConstantStringInfo() needed fixes to SelectionDAG memcpy lowering and may lead to some cases for out-of-bounds zeroinitializer accesses not getting optimized anymore. So some code with UB may produce out of bound memory reads now instead of just producing zeros. The refactoring "accidentally" fixes http://llvm.org/PR32124 Differential Revision: https://reviews.llvm.org/D32839 llvm-svn: 303461
* [SelectionDAG] Added support for EXTRACT_SUBVECTOR/CONCAT_VECTORS ↵Simon Pilgrim2017-05-131-7/+29
| | | | | | demandedelts in ComputeNumSignBits llvm-svn: 302997
* [SelectionDAG] Add VECTOR_SHUFFLE support to ComputeNumSignBitsSimon Pilgrim2017-05-131-0/+34
| | | | llvm-svn: 302993
* [ValueTracking] Remove const_casts on several calls to computeKnownBits and ↵Craig Topper2017-05-131-2/+1
| | | | | | ComputeSignBit. NFC llvm-svn: 302991
* [KnownBits] Add bit counting methods to KnownBits struct and use them where ↵Craig Topper2017-05-121-30/+25
| | | | | | | | | | | | possible This patch adds min/max population count, leading/trailing zero/one bit counting methods. The min methods return answers based on bits that are known without considering unknown bits. The max methods give answers taking into account the largest count that unknown bits could give. Differential Revision: https://reviews.llvm.org/D32931 llvm-svn: 302925
* Use SDValue::getOperand() helper. NFCI.Simon Pilgrim2017-05-121-16/+14
| | | | llvm-svn: 302896
* Strip trailing whitespace. NFCI.Simon Pilgrim2017-05-111-1/+1
| | | | llvm-svn: 302784
* Introduce experimental generic intrinsics for horizontal vector reductions.Amara Emerson2017-05-091-1/+1
| | | | | | | | | | | | | | - This change allows targets to opt-in to using them instead of the log2 shufflevector algorithm. - The SLP and Loop vectorizers have the common code to do shuffle reductions factored out into LoopUtils, and now have a unified interface for generating reductions regardless of the preference of the target. LoopUtils now uses TTI to determine what kind of reductions the target wants to handle. - For CodeGen, basic legalization support is added. Differential Revision: https://reviews.llvm.org/D30086 llvm-svn: 302514
* [KnownBits] Add wrapper methods for setting and clear all bits in the ↵Craig Topper2017-05-051-6/+3
| | | | | | | | | | underlying APInts in KnownBits. This adds routines for reseting KnownBits to unknown, making the value all zeros or all ones. It also adds methods for querying if the value is zero, all ones or unknown. Differential Revision: https://reviews.llvm.org/D32637 llvm-svn: 302262
* [SelectionDAG] Improve known bits support for CTPOP.Craig Topper2017-05-041-1/+4
| | | | | | This is based on the same concept from ValueTracking's version of computeKnownBits. llvm-svn: 302110
* [KnownBits] Add zext, sext, and trunc methods to KnownBitsCraig Topper2017-05-031-32/+16
| | | | | | | | This patch adds zext, sext, and trunc methods to KnownBits and uses them where possible. Differential Revision: https://reviews.llvm.org/D32784 llvm-svn: 302088
* [SelectionDAG] Improve support for promotion of <1 x fX> floating point ↵Simon Pilgrim2017-05-021-0/+6
| | | | | | | | | | | | argument types (PR31088) PR31088 demonstrated that we were assuming that only integers require promotion from <1 x iX> types, when in fact float types may require it as well - in this case half floats. This patch adds support for extension/truncation for both integer and float types. Differential Revision: https://reviews.llvm.org/D32391 llvm-svn: 301910
* [SelectionDAG] Use known ones to provide a better bound for the known zeros ↵Craig Topper2017-05-011-2/+16
| | | | | | | | | | | | | | for CTTZ/CTLZ operations. This is the SelectionDAG version of D32521. If know where at least one 1 is located in the input to these intrinsics we can place an upper bound on the number of bits needed to represent the count and thus increase the number of known zeros in the output. I think we can also refine this further for CTTZ_UNDEF/CTLZ_UNDEF by assuming that the answer will never be BitWidth. I've left this out for now because it caused other test failures across multiple targets. Usually because of turning ADD into OR based on this new information. I'll fix CTPOP in a future patch. Differential Revision: https://reviews.llvm.org/D32692 llvm-svn: 301806
* Generalize the specialized flag-carrying SDNodes by moving flags into SDNode.Amara Emerson2017-05-011-56/+26
| | | | | | | | This removes BinaryWithFlagsSDNode, and flags are now all passed by value. Differential Revision: https://reviews.llvm.org/D32527 llvm-svn: 301803
* Do not legalize large add with addc/adde, introduce addcarry and do it with ↵Amaury Sechet2017-04-301-5/+6
| | | | | | | | | | | | | | uaddo/addcarry Summary: As per discution on how to get better codegen an large int legalization, it became clear that using a glue for the carry was preventing several desirable optimizations. Passing the carry down as a value allow for more flexibility. Reviewers: jyknight, nemanjai, mkuper, spatel, RKSimon, zvi, bkramer Subscribers: igorb, llvm-commits Differential Revision: https://reviews.llvm.org/D29872 llvm-svn: 301775
* [APInt] Replace calls to setBits with more specific calls to setBitsFrom and ↵Craig Topper2017-04-301-1/+1
| | | | | | setLowBits where possible. llvm-svn: 301768
* [KnownBits] Add methods for determining if the known bits represent a ↵Craig Topper2017-04-291-5/+5
| | | | | | | | | | | | | | | | negative/nonnegative number and add methods for changing the negative/nonnegative state Summary: This patch adds isNegative, isNonNegative for querying whether the sign bit is known. It also adds makeNegative and makeNonNegative for controlling the sign bit. Reviewers: RKSimon, spatel, davide Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D32651 llvm-svn: 301747
* [APInt] Add clearSignBit method. Use it and setSignBit in a few places. NFCICraig Topper2017-04-281-1/+1
| | | | llvm-svn: 301656
* [DAGCombiner] Add ComputeNumSignBits vector demanded elements support to ↵Simon Pilgrim2017-04-281-1/+39
| | | | | | | | ASHR and INSERT_VECTOR_ELT (reapplied) Reapplied r299221 after fix for nondeterminism in ThinLTO builder (rL301599), with extra check for implicit truncation of inserted element. llvm-svn: 301644
* [ValueTracking] Convert computeKnownBitsFromRangeMetadata to use KnownBits ↵Craig Topper2017-04-281-1/+1
| | | | | | struct. llvm-svn: 301626
* [SelectionDAG] Use KnownBits struct in DAG's computeKnownBits and ↵Craig Topper2017-04-281-310/+276
| | | | | | | | | | | | simplifyDemandedBits This patch replaces the separate APInts for KnownZero/KnownOne with a single KnownBits struct. This is similar to what was done to ValueTracking's version recently. This is largely a mechanical transformation from KnownZero to Known.Zero. Differential Revision: https://reviews.llvm.org/D32569 llvm-svn: 301620
* [SelectionDAG] Use various APInt methods to reduce temporary APInt creationCraig Topper2017-04-281-7/+7
| | | | | | This patch uses various APInt methods to reduce the number of temporary APInts. These were all found while working through converting SelectionDAG's computeKnownBits to also use the KnownBits struct recently added to the ValueTracking version. llvm-svn: 301618
* [APInt] Use inplace shift methods where possible. NFCICraig Topper2017-04-281-3/+3
| | | | llvm-svn: 301612
* [ValueTracking] Introduce a KnownBits struct to wrap the two APInts for ↵Craig Topper2017-04-261-3/+4
| | | | | | | | | | | | | | | | computeKnownBits This patch introduces a new KnownBits struct that wraps the two APInt used by computeKnownBits. This allows us to treat them as more of a unit. Initially I've just altered the signatures of computeKnownBits and InstCombine's simplifyDemandedBits to pass a KnownBits reference instead of two separate APInt references. I'll do similar to the SelectionDAG version of computeKnownBits/simplifyDemandedBits as a separate patch. I've added a constructor that allows initializing both APInts to the same bit width with a starting value of 0. This reduces the repeated pattern of initializing both APInts. Once place default constructed the APInts so I added a default constructor for those cases. Going forward I would like to add more methods that will work on the pairs. For example trunc, zext, and sext occur on both APInts together in several places. We should probably add a clear method that can be used to clear both pieces. Maybe a method to check for conflicting information. A method to return (Zero|One) so we don't write it out everywhere. Maybe a method for (Zero|One).isAllOnesValue() to determine if all bits are known. I'm sure there are many other methods we can come up with. Differential Revision: https://reviews.llvm.org/D32376 llvm-svn: 301432
* [DAG] add FIXME comments for splat detection; NFCSanjay Patel2017-04-261-0/+4
| | | | llvm-svn: 301403
* [DAG] fix formatting of isConstantSplat(); NFCSanjay Patel2017-04-251-27/+23
| | | | llvm-svn: 301366
* [SelectionDAG] Recognise splat vector isKnownToBeAPowerOfTwo one/sign bit ↵Simon Pilgrim2017-04-251-2/+2
| | | | | | shift cases. llvm-svn: 301303
* Revert "[APInt] Fix a few places that use APInt::getRawData to operate ↵Renato Golin2017-04-231-3/+3
| | | | | | | | | | | | | | | | within the normal API." This reverts commit r301105, 4, 3 and 1, as a follow up of the previous revert, which broke even more bots. For reference: Revert "[APInt] Use operator<<= where possible. NFC" Revert "[APInt] Use operator<<= instead of shl where possible. NFC" Revert "[APInt] Use ashInPlace where possible." PR32754. llvm-svn: 301111
* [APInt] Use operator<<= where possible. NFCCraig Topper2017-04-231-2/+2
| | | | llvm-svn: 301104
* [APInt] Use ashInPlace where possible.Craig Topper2017-04-231-1/+1
| | | | llvm-svn: 301101
* [Recycler] Add asan/msan annotations.Benjamin Kramer2017-04-201-2/+5
| | | | | | | | | | This enables use after free and uninit memory checking for memory returned by a recycler. SelectionDAG currently relies on the opcode of a free'd node being ISD::DELETED_NODE, so poke a hole in the asan poison for SDNode opcodes. This means that we won't find some issues, but only in SDag. llvm-svn: 300868
* CodeGen: Let frame index value type match alloca addr spaceYaxun Liu2017-04-201-2/+2
| | | | | | | | | | | | | | | | | | | | | | Recently alloca address space has been added to data layout. Due to this change, pointer returned by alloca may have different size as pointer in address space 0. However, currently the value type of frame index is assumed to be of the same size as pointer in address space 0. This patch fixes that. Most targets assume alloca returning pointer in address space 0, which is the default alloca address space. Therefore it is NFC for them. AMDGCN target with amdgiz environment requires this change since it assumes alloca returning pointer to addr space 5 and its size is 32, which is different from the size of pointer in addr space 0 which is 64. Differential Revision: https://reviews.llvm.org/D32021 llvm-svn: 300864
* [APInt] Rename getSignBit to getSignMaskCraig Topper2017-04-201-12/+12
| | | | | | | | getSignBit is a static function that creates an APInt with only the sign bit set. getSignMask seems like a better name to convey its functionality. In fact several places use it and then store in an APInt named SignMask. Differential Revision: https://reviews.llvm.org/D32108 llvm-svn: 300856
* [MVT][SVE] Scalable vector MVTs (1/3)Amara Emerson2017-04-201-7/+4
| | | | | | | | | | | | | | | | This patch adds a few helper functions to obtain new vector value types based on existing ones without needing to care about whether they are scalable or not. I've confined their use to a few common locations right now, and targets that don't have scalable vectors should never need to care about these. Patch by Graham Hunter. Differential Revision: https://reviews.llvm.org/D32017 llvm-svn: 300838
* Temporarily revert r299221 to fix nondeterminism in ThinLTO builder.Galina Kistanova2017-04-191-35/+1
| | | | llvm-svn: 300783
* [APInt] Cast calls to add/sub/mul overflow methods to void if only their ↵Craig Topper2017-04-191-1/+1
| | | | | | | | | | overflow bool out param is used. This is preparation for a clang change to improve the [[nodiscard]] warning to not be ignored on methods that return a class marked [[nodiscard]] that are defined in the class itself. See D32207. We should consider adding wrapper methods to APInt that return the overflow flag directly and discard the APInt result. This would eliminate the void casts and the need to create a bool before the call to pass to the out param. llvm-svn: 300758
* [APInt] Use lshrInPlace to replace lshr where possibleCraig Topper2017-04-181-5/+5
| | | | | | | | | | This patch uses lshrInPlace to replace code where the object that lshr is called on is being overwritten with the result. This adds an lshrInPlace(const APInt &) version as well. Differential Revision: https://reviews.llvm.org/D32155 llvm-svn: 300566
* [SelectionDAG] NFC patch removing a redundant check.Jonas Paulsson2017-04-061-8/+4
| | | | | | | | | Since the BUILD_VECTOR has already been checked by isBuildVectorOfConstantSDNodes() in SelectionDAG::getNode() for a SIGN_EXTEND_INREG, it can be assumed that Op is always either undef or a ConstantSDNode, and Ops.size() will always equal VT.getVectorNumElements(). llvm-svn: 299647
* [DAGCombiner] Don't make a BUILD_VECTOR with operands of illegal type.Jonas Paulsson2017-04-051-6/+6
| | | | | | | | | | | | | | | | | | When DAGCombiner visits a SIGN_EXTEND_INREG of a BUILD_VECTOR with constant operands, a new BUILD_VECTOR node will be created transformed constants. Llvm-stress found a case where the new BUILD_VECTOR had constant operands of an illegal type, because the (legal) element type is in fact not a legal scalar type. This patch changes this so that the new BUILD_VECTOR has the same operand type as the old one. Review: Eli Friedman, Nirav Dave https://bugs.llvm.org//show_bug.cgi?id=32422 llvm-svn: 299540
* [DAGCombiner] Add ComputeNumSignBits vector demanded elements support to ↵Simon Pilgrim2017-03-311-1/+35
| | | | | | | | ASHR and INSERT_VECTOR_ELT Followup to D31311 llvm-svn: 299221
* [DAGCombiner] Add vector demanded elements support to ComputeNumSignBitsSimon Pilgrim2017-03-311-9/+35
| | | | | | | | | | | | | | Currently ComputeNumSignBits returns the minimum number of sign bits for all elements of vector data, when we may only be interested in one/some of the elements. This patch adds a DemandedElts argument that allows us to specify the elements we actually care about. The original ComputeNumSignBits implementation calls with a DemandedElts demanding all elements to match current behaviour. Scalar types set this to 1. I've only added support for BUILD_VECTOR and EXTRACT_VECTOR_ELT so far, all others will default to demanding all elements but can be updated in due course. Followup to D25691. Differential Revision: https://reviews.llvm.org/D31311 llvm-svn: 299219
* [DAGCombiner] Add vector demanded elements support to ↵Simon Pilgrim2017-03-311-3/+2
| | | | | | | | | | computeKnownBitsForTargetNode Follow up to D25691, this sets up the plumbing necessary to support vector demanded elements support in known bits calculations in target nodes. Differential Revision: https://reviews.llvm.org/D31249 llvm-svn: 299201
* [CodeGen] Pass SDAG an ORE, and replace FastISel stats with remarks.Ahmed Bougacha2017-03-301-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In the long-term, we want to replace statistics with something finer-grained that lets us gather per-function data. Remarks are that replacement. Create an ORE instance in SelectionDAGISel, and pass it to SelectionDAG. SelectionDAG was used so that we can emit remarks from all SelectionDAG-related code, including TargetLowering and DAGCombiner. This isn't used in the current patch but Adam tells me he's interested for the fp-contract combines. Use the ORE instance to emit FastISel failures as remarks (instead of the mix of dbgs() dumps and statistics that we currently have). Eventually, we want to have an API that tells us whether remarks are enabled (http://llvm.org/PR32352) so that we don't emit expensive remarks (in this case, dumping IR) when it's not needed. For now, use 'isEnabled' as a crude replacement. This does mean that the replacement for '-fast-isel-verbose' is now '-pass-remarks-missed=isel'. Additionally, clang users also need to enable remark diagnostics, using '-Rpass-missed=isel'. This also removes '-fast-isel-verbose2': there are no static statistics that we want to only enable in asserts builds, so we can always use the remarks regardless of the build type. Differential Revision: https://reviews.llvm.org/D31405 llvm-svn: 299093
* [DAG] fix formatting; NFCSanjay Patel2017-03-281-8/+8
| | | | llvm-svn: 298950
* Apply clang-format as commented in D31311. NFCI.Simon Pilgrim2017-03-241-1/+2
| | | | llvm-svn: 298751
* Fix constant folding of fp2int to large integersSimon Pilgrim2017-03-191-8/+5
| | | | | | | | | | | | We make the assumption in most of our constant folding code that a fp2int will target an integer of 128-bits or less, calling the APFloat::convertToInteger with only uint64_t[2] of raw bits for the result. Fuzz testing (PR24662) showed that we don't handle other cases at all, resulting in stack overflows and all sorts of crashes. This patch uses the APSInt version of APFloat::convertToInteger instead to better handle such cases. Differential Revision: https://reviews.llvm.org/D31074 llvm-svn: 298226
* Make library calls sensitive to regparm module flag (Fixes PR3997).Nirav Dave2017-03-181-15/+15
| | | | | | | | | | Reviewers: mkuper, rnk Subscribers: mehdi_amini, jyknight, aemerson, llvm-commits, rengolin Differential Revision: https://reviews.llvm.org/D27050 llvm-svn: 298179
* [SelectionDAG] Remove redundant stores more aggressively.Eli Friedman2017-03-171-7/+25
| | | | | | | | | | | | | | | | Handle TokenFactors more aggressively in SDValue::reachesChainWithoutSideEffects. This isn't really a very effective change anymore because of other changes to chain handling, but it's a cheap check, and the expanded comments are still useful. It might be possible to loosen the hasOneUse() requirement with a deeper analysis, but a naive implementation of that check would be expensive. Differential Revision: https://reviews.llvm.org/D29845 llvm-svn: 298156
OpenPOWER on IntegriCloud