summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG
Commit message (Collapse)AuthorAgeFilesLines
...
* [CodeGen] Add comment about FSUB <-> FNEG xformsCameron McInally2019-05-091-0/+4
| | | | | | Differential Revision: https://reviews.llvm.org/D61741 llvm-svn: 360366
* [DAGCombiner] Limit number of nodes explored as store candidates.Florian Hahn2019-05-091-2/+5
| | | | | | | | | | | | | | To find the candidates to merge stores we iterate over all nodes in a chain for each store, which leads to quadratic compile times for large basic blocks with a large number of stores. Reviewers: niravd, spatel, craig.topper Reviewed By: niravd Differential Revision: https://reviews.llvm.org/D61511 llvm-svn: 360357
* [SelectionDAG] Expand ADD/SUBCARRYLeonard Chan2019-05-091-0/+42
| | | | | | | | This patch allows for expansion of ADDCARRY and SUBCARRY when the target does not support it. Differential Revision: https://reviews.llvm.org/D61411 llvm-svn: 360303
* [SelectionDAG] fold 'fneg undef' to undefSanjay Patel2019-05-081-0/+4
| | | | | | | | | | | | | | | | | This is extracted from the original draft of D61419 with some additional tests. We don't currently get this in IR (it's conservatively turned into a NaN), but presumably that'll get updated as we add real IR support for 'fneg' rather than 'fsub -0.0, x'. The x86-32 run shows the following, and I haven't looked further to see why, but that seems to be independent: Legalizing: t1: f32 = undef Trying to expand node Creating fp constant: t4: f32 = ConstantFP<0.000000e+00> Differential Revision: https://reviews.llvm.org/D61516 llvm-svn: 360296
* [FastISel][X86] Support FNeg instruction in target independent fast isel ↵Craig Topper2019-05-081-0/+3
| | | | | | | | | | handling This patch adds support for calling selectFNeg for FNeg instructions in addition to the fsub idiom Differential Revision: https://reviews.llvm.org/D61624 llvm-svn: 360273
* [LegalizeDAG] Assert non-power-of-2 load/store op splits are in range. NFCI.Simon Pilgrim2019-05-081-2/+6
| | | | | | Fixes static analyzer undefined/out-of-range shift warnings. llvm-svn: 360245
* Fix cppcheck operator precedence warning. NFCI.Simon Pilgrim2019-05-081-2/+2
| | | | llvm-svn: 360234
* [NFC] Add a static function to do the endian checkQingShan Zhang2019-05-081-15/+37
| | | | | | | | Add a new function to do the endian check, as I will commit another patch later, which will also need the endian check. Differential Revision: https://reviews.llvm.org/D61236 llvm-svn: 360226
* [DAGCombiner] Avoid creating large tokenfactors in visitTokenFactorFlorian Hahn2019-05-071-2/+3
| | | | | | | | | | | | | | | | | | When simplifying TokenFactors, we potentially iterate over all operands of a large number of TokenFactors. This causes quadratic compile times in some cases and the large token factors cause additional scalability problems elsewhere. This patch adds some limits to the number of nodes explored for the cases mentioned above. Reviewers: niravd, spatel, craig.topper Reviewed By: niravd Differential Revision: https://reviews.llvm.org/D61397 llvm-svn: 360171
* Avoid use-after-move warnings by using swap instead. NFCI.Simon Pilgrim2019-05-071-2/+5
| | | | | | Swap should be as quick in these cases, and leaves the original variables in a known (empty) state. llvm-svn: 360164
* [FastISel][X86] If selectFNeg fails, fall back to SelectionDAG not treating ↵Craig Topper2019-05-071-8/+9
| | | | | | | | | | | | | | | | | | | | | it as an fsub. Summary: If fneg lowering for fsub -0.0, x fails we currently fall back to treating it as an fsub. This has different behavior for nans than the xor with sign bit trick we normally try to do. On X86, the xor trick for double fails fast-isel in 32-bit mode with sse2 due to 64 bit integer types not being available. With -O2 we would always use an xorpd for this case. If we use subsd, this creates an observable behavior difference between -O0 and -O2. So fall back to SelectionDAG if we can't fast-isel it, that way SelectionDAG will use the xorpd. I believe this patch is restoring the behavior prior to r345295 from last October. This was missed then because our fast isel case in 32-bit mode aborted fast-isel earlier for another reason. But I've added new tests to cover that. Reviewers: andrew.w.kaylor, cameron.mcinally, spatel, efriedma Reviewed By: cameron.mcinally Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D61622 llvm-svn: 360111
* [FastISel] Pass the fneg input operand to hasTrivialKill in ↵Craig Topper2019-05-061-1/+1
| | | | | | | | FastISel::selectFNeg. We're trying to calculate the kill flag for OpReg which is the input so we need to pass the input here. llvm-svn: 360097
* Fix pr33010, a 2 year old crashing regressionPhilip Reames2019-05-061-0/+4
| | | | | | | | The problem was that we were creating a CMOV64rr <TargetFrameIndex>, <TargetFrameIndex>. The entire point of a TFI is that address code is not generated, so there's no way to legalize/lower this. Instead, simply prevent it's creation. Arguably, we shouldn't be using *Target*FrameIndices in StatepointLowering at all, but that's a much deeper change. llvm-svn: 360090
* [SelectionDAG][X86] Support inline assembly returning an mmx register into a ↵Craig Topper2019-05-061-0/+8
| | | | | | | | | | | | | | | | | | type with fewer than 64 bits. It's possible to use the 'y' mmx constraint with a type narrower than 64-bits. This patch supports this by bitcasting the mmx type to 64-bits and then truncating to the desired type. There are probably other missing type combinations we need to support, but this is the case we have a bug report for. Fixes PR41748. Differential Revision: https://reviews.llvm.org/D61582 llvm-svn: 360069
* Revert r359392 and r358887Craig Topper2019-05-061-25/+1
| | | | | | | | | | | | | | | | | | | | Reverts "[X86] Remove (V)MOV64toSDrr/m and (V)MOVDI2SSrr/m. Use 128-bit result MOVD/MOVQ and COPY_TO_REGCLASS instead" Reverts "[TargetLowering][AMDGPU][X86] Improve SimplifyDemandedBits bitcast handling" Eric Christopher and Jorge Gorbe Moya reported some issues with these patches to me off list. Removing the CodeGenOnly instructions has changed how fneg is handled during fast-isel with sse/sse2. We're now emitting fsub -0.0, x instead moving to the integer domain(in a GPR), xoring the sign bit, and then moving back to xmm. This is because the fast isel table no longer contains an entry for (f32/f64 bitcast (i32/i64)) so the target independent fneg code fails. The use of fsub changes the behavior of nan with respect to -O2 codegen which will always use a pxor. NOTE: We still have a difference with double with -m32 since the move to GPR doesn't work there. I'll file a separate PR for that and add test cases. Since removing the CodeGenOnly instructions was fixing PR41619, I'm reverting r358887 which exposed that PR. Though I wouldn't be surprised if that bug can still be hit independent of that. This should hopefully get Google back to green. I'll work with Simon and other X86 folks to figure out how to move forward again. llvm-svn: 360066
* [SDAG][AArch64] Boolean and/or reduce to umax/min reduce (PR41635)Nikita Popov2019-05-061-0/+12
| | | | | | | | | | | This addresses one half of https://bugs.llvm.org/show_bug.cgi?id=41635 by combining a VECREDUCE_AND/OR into VECREDUCE_UMIN/UMAX (if latter is legal but former is not) for zero-or-all-ones boolean reductions (which are detected based on sign bits). Differential Revision: https://reviews.llvm.org/D61398 llvm-svn: 360054
* [SelectionDAG] Replace llvm_unreachable at the end of getCopyFromParts with ↵Craig Topper2019-05-061-1/+1
| | | | | | | | | | | | | | | | a report_fatal_error. Based on PR41748, not all cases are handled in this function. llvm_unreachable is treated as an optimization hint than can prune code paths in a release build. This causes weird behavior when PR41748 is encountered on a release build. It appears to generate an fp_round instruction from the floating point code. Making this a report_fatal_error prevents incorrect optimization of the code and will instead generate a message to file a bug report. llvm-svn: 360008
* [SelectionDAG] Use any_of/all_of where possible. NFCI.Simon Pilgrim2019-05-051-14/+4
| | | | llvm-svn: 359974
* [DAGCombine] Remove repeated variables. NFCI.Simon Pilgrim2019-05-031-8/+3
| | | | llvm-svn: 359915
* [TargetLowering] SimplifySetCC - remove repeated variable. NFCI.Simon Pilgrim2019-05-031-2/+1
| | | | | | Also reduce scope of Temp variable. llvm-svn: 359911
* [SelectionDAG] CreateTopologicalOrder - don't use iteratorSimon Pilgrim2019-05-031-10/+6
| | | | | | | | We shouldn't use an iterator to loop across a std::vector when the same loop is adding elements to that std::vector Found by cppcheck llvm-svn: 359900
* [TargetLowering] ShrinkDemandedConstant - reduce scope of TLO.DAG variable. ↵Simon Pilgrim2019-05-031-3/+2
| | | | | | | | NFCI. Only ever used in one block llvm-svn: 359890
* [TargetLowering] expandUnalignedStore - cleanup EVT variables. NFCI.Simon Pilgrim2019-05-031-23/+18
| | | | | | Avoid duplicated EVTs and rename Store/Load VTs to avoid -Wshadow warnings. llvm-svn: 359877
* [SelectionDAG] Use INT_MIN as (1 << 31) is UB for signed integers. NFCI.Simon Pilgrim2019-05-031-2/+2
| | | | llvm-svn: 359873
* [SelectionDAG] computeKnownBits - remove some duplicate/shadow variables. NFCI.Simon Pilgrim2019-05-031-6/+4
| | | | llvm-svn: 359872
* [SelectionDAG] Add asserts to verify the vectorness of input and output ↵Craig Topper2019-05-021-0/+12
| | | | | | | | | | types of TRUNCATE/ZERO_EXTEND/ANY_EXTEND/SIGN_EXTEND agree As a result of the underlying cause of PR41678 we created an ANY_EXTEND node with a scalar result type and v1i1 input type. Ideally we would have asserted for this instead of letting it go through to instruction selection and generate bad machine IR Differential Revision: https://reviews.llvm.org/D61463 llvm-svn: 359836
* [DAGCombiner] try repeated fdiv divisor transform before building estimate ↵Sanjay Patel2019-05-021-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | (2nd try) The original patch was committed at rL359398 and reverted at rL359695 because of infinite looping. This includes a fix to check for a vector splat of "1.0" to avoid the infinite loop. Original commit message: This was originally part of D61028, but it's an independent diff. If we try the repeated divisor reciprocal transform before producing an estimate sequence, then we have an opportunity to use scalar fdiv. On x86, the trade-off is 1 divss vs. 5 vector FP ops in the default estimate sequence. On recent chips (Skylake, Ryzen), the full-precision division is only 3 cycle throughput, so that's probably the better perf default option and avoids problems from x86's inaccurate estimates. The last 2 tests show that users still have the option to override the defaults by using the function attributes for reciprocal estimates, but those patterns are potentially made faster by converting the vector ops (including ymm ops) to scalar math. Differential Revision: https://reviews.llvm.org/D61149 llvm-svn: 359793
* [SelectionDAG] remove constant folding limitations based on FP exceptionsSanjay Patel2019-05-021-26/+16
| | | | | | | | | | | | | | | | | We don't have FP exception limits in the IR constant folder for the binops (apart from strict ops), so it does not make sense to have them here in the DAG either. Nothing else in the backend tries to preserve exceptions (again outside of strict ops), so I don't see how this could have ever worked for real code that cares about FP exceptions. There are still cases (examples: unary opcodes in SDAG, FMA in IR) where we are trying (at least partially) to preserve exceptions without even asking if the target supports FP exceptions. Those should be corrected in subsequent patches. Real support for FP exceptions requires several changes to handle the constrained/strict FP ops. Differential Revision: https://reviews.llvm.org/D61331 llvm-svn: 359791
* Revert "[DAGCombiner] try repeated fdiv divisor transform before building ↵Sanjay Patel2019-05-011-3/+3
| | | | | | | | | estimate" This reverts commit fb9a5307a94e6f1f850e4d89f79103b123f16279 (rL359398) because it can cause an infinite loop due to opposing combines. llvm-svn: 359695
* DAG: allow DAG pointer size different from memory representation.Tim Northover2019-05-012-44/+120
| | | | | | | | | | | | | | | | | | | | | In preparation for supporting ILP32 on AArch64, this modifies the SelectionDAG builder code so that pointers are allowed to have a larger type when "live" in the DAG compared to memory. Pointers get zero-extended whenever they are loaded, and truncated prior to stores. In addition, a few not quite so obvious locations need updating: * A GEP that has not been marked inbounds needs to enforce the IR-documented 2s-complement wrapping at the memory pointer size. Inbounds GEPs are undefined if they overflow the address space, so no additional operations are needed. * Signed comparisons would give incorrect results if performed on the zero-extended values. This shouldn't affect CodeGen for now, but will become active when the AArch64 ILP32 support is committed. llvm-svn: 359676
* [SelectionDAG] remove div-by-zero constant folding restrictionSanjay Patel2019-04-301-7/+3
| | | | | | | | | | | | | | | | We don't have this restriction in IR, so it should not be here either simply out of consistency. Code that wants to handle FP exceptions is expected to use the 'strict' variants of these nodes. We don't get the frem case because frem by 0.0 produces NaN (invalid), and that's the remaining check here (so the removed check for frem was dead code AFAIK). This is the only place in SDAG that uses "HasFPExceptions", so I think we should remove that entirely as a follow-up patch. llvm-svn: 359566
* [TargetLowering] findOptimalMemOpLowering. NFCI.Sjoerd Meijer2019-04-302-123/+119
| | | | | | | | | | This was a local static funtion in SelectionDAG, which I've promoted to TargetLowering so that I can reuse it to estimate the cost of a memory operation in D59787. Differential Revision: https://reviews.llvm.org/D59766 llvm-svn: 359543
* [TargetLowering] Change getOptimalMemOpType to take a function attribute listSjoerd Meijer2019-04-301-1/+2
| | | | | | | | | | | | The MachineFunction wasn't used in getOptimalMemOpType, but more importantly, this allows reuse of findOptimalMemOpLowering that is calling getOptimalMemOpType. This is the groundwork for the changes in D59766 and D59787, that allows implementation of TTI::getMemcpyCost. Differential Revision: https://reviews.llvm.org/D59785 llvm-svn: 359537
* [DAGCombiner] Do not generate ISD::ADDE node if adde is not legal for the ↵Zi Xuan Wu2019-04-301-1/+3
| | | | | | | | | | | | | | | target when combine ISD::TRUNC node Do not combine (trunc adde(X, Y, Carry)) into (adde trunc(X), trunc(Y), Carry), if adde is not legal for the target. Even it's at type-legalize phase. Because adde is special and will not be legalized at operation-legalize phase later. This fixes: PR40922 https://bugs.llvm.org/show_bug.cgi?id=40922 Differential Revision: https://reviews.llvm.org//D60854 llvm-svn: 359532
* [DAG] Refactor DAGCombiner::ReassociateOpsBjorn Pettersson2019-04-291-45/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Extract the logic for doing reassociations from DAGCombiner::reassociateOps into a helper function DAGCombiner::reassociateOpsCommutative, and use that helper to trigger reassociation on the original operand order, or the commuted operand order. Codegen is not identical since the operand order will be different when doing the reassociations for the commuted case. That causes some unfortunate churn in some test cases. Apart from that this should be NFC. Reviewers: spatel, craig.topper, tstellar Reviewed By: spatel Subscribers: dmgreen, dschuff, jvesely, nhaehnle, javed.absar, sbc100, jgravelle-google, hiraditya, aheejin, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D61199 llvm-svn: 359476
* [DAGCombiner] try repeated fdiv divisor transform before building estimateSanjay Patel2019-04-281-3/+3
| | | | | | | | | | | | | | | | | | This was originally part of D61028, but it's an independent diff. If we try the repeated divisor reciprocal transform before producing an estimate sequence, then we have an opportunity to use scalar fdiv. On x86, the trade-off is 1 divss vs. 5 vector FP ops in the default estimate sequence. On recent chips (Skylake, Ryzen), the full-precision division is only 3 cycle throughput, so that's probably the better perf default option and avoids problems from x86's inaccurate estimates. The last 2 tests show that users still have the option to override the defaults by using the function attributes for reciprocal estimates, but those patterns are potentially made faster by converting the vector ops (including ymm ops) to scalar math. Differential Revision: https://reviews.llvm.org/D61149 llvm-svn: 359398
* [DAGCombine] Cleanup visitEXTRACT_SUBVECTOR. NFCI.Simon Pilgrim2019-04-261-10/+11
| | | | | | Use ArrayRef::slice, reduce some rather awkward long lines for legibility and run clang-format. llvm-svn: 359326
* [X86][SSE] Disable shouldFoldConstantShiftPairToMask for btver1/btver2 ↵Simon Pilgrim2019-04-261-0/+3
| | | | | | | | | | targets (PR40758) As detailed on PR40758, Bobcat/Jaguar can perform vector immediate shifts on the same pipes as vector ANDs with the same latency - so it doesn't make sense to replace a shl+lshr with a shift+and pair as it requires an additional mask (with the extra constant pool, loading and register pressure costs). Differential Revision: https://reviews.llvm.org/D61068 llvm-svn: 359293
* [SelectionDAG][X86] Use stack load/store in PromoteIntRes_BITCAST when the ↵Craig Topper2019-04-251-15/+18
| | | | | | | | | | | | | | | | | | input needs to be be split and the output type is a vector. We had special case handling here, but it uses a scalar any_extend for the promotion then bitcasts to the final type. This won't split up the input data into multiple promoted elements like we need. This patch falls back to doing the conversion through memory. Fixes PR41594 which I believe was reflected in the bitcast-vector-bool.ll changes. The changes to vector-half-conversions.ll are fixing a previously unknown miscompile from this issue. Differential Revision: https://reviews.llvm.org/D61114 llvm-svn: 359219
* Recommitting r358783 and r358786 "[MS] Emit S_HEAPALLOCSITE debug info" with ↵Amy Huang2019-04-241-0/+6
| | | | | | | | | | | | | | | | | | | fixes for buildbot error (undefined assembler label). Summary: This emits labels around heapallocsite calls and S_HEAPALLOCSITE debug info in codeview. Currently only changes FastISel, so emitting labels still needs to be implemented in SelectionDAG. Reviewers: rnk Subscribers: aprantl, hiraditya, cfe-commits, llvm-commits Tags: #clang, #llvm Differential Revision: https://reviews.llvm.org/D61083 llvm-svn: 359149
* [DAGCombiner] scale repeated FP divisor by splat factorSanjay Patel2019-04-241-3/+13
| | | | | | | | | | | | | | If we have a vector FP division with a splatted divisor, use the existing transform that converts 'x/y' into 'x * (1.0/y)' to allow more conversions. This can then potentially be converted into a scalar FP division by existing combines (rL358984) as seen in the tests here. That can be a potentially big perf difference if scalar fdiv has better timing (including avoiding possible frequency throttling for vector ops). Differential Revision: https://reviews.llvm.org/D61028 llvm-svn: 359147
* Add "const" in GetUnderlyingObjects. NFCBjorn Pettersson2019-04-241-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Both the input Value pointer and the returned Value pointers in GetUnderlyingObjects are now declared as const. It turned out that all current (in-tree) uses of GetUnderlyingObjects were trivial to update, being satisfied with have those Value pointers declared as const. Actually, in the past several of the users had to use const_cast, just because of ValueTracking not providing a version of GetUnderlyingObjects with "const" Value pointers. With this patch we get rid of those const casts. Reviewers: hfinkel, materi, jkorous Reviewed By: jkorous Subscribers: dexonsmith, jkorous, jholewinski, sdardis, eraman, hiraditya, jrtc27, atanasyan, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D61038 llvm-svn: 359072
* Revert "[MS] Emit S_HEAPALLOCSITE debug info" because of ToTWin64(db)Amy Huang2019-04-231-6/+0
| | | | | | | | | buildbot failure. This reverts commit d07d6d617713bececf57f3547434dd52f0f13f9e and c774f687b6880484a126ed3e3d737e74c926f0ae. llvm-svn: 359034
* Use llvm::stable_sortFangrui Song2019-04-231-1/+1
| | | | | | While touching the code, simplify if feasible. llvm-svn: 358996
* [DAGCombiner] generalize binop-of-splats scalarizationSanjay Patel2019-04-231-46/+38
| | | | | | | | | | | | | | If we only match build vectors, we can miss some patterns that use shuffles as seen in the affected tests. Note that the underlying calls within getSplatSourceVector() have the potential for compile-time explosion because of exponential recursion looking through binop opcodes, but currently the list of supported opcodes is very limited. Both of those problems should be addressed in follow-up patches. llvm-svn: 358984
* [DAGCombiner] Combine OR as ADD when no common bits are setBjorn Pettersson2019-04-231-16/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The DAGCombiner is rewriting (canonicalizing) an ISD::ADD with no common bits set in the operands as an ISD::OR node. This could sometimes result in "missing out" on some combines that normally are performed for ADD. To be more specific this could happen if we already have rewritten an ADD into OR, and later (after legalizations or combines) we expose patterns that could have been optimized if we had seen the OR as an ADD (e.g. reassociations based on ADD). To make the DAG combiner less sensitive to if ADD or OR is used for these "no common bits set" ADD/OR operations we now apply most of the ADD combines also to an OR operation, when value tracking indicates that the operands have no common bits set. Reviewers: spatel, RKSimon, craig.topper, kparzysz Reviewed By: spatel Subscribers: arsenm, rampitec, lebedev.ri, jvesely, nhaehnle, hiraditya, javed.absar, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D59758 llvm-svn: 358965
* [SelectionDAG] move splat util functions up from x86 loweringSanjay Patel2019-04-221-0/+52
| | | | | | | | | | This was supposed to be NFC, but the change in SDLoc definitions causes instruction scheduling changes. There's nothing x86-specific in this code, and it can likely be used from DAGCombiner's simplifyVBinOp(). llvm-svn: 358930
* [TargetLowering][AMDGPU][X86] Improve SimplifyDemandedBits bitcast handlingSimon Pilgrim2019-04-221-1/+25
| | | | | | | | | | | | This patch adds support for BigBitWidth -> SmallBitWidth bitcasts, splitting the DemandedBits/Elts accordingly. The AMDGPU backend needed an extra (srl (and x, c1 << c2), c2) -> (and (srl(x, c2), c1) combine to encourage BFE creation, I investigated putting this in DAGCombine but it caused a lot of noise on other targets - some improvements, some regressions. The X86 changes are all definite wins. Differential Revision: https://reviews.llvm.org/D60462 llvm-svn: 358887
* [DAGCombiner] make variable name less ambiguous; NFCSanjay Patel2019-04-221-4/+4
| | | | llvm-svn: 358886
* [DAGCombiner] prepare shuffle-of-splat to handle more patterns; NFCSanjay Patel2019-04-221-11/+16
| | | | llvm-svn: 358884
OpenPOWER on IntegriCloud