summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG
Commit message (Collapse)AuthorAgeFilesLines
...
* Reland "[SelectionDAG] Enable target specific vector scalarization of calls ↵Simon Dardis2017-06-093-64/+185
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | and returns" By target hookifying getRegisterType, getNumRegisters, getVectorBreakdown, backends can request that LLVM to scalarize vector types for calls and returns. The MIPS vector ABI requires that vector arguments and returns are passed in integer registers. With SelectionDAG's new hooks, the MIPS backend can now handle LLVM-IR with vector types in calls and returns. E.g. 'call @foo(<4 x i32> %4)'. Previously these cases would be scalarized for the MIPS O32/N32/N64 ABI for calls and returns if vector types were not legal. If vector types were legal, a single 128bit vector argument would be assigned to a single 32 bit / 64 bit integer register. By teaching the MIPS backend to inspect the original types, it can now implement the MIPS vector ABI which requires a particular method of scalarizing vectors. Previously, the MIPS backend relied on clang to scalarize types such as "call @foo(<4 x float> %a) into "call @foo(i32 inreg %1, i32 inreg %2, i32 inreg %3, i32 inreg %4)". This patch enables the MIPS backend to take either form for vector types. The previous version of this patch had a "conditional move or jump depends on uninitialized value". Reviewers: zoran.jovanovic, jaydeep, vkalintiris, slthakur Differential Revision: https://reviews.llvm.org/D27845 llvm-svn: 305083
* Prevent RemoveDeadNodes from deleted already deleted node.Nirav Dave2017-06-091-0/+5
| | | | | | | | | | | | | | | | | | | | | This prevents against assertion errors like PR32659 which occur from a replacement deleting a node after it's been added to the list argument of RemoveDeadNodes. The specific failure from PR32659 does not currently happen, but it is still potentially possible. The underlying cause is that the callers of the change dfunction builds up a list of nodes to delete after having moved their uses and it possible that a move of a later node will cause a previously deleted nodes to be deleted. Reviewers: bkramer, spatel, davide Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D33731 llvm-svn: 305070
* [CodeGen] Fix some Clang-tidy modernize-use-using and Include What You Use ↵Eugene Zelenko2017-06-071-6/+9
| | | | | | warnings; other minor fixes (NFC). llvm-svn: 304954
* [DAG] Improve Store Merge candidate pruning. NFC.Nirav Dave2017-06-071-3/+15
| | | | | | | | | When considering merging stores values are the results of loads only consider stores whose values come from loads from the same base. This fixes much of the longer compile times in PR33330. llvm-svn: 304934
* [DAG] Move SelectionDAG::isCommutativeBinOp to TargetLowering.Simon Pilgrim2017-06-073-7/+7
| | | | | | | | This will allow commutation of target-specific DAG nodes in future patches Differential Revision: https://reviews.llvm.org/D33882 llvm-svn: 304911
* [DAG] remove duplicated code for isOnlyUsedInZeroEqualityComparison(); NFCISanjay Patel2017-06-061-15/+1
| | | | llvm-svn: 304822
* Sort the remaining #include lines in include/... and lib/....Chandler Carruth2017-06-069-11/+11
| | | | | | | | | | | | | | | | | | | | | | | | | I did this a long time ago with a janky python script, but now clang-format has built-in support for this. I fed clang-format every line with a #include and let it re-sort things according to the precise LLVM rules for include ordering baked into clang-format these days. I've reverted a number of files where the results of sorting includes isn't healthy. Either places where we have legacy code relying on particular include ordering (where possible, I'll fix these separately) or where we have particular formatting around #include lines that I didn't want to disturb in this patch. This patch is *entirely* mechanical. If you get merge conflicts or anything, just ignore the changes in this patch and run clang-format over your #include lines in the files. Sorry for any noise here, but it is important to keep these things stable. I was seeing an increasing number of patches with irrelevant re-ordering of #include lines because clang-format was used. This patch at least isolates that churn, makes it easy to skip when resolving conflicts, and gets us to a clean baseline (again). llvm-svn: 304787
* [llvm] Remove double semicolonsMandeep Singh Grang2017-06-061-2/+2
| | | | | | | | | | | | Reviewers: craig.topper, arsenm, mehdi_amini Reviewed By: mehdi_amini Subscribers: mehdi_amini, wdng, nhaehnle, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D33924 llvm-svn: 304767
* [SelectionDAG] Update the dominator after splitting critical edges.Davide Italiano2017-06-051-5/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Running `llc -verify-dom-info` on the attached testcase results in a crash in the verifier, due to a stale dominator tree. i.e. DominatorTree is not up to date! Computed: =============================-------------------------------- Inorder Dominator Tree: [1] %safe_mod_func_uint8_t_u_u.exit.i.i.i {0,7} [2] %lor.lhs.false.i61.i.i.i {1,2} [2] %safe_mod_func_int8_t_s_s.exit.i.i.i {3,6} [3] %safe_div_func_int64_t_s_s.exit66.i.i.i {4,5} Actual: =============================-------------------------------- Inorder Dominator Tree: [1] %safe_mod_func_uint8_t_u_u.exit.i.i.i {0,9} [2] %lor.lhs.false.i61.i.i.i {1,2} [2] %safe_mod_func_int8_t_s_s.exit.i.i.i {3,8} [3] %safe_div_func_int64_t_s_s.exit66.i.i.i {4,5} [3] %safe_mod_func_int8_t_s_s.exit.i.i.i.lor.lhs.false.i61.i.i.i_crit_edge {6,7} This is because in `SelectionDAGIsel` we split critical edges without updating the corresponding dominator for the function (and we claim in `MachineFunctionPass::getAnalysisUsage()` that the domtree is preserved). We could either stop preserving the domtree in `getAnalysisUsage` or tell `splitCriticalEdge()` to update it. As the second option is easy to implement, that's the one I chose. Differential Revision: https://reviews.llvm.org/D33800 llvm-svn: 304742
* [DAGCombine] Fix unchecked calls to DAGCombiner::*ExtPromoteOperandSanjay Patel2017-06-051-6/+6
| | | | | | | | | | | | | | | | | Other calls to DAGCombiner::*PromoteOperand check the result, but here it could cause an assertion in getNode. Falling back to any extend in this case instead of failing outright seems correct to me. No test case because: The failure was triggered by an out of tree backend. In order to trigger it, a backend would need to overload TargetLowering::IsDesirableToPromoteOp to return true for a type for which ISD::SIGN_EXTEND_INREG is marked illegal. In tree, only X86 overloads and sometimes returns true for MVT::i16 yet it marks setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::i16 , Legal);. Patch by Jacob Young! Differential Revision: https://reviews.llvm.org/D33633 llvm-svn: 304723
* Added LLVM_FALLTHROUGH to address warning: this statement may fall through. NFC.Galina Kistanova2017-06-031-0/+1
| | | | llvm-svn: 304635
* [CodeGen] Fix Windows builds which treat warnings as errors, broken in r304621.Eugene Zelenko2017-06-031-1/+1
| | | | llvm-svn: 304627
* [CodeGen] Fix some Clang-tidy modernize-use-using and Include What You Use ↵Eugene Zelenko2017-06-031-59/+62
| | | | | | warnings; other minor fixes (NFC). llvm-svn: 304621
* [Statepoint] Be consistent about using deopt naming [NFCI]Philip Reames2017-06-021-1/+1
| | | | | | We'd called this "vm state" in the early days, but have long since standardized on calling it "deopt" in line with the operand bundle tag. Fix a few cases we'd missed. llvm-svn: 304607
* [TargetLowering] fix formatting; NFCSanjay Patel2017-06-021-2/+1
| | | | llvm-svn: 304569
* nits in TargetLowering.cpp . NFCAmaury Sechet2017-06-021-13/+20
| | | | llvm-svn: 304532
* [SelectionDAG] Get rid of recursion in findNonImmUseMax Kazantsev2017-06-021-20/+26
| | | | | | | | | | | | The recursive implementation of findNonImmUse may overflow stack on extremely long use chains. This patch replaces it with an equivalent iterative implementation. Reviewed By: bogner Differential Revision: https://reviews.llvm.org/D33775 llvm-svn: 304522
* [SDAG] Fix CombineTo ordering in visitZERO_EXTEND and visitSIGN_EXTENDNirav Dave2017-06-011-15/+8
| | | | | | | | | | | | Reorder CombineTo Calls to prevent references to stale/deleted SDNodes which caused undue assertions. Reviewers: dbabokin Subscribers: aemerson, rengolin, llvm-commits Differential Revision: https://reviews.llvm.org/D31625 llvm-svn: 304460
* DAG: Remove pointless type checkMatt Arsenault2017-06-011-1/+1
| | | | | | These are only integer operations. llvm-svn: 304417
* Only generate addcarry node when it is legal.Amaury Sechet2017-06-011-7/+11
| | | | | | | | | | | | | | | Summary: This is a problem uncovered by stage2 testing. ADDCARRY end up being generated on target that do not support it. The patch that introduced the problem has other patches layed on top of it, so we want to fix the issue rather than revert it to avoid creating a lor of churn. A regression test will be added shortly, but this is committed as this in order to get the build back to green promptly. Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D33770 llvm-svn: 304409
* Do not legalize large setcc with setcce, introduce setcccarry and do it with ↵Amaury Sechet2017-06-014-24/+63
| | | | | | | | | | | | | | | | | usubo/setcccarry. Summary: This is a continuation of the work started in D29872 . Passing the carry down as a value rather than as a glue allows for further optimizations. Introducing setcccarry makes the use of addc/subc unecessary and we can start the removal process. This patch only introduce the optimization strictly required to get the same level of optimization as was available before nothing more. Reviewers: jyknight, nemanjai, mkuper, spatel, RKSimon, zvi, bkramer Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D33374 llvm-svn: 304404
* [DAGCombine] Refactor common addcarry pattern.Amaury Sechet2017-06-011-0/+35
| | | | | | | | | | | | Summary: This pattern is no very useful per se, but it exposes optimization for toehr patterns that wouldn't kick in otherwize. It's very common and worth optimizing for. Reviewers: jyknight, nemanjai, mkuper, spatel, RKSimon, zvi, bkramer Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D32756 llvm-svn: 304402
* [DAGCombine] (add/uaddo X, Carry) -> (addcarry X, 0, Carry)Amaury Sechet2017-06-011-0/+49
| | | | | | | | | | | | | | | Summary: This enables further transforms. Depends on D32916 Reviewers: jyknight, nemanjai, mkuper, spatel, RKSimon, zvi, bkramer Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D32925 llvm-svn: 304401
* [ScheduleDAG] Deal with already scheduled loads in ScheduleDAG.Nirav Dave2017-05-311-128/+150
| | | | | | | | | | | | | | | | | Summary: If we attempt to unfold an SUnit in ScheduleDAG that results in finding an already scheduled load, we must should abort the unfold as it will not improve scheduling. This fixes PR32610. Reviewers: jmolloy, sunfish, bogner, spatel Subscribers: llvm-commits, MatzeB Differential Revision: https://reviews.llvm.org/D32911 llvm-svn: 304321
* [DAG] Avoid use of stale store.Nirav Dave2017-05-311-2/+2
| | | | | | | | | | | | | | | | | | Correct references to alignment of store which may be deleted in a previous iteration of merge. Instead use first store that would be merged. Corrects pr33172's use-after-poison caught by ASan. Reviewers: spatel, hfinkel, RKSimon Reviewed By: RKSimon Subscribers: thegameg, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D33686 llvm-svn: 304299
* [SelectionDAG] Remove special case for ISD::FPOWI from the strict FP ↵Craig Topper2017-05-301-4/+0
| | | | | | | | intrinsic handling. This code was compensating for FPOWI defaulting to Legal and many targets not changing it to Expand. This was fixed in r304215 to default to Expand so this special handling should no longer be necessary. llvm-svn: 304221
* [SelectionDAG] Set ISD::FPOWI to Expand by defaultCraig Topper2017-05-301-1/+0
| | | | | | | | | | | | | | | | | Summary: Currently FPOWI defaults to Legal and LegalizeDAG.cpp turns Legal into Expand for this opcode because Legal is a "lie". This patch changes the default for this opcode to Expand and removes the hack from LegalizeDAG.cpp. It also removes all the code in the targets that set this opcode to Expand themselves since they can just rely on the default. Reviewers: spatel, RKSimon, efriedma Reviewed By: RKSimon Subscribers: jfb, dschuff, sbc100, jgravelle-google, nemanjai, javed.absar, andrew.w.kaylor, llvm-commits Differential Revision: https://reviews.llvm.org/D33530 llvm-svn: 304215
* [DAGCombiner] fix load narrowing transform to exclude loads with extensionSanjay Patel2017-05-291-1/+2
| | | | | | | | | | The extending load possibility was missed in: https://reviews.llvm.org/rL304072 We might want to handle this cases as a follow-up, but bailing out for now to avoid miscompiling. llvm-svn: 304153
* [DAGCombiner] use narrow load to avoid vector extractSanjay Patel2017-05-271-0/+50
| | | | | | | | | | | | | | | | | | If we have (extract_subvector(load wide vector)) with no other users, that can just be (load narrow vector). This is intentionally conservative. Follow-ups may loosen the one-use constraint to account for the extract cost or just remove the one-use check. The memop chain updating is based on code that already exists multiple times in x86 lowering, so that should be pulled into a helper function as a follow-up. Background: this is a potential improvement noticed via regressions caused by making x86's peekThroughBitcasts() not loop on consecutive bitcasts (see comments in D33137). Differential Revision: https://reviews.llvm.org/D33578 llvm-svn: 304072
* Make helper functions static. NFC.Benjamin Kramer2017-05-261-5/+6
| | | | llvm-svn: 304029
* [DAGCombiner] use narrow vector ops to eliminate concat/extract (PR32790)Sanjay Patel2017-05-261-0/+96
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In the best case: extract (binop (concat X1, X2), (concat Y1, Y2)), N --> binop XN, YN ...we kill all of the extract/concat and just have narrow binops remaining. If only one of the binop operands is amenable, this transform is still worthwhile because we kill some of the extract/concat. Optional bitcasting makes the code more complicated, but there doesn't seem to be a way to avoid that. The TODO about extending to more than bitwise logic is there because we really will regress several x86 tests including madd, psad, and even a plain integer-multiply-by-2 or shift-left-by-1. I don't think there's anything fundamentally wrong with this patch that would cause those regressions; those folds are just missing or brittle. If we extend to more binops, I found that this patch will fire on at least one non-x86 regression test. There's an ARM NEON test in test/CodeGen/ARM/coalesce-subregs.ll with a pattern like: t5: v2f32 = vector_shuffle<0,3> t2, t4 t6: v1i64 = bitcast t5 t8: v1i64 = BUILD_VECTOR Constant:i64<0> t9: v2i64 = concat_vectors t6, t8 t10: v4f32 = bitcast t9 t12: v4f32 = fmul t11, t10 t13: v2i64 = bitcast t12 t16: v1i64 = extract_subvector t13, Constant:i32<0> There was no functional change in the codegen from this transform from what I could see though. For the x86 test changes: 1. PR32790() is the closest call. We don't reduce the AVX1 instruction count in that case, but we improve throughput. Also, on a core like Jaguar that double-pumps 256-bit ops, there's an unseen win because two 128-bit ops have the same cost as the wider 256-bit op. SSE/AVX2/AXV512 are not affected which is expected because only AVX1 has the extract/concat ops to match the pattern. 2. do_not_use_256bit_op() is the best case. Everyone wins by avoiding the concat/extract. Related bug for IR filed as: https://bugs.llvm.org/show_bug.cgi?id=33026 3. The SSE diffs in vector-trunc-math.ll are just scheduling/RA, so nothing real AFAICT. 4. The AVX1 diffs in vector-tzcnt-256.ll are all the same pattern: we reduced the instruction count by one in each case by eliminating two insert/extract while adding one narrower logic op. https://bugs.llvm.org/show_bug.cgi?id=32790 Differential Revision: https://reviews.llvm.org/D33137 llvm-svn: 303997
* [DAG] Move legal type checks in store merge to be checked onlyNirav Dave2017-05-261-2/+4
| | | | | | on non-legal cases. NFC. llvm-svn: 303994
* [ARM] Fix lowering of misaligned memcpy/memsetJohn Brawn2017-05-261-12/+12
| | | | | | | | | | | | | | | | | Currently getOptimalMemOpType returns i32 for large enough sizes without checking for alignment, leading to poor code generation when misaligned accesses aren't permitted as we generate a word store then later split it up into byte stores. This means we inadvertantly go over the MaxStoresPerMemcpy limit and for memset we splat the memset value into a word then immediately split it up again. Fix this by leaving it up to FindOptimalMemOpLowering to figure out which type to use, but also fix a bug there where it wasn't correctly checking if misaligned memory accesses are allowed. Differential Revision: https://reviews.llvm.org/D33442 llvm-svn: 303990
* Add constrained intrinsics for some libm-equivalent operationsAndrew Kaylor2017-05-255-61/+191
| | | | | | Differential revision: https://reviews.llvm.org/D32319 llvm-svn: 303922
* Fix SelectionDAGBuilder::getDbgValue to not expect DW_OP_deref on FI varsAdrian Prantl2017-05-251-14/+5
| | | | | | | | | | | | This fixes an oversight in r300522, which changed alloca dbg.values to no longer emit a DW_OP_deref. The array.ll testcase was regenerated from source. Fixes PR33166: https://bugs.llvm.org/show_bug.cgi?id=33166 llvm-svn: 303897
* [DAG] Prevent crashes when merging constant stores with high-bit set. NFC.Nirav Dave2017-05-241-2/+2
| | | | llvm-svn: 303802
* Revert LLVM changes for "Sema: allow imaginary constants via GNU extension ↵Tim Northover2017-05-231-4/+1
| | | | | | | | if UDL overloads not present." The changes accidentally crept into a Clang commit I was making. llvm-svn: 303697
* Sema: allow imaginary constants via GNU extension if UDL overloads not present.Tim Northover2017-05-231-1/+4
| | | | | | | | | | | | | C++14 added user-defined literal support for complex numbers so that you can write something like "complex<double> val = 2i". However, there is an existing GNU extension supporting this syntax and interpreting the result as a _Complex type. This changes parsing so that such literals are interpreted in terms of C++14's operators if an overload is present but otherwise falls back to the original GNU extension. llvm-svn: 303694
* [DAG] Add AddressSpace parameter to canMergeStoresTo. NFC.Nirav Dave2017-05-231-7/+10
| | | | llvm-svn: 303673
* [DAG] Add canMergeStoresTo predicate checks. NFCI.Nirav Dave2017-05-231-4/+6
| | | | | | Propagate canMergeStoresTo checks to missing cases in StoreMerge. llvm-svn: 303668
* [KnownBits] Use !hasConflict() in asserts in place of Zero & One == 0 or ↵Craig Topper2017-05-231-17/+17
| | | | | | similar. NFC llvm-svn: 303614
* [DAG] Rework store merge to loop on load candidates. NFCI.Nirav Dave2017-05-221-202/+225
| | | | | | | Continue to consider remaining candidate merges until all possible merges have been considered. llvm-svn: 303560
* SimplifyLibCalls: Optimize wcslenMatthias Braun2017-05-191-19/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | Refactor the strlen optimization code to work for both strlen and wcslen. This especially helps with programs in the wild where people pass L"string"s to const std::wstring& function parameters and the wstring constructor gets inlined. This also fixes a lingerind API problem/bug in getConstantStringInfo() where zeroinitializers would always give you an empty string (without a length) back regardless of the actual length of the initializer which did not work well in the TrimAtNul==false causing the PR mentioned below. Note that the fixed getConstantStringInfo() needed fixes to SelectionDAG memcpy lowering and may lead to some cases for out-of-bounds zeroinitializer accesses not getting optimized anymore. So some code with UB may produce out of bound memory reads now instead of just producing zeros. The refactoring "accidentally" fixes http://llvm.org/PR32124 Differential Revision: https://reviews.llvm.org/D32839 llvm-svn: 303461
* [DAGCombine] (addcarry 0, 0, X) -> (ext/trunc X)Amaury Sechet2017-05-191-0/+11
| | | | | | | | | | | | | | | Summary: While this makes some case better and some case worse - so it's unclear if it is a worthy combine just by itself - this is a useful canonicalisation. As per discussion in D32756 . Reviewers: jyknight, nemanjai, mkuper, spatel, RKSimon, zvi, bkramer Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D32916 llvm-svn: 303441
* [Statistics] Add a method to atomically update a statistic that contains a ↵Craig Topper2017-05-181-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | maximum Summary: There are several places in the codebase that try to calculate a maximum value in a Statistic object. We currently do this in one of two ways: MaxNumFoo = std::max(MaxNumFoo, NumFoo); or MaxNumFoo = (MaxNumFoo > NumFoo) ? MaxNumFoo : NumFoo; The first version reads from MaxNumFoo one time and uncontionally rwrites to it. The second version possibly reads it twice depending on the result of the first compare. But we have no way of knowing if the value was changed by another thread between the reads and the writes. This patch adds a method to the Statistic object that can ensure that we only store if our value is the max and the previous max didn't change after we read it. If it changed we'll recheck if our value should still be the max or not and try again. This spawned from an audit I'm trying to do of all places we uses the implicit conversion to unsigned on the Statistics objects. See my previous thread on llvm-dev https://groups.google.com/forum/#!topic/llvm-dev/yfvxiorKrDQ Reviewers: dberlin, chandlerc, hfinkel, dblaikie Reviewed By: chandlerc Subscribers: llvm-commits, sanjoy Differential Revision: https://reviews.llvm.org/D33301 llvm-svn: 303318
* Elide stores which are overwritten without being observed.Nirav Dave2017-05-161-7/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: In SelectionDAG, when a store is immediately chained to another store to the same address, elide the first store as it has no observable effects. This is causes small improvements dealing with intrinsics lowered to stores. Test notes: * Many testcases overwrite store addresses multiple times and needed minor changes, mainly making stores volatile to prevent the optimization from optimizing the test away. * Many X86 test cases optimized out instructions associated with associated with va_start. * Note that test_splat in CodeGen/AArch64/misched-stp.ll no longer has dependencies to check and can probably be removed and potentially replaced with another test. Reviewers: rnk, john.brawn Subscribers: aemerson, rengolin, qcolombet, jyknight, nemanjai, nhaehnle, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D33206 llvm-svn: 303198
* [DAG] Prune deleted nodes in TokenFactorNirav Dave2017-05-161-8/+3
| | | | | | Fix visitTokenFactor to correctly remove deleted nodes. NFC. llvm-svn: 303181
* IR: Give function GlobalValue::getRealLinkageName() a less misleading name: ↵Peter Collingbourne2017-05-161-2/+2
| | | | | | | | | | | | dropLLVMManglingEscape(). This function gives the wrong answer on some non-ELF platforms in some cases. The function that does the right thing lives in Mangler.h. To try to discourage people from using this function, give it a different name. Differential Revision: https://reviews.llvm.org/D33162 llvm-svn: 303134
* [SelectionDAG] Added support for EXTRACT_SUBVECTOR/CONCAT_VECTORS ↵Simon Pilgrim2017-05-131-7/+29
| | | | | | demandedelts in ComputeNumSignBits llvm-svn: 302997
* [SelectionDAG] Add VECTOR_SHUFFLE support to ComputeNumSignBitsSimon Pilgrim2017-05-131-0/+34
| | | | llvm-svn: 302993
OpenPOWER on IntegriCloud