summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/SelectionDAG
Commit message (Collapse)AuthorAgeFilesLines
...
* [DAG] Move BaseIndexOffset into separate Libarary. NFC.Nirav Dave2017-06-213-114/+97
| | | | | | | Move BaseIndexOffset analysis out of DAGCombiner for use in other files. llvm-svn: 305921
* [DAG] Remove Node csonstruction from BaseIndexOffset match. NFCI.Nirav Dave2017-06-211-52/+69
| | | | | | | | Move GlobalAddress Offset decomposition from initial match into comparision check and removing the possibility of constructing a new offseted global address when examining addresses. llvm-svn: 305917
* [DAGCombiner] Add another combine from build vector to shuffleGuy Blank2017-06-211-0/+5
| | | | | | | Add support for combining a build vector to a shuffle. When the build vector is of extracted elements from 2 vectors (vec1, vec2) where vec2 is 2 times smaller than vec1. llvm-svn: 305883
* DAG: correctly legalize UMULO.Tim Northover2017-06-201-11/+18
| | | | | | | | | We were incorrectly sign extending into the high word (as you would for SMULO) when legalizing UMULO in terms of a wider full multiplication. Patch by James Duley. llvm-svn: 305800
* [SelectionDAG] Fix an use-after-free issue introduced in r305775.Haojian Wu2017-06-201-2/+2
| | | | | | vector.back() will be invalidated when memory reallocation happens. llvm-svn: 305785
* [SelectionDAG] Get rid of recursion in CalcNodeSethiUllmanNumberMax Kazantsev2017-06-201-19/+59
| | | | | | | | | | The recursive implementation of CalcNodeSethiUllmanNumber may overflow stack on extremely long pred chains. This patch replaces it with an equivalent iterative implementation. Differential Revision: https://reviews.llvm.org/D33769 llvm-svn: 305775
* [DAG] Simplify BaseIndexOffset. NFCI.Nirav Dave2017-06-201-59/+57
| | | | | | Remove tail calls and cleanup codeflow. llvm-svn: 305768
* Allow truncated and extend memory operations in Store Merge. NFCI.Nirav Dave2017-06-191-24/+46
| | | | | | | | | | As all store merges checks are based on the memory operation performed, allow use of truncated stores and extended loads as valid input candidates for merging. Relanding after fixing selection between truncated and normal store. llvm-svn: 305701
* [SelectionDAG] Update Loop info after splitting critical edges.Davide Italiano2017-06-171-6/+9
| | | | | | The analysis is expected to be preserved by SelectionDAG. llvm-svn: 305621
* [SelectionDAG] Use APInt::isSubsetOf. NFCCraig Topper2017-06-162-4/+4
| | | | llvm-svn: 305606
* [SelectionDAG] Use APInt::isNullValue/isOneValue. NFCCraig Topper2017-06-162-5/+5
| | | | llvm-svn: 305605
* [TargetLowering] Use ConstantSDNode::isOne and getSExtValue instead of ↵Craig Topper2017-06-161-6/+6
| | | | | | getting the underlying APInt first. NFC llvm-svn: 305604
* [Atomics] Rename and change prototype for atomic memcpy intrinsicDaniel Neilson2017-06-161-14/+10
| | | | | | | | | | | | | | | | | | Summary: Background: http://lists.llvm.org/pipermail/llvm-dev/2017-May/112779.html This change is to alter the prototype for the atomic memcpy intrinsic. The prototype itself is being changed to more closely resemble the semantics and parameters of the llvm.memcpy intrinsic -- to ease later combination of the llvm.memcpy and atomic memcpy intrinsics. Furthermore, the name of the atomic memcpy intrinsic is being changed to make it clear that it is not a generic atomic memcpy, but specifically a memcpy is unordered atomic. Reviewers: reames, sanjoy, efriedma Reviewed By: reames Subscribers: mzolotukhin, anna, llvm-commits, skatkov Differential Revision: https://reviews.llvm.org/D33240 llvm-svn: 305558
* Revert "[DAG] Allow truncated and extend memory operations in Store Merge. ↵Ahmed Bougacha2017-06-151-21/+10
| | | | | | | | NFCI." This reverts commit r305468, as it caused PR33475. llvm-svn: 305527
* Fold variable into assert.Benjamin Kramer2017-06-151-2/+1
| | | | | | Silences an unused variable warning in Release builds. llvm-svn: 305488
* ISel: Fix FastISel of swifterror valuesArnold Schwaighofer2017-06-153-14/+125
| | | | | | | | | | | | The code assumed that we process instructions in basic block order. FastISel processes instructions in reverse basic block order. We need to pre-assign virtual registers before selecting otherwise we get def-use relationships wrong. This only affects code with swifterror registers. rdar://32659327 llvm-svn: 305484
* [DAG] As StoreMerge now generates only legal nodes remove unecessary guard ↵Nirav Dave2017-06-151-4/+2
| | | | | | when run post-legalization NFCI. llvm-svn: 305477
* [DAG] Defer Pre/Post IndexStore merge to after mergestore. NFCI.Nirav Dave2017-06-151-4/+4
| | | | | | | | In preparation for doing storemerge post-legalization, reorder visitSTORE passes to move pre/post-index combining after store merge. Reordered passes other than store merge are unaffected. llvm-svn: 305473
* [DAG] Allow truncated and extend memory operations in Store Merge. NFCI.Nirav Dave2017-06-151-10/+21
| | | | | | | | As all store merges checks are based on the memory operation performed, allow use of truncated stores and extended loads as valid input candidates for merging. llvm-svn: 305468
* [DAG] Make MergeStores generate legalized stores. NFCI.Nirav Dave2017-06-151-4/+21
| | | | | | | Realized merged stores as truncstores if store will be realized as such by legalization. llvm-svn: 305467
* [DAG] Use correct size for truncated store merge of load. NFCI.Nirav Dave2017-06-151-2/+2
| | | | | | | Avoid non-legal memory ops by checking correct size when merging stores of loads into a extload-truncstore pair. llvm-svn: 305466
* [mips] Fix multiprecision arithmetic.Simon Dardis2017-06-141-4/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | For multiprecision arithmetic on MIPS, rather than using ISD::ADDE / ISD::ADDC, get SelectionDAG to break down the operation into ISD::ADDs and ISD::SETCCs. For MIPS, only the DSP ASE has a carry flag, so in the general case it is not useful to directly support ISD::{ADDE, ADDC, SUBE, SUBC} nodes. Also improve the generation code in such cases for targets with TargetLoweringBase::ZeroOrOneBooleanContent by directly using the result of the comparison node rather than using it in selects. Similarly for ISD::SUBE / ISD::SUBC. Address optimization breakage by moving the generation of MIPS specific integer multiply-accumulate nodes to before legalization. This revolves PR32713 and PR33424. Thanks to Simonas Kazlauskas and Pirama Arumuga Nainar for reporting the issue! Reviewers: slthakur Differential Revision: https://reviews.llvm.org/D33494 llvm-svn: 305389
* [SelectionDAG] Allow sin/cos -> sincos optimization on GNU triples w/ just ↵Geoff Berry2017-06-121-14/+1
| | | | | | | | | | | | | | | | | | | | | -fno-math-errno Summary: This change enables the sin(x) cos(x) -> sincos(x) optimization on GNU target triples. This optimization was being inhibited when -ffast-math wasn't set because sincos in GLibC does not set errno, while sin and cos do. However, this optimization will only run if the attributes on the sin/cos calls include readnone, which is how clang represents the fact that it doesn't care about the errno values set by these functions (via the -fno-math-errno flag). Reviewers: hfinkel, bogner Subscribers: mcrosier, javed.absar, llvm-commits, paul.redmond Differential Revision: https://reviews.llvm.org/D32921 llvm-svn: 305204
* [DAG] add helper to bind memop chains; NFCISanjay Patel2017-06-122-15/+19
| | | | | | | | | | This step is just intended to reduce code duplication rather than change any functionality. A follow-up would be to replace PPCTargetLowering::spliceIntoChain() usage with this new helper. Differential Revision: https://reviews.llvm.org/D33649 llvm-svn: 305192
* [DAGCombine] Make sure we check the ResNo from UADDO before combiningAmaury Sechet2017-06-111-1/+2
| | | | | | | | | | | | Summary: UADDO has 2 result, and one must check the result no before doing any kind of combine. Without it, the transform is invalid. Reviewers: joerg Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34088 llvm-svn: 305162
* SelectionDAG: Remove deleted nodes from legalized set to avoid clash with ↵Zvi Rackover2017-06-091-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | newly created nodes Summary: During DAG legalization loop in SelectionDAG::Legalize(), bookkeeping of the SDNodes that were already legalized is implemented with SmallPtrSet (LegalizedNodes). This kind of set stores only pointers to objects, not the objects themselves. Unfortunately, if SDNode is deleted during legalization for some reason, LegalizedNodes set is not informed about this fact. This wouldn’t be so bad, if SelectionDAG wouldn’t reuse space deallocated after deletion of unused nodes, for creation of new ones. Because of this, new nodes, created during legalization often can have pointers identical to ones that have been previously legalized, added to the LegalizedNodes set, and deleted afterwards. This in turn causes, that newly created nodes, sharing the same pointer as deleted old ones, are present in LegalizedNodes *already at the moment of creation*, so we never call Legalize on them. The fix facilitates the fact, that DAG notifies listeners about each modification. I have registered DAGNodeDeletedListener inside SelectionDAG::Legalize, with a callback function that removes any pointer of any deleted SDNode from the LegalizedNodes set. With this modification, LegalizeNodes set does not contain pointers to nodes that were deleted, so newly created nodes can always be inserted to it, even if they share pointers with old deleted nodes. Patch by pawel.szczerbuk@intel.com The issue this patch addresses causes failures in an out-of-tree target, and i was not able to create a reproducer for an in-tree target, hence there is no test-case. Reviewers: delena, spatel, RKSimon, hfinkel, davide, qcolombet Reviewed By: delena Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D33891 llvm-svn: 305084
* Reland "[SelectionDAG] Enable target specific vector scalarization of calls ↵Simon Dardis2017-06-093-64/+185
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | and returns" By target hookifying getRegisterType, getNumRegisters, getVectorBreakdown, backends can request that LLVM to scalarize vector types for calls and returns. The MIPS vector ABI requires that vector arguments and returns are passed in integer registers. With SelectionDAG's new hooks, the MIPS backend can now handle LLVM-IR with vector types in calls and returns. E.g. 'call @foo(<4 x i32> %4)'. Previously these cases would be scalarized for the MIPS O32/N32/N64 ABI for calls and returns if vector types were not legal. If vector types were legal, a single 128bit vector argument would be assigned to a single 32 bit / 64 bit integer register. By teaching the MIPS backend to inspect the original types, it can now implement the MIPS vector ABI which requires a particular method of scalarizing vectors. Previously, the MIPS backend relied on clang to scalarize types such as "call @foo(<4 x float> %a) into "call @foo(i32 inreg %1, i32 inreg %2, i32 inreg %3, i32 inreg %4)". This patch enables the MIPS backend to take either form for vector types. The previous version of this patch had a "conditional move or jump depends on uninitialized value". Reviewers: zoran.jovanovic, jaydeep, vkalintiris, slthakur Differential Revision: https://reviews.llvm.org/D27845 llvm-svn: 305083
* Prevent RemoveDeadNodes from deleted already deleted node.Nirav Dave2017-06-091-0/+5
| | | | | | | | | | | | | | | | | | | | | This prevents against assertion errors like PR32659 which occur from a replacement deleting a node after it's been added to the list argument of RemoveDeadNodes. The specific failure from PR32659 does not currently happen, but it is still potentially possible. The underlying cause is that the callers of the change dfunction builds up a list of nodes to delete after having moved their uses and it possible that a move of a later node will cause a previously deleted nodes to be deleted. Reviewers: bkramer, spatel, davide Reviewed By: spatel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D33731 llvm-svn: 305070
* [CodeGen] Fix some Clang-tidy modernize-use-using and Include What You Use ↵Eugene Zelenko2017-06-071-6/+9
| | | | | | warnings; other minor fixes (NFC). llvm-svn: 304954
* [DAG] Improve Store Merge candidate pruning. NFC.Nirav Dave2017-06-071-3/+15
| | | | | | | | | When considering merging stores values are the results of loads only consider stores whose values come from loads from the same base. This fixes much of the longer compile times in PR33330. llvm-svn: 304934
* [DAG] Move SelectionDAG::isCommutativeBinOp to TargetLowering.Simon Pilgrim2017-06-073-7/+7
| | | | | | | | This will allow commutation of target-specific DAG nodes in future patches Differential Revision: https://reviews.llvm.org/D33882 llvm-svn: 304911
* [DAG] remove duplicated code for isOnlyUsedInZeroEqualityComparison(); NFCISanjay Patel2017-06-061-15/+1
| | | | llvm-svn: 304822
* Sort the remaining #include lines in include/... and lib/....Chandler Carruth2017-06-069-11/+11
| | | | | | | | | | | | | | | | | | | | | | | | | I did this a long time ago with a janky python script, but now clang-format has built-in support for this. I fed clang-format every line with a #include and let it re-sort things according to the precise LLVM rules for include ordering baked into clang-format these days. I've reverted a number of files where the results of sorting includes isn't healthy. Either places where we have legacy code relying on particular include ordering (where possible, I'll fix these separately) or where we have particular formatting around #include lines that I didn't want to disturb in this patch. This patch is *entirely* mechanical. If you get merge conflicts or anything, just ignore the changes in this patch and run clang-format over your #include lines in the files. Sorry for any noise here, but it is important to keep these things stable. I was seeing an increasing number of patches with irrelevant re-ordering of #include lines because clang-format was used. This patch at least isolates that churn, makes it easy to skip when resolving conflicts, and gets us to a clean baseline (again). llvm-svn: 304787
* [llvm] Remove double semicolonsMandeep Singh Grang2017-06-061-2/+2
| | | | | | | | | | | | Reviewers: craig.topper, arsenm, mehdi_amini Reviewed By: mehdi_amini Subscribers: mehdi_amini, wdng, nhaehnle, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D33924 llvm-svn: 304767
* [SelectionDAG] Update the dominator after splitting critical edges.Davide Italiano2017-06-051-5/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Running `llc -verify-dom-info` on the attached testcase results in a crash in the verifier, due to a stale dominator tree. i.e. DominatorTree is not up to date! Computed: =============================-------------------------------- Inorder Dominator Tree: [1] %safe_mod_func_uint8_t_u_u.exit.i.i.i {0,7} [2] %lor.lhs.false.i61.i.i.i {1,2} [2] %safe_mod_func_int8_t_s_s.exit.i.i.i {3,6} [3] %safe_div_func_int64_t_s_s.exit66.i.i.i {4,5} Actual: =============================-------------------------------- Inorder Dominator Tree: [1] %safe_mod_func_uint8_t_u_u.exit.i.i.i {0,9} [2] %lor.lhs.false.i61.i.i.i {1,2} [2] %safe_mod_func_int8_t_s_s.exit.i.i.i {3,8} [3] %safe_div_func_int64_t_s_s.exit66.i.i.i {4,5} [3] %safe_mod_func_int8_t_s_s.exit.i.i.i.lor.lhs.false.i61.i.i.i_crit_edge {6,7} This is because in `SelectionDAGIsel` we split critical edges without updating the corresponding dominator for the function (and we claim in `MachineFunctionPass::getAnalysisUsage()` that the domtree is preserved). We could either stop preserving the domtree in `getAnalysisUsage` or tell `splitCriticalEdge()` to update it. As the second option is easy to implement, that's the one I chose. Differential Revision: https://reviews.llvm.org/D33800 llvm-svn: 304742
* [DAGCombine] Fix unchecked calls to DAGCombiner::*ExtPromoteOperandSanjay Patel2017-06-051-6/+6
| | | | | | | | | | | | | | | | | Other calls to DAGCombiner::*PromoteOperand check the result, but here it could cause an assertion in getNode. Falling back to any extend in this case instead of failing outright seems correct to me. No test case because: The failure was triggered by an out of tree backend. In order to trigger it, a backend would need to overload TargetLowering::IsDesirableToPromoteOp to return true for a type for which ISD::SIGN_EXTEND_INREG is marked illegal. In tree, only X86 overloads and sometimes returns true for MVT::i16 yet it marks setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::i16 , Legal);. Patch by Jacob Young! Differential Revision: https://reviews.llvm.org/D33633 llvm-svn: 304723
* Added LLVM_FALLTHROUGH to address warning: this statement may fall through. NFC.Galina Kistanova2017-06-031-0/+1
| | | | llvm-svn: 304635
* [CodeGen] Fix Windows builds which treat warnings as errors, broken in r304621.Eugene Zelenko2017-06-031-1/+1
| | | | llvm-svn: 304627
* [CodeGen] Fix some Clang-tidy modernize-use-using and Include What You Use ↵Eugene Zelenko2017-06-031-59/+62
| | | | | | warnings; other minor fixes (NFC). llvm-svn: 304621
* [Statepoint] Be consistent about using deopt naming [NFCI]Philip Reames2017-06-021-1/+1
| | | | | | We'd called this "vm state" in the early days, but have long since standardized on calling it "deopt" in line with the operand bundle tag. Fix a few cases we'd missed. llvm-svn: 304607
* [TargetLowering] fix formatting; NFCSanjay Patel2017-06-021-2/+1
| | | | llvm-svn: 304569
* nits in TargetLowering.cpp . NFCAmaury Sechet2017-06-021-13/+20
| | | | llvm-svn: 304532
* [SelectionDAG] Get rid of recursion in findNonImmUseMax Kazantsev2017-06-021-20/+26
| | | | | | | | | | | | The recursive implementation of findNonImmUse may overflow stack on extremely long use chains. This patch replaces it with an equivalent iterative implementation. Reviewed By: bogner Differential Revision: https://reviews.llvm.org/D33775 llvm-svn: 304522
* [SDAG] Fix CombineTo ordering in visitZERO_EXTEND and visitSIGN_EXTENDNirav Dave2017-06-011-15/+8
| | | | | | | | | | | | Reorder CombineTo Calls to prevent references to stale/deleted SDNodes which caused undue assertions. Reviewers: dbabokin Subscribers: aemerson, rengolin, llvm-commits Differential Revision: https://reviews.llvm.org/D31625 llvm-svn: 304460
* DAG: Remove pointless type checkMatt Arsenault2017-06-011-1/+1
| | | | | | These are only integer operations. llvm-svn: 304417
* Only generate addcarry node when it is legal.Amaury Sechet2017-06-011-7/+11
| | | | | | | | | | | | | | | Summary: This is a problem uncovered by stage2 testing. ADDCARRY end up being generated on target that do not support it. The patch that introduced the problem has other patches layed on top of it, so we want to fix the issue rather than revert it to avoid creating a lor of churn. A regression test will be added shortly, but this is committed as this in order to get the build back to green promptly. Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D33770 llvm-svn: 304409
* Do not legalize large setcc with setcce, introduce setcccarry and do it with ↵Amaury Sechet2017-06-014-24/+63
| | | | | | | | | | | | | | | | | usubo/setcccarry. Summary: This is a continuation of the work started in D29872 . Passing the carry down as a value rather than as a glue allows for further optimizations. Introducing setcccarry makes the use of addc/subc unecessary and we can start the removal process. This patch only introduce the optimization strictly required to get the same level of optimization as was available before nothing more. Reviewers: jyknight, nemanjai, mkuper, spatel, RKSimon, zvi, bkramer Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D33374 llvm-svn: 304404
* [DAGCombine] Refactor common addcarry pattern.Amaury Sechet2017-06-011-0/+35
| | | | | | | | | | | | Summary: This pattern is no very useful per se, but it exposes optimization for toehr patterns that wouldn't kick in otherwize. It's very common and worth optimizing for. Reviewers: jyknight, nemanjai, mkuper, spatel, RKSimon, zvi, bkramer Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D32756 llvm-svn: 304402
* [DAGCombine] (add/uaddo X, Carry) -> (addcarry X, 0, Carry)Amaury Sechet2017-06-011-0/+49
| | | | | | | | | | | | | | | Summary: This enables further transforms. Depends on D32916 Reviewers: jyknight, nemanjai, mkuper, spatel, RKSimon, zvi, bkramer Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D32925 llvm-svn: 304401
* [ScheduleDAG] Deal with already scheduled loads in ScheduleDAG.Nirav Dave2017-05-311-128/+150
| | | | | | | | | | | | | | | | | Summary: If we attempt to unfold an SUnit in ScheduleDAG that results in finding an already scheduled load, we must should abort the unfold as it will not improve scheduling. This fixes PR32610. Reviewers: jmolloy, sunfish, bogner, spatel Subscribers: llvm-commits, MatzeB Differential Revision: https://reviews.llvm.org/D32911 llvm-svn: 304321
OpenPOWER on IntegriCloud