summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
...
* Re-apply fixed "[Attributor] Fix: Make sure we set the changed flag"Johannes Doerfert2019-08-191-4/+4
| | | | | | | | | This reverts commit cedd0d9a6e4b433e1cd6585d1d4d152eb5e60b11. Re-apply the original commit but make sure the variables are initialized (even if they are not used) so UBSan is not complaining. llvm-svn: 369294
* [MemorySSA] Rename uses when inserting memory uses.Alina Sbirlea2019-08-191-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: When inserting uses from outside the MemorySSA creation, we don't normally need to rename uses, based on the assumption that there will be no inserted Phis (if Def existed that required a Phi, that Phi already exists). However, when dealing with unreachable blocks, MemorySSA will optimize away Phis whose incoming blocks are unreachable, and these Phis end up being re-added when inserting a Use. There are two potential solutions here: 1. Analyze the inserted Phis and clean them up if they are unneeded (current method for cleaning up trivial phis does not cover this) 2. Leave the Phi in place and rename uses, the same way as whe inserting defs. This patch use approach 2. Resolves first test in PR42940. Reviewers: george.burgess.iv Subscribers: Prazek, sanjoy.google, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D66033 llvm-svn: 369291
* [SLP] reduce duplicated code; NFCSanjay Patel2019-08-191-2/+4
| | | | llvm-svn: 369250
* Revert [Attributor] Fix: Make sure we set the changed flagDavid L. Jones2019-08-191-4/+4
| | | | | | | | This reverts r369159 (git commit cbaf1fdea2de891bdbc49cdec89ae2077e6b9ed0) r369160 caused a test to fail under UBSAN. See thread on llvm-commits. llvm-svn: 369241
* Revert [Attributor] Fix: Do not partially resolve returned calls.David L. Jones2019-08-191-28/+11
| | | | | | | | This reverts r369160 (git commit f72d9b1c97b41fff48ad1eecbba59a29c171bff4) r369160 caused some tests to fail under UBSAN. See thread on llvm-commits. llvm-svn: 369236
* [InstCombine] Cherry-pick NFC cleanups of ↵Roman Lebedev2019-08-181-5/+8
| | | | | | foldShiftIntoShiftInAnotherHandOfAndInICmp() from D66383 llvm-svn: 369207
* [MemorySSA] Loop passes should mark MSSA preserved when available.Alina Sbirlea2019-08-175-6/+6
| | | | | | | | This patch applies only to the new pass manager. Currently, when MSSA Analysis is available, and pass to each loop pass, it will be preserved by that loop pass. Hence, mark the analysis preserved based on that condition, vs the current `EnableMSSALoopDependency`. This leaves the global flag to affect only the entry point in the loop pass manager (in FunctionToLoopPassAdaptor). llvm-svn: 369181
* Revert r367891 - "[InstCombine] combine mul+shl separated by zext"Sanjay Patel2019-08-161-13/+2
| | | | | | | | | | | | | This reverts commit 5dbb90bfe14ace30224239cac7c61a1422fa5144. As noted in the post-commit thread for r367891, this can create a multiply that is lowered to a libcall that may not exist. We need to improve the backend decomposition for integer multiply before trying to re-land this (if it's still worthwhile after doing the backend work). llvm-svn: 369174
* Reland "[ARM] push LR before __gnu_mcount_nc"Jian Cai2019-08-161-1/+1
| | | | | | | | This relands r369147 with fixes to unit tests. https://reviews.llvm.org/D65019 llvm-svn: 369173
* [Attributor] Fix: Do not partially resolve returned calls.Johannes Doerfert2019-08-161-11/+28
| | | | | | | | | | By partially resolving returned calls we did not record that they were not fully resolved which caused odd behavior down the line. We could also end up with some, but not all, returned values of the callee in the returned values map of the caller, another odd behavior we want to avoid. llvm-svn: 369160
* [Attributor] Fix: Make sure we set the changed flagJohannes Doerfert2019-08-161-4/+4
| | | | | | | The flag was updated *before* we actually run the visitor callback so we might miss updates. llvm-svn: 369159
* [Attributor] Add all missing attribute definitions/symbolsJohannes Doerfert2019-08-161-35/+117
| | | | | | | | | | | | | | | | As a preparation to "on-demand" abstract attribute generation we need implementations for all attributes (as they can be queried and then created on-demand where we now fail to find one). Reviewers: uenoku, sstefan1 Subscribers: hiraditya, bollu, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D66129 llvm-svn: 369155
* Revert "[ARM] push LR before __gnu_mcount_nc"Jian Cai2019-08-161-1/+1
| | | | | | This reverts commit f4cf3b959333f62b7a7b2d7771f7010c9d8da388. llvm-svn: 369149
* [ARM] push LR before __gnu_mcount_ncJian Cai2019-08-161-1/+1
| | | | | | | | | Push LR register before calling __gnu_mcount_nc as it expects the value of LR register to be the top value of the stack on ARM32. Differential Revision: https://reviews.llvm.org/D65019 llvm-svn: 369147
* [Attributor] Towards a more structured deduction patternJohannes Doerfert2019-08-161-111/+222
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This is the first commit aiming to structure the attribute deduction. The base idea is that we have default propagation patterns as listed below on top of which we can add specific, e.g., context sensitive, logic. Deduction patterns used in this patch: - argument states are determined from call site argument states, see AAAlignArgument and AAArgumentFromCallSiteArguments. - call site argument states are determined as if they were floating values, see AAAlignCallSiteArgument and AAAlignFloating. - floating value states are determined by traversing the def-use chain and combining the states determined for the leaves, see AAAlignFloating and genericValueTraversal. - call site return states are determined from function return states, see AAAlignCallSiteReturned and AACallSiteReturnedFromReturned. - function return states are determined from returned value states, see AAAlignReturned and AAReturnedFromReturnedValues. Through this strategy all logic for alignment is concentrated in the AAAlignFloating::updateImpl method. Note: This commit works on its own but is part of a larger change that involves "on-demand" creation of abstract attributes that will participate in the fixpoint iteration. Without this part, we sometimes do not have an AAAlign abstract attribute to query, loosing information we determined before. All tests have appropriate FIXMEs and the information will be recovered once we added all parts. Reviewers: sstefan1, uenoku Subscribers: hiraditya, bollu, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D66126 llvm-svn: 369144
* [Attributor][NFC] Introduce aliases for call site attributesJohannes Doerfert2019-08-161-7/+40
| | | | | | | | | Until we have call site specific liveness and/or value information there is no need to do call site specific deduction. Though, we need the symbols in follow up patches that make Attributor::getAAFor return a reference. llvm-svn: 369143
* [Attributor] Introduce initialize calls and move code to keep attributes conciseJohannes Doerfert2019-08-161-179/+180
| | | | | | | | | | | | | | | | | | Summary: This patch should not change the behavior except that the added initialize methods might indicate an optimistic fixpoint earlier. The code movement is done to keep the attribute definitions in a single block where it makes sense. No functional changes intended there. Reviewers: uenoku, sstefan1 Subscribers: hiraditya, bollu, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D66258 llvm-svn: 369142
* [InstCombine] canonicalize a scalar-select-of-vectors to vector selectSanjay Patel2019-08-161-0/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This pattern may arise more frequently with an enhancement to SLP vectorization suggested in PR42755: https://bugs.llvm.org/show_bug.cgi?id=42755 ...but we should handle this pattern to make things easier for the backend either way. For all in-tree targets that I looked at, codegen for typical vector sizes looks better when we change to a vector select, so this is safe to do without a cost model (in other words, as a target-independent canonicalization). For example, if the condition of the select is a scalar, we end up with something like this on x86: vpcmpgtd %xmm0, %xmm1, %xmm0 vpextrb $12, %xmm0, %eax testb $1, %al jne LBB0_2 ## %bb.1: vmovaps %xmm3, %xmm2 LBB0_2: vmovaps %xmm2, %xmm0 Rather than the splat-condition variant: vpcmpgtd %xmm0, %xmm1, %xmm0 vpshufd $255, %xmm0, %xmm0 ## xmm0 = xmm0[3,3,3,3] vblendvps %xmm0, %xmm2, %xmm3, %xmm0 Differential Revision: https://reviews.llvm.org/D66095 llvm-svn: 369140
* [SLPVectorizer] Make the scheduler aware of the TreeEntry operands.Vasileios Porpodas2019-08-161-79/+171
| | | | | | | | | | | | | | | | | | | | | | Summary: The scheduler's dependence graph gets the use-def dependencies by accessing the operands of the instructions in a bundle. However, buildTree_rec() may change the order of the operands in TreeEntry, and the scheduler is currently not aware of this. This is not causing any functional issues currently, because reordering is restricted to the operands of a single instruction. Once we support operand reordering across multiple TreeEntries, as shown here: http://www.llvm.org/devmtg/2019-04/slides/Poster-Porpodas-Supernode_SLP.pdf , the scheduler will need to get the correct operands from TreeEntry and not from the individual instructions. In short, this patch: - Connects the scheduler's bundle with the corresponding TreeEntry. It introduces new TE and Lane fields in ScheduleData. - Moves the location where the operands of the TreeEntry are initialized. This used to take place in newTreeEntry() setting one operand at a time, but is now moved pre-order just before the recursion of buildTree_rec(). This is required because the scheduler needs to access both operands of the TreeEntry in tryScheduleBundle(). - Updates the scheduler to access the instruction operands through the TreeEntry operands instead of accessing the instruction operands directly. Reviewers: ABataev, RKSimon, dtemirbulatov, Ayal, dorit, hfinkel Reviewed By: ABataev Subscribers: hiraditya, llvm-commits, lebedev.ri, rcorcs Tags: #llvm Differential Revision: https://reviews.llvm.org/D62432 llvm-svn: 369131
* [InstCombine] Simplify pow(2.0, itofp(y)) to ldexp(1.0, y)Evandro Menezes2019-08-161-22/+32
| | | | | | | | Simplify `pow(2.0, itofp(y))` to `ldexp(1.0, y)`. Differential revision: https://reviews.llvm.org/D65979 llvm-svn: 369120
* [InstCombine] Shift amount reassociation in bittest: trunc-of-shl (PR42399)Roman Lebedev2019-08-161-19/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This is continuation of D63829 / https://bugs.llvm.org/show_bug.cgi?id=42399 I thought naive pattern would solve my issue, but nope, it involved truncation, thus more folds needed.. This isn't really the fold i'm interested in, i need trunc-of-lshr, but i'we decided to start with `shl` because it's simpler. In this case, no extra legality checks are needed: https://rise4fun.com/Alive/CAb We should be careful about not increasing instruction count, since we need to produce `zext` because `and` is done in wider type. Reviewers: spatel, nikic, xbolva00 Reviewed By: spatel Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D66057 llvm-svn: 369117
* [SLPVectorizer] Silence null dereference warning. NFCI.Simon Pilgrim2019-08-161-0/+1
| | | | | | cppcheck + MSVC analyzer both over zealously warn that we might dereference a null Bundle pointer - add an assertion to check for null to silence the warning, plus its a good idea to check that we succeeded in finding a schedule bundle anyway.... llvm-svn: 369094
* Move isPointerOffset function to ValueTracking (NFC).Evgeniy Stepanov2019-08-151-86/+2
| | | | | | | | | | | | | | Summary: To be reused in MemTag sanitizer. Reviewers: pcc, vitalybuka, ostannard Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D66165 llvm-svn: 369062
* [llvm] Migrate llvm::make_unique to std::make_uniqueJonas Devlieghere2019-08-1531-52/+52
| | | | | | | | Now that we've moved to C++14, we no longer need the llvm::make_unique implementation from STLExtras.h. This patch is a mechanical replacement of (hopefully) all the llvm::make_unique instances across the monorepo. llvm-svn: 369013
* [LV] fold-tail predication should be respected even with assume_safetyDorit Nuzman2019-08-152-5/+5
| | | | | | | | | | | | | | | assume_safety implies that loads under "if's" can be safely executed speculatively (unguarded, unmasked). However this assumption holds only for the original user "if's", not those introduced by the compiler, such as the fold-tail "if" that guards us from loading beyond the original loop trip-count. Currently the combination of fold-tail and assume-safety pragmas results in ignoring the fold-tail predicate that guards the loads, generating unmasked loads. This patch fixes this behavior. Differential Revision: https://reviews.llvm.org/D66106 Reviewers: Ayal, hsaito, fhahn llvm-svn: 368973
* [coroutine] Fixes "cannot move instruction since its users are not dominated ↵Gor Nishanov2019-08-152-148/+105
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | by CoroBegin" problem. Summary: Fixes https://bugs.llvm.org/show_bug.cgi?id=36578 and https://bugs.llvm.org/show_bug.cgi?id=36296. Supersedes: https://reviews.llvm.org/D55966 One of the fundamental transformation that CoroSplit pass performs before splitting the coroutine is to find which values need to survive between suspend and resume and provide a slot for them in the coroutine frame to spill and restore the value as needed. Coroutine frame becomes available once the storage for it was allocated and that point is marked in the pre-split coroutine with a llvm.coro.begin intrinsic. FE normally puts all of the user-authored code that would be accessing those values after llvm.coro.begin, however, sometimes instructions accessing those values would end up prior to coro.begin. For example, writing out a value of the parameter into the alloca done by the FE or instructions that are added by the optimization passes such as SROA when it rewrites allocas. Prior to this change, CoroSplit pass would try to move instructions that may end up accessing the values in the coroutine frame after CoroBegin. However it would run into problems (report_fatal_error) if some of the values would be used both in the allocation function (for example allocator is passed as a parameter to a coroutine) and in the use-authored body of the coroutine. To handle this case and to simplify the instruction moving logic, this change removes all of the instruction moving. Instead, we only change the uses of the spilled values that are dominated by coro.begin and leave other instructions intact. Before: ``` %var = alloca i32 %1 = getelementptr .. %var; ; will move this one after coro.begin %f = call i8* @llvm.coro.begin( ``` After: ``` %var = alloca i32 %1 = getelementptr .. %var; stays put %f = call i8* @llvm.coro.begin( ``` If we discover that there is a potential write into an alloca, prior to coro.begin we would copy its value from the alloca into the spill slot in the coroutine frame. Before: ``` %var = alloca i32 store .. %var ; will move this one after coro.begin %f = call i8* @llvm.coro.begin( ``` After: ``` %var = alloca i32 store .. %var ;stays put %f = call i8* @llvm.coro.begin( %tmp = load %var store %tmp, %spill.slot.for.var ``` Note: This change does not handle array allocas as that is something that C++ FE does not produce, but, it can be added in the future if need arises Reviewers: llvm-commits, modocache, ben-clayton, tks2103, rjmccall Reviewed By: modocache Subscribers: bartdesmet Differential Revision: https://reviews.llvm.org/D66230 llvm-svn: 368949
* [Attributor] Try to fix "missing field 'RetInsts' initializer" warningJohannes Doerfert2019-08-141-1/+1
| | | | | | http://lab.llvm.org:8011/builders/lld-x86_64-darwin13/builds/35674/steps/build_Lld/logs/stdio llvm-svn: 368938
* [Attributor][NFC] Make debug output consistentJohannes Doerfert2019-08-141-4/+4
| | | | llvm-svn: 368931
* [SCEV] Rename getMaxBackedgeTakenCount to getConstantMaxBackedgeTakenCount [NFC]Philip Reames2019-08-143-5/+5
| | | | llvm-svn: 368930
* [Attributor][NFC] Try to eliminate warnings (debug build + fall through)Johannes Doerfert2019-08-141-1/+3
| | | | llvm-svn: 368928
* [Attributor][NFC] Introduce statistics macros for new positionsJohannes Doerfert2019-08-141-54/+42
| | | | llvm-svn: 368927
* [Attributor][NFC] Add merge/join/clamp operators to the IntegerStateJohannes Doerfert2019-08-141-0/+24
| | | | | | Differential Revision: https://reviews.llvm.org/D66146 llvm-svn: 368925
* [Attributor] Use the AANoNull attribute directly in AADereferenceableJohannes Doerfert2019-08-141-78/+34
| | | | | | | | | | | | | | | | | Summary: Instead of constantly keeping track of the nonnull status with the dereferenceable information we can simply query the nonnull attribute whenever we need the information (debug + manifest). Reviewers: sstefan1, uenoku Subscribers: hiraditya, bollu, jfb, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D66113 llvm-svn: 368924
* [Attributor] Use liveness during the creation of AAReturnedValuesJohannes Doerfert2019-08-141-227/+182
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: As one of the first attributes, and one of the complex ones, AAReturnedValues was not using liveness but we filtered the result after the fact. This change adds liveness usage during the creation. The algorithm is also improved and shorter. The new algorithm will collect returned values over time using the generic facilities that work with liveness already, e.g., genericValueTraversal which does not look at dead PHI node predecessors. A test to show how this leads to better results is included. Note: Unresolved calls and resolved calls are now tracked explicitly. Reviewers: uenoku, sstefan1 Subscribers: hiraditya, bollu, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D66120 llvm-svn: 368922
* [Attributor] Do not update or manifest dead attributesJohannes Doerfert2019-08-141-3/+23
| | | | | | | | | | | | | | | | Summary: If the associated context instruction is assumed dead we do not need to update or manifest the state. Reviewers: sstefan1, uenoku Subscribers: hiraditya, bollu, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D66116 llvm-svn: 368921
* [Attributor] Use IRPosition consistentlyJohannes Doerfert2019-08-141-234/+367
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The next attempt to clean up the Attributor interface before we grow it further. Before, we used a combination of two values (associated + anchor) and an argument number (or -1) to determine a location. This was very fragile. The new system uses exclusively IR positions and we restrict the generation of IR positions to special constructor methods that verify internal constraints we have. This will catch misuse early. The auto-conversion, e.g., in getAAFor, is now performed through the SubsumingPositionIterator. This iterator takes an IR position and allows to visit all IR positions that "subsume" the given one, e.g., function attributes "subsume" argument attributes of that function. For a detailed breakdown see the class comment of SubsumingPositionIterator. This patch also introduces the IRPosition::getAttrs() to extract IR attributes at a certain position. The method knows how to look up in different positions that are equivalent, e.g., the argument position for call site arguments. We also introduce three new positions kinds such that we have all IR positions where attributes can be placed and one for "floating" values. Reviewers: sstefan1, uenoku Subscribers: hiraditya, bollu, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D65977 llvm-svn: 368919
* [SLP][NFC] Use pointers to address to ScalarToTreeEntry elements, instead of ↵Dinar Temirbulatov2019-08-141-4/+4
| | | | | | indexes. llvm-svn: 368906
* [RLEV] Rewrite loop exit values for multiple exit loops w/o overall loop ↵Philip Reames2019-08-141-4/+20
| | | | | | | | | | | | exit count We already supported rewriting loop exit values for multiple exit loops, but if any of the loop exits were not computable, we gave up on all loop exit values. This patch generalizes the existing code to handle individual computable loop exits where possible. As discussed in the review, this is a starting point for figuring out a better API. The code is a bit ugly, but getting it in lets us test as we go. Differential Revision: https://reviews.llvm.org/D65544 llvm-svn: 368898
* InferAddressSpaces: Move target intrinsic handling to TTIMatt Arsenault2019-08-141-23/+15
| | | | | | | | I'm planning on handling intrinsics that will benefit from checking the address space enums. Don't bother moving the address collection for now, since those won't need th enums. llvm-svn: 368895
* InferAddressSpaces: Remove unnecessary check for ConstantIntMatt Arsenault2019-08-141-2/+2
| | | | | | The IR is invalid if this isn't a constant since immarg was added. llvm-svn: 368893
* [SLC] Dereferenceable annonation - handle valid null pointersDavid Bolvansky2019-08-141-4/+11
| | | | | | | | | | | | | | Reviewers: jdoerfert, reames Reviewed By: jdoerfert Subscribers: llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D66161 llvm-svn: 368884
* [BuildLibCalls] Noalias annotationDavid Bolvansky2019-08-142-9/+16
| | | | | | | | | | | | | | | | Summary: I think this is better solution than annotating callsites in IC/SLC. Reviewers: jdoerfert Reviewed By: jdoerfert Subscribers: MaskRay, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D66217 llvm-svn: 368875
* Ignore indirect branches from callbr.Bill Wendling2019-08-141-2/+4
| | | | | | | | | | | | | | | | | | Summary: We can't speculate around indirect branches: indirectbr and invoke. The callbr instruction needs to be included here. Reviewers: nickdesaulniers, manojgupta, chandlerc Reviewed By: chandlerc Subscribers: llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D66200 llvm-svn: 368873
* Fix "not all control paths return a value" MSVC warnings. NFCI.Simon Pilgrim2019-08-141-1/+4
| | | | llvm-svn: 368831
* Fix "not all control paths return a value" MSVC warning. NFCI.Simon Pilgrim2019-08-141-0/+1
| | | | llvm-svn: 368830
* Fix "not all control paths return a value" MSVC warnings. NFCI.Simon Pilgrim2019-08-141-0/+2
| | | | llvm-svn: 368829
* [InstCombine] Refactor getFlippedStrictnessPredicateAndConstant() out of ↵Roman Lebedev2019-08-142-32/+50
| | | | | | | | | canonicalizeCmpWithConstant(), NFCI I'd like to use it elsewhere, hopefully without reinventing the wheel. No functional change intended so far. llvm-svn: 368820
* [LV] Fold-tail flagDorit Nuzman2019-08-141-5/+13
| | | | | | | | | | | This is the compiler-flag equivalent of the Predicate pragma (https://reviews.llvm.org/D65197), to direct the vectorizer to fold the remainder-loop into the main-loop using predication. Differential Revision: https://reviews.llvm.org/D66108 Reviewers: Ayal, hsaito, fhahn, SjoerdMeije llvm-svn: 368801
* Revert '[LICM] Make Loop ICM profile aware' and 'Fix pass dependency for LICM'David L. Jones2019-08-141-75/+18
| | | | | | | This reverts r368526 (git commit 7e71aa24bc0788690fea7f0d7eab400c6a784deb) This reverts r368542 (git commit cb5a90fd314a7914cf293797bb4fd7a6841052cf) llvm-svn: 368800
* Coroutines: adjust for SVN r358739John McCall2019-08-141-4/+6
| | | | | | | CallSite has been removed in favour of CallBase. Adjust the coroutine split to account for that. llvm-svn: 368798
OpenPOWER on IntegriCloud