summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
* [GVN] When merging blocks update LoopInfo if it's availableAdam Nemet2016-12-011-6/+10
| | | | | | | | | | | | | If LoopInfo is available during GVN, BasicAA will use it. However MergeBlockIntoPredecessor does not update LI as it merges blocks. This didn't use to cause problems because LI was freed before GVN/BasicAA. Now with OptimizationRemarkEmitter, the lifetime of LI is extended so LI needs to be kept up-to-date during GVN. Differential Revision: https://reviews.llvm.org/D27288 llvm-svn: 288307
* Fix LSR best register search algorithm.Evgeny Stupachenko2016-11-301-2/+3
| | | | | | | | | | | | | Summary: Fix a case when first register in a search has maximum RegUses.getUsedByIndices(Reg).count() Reviewers: qcolombet Differential Revision: http://reviews.llvm.org/D26877 From: Evgeny Stupachenko <evstupac@gmail.com> llvm-svn: 288278
* [LoopUnroll] Implement profile-based loop peelingMichael Kuperstein2016-11-305-31/+474
| | | | | | | | | | | | | | | | | | | This implements PGO-driven loop peeling. The basic idea is that when the average dynamic trip-count of a loop is known, based on PGO, to be low, we can expect a performance win by peeling off the first several iterations of that loop. Unlike unrolling based on a known trip count, or a trip count multiple, this doesn't save us the conditional check and branch on each iteration. However, it does allow us to simplify the straight-line code we get (constant-folding, etc.). This is important given that we know that we will usually only hit this code, and not the actual loop. This is currently disabled by default. Differential Revision: https://reviews.llvm.org/D25963 llvm-svn: 288274
* [InstCombine] allow more narrowing transforms for logic opsSanjay Patel2016-11-302-9/+24
| | | | | | | We had a limited version of this for scalar 'and'; this expands the transform to 'or' and 'xor' and allows vectors types too. llvm-svn: 288273
* Fix some Clang-tidy and Include What You Use warnings; other minor fixes (NFC).Eugene Zelenko2016-11-303-41/+133
| | | | | | This preparation to remove SetVector.h dependency on SmallSet.h. llvm-svn: 288256
* Revert "[GVN] Basic optimization remark support"Adam Nemet2016-11-301-20/+3
| | | | | | | | This reverts commit r288210. The failure on the stage2 LTO build is back. llvm-svn: 288226
* [GVN] Basic optimization remark supportAdam Nemet2016-11-291-3/+20
| | | | | | | | | | | | | | | | [recommiting patches one-by-one to see which breaks the stage2 LTO bot] Follow-on patches will add more interesting cases. The goal of this patch-set is to get the GVN messages printed in opt-viewer from Dhrystone as was presented in my Dev Meeting talk. This is the optimization view for the function (the last remark in the function has a bug which is fixed in this series): http://lab.llvm.org:8080/artifacts/opt-view_test-suite/build/SingleSource/Benchmarks/Dhrystone/CMakeFiles/dry.dir/html/_org_test-suite_SingleSource_Benchmarks_Dhrystone_dry.c.html#L430 Differential Revision: https://reviews.llvm.org/D26488 llvm-svn: 288210
* [StructurizeCFG] Fix infinite loop in rebuildSSA.Justin Lebar2016-11-291-1/+4
| | | | | | | | | Michel Dänzer reported that r288051, "[StructurizeCFG] Use range-based for loops", introduced a bug into rebuildSSA, wherein we were iterating over an instruction's use list while modifying it, without taking care to do this correctly. llvm-svn: 288200
* Use CallSite to simplify codeDavid Blaikie2016-11-291-5/+3
| | | | llvm-svn: 288192
* Revert "[GVN] Basic optimization remark support"Adam Nemet2016-11-291-20/+3
| | | | | | | | | This reverts commit r288046. Trying to see if the revert fixes a compiler crash during a stage2 LTO build with a GVN backtrace. llvm-svn: 288179
* Revert "[GVN, OptDiag] Include the value that is forwarded in load elimination"Adam Nemet2016-11-291-8/+5
| | | | | | | | | This reverts commit r288047. Trying to see if the revert fixes a compiler crash during a stage2 LTO build with a GVN backtrace. llvm-svn: 288178
* Revert "[GVN, OptDiag] Print the interesting instructions involved in missed ↵Adam Nemet2016-11-291-36/+0
| | | | | | | | | | | load-elimination" This reverts commit r288090. Trying to see if the revert fixes a compiler crash during a stage2 LTO build with a GVN backtrace. llvm-svn: 288177
* [CVP] Remove cvp-dont-process-adds flagArtur Pilipenko2016-11-291-5/+0
| | | | | | The flag was introduced because the optimization controlled by the flag initially caused regressions. All the regressions were fixed some time ago and the flag has been false for quite a while. llvm-svn: 288154
* [GVNHoist] Rename variables.Aditya Kumar2016-11-291-14/+16
| | | | | | Differential Revision: https://reviews.llvm.org/D27110 llvm-svn: 288142
* [GVNHoist] Enable aggressive hoisting when optimizing for code-sizeAditya Kumar2016-11-291-5/+13
| | | | | | | | | Enable scalar hoisting at -Oz as it is safe to hoist scalars to a place where they are partially needed. Differential Revision: https://reviews.llvm.org/D27111 llvm-svn: 288141
* [SLPVectorizer] Improved support of partial tree vectorization.Alexey Bataev2016-11-291-87/+178
| | | | | | | | | | | Currently SLP vectorizer tries to vectorize a binary operation and dies immediately after unsuccessful the first unsuccessfull attempt. Patch tries to improve the situation, trying to vectorize all binary operations of all children nodes in the binop tree. Differential Revision: https://reviews.llvm.org/D25517 llvm-svn: 288115
* [asan/win] Align global registration metadata to its sizeReid Kleckner2016-11-291-2/+13
| | | | | | | | This way, when the linker adds padding between globals, we can skip over the zero padding bytes and reliably find the start of the next metadata global. llvm-svn: 288096
* [GVN, OptDiag] Print the interesting instructions involved in missed ↵Adam Nemet2016-11-291-0/+36
| | | | | | | | | | | | | | | | | | | load-elimination This includes the intervening store and the load/store that we're trying to forward from in the optimization remark for the missed load elimination. This is hooked up under a new mode in ORE that allows for compile-time budget for a bit more analysis to print more insightful messages. This mode is currently enabled for -fsave-optimization-record (-Rpass is trickier since it is controlled in the front-end). With this we can now print the red remark in http://lab.llvm.org:8080/artifacts/opt-view_test-suite/build/SingleSource/Benchmarks/Dhrystone/CMakeFiles/dry.dir/html/_org_test-suite_SingleSource_Benchmarks_Dhrystone_dry.c.html#L446 Differential Revision: https://reviews.llvm.org/D26490 llvm-svn: 288090
* [SROA] Drop lifetime.start/end intrinsics when they block promotion.Eli Friedman2016-11-281-6/+12
| | | | | | | | | | | | | | Preserving lifetime markers isn't as important as allowing promotion, so just drop the lifetime markers if necessary. This also fixes an assertion failure where other parts of SROA assumed that lifetime markers never block promotion. Fixes https://llvm.org/bugs/show_bug.cgi?id=29139. Differential Revision: https://reviews.llvm.org/D24854 llvm-svn: 288074
* [StructurizeCFG] Use range-based for loops.Justin Lebar2016-11-281-72/+51
| | | | | | | | | | Reviewers: arsenm Subscribers: wdng, llvm-commits Differential Revision: https://reviews.llvm.org/D27000 llvm-svn: 288051
* [StructurizeCFG] Refactor NearestCommonDominator.Justin Lebar2016-11-281-56/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: As far as I can tell, doing our own computations in NearestCommonDominator is a false optimization -- DomTree will build up what appears to be exactly this data when it decides it's worthwhile. Moreover, by building the cache ourselves, we cannot take advantage of the cache that the domtree might have available. In addition, I am not convinced of the correctness of the original code. In particular, setting ResultIndex = 1 on the first addBlock instead of setting it to 0 is quite fishy. Similarly, it's not clear to me that setting IndexMap[Node] = 0 for every node as we walk up the tree finding a common parent is correct. But rather than ponder over these questions, I'd rather just make the code do the obviously-correct thing. This patch also changes the NearestCommonDominator API a bit, improving the names and getting rid of the boolean parameter in addBlock -- see http://jlebar.com/2011/12/16/Boolean_parameters_to_API_functions_considered_harmful..html Reviewers: arsenm Subscribers: aemerson, wdng, llvm-commits Differential Revision: https://reviews.llvm.org/D26998 llvm-svn: 288050
* [GVN, OptDiag] Include the value that is forwarded in load eliminationAdam Nemet2016-11-281-5/+8
| | | | | | | | | | | | | | | | | | | | | This requires some changes to the opt-diag API. Hal and I have discussed this at the Dev Meeting and came up with a streaming delimiter (setExtraArgs) to solve this. Arguments after this delimiter are only included in the optimization records and not in the remarks printed in the compiler output. (Note, how in the test the content of the YAML file changes but the remarks on the compiler output don't.) This implements the green GVN message with a bug fix at line http://lab.llvm.org:8080/artifacts/opt-view_test-suite/build/SingleSource/Benchmarks/Dhrystone/CMakeFiles/dry.dir/html/_org_test-suite_SingleSource_Benchmarks_Dhrystone_dry.c.html#L446 The fix is that now we properly include the constant value in the message: "load of type i32 eliminated in favor of 7" Differential Revision: https://reviews.llvm.org/D26489 llvm-svn: 288047
* [GVN] Basic optimization remark supportAdam Nemet2016-11-281-3/+20
| | | | | | | | | | | | | | Follow-on patches will add more interesting cases. The goal of this patch-set is to get the GVN messages printed in opt-viewer from Dhrystone as was presented in my Dev Meeting talk. This is the optimization view for the function (the last remark in the function has a bug which is fixed in this series): http://lab.llvm.org:8080/artifacts/opt-view_test-suite/build/SingleSource/Benchmarks/Dhrystone/CMakeFiles/dry.dir/html/_org_test-suite_SingleSource_Benchmarks_Dhrystone_dry.c.html#L430 Differential Revision: https://reviews.llvm.org/D26488 llvm-svn: 288046
* [InstSimplify] allow integer vector types to use computeKnownBitsSanjay Patel2016-11-271-5/+5
| | | | | | | | Note that the non-splat lshr+lshr test folded, but that does not work in general. Something is missing or wrong in computeKnownBits as the non-splat shl+shl test still shows. llvm-svn: 288005
* fix formatting; NFCSanjay Patel2016-11-271-13/+15
| | | | llvm-svn: 287997
* [InstCombine] don't drop metadata in FoldOpIntoSelect()Sanjay Patel2016-11-261-3/+3
| | | | llvm-svn: 287980
* add optional param to copy metadata when creating selects; NFCSanjay Patel2016-11-261-7/+3
| | | | | | | | | | | There are other spots where we can use this; we're currently dropping metadata in some places, and there are proposed changes where we will want to propagate metadata. IRBuilder's CreateSelect() already has a parameter like this, so this change makes the regular 'Create' API line up with that. llvm-svn: 287976
* Replace some callers of setTailCall with setTailCallKindDavid Majnemer2016-11-254-13/+12
| | | | | | | We were a little sloppy with adding tailcall markers. Be more consistent by using setTailCallKind instead of setTailCall. llvm-svn: 287955
* [Loop Unswitch] Patch to selective unswitch only the reachable branch ↵Abhilash Bhandari2016-11-251-1/+36
| | | | | | | | | | | | | | | | instructions. Summary: The iterative algorithm for Loop Unswitching may render some of the branches unreachable in the unswitched loops. Given the exponential nature of the algorithm, this is quite an overhead. This patch fixes this problem by selectively unswitching only those branches within a loop that are reachable from the loop header. Reviewers: Michael Zolothukin, Anna Thomas, Weiming Zhao. Subscribers: llvm-commits. Differential Revision: http://reviews.llvm.org/D26299 llvm-svn: 287925
* [LoopUnroll] Move code to exit early. NFC.Haicheng Wu2016-11-231-10/+8
| | | | | | | | Just to save some compilation time. Differential Revision: https://reviews.llvm.org/D26784 llvm-svn: 287800
* [PM] Change the static object whose address is used to uniquely identifyChandler Carruth2016-11-231-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | analyses to have a common type which is enforced rather than using a char object and a `void *` type when used as an identifier. This has a number of advantages. First, it at least helps some of the confusion raised in Justin Lebar's code review of why `void *` was being used everywhere by having a stronger type that connects to documentation about this. However, perhaps more importantly, it addresses a serious issue where the alignment of these pointer-like identifiers was unknown. This made it hard to use them in pointer-like data structures. We were already dodging this in dangerous ways to create the "all analyses" entry. In a subsequent patch I attempted to use these with TinyPtrVector and things fell apart in a very bad way. And it isn't just a compile time or type system issue. Worse than that, the actual alignment of these pointer-like opaque identifiers wasn't guaranteed to be a useful alignment as they were just characters. This change introduces a type to use as the "key" object whose address forms the opaque identifier. This both forces the objects to have proper alignment, and provides type checking that we get it right everywhere. It also makes the types somewhat less mysterious than `void *`. We could go one step further and introduce a truly opaque pointer-like type to return from the `ID()` static function rather than returning `AnalysisKey *`, but that didn't seem to be a clear win so this is just the initial change to get to a reliably typed and aligned object serving is a key for all the analyses. Thanks to Richard Smith and Justin Lebar for helping pick plausible names and avoid making this refactoring many times. =] And thanks to Sean for the super fast review! While here, I've tried to move away from the "PassID" nomenclature entirely as it wasn't really helping and is overloaded with old pass manager constructs. Now we have IDs for analyses, and key objects whose address can be used as IDs. Where possible and clear I've shortened this to just "ID". In a few places I kept "AnalysisID" to make it clear what was being identified. Differential Revision: https://reviews.llvm.org/D27031 llvm-svn: 287783
* [LoadStoreVectorizer] Enable vectorization of stores in the presence of an ↵Alina Sbirlea2016-11-231-3/+25
| | | | | | | | | | | | | | | | | | aliasing load Summary: The "getVectorizablePrefix" method would give up if it found an aliasing load for a store chain. In practice, the aliasing load can be treated as a memory barrier and all stores that precede it are a valid vectorizable prefix. Issue found by volkan in D26962. Testcase is a pruned version of the one in the original patch. Reviewers: jlebar, arsenm, tstellarAMD Subscribers: mzolotukhin, wdng, nhaehnle, anna, volkan, llvm-commits Differential Revision: https://reviews.llvm.org/D27008 llvm-svn: 287781
* [StructurizeCFG] Refactor OrderNodes.Justin Lebar2016-11-221-12/+6
| | | | | | | | | | | Summary: No need to copy the RPOT vector before using it. Switch from std::map to SmallDenseMap. Get rid of an unused variable (TempVisited). Get rid of a typedef, RNVector, which is now used only once. Differential Revision: https://reviews.llvm.org/D26997 llvm-svn: 287721
* [StructurizeCFG] Add whitespace in getAnalysisUsage.Justin Lebar2016-11-221-0/+1
| | | | | | | | | | | | | | | Summary: "addRequired" and "addPreserved" look very similar when squished up next to each other -- without the newline this code looked to me like it was addRequired'ing DominatorTreeWrapperPass twice. Reviewers: arsenm Subscribers: wdng, llvm-commits Differential Revision: https://reviews.llvm.org/D26996 llvm-svn: 287720
* [StructurizeCFG] Remove unnecessary "using" in class.Justin Lebar2016-11-221-1/+0
| | | | | | | | | | Reviewers: arsenm Subscribers: wdng, llvm-commits Differential Revision: https://reviews.llvm.org/D26995 llvm-svn: 287719
* [StructurizeCFG] Merge the two constructors into one.Justin Lebar2016-11-221-7/+2
| | | | | | | | | | Reviewers: arsenm Subscribers: wdng, llvm-commits Differential Revision: https://reviews.llvm.org/D26994 llvm-svn: 287718
* [StructurizeCFG] Use a for-each loop instead of iterators in runOnRegion.Justin Lebar2016-11-221-4/+3
| | | | | | | | | | | | Summary: Reviewers: arsenm Subscribers: wdng, llvm-commits Differential Revision: https://reviews.llvm.org/D26993 llvm-svn: 287717
* [StructurizeCFG] Make hasOnlyUniformBranches a non-member function.Justin Lebar2016-11-221-7/+5
| | | | | | | | | | | | Summary: Lets us get rid of one member variable too. Reviewers: arsenm Subscribers: wdng, llvm-commits Differential Revision: https://reviews.llvm.org/D26992 llvm-svn: 287716
* add and use isBitwiseLogicOp() helper function; NFCISanjay Patel2016-11-223-33/+14
| | | | llvm-svn: 287712
* Before sample pgo annotation, do not inline a function that has no debug ↵Dehao Chen2016-11-221-0/+2
| | | | | | | | info. (NFC) If there is no debug info in the callee, inlining it will not help annotator. This avoids infinite loop as reported in PR/31119. llvm-svn: 287710
* [SCCP] Remove code in visitBinaryOperator (and add tests).Davide Italiano2016-11-221-11/+3
| | | | | | | | | | We visit and/or, we try to derive a lattice value for the instruction even if one of the operands is overdefined. If the non-overdefined value is still 'unknown' just return and wait for ResolvedUndefsIn to "plug in" the correct value. This simplifies the logic a bit. While I'm here add tests for missing cases. llvm-svn: 287709
* [InstCombine] change bitwise logic type to eliminate bitcastsSanjay Patel2016-11-221-0/+43
| | | | | | | | | | | | | | | | | | | | In PR27925: https://llvm.org/bugs/show_bug.cgi?id=27925 ...we proposed adding this fold to eliminate a bitcast. In D20774, there was some concern about changing the type of a bitwise op as well as creating bitcasts that might not be free for a target. However, if we're strictly eliminating an instruction (by limiting this to one-use ops), then we should be able to do this in InstCombine. But we're cautiously restricting the transform for now to vector types to avoid possible backend problems. A transform to make sure the logic op is legal for the target should be added to reverse this transform and improve codegen. Differential Revision: https://reviews.llvm.org/D26641 llvm-svn: 287707
* Fixed the lost FastMathFlags in GVN(Global Value Numbering).Vyacheslav Klochkov2016-11-221-1/+6
| | | | | | | Reviewer: Hal Finkel. Differential Revision: https://reviews.llvm.org/D26952 llvm-svn: 287700
* Fixed the lost FastMathFlags in Reassociate optimization.Vyacheslav Klochkov2016-11-221-0/+6
| | | | | | | Reviewer: Hal Finkel. Differential Revision: https://reviews.llvm.org/D26957 llvm-svn: 287695
* [LoopReroll] Make root-finding more aggressive.Eli Friedman2016-11-211-50/+58
| | | | | | | | | | Allow using an instruction other than a mul or phi as the base for root-finding. For example, the included testcase includes a loop which requires using a getelementptr as the base for root-finding. Differential Revision: https://reviews.llvm.org/D26529 llvm-svn: 287588
* [InstCombine] canonicalize min/max constant to select's false valueSanjay Patel2016-11-211-0/+42
| | | | | | | | | | | | | | | | | | | | This is a first step towards canonicalization and improved folding/codegen for integer min/max as discussed here: http://lists.llvm.org/pipermail/llvm-dev/2016-November/106868.html Here, we're just matching the simplest min/max patterns and adjusting the icmp predicate while swapping the select operands. I've included FIXME tests in test/Transforms/InstCombine/select_meta.ll so it's easier to see how this might be extended (corresponds to the TODO comment in the code). That's also why I'm using matchSelectPattern() rather than a simpler check; once the backend is patched, we can just remove some of the restrictions to allow the obfuscated min/max patterns in the FIXME tests to be matched. Differential Revision: https://reviews.llvm.org/D26525 llvm-svn: 287585
* LSR debug fix.Evgeny Stupachenko2016-11-211-1/+1
| | | | | | | | | | | Summary: Dump instruction instead of address. Reviewers: hfinkel Differential Revision: http://reviews.llvm.org/D26877 From: Evgeny Stupachenko <evstupac@gmail.com> llvm-svn: 287584
* fix formatting; NFCSanjay Patel2016-11-211-1/+0
| | | | llvm-svn: 287582
* [asan] Make ASan compatible with linker dead stripping on WindowsReid Kleckner2016-11-211-47/+97
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This is similar to what was done for Darwin in rL264645 / http://reviews.llvm.org/D16737, but it uses COFF COMDATs to achive the same result instead of relying on new custom linker features. As on MachO, this creates one metadata global per instrumented global. The metadata global is placed in the custom .ASAN$GL section, which the ASan runtime will iterate over during initialization. There are no other references to the metadata, so normal linker dead stripping would discard it. However, the metadata is put in a COMDAT group with the instrumented global, so that it will be discarded if and only if the instrumented global is discarded. I didn't update the ASan ABI version check since this doesn't affect non-Windows platforms, and the WinASan ABI isn't really stable yet. Implementing this for ELF will require extending LLVM IR and MC a bit so that we can use non-COMDAT section groups. Reviewers: pcc, kcc, mehdi_amini, kubabrecka Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D26770 llvm-svn: 287576
* [MemorySSA] Fix for non-determinism in codegenMandeep Singh Grang2016-11-211-2/+11
| | | | | | | | | | | | | | | | | | | | This patch fixes the non-determinism caused due to iterating SmallPtrSet's which was uncovered due to the experimental "reverse iteration order " patch: https://reviews.llvm.org/D26718 The following unit tests failed because of the undefined order of iteration. LLVM :: Transforms/Util/MemorySSA/cyclicphi.ll LLVM :: Transforms/Util/MemorySSA/many-dom-backedge.ll LLVM :: Transforms/Util/MemorySSA/many-doms.ll LLVM :: Transforms/Util/MemorySSA/phi-translation.ll Reviewers: dberlin, mgrang Subscribers: dberlin, llvm-commits, david2050 Differential Revision: https://reviews.llvm.org/D26704 llvm-svn: 287563
OpenPOWER on IntegriCloud