summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
* [LoopUnroll] Ensure we create prolog loops in simplified form.Michael Zolotukhin2016-08-021-0/+12
| | | | llvm-svn: 277502
* MSVC 2013 does not implement C++11 unions properly, so remove the anoymous ↵Daniel Berlin2016-08-021-2/+2
| | | | | | | | union for now, and leave a FIXME. llvm-svn: 277485
* Rewrite the use optimizer to be less memory intensive and 50% faster.Daniel Berlin2016-08-021-31/+311
| | | | | | | | | | | | | | | | | | | | | | | | Fixes PR28670 Summary: Rewrite the use optimizer to be less memory intensive and 50% faster. Fixes PR28670 The new use optimizer works like a standard SSA renaming pass, storing all possible versions a MemorySSA use could get in a stack, and just tracking indexes into the stack. This uses much less memory than caching N^2 alias query results. It's also a lot faster. The current version defers phi node walking to the normal walker. Reviewers: george.burgess.iv Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D23032 llvm-svn: 277480
* [LV] Generate both scalar and vector integer induction variablesMatthew Simpson2016-08-021-34/+66
| | | | | | | | | | | | | | | | This patch enables the vectorizer to generate both scalar and vector versions of an integer induction variable for a given loop. Previously, we only generated a scalar induction variable if we knew all its users were going to be scalar. Otherwise, we generated a vector induction variable. In the case of a loop with both scalar and vector users of the induction variable, we would generate the vector induction variable and extract scalar values from it for the scalar users. With this patch, we now generate both versions of the induction variable when there are both scalar and vector users and select which version to use based on whether the user is scalar or vector. Differential Revision: https://reviews.llvm.org/D22869 llvm-svn: 277474
* [LV] Untangle the concepts of uniform and scalarMatthew Simpson2016-08-021-63/+107
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch refactors the logic in collectLoopUniforms and collectValuesToIgnore, untangling the concepts of "uniform" and "scalar". It adds isScalarAfterVectorization along side isUniformAfterVectorization to distinguish the two. Known scalar values include those that are uniform, getelementptr instructions that won't be vectorized, and induction variables and induction variable update instructions whose users are all known to be scalar. This patch includes the following functional changes: - In collectLoopUniforms, we mark uniform the pointer operands of interleaved accesses. Although non-consecutive, these pointers are treated like consecutive pointers during vectorization. - In collectValuesToIgnore, we insert a value into VecValuesToIgnore if it isScalarAfterVectorization rather than isUniformAfterVectorization. This differs from the previous functionaly in that we now add getelementptr instructions that will not be vectorized into VecValuesToIgnore. This patch also removes the ValuesNotWidened set used for induction variable scalarization since, after the above changes, it is now equivalent to isScalarAfterVectorization. Differential Revision: https://reviews.llvm.org/D22867 llvm-svn: 277460
* [LoadStoreVectorizer] Don't use a linear walk for an existence check in a ↵Benjamin Kramer2016-08-021-1/+1
| | | | | | | | SmallPtrSet No functionality change intended. llvm-svn: 277436
* Minor code cleanups. NFC.Junmo Park2016-08-021-2/+2
| | | | llvm-svn: 277415
* CodeExtractor : Add ability to preserve profile data.Sean Silva2016-08-022-19/+122
| | | | | | | | | | | Added ability to estimate the entry count of the extracted function and the branch probabilities of the exit branches. Patch by River Riddle! Differential Revision: https://reviews.llvm.org/D22744 llvm-svn: 277411
* [ADT] NFC: Generalize GraphTraits requirement of "NodeType *" in interfaces ↵Tim Shen2016-08-011-0/+1
| | | | | | | | | | | | | | to "NodeRef", and migrate SCCIterator.h to use NodeRef Summary: By generalize the interface, users are able to inject more flexible Node token into the algorithm, for example, a pair of vector<Node>* and index integer. Currently I only migrated SCCIterator to use NodeRef, but more is coming. It's a NFC. Reviewers: dblaikie, chandlerc Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D22937 llvm-svn: 277399
* [WebAssembly] Support CFI for WebAssembly targetDerek Schuff2016-08-011-25/+68
| | | | | | | | | | | | | | | Summary: This patch implements CFI for WebAssembly. It modifies the LowerTypeTest pass to pre-assign table indexes to functions that are called indirectly, and lowers type checks to test against the appropriate table indexes. It also modifies the WebAssembly backend to support a special ".indidx" assembly directive that propagates the table index assignments out to the linker. Patch by Dominic Chen Differential Revision: https://reviews.llvm.org/D21768 llvm-svn: 277398
* [PM] Port SpeculativeExecution to the new PMMichael Kuperstein2016-08-012-40/+58
| | | | | | Differential Revision: https://reviews.llvm.org/D23033 llvm-svn: 277393
* [Profile] IR profiling minor cleanup /nfcXinliang David Li2016-08-011-5/+15
| | | | | | Differential Revision: http://reviews.llvm.org/D22995 llvm-svn: 277379
* [LV] Move isGatherOrScatterLegal into LoopVectorizationLegality (NFC)Matthew Simpson2016-08-011-15/+13
| | | | llvm-svn: 277376
* [LV] Use getPointerOperand helper where appropriate (NFC)Matthew Simpson2016-08-011-27/+21
| | | | llvm-svn: 277375
* [SimplifyCFG] Fix nasty RAUW bug from r277325James Molloy2016-08-011-3/+4
| | | | | | | | | | | | | | | Using RAUW was wrong here; if we have a switch transform such as: 18 -> 6 then 6 -> 0 If we use RAUW, while performing the second transform the *transformed* 6 from the first will be also replaced, so we end up with: 18 -> 0 6 -> 0 Found by clang stage2 bootstrap; testcase added. llvm-svn: 277332
* [SimplifyCFG] Range reduce switchesJames Molloy2016-08-011-0/+106
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If a switch is sparse and all the cases (once sorted) are in arithmetic progression, we can extract the common factor out of the switch and create a dense switch. For example: switch (i) { case 5: ... case 9: ... case 13: ... case 17: ... } can become: if ( (i - 5) % 4 ) goto default; switch ((i - 5) / 4) { case 0: ... case 1: ... case 2: ... case 3: ... } or even better: switch ( ROTR(i - 5, 2) { case 0: ... case 1: ... case 2: ... case 3: ... } The division and remainder operations could be costly so we only do this if the factor is a power of two, and emit a right-rotate instead of a divide/remainder sequence. Dense switches can be lowered significantly better than sparse switches and can even be transformed into lookup tables. llvm-svn: 277325
* Revert r277313 and r277314.Sean Silva2016-08-012-143/+22
| | | | | | | | | | | | | | | They seem to trigger an LSan failure: http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/15140/steps/check-llvm%20asan/logs/stdio Revert "Add the tests for r277313" This reverts commit r277314. Revert "CodeExtractor : Add ability to preserve profile data." This reverts commit r277313. llvm-svn: 277317
* Fix - CodeExtractor : Inherit Target Dependent Attributes from the parent ↵Sean Silva2016-08-011-1/+16
| | | | | | | | | | | | | | | | | | | function. When extracting a set of blocks make sure to inherit all of the target dependent attributes to make sure that the function will be valid for lowering. One example is the "target-features" attribute for x86, if the extracted region has functionality that relies on a specific feature it will fail to be lowered. This also allows for extracted functions to be valid for inlining, at least back into the parent function, as the target attributes are tested when inlining for compatibility. Patch by River Riddle! Differential Revision: https://reviews.llvm.org/D22713 llvm-svn: 277315
* CodeExtractor : Add ability to preserve profile data.Sean Silva2016-08-012-22/+143
| | | | | | | | | | | Added ability to estimate the entry count of the extracted function and the branch probabilities of the exit branches. Patch by River Riddle! Differential Revision: https://reviews.llvm.org/D22744 llvm-svn: 277313
* Fix the MemorySSA updating API to enable people to create memory accesses ↵Daniel Berlin2016-07-311-4/+6
| | | | | | before removing old ones llvm-svn: 277309
* [LoopUnroll] Include hotness of region in opt remarkAdam Nemet2016-07-293-34/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | LoopUnroll is a loop pass, so the analysis of OptimizationRemarkEmitter is added to the common function analysis passes that loop passes depend on. The BFI and indirectly BPI used in this pass is computed lazily so no overhead should be observed unless -pass-remarks-with-hotness is used. This is how the patch affects the O3 pipeline: Dominator Tree Construction Natural Loop Information Canonicalize natural loops Loop-Closed SSA Form Pass Basic Alias Analysis (stateless AA impl) Function Alias Analysis Results Scalar Evolution Analysis + Lazy Branch Probability Analysis + Lazy Block Frequency Analysis + Optimization Remark Emitter Loop Pass Manager Rotate Loops Loop Invariant Code Motion Unswitch loops Simplify the CFG Dominator Tree Construction Basic Alias Analysis (stateless AA impl) Function Alias Analysis Results Combine redundant instructions Natural Loop Information Canonicalize natural loops Loop-Closed SSA Form Pass Scalar Evolution Analysis + Lazy Branch Probability Analysis + Lazy Block Frequency Analysis + Optimization Remark Emitter Loop Pass Manager Induction Variable Simplification Recognize loop idioms Delete dead loops Unroll loops ... llvm-svn: 277203
* Recommitting r275284: add support to inline __builtin_mempcpyAndrew Kaylor2016-07-291-0/+1
| | | | | | | | Patch by Sunita Marathe Third try, now following fixes to MSan to handle mempcy in such a way that this commit won't break the MSan buildbots. (Thanks, Evegenii!) llvm-svn: 277189
* [EarlyCSE] Correctly handle simplified, but live, instructionsDavid Majnemer2016-07-291-2/+4
| | | | | | | | | Some instructions may have their uses replaced with a symbolic constant. However, the instruction may still have side effects which percludes it from being removed from the function. EarlyCSE treated such an instruction as if it were removed, resulting in PR28763. llvm-svn: 277114
* [ConstnatFolding] Teach the folder how to fold ConstantVectorDavid Majnemer2016-07-296-29/+33
| | | | | | | | | | | A ConstantVector can have ConstantExpr operands and vice versa. However, the folder had no ability to fold ConstantVectors which, in some cases, was an optimization barrier. Instead, rephrase the folder in terms of Constants instead of ConstantExprs and teach callers how to deal with failure. llvm-svn: 277099
* Added ThinLTO inlining statisticsPiotr Padlewski2016-07-293-9/+242
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: copypasta doc of ImportedFunctionsInliningStatistics class \brief Calculate and dump ThinLTO specific inliner stats. The main statistics are: (1) Number of inlined imported functions, (2) Number of imported functions inlined into importing module (indirect), (3) Number of non imported functions inlined into importing module (indirect). The difference between first and the second is that first stat counts all performed inlines on imported functions, but the second one only the functions that have been eventually inlined to a function in the importing module (by a chain of inlines). Because llvm uses bottom-up inliner, it is possible to e.g. import function `A`, `B` and then inline `B` to `A`, and after this `A` might be too big to be inlined into some other function that calls it. It calculates this statistic by building graph, where the nodes are functions, and edges are performed inlines and then by marking the edges starting from not imported function. If `Verbose` is set to true, then it also dumps statistics per each inlined function, sorted by the greatest inlines count like - number of performed inlines - number of performed inlines to importing module Reviewers: eraman, tejohnson, mehdi_amini Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D22491 llvm-svn: 277089
* [sanitizer] Simplify and future-proof maybeMarkSanitizerLibraryCallNoBuiltin().Evgeniy Stepanov2016-07-281-17/+6
| | | | | | | | | | | | | | | | | | | | Sanitizers set nobuiltin attribute on certain library functions to avoid a situation where such function is neither instrumented nor intercepted. At the moment the list of interesting functions is hardcoded. This change replaces it with logic based on TargetLibraryInfo::hasOptimizedCodegen and the presense of readnone function attribute (sanitizers are generally interested in memory behavior of library functions). This is expected to be a no-op change: the new logic matches exactly the same set of functions. r276771 (currently reverted) added mempcpy() to the list, breaking MSan tests. With this change, r276771 can be safely re-landed. llvm-svn: 277086
* Do not remove empty lifetime.start/lifetime.end rangesVitaly Buka2016-07-281-0/+5
| | | | | | | | | | | | | | | | | | | | | | | Summary: Asan stack-use-after-scope check should poison alloca even if there is no access between start and end. This is possible for code like this: for (int i = 0; i < 3; i++) { int x; p = &x; } "Loop Invariant Code Motion" will move "p = &x;" out of the loop, making start/end range empty. PR27453 Reviewers: eugenis Differential Revision: https://reviews.llvm.org/D22842 llvm-svn: 277072
* Should be committed as one CL.Vitaly Buka2016-07-281-5/+0
| | | | | | This reverts commits r277068 r277067 r277066. llvm-svn: 277071
* [asan] Add const into few methodsVitaly Buka2016-07-281-10/+10
| | | | | | | | | | | | Summary: No functional changes Reviewers: eugenis Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D22899 llvm-svn: 277069
* Do not remove empty lifetime.start/lifetime.end rangesVitaly Buka2016-07-281-8/+5
| | | | | | | | | | | | | | | | | | | | | | | Summary: Asan stack-use-after-scope check should poison alloca even if there is no access between start and end. This is possible for code like this: for (int i = 0; i < 3; i++) { int x; p = &x; } "Loop Invariant Code Motion" will move "p = &x;" out of the loop, making start/end range empty. PR27453 Reviewers: eugenis Differential Revision: https://reviews.llvm.org/D22842 llvm-svn: 277068
* manedVitaly Buka2016-07-281-6/+8
| | | | llvm-svn: 277067
* rangeVitaly Buka2016-07-281-0/+6
| | | | llvm-svn: 277066
* [PM] Port LowerGuardIntrinsic to the new PM.Michael Kuperstein2016-07-282-9/+23
| | | | llvm-svn: 277057
* Revert r277038 until clearing why tests fail.Alina Sbirlea2016-07-281-9/+10
| | | | llvm-svn: 277039
* Remove TargetBaseAlign. Keep alignment for stack adjustments.Alina Sbirlea2016-07-281-10/+9
| | | | | | | | | | | | | | Summary: TargetBaseAlign is no longer required since LSV checks if target allows misaligned accesses. A constant defining a base alignment is still needed for stack accesses where alignment can be adjusted. Reviewers: llvm-commits, jlebar Subscribers: mzolotukhin, arsenm Differential Revision: https://reviews.llvm.org/D22936 llvm-svn: 277038
* Really try to pacify the build bots :/David Majnemer2016-07-281-1/+1
| | | | llvm-svn: 277037
* Try to passify the buildersDavid Majnemer2016-07-281-0/+1
| | | | llvm-svn: 277036
* [coroutines] Part 3 of N: Adding Boilerplate for Coroutine PassesDavid Majnemer2016-07-2810-1/+307
| | | | | | | | | | | | | This adds boilerplate code for all coroutine passes, the passes are no-ops for now. Also, a small test has been added to verify that passes execute in the expected order or not at all if coroutine support is disabled. Patch by Gor Nishanov! Differential Revision: https://reviews.llvm.org/D22847 llvm-svn: 277033
* Add EP_CGSCCOptimizerLate extension point to PassManagerBuilderDavid Majnemer2016-07-281-0/+1
| | | | | | | | | | | | The EP_CGSCCOptimizerLate extension point allows adding CallGraphSCC passes at the end of the main CallGraphSCC passes and before any function simplification passes run by CGPassManager. Patch by Gor Nishanov! Differential Revision: https://reviews.llvm.org/D22897 llvm-svn: 276953
* [InstCombine] Handle failures from ConstantFoldConstantExpressionDavid Majnemer2016-07-281-1/+2
| | | | | | | | ConstantFoldConstantExpression returns null when folding fails. This fixes PR28745. llvm-svn: 276952
* Fix the assertion error in collectLoopUniforms caused by empty Worklist ↵Wei Mi2016-07-271-2/+2
| | | | | | | | | | before expanding. Contributed-by: David Callahan Differential Revision: https://reviews.llvm.org/D22886 llvm-svn: 276943
* Add verifyAnalysis for LCSSA.Michael Zolotukhin2016-07-271-0/+5
| | | | | | | | | | | | | | | Summary: LCSSAWrapperPass currently doesn't override verifyAnalysis method, so pass manager doesn't verify LCSSA. This patch adds the method so that we start verifying LCSSA between loop passes. Reviewers: chandlerc, sanjoy, hfinkel Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D22888 llvm-svn: 276941
* [LSV] Use Instruction*s rather than Value*s where possible.Justin Lebar2016-07-271-87/+94
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: Given the crash in D22878, this patch converts the load/store vectorizer to use explicit Instruction*s wherever possible. This is an overall simplification and should be an improvement in safety, as we have fewer naked cast<>s, and now where we use Value*, we really mean something different from Instruction*. This patch also gets rid of some cast<>s around Value*s returned by Builder. Given that Builder constant-folds everything, we can't assume much about what we get out of it. One downside of this patch is that we have to copy our chain before calling propagateMetadata. But I don't think this is a big deal, as our chains are very small (usually 2 or 4 elems). Reviewers: asbirlea Subscribers: mzolotukhin, llvm-commits, arsenm Differential Revision: https://reviews.llvm.org/D22887 llvm-svn: 276938
* [LSV] Don't assume that bitcast ops are Instructions.Justin Lebar2016-07-271-9/+6
| | | | | | | | | | | | | | Summary: When we ask the builder to create a bitcast on a constant, we get back a constant, not an instruction. Reviewers: asbirlea Subscribers: jholewinski, mzolotukhin, llvm-commits, arsenm Differential Revision: https://reviews.llvm.org/D22878 llvm-svn: 276922
* [DSE] Fix bug in updating MadeChange flagJun Bum Lim2016-07-271-2/+2
| | | | | | | | | | | | Summary: The MadeChange flag should be ORed to keep the previous result. Reviewers: mcrosier Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D22873 llvm-svn: 276894
* Refactor - CodeExtractor : Move check for valid block to static utilitySean Silva2016-07-271-2/+2
| | | | | | | | | | | This lets you actually check to see if a block is valid before trying to extract. Patch by River Riddle! Differential Revision: https://reviews.llvm.org/D22699 llvm-svn: 276846
* [GVNHoist] Fix typo in assert.George Burgess IV2016-07-271-1/+1
| | | | | | This fixes PR28730. llvm-svn: 276844
* GVN-hoist: improve code generation for recursive GEPsSebastian Pop2016-07-271-42/+82
| | | | | | | | | | | | | | | | | | | When loading or storing in a field of a struct like "a.b.c", GVN is able to detect the equivalent expressions, and GVN-hoist would fail in the code generation. This is because the GEPs are not hoisted as scalar operations to avoid moving the GEPs too far from their ld/st instruction when the ld/st is not movable. So we end up having to generate code for the GEP of a ld/st when we move the ld/st. In the case of a GEP referring to another GEP as in "a.b.c" we need to code generate all the GEPs necessary to make all the operands available at the new location for the ld/st. With this patch we recursively walk through the GEP operands checking whether all operands are available, and in the case of a GEP operand, it recursively makes all its operands available. Code generation happens from the inner GEPs out until reaching the GEP that appears as an operand of the ld/st. Differential Revision: https://reviews.llvm.org/D22599 llvm-svn: 276841
* GVN-hoist: use DFS numbers instead of walking the instruction streamSebastian Pop2016-07-271-10/+3
| | | | | | | | | The patch replaces a function that walks the IR with a call to firstInBB() that uses the DFS numbering. NFC. Differential Revision: https://reviews.llvm.org/D22809 llvm-svn: 276840
* Reverting r276771 due to MSan failures.Andrew Kaylor2016-07-271-1/+0
| | | | llvm-svn: 276824
OpenPOWER on IntegriCloud