summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
...
* [LV] Prevent interleaving if computeMaxVF returned None.Florian Hahn2019-02-072-22/+35
| | | | | | | | | | | | | | | As discussed in D57382, interleaving should be avoided if computeMaxVF returns None, same as we currently do for vectorization. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=6477 Reviewers: Ayal, dcaballe, hsaito, mkuper, rengolin Reviewed By: Ayal Differential Revision: https://reviews.llvm.org/D57837 llvm-svn: 353461
* [InstrProf] Avoid reconstructing Triple, NFCReid Kleckner2019-02-071-17/+15
| | | | llvm-svn: 353439
* [HotColdSplit] With PGO add profile entry metadata to split cold functionTeresa Johnson2019-02-071-2/+11
| | | | | | | | | | | | | | | | | | Summary: When compiling with profile data, ensure the split cold function gets cold function_entry_count metadata (just use 0 since it should be cold). Otherwise with function sections it will not be placed in the unlikely text section with other cold code. Reviewers: vsk Subscribers: sebpop, hiraditya, davidxl, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D57900 llvm-svn: 353434
* [LSR] Generate cross iteration indexesSam Parker2019-02-071-23/+67
| | | | | | | | | | | | | | Modify GenerateConstantOffsetsImpl to create offsets that can be used by indexed addressing modes. If formulae can be generated which result in the constant offset being the same size as the recurrence, we can generate a pre-indexed access. This allows the pointer to be updated via the single pre-indexed access so that (hopefully) no add/subs are required to update it for the next iteration. For small cores, this can significantly improve performance DSP-like loops. Differential Revision: https://reviews.llvm.org/D55373 llvm-svn: 353403
* [LICM/MSSA] Add promotion to scalars by building an AliasSetTracker with ↵Alina Sbirlea2019-02-063-45/+120
| | | | | | | | | | | | | | | | | | | | | | | | MemorySSA. Summary: Experimentally we found that promotion to scalars carries less benefits than sinking and hoisting in LICM. When using MemorySSA, we build an AliasSetTracker on demand in order to reuse the current infrastructure. We only build it if less than AccessCapForMSSAPromotion exist in the loop, a cap that is by default set to 250. This value ensures there are no runtime regressions, and there are small compile time gains for pathological cases. A much lower value (20) was found to yield a single regression in the llvm-test-suite and much higher benefits for compile times. Conservatively we set the current cap to a high value, but we will explore lowering it when MemorySSA is enabled by default. Reviewers: sanjoy, chandlerc Subscribers: nemanjai, jlebar, Prazek, george.burgess.iv, jfb, jsji, llvm-commits Differential Revision: https://reviews.llvm.org/D56625 llvm-svn: 353339
* [InstCombine] X | C == C --> (X & ~C) == 0Sanjay Patel2019-02-061-9/+18
| | | | | | | | | | We should canonicalize to one of these forms, and compare-with-zero could be more conducive to follow-on transforms. This also leads to generally better codegen as shown in PR40611: https://bugs.llvm.org/show_bug.cgi?id=40611 llvm-svn: 353313
* [NFC] Simplify check in guard wideningMax Kazantsev2019-02-061-9/+3
| | | | llvm-svn: 353290
* [NFC] Factor out detatchment of dead blocks from their erasingMax Kazantsev2019-02-061-18/+26
| | | | llvm-svn: 353277
* [LoopSimplifyCFG] Do not count dead exit blocks twice, make CFG simplerMax Kazantsev2019-02-061-1/+3
| | | | llvm-svn: 353276
* [NFC] Revert rL353274Max Kazantsev2019-02-061-10/+5
| | | | llvm-svn: 353275
* [NFC] Extend API of DeleteDeadBlock(s) to collect updates without DTUMax Kazantsev2019-02-061-5/+10
| | | | llvm-svn: 353274
* [NFC] Replace readonly SmallVectorImpl with ArrayRefMax Kazantsev2019-02-061-3/+2
| | | | llvm-svn: 353273
* [HotColdSplit] Move splitting after instrumented PGO useTeresa Johnson2019-02-061-5/+5
| | | | | | | | | | | | | | | | | | | | | Summary: Follow up to D57082 which moved splitting earlier in the pipeline, in order to perform it before inlining. However, it was moved too early, before the IR is annotated with instrumented PGO data. This caused the splitting to incorrectly determine cold functions. Move it to just after PGO annotation (still before inlining), in both pass managers. Reviewers: vsk, hiraditya, sebpop Subscribers: mehdi_amini, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D57805 llvm-svn: 353270
* Move DomTreeUpdater from IR to AnalysisRichard Trieu2019-02-0613-13/+13
| | | | | | | | DomTreeUpdater depends on headers from Analysis, but is in IR. This is a layering violation since Analysis depends on IR. Relocate this code from IR to Analysis to fix the layering violation. llvm-svn: 353265
* [HotColdSplit] Do not split out `resume` instructionsVedant Kumar2019-02-051-2/+6
| | | | | | | | | Resumes that are not reachable from a cleanup landing pad are considered to be unreachable. It’s not safe to split them out. rdar://47808235 llvm-svn: 353242
* [InstCombine] limit extracting shuffle transform based on usesSanjay Patel2019-02-051-0/+5
| | | | | | | | | | As discussed in D53037, this can lead to worse codegen, and we don't generally expect the backend to be able to optimize arbitrary shuffles. If there's only one use of the 1st shuffle, that means it's getting removed, so that should always be safe. llvm-svn: 353235
* [PGO] Use a function for creating variable for profile file name. NFC.Rong Xu2019-02-051-16/+2
| | | | | | | Factored out the code for creating variable for profile file name to a function. llvm-svn: 353230
* [DebugInfo][NFCI] Split salvageDebugInfo into helper functionsJeremy Morse2019-02-051-82/+82
| | | | | | | | | | | | | | | Some use cases are appearing where salvaging is needed that does not correspond to an instruction being deleted -- for example an instruction being sunk, or a Value not being available in a block being isel'd. Enable more fine grained control over how salavging occurs by splitting the logic into helper functions, separating things that are specific to working on DbgVariableIntrinsics from those specific to interpreting IR and building DIExpressions. Differential Revision: https://reviews.llvm.org/D57696 llvm-svn: 353156
* [LSR] Check SCEV on isZero() after extend. PR40514Max Kazantsev2019-02-051-3/+21
| | | | | | | | | | | | | | | | | | | | | When LSR first adds SCEVs to BaseRegs, it only does it if `isZero()` has returned false. In the end, in invocation of `InsertFormula`, it asserts that all values there are still not zero constants. However between these two points, it makes some transformations, in particular extends them to wider type. SCEV does not give us guarantee that if `S` is not a constant zero, then `sext(S)` is also not a constant zero. It might have missed some optimizing transforms when it was calculating `S` and then made them when it took `sext`. For example, it may happen if previously optimizing transforms were limited by depth or somehow else. This patch adds a bailout when we may end up with a zero SCEV after extension. Differential Revision: https://reviews.llvm.org/D57565 Reviewed By: samparker llvm-svn: 353136
* Fix narrowing issue from r353129Richard Trieu2019-02-051-1/+2
| | | | llvm-svn: 353134
* [SamplePGO][NFC] Minor improvement to replace a temporary vector with aWei Mi2019-02-051-3/+2
| | | | | | | | brace-enclosed init list. Differential Revision: https://reviews.llvm.org/D57726 llvm-svn: 353129
* [SamplePGO] Minor efficiency improvement in samplePGO ICPTeresa Johnson2019-02-051-3/+3
| | | | | | | | | | | | | | | | | | | | Summary: When attaching prof metadata to promoted direct calls in SamplePGO mode, no need to construct and use a SmallVector to pass a single count to the ArrayRef parameter, we can simply use a brace-enclosed init list. This made a small but consistent improvement for a ThinLTO backend compile I was measuring. Reviewers: wmi Subscribers: mehdi_amini, dexonsmith, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D57706 llvm-svn: 353123
* [SanitizerCoverage] Clang crashes if user declares `__sancov_lowest_stack` ↵Julian Lettner2019-02-041-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | variable Summary: If the user declares or defines `__sancov_lowest_stack` with an unexpected type, then `getOrInsertGlobal` inserts a bitcast and the following cast fails: ``` Constant *SanCovLowestStackConstant = M.getOrInsertGlobal(SanCovLowestStackName, IntptrTy); SanCovLowestStack = cast<GlobalVariable>(SanCovLowestStackConstant); ``` This variable is a SanitizerCoverage implementation detail and the user should generally never have a need to access it, so we emit an error now. rdar://problem/44143130 Reviewers: morehouse Differential Revision: https://reviews.llvm.org/D57633 llvm-svn: 353100
* [InstCombine] Cleanup the TFE/LWE check in AMDGPU SimplifyDemandedNicolai Haehnle2019-02-042-18/+14
| | | | | | | | | | | | | | | | | | | | | | | | Summary: The fix added in r352904 is not quite correct, or rather misleading: 1. When the texfailctrl (TFC) argument was non-constant, the fix assumed non-TFE/LWE, which is incorrect. 2. Regardless, this code path cannot even be hit for correct TFE/LWE-enabled calls, because those return a struct. Added a test case for those for completeness. Change-Id: I92d314dbc67a2670f6d7adaab765ef45f56a49cf Reviewers: hliao, dstuttard, arsenm Subscribers: kzhuravl, jvesely, wdng, yaxunl, tpr, t-tye, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D57681 llvm-svn: 353097
* [NewPM][MSan] Add Options HandlingPhilip Pfaffe2019-02-041-18/+12
| | | | | | | | | | | | | | Summary: This patch enables passing options to msan via the passes pipeline, e.e., -passes=msan<recover;kernel;track-origins=4>. Reviewers: chandlerc, fedor.sergeev, leonardchan Subscribers: hiraditya, bollu, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D57640 llvm-svn: 353090
* [WarnMissedTransforms] Do not warn about already vectorized loops.Michael Kruse2019-02-041-7/+7
| | | | | | | | | | | | | LoopVectorize adds llvm.loop.isvectorized, but leaves llvm.loop.vectorize.enable. Do not consider such a loop for user-forced vectorization since vectorization already happened -- by prioritizing llvm.loop.isvectorized except for TM_SuppressedByUser. Fixes http://llvm.org/PR40546 Differential Revision: https://reviews.llvm.org/D57542 llvm-svn: 353082
* [NFC] Make a check in GuardWidening more obviousMax Kazantsev2019-02-041-1/+1
| | | | llvm-svn: 353038
* [NFC] Rename variables to reflect the actual status of GuardWideningMax Kazantsev2019-02-041-35/+34
| | | | llvm-svn: 353036
* [NFC] Remove redundant parameters for better readabilityMax Kazantsev2019-02-041-12/+8
| | | | llvm-svn: 353034
* [NFC] Replace equivalent condition for better readabilityMax Kazantsev2019-02-041-1/+2
| | | | llvm-svn: 353032
* [LoopIdiomRecognize] @llvm.dbg values shouldn't affect the transformation.Davide Italiano2019-02-031-2/+8
| | | | | | | | | | | | | | Summary: PR40564 Reviewers: aprantl, rnk Subscribers: llvm-commits, hiraditya Tags: #llvm Differential Revision: https://reviews.llvm.org/D57629 llvm-svn: 353007
* [LCSSA] Handle case with single new PHI faster.Florian Hahn2019-02-021-3/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If there is only a single available value, all uses must be dominated by the single value and there is no need to search for a reaching definition. This drastically speeds up LCSSA in some cases. For the test case from PR37202, it speeds up LCSSA construction by 4 times. Time-passes without this patch for test case from PR37202: Total Execution Time: 29.9285 seconds (29.9276 wall clock) ---User Time--- --System Time-- --User+System-- ---Wall Time--- --- Name --- 5.2786 ( 17.7%) 0.0021 ( 1.2%) 5.2806 ( 17.6%) 5.2808 ( 17.6%) Unswitch loops 4.3739 ( 14.7%) 0.0303 ( 18.1%) 4.4042 ( 14.7%) 4.4042 ( 14.7%) Loop-Closed SSA Form Pass 4.2658 ( 14.3%) 0.0192 ( 11.5%) 4.2850 ( 14.3%) 4.2851 ( 14.3%) Loop-Closed SSA Form Pass #2 2.2307 ( 7.5%) 0.0013 ( 0.8%) 2.2320 ( 7.5%) 2.2318 ( 7.5%) Loop Invariant Code Motion 2.0888 ( 7.0%) 0.0012 ( 0.7%) 2.0900 ( 7.0%) 2.0897 ( 7.0%) Unroll loops 1.6761 ( 5.6%) 0.0013 ( 0.8%) 1.6774 ( 5.6%) 1.6774 ( 5.6%) Value Propagation 1.3686 ( 4.6%) 0.0029 ( 1.8%) 1.3716 ( 4.6%) 1.3714 ( 4.6%) Induction Variable Simplification 1.1457 ( 3.8%) 0.0010 ( 0.6%) 1.1468 ( 3.8%) 1.1468 ( 3.8%) Loop-Closed SSA Form Pass #4 1.1384 ( 3.8%) 0.0005 ( 0.3%) 1.1389 ( 3.8%) 1.1389 ( 3.8%) Loop-Closed SSA Form Pass #6 1.1360 ( 3.8%) 0.0027 ( 1.6%) 1.1387 ( 3.8%) 1.1387 ( 3.8%) Loop-Closed SSA Form Pass #5 1.1331 ( 3.8%) 0.0010 ( 0.6%) 1.1341 ( 3.8%) 1.1340 ( 3.8%) Loop-Closed SSA Form Pass #3 Time passes with this patch Total Execution Time: 19.2802 seconds (19.2813 wall clock) ---User Time--- --System Time-- --User+System-- ---Wall Time--- --- Name --- 4.4234 ( 23.2%) 0.0038 ( 2.0%) 4.4272 ( 23.0%) 4.4273 ( 23.0%) Unswitch loops 2.3828 ( 12.5%) 0.0020 ( 1.1%) 2.3848 ( 12.4%) 2.3847 ( 12.4%) Unroll loops 1.8714 ( 9.8%) 0.0020 ( 1.1%) 1.8734 ( 9.7%) 1.8735 ( 9.7%) Loop Invariant Code Motion 1.7973 ( 9.4%) 0.0022 ( 1.2%) 1.7995 ( 9.3%) 1.8003 ( 9.3%) Value Propagation 1.4010 ( 7.3%) 0.0033 ( 1.8%) 1.4043 ( 7.3%) 1.4044 ( 7.3%) Induction Variable Simplification 0.9978 ( 5.2%) 0.0244 ( 13.1%) 1.0222 ( 5.3%) 1.0224 ( 5.3%) Loop-Closed SSA Form Pass #2 0.9611 ( 5.0%) 0.0257 ( 13.8%) 0.9868 ( 5.1%) 0.9868 ( 5.1%) Loop-Closed SSA Form Pass 0.5856 ( 3.1%) 0.0015 ( 0.8%) 0.5871 ( 3.0%) 0.5869 ( 3.0%) Unroll loops #2 0.4132 ( 2.2%) 0.0012 ( 0.7%) 0.4145 ( 2.1%) 0.4143 ( 2.1%) Loop Invariant Code Motion #3 Reviewers: efriedma, davide, mzolotukhin Reviewed By: efriedma Differential Revision: https://reviews.llvm.org/D57033 llvm-svn: 352960
* [LCSSA] Add expensive verification of LCSSA form for sub-loops.Florian Hahn2019-02-021-0/+6
| | | | | | | | | | | | | | This assertion makes sure all sub-loops are in LCSSA form before bringing their parent in LCSSA form. This precondition was added to formLCSSA in D56848. Reviewers: davide, efriedma, mzolotukhin Reviewed By: davide Differential Revision: https://reviews.llvm.org/D56921 llvm-svn: 352958
* [ASan] Do not instrument other runtime functions with `__asan_handle_no_return`Julian Lettner2019-02-021-2/+3
| | | | | | | | | | | | | | Summary: Currently, ASan inserts a call to `__asan_handle_no_return` before every `noreturn` function call/invoke. This is unnecessary for calls to other runtime funtions. This patch changes ASan to skip instrumentation for functions calls marked with `!nosanitize` metadata. Reviewers: TODO Differential Revision: https://reviews.llvm.org/D57489 llvm-svn: 352948
* [opaque pointer types] Pass function type for CallBase::setCalledFunction.James Y Knight2019-02-014-14/+14
| | | | | | Differential Revision: https://reviews.llvm.org/D57174 llvm-svn: 352914
* [opaque pointer types] Pass value type to GetElementPtr creation.James Y Knight2019-02-0122-131/+167
| | | | | | | | | This cleans up all GetElementPtr creation in LLVM to explicitly pass a value type rather than deriving it from the pointer's element-type. Differential Revision: https://reviews.llvm.org/D57173 llvm-svn: 352913
* [opaque pointer types] Pass value type to LoadInst creation.James Y Knight2019-02-0137-213/+288
| | | | | | | | | This cleans up all LoadInst creation in LLVM to explicitly pass the value type rather than deriving it from the pointer's element-type. Differential Revision: https://reviews.llvm.org/D57172 llvm-svn: 352911
* [opaque pointer types] Pass function types to InvokeInst creation.James Y Knight2019-02-013-5/+6
| | | | | | | | | This cleans up all InvokeInst creation in LLVM to explicitly pass a function type rather than deriving it from the pointer's element-type. Differential Revision: https://reviews.llvm.org/D57171 llvm-svn: 352910
* [opaque pointer types] Pass function types to CallInst creation.James Y Knight2019-02-0120-80/+97
| | | | | | | | | This cleans up all CallInst creation in LLVM to explicitly pass a function type rather than deriving it from the pointer's element-type. Differential Revision: https://reviews.llvm.org/D57170 llvm-svn: 352909
* [InstCombine] Extra null-checking on TFE/LWE supportMichael Liao2019-02-011-4/+3
| | | | | | | | - If that operand is not ConstantInt, skip enabling TFE/LWE. Differential Revision: https://reviews.llvm.org/D57539 llvm-svn: 352904
* [InstCombine] reduce duplicate code; NFCSanjay Patel2019-02-011-2/+1
| | | | | | | | | An unused variable problem was introduced with rL352870 and stubbed out with rL352871, but we can make a better fix by actually using the local variable in code rather than just the assert. llvm-svn: 352873
* [InstCombine] Fix -Wunused-variable when -DLLVM_ENABLE_ASSERTIONS=offFangrui Song2019-02-011-0/+1
| | | | llvm-svn: 352871
* [InstCombine] try to reduce x86 addcarry to generic uaddo intrinsicSanjay Patel2019-02-011-0/+33
| | | | | | | | | | | | | | | | | | | If we can reduce the x86-specific intrinsic to the generic op, it allows existing simplifications and value tracking folds. AFAICT, this always results in identical x86 codegen in the non-reduced case...which should be true because we semi-generically (too aggressively IMO) convert to llvm.uadd.with.overflow in CGP, so the DAG/isel must already combine/lower this intrinsic as expected. This isn't quite what was requested in: https://bugs.llvm.org/show_bug.cgi?id=40486 ...but we want to have these kinds of folds early for efficiency and to enable greater simplifications. For the case in the bug report where we have: _addcarry_u64(0, ahi, 0, &ahi) ...this gets completely simplified away in IR. Differential Revision: https://reviews.llvm.org/D57453 llvm-svn: 352870
* Provide reason messages for unviable inliningYevgeny Rouban2019-02-011-6/+15
| | | | | | | | | | | | | InlineCost's isInlineViable() is changed to return InlineResult instead of bool. This provides messages for failure reasons and allows to get more specific messages for cases where callsites are not viable for inlining. Reviewed By: xbolva00, anemet Differential Revision: https://reviews.llvm.org/D57089 llvm-svn: 352849
* [SLPVectorizer] Get rid of IndexQueue array from vectorizeStores. NFCI.Yevgeny Rouban2019-02-011-27/+18
| | | | | | | | Indices are checked as they are generated. No need to fill the whole array of indices. Differential Revision: https://reviews.llvm.org/D57144 llvm-svn: 352839
* [opaque pointer types] Add a FunctionCallee wrapper type, and use it.James Y Knight2019-02-0120-527/+523
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Recommit r352791 after tweaking DerivedTypes.h slightly, so that gcc doesn't choke on it, hopefully. Original Message: The FunctionCallee type is effectively a {FunctionType*,Value*} pair, and is a useful convenience to enable code to continue passing the result of getOrInsertFunction() through to EmitCall, even once pointer types lose their pointee-type. Then: - update the CallInst/InvokeInst instruction creation functions to take a Callee, - modify getOrInsertFunction to return FunctionCallee, and - update all callers appropriately. One area of particular note is the change to the sanitizer code. Previously, they had been casting the result of `getOrInsertFunction` to a `Function*` via `checkSanitizerInterfaceFunction`, and storing that. That would report an error if someone had already inserted a function declaraction with a mismatching signature. However, in general, LLVM allows for such mismatches, as `getOrInsertFunction` will automatically insert a bitcast if needed. As part of this cleanup, cause the sanitizer code to do the same. (It will call its functions using the expected signature, however they may have been declared.) Finally, in a small number of locations, callers of `getOrInsertFunction` actually were expecting/requiring that a brand new function was being created. In such cases, I've switched them to Function::Create instead. Differential Revision: https://reviews.llvm.org/D57315 llvm-svn: 352827
* [sanitizer-coverage] prune trace-cmp instrumentation for CMP isntructions ↵Kostya Serebryany2019-01-311-2/+34
| | | | | | that feed into the backedge branch. Instrumenting these CMP instructions is almost always useless (and harmful) for fuzzing llvm-svn: 352818
* Revert "[opaque pointer types] Add a FunctionCallee wrapper type, and use it."James Y Knight2019-01-3120-523/+527
| | | | | | | | | This reverts commit f47d6b38c7a61d50db4566b02719de05492dcef1 (r352791). Seems to run into compilation failures with GCC (but not clang, where I tested it). Reverting while I investigate. llvm-svn: 352800
* [EarlyCSE & MSSA] Cleanup special handling for removing MemoryAccesses.Alina Sbirlea2019-01-311-30/+5
| | | | | | | | | | | | Summary: Moving special handling to MemorySSAUpdater in D57199. Reviewers: gberry, george.burgess.iv Subscribers: sanjoy, jlebar, Prazek, llvm-commits Differential Revision: https://reviews.llvm.org/D57200 llvm-svn: 352794
* [opaque pointer types] Add a FunctionCallee wrapper type, and use it.James Y Knight2019-01-3120-527/+523
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The FunctionCallee type is effectively a {FunctionType*,Value*} pair, and is a useful convenience to enable code to continue passing the result of getOrInsertFunction() through to EmitCall, even once pointer types lose their pointee-type. Then: - update the CallInst/InvokeInst instruction creation functions to take a Callee, - modify getOrInsertFunction to return FunctionCallee, and - update all callers appropriately. One area of particular note is the change to the sanitizer code. Previously, they had been casting the result of `getOrInsertFunction` to a `Function*` via `checkSanitizerInterfaceFunction`, and storing that. That would report an error if someone had already inserted a function declaraction with a mismatching signature. However, in general, LLVM allows for such mismatches, as `getOrInsertFunction` will automatically insert a bitcast if needed. As part of this cleanup, cause the sanitizer code to do the same. (It will call its functions using the expected signature, however they may have been declared.) Finally, in a small number of locations, callers of `getOrInsertFunction` actually were expecting/requiring that a brand new function was being created. In such cases, I've switched them to Function::Create instead. Differential Revision: https://reviews.llvm.org/D57315 llvm-svn: 352791
OpenPOWER on IntegriCloud