summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/Utils
Commit message (Collapse)AuthorAgeFilesLines
* [Loop Peeling] Enable peeling of multiple exits by default.Serguei Katkov2019-07-191-1/+1
| | | | | | | | | | | | Enable loop peeling with multiple exits where all non-latch exits ends up with deopt by default. Reviewers: reames, fhahn Reviewed By: reames Subscribers: xbolva00, hiraditya, zzheng, llvm-commits Differential Revision: https://reviews.llvm.org/D64619 llvm-svn: 366542
* [LoopInfo] Use early return in branch weight update functions. NFC.Serguei Katkov2019-07-181-29/+30
| | | | llvm-svn: 366411
* Fix parameter name comments using clang-tidy. NFC.Rui Ueyama2019-07-162-2/+2
| | | | | | | | | | | | | | | | | | | | | This patch applies clang-tidy's bugprone-argument-comment tool to LLVM, clang and lld source trees. Here is how I created this patch: $ git clone https://github.com/llvm/llvm-project.git $ cd llvm-project $ mkdir build $ cd build $ cmake -GNinja -DCMAKE_BUILD_TYPE=Debug \ -DLLVM_ENABLE_PROJECTS='clang;lld;clang-tools-extra' \ -DCMAKE_EXPORT_COMPILE_COMMANDS=On -DLLVM_ENABLE_LLD=On \ -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ ../llvm $ ninja $ parallel clang-tidy -checks='-*,bugprone-argument-comment' \ -config='{CheckOptions: [{key: StrictMode, value: 1}]}' -fix \ ::: ../llvm/lib/**/*.{cpp,h} ../clang/lib/**/*.{cpp,h} ../lld/**/*.{cpp,h} llvm-svn: 366177
* ARM MTE stack sanitizer.Evgeniy Stepanov2019-07-151-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | Add "memtag" sanitizer that detects and mitigates stack memory issues using armv8.5 Memory Tagging Extension. It is similar in principle to HWASan, which is a software implementation of the same idea, but there are enough differencies to warrant a new sanitizer type IMHO. It is also expected to have very different performance properties. The new sanitizer does not have a runtime library (it may grow one later, along with a "debugging" mode). Similar to SafeStack and StackProtector, the instrumentation pass (in a follow up change) will be inserted in all cases, but will only affect functions marked with the new sanitize_memtag attribute. Reviewers: pcc, hctim, vitalybuka, ostannard Subscribers: srhines, mehdi_amini, javed.absar, kristof.beyls, hiraditya, cryptoad, steven_wu, dexonsmith, cfe-commits, llvm-commits Tags: #clang, #llvm Differential Revision: https://reviews.llvm.org/D64169 llvm-svn: 366123
* [Loop Peeling] Fix the bug with IDom setting for exit loopsSerguei Katkov2019-07-151-3/+18
| | | | | | | | | | | | | It is possible that loop exit has two predecessors in a loop body. In this case after the peeling the iDom of the exit should be a clone of iDom of original exit but no a clone of a block coming to this exit. Reviewers: reames, fhahn Reviewed By: reames Subscribers: hiraditya, zzheng, llvm-commits Differential Revision: https://reviews.llvm.org/D64618 llvm-svn: 366050
* [Loop Peeling] Enable peeling for loops with multiple exitsSerguei Katkov2019-07-152-1/+22
| | | | | | | | | | | | | | | This CL enables peeling of the loop with multiple exits where one exit should be from latch and others are basic blocks with call to deopt. The peeling is enabled under the flag which is false by default. Reviewers: reames, mkuper, iajbar, fhahn Reviewed By: reames Subscribers: xbolva00, hiraditya, zzheng, llvm-commits Differential Revision: https://reviews.llvm.org/D63923 llvm-svn: 366048
* [LoopUtils] Extend the scope of getLoopEstimatedTripCountSerguei Katkov2019-07-151-6/+14
| | | | | | | | | | | | | | | With this patch the getLoopEstimatedTripCount function will accept also the loops where there are more than one exit but all exits except latch block should ends up with a call to deopt. This side exits should not impact the estimated trip count. Reviewers: reames, mkuper, danielcdh Reviewed By: reames Subscribers: fhahn, lebedev.ri, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D64553 llvm-svn: 366042
* [LoopInfo] Introduce getUniqueNonLatchExitBlocks utility functionSerguei Katkov2019-07-151-13/+7
| | | | | | | | | | | | Extract the code from LoopUnrollRuntime into utility function to re-use it in D63923. Reviewers: reames, mkuper Reviewed By: reames Subscribers: fhahn, hiraditya, zzheng, dmgreen, llvm-commits Differential Revision: https://reviews.llvm.org/D64548 llvm-svn: 366040
* [MemorySSA] Use SetVector to avoid nondeterminism.Alina Sbirlea2019-07-122-2/+3
| | | | | | | | | | | | | | | | Summary: Use a SetVector for DeadBlockSet. Resolves PR42574. Reviewers: george.burgess.iv, uabelho, dblaikie Subscribers: jlebar, Prazek, mgrang, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D64601 llvm-svn: 365970
* Delete dead storesFangrui Song2019-07-121-1/+1
| | | | llvm-svn: 365903
* [InstCombine] Reorder pow() transformations (NFC)Evandro Menezes2019-07-121-23/+19
| | | | | | | Move the transformation from `powf(x, itofp(y))` to `powi(x, y)` to the group of transformations related to the exponent. llvm-svn: 365851
* [Attributor] Deduce "nosync" function attribute.Stefan Stipanovic2019-07-111-0/+1
| | | | | | | | | | | | | | Introduce and deduce "nosync" function attribute to indicate that a function does not synchronize with another thread in a way that other thread might free memory. Reviewers: jdoerfert, jfb, nhaehnle, arsenm Subscribers: wdng, hfinkel, nhaenhle, mehdi_amini, steven_wu, dexonsmith, arsenm, uenoku, hiraditya, jfb, llvm-commits Differential Revision: https://reviews.llvm.org/D62766 llvm-svn: 365830
* [InstCombine] Reorder recently added/improved pow transformationsDavid Bolvansky2019-07-111-3/+3
| | | | | | Changed cases are now faster with exp2. llvm-svn: 365758
* [LoopRotate + MemorySSA] Keep an <instruction-cloned instruction> map.Alina Sbirlea2019-07-101-4/+8
| | | | | | | | | | | | | | | | | | | | Summary: The map kept in loop rotate is used for instruction remapping, in order to simplify the clones of instructions. Thus, if an instruction can be simplified, its simplified value is placed in the map, even when the clone is added to the IR. MemorySSA in contrast needs to know about that clone, so it can add an access for it. To resolve this: keep a different map for MemorySSA. Reviewers: george.burgess.iv Subscribers: jlebar, Prazek, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D63680 llvm-svn: 365672
* [CodeExtractor] Fix sinking of allocas with multiple bitcast uses (PR42451)Vedant Kumar2019-07-101-13/+29
| | | | | | | | | | | | | | An alloca which can be sunk into the extraction region may have more than one bitcast use. Move these uses along with the alloca to prevent use-before-def. Testing: check-llvm, stage2 build of clang Fixes llvm.org/PR42451. Differential Revision: https://reviews.llvm.org/D64463 llvm-svn: 365660
* [CodeExtractor] Simplify findAllocas, NFCVedant Kumar2019-07-101-73/+91
| | | | | | | | | Split getLifetimeMarkers out into its own method and have it return a struct. Differential Revision: https://reviews.llvm.org/D64467 llvm-svn: 365659
* [InstCombine] pow(C,x) -> exp2(log2(C)*x)David Bolvansky2019-07-101-1/+24
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: Transform pow(C,x) To exp2(log2(C)*x) if C > 0, C != inf, C != NaN (and C is not power of 2, since we have some fold for such case already). log(C) is folded by the compiler and exp2 is much faster to compute than pow. Reviewers: spatel, efriedma, evandro Reviewed By: evandro Subscribers: lebedev.ri, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D64099 llvm-svn: 365637
* [Loop Peeling] Add support for peeling of loops with multiple exitsSerguei Katkov2019-07-092-23/+31
| | | | | | | | | | | | | | | | | | | | | This patch modifies the loop peeling transformation so that it does not expect that there is only one loop exit from latch. It modifies only transformation. Update of branch weights remains only for exit from latch. The motivation is that in follow-up patch I plan to enable loop peeling for loops with multiple exits but only if other exits then from latch one goes to block with call to deopt. For now this patch is NFC. Reviewers: reames, mkuper, iajbar, fhahn Reviewed By: reames, fhahn Subscribers: zzheng, llvm-commits Differential Revision: https://reviews.llvm.org/D63921 llvm-svn: 365441
* Keep the order of the basic blocks in the cloned loop as the originalWhitney Tsang2019-07-081-24/+25
| | | | | | | | | | | | | | loop Summary: Do the cloning in two steps, first allocate all the new loops, then clone the basic blocks in the same order as the original loop. Reviewer: Meinersbur, fhahn, kbarton, hfinkel Reviewed By: hfinkel Subscribers: hfinkel, hiraditya, llvm-commits Tag: https://reviews.llvm.org/D64224 Differential Revision: llvm-svn: 365366
* Add, and infer, a nofree function attributeBrian Homerding2019-07-082-0/+12
| | | | | | | | | | | | This patch adds a function attribute, nofree, to indicate that a function does not, directly or indirectly, call a memory-deallocation function (e.g., free, C++'s operator delete). Reviewers: jdoerfert Differential Revision: https://reviews.llvm.org/D49165 llvm-svn: 365336
* [ThinLTO] Attempt to recommit r365188 after alignment fixEugene Leviant2019-07-051-6/+7
| | | | llvm-svn: 365215
* Reverted r365188 due to alignment problems on i686-androidEugene Leviant2019-07-051-7/+6
| | | | llvm-svn: 365206
* [ThinLTO] Attempt to recommit r365040 after caching fixEugene Leviant2019-07-051-6/+7
| | | | | | | | | | | | | | It's possible that some function can load and store the same variable using the same constant expression: store %Derived* @foo, %Derived** bitcast (%Base** @bar to %Derived**) %42 = load %Derived*, %Derived** bitcast (%Base** @bar to %Derived**) The bitcast expression was mistakenly cached while processing loads, and never examined later when processing store. This caused @bar to be mistakenly treated as read-only variable. See load-store-caching.ll. llvm-svn: 365188
* [LoopPeel] Some small comment update. NFC.Serguei Katkov2019-07-041-3/+3
| | | | | | | Follow-up change of comment after https://reviews.llvm.org/D63917 is landed. llvm-svn: 365107
* Revert [ThinLTO] Optimize writeonly globals outReid Kleckner2019-07-041-7/+6
| | | | | | | | | This reverts r365040 (git commit 5cacb914758c7f436b47c8362100f10cef14bbc4) Speculatively reverting, since this appears to have broken check-lld on Linux. Partial analysis in https://crbug.com/981168. llvm-svn: 365097
* [ThinLTO] Optimize writeonly globals outEugene Leviant2019-07-031-6/+7
| | | | | | Differential revision: https://reviews.llvm.org/D63444 llvm-svn: 365040
* [LoopPeel] Re-factor llvm::peelLoop method. NFC.Serguei Katkov2019-07-031-25/+49
| | | | | | | | | | | Extract code dealing with branch weights in separate functions. Reviewers: reames, mkuper, iajbar, fhahn Reviewed By: reames, fhahn Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D63917 llvm-svn: 365002
* [NFC] Strenghten isInteger condition for rL364940David Bolvansky2019-07-021-2/+3
| | | | llvm-svn: 364969
* [SimplifyLibCalls] powf(x, sitofp(n)) -> powi(x, n)David Bolvansky2019-07-021-12/+47
| | | | | | | | | | | | | | | | | | | Summary: Partially solves https://bugs.llvm.org/show_bug.cgi?id=42190 Reviewers: spatel, nikic, efriedma Reviewed By: efriedma Subscribers: efriedma, nikic, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D63038 llvm-svn: 364940
* [PGO] Update ICP pass for recent byval type changesReid Kleckner2019-07-011-0/+9
| | | | | | | | | | Fixes verifier errors encountered in PR42413. Reviewers: xur, t.p.northover, inglorion, gbiv, george.burgess.iv Differential Revision: https://reviews.llvm.org/D63842 llvm-svn: 364861
* [InstCombine] canonicalize fmin/fmax to LLVM intrinsics minnum/maxnumSanjay Patel2019-06-291-24/+14
| | | | | | | | | | | | | | This transform came up in D62414, but we should deal with it first. We have LLVM intrinsics that correspond exactly to libm calls (unlike most libm calls, these libm calls never set errno). This holds without any fast-math-flags, so we should always canonicalize to those intrinsics directly for better optimization. Currently, we convert to fcmp+select only when we have FMF (nnan) because fcmp+select does not preserve the semantics of the call in the general case. Differential Revision: https://reviews.llvm.org/D63214 llvm-svn: 364714
* [Attr] Add "willreturn" function attributeJohannes Doerfert2019-06-271-0/+1
| | | | | | | | | | | | | | | | | | | | | | This patch introduces a new function attribute, willreturn, to indicate that a call of this function will either exhibit undefined behavior or comes back and continues execution at a point in the existing call stack that includes the current invocation. This attribute guarantees that the function does not have any endless loops, endless recursion, or terminating functions like abort or exit. Patch by Hideto Ueno (@uenoku) Reviewers: jdoerfert Subscribers: mehdi_amini, hiraditya, steven_wu, dexonsmith, lebedev.ri, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62801 llvm-svn: 364555
* IR: compare type attributes deeply when looking into functions.Tim Northover2019-06-271-0/+13
| | | | | | | | | FunctionComparator attempts to produce a stable comparison of two Function instances by looking at all available properties. Since ByVal attributes now contain a Type pointer, they are not trivially ordered and FunctionComparator should use its own Type comparison logic to sort them. llvm-svn: 364523
* [LoopUnroll] Add support for loops with exiting headers and uncond latches.Florian Hahn2019-06-261-60/+170
| | | | | | | | | | | | | | | | | | | | | | | | | | This patch generalizes the UnrollLoop utility to support loops that exit from the header instead of the latch. Usually, LoopRotate would take care of must of those cases, but in some cases (e.g. -Oz), LoopRotate does not kick in. Codesize impact looks relatively neutral on ARM64 with -Oz + LTO. Program master patch diff External/S.../CFP2006/447.dealII/447.dealII 629060.00 627676.00 -0.2% External/SPEC/CINT2000/176.gcc/176.gcc 1245916.00 1244932.00 -0.1% MultiSourc...Prolangs-C/simulator/simulator 86100.00 86156.00 0.1% MultiSourc...arks/Rodinia/backprop/backprop 66212.00 66252.00 0.1% MultiSourc...chmarks/Prolangs-C++/life/life 67276.00 67312.00 0.1% MultiSourc...s/Prolangs-C/compiler/compiler 69824.00 69788.00 -0.1% MultiSourc...Prolangs-C/assembler/assembler 86672.00 86696.00 0.0% Reviewers: efriedma, vsk, paquette Reviewed By: paquette Differential Revision: https://reviews.llvm.org/D61962 llvm-svn: 364398
* Expand cloneLoopWithPreheader() to support cloning loop nestWhitney Tsang2019-06-251-10/+31
| | | | | | | | | | | | Summary: cloneLoopWithPreheader() currently only support innermost loop, and assert otherwise. Reviewers: Meinersbur, fhahn, kbarton Reviewed By: Meinersbur Subscribers: hiraditya, jsji, llvm-commits Tag: LLVM Differential Revision: https://reviews.llvm.org/D63446 llvm-svn: 364310
* [DebugInfo@O2][LoopVectorize] pr39024: Vectorized code linenos step through ↵Orlando Cazalet-Hyams2019-06-191-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | loop even after completion Summary: Bug: https://bugs.llvm.org/show_bug.cgi?id=39024 The bug reports that a vectorized loop is stepped through 4 times and each step through the loop seemed to show a different path. I found two problems here: A) An incorrect line number on a preheader block (for.body.preheader) instruction causes a step into the loop before it begins. B) Instructions in the middle block have different line numbers which give the impression of another iteration. In this patch I give all of the middle block instructions the line number of the scalar loop latch terminator branch. This seems to provide the smoothest debugging experience because the vectorized loops will always end on this line before dropping into the scalar loop. To solve problem A I have altered llvm::SplitBlockPredecessors to accommodate loop header blocks. I have set up a separate review D61933 for a fix which is required for this patch. Reviewers: samsonov, vsk, aprantl, probinson, anemet, hfinkel, jmorse Reviewed By: hfinkel, jmorse Subscribers: jmorse, javed.absar, eraman, kcc, bjope, jmellorcrummey, hfinkel, gbedwell, hiraditya, zzheng, llvm-commits Tags: #llvm, #debug-info Differential Revision: https://reviews.llvm.org/D60831 > llvm-svn: 363046 llvm-svn: 363786
* [SimplifyCFG] NFC, prof branch_weighs handling is simplifiedYevgeny Rouban2019-06-181-49/+15
| | | | | | | | | Using the new SwitchInstProfUpdateWrapper this patch simplifies 3 places of prof branch_weights handling. Differential Revision: https://reviews.llvm.org/D62123 llvm-svn: 363652
* PHINode: introduce setIncomingValueForBlock() function, and use it.Whitney Tsang2019-06-173-8/+6
| | | | | | | | | | | | | | | | Summary: There is PHINode::getBasicBlockIndex() and PHINode::setIncomingValue() but no function to replace incoming value for a specified BasicBlock* predecessor. Clearly, there are a lot of places that could use that functionality. Reviewer: craig.topper, lebedev.ri, Meinersbur, kbarton, fhahn Reviewed By: Meinersbur, fhahn Subscribers: fhahn, hiraditya, zzheng, jsji, llvm-commits Tag: LLVM Differential Revision: https://reviews.llvm.org/D63338 llvm-svn: 363566
* Re-commit r357452 (take 3): "SimplifyCFG SinkCommonCodeFromPredecessors: ↵Hans Wennborg2019-06-171-14/+15
| | | | | | | | | | | Also sink function calls without used results (PR41259)" Third time's the charm. This was reverted in r363220 due to being suspected of an internal benchmark regression and a test failure, none of which turned out to be caused by this. llvm-svn: 363529
* [SimplifyCFG] Fix prof branch_weights MD while removing unreachable switch casesYevgeny Rouban2019-06-171-4/+5
| | | | | | | | | | | SimplifyCFG has a bug that results in inconsistent prof branch_weights metadata if unreachable switch cases are removed. This patch fixes this bug by making use of the newly introduced SwitchInstProfUpdateWrapper class (see patch D62122). A new test is created. Differential Revision: https://reviews.llvm.org/D62186 llvm-svn: 363527
* [SimplifyIndVar] Simplify non-overflowing saturating add/subNikita Popov2019-06-151-0/+24
| | | | | | | | | | | If we can detect that saturating math that depends on an IV cannot overflow, replace it with simple math. This is similar to the CVP optimization from D62703, just based on a different underlying analysis (SCEV vs LVI) that catches different cases. Differential Revision: https://reviews.llvm.org/D62792 llvm-svn: 363489
* [SimpligyCFG] NFC intended, remove GCD that was only used for powers of twoShawn Landden2019-06-141-13/+11
| | | | | | | | | | | | and replace with an equilivent countTrailingZeros. GCD is much more expensive than this, with repeated division. This depends on D60823 Differential Revision: https://reviews.llvm.org/D61151 llvm-svn: 363422
* [SimplifyCFG] reverting preliminary Switch patches againShawn Landden2019-06-131-11/+13
| | | | | | | | | This reverts 363226 and 363227, both NFC intended I swear I fixed the test case that is failing, and ran the tests, but I will look into it again. llvm-svn: 363229
* [SimpligyCFG] NFC intended, remove GCD that was only used for powers of twoShawn Landden2019-06-131-13/+11
| | | | | | | | | | | | and replace with an equilivent countTrailingZeros. GCD is much more expensive than this, with repeated division. This depends on D60823 Differential Revision: https://reviews.llvm.org/D61151 llvm-svn: 363227
* Revert r361811: 'Re-commit r357452 (take 2): "SimplifyCFG ↵David L. Jones2019-06-131-15/+14
| | | | | | | | | | | | | | | | SinkCommonCodeFromPredecessors ...' We have observed some failures with internal builds with this revision. - Performance regressions: - llvm's SingleSource/Misc evalloop shows performance regressions (although these may be red herrings). - Benchmarks for Abseil's SwissTable. - Correctness: - Failures for particular libicu tests when building the Google AppEngine SDK (for PHP). hwennborg has already been notified, and is aware of reproducer failures. llvm-svn: 363220
* LoopVersioning: Respect convergentMatt Arsenault2019-06-121-2/+3
| | | | | | | | | This changes the standalone pass only. Arguably the utility class itself should assert there are no convergent calls. However, a target pass with additional context may still be able to version a loop if all of the dynamic conditions are sufficiently uniform. llvm-svn: 363165
* Revert "[DebugInfo@O2][LoopVectorize] pr39024: Vectorized code linenos step ↵Orlando Cazalet-Hyams2019-06-121-7/+1
| | | | | | | | | through loop even after completion" This reverts commit 1a0f7a2077b70c9864faa476e15b048686cf1ca7. See phabricator thread for D60831. llvm-svn: 363132
* Generalize icmp matching in IndVars' eliminateTruncPhilip Reames2019-06-111-14/+15
| | | | | | | | We were only matching RHS being a loop invariant value, not the inverse. Since there's nothing which appears to canonicalize loop invariant values to RHS, this means we missed cases. Differential Revision: https://reviews.llvm.org/D63112 llvm-svn: 363108
* [DebugInfo@O2][LoopVectorize] pr39024: Vectorized code linenos step through ↵Orlando Cazalet-Hyams2019-06-111-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | loop even after completion Summary: Bug: https://bugs.llvm.org/show_bug.cgi?id=39024 The bug reports that a vectorized loop is stepped through 4 times and each step through the loop seemed to show a different path. I found two problems here: A) An incorrect line number on a preheader block (for.body.preheader) instruction causes a step into the loop before it begins. B) Instructions in the middle block have different line numbers which give the impression of another iteration. In this patch I give all of the middle block instructions the line number of the scalar loop latch terminator branch. This seems to provide the smoothest debugging experience because the vectorized loops will always end on this line before dropping into the scalar loop. To solve problem A I have altered llvm::SplitBlockPredecessors to accommodate loop header blocks. I have set up a separate review D61933 for a fix which is required for this patch. Reviewers: samsonov, vsk, aprantl, probinson, anemet, hfinkel, jmorse Reviewed By: hfinkel, jmorse Subscribers: jmorse, javed.absar, eraman, kcc, bjope, jmellorcrummey, hfinkel, gbedwell, hiraditya, zzheng, llvm-commits Tags: #llvm, #debug-info Differential Revision: https://reviews.llvm.org/D60831 llvm-svn: 363046
* Change semantics of fadd/fmul vector reductions.Sander de Smalen2019-06-111-8/+4
| | | | | | | | | | | | | | | | | | | | This patch changes how LLVM handles the accumulator/start value in the reduction, by never ignoring it regardless of the presence of fast-math flags on callsites. This change introduces the following new intrinsics to replace the existing ones: llvm.experimental.vector.reduce.fadd -> llvm.experimental.vector.reduce.v2.fadd llvm.experimental.vector.reduce.fmul -> llvm.experimental.vector.reduce.v2.fmul and adds functionality to auto-upgrade existing LLVM IR and bitcode. Reviewers: RKSimon, greened, dmgreen, nikic, simoll, aemerson Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D60261 llvm-svn: 363035
OpenPOWER on IntegriCloud