summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
...
* Pointer types were treated as zero-size by MergeICmpsChristy Lee2018-10-261-1/+2
| | | | | | | | | | | | | | | | Summary: The visitICmp analysis function would record compares of pointer types, as size 0. This causes the resulting memcmp() call to have the wrong total size. Found with "self-build" of clang/LLVM on Windows. Reviewers: christylee, trentxintong, courbet Reviewed By: courbet Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D53536 llvm-svn: 345413
* [SimpleLoopUnswitch] Unswitch by experimental.guard intrinsicsMax Kazantsev2018-10-261-2/+107
| | | | | | | | | | | | | | | | | | This patch adds support of `llvm.experimental.guard` intrinsics to non-trivial simple loop unswitching. These intrinsics represent implicit control flow which has pretty much the same semantics as usual conditional branches. The algorithm of dealing with them is following: - Consider guards as unswitching candidates; - If a guard is considered the best candidate, turn it into a branch; - Apply normal unswitching algorithm on this branch. The patch has no compile time effect on code that does not contain any guards. Differential Revision: https://reviews.llvm.org/D53744 Reviewed By: chandlerc llvm-svn: 345387
* [SimpleLoopUnswitch] Make all checks before actual non-trivial unswitchMax Kazantsev2018-10-261-18/+20
| | | | | | | | | | | We should be able to make all relevant checks before we actually start the non-trivial unswitching, so that we could guarantee that once we have started the transform, it will always succeed. Reviewed By: chandlerc Differential Revision: https://reviews.llvm.org/D53747 llvm-svn: 345375
* [FPEnv] Last BinaryOperator::isFNeg(...) to m_FNeg(...) changesCameron McInally2018-10-251-2/+3
| | | | | | | | | Replacing BinaryOperator::isFNeg(...) to avoid regressions when we separate FNeg from the FSub IR instruction. Differential Revision: https://reviews.llvm.org/D53650 llvm-svn: 345295
* [DebugInfo][Dexter] Unreachable line stepped onto after SimplifyCFG.Carlos Alberto Enciso2018-10-252-18/+45
| | | | | | | | When SimplifyCFG changes the PHI node into a select instruction, the debug line records becomes ambiguous. It causes the debugger to display unreachable source lines. Differential Revision: https://reviews.llvm.org/D53287 llvm-svn: 345250
* Add -instcombine-code-sinking optionGabor Buella2018-10-251-1/+5
| | | | | | | | | | Reviewers: craig.topper, andrew.w.kaylor, efriedma Reviewed By: craig.topper, andrew.w.kaylor, efriedma Differential Revision: https://reviews.llvm.org/D52709 llvm-svn: 345248
* Update MemorySSA in LoopRotate.Alina Sbirlea2018-10-242-13/+75
| | | | | | | | | | | | Summary: Teach LoopRotate to preserve MemorySSA. Enable tests for correctness, dependency disabled by default. Subscribers: sanjoy, jlebar, Prazek, george.burgess.iv, llvm-commits Differential Revision: https://reviews.llvm.org/D51718 llvm-svn: 345216
* [HotColdSplitting] Identify larger cold regions using domtree queriesVedant Kumar2018-10-242-201/+192
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The current splitting algorithm works in three stages: 1) Identify cold blocks, then 2) Use forward/backward propagation to mark hot blocks, then 3) Grow a SESE region of blocks *outside* of the set of hot blocks and start outlining. While testing this pass on Apple internal frameworks I noticed that some kinds of control flow (e.g. loops) are never outlined, even though they unconditionally lead to / follow cold blocks. I noticed two other issues related to how cold regions are identified: - An inconsistency can arise in the internal state of the hotness propagation stage, as a block may end up in both the ColdBlocks set and the HotBlocks set. Further inconsistencies can arise as these sets do not match what's in ProfileSummaryInfo. - It isn't necessary to limit outlining to single-exit regions. This patch teaches the splitting algorithm to identify maximal cold regions and outline them. A maximal cold region is defined as the set of blocks post-dominated by a cold sink block, or dominated by that sink block. This approach can successfully outline loops in the cold path. As a side benefit, it maintains less internal state than the current approach. Due to a limitation in CodeExtractor, blocks within the maximal cold region which aren't dominated by a single entry point (a so-called "max ancestor") are filtered out. Results: - X86 (LNT + -Os + externals): 134KB of TEXT were outlined compared to 47KB pre-patch, or a ~3x improvement. Did not see a performance impact across two runs. - AArch64 (LNT + -Os + externals + Apple-internal benchmarks): 149KB of TEXT were outlined. Ditto re: performance impact. - Outlining results improve marginally in the internal frameworks I tested. Follow-ups: - Outline more than once per function, outline large single basic blocks, & try to remove unconditional branches in outlined functions. Differential Revision: https://reviews.llvm.org/D53627 llvm-svn: 345209
* [hot-cold-split] Name split functions with ".cold" suffixTeresa Johnson2018-10-242-12/+20
| | | | | | | | | | | | | | | | | | | | | | | Summary: The current default of appending "_"+entry block label to the new extracted cold function breaks demangling. Change the deliminator from "_" to "." to enable demangling. Because the header block label will be empty for release compile code, use "extracted" after the "." when the label is empty. Additionally, add a mechanism for the client to pass in an alternate suffix applied after the ".", and have the hot cold split pass use "cold."+Count, where the Count is currently 1 but can be used to uniquely number multiple cold functions split out from the same function with D53588. Reviewers: sebpop, hiraditya Subscribers: llvm-commits, erik.pilkington Differential Revision: https://reviews.llvm.org/D53534 llvm-svn: 345178
* [InstCombine] try harder to form select from logic ops (2nd try)Sanjay Patel2018-10-242-30/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The original patch was committed here: rL344609 ...and reverted: rL344612 ...because it did not properly check/test data types before calling ComputeNumSignBits(). The tests that caused bot failures for the previous commit are over-reaching front-end tests that run the entire -O optimizer pipeline: Clang :: CodeGen/builtins-systemz-zvector.c Clang :: CodeGen/builtins-systemz-zvector2.c I've added a negative test here to ensure coverage for that case. The new early exit check also tests the type of the 'B' parameter, so we don't waste time on matching if either value is unsuitable. Original commit message: This is part of solving PR37549: https://bugs.llvm.org/show_bug.cgi?id=37549 The patterns shown here are a special case of something that we already convert to select. Using ComputeNumSignBits() catches that case (but not the more complicated motivating patterns yet). The backend has hooks/logic to convert back to logic ops if that's better for the target. llvm-svn: 345149
* [FPEnv] Convert more BinaryOperator::isFNeg(...) to m_FNeg(...)Cameron McInally2018-10-241-10/+7
| | | | | | | | This work is to avoid regressions when we seperate FNeg from the FSub IR instruction. Differential Revision: https://reviews.llvm.org/D53205 llvm-svn: 345146
* Revert r345114Gil Rapaport2018-10-241-40/+12
| | | | | | Investigating fails. llvm-svn: 345123
* [LV] Don't have fold-tail under optsize invalidate interleave-groups whenDorit Nuzman2018-10-241-1/+7
| | | | | | | | | | | | | | | | | | | | masked-interleaving is enabled Enable interleave-groups under fold-tail scenario for Opt for size compilation; D50480 added support for vectorizing loops of arbitrary trip-count without a remiander, which in turn makes everything in the loop conditional, including interleave-groups if any. It therefore invalidated all interleave-groups because we didn't have support for vectorizing predicated interleaved-groups at the time. In the meantime, D53011 introduced this support, so we don't have to invalidate interleave-groups when masked-interleaved support is enabled. Reviewers: Ayal, hsaito, dcaballe, fhahn Reviewed By: hsaito Differential Revision: https://reviews.llvm.org/D53559 llvm-svn: 345115
* [LSR] Combine unfolded offset into invariant registerGil Rapaport2018-10-241-12/+40
| | | | | | | | | | | | LSR reassociates constants as unfolded offsets when the constants fit as immediate add operands, which currently prevents such constants from being combined later with loop invariant registers. This patch modifies GenerateCombinations() to generate a second formula which includes the unfolded offset in the combined loop-invariant register. Differential Revision: https://reviews.llvm.org/D51861 llvm-svn: 345114
* [PM] keeping history when original SCC split and then merge into itselfWei Mi2018-10-231-2/+11
| | | | | | | | | | | | | | | | | | | | in the same round of SCC update. In https://reviews.llvm.org/rL309784, inline history is added to prevent infinite inlining across multiple run of inliner and SCC update, but the history will only be kept when new SCC is actually generated during SCC update. We found a case that SCC can be split and then merge into itself in the same round of SCC update, so the same SCC will be pop out from UR.CWorklist and then added back immediately, without any new SCC generated, that is why the existing patch cannot catch the infinite inline case. What the patch does is even if no new SCC is generated, if only the current SCC appears in UR.CWorklist again, then keep the inline history. Differential Revision: https://reviews.llvm.org/D52915 llvm-svn: 345103
* [HotColdSplitting] Attach MinSize to outlined codeVedant Kumar2018-10-231-0/+7
| | | | | | | | | | | | | | | | | | Outlined code is cold by assumption, so it makes sense to optimize it for minimal code size rather than performance. After r344869 moved the splitting pass to the end of the IR pipeline, this does not result in much of a code size reduction. This is probably because a comparatively small number backend transforms make use of the MinSize hint. Running LNT on x86_64, I see that 33/1020 binaries shrink for a total of 919 bytes of TEXT reduction. I didn't measure a significant performance impact. Differential Revision: https://reviews.llvm.org/D53518 llvm-svn: 345072
* [InstCombine] use 'match' to simplify codeSanjay Patel2018-10-231-1/+1
| | | | | | | | | | | | There's probably some vector-with-undef-element pattern that shows an improvement, so this is probably not quite 'NFC'. This is the last step towards removing the fake binop queries for not/neg. Ie, there are no more uses of those functions in trunk. Fneg should follow. llvm-svn: 345050
* [DebugInfo][GlobalOpt] Fix -debugify for globalopt shrinking globals to ↵Jordan Rupprecht2018-10-231-3/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | booleans. Summary: TryToShrinkGlobalToBoolean, when possible, will split store <value> + load <value> into store <bool> + select <bool ? value : 0>. This preserves DebugLoc during that pass. Fixes PR37959. The test case here is the simplified .ll for: ``` static int foo; int bar() { foo = 5; return foo; } ``` Reviewers: dblaikie, gbedwell, aprantl Reviewed By: dblaikie Subscribers: mehdi_amini, JDevlieghere, dexonsmith, llvm-commits Tags: #debug-info Differential Revision: https://reviews.llvm.org/D53531 llvm-svn: 345046
* [Reassociate] replace fake binop queries with 'match' APISanjay Patel2018-10-231-18/+16
| | | | | | | | | | | | | | | | | We need to update this code before introducing an 'fneg' instruction in IR, so we might as well kill off the integer neg/not queries too. This is no-functional-change-intended for scalar code and most vector code. For vectors, we can see that the 'match' API allows for undef elements in constants, so we optimize those cases better. Ideally, there would be a test for each code diff, but I don't see evidence of that for the existing code, so I didn't try very hard to come up with new vector tests for each code change. Differential Revision: https://reviews.llvm.org/D53533 llvm-svn: 345042
* [SLPVectorizer] Add basic support for mul/and/or/xor horizontal reductionsSimon Pilgrim2018-10-231-2/+5
| | | | | | | | | | | | Expand arithmetic reduction to include mul/and/or/xor instructions. This patch just fixes the SLPVectorizer - the effective reduction costs for AVX1+ are still poor (see rL344846) and will need to be improved before SLP sees this as a valid transform - but we can already see the effect on SSE2 tests. This partially helps PR37731, but doesn't fix it all as it still falls over on the extraction/reduction order for some reason. Differential Revision: https://reviews.llvm.org/D53473 llvm-svn: 345037
* [InstCombine] use 'match' to handle vectors and simplify codeSanjay Patel2018-10-231-2/+3
| | | | | | | This is another step towards completely removing the fake binop queries for not/neg/fneg. llvm-svn: 345036
* [InstCombine] swap select profile metadata when swapping select opsSanjay Patel2018-10-231-0/+1
| | | | llvm-svn: 345034
* [SLSR] use 'match' to simplify code; NFCSanjay Patel2018-10-231-4/+5
| | | | | | | | | This pass could probably be modified slightly to allow vector splat transforms for practically no cost, but it only works on scalars for now. So the use of the newer 'match' API should make no functional difference. llvm-svn: 345030
* Leftover bits from https://reviews.llvm.org/D53420 that were accidentally leftDorit Nuzman2018-10-231-2/+1
| | | | | | out of revision 344883 llvm-svn: 345021
* [hwasan] add stack frame descriptions.Kostya Serebryany2018-10-231-1/+75
| | | | | | | | | | | | | | | | | | | Summary: At compile-time, create an array of {PC,HumanReadableStackFrameDescription} for every function that has an instrumented frame, and pass this array to the run-time at the module-init time. Similar to how we handle pc-table in SanitizerCoverage. The run-time is dummy, will add the actual logic in later commits. Reviewers: morehouse, eugenis Reviewed By: eugenis Subscribers: srhines, llvm-commits, kubamracek Differential Revision: https://reviews.llvm.org/D53227 llvm-svn: 344985
* [Reassociate] add 'using namespace' to reduce bloat; NFCSanjay Patel2018-10-221-3/+4
| | | | llvm-svn: 344959
* [hot-cold-split] Add opt remark on successTeresa Johnson2018-10-221-0/+8
| | | | | | | | | | | | Summary: Emit optimization remark on successful hot cold split. Reviewers: sebpop, hiraditya Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D53512 llvm-svn: 344938
* [CGProfile] Turn constant-size SmallVector into arrayBenjamin Kramer2018-10-221-5/+4
| | | | | | No functionality change. llvm-svn: 344893
* [IAI,LV] Avoid creating a scalar epilogue due to gaps in interleave-groups when Dorit Nuzman2018-10-221-2/+8
| | | | | | | | | | | | | | | | | | | | | | | optimizing for size LV is careful to respect -Os and not to create a scalar epilog in all cases (runtime tests, trip-counts that require a remainder loop) except for peeling due to gaps in interleave-groups. This patch fixes that; -Os will now have us invalidate such interleave-groups and vectorize without an epilog. The patch also removes a related FIXME comment that is now obsolete, and was also inaccurate: "FIXME: return None if loop requiresScalarEpilog(<MaxVF>), or look for a smaller MaxVF that does not require a scalar epilog." (requiresScalarEpilog() has nothing to do with VF). Reviewers: Ayal, hsaito, dcaballe, fhahn Reviewed By: Ayal Differential Revision: https://reviews.llvm.org/D53420 llvm-svn: 344883
* Schedule Hot Cold Splitting pass after most optimization passesAditya Kumar2018-10-211-3/+3
| | | | | | | | | | | | | | | | Summary: In the new+old pass manager, hot cold splitting was schedule too early. Thanks to Vedant for pointing this out. Reviewers: sebpop, vsk Reviewed By: sebpop, vsk Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D53437 llvm-svn: 344869
* [InstCombine] use 'match' to simplify code; NFCSanjay Patel2018-10-201-59/+56
| | | | llvm-svn: 344855
* [InstCombine] make code more flexible with lambda; NFCSanjay Patel2018-10-201-4/+10
| | | | | | | | | | | | | | | | I couldn't tell from svn history when these checks were added, but it pre-dates the split of instcombine into its own directory at rL92459. The motivation for changing the check is partly shown by the code in PR34724: https://bugs.llvm.org/show_bug.cgi?id=34724 There are also existing regression tests for SLPVectorizer with sequences of extract+insert that are likely assumed to become shuffles by the vectorizer cost models. llvm-svn: 344854
* [InstCombine] add explanatory comment for strange vector logic; NFCSanjay Patel2018-10-201-0/+16
| | | | llvm-svn: 344852
* [NFC][InstCombine] Undo stray changeEvandro Menezes2018-10-191-2/+2
| | | | | | Undo stray change introduced by r344725. llvm-svn: 344814
* [InstCombine] InstCombine and InstSimplify for minimum and maximumThomas Lively2018-10-191-5/+22
| | | | | | | | | | | | Summary: Depends on D52765 Reviewers: aheejin, dschuff Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D52766 llvm-svn: 344799
* [InstCombine] use m_Neg() in dyn_castNegVal() to match vectors with undef eltsSanjay Patel2018-10-191-2/+3
| | | | llvm-svn: 344793
* Use llvm::{all,any,none}_of instead std::{all,any,none}_of. NFCFangrui Song2018-10-192-17/+13
| | | | llvm-svn: 344774
* [LV] Fold tail by masking to vectorize loops of arbitrary trip count under ↵Ayal Zaks2018-10-184-38/+188
| | | | | | | | | | | | | | | | | | | | | | | | opt for size When optimizing for size, a loop is vectorized only if the resulting vector loop completely replaces the original scalar loop. This holds if no runtime guards are needed, if the original trip-count TC does not overflow, and if TC is a known constant that is a multiple of the VF. The last two TC-related conditions can be overcome by 1. rounding the trip-count of the vector loop up from TC to a multiple of VF; 2. masking the vector body under a newly introduced "if (i <= TC-1)" condition. The patch allows loops with arbitrary trip counts to be vectorized under -Os, subject to the existing cost model considerations. It also applies to loops with small trip counts (under -O2) which are currently handled as if under -Os. The patch does not handle loops with reductions, live-outs, or w/o a primary induction variable, and disallows interleave groups. (Third, final and main part of -) Differential Revision: https://reviews.llvm.org/D50480 llvm-svn: 344743
* Add a emitUnaryFloatFnCall version that fetches the function name from TLIMikael Holmen2018-10-183-11/+65
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: In several places in the code we use the following pattern: if (hasUnaryFloatFn(&TLI, Ty, LibFunc_tan, LibFunc_tanf, LibFunc_tanl)) { [...] Value *Res = emitUnaryFloatFnCall(X, TLI.getName(LibFunc_tan), B, Attrs); [...] } In short, we check if there is a lib-function for a certain type, and then we _always_ fetch the name of the "double" version of the lib function and construct a call to the appropriate function, that we just checked exists, using that "double" name as a basis. This is of course a problem in cases where the target doesn't support the "double" version, but e.g. only the "float" version. In that case TLI.getName(LibFunc_tan) returns "", and emitUnaryFloatFnCall happily appends an "f" to "", and we erroneously end up with a call to a function called "f". To solve this, the above pattern is changed to if (hasUnaryFloatFn(&TLI, Ty, LibFunc_tan, LibFunc_tanf, LibFunc_tanl)) { [...] Value *Res = emitUnaryFloatFnCall(X, &TLI, LibFunc_tan, LibFunc_tanf, LibFunc_tanl, B, Attrs); [...] } I.e instead of first fetching the name of the "double" version and then letting emitUnaryFloatFnCall() add the final "f" or "l", we let emitUnaryFloatFnCall() fetch the right name from TLI. Reviewers: eli.friedman, efriedma Reviewed By: efriedma Subscribers: efriedma, bjope, llvm-commits Differential Revision: https://reviews.llvm.org/D53370 llvm-svn: 344725
* [TI removal] Switch simple loop unswitch to `Instruction`.Chandler Carruth2018-10-181-5/+5
| | | | llvm-svn: 344719
* [TI removal] Switch NewGVN to directly use `Instruction`.Chandler Carruth2018-10-181-3/+3
| | | | llvm-svn: 344718
* [TI removal] Use `Instruction` instead of `TerminatorInst` forChandler Carruth2018-10-181-2/+2
| | | | | | a variable's type. llvm-svn: 344717
* [TI removal] Update CodeExtractor to use Instruction directly.Chandler Carruth2018-10-181-4/+4
| | | | llvm-svn: 344716
* [TI removal] Switch ObjCARC code to directly use the nice range-basedChandler Carruth2018-10-182-16/+9
| | | | | | | successors API or directly build the iterators out of the terminator instruction and avoid requiring a TerminatorInst variable. llvm-svn: 344715
* [TI removal] Switch MergeFunctions to directly use Instruction API.Chandler Carruth2018-10-181-1/+1
| | | | llvm-svn: 344714
* StructurizeCFG: Simplify inserted PHI nodesNicolai Haehnle2018-10-171-1/+23
| | | | | | | | | | | | | | | Summary: This improves subsequent divergence analysis in some cases. Change-Id: I5e95e7ec7fd3fa80d414d1a53a02fea23e3d67d3 Reviewers: arsenm, rampitec Subscribers: jvesely, wdng, llvm-commits Differential Revision: https://reviews.llvm.org/D53316 llvm-svn: 344697
* [LoopPredication] add some simple statsFedor Sergeev2018-10-171-0/+8
| | | | | | | Just adding some useful statistics to LoopPredication pass which was lacking any of these. llvm-svn: 344681
* [Sanitizer][PassManager] Fix for failing ASan tests on arm-linux-gnueabihfLeonard Chan2018-10-171-1/+3
| | | | | | | | Forgot to initialize the legacy pass in it's constructor. Differential Revision: https://reviews.llvm.org/D53350 llvm-svn: 344659
* [ThinLTO] Add importing stats to thin linkTeresa Johnson2018-10-161-5/+27
| | | | | | | | | | | | | | | Summary: Previously we could only get the number of imported functions and variables from the backend. This adds stats to the thin link where the importing is decided. Reviewers: wmi Subscribers: inglorion, dexonsmith, llvm-commits Differential Revision: https://reviews.llvm.org/D53337 llvm-svn: 344658
* [SanitizerCoverage] Don't duplicate code to get section pointersJonathan Metzman2018-10-161-33/+15
| | | | | | | | | | | | | | | | | Summary: Merge code used to get section start and section end pointers for SanitizerCoverage constructors. This includes code that handles getting the start pointers when targeting MSVC. Reviewers: kcc, morehouse Reviewed By: morehouse Subscribers: kcc, hiraditya Differential Revision: https://reviews.llvm.org/D53211 llvm-svn: 344657
OpenPOWER on IntegriCloud