summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
* Transforms: Introduce Transforms/Utils.h rather than spreading the ↵David Blaikie2018-03-2830-29/+43
| | | | | | | | | declarations amongst Scalar.h and IPO.h Fixes layering - Transforms/Utils shouldn't depend on including a Scalar or IPO header, because Scalar and IPO depend on Utils. llvm-svn: 328717
* [MSan] Introduce ActualFnStart. NFCAlexander Potapenko2018-03-281-8/+10
| | | | | | | | | | | | | This is a step towards the upcoming KMSAN implementation patch. KMSAN is going to prepend a special basic block containing tool-specific calls to each function. Because we still want to instrument the original entry block, we'll need to store it in ActualFnStart. For MSan this will still be F.getEntryBlock(), whereas for KMSAN it'll contain the second BB. llvm-svn: 328697
* [MSan] Add an isStore argument to getShadowOriginPtr(). NFCAlexander Potapenko2018-03-281-38/+47
| | | | | | | | | | | | | | | | This is a step towards the upcoming KMSAN implementation patch. The isStore argument is to be used by getShadowOriginPtrKernel(), it is ignored by getShadowOriginPtrUserspace(). Depending on whether a memory access is a load or a store, KMSAN instruments it with different functions, __msan_metadata_ptr_for_load_X() and __msan_metadata_ptr_for_store_X(). Those functions may return different values for a single address, which is necessary in the case the runtime library decides to ignore particular accesses. llvm-svn: 328692
* 80-line wrap. NFCXin Tong2018-03-271-1/+2
| | | | llvm-svn: 328660
* [PGO] Fix branch probability remarks assertRong Xu2018-03-271-7/+9
| | | | | | | | | Fixed counter/weight overflow that leads to an assertion. Also fixed the help string for pgo-emit-branch-prob option. Differential Revision: https://reviews.llvm.org/D44809 llvm-svn: 328653
* [LV] Add TTI::shouldMaximizeVectorBandwidth to allow enabling it per targetKrzysztof Parzyszek2018-03-271-1/+2
| | | | | | | | The default implementation returns false and keeps the current behavior. Differential Revision: https://reviews.llvm.org/D44735 llvm-svn: 328632
* [LoopUnroll][NFC] Remove redundant canPeel checkMax Kazantsev2018-03-271-2/+2
| | | | | | | | | | | | We check `canPeel` twice: when evaluating the number of iterations to be peeled and within the method `peelLoop` that performs peeling. This method is only executed if the calculated peel count is positive. Thus, the check in `peelLoop` can never fail. This patch replaces this check with an assert. Differential Revision: https://reviews.llvm.org/D44919 Reviewed By: fhahn llvm-svn: 328615
* [IRCE] Enable decreasing loops of non-const boundSam Parker2018-03-271-52/+74
| | | | | | | | | | | | | | As a follow-up to r328480, this updates the logic for the decreasing safety checks in a similar manner: - CanBeMax is replaced by CannotBeMaxInLoop which queries isLoopEntryGuardedByCond on the maximum value. - SumCanReachMin is replaced by isSafeDecreasingBound which includes some logic from parseLoopStructure and, again, has been updated to use isLoopEntryGuardedByCond on the given bounds. Differential Revision: https://reviews.llvm.org/D44776 llvm-svn: 328613
* [InstCombine] improve code comment; NFCSanjay Patel2018-03-261-2/+2
| | | | llvm-svn: 328560
* [InstCombine] reassociate loop invariant GEP chains to enable LICMSebastian Pop2018-03-261-0/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | This change brings performance of zlib up by 10%. The example below is from a hot loop in longest_match() from zlib. do.body: %cur_match.addr.0 = phi i32 [ %cur_match, %entry ], [ %2, %do.cond ] %idx.ext = zext i32 %cur_match.addr.0 to i64 %add.ptr = getelementptr inbounds i8, i8* %win, i64 %idx.ext %add.ptr2 = getelementptr inbounds i8, i8* %add.ptr, i64 %idx.ext1 %add.ptr3 = getelementptr inbounds i8, i8* %add.ptr2, i64 -1 In this example %idx.ext1 is a loop invariant. It will be moved above the use of loop induction variable %idx.ext such that it can be hoisted out of the loop by LICM. The operands that have dependences carried by the loop will be sinked down in the GEP chain. This patch will produce the following output: do.body: %cur_match.addr.0 = phi i32 [ %cur_match, %entry ], [ %2, %do.cond ] %idx.ext = zext i32 %cur_match.addr.0 to i64 %add.ptr = getelementptr inbounds i8, i8* %win, i64 %idx.ext1 %add.ptr2 = getelementptr inbounds i8, i8* %add.ptr, i64 -1 %add.ptr3 = getelementptr inbounds i8, i8* %add.ptr2, i64 %idx.ext llvm-svn: 328539
* [InstCombine] distribute fmul over fadd/fsubSanjay Patel2018-03-262-100/+15
| | | | | | | | | | This replaces a large chunk of code that was looking for compound patterns that include these sub-patterns. Existing tests ensure that all of the previous examples are still folded as expected. We still need to loosen the FMF check. llvm-svn: 328502
* [InstCombine] check uses before creating instructions for fmul distributionSanjay Patel2018-03-261-1/+1
| | | | | | As the tests show, we could create extra instructions without any obvious benefit. llvm-svn: 328498
* [LSR] Allow giving priority to post-incrementing addressing modesKrzysztof Parzyszek2018-03-261-9/+66
| | | | | | | | | | | Implement TTI interface for targets to indicate that the LSR should give priority to post-incrementing addressing modes. Combination of patches by Sebastian Pop and Brendon Cahoon. Differential Revision: https://reviews.llvm.org/D44758 llvm-svn: 328490
* [LoopUnroll] Fix dangling pointers in SCEVMax Kazantsev2018-03-261-28/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Current logic of loop SCEV invalidation in Loop Unroller implicitly relies on fact that exit count of outer loops cannot rely on exiting blocks of inner loops, which is true in current implementation of backedge taken count calculation but is wrong in general. As result, when we only forget the loop that we have just unrolled, we may still have cached data for its outer loops (in particular, exit counts) which keeps references on blocks of inner loop that could have been changed or even deleted. The attached test demonstrates a situaton when after unrolling of innermost loop the outermost loop contains a dangling pointer on non-existant block. The problem shows up when we apply patch https://reviews.llvm.org/D44677 that makes SCEV smarter about exit count calculation. I am not sure if the bug exists without this patch, it appears that now it is accidentally correct just because in practice exact backedge taken count for outer loops with complex control flow inside is never calculated. But when SCEV learns to do so, this problem shows up. This patch replaces existing logic of SCEV loop invalidation with a correct one, which happens to be invalidation of outermost loop (which also leads to invalidation of all loops inside of it). It is the only way to ensure that no outer loop keeps dangling pointers on removed blocks, or just outdated information that has changed after unrolling. Differential Revision: https://reviews.llvm.org/D44818 Reviewed By: samparker llvm-svn: 328483
* [DeadArgElim] Strip allocsize attributes when deleting an argument.Benjamin Kramer2018-03-261-2/+6
| | | | | | | Since allocsize refers to the argument number it gets invalidated when an argument is removed and the numbers shift. llvm-svn: 328481
* [IRCE] Enable increasing loops of variable boundsSam Parker2018-03-261-58/+78
| | | | | | | | | | | | | | | | | | | | | CanBeMin is currently used which will report true for any unknown values, but often a check is performed outside the loop which covers this situation: for (int i = 0; i < N; ++i) ... if (N > 0) for (int i = 0; i < N; ++i) ... So I've add 'LoopGuardedAgainstMin' which reports whether N is greater than the minimum value which then allows loop with a variable loop count to be optimised. I've also moved the increasing bound checking into its own function and replaced SumCanReachMax is another isLoopEntryGuardedByCond function. llvm-svn: 328480
* [PatternMatch] allow undef elements when matching vector FP +0.0Sanjay Patel2018-03-255-11/+11
| | | | | | | | | | | | | This continues the FP constant pattern matching improvements from: https://reviews.llvm.org/rL327627 https://reviews.llvm.org/rL327339 https://reviews.llvm.org/rL327307 Several integer constant matchers also have this ability. I'm separating matching of integer/pointer null from FP positive zero and renaming/commenting to make the functionality clearer. llvm-svn: 328461
* [InstCombine] peek through more icmp of FP cast + bitcastSanjay Patel2018-03-251-4/+14
| | | | | | This is an extension of rL328426 as noted in D44367. llvm-svn: 328448
* [InstCombine] peek through FP casts for sign-bit compares (PR36682)Sanjay Patel2018-03-241-0/+9
| | | | | | | | | | | | This pattern came up in PR36682: https://bugs.llvm.org/show_bug.cgi?id=36682 https://godbolt.org/g/LhuD9A Equality checks are planned as a follow-up enhancement. Differential Revision: https://reviews.llvm.org/D44367 llvm-svn: 328426
* [InstCombine] fix formatting; NFCSanjay Patel2018-03-241-37/+30
| | | | llvm-svn: 328425
* Remove unused header from EntryExitInstrumenterDavid Blaikie2018-03-241-1/+0
| | | | | | | Fixes layering, since Transforms/Utils doesn't depend on CodeGen, so shouldn't include headers from it. llvm-svn: 328399
* [GuardWidening] Group code by class [NFC]Philip Reames2018-03-231-39/+40
| | | | llvm-svn: 328387
* Fix Layering, move instrumentation transform headers into Instrumentation ↵David Blaikie2018-03-235-5/+5
| | | | | | subdirectory llvm-svn: 328379
* [PM][FunctionAttrs] add NoUnwind attribute inference to ↵Fedor Sergeev2018-03-231-32/+201
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | PostOrderFunctionAttrs pass Summary: This was motivated by absence of PrunEH functionality in new PM. It was decided that a proper way to do PruneEH is to add NoUnwind inference into PostOrderFunctionAttrs and then perform normal SimplifyCFG on top. This change generalizes attribute handling implemented for (a removal of) Convergent attribute, by introducing a generic builder-like class AttributeInferer It registers all the attribute inference requests, storing per-attribute predicates into a vector, and then goes through an SCC Node, scanning all the instructions for not breaking attribute assumptions. The main idea is that as soon all the instructions from all the functions of SCC Node conform to attribute assumptions then we are free to infer the attribute as set for all the functions of SCC Node. It handles two distinct cases of attributes: - those that might break due to derefinement of the function code for these attributes we are allowed to apply inference only if all the functions are "exact definitions". Example - NoUnwind. - those that do not care about derefinement for these attributes we are allowed to apply inference as soon as we see any function definition. Example - removal of Convergent attribute. Also in this commit: * Converted all the FunctionAttrs tests to use FileCheck and added new-PM invocations to them * FunctionAttrs/convergent.ll test demonstrates a difference in behavior between new and old PM implementations. Marked with FIXME. * PruneEH tests were converted to new-PM as well, using function-attrs+simplify-cfg combo as intended * some of "other" tests were updated since function-attrs now infers 'nounwind' even for old PM pipeline * -disable-nounwind-inference hidden option added as a possible workaround for a supposedly rare case when nounwind being inferred by default presents a problem Reviewers: chandlerc, jlebar Reviewed By: jlebar Subscribers: eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D44415 llvm-svn: 328377
* [InstCombine] simplify code for FP intrinsic shrinking; NFCISanjay Patel2018-03-231-10/+5
| | | | llvm-svn: 328372
* [HWASan] Port HWASan to Linux x86-64 (LLVM)Alex Shlyapnikov2018-03-231-13/+58
| | | | | | | | | | | | | | | | | | | | | Summary: Porting HWASan to Linux x86-64, first of the three patches, LLVM part. The approach is similar to ARM case, trap signal is used to communicate memory tag check failure. int3 instruction is used to generate a signal, access parameters are stored in nop [eax + offset] instruction immediately following the int3 one. One notable difference is that x86-64 has to untag the pointer before use due to the lack of feature comparable to ARM's TBI (Top Byte Ignore). Reviewers: eugenis Subscribers: kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D44699 llvm-svn: 328342
* Fix a block copying problem in LICMAndrew Kaylor2018-03-231-2/+8
| | | | | | Differential Revision: https://reviews.llvm.org/D44817 llvm-svn: 328336
* [InstCombine] reduce code duplication; NFCSanjay Patel2018-03-231-56/+49
| | | | llvm-svn: 328323
* [InstCombine] improve variable name; NFCSanjay Patel2018-03-231-12/+10
| | | | llvm-svn: 328322
* [SLP] Stop counting cost of gather sequences with multiple usesMatthew Simpson2018-03-231-1/+22
| | | | | | | | | | | | | | | When building the SLP tree, we look for reuse among the vectorized tree entries. However, each gather sequence is represented by a unique tree entry, even though the sequence may be identical to another one. This means, for example, that a gather sequence with two uses will be counted twice when computing the cost of the tree. We should only count the cost of the definition of a gather sequence rather than its uses. During code generation, the redundant gather sequences are emitted, but we optimize them away with CSE. So it looks like this problem just affects the cost model. Differential Revision: https://reviews.llvm.org/D44742 llvm-svn: 328316
* Revert r328307: [IPSCCP] Use constant range information for comparisons of ↵Florian Hahn2018-03-231-17/+50
| | | | | | | | parameters. Reverted for now, due to it causing verifier failures. llvm-svn: 328312
* [IPSCCP] Use constant range information for comparisons of parameters.Florian Hahn2018-03-231-50/+17
| | | | | | | | | | | | | | | | | | For comparisons with parameters, we can use the ParamState lattice elements which also provide constant range information. This improves the code for PR33253 further and gets us closer to use ValueLatticeElement for all values. Also, as we are using the range information in the solver directly, we do not need tryToReplaceWithConstantRange afterwards anymore. Reviewers: dberlin, mssimpso, davide, efriedma Reviewed By: mssimpso Differential Revision: https://reviews.llvm.org/D43762 llvm-svn: 328307
* [LoopUnroll] Simplify induction variables after peeling too.Florian Hahn2018-03-231-2/+3
| | | | | | | | | | | | | Loop peeling also has an impact on the induction variables, so we should benefit from induction variable simplification after peeling too. Reviewers: sanjoy, bogner, mzolotukhin, efriedma Reviewed By: efriedma Differential Revision: https://reviews.llvm.org/D43878 llvm-svn: 328301
* Move SampleProfile.h into IPO along with the rest of the IPO pass headersDavid Blaikie2018-03-221-1/+1
| | | | llvm-svn: 328262
* Finish moving the IPSCCP pass from Scalar to IPO - moving the registrationDavid Blaikie2018-03-222-1/+1
| | | | llvm-svn: 328259
* Fix layering between SCCP and IPO SCCPDavid Blaikie2018-03-223-57/+62
| | | | | | | | | | Transforms/Scalar/SCCP.cpp implemented both the Scalar and IPO SCCP, but this meant Transforms/Scalar including Transfroms/IPO headers, creating a circular dependency. (IPO depends on Scalar already) - so move the IPO SCCP shims out into IPO and the basic library implementation accessible from Scalar/SCCP.h to be used from the IPO/SCCP.cpp implementation. llvm-svn: 328250
* Move the initialization of the Meta Renamer pass over to IPO along with the ↵David Blaikie2018-03-222-1/+1
| | | | | | rest of it that was moved in r328209 llvm-svn: 328234
* [InstCombineCalls] Update deprecated API usage (NFC)Daniel Neilson2018-03-221-1/+1
| | | | | | | | Summary: Just updating a call to MemSetInst::getAlignment() to MemSetInst::getDestAlignment(). The former has been deprecated. llvm-svn: 328227
* [SimplifyCFG] Create attribute for fuzzing-specific optimizations.Matt Morehouse2018-03-223-0/+9
| | | | | | | | | | | | | | | | | | | | | | Summary: When building with libFuzzer, converting control flow to selects or obscuring the original operands of CMPs reduces the effectiveness of libFuzzer's heuristics. This patch provides an attribute to disable or modify certain optimizations for optimal fuzzing signal. Provides a less aggressive alternative to https://reviews.llvm.org/D44057. Reviewers: vitalybuka, davide, arsenm, hfinkel Reviewed By: vitalybuka Subscribers: junbuml, mehdi_amini, wdng, javed.absar, hiraditya, llvm-commits, kcc Differential Revision: https://reviews.llvm.org/D44232 llvm-svn: 328214
* [LoopPredication] Add profitability check based on BPIAnna Thomas2018-03-221-3/+92
| | | | | | | | | | | | | | | | | | | | | | | | Summary: LoopPredication is not profitable when the loop is known to always exit through some block other than the latch block. A coarse grained latch check can cause loop predication to predicate the loop, and unconditionally deoptimize. However, without predicating the loop, the guard may never fail within the loop during the dynamic execution because the non-latch loop termination condition exits the loop before the latch condition causes the loop to exit. We teach LP about this using BranchProfileInfo pass. Reviewers: apilipenko, skatkov, mkazantsev, reames Reviewed by: skatkov Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D44667 llvm-svn: 328210
* Move MetaRenamer from Transforms/UTils to Transforms/IPO since it implements ↵David Blaikie2018-03-223-1/+1
| | | | | | part of IPO.h llvm-svn: 328209
* [CallSiteSplitting] Preserve DominatorTreeAnalysis.Florian Hahn2018-03-221-13/+21
| | | | | | | | | | | | | | The dominator tree analysis can be preserved easily. Some other kinds of analysis can probably be preserved too. Reviewers: junbuml, dberlin Reviewed By: dberlin Differential Revision: https://reviews.llvm.org/D43173 llvm-svn: 328206
* [InstCombine] add folds for xor-of-icmp signbit tests (PR36682)Sanjay Patel2018-03-221-0/+28
| | | | | | | | | | | | | | | | | | This is a retry of r328119 which was reverted at r328145 because it could crash by trying to combine icmps with different operand types. This version has a check for that and additional tests. Original commit message: This is part of solving: https://bugs.llvm.org/show_bug.cgi?id=36682 There's also a leftover improvement from the long-ago-closed: https://bugs.llvm.org/show_bug.cgi?id=5438 https://rise4fun.com/Alive/dC1 llvm-svn: 328197
* [CloneFunction] Preserve DT in DuplicateInstructionsInSplitBetween.Florian Hahn2018-03-221-2/+3
| | | | | | | | | | | | | DuplicateInstructionsInSplitBetween can preserve the DT by passing through DT to SplitEdge. Reviewers: sanjoy, junbuml, anna, kuhar Reviewed By: kuhar Differential Revision: https://reviews.llvm.org/D44629 llvm-svn: 328189
* Fix a couple of layering violations in TransformsDavid Blaikie2018-03-2171-79/+75
| | | | | | | | | | | | | Remove #include of Transforms/Scalar.h from Transform/Utils to fix layering. Transforms depends on Transforms/Utils, not the other way around. So remove the header and the "createStripGCRelocatesPass" function declaration (& definition) that is unused and motivated this dependency. Move Transforms/Utils/Local.h into Analysis because it's used by Analysis/MemoryBuiltins.cpp. llvm-svn: 328165
* Revert r328119 "[InstCombine] add folds for xor-of-icmp signbit tests (PR36682)"Reid Kleckner2018-03-211-30/+0
| | | | | | | This asserts when compiling safe_numerics_unittest.cpp in Chromium with MSan. llvm-svn: 328145
* [InstCombine] add folds for xor-of-icmp signbit tests (PR36682)Sanjay Patel2018-03-211-0/+30
| | | | | | | | | | | | This is part of solving: https://bugs.llvm.org/show_bug.cgi?id=36682 There's also a leftover improvement from the long-ago-closed: https://bugs.llvm.org/show_bug.cgi?id=5438 https://rise4fun.com/Alive/dC1 llvm-svn: 328119
* [MemCpyOpt] Update to new API for memory intrinsic alignmentDaniel Neilson2018-03-211-25/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This change is part of step five in the series of changes to remove alignment argument from memcpy/memmove/memset in favour of alignment attributes. In particular, this changes the MemCpyOpt pass to cease using: 1) The old getAlignment() API of MemoryIntrinsic in favour of getting source & dest specific alignments through the new API. 2) The old IRBuilder CreateMemCpy/CreateMemMove single-alignment APIs in favour of the new API that allows setting source and destination alignments independently. We also add a few tests to fill gaps in the testing of this pass. Steps: Step 1) Remove alignment parameter and create alignment parameter attributes for memcpy/memmove/memset. ( rL322965, rC322964, rL322963 ) Step 2) Expand the IRBuilder API to allow creation of memcpy/memmove with differing source and dest alignments. ( rL323597 ) Step 3) Update Clang to use the new IRBuilder API. ( rC323617 ) Step 4) Update Polly to use the new IRBuilder API. ( rL323618 ) Step 5) Update LLVM passes that create memcpy/memmove calls to use the new IRBuilder API, and those that use use MemIntrinsicInst::[get|set]Alignment() to use [get|set]DestAlignment() and [get|set]SourceAlignment() instead. ( rL323886, rL323891, rL324148, rL324273, rL324278, rL324384, rL324395, rL324402, rL324626, rL324642, rL324653, rL324654, rL324773, rL324774, rL324781, rL324784, rL324955, rL324960, rL325816, rL327398, rL327421 ) Step 6) Remove the single-alignment IRBuilder API for memcpy/memmove, and the MemIntrinsicInst::[get|set]Alignment() methods. Reference http://lists.llvm.org/pipermail/llvm-dev/2015-August/089384.html http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20151109/312083.html llvm-svn: 328097
* Re-re-land: Teach CorrelatedValuePropagation to reduce the width of ↵Justin Lebar2018-03-211-0/+54
| | | | | | | | | | | | | | | udiv/urem instructions. Summary: If the operands of a udiv/urem can be proved to fit within a smaller power-of-two-sized type, reduce the width of the udiv/urem. Backed out for causing performance regressions. Re-landing because we've determined that these regressions were noise. Original Differential Revision: https://reviews.llvm.org/D44102 llvm-svn: 328096
* [MustExecute] Move isGuaranteedToExecute and related rourtines to AnalysisPhilip Reames2018-03-201-133/+1
| | | | | | Next step is to actually merge the implementations and get both implementations tested through the new printer. llvm-svn: 328055
OpenPOWER on IntegriCloud