summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
* [LoadCombine] Fix combining of loads which span an aliasing store.Michael J. Spencer2017-02-091-1/+5
| | | | | | | | Fixes PR31517 Differential Revision: https://reviews.llvm.org/D28922 llvm-svn: 294632
* Rename LowerTypeTestsSummaryAction to PassSummaryAction. NFCI.Peter Collingbourne2017-02-092-22/+19
| | | | | | | | | I intend to use the same type with the same semantics in the WholeProgramDevirt pass. Differential Revision: https://reviews.llvm.org/D29746 llvm-svn: 294629
* [InstCombine] use m_APInt to allow demanded bits analysis on splat constantsSanjay Patel2017-02-091-10/+13
| | | | llvm-svn: 294628
* [JumpThreading] Thread through guardsSanjoy Das2017-02-092-15/+189
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch allows JumpThreading also thread through guards. Virtually, guard(cond) is equivalent to the following construction: if (cond) { do something } else {deoptimize} Yet it is not explicitly converted into IFs before lowering. This patch enables early threading through guards in simple cases. Currently it covers the following situation: if (cond1) { // code A } else { // code B } // code C guard(cond2) // code D If there is implication cond1 => cond2 or !cond1 => cond2, we can transform this construction into the following: if (cond1) { // code A // code C } else { // code B // code C guard(cond2) } // code D Thus, removing the guard from one of execution branches. Patch by Max Kazantsev! Reviewers: reames, apilipenko, igor-laevsky, anna, sanjoy Reviewed By: sanjoy Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29620 llvm-svn: 294617
* [sancov] using comdat only when it is enabledMike Aizatsky2017-02-081-3/+7
| | | | | | Differential Revision: https://reviews.llvm.org/D29733 llvm-svn: 294529
* [sancov] specifying comdat for sancov constructorsMike Aizatsky2017-02-081-1/+3
| | | | | | Differential Revision: https://reviews.llvm.org/D29662 llvm-svn: 294517
* ThinLTOBitcodeWriter: Strip debug info from merged module.Peter Collingbourne2017-02-081-0/+2
| | | | | | | | | | This module will contain nothing but vtable definitions and (soon) available_externally function definitions, so there is no point in keeping debug info in the module. Differential Revision: https://reviews.llvm.org/D28913 llvm-svn: 294511
* [Loop Vectorizer] Cost-based decision for vectorization form of memory ↵Elena Demikhovsky2017-02-081-269/+485
| | | | | | | | | | | | | | | | | | instruction. Making the cost model selecting between Interleave, GatherScatter or Scalar vectorization form of memory instruction. The right decision should be done for non-consecutive memory access instrcuctions that may have more than one vectorization solution. This patch includes the following changes: - Cost Model calculates the cost of Load/Store vector form and choose the better option between Widening, Interleave, GatherScactter and Scalarization. Cost Model keeps the widening decision. - Arrays of Uniform and Scalar values are moved from Legality to Cost Model. - Cost Model collects Uniforms and Scalars per VF. The collection is based on CM decision map of Loadis/Stores vectorization form. - Vectorization of memory instruction is performed according to the CM decision. Differential Revision: https://reviews.llvm.org/D27919 llvm-svn: 294503
* NVPTX: Extract mem intrinsic expansions into utilitiesMatt Arsenault2017-02-082-0/+232
| | | | llvm-svn: 294490
* [Reassociate] Remove an unused argument. NFC.Chad Rosier2017-02-081-5/+4
| | | | llvm-svn: 294489
* [InstCombine] add local name for repeated calls; NFC Sanjay Patel2017-02-081-6/+4
| | | | llvm-svn: 294470
* [InstComobineCalls] Fix buildbot failures after r294453.Igor Laevsky2017-02-081-1/+1
| | | | | | | | Some targets don't support uint64_t options. Change type to unsigned. Differential Revision: https://reviews.llvm.org/D28909 llvm-svn: 294461
* [InstCombineCalls] Unfold element atomic memcpy instructionIgor Laevsky2017-02-082-0/+83
| | | | | | Differential Revision: https://reviews.llvm.org/D28909 llvm-svn: 294453
* [InstCombineCalls] Remove zero length atomic memcpy intrinsicsIgor Laevsky2017-02-081-0/+6
| | | | | | Differential Revision: https://reviews.llvm.org/D28909 llvm-svn: 294452
* LSR: Check atomic instruction pointer operandsMatt Arsenault2017-02-081-1/+11
| | | | llvm-svn: 294410
* Revert "CVP: Make CVP iterate in an order that maximizes reuse of LVI cache"Daniel Berlin2017-02-081-5/+6
| | | | | | This reverts commit r294398, it seems to be failing on the bots. llvm-svn: 294399
* CVP: Make CVP iterate in an order that maximizes reuse of LVI cacheDaniel Berlin2017-02-081-6/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: After the DFS order change for LVI, i have a few testcases that now take forever. The TL;DR - This is mainly due to the overdefined cache, but that requires predicateinfo to fix[1] In order to maximize reuse of the LVI cache for now, change the order we iterate in. This reduces my testcase from 5 minutes to 4 seconds. I have verified cases like gmic do not get slower. I am playing with whether the order should be postorder or idf. [1] In practice, overdefined anywhere should be overdefined everywhere, so this cache should be global. That also fixes this bug. The problem, however, is that LVI relies on this cache being filled in per-block because it wants different values in different blocks due to precisely the naming issue that predicateinfo fixes. With predicateinfo, making the cache global works fine on individual passes, and also resolves this issue. Reviewers: davide, sanjoy, chandlerc Subscribers: llvm-commits, djasper Differential Revision: https://reviews.llvm.org/D29679 llvm-svn: 294398
* [IRCE] Add a missing invariant checkSanjoy Das2017-02-071-5/+39
| | | | | | | | | | | | | | | | | | | | | | | Currently IRCE relies on the loops it transforms to be (semantically) of the form: for (i = START; i < END; i++) ... or for (i = START; i > END; i--) ... However, we were not verifying the presence of the START < END entry check (i.e. check before the first iteration). We were only verifying that the backedge was guarded by (i + 1) < END. Usually this would work "fine" since (especially in Java) most loops do actually have the START < END check, but of course that is not guaranteed. llvm-svn: 294375
* PredicateInfo: Some compilers are unhappy with naming Use *'s Use. Change ↵Daniel Berlin2017-02-071-13/+13
| | | | | | the name. llvm-svn: 294364
* Add PredicateInfo utility and printing passDaniel Berlin2017-02-073-0/+642
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch adds a utility to build extended SSA (see "ABCD: eliminating array bounds checks on demand"), and an intrinsic to support it. This is then used to get functionality equivalent to propagateEquality in GVN, in NewGVN (without having to replace instructions as we go). It would work similarly in SCCP or other passes. This has been talked about a few times, so i built a real implementation and tried to productionize it. Copies are inserted for operands used in assumes and conditional branches that are based on comparisons (see below for more) Every use affected by the predicate is renamed to the appropriate intrinsic result. E.g. %cmp = icmp eq i32 %x, 50 br i1 %cmp, label %true, label %false true: ret i32 %x false: ret i32 1 will become %cmp = icmp eq i32, %x, 50 br i1 %cmp, label %true, label %false true: ; Has predicate info ; branch predicate info { TrueEdge: 1 Comparison: %cmp = icmp eq i32 %x, 50 } %x.0 = call @llvm.ssa_copy.i32(i32 %x) ret i32 %x.0 false: ret i23 1 (you can use -print-predicateinfo to get an annotated-with-predicateinfo dump) This enables us to easily determine what operations are affected by a given predicate, and how operations affected by a chain of predicates. Reviewers: davide, sanjoy Subscribers: mgorny, llvm-commits, Prazek Differential Revision: https://reviews.llvm.org/D29519 Update for review comments Fix a bug Nuno noticed where we are giving information about and/or on edges where the info is not useful and easy to use wrong Update for review comments llvm-svn: 294351
* Fix the -Werror build for some sign-comparisonsDavid Blaikie2017-02-071-1/+1
| | | | llvm-svn: 294331
* [InstCombine] Make max size array combine a tunable.Davide Italiano2017-02-074-3/+13
| | | | | | | Requested by Sanjoy/Hal a while ago, and forgotten by me (r283612). llvm-svn: 294323
* Revert "[GVNHoist] Merge DebugLoc metadata on hoisted instructions"Reid Kleckner2017-02-071-7/+0
| | | | | | | | | This reverts commit r294250. It caused PR31891. Add a test case that shows that inlinable calls retain location information with an accurate scope. llvm-svn: 294317
* LowerTypeTests: Simplify. NFC.Peter Collingbourne2017-02-071-5/+3
| | | | llvm-svn: 294273
* Fix the samplepgo indirect call promotion bug: we should not promote a ↵Dehao Chen2017-02-061-1/+2
| | | | | | | | | | | | | | | | direct call. Summary: Checking CS.getCalledFunction() == nullptr does not necessary indicate indirect call. We also need to check if CS.getCalledValue() is not a constant. Reviewers: davidxl Reviewed By: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29570 llvm-svn: 294260
* Merge DebugLoc on combined stores; in this case, when combining storesPaul Robinson2017-02-061-1/+4
| | | | | | | | from the end of two blocks, merge instead of arbitrarily picking one. Differential Revision: http://reviews.llvm.org/D29504 llvm-svn: 294251
* [GVNHoist] Merge DebugLoc metadata on hoisted instructionsTaewook Oh2017-02-061-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: When instructions are hoisted, current implementation keeps DebugLoc metadata of the instruction that chosen as Repl (and its GEP operand if Repl is a load or a store). However, DebugLoc metadata should be updated to the 'merged' location across all hoisted instructions. See the following example code: ``` 1: typedef struct { 2: int a[10]; 3: } S1; 4: 5: extern S1 *s1[10]; 6: 7: void foo(int x, int y, int i) { 8: if (y) 9: s1[i]->a[i] = x + y; 10: else 11: s1[i]->a[i] = x; 12: } ``` Below is LLVM IR representation of the program before gvn-hoist: ``` %struct.S1 = type { [10 x i32] } @s1 = external local_unnamed_addr global [10 x %struct.S1*], align 16 define void @foo(i32 %x, i32 %y, i32 %i) !dbg !4 { entry: %tobool = icmp ne i32 %y, 0, !dbg !8 br i1 %tobool, label %if.then, label %if.else, !dbg !10 if.then: ; preds = %entry %add = add nsw i32 %x, %y, !dbg !11 %idxprom = sext i32 %i to i64, !dbg !12 %arrayidx = getelementptr inbounds [10 x %struct.S1*], [10 x %struct.S1*]* @s1, i64 0, i64 %idxprom, !dbg !12 %0 = load %struct.S1*, %struct.S1** %arrayidx, align 8, !dbg !12, !tbaa !13 %a = getelementptr inbounds %struct.S1, %struct.S1* %0, i32 0, i32 0, !dbg !17 br label %if.end, !dbg !12 if.else: ; preds = %entry %idxprom3 = sext i32 %i to i64, !dbg !18 %arrayidx4 = getelementptr inbounds [10 x %struct.S1*], [10 x %struct.S1*]* @s1, i64 0, i64 %idxprom3, !dbg !18 %1 = load %struct.S1*, %struct.S1** %arrayidx4, align 8, !dbg !18, !tbaa !13 %a5 = getelementptr inbounds %struct.S1, %struct.S1* %1, i32 0, i32 0, !dbg !19 br label %if.end if.end: ; preds = %if.else, %if.then %a5.sink = phi [10 x i32]* [ %a5, %if.else ], [ %a, %if.then ] %.sink = phi i32 [ %x, %if.else ], [ %add, %if.then ] %idxprom6 = sext i32 %i to i64 %arrayidx7 = getelementptr inbounds [10 x i32], [10 x i32]* %a5.sink, i64 0, i64 %idxprom6 store i32 %.sink, i32* %arrayidx7, align 4, !tbaa !20 ret void, !dbg !22 } ``` where ``` !11 = !DILocation(line: 9, column: 18, scope: !9) !12 = !DILocation(line: 9, column: 5, scope: !9) !18 = !DILocation(line: 11, column: 5, scope: !9) !19 = !DILocation(line: 11, column: 9, scope: !9) ``` . And below is after gvn-hoist: ``` define void @foo(i32 %x, i32 %y, i32 %i) !dbg !4 { entry: %tobool = icmp ne i32 %y, 0, !dbg !8 %idxprom = sext i32 %i to i64, !dbg !10 %0 = getelementptr inbounds [10 x %struct.S1*], [10 x %struct.S1*]* @s1, i64 0, i64 %idxprom, !dbg !10 %1 = load %struct.S1*, %struct.S1** %0, align 8, !dbg !10, !tbaa !11 br i1 %tobool, label %if.then, label %if.else, !dbg !15 if.then: ; preds = %entry %add = add nsw i32 %x, %y, !dbg !16 %arrayidx = getelementptr inbounds [10 x %struct.S1*], [10 x %struct.S1*]* @s1, i64 0, i64 %idxprom, !dbg !10 %a = getelementptr inbounds %struct.S1, %struct.S1* %1, i32 0, i32 0, !dbg !17 br label %if.end, !dbg !10 if.else: ; preds = %entry %arrayidx4 = getelementptr inbounds [10 x %struct.S1*], [10 x %struct.S1*]* @s1, i64 0, i64 %idxprom, !dbg !18 %a5 = getelementptr inbounds %struct.S1, %struct.S1* %1, i32 0, i32 0, !dbg !19 br label %if.end if.end: ; preds = %if.else, %if.then %a5.sink = phi [10 x i32]* [ %a5, %if.else ], [ %a, %if.then ] %.sink = phi i32 [ %x, %if.else ], [ %add, %if.then ] %arrayidx7 = getelementptr inbounds [10 x i32], [10 x i32]* %a5.sink, i64 0, i64 %idxprom store i32 %.sink, i32* %arrayidx7, align 4, !tbaa !20 ret void, !dbg !22 } ``` As you see, loads and their GEPs have been hosited from if.then/if.else block to entry block. However, DebugLoc metadata of these new instructions are still same as the instructions in if.then block, as they are moved/cloned from if.then block. This may result incorrect stepping and imprecise sample profile result. Reviewers: majnemer, pcc, sebpop Reviewed By: sebpop Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29377 llvm-svn: 294250
* [SLP] Revert "Allow using of extra values in horizontal reductions."Michael Kuperstein2017-02-061-67/+12
| | | | | | | | This breaks when one of the extra values is also a scalar that participates in the same vectorization tree which we'll end up reducing. llvm-svn: 294245
* [ValueTracking] emit a remark when we detect a conflicting assumption (PR31809)Sanjay Patel2017-02-061-4/+12
| | | | | | | | | | | | This is a follow-up to D29395 where we try to be good citizens and let the user know that we've probably gone off the rails. This should allow us to resolve: https://llvm.org/bugs/show_bug.cgi?id=31809 Differential Revision: https://reviews.llvm.org/D29404 llvm-svn: 294208
* Fix the bug of samplepgo indirect call promption when type casting of the ↵Dehao Chen2017-02-061-1/+3
| | | | | | | | | | | | | | | | return value is needed. Summary: When type casting of the return value is needed, promoteIndirectCall will return the type casting instruction instead of the direct call. This patch changed to return the direct call instruction instead. Reviewers: davidxl Reviewed By: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29569 llvm-svn: 294205
* [InstCombine] simplify dyn_cast + isa; NFCISanjay Patel2017-02-061-6/+4
| | | | llvm-svn: 294198
* Refactor SampleProfile.cpp to make it cleaner. (NFC)Dehao Chen2017-02-051-32/+14
| | | | llvm-svn: 294118
* [IPCP] Don't propagate return value for naked functions.Davide Italiano2017-02-041-1/+7
| | | | | | This is pretty much the same change made in SCCP. llvm-svn: 294098
* Fix variable name /NFCXinliang David Li2017-02-041-2/+2
| | | | llvm-svn: 294090
* [InstCombine] treat i1 as a special type in shouldChangeType()Sanjay Patel2017-02-031-4/+8
| | | | | | | | | | | | | | | | | | | | This patch is based on the llvm-dev discussion here: http://lists.llvm.org/pipermail/llvm-dev/2017-January/109631.html Folding to i1 should always be desirable because that's better for value tracking and we have special folds for i1 types. I checked for other users of shouldChangeType() where this might have an effect, but we already handle the i1 case differently than other types in all of those cases. Side note: the default datalayout includes i1, so it seems we only find this gap in shouldChangeType + phi folding for the case when there is (1) an explicit datalayout without i1, (2) casting to i1 from a legal type, and (3) a phi with exactly 2 incoming casted operands (as Björn mentioned). Differential Revision: https://reviews.llvm.org/D29336 llvm-svn: 294066
* [PGO] Add select instr profile in graph dumpXinliang David Li2017-02-031-3/+32
| | | | | | Differential Revision: http://reviews.llvm.org/D29474 llvm-svn: 294055
* [InstCombine] fix operand-complexity-based canonicalization (PR28296)Sanjay Patel2017-02-031-7/+15
| | | | | | | | | | | | | | | | | | | The code comments didn't match the code logic, and we didn't actually distinguish the fake unary (not/neg/fneg) operators from arguments. Adding another level to the weighting scheme provides more structure and can help simplify the pattern matching in InstCombine and other places. I fixed regressions that would have shown up from this change in: rL290067 rL290127 But that doesn't mean there are no pattern-matching logic holes left; some combines may just be missing regression tests. Should fix: https://llvm.org/bugs/show_bug.cgi?id=28296 Differential Revision: https://reviews.llvm.org/D27933 llvm-svn: 294049
* [SLP] Make sortMemAccesses explicitly return an error. NFC.Michael Kuperstein2017-02-031-12/+15
| | | | llvm-svn: 294029
* NFC: [LoopUnroll] More meaningful message in tracingAnna Thomas2017-02-031-1/+1
| | | | llvm-svn: 294017
* IRMover: Merge flags LinkModuleInlineAsm and IsPerformingImport.Peter Collingbourne2017-02-031-1/+1
| | | | | | | | | Currently these flags are always the inverse of each other, so there is no need to keep them separate. Differential Revision: https://reviews.llvm.org/D29471 llvm-svn: 294016
* FunctionImport: Use IRMover directly.Peter Collingbourne2017-02-032-14/+19
| | | | | | | | | | | | The importer was previously using ModuleLinker in a sort of "IRMover mode". Use IRMover directly instead in order to remove a level of indirection. I will remove all importing support from ModuleLinker in a separate change. Differential Revision: https://reviews.llvm.org/D29468 llvm-svn: 294014
* [SLP] Fix for PR31690: Allow using of extra values in horizontal reductions.Alexey Bataev2017-02-031-12/+67
| | | | | | | | | | | | | | | | | | | | | Currently LLVM supports vectorization of horizontal reduction instructions with initial value set to 0. Patch supports vectorization of reduction with non-zero initial values. Also it supports a vectorization of instructions with some extra arguments, like: float f(float x[], int a, int b) { float p = a % b; p += x[0] + 3; for (int i = 1; i < 32; i++) p += x[i]; return p; } Patch allows vectorization of this kind of horizontal reductions. Differential Revision: https://reviews.llvm.org/D28961 llvm-svn: 293994
* Revert "[ThinLTO] Add an auto-hide feature"Mehdi Amini2017-02-031-7/+0
| | | | | | | | | This reverts commit r293970. After more discussion, this belongs to the linker side and there is no added value to do it at this level. llvm-svn: 293993
* [sanitizer coverage] Fix Instrumentation to work on Windows.Marcos Pividori2017-02-031-21/+29
| | | | | | | | | | | | | | | | | | | | | On Windows, the symbols "___stop___sancov_guards" and "___start___sancov_guards" are not defined automatically. So, we need to take a different approach. We define 3 sections: Section ".SCOV$A" will only hold a variable ___start___sancov_guard. Section ".SCOV$M" will hold the main data. Section ".SCOV$Z" will only hold a variable ___stop___sancov_guards. When linking, they will be merged sorted by the characters after the $, so we can use the pointers of the variables ___[start|stop]___sancov_guard to know the actual range of addresses of that section. In this diff, I updated instrumentation to include all the guard arrays in section ".SCOV$M". Differential Revision: https://reviews.llvm.org/D28434 llvm-svn: 293987
* [ThinLTO] Add an auto-hide featureMehdi Amini2017-02-031-0/+7
| | | | | | | | | | | | | | | When a symbol is not exported outside of the DSO, it is can be hidden. Usually we try to internalize as much as possible, but it is not always possible, for instance a symbol can be referenced outside of the LTO unit, or there can be cross-module reference in ThinLTO. This is a recommit of r293912 after fixing build failures, and a recommit of r293918 after fixing LLD tests. Differential Revision: https://reviews.llvm.org/D28978 llvm-svn: 293970
* Revert "[ThinLTO] Add an auto-hide feature"Mehdi Amini2017-02-021-7/+0
| | | | | | This reverts commit r293918, one lld test does not pass. llvm-svn: 293961
* [PGO] internal option cleanupsXinliang David Li2017-02-021-23/+37
| | | | | | | | | | 1. Added comments for options 2. Added missing option cl::desc field 3. Uniified function filter option for graph viewing. Now PGO count/raw-counts share the same filter option: -view-bfi-func-name=. llvm-svn: 293938
* [PGO] make graph view internal options available for all buildsXinliang David Li2017-02-021-2/+0
| | | | | | Differential Revision: https://reviews.llvm.org/D29259 llvm-svn: 293921
* FunctionImport: Remove the -disable-force-link-odr flag and change ↵Peter Collingbourne2017-02-021-13/+3
| | | | | | | | | | importFunctions to never force link. This removes some functionality that was only being used by tests. Differential Revision: https://reviews.llvm.org/D29439 llvm-svn: 293919
* [ThinLTO] Add an auto-hide featureMehdi Amini2017-02-021-0/+7
| | | | | | | | | | | | | | When a symbol is not exported outside of the DSO, it is can be hidden. Usually we try to internalize as much as possible, but it is not always possible, for instance a symbol can be referenced outside of the LTO unit, or there can be cross-module reference in ThinLTO. This is a recommit of r293912 after fixing build failures. Differential Revision: https://reviews.llvm.org/D28978 llvm-svn: 293918
OpenPOWER on IntegriCloud