summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms
Commit message (Collapse)AuthorAgeFilesLines
* [SLP] Vectorize jumbled memory loads.Mohammad Shahid2017-09-201-83/+182
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch tries to vectorize loads of consecutive memory accesses, accessed in non-consecutive or jumbled way. An earlier attempt was made with patch D26905 which was reverted back due to some basic issue with representing the 'use mask' of jumbled accesses. This patch fixes the mask representation by recording the 'use mask' in the usertree entry. Change-Id: I9fe7f5045f065d84c126fa307ef6ebe0787296df Reviewers: mkuper, loladiro, Ayal, zvi, danielcdh Reviewed By: Ayal Subscribers: mzolotukhin Differential Revision: https://reviews.llvm.org/D36130 Commit after rebase for patch D36130 Change-Id: I8add1c265455669ef288d880f870a9522c8c08ab llvm-svn: 313736
* Tighten the invariants around LoopBase::invalidateSanjoy Das2017-09-202-26/+25
| | | | | | | | | | | | | | | | | Summary: With this change: - Methods in LoopBase trip an assert if the receiver has been invalidated - LoopBase::clear frees up the memory held the LoopBase instance This change also shuffles things around as necessary to work with this stricter invariant. Reviewers: chandlerc Subscribers: mehdi_amini, mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D38055 llvm-svn: 313708
* GVNSink: Make ModelledPHIs constructor linear (and avoid edge case it ↵Daniel Berlin2017-09-201-7/+8
| | | | | | worries about) by avoiding getIncomingValueForBlock llvm-svn: 313702
* Revert "[GVNSink] Remove dependency on SmallPtrSet iteration order."Daniel Berlin2017-09-201-2/+0
| | | | | | This reverts commit r312156, because now the op and block arrays are not in the same order :(. llvm-svn: 313701
* NewGVN: Remove unused includesDaniel Berlin2017-09-201-21/+0
| | | | llvm-svn: 313700
* [LoopInfo] Make LoopBase and Loop destructors non-publicSanjoy Das2017-09-193-4/+4
| | | | | | | | | | | | | | | | | | | Summary: See comment for why I think this is a good idea. This change also: - Removes an SCEV test case. The SCEV test was not testing anything useful (most of it was `#if 0` ed out) and it would need to be updated to deal with a private ~Loop::Loop. - Updates the loop pass manager test case to deal with a private ~Loop::Loop. - Renames markAsRemoved to markAsErased to contrast with removeLoop, via the usual remove vs. erase idiom we already have for instructions and basic blocks. Reviewers: chandlerc Subscribers: mehdi_amini, mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D37996 llvm-svn: 313695
* Allow ORE.emit to take a closure to delay building the remark objectAdam Nemet2017-09-193-16/+31
| | | | | | | | | | | In the lambda we are now returning the remark by value so we need to preserve its type in the insertion operator. This requires making the insertion operator generic. I've also converted a few cases to use the new API. It seems to work pretty well. See the LoopUnroller for a slightly more interesting case. llvm-svn: 313691
* Import all inlined indirect call targets for SamplePGO.Dehao Chen2017-09-191-3/+5
| | | | | | | | | | | | | | Summary: In the ThinLTO compilation, if a function is inlined in the profiling binary, we need to inline it before annotation. If the callee is not available in the primary module, a first step is needed to import that callee function. For the current implementation, if the call is an indirect call, which has been promoted to >1 targets and inlined, SamplePGO will only import one target with the largest sample count. This patch fixed the bug to import all targets instead. Reviewers: tejohnson, davidxl Reviewed By: tejohnson Subscribers: sanjoy, llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D36637 llvm-svn: 313678
* [SimplifyCFG] fix typos/formatting; NFCSanjay Patel2017-09-191-24/+22
| | | | llvm-svn: 313671
* Handle profile mismatch correctly for SamplePGO.Dehao Chen2017-09-191-1/+6
| | | | | | | | | | | | | | Summary: Fix the bug when promoted call return type mismatches with the promoted function, we should not try to inline it. Otherwise it may lead to compiler crash. Reviewers: davidxl, tejohnson, eraman Reviewed By: tejohnson Subscribers: llvm-commits, sanjoy Differential Revision: https://reviews.llvm.org/D38018 llvm-svn: 313658
* [gcov] Emit errors when opening the notes file failsReid Kleckner2017-09-181-0/+6
| | | | | | | | No time to write a test case, on to the next bug. =P Discovered while investigating PR34659 llvm-svn: 313571
* [SLP] clean up for vector store case; NFCI Sanjay Patel2017-09-181-12/+11
| | | | llvm-svn: 313541
* [X86] Remove VPERM2F128/VPERM2I128 intrinsics and autoupgrade to native ↵Craig Topper2017-09-161-74/+0
| | | | | | | | shuffles. I've moved the test cases from the InstCombine optimizations to the backend to keep the coverage we had there. It covered every possible immediate so I've preserved the resulting shuffle mask for each of those immediates. llvm-svn: 313450
* [SLP] Revert r312791 and other necessary commits, except for TTI andChandler Carruth2017-09-151-245/+49
| | | | | | | | | | | | | | | | | | | | | | | | | | CostModel. The original patch added support for horizontal min/max reductions to the SLP vectorizer. This patch causes LLVM to miscompile fairly simple signed min reductions. I have attached a test progrom to http://llvm.org/PR34635 that shows the behavior change after this patch. We found this in a test for the open source Eigen library, but also in other code. Unfortunately, the revert is moderately challenging. It required reverting: r313042: [SLP] Test with multiple uses of conditional op and wrong parent. r312853: [SLP] Fix buildbots, NFC. r312793: [SLP] Fix the warning about paths not returning the value, NFC. r312791: [SLP] Support for horizontal min/max reduction. And even then, I had to completely skip reverting the changes to TTI and CostModel because r312832 rewrote so much of this code. Plus, the cost modeling changes aren implicated in the miscompile, so they should be fine and will just not be used until this gets re-introduced. llvm-svn: 313409
* This patch fixes https://bugs.llvm.org/show_bug.cgi?id=32352 Vivek Pandya2017-09-152-11/+13
| | | | | | | | | | | It enables OptimizationRemarkEmitter::allowExtraAnalysis and MachineOptimizationRemarkEmitter::allowExtraAnalysis to return true not only for -fsave-optimization-record but when specific remarks are requested with command line options. The diagnostic handler used to be callback now this patch adds a class DiagnosticHandler. It has virtual method to provide custom diagnostic handler and methods to control which particular remarks are enabled. However LLVM-C API users can still provide callback function for diagnostic handler. llvm-svn: 313390
* This reverts r313381Vivek Pandya2017-09-152-13/+11
| | | | llvm-svn: 313387
* This patch fixes https://bugs.llvm.org/show_bug.cgi?id=32352 Vivek Pandya2017-09-152-11/+13
| | | | | | | | | | | It enables OptimizationRemarkEmitter::allowExtraAnalysis and MachineOptimizationRemarkEmitter::allowExtraAnalysis to return true not only for -fsave-optimization-record but when specific remarks are requested with command line options. The diagnostic handler used to be callback now this patch adds a class DiagnosticHandler. It has virtual method to provide custom diagnostic handler and methods to control which particular remarks are enabled. However LLVM-C API users can still provide callback function for diagnostic handler. llvm-svn: 313382
* [RuntimeUnroll] Add heuristic for unrolling multi-exit loopAnna Thomas2017-09-151-2/+34
| | | | | | | | | | | | | | | | | | | | | Add a profitability heuristic to enable runtime unrolling of multi-exit loop: There can be atmost two unique exit blocks for the loop and the second exit block should be a deoptimizing block. Also, there can be one other exiting block other than the latch exiting block. The reason for the latter is so that we limit the number of branches in the unrolled code to being at most the unroll factor. Deoptimizing blocks are rarely taken so these additional number of branches created due to the unrolling are predictable, since one of their target is the deopt block. Reviewers: apilipenko, reames, evstupac, mkuper Subscribers: llvm-commits Reviewed by: reames Differential Revision: https://reviews.llvm.org/D35380 llvm-svn: 313363
* [RuntimeUnrolling] Populate the VMap entry correctly when default generated ↵Anna Thomas2017-09-151-3/+7
| | | | | | | | | | | | | | | | | through lookup During runtime unrolling on loops with multiple exits, we update the exit blocks with the correct phi values from both original and remainder loop. In this process, we lookup the VMap for the mapped incoming phi values, but did not update the VMap if a default entry was generated in the VMap during the lookup. This default value is generated when constants or values outside the current loop are looked up. This patch fixes the assertion failure when null entries are present in the VMap because of this lookup. Added a testcase that showcases the problem. llvm-svn: 313358
* Revert "[SLPVectorizer] Failure to beneficially vectorize 'copyable' ↵Ilya Biryukov2017-09-151-317/+142
| | | | | | | | | elements in integer binary ops." This reverts commit r313348. Reason: it caused buildbot failures. llvm-svn: 313352
* [SLPVectorizer] Failure to beneficially vectorize 'copyable' elements in ↵Dinar Temirbulatov2017-09-151-142/+317
| | | | | | | | | | | | | | | | | | | | | | integer binary ops. Patch tries to improve vectorization of the following code: void add1(int * __restrict dst, const int * __restrict src) { *dst++ = *src++; *dst++ = *src++ + 1; *dst++ = *src++ + 2; *dst++ = *src++ + 3; } Allows to vectorize even if the very first operation is not a binary add, but just a load. Reviewers: spatel, mzolotukhin, mkuper, hfinkel, RKSimon, filcab, ABataev, davide Subscribers: llvm-commits, RKSimon Differential Revision: https://reviews.llvm.org/D28907 llvm-svn: 313348
* [SLPVectorizer] Remove duplicated functionality code in initScheduleData ↵Dinar Temirbulatov2017-09-151-6/+0
| | | | | | function, NFCI. llvm-svn: 313341
* Refactor collectChildrenInLoop to LoopUtils [NFC]Alina Sbirlea2017-09-152-23/+21
| | | | | | | | | | | | Summary: Move to LoopUtils method that collects all children of a node inside a loop. Reviewers: majnemer, sanjoy Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D37870 llvm-svn: 313322
* Invoke GetInlineCost for legality check before inline functions in ↵Dehao Chen2017-09-141-6/+37
| | | | | | | | | | | | | | | | SampleProfileLoader. Summary: SampleProfileLoader inlines hot functions if it is inlined in the profiled binary. However, the inline needs to be guarded by legality check, otherwise it could lead to correctness issues. Reviewers: eraman, davidxl Reviewed By: eraman Subscribers: vitalybuka, sanjoy, llvm-commits Differential Revision: https://reviews.llvm.org/D37779 llvm-svn: 313277
* [LV] Fix maximum legal VF calculationAlon Kom2017-09-141-28/+18
| | | | | | | | | | | | | | | | This patch fixes pr34283, which exposed that the computation of maximum legal width for vectorization was wrong, because it relied on MaxInterleaveFactor to obtain the maximum stride used in the loop, however not all strided accesses in the loop have an interleave-group associated with them. Instead of recording the maximum stride in the loop, which can be over conservative (e.g. if the access with the maximum stride is not involved in the dependence limitation), this patch tracks the actual maximum legal width imposed by accesses that are involved in dependencies. Differential Revision: https://reviews.llvm.org/D37507 llvm-svn: 313237
* Revert "Invoke GetInlineCost for legality check before inline functions in ↵Vitaly Buka2017-09-141-37/+6
| | | | | | | | | | SampleProfileLoader." Patch introduced uninitialized value. This reverts commit r313195. llvm-svn: 313230
* Reland r313157, "ThinLTO: Correctly follow aliasee references when dead ↵Peter Collingbourne2017-09-142-17/+10
| | | | | | | | | | | stripping." which was reverted in r313222. This reland includes a fix for the LowerTypeTests pass so that it looks past aliases when determining which type identifiers are live. Differential Revision: https://reviews.llvm.org/D37842 llvm-svn: 313229
* [SLPVectorizer] Prefer auto over explicit type for VL0, NFCI.Dinar Temirbulatov2017-09-141-1/+1
| | | | llvm-svn: 313228
* Revert r313157 "ThinLTO: Correctly follow aliasee references when dead ↵Hans Wennborg2017-09-141-5/+8
| | | | | | | | | | | | | | | | | | | | | | stripping." This broke Chromium's CFI build; see crbug.com/765004. > We were previously handling aliases during dead stripping by adding > the aliased global's "original name" GUID to the worklist. This will > lead to incorrect behaviour if the global has local linkage because > the original name GUID will not correspond to the global's GUID in > the summary. > > Because an alias is just another name for the global that it > references, there is no need to mark the referenced global as used, > or to follow references from any other copies of the global. So all > we need to do is to follow references from the aliasee's summary > instead of the alias. > > Differential Revision: https://reviews.llvm.org/D37789 llvm-svn: 313222
* [Transforms] Fix some Clang-tidy modernize-use-using and Include What You ↵Eugene Zelenko2017-09-133-63/+133
| | | | | | Use warnings; other minor fixes (NFC). llvm-svn: 313198
* Invoke GetInlineCost for legality check before inline functions in ↵Dehao Chen2017-09-131-6/+37
| | | | | | | | | | | | | | | | SampleProfileLoader. Summary: SampleProfileLoader inlines hot functions if it is inlined in the profiled binary. However, the inline needs to be guarded by legality check, otherwise it could lead to correctness issues. Reviewers: eraman, davidxl Reviewed By: eraman Subscribers: sanjoy, llvm-commits Differential Revision: https://reviews.llvm.org/D37779 llvm-svn: 313195
* [LV] Avoid computing the register usage for default VF. NFCAnna Thomas2017-09-131-2/+4
| | | | | | | | | | | These are changes to reduce redundant computations when calculating a feasible vectorization factor: 1. early return when target has no vector registers 2. don't compute register usage for the default VF. Suggested during review for D37702. llvm-svn: 313176
* Add options to dump PGO counts in text.Hiroshi Yamauchi2017-09-131-20/+38
| | | | | | | | | | | | | | | | | Summary: Added text options to -pgo-view-counts and -pgo-view-raw-counts that dump block frequency and branch probability info in text. This is useful when the graph is very large and complex (the dot command crashes, lines/edges too close to tell apart, hard to navigate without textual search) or simply when text is preferred. Reviewers: davidxl Reviewed By: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D37776 llvm-svn: 313159
* ThinLTO: Correctly follow aliasee references when dead stripping.Peter Collingbourne2017-09-131-8/+5
| | | | | | | | | | | | | | | | | | We were previously handling aliases during dead stripping by adding the aliased global's "original name" GUID to the worklist. This will lead to incorrect behaviour if the global has local linkage because the original name GUID will not correspond to the global's GUID in the summary. Because an alias is just another name for the global that it references, there is no need to mark the referenced global as used, or to follow references from any other copies of the global. So all we need to do is to follow references from the aliasee's summary instead of the alias. Differential Revision: https://reviews.llvm.org/D37789 llvm-svn: 313157
* [ThinLTO] For SamplePGO, need to handle ICP targets consistently in thin linkTeresa Johnson2017-09-131-11/+29
| | | | | | | | | | | | | | | | | | | | | | Summary: SamplePGO indirect call profiles record the target as the original GUID for statics. The importer had special handling to map to the normal GUID in that case. The dead global analysis needs the same treatment or inconsistencies arise, resulting in linker unsats due to some dead symbols being exported and kept, leaving in references to other dead symbols that are removed. This can happen when a SamplePGO profile collected by one binary is used for a different binary, so the indirect call profiles may not accurately reflect live targets. Reviewers: danielcdh Subscribers: mehdi_amini, inglorion, llvm-commits, eraman Differential Revision: https://reviews.llvm.org/D37783 llvm-svn: 313151
* [LV] Fix PR34523 - avoid generating redundant selectsAyal Zaks2017-09-131-3/+3
| | | | | | | | | | | | | | | | | When converting a PHI into a series of 'select' instructions to combine the incoming values together according their edge masks, initialize the first value to the incoming value In0 of the first predecessor, instead of generating a redundant assignment 'select(Cond[0], In0, In0)'. The latter fails when the Cond[0] mask is null, representing a full mask, which can happen only when there's a single incoming value. No functional changes intended nor expected other than surviving null Cond[0]'s. This fix follows D35725, which introduced using null to represent full masks. Differential Revision: https://reviews.llvm.org/D37619 llvm-svn: 313119
* [GVNHoist] Factor out reachability to search for anticipable instructions ↵Aditya Kumar2017-09-131-288/+418
| | | | | | | | | | | | | | | | | | | | | | | quickly Factor out the reachability such that multiple queries to find reachability of values are fast. This is based on finding the ANTIC points in the CFG which do not change during hoisting. The ANTIC points are basically the dominance-frontiers in the inverse graph. So we introduce a data structure (CHI nodes) to keep track of values flowing out of a basic block. We only do this for values with multiple occurrences in the function as they are the potential hoistable candidates. This patch allows us to hoist instructions to a basic block with >2 successors, as well as deal with infinite loops in a trivial way. Relevant test cases are added to show the functionality as well as regression fixes from PR32821. Regression from previous GVNHoist: We do not hoist fully redundant expressions because fully redundant expressions are already handled by NewGVN Differential Revision: https://reviews.llvm.org/D35918 Reviewers: dberlin, sebpop, gberry, llvm-svn: 313116
* [InstCombine] Add a flag to disable LowerDbgDeclareReid Kleckner2017-09-131-1/+30
| | | | | | | | | | | | | | | | | | | | | | | | Summary: This should improve optimized debug info for address-taken variables at the cost of inaccurate debug info in some situations. We patched this into clang and deployed this change to Chromium developers, and this significantly improved debuggability of optimized code. The long-term solution to PR34136 seems more and more like it's going to take a while, so I would like to commit this change under a flag so that it can be used as a stop-gap measure. This flag should really help so for C++ aggregates like std::string and std::vector, which are typically address-taken, even after inlining, and cannot be SROA-ed. Reviewers: aprantl, dblaikie, probinson, dberlin Subscribers: hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D36596 llvm-svn: 313108
* Refactor the code to pass down ACT to SampleProfileLoader correctly.Dehao Chen2017-09-121-13/+23
| | | | | | | | | | | | | | Summary: This change passes down ACT to SampleProfileLoader for the new PM. Also remove the default value for SampleProfileLoader class as it is not used. Reviewers: eraman, davidxl Reviewed By: eraman Subscribers: sanjoy, llvm-commits Differential Revision: https://reviews.llvm.org/D37773 llvm-svn: 313080
* Make promoteLoopAccessesToScalars independent of AliasSet [NFC]Alina Sbirlea2017-09-121-47/+52
| | | | | | | | | | | | | | | | | | | | | | | Summary: The current promoteLoopAccessesToScalars method receives an AliasSet, but the information used is in fact a list of Value*, known to must alias. Create the list ahead of time to make this method independent of the AliasSet class. While there is no functionality change, this adds overhead for creating a set of Value*, when promotion would normally exit earlier. This is meant to be as a first refactoring step in order to start replacing AliasSetTracker with MemorySSA. And while the end goal is to redesign LICM, the first few steps will focus on adding MemorySSA as an alternative to the AliasSetTracker using most of the existing functionality. Reviewers: mkuper, danielcdh, dberlin Subscribers: sanjoy, chandlerc, gberry, davide, llvm-commits Differential Revision: https://reviews.llvm.org/D35439 llvm-svn: 313075
* [LV] Clamp the VF to the trip countAnna Thomas2017-09-121-7/+12
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: When the MaxVectorSize > ConstantTripCount, we should just clamp the vectorization factor to be the ConstantTripCount. This vectorizes loops where the TinyTripCountThreshold >= TripCount < MaxVF. Earlier we were finding the maximum vector width, which could be greater than the trip count itself. The Loop vectorizer does all the work for generating a vectorizable loop, but in the end we would always choose the scalar loop (since the VF > trip count). This allows us to choose the VF keeping in mind the trip count if available. This is a fix on top of rL312472. Reviewers: Ayal, zvi, hfinkel, dneilson Reviewed by: Ayal Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D37702 llvm-svn: 313046
* [SLP] Fix for PHINode during horizontal reduction scanning, NFC.Alexey Bataev2017-09-121-1/+1
| | | | | | Reduces number of loops during instructions analysis. llvm-svn: 313035
* LowerTypeTests: Add import/export support for targets without absolute ↵Peter Collingbourne2017-09-111-26/+68
| | | | | | | | | | symbol constants. The rationale is the same as for r312967. Differential Revision: https://reviews.llvm.org/D37408 llvm-svn: 312968
* WholeProgramDevirt: Add import/export support for targets without absolute ↵Peter Collingbourne2017-09-111-16/+57
| | | | | | | | | | | | | | | | | | | symbol constants. Not all targets support the use of absolute symbols to export constants. In particular, ARM has a wide variety of constant encodings that cannot currently be relocated by linkers. So instead of exporting the constants using symbols, export them directly in the summary. The values of the constants are left as zeroes on targets that support symbolic exports. This may result in more cache misses when targeting those architectures as a result of arbitrary changes in constant values, but this seems somewhat unavoidable for now. Differential Revision: https://reviews.llvm.org/D37407 llvm-svn: 312967
* Test commitUriel Korach2017-09-101-1/+1
| | | | llvm-svn: 312878
* Merge isKnownNonNull into isKnownNonZeroNuno Lopes2017-09-093-10/+13
| | | | | | | | | It now knows the tricks of both functions. Also, fix a bug that considered allocas of non-zero address space to be always non null Differential Revision: https://reviews.llvm.org/D37628 llvm-svn: 312869
* [DivRempairs] add a pass to optimize div/rem pairs (PR31028)Sanjay Patel2017-09-094-0/+213
| | | | | | | | | | | | | | | | | | This is intended to be a superset of the functionality from D31037 (EarlyCSE) but implemented as an independent pass, so there's no stretching of scope and feature creep for an existing pass. I also proposed a weaker version of this for SimplifyCFG in D30910. And I initially had almost this same functionality as an addition to CGP in the motivating example of PR31028: https://bugs.llvm.org/show_bug.cgi?id=31028 The advantage of positioning this ahead of SimplifyCFG in the pass pipeline is that it can allow more flattening. But it needs to be after passes (InstCombine) that could sink a div/rem and undo the hoisting that is done here. Decomposing remainder may allow removing some code from the backend (PPC and possibly others). Differential Revision: https://reviews.llvm.org/D37121 llvm-svn: 312862
* [sanitizer-coverage] call appendToUsed once per module, not once per ↵Kostya Serebryany2017-09-091-8/+8
| | | | | | function (which is too slow) llvm-svn: 312855
* [SLP] Fix buildbots, NFC.Alexey Bataev2017-09-091-2/+2
| | | | llvm-svn: 312853
* [SLPVectorizer] Add struct InstructionsState that holds information about ↵Dinar Temirbulatov2017-09-081-88/+120
| | | | | | | | | | | | analysis of vector to be vectorized. Reviewers: spatel, mzolotukhin, mkuper, hfinkel, RKSimon, filcab, ABataev, davide Subscribers: llvm-commits, rengolin Differential Revision: https://reviews.llvm.org/D37212 llvm-svn: 312802
OpenPOWER on IntegriCloud