summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/IPO/PartialInlining.cpp
Commit message (Collapse)AuthorAgeFilesLines
* [NFC] Fix trivial typos in commentsJames Henderson2020-01-061-1/+1
| | | | | | | | Reviewed By: jhenderson Differential Revision: https://reviews.llvm.org/D72143 Patch by Kazuaki Ishizaki.
* Sink all InitializePasses.h includesReid Kleckner2019-11-131-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This file lists every pass in LLVM, and is included by Pass.h, which is very popular. Every time we add, remove, or rename a pass in LLVM, it caused lots of recompilation. I found this fact by looking at this table, which is sorted by the number of times a file was changed over the last 100,000 git commits multiplied by the number of object files that depend on it in the current checkout: recompiles touches affected_files header 342380 95 3604 llvm/include/llvm/ADT/STLExtras.h 314730 234 1345 llvm/include/llvm/InitializePasses.h 307036 118 2602 llvm/include/llvm/ADT/APInt.h 213049 59 3611 llvm/include/llvm/Support/MathExtras.h 170422 47 3626 llvm/include/llvm/Support/Compiler.h 162225 45 3605 llvm/include/llvm/ADT/Optional.h 158319 63 2513 llvm/include/llvm/ADT/Triple.h 140322 39 3598 llvm/include/llvm/ADT/StringRef.h 137647 59 2333 llvm/include/llvm/Support/Error.h 131619 73 1803 llvm/include/llvm/Support/FileSystem.h Before this change, touching InitializePasses.h would cause 1345 files to recompile. After this change, touching it only causes 550 compiles in an incremental rebuild. Reviewers: bkramer, asbirlea, bollu, jdoerfert Differential Revision: https://reviews.llvm.org/D70211
* [CodeExtractor] Factor out and reuse shrinkwrap analysisVedant Kumar2019-10-081-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Factor out CodeExtractor's analysis of allocas (for shrinkwrapping purposes), and allow the analysis to be reused. This resolves a quadratic compile-time bug observed when compiling AMDGPUDisassembler.cpp.o. Pre-patch (Release + LTO clang): ``` ---User Time--- --System Time-- --User+System-- ---Wall Time--- --- Name --- 176.5278 ( 57.8%) 0.4915 ( 18.5%) 177.0192 ( 57.4%) 177.4112 ( 57.3%) Hot Cold Splitting ``` Post-patch (ReleaseAsserts clang): ``` ---User Time--- --System Time-- --User+System-- ---Wall Time--- --- Name --- 1.4051 ( 3.3%) 0.0079 ( 0.3%) 1.4129 ( 3.2%) 1.4129 ( 3.2%) Hot Cold Splitting ``` Testing: check-llvm, and comparing the AMDGPUDisassembler.cpp.o binary pre- vs. post-patch. An alternate approach is to hide CodeExtractorAnalysisCache from clients of CodeExtractor, and to recompute the analysis from scratch inside of CodeExtractor::extractCodeRegion(). This eliminates some redundant work in the shrinkwrapping legality check. However, some clients continue to exhibit O(n^2) compile time behavior as computing the analysis is O(n). rdar://55912966 Differential Revision: https://reviews.llvm.org/D68616 llvm-svn: 374089
* Second attempt to add iterator_range::empty()Jordan Rose2019-10-071-2/+2
| | | | | | | | | | | | Doing this makes MSVC complain that `empty(someRange)` could refer to either C++17's std::empty or LLVM's llvm::empty, which previously we avoided via SFINAE because std::empty is defined in terms of an empty member rather than begin and end. So, switch callers over to the new method as it is added. https://reviews.llvm.org/D68439 llvm-svn: 373935
* [llvm] Migrate llvm::make_unique to std::make_uniqueJonas Devlieghere2019-08-151-4/+4
| | | | | | | | Now that we've moved to C++14, we no longer need the llvm::make_unique implementation from STLExtras.h. This patch is a mechanical replacement of (hopefully) all the llvm::make_unique instances across the monorepo. llvm-svn: 369013
* [CallSite removal] move InlineCost to CallBase usageFedor Sergeev2019-04-231-6/+7
| | | | | | | | | | | Converting InlineCost interface and its internals into CallBase usage. Inliners themselves are still not converted. Reviewed By: reames Tags: #llvm Differential Revision: https://reviews.llvm.org/D60636 llvm-svn: 358982
* [Inliner] Pass nullptr for the ORE param of getInlineCost if RemarkEnabledWei Mi2019-02-211-2/+6
| | | | | | | | | | | | | | | | | | | is false. Right now for inliner and partial inliner, we always pass the address of a valid ORE object to getInlineCost even if RemarkEnabled is false because of no -Rpass is specified. Since ComputeFullInlineCost will be set to true if ORE is non-null in getInlineCost, this introduces the problem that in getInlineCost we cannot return early even if we already know the cost is definitely higher than the threshold. It is a general problem for compile time. This patch fixes that by pass nullptr as the ORE argument if RemarkEnabled is false. Differential Revision: https://reviews.llvm.org/D58399 llvm-svn: 354542
* [NFC] Avoid passing blocks vector to the OutlineRegionInfo constructor by value.Sergey Dmitriev2019-02-081-4/+3
| | | | | | | | | | | | | | Reviewers: vsk, fhahn, davidxl Reviewed By: vsk Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D57957 llvm-svn: 353582
* [CodeExtractor] Update function's assumption cache after extracting blocks ↵Sergey Dmitriev2019-02-081-13/+33
| | | | | | | | | | | | | | | | | | from it Summary: Assumption cache's self-updating mechanism does not correctly handle the case when blocks are extracted from the function by the CodeExtractor. As a result function's assumption cache may have stale references to the llvm.assume calls that were moved to the outlined function. This patch fixes this problem by removing extracted llvm.assume calls from the function’s assumption cache. Reviewers: hfinkel, vsk, fhahn, davidxl, sanjoy Reviewed By: hfinkel, vsk Subscribers: llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D57215 llvm-svn: 353500
* Update the file headers across all of the LLVM projects in the monorepoChandler Carruth2019-01-191-4/+3
| | | | | | | | | | | | | | | | | to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
* [IR] Add Instruction::isLifetimeStartOrEnd, NFCVedant Kumar2018-12-211-6/+2
| | | | | | | | | | | Instruction::isLifetimeStartOrEnd() checks whether an Instruction is an llvm.lifetime.start or an llvm.lifetime.end intrinsic. This was suggested as a cleanup in D55967. Differential Revision: https://reviews.llvm.org/D56019 llvm-svn: 349964
* [PartialInliner] Make PHIs free in cost computation.Florian Hahn2018-11-271-10/+9
| | | | | | | | | | | | | | | | InlineCost also treats them as free and the current implementation can cause assertion failures if PHI nodes are moved outside the region from entry BBs to the region. It also updates the code to use the instructionsWithoutDebug iterator. Reviewers: davidxl, davide, vsk, graham-yiu-huawei Reviewed By: davidxl Differential Revision: https://reviews.llvm.org/D54748 llvm-svn: 347683
* [IR] Add hasNPredecessors, hasNPredecessorsOrMore to BasicBlockVedant Kumar2018-11-191-1/+1
| | | | | | | | | | | | | | | | | | | | | | | Add methods to BasicBlock which make it easier to efficiently check whether a block has N (or more) predecessors. This can be more efficient than using pred_size(), which is a linear time operation. We might consider adding similar methods for successors. I haven't done so in this patch because succ_size() is already O(1). With this patch applied, I measured a 0.065% compile-time reduction in user time for running `opt -O3` on the sqlite3 amalgamation (30 trials). The change in mergeStoreIntoSuccessor alone saves 45 million linked list iterations in a stage2 Release build of llc. See llvm.org/PR39702 for a harder but more general way of achieving similar results. Differential Revision: https://reviews.llvm.org/D54686 llvm-svn: 347256
* [ProfileSummary] Standardize methods and fix commentVedant Kumar2018-11-191-2/+2
| | | | | | | | | | | | | | | | | | | | | Every Analysis pass has a get method that returns a reference of the Result of the Analysis, for example, BlockFrequencyInfo &BlockFrequencyInfoWrapperPass::getBFI(). I believe that ProfileSummaryInfo::getPSI() is the only exception to that, as it was returning a pointer. Another change is renaming isHotBB and isColdBB to isHotBlock and isColdBlock, respectively. Most methods use BB as the argument of variable names while methods usually refer to Basic Blocks as Blocks, instead of BB. For example, Function::getEntryBlock, Loop:getExitBlock, etc. I also fixed one of the comments. Patch by Rodrigo Caetano Rocha! Differential Revision: https://reviews.llvm.org/D54669 llvm-svn: 347182
* ADT/STLExtras: Introduce llvm::empty; NFCMatthias Braun2018-10-311-2/+2
| | | | | | | | This is modeled after C++17 std::empty(). Differential Revision: https://reviews.llvm.org/D53909 llvm-svn: 345679
* [TI removal] Make variables declared as `TerminatorInst` and initializedChandler Carruth2018-10-151-1/+1
| | | | | | | | | | | | | by `getTerminator()` calls instead be declared as `Instruction`. This is the biggest remaining chunk of the usage of `getTerminator()` that insists on the narrow type and so is an easy batch of updates. Several files saw more extensive updates where this would cascade to requiring API updates within the file to use `Instruction` instead of `TerminatorInst`. All of these were trivial in nature (pervasively using `Instruction` instead just worked). llvm-svn: 344502
* [Partial Inlining] Fix small bug in detecting if we did somethingGraham Yiu2018-08-031-3/+1
| | | | | | | - It's possible for 'Changed' to return as false even if we did partial inline something. Fixed to accumulate return values llvm-svn: 338896
* [STLExtras] Add distance() for ranges, pred_size(), and succ_size()Vedant Kumar2018-05-101-8/+3
| | | | | | | | | | | This commit adds a wrapper for std::distance() which works with ranges. As it would be a common case to write `distance(predecessors(BB))`, this also introduces `pred_size()` and `succ_size()` helpers to make that easier to write. Differential Revision: https://reviews.llvm.org/D46668 llvm-svn: 332057
* [PartialInlining] Fix Crash from holding a reference to a destructed ORE.Sean Fertile2018-04-201-37/+20
| | | | | | | | | | | | | The callback used to create an ORE for the legacy PI pass caches the allocated object in a unique_ptr in the runOnModule function, and returns a reference to that object. Under certian circumstances we can end up holding onto that reference after the OREs destruction. Rather then allowing the new and legacy passes to create ORE object in diffrent ways, create the ORE at the point of use. Differential Revision: https://reviews.llvm.org/D43219 llvm-svn: 330473
* [PartialInlining] Use isInlineViable to detect constructs preventing inlining.Florian Hahn2018-03-101-3/+3
| | | | | | | | | | | | | Use isInlineViable to prevent inlining of functions with non-inlinable constructs, in case cost analysis is skipped. Reviewers: efriedma, sfertile, davide, davidxl Reviewed By: efriedma Differential Revision: https://reviews.llvm.org/D42846 llvm-svn: 327207
* [NFC] fix trivial typos in commentsHiroshi Inoue2018-01-191-1/+1
| | | | | | "the the" -> "the" llvm-svn: 322934
* Add a ProfileCount class to represent entry counts.Easwaran Raman2018-01-171-2/+4
| | | | | | | | | | | | | | Summary: The class wraps a uint64_t and an enum to represent the type of profile count (real and synthetic) with some helper methods. Reviewers: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41883 llvm-svn: 322771
* Add hasProfileData() to check if a function has profile data. NFC.Easwaran Raman2017-12-221-4/+4
| | | | | | | | | | | | | | | | | | | Summary: This replaces calls to getEntryCount().hasValue() with hasProfileData that does the same thing. This refactoring is useful to do before adding synthetic function entry counts but also a useful cleanup IMO even otherwise. I have used hasProfileData instead of hasRealProfileData as David had earlier suggested since I think profile implies "real" and I use the phrase "synthetic entry count" and not "synthetic profile count" but I am fine calling it hasRealProfileData if you prefer. Reviewers: davidxl, silvas Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41461 llvm-svn: 321331
* Silence a bunch of implicit fallthrough warningsAdrian Prantl2017-12-191-0/+1
| | | | llvm-svn: 321114
* Removed unused DominanceFrontierMatt Arsenault2017-12-181-3/+0
| | | | llvm-svn: 321001
* - Removed unused lamba (IsReturnBlock) causing build bots to fail for r319398Graham Yiu2017-11-301-7/+1
| | | | | | - Added lit testcases that were supposed to be part of r319398 llvm-svn: 319399
* With PGO information, we can do more aggressive outlining of cold regions in ↵Graham Yiu2017-11-301-88/+579
| | | | | | | | | | | | the inline candidate function. This contrasts with the scheme of keeping only the 'early return' portion of the inline candidate and outlining the rest of the function as a single function call. Support for outlining multiple regions of each function is added, as well as some basic heuristics to determine which regions are good to outline. Outline candidates limited to regions that are single-entry & single-exit. We also avoid outlining regions that produce live-exit variables, which may inhibit some forms of code motion (like commoning). Fallback to the regular partial inlining scheme is retained when either i) no regions are identified for outlining in the function, or ii) the outlined function could not be inlined in any of its callers. Differential Revision: https://reviews.llvm.org/D38190 llvm-svn: 319398
* [PartialInliner] Inline vararg functions that forward varargs.Florian Hahn2017-11-131-3/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch extends the partial inliner to support inlining parts of vararg functions, if the vararg handling is done in the outlined part. It adds a `ForwardVarArgsTo` argument to InlineFunction. If it is non-null, all varargs passed to the inlined function will be added to all calls to `ForwardVarArgsTo`. The partial inliner takes care to only pass `ForwardVarArgsTo` if the varargs handing is done in the outlined function. It checks that vastart is not part of the function to be inlined. `test/Transforms/CodeExtractor/PartialInlineNoInline.ll` (already part of the repo) checks we do not do partial inlining if vastart is used in a basic block that will be inlined. Reviewers: davide, davidxl, grosser Reviewed By: davide, davidxl, grosser Subscribers: gyiu, grosser, eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D39607 llvm-svn: 318028
* [PartialInliner] Skip call sites where inlining fails.Florian Hahn2017-11-031-7/+9
| | | | | | | | | | | | | | | | | | Summary: InlineFunction can fail, for example when trying to inline vararg fuctions. In those cases, we do not want to bump partial inlining counters or set AnyInlined to true, because this could leave an unused function hanging around. Reviewers: davidxl, davide, gyiu Reviewed By: davide Subscribers: llvm-commits, eraman Differential Revision: https://reviews.llvm.org/D39581 llvm-svn: 317314
* [NFC] Convert OptimizationRemarkEmitter old emit() calls to new closureVivek Pandya2017-10-111-22/+37
| | | | | | | | | | | | | | parameterized emit() calls Summary: This is not functional change to adopt new emit() API added in r313691. Reviewed By: anemet Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D38285 llvm-svn: 315476
* [Transforms] Fix some Clang-tidy modernize and Include What You Use ↵Eugene Zelenko2017-10-101-15/+46
| | | | | | warnings; other minor fixes (NFC). llvm-svn: 315383
* Rename OptimizationDiagnosticInfo.* to OptimizationRemarkEmitter.*Adam Nemet2017-10-091-1/+1
| | | | | | | Sync it up with the name of the class actually defined here. This has been bothering me for a while... llvm-svn: 315249
* Sink some IntrinsicInst.h and Intrinsics.h out of llvm/includeReid Kleckner2017-09-071-0/+1
| | | | | | | Many of these uses can get by with forward declarations. Hopefully this speeds up compilation after adding a single intrinsic. llvm-svn: 312759
* [PartialInlining] Formatting. NFC.Chad Rosier2017-08-241-1/+1
| | | | llvm-svn: 311702
* [PartialInlining] Type. NFC.Chad Rosier2017-08-241-1/+1
| | | | llvm-svn: 311699
* [InlineCost] Add cl::opt to allow full inline cost to be computed for ↵Haicheng Wu2017-08-211-1/+1
| | | | | | | | | | | | | | | debugging purposes. Currently, the inline cost model will bail once the inline cost exceeds the inline threshold in order to avoid unnecessary compile-time. However, when debugging it is useful to compute the full cost, so this command line option is added to override the default behavior. I took over this work from Chad Rosier (mcrosier@codeaurora.org). Differential Revision: https://reviews.llvm.org/D35850 llvm-svn: 311371
* Fix function name /NFCXinliang David Li2017-06-161-3/+3
| | | | llvm-svn: 305564
* [PartialInlining] Code RefactoringXinliang David Li2017-06-151-191/+223
| | | | | | | This is a NFC code refactoring and interface cleanup. This paves the way to enable outlining-only mode for the partial inliner. llvm-svn: 305530
* Fix debug build test failureXinliang David Li2017-06-021-2/+3
| | | | llvm-svn: 304600
* [PartialInlining] Minor cost anaysis tuningXinliang David Li2017-06-021-9/+56
| | | | | | Also added a test option and 2 cost analysis related tests. llvm-svn: 304599
* [PartialInlining] Reduce outlining overhead by removing unneeded live-out(s)Xinliang David Li2017-06-011-0/+23
| | | | | | Differential Revision: http://reviews.llvm.org/D33694 llvm-svn: 304375
* [PartialInlining] Replace delete with unique_ptr in ↵Vitaly Buka2017-05-271-7/+3
| | | | | | | | | | | | | | computeCallsiteToProfCountMap Reviewers: davidxl Reviewed By: davidxl Subscribers: vsk, llvm-commits Differential Revision: https://reviews.llvm.org/D33220 llvm-svn: 304064
* Fix memory leakXinliang David Li2017-05-151-0/+4
| | | | llvm-svn: 303126
* Fix test failure on windows -- do not return deleted funcXinliang David Li2017-05-141-2/+8
| | | | llvm-svn: 302999
* [PartialInlining] Profile based cost analysisXinliang David Li2017-05-121-45/+363
| | | | | | | | | | | | Implemented frequency based cost/saving analysis and related options. The pass is now in a state ready to be turne on in the pipeline (in follow up). Differential Revision: http://reviews.llvm.org/D32783 llvm-svn: 302967
* [ProfileSummary] Make getProfileCount a non-static member function.Easwaran Raman2017-05-091-1/+1
| | | | | | | | | | This change is required because the notion of count is different for sample profiling and getProfileCount will need to determine the underlying profile type. Differential revision: https://reviews.llvm.org/D33012 llvm-svn: 302597
* [PartialInlining] Capture by reference rather than by value.Davide Italiano2017-05-081-3/+3
| | | | llvm-svn: 302464
* [PartialInlining] Add more early filteringXinliang David Li2017-05-021-0/+10
| | | | | | | This is a follow up to the previous inline cost patch for quicker filtering. llvm-svn: 301959
* [PartialInlining] Hook up inline cost analysisXinliang David Li2017-05-021-11/+98
| | | | | | Differential Revision: http://reviews.llvm.org/D32666 llvm-svn: 301894
* [PartialInlining]: Improve partial inlining to handle complex conditionsXinliang David Li2017-04-271-41/+243
| | | | | | Differential Revision: http://reviews.llvm.org/D32249 llvm-svn: 301561
OpenPOWER on IntegriCloud