summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/IPO/FunctionImport.cpp
Commit message (Collapse)AuthorAgeFilesLines
* [ThinLTO] Thin link efficiency: More efficient export list computationTeresa Johnson2016-12-161-32/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Instead of checking whether a global referenced by a function being imported is defined in the same module, speculatively always add the referenced globals to the module's export list. After all imports are computed, for each module prune any not in its defined set from its export list. For a huge C++ app with aggressive importing thresholds, even with D27687 we spent a lot of time invoking modulePath() from exportGlobalInModule (modulePath() was still the 2nd hottest routine in profile). The reason is that with comdat/linkonce the summary lists for each GUID can be long. For the app in question, for example, we were invoking exportGlobalInModule almost 2 million times, and we traversed an average of 63 entries in the summary list each time. This patch reduced the thin link time for the app by about 10% (on top of D27687) when using aggressive importing thresholds, and about 3.5% on average with default importing thresholds. Reviewers: mehdi_amini Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D27755 llvm-svn: 289918
* [ThinLTO] Thin link efficiency improvement: don't re-export globals (NFC)Teresa Johnson2016-12-151-9/+13
| | | | | | | | | | | | | | | | | Summary: We were reinvoking exportGlobalInModule numerous times redundantly. No need to re-export globals referenced by a global that was already imported from its module. This resulted in a large speedup in the thin link for a big application, particularly when importing aggressiveness was cranked up. Reviewers: mehdi_amini Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D27687 llvm-svn: 289896
* [ThinLTO] Revert part of r289843 that belonged to another patch.Teresa Johnson2016-12-151-13/+9
| | | | | | | | The code change for D27687 accidentally got committed along with the main change in r289843. Revert it temporarily, so that I can recommit it along with its test as intended. llvm-svn: 289875
* [ThinLTO] Remove stale comment (NFC)Teresa Johnson2016-12-151-2/+1
| | | | | | This should have been removed with r288446. llvm-svn: 289871
* [ThinLTO] Thin link efficiency: skip candidate added later with higher ↵Teresa Johnson2016-12-151-4/+13
| | | | | | | | | | | | | | | | | | | | | | | threshold (NFC) Summary: Thin link efficiency improvement. After adding an importing candidate to the worklist we might have later added it again with a higher threshold. Skip it when popped from the worklist if we recorded a higher threshold than the current worklist entry, it will get processed again at the higher threshold when that entry is popped. This required adding the summary's GUID to the worklist, so that it can be used to query the recorded highest threshold for it when we pop from the worklist. Reviewers: mehdi_amini Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D27696 llvm-svn: 289867
* [ThinLTO] Ensure callees get hot threshold when first seen on cold pathTeresa Johnson2016-12-151-24/+28
| | | | | | | | | | | | | | | | | This is split out from D27696, since it turned out to be a bug fix and not part of the NFC efficiency change. Keep the same adjusted (possibly decayed) threshold in both the worklist and the ImportList. Otherwise if we encountered it first along a cold path, the callee would be added to the worklist with a lower decayed threshold than when it is later encountered along a hot path. But the logic uses the threshold recorded in the ImportList entry to check if we should re-add it, and without this patch the threshold recorded there is the same along both paths so we don't re-add it. Using the same possibly decayed threshold in the ImportList ensures we re-add it later with the higher non-decayed hot path threshold. llvm-svn: 289843
* [ThinLTO] Remove useless code (NFC)Teresa Johnson2016-12-121-4/+0
| | | | | | Should have been removed in r288446. llvm-svn: 289466
* [ThinLTO] Stop importing constant global vars as copies in the backendTeresa Johnson2016-12-021-12/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: We were doing an optimization in the ThinLTO backends of importing constant unnamed_addr globals unconditionally as a local copy (regardless of whether the thin link decided to import them). This should be done in the thin link instead, so that resulting exported references are marked and promoted appropriately, but will need a summary enhancement to mark these variables as constant unnamed_addr. The function import logic during the thin link was trying to handle this proactively, by conservatively marking all values referenced in the initializer lists of exported global variables as also exported. However, this only handled values referenced directly from the initializer list of an exported global variable. If the value is itself a constant unnamed_addr variable, we could end up exporting its references as well. This caused multiple issues. The first is that the transitively exported references weren't promoted. Secondly, some could not be promoted/renamed (e.g. they had a section or other constraint). recursively, instead of just adding the first level of initializer list references to the ExportList directly. Remove this optimization and the associated handling in the function import backend. SPEC measurements indicate we weren't getting much from it in any case. Fixes PR31052. Reviewers: mehdi_amini Subscribers: krasin, llvm-commits Differential Revision: https://reviews.llvm.org/D26880 llvm-svn: 288446
* Object: Extract a ModuleSymbolTable class from IRObjectFile.Peter Collingbourne2016-12-011-1/+1
| | | | | | | | | | | | This class represents a symbol table built from in-memory IR. It provides access to GlobalValues and should only be used if such access is required (e.g. in the LTO implementation). We will eventually change IRObjectFile to read from a bitcode symbol table rather than using ModuleSymbolTable, so it would not be able to expose the module. Differential Revision: https://reviews.llvm.org/D27073 llvm-svn: 288319
* [ThinLTO] Only promote exported locals as marked in indexTeresa Johnson2016-11-141-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: We have always speculatively promoted all renamable local values (except const non-address taken variables) for both the exporting and importing module. We would then internalize them back based on the ThinLink results if they weren't actually exported. This is inefficient, and results in unnecessary renames. It also meant we had to check the non-renamability of a value in the summary, which was already checked during function importing analysis in the ThinLink. Made renameModuleForThinLTO (which does the promotion/renaming) instead use the index when exporting, to avoid unnecessary renames/promotions. For importing modules, we can simply promoted all values as any local we import by definition is exported and needs promotion. This required changes to the method used by the FunctionImport pass (only invoked from 'opt' for testing) and when invoked from llvm-link, since neither does a ThinLink. We simply conservatively mark all locals in the index as promoted, which preserves the current aggressive promotion behavior. I also needed to change an llvm-lto based test where we had previously been aggressively promoting values that weren't importable (aliasees), but now will not promote. Reviewers: mehdi_amini Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D26467 llvm-svn: 286871
* [ThinLTO] Make inline assembly handling more efficient in summaryTeresa Johnson2016-11-141-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The change in r285513 to prevent exporting of locals used in inline asm added all locals in the llvm.used set to the reference set of functions containing inline asm. Since these locals were marked NoRename, this automatically prevented importing of the function. Unfortunately, this caused an explosion in the summary reference lists in some cases. In my particular example, it happened for a large protocol buffer generated C++ file, where many of the generated functions contained an inline asm call. It was exacerbated when doing a ThinLTO PGO instrumentation build, where the PGO instrumentation included thousands of private __profd_* values that were added to llvm.used. We really only need to include a single llvm.used local (NoRename) value in the reference list of a function containing inline asm to block it being imported. However, it seems cleaner to add a flag to the summary that explicitly describes this situation, which is what this patch does. Reviewers: mehdi_amini Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D26402 llvm-svn: 286840
* Bitcode: Change module reader functions to return an llvm::Expected.Peter Collingbourne2016-11-131-1/+4
| | | | | | Differential Revision: https://reviews.llvm.org/D26562 llvm-svn: 286752
* Bitcode: Change getModuleSummaryIndex() to return an llvm::Expected.Peter Collingbourne2016-11-111-35/+6
| | | | | | Differential Revision: https://reviews.llvm.org/D26539 llvm-svn: 286624
* Bitcode: Change the materializer interface to return llvm::Error.Peter Collingbourne2016-11-091-8/+22
| | | | | | Differential Revision: https://reviews.llvm.org/D26439 llvm-svn: 286382
* [ThinLTO] Rename HasSection to NoRename (NFC)Teresa Johnson2016-10-281-2/+3
| | | | | | | | | | | | | | Summary: This is in preparation for a change to utilize this flag for symbols referenced/defined in either inline or module level assembly. Reviewers: mehdi_amini Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D26048 llvm-svn: 285376
* Use StringRef in Pass/PassManager APIs (NFC)Mehdi Amini2016-10-011-1/+1
| | | | llvm-svn: 283004
* [thinlto] Don't decay threshold for hot callsitesPiotr Padlewski2016-09-301-5/+22
| | | | | | | | | | | | | | Summary: We don't want to decay hot callsites to import chains of hot callsites. The same mechanism is used in LIPO. Reviewers: tejohnson, eraman, mehdi_amini Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D24976 llvm-svn: 282833
* [thinlto] Add cold-callsite import heuristicPiotr Padlewski2016-09-291-6/+16
| | | | | | | | | | | | | | Summary: Not tunned up heuristic, but with this small heuristic there is about +0.10% improvement on SPEC 2006 Reviewers: tejohnson, mehdi_amini, eraman Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D24940 llvm-svn: 282733
* [thinlto] Basic thinlto fdo heuristicPiotr Padlewski2016-09-261-3/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch improves thinlto importer by importing 3x larger functions that are called from hot block. I compared performance with the trunk on spec, and there were about 2% on povray and 3.33% on milc. These results seems to be consistant and match the results Teresa got with her simple heuristic. Some benchmarks got slower but I think they are just noisy (mcf, xalancbmki, omnetpp)- running the benchmarks again with more iterations to confirm. Geomean of all benchmarks including the noisy ones were about +0.02%. I see much better improvement on google branch with Easwaran patch for pgo callsite inlining (the inliner actually inline those big functions) Over all I see +0.5% improvement, and I get +8.65% on povray. So I guess we will see much bigger change when Easwaran patch will land (it depends on new pass manager), but it is still worth putting this to trunk before it. Implementation details changes: - Removed CallsiteCount. - ProfileCount got replaced by Hotness - hot-import-multiplier is set to 3.0 for now, didn't have time to tune it up, but I see that we get most of the interesting functions with 3, so there is no much performance difference with higher, and binary size doesn't grow as much as with 10.0. Reviewers: eraman, mehdi_amini, tejohnson Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D24638 llvm-svn: 282437
* FunctionImport: missed one occurence of ImportListForModule to rename (NFC)Mehdi Amini2016-08-161-1/+1
| | | | llvm-svn: 278778
* FunctionImport: rename ImportsForModule to ImportList for consistency (NFC)Mehdi Amini2016-08-161-7/+7
| | | | llvm-svn: 278777
* [LTO] Simplify APIs and constify (NFC)Mehdi Amini2016-08-161-21/+16
| | | | | | | | | | | | | | | | | Summary: Multiple APIs were taking a StringMap for the ImportLists containing the entries for for all the modules while operating on a single entry for the current module. Instead we can pass the desired ModuleImport directly. Also some of the APIs were not const, I believe just to be able to use operator[] on the StringMap. Reviewers: tejohnson Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D23537 llvm-svn: 278776
* [ThinLTO] Remove functions resolved to available_externally from comdatsTeresa Johnson2016-08-151-0/+9
| | | | | | | | | | | | | | | | Summary: thinLTOResolveWeakForLinkerModule needs to drop any preempted weak symbols that were converted to available_externally from comdats, otherwise we will get a verification failure (since available_externally is a declaration for the linker, and no declarations can be in a comdat). Reviewers: mehdi_amini Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D23015 llvm-svn: 278739
* Don't import variadic functionsPiotr Padlewski2016-08-111-0/+4
| | | | | | | | | | | | | | | | | | | | Summary: This patch adds IsVariadicFunction bit to summary in order to not import variadic functions. Inliner doesn't inline variadic functions because it is hard to reason about it. This one small fix improves Importer by about 16% (going from 86% to 100% of imported functions that are inlined anywhere) on some spec benchmarks like 'int' and others. Reviewers: eraman, mehdi_amini, tejohnson Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D23339 llvm-svn: 278432
* Consistently use ModuleAnalysisManagerSean Silva2016-08-091-1/+1
| | | | | | | | | | | Besides a general consistently benefit, the extra layer of indirection allows the mechanical part of https://reviews.llvm.org/D23256 that requires touching every transformation and analysis to be factored out cleanly. Thanks to David for the suggestion. llvm-svn: 278078
* [PM] Port FunctionImport Pass to new PMTeresa Johnson2016-07-181-45/+54
| | | | | | | | | | | | Summary: Port FunctionImport Pass to new PM. Reviewers: mehdi_amini, davide Subscribers: davidxl, llvm-commits Differential Revision: https://reviews.llvm.org/D22475 llvm-svn: 275916
* Add 'thinlto_src_module' md with asserts or -enable-import-metadataPiotr Padlewski2016-07-081-6/+19
| | | | | | | | | | | | | | | | Summary: This way the metadata will be only generated when asserts enabled, or when -enable-import-metadata specified FIXED missing colon on requires. Reviewers: tejohnson, eraman, mehdi_amini Subscribers: mehdi_amini, llvm-commits Differential Revision: http://reviews.llvm.org/D22167 llvm-svn: 274947
* Revert "Add 'thinlto_src_module' md with asserts or -enable-import-metadata"Piotr Padlewski2016-07-081-19/+6
| | | | | | | | | Reverting because of 17463 http://lab.llvm.org:8011/builders/clang-x86_64-linux-selfhost-modules/builds/17463 This reverts commit d20cb431bba2ba43b4c65a8556cff445bfefbb7c. llvm-svn: 274946
* Add 'thinlto_src_module' md with asserts or -enable-import-metadataPiotr Padlewski2016-07-081-6/+19
| | | | | | | | | | | | | | Summary: This way the metadata will be only generated when asserts enabled, or when -enable-import-metadata specified Reviewers: tejohnson, eraman, mehdi_amini Subscribers: mehdi_amini, llvm-commits Differential Revision: http://reviews.llvm.org/D22167 llvm-svn: 274938
* Add 'thinlto_src_module' metadata to imported functionPiotr Padlewski2016-07-061-0/+6
| | | | | | | | | | | | | | Added metadata to be able to make statistics on how many functions that have been imported have been removed. Also module name might be helpfull when debugging. Reviewers: tejohnson, eraman Subscribers: mehdi_amini, llvm-commits Differential Revision: http://reviews.llvm.org/D21943 llvm-svn: 274668
* NFC changed names in FunctionImportPiotr Padlewski2016-07-061-14/+14
| | | | llvm-svn: 274649
* Apply another batch of fixes from clang-tidy's ↵Benjamin Kramer2016-06-171-3/+3
| | | | | | | | performance-unnecessary-value-param. Contains some manual fixes. No functionality change intended. llvm-svn: 273047
* [ThinLTO/gold] Enable summary-based internalizationTeresa Johnson2016-06-091-2/+13
| | | | | | | | | | | | Summary: Enable existing summary-based importing support in the gold-plugin. Reviewers: mehdi_amini Subscribers: llvm-commits, mehdi_amini Differential Revision: http://reviews.llvm.org/D21080 llvm-svn: 272239
* [ThinLTO] Refactor ODR resolution and internalization (NFC)Teresa Johnson2016-05-251-0/+77
| | | | | | | | | Move the now index-based ODR resolution and internalization routines out of ThinLTOCodeGenerator.cpp and into either LTO.cpp (index-based analysis) or FunctionImport.cpp (index-driven optimizations). This is to enable usage by other linkers. llvm-svn: 270698
* [ThinLTO] Don't re-analyze callee at same threshold unnecessarilyTeresa Johnson2016-05-111-1/+1
| | | | | | | | | This should just be a compile-time change. Correct the check for whether we have already analyzed the callee when making summary based decisions. There is no need to reprocess one at the same threshold as when it was last processed. llvm-svn: 269251
* Delete mayBeOverridden.Rafael Espindola2016-05-111-1/+1
| | | | | | It is the same as isInterposable which seems to be the preferred name. llvm-svn: 269150
* [ThinLTO] Add option to emit imports files for distributed backendsTeresa Johnson2016-05-101-0/+15
| | | | | | | | | | | | | | | | | | | | | Summary: Add support for emission of plaintext lists of the imported files for each distributed backend compilation. Used for distributed build file staging. Invoked with new gold-plugin thinlto-emit-imports-files option, which is only valid with thinlto-index-only (i.e. for distributed builds), or from llvm-lto with new -thinlto-action=emitimports value. Depends on D19556. Reviewers: joker.eph Subscribers: llvm-commits, joker.eph Differential Revision: http://reviews.llvm.org/D19636 llvm-svn: 269067
* Restore "[ThinLTO] Emit individual index files for distributed backends"Teresa Johnson2016-05-101-0/+27
| | | | | | | | | | | | | | | | | | | This restores commit r268627: Summary: When launching ThinLTO backends in a distributed build (currently supported in gold via the thinlto-index-only plugin option), emit an individual index file for each backend process as described here: http://lists.llvm.org/pipermail/llvm-dev/2016-April/098272.html ... Differential Revision: http://reviews.llvm.org/D19556 Address msan failures by avoiding std::prev on map.end(), the theory is that this is causing issues due to some known UB problems in __tree. llvm-svn: 269059
* Revert "[ThinLTO] Emit individual index files for distributed backends"Vitaly Buka2016-05-051-27/+0
| | | | | | | | | MemorySanitizer: use-of-uninitialized-value in lib/Bitcode/Writer/BitcodeWriter.cpp:364:70 http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/12544/steps/check-llvm%20msan/logs/stdio This reverts commit 0c4a898ea550699d1b2f4fe3767251c8f9a48d52. llvm-svn: 268660
* [ThinLTO] Emit individual index files for distributed backendsTeresa Johnson2016-05-051-0/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: When launching ThinLTO backends in a distributed build (currently supported in gold via the thinlto-index-only plugin option), emit an individual index file for each backend process as described here: http://lists.llvm.org/pipermail/llvm-dev/2016-April/098272.html The individual index file encodes the summary and module information required for implementing the importing/exporting decisions made for a given module in the thin link step. This is in place of the current mechanism that uses the combined index to make importing decisions in each back end independently. It is an enabler for doing global summary based optimizations in the thin link step (which will be recorded in the individual index files), and reduces the size of the index that must be sent to each backend process, and the amount of work to scan it in the backends. Rather than create entirely new ModuleSummaryIndex structures (and all the included unique_ptrs) for each backend index file, a map is created to record all of the GUID and summary pointers needed for a particular index file. The IndexBitcodeWriter walks this map instead of the full index (hiding the details of managing the appropriate summary iteration in a new iterator subclass). This is more efficient than walking the entire combined index and filtering out just the needed summaries during each backend bitcode index write. Depends on D19481. Reviewers: joker.eph Subscribers: llvm-commits, joker.eph Differential Revision: http://reviews.llvm.org/D19556 llvm-svn: 268627
* ThinLTO: do not import function whose linkage prevents inlining.Mehdi Amini2016-05-031-2/+2
| | | | | | | | | | | There is not point in importing a "weak" or a "linkonce" function since we won't be able to inline it anyway. We already had a targeted check for WeakAny, this is using the same check on GlobalValue as the inline, i.e. isMayBeOverriddenLinkage() From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 268341
* Revert "ThinLTO: do not import function whose linkage prevents inlining."Mehdi Amini2016-05-021-2/+2
| | | | | | | This reverts commit r268315, the tests are not passing. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 268317
* ThinLTO: do not import function whose linkage prevents inlining.Mehdi Amini2016-05-021-2/+2
| | | | | | | | | | | There is not point in importing a "weak" or a "linkonce" function since we won't be able to inline it anyway. We already had a targeted check for WeakAny, this is using the same check on GlobalValue as the inline, i.e. isMayBeOverriddenLinkage() From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 268315
* ThinLTO: do not promote GlobalVariable that have a specific section.Mehdi Amini2016-04-271-4/+70
| | | | | | | Differential Revision: http://reviews.llvm.org/D18298 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 267646
* [ThinLTO] Introduce typedef for commonly-used map type (NFC)Teresa Johnson2016-04-251-7/+4
| | | | | | | | Add a typedef for the std::map<GlobalValue::GUID, GlobalValueSummary *> map that is passed around to identify summaries for values defined in a particular module. This shortens up declarations in a variety of places. llvm-svn: 267471
* [ThinLTO] Remove GlobalValueInfo class from indexTeresa Johnson2016-04-241-32/+29
| | | | | | | | | | | | | | | | Summary: Remove the GlobalValueInfo and change the ModuleSummaryIndex to directly reference summary objects. The info structure was there to support lazy parsing of the combined index summary objects, which is no longer needed and not supported. Reviewers: joker.eph Subscribers: joker.eph, llvm-commits Differential Revision: http://reviews.llvm.org/D19462 llvm-svn: 267344
* Always traverse GlobalVariable initializer when computing the export listMehdi Amini2016-04-231-21/+50
| | | | | | | | | | | | | | | | Summary: We are always importing the initializer for a GlobalVariable. So if a GlobalVariable is in the export-list, we pull in any refs as well. Reviewers: tejohnson Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D19102 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 267303
* Re-commit optimization bisect support (r267022) without new pass manager ↵Andrew Kaylor2016-04-221-0/+3
| | | | | | | | | | support. The original commit was reverted because of a buildbot problem with LazyCallGraph::SCC handling (not related to the OptBisect handling). Differential Revision: http://reviews.llvm.org/D19172 llvm-svn: 267231
* Revert "Initial implementation of optimization bisect support."Vedant Kumar2016-04-221-3/+0
| | | | | | | | This reverts commit r267022, due to an ASan failure: http://lab.llvm.org:8080/green/job/clang-stage2-cmake-RgSan_check/1549 llvm-svn: 267115
* Initial implementation of optimization bisect support.Andrew Kaylor2016-04-211-0/+3
| | | | | | | | | | | | This patch implements a optimization bisect feature, which will allow optimizations to be selectively disabled at compile time in order to track down test failures that are caused by incorrect optimizations. The bisection is enabled using a new command line option (-opt-bisect-limit). Individual passes that may be skipped call the OptBisect object (via an LLVMContext) to see if they should be skipped based on the bisect limit. A finer level of control (disabling individual transformations) can be managed through an addition OptBisect method, but this is not yet used. The skip checking in this implementation is based on (and replaces) the skipOptnoneFunction check. Where that check was being called, a new call has been inserted in its place which checks the bisect limit and the optnone attribute. A new function call has been added for module and SCC passes that behaves in a similar way. Differential Revision: http://reviews.llvm.org/D19172 llvm-svn: 267022
OpenPOWER on IntegriCloud