summaryrefslogtreecommitdiffstats
path: root/llvm/lib/LTO/ThinLTOCodeGenerator.cpp
Commit message (Collapse)AuthorAgeFilesLines
* [UpdateCompilerUsed] API rename and cleanup, suggested by Rafaael.Davide Italiano2016-06-221-1/+0
| | | | | | | | | * UpdateCompilerUsed() -> updateCompilerUsed() * ThinLTO doesn't use the API so we can remove the include * Clean up unused #include <functional> from the header * Rename #ifdef guard comment to be correct. llvm-svn: 273461
* [LTO] Move UpdateCompilerUsed.h from lib/ to include/Davide Italiano2016-06-221-1/+1
| | | | | | | | I plan to use it in lld soon. Differential Revision: http://reviews.llvm.org/D21575 llvm-svn: 273380
* Apply most suggestions of clang-tidy's performance-unnecessary-value-paramBenjamin Kramer2016-06-081-1/+1
| | | | | | | Avoids unnecessary copies. All changes audited & pass tests with asan. No functional change intended. llvm-svn: 272190
* [ThinLTO] Refactor ODR resolution and internalization (NFC)Teresa Johnson2016-05-251-174/+0
| | | | | | | | | Move the now index-based ODR resolution and internalization routines out of ThinLTOCodeGenerator.cpp and into either LTO.cpp (index-based analysis) or FunctionImport.cpp (index-driven optimizations). This is to enable usage by other linkers. llvm-svn: 270698
* [ThinLTO] Handle empty exports list for module.Teresa Johnson2016-05-241-6/+9
| | | | | | | We might have an empty exports list for a module and shouldn't assert when checking if a symbol is exported. llvm-svn: 270600
* [ThinLTO] Fix bot failures by avoiding threaded map modificationsTeresa Johnson2016-05-241-3/+5
| | | | | | | | Ensure multiple threads won't have to modify map, probable cause of bot failures from r270584 such as: http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/13095 llvm-svn: 270595
* [ThinLTO] Change ODR resolution and internalization to be index-basedTeresa Johnson2016-05-241-194/+272
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch changes the ODR resolution and internalization to be based on updates to the Index, which are consumed by the backend portion of the transformations. It will be followed by an NFC change to move these out of libLTO's ThinLTOCodeGenerator so that it can be used by other linkers (gold and lld) and by ThinLTO distributed backends. The global summary-based portions use callbacks so that the client can determine the prevailing copy and other information in a client-specific way. Eventually, with the API being developed in D20268, these may be modified to use information such as symbol resolutions, supplied by the clients to the API. Reviewers: joker-eph Subscribers: joker.eph, pcc, llvm-commits Differential Revision: http://reviews.llvm.org/D20290 llvm-svn: 270584
* [ThinLTO] Refactor module loader handling into new LTO file (NFC)Teresa Johnson2016-05-231-43/+1
| | | | | | | | | | | | Moved the ModuleLoader and supporting helper loadModuleFromBuffer out of ThinLTOCodeGenerator and into new LTO.h/LTO.cpp files. This is in preparation for a patch that will utilize these in the gold-plugin. Note that there are some other pending patches (D20268 and D20290) that also plan to refactor common interfaces and functionality into this same pair of new files. llvm-svn: 270509
* ThinLTO: sort inputs and schedule by decreasing sizeMehdi Amini2016-05-161-4/+18
| | | | | | | | | | | This is a compile time optimization: keeping a large file to process at the end hurts parallelism. The heurisitic used right now is the input buffer size, however we may want to consider the number of functions to import or the different number of files to load for importing as well. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 269684
* ThinLTO caching: reload cached file with mmap and drop heap-allocated memory ↵Mehdi Amini2016-05-161-5/+14
| | | | | | | | | | | | buffer This is reducing pressure on the OS memory system, and is NFC when not using a cache. I measure a 10x memory consumption reduction when linking opt with full debug info. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 269682
* ThinLTOCodeGenerator: fix race condition found using TSANMehdi Amini2016-05-151-0/+5
| | | | | From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 269588
* ThinLTOCodeGenerator: handle cases where temporary files can't be renamedMehdi Amini2016-05-141-2/+6
| | | | | | | For instance when they're on different filesystem. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 269544
* ThinLTOCodeGenerator: handle std::error_code instead of silently dropping it.Mehdi Amini2016-05-141-1/+5
| | | | | From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 269541
* [ThinLTO] Add option to emit imports files for distributed backendsTeresa Johnson2016-05-101-0/+24
| | | | | | | | | | | | | | | | | | | | | Summary: Add support for emission of plaintext lists of the imported files for each distributed backend compilation. Used for distributed build file staging. Invoked with new gold-plugin thinlto-emit-imports-files option, which is only valid with thinlto-index-only (i.e. for distributed builds), or from llvm-lto with new -thinlto-action=emitimports value. Depends on D19556. Reviewers: joker.eph Subscribers: llvm-commits, joker.eph Differential Revision: http://reviews.llvm.org/D19636 llvm-svn: 269067
* Restore "[ThinLTO] Emit individual index files for distributed backends"Teresa Johnson2016-05-101-0/+23
| | | | | | | | | | | | | | | | | | | This restores commit r268627: Summary: When launching ThinLTO backends in a distributed build (currently supported in gold via the thinlto-index-only plugin option), emit an individual index file for each backend process as described here: http://lists.llvm.org/pipermail/llvm-dev/2016-April/098272.html ... Differential Revision: http://reviews.llvm.org/D19556 Address msan failures by avoiding std::prev on map.end(), the theory is that this is causing issues due to some known UB problems in __tree. llvm-svn: 269059
* Revert "[ThinLTO] Emit individual index files for distributed backends"Vitaly Buka2016-05-051-23/+0
| | | | | | | | | MemorySanitizer: use-of-uninitialized-value in lib/Bitcode/Writer/BitcodeWriter.cpp:364:70 http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/12544/steps/check-llvm%20msan/logs/stdio This reverts commit 0c4a898ea550699d1b2f4fe3767251c8f9a48d52. llvm-svn: 268660
* [ThinLTO] Emit individual index files for distributed backendsTeresa Johnson2016-05-051-0/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: When launching ThinLTO backends in a distributed build (currently supported in gold via the thinlto-index-only plugin option), emit an individual index file for each backend process as described here: http://lists.llvm.org/pipermail/llvm-dev/2016-April/098272.html The individual index file encodes the summary and module information required for implementing the importing/exporting decisions made for a given module in the thin link step. This is in place of the current mechanism that uses the combined index to make importing decisions in each back end independently. It is an enabler for doing global summary based optimizations in the thin link step (which will be recorded in the individual index files), and reduces the size of the index that must be sent to each backend process, and the amount of work to scan it in the backends. Rather than create entirely new ModuleSummaryIndex structures (and all the included unique_ptrs) for each backend index file, a map is created to record all of the GUID and summary pointers needed for a particular index file. The IndexBitcodeWriter walks this map instead of the full index (hiding the details of managing the appropriate summary iteration in a new iterator subclass). This is more efficient than walking the entire combined index and filtering out just the needed summaries during each backend bitcode index write. Depends on D19481. Reviewers: joker.eph Subscribers: llvm-commits, joker.eph Differential Revision: http://reviews.llvm.org/D19556 llvm-svn: 268627
* ThinLTOCodeGenerator: remove useless temporary file that was emitted.Mehdi Amini2016-05-051-7/+4
| | | | | | | | This was a remaining of a previous scheme where some IPOs were taking place before we enter this code. This is not relevant anymore. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 268605
* Clean up to avoid compiler warnings for casting away const qualifiers.Sjoerd Meijer2016-04-271-3/+3
| | | | | | Differential Revision: http://reviews.llvm.org/D19598 llvm-svn: 267753
* ThinLTOCodeGenerator: preserve linkonce when in "MustPreserved" setMehdi Amini2016-04-261-4/+10
| | | | | | | | | If the linker specifically requested for a linkonce to be preserved, we need to make sure we won't drop it even if all the uses in the current module disappear. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 267543
* [ThinLTO] Introduce typedef for commonly-used map type (NFC)Teresa Johnson2016-04-251-11/+6
| | | | | | | | Add a typedef for the std::map<GlobalValue::GUID, GlobalValueSummary *> map that is passed around to identify summaries for values defined in a particular module. This shortens up declarations in a variety of places. llvm-svn: 267471
* [ThinLTO] Remove GlobalValueInfo class from indexTeresa Johnson2016-04-241-11/+13
| | | | | | | | | | | | | | | | Summary: Remove the GlobalValueInfo and change the ModuleSummaryIndex to directly reference summary objects. The info structure was there to support lazy parsing of the combined index summary objects, which is no longer needed and not supported. Reviewers: joker.eph Subscribers: joker.eph, llvm-commits Differential Revision: http://reviews.llvm.org/D19462 llvm-svn: 267344
* Add an internalization step to the ThinLTOCodeGeneratorMehdi Amini2016-04-241-18/+151
| | | | | | | | | | | | | Keeping as much as possible internal/private is known to help the optimizer. Let's try to benefit from this in ThinLTO. Note: this is early work, but is enough to build clang (and all the LLVM tools). I still need to write some lit-tests... Differential Revision: http://reviews.llvm.org/D19103 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 267317
* [ThinLTO] Remove unused/incomplete lazy summary reading support (NFC)Teresa Johnson2016-04-221-1/+1
| | | | | | | | | | | | | | This removes the interfaces added (and not yet complete) to support lazy reading of summaries. This support is not expected to be needed since we are moving to a model where the full index is only being traversed in the thin link step, instead of the back ends. (The second part of this that I plan to do next is remove the GlobalValueInfo from the ModuleSummaryIndex - it was mostly needed to support lazy parsing of summaries. The index can instead reference the summary structures directly.) llvm-svn: 267097
* ThinLTO: add module caching handling.Mehdi Amini2016-04-211-11/+148
| | | | | | | Differential Revision: http://reviews.llvm.org/D18494 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 266959
* ThinLTO: Resolve linkonce_odr aliases just like functionsMehdi Amini2016-04-211-11/+38
| | | | | | | | | | | | This help to streamline the process of handling importing since we don't need to special case alias everywhere: just like linkonce_odr function, make sure at least one alias is emitted by turning it weak. Differential Revision: http://reviews.llvm.org/D19308 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 266958
* Enable ODR uniquing of DITypes in more placesTeresa Johnson2016-04-191-0/+1
| | | | | | | | | | | | | | | | | | | | | | | Summary: This is a follow-on to apply Duncan's new DIType ODR uniquing from r266549 and r266713 in more places. Enable enableDebugTypeODRUniquing() for ThinLTO backends invoked via libLTO, similar to the way r266549 enabled this for ThinLTO backend threads launched from gold-plugin. Also enable enableDebugTypeODRUniquing in opt, similar to the way r266549 enabled this for llvm-link (on by default, can be disabled with new -disable-debug-info-type-map option), since we may perform ThinLTO importing from opt. Reviewers: dexonsmith, joker.eph Subscribers: joker.eph, llvm-commits Differential Revision: http://reviews.llvm.org/D19263 llvm-svn: 266746
* ThinLTO: Move the ODR resolution to be based purely on the summary.Mehdi Amini2016-04-161-47/+91
| | | | | | | | | This is a requirement for the cache handling in D18494 Differential Revision: http://reviews.llvm.org/D18908 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 266519
* ThinLTO: linkonce compile-time optimization, do not bother when there is ↵Mehdi Amini2016-04-141-0/+4
| | | | | | | only one input file From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 266281
* [ThinLTO] Move summary computation from BitcodeWriter to new passTeresa Johnson2016-04-111-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This is the first step in also serializing the index out to LLVM assembly. The per-module summary written to bitcode is moved out of the bitcode writer and to a new analysis pass (ModuleSummaryIndexWrapperPass). The pass itself uses a new builder class to compute index, and the builder class is used directly in places where we don't have a pass manager (e.g. llvm-as). Because we are computing summaries outside of the bitcode writer, we no longer can use value ids created by the bitcode writer's ValueEnumerator. This required changing the reference graph edge type to use a new ValueInfo class holding a union between a GUID (combined index) and Value* (permodule index). The Value* are converted to the appropriate value ID during bitcode writing. Also, this enables removal of the BitWriter library's dependence on the Analysis library that was previously required for the summary computation. Reviewers: joker.eph Subscribers: joker.eph, llvm-commits Differential Revision: http://reviews.llvm.org/D18763 llvm-svn: 265941
* Clean up calls to WriteBitcodeToFile (NFC)Teresa Johnson2016-04-041-1/+1
| | | | | | | | | | | Remove a default parameter value being passed unnecessarily, which also reduces the changes required when this parameter is changed in D18763. Document the remaining non-default bool value passed for another parameter. llvm-svn: 265348
* Revert "ThinLTO: add module caching handling."Mehdi Amini2016-04-021-95/+1
| | | | | | | This reverts commit r265214, unintentionally commited. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 265216
* ThinLTO: add module caching handling.Mehdi Amini2016-04-021-1/+95
| | | | | | | | | | | Reviewers: tejohnson Subscribers: llvm-commits, joker.eph Differential Revision: http://reviews.llvm.org/D18494 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 265214
* Fix -Wpedantic warning about extra semi-colonHans Wennborg2016-04-021-1/+1
| | | | llvm-svn: 265204
* ThinLTO: special handling for LinkOnce functionsMehdi Amini2016-04-011-0/+108
| | | | | | | | | | | | | | | | | | | | | | | | | | These function can be dropped by the compiler if they are no longer referenced in the current module. However there is a change that another module is still referencing them because of the import. Multiple solutions can be used: - Always import LinkOnce when a caller is imported. This ensure that every module with a call to a LinkOnce has the definition and will be able to emit it if it emits the call. - Turn the LinkOnce into Weak, so that it is always emitted. - Turn all LinkOnce into available_externally and come back after all modules are codegen'ed to emit only one copy of the linkonce, when there is still a reference to it. This patch implement the second option, with am optimization that only *one* module will turn the LinkOnce into Weak, while the others will turn it into available_externally, so that there is exactly one copy emitted for the whole compilation. http://reviews.llvm.org/D18346 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 265190
* ThinLTO: move ObjCARCContractPass in the CodeGen pipelineMehdi Amini2016-04-011-1/+6
| | | | | | | This is to be coherent with Full LTO. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 265118
* Add a libLTO API to stop/restart ThinLTO between optimizations and CodeGenMehdi Amini2016-04-011-2/+35
| | | | | | | | This allows the linker to instruct ThinLTO to perform only the optimization part or only the codegen part of the process. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 265113
* ThinLTO: use the callgraph from the combined index to drive the FunctionImporterMehdi Amini2016-03-261-8/+26
| | | | | | | | | | | | | | | | | | Summary: Now that the summary contains the full reference/call graph, we can replace the existing function importer that loads and inspect the IR to iteratively walk the call graph by a traversal based purely on the summary information. Decouple the actual importing decision from any IR manipulation. Reviewers: tejohnson Subscribers: llvm-commits, joker.eph Differential Revision: http://reviews.llvm.org/D18343 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 264503
* [ThinLTO] Renaming of function index to module summary index (NFC)Teresa Johnson2016-03-151-16/+16
| | | | | | | | | | | | | | | (Resubmitting after fixing missing file issue) With the changes in r263275, there are now more than just functions in the summary. Completed the renaming of data structures (started in r263275) to reflect the wider scope. In particular, changed the FunctionIndex* data structures to ModuleIndex*, and renamed related variables and comments. Also renamed the files to reflect the changes. A companion clang patch will immediately succeed this patch to reflect this renaming. llvm-svn: 263513
* Revert "[ThinLTO] Renaming of function index to module summary index (NFC)"Teresa Johnson2016-03-141-16/+16
| | | | | | This reverts commit r263490. Missed a file. llvm-svn: 263493
* [ThinLTO] Renaming of function index to module summary index (NFC)Teresa Johnson2016-03-141-16/+16
| | | | | | | | | | | | | With the changes in r263275, there are now more than just functions in the summary. Completed the renaming of data structures (started in r263275) to reflect the wider scope. In particular, changed the FunctionIndex* data structures to ModuleIndex*, and renamed related variables and comments. Also renamed the files to reflect the changes. A companion clang patch will immediately succeed this patch to reflect this renaming. llvm-svn: 263490
* [ThinLTO] Support for reference graph in per-module and combined summary.Teresa Johnson2016-03-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch adds support for including a full reference graph including call graph edges and other GV references in the summary. The reference graph edges can be used to make importing decisions without materializing any source modules, can be used in the plugin to make file staging decisions for distributed build systems, and is expected to have other uses. The call graph edges are recorded in each function summary in the bitcode via a list of <CalleeValueIds, StaticCount> tuples when no PGO data exists, or <CalleeValueId, StaticCount, ProfileCount> pairs when there is PGO, where the ValueId can be mapped to the function GUID via the ValueSymbolTable. In the function index in memory, the call graph edges reference the target via the CalleeGUID instead of the CalleeValueId. The reference graph edges are recorded in each summary record with a list of referenced value IDs, which can be mapped to value GUID via the ValueSymbolTable. Addtionally, a new summary record type is added to record references from global variable initializers. A number of bitcode records and data structures have been renamed to reflect the newly expanded scope of the summary beyond functions. More cleanup will follow. Reviewers: joker.eph, davidxl Subscribers: joker.eph, llvm-commits Differential Revision: http://reviews.llvm.org/D17212 llvm-svn: 263275
* Add a flag to the LLVMContext to disable name for Value other than GlobalValueMehdi Amini2016-03-101-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This is intended to be a performance flag, on the same level as clang cc1 option "--disable-free". LLVM will never initialize it by default, it will be up to the client creating the LLVMContext to request this behavior. Clang will do it by default in Release build (just like --disable-free). "opt" and "llc" can opt-in using -disable-named-value command line option. When performing LTO on llvm-tblgen, the initial merging of IR peaks at 92MB without this patch, and 86MB after this patch,setNameImpl() drops from 6.5MB to 0.5MB. The total link time goes from ~29.5s to ~27.8s. Compared to a compile-time flag (like the IRBuilder one), it performs very close. I profiled on SROA and obtain these results: 420ms with IRBuilder that preserve name 372ms with IRBuilder that strip name 375ms with IRBuilder that preserve name, and a runtime flag to strip Reviewers: chandlerc, dexonsmith, bogner Subscribers: joker.eph, llvm-commits Differential Revision: http://reviews.llvm.org/D17946 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 263086
* libLTO: add a ThinLTOCodeGenerator on the model of LTOCodeGenerator.Mehdi Amini2016-03-091-0/+384
This is intended to provide a parallel (threaded) ThinLTO scheme for linker plugin use through the libLTO C API. The intent of this patch is to provide a first implementation as a proof-of-concept and allows linker to start supporting ThinLTO by definiing the libLTO C API. Some part of the libLTO API are left unimplemented yet. Following patches will add support for these. The current implementation can link all clang/llvm binaries. Differential Revision: http://reviews.llvm.org/D17066 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 262977
OpenPOWER on IntegriCloud