summaryrefslogtreecommitdiffstats
path: root/llvm/lib/LTO/LTO.cpp
Commit message (Collapse)AuthorAgeFilesLines
...
* [ThinLTO] Import composite types as declarationsTeresa Johnson2016-12-161-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | Summary: When reading the metadata bitcode, create a type declaration when possible for composite types when we are importing. Doing this in the bitcode reader saves memory. Also it works naturally in the case when the type ODR map contains a definition for the same composite type because it was used in the importing module (buildODRType will automatically use the existing definition and not create a type declaration). For Chromium built with -g2, this reduces the aggregate size of the generated native object files by 66% (from 31G to 10G). It reduced the time through the ThinLTO link and backend phases by about 20% on my machine. Reviewers: mehdi_amini, dblaikie, aprantl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D27775 llvm-svn: 289993
* Pass sample pgo flags to thinlto.Dehao Chen2016-12-161-0/+6
| | | | | | | | | | | | Summary: ThinLTO needs to invoke SampleProfileLoader pass during link time in order to annotate profile correctly after module importing. Reviewers: davidxl, mehdi_amini, tejohnson Subscribers: pcc, davide, llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D27790 llvm-svn: 289957
* [LTO] Reject modules without datalayout.Davide Italiano2016-12-141-0/+5
| | | | | | | | | | | Also, udpate the ~60 failing tests in the tree which did not contain a valid datalayout. This fixes PR31123. lld will be updated in a following patch, immediately after this is committed. Differential Revision: https://reviews.llvm.org/D27082 llvm-svn: 289719
* LTO: Add support for multi-module bitcode files.Peter Collingbourne2016-12-141-58/+112
| | | | | | Differential Revision: https://reviews.llvm.org/D27313 llvm-svn: 289621
* LTO: Port the new LTO API to ModuleSymbolTable.Peter Collingbourne2016-12-131-44/+48
| | | | | | Differential Revision: https://reviews.llvm.org/D27077 llvm-svn: 289574
* [ThinLTO] Import only necessary DICompileUnit fieldsTeresa Johnson2016-12-121-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: As discussed on mailing list, for ThinLTO importing we don't need to import all the fields of the DICompileUnit. Don't import enums, macros, retained types lists. Also only import local scoped imported entities. Since we don't currently import any global variables, we also don't need to import the list of global variables (added an assert to verify none are being imported). This is being done by pre-populating the value map entries to map the unneeded metadata to nullptr. For the imported entities, we can simply replace the source module's list with a new list containing only those needed imported entities. This is done in the IRLinker constructor so that value mapping automatically does the desired mapping. Reviewers: mehdi_amini, dexonsmith, dblaikie, aprantl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D27635 llvm-svn: 289441
* LTO: Hash the parts of the LTO configuration that affect code generation.Peter Collingbourne2016-12-081-3/+36
| | | | | | | | | Most importantly, we need to hash the relocation model, otherwise we can end up trying to link non-PIC object files into PIEs or DSOs. Differential Revision: https://reviews.llvm.org/D27556 llvm-svn: 289024
* LTO: Remove ModuleLoader, make loadModuleFromBuffer static and move into its ↵Peter Collingbourne2016-12-011-20/+0
| | | | | | | | | | only client, ThinLTOCodeGenerator. This is no longer the recommended way to load modules for importing, so it should not be public API. Differential Revision: https://reviews.llvm.org/D27292 llvm-svn: 288316
* LTO: Remove Symbol::getIRName().Peter Collingbourne2016-12-011-1/+4
| | | | | | | Its only use was in the LTO implementation. Also document Symbol::getName(). llvm-svn: 288302
* Give some helper classes/functions internal linkage. NFC.Benjamin Kramer2016-11-191-0/+4
| | | | llvm-svn: 287462
* Bitcode: Change module reader functions to return an llvm::Expected.Peter Collingbourne2016-11-131-29/+17
| | | | | | Differential Revision: https://reviews.llvm.org/D26562 llvm-svn: 286752
* Bitcode: Change getModuleSummaryIndex() to return an llvm::Expected.Peter Collingbourne2016-11-111-4/+3
| | | | | | Differential Revision: https://reviews.llvm.org/D26539 llvm-svn: 286624
* Bitcode: Clean up error handling for certain bitcode query functions.Peter Collingbourne2016-11-111-2/+4
| | | | | | | | | | | | | The functions getBitcodeTargetTriple(), isBitcodeContainingObjCCategory(), getBitcodeProducerString() and hasGlobalValueSummary() now return errors via their return value rather than via the diagnostic handler. To make this work, re-implement these functions using non-member functions so that they can be used without the LLVMContext required by BitcodeReader. Differential Revision: https://reviews.llvm.org/D26532 llvm-svn: 286623
* Split Bitcode/ReaderWriter.h into separate reader and writer headersTeresa Johnson2016-11-111-1/+2
| | | | | | | | | | | | | | | | | | | | | Summary: Split ReaderWriter.h which contains the APIs into both the BitReader and BitWriter libraries into BitcodeReader.h and BitcodeWriter.h. This is to address Chandler's concern about sharing the same API header between multiple libraries (BitReader and BitWriter). That concern is why we create a single bitcode library in our downstream build of clang, which led to r286297 being reverted as it added a dependency that created a cycle only when there is a single bitcode library (not two as in upstream). Reviewers: mehdi_amini Subscribers: dlj, mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D26502 llvm-svn: 286566
* Make the Error class constructor protectedMehdi Amini2016-11-111-10/+10
| | | | | | | | | This is forcing to use Error::success(), which is in a wide majority of cases a lot more readable. Differential Revision: https://reviews.llvm.org/D26481 llvm-svn: 286561
* Bitcode: Change the materializer interface to return llvm::Error.Peter Collingbourne2016-11-091-1/+2
| | | | | | Differential Revision: https://reviews.llvm.org/D26439 llvm-svn: 286382
* IR, Bitcode: Change bitcode reader to no longer own its memory buffer.Peter Collingbourne2016-11-081-3/+2
| | | | | | | | | | | | | | | | | | Unique ownership is just one possible ownership pattern for the memory buffer underlying the bitcode reader. In practice, as this patch shows, ownership can often reside at a higher level. With the upcoming change to allow multiple modules in a single bitcode file, it will no longer be appropriate for modules to generally have unique ownership of their memory buffer. The C API exposes the ownership relation via the LLVMGetBitcodeModuleInContext and LLVMGetBitcodeModuleInContext2 functions, so we still need some way for the module to own the memory buffer. This patch does so by adding an owned memory buffer field to Module, and using it in a few other places where it is convenient. Differential Revision: https://reviews.llvm.org/D26384 llvm-svn: 286214
* [ThinLTO] Disable importing and other cross-module optis at -O0Teresa Johnson2016-10-311-27/+31
| | | | | | | | | | | | | | | | Summary: There is no point to importing at -O0, since we won't inline. We should also disable other cross-module optimizations. (Plan to backport this fix to the 3.9 branch to fix PR30774) Reviewers: pcc Subscribers: johanengelen, mehdi_amini Differential Revision: https://reviews.llvm.org/D25918 llvm-svn: 285648
* [ThinLTO] Correctly resolve linkonce when importing aliaseeTeresa Johnson2016-10-301-3/+8
| | | | | | | | | | | | | | | | | | | Summary: When we have an aliasee that is linkonce, while we can't convert the non-prevailing copies to available_externally, we still need to convert the prevailing copy to weak. If a reference to the aliasee is exported, not converting a copy to weak will result in undefined references when the linkonce is removed in its original module. Add a new test and update existing tests. Reviewers: mehdi_amini Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D26076 llvm-svn: 285512
* fix warningRafael Espindola2016-10-251-0/+1
| | | | llvm-svn: 285064
* Make the LTO comdat api more symbol table friendly.Rafael Espindola2016-10-251-2/+29
| | | | | | | | | | | | | | | | | | | | | | In an IR symbol table I would expect the comdats to be represented as: - A table of strings, one for each comdat name. - Each symbol has an optional index into that table. The natural api for accessing that would be InputFile: ArrayRef<StringRef> getComdatTable() const; Symbol: int getComdatIndex() const; This patch implements an API as close to that as possible. The implementation on top of the current IRObjectFile is a bit hackish, but should map just fine over a symbol table and is very convenient to use. llvm-svn: 285061
* [ThinLTO] Default backend threads to heavyweight_hardware_concurrencyTeresa Johnson2016-10-191-1/+3
| | | | | | | | | | | | | | | | | Summary: Changes default backend parallelism from thread::hardware_concurrency to the new llvm::heavyweight_hardware_concurrency, which for X86 Linux defaults to the number of physical cores (and will fall back to thread::hardware_concurrency otherwise). This avoid oversubscribing the physical cores using hyperthreading. Reviewers: mehdi_amini, pcc Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D25775 llvm-svn: 284618
* [ThinLTO] Don't link module level assembly when importingTeresa Johnson2016-10-121-1/+2
| | | | | | | | | | | | | | | | | | | | | Module inline asm was always being linked/concatenated when running the IRLinker. This is correct for full LTO but not when we are importing for ThinLTO, as it can result in multiply defined symbols when the module asm defines a global symbol. In order to test with llvm-lto2, I had to work around PR30396, where a symbol that is defined in module assembly but defined in the LLVM IR appears twice. Added workaround to llvm-lto2 with a FIXME. Fixes PR30610. Reviewers: mehdi_amini Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D25359 llvm-svn: 284030
* ThinLTO: don't perform incremental LTO on module without a hashMehdi Amini2016-10-081-2/+6
| | | | | | | | | Clang always emit a hash for ThinLTO, but as other frontend are starting to use ThinLTO, this could be a serious bug. Differential Revision: https://reviews.llvm.org/D25379 llvm-svn: 283655
* ThinLTO: handles modules with empty summariesMehdi Amini2016-10-081-3/+5
| | | | | | | | | | | | | We need to add an entry in the combined-index for modules that have a hash but otherwise empty summary, this is needed so that we can get the hash for the module. Also, if no entry is present in the combined index for a module, we need to skip it when trying to compute a cache entry. Differential Revision: https://reviews.llvm.org/D25300 llvm-svn: 283654
* LTO: Simplify caching interface.Peter Collingbourne2016-09-231-38/+39
| | | | | | | | | | | | | | | | | The NativeObjectOutput class has a design problem: it mixes up the caching policy with the interface for output streams, which makes the client-side code hard to follow and would for example make it harder to replace the cache implementation in an arbitrary client. This change separates the two aspects by moving the caching policy to a separate field in Config, replacing NativeObjectOutput with a NativeObjectStream class which only deals with streams and does not need to be overridden by most clients and introducing an AddFile callback for adding files (e.g. from the cache) to the link. Differential Revision: https://reviews.llvm.org/D24622 llvm-svn: 282299
* [ThinLTO] Emit files for distributed builds for all modulesTeresa Johnson2016-09-211-20/+20
| | | | | | | | | | | | | | | | | | | | With the new LTO API in r278338, we stopped emitting the individual index files and imports files for some modules in the distributed backend case (thinlto-index-only plugin option). Specifically, this is when the linker decides not to include a module in the link, because it was in an archive library and did not have a strong reference to it. Not creating the expected output files makes the distributed build system implementation more difficult, in terms of checking for the expected outputs of the thin link, and scheduling the backend jobs. To address this, the gold-plugin will write dummy empty .thinlto.bc and .imports files for modules not included in the link (which LTO never sees). Augmented a gold v1.12+ test, since that version of gold has the handling for notifying on modules not being included in the link. llvm-svn: 282100
* [ThinLTO] Always emit a summary when compiling in ThinLTO modeTeresa Johnson2016-09-201-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Emit an empty summary section, instead of no summary section, when there are no global variables in the index. This ensures that LTO will treat these files as ThinLTO inputs, instead of as regular LTO inputs. In addition to not being what the user likely intended when compiling with -flto=thin, the current behavior is problematic for distributed build systems that expect to get ThinLTO index and imports files back for each input compiled with -flto=thin. Combining into a single regular LTO module also reduces the backend parallelism. And in the case where the index was suppressed due to uses in inline assembly, combining into a single LTO module could provoke renaming of duplicates that we were trying to prevent by suppressing the index. This change required a couple of fixes to handle the empty summary section. Reviewers: mehdi_amini Subscribers: mehdi_amini, llvm-commits, pcc Differential Revision: https://reviews.llvm.org/D24779 llvm-svn: 282037
* [LTO] Fix handling of mixed (regular and thin) mode LTOTeresa Johnson2016-09-161-6/+9
| | | | | | | | | | | | | | | | | | | | | Summary: In runThinLTO we start the task numbering for ThinLTO backend tasks depending on whether there was also a regular LTO object (CombinedModule). However, the CombinedModule is moved at the end of runRegularLTO, so we need to save this information and pass it into runThinLTO. Otherwise the AddOutput callback to the client will use the same task number for both the regular LTO object and the first ThinLTO object, which in gold-plugin caused only one to be end up in the output filename array and therefore passed back to gold for the final native link. Reviewers: pcc, mehdi_amini Subscribers: mehdi_amini, kromanova Differential Revision: https://reviews.llvm.org/D24643 llvm-svn: 281725
* [LTO] Fix commons handlingMehdi Amini2016-09-141-3/+7
| | | | | | | | | | | | | | Previously the prevailing information was not honored, and commons symbols could override a strong definition. This patch fixes it and propose the following semantic for commons: the client should mark as prevailing the commons that it expects the LTO implementation to merge (i.e. take the maximum size and alignment). It implies that commons are allowed to have multiple prevailing definitions. Differential Revision: https://reviews.llvm.org/D24545 llvm-svn: 281538
* [LTO] Don't pass SF_Undefined symbols to the IRmover.Davide Italiano2016-09-131-0/+2
| | | | | | This should fix PR 30363. llvm-svn: 281366
* [LTO] Constify (NFC)Mehdi Amini2016-09-061-16/+20
| | | | llvm-svn: 280687
* [LTO] Don't create a new common unless merged has different sizeTeresa Johnson2016-08-271-3/+4
| | | | | | | | | | | | | | | | | Summary: This addresses a regression in common handling from the new LTO API in r278338. Only create a new common if the size is different. The type comparison against an array type fails when the size is different but not an array. GlobalMerge does not handle the array types as well and we lose some global merging opportunities. Reviewers: mehdi_amini Subscribers: junbuml, llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D23955 llvm-svn: 279911
* Make writeToResolutionFile a static helper.Rafael Espindola2016-08-261-10/+10
| | | | llvm-svn: 279859
* [ThinLTO] Add caching to the new LTO APIMehdi Amini2016-08-231-28/+121
| | | | | | | | | | | | Add the ability to plug a cache on the LTO API. I tried to write such that a linker implementation can control the cache backend. This is intrusive and I'm not totally happy with it, but I can't figure out a better design right now. Differential Revision: https://reviews.llvm.org/D23599 llvm-svn: 279576
* Stop always creating and running an LTO compilation if there is not a single ↵Mehdi Amini2016-08-231-21/+13
| | | | | | | | | | | | | | | | | | | | LTO object Summary: I assume there was a use case, so maybe this strawman patch will help clarifying if it is legit. In any case the current situation is not legit: a ThinLTO compilation should not trigger an unexpected full LTO compilation. Right now, adding a --save-temps option triggers this and makes the number of output differs. Reviewers: tejohnson Subscribers: pcc, llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D23600 llvm-svn: 279550
* [ThinLTO] Make sure the Context used for the ThinLTO backend has all the ↵Mehdi Amini2016-08-231-1/+1
| | | | | | | | | | | | | appropriate options An important performance setting on the LLVMContext for LTO is enableDebugTypeODRUniquing(), this adds an automatic merging of debug information in the context based on type ids. Also, the lto::Config includes a diagnostic handler that needs to be set on the Context, as well as the setDiscardValueNames() setting. llvm-svn: 279532
* [LTO] Handles commons in monolithic LTOMehdi Amini2016-08-221-0/+33
| | | | | | | | | The gold-plugin was doing this internally, now the API is handling commons correctly based on the given resolution. Differential Revision: https://reviews.llvm.org/D23739 llvm-svn: 279417
* [LTO] Add a "CodeGenOnly" option. Allows the client to skip the optimizer.Mehdi Amini2016-08-221-19/+21
| | | | | | | | | | | | Summary: Slowly getting on par with libLTO Reviewers: tejohnson Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D23615 llvm-svn: 279416
* [LTO] Remove dead-code: collectUsedGlobalVariables has been moved to Thin ↵Mehdi Amini2016-08-191-3/+1
| | | | | | and LTO specifc path (NFC) llvm-svn: 279261
* [LTO] Move callback member from base class to the derived where it is used (NFC)Mehdi Amini2016-08-191-12/+10
| | | | llvm-svn: 279212
* [LTO] Add a move to inialize member in ctor initialization list (NFC)Mehdi Amini2016-08-191-1/+1
| | | | llvm-svn: 279210
* [LTO] Introduce an Output class to wrap the output stream creation (NFC)Mehdi Amini2016-08-171-21/+21
| | | | | | | | | | | | | | | | | Summary: While NFC for now, this will allow more flexibility on the client side to hold state necessary to back up the stream. Also when adding caching, this class will grow in complexity. Note I blindly modified the gold-plugin as I can't compile it. Reviewers: tejohnson Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D23542 llvm-svn: 278907
* [LTO] Simplify APIs and constify (NFC)Mehdi Amini2016-08-161-8/+8
| | | | | | | | | | | | | | | | | Summary: Multiple APIs were taking a StringMap for the ImportLists containing the entries for for all the modules while operating on a single entry for the current module. Instead we can pass the desired ModuleImport directly. Also some of the APIs were not const, I believe just to be able to use operator[] on the StringMap. Reviewers: tejohnson Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D23537 llvm-svn: 278776
* Fix type truncation warningsTeresa Johnson2016-08-111-8/+8
| | | | | | | | | Avoid type truncation warnings from a 32-bit bot due to size_t not being unsigned long long, by converting the variables and constants to unsigned. This was introduced by r278338 and caused warnings here: http://bb.pgr.jp/builders/i686-mingw32-RA-on-linux/builds/15527/steps/build_llvmclang/logs/warnings%20%287%29 llvm-svn: 278406
* Restore "Resolution-based LTO API."Teresa Johnson2016-08-111-5/+531
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This restores commit r278330, with fixes for a few bot failures: - Fix a late change I had made to the save temps output file that I missed due to existing files sitting on my disk - Fix a bunch of Windows bot failures with "ambiguous call to overloaded function" due to confusion between llvm::make_unique vs std::make_unique (preface the new make_unique calls with "llvm::") - Attempt to fix a modules bot failure by adding a missing include to LTO/Config.h. Original change: Resolution-based LTO API. Summary: This introduces a resolution-based LTO API. The main advantage of this API over existing APIs is that it allows the linker to supply a resolution for each symbol in each object, rather than the combined object as a whole. This will become increasingly important for use cases such as ThinLTO which require us to process symbol resolutions in a more complicated way than just adjusting linkage. Patch by Peter Collingbourne. Reviewers: rafael, tejohnson, mehdi_amini Subscribers: lhames, tejohnson, mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D20268 llvm-svn: 278338
* Revert "Resolution-based LTO API."Teresa Johnson2016-08-111-531/+5
| | | | | | | | | | This reverts commit r278330. I made a change to the save temps output that is causing issues with the bots. Didn't realize this because I had older output files sitting on disk in my test output directory. llvm-svn: 278331
* Resolution-based LTO API.Teresa Johnson2016-08-111-5/+531
| | | | | | | | | | | | | | | | | | | | | | Summary: This introduces a resolution-based LTO API. The main advantage of this API over existing APIs is that it allows the linker to supply a resolution for each symbol in each object, rather than the combined object as a whole. This will become increasingly important for use cases such as ThinLTO which require us to process symbol resolutions in a more complicated way than just adjusting linkage. Patch by Peter Collingbourne. Reviewers: rafael, tejohnson, mehdi_amini Subscribers: lhames, tejohnson, mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D20268 Address review comments llvm-svn: 278330
* ThinLTO: Do not take into account whether a definition has multiple copies ↵Peter Collingbourne2016-07-071-13/+3
| | | | | | | | | | | | | | | | | | | | | | when promoting. We currently do not touch a symbol's linkage in the case where a definition has a single copy. However, this code is effectively unnecessary: either the definition is not exported, in which case the internalize phase sets its linkage to internal, or it is exported, in which case we need to promote linkage to weak. Those two cases are already handled by existing code. I believe that the only real functional change here is in the case where we have a single definition which does not prevail (e.g. because the definition in a native object file prevails). In that case we now lower linkage to available_externally following the existing code path for that case. As a result we can remove the isExported function parameter from the thinLTOResolveWeakForLinkerInIndex function. Differential Revision: http://reviews.llvm.org/D21883 llvm-svn: 274784
* ThinLTO: Remove check for multiple modules before applying weak resolutions.Peter Collingbourne2016-07-071-4/+0
| | | | | | | | | | This check is not only unnecessary, it can produce the wrong result. If we are linking a single module and it has an exported linkonce symbol, we need to promote to weak in order to avoid PR19901-style problems. Differential Revision: http://reviews.llvm.org/D21917 llvm-svn: 274722
OpenPOWER on IntegriCloud