summaryrefslogtreecommitdiffstats
path: root/llvm/lib/LTO/LTO.cpp
Commit message (Collapse)AuthorAgeFilesLines
* [ThinLTO] Fix handling of weak interposable symbolsTeresa Johnson2019-08-231-5/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Keep aliasees alive if their alias is live, otherwise we end up with an alias to a declaration, which is invalid. This can happen when the aliasee is weak and non-prevailing. This fix exposed the fact that we were then attempting to internalize the weak symbol, which was not exported as it was not prevailing. We should not internalize interposable symbols in general, unless this is the prevailing copy, since it can lead to incorrect inlining and other optimizations. Most of the changes in this patch are due to the restructuring required to pass down the prevailing callback. Finally, while implementing the test cases, I found that in the case of a weak aliasee that is still marked not live because its alias isn't live, after dropping the definition we incorrectly marked the declaration with weak linkage when resolving prevailing symbols in the module. This was due to some special case handling for symbols marked WeakLinkage in the summary located before instead of after a subsequent check for the symbol being a declaration. It turns out that we don't actually need this special case handling any more (looking back at the history, when that was added the code was structured quite differently) - we will correctly mark with weak linkage further below when the definition hasn't been dropped. Fixes PR42542. Reviewers: pcc Subscribers: mehdi_amini, inglorion, steven_wu, dexonsmith, dang, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D66264 llvm-svn: 369766
* [llvm] Migrate llvm::make_unique to std::make_uniqueJonas Devlieghere2019-08-151-5/+5
| | | | | | | | Now that we've moved to C++14, we no longer need the llvm::make_unique implementation from STLExtras.h. This patch is a mechanical replacement of (hopefully) all the llvm::make_unique instances across the monorepo. llvm-svn: 369013
* Rename F_{None,Text,Append} to OF_{None,Text,Append}. NFCFangrui Song2019-08-051-2/+2
| | | | | | F_{None,Text,Append} are kept for compatibility since r334221. llvm-svn: 367800
* [ThinLTO] Implement index-based WPDTeresa Johnson2019-08-021-1/+15
| | | | | | | | | | | | | | | | | | | This patch adds support to the WholeProgramDevirt pass to perform index-based WPD, which is invoked from ThinLTO during the thin link. The ThinLTO backend (WPD import phase) behaves the same regardless of whether the WPD decisions were made with the index-based or (the existing) IR-based analysis. Depends on D54815. Reviewers: pcc Subscribers: mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, arphaman, dang, llvm-commits Differential Revision: https://reviews.llvm.org/D55153 llvm-svn: 367679
* [ThinLTO] Attempt to recommit r365188 after alignment fixEugene Leviant2019-07-051-9/+12
| | | | llvm-svn: 365215
* Reverted r365188 due to alignment problems on i686-androidEugene Leviant2019-07-051-12/+9
| | | | llvm-svn: 365206
* [ThinLTO] Attempt to recommit r365040 after caching fixEugene Leviant2019-07-051-9/+12
| | | | | | | | | | | | | | It's possible that some function can load and store the same variable using the same constant expression: store %Derived* @foo, %Derived** bitcast (%Base** @bar to %Derived**) %42 = load %Derived*, %Derived** bitcast (%Base** @bar to %Derived**) The bitcast expression was mistakenly cached while processing loads, and never examined later when processing store. This caused @bar to be mistakenly treated as read-only variable. See load-store-caching.ll. llvm-svn: 365188
* Revert [ThinLTO] Optimize writeonly globals outReid Kleckner2019-07-041-12/+9
| | | | | | | | | This reverts r365040 (git commit 5cacb914758c7f436b47c8362100f10cef14bbc4) Speculatively reverting, since this appears to have broken check-lld on Linux. Partial analysis in https://crbug.com/981168. llvm-svn: 365097
* [ThinLTO] Optimize writeonly globals outEugene Leviant2019-07-031-9/+12
| | | | | | Differential revision: https://reviews.llvm.org/D63444 llvm-svn: 365040
* [Remarks] Extend -fsave-optimization-record to specify the formatFrancis Visoiu Mistrih2019-06-171-3/+3
| | | | | | | | | Use -fsave-optimization-record=<format> to specify a different format than the default, which is YAML. For now, only YAML is supported. llvm-svn: 363573
* Reland: [Remarks] Refactor optimization remarks setupFrancis Visoiu Mistrih2019-06-141-24/+12
| | | | | | | | * Add a common function to setup opt-remarks * Rename common options to the same names * Add error types to distinguish between file errors and regex errors llvm-svn: 363415
* Revert "[Remarks] Refactor optimization remarks setup"Francis Visoiu Mistrih2019-06-141-12/+24
| | | | | | | | This reverts commit 6e6e3af55bb97e1a4c97375c15a2b0099120c5a7. This breaks greendragon. llvm-svn: 363343
* [Remarks] Refactor optimization remarks setupFrancis Visoiu Mistrih2019-06-131-24/+12
| | | | | | | | * Add a common function to setup opt-remarks * Rename common options to the same names * Add error types to distinguish between file errors and regex errors llvm-svn: 363328
* [Remarks][NFC] Move the serialization to lib/RemarksFrancis Visoiu Mistrih2019-05-301-2/+3
| | | | | | | | | | | | Separate the remark serialization to YAML from the LLVM Diagnostics. This adds a new serialization abstraction: remarks::Serializer. It's completely independent from lib/IR and it provides an easy way to replace YAML by providing a new remarks::Serializer. Differential Revision: https://reviews.llvm.org/D62632 llvm-svn: 362160
* [ELF] Implement Dependent Libraries FeatureBen Dunbobbin2019-05-171-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch implements a limited form of autolinking primarily designed to allow either the --dependent-library compiler option, or "comment lib" pragmas ( https://docs.microsoft.com/en-us/cpp/preprocessor/comment-c-cpp?view=vs-2017) in C/C++ e.g. #pragma comment(lib, "foo"), to cause an ELF linker to automatically add the specified library to the link when processing the input file generated by the compiler. Currently this extension is unique to LLVM and LLD. However, care has been taken to design this feature so that it could be supported by other ELF linkers. The design goals were to provide: - A simple linking model for developers to reason about. - The ability to to override autolinking from the linker command line. - Source code compatibility, where possible, with "comment lib" pragmas in other environments (MSVC in particular). Dependent library support is implemented differently for ELF platforms than on the other platforms. Primarily this difference is that on ELF we pass the dependent library specifiers directly to the linker without manipulating them. This is in contrast to other platforms where they are mapped to a specific linker option by the compiler. This difference is a result of the greater variety of ELF linkers and the fact that ELF linkers tend to handle libraries in a more complicated fashion than on other platforms. This forces us to defer handling the specifiers to the linker. In order to achieve a level of source code compatibility with other platforms we have restricted this feature to work with libraries that meet the following "reasonable" requirements: 1. There are no competing defined symbols in a given set of libraries, or if they exist, the program owner doesn't care which is linked to their program. 2. There may be circular dependencies between libraries. The binary representation is a mergeable string section (SHF_MERGE, SHF_STRINGS), called .deplibs, with custom type SHT_LLVM_DEPENDENT_LIBRARIES (0x6fff4c04). The compiler forms this section by concatenating the arguments of the "comment lib" pragmas and --dependent-library options in the order they are encountered. Partial (-r, -Ur) links are handled by concatenating .deplibs sections with the normal mergeable string section rules. As an example, #pragma comment(lib, "foo") would result in: .section ".deplibs","MS",@llvm_dependent_libraries,1 .asciz "foo" For LTO, equivalent information to the contents of a the .deplibs section can be retrieved by the LLD for bitcode input files. LLD processes the dependent library specifiers in the following way: 1. Dependent libraries which are found from the specifiers in .deplibs sections of relocatable object files are added when the linker decides to include that file (which could itself be in a library) in the link. Dependent libraries behave as if they were appended to the command line after all other options. As a consequence the set of dependent libraries are searched last to resolve symbols. 2. It is an error if a file cannot be found for a given specifier. 3. Any command line options in effect at the end of the command line parsing apply to the dependent libraries, e.g. --whole-archive. 4. The linker tries to add a library or relocatable object file from each of the strings in a .deplibs section by; first, handling the string as if it was specified on the command line; second, by looking for the string in each of the library search paths in turn; third, by looking for a lib<string>.a or lib<string>.so (depending on the current mode of the linker) in each of the library search paths. 5. A new command line option --no-dependent-libraries tells LLD to ignore the dependent libraries. Rationale for the above points: 1. Adding the dependent libraries last makes the process simple to understand from a developers perspective. All linkers are able to implement this scheme. 2. Error-ing for libraries that are not found seems like better behavior than failing the link during symbol resolution. 3. It seems useful for the user to be able to apply command line options which will affect all of the dependent libraries. There is a potential problem of surprise for developers, who might not realize that these options would apply to these "invisible" input files; however, despite the potential for surprise, this is easy for developers to reason about and gives developers the control that they may require. 4. This algorithm takes into account all of the different ways that ELF linkers find input files. The different search methods are tried by the linker in most obvious to least obvious order. 5. I considered adding finer grained control over which dependent libraries were ignored (e.g. MSVC has /nodefaultlib:<library>); however, I concluded that this is not necessary: if finer control is required developers can fall back to using the command line directly. RFC thread: http://lists.llvm.org/pipermail/llvm-dev/2019-March/131004.html. Differential Revision: https://reviews.llvm.org/D60274 llvm-svn: 360984
* [ThinLTO] Don't internalize weak writeable variablesEugene Leviant2019-05-131-1/+14
| | | | | | | | | | Variables with linkonce_odr and weak_odr linkage shouldn't be internalized if they're not readonly. Otherwise we may end up with multiple copies of such variable, so reads and writes will become inconsistent Differential revision: https://reviews.llvm.org/D61255 llvm-svn: 360577
* [ThinLTO] Auto-hide prevailing linkonce_odr only when all copies eligibleTeresa Johnson2019-05-101-14/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: We hit undefined references building with ThinLTO when one source file contained explicit instantiations of a template method (weak_odr) but there were also implicit instantiations in another file (linkonce_odr), and the latter was the prevailing copy. In this case the symbol was marked hidden when the prevailing linkonce_odr copy was promoted to weak_odr. It led to unsats when the resulting shared library was linked with other code that contained a reference (expecting to be resolved due to the explicit instantiation). Add a CanAutoHide flag to the GV summary to allow the thin link to identify when all copies are eligible for auto-hiding (because they were all originally linkonce_odr global unnamed addr), and only do the auto-hide in that case. Most of the changes here are due to plumbing the new flag through the bitcode and llvm assembly, and resulting test changes. I augmented the existing auto-hide test to check for this situation. Reviewers: pcc Subscribers: mehdi_amini, inglorion, eraman, dexonsmith, arphaman, dang, llvm-commits, steven_wu, wmi Tags: #llvm Differential Revision: https://reviews.llvm.org/D59709 llvm-svn: 360466
* [LTO] Add plumbing to save stats during LTO on Darwin.Florian Hahn2019-04-191-10/+21
| | | | | | | | | | | | | | | | | | | | | Gold and ld on Linux already support saving stats, but the infrastructure is missing on Darwin. Unfortunately it seems like the configuration from lib/LTO/LTO.cpp is not used. This patch adds a new LTOStatsFile option and adds plumbing in Clang to use it on Darwin, similar to the way remarks are handled. Currnetly the handling of LTO flags seems quite spread out, with a bunch of duplication. But I am not sure if there is an easy way to improve that? Reviewers: anemet, tejohnson, thegameg, steven_wu Reviewed By: steven_wu Differential Revision: https://reviews.llvm.org/D60516 llvm-svn: 358753
* [ThinLTO] Fix ThinLTOCodegenerator to export llvm.used symbolsSteven Wu2019-04-171-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Reapply r357931 with fixes to ThinLTO testcases and llvm-lto tool. ThinLTOCodeGenerator currently does not preserve llvm.used symbols and it can internalize them. In order to pass the necessary information to the legacy ThinLTOCodeGenerator, the input to the code generator is rewritten to be based on lto::InputFile. Now ThinLTO using the legacy LTO API will requires data layout in Module. "internalize" thinlto action in llvm-lto is updated to run both "promote" and "internalize" with the same configuration as ThinLTOCodeGenerator. The old "promote" + "internalize" option does not produce the same output as ThinLTOCodeGenerator. This fixes: PR41236 rdar://problem/49293439 Reviewers: tejohnson, pcc, kromanova, dexonsmith Reviewed By: tejohnson Subscribers: ormris, bd1976llvm, mehdi_amini, inglorion, eraman, hiraditya, jkorous, dexonsmith, arphaman, dang, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60421 llvm-svn: 358601
* Revert [ThinLTO] Fix ThinLTOCodegenerator to export llvm.used symbolsSteven Wu2019-04-081-5/+0
| | | | | | This reverts r357931 (git commit 8b70a5c11e08116955a875b9085433f14737bcaf) llvm-svn: 357932
* [ThinLTO] Fix ThinLTOCodegenerator to export llvm.used symbolsSteven Wu2019-04-081-0/+5
| | | | | | | | | | | | | | | | | | | | | | | Summary: ThinLTOCodeGenerator currently does not preserve llvm.used symbols and it can internalize them. In order to pass the necessary information to the legacy ThinLTOCodeGenerator, the input to the code generator is rewritten to be based on lto::InputFile. This fixes: PR41236 rdar://problem/49293439 Reviewers: tejohnson, pcc, dexonsmith Reviewed By: tejohnson Subscribers: mehdi_amini, inglorion, eraman, hiraditya, jkorous, dang, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60226 llvm-svn: 357931
* Reland "[Remarks] Add -foptimization-record-passes to filter remark emission"Francis Visoiu Mistrih2019-03-121-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | Currently we have -Rpass for filtering the remarks that are displayed as diagnostics, but when using -fsave-optimization-record, there is no way to filter the remarks while generating them. This adds support for filtering remarks by passes using a regex. Ex: `clang -fsave-optimization-record -foptimization-record-passes=inline` will only emit the remarks coming from the pass `inline`. This adds: * `-fsave-optimization-record` to the driver * `-opt-record-passes` to cc1 * `-lto-pass-remarks-filter` to the LTOCodeGenerator * `--opt-remarks-passes` to lld * `-pass-remarks-filter` to llc, opt, llvm-lto, llvm-lto2 * `-opt-remarks-passes` to gold-plugin Differential Revision: https://reviews.llvm.org/D59268 Original llvm-svn: 355964 llvm-svn: 355984
* Revert "[Remarks] Add -foptimization-record-passes to filter remark emission"Francis Visoiu Mistrih2019-03-121-6/+0
| | | | | | This reverts commit 20fff32b7d1f1a1bd417b22aa9f26ededd97a3e5. llvm-svn: 355976
* [Remarks] Add -foptimization-record-passes to filter remark emissionFrancis Visoiu Mistrih2019-03-121-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | Currently we have -Rpass for filtering the remarks that are displayed as diagnostics, but when using -fsave-optimization-record, there is no way to filter the remarks while generating them. This adds support for filtering remarks by passes using a regex. Ex: `clang -fsave-optimization-record -foptimization-record-passes=inline` will only emit the remarks coming from the pass `inline`. This adds: * `-fsave-optimization-record` to the driver * `-opt-record-passes` to cc1 * `-lto-pass-remarks-filter` to the LTOCodeGenerator * `--opt-remarks-passes` to lld * `-pass-remarks-filter` to llc, opt, llvm-lto, llvm-lto2 * `-opt-remarks-passes` to gold-plugin Differential Revision: https://reviews.llvm.org/D59268 llvm-svn: 355964
* Reland "[Remarks] Refactor remark diagnostic emission in a RemarkStreamer"Francis Visoiu Mistrih2019-03-061-2/+3
| | | | | | | | | | | | This allows us to store more info about where we're emitting the remarks without cluttering LLVMContext. This is needed for future support for the remark section. Differential Revision: https://reviews.llvm.org/D58996 Original llvm-svn: 355507 llvm-svn: 355514
* Revert "[Remarks] Refactor remark diagnostic emission in a RemarkStreamer"Francis Visoiu Mistrih2019-03-061-3/+2
| | | | | | | | This reverts commit 2e8c4997a2089f8228c843fd81b148d903472e02. Breaks bots. llvm-svn: 355511
* [Remarks] Refactor remark diagnostic emission in a RemarkStreamerFrancis Visoiu Mistrih2019-03-061-2/+3
| | | | | | | | | | This allows us to store more info about where we're emitting the remarks without cluttering LLVMContext. This is needed for future support for the remark section. Differential Revision: https://reviews.llvm.org/D58996 llvm-svn: 355507
* [ThinLTO] Detect partially split modules during the thin linkTeresa Johnson2019-02-141-5/+51
| | | | | | | | | | | | | | | | | | | | | | | | Summary: The changes to disable LTO unit splitting by default (r350949) and detect inconsistently split LTO units (r350948) are causing some crashes when the inconsistency is detected in multiple threads simultaneously. Fix that by having the code always look for the inconsistently split LTO units during the thin link, by checking for the presence of type tests recorded in the summaries. Modify test added in r350948 to remove single threading required to fix a bot failure due to this issue (and some debugging options added in the process of diagnosing it). Reviewers: pcc Subscribers: mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D57561 llvm-svn: 354062
* Update the file headers across all of the LLVM projects in the monorepoChandler Carruth2019-01-191-4/+3
| | | | | | | | | | | | | | | | | to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
* [LTO] Record whether LTOUnit splitting is enabled in indexTeresa Johnson2019-01-111-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Records in the module summary index whether the bitcode was compiled with the option necessary to enable splitting the LTO unit (e.g. -fsanitize=cfi, -fwhole-program-vtables, or -fsplit-lto-unit). The information is passed down to the ModuleSummaryIndex builder via a new module flag "EnableSplitLTOUnit", which is propagated onto a flag on the summary index. This is then used during the LTO link to check whether all linked summaries were built with the same value of this flag. If not, an error is issued when we detect a situation requiring whole program visibility of the class hierarchy. This is the case when both of the following conditions are met: 1) We are performing LowerTypeTests or Whole Program Devirtualization. 2) There are type tests or type checked loads in the code. Note I have also changed the ThinLTOBitcodeWriter to also gate the module splitting on the value of this flag. Reviewers: pcc Subscribers: ormris, mehdi_amini, Prazek, inglorion, eraman, steven_wu, dexonsmith, arphaman, dang, llvm-commits Differential Revision: https://reviews.llvm.org/D53890 llvm-svn: 350948
* [ThinLTO] Remove dllimport attribute from locally defined symbolsMatthew Voss2018-12-191-1/+5
| | | | | | | | | | | | | | | | | | | | Summary: The LTO/ThinLTO driver currently creates invalid bitcode by setting symbols marked dllimport as dso_local. The compiler often has access to the definition (often dllexport) and the declaration (often dllimport) of an object at link-time, leading to a conflicting declaration. This patch resolves the inconsistency by removing the dllimport attribute. Reviewers: tejohnson, pcc, rnk, echristo Reviewed By: rnk Subscribers: dmikulin, wristow, mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, dang, llvm-commits Differential Revision: https://reviews.llvm.org/D55627 llvm-svn: 349667
* [ThinLTO] Compute synthetic function entry countEaswaran Raman2018-12-131-0/+5
| | | | | | | | | | | | | | | | | Summary: This patch computes the synthetic function entry count on the whole program callgraph (based on module summary) and writes the entry counts to the summary. After function importing, this count gets attached to the IR as metadata. Since it adds a new field to the summary, this bumps up the version. Reviewers: tejohnson Subscribers: mehdi_amini, inglorion, llvm-commits Differential Revision: https://reviews.llvm.org/D43521 llvm-svn: 349076
* LTO: Don't internalize available_externally globals.Peter Collingbourne2018-12-051-1/+4
| | | | | | | | | | This breaks C and C++ semantics because it can cause the address of the global inside the module to differ from the address outside of the module. Differential Revision: https://reviews.llvm.org/D55237 llvm-svn: 348321
* [ThinLTO] Look through aliases when computing hash keysGeorge Burgess IV2018-12-041-2/+8
| | | | | | | | | | | Without this, we don't consider types used by aliasees in our cache key. This caused issues when using the same cache for thin-linking the same TU with different sets of virtual call candidates for a virtual call inside of a constructor. That's sort of a mouthful. :) Differential Revision: https://reviews.llvm.org/D55060 llvm-svn: 348216
* [ThinLTO] Consolidate cache key computation between new/old LTO APIsTeresa Johnson2018-11-261-5/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The old legacy LTO API had a separate cache key computation, which was a subset of the cache key computation in the new LTO API (from what I can tell this is largely just because certain features such as CFI, dsoLocal, etc are only utilized via the new LTO API). However, having separate computations is unnecessary (much of the code is duplicated), and can lead to bugs when adding new optimizations if both cache computation algorithms aren't updated properly - it's much easier to maintain if we have a single facility. This patch refactors the old LTO API code to use the cache key computation from the new LTO API. To do this, we set up an lto::Config object and fill in the fields that the old LTO was hashing (the others will just use the defaults). There are two notable changes: - I added a Freestanding flag to the LTO Config. Currently this is only used by the legacy LTO API. In the patch that added it (D30791) I had asked about adding it to the new LTO API, but it looks like that was not addressed. This should probably be discussed as a follow up to this change, as it is orthogonal. - The legacy LTO API had some code that was hashing the GUID of all preserved symbols defined in the module. I looked back at the history of this (which was added with the original hashing in the legacy LTO API in D18494), and there is a comment in the review thread that it was added in preparation for future internalization. We now do the internalization of course, and that is handled in the new LTO API cache key computation by hashing the recorded linkage type of all defined globals. Therefore I didn't try to move over and keep the preserved symbols handling. Reviewers: steven_wu, pcc Subscribers: mehdi_amini, inglorion, eraman, dexonsmith, dang, llvm-commits Differential Revision: https://reviews.llvm.org/D54635 llvm-svn: 347592
* [ThinLTO] Internalize readonly globalsEugene Leviant2018-11-161-1/+4
| | | | | | | | An attempt to recommit r346584 after failure on OSX build bot. Fixed cache key computation in ThinLTOCodeGenerator and added test case llvm-svn: 347033
* Revert "[ThinLTO] Internalize readonly globals"Steven Wu2018-11-131-4/+1
| | | | | | This reverts commit 10c84a8f35cae4a9fc421648d9608fccda3925f2. llvm-svn: 346768
* [ThinLTO] Internalize readonly globalsEugene Leviant2018-11-101-1/+4
| | | | | | | | | This patch allows internalising globals if all accesses to them (from live functions) are from non-volatile load instructions Differential revision: https://reviews.llvm.org/D49362 llvm-svn: 346584
* [LTO] Drop non-prevailing definitions only if linkage is not local or appendingPirama Arumuga Nainar2018-11-081-10/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This fixes PR 37422 In ELF, non-weak symbols can also be non-prevailing. In this particular PR, the __llvm_profile_* symbols are non-prevailing but weren't getting dropped - causing multiply-defined errors with lld. Also add a test, strong_non_prevailing.ll, to ensure that multiple copies of a strong symbol are dropped. To fix the test regressions exposed by this fix, - do not mark prevailing copies for symbols with 'appending' linkage. There's no one prevailing copy for such symbols. - fix the prevailing version in dead-strip-fulllto.ll - explicitly pass exported symbols to llvm-lto in fumcimport.ll and funcimport_var.ll Reviewers: tejohnson, pcc Subscribers: mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, dang, srhines, llvm-commits Differential Revision: https://reviews.llvm.org/D54125 llvm-svn: 346436
* [ThinLTO] Add an option to disable (thin)lto internalization.Xin Tong2018-11-051-2/+8
| | | | | | | | | | | | | | | | | | | | | | Summary: LTO and ThinLTO optimizes the IR differently. One source of differences is the amount of internalizations that can happen. Add an option to enable/disable internalization so that other differences can be studied in isolation. e.g. inlining. There are other things lto and thinlto do differently, I will add flags to enable/disable them as needed. Reviewers: tejohnson, pcc, steven_wu Subscribers: mehdi_amini, inglorion, steven_wu, dexonsmith, dang, llvm-commits Differential Revision: https://reviews.llvm.org/D53294 llvm-svn: 346140
* Add a flag to remap manglings when reading profile data information.Richard Smith2018-10-101-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This can be used to preserve profiling information across codebase changes that have widespread impact on mangled names, but across which most profiling data should still be usable. For example, when switching from libstdc++ to libc++, or from the old libstdc++ ABI to the new ABI, or even from a 32-bit to a 64-bit build. The user can provide a remapping file specifying parts of mangled names that should be treated as equivalent (eg, std::__1 should be treated as equivalent to std::__cxx11), and profile data will be treated as applying to a particular function if its name is equivalent to the name of a function in the profile data under the provided equivalences. See the documentation change for a description of how this is configured. Remapping is supported for both sample-based profiling and instruction profiling. We do not support remapping indirect branch target information, but all other profile data should be remapped appropriately. Support is only added for the new pass manager. If someone wants to also add support for this for the old pass manager, doing so should be straightforward. This is the LLVM side of Clang r344199. Reviewers: davidxl, tejohnson, dlj, erik.pilkington Subscribers: mehdi_amini, steven_wu, dexonsmith, llvm-commits Differential Revision: https://reviews.llvm.org/D51249 llvm-svn: 344200
* [ThinLTO] Efficiency fix for writing type id records in per-module indexesTeresa Johnson2018-09-251-27/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: In D49565/r337503, the type id record writing was fixed so that only referenced type ids were emitted into each per-module index for ThinLTO distributed builds. However, this still left an efficiency issue: each per-module index checked all type ids for membership in the referenced set, yielding O(M*N) performance (M indexes and N type ids). Change the TypeIdMap in the summary to be indexed by GUID, to facilitate correlating with type identifier GUIDs referenced in the function summary TypeIdInfo structures. This allowed simplifying other places where a map from type id GUID to type id map entry was previously being used to aid this correlation. Also fix AsmWriter code to handle the rare case of type id GUID collision. For a large internal application, this reduced the thin link time by almost 15%. Reviewers: pcc, vitalybuka Subscribers: mehdi_amini, inglorion, steven_wu, dexonsmith, llvm-commits Differential Revision: https://reviews.llvm.org/D51330 llvm-svn: 343021
* [LTO] Don't internalize declarationsBob Haarman2018-07-271-1/+2
| | | | | | | | | | | | | | | | | | | | Summary: Some links were failing with "Global is external, but doesn't have external or weak linkage!" in ThinLTO builds with debug information. This happened when we elide the body of a global that is referenced by debug info. This results in a declaration, which we would then internalize - but declarations cannot be internal. This change avoids the problem by not internalizing these declarations. Fixes PR38046. Reviewers: pcc, tejohnson Subscribers: mehdi_amini, aprantl, hiraditya, JDevlieghere, steven_wu, dexonsmith, llvm-commits Differential Revision: https://reviews.llvm.org/D49777 llvm-svn: 338100
* [LTO] Handle __imp_ (dllimport) symbols consistently with lldTeresa Johnson2018-07-231-1/+8
| | | | | | | | | | | | | | | | | | | Summary: Similar to what lld already does for dllimport symbols which are prefaced with __imp_ (see lld patch r240620), strip off the __imp_ prefix in LTO. Otherwise we can get 2 separate GlobalResolution for a single symbol, the dllimport declaration, and the definition, which leads to incorrect LTO handling. Fixes PR38105. Reviewers: pcc Subscribers: mehdi_amini, inglorion, steven_wu, dexonsmith, llvm-commits Differential Revision: https://reviews.llvm.org/D49138 llvm-svn: 337762
* Restore "[ThinLTO] Ensure we always select the same function copy to import"Teresa Johnson2018-07-161-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit r337081, therefore restoring r337050 (and fix in r337059), with test fix for bot failure described after the original description below. In order to always import the same copy of a linkonce function, even when encountering it with different thresholds (a higher one then a lower one), keep track of the summary we decided to import. This ensures that the backend only gets a single definition to import for each GUID, so that it doesn't need to choose one. Move the largest threshold the GUID was considered for import into the current module out of the ImportMap (which is part of a larger map maintained across the whole index), and into a new map just maintained for the current module we are computing imports for. This saves some memory since we no longer have the thresholds maintained across the whole index (and throughout the in-process backends when doing a normal non-distributed ThinLTO build), at the cost of some additional information being maintained for each invocation of ComputeImportForModule (the selected summary pointer for each import). There is an additional map lookup for each callee being considered for importing, however, this was able to subsume a map lookup in the Worklist iteration that invokes computeImportForFunction. We also are able to avoid calling selectCallee if we already failed to import at the same or higher threshold. I compared the run time and peak memory for the SPEC2006 471.omnetpp benchmark (running in-process ThinLTO backends), as well as for a large internal benchmark with a distributed ThinLTO build (so just looking at the thin link time/memory). Across a number of runs with and without this change there was no significant change in the time and memory. (I tried a few other variations of the change but they also didn't improve time or peak memory). The new commit removes a test that no longer makes sense (Transforms/FunctionImport/hotness_based_import2.ll), as exposed by the reverse-iteration bot. The test depends on the order of processing the summary call edges, and actually depended on the old problematic behavior of selecting more than one summary for a given GUID when encountered with different thresholds. There was no guarantee even before that we would eventually pick the linkonce copy with the hottest call edges, it just happened to work with the test and the old code, and there was no guarantee that we would end up importing the selected version of the copy that had the hottest call edges (since the backend would effectively import only one of the selected copies). Reviewers: davidxl Subscribers: mehdi_amini, inglorion, llvm-commits Differential Revision: https://reviews.llvm.org/D48670 llvm-svn: 337184
* Revert "[ThinLTO] Ensure we always select the same function copy to import"Teresa Johnson2018-07-141-2/+2
| | | | | | | This reverts commits r337050 and r337059. Caused failure in reverse-iteration bot that needs more investigation. llvm-svn: 337081
* [ThinLTO] Ensure we always select the same function copy to importTeresa Johnson2018-07-131-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In order to always import the same copy of a linkonce function, even when encountering it with different thresholds (a higher one then a lower one), keep track of the summary we decided to import. This ensures that the backend only gets a single definition to import for each GUID, so that it doesn't need to choose one. Move the largest threshold the GUID was considered for import into the current module out of the ImportMap (which is part of a larger map maintained across the whole index), and into a new map just maintained for the current module we are computing imports for. This saves some memory since we no longer have the thresholds maintained across the whole index (and throughout the in-process backends when doing a normal non-distributed ThinLTO build), at the cost of some additional information being maintained for each invocation of ComputeImportForModule (the selected summary pointer for each import). There is an additional map lookup for each callee being considered for importing, however, this was able to subsume a map lookup in the Worklist iteration that invokes computeImportForFunction. We also are able to avoid calling selectCallee if we already failed to import at the same or higher threshold. I compared the run time and peak memory for the SPEC2006 471.omnetpp benchmark (running in-process ThinLTO backends), as well as for a large internal benchmark with a distributed ThinLTO build (so just looking at the thin link time/memory). Across a number of runs with and without this change there was no significant change in the time and memory. (I tried a few other variations of the change but they also didn't improve time or peak memory). Reviewers: davidxl Subscribers: mehdi_amini, inglorion, llvm-commits Differential Revision: https://reviews.llvm.org/D48670 llvm-svn: 337050
* [ThinLTO] Use std::map to get determistic imports filesTeresa Johnson2018-07-101-1/+2
| | | | | | | | | | | | | | | | | | | | | | Summary: I noticed that the .imports files emitted for distributed ThinLTO backends do not have consistent ordering. This is because StringMap iteration order is not guaranteed to be deterministic. Since we already have a std::map with this information, used when emitting the individual index files (ModuleToSummariesForIndex), use it for the imports files as well. This issue is likely causing some unnecessary rebuilds of the ThinLTO backends in our distributed build system as the imports files are inputs to those backends. Reviewers: pcc, steven_wu, mehdi_amini Subscribers: mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, llvm-commits Differential Revision: https://reviews.llvm.org/D48783 llvm-svn: 336721
* [ThinLTO] Rename index IsAnalysis flag to HaveGVs (NFC)Teresa Johnson2018-06-061-1/+1
| | | | | | | | | With the upcoming patch to add summary parsing support, IsAnalysis would be true in contexts where we are not performing module summary analysis. Rename to the more specific and approprate HaveGVs, which is essentially what this flag is indicating. llvm-svn: 334140
* [LTO] Allow pass remarks with hotness to be set when emitting to stderrTeresa Johnson2018-05-041-2/+2
| | | | | | | | | | | | | | | | | | | | | Summary: Set setDiagnosticsHotnessRequested before the early exit check for a diagnostic output file, so that pass remarks with hotness works when emitting pass remarks to stderr (e.g. via -pass-remarks=.). Also fix the llvm-lto2 diagnistic handler so that it only calls exit(1) when the diagnistic is an error type. Otherwise the new test invocation of llvm-lto2 with -pass-remarks causes it to fail. The new code is consistent with the diagnostic handler elsewhere (e.g. on the LLVMContext). Reviewers: pcc, davide Subscribers: fhahn, mehdi_amini, llvm-commits, inglorion Differential Revision: https://reviews.llvm.org/D46387 llvm-svn: 331569
OpenPOWER on IntegriCloud