summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/IPO/WholeProgramDevirt.cpp
Commit message (Collapse)AuthorAgeFilesLines
* [ThinLTO] Detect partially split modules during the thin linkTeresa Johnson2019-02-141-11/+8
| | | | | | | | | | | | | | | | | | | | | | | | Summary: The changes to disable LTO unit splitting by default (r350949) and detect inconsistently split LTO units (r350948) are causing some crashes when the inconsistency is detected in multiple threads simultaneously. Fix that by having the code always look for the inconsistently split LTO units during the thin link, by checking for the presence of type tests recorded in the summaries. Modify test added in r350948 to remove single threading required to fix a bot failure due to this issue (and some debugging options added in the process of diagnosing it). Reviewers: pcc Subscribers: mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D57561 llvm-svn: 354062
* [opaque pointer types] Pass value type to LoadInst creation.James Y Knight2019-02-011-1/+1
| | | | | | | | | This cleans up all LoadInst creation in LLVM to explicitly pass the value type rather than deriving it from the pointer's element-type. Differential Revision: https://reviews.llvm.org/D57172 llvm-svn: 352911
* [opaque pointer types] Pass function types to InvokeInst creation.James Y Knight2019-02-011-1/+1
| | | | | | | | | This cleans up all InvokeInst creation in LLVM to explicitly pass a function type rather than deriving it from the pointer's element-type. Differential Revision: https://reviews.llvm.org/D57171 llvm-svn: 352910
* [opaque pointer types] Pass function types to CallInst creation.James Y Knight2019-02-011-5/+6
| | | | | | | | | This cleans up all CallInst creation in LLVM to explicitly pass a function type rather than deriving it from the pointer's element-type. Differential Revision: https://reviews.llvm.org/D57170 llvm-svn: 352909
* [opaque pointer types] Add a FunctionCallee wrapper type, and use it.James Y Knight2019-02-011-4/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Recommit r352791 after tweaking DerivedTypes.h slightly, so that gcc doesn't choke on it, hopefully. Original Message: The FunctionCallee type is effectively a {FunctionType*,Value*} pair, and is a useful convenience to enable code to continue passing the result of getOrInsertFunction() through to EmitCall, even once pointer types lose their pointee-type. Then: - update the CallInst/InvokeInst instruction creation functions to take a Callee, - modify getOrInsertFunction to return FunctionCallee, and - update all callers appropriately. One area of particular note is the change to the sanitizer code. Previously, they had been casting the result of `getOrInsertFunction` to a `Function*` via `checkSanitizerInterfaceFunction`, and storing that. That would report an error if someone had already inserted a function declaraction with a mismatching signature. However, in general, LLVM allows for such mismatches, as `getOrInsertFunction` will automatically insert a bitcast if needed. As part of this cleanup, cause the sanitizer code to do the same. (It will call its functions using the expected signature, however they may have been declared.) Finally, in a small number of locations, callers of `getOrInsertFunction` actually were expecting/requiring that a brand new function was being created. In such cases, I've switched them to Function::Create instead. Differential Revision: https://reviews.llvm.org/D57315 llvm-svn: 352827
* Revert "[opaque pointer types] Add a FunctionCallee wrapper type, and use it."James Y Knight2019-01-311-10/+4
| | | | | | | | | This reverts commit f47d6b38c7a61d50db4566b02719de05492dcef1 (r352791). Seems to run into compilation failures with GCC (but not clang, where I tested it). Reverting while I investigate. llvm-svn: 352800
* [opaque pointer types] Add a FunctionCallee wrapper type, and use it.James Y Knight2019-01-311-4/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The FunctionCallee type is effectively a {FunctionType*,Value*} pair, and is a useful convenience to enable code to continue passing the result of getOrInsertFunction() through to EmitCall, even once pointer types lose their pointee-type. Then: - update the CallInst/InvokeInst instruction creation functions to take a Callee, - modify getOrInsertFunction to return FunctionCallee, and - update all callers appropriately. One area of particular note is the change to the sanitizer code. Previously, they had been casting the result of `getOrInsertFunction` to a `Function*` via `checkSanitizerInterfaceFunction`, and storing that. That would report an error if someone had already inserted a function declaraction with a mismatching signature. However, in general, LLVM allows for such mismatches, as `getOrInsertFunction` will automatically insert a bitcast if needed. As part of this cleanup, cause the sanitizer code to do the same. (It will call its functions using the expected signature, however they may have been declared.) Finally, in a small number of locations, callers of `getOrInsertFunction` actually were expecting/requiring that a brand new function was being created. In such cases, I've switched them to Function::Create instead. Differential Revision: https://reviews.llvm.org/D57315 llvm-svn: 352791
* Update the file headers across all of the LLVM projects in the monorepoChandler Carruth2019-01-191-4/+3
| | | | | | | | | | | | | | | | | to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
* [LTO] Record whether LTOUnit splitting is enabled in indexTeresa Johnson2019-01-111-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Records in the module summary index whether the bitcode was compiled with the option necessary to enable splitting the LTO unit (e.g. -fsanitize=cfi, -fwhole-program-vtables, or -fsplit-lto-unit). The information is passed down to the ModuleSummaryIndex builder via a new module flag "EnableSplitLTOUnit", which is propagated onto a flag on the summary index. This is then used during the LTO link to check whether all linked summaries were built with the same value of this flag. If not, an error is issued when we detect a situation requiring whole program visibility of the class hierarchy. This is the case when both of the following conditions are met: 1) We are performing LowerTypeTests or Whole Program Devirtualization. 2) There are type tests or type checked loads in the code. Note I have also changed the ThinLTOBitcodeWriter to also gate the module splitting on the value of this flag. Reviewers: pcc Subscribers: ormris, mehdi_amini, Prazek, inglorion, eraman, steven_wu, dexonsmith, arphaman, dang, llvm-commits Differential Revision: https://reviews.llvm.org/D53890 llvm-svn: 350948
* [IPO][AVR] Create new Functions in the default address space specified in ↵Dylan McKay2018-12-181-1/+4
| | | | | | | | | | | | | | | | | | | | | the data layout This modifies the IPO pass so that it respects any explicit function address space specified in the data layout. In targets with nonzero program address spaces, all functions should, by default, be placed into the default program address space. This is required for Harvard architectures like AVR. Without this, the functions will be marked as residing in data space, and thus not be callable. This has no effect to any in-tree official backends, as none use an explicit program address space in their data layouts. Patch by Tim Neumann. llvm-svn: 349469
* [WPD] Fix incorrect devirtualization after indirect call promotionTeresa Johnson2018-09-271-20/+41
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: Add a dominance check to ensure that the possible devirtualizable call is actually dominated by the type test/checked load intrinsic being analyzed. With PGO, after indirect call promotion is performed during the compile step, followed by inlining, we may have a type test in the promoted and inlined sequence that allows an indirect call in that sequence to be devirtualized. That indirect call (inserted by inlining after promotion) will share the same vtable pointer as the fallback indirect call that cannot be devirtualized. Before this patch the code was incorrectly devirtualizing the fallback indirect call. See the new test and the example described there for more details. Reviewers: pcc, vitalybuka Subscribers: mehdi_amini, Prazek, eraman, steven_wu, dexonsmith, llvm-commits Differential Revision: https://reviews.llvm.org/D52514 llvm-svn: 343226
* [WholeProgramDevirt] Don't process declarations when building type id mapEugene Leviant2018-09-231-1/+1
| | | | | | Differential revision: https://reviews.llvm.org/D52175 llvm-svn: 342836
* [LTO] Make detection of WPD remark enablement more robustTeresa Johnson2018-09-181-9/+8
| | | | | | | | | | | | | | | | Summary: Currently only the first function in the module is checked to see if it has remarks enabled. If that first function is a declaration, remarks will be incorrectly skipped. Change to look for the first non-empty function. Reviewers: pcc Subscribers: mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, llvm-commits Differential Revision: https://reviews.llvm.org/D51556 llvm-svn: 342477
* [ThinLTO] Fix printing of WPD remarksTeresa Johnson2018-08-141-2/+4
| | | | | | | | | | | | | | | | | Summary: When WPD is performed in a ThinLTO backend, the function may be created if it isn't already in that module. Module::getOrInsertFunction may add a bitcast, in which case the returned Constant is not a Function and doesn't have a name. Invoke stripPointerCasts() on the returned value where we access its name. Reviewers: pcc Subscribers: mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, llvm-commits Differential Revision: https://reviews.llvm.org/D49959 llvm-svn: 339640
* [ThinLTO] Enable ThinLTO WholeProgramDevirt and LowerTypeTests in new PMTeresa Johnson2018-07-191-1/+2
| | | | | | | | | | | | | | | | Summary: Enable these passes for CFI and WPD in ThinLTO and LTO with the new pass manager. Add a couple of tests for both PMs based on the clang tests tools/clang/test/CodeGen/thinlto-distributed-cfi*.ll, but just test through llvm-lto2 and not with distributed ThinLTO. Reviewers: pcc Subscribers: mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, llvm-commits Differential Revision: https://reviews.llvm.org/D49429 llvm-svn: 337461
* [ThinLTO] Rename index IsAnalysis flag to HaveGVs (NFC)Teresa Johnson2018-06-061-1/+1
| | | | | | | | | With the upcoming patch to add summary parsing support, IsAnalysis would be true in contexts where we are not performing module summary analysis. Rename to the more specific and approprate HaveGVs, which is essentially what this flag is indicating. llvm-svn: 334140
* Extend the GlobalObject metadata interfaceBenjamin Kramer2018-05-311-1/+1
| | | | | | | | | | | | - Make eraseMetadata return whether it changed something - Wire getMetadata for a single MDNode efficiently into the attachment map - Add hasMetadata, which is less weird than checking getMetadata == nullptr on a multimap. Use it to simplify code. llvm-svn: 333649
* Fix warning by cl::opt<int> -> cl::opt<unsigned>Vitaly Buka2018-04-061-4/+5
| | | | llvm-svn: 329461
* Runtime flag to control branch funnel thresholdVitaly Buka2018-04-061-2/+6
| | | | | | | | | | Reviewers: pcc Subscribers: hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D45193 llvm-svn: 329459
* Use branch funnels for virtual calls when retpoline mitigation is enabled.Peter Collingbourne2018-03-091-22/+189
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The retpoline mitigation for variant 2 of CVE-2017-5715 inhibits the branch predictor, and as a result it can lead to a measurable loss of performance. We can reduce the performance impact of retpolined virtual calls by replacing them with a special construct known as a branch funnel, which is an instruction sequence that implements virtual calls to a set of known targets using a binary tree of direct branches. This allows the processor to speculately execute valid implementations of the virtual function without allowing for speculative execution of of calls to arbitrary addresses. This patch extends the whole-program devirtualization pass to replace certain virtual calls with calls to branch funnels, which are represented using a new llvm.icall.jumptable intrinsic. It also extends the LowerTypeTests pass to recognize the new intrinsic, generate code for the branch funnels (x86_64 only for now) and lay out virtual tables as required for each branch funnel. The implementation supports full LTO as well as ThinLTO, and extends the ThinLTO summary format used for whole-program devirtualization to support branch funnels. For more details see RFC: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120672.html Differential Revision: https://reviews.llvm.org/D42453 llvm-svn: 327163
* [ThinLTO] Re-commit of dot dumper after test fixEugene Leviant2018-01-221-1/+1
| | | | llvm-svn: 323116
* Temporarily revert r323062 to investigate buildbot failuresEugene Leviant2018-01-211-1/+1
| | | | llvm-svn: 323065
* [ThinLTO] Implement summary visualizerEugene Leviant2018-01-211-1/+1
| | | | | | Differential revision: https://reviews.llvm.org/D41297 llvm-svn: 323062
* WholeProgramDevirt: Simplify ORE getter mechanism for old PM. NFCI.Peter Collingbourne2018-01-051-34/+17
| | | | llvm-svn: 321841
* Silence a bunch of implicit fallthrough warningsAdrian Prantl2017-12-191-0/+1
| | | | llvm-svn: 321114
* Remove redundant includes from lib/Transforms.Michael Zolotukhin2017-12-131-1/+0
| | | | llvm-svn: 320628
* Rename OptimizationDiagnosticInfo.* to OptimizationRemarkEmitter.*Adam Nemet2017-10-091-1/+1
| | | | | | | Sync it up with the name of the class actually defined here. This has been bothering me for a while... llvm-svn: 315249
* WholeProgramDevirt: Add import/export support for targets without absolute ↵Peter Collingbourne2017-09-111-16/+57
| | | | | | | | | | | | | | | | | | | symbol constants. Not all targets support the use of absolute symbols to export constants. In particular, ARM has a wide variety of constant encodings that cannot currently be relocated by linkers. So instead of exporting the constants using symbols, export them directly in the summary. The values of the constants are left as zeroes on targets that support symbolic exports. This may result in more cache misses when targeting those architectures as a result of arbitrary changes in constant values, but this seems somewhat unavoidable for now. Differential Revision: https://reviews.llvm.org/D37407 llvm-svn: 312967
* WholeProgramDevirt: When promoting for single-impl devirt, also rename the ↵Peter Collingbourne2017-09-081-1/+16
| | | | | | | | | | | comdat. This is required when targeting COFF, as the comdat name must match one of the names of the symbols in the comdat. Differential Revision: https://reviews.llvm.org/D37550 llvm-svn: 312767
* WholeProgramDevirt: Create bitcast to i8* at each virtual call site.Peter Collingbourne2017-08-221-5/+6
| | | | | | | | | We can't reuse the llvm.assume instruction's bitcast because it may not dominate every user of the vtable pointer. Differential Revision: https://reviews.llvm.org/D36994 llvm-svn: 311491
* Migrate WholeProgramDevirt to new Optimization Remark APISam Elliott2017-08-211-25/+75
| | | | | | | | | | | | | | | | | Summary: This is an attempt to move WholeProgramDevirt to the new remark API. https://bugs.llvm.org/show_bug.cgi?id=33793 Reviewers: anemet Reviewed By: anemet Subscribers: fhahn, llvm-commits Differential Revision: https://reviews.llvm.org/D36943 llvm-svn: 311352
* Sort the remaining #include lines in include/... and lib/....Chandler Carruth2017-06-061-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | I did this a long time ago with a janky python script, but now clang-format has built-in support for this. I fed clang-format every line with a #include and let it re-sort things according to the precise LLVM rules for include ordering baked into clang-format these days. I've reverted a number of files where the results of sorting includes isn't healthy. Either places where we have legacy code relying on particular include ordering (where possible, I'll fix these separately) or where we have particular formatting around #include lines that I didn't want to disturb in this patch. This patch is *entirely* mechanical. If you get merge conflicts or anything, just ignore the changes in this patch and run clang-format over your #include lines in the files. Sorry for any noise here, but it is important to keep these things stable. I was seeing an increasing number of patches with irrelevant re-ordering of #include lines because clang-format was used. This patch at least isolates that churn, makes it easy to skip when resolving conflicts, and gets us to a clean baseline (again). llvm-svn: 304787
* Re-apply r302108, "IR: Use pointers instead of GUIDs to represent edges in ↵Peter Collingbourne2017-05-041-1/+1
| | | | | | | | the module summary. NFCI." with a fix for the clang backend. llvm-svn: 302176
* Revert "IR: Use pointers instead of GUIDs to represent edges in the module ↵Eric Liu2017-05-041-1/+1
| | | | | | | | | | summary. NFCI." This reverts commit r302108. This causes crash in clang bootstrap with LTO. Contacted the auther in the original commit. llvm-svn: 302140
* IR: Use pointers instead of GUIDs to represent edges in the module summary. ↵Peter Collingbourne2017-05-041-1/+1
| | | | | | | | | | | | | | | | | | | | NFCI. When profiling a no-op incremental link of Chromium I found that the functions computeImportForFunction and computeDeadSymbols were consuming roughly 10% of the profile. The goal of this change is to improve the performance of those functions by changing the map lookups that they were previously doing into pointer dereferences. This is achieved by changing the ValueInfo data structure to be a pointer to an element of the global value map owned by ModuleSummaryIndex, and changing reference lists in the GlobalValueSummary to hold ValueInfos instead of GUIDs. This means that a ValueInfo will take a client directly to the summary list for a given GUID. Differential Revision: https://reviews.llvm.org/D32471 llvm-svn: 302108
* Module::getOrInsertFunction is using C-style vararg instead of variadic ↵Serge Guelton2017-04-111-1/+1
| | | | | | | | | | | templates. From a user prospective, it forces the use of an annoying nullptr to mark the end of the vararg, and there's not type checking on the arguments. The variadic template is an obvious solution to both issues. Differential Revision: https://reviews.llvm.org/D31070 llvm-svn: 299949
* Revert "Turn some C-style vararg into variadic templates"Diana Picus2017-04-111-1/+1
| | | | | | | This reverts commit r299925 because it broke the buildbots. See e.g. http://lab.llvm.org:8011/builders/clang-cmake-armv7-a15/builds/6008 llvm-svn: 299928
* Turn some C-style vararg into variadic templatesSerge Guelton2017-04-111-1/+1
| | | | | | | | | | | | Module::getOrInsertFunction is using C-style vararg instead of variadic templates. From a user prospective, it forces the use of an annoying nullptr to mark the end of the vararg, and there's not type checking on the arguments. The variadic template is an obvious solution to both issues. llvm-svn: 299925
* Revert "Turn some C-style vararg into variadic templates"Mehdi Amini2017-04-061-2/+2
| | | | | | This reverts commit r299699, the examples needs to be updated. llvm-svn: 299702
* Turn some C-style vararg into variadic templatesMehdi Amini2017-04-061-2/+2
| | | | | | | | | | | | | | | | Module::getOrInsertFunction is using C-style vararg instead of variadic templates. From a user prospective, it forces the use of an annoying nullptr to mark the end of the vararg, and there's not type checking on the arguments. The variadic template is an obvious solution to both issues. Patch by: Serge Guelton <serge.guelton@telecom-bretagne.eu> Differential Revision: https://reviews.llvm.org/D31070 llvm-svn: 299699
* IPO: Const correctness for summaries passed into passes.Peter Collingbourne2017-03-221-24/+36
| | | | | | | | | Pass const qualified summaries into importers and unqualified summaries into exporters. This lets us const-qualify the summary argument to thinBackend. Differential Revision: https://reviews.llvm.org/D31230 llvm-svn: 298534
* IR: Fix a race condition in type id clients of ModuleSummaryIndex.Peter Collingbourne2017-03-221-7/+12
| | | | | | | | | Add a const version of the getTypeIdSummary accessor that avoids mutating the TypeIdMap. Differential Revision: https://reviews.llvm.org/D31226 llvm-svn: 298531
* WholeProgramDevirt: Implement export/import support for VCP.Peter Collingbourne2017-03-101-4/+36
| | | | | | Differential Revision: https://reviews.llvm.org/D30017 llvm-svn: 297503
* WholeProgramDevirt: Implement export/import support for unique ret val opt.Peter Collingbourne2017-03-101-13/+80
| | | | | | Differential Revision: https://reviews.llvm.org/D29917 llvm-svn: 297502
* WholeProgramDevirt: Fixed compilation error under MSVS2015.George Rimar2017-03-101-9/+18
| | | | | | | | | | | | | | | | | | | | | | It was introduced in: r296945 WholeProgramDevirt: Implement exporting for single-impl devirtualization. --------------------- r296939 WholeProgramDevirt: Add any unsuccessful llvm.type.checked.load devirtualizations to the list of llvm.type.test users. --------------------- Microsoft Visual Studio Community 2015 Version 14.0.23107.0 D14REL Does not compile that code without additional brackets, showing multiple error like below: WholeProgramDevirt.cpp(1216): error C2958: the left bracket '[' found at 'c:\access_softek\llvm\lib\transforms\ipo\wholeprogramdevirt.cpp(1216)' was not matched correctly WholeProgramDevirt.cpp(1216): error C2143: syntax error: missing ']' before '}' WholeProgramDevirt.cpp(1216): error C2143: syntax error: missing ';' before '}' WholeProgramDevirt.cpp(1216): error C2059: syntax error: ']' llvm-svn: 297451
* WholeProgramDevirt: Implement importing for uniform ret val opt.Peter Collingbourne2017-03-091-0/+18
| | | | | | Differential Revision: https://reviews.llvm.org/D29854 llvm-svn: 297350
* WholeProgramDevirt: Implement importing for single-impl devirtualization.Peter Collingbourne2017-03-091-11/+47
| | | | | | Differential Revision: https://reviews.llvm.org/D29844 llvm-svn: 297333
* Fix build.Peter Collingbourne2017-03-041-1/+1
| | | | llvm-svn: 296949
* WholeProgramDevirt: Implement exporting for uniform ret val opt.Peter Collingbourne2017-03-041-6/+19
| | | | | | Differential Revision: https://reviews.llvm.org/D29846 llvm-svn: 296948
* WholeProgramDevirt: Implement exporting for single-impl devirtualization.Peter Collingbourne2017-03-041-6/+54
| | | | | | Differential Revision: https://reviews.llvm.org/D29811 llvm-svn: 296945
OpenPOWER on IntegriCloud