summaryrefslogtreecommitdiffstats
path: root/llvm/test/Transforms/WholeProgramDevirt
Commit message (Collapse)AuthorAgeFilesLines
* IR: print value numbers for unnamed function argumentsTim Northover2019-08-032-4/+4
| | | | | | | | | | For consistency with normal instructions and clarity when reading IR, it's best to print the %0, %1, ... names of function arguments in definitions. Also modifies the parser to accept IR in that form for obvious reasons. llvm-svn: 367755
* WholeProgramDevirt: Teach the pass to respect the global's alignment.Peter Collingbourne2019-07-222-14/+16
| | | | | | | | | | | | | | | | | | | The bytes inserted before an overaligned global need to be padded according to the alignment set on the original global in order for the initializer to meet the global's alignment requirements. The previous implementation that padded to the pointer width happened to be correct for vtables on most platforms but may do the wrong thing if the vtable has a larger alignment. This issue is visible with a prototype implementation of HWASAN for globals, which will overalign all globals including vtables to 16 bytes. There is also no padding requirement for the bytes inserted after the global because they are never read from nor are they significant for alignment purposes, so stop inserting padding there. Differential Revision: https://reviews.llvm.org/D65031 llvm-svn: 366725
* [ThinLTO] Auto-hide prevailing linkonce_odr only when all copies eligibleTeresa Johnson2019-05-101-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: We hit undefined references building with ThinLTO when one source file contained explicit instantiations of a template method (weak_odr) but there were also implicit instantiations in another file (linkonce_odr), and the latter was the prevailing copy. In this case the symbol was marked hidden when the prevailing linkonce_odr copy was promoted to weak_odr. It led to unsats when the resulting shared library was linked with other code that contained a reference (expecting to be resolved due to the explicit instantiation). Add a CanAutoHide flag to the GV summary to allow the thin link to identify when all copies are eligible for auto-hiding (because they were all originally linkonce_odr global unnamed addr), and only do the auto-hide in that case. Most of the changes here are due to plumbing the new flag through the bitcode and llvm assembly, and resulting test changes. I augmented the existing auto-hide test to check for this situation. Reviewers: pcc Subscribers: mehdi_amini, inglorion, eraman, dexonsmith, arphaman, dang, llvm-commits, steven_wu, wmi Tags: #llvm Differential Revision: https://reviews.llvm.org/D59709 llvm-svn: 360466
* Revert "Temporarily Revert "Add basic loop fusion pass.""Eric Christopher2019-04-1743-0/+2488
| | | | | | | | The reversion apparently deleted the test/Transforms directory. Will be re-reverting again. llvm-svn: 358552
* Temporarily Revert "Add basic loop fusion pass."Eric Christopher2019-04-1743-2488/+0
| | | | | | | | As it's causing some bot failures (and per request from kbarton). This reverts commit r358543/ab70da07286e618016e78247e4a24fcb84077fda. llvm-svn: 358546
* [ThinLTO] Efficiency fix for writing type id records in per-module indexesTeresa Johnson2018-09-252-11/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: In D49565/r337503, the type id record writing was fixed so that only referenced type ids were emitted into each per-module index for ThinLTO distributed builds. However, this still left an efficiency issue: each per-module index checked all type ids for membership in the referenced set, yielding O(M*N) performance (M indexes and N type ids). Change the TypeIdMap in the summary to be indexed by GUID, to facilitate correlating with type identifier GUIDs referenced in the function summary TypeIdInfo structures. This allowed simplifying other places where a map from type id GUID to type id map entry was previously being used to aid this correlation. Also fix AsmWriter code to handle the rare case of type id GUID collision. For a large internal application, this reduced the thin link time by almost 15%. Reviewers: pcc, vitalybuka Subscribers: mehdi_amini, inglorion, steven_wu, dexonsmith, llvm-commits Differential Revision: https://reviews.llvm.org/D51330 llvm-svn: 343021
* [WholeProgramDevirt] Don't process declarations when building type id mapEugene Leviant2018-09-231-0/+25
| | | | | | Differential revision: https://reviews.llvm.org/D52175 llvm-svn: 342836
* Runtime flag to control branch funnel thresholdVitaly Buka2018-04-061-0/+100
| | | | | | | | | | Reviewers: pcc Subscribers: hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D45193 llvm-svn: 329459
* Don't inline @llvm.icall.branch.funnelVitaly Buka2018-04-041-6/+14
| | | | | | | | | | | | | | Summary: @llvm.icall.branch.funnel is musttail with variable number of arguments. After inlining current backend can't separate call targets from call arguments. Reviewers: pcc Subscribers: hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D45116 llvm-svn: 329235
* Use branch funnels for virtual calls when retpoline mitigation is enabled.Peter Collingbourne2018-03-094-12/+209
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The retpoline mitigation for variant 2 of CVE-2017-5715 inhibits the branch predictor, and as a result it can lead to a measurable loss of performance. We can reduce the performance impact of retpolined virtual calls by replacing them with a special construct known as a branch funnel, which is an instruction sequence that implements virtual calls to a set of known targets using a binary tree of direct branches. This allows the processor to speculately execute valid implementations of the virtual function without allowing for speculative execution of of calls to arbitrary addresses. This patch extends the whole-program devirtualization pass to replace certain virtual calls with calls to branch funnels, which are represented using a new llvm.icall.jumptable intrinsic. It also extends the LowerTypeTests pass to recognize the new intrinsic, generate code for the branch funnels (x86_64 only for now) and lay out virtual tables as required for each branch funnel. The implementation supports full LTO as well as ThinLTO, and extends the ThinLTO summary format used for whole-program devirtualization to support branch funnels. For more details see RFC: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120672.html Differential Revision: https://reviews.llvm.org/D42453 llvm-svn: 327163
* Make GlobalValues with non-default visibilility dso_local.Rafael Espindola2018-01-181-2/+2
| | | | | | | | | | | | This is similar to r322317, but for visibility. It is not as neat because we have to special case extern_weak. The idea is the same as the previous change, make the transition to explicit dso_local easier for the frontends. With this they only have to add dso_local to symbols where we need some external information to decide if it is dso_local (like it being part of an ELF executable). llvm-svn: 322806
* Make internal/private GVs implicitly dso_local.Rafael Espindola2018-01-111-2/+2
| | | | | | | | | | | | | | | | While updating clang tests for having clang set dso_local I noticed that: - There are *a lot* of tests to update. - Many of the updates are redundant. They are redundant because a GV is "obviously dso_local". This patch starts formalizing that a bit by requiring that internal and private GVs be dso_local too. Since they all are, we don't have to print dso_local to the textual representation, making it a bit more compact and easier to read. llvm-svn: 322317
* [LTO][ThinLTO] Use the linker resolutions to mark global values as dso_local.Sean Fertile2017-11-041-0/+1
| | | | | | | | | | | | | Now that we have a way to mark GlobalValues as local we can use the symbol resolutions that the linker plugin provides as part of lto/thinlto link step to refine the compilers view on what symbols will end up being local. Originally commited as r317374, but reverted in r317395 to update some missed tests. Differential Revision: https://reviews.llvm.org/D35702 llvm-svn: 317408
* Revert "[LTO][ThinLTO] Use the linker resolutions to mark global values ..."Sean Fertile2017-11-041-1/+0
| | | | | | | | | Changes more tests then expected on one of the build bots. reverting to investigate. This reverts https://llvm.org/svn/llvm-project/llvm/trunk@317374 llvm-svn: 317395
* [LTO][ThinLTO] Use the linker resolutions to mark global values as dso_local.Sean Fertile2017-11-031-0/+1
| | | | | | | | | | Now that we have a way to mark GlobalValues as local we can use the symbol resolutions that the linker plugin provides as part of lto/thinlto link step to refine the compilers view on what symbols will end up being local. Differential Revision: https://reviews.llvm.org/D35702 llvm-svn: 317374
* LowerTypeTests: Add import/export support for targets without absolute ↵Peter Collingbourne2017-09-115-0/+40
| | | | | | | | | | symbol constants. The rationale is the same as for r312967. Differential Revision: https://reviews.llvm.org/D37408 llvm-svn: 312968
* WholeProgramDevirt: Add import/export support for targets without absolute ↵Peter Collingbourne2017-09-116-16/+47
| | | | | | | | | | | | | | | | | | | symbol constants. Not all targets support the use of absolute symbols to export constants. In particular, ARM has a wide variety of constant encodings that cannot currently be relocated by linkers. So instead of exporting the constants using symbols, export them directly in the summary. The values of the constants are left as zeroes on targets that support symbolic exports. This may result in more cache misses when targeting those architectures as a result of arbitrary changes in constant values, but this seems somewhat unavoidable for now. Differential Revision: https://reviews.llvm.org/D37407 llvm-svn: 312967
* WholeProgramDevirt: When promoting for single-impl devirt, also rename the ↵Peter Collingbourne2017-09-081-6/+12
| | | | | | | | | | | comdat. This is required when targeting COFF, as the comdat name must match one of the names of the symbols in the comdat. Differential Revision: https://reviews.llvm.org/D37550 llvm-svn: 312767
* WholeProgramDevirt: Create bitcast to i8* at each virtual call site.Peter Collingbourne2017-08-225-0/+45
| | | | | | | | | We can't reuse the llvm.assume instruction's bitcast because it may not dominate every user of the vtable pointer. Differential Revision: https://reviews.llvm.org/D36994 llvm-svn: 311491
* Skip CFI for dead functions.Evgeniy Stepanov2017-06-028-8/+12
| | | | | | Differential Revision: https://reviews.llvm.org/D33805 llvm-svn: 304578
* Align all scalar numbers to LLVM_YAML_IS_FLOW_SEQUENCE_VECTORJonas Hahnfeld2017-04-042-9/+3
| | | | | | | | | Otherwise, yamlize in YAMLTraits.h might be wrongly defined. This makes some AMDGPU tests fail when LLVM_LINK_LLVM_DYLIB is set. Differential Revision: https://reviews.llvm.org/D30508 llvm-svn: 299415
* WholeProgramDevirt: Implement export/import support for VCP.Peter Collingbourne2017-03-103-0/+126
| | | | | | Differential Revision: https://reviews.llvm.org/D30017 llvm-svn: 297503
* WholeProgramDevirt: Implement export/import support for unique ret val opt.Peter Collingbourne2017-03-104-0/+127
| | | | | | Differential Revision: https://reviews.llvm.org/D29917 llvm-svn: 297502
* WholeProgramDevirt: Implement importing for uniform ret val opt.Peter Collingbourne2017-03-092-0/+28
| | | | | | Differential Revision: https://reviews.llvm.org/D29854 llvm-svn: 297350
* WholeProgramDevirt: Implement importing for single-impl devirtualization.Peter Collingbourne2017-03-092-0/+62
| | | | | | Differential Revision: https://reviews.llvm.org/D29844 llvm-svn: 297333
* WholeProgramDevirt: Implement exporting for uniform ret val opt.Peter Collingbourne2017-03-041-0/+36
| | | | | | Differential Revision: https://reviews.llvm.org/D29846 llvm-svn: 296948
* WholeProgramDevirt: Implement exporting for single-impl devirtualization.Peter Collingbourne2017-03-041-0/+78
| | | | | | Differential Revision: https://reviews.llvm.org/D29811 llvm-svn: 296945
* WholeProgramDevirt: Add any unsuccessful llvm.type.checked.load ↵Peter Collingbourne2017-03-042-0/+50
| | | | | | | | | | | | | devirtualizations to the list of llvm.type.test users. Any unsuccessful llvm.type.checked.load devirtualizations will be translated into uses of llvm.type.test, so we need to add the resulting llvm.type.test intrinsics to the function summaries so that the LowerTypeTests pass will export them. Differential Revision: https://reviews.llvm.org/D29808 llvm-svn: 296939
* WholeProgramDevirt: For VCP use a 32-bit ConstantInt for the byte offset.Peter Collingbourne2017-02-173-9/+9
| | | | | | | | | | | | | | | A future change will cause this byte offset to be inttoptr'd and then exported via an absolute symbol. On the importing end we will expect the symbol to be in range [0,2^32) so that it will fit into a 32-bit relocation. The problem is that on 64-bit architectures if the offset is negative it will not be in the correct range once we inttoptr it. This change causes us to use a 32-bit integer so that it can be inttoptr'd (which zero extends) into the correct range. Differential Revision: https://reviews.llvm.org/D30016 llvm-svn: 295487
* WholeProgramDevirt: Examine the function body when deciding whether ↵Peter Collingbourne2017-02-171-6/+40
| | | | | | | | | | functions are readnone. The goal is to get an analysis result even for de-refineable functions. Differential Revision: https://reviews.llvm.org/D29803 llvm-svn: 295472
* Re-apply r295110 and r295144 with a fix for the ASan issue.Peter Collingbourne2017-02-153-20/+59
| | | | llvm-svn: 295241
* Revert r295110 and r295144.Daniel Jasper2017-02-153-59/+20
| | | | | | | This fails under ASAN: http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/builds/798/steps/check-llvm%20asan/logs/stdio llvm-svn: 295162
* WholeProgramDevirt: Change internal vcall data structures to match summary.Peter Collingbourne2017-02-143-20/+59
| | | | | | | | | | | | | | | | | | Group calls into constant and non-constant arguments up front, and use uint64_t instead of ConstantInt to represent constant arguments. The goal is to allow the information from the summary to fit naturally into this data structure in a future change (specifically, it will be added to CallSiteInfo). This has two side effects: - We disallow VCP for constant integer arguments of width >64 bits. - We remove the restriction that the bitwidth of a vcall's argument and return types must match those of the vfunc definitions. I don't expect either of these to matter in practice. The first case is uncommon, and the second one will lead to UB (so we can do anything we like). Differential Revision: https://reviews.llvm.org/D29744 llvm-svn: 295110
* IR: Type ID summary extensions for WPD; thread summary into WPD pass.Peter Collingbourne2017-02-133-0/+147
| | | | | | | | | | Make the whole thing testable by adding YAML I/O support for the WPD summary information and adding some negative tests that exercise the YAML support. Differential Revision: https://reviews.llvm.org/D29782 llvm-svn: 294981
* WholeProgramDevirt: Check that VCP candidate functions are defined before ↵Peter Collingbourne2017-02-091-0/+32
| | | | | | | | evaluating them. This was crashing before. llvm-svn: 294666
* LowerTypeTests: Change a few vtable globals in tests to constants.Peter Collingbourne2017-02-097-14/+14
| | | | | | | | | It turns out that some of our negative tests were not in fact providing the test coverage we expected: they were passing because the vtables were failing an early check that they were constant. Fix this by changing the globals in these tests to constants. llvm-svn: 294550
* WholeProgramDevirt: Teach the pass to handle structs of arrays.Peter Collingbourne2016-12-093-4/+74
| | | | | | This will become necessary in some cases once D22296 lands. llvm-svn: 289165
* Make WholeProgramDevirt understand ConstStruct vtables.Peter Collingbourne2016-12-092-0/+63
| | | | | | | | Based on a patch by LemonBoy! Differential Revision: https://reviews.llvm.org/D26581 llvm-svn: 289162
* WholeProgramDevirt: generate more detailed and accurate remarks.Ivan Krasin2016-08-113-23/+30
| | | | | | | | | | | | | | | | | | | Summary: Keep track of all methods for which we have devirtualized at least one call and then print them sorted alphabetically. That allows to avoid duplicates and also makes the order deterministic. Add optimization names into the remarks, so that it's easier to understand how has each method been devirtualized. Fix a bug when wrong methods could have been reported for tryVirtualConstProp. Reviewers: kcc, mehdi_amini Differential Revision: https://reviews.llvm.org/D23297 llvm-svn: 278389
* WholeProgramDevirt: print remarks with devirtualized method names.Ivan Krasin2016-08-052-1/+17
| | | | | | | | | | | | | | | | | | Summary: Chrome on Linux uses WholeProgramDevirt for speed ups, and it's important to detect regressions on both sides: the toolchain, if fewer methods get devirtualized after an update, and Chrome, if an innocently looking change caused many hot methods become virtual again. The need to track devirtualized methods is not Chrome-specific, but it's probably the only user of the pass at this time. Reviewers: kcc Differential Revision: https://reviews.llvm.org/D23219 llvm-svn: 277856
* Print remarks from WholeProgramDevirt pass for each call site.Ivan Krasin2016-07-123-3/+13
| | | | | | | | | | | | | Summary: It's useful to have some visibility about which call sites are devirtualized, especially for debug purposes. Another use case is a regression test on the application side (like, Chromium). Reviewers: pcc Differential Revision: http://reviews.llvm.org/D22252 llvm-svn: 275145
* IR: Introduce llvm.type.checked.load intrinsic.Peter Collingbourne2016-06-256-12/+257
| | | | | | | | | | | | | | | | This intrinsic safely loads a function pointer from a virtual table pointer using type metadata. This intrinsic is used to implement control flow integrity in conjunction with virtual call optimization. The virtual call optimization pass will optimize away llvm.type.checked.load intrinsics associated with devirtualized calls, thereby removing the type check in cases where it is not needed to enforce the control flow integrity constraint. This patch also introduces the capability to copy type metadata between global variables, and teaches the virtual call optimization pass to do so. Differential Revision: http://reviews.llvm.org/D21121 llvm-svn: 273756
* IR: New representation for CFI and virtual call optimization pass metadata.Peter Collingbourne2016-06-2416-136/+98
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The bitset metadata currently used in LLVM has a few problems: 1. It has the wrong name. The name "bitset" refers to an implementation detail of one use of the metadata (i.e. its original use case, CFI). This makes it harder to understand, as the name makes no sense in the context of virtual call optimization. 2. It is represented using a global named metadata node, rather than being directly associated with a global. This makes it harder to manipulate the metadata when rebuilding global variables, summarise it as part of ThinLTO and drop unused metadata when associated globals are dropped. For this reason, CFI does not currently work correctly when both CFI and vcall opt are enabled, as vcall opt needs to rebuild vtable globals, and fails to associate metadata with the rebuilt globals. As I understand it, the same problem could also affect ASan, which rebuilds globals with a red zone. This patch solves both of those problems in the following way: 1. Rename the metadata to "type metadata". This new name reflects how the metadata is currently being used (i.e. to represent type information for CFI and vtable opt). The new name is reflected in the name for the associated intrinsic (llvm.type.test) and pass (LowerTypeTests). 2. Attach metadata directly to the globals that it pertains to, rather than using the "llvm.bitsets" global metadata node as we are doing now. This is done using the newly introduced capability to attach metadata to global variables (r271348 and r271358). See also: http://lists.llvm.org/pipermail/llvm-dev/2016-June/100462.html Differential Revision: http://reviews.llvm.org/D21053 llvm-svn: 273729
* [PM] Port WholeProgramDevirt to the new pass manager.Davide Italiano2016-06-141-0/+1
| | | | llvm-svn: 272721
* WholeProgramDevirt: introduce.Peter Collingbourne2016-02-0916-0/+904
This pass implements whole program optimization of virtual calls in cases where we know (via bitset information) that the list of callees is fixed. This includes the following: - Single implementation devirtualization: if a virtual call has a single possible callee, replace all calls with a direct call to that callee. - Virtual constant propagation: if the virtual function's return type is an integer <=64 bits and all possible callees are readnone, for each class and each list of constant arguments: evaluate the function, store the return value alongside the virtual table, and rewrite each virtual call as a load from the virtual table. - Uniform return value optimization: if the conditions for virtual constant propagation hold and each function returns the same constant value, replace each virtual call with that constant. - Unique return value optimization for i1 return values: if the conditions for virtual constant propagation hold and a single vtable's function returns 0, or a single vtable's function returns 1, replace each virtual call with a comparison of the vptr against that vtable's address. Differential Revision: http://reviews.llvm.org/D16795 llvm-svn: 260312
OpenPOWER on IntegriCloud