summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/IPO
Commit message (Collapse)AuthorAgeFilesLines
...
* Added ThinLTO inlining statisticsPiotr Padlewski2016-07-291-9/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: copypasta doc of ImportedFunctionsInliningStatistics class \brief Calculate and dump ThinLTO specific inliner stats. The main statistics are: (1) Number of inlined imported functions, (2) Number of imported functions inlined into importing module (indirect), (3) Number of non imported functions inlined into importing module (indirect). The difference between first and the second is that first stat counts all performed inlines on imported functions, but the second one only the functions that have been eventually inlined to a function in the importing module (by a chain of inlines). Because llvm uses bottom-up inliner, it is possible to e.g. import function `A`, `B` and then inline `B` to `A`, and after this `A` might be too big to be inlined into some other function that calls it. It calculates this statistic by building graph, where the nodes are functions, and edges are performed inlines and then by marking the edges starting from not imported function. If `Verbose` is set to true, then it also dumps statistics per each inlined function, sorted by the greatest inlines count like - number of performed inlines - number of performed inlines to importing module Reviewers: eraman, tejohnson, mehdi_amini Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D22491 llvm-svn: 277089
* Add EP_CGSCCOptimizerLate extension point to PassManagerBuilderDavid Majnemer2016-07-281-0/+1
| | | | | | | | | | | | The EP_CGSCCOptimizerLate extension point allows adding CallGraphSCC passes at the end of the main CallGraphSCC passes and before any function simplification passes run by CGPassManager. Patch by Gor Nishanov! Differential Revision: https://reviews.llvm.org/D22897 llvm-svn: 276953
* Cleanup : Reformat PartialInliner.cpp to have current LLVM style conventionsSean Silva2016-07-251-71/+74
| | | | | | | | | | Modify the variable names and code style to be that of modern LLVM. Patch by River Riddle! Differential Revision: https://reviews.llvm.org/D22743 llvm-svn: 276610
* Fix : Partial Inliner requires AssumptionCacheTrackerSean Silva2016-07-251-35/+58
| | | | | | | | | | | The public InlineFunction utility assumes that the passed in InlineFunctionInfo has a valid AssumptionCacheTracker. Patch by River Riddle! Differential Revision: https://reviews.llvm.org/D22706 llvm-svn: 276609
* [Profile] Use explicit flag to enable IR PGOXinliang David Li2016-07-231-8/+13
| | | | | | | | Patch by Jake VanAdrighem Differential Revision: http://reviews.llvm.org/D22607 llvm-svn: 276516
* Avoid using a raw AssumptionCacheTracker in various inliner functions.Sean Silva2016-07-233-28/+64
| | | | | | | | | | This unblocks the new PM part of River's patch in https://reviews.llvm.org/D22706 Conveniently, this same change was needed for D21921 and so these changes are just spun out from there. llvm-svn: 276515
* Add flag to PassManagerBuilder to disable GVN Hoist Pass.Alina Sbirlea2016-07-221-1/+6
| | | | | | | | | | | | | | Summary: Adding a flag to diable GVN Hoisting by default. Note: The GVN Hoist Pass causes some Halide tests to hang. Halide will disable the pass while investigating. Reviewers: llvm-commits, chandlerc, spop, dberlin Subscribers: mehdi_amini Differential Revision: https://reviews.llvm.org/D22639 llvm-svn: 276479
* Don't remove side effecting instructions due to ConstantFoldInstructionDavid Majnemer2016-07-221-1/+3
| | | | | | | | | Just because we can constant fold the result of an instruction does not imply that we can delete the instruction. It may have side effects. This fixes PR28655. llvm-svn: 276389
* Make help text more consistent. NFC.George Burgess IV2016-07-201-1/+1
| | | | llvm-svn: 276205
* [FunctionAttrs] Correct the safety analysis for inference of 'returned'David Majnemer2016-07-191-0/+51
| | | | | | | | | | | | We skipped over ReturnInsts which didn't return an argument which would lead us to incorrectly conclude that an argument returned by another ReturnInst was 'returned'. This reverts commit r275756. This fixes PR28610. llvm-svn: 276008
* [PM] Port FunctionImport Pass to new PMTeresa Johnson2016-07-182-46/+55
| | | | | | | | | | | | Summary: Port FunctionImport Pass to new PM. Reviewers: mehdi_amini, davide Subscribers: davidxl, llvm-commits Differential Revision: https://reviews.llvm.org/D22475 llvm-svn: 275916
* Revert r275678, "Revert "Revert r275027 - Let FuncAttrs infer the 'returned' ↵NAKAMURA Takumi2016-07-181-50/+0
| | | | | | | | | | argument attribute"" This reverts also r275029, "Update Clang tests after adding inference for the returned argument attribute" It broke LTO build. Seems miscompilation. llvm-svn: 275756
* [ThinLTO] Perform profile-guided indirect call promotionTeresa Johnson2016-07-171-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: To enable profile-guided indirect call promotion in ThinLTO mode, we simply add call graph edges for each profitable target from the profile to the summaries, then the summary-guided importing will consider the callee for importing as usual. Also we need to enable the indirect call promotion pass creation in the PassManagerBuilder when PerformThinLTO=true (we are in the ThinLTO backend), so that the newly imported functions are considered for promotion in the backends. The IC promotion profiles refer to callees by GUID, which required adding GUIDs to the per-module VST in bitcode (and assigning them valueIds similar to how they are assigned valueIds in the combined index). Reviewers: mehdi_amini, xur Subscribers: mehdi_amini, davidxl, llvm-commits Differential Revision: http://reviews.llvm.org/D21932 llvm-svn: 275707
* Revert "Revert r275027 - Let FuncAttrs infer the 'returned' argument attribute"Hal Finkel2016-07-161-0/+50
| | | | | | | | | | | | | | | | This reverts commit r275042; the initial commit triggered self-hosting failures on ARM/AArch64. James Molloy identified the problematic backend code, which has been disabled in r275677. Trying again... Original commit message: Let FuncAttrs infer the 'returned' argument attribute A function can have one argument with the 'returned' attribute, indicating that the associated argument is always the return value of the function. Add FuncAttrs inference logic. llvm-svn: 275678
* [PGO] IRPGO pre-cleanup pass changesRong Xu2016-07-151-0/+22
| | | | | | | | | | | | | | This patch adds a selected set of cleanup passes including a pre-inline pass before LLVM IR PGO instrumentation. The inline is only intended to apply those obvious/trivial ones before instrumentation so that much less instrumentation is needed to get better profiling information. This will drastically improve the instrumented code performance for large C++ applications. Another benefit is the context sensitive counts that can potentially improve the PGO optimization. Differential Revision: http://reviews.llvm.org/D21405 llvm-svn: 275588
* code hoisting pass based on GVNSebastian Pop2016-07-151-0/+1
| | | | | | | | | | | | | This pass hoists duplicated computations in the program. The primary goal of gvn-hoist is to reduce the size of functions before inline heuristics to reduce the total cost of function inlining. Pass written by Sebastian Pop, Aditya Kumar, Xiaoyu Hu, and Brian Rzycki. Important algorithmic contributions by Daniel Berlin under the form of reviews. Differential Revision: http://reviews.llvm.org/D19338 llvm-svn: 275561
* Revert r275401, it caused PR28551.Nico Weber2016-07-141-1/+0
| | | | llvm-svn: 275420
* code hoisting pass based on GVNSebastian Pop2016-07-141-0/+1
| | | | | | | | | | | | | This pass hoists duplicated computations in the program. The primary goal of gvn-hoist is to reduce the size of functions before inline heuristics to reduce the total cost of function inlining. Pass written by Sebastian Pop, Aditya Kumar, Xiaoyu Hu, and Brian Rzycki. Important algorithmic contributions by Daniel Berlin under the form of reviews. Differential Revision: http://reviews.llvm.org/D19338 llvm-svn: 275401
* Print remarks from WholeProgramDevirt pass for each call site.Ivan Krasin2016-07-121-0/+10
| | | | | | | | | | | | | Summary: It's useful to have some visibility about which call sites are devirtualized, especially for debug purposes. Another use case is a regression test on the application side (like, Chromium). Reviewers: pcc Differential Revision: http://reviews.llvm.org/D22252 llvm-svn: 275145
* [PM/IPO] Port LowerTypeTests to the new PassManager.Davide Italiano2016-07-111-17/+28
| | | | | | | | There's a little bit of churn in this patch because the initialization mechanism is now shared between the old and the new PM. Other than that, it's just a pretty mechanical translation. llvm-svn: 275082
* [LowerTypeTests] Don't rely on doInitialization().Davide Italiano2016-07-111-23/+16
| | | | | | | | | In preparation for porting this pass to the new PM (which has no doInitialization()). Differential Revision: http://reviews.llvm.org/D22223 llvm-svn: 275074
* Implement callsite-hotness based inline cost for Sample-based PGODehao Chen2016-07-111-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: For sample-based PGO, using BFI to calculate callsite count is sometime not accurate. This is because with sampling based approach, if a callsite resides in a hot loop deeply nested in a bunch of cold branches, the callsite's BFI frequency would be inaccurately calculated due to lack of samples in the cold branch. E.g. if (A1 && A2 && A3 && ..... && A10) { for (i=0; i < 100000000; i++) { callsite(); } } Assume that A1 to A100 are all 100% taken, and callsite has 1000 samples and thus is considerred hot. Because the loop's trip count is huge, it's normal that all branches outside the loop has no sample at all. As a result, we can only use static branch probability to derive the the frequency of the loop header. Assuming that static heuristic thinks each branch is 50% taken, then the count calculated from BFI will be 1/(2^10) of the actual value. In order to get more accurate callsite count, we directly annotate the weight on the call instruction, and directly use it when checking callsite hotness. Note that this mechanism can also be shared by instrumentation based callsite hotness analysis. The side benefit is that it breaks the dependency from Inliner to BFI as call count is embedded in the IR. Reviewers: davidxl, eraman, dnovillo Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D22118 llvm-svn: 275073
* Tune the weight propagation algorithm for sample profile.Dehao Chen2016-07-111-12/+26
| | | | | | | | | | | | Summary: Handle the case when there is only one incoming/outgoing edge for a visited basic block: use the block weight to adjust edge weight even when the edge has been visited before. This can help reduce inaccuracies introduced by incorrect basic block profile, as shown in the updated unittest. Reviewers: davidxl, dnovillo Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D22180 llvm-svn: 275072
* Revert r275027 - Let FuncAttrs infer the 'returned' argument attributeHal Finkel2016-07-111-50/+0
| | | | | | Reverting r275027 and r275033. These seem to cause miscompiles on the AArch64 buildbot. llvm-svn: 275042
* Don't use a SmallSet for returned attribute inferenceHal Finkel2016-07-111-11/+19
| | | | | | | Suggested post-commit by David Majnemer on IRC (following-up on a pre-commit review comment). llvm-svn: 275033
* Let FuncAttrs infer the 'returned' argument attributeHal Finkel2016-07-101-0/+42
| | | | | | | | | | A function can have one argument with the 'returned' attribute, indicating that the associated argument is always the return value of the function. Add FuncAttrs inference logic. Differential Revision: http://reviews.llvm.org/D22202 llvm-svn: 275027
* [ArgPromote] Use function_ref and for-range loops.Benjamin Kramer2016-07-091-13/+10
| | | | | | No functionality change intended. llvm-svn: 274973
* [PM] Port CrossDSOCFI to the new pass manager.Davide Italiano2016-07-091-1/+10
| | | | llvm-svn: 274962
* Add 'thinlto_src_module' md with asserts or -enable-import-metadataPiotr Padlewski2016-07-081-6/+19
| | | | | | | | | | | | | | | | Summary: This way the metadata will be only generated when asserts enabled, or when -enable-import-metadata specified FIXED missing colon on requires. Reviewers: tejohnson, eraman, mehdi_amini Subscribers: mehdi_amini, llvm-commits Differential Revision: http://reviews.llvm.org/D22167 llvm-svn: 274947
* Revert "Add 'thinlto_src_module' md with asserts or -enable-import-metadata"Piotr Padlewski2016-07-081-19/+6
| | | | | | | | | Reverting because of 17463 http://lab.llvm.org:8011/builders/clang-x86_64-linux-selfhost-modules/builds/17463 This reverts commit d20cb431bba2ba43b4c65a8556cff445bfefbb7c. llvm-svn: 274946
* Add 'thinlto_src_module' md with asserts or -enable-import-metadataPiotr Padlewski2016-07-081-6/+19
| | | | | | | | | | | | | | Summary: This way the metadata will be only generated when asserts enabled, or when -enable-import-metadata specified Reviewers: tejohnson, eraman, mehdi_amini Subscribers: mehdi_amini, llvm-commits Differential Revision: http://reviews.llvm.org/D22167 llvm-svn: 274938
* Remove inline hints computation from SampleProfile.cppDehao Chen2016-07-081-70/+0
| | | | | | | | | | | | Summary: As we will move to use uniformed hotness check in inliner, we do not need inline hints in SampleProfile pass any more. Reviewers: dnovillo, davidxl Subscribers: eraman, llvm-commits Differential Revision: http://reviews.llvm.org/D19287 llvm-svn: 274918
* [CrossDSOCFI] Change the pass so that it doesn't require doInitialization()Davide Italiano2016-07-081-19/+10
| | | | | | Differential Revision: http://reviews.llvm.org/D21357 llvm-svn: 274910
* Add 'thinlto_src_module' metadata to imported functionPiotr Padlewski2016-07-061-0/+6
| | | | | | | | | | | | | | Added metadata to be able to make statistics on how many functions that have been imported have been removed. Also module name might be helpfull when debugging. Reviewers: tejohnson, eraman Subscribers: mehdi_amini, llvm-commits Differential Revision: http://reviews.llvm.org/D21943 llvm-svn: 274668
* NFC changed names in FunctionImportPiotr Padlewski2016-07-061-14/+14
| | | | llvm-svn: 274649
* [CFLAA] Split into Anders+Steens analysis.George Burgess IV2016-07-061-6/+30
| | | | | | | | | | | | | | | | | | | | | StratifiedSets (as implemented) is very fast, but its accuracy is also limited. If we take a more aggressive andersens-like approach, we can be way more accurate, but we'll also end up being slower. So, we've decided to split CFLAA into CFLSteensAA and CFLAndersAA. Long-term, we want to end up in a place where CFLSteens is queried first; if it can provide an answer, great (since queries are basically map lookups). Otherwise, we'll fall back to CFLAnders, BasicAA, etc. This patch splits everything out so we can try to do something like that when we get a reasonable CFLAnders implementation. Patch by Jia Chen. Differential Revision: http://reviews.llvm.org/D21910 llvm-svn: 274589
* [PM] Some preparatory refactoring to minimize the diff of D21921Sean Silva2016-07-031-14/+20
| | | | llvm-svn: 274456
* Remove dead TLI arg of isKnownNonNull and propagate deadness. NFC.Sean Silva2016-07-021-21/+6
| | | | | | | | | | | | | | This actually uncovered a surprisingly large chain of ultimately unused TLI args. From what I can gather, this argument is a remnant of when isKnownNonNull would look at the TLI directly. The current approach seems to be that InferFunctionAttrs runs early in the pipeline and uses TLI to annotate the TLI-dependent non-null information as return attributes. This also removes the dependence of functionattrs on TLI altogether. llvm-svn: 274455
* [PM] Some preparatory refactoring to minimize the diff of D21921Sean Silva2016-07-021-15/+17
| | | | | | The main change here is just moving stuff to static functions. llvm-svn: 274446
* [PM] Preparatory cleanups to ArgumentPromotion.Sean Silva2016-07-021-54/+74
| | | | | | | This pulls some obvious changes out of http://reviews.llvm.org/D21921 to minimize the diff. llvm-svn: 274445
* Revert "code hoisting pass based on GVN"Duncan P. N. Exon Smith2016-07-011-1/+0
| | | | | | | | | | | | This reverts commit r274305, since it breaks self-hosting: http://lab.llvm.org:8080/green/job/clang-stage1-configure-RA_build/22349/ http://lab.llvm.org:8011/builders/clang-x86_64-linux-selfhost-modules/builds/17232 Note that the blamelist on lab.llvm.org:8011 is incorrect. The previous build was r274299, but somehow r274305 wasn't included in the blamelist: http://lab.llvm.org:8011/builders/clang-x86_64-linux-selfhost-modules llvm-svn: 274320
* code hoisting pass based on GVNSebastian Pop2016-07-011-0/+1
| | | | | | | | | | | | | This pass hoists duplicated computations in the program. The primary goal of gvn-hoist is to reduce the size of functions before inline heuristics to reduce the total cost of function inlining. Pass written by Sebastian Pop, Aditya Kumar, Xiaoyu Hu, and Brian Rzycki. Important algorithmic contributions by Daniel Berlin under the form of reviews. Differential Revision: http://reviews.llvm.org/D19338 llvm-svn: 274305
* [PM] Port PartialInlining to the new PMEaswaran Raman2016-06-272-22/+29
| | | | | | Differential revision: http://reviews.llvm.org/D21699 llvm-svn: 273894
* Apply clang-tidy's modernize-loop-convert to most of lib/Transforms.Benjamin Kramer2016-06-2611-100/+85
| | | | | | Only minor manual fixes. No functionality change intended. llvm-svn: 273808
* Revert "[SimplifyCFG] Stop inserting calls to llvm.trap for UB"David Majnemer2016-06-251-1/+1
| | | | | | This reverts commit r273778, it seems to break UBSan :/ llvm-svn: 273779
* [SimplifyCFG] Stop inserting calls to llvm.trap for UBDavid Majnemer2016-06-251-1/+1
| | | | | | | | | | | | | | | | | SimplifyCFG had logic to insert calls to llvm.trap for two very particular IR patterns: stores and invokes of undef/null. While InstCombine canonicalizes certain undefined behavior IR patterns to stores of undef, phase ordering means that this cannot be relied upon in general. There are much better tools than llvm.trap: UBSan and ASan. N.B. I could be argued into reverting this change if a clear argument as to why it is important that we synthesize llvm.trap for stores, I'd be hard pressed to see why it'd be useful for invokes... llvm-svn: 273778
* IR: Introduce llvm.type.checked.load intrinsic.Peter Collingbourne2016-06-251-12/+142
| | | | | | | | | | | | | | | | This intrinsic safely loads a function pointer from a virtual table pointer using type metadata. This intrinsic is used to implement control flow integrity in conjunction with virtual call optimization. The virtual call optimization pass will optimize away llvm.type.checked.load intrinsics associated with devirtualized calls, thereby removing the type check in cases where it is not needed to enforce the control flow integrity constraint. This patch also introduces the capability to copy type metadata between global variables, and teaches the virtual call optimization pass to do so. Differential Revision: http://reviews.llvm.org/D21121 llvm-svn: 273756
* Hookup ProfileSummary with SampleProfilerLoaderDehao Chen2016-06-241-0/+1
| | | | | | | | | | | | Summary: Set ProfileSummary in SampleProfilerLoader. Reviewers: davidxl, eraman Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D21702 llvm-svn: 273745
* Fix unused variable warning in -asserts builds.Peter Collingbourne2016-06-241-2/+1
| | | | llvm-svn: 273737
* IR: New representation for CFI and virtual call optimization pass metadata.Peter Collingbourne2016-06-246-317/+270
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The bitset metadata currently used in LLVM has a few problems: 1. It has the wrong name. The name "bitset" refers to an implementation detail of one use of the metadata (i.e. its original use case, CFI). This makes it harder to understand, as the name makes no sense in the context of virtual call optimization. 2. It is represented using a global named metadata node, rather than being directly associated with a global. This makes it harder to manipulate the metadata when rebuilding global variables, summarise it as part of ThinLTO and drop unused metadata when associated globals are dropped. For this reason, CFI does not currently work correctly when both CFI and vcall opt are enabled, as vcall opt needs to rebuild vtable globals, and fails to associate metadata with the rebuilt globals. As I understand it, the same problem could also affect ASan, which rebuilds globals with a red zone. This patch solves both of those problems in the following way: 1. Rename the metadata to "type metadata". This new name reflects how the metadata is currently being used (i.e. to represent type information for CFI and vtable opt). The new name is reflected in the name for the associated intrinsic (llvm.type.test) and pass (LowerTypeTests). 2. Attach metadata directly to the globals that it pertains to, rather than using the "llvm.bitsets" global metadata node as we are doing now. This is done using the newly introduced capability to attach metadata to global variables (r271348 and r271358). See also: http://lists.llvm.org/pipermail/llvm-dev/2016-June/100462.html Differential Revision: http://reviews.llvm.org/D21053 llvm-svn: 273729
OpenPOWER on IntegriCloud