summaryrefslogtreecommitdiffstats
path: root/llvm/test/Transforms/PGOProfile
Commit message (Collapse)AuthorAgeFilesLines
* [PGO] Implementate profile counter regiser promotionXinliang David Li2017-06-253-0/+222
| | | | | | Differential Revision: http://reviews.llvm.org/D34085 llvm-svn: 306231
* [PATCH] [PGO] Fixed cast operation in ↵Ana Pazos2017-06-191-0/+14
| | | | | | | | | | | | | | emIntrinsicVisitor::instrumentOneMemIntrinsic. Reviewers: xur, efriedma, davidxl Reviewed By: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34293 llvm-svn: 305737
* [PGO] Update VP metadata after memory intrinsic optimizationTeresa Johnson2017-06-131-2/+7
| | | | | | | | | | | | | | | Summary: Leave an updated VP metadata on the fallback memcpy intrinsic after specialization. This can be used for later possible expansion based on the average of the remaining values. Reviewers: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34164 llvm-svn: 305321
* [PartialInlining] Emit branch info and profile data as remarksXinliang David Li2017-06-011-0/+5
| | | | | | | | | This allows us to collect profile statistics to tune static branch prediction. Differential Revision: http://reviews.llvm.org/D33746 llvm-svn: 304452
* Memory intrinsic value profile optimization: Avoid divide by 0Teresa Johnson2017-04-281-0/+19
| | | | | | | | | | | | | | Summary: Skip memops if the total value profiled count is 0, we can't correctly scale up the counts and there is no point anyway. Reviewers: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D32624 llvm-svn: 301645
* Update profile during memory instrinsic optimizationTeresa Johnson2017-04-241-5/+16
| | | | | | | | | | | | | | | | Summary: Ensure that the new merge BB (which contains the rest of the original BB after the mem op being optimized) gets a profile frequency, in case there are additional mem ops later in the BB. Otherwise they get skipped as the merge BB looks cold. Reviewers: davidxl, xur Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D32447 llvm-svn: 301244
* [PGO] Memory intrinsic calls optimization based on profiled sizeRong Xu2017-04-041-0/+100
| | | | | | | | | | | | | | | | | | | | | | | | This patch optimizes two memory intrinsic operations: memset and memcpy based on the profiled size of the operation. The high level transformation is like: mem_op(..., size) ==> switch (size) { case s1: mem_op(..., s1); goto merge_bb; case s2: mem_op(..., s2); goto merge_bb; ... default: mem_op(..., size); goto merge_bb; } merge_bb: Differential Revision: http://reviews.llvm.org/D28966 llvm-svn: 299446
* [PGO] Add omitted test cases.Rong Xu2017-03-171-0/+35
| | | | llvm-svn: 298115
* [PGO] Value profile for size of memory intrinsic callsRong Xu2017-03-172-0/+86
| | | | | | | | This patch annotates the valuesites profile to memory intrinsics. Differential Revision: http://reviews.llvm.org/D31002 llvm-svn: 298110
* Resubmit r297897: [PGO] Value profile for size of memory intrinsic callsRong Xu2017-03-161-2/+2
| | | | | | | R297897 inadvertently enabled annotation for memop profiling. This new patch fixed it. llvm-svn: 297996
* Revert "[PGO] Value profile for size of memory intrinsic calls"Eric Liu2017-03-161-2/+2
| | | | | | This commit reverts r297897 and r297909. llvm-svn: 297951
* [PGO] Value profile for size of memory intrinsic callsRong Xu2017-03-151-2/+2
| | | | | | | | | This patch adds the value profile support to profile the size parameter of memory intrinsic calls: memcpy, memcmp, and memmov. Differential Revision: http://reviews.llvm.org/D28965 llvm-svn: 297897
* SamplePGO ThinLTO ICP fix for local functions.Dehao Chen2017-03-142-0/+90
| | | | | | | | | | | | | | | | | | | | | | | | Summary: In SamplePGO, if the profile is collected from non-LTO binary, and used to drive ThinLTO, the indirect call promotion may fail because ThinLTO adjusts local function names to avoid conflicts. There are two places of where the mismatch can happen: 1. thin-link prepends SourceFileName to front of FuncName to build the GUID (GlobalValue::getGlobalIdentifier). Unlike instrumentation FDO, SamplePGO does not use the PGOFuncName scheme and therefore the indirect call target profile data contains a hash of the OriginalName. 2. backend compiler promotes some local functions to global and appends .llvm.{$ModuleHash} to the end of the FuncName to derive PromotedFunctionName This patch tries at the best effort to find the GUID from the original local function name (in profile), and use that in ICP promotion, and in SamplePGO matching that happens in the backend after importing/inlining: 1. in thin-link, it builds the map from OriginalName to GUID so that when thin-link reads in indirect call target profile (represented by OriginalName), it knows which GUID to import. 2. in backend compiler, if sample profile reader cannot find a profile match for PromotedFunctionName, it will try to find if there is a match for OriginalFunctionName. 3. in backend compiler, we build symbol table entry for OriginalFunctionName and pointer to the same symbol of PromotedFunctionName, so that ICP can find the correct target to promote. Reviewers: mehdi_amini, tejohnson Reviewed By: tejohnson Subscribers: llvm-commits, Prazek Differential Revision: https://reviews.llvm.org/D30754 llvm-svn: 297757
* [PGO] Directory name stripping in global identifier for static functionsRong Xu2017-02-251-3/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Current internal option -static-func-full-module-prefix keeps all the directory path the profile counter names for static functions. The default of this option is false. This strips the directory names from the source filename which is problematic: (1) it creates linker errors for profile-generation compilation, exposed in our internal benchmarks. We are seeing messages like "warning: relocation refers to discarded section". This is due to the name conflicts after the stripping. (2) the stripping only applies to getPGOFuncName. Current Thin-LTO module importing for the indirect-calls assumes the source directory name not being stripped. Current default value for this option can potentially prevent some inter-module indirect-call-promotions. This patch turns the default value for -static-func-full-module-prefix to true. The second part of the patch is to have an alternative implementation under the internal option -static-func-strip-dirname-prefix=<value> This options specifies level of directories to be stripped from the source filename. Using a large value as the parameter has the same effect as -static-func-full-module-prefix. Differential Revision: http://reviews.llvm.org/D29512 llvm-svn: 296206
* Add call branch annotation for ICP promoted direct call in SamplePGO mode.Dehao Chen2017-02-231-0/+4
| | | | | | | | | | | | | | Summary: SamplePGO uses branch_weight annotation to represent callsite hotness. When ICP promotes an indirect call to direct call, we need to make sure the direct call is annotated with branch_weight in SamplePGO mode, so that downstream function inliner can use hot callsite heuristic. Reviewers: davidxl, eraman, xur Reviewed By: davidxl, xur Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D30282 llvm-svn: 296028
* Re-apply "[profiling] Remove dead profile name vars after emitting name data"Vedant Kumar2017-02-141-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | This reverts 295092 (re-applies 295084), with a fix for dangling references from the array of coverage names passed down from frontends. I missed this in my initial testing because I only checked test/Profile, and not test/CoverageMapping as well. Original commit message: The profile name variables passed to counter increment intrinsics are dead after we emit the finalized name data in __llvm_prf_nm. However, we neglect to erase these name variables. This causes huge size increases in the __TEXT,__const section as well as slowdowns when linker dead stripping is disabled. Some affected projects are so massive that they fail to link on Darwin, because only the small code model is supported. Fix the issue by throwing away the name constants as soon as we're done with them. Differential Revision: https://reviews.llvm.org/D29921 llvm-svn: 295099
* Revert "[profiling] Remove dead profile name vars after emitting name data"Vedant Kumar2017-02-141-1/+1
| | | | | | | | This reverts commit r295084. There is a test failure on: http://lab.llvm.org:8011/builders/clang-atom-d525-fedora-rel/builds/2620/ llvm-svn: 295092
* [profiling] Remove dead profile name vars after emitting name dataVedant Kumar2017-02-141-1/+1
| | | | | | | | | | | | | | | | The profile name variables passed to counter increment intrinsics are dead after we emit the finalized name data in __llvm_prf_nm. However, we neglect to erase these name variables. This causes huge size increases in the __TEXT,__const section as well as slowdowns when linker dead stripping is disabled. Some affected projects are so massive that they fail to link on Darwin, because only the small code model is supported. Fix the issue by throwing away the name constants as soon as we're done with them. Differential Revision: https://reviews.llvm.org/D29921 llvm-svn: 295084
* Fix some broken CHECK lines.Benjamin Kramer2017-01-221-2/+2
| | | | | | The colon is important. llvm-svn: 292761
* Resubmit "[PGO] Turn off comdat renaming in IR PGO by default"Rong Xu2017-01-115-21/+81
| | | | | | This patch resubmits the changes in r291588. llvm-svn: 291696
* Revert "[PGO] Turn off comdat renaming in IR PGO by default"Rong Xu2017-01-105-81/+21
| | | | | | | This patch reverts r291588: [PGO] Turn off comdat renaming in IR PGO by default, as we are seeing some hash mismatches in our internal tests. llvm-svn: 291621
* [PGO] Turn off comdat renaming in IR PGO by defaultRong Xu2017-01-105-21/+81
| | | | | | | | | | | | | | | | | | | | | | | Summary: In IR PGO we append the function hash to comdat functions to avoid the potential hash mismatch. This turns out not legal in some cases: if the comdat function is address-taken and used in comparison. Renaming changes the semantic. This patch turns off comdat renaming by default. To alleviate the hash mismatch issue, we now rename the profile variable for comdat functions. Profile allows co-existing multiple versions of profiles with different hash value. The inlined copy will always has the correct profile counter. The out-of-line copy might not have the correct count. But we will not have the bogus mismatch warning. Reviewers: davidxl Subscribers: llvm-commits, xur Differential Revision: https://reviews.llvm.org/D28416 llvm-svn: 291588
* Fix the test cases committed in r289521. Rong Xu2016-12-131-10/+8
| | | | llvm-svn: 289556
* llvm/test/Transforms/PGOProfile/noreturncall.ll REQUIRES asserts due to ↵NAKAMURA Takumi2016-12-131-0/+1
| | | | | | -debug-only. llvm-svn: 289522
* [PGO] Fix insane counts due to nonreturn callsRong Xu2016-12-132-0/+57
| | | | | | | | | | | | | | | | Summary: Since we don't break BBs for function calls. We might get some insane counts (wrap of unsigned) in the presence of noreturn calls. This patch sets these counts to zero instead of the wrapped number. Reviewers: davidxl Subscribers: xur, eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D27602 llvm-svn: 289521
* [PGO] Fix PGO use ICE when there are unreachable BBsRong Xu2016-12-022-0/+32
| | | | | | | | | | | | | | For -O0 there might be unreachable BBs, which breaks the assumption that all the BBs have an auxiliary data structure. In this patch, we add another interface called findBBInfo() so that a nullptr can be returned for the unreachable BBs (and the callers can ignore those BBs). This fixes the bug reported https://llvm.org/bugs/show_bug.cgi?id=31209 Differential Revision: https://reviews.llvm.org/D27280 llvm-svn: 288528
* [PGO] Fix select instruction annotationRong Xu2016-10-252-0/+48
| | | | | | | | | | | | | | | | Summary: Select instruction annotation in IR PGO uses the edge count to infer the branch count. It's currently placed in setInstrumentedCounts() where no all the BB counts have been computed. This leads to wrong branch weights. Move the annotation after all BB counts are populated. Reviewers: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D25961 llvm-svn: 285128
* [PGO] Create weak alias for the renamed Comdat functionRong Xu2016-10-061-0/+6
| | | | | | | | | | Add a weak alias to the renamed Comdat function in IR level instrumentation, using it's original name. This ensures the same behavior w/ and w/o IR instrumentation, even for non standard conforming code. Differential Revision: http://reviews.llvm.org/D25339 llvm-svn: 283490
* [Profile] Implement select instruction instrumentation in IR PGOXinliang David Li2016-09-182-0/+42
| | | | | | Differential Revision: http://reviews.llvm.org/D23727 llvm-svn: 281858
* [ThinLTO] Indirect call promotion fixes for promoted local functionsTeresa Johnson2016-08-292-3/+19
| | | | | | | | | | | | | | | | | | | Summary: Fix a couple issues limiting the application of indirect call promotion in ThinLTO mode: - Invoke indirect call promotion before globalopt, since it may eliminate imported functions which appear unreferenced. - Invoke indirect call promotion with InLTO=true so that the PGOFuncName metadata is used to get the name for locals which would have been renamed during promotion. Reviewers: davidxl, mehdi_amini Subscribers: Prazek, llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D24004 llvm-svn: 280024
* [Profile] add test with large countsXinliang David Li2016-08-202-0/+15
| | | | llvm-svn: 279361
* [Profile] improve warning control option Xinliang David Li2016-08-111-2/+6
| | | | | | | | | Change --no-pgo-warn-missing to -pgo-warn-missing-function and negate the default. /NFC Add more test to make sure the warning is off by default llvm-svn: 278314
* [Profile] turn off verbose warnings by defaultXinliang David Li2016-08-091-2/+2
| | | | | | | | | no prof data for func warning is turned off by default due to its high verbosity and minimal usefulness. Differential Revision: http://reviews.llvm.org/D23295 llvm-svn: 278127
* [PGO] Fix profile mismatch in COMDAT function with pre-inlinerRong Xu2016-07-254-11/+70
| | | | | | | | | | | | | | | | | | Pre-instrumentation inline (pre-inliner) greatly improves the IR instrumentation code performance, among other benefits. One issue of the pre-inliner is it can introduce CFG-mismatch for COMDAT functions. This is due to the fact that the same COMDAT function may have different early inline decisions across different modules -- that means different copies of COMDAT functions will have different CFG checksum. In this patch, we propose a partially renaming the COMDAT group and its member function/variable so we have different profile counter for each version. We will post-fix the COMDAT function and the group name with its FunctionHash. Differential Revision: http://reviews.llvm.org/D22600 llvm-svn: 276673
* [Profile] Use explicit flag to enable IR PGOXinliang David Li2016-07-231-1/+2
| | | | | | | | Patch by Jake VanAdrighem Differential Revision: http://reviews.llvm.org/D22607 llvm-svn: 276516
* [ThinLTO] Perform profile-guided indirect call promotionTeresa Johnson2016-07-172-0/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: To enable profile-guided indirect call promotion in ThinLTO mode, we simply add call graph edges for each profitable target from the profile to the summaries, then the summary-guided importing will consider the callee for importing as usual. Also we need to enable the indirect call promotion pass creation in the PassManagerBuilder when PerformThinLTO=true (we are in the ThinLTO backend), so that the newly imported functions are considered for promotion in the backends. The IC promotion profiles refer to callees by GUID, which required adding GUIDs to the per-module VST in bitcode (and assigning them valueIds similar to how they are assigned valueIds in the combined index). Reviewers: mehdi_amini, xur Subscribers: mehdi_amini, davidxl, llvm-commits Differential Revision: http://reviews.llvm.org/D21932 llvm-svn: 275707
* [PGO] IRPGO pre-cleanup pass changesRong Xu2016-07-151-0/+22
| | | | | | | | | | | | | | This patch adds a selected set of cleanup passes including a pre-inline pass before LLVM IR PGO instrumentation. The inline is only intended to apply those obvious/trivial ones before instrumentation so that much less instrumentation is needed to get better profiling information. This will drastically improve the instrumented code performance for large C++ applications. Another benefit is the context sensitive counts that can potentially improve the PGO optimization. Differential Revision: http://reviews.llvm.org/D21405 llvm-svn: 275588
* [PGO] Don't include full file path in static function profile counter namesXinliang David Li2016-07-121-0/+11
| | | | | | | Patch by Jake VanAdrighem Differential Revision: http://reviews.llvm.org/D22028 llvm-svn: 275193
* Fix PR28219: Use profile summary from reader and not compute itEaswaran Raman2016-06-212-0/+22
| | | | | | Differentiaal revision: http://reviews.llvm.org/D21546 llvm-svn: 273301
* [profile] value profiling bug fix -- missing icall targets in profile-useXinliang David Li2016-06-021-0/+11
| | | | | | | | | | | | | | | | | Inline virtual functions has linkeonceodr linkage (emitted in comdat on supporting targets). If the vtable for the class is not emitted in the defining module, function won't be address taken thus its address is not recorded. At the mercy of the linker, if the per-func prf_data from this module (in comdat) is picked at link time, we will lose mapping from function address to its hash val. This leads to missing icall promotion. The second test case (currently disabled) in compiler_rt (r271528): instrprof-icall-prom.test demostrates the bug. The first profile-use subtest is fine due to linker order difference. With this change, no missing icall targets is found in instrumented clang's raw profile. llvm-svn: 271532
* make icall pass name consistent /NFCXinliang David Li2016-06-022-4/+4
| | | | llvm-svn: 271467
* Bring back r271090 in a way that doesn't depend on r271089.Sean Silva2016-05-281-0/+3
| | | | llvm-svn: 271092
* Revert r271089 and r271090.Sean Silva2016-05-281-3/+0
| | | | | | | | | | | | | | It was triggering an msan bot. Revert "[IRPGO] Set the function entry count metadata." This reverts commit r271090. Revert "[IRPGO] Centralize the function attribute inliner hint logic. NFC." This reverts commit r271089. llvm-svn: 271091
* [IRPGO] Set the function entry count metadata.Sean Silva2016-05-281-0/+3
| | | | llvm-svn: 271090
* Attach profile summary in IR based instrumentation pass.Easwaran Raman2016-05-261-1/+3
| | | | | | Differential revision: http://reviews.llvm.org/D20655 llvm-svn: 270933
* [PM] Port indirect call promotion pass to new pass managerXinliang David Li2016-05-166-0/+7
| | | | llvm-svn: 269660
* [PM]: port IR based profUse pass to new pass managerXinliang David Li2016-05-1012-0/+16
| | | | llvm-svn: 269129
* [PGO] resubmit r268969Rong Xu2016-05-102-0/+13
| | | | | | Put the test into a target specific directory. llvm-svn: 269090
* Revert "[PGO] Fix __llvm_profile_raw_version linkage in MACHO IR ↵Renato Golin2016-05-101-10/+0
| | | | | | | | | | | | | instrumentation generates a COMDAT symbol __llvm_profile_raw_version to overwrite the same symbol in profile run-time to distinguish IR profiles from Clang generated profiles. In MACHO, LinkOnceODR linkage is used due to the lack of COMDAT support." This reverts commits r268969, r268979 and r268984. They had target specific test in generic directories without the correct specifiers and made it hard for us to come up with a good solution by rapidly committing untested changes. This test needs to be in a target specific directory or have the correct REQUIRED identifier. llvm-svn: 269027
* Fix buildbot failure from r268968.Rong Xu2016-05-091-1/+2
| | | | llvm-svn: 268984
OpenPOWER on IntegriCloud