summaryrefslogtreecommitdiffstats
path: root/llvm/lib/ProfileData
Commit message (Collapse)AuthorAgeFilesLines
* [SampleFDO] Add indexing for function profiles so they can be loaded on demandWei Mi2019-10-092-34/+125
| | | | | | | | | | | | | | | | | | | | | | | | in ExtBinary format Currently for Text, Binary and ExtBinary format profiles, when we compile a module with samplefdo, even if there is no function showing up in the profile, we have to load all the function profiles from the profile input. That is a waste of compile time. CompactBinary format profile has already had the support of loading function profiles on demand. In this patch, we add the support to load profile on demand for ExtBinary format. It will work no matter the sections in ExtBinary format profile are compressed or not. Experiment shows it reduces the time to compile a server benchmark by 30%. When profile remapping and loading function profiles on demand are both used, extra work needs to be done so that the loading on demand process will take the name remapping into consideration. It will be addressed in a follow-up patch. Differential Revision: https://reviews.llvm.org/D68601 llvm-svn: 374233
* Fix build errors caused by rL373914.Wei Mi2019-10-071-1/+2
| | | | llvm-svn: 373919
* [llvm-profdata] Minor format fixWenlei He2019-10-071-0/+1
| | | | | | | | | | | | | | Summary: Minor format fix for output of "llvm-profdata -show" Reviewers: wmi Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68440 llvm-svn: 373917
* [SampleFDO] Add compression support for any section in ExtBinary profile formatWei Mi2019-10-073-83/+158
| | | | | | | | | | | | | Previously ExtBinary profile format only supports compression using zlib for profile symbol list. In this patch, we extend the compression support to any section. User can select some or all of the sections to compress. In an experiment, for a 45M profile in ExtBinary format, compressing name table reduced its size to 24M, and compressing all the sections reduced its size to 11M. Differential Revision: https://reviews.llvm.org/D68253 llvm-svn: 373914
* [PGO] Fix typos from r359612. NFC.Rong Xu2019-10-013-9/+9
| | | | llvm-svn: 373369
* InstrProf - avoid static analyzer dyn_cast<ConstantInt> null dereference ↵Simon Pilgrim2019-10-011-4/+2
| | | | | | | | | | warning. The static analyzer is warning about a potential null dereference, as we're already earlying-out for a null Constant pointer I've just folded this into a dyn_cast_or_null<ConstantInt>. No test case, this is by inspection only. llvm-svn: 373322
* Recommit [SampleFDO] Expose an interface to return the size of a sectionWei Mi2019-09-211-0/+30
| | | | | | | | | | | | | | | | | | | | | or the size of the profile for profile in ExtBinary format. Fix a test failure on Mac. [SampleFDO] Expose an interface to return the size of a section or the size of the profile for profile in ExtBinary format. Sometimes we want to limit the size of the profile by stripping some functions with low sample count or by stripping some function names with small text size from profile symbol list. That requires the profile reader to have the interfaces returning the size of a section or the size of total profile. The patch add those interfaces. At the same time, add some dump facility to show the size of each section. Differential revision: https://reviews.llvm.org/D67726 llvm-svn: 372478
* Revert "[SampleFDO] Expose an interface to return the size of a section or ↵Amara Emerson2019-09-211-30/+0
| | | | | | | | | | the size" This reverts commit f118852046a1d255ed8c65c6b5db320e8cea53a0. Broke the macOS build/greendragon bots. llvm-svn: 372464
* [SampleFDO] Expose an interface to return the size of a section or the sizeWei Mi2019-09-201-0/+30
| | | | | | | | | | | | | | of the profile for profile in ExtBinary format. Sometimes we want to limit the size of the profile by stripping some functions with low sample count or by stripping some function names with small text size from profile symbol list. That requires the profile reader to have the interfaces returning the size of a section or the size of total profile. The patch add those interfaces. At the same time, add some dump facility to show the size of each section. llvm-svn: 372439
* [Object] Extend MachOUniversalBinary::getObjectForArchAlexander Shaposhnikov2019-09-191-1/+1
| | | | | | | | | | | | Make the method MachOUniversalBinary::getObjectForArch return MachOUniversalBinary::ObjectForArch and add helper methods MachOUniversalBinary::getMachOObjectForArch, MachOUniversalBinary::getArchiveForArch for those who explicitly expect to get a MachOObjectFile or an Archive. Differential revision: https://reviews.llvm.org/D67700 Test plan: make check-all llvm-svn: 372278
* [PGO] Change hardcoded thresholds for cold/inlinehint to use summaryTeresa Johnson2019-09-171-6/+6
| | | | | | | | | | | | | | | | | | | | | | | Summary: The PGO counter reading will add cold and inlinehint (hot) attributes to functions that are very cold or hot. This was using hardcoded thresholds, instead of the profile summary cutoffs which are used in other hot/cold detection and are more dynamic and adaptable. Switch to using the summary-based cold/hot detection. The hardcoded limits were causing some code that had a medium level of hotness (per the summary) to be incorrectly marked with a cold attribute, blocking inlining. Reviewers: davidxl Subscribers: llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D67673 llvm-svn: 372189
* [Coverage] Speed up file-based queries for coverage info, NFCVedant Kumar2019-09-161-2/+35
| | | | | | | | | | | | | | Speed up queries for coverage info in a file by reducing the amount of time spent determining whether a function record corresponds to a file. This gives a 36% speedup when generating a coverage report for `llc`. The reduction is entirely in user time. rdar://54758110 Differential Revision: https://reviews.llvm.org/D67575 llvm-svn: 372025
* [Coverage] Assert that filenames in a TU are unique, NFCVedant Kumar2019-09-161-0/+10
| | | | llvm-svn: 372024
* [InstrProf] Tighten a check for malformed data records in raw profilesVedant Kumar2019-09-031-4/+10
| | | | | | | | | | | | | The check needs to validate a counter offset before performing pointer arithmetic with the (potentially corrupt) offset. Found by UBSan's pointer overflow check. rdar://54843625 Differential Revision: https://reviews.llvm.org/D66979 llvm-svn: 370826
* Fix some errors introduced by rL370563 which were not exposed on my local ↵Wei Mi2019-08-311-1/+2
| | | | | | | | | machine. 1. zlib::compress accept &size_t but the param is an uint64_t. 2. Some systems don't have zlib installed. Don't use compression by default. llvm-svn: 370564
* [SampleFDO] Add profile symbol list section to discriminate function beingWei Mi2019-08-313-1/+115
| | | | | | | | | | | | | | | | | | | cold versus function being newly added. This is the second half of https://reviews.llvm.org/D66374. Profile symbol list is the collection of function symbols showing up in the binary which generates the current profile. It is used to discriminate function being cold versus function being newly added. Profile symbol list is only added for profile with ExtBinary format. During profile use compilation, when profile-sample-accurate is enabled, a function without profile will be regarded as cold only when it is contained in that list. Differential Revision: https://reviews.llvm.org/D66766 llvm-svn: 370563
* Ignore object files that lack coverage information.James Y Knight2019-08-281-2/+21
| | | | | | | | | | Before this change, if multiple binary files were presented, all of them must have been instrumented or the load would fail with coverage_map_error::no_data_found. Patch by Dean Sturtevant. Differential Revision: https://reviews.llvm.org/D66763 llvm-svn: 370257
* [SampleFDO] Extract the code calling each section reader to readOneSection.Wei Mi2019-08-261-20/+29
| | | | | | | | | | | | This is a followup of https://reviews.llvm.org/D66513. The code calling each section reader should be put into a separate function (readOneSection), so SampleProfileExtBinaryReader can override it. Otherwise, the base class SampleProfileExtBinaryBaseReader will need to be aware of all different kinds of section readers. That is not right. Differential Revision: https://reviews.llvm.org/D66693 llvm-svn: 369919
* [SampleFDO] Add ExtBinary format to support extension of binary profile.Wei Mi2019-08-232-29/+258
| | | | | | | | | | | | This is a patch split from https://reviews.llvm.org/D66374. It tries to add a new format of profile called ExtBinary. The format adds a section header table to the profile and organize the profile in sections, so the future extension like adding a new section or extending an existing section will be easier while keeping backward compatiblity feasible. Differential Revision: https://reviews.llvm.org/D66513 llvm-svn: 369798
* [AutoFDO] Make call targets order deterministic for sample profileWenlei He2019-08-202-6/+6
| | | | | | | | | | | | | | | | | Summary: StringMap is used for storing call target to frequency map for AutoFDO. However the iterating order of StringMap is non-deterministic, which leads to non-determinism in AutoFDO profile output. Now new API getSortedCallTargets and SortCallTargets are added for deterministic ordering and output. Roundtrip test for text profile and binary profile is added. Reviewers: wmi, davidxl, danielcdh Subscribers: hiraditya, mgrang, llvm-commits, twoh Tags: #llvm Differential Revision: https://reviews.llvm.org/D66191 llvm-svn: 369440
* [llvm] Migrate llvm::make_unique to std::make_uniqueJonas Devlieghere2019-08-155-20/+20
| | | | | | | | Now that we've moved to C++14, we no longer need the llvm::make_unique implementation from STLExtras.h. This patch is a mechanical replacement of (hopefully) all the llvm::make_unique instances across the monorepo. llvm-svn: 369013
* Recommit r368812 "[llvm/Object] - Convert SectionRef::getName() to return ↵George Rimar2019-08-141-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Expected<>" Changes: no changes. A fix for the clang code will be landed right on top. Original commit message: SectionRef::getName() returns std::error_code now. Returning Expected<> instead has multiple benefits. For example, it forces user to check the error returned. Also Expected<> may keep a valuable string error message, what is more useful than having a error code. (Object\invalid.test was updated to show the new messages printed.) This patch makes a change for all users to switch to Expected<> version. Note: in a few places the error returned was ignored before my changes. In such places I left them ignored. My intention was to convert the interface used, and not to improve and/or the existent users in this patch. (Though I think this is good idea for a follow-ups to revisit such places and either remove consumeError calls or comment each of them to clarify why it is OK to have them). Differential revision: https://reviews.llvm.org/D66089 llvm-svn: 368826
* Revert r368812 "[llvm/Object] - Convert SectionRef::getName() to return ↵George Rimar2019-08-141-4/+4
| | | | | | | | Expected<>" It broke clang BB: http://lab.llvm.org:8011/builders/clang-x86_64-debian-fast/builds/16455 llvm-svn: 368813
* [llvm/Object] - Convert SectionRef::getName() to return Expected<>George Rimar2019-08-141-4/+4
| | | | | | | | | | | | | | | | | | | | | | | SectionRef::getName() returns std::error_code now. Returning Expected<> instead has multiple benefits. For example, it forces user to check the error returned. Also Expected<> may keep a valuable string error message, what is more useful than having a error code. (Object\invalid.test was updated to show the new messages printed.) This patch makes a change for all users to switch to Expected<> version. Note: in a few places the error returned was ignored before my changes. In such places I left them ignored. My intention was to convert the interface used, and not to improve and/or the existent users in this patch. (Though I think this is good idea for a follow-ups to revisit such places and either remove consumeError calls or comment each of them to clarify why it is OK to have them). Differential revision: https://reviews.llvm.org/D66089 llvm-svn: 368812
* [llvm-profdata] Profile dump for compact binary formatWenlei He2019-08-131-6/+19
| | | | | | | | | | | | | | Summary: Fix "llvm-profdata show" so it can work with compact binary format profile. The change is to mark all functions "used" so SampleProfileReaderCompactBinary::read will read in all profiles available for dumping. The function names will be MD5 hash for compact binary format. Reviewers: wmi, davidxl, danielcdh Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D65162 llvm-svn: 368731
* [ThinLTO][AutoFDO] Fix memory corruption due to race condition from thin ↵Wenlei He2019-08-121-2/+0
| | | | | | | | | | | | | | | | | | backends Summary: This commit fixed a race condition from multi-threaded thinLTO backends that causes non-deterministic memory corruption for a data structure used only by AutoFDO with compact binary profile. GUIDToFuncNameMap, a static data member of type DenseMap in FunctionSamples is used as a per-module mapping from function name MD5 to name string when input AutoFDO profile is in compact binary format. However with ThinLTO, we can have parallel backends modifying and accessing the class static map concurrently. The fix is to make GUIDToFuncNameMap a member of SampleProfileLoader instead of a file static data. Reviewers: wmi, davidxl, danielcdh Subscribers: mehdi_amini, inglorion, hiraditya, dexonsmith, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D65848 llvm-svn: 368596
* Revert "test commit"Wenlei He2019-08-111-2/+0
| | | | | | This reverts commit ad92a4a2769425ad0d39ac1dbb6282f6f51a1af7. llvm-svn: 368525
* test commitWenlei He2019-08-111-0/+2
| | | | llvm-svn: 368524
* Rename F_{None,Text,Append} to OF_{None,Text,Append}. NFCFangrui Song2019-08-052-3/+3
| | | | | | F_{None,Text,Append} are kept for compatibility since r334221. llvm-svn: 367800
* [Profile] Support raw/indexed profiles larger than 4GBVedant Kumar2019-07-091-2/+2
| | | | | | rdar://45955976 llvm-svn: 365565
* Cleanup: llvm::bsearch -> llvm::partition_point after r364719Fangrui Song2019-06-301-6/+5
| | | | llvm-svn: 364720
* Simplify std::lower_bound with llvm::{bsearch,lower_bound}. NFCFangrui Song2019-06-211-3/+3
| | | | llvm-svn: 364006
* [Coverage] Speculative fix for r363325 for an older compilerVedant Kumar2019-06-141-4/+4
| | | | | | | It looks like an older version of gcc can't figure out that it needs to move a unique_ptr while implicitly constructing an Expected object. llvm-svn: 363342
* [Coverage] Load code coverage data from archivesVedant Kumar2019-06-132-71/+136
| | | | | | | | | | | | | Support loading code coverage data from regular archives, thin archives, and from MachO universal binaries which contain archives. Testing: check-llvm, check-profile (with {A,UB}San enabled) rdar://51538999 Differential Revision: https://reviews.llvm.org/D63232 llvm-svn: 363325
* Recommit [Object] Change object::SectionRef::getContents() to return ↵Fangrui Song2019-05-161-4/+9
| | | | | | | | | | | | Expected<StringRef> r360876 didn't fix 2 call sites in clang. Expected<ArrayRef<uint8_t>> may be better but use Expected<StringRef> for now. Follow-up of D61781. llvm-svn: 360892
* Revert r360876 "[Object] Change object::SectionRef::getContents() to return ↵Hans Wennborg2019-05-161-9/+4
| | | | | | | | | | | | Expected<StringRef>" It broke the Clang build, see llvm-commits thread. > Expected<ArrayRef<uint8_t>> may be better but use Expected<StringRef> for now. > > Follow-up of D61781. llvm-svn: 360878
* [Object] Change object::SectionRef::getContents() to return Expected<StringRef>Fangrui Song2019-05-161-4/+9
| | | | | | | | Expected<ArrayRef<uint8_t>> may be better but use Expected<StringRef> for now. Follow-up of D61781. llvm-svn: 360876
* Fix a few -Werror warnings:Eric Christopher2019-04-301-4/+3
| | | | | | | - Remove a variable only used in an assert - Fix pessimizing move warning around copy elision llvm-svn: 359617
* [llvm-profdata] Add overlap command to compute similarity b/w two profile filesRong Xu2019-04-303-0/+283
| | | | | | | | | Add overlap functionality to llvm-profdata tool to compute the similarity between two profile files. Differential Revision: https://reviews.llvm.org/D60977 llvm-svn: 359612
* The error message for mismatched value sites is very cryptic.Dmitry Mikulin2019-04-231-1/+1
| | | | | | | | Make it more readable for an average user. Differential Revision: https://reviews.llvm.org/D60896 llvm-svn: 359043
* Use llvm::stable_sortFangrui Song2019-04-233-16/+14
| | | | | | While touching the code, simplify if feasible. llvm-svn: 358996
* Change some StringRef::data() reinterpret_cast to bytes_begin() or ↵Fangrui Song2019-04-072-8/+5
| | | | | | arrayRefFromStringRef() llvm-svn: 357852
* [ProfileSummary] Count callsite samples when computing total samples.Taewook Oh2019-04-031-4/+9
| | | | | | | | | | | | | | Summary: Currently ProfileSummaryBuilder doesn't count into callsite samples when computing total samples. Considering that ProfileSummaryInfo is used to checked the hotness of not only body samples but also callsite samples (from SampleProfileLoader), I think the callsite sample counts should be considered when computing total samples. Reviewers: eraman, danielcdh, wmi Subscribers: hiraditya, jdoerfert, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D59835 llvm-svn: 357627
* [SampleFDO] add suffix elision control for fcn namesThan McIntosh2019-03-141-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Add hooks for determining the policy used to decide whether/how to chop off symbol 'suffixes' when locating a given function in a sample profile. Prior to this change, any function symbols of the form "X.Y" were elided/truncated into just "X" when looking up things in a sample profile data file. With this change, the policy on suffixes can be changed by adding a new attribute "sample-profile-suffix-elision-policy" to the function: this attribute can have the value "all" (the default), "selected", or "none". A value of "all" preserves the previous behavior (chop off everything after the first "." character, then treat that as the symbol name). A value of "selected" chops off only the rightmost ".llvm.XXXX" suffix (where "XXX" is any string not containing a "." char). A value of "none" indicates that names should be left as is. Subscribers: jdoerfert, wmi, mtrofin, danielcdh, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D58832 llvm-svn: 356146
* [PGO] Context sensitive PGO (part 4)Rong Xu2019-03-061-1/+1
| | | | | | | | | | | Part 4 of CSPGO changes: (1) add support in cmake for cspgo build. (2) fix an issue in big endian. (3) test cases. Differential Revision: https://reviews.llvm.org/D54175 llvm-svn: 355541
* [ProfileData] Sort FuncData before iteration to remove non-determinismMandeep Singh Grang2019-03-021-6/+22
| | | | | | | | | | | | | | Reviewers: rsmith, bogner, dblaikie Reviewed By: dblaikie Subscribers: Hahnfeld, jdoerfert, vsk, dblaikie, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D57986 llvm-svn: 355252
* [PGO] Context sensitive PGO (part 2)Rong Xu2019-02-282-8/+50
| | | | | | | | | | | Part 2 of CSPGO changes (mostly related to ProfileSummary). Note that I use a default parameter in setProfileSummary() and getSummary(). This is to break the dependency in clang. I will make the parameter explicit after changing clang in a separated patch. Differential Revision: https://reviews.llvm.org/D54175 llvm-svn: 355131
* Recommit r354930 "[PGO] Context sensitive PGO (part 1)"Rong Xu2019-02-271-0/+19
| | | | | | Fixed UBSan failures. llvm-svn: 355005
* Revert "[PGO] Context sensitive PGO (part 1)"Vlad Tsyrklevich2019-02-271-19/+0
| | | | | | This reverts commit r354930, it was causing UBSan failures. llvm-svn: 354953
* [PGO] Context sensitive PGO (part 1)Rong Xu2019-02-261-0/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Current PGO profile counts are not context sensitive. The branch probabilities for the inlined functions are kept the same for all call-sites, and they might be very different from the actual branch probabilities. These suboptimal profiles can greatly affect some downstream optimizations, in particular for the machine basic block placement optimization. In this patch, we propose to have a post-inline PGO instrumentation/use pass, which we called Context Sensitive PGO (CSPGO). For the users who want the best possible performance, they can perform a second round of PGO instrument/use on the top of the regular PGO. They will have two sets of profile counts. The first pass profile will be manly for inline, indirect-call promotion, and CGSCC simplification pass optimizations. The second pass profile is for post-inline optimizations and code-gen optimizations. A typical usage: // Regular PGO instrumentation and generate pass1 profile. > clang -O2 -fprofile-generate source.c -o gen > ./gen > llvm-profdata merge default.*profraw -o pass1.profdata // CSPGO instrumentation. > clang -O2 -fprofile-use=pass1.profdata -fcs-profile-generate -o gen2 > ./gen2 // Merge two sets of profiles > llvm-profdata merge default.*profraw pass1.profdata -o profile.profdata // Use the combined profile. Pass manager will invoke two PGO use passes. > clang -O2 -fprofile-use=profile.profdata -o use This change touches many components in the compiler. The reviewed patch (D54175) will committed in phrases. Differential Revision: https://reviews.llvm.org/D54175 llvm-svn: 354930
OpenPOWER on IntegriCloud