summaryrefslogtreecommitdiffstats
path: root/llvm/test/tools/llvm-profdata
Commit message (Collapse)AuthorAgeFilesLines
* [test] do not parse ls output for file size; NFCIBryan Chan2019-12-301-1/+1
| | | | | | | | Parsing `ls -l` output to obtain the size of a file is unreliable; the exact output format is not specified, and some user or group names may contain multiple words, causing `cut -f5 -d' '` to extract an incorrect value. `wc -c`, on the other hand, is portable, and there are precendents of its use in test cases.
* Add more diff -b to roundtrip-compress.testHans Wennborg2019-12-101-1/+1
| | | | | It was missing on the first test invocation. The flag is necessary to ignore line-ending differences on Windows.
* Use diff -b on zlib tests so they pass on WindowsAmy Huang2019-12-052-2/+2
| | | | | | | | | | Reviewers: hubert.reinterpretcast Subscribers: llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71036
* Revert "Add --strip-trailing-cr to compression tests so they pass on Windows."Xiangling Liao2019-12-042-2/+2
| | | | | | This reverts commit d6cbc9528d46d30416a6f9cd6c8570b704a0bd33. It causes the tests to fail on AIX.
* Add --strip-trailing-cr to compression tests so they pass on Windows.Amy Huang2019-12-042-2/+2
|
* Revert "Fixed a profdata file size detection on Windows system."Vladimir Vereschaka2019-11-101-1/+1
| | | | | | | | | | This reverts commit bcbb121ff6c8440382abfce8f2911a095f14602b. Using 'ls -o' is not compatible way to fix the problem. FreeBSD and OSX version of 'ls' do not support -o flag and test gets failed on these platforms. Differential Revision: https://reviews.llvm.org/D69317
* Fixed a profdata file size detection on Windows system.Vladimir Vereschaka2019-11-051-1/+1
| | | | | | | | | | | The space symbols are allowed in the group names on Windows system (as example: Domain Users). In that case the test extracts a wrong field from the output to get a size of the profdata file. This patch avoids a printing of the group names in the test output and extracts a proper field as a file size. Differential Revision: https://reviews.llvm.org/D69317
* [profile] Fourth fix for toolchains without zlib after D68351Vedant Kumar2019-10-311-0/+0
| | | | | Fix llvm/test/tools/llvm-profdata/nocompress.test, which I missed when authoring D68351 because my toolchain is configured with zlib support.
* [profile] Add a mode to continuously sync counter updates to a fileVedant Kumar2019-10-318-9/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add support for continuously syncing profile counter updates to a file. The motivation for this is that programs do not always exit cleanly. On iOS, for example, programs are usually killed via a signal from the OS. Running atexit() handlers after catching a signal is unreliable, so some method for progressively writing out profile data is necessary. The approach taken here is to mmap() the `__llvm_prf_cnts` section onto a raw profile. To do this, the linker must page-align the counter and data sections, and the runtime must ensure that counters are mapped to a page-aligned offset within a raw profile. Continuous mode is (for the moment) incompatible with the online merging mode. This limitation is lifted in https://reviews.llvm.org/D69586. Continuous mode is also (for the moment) incompatible with value profiling, as I'm not sure whether there is interest in this and the implementation may be tricky. As I have not been able to test extensively on non-Darwin platforms, only Darwin support is included for the moment. However, continuous mode may "just work" without modification on Linux and some UNIX-likes. AIUI the default value for the GNU linker's `--section-alignment` flag is set to the page size on many systems. This appears to be true for LLD as well, as its `no_nmagic` option is on by default. Continuous mode will not "just work" on Fuchsia or Windows, as it's not possible to mmap() a section on these platforms. There is a proposal to add a layer of indirection to the profile instrumentation to support these platforms. rdar://54210980 Differential Revision: https://reviews.llvm.org/D68351
* Fix non-portable GNU diff optionstevewan2019-10-233-7/+7
| | | | | | | | | | | | | | Summary: This is a fix to revision D68839 and rL375023. This patch substitutes POSIX option "-b" for the non-portable GNU option "--strip-trailing-cr". Reviewers: daltenty, hubert.reinterpretcast Reviewed By: daltenty Subscribers: mehdi_amini, hiraditya, steven_wu, dexonsmith, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69342
* [lit] Fix internal diff's --strip-trailing-cr and use itJoel E. Denny2019-10-163-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | Using GNU diff, `--strip-trailing-cr` removes a `\r` appearing before a `\n` at the end of a line. Without this patch, lit's internal diff only removes `\r` if it appears as the last character. That seems useless. This patch fixes that. This patch also adds `--strip-trailing-cr` to some tests that fail on Windows bots when D68664 is applied. Based on what I see in the bot logs, I think the following is happening. In each test there, lit diff is comparing a file with `\r\n` line endings to a file with `\n` line endings. Without D68664, lit diff reads those files in text mode, which in Windows causes `\r\n` to be replaced with `\n`. However, with D68664, lit diff reads the files in binary mode instead and thus reports that every line is different, just as GNU diff does (at least under Ubuntu). Adding `--strip-trailing-cr` to those tests restores the previous behavior while permitting the behavior of lit diff to be more like GNU diff. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D68839 llvm-svn: 375020
* [llvm-profdata] Reinstate ↵Vedant Kumar2019-10-151-0/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | tools/llvm-profdata/malformed-ptr-to-counter-array.test I removed this test to unblock the ARM bots while looking into failures (r374915), and am reinstating it now with a fix. I believe the problem was that counter ptr address I used, '\0\0\6\0\1\0\0\1', set the high bits of the pointer, not the low bits like I wanted. On x86_64 this superficially looks like it tests r370826, but it doesn't, as it would have been caught before r370826. However, on ARM (or, 32-bit hosts more generally), I suspect the high bits were cleared, and you get a 'valid' profile. I verified that setting the *low* bits of the pointer does trigger the new condition: -// Note: The CounterPtr here is off-by-one. This should trigger a malformed profile error. -RUN: printf '\0\0\6\0\1\0\0\1' >> %t.profraw +// Note: The CounterPtr here is off-by-one. +// +// Octal '\11' is 9 in decimal: this should push CounterOffset to 1. As there are two counters, +// the profile reader should error out. +RUN: printf '\11\0\6\0\1\0\0\0' >> %t.profraw This reverts commit c7cf5b3e4b918c9769fd760f28485b8d943ed968. llvm-svn: 374927
* [llvm-profdata] Remove tools/llvm-profdata/malformed-ptr-to-counter-array.testVedant Kumar2019-10-151-48/+0
| | | | | | | This test is still failing on the ARM bots and I need time to investigate. llvm-svn: 374915
* [llvm-profdata] Weaken "malformed-ptr-to-counter-array.test" to appease arm botsVedant Kumar2019-10-141-3/+1
| | | | | | | | | | | | | | There are a number arm bots failing after r374617 landed, and I'm not sure why. It looks a bit like the error message llvm-profdata is expected to print to stderr isn't flushed. Weaken the test in an attempt to appease the arm bots: if this doesn't work, that means that llvm-profdata is actually *not failing*, and that will be a clear indication that some logic error is actually happening. http://lab.llvm.org:8011/builders/clang-cmake-armv7-global-isel/builds/5604/ llvm-svn: 374792
* Revert r374652: "[lit] Fix internal diff's --strip-trailing-cr and use it"Joel E. Denny2019-10-123-7/+7
| | | | | | This series of patches still breaks a Windows bot. llvm-svn: 374679
* [lit] Fix internal diff's --strip-trailing-cr and use itJoel E. Denny2019-10-123-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | Using GNU diff, `--strip-trailing-cr` removes a `\r` appearing before a `\n` at the end of a line. Without this patch, lit's internal diff only removes `\r` if it appears as the last character. That seems useless. This patch fixes that. This patch also adds `--strip-trailing-cr` to some tests that fail on Windows bots when D68664 is applied. Based on what I see in the bot logs, I think the following is happening. In each test there, lit diff is comparing a file with `\r\n` line endings to a file with `\n` line endings. Without D68664, lit diff reads those files with Python's universal newlines support activated, causing `\r` to be dropped. However, with D68664, lit diff reads the files in binary mode instead and thus reports that every line is different, just as GNU diff does (at least under Ubuntu). Adding `--strip-trailing-cr` to those tests restores the previous behavior while permitting the behavior of lit diff to be more like GNU diff. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D68839 llvm-svn: 374652
* [llvm-profdata] Make "malformed-ptr-to-counter-array.test" textualVedant Kumar2019-10-122-2/+47
| | | | | | | | | As pointed out in https://reviews.llvm.org/D66979 post-commit, making this test textual would make it more maintainable. Differential Revision: https://reviews.llvm.org/D68718 llvm-svn: 374617
* [SampleFDO] Add compression support for any section in ExtBinary profile formatWei Mi2019-10-072-0/+16
| | | | | | | | | | | | | Previously ExtBinary profile format only supports compression using zlib for profile symbol list. In this patch, we extend the compression support to any section. User can select some or all of the sections to compress. In an experiment, for a 45M profile in ExtBinary format, compressing name table reduced its size to 24M, and compressing all the sections reduced its size to 11M. Differential Revision: https://reviews.llvm.org/D68253 llvm-svn: 373914
* Recommit [SampleFDO] Expose an interface to return the size of a sectionWei Mi2019-09-211-0/+7
| | | | | | | | | | | | | | | | | | | | | or the size of the profile for profile in ExtBinary format. Fix a test failure on Mac. [SampleFDO] Expose an interface to return the size of a section or the size of the profile for profile in ExtBinary format. Sometimes we want to limit the size of the profile by stripping some functions with low sample count or by stripping some function names with small text size from profile symbol list. That requires the profile reader to have the interfaces returning the size of a section or the size of total profile. The patch add those interfaces. At the same time, add some dump facility to show the size of each section. Differential revision: https://reviews.llvm.org/D67726 llvm-svn: 372478
* Revert "[SampleFDO] Expose an interface to return the size of a section or ↵Amara Emerson2019-09-211-7/+0
| | | | | | | | | | the size" This reverts commit f118852046a1d255ed8c65c6b5db320e8cea53a0. Broke the macOS build/greendragon bots. llvm-svn: 372464
* [SampleFDO] Expose an interface to return the size of a section or the sizeWei Mi2019-09-201-0/+7
| | | | | | | | | | | | | | of the profile for profile in ExtBinary format. Sometimes we want to limit the size of the profile by stripping some functions with low sample count or by stripping some function names with small text size from profile symbol list. That requires the profile reader to have the interfaces returning the size of a section or the size of total profile. The patch add those interfaces. At the same time, add some dump facility to show the size of each section. llvm-svn: 372439
* [llvm-profdata] Add mode to recover from profile read failuresVedant Kumar2019-09-032-7/+16
| | | | | | | | | | Add a mode in which profile read errors are not immediately treated as fatal. In this mode, merging makes forward progress and reports failure only if no inputs can be read. Differential Revision: https://reviews.llvm.org/D66985 llvm-svn: 370827
* [InstrProf] Tighten a check for malformed data records in raw profilesVedant Kumar2019-09-032-0/+5
| | | | | | | | | | | | | The check needs to validate a counter offset before performing pointer arithmetic with the (potentially corrupt) offset. Found by UBSan's pointer overflow check. rdar://54843625 Differential Revision: https://reviews.llvm.org/D66979 llvm-svn: 370826
* [SampleFDO] Add profile symbol list section to discriminate function beingWei Mi2019-08-314-0/+56
| | | | | | | | | | | | | | | | | | | cold versus function being newly added. This is the second half of https://reviews.llvm.org/D66374. Profile symbol list is the collection of function symbols showing up in the binary which generates the current profile. It is used to discriminate function being cold versus function being newly added. Profile symbol list is only added for profile with ExtBinary format. During profile use compilation, when profile-sample-accurate is enabled, a function without profile will be regarded as cold only when it is contained in that list. Differential Revision: https://reviews.llvm.org/D66766 llvm-svn: 370563
* [SampleFDO] Add ExtBinary format to support extension of binary profile.Wei Mi2019-08-231-1/+10
| | | | | | | | | | | | This is a patch split from https://reviews.llvm.org/D66374. It tries to add a new format of profile called ExtBinary. The format adds a section header table to the profile and organize the profile in sections, so the future extension like adding a new section or extending an existing section will be easier while keeping backward compatiblity feasible. Differential Revision: https://reviews.llvm.org/D66513 llvm-svn: 369798
* [AutoFDO] Make call targets order deterministic for sample profileWenlei He2019-08-203-6/+9
| | | | | | | | | | | | | | | | | Summary: StringMap is used for storing call target to frequency map for AutoFDO. However the iterating order of StringMap is non-deterministic, which leads to non-determinism in AutoFDO profile output. Now new API getSortedCallTargets and SortCallTargets are added for deterministic ordering and output. Roundtrip test for text profile and binary profile is added. Reviewers: wmi, davidxl, danielcdh Subscribers: hiraditya, mgrang, llvm-commits, twoh Tags: #llvm Differential Revision: https://reviews.llvm.org/D66191 llvm-svn: 369440
* [llvm-profdata] Profile dump for compact binary formatWenlei He2019-08-132-0/+8
| | | | | | | | | | | | | | Summary: Fix "llvm-profdata show" so it can work with compact binary format profile. The change is to mark all functions "used" so SampleProfileReaderCompactBinary::read will read in all profiles available for dumping. The function names will be MD5 hash for compact binary format. Reviewers: wmi, davidxl, danielcdh Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D65162 llvm-svn: 368731
* [llvm-profdata] Don't make the output overwrite the input file.Haojian Wu2019-07-091-4/+4
| | | | | | | Some file systems may not allow this behavior, the test fails on our internal system ("Permission denied"). llvm-svn: 365450
* [llvm-profdata] Fix buildbot failure on llvm-clang-x86_64-expensive-checks-winRong Xu2019-07-081-1/+1
| | | | | | | This fixes buildbot failure in LLVM on llvm-clang-x86_64-expensive-checks-win from r365386. llvm-svn: 365401
* llvm-profdata] Handle the cases of overlapping input file and output fileRong Xu2019-07-085-0/+79
| | | | | | | | | | | | | | | | | | | | Currently llvm-profdata does not expect the same file name for the input profile and the output profile. >llvm-profdata merge A.profraw B.profraw -o B.profraw The above command runs successfully but the resulted B.profraw is not correct. This patch fixes the issue by moving the initialization of writer after loading the profile. For the show command, the following will report a confusing error of "Empty raw profile file": >llvm-profdata show B.profraw -o B.profraw It's harder to fix as we need to output something before loading the input profile. I don't think that a fix for this is worth the effort. I just make the error explicit for the show command. Differential Revision: https://reviews.llvm.org/D64360 llvm-svn: 365386
* Replace 'REQUIRES: nozlib' with '!zlib' because we don't need two waysPaul Robinson2019-05-101-1/+1
| | | | | | to say the same thing. llvm-svn: 360455
* [tests] Add host-byteorder-*-endian; update XFAILs of big-endian triplesHubert Tong2019-05-011-1/+1
| | | | | | | | | | | | | | | | | | | | Summary: Triple components in `XFAIL` lines are tested against the target triple. Various tests that are expected to fail on big-endian hosts are marked as being `XFAIL` for big-endian targets. This patch corrects these tests by having them test against a new `host-byteorder-big-endian` feature. Reviewers: xingxue, sfertile, jasonliu Reviewed By: xingxue Subscribers: jvesely, nhaehnle, fedor.sergeev, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60551 llvm-svn: 359689
* [llvm-profdata] Add overlap command to compute similarity b/w two profile filesRong Xu2019-04-309-0/+191
| | | | | | | | | Add overlap functionality to llvm-profdata tool to compute the similarity between two profile files. Differential Revision: https://reviews.llvm.org/D60977 llvm-svn: 359612
* [PGO] Context sensitive PGO (part 4)Rong Xu2019-03-064-0/+63
| | | | | | | | | | | Part 4 of CSPGO changes: (1) add support in cmake for cspgo build. (2) fix an issue in big endian. (3) test cases. Differential Revision: https://reviews.llvm.org/D54175 llvm-svn: 355541
* [llvm-profdata] add value-cutoff functionality in show commandRong Xu2019-01-083-3/+47
| | | | | | | | | | | | | This patch improves llvm-profdata show command: (1) add -value-cutoff=<N> option: Show only those functions whose max count values are greater or equal to N. (2) add -list-below-cutoff option: Only output names of functions whose max count value are below the cutoff. (3) formats value-profile counts and prints out percentage. Differential Revision: https://reviews.llvm.org/D56342 llvm-svn: 350673
* [PGO] Revert r350579 to fix commit message.Rong Xu2019-01-083-47/+3
| | | | | | Will re-commit it using the correct commit message. llvm-svn: 350670
* [PGO] Use SourceFileName rather module name in PGOFuncNameRong Xu2019-01-073-3/+47
| | | | | | | | | | In LTO or Thin-lto mode (though linker plugin), the module names are of temp file names which are different for different compilations. Using SourceFileName avoids the issue. This should not change any functionality for current PGO as all the current callers of getPGOFuncName() is before LTO. llvm-svn: 350579
* Add flag to llvm-profdata to allow symbols in profile data to be remapped, andRichard Smith2018-09-138-0/+95
| | | | | | | | | | | | | | | | | add a tool to generate symbol remapping files. Summary: The new tool llvm-cxxmap builds a symbol mapping table from a file containing a description of partial equivalences to apply to mangled names and files containing old and new symbol tables. Reviewers: davidxl Subscribers: mgorny, llvm-commits Differential Revision: https://reviews.llvm.org/D51470 llvm-svn: 342168
* Make llvm-profdata show -text work as advertised in the documentation.Richard Smith2018-08-241-0/+6
| | | | | | | | | | | | | | | | | | | Per LLVM's CommandGuide, llvm-profdata show -text is supposed to produce textual output that can be passed as input to further llvm-profdata invocations. This previously didn't work for two reasons: 1) -text was not sufficient to enable the machine-readable text format output; instead, -text was effectively ignored if -counts was not also specified. (With this patch, -counts is instead ignored if -text is specified, because the machine-readable text format always includes counts.) 2) When the input data was an IR-level profile, the :ir marker was missing from the output, resulting in a text format output that would not be usable as profiling data due to function hash mismatches. Differential Revision: https://reviews.llvm.org/D51188 llvm-svn: 340592
* Revert "Revert "[lit] Generalized /dev/null support on Windows.""Mircea Trofin2018-03-271-1/+1
| | | | | | | | | | | | | | | | | Summary: This reverts commit r328596. Checking if the arguments are strings before testing if they contain "/dev/null". Reviewers: rnk Reviewed By: rnk Subscribers: delcypher, llvm-commits Differential Revision: https://reviews.llvm.org/D44914 llvm-svn: 328603
* Revert "[lit] Generalized /dev/null support on Windows."Mircea Trofin2018-03-261-1/+1
| | | | | | This reverts commit ca7fdbb974384ce5a05528b22a41d46b1cc13e92. llvm-svn: 328596
* [lit] Generalized /dev/null support on Windows.Mircea Trofin2018-03-261-1/+1
| | | | | | | | | | | | | | Generalized /dev/null remapping on Windows, and added test. Reviewers: rnk Reviewed By: rnk Subscribers: amccarth, zturner, delcypher, llvm-commits Differential Revision: https://reviews.llvm.org/D44771 llvm-svn: 328589
* Revert "Revert "[InstrProf] Support for external functions in text format.""Mircea Trofin2018-03-221-0/+50
| | | | | | | | | | | | | | | | | | | | | Summary: This reverts commit 364eb09576a7667bc6d3ff80c52a83014ccac976 and separates out the portion that was fixing binary reader error propagation - turns out, there are production cases where that causes a regression. Will re-introduce the error propagation fix separately. The fix to the text reader error propagation is still "in". Reviewers: bkramer Reviewed By: bkramer Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D44807 llvm-svn: 328244
* Revert "[InstrProf] Support for external functions in text format."Benjamin Kramer2018-03-221-50/+0
| | | | | | | This reverts commit r328132. Breaks FDO selfhost. I'm seeing error: /tmp/profraw: Invalid instrumentation profile data (bad magic) llvm-svn: 328207
* [llvm-profdata] Use "-o /dev/null" in invalid-profdata.testReid Kleckner2018-03-211-1/+1
| | | | | | Lit automatically rewrites /dev/null to a temp file on Windows. llvm-svn: 328157
* [InstrProf] Support for external functions in text format.Mircea Trofin2018-03-211-0/+50
| | | | | | | | | | | | | | | | | | | | | | | Summary: External functions appearing as indirect call targets could not be found in the SymTab, and the value:counter record was represented, in the text format, using an empty string for the name. This would then cause a silent parsing error when reading. This CL: - adds explicit support for such functions - fixes the places where we would not propagate errors when reading - addresses a performance issue due to eager resorting of the SymTab. Reviewers: xur, eraman, davidxl Reviewed By: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D44717 llvm-svn: 328132
* Profiling tests: Endianess XFAIL for powerpc- (32-bit)Hubert Tong2018-01-091-1/+1
| | | | | | | | | | | | Add powerpc- (32-bit) as XFAIL for tests that are documented either in- line or via commit messages as expected to fail on big-endian systems. Tests not documented in-line are documented in commit messages as follows: r211172 - test/tools/llvm-cov/llvm-cov.test r247920 - test/Transforms/SampleProfile/gcc-simple.ll llvm-svn: 322114
* [llvm-profdata] Don't treat non-fatal merge errors as fatalVedant Kumar2017-11-172-1/+10
| | | | | | | | | This fixes an issue seen on the coverage bot: http://lab.llvm.org:8080/green/view/Experimental/job/clang-stage2-coverage-R/1930 Profile merging shouldn't fail if a single counter mismatch is detected. llvm-svn: 318555
* [llvm-profdata] Fix a dangling reference to an error stringVedant Kumar2017-11-175-0/+32
| | | | llvm-svn: 318502
* Adjust test after r318159Adam Nemet2017-11-141-0/+1
| | | | llvm-svn: 318160
OpenPOWER on IntegriCloud