| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
When forking in several threads, the counters were written out in using the same global static variables (see GCDAProfiling.c): that leads to crashes.
So when there is a fork, the counters are resetted in the child process and they will be dumped at exit using the interprocess file locking.
When there is an exec, the counters are written out and in case of failures they're resetted.
Reviewers: jfb, vsk, marco-c, serge-sans-paille
Reviewed By: marco-c, serge-sans-paille
Subscribers: llvm-commits, serge-sans-paille, dmajor, cfe-commits, hiraditya, dexonsmith, #sanitizers, marco-c, sylvestre.ledru
Tags: #sanitizers, #clang, #llvm
Differential Revision: https://reviews.llvm.org/D78477
(cherry picked from commit bec223a9bc4eb9747993ee9a4c1aa135c32123e6)
|
|
|
|
|
|
| |
See the discussion on PR44792.
This reverts commit 02ce9d8ef5a84bc884de4105eae5f8736ef67634.
|
|
|
|
|
|
|
|
|
|
|
| |
The executable acquires an advisory record lock (`fcntl(fd, F_SETLKW, *)`) on a profile file.
Merge pool size >= 10 may be beneficial when the concurrency is large.
Also fix a small problem about snprintf. It can cause the filename to be truncated after %m.
Reviewed By: davidxl
Differential Revision: https://reviews.llvm.org/D71970
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
Counters can be flushed in a multi-threaded context for example when the process is forked in different threads (https://github.com/llvm/llvm-project/blob/master/llvm/lib/Transforms/Instrumentation/GCOVProfiling.cpp#L632-L663).
In order to avoid pretty bad things, a critical section is needed around the flush.
We had a lot of crashes in this code in Firefox CI when we switched to clang for linux ccov builds and those crashes disappeared with this patch.
Reviewers: marco-c, froydnj, dmajor, davidxl, vsk
Reviewed By: marco-c, dmajor
Subscribers: ahatanak, froydnj, dmajor, dberris, jfb, #sanitizers, llvm-commits, sylvestre.ledru
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D70910
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When writing out a profile, avoid allocating a page on the stack for the
purpose of writing out zeroes, as some embedded environments do not have
enough stack space to accomodate this.
Instead, use a small, fixed-size zero buffer that can be written
repeatedly.
For a synthetic file with >100,000 functions, I did not measure a
significant difference in profile write times. We are removing a
page-length zero-fill `memset()` in favor of several smaller buffered
`fwrite()` calls: in practice, I am not sure there is much of a
difference. The performance impact is only expected to affect the
continuous sync mode (%c) -- zero padding is less than 8 bytes in all
other cases.
rdar://57810014
Differential Revision: https://reviews.llvm.org/D71323
|
|
|
|
|
|
|
| |
This reverts commit 88f5bf77f92899b19fdafdffc7b060f930c1cb8b as it broke
green dragon bots.
http://lab.llvm.org:8080/green/job/clang-stage1-RA/4401/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
Counters can be flushed in a multi-threaded context for example when the process is forked in different threads (https://github.com/llvm/llvm-project/blob/master/llvm/lib/Transforms/Instrumentation/GCOVProfiling.cpp#L632-L663).
In order to avoid pretty bad things, a critical section is needed around the flush.
We had a lot of crashes in this code in Firefox CI when we switched to clang for linux ccov builds and those crashes disappeared with this patch.
Reviewers: marco-c, froydnj, dmajor, davidxl
Reviewed By: marco-c, dmajor
Subscribers: froydnj, dmajor, dberris, jfb, #sanitizers, llvm-commits, sylvestre.ledru
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D70910
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
-fprofile-generate
See PR43425:
https://bugs.llvm.org/show_bug.cgi?id=43425
When writing profile data on Windows we were opening profile file with
exclusive read/write access.
In case we are trying to write to the file from multiple processes
simultaneously, subsequent calls to CreateFileA would return
INVALID_HANDLE_VALUE.
To fix this, I changed to open without exclusive access and then take a
lock.
Patch by Michael Holman!
Differential revision: https://reviews.llvm.org/D70330
|
|
|
|
|
|
|
|
|
|
|
| |
This header fragment is useful on its own for any consumer that wants
to use custom instruction profile runtime with the LLVM instrumentation.
The concrete use case is in Fuchsia's kernel where we want to use
instruction profile instrumentation, but we cannot use the compiler-rt
runtime because it's not designed for use in the kernel environment.
This change allows installing this header as part of compiler-rt.
Differential Revision: https://reviews.llvm.org/D64532
|
|
|
|
|
| |
This reverts commit f11bc1776fd2815b60e0b1ed97be00b517348162 since it's
failing to build on some bots.
|
|
|
|
|
|
|
|
|
|
|
| |
This header fragment is useful on its own for any consumer that wants
to use custom instruction profile runtime with the LLVM instrumentation.
The concrete use case is in Fuchsia's kernel where we want to use
instruction profile instrumentation, but we cannot use the compiler-rt
runtime because it's not designed for use in the kernel environment.
This change allows installing this header as part of compiler-rt.
Differential Revision: https://reviews.llvm.org/D64532
|
|
|
|
|
| |
This '#ifdef's out two functions which are unused on Windows, to prevent
-Wunused-function warnings.
|
|
|
|
|
|
| |
Continuous mode is not yet supported on Fuchsia/Windows, however an
error should not be reported unless the user attempted to actually
enable continuous mode.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Make it possible to use online profile merging ("%m" mode) with
continuous sync ("%c" mode).
To implement this, the merged profile is locked in the runtime
initialization step and either a) filled out for the first time or b)
checked for compatibility. Then, the profile can simply be mmap()'d with
MAP_SHARED set. With the mmap() in place, counter updates from every
process which uses an image are mapped onto the same set of physical
pages assigned by the filesystem cache. After the mmap() is set up, the
profile is unlocked.
Differential Revision: https://reviews.llvm.org/D69586
|
|
|
|
|
|
|
|
|
|
|
| |
Split out the logic to get the size of a merged profile and to do a
compatibility check. This can be shared with both the continuous+merging
mode implementation, as well as the runtime-allocated counters
implementation planned for Fuchsia.
Lifted out of D69586.
Differential Revision: https://reviews.llvm.org/D70135
|
|
|
|
|
|
|
|
| |
A few years back a similar change was made for getenv since neither function is supported on the PS4 platform.
Recently, commit d889d1e added a call to setenv in compiler-rt which was causing linking errors because the symbol was not found. This fixes that issue by putting in a shim similar to how we previously dealt with the lack of getenv.
Differential Revision: https://reviews.llvm.org/D70033
|
|
|
|
|
|
|
|
| |
This commit added use of a Windows API in InstrProfilingPort.h.
When _MSC_VER is defined (for MSVC), windows.h is already included
earlier in the same header (for atomics), but MinGW, the gcc
atomics builtins are used instead. Therefore explicitly include
windows.h here, where the API is used.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Use the printf macros from inttypes.h to sidestep -Wformat issues:
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/profile/InstrProfilingFile.c:425:14: error: format specifies type 'long long' but the argument has type 'off_t' (aka 'long') [-Werror,-Wformat]
CurrentFileOffset, PageSize);
^~~~~~~~~~~~~~~~~
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/profile/InstrProfilingPort.h:114:50: note: expanded from macro 'PROF_ERR'
fprintf(stderr, "LLVM Profile Error: " Format, __VA_ARGS__);
~~~~~~ ^~~~~~~~~~~
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/profile/InstrProfilingFile.c:461:41: error: format specifies type 'unsigned long long' but the argument has type 'uint64_t' (aka 'unsigned long') [-Werror,-Wformat]
strerror(errno), CountersBegin, PageAlignedCountersLength, Fileno,
^~~~~~~~~~~~~~~~~~~~~~~~~
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/profile/InstrProfilingPort.h:114:50: note: expanded from macro 'PROF_ERR'
fprintf(stderr, "LLVM Profile Error: " Format, __VA_ARGS__);
~~~~~~ ^~~~~~~~~~~
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/profile/InstrProfilingFile.c:462:9: error: format specifies type 'unsigned long long' but the argument has type 'uint64_t' (aka 'unsigned long') [-Werror,-Wformat]
FileOffsetToCounters);
^~~~~~~~~~~~~~~~~~~~
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/profile/InstrProfilingPort.h:114:50: note: expanded from macro 'PROF_ERR'
fprintf(stderr, "LLVM Profile Error: " Format, __VA_ARGS__);
|
|
|
|
|
|
|
| |
_putenv on Windows takes 1 argument, whereas setenv elsewhere takes 3.
Just treat the two platforms differently.
http://lab.llvm.org:8011/builders/sanitizer-windows/builds/53547
|
|
|
|
|
|
|
|
| |
VLAs in C appear to not work on Windows, so use COMPILER_RT_ALLOCA:
C:\b\slave\sanitizer-windows\llvm-project\compiler-rt\lib\profile\InstrProfilingWriter.c(264): error C2057: expected constant expression
C:\b\slave\sanitizer-windows\llvm-project\compiler-rt\lib\profile\InstrProfilingWriter.c(264): error C2466: cannot allocate an array of constant size 0
C:\b\slave\sanitizer-windows\llvm-project\compiler-rt\lib\profile\InstrProfilingWriter.c(264): error C2133: 'Zeroes': unknown size
|
|
|
|
|
|
| |
setenv() appears to not be available on Windows:
http://lab.llvm.org:8011/builders/sanitizer-windows/builds/53545/steps/stage%201%20build/logs/stdio
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add support for continuously syncing profile counter updates to a file.
The motivation for this is that programs do not always exit cleanly. On
iOS, for example, programs are usually killed via a signal from the OS.
Running atexit() handlers after catching a signal is unreliable, so some
method for progressively writing out profile data is necessary.
The approach taken here is to mmap() the `__llvm_prf_cnts` section onto
a raw profile. To do this, the linker must page-align the counter and
data sections, and the runtime must ensure that counters are mapped to a
page-aligned offset within a raw profile.
Continuous mode is (for the moment) incompatible with the online merging
mode. This limitation is lifted in https://reviews.llvm.org/D69586.
Continuous mode is also (for the moment) incompatible with value
profiling, as I'm not sure whether there is interest in this and the
implementation may be tricky.
As I have not been able to test extensively on non-Darwin platforms,
only Darwin support is included for the moment. However, continuous mode
may "just work" without modification on Linux and some UNIX-likes. AIUI
the default value for the GNU linker's `--section-alignment` flag is set
to the page size on many systems. This appears to be true for LLD as
well, as its `no_nmagic` option is on by default. Continuous mode will
not "just work" on Fuchsia or Windows, as it's not possible to mmap() a
section on these platforms. There is a proposal to add a layer of
indirection to the profile instrumentation to support these platforms.
rdar://54210980
Differential Revision: https://reviews.llvm.org/D68351
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When the %m filename pattern is used, the filename is unique to each
image, so the cached value is wrong.
It struck me that the full filename isn't something that's recomputed
often, so perhaps it doesn't need to be cached at all. David Li pointed
out we can go further and just hide lprofCurFilename. This may regress
workflows that depend on using the set-filename API to change filenames
across all loaded DSOs, but this is expected to be very rare.
rdar://55137071
Differential Revision: https://reviews.llvm.org/D69137
llvm-svn: 375301
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Annotations in LLVM"
This reverts commit r371584. It introduced a dependency from compiler-rt
to llvm/include/ADT, which is problematic for multiple reasons.
One is that it is a novel dependency edge, which needs cross-compliation
machinery for llvm/include/ADT (yes, it is true that right now
compiler-rt included only header-only libraries, however, if we allow
compiler-rt to depend on anything from ADT, other libraries will
eventually get used).
Secondly, depending on ADT from compiler-rt exposes ADT symbols from
compiler-rt, which would cause ODR violations when Clang is built with
the profile library.
llvm-svn: 371598
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch contains the basic functionality for reporting potentially
incorrect usage of __builtin_expect() by comparing the developer's
annotation against a collected PGO profile. A more detailed proposal and
discussion appears on the CFE-dev mailing list
(http://lists.llvm.org/pipermail/cfe-dev/2019-July/062971.html) and a
prototype of the initial frontend changes appear here in D65300
We revised the work in D65300 by moving the misexpect check into the
LLVM backend, and adding support for IR and sampling based profiles, in
addition to frontend instrumentation.
We add new misexpect metadata tags to those instructions directly
influenced by the llvm.expect intrinsic (branch, switch, and select)
when lowering the intrinsics. The misexpect metadata contains
information about the expected target of the intrinsic so that we can
check against the correct PGO counter when emitting diagnostics, and the
compiler's values for the LikelyBranchWeight and UnlikelyBranchWeight.
We use these branch weight values to determine when to emit the
diagnostic to the user.
A future patch should address the comment at the top of
LowerExpectIntrisic.cpp to hoist the LikelyBranchWeight and
UnlikelyBranchWeight values into a shared space that can be accessed
outside of the LowerExpectIntrinsic pass. Once that is done, the
misexpect metadata can be updated to be smaller.
In the long term, it is possible to reconstruct portions of the
misexpect metadata from the existing profile data. However, we have
avoided this to keep the code simple, and because some kind of metadata
tag will be required to identify which branch/switch/select instructions
are influenced by the use of llvm.expect
Patch By: paulkirth
Differential Revision: https://reviews.llvm.org/D66324
llvm-svn: 371584
|
|
|
|
|
|
|
| |
Fixes "warning: implicit declaration of function 'flock' is invalid in C99"
for flock().
llvm-svn: 369534
|
|
|
|
|
|
|
|
|
|
|
| |
We want to avoid doing expensive work during atexit since the process
might be terminated before we can publish the VMO and write out the
symbolizer markup, so move the VMO creation to the initialization
phase and only write data during the atexit phase.
Differential Revision: https://reviews.llvm.org/D66323
llvm-svn: 369180
|
|
|
|
|
|
|
|
| |
See https://reviews.llvm.org/D58620 for discussion.
Note how the comment in the file already said ".cpp" :)
llvm-svn: 367460
|
|
|
|
|
|
| |
Differential Revision: https://reviews.llvm.org/D65518
llvm-svn: 367450
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
In Android, attempting to mkdir() or even stat() top-level directories
like /data causes noisy selinux denials. During whole-system coverage
instrumentation, this causes a deluge of noisy messages that drown out
legitimate selinux denials, that should be audited and fixed.
To avoid this, skip creating any directory in GCOV_PREFIX (thereby
assuming that it exists).
- Android platform ensures that the GCOV_PREFIX used in Android is
created and read/writable by all processes.
- This only affects the Android platform (by checking against
__ANDROID_API_FUTURE__) and for apps built with Clang coverage, the
runtime will still create any non-existent parent directories for the
coverage files.
Reviewers: srhines, davidxl
Subscribers: krytarowski, #sanitizers, danalbert, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D65245
llvm-svn: 367064
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
User code can open a file on its own and pass it to the runtime, rather than
specifying a name and having the runtime open the file. This supports the use
case where a process cannot open a file on its own but can receive a file
descriptor from another process.
Relanding https://reviews.llvm.org/D62541. The original revision unlocked
the file before calling flush, this revision fixes that.
Reviewers: Dor1s, davidxl
Reviewed By: Dor1s
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D63581
llvm-svn: 364231
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
profile data."
This caused instrumented Clang to become crashy. See llvm-commits thread
for repro steps.
This also reverts follow-up r362716 which added test cases.
> Author: Sajjad Mirza
>
> Differential Revision: http://reviews.llvm.org/D62541
llvm-svn: 363134
|
|
|
|
|
|
|
|
| |
Author: Sajjad Mirza
Differential Revision: http://reviews.llvm.org/D62541
llvm-svn: 362676
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
profile data has been dumped to the file.
Summary:
The main use is for users to disable dumping profile data to the file
for certain processes in case the processes don't have permission to
write to the disks, and trying to do so would result in side effects
such as crashes.
Patch by Yuke Liao (@liaoyuke).
Additional context (Chromium use case):
- https://bugs.chromium.org/p/chromium/issues/detail?id=842424
- https://bugs.chromium.org/p/chromium/issues/detail?id=957655
- https://chromium-review.googlesource.com/c/chromium/src/+/1610093
Reviewers: Dor1s, vsk, davidxl
Reviewed By: Dor1s, davidxl
Subscribers: delcypher, davidxl, sajjadm, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D62078
llvm-svn: 361194
|
|
|
|
|
|
|
|
| |
- Several "warning: extra ';' [-Wpedantic]"
- One "C++ style comments are not allowed in ISO C90 [enabled by default]"
in a file that uses C style comments everywhere but in one place
llvm-svn: 360430
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently VMO in Zircon create using the zx_vmo_create is resizable
by default, but we'll be changing this in the future, requiring an
explicit flag to make the VMO resizable.
Prepare for this change by passing ZX_VMO_RESIZABLE option to all
zx_vmo_create calls that need resizable VMO.
Differential Revision: https://reviews.llvm.org/D61450
llvm-svn: 359803
|
|
|
|
|
|
|
|
| |
Make it more readable for an average user.
Differential Revision: https://reviews.llvm.org/D60896
llvm-svn: 359043
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
r355343 was landed and was reverted in r355363 due to build breakage.
This patch adds Linux/Windows support on top of r355343.
In this patch, Darwin should be working with testing case. Linux should be working,
I will enable the testing case in a follwup diff. Windows/Other should be building.
Correct implementation for Other platforms will be added.
Thanks David for reviewing the original diff, helping me with issues on Linux, and
giving suggestions for adding support for Other platforms.
llvm-svn: 355701
|
|
|
|
|
|
|
|
| |
This caused issues on Linux/Windows and other platforms.
r355343 355350 355350
llvm-svn: 355363
|
|
|
|
|
|
| |
Followup to D57530.
llvm-svn: 355357
|
|
|
|
|
|
| |
By adding implementations for __llvm_profile_begin_orderfile for non-Darwin platforms.
llvm-svn: 355350
|
|
|
|
|
|
|
|
| |
The profile data will be dumped in a file default_xxx.profraw.order.
Differential Revision: https://reviews.llvm.org/D57530
llvm-svn: 355343
|
|
|
|
| |
llvm-svn: 355119
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
The llvm-cov tool needs to be able to find coverage names in the
executable, so the .lprfn and .lcovmap sections cannot be merged into
.rdata.
Also, the linker merges .lprfn$M into .lprfn, so llvm-cov needs to
handle that when looking up sections. It has to support running on both
relocatable object files and linked PE files.
Lastly, when loading .lprfn from a PE file, llvm-cov needs to skip the
leading zero byte added by the profile runtime.
Reviewers: vsk
Subscribers: hiraditya, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D58661
llvm-svn: 354840
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
The changes in https://reviews.llvm.org/D44847 cause load time failure
due to lprofMergeValueProfData in Android libs enabled with profile generation:
"dlopen failed: cannot locate symbol "lprofMergeValueProfData" referenced by..."
Marking lprofMergeValueProfData as hidden so the correct in-module definition
is picked by the linker.
Reviewers: davidxl
Reviewed By: davidxl
Subscribers: efriedma, xur, davidxl, llvm-commits
Differential Revision: https://reviews.llvm.org/D55893
llvm-svn: 354064
|
|
|
|
|
|
|
|
|
| |
This function doesn't use anything MSVC specific but works fine
for any _WIN32 target.
Differential Revision: https://reviews.llvm.org/D58106
llvm-svn: 353918
|
|
|
|
|
|
|
|
|
|
| |
The macros were changed but not updated in the Darwin version.
Fail here: http://green.lab.llvm.org/green/job/clang-stage1-configure-RA/53739/
rdar://47925820
llvm-svn: 353560
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
The motivating use case is eliminating duplicate profile data registered
for the same inline function in two object files. Before this change,
users would observe multiple symbol definition errors with VC link, but
links with LLD would succeed.
Users (Mozilla) have reported that PGO works well with clang-cl and LLD,
but when using LLD without this static registration, we would get into a
"relocation against a discarded section" situation. I'm not sure what
happens in that situation, but I suspect that duplicate, unused profile
information was retained. If so, this change will reduce the size of
such binaries with LLD.
Now, Windows uses static registration and is in line with all the other
platforms.
Reviewers: davidxl, wmi, inglorion, void, calixte
Subscribers: mgorny, krytarowski, eraman, fedor.sergeev, hiraditya, #sanitizers, dmajor, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D57929
llvm-svn: 353547
|
|
|
|
| |
llvm-svn: 353485
|
|
|
|
|
|
|
|
|
|
|
| |
There is no reason for these sections to remain separate in the final
DLL or EXE.
I have not yet added a InstrProfilingPlatformWindows.c for these, since
avoiding dynamic profile data registration is a larger project for
later.
llvm-svn: 353221
|