Project Ortega BCM5719 LLVM https://git.raptorcs.com/git/bcm5719-llvm/atom?h=meklort-10.0.1 2019-12-04T18:35:14+00:00 2019-12-04T18:35:14+00:00 Vedant Kumar vsk@apple.com 2019-12-04T18:35:14+00:00 urn:sha1:f208b70fbc4dee78067b3c5bd6cb92aa3ba58a1e This reverts commit e18531595bba495946aa52c0a16b9f9238cff8bc. On Windows, there is an error: http://lab.llvm.org:8011/builders/sanitizer-windows/builds/54963/steps/stage%201%20check/logs/stdio error: C:\b\slave\sanitizer-windows\build\stage1\projects\compiler-rt\test\profile\Profile-x86_64\Output\instrprof-merging.cpp.tmp.v1.o: Failed to load coverage: Malformed coverage data 2019-12-04T18:10:55+00:00 Vedant Kumar vsk@apple.com 2019-10-21T18:48:38+00:00 urn:sha1:e18531595bba495946aa52c0a16b9f9238cff8bc Revise the coverage mapping format to reduce binary size by: 1. Naming function records and marking them `linkonce_odr`, and 2. Compressing filenames. This shrinks the size of llc's coverage segment by 82% (334MB -> 62MB) and speeds up end-to-end single-threaded report generation by 10%. For reference the compressed name data in llc is 81MB (__llvm_prf_names). Rationale for changes to the format: - With the current format, most coverage function records are discarded. E.g., more than 97% of the records in llc are *duplicate* placeholders for functions visible-but-not-used in TUs. Placeholders *are* used to show under-covered functions, but duplicate placeholders waste space. - We reached general consensus about giving (1) a try at the 2017 code coverage BoF [1]. The thinking was that using `linkonce_odr` to merge duplicates is simpler than alternatives like teaching build systems about a coverage-aware database/module/etc on the side. - Revising the format is expensive due to the backwards compatibility requirement, so we might as well compress filenames while we're at it. This shrinks the encoded filenames in llc by 86% (12MB -> 1.6MB). See CoverageMappingFormat.rst for the details on what exactly has changed. Fixes PR34533 [2], hopefully. [1] http://lists.llvm.org/pipermail/llvm-dev/2017-October/118428.html [2] https://bugs.llvm.org/show_bug.cgi?id=34533 Differential Revision: https://reviews.llvm.org/D69471 2019-09-30T18:11:22+00:00 Rong Xu xur@google.com 2019-09-30T18:11:22+00:00 urn:sha1:367405008755640eac6114b18ec8c98be0cf5392 With this patch, compiler generated profile variables will have its own COMDAT name for ELF format, which syncs the behavior with COFF. Tested with clang PGO bootstrap. This shows a modest reduction in object sizes in ELF format. Differential Revision: https://reviews.llvm.org/D68041 llvm-svn: 373241 2019-09-17T21:10:49+00:00 Reid Kleckner rnk@google.com 2019-09-17T21:10:49+00:00 urn:sha1:23e872a3d0548dac30b5d282506198dddfe89b2a For COFF, a comdat group is really a symbol marked IMAGE_COMDAT_SELECT_ANY and zero or more other symbols marked IMAGE_COMDAT_SELECT_ASSOCIATIVE. Typically the associative symbols in the group are not external and are not referenced by other TUs, they are things like debug info, C++ dynamic initializers, or other section registration schemes. The Visual C++ linker reports a duplicate symbol error for symbols marked IMAGE_COMDAT_SELECT_ASSOCIATIVE even if they would be discarded after handling the leader symbol. Fixes coverage-inline.cpp in check-profile after r372020. llvm-svn: 372182 2019-09-16T18:49:09+00:00 Reid Kleckner rnk@google.com 2019-09-16T18:49:09+00:00 urn:sha1:32837a0c93ecdb33faaba253c5c7e868db9c529f This fixes relocations against __profd_ symbols in discarded sections, which is PR41380. In general, instrumentation happens very early, and optimization and inlining happens afterwards. The counters for a function are calculated early, and after inlining, counters for an inlined function may be widely referenced by other functions. For C++ inline functions of all kinds (linkonce_odr & available_externally mainly), instr profiling wants to deduplicate these __profc_ and __profd_ globals. Otherwise the binary would be quite large. I made __profd_ and __profc_ comdat in r355044, but I chose to make __profd_ internal. At the time, I was only dealing with coverage, and in that case, none of the instrumentation needs to reference __profd_. However, if you use PGO, then instrumentation passes add calls to __llvm_profile_instrument_range which reference __profd_ globals. The solution is to make these globals externally visible by using linkonce_odr linkage for data as was done for counters. This is safe because PGO adds a CFG hash to the names of the data and counter globals, so if different TUs have different globals, they will get different data and counter arrays. Reviewers: xur, hans Differential Revision: https://reviews.llvm.org/D67579 llvm-svn: 372020 2019-06-20T21:27:06+00:00 Rainer Orth ro@gcc.gnu.org 2019-06-20T21:27:06+00:00 urn:sha1:6fde832b82daa0ddc19b88b281528b13dbf3958d Currently, many profiling tests on Solaris FAIL like Command Output (stderr): -- Undefined first referenced symbol in file __llvm_profile_register_names_function /tmp/lit_tmp_Nqu4eh/infinite_loop-9dc638.o __llvm_profile_register_function /tmp/lit_tmp_Nqu4eh/infinite_loop-9dc638.o Solaris 11.4 ld supports the non-standard GNU ld extension of adding __start_SECNAME and __stop_SECNAME labels to sections whose names are valid as C identifiers. Given that we already use Solaris 11.4-only features like ld -z gnu-version-script-compat and fully working .preinit_array support in compiler-rt, we don't need to worry about older versions of Solaris ld. The patch documents that support (although the comment in lib/Transforms/Instrumentation/InstrProfiling.cpp (needsRuntimeRegistrationOfSectionRange) is quite cryptic what it's actually about), and adapts the affected testcase not to expect the alternativeq __llvm_profile_register_functions and __llvm_profile_init. It fixes all affected tests. Tested on amd64-pc-solaris2.11. Differential Revision: https://reviews.llvm.org/D41111 llvm-svn: 363984 2019-06-17T09:51:07+00:00 Fangrui Song maskray@google.com 2019-06-17T09:51:07+00:00 urn:sha1:ac14f7b10cffe2be548607269e036244cd16acc3 llvm-svn: 363538 2019-02-27T23:38:44+00:00 Reid Kleckner rnk@google.com 2019-02-27T23:38:44+00:00 urn:sha1:4fb3502bc9faf5b32024f61cc3f043b563d7b66b Summary: I hadn't realized that instrumentation runs before inlining, so we can't use the function as the comdat group. Doing so can create relocations against discarded sections when references to discarded __profc_ variables are inlined into functions outside the function's comdat group. In the future, perhaps we should consider standardizing the comdat group names that ELF and COFF use. It will save object file size, since __profv_$sym won't appear in the symbol table again. Reviewers: xur, vsk Subscribers: eraman, hiraditya, cfe-commits, #sanitizers, llvm-commits Tags: #clang, #sanitizers, #llvm Differential Revision: https://reviews.llvm.org/D58737 llvm-svn: 355044 2019-02-08T19:03:50+00:00 Reid Kleckner rnk@google.com 2019-02-08T19:03:50+00:00 urn:sha1:987d331fab577b33255e93ce5a53850e36b9b180 Summary: The motivating use case is eliminating duplicate profile data registered for the same inline function in two object files. Before this change, users would observe multiple symbol definition errors with VC link, but links with LLD would succeed. Users (Mozilla) have reported that PGO works well with clang-cl and LLD, but when using LLD without this static registration, we would get into a "relocation against a discarded section" situation. I'm not sure what happens in that situation, but I suspect that duplicate, unused profile information was retained. If so, this change will reduce the size of such binaries with LLD. Now, Windows uses static registration and is in line with all the other platforms. Reviewers: davidxl, wmi, inglorion, void, calixte Subscribers: mgorny, krytarowski, eraman, fedor.sergeev, hiraditya, #sanitizers, dmajor, llvm-commits Tags: #sanitizers, #llvm Differential Revision: https://reviews.llvm.org/D57929 llvm-svn: 353547 2018-08-16T22:24:47+00:00 Vedant Kumar vsk@apple.com 2018-08-16T22:24:47+00:00 urn:sha1:ee6c233ae045f36f5bb5c7a8d14cfc9d41c634d0 Thread sanitizer instrumentation fails to skip all loads and stores to profile counters. This can happen if profile counter updates are merged: %.sink = phi i64* ... %pgocount5 = load i64, i64* %.sink %27 = add i64 %pgocount5, 1 %28 = bitcast i64* %.sink to i8* call void @__tsan_write8(i8* %28) store i64 %27, i64* %.sink To suppress TSan diagnostics about racy counter updates, make the counter updates atomic when TSan is enabled. If there's general interest in this mode it can be surfaced as a clang/swift driver option. Testing: check-{llvm,clang,profile} rdar://40477803 Differential Revision: https://reviews.llvm.org/D50867 llvm-svn: 339955