summaryrefslogtreecommitdiffstats
path: root/clang/utils/perf-training
Commit message (Collapse)AuthorAgeFilesLines
* [perf-training] Change profile file pattern string to use %4m instead of %pXin-Xin Wang2019-12-171-1/+1
| | | | | | | | | | | | | | Summary: With %p, each test file that we're using to generate profile data will make its own profraw file which is around 60 MB in size. If we have a lot of test files, that quickly uses a lot of space. Use %4m instead to share the profraw files used to store the profile data. We use 4 here based on the default value in https://reviews.llvm.org/source/llvm-github/browse/master/llvm/CMakeLists.txt$604 Reviewers: beanz, phosek, xiaobai, smeenai, vsk Reviewed By: vsk Subscribers: vsk, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D71585
* [perf-training] Make training data location configurableShoaib Meenai2019-12-145-5/+11
| | | | | | | We may wish to keep the PGO training data outside the repository. Add a CMake variable to allow referencing an external lit testsuite. Differential Revision: https://reviews.llvm.org/D71507
* Revert "NFC: utils/perf-training: Python 3 compatibility for lit.cfg"Azharuddin Mohammed2019-07-241-1/+1
| | | | | | | | | This reverts commit 9178b10163f758cbf8a5290ea6a827990427ddc0 (r365969). We are back to using Python2 and this is failing. This should instead be made to be compatible with both Python 2 and 3. llvm-svn: 366953
* NFC: utils/perf-training: Python 3 compatibility for lit.cfgAlex Lorenz2019-07-121-1/+1
| | | | | | The output of subprocess.check_output is now bytes. We need to decode it. llvm-svn: 365969
* Update the file headers across all of the LLVM projects in the monorepoChandler Carruth2019-01-191-4/+3
| | | | | | | | | | | | | | | | | to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
* [Bug 39548][Clang] PGO bootstrap fails with python3: errors in perf-helper.pySerge Guelton2019-01-111-2/+2
| | | | | | | | | | | Current clang fail to bootstrap in PGO mode when only python3 is available, because perf-helper.py is not compatible with python3. Commited on behalf of Romain Geissler. Differential Revision: https://reviews.llvm.org/D54071 llvm-svn: 350955
* Portable Python script across Python versionSerge Guelton2018-12-181-1/+1
| | | | | | | | Make scripts more future-proof by importing most __future__ stuff. Differential Revision: https://reviews.llvm.org/D55208 llvm-svn: 349504
* Portable Python script across Python versionSerge Guelton2018-12-181-3/+3
| | | | | | | | | | | In Python3, dict.items, dict.keys, dict.values, zip, map and filter no longer return lists, they create generator instead. The portability patch consists in forcing an extra `list` call if the result is actually used as a list. `map` are replaced by list comprehension and `filter` by filtered list comprehension. Differential Revision: https://reviews.llvm.org/D55197 llvm-svn: 349501
* [clang] Enable clang build with LLVM_BUILD_INSTRUMENTED without setting ↵Alexander Shaposhnikov2017-10-131-6/+6
| | | | | | | | | | | | | | | | | | LLVM_PROFTDATA At the moment if LLVM_BUILD_INSTRUMENTED is set to True one has to set LLVM_PROFTDATA even if it's not really used (because of message(FATAL_ERROR ...)). Building the instrumented version of Clang can be useful even if one doesn't plan to build the target generate-profdata (currently that target would only compile utils/perf-training/cxx/hello_world.cpp). For example, one can run the instrumented version of Clang via a separate build system against a different codebase, collect/analyze the profiles and merge them by llvm-profdata later. Differential revision: https://reviews.llvm.org/D38859 llvm-svn: 315665
* Fix one more reference to lit.util.capture()Michael Zolotukhin2017-07-052-2/+4
| | | | | | The capture method was removed in r306643. llvm-svn: 307201
* [clang] Update lit config in utils/perf-trainingAlexander Shaposhnikov2017-01-042-4/+3
| | | | | | | | | | | | This diff replaces --driver-mode=cpp in utils/perf-training/order-files.lit.cfg and utils/perf-training/lit.cfg with --driver-mode=g++. clang --driver-mode=cpp will call the preprocessor and will not trigger compilation. Differential revision: https://reviews.llvm.org/D28269 llvm-svn: 290936
* [clang] Fix clean build of generate-order-fileAlexander Shaposhnikov2016-12-311-1/+5
| | | | | | | | | | | | | This diff fixes the clean build of the target generate-order-file. In llvm/tools/clang/CMakeLists.txt add_subdirectory(utils/perf-training) should go after the block where the value of the variable CLANG_ORDER_FILE is set - otherwise (tested with cmake's version 3.6.2) the arguments of perf-helper.py gen-order-file will be ill-formed (CLANG_ORDER_FILE will be empty). Differential revision: https://reviews.llvm.org/D28153 llvm-svn: 290781
* [Order Files] On Darwin use DTrace's oneshot probeChris Bieneman2016-08-241-0/+5
| | | | | | The oneshot probe only gets executed the first time the probe is hit in the process. For order file generation this is really all we care about. llvm-svn: 279673
* Revert "[Order Files] Remove dtrace predicate"Chris Bieneman2016-08-021-1/+2
| | | | | | | | This reverts commit r277487. Removing the probe predicate was a red herring. It results in more symbols being placed in the final order file, but they are symbols from outside the clang image. llvm-svn: 277492
* [Order Files] Remove dtrace predicateChris Bieneman2016-08-021-2/+1
| | | | | | Having the dtrace predicate setup to only show probes in clang filters out static initializers executed by dyld, which we do want included in the order files. llvm-svn: 277487
* [Order Files] Fixing an error in the perf-helper scriptChris Bieneman2016-08-011-1/+1
| | | | | | Dtrace probemod needs to be based on the first argument of the command, not the first argument of the args. This error was introduced a while back when I added support for skipping the driver and invoking cc1 directly. llvm-svn: 277401
* [Perf-Helper] Add logging for dtrace commandsChris Bieneman2016-07-291-0/+1
| | | | | | Logging the dtrace command into the top of the dtrace log is useful when debugging why the order file generation is flaky. llvm-svn: 277234
* [perf-training] Ignore 'Profile Note' warnings from the runtimeVedant Kumar2016-06-141-0/+1
| | | | | | | | | | | | | | | After r272599, -DLLVM_BUILD_INSTRUMENTED passes a default argument to -fprofile-instr-generate. This confuses the perf-helper script because the runtime emits a note stating that the default is overridden by the LLVM_PROFILE_FILE environment variable. Change the perf-helper script s.t it does not treat these notes as failures. This isn't a strictly NFC change, but I don't see a simple way to add a test for it. llvm-svn: 272695
* Remove stray semi-colon in *.py file, NFCVedant Kumar2016-06-141-1/+1
| | | | llvm-svn: 272610
* Replace hardcoded comment at 'lit.site.cfg.in'Alex Denisov2016-04-162-4/+4
| | | | | | | | | | | | At the moment almost every lit.site.cfg.in contains two lines comment: ## Autogenerated by LLVM/Clang configuration. # Do not edit! The patch adds variable LIT_SITE_CFG_IN_HEADER, that is replaced from configure_lit_site_cfg with the note and some useful information. llvm-svn: 266516
* [OrderFiles] Don't allow lit to run dtrace multithreadedChris Bieneman2016-04-131-0/+1
| | | | | | Dtrace is implemented to try and minimize performance impact on the process being traced. This results in dtrace dropping samples if it is taking too many CPU resources. Multi-threading dtrace increases the sample drop rate dramatically. llvm-svn: 266213
* [Perf-Training] Reworked workflow improvements for order-file generationChris Bieneman2016-04-081-2/+1
| | | | | | | | | | | | | | | | | | | This is re-landing r260742. I've reworked the conditionals so that it only hits when targeting Apple platforms with ld64. Original Summary: With this change generating clang order files using dtrace uses the following workflow: cmake <whatever options you want> ninja generate-order-file ninja clang This patch works by setting a default path to the order file (which can be overridden by the user). If the order file doesn't exist during configuration CMake will create an empty one. CMake then ties up the dependencies between the clang link job and the order file, and generate-order-file overwrites CLANG_ORDER_FILE with the new order file. llvm-svn: 265864
* [Perf-training] Using os.devnull instead of a temp fileChris Bieneman2016-03-221-4/+1
| | | | | | | | This is based on post-commit feedback from Vedant. Totally didn't know that existed and worked on Windows. Thanks Vedant! llvm-svn: 264064
* [Perf-training] Cleanup based on feedback from Sean SilvasChris Bieneman2016-03-221-11/+6
| | | | | | Sean provided feedback based on r257934 on cfe-commits. This change addresses that feedback. llvm-svn: 264063
* [Perf-training] Fixing an issue with multi-threading PGO generationChris Bieneman2016-03-221-2/+5
| | | | | | When LIT parallelizes the profraw file generation we need to generate unique temp filenames then clean them up after the driver executes. llvm-svn: 264021
* [Perf-training] Adding support for tests to skip the clang driverChris Bieneman2016-03-215-2/+69
| | | | | | | | | | This patch adds a new set of substitutions to the lit run lines for order files and PGO generation which run the clang driver to get the cc1 command, then execute the cc1 command directly. This allows the scripts to bypass profiling the clang driver over and over again. The approach in this patch was discussed via IRC with Sean Silvas. Special thanks to Daniel Dunbar whose out-of-tree code I liberally plagiarized. llvm-svn: 263997
* [cmake] Revert r260742 (and r260744) to improve order file support.Chandler Carruth2016-02-171-1/+2
| | | | | | | | | | This appears to be passing '-Wl,-order_file' to Linux link commands, which then causes the linker to silently, behind the scenes, write the output to 'rder_file' instead of somewhere else. Will work with Chris to figure out the proper support for this, but so far there are numerous people who can't get Clang to update when they build because of this. llvm-svn: 261054
* [CMake] Improve the clang order-file generation workflowChris Bieneman2016-02-121-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This commit re-lands r259862. The underlying cause of the build breakage was an incorrectly written capabilities test. In tools/Driver/CMakeLists.txt I was attempting to check if a linker flag worked, the test was passing it to the compiler, not the linker. CMake doesn't have a linker test, so we have a hand-rolled one. Original Patch Review: http://reviews.llvm.org/D16896 Original Summary: With this change generating clang order files using dtrace uses the following workflow: cmake <whatever options you want> ninja generate-order-file ninja clang This patch works by setting a default path to the order file (which can be overridden by the user). If the order file doesn't exist during configuration CMake will create an empty one. CMake then ties up the dependencies between the clang link job and the order file, and generate-order-file overwrites CLANG_ORDER_FILE with the new order file. Reviewers: bogner Subscribers: cfe-commits Differential Revision: http://reviews.llvm.org/D16999 llvm-svn: 260742
* Revert "[CMake] Improve the clang order-file generation workflow"Chris Bieneman2016-02-051-1/+2
| | | | | | | | | | | | | This reverts commit r259862, and attempts to fix builder CMakeCaches. Will try this again some other time... Conflicts: CMakeLists.txt tools/driver/CMakeLists.txt llvm-svn: 259872
* [CMake] Improve the clang order-file generation workflowChris Bieneman2016-02-051-2/+1
| | | | | | | | | | | | | | | | | | | | | | | Summary: With this change generating clang order files using dtrace uses the following workflow: cmake <whatever options you want> ninja generate-order-file ninja clang This patch works by setting a default path to the order file (which can be overridden by the user). If the order file doesn't exist during configuration CMake will create an empty one. CMake then ties up the dependencies between the clang link job and the order file, and generate-order-file overwrites CLANG_ORDER_FILE with the new order file. Reviewers: bogner Subscribers: cfe-commits Differential Revision: http://reviews.llvm.org/D16896 llvm-svn: 259862
* [CMake] Fix bots broken by including order file generation in check-allChris Bieneman2016-01-151-0/+4
| | | | llvm-svn: 257948
* [CMake] [Order-files] Use print_function as an attempt at being forward ↵Chris Bieneman2016-01-151-17/+19
| | | | | | | | compatible. Based on feedback from bogner. llvm-svn: 257936
* [CMake] Support generation of linker order files using dtraceChris Bieneman2016-01-154-22/+408
| | | | | | | | | | | | | | | | | | | Summary: This patch extends the lit-based perf-training tooling supplied for PGO data generation to also generate linker order files using dtrace. This patch should work on any system that has dtrace. If CMake can find the dtrace tool it will generate a target 'generate-order-file' which will run the per-training tests wrapped by dtrace to capture function entries. There are several algorithms implemented for sorting the order files which can be experimented with for best performance. The dtrace wrapper also supports bot oneshot and pid probes. The perf-helper.py changes to support order file construction are ported from internal changes by ddunbar; he gets all the credit for the hard work here, I just copy and pasted. Note: I've tested these patches on FreeBSD and OS X 10.10. Reviewers: ddunbar, bogner, silvas Subscribers: llvm-commits, emaste Differential Revision: http://reviews.llvm.org/D16134 llvm-svn: 257934
* [CMake] Fixing a typo in a flagChris Bieneman2015-12-191-1/+1
| | | | | | Turns out cc1's flag has 1 - not 2... llvm-svn: 256070
* [CMake] PGO training dataChris Bieneman2015-12-181-0/+2
| | | | | | Adding in a few more lit substitutions for cc1 and the test exec path. llvm-svn: 256057
* [CMake] Add support for generating profdata for clang from training filesChris Bieneman2015-12-166-0/+150
Summary: This patch adds support for using LIT to drive generating PGO profile data for clang. This first pass implementation should work on Linux and Unix based platforms. If you build clang using CMake with LLVM_BUILD_INSTRUMENTED=On the CMake build generates a generate-profdata target that will use the just-built clang to build any test files (see hello_world.cpp as an example). Each test compile will generate profraw files for each clang process. After all tests have run CMake will merge the profraw files using llvm-profdata. Future opportunities for extension: * Support for Build->Profile->Build bootstrapping * Support for linker order file generation using a similar mechanism and the same training data * Support for Windows Reviewers: dexonsmith, friss, bogner, cmatthews, vsk, silvas Subscribers: cfe-commits Differential Revision: http://reviews.llvm.org/D15462 llvm-svn: 255740
OpenPOWER on IntegriCloud