summaryrefslogtreecommitdiffstats
path: root/llvm/utils/opt-viewer
Commit message (Collapse)AuthorAgeFilesLines
* [opt-viewer] Move under tools, install itAdam Nemet2017-07-066-872/+0
| | | | | | | | | | | | | | | | | | | | We weren't installing opt-viewer and co before, this fixes the omission. I am also moving the tools from utils/ to tools/. I believe that this is more appropriate since these tools have matured greatly in the past year through contributions by multiple people (thanks!) so they are ready to become external tools. The tools are installed under <install>/share/opt-viewer/. I am *not* adding the llvm- prefix. If people feel strongly about adding that, this is probably a good time since the new location will require some mental adjustment anyway. Fixes PR33521 Differential Revision: https://reviews.llvm.org/D35048 llvm-svn: 307285
* [opt-viewer] Add progress indicators (PR33522)Brian Gesiak2017-06-295-44/+128
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Provide feedback to users of opt-diff.py, opt-stats.py, and opt-viewer.py, on how many YAML files have finished being processed, and how many HTML files have been generated. This feedback is particularly helpful for opt-viewer.py, which may take a long time to complete when given many large YAML files as input. The progress indicators use simple output such as the following: ``` Reading YAML files... 9 of 1197 ``` Test plan: Run `utils/opt-viewer/opt-*.py` on a CentOS and macOS machine, using Python 3.4 and Python 2.7 respectively, and ensure the output is formatted well on both. Reviewers: anemet, davidxl Reviewed By: anemet Subscribers: simon.f.whittaker, llvm-commits Differential Revision: https://reviews.llvm.org/D34735 llvm-svn: 306726
* [opt-viewer] Python 3 support in opt-viewer.pyBrian Gesiak2017-06-292-5/+6
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: Minor changes that allow opt-stats.py to support both Python 2 and 3. In addition to the same dictionary iterator changes that were necessary in https://reviews.llvm.org/D34564, this diff also: * Explcitly converts strings to bytes when reading from and writing to stdin and stdout. * No longer uses dictionaries as a sort key for optimization remarks. Dictionary sort order in Python 2 is pretty esoteric anyway, so it's not clear that the additional sorting had a benefit for end users (for details, https://stackoverflow.com/a/3484456/679254 is a good resource on Python 2 dictionary sort order). Reviewers: anemet, davidxl Reviewed By: anemet Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34647 llvm-svn: 306720
* [opt-viewer] opt-viewer.py takes -o argumentBrian Gesiak2017-06-291-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | Summary: Change how the output directory is specified when invoking opt-viewer.py, from `opt-viewer.py yaml_file_one yaml_file_two output_dir` to `opt-viewer.py -o output_dir yaml_file_one yaml_file_two`. This makes it easier to pipe the results of another command into opt-viewer.py. For example: ``` find . -name "*.yaml" -print | xargs /path/to/opt-viewer.py -o html ``` Reviewers: anemet, davidxl Reviewed By: anemet Subscribers: fhahn, llvm-commits Differential Revision: https://reviews.llvm.org/D34711 llvm-svn: 306694
* [opt-viewer] Python 3 support in opt-diff.pyBrian Gesiak2017-06-271-2/+2
| | | | | | | | | | | | | | | | Summary: The `file()` builtin is not available in Python 3; use `open()` instead. https://docs.python.org/3.0/whatsnew/3.0.html#builtins Reviewers: anemet, davidxl, davide Reviewed By: davide Subscribers: davide, fhahn, llvm-commits Differential Revision: https://reviews.llvm.org/D34670 llvm-svn: 306423
* [opt-viewer] Python 3 support in opt-stats.pyBrian Gesiak2017-06-262-4/+22
| | | | | | | | | | | | | | Summary: Minor changes that allow opt-stats.py to support both Python 2 and 3. Reviewers: anemet, davidxl Reviewed By: anemet Subscribers: llvm-commits, fhahn Differential Revision: https://reviews.llvm.org/D34564 llvm-svn: 306306
* [opt-viewer] Remove positional arg checks (NFC)Brian Gesiak2017-06-232-8/+0
| | | | | | | | | | | | | | | | | | | | | | Summary: opt-stats.py and opt-viewer.py's argument parsers both take a positional argument 'yaml_files'. Positional arguments in Python's argparse module are required by default, so the subsequent checks for `len(args.yaml_files) == 0` are unnecessary -- if the length was zero, then the call to `parser.parse_args()` would have thrown an error already. Because there is no way for `len(args.yaml_files)` to be zero at these points, removing the code is NFC. Reviewers: anemet, davidxl Reviewed By: anemet Subscribers: llvm-commits, fhahn Differential Revision: https://reviews.llvm.org/D34567 llvm-svn: 306147
* [opt-viewer] Include default values in help outputBrian Gesiak2017-06-103-3/+3
| | | | | | | | | | | | | | | | | Summary: Python's argparse module includes a `%(default)s` format specifier that can be used to print the default value of an option in its help text. Use this for opt-viewer utilities' `--jobs` arguments. Reviewers: anemet Reviewed By: anemet Subscribers: llvm-commits, fhahn Differential Revision: https://reviews.llvm.org/D34081 llvm-svn: 305155
* PR33331 - opt-viewer.py produces broken output for directories with spacesFilipe Cabecinhas2017-06-071-1/+1
| | | | | | | | Fix: Properly quote href attributes. Patch by Simon Whittaker! llvm-svn: 304919
* [opt-diff] Fix the case when the script is invoked with directoriesAdam Nemet2017-03-061-1/+3
| | | | llvm-svn: 297055
* New tool: opt-diff.pyAdam Nemet2017-03-023-3/+88
| | | | | | | | | | This tool allows generating the different between two optimization record files. The result is a YAML file too that can be visualized with opt-viewer. This is very useful to see what optimization were added and removed by a change. llvm-svn: 296767
* [opt-viewer] Treat remarks with different attributes as differentAdam Nemet2017-03-021-1/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We used to exclude arguments but for a diffed YAML file, it's interesting to show these as changes. Turns out this also affects gvn/LoadClobbered because we used to squash multiple entries of this on the same line even if they reported clobbers by *different* instructions. This increases the number of unique entries now and the share of gvn/LoadClobbered. Total number of remarks 902287 Top 10 remarks by pass: inline 43% gvn 37% licm 11% loop-vectorize 4% asm-printer 3% regalloc 1% loop-unroll 1% inline-cost 0% slp-vectorizer 0% loop-delete 0% Top 10 remarks: gvn/LoadClobbered 33% inline/Inlined 16% inline/CanBeInlined 14% inline/NoDefinition 7% licm/Hoisted 6% licm/LoadWithLoopInvariantAddressInvalidated 5% gvn/LoadElim 3% asm-printer/InstructionCount 3% inline/TooCostly 2% loop-vectorize/MissedDetails 2% llvm-svn: 296766
* [opt-viewer] Don't use __getattr__ for missing YAML attributesAdam Nemet2017-03-021-5/+6
| | | | | | | __getattr__ does not work well with debugging. If the attribute function has a run-time error, a missing attribute is reported instead. llvm-svn: 296765
* [opt-viewer] Sort entries with identical hotness by source lineAdam Nemet2017-03-021-1/+1
| | | | | | | We want entries that are close to each other in the source appear next to each other. llvm-svn: 296764
* New tool: opt-stats.pyAdam Nemet2017-03-013-188/+246
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I am planning to use this tool to find too noisy (missed) optimization remarks. Long term it may actually be better to just have another tool that exports the remarks into an sqlite database and perform queries like this in SQL. This splits out the YAML parsing from opt-viewer.py into a new Python module optrecord.py. This is the result of the script on the LLVM testsuite: Total number of remarks 714433 Top 10 remarks by pass: inline 52% gvn 24% licm 13% loop-vectorize 5% asm-printer 3% loop-unroll 1% regalloc 1% inline-cost 0% slp-vectorizer 0% loop-delete 0% Top 10 remarks: gvn/LoadClobbered 20% inline/Inlined 19% inline/CanBeInlined 18% inline/NoDefinition 9% licm/LoadWithLoopInvariantAddressInvalidated 6% licm/Hoisted 6% asm-printer/InstructionCount 3% inline/TooCostly 3% gvn/LoadElim 3% loop-vectorize/MissedDetails 2% Beside some refactoring, I also changed optrecords not to use context to access global data (max_hotness). Because of the separate module this would have required splitting context into two. However it's not possible to access the optrecord context from the SourceFileRenderer when calling back to Remark.RelativeHotness. llvm-svn: 296682
* [opt-viewer] Suggest installing the faster parser (libYAML)Adam Nemet2017-03-011-3/+2
| | | | llvm-svn: 296553
* [opt-viewer] Handle column number 0Adam Nemet2017-02-281-1/+1
| | | | | | | The asm-printer now emits remarks with function location which have unspecified (0) source column number. llvm-svn: 296547
* opt-viewer: Fix syntax highlightingBrian Cain2017-02-181-9/+18
| | | | | | | | | | | | | | | Syntax highlighting has been done line-at-a-time. Done this way, the lexer resets the context at each line, distorting the formatting. This change will render the whole file at once and feed the highlighted text line-at-a-time to be wrapped by the SourceFileRenderer. Leading/trailing newlines were being ignored by Pygments but since each line was rendered in its own row, it didn't matter. This bug was masked by the line-at-a-time algorithm. So now we need to add "stripnl=False" to the CppLexer to change its behavior to match the expectation. llvm-svn: 295546
* [opt-viewer] For single-process, fall back on map instead of Pool.mapAdam Nemet2017-02-141-7/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows for nicer backtrace and debugging when -j1 is passed: $ opt-viewer.py CMakeFiles/LLVMScalarOpts.dir/LoopVersioningLICM.cpp.opt.yaml html Traceback (most recent call last): File "/org/llvm/utils/opt-viewer/opt-viewer.py", line 405, in <module> generate_report(pmap, all_remarks, file_remarks, args.source_dir, args.output_dir) File "/org/llvm/utils/opt-viewer/opt-viewer.py", line 362, in generate_report pmap(_render_file_bound, file_remarks.items()) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 251, in map return self.map_async(func, iterable, chunksize).get() File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 567, in get raise self._value Exception: blah $ opt-viewer.py -j 1 CMakeFiles/LLVMScalarOpts.dir/LoopVersioningLICM.cpp.opt.yaml html Traceback (most recent call last): File "/org/llvm/utils/opt-viewer/opt-viewer.py", line 405, in <module> generate_report(pmap, all_remarks, file_remarks, args.source_dir, args.output_dir) File "/org/llvm/utils/opt-viewer/opt-viewer.py", line 362, in generate_report pmap(_render_file_bound, file_remarks.items()) File "/org/llvm/utils/opt-viewer/opt-viewer.py", line 317, in _render_file SourceFileRenderer(source_dir, output_dir, filename).render(remarks) File "/org/llvm/utils/opt-viewer/opt-viewer.py", line 168, in __init__ raise Exception("blah") Exception: blah llvm-svn: 295080
* opt-viewer: fix HtmlFormatter encodingAdam Nemet2017-02-101-1/+1
| | | | | | | | | | | | | | | | Summary: Small fix to HtmlFormatter, defaults to ascii encoding, so utf-8 output may get `UnicodeEncodeError: 'ascii' codec can't encode character ... ordinal not in range(128)` during write. Patch by Brian Cain! Reviewers: anemet, fhahn Reviewed By: anemet Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29802 llvm-svn: 294710
* [opt-viewer] HTML-escape function namesAdam Nemet2017-02-021-2/+5
| | | | llvm-svn: 293869
* [opt-viewer] Alternating row background colors on index pageAdam Nemet2017-02-022-6/+12
| | | | llvm-svn: 293868
* [opt-viewer] Introduce global contextAdam Nemet2017-01-271-18/+24
| | | | | | | This is necessary since globals (max_hotness, caller_loc) need to be explicitly passed to the subprocesses. llvm-svn: 293266
* [opt-viewer] Remove message from the keyAdam Nemet2017-01-271-1/+1
| | | | | | | | This is causing problems because the rendering of the text will depend on varying global state to show relative hotness or a link in the inlining context. llvm-svn: 293265
* [opt-viewer] Unique across the different jobs as wellAdam Nemet2017-01-271-24/+11
| | | | llvm-svn: 293264
* [opt-viewer] Make sorting for the index page deterministicAdam Nemet2017-01-271-2/+2
| | | | | | Break the tie between entries with identical hotness deterministically. llvm-svn: 293263
* [opt-viewer] Include the function in the remark keyAdam Nemet2017-01-271-1/+1
| | | | | | Avoid uniquing remarks with different the inlining context (Function). llvm-svn: 293262
* [opt-viewer] Put critical items in parallelAdam Nemet2017-01-271-42/+127
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Put opt-viewer critical items in parallel Patch by Brian Cain! Requires features from Python 2.7 **Performance** Below are performance results across various configurations. These were taken on an i5-5200U (dual core + HT). They were taken with a small subset of the YAML output of building Python 3.6.0b3 with LTO+PGO. 60 YAML files. "multiprocessing" is the current submission contents. "baseline" is as of 544f14c6b2a07a94168df31833dba9dc35fd8289 (I think this is aka r287505). "ImportError" vs "class<...CLoader>" below are just confirming the expected configuration (with/without CLoader). The below was measured on AMD A8-5500B (4 cores) with 224 input YAML files, showing a ~1.75x speed increase over the baseline with libYAML. I suspect it would scale well on high-end servers. ``` **************************************** MULTIPROCESSING **************************************** PyYAML: Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: cannot import name CLoader Python 2.7.10 489.42user 5.53system 2:38.03elapsed 313%CPU (0avgtext+0avgdata 400308maxresident)k 0inputs+31392outputs (0major+473540minor)pagefaults 0swaps PyYAML+libYAML: <class 'yaml.cyaml.CLoader'> Python 2.7.10 78.69user 5.45system 0:32.63elapsed 257%CPU (0avgtext+0avgdata 398560maxresident)k 0inputs+31392outputs (0major+542022minor)pagefaults 0swaps PyPy/PyYAML: Traceback (most recent call last): File "<builtin>/app_main.py", line 75, in run_toplevel File "<builtin>/app_main.py", line 601, in run_it File "<string>", line 1, in <module> ImportError: cannot import name 'CLoader' Python 2.7.9 (2.6.0+dfsg-3, Jul 04 2015, 05:43:17) [PyPy 2.6.0 with GCC 4.9.3] 154.27user 8.12system 0:53.83elapsed 301%CPU (0avgtext+0avgdata 627960maxresident)k 808inputs+30376outputs (0major+727994minor)pagefaults 0swaps **************************************** BASELINE **************************************** PyYAML: Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: cannot import name CLoader Python 2.7.10 358.08user 4.05system 6:08.37elapsed 98%CPU (0avgtext+0avgdata 315004maxresident)k 0inputs+31392outputs (0major+85252minor)pagefaults 0swaps PyYAML+libYAML: <class 'yaml.cyaml.CLoader'> Python 2.7.10 50.32user 3.30system 0:56.59elapsed 94%CPU (0avgtext+0avgdata 307296maxresident)k 0inputs+31392outputs (0major+79335minor)pagefaults 0swaps PyPy/PyYAML: Traceback (most recent call last): File "<builtin>/app_main.py", line 75, in run_toplevel File "<builtin>/app_main.py", line 601, in run_it File "<string>", line 1, in <module> ImportError: cannot import name 'CLoader' Python 2.7.9 (2.6.0+dfsg-3, Jul 04 2015, 05:43:17) [PyPy 2.6.0 with GCC 4.9.3] 72.94user 5.18system 1:23.41elapsed 93%CPU (0avgtext+0avgdata 455312maxresident)k 0inputs+30392outputs (0major+110280minor)pagefaults 0swaps ``` Reviewers: fhahn, anemet Reviewed By: anemet Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D26967 llvm-svn: 293261
* [opt-viewer] Add support for libYAML for faster parsingAdam Nemet2016-11-151-2/+12
| | | | | | | | | | | | | | | | | | | | | | This results in a speed-up of over 6x on sqlite3. Before: $ time -p /org/llvm/utils/opt-viewer/opt-viewer.py ./MultiSource/Applications/sqlite3/CMakeFiles/sqlite3.dir/sqlite3.c.opt.yaml html real 415.07 user 410.00 sys 4.66 After with libYAML: $ time -p /org/llvm/utils/opt-viewer/opt-viewer.py ./MultiSource/Applications/sqlite3/CMakeFiles/sqlite3.dir/sqlite3.c.opt.yaml html real 63.96 user 60.03 sys 3.67 I followed these steps to get libYAML working with PyYAML: http://rmcgibbo.github.io/blog/2013/05/23/faster-yaml-parsing-with-libyaml/ llvm-svn: 286942
* [opt-viewer] Don't fail with remarks without debug locationAdam Nemet2016-11-141-2/+2
| | | | llvm-svn: 286861
* [opt-viewer] Make it work in the absence of hotness informationAdam Nemet2016-11-111-12/+30
| | | | | | In this case the index page is sorted by the source location. llvm-svn: 286572
* [opt-viewer] PEPify opt-viewer.pyMandeep Singh Grang2016-11-111-5/+18
| | | | | | | | | | Reviewers: anemet Subscribers: fhahn Differential Revision: https://reviews.llvm.org/D26535 llvm-svn: 286564
* [opt-viewer] Add column number supportAdam Nemet2016-11-111-3/+8
| | | | | | | With this the yellow (bubble) part of the remark shows up under the corresponding expression. llvm-svn: 286545
* [opt-viewer] Display inlining contextAdam Nemet2016-11-111-0/+18
| | | | | | | | | | | When a function is inlined, each instance is optimized in their own inlining context. This can produce different remarks all pointing to the same source line. This adds a new column on the source view to display the inlining context. llvm-svn: 286537
* [opt-viewer] Add option to set source directoryAdam Nemet2016-11-111-1/+22
| | | | llvm-svn: 286536
* [opt-viewer] Mention Pygments in the descriptionAdam Nemet2016-11-111-1/+1
| | | | llvm-svn: 286535
* [opt-viewer] Add syntax highlightingAdam Nemet2016-11-112-1/+74
| | | | | | Uses pygments. llvm-svn: 286532
* [opt-viewer] Avoid duplicated remarksAdam Nemet2016-11-101-4/+13
| | | | | | | This can happen if a pass is run multiple times or if the code is in a header file which is included multiple times. llvm-svn: 286489
* [opt-viewer] Avoid division by zeroAdam Nemet2016-11-071-1/+1
| | | | llvm-svn: 286172
* [OptDiag, opt-viewer] Save callee's location and display as linkAdam Nemet2016-11-071-8/+24
| | | | | | | | | | | | | With this we get a new field in the YAML record if the value being streamed out has a debug location. For examples, please see the changes to the tests. This is then used in opt-viewer to display a link for the callee function in the inlining remarks. Differential Revision: https://reviews.llvm.org/D26366 llvm-svn: 286169
* [opt-viewer] Remove unnecessary call to demangleAdam Nemet2016-10-111-1/+1
| | | | llvm-svn: 283898
* [opt-viewer] Print hotness as percentage of the maximum hotnessAdam Nemet2016-10-111-2/+9
| | | | llvm-svn: 283897
* [opt-viewer] Convert another HTML output to use a multiline stringAdam Nemet2016-10-111-10/+16
| | | | llvm-svn: 283896
* New utility to visualize optimization recordsAdam Nemet2016-10-072-0/+314
This is a new tool built on top of the new YAML ouput generated from optimization remarks. It produces HTML for easy navigation and visualization. The tool assumes that hotness information for the remarks is available (the YAML file was produced with PGO). It uses hotness to list the remarks prioritized by the hotness on the index page. Clicking the source location of the remark in the list takes you the source where the remarks are rendedered inline in the source. For now, the tool is meant as prototype. It's written in Python. It uses PyYAML to parse the input. Differential Revision: https://reviews.llvm.org/D25348 llvm-svn: 283571
OpenPOWER on IntegriCloud