summaryrefslogtreecommitdiffstats
path: root/llvm/docs/CommandGuide/FileCheck.rst
Commit message (Collapse)AuthorAgeFilesLines
* [FileCheck] Annotate input dump (1/7)Joel E. Denny2018-12-181-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Extend FileCheck to dump its input annotated with FileCheck's diagnostics: errors, good matches if -v, and additional information if -vv. The goal is to make it easier to visualize FileCheck's matching behavior when debugging. Each patch in this series implements input annotations for a particular category of FileCheck diagnostics. While the first few patches alone are somewhat useful, the annotations become much more useful as later patches implement annotations for -v and -vv diagnostics, which show the matching behavior leading up to the error. This first patch implements boilerplate plus input annotations for error diagnostics reporting that no matches were found for a directive. These annotations mark the search ranges of the failed directives. Instead of using the usual `^~~`, which is used by later patches for good matches, these annotations use `X~~` so that this category of errors is visually distinct. For example: ``` $ FileCheck -dump-input=help The following description was requested by -dump-input=help to explain the input annotations printed by -dump-input=always and -dump-input=fail: - L: labels line number L of the input file - T:L labels the match result for a pattern of type T from line L of the check file - X~~ marks search range when no match is found - colors error If you are not seeing color above or in input dumps, try: -color $ FileCheck -v -dump-input=always check1 < input1 |& sed -n '/^Input file/,$p' Input file: <stdin> Check file: check1 -dump-input=help describes the format of the following dump. Full input was: <<<<<< 1: ; abc def 2: ; ghI jkl next:3 X~~~~~~~~ error: no match found >>>>>> $ cat check1 CHECK: abc CHECK-SAME: def CHECK-NEXT: ghi CHECK-SAME: jkl $ cat input1 ; abc def ; ghI jkl ``` Some additional details related to the boilerplate: * Enabling: The annotated input dump is enabled by `-dump-input`, which can also be set via the `FILECHECK_OPTS` environment variable. Accepted values are `help`, `always`, `fail`, or `never`. As shown above, `help` describes the format of the dump. `always` is helpful when you want to investigate a successful FileCheck run, perhaps for an unexpected pass. `-dump-input-on-failure` and `FILECHECK_DUMP_INPUT_ON_FAILURE` remain as a deprecated alias for `-dump-input=fail`. * Diagnostics: The usual diagnostics are not suppressed in this mode and are printed first. For brevity in the example above, I've omitted them using a sed command. Sometimes they're perfectly sufficient, and then they make debugging quicker than if you were forced to hunt through a dump of long input looking for the error. If you think they'll get in the way sometimes, keep in mind that it's pretty easy to grep for the start of the input dump, which is `<<<`. * Colored Annotations: The annotated input is colored if colors are enabled (enabling colors can be forced using -color). For example, errors are red. However, as in the above example, colors are not vital to reading the annotations. I don't know how to test color in the output, so any hints here would be appreciated. Reviewed By: george.karpenkov, zturner, probinson Differential Revision: https://reviews.llvm.org/D52999 llvm-svn: 349418
* [FileCheck] fixing docs buildbot - use proper code-block typeFedor Sergeev2018-11-131-1/+1
| | | | llvm-svn: 346740
* [FileCheck] fixing small formatting error in docsFedor Sergeev2018-11-131-1/+1
| | | | llvm-svn: 346725
* [FileCheck] introduce CHECK-COUNT-<num> repetition directiveFedor Sergeev2018-11-131-0/+23
| | | | | | | | | | | | | | | | | | | | | | | | In some cases it is desirable to match the same pattern repeatedly many times. Currently the only way to do it is to copy the same check pattern as many times as needed. And that gets pretty unwieldy when its more than count is big. Introducing CHECK-COUNT-<num> directive which acts like a plain CHECK directive yet matches the same pattern exactly <num> times. Extended FileCheckType to a struct to add Count there. Changed some parsing routines to handle non-fixed length of directive (all currently existing directives were fixed-length). The code is generic enough to allow future support for COUNT in more than just PlainCheck directives. See motivating example for this feature in reviews.llvm.org/D54223. Reviewed By: chandlerc, dblaikie Differential Revision: https://reviews.llvm.org/D54336 llvm-svn: 346722
* [FileCheck] Parse command-line options from FILECHECK_OPTSJoel E. Denny2018-11-061-0/+3
| | | | | | | | | | | | | | | | | This feature makes it easy to tune FileCheck diagnostic output when running the test suite via ninja, a bot, or an IDE. For example: ``` $ FILECHECK_OPTS='-color -v -dump-input-on-failure' \ LIT_FILTER='OpenMP/for_codegen.cpp' ninja check-clang \ | less -R ``` Reviewed By: probinson Differential Revision: https://reviews.llvm.org/D53517 llvm-svn: 346272
* [SourceMgr][FileCheck] Obey -color by extending WithColorJoel E. Denny2018-10-241-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | (Relands r344930, reverted in r344935, and now hopefully fixed for Windows.) While this change specifically targets FileCheck, it affects any tool using the same SourceMgr facilities. Previously, -color was documented in FileCheck's -help output, but -color had no effect. Now, -color obeys its documentation: it forces colors to be used in FileCheck diagnostics even when stderr is not a terminal. -color is especially helpful when combined with FileCheck's -v, which can produce a long series of diagnostics that you might wish to pipe to a pager, such as less -R. The WithColor extensions here will also help to clean up color usage in FileCheck's annotated dump of input, which is proposed in D52999. Reviewed By: JDevlieghere, zturner Differential Revision: https://reviews.llvm.org/D53419 llvm-svn: 345202
* Revert r344930 as it broke some of the bots on Windows.Aaron Ballman2018-10-221-4/+0
| | | | | | http://lab.llvm.org:8011/builders/clang-x64-windows-msvc/builds/739 llvm-svn: 344935
* [SourceMgr][FileCheck] Obey -color by extending WithColorJoel E. Denny2018-10-221-0/+4
| | | | | | | | | | | | | | | | | | | | | | While this change specifically targets FileCheck, it affects any tool using the same SourceMgr facilities. Previously, -color was documented in FileCheck's -help output, but -color had no effect. Now, -color obeys its documentation: it forces colors to be used in FileCheck diagnostics even when stderr is not a terminal. -color is especially helpful when combined with FileCheck's -v, which can produce a long series of diagnostics that you might wish to pipe to a pager, such as less -R. The WithColor extensions here will also help to clean up color usage in FileCheck's annotated dump of input, which is proposed in D52999. Reviewed By: JDevlieghere Differential Revision: https://reviews.llvm.org/D53419 llvm-svn: 344930
* [docs] Fix an LLVM-syntax code block to actually be valid LLVM synatx.Chandler Carruth2018-08-061-2/+2
| | | | | | Hopefully fixes an issue on the docs build bot. llvm-svn: 338980
* [FileCheck] Provide an option for FileCheck to dump original input to stderr ↵George Karpenkov2018-07-201-0/+4
| | | | | | | | | | | | | | | | on failure The option can be either set using environment variable (e.g. env FILECHECK_DUMP_INPUT_ON_FAILURE=1 ninja check-fuzzer) or with a FileCheck flag. This can be extremely useful for debugging, cf. https://groups.google.com/forum/#!topic/llvm-dev/kLrzg8OM_h8 for discussion. Differential Revision: https://reviews.llvm.org/D49328 llvm-svn: 337609
* [FileCheck] Implement -v and -vv for tracing matchesJoel E. Denny2018-07-131-0/+10
| | | | | | | | | | | | | | | | | | -v prints all directive pattern matches. -vv additionally prints info that might be noise to users but that can be helpful to FileCheck developers. To maximize code reuse and to make diagnostics more consistent, this patch also adjusts and extends some of the existing diagnostics. CHECK-NOT failures now report variables uses. Many more diagnostics now report the check prefix and kind of directive. Reviewed By: probinson Differential Revision: https://reviews.llvm.org/D47114 llvm-svn: 336967
* [FileCheck] Don't permit overlapping CHECK-DAGJoel E. Denny2018-07-111-0/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | That is, make CHECK-DAG skip matches that overlap the matches of any preceding consecutive CHECK-DAG directives. This change makes CHECK-DAG more consistent with other directives, and there is evidence it makes CHECK-DAG more intuitive and less error-prone. See the RFC discussion starting at: http://lists.llvm.org/pipermail/llvm-dev/2018-May/123010.html Moreover, this behavior enables CHECK-DAG groups for unordered, non-unique strings or patterns. For example, it is useful for verifying output or logs from a parallel program, such as the OpenMP runtime. This patch also implements the command-line option -allow-deprecated-dag-overlap, which reverts CHECK-DAG to the old overlapping behavior. This option should not be used in new tests. It is meant only for the existing tests that are broken by this change and that need time to update. See the following bugzilla issue for tracking of such tests: https://bugs.llvm.org/show_bug.cgi?id=37532 Patches to add -allow-deprecated-dag-overlap to those tests will follow immediately. Reviewed By: probinson Differential Revision: https://reviews.llvm.org/D47106 llvm-svn: 336847
* Revert r336830: [FileCheck] Don't permit overlapping CHECK-DAGJoel E. Denny2018-07-111-26/+0
| | | | | | | Companion patches are failing to commit, and this patch alone breaks many tests. llvm-svn: 336833
* [FileCheck] Don't permit overlapping CHECK-DAGJoel E. Denny2018-07-111-0/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | That is, make CHECK-DAG skip matches that overlap the matches of any preceding consecutive CHECK-DAG directives. This change makes CHECK-DAG more consistent with other directives, and there is evidence it makes CHECK-DAG more intuitive and less error-prone. See the RFC discussion starting at: http://lists.llvm.org/pipermail/llvm-dev/2018-May/123010.html Moreover, this behavior enables CHECK-DAG groups for unordered, non-unique strings or patterns. For example, it is useful for verifying output or logs from a parallel program, such as the OpenMP runtime. This patch also implements the command-line option -allow-deprecated-dag-overlap, which reverts CHECK-DAG to the old overlapping behavior. This option should not be used in new tests. It is meant only for the existing tests that are broken by this change and that need time to update. See the following bugzilla issue for tracking of such tests: https://bugs.llvm.org/show_bug.cgi?id=37532 Patches to add -allow-deprecated-dag-overlap to those tests will follow immediately. Reviewed By: probinson Differential Revision: https://reviews.llvm.org/D47106 llvm-svn: 336830
* Fix doc title underlining.James Henderson2018-06-261-1/+1
| | | | llvm-svn: 335615
* [FileCheck] Add CHECK-EMPTY directive for checking for blank linesJames Henderson2018-06-261-0/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Prior to this change, there was no clean way of getting FileCheck to check that a line is completely empty. The expected way of using "CHECK: {{^$}}" does not work because the '^' matches the end of the previous match (this behaviour may be desirable in certain instances). For the same reason, "CHECK-NEXT: {{^$}}" will fail when the previous match was at the end of the line, as the pattern will match there. Using the recommended [[:space:]] to match an explicit new line could also match a space, and thus is not always desired. Literal '\n' matches also do not work. A workaround was suggested in the review, but it is a little clunky. This change adds a new directive that behaves the same as CHECK-NEXT, except that it only matches against empty lines (nothing, not even whitespace, is allowed). As with CHECK-NEXT, it will fail if more than one newline occurs before the next blank line. Example usage: ; test.txt foo bar ; CHECK: foo ; CHECK-EMPTY: ; CHECK-NEXT: bar Differential Revision: https://reviews.llvm.org/D28896 Reviewed by: probinson llvm-svn: 335613
* Add a -D flag to FileCheck to define variablesAlexander Richardson2017-11-071-0/+5
| | | | | | | | | | | | | | | | Summary: This makes it very easy to test files that only differ in a constant value somewhere in the test case. Reviewers: jlebar, hfinkel, chandlerc, probinson Reviewed By: probinson Subscribers: probinson, llvm-commits Differential Revision: https://reviews.llvm.org/D39629 llvm-svn: 317572
* [FileCheck] regexp doc update/fixSjoerd Meijer2017-10-131-5/+6
| | | | | | | | | | Minor doc update that the FileCheck matcher supports POSIX ERE. It also fixes a minor issue in the regexp describing a variable name: underscores are allowed too. Differential Revision: https://reviews.llvm.org/D38787 llvm-svn: 315679
* [FileCheck] Added --enable-var-scope option to enable scope for regex variables.Artem Belevich2017-03-091-0/+19
| | | | | | | | | | | | | If `--enable-var-scope` is in effect, variables with names that start with `$` are considered to be global. All other variables are local. All local variables get undefined at the beginning of each CHECK-LABEL block. Global variables are not affected by CHECK-LABEL. This makes it easier to ensure that individual tests are not affected by variables set in preceding tests. Differential Revision: https://reviews.llvm.org/D30749 llvm-svn: 297396
* [docs] Fixing Sphinx warnings to unclog the buildbotRenato Golin2016-07-201-4/+4
| | | | | | | | | | | | | | | | | | | | | | | Lots of blocks had "llvm" or "nasm" syntax types but either weren't following the syntax, or the syntax has changed (and sphinx hasn't keep up) or the type doesn't even exist (nasm?). Other documents had :options: what were invalid. I only removed those that had warnings, and left the ones that didn't, in order to follow the principle of least surprise. This is like this for ages, but the buildbot is now failing on errors. It may take a while to upgrade the buildbot's sphinx, if that's even possible, but that shouldn't stop us from getting docs updates (which seem down for quite a while). Also, we're not losing any syntax highlight, since when it doesn't parse, it doesn't colour. Ie. those blocks are not being highlighted anyway. I'm trying to get all docs in one go, so that it's easy to revert later if we do fix, or at least easy to know what's to fix. llvm-svn: 276109
* Document the ability to perform multi-line pattern matching in FileCheck.Wolfgang Pieb2016-06-271-0/+19
| | | | | | Differential review: http://reviews.llvm.org/D21522 llvm-svn: 273962
* [FileCheck] Document --check-prefixes.Daniel Sanders2016-06-141-0/+5
| | | | llvm-svn: 272683
* Add -match-full-lines argument to FileCheck.James Y Knight2016-02-111-0/+12
| | | | | | | This is useful for some tests where more-exact matching is useful, such as clang's Preprocessor tests. llvm-svn: 260540
* IR: Give 'DI' prefix to debug info metadataDuncan P. N. Exon Smith2015-04-291-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Finish off PR23080 by renaming the debug info IR constructs from `MD*` to `DI*`. The last of the `DIDescriptor` classes were deleted in r235356, and the last of the related typedefs removed in r235413, so this has all baked for about a week. Note: If you have out-of-tree code (like a frontend), I recommend that you get everything compiling and tests passing with the *previous* commit before updating to this one. It'll be easier to keep track of what code is using the `DIDescriptor` hierarchy and what you've already updated, and I think you're extremely unlikely to insert bugs. YMMV of course. Back to *this* commit: I did this using the rename-md-di-nodes.sh upgrade script I've attached to PR23080 (both code and testcases) and filtered through clang-format-diff.py. I edited the tests for test/Assembler/invalid-generic-debug-node-*.ll by hand since the columns were off-by-three. It should work on your out-of-tree testcases (and code, if you've followed the advice in the previous paragraph). Some of the tests are in badly named files now (e.g., test/Assembler/invalid-mdcompositetype-missing-tag.ll should be 'dicompositetype'); I'll come back and move the files in a follow-up commit. llvm-svn: 236120
* All FileCheck directives allow patterns.Paul Robinson2015-03-051-1/+1
| | | | llvm-svn: 231418
* FileCheck: Document CHECK-SAME, follow-up to r230612Duncan P. N. Exon Smith2015-03-051-0/+25
| | | | llvm-svn: 231379
* Fix Sphinx warnings.Dan Liew2014-07-211-1/+1
| | | | llvm-svn: 213559
* Add FileCheck -implicit-check-not option to allow stricter tests without ↵Alexander Kornienko2014-07-111-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | adding too many CHECK-NOTs manually. Summary: Add FileCheck -implicit-check-not option which allows specifying a pattern that should only occur in the input when explicitly matched by a positive check. This feature allows checking tool diagnostics in a way clang -verify does it for compiler diagnostics. The option has been tested on a number of clang-tidy checks, I'll post a link to the clang-tidy patch to this thread. Once there's an agreement on the general direction, I can add tests and documentation. Reviewers: djasper, bkramer Reviewed By: bkramer Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D4462 llvm-svn: 212810
* Allow multiple check prefixes in FileCheck.Matt Arsenault2013-11-101-5/+7
| | | | | | | | This is useful if you want to run multiple variations of a single test, and the majority of check lines should be the same. llvm-svn: 194343
* Add warning about CHECK-DAG with variable definitionRenato Golin2013-10-111-9/+37
| | | | llvm-svn: 192479
* Fix underscore to be the proper length.Bill Wendling2013-07-301-1/+1
| | | | llvm-svn: 187406
* Fix FileCheck CHECK-LABEL documentation wording slightly; also mention that ↵Stephen Lin2013-07-181-8/+10
| | | | | | it allows error recovery. llvm-svn: 186628
* Correct inaccurate statement in FileCheck docs.Stephen Lin2013-07-141-1/+2
| | | | llvm-svn: 186290
* Add new directive called CHECK-LABEL to FileCheck.Stephen Lin2013-07-121-0/+49
| | | | | | | | CHECK-LABEL is meant to be used in place on CHECK on lines containing identifiers or other unique labels (they need not actually be labels in the source or output language, though.) This is used to break up the input stream into separate blocks delineated by CHECK-LABEL lines, each of which is checked independently. This greatly improves the accuracy of errors and fix-it hints in many cases, and allows for FileCheck to recover from errors in one block by continuing to subsequent blocks. Some tests will be converted to use this new directive in forthcoming patches. llvm-svn: 186162
* [docs] Fix formatting.Sean Silva2013-06-211-1/+1
| | | | | | '\n' was displaying as 'n' llvm-svn: 184507
* Add 'CHECK-DAG' supportMichael Liao2013-05-141-0/+49
| | | | | | Refer to 'FileCheck.rst'f for details of 'CHECK-DAG'. llvm-svn: 181827
* Canonicalize line endings to Linux style also when the --strict-whitespace ↵Guy Benyei2013-02-061-1/+2
| | | | | | flag is in use. This flag is supposed to affect horizontal whitespaces only. llvm-svn: 174541
* Update FileCheck's documentation to mention recently added feature ofEli Bendersky2012-12-011-15/+16
| | | | | | matching a variable defined on the same line. llvm-svn: 169103
* Documentation for FileCheck: use 'option' and 'program' directives.Dmitri Gribenko2012-11-291-36/+32
| | | | | | This enables option cross-referencing and now '--' in option names are no more turned into en dashes. llvm-svn: 168926
* Fix a typo in FileCheck.rstEli Bendersky2012-11-211-1/+1
| | | | llvm-svn: 168466
* Support for [[@LINE]], [[@LINE+<offset>]], [[@LINE-<offset>]] expressions inAlexander Kornienko2012-11-141-0/+27
| | | | | | FileCheck. llvm-svn: 167978
* FileCheck.rst: change formatting of code-like constructs to use a monospacedDmitri Gribenko2012-11-141-86/+50
| | | | | | font. These were formatted in bold, but that's not correct. llvm-svn: 167964
* Fix a broken sentenceEli Bendersky2012-11-071-2/+3
| | | | llvm-svn: 167521
* Document the -input-file option of FileCheckEli Bendersky2012-11-071-0/+5
| | | | llvm-svn: 167517
* Use correct syntax highliter in code blocks. Noticed by Sean Silva.Dmitri Gribenko2012-06-121-7/+7
| | | | llvm-svn: 158359
* FileCheck docs: remove leftover HTML markup.Dmitri Gribenko2012-06-121-9/+9
| | | | llvm-svn: 158344
* Typo. Patch by Nicklas Bo Jensen <nbjensen@gmail.com>.Chad Rosier2012-05-241-1/+1
| | | | llvm-svn: 157415
* [docs] Add ReST version of all the man pages.Daniel Dunbar2012-05-081-0/+284
- The POD versions are slated for execution, but are still around until llvm.org machinery is in place. llvm-svn: 156384
OpenPOWER on IntegriCloud