summaryrefslogtreecommitdiffstats
path: root/llvm/utils/FileCheck
Commit message (Collapse)AuthorAgeFilesLines
* [FileCheck] Given multiple -dump-input, prefer most verboseJoel E. Denny2019-12-031-7/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: `FILECHECK_OPTS` was implemented so that a test runner, such as a bot, can specify FileCheck debugging options, such as `-dump-input=fail`. However, some existing test suites have FileCheck calls that already specify `-dump-input=fail` or `-dump-input=always`. Without this patch, such tests fail under such a test runner because FileCheck doesn't accept multiple occurrences of `-dump-input`. Solution: This patch permits multiple occurrences of `-dump-input` by assigning precedence to its values in the following descending order: `help`, `always`, `fail`, and `never`. That is, any occurrence of `help` always obtains help, and otherwise the behavior is similar to `-v` vs. `-vv` in that the option specifying the greatest verbosity has precedence. Rationale: My justification for the new behavior is as follows. I have not experienced use cases where, either as a test runner or as a test author, I want to **limit** the permitted debugging verbosity (except as a test author in FileCheck's or lit's test suites where the FileCheck debugging output itself is under test, but the solution there is `env FILECHECK_OPTS=`, and I imagine we should use the same solution anywhere else this need might occur). Of course, as either a test runner or test author, it is useful to **increase** debugging verbosity. Reviewed By: probinson Differential Revision: https://reviews.llvm.org/D70784
* [FileCheck] Implement --ignore-case option.Kai Nacke2019-10-111-0/+5
| | | | | | | | | | | | The FileCheck utility is enhanced to support a `--ignore-case` option. This is useful in cases where the output of Unix tools differs in case (e.g. case not specified by Posix). Reviewers: Bigcheese, jakehehrlich, rupprecht, espindola, alexshap, jhenderson, MaskRay Differential Revision: https://reviews.llvm.org/D68146 llvm-svn: 374538
* Revert "[FileCheck] Implement --ignore-case option."Dmitri Gribenko2019-10-101-656/+651
| | | | | | | This reverts commit r374339. It broke tests: http://lab.llvm.org:8011/builders/clang-x86_64-debian-fast/builds/19066 llvm-svn: 374359
* [FileCheck] Implement --ignore-case option.Kai Nacke2019-10-101-651/+656
| | | | | | | | | | | | The FileCheck utility is enhanced to support a `--ignore-case` option. This is useful in cases where the output of Unix tools differs in case (e.g. case not specified by Posix). Reviewers: Bigcheese, jakehehrlich, rupprecht, espindola, alexshap, jhenderson, MaskRay Differential Revision: https://reviews.llvm.org/D68146 llvm-svn: 374339
* [FileCheck] Remove implementation types from APIThomas Preud'homme2019-09-301-3/+2
| | | | | | | | | | | | | | | | | Summary: Remove use of FileCheckPatternContext and FileCheckString concrete types from FileCheck API to allow moving it and the other implementation only only declarations into a private header file. Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68186 llvm-svn: 373211
* [FileCheck] Document FILECHECK_OPTS in -helpJoel E. Denny2019-08-141-0/+4
| | | | | | | | Reviewed By: thopre Differential Revision: https://reviews.llvm.org/D65707 llvm-svn: 368787
* [FileCheck] Move -dump-input diagnostic to first lineJoel E. Denny2019-08-141-10/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Without this patch, `-dump-input` prints a diagnostic at the end of its marker range. For example: ``` 1: Start. check:1 ^~~~~~ 2: Bad. next:2 X~~~ 3: Many lines next:2 ~~~~~~~~~~ 4: of input. next:2 ~~~~~~~~~ 5: End. next:2 ~~~~ error: no match found ``` This patch moves it to the beginning like this: ``` 1: Start. check:1 ^~~~~~ 2: Bad. next:2 X~~~ error: no match found 3: Many lines next:2 ~~~~~~~~~~ 4: of input. next:2 ~~~~~~~~~ 5: End. next:2 ~~~~ ``` The former somehow looks nicer because the diagnostic doesn't appear to be somewhere within the marker range. However, the latter is more practical, especially when the marker range includes the remainder of a very long dump. First, in the case of an error, this patch enables me to search the dump for `error:` and usually immediately land where the detected error began. Second, when trying to follow FileCheck's logic, it's best to read top down, so this patch enables me to see each diagnostic as soon as I encounter its marker. Reviewed By: thopre Differential Revision: https://reviews.llvm.org/D65702 llvm-svn: 368786
* Revert r367649: Improve raw_ostream so that you can "write" colors using ↵Rui Ueyama2019-08-021-2/+3
| | | | | | | | operator<< This reverts commit r367649 in an attempt to unbreak Windows bots. llvm-svn: 367658
* Improve raw_ostream so that you can "write" colors using operator<<Rui Ueyama2019-08-021-3/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1. raw_ostream supports ANSI colors so that you can write messages to the termina with colors. Previously, in order to change and reset color, you had to call `changeColor` and `resetColor` functions, respectively. So, if you print out "error: " in red, for example, you had to do something like this: OS.changeColor(raw_ostream::RED); OS << "error: "; OS.resetColor(); With this patch, you can write the same code as follows: OS << raw_ostream::RED << "error: " << raw_ostream::RESET; 2. Add a boolean flag to raw_ostream so that you can disable colored output. If you disable colors, changeColor, operator<<(Color), resetColor and other color-related functions have no effect. Most LLVM tools automatically prints out messages using colors, and you can disable it by passing a flag such as `--disable-colors`. This new flag makes it easy to write code that works that way. Differential Revision: https://reviews.llvm.org/D65564 llvm-svn: 367649
* [llvm] [FileCheck] Use FILECHECK_DUMP_INPUT_ON_FAILURE only when non-emptyMichal Gorny2019-07-261-1/+2
| | | | | | | | | | | Enable dumping output only if FILECHECK_DUMP_INPUT_ON_FAILURE is set to a non-empty value. This is necessary to support disabling it via POSIX-compliant env(1) that does not support '-u' argument, and therefore fix regression caused by r366980. Differential Revision: https://reviews.llvm.org/D65334 llvm-svn: 367122
* FileCheck: Improve FileCheck variable terminologyThomas Preud'homme2019-05-231-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Terminology introduced by [[#]] blocks is confusing and does not integrate well with existing terminology. First, variables referred by [[]] blocks are called "pattern variables" while the text a CHECK directive needs to match is called a "CHECK pattern". This is inconsistent with variables in [[#]] blocks since [[#]] blocks are also found in CHECK pattern yet those variables are called "numeric variable". Second, the replacing of both [[]] and [[#]] blocks by the value of the variable or expression they contain is represented by a FileCheckPatternSubstitution class. The naming refers to being a substitution in a CHECK pattern but could be wrongly understood as being a substitution of a pattern variable. Third and lastly, comments use "numeric expression" to refer both to the [[#]] blocks as well as to the numeric expressions these blocks contain which get evaluated at match time. This patch solves these confusions by - calling variables in [[]] and [[#]] blocks as string and numeric variables respectively; - referring to [[]] and [[#]] as substitution *blocks*, with the former being a string substitution block and the latter a numeric substitution block; - calling [[]] and [[#]] blocks to be replaced by the value of a variable or expression they contain a substitution (as opposed to definition when these blocks are used to defined a variable), with the former being a string substitution and the latter a numeric substitution; - renaming the FileCheckPatternSubstitution as a FileCheckSubstitution class with FileCheckStringSubstitution and FileCheckNumericSubstitution subclasses; - restricting the use of "numeric expression" to refer to the expression that is evaluated in a numeric substitution. While numeric substitution blocks only support numeric substitutions of numeric expressions at the moment there are plans to augment numeric substitution blocks to support numeric definitions as well as both a numeric definition and numeric substitution in the same numeric substitution block. Reviewers: jhenderson, jdenny, probinson, arichardson Subscribers: hiraditya, arichardson, probinson, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62146 llvm-svn: 361445
* [FileCheck] Fix FileCheck.cpp compilation on SolarisRainer Orth2019-04-021-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Both LLVM 8.0.0 and current trunk fail to compile on Solaris with GCC 8.1.0: /vol/llvm/src/llvm/dist/utils/FileCheck/FileCheck.cpp: In function ‘void DumpAnnotatedInput(llvm::raw_ostream&, const llvm::FileCheckRequest&, llvm::StringRef, std::vector<InputAnnotation>&, unsigned int)’: /vol/llvm/src/llvm/dist/utils/FileCheck/FileCheck.cpp:408:41: error: call of overloaded ‘log10(unsigned int&)’ is ambiguous unsigned LineNoWidth = log10(LineCount) + 1; ^ In file included from /vol/gcc-8/lib/gcc/i386-pc-solaris2.11/8.1.0/include-fixed/math.h:24, from /vol/gcc-8/include/c++/8.1.0/cmath:45, from /vol/llvm/src/llvm/dist/include/llvm-c/DataTypes.h:28, from /vol/llvm/src/llvm/dist/include/llvm/Support/DataTypes.h:16, from /vol/llvm/src/llvm/dist/include/llvm/ADT/Hashing.h:47, from /vol/llvm/src/llvm/dist/include/llvm/ADT/ArrayRef.h:12, from /vol/llvm/src/llvm/dist/include/llvm/Support/CommandLine.h:22, from /vol/llvm/src/llvm/dist/utils/FileCheck/FileCheck.cpp:18: /vol/gcc-8/lib/gcc/i386-pc-solaris2.11/8.1.0/include-fixed/iso/math_iso.h:209:21: note: candidate: ‘long double std::log10(long double)’ inline long double log10(long double __X) { return __log10l(__X); } ^~~~~ /vol/gcc-8/lib/gcc/i386-pc-solaris2.11/8.1.0/include-fixed/iso/math_iso.h:170:15: note: candidate: ‘float std::log10(float)’ inline float log10(float __X) { return __log10f(__X); } ^~~~~ /vol/gcc-8/lib/gcc/i386-pc-solaris2.11/8.1.0/include-fixed/iso/math_iso.h:70:15: note: candidate: ‘double std::log10(double)’ extern double log10 __P((double)); ^~~~~ Fixed by using std::log10 instead, which allowed the compilation on i386-pc-solaris2.11 and sparc-sun-solaris2.11 to continue. Differential Revision: https://reviews.llvm.org/D60043 llvm-svn: 357509
* Recommit: Detect incorrect FileCheck variable CLI definitionThomas Preud'homme2019-02-051-4/+22
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: While the backend code of FileCheck relies on definition of variable from the command-line to have an equal sign '=' and a variable name before that, the frontend does not actually enforce it. This leads to FileCheck crashing when invoked with invalid syntax for the -D option. This patch adds the missing validation in the frontend. It also makes the -D option an AlwaysPrefix option to be able to detect -D=FOO as being a define without variable and -D as missing its value. Copyright: - Linaro (changes in version 2 of revision D55940) - GraphCore (changes in later versions) Reviewers: jdenny Subscribers: JonChesterfield, hiraditya, kristina, probinson, llvm-commits Differential Revision: https://reviews.llvm.org/D55940 llvm-svn: 353173
* Revert "Detect incorrect FileCheck variable CLI definition"Thomas Preud'homme2019-01-271-22/+4
| | | | | | This reverts commit r351039. llvm-svn: 352309
* [FileCheck] Suppress old -v/-vv diags if dumping inputJoel E. Denny2019-01-221-3/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The old diagnostic form of the trace produced by -v and -vv looks like: ``` check1:1:8: remark: CHECK: expected string found in input CHECK: abc ^ <stdin>:1:3: note: found here ; abc def ^~~ ``` When dumping annotated input is requested (via -dump-input), I find that this old trace is not useful and is sometimes harmful: 1. The old trace is mostly redundant because the same basic information also appears in the input dump's annotations. 2. The old trace buries any error diagnostic between it and the input dump, but I find it useful to see any error diagnostic up front. 3. FILECHECK_OPTS=-dump-input=fail requests annotated input dumps only for failed FileCheck calls. However, I have to also add -v or -vv to get a full set of annotations, and that can produce massive output from all FileCheck calls in all tests. That's a real problem when I run this in the IDE I use, which grinds to a halt as it tries to capture all that output. When -dump-input=fail|always, this patch suppresses the old trace from -v or -vv. Error diagnostics still print as usual. If you want the old trace, perhaps to see variable expansions, you can set -dump-input=none (the default). Reviewed By: probinson Differential Revision: https://reviews.llvm.org/D55825 llvm-svn: 351881
* Update the file headers across all of the LLVM projects in the monorepoChandler Carruth2019-01-191-4/+3
| | | | | | | | | | | | | | | | | to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
* Detect incorrect FileCheck variable CLI definitionThomas Preud'homme2019-01-141-4/+22
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: While the backend code of FileCheck relies on definition of variable from the command-line to have an equal sign '=' and a variable name before that, the frontend does not actually enforce it. This leads to FileCheck crashing when invoked with invalid syntax for the -D option. This patch adds the missing validation in the frontend. It also makes the -D option an AlwaysPrefix option to be able to detect -D=FOO as being a define without variable and -D as missing its value. Copyright: - Linaro (changes in version 2 of revision D55940) - GraphCore (changes in later versions) Reviewers: jdenny Subscribers: JonChesterfield, hiraditya, kristina, probinson, llvm-commits Differential Revision: https://reviews.llvm.org/D55940 llvm-svn: 351039
* [FileCheck] Annotate input dump (final tweaks)Joel E. Denny2018-12-181-16/+16
| | | | | | | | | | | | | | | | | | | Apply final suggestions from probinson for this patch series plus a few more tweaks: * Improve various docs, for MatchType in particular. * Rename some members of MatchType. The main problem was that the term "final match" became a misnomer when CHECK-COUNT-<N> was created. * Split InputStartLine, etc. declarations into multiple lines. Differential Revision: https://reviews.llvm.org/D55738 Reviewed By: probinson llvm-svn: 349425
* [FileCheck] Annotate input dump (7/7)Joel E. Denny2018-12-181-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch implements annotations for diagnostics reporting CHECK-NOT failed matches. These diagnostics are enabled by -vv. As for diagnostics reporting failed matches for other directives, these annotations mark the search ranges using `X~~`. The difference here is that failed matches for CHECK-NOT are successes not errors, so they are green not red when colors are enabled. For example: ``` $ FileCheck -dump-input=help The following description was requested by -dump-input=help to explain the input annotations printed by -dump-input=always and -dump-input=fail: - L: labels line number L of the input file - T:L labels the only match result for a pattern of type T from line L of the check file - T:L'N labels the Nth match result for a pattern of type T from line L of the check file - ^~~ marks good match (reported if -v) - !~~ marks bad match, such as: - CHECK-NEXT on same line as previous match (error) - CHECK-NOT found (error) - CHECK-DAG overlapping match (discarded, reported if -vv) - X~~ marks search range when no match is found, such as: - CHECK-NEXT not found (error) - CHECK-NOT not found (success, reported if -vv) - CHECK-DAG not found after discarded matches (error) - ? marks fuzzy match when no match is found - colors success, error, fuzzy match, discarded match, unmatched input If you are not seeing color above or in input dumps, try: -color $ FileCheck -vv -dump-input=always check5 < input5 |& sed -n '/^<<<</,$p' <<<<<< 1: abcdef check:1 ^~~ not:2 X~~ 2: ghijkl not:2 ~~~ check:3 ^~~ 3: mnopqr not:4 X~~~~~ 4: stuvwx not:4 ~~~~~~ 5: eof:4 ^ >>>>>> $ cat check5 CHECK: abc CHECK-NOT: foobar CHECK: jkl CHECK-NOT: foobar $ cat input5 abcdef ghijkl mnopqr stuvwx ``` Reviewed By: george.karpenkov, probinson Differential Revision: https://reviews.llvm.org/D53899 llvm-svn: 349424
* [FileCheck] Annotate input dump (6/7)Joel E. Denny2018-12-181-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch implements input annotations for diagnostics reporting CHECK-DAG discarded matches. These diagnostics are enabled by -vv. These annotations mark discarded match ranges using `!~~` because they are bad matches even though they are not errors. CHECK-DAG discarded matches create another case where there can be multiple match results for the same directive. For example: ``` $ FileCheck -dump-input=help The following description was requested by -dump-input=help to explain the input annotations printed by -dump-input=always and -dump-input=fail: - L: labels line number L of the input file - T:L labels the only match result for a pattern of type T from line L of the check file - T:L'N labels the Nth match result for a pattern of type T from line L of the check file - ^~~ marks good match (reported if -v) - !~~ marks bad match, such as: - CHECK-NEXT on same line as previous match (error) - CHECK-NOT found (error) - CHECK-DAG overlapping match (discarded, reported if -vv) - X~~ marks search range when no match is found, such as: - CHECK-NEXT not found (error) - CHECK-DAG not found after discarded matches (error) - ? marks fuzzy match when no match is found - colors success, error, fuzzy match, discarded match, unmatched input If you are not seeing color above or in input dumps, try: -color $ FileCheck -vv -dump-input=always check4 < input4 |& sed -n '/^<<<</,$p' <<<<<< 1: abcdef dag:1 ^~~~ dag:2'0 !~~~ discard: overlaps earlier match 2: cdefgh dag:2'1 ^~~~ check:3 X~ error: no match found >>>>>> $ cat check4 CHECK-DAG: abcd CHECK-DAG: cdef CHECK: efgh $ cat input4 abcdef cdefgh ``` This shows that the line 3 CHECK fails to match even though its pattern appears in the input because its search range starts after the line 2 CHECK-DAG's match range. The trouble might be that the line 2 CHECK-DAG's match range is later than expected because its first match range overlaps with the line 1 CHECK-DAG match range and thus is discarded. Because `!~~` for CHECK-DAG does not indicate an error, it is not colored red. Instead, when colors are enabled, it is colored cyan, which suggests a match that went cold. Reviewed By: george.karpenkov, probinson Differential Revision: https://reviews.llvm.org/D53898 llvm-svn: 349423
* [FileCheck] Annotate input dump (5/7)Joel E. Denny2018-12-181-15/+68
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch implements input annotations for diagnostics enabled by -v, which report good matches for directives. These annotations mark match ranges using `^~~`. For example: ``` $ FileCheck -dump-input=help The following description was requested by -dump-input=help to explain the input annotations printed by -dump-input=always and -dump-input=fail: - L: labels line number L of the input file - T:L labels the only match result for a pattern of type T from line L of the check file - T:L'N labels the Nth match result for a pattern of type T from line L of the check file - ^~~ marks good match (reported if -v) - !~~ marks bad match, such as: - CHECK-NEXT on same line as previous match (error) - CHECK-NOT found (error) - X~~ marks search range when no match is found, such as: - CHECK-NEXT not found (error) - ? marks fuzzy match when no match is found - colors success, error, fuzzy match, unmatched input If you are not seeing color above or in input dumps, try: -color $ FileCheck -v -dump-input=always check3 < input3 |& sed -n '/^<<<</,$p' <<<<<< 1: abc foobar def check:1 ^~~ not:2 !~~~~~ error: no match expected check:3 ^~~ >>>>>> $ cat check3 CHECK: abc CHECK-NOT: foobar CHECK: def $ cat input3 abc foobar def ``` -vv enables these annotations for FileCheck's implicit EOF patterns as well. For an example where EOF patterns become relevant, see patch 7 in this series. If colors are enabled, `^~~` is green to suggest success. -v plus color enables highlighting of input text that has no final match for any expected pattern. The highlight uses a cyan background to suggest a cold section. This highlighting can make it easier to spot text that was intended to be matched but that failed to be matched in a long series of good matches. CHECK-COUNT-<num> good matches are another case where there can be multiple match results for the same directive. Reviewed By: george.karpenkov, probinson Differential Revision: https://reviews.llvm.org/D53897 llvm-svn: 349422
* [FileCheck] Annotate input dump (4/7)Joel E. Denny2018-12-181-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch implements input annotations for diagnostics that report unexpected matches for CHECK-NOT. Like wrong-line matches for CHECK-NEXT, CHECK-SAME, and CHECK-EMPTY, these annotations mark match ranges using red `!~~` to indicate bad matches that are errors. For example: ``` $ FileCheck -dump-input=help The following description was requested by -dump-input=help to explain the input annotations printed by -dump-input=always and -dump-input=fail: - L: labels line number L of the input file - T:L labels the only match result for a pattern of type T from line L of the check file - T:L'N labels the Nth match result for a pattern of type T from line L of the check file - !~~ marks bad match, such as: - CHECK-NEXT on same line as previous match (error) - CHECK-NOT found (error) - X~~ marks search range when no match is found, such as: - CHECK-NEXT not found (error) - ? marks fuzzy match when no match is found - colors error, fuzzy match If you are not seeing color above or in input dumps, try: -color $ FileCheck -v -dump-input=always check3 < input3 |& sed -n '/^<<<</,$p' <<<<<< 1: abc foobar def not:2 !~~~~~ error: no match expected >>>>>> $ cat check3 CHECK: abc CHECK-NOT: foobar CHECK: def $ cat input3 abc foobar def ``` Reviewed By: george.karpenkov, probinson Differential Revision: https://reviews.llvm.org/D53896 llvm-svn: 349421
* [FileCheck] Annotate input dump (3/7)Joel E. Denny2018-12-181-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch implements input annotations for diagnostics that report wrong-line matches for the directives CHECK-NEXT, CHECK-SAME, and CHECK-EMPTY. Instead of the usual `^~~`, which is used by later patches for good matches, these annotations use `!~~` to mark the bad match ranges so that this category of errors is visually distinct. Because such matches are errors, these annotates are red when colors are enabled. For example: ``` $ FileCheck -dump-input=help The following description was requested by -dump-input=help to explain the input annotations printed by -dump-input=always and -dump-input=fail: - L: labels line number L of the input file - T:L labels the only match result for a pattern of type T from line L of the check file - T:L'N labels the Nth match result for a pattern of type T from line L of the check file - !~~ marks bad match, such as: - CHECK-NEXT on same line as previous match (error) - X~~ marks search range when no match is found, such as: - CHECK-NEXT not found (error) - ? marks fuzzy match when no match is found - colors error, fuzzy match If you are not seeing color above or in input dumps, try: -color $ FileCheck -v -dump-input=always check2 < input2 |& sed -n '/^<<<</,$p' <<<<<< 1: foo bar next:2 !~~ error: match on wrong line >>>>>> $ cat check2 CHECK: foo CHECK-NEXT: bar $ cat input2 foo bar ``` Reviewed By: george.karpenkov, probinson Differential Revision: https://reviews.llvm.org/D53894 llvm-svn: 349420
* [FileCheck] Annotate input dump (2/7)Joel E. Denny2018-12-181-8/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch implements input annotations for diagnostics that suggest fuzzy matches for directives for which no matches were found. Instead of using the usual `^~~`, which is used by later patches for good matches, these annotations use `?` so that fuzzy matches are visually distinct. No tildes are included as these diagnostics (independently of this patch) currently identify only the start of the match. For example: ``` $ FileCheck -dump-input=help The following description was requested by -dump-input=help to explain the input annotations printed by -dump-input=always and -dump-input=fail: - L: labels line number L of the input file - T:L labels the only match result for a pattern of type T from line L of the check file - T:L'N labels the Nth match result for a pattern of type T from line L of the check file - X~~ marks search range when no match is found - ? marks fuzzy match when no match is found - colors error, fuzzy match If you are not seeing color above or in input dumps, try: -color $ FileCheck -v -dump-input=always check1 < input1 |& sed -n '/^<<<</,$p' <<<<<< 1: ; abc def 2: ; ghI jkl next:3'0 X~~~~~~~~ error: no match found next:3'1 ? possible intended match >>>>>> $ cat check1 CHECK: abc CHECK-SAME: def CHECK-NEXT: ghi CHECK-SAME: jkl $ cat input1 ; abc def ; ghI jkl ``` This patch introduces the concept of multiple "match results" per directive. In the above example, the first match result for the CHECK-NEXT directive is the failed match, for which the annotation shows the search range. The second match result is the fuzzy match. Later patches will introduce other cases of multiple match results per directive. When colors are enabled, `?` is colored magenta. That is, it doesn't indicate the actual error, which a red `X~~` marker indicates, but its color suggests it's closely related. Reviewed By: george.karpenkov, probinson Differential Revision: https://reviews.llvm.org/D53893 llvm-svn: 349419
* [FileCheck] Annotate input dump (1/7)Joel E. Denny2018-12-181-7/+317
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Extend FileCheck to dump its input annotated with FileCheck's diagnostics: errors, good matches if -v, and additional information if -vv. The goal is to make it easier to visualize FileCheck's matching behavior when debugging. Each patch in this series implements input annotations for a particular category of FileCheck diagnostics. While the first few patches alone are somewhat useful, the annotations become much more useful as later patches implement annotations for -v and -vv diagnostics, which show the matching behavior leading up to the error. This first patch implements boilerplate plus input annotations for error diagnostics reporting that no matches were found for a directive. These annotations mark the search ranges of the failed directives. Instead of using the usual `^~~`, which is used by later patches for good matches, these annotations use `X~~` so that this category of errors is visually distinct. For example: ``` $ FileCheck -dump-input=help The following description was requested by -dump-input=help to explain the input annotations printed by -dump-input=always and -dump-input=fail: - L: labels line number L of the input file - T:L labels the match result for a pattern of type T from line L of the check file - X~~ marks search range when no match is found - colors error If you are not seeing color above or in input dumps, try: -color $ FileCheck -v -dump-input=always check1 < input1 |& sed -n '/^Input file/,$p' Input file: <stdin> Check file: check1 -dump-input=help describes the format of the following dump. Full input was: <<<<<< 1: ; abc def 2: ; ghI jkl next:3 X~~~~~~~~ error: no match found >>>>>> $ cat check1 CHECK: abc CHECK-SAME: def CHECK-NEXT: ghi CHECK-SAME: jkl $ cat input1 ; abc def ; ghI jkl ``` Some additional details related to the boilerplate: * Enabling: The annotated input dump is enabled by `-dump-input`, which can also be set via the `FILECHECK_OPTS` environment variable. Accepted values are `help`, `always`, `fail`, or `never`. As shown above, `help` describes the format of the dump. `always` is helpful when you want to investigate a successful FileCheck run, perhaps for an unexpected pass. `-dump-input-on-failure` and `FILECHECK_DUMP_INPUT_ON_FAILURE` remain as a deprecated alias for `-dump-input=fail`. * Diagnostics: The usual diagnostics are not suppressed in this mode and are printed first. For brevity in the example above, I've omitted them using a sed command. Sometimes they're perfectly sufficient, and then they make debugging quicker than if you were forced to hunt through a dump of long input looking for the error. If you think they'll get in the way sometimes, keep in mind that it's pretty easy to grep for the start of the input dump, which is `<<<`. * Colored Annotations: The annotated input is colored if colors are enabled (enabling colors can be forced using -color). For example, errors are red. However, as in the above example, colors are not vital to reading the annotations. I don't know how to test color in the output, so any hints here would be appreciated. Reviewed By: george.karpenkov, zturner, probinson Differential Revision: https://reviews.llvm.org/D52999 llvm-svn: 349418
* [FileCheck] Parse command-line options from FILECHECK_OPTSJoel E. Denny2018-11-061-1/+2
| | | | | | | | | | | | | | | | | This feature makes it easy to tune FileCheck diagnostic output when running the test suite via ninja, a bot, or an IDE. For example: ``` $ FILECHECK_OPTS='-color -v -dump-input-on-failure' \ LIT_FILTER='OpenMP/for_codegen.cpp' ninja check-clang \ | less -R ``` Reviewed By: probinson Differential Revision: https://reviews.llvm.org/D53517 llvm-svn: 346272
* [SourceMgr][FileCheck] Obey -color by extending WithColorJoel E. Denny2018-10-241-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | (Relands r344930, reverted in r344935, and now hopefully fixed for Windows.) While this change specifically targets FileCheck, it affects any tool using the same SourceMgr facilities. Previously, -color was documented in FileCheck's -help output, but -color had no effect. Now, -color obeys its documentation: it forces colors to be used in FileCheck diagnostics even when stderr is not a terminal. -color is especially helpful when combined with FileCheck's -v, which can produce a long series of diagnostics that you might wish to pipe to a pager, such as less -R. The WithColor extensions here will also help to clean up color usage in FileCheck's annotated dump of input, which is proposed in D52999. Reviewed By: JDevlieghere, zturner Differential Revision: https://reviews.llvm.org/D53419 llvm-svn: 345202
* Refactor FileCheck to make it usable as an APIAditya Nandakumar2018-08-071-1442/+29
| | | | | | | | | | https://reviews.llvm.org/D50283 reviewed by bogner This patch refactors FileCheck's implementation into support so it can be used from C++ in other places (Unit tests). llvm-svn: 339192
* [FileCheck] Provide an option for FileCheck to dump original input to stderr ↵George Karpenkov2018-07-201-1/+12
| | | | | | | | | | | | | | | | on failure The option can be either set using environment variable (e.g. env FILECHECK_DUMP_INPUT_ON_FAILURE=1 ninja check-fuzzer) or with a FileCheck flag. This can be extremely useful for debugging, cf. https://groups.google.com/forum/#!topic/llvm-dev/kLrzg8OM_h8 for discussion. Differential Revision: https://reviews.llvm.org/D49328 llvm-svn: 337609
* [FileCheck] Fix search ranges for DAG-NOT-DAGJoel E. Denny2018-07-201-46/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A DAG-NOT-DAG is a CHECK-DAG group, X, followed by a CHECK-NOT group, N, followed by a CHECK-DAG group, Y. Let y be the initial directive of Y. This patch makes the following changes to the behavior: 1. Directives in N can no longer match within part of Y's match range just because y happens not to be the earliest match from Y. Specifically, this patch withdraws N's search range end from y's match range start to Y's match range start. 2. y can no longer match within X's match range, where a y match produced a reordering complaint, which is thus no longer possible. Specifically, this patch withdraws y's search range start from X's permitted range start to X's match range end, which was already the search range start for other members of Y. Both of these changes can only increase the number of test passes: #1 constrains the ability of CHECK-NOTs to match, and #2 expands the ability of CHECK-DAGs to match without complaints. These changes are based on discussions at: <http://lists.llvm.org/pipermail/llvm-dev/2018-May/123550.html> <https://reviews.llvm.org/D47106> which conclude that: 1. These changes simplify the FileCheck conceptual model. First, it makes search ranges for DAG-NOT-DAG more consistent with other cases. Second, it was confusing that y was treated differently from the rest of Y. 2. These changes add theoretical use cases for DAG-NOT-DAG that had no obvious means to be expressed otherwise. We can justify the first half of this assertion with the observation that these changes can only increase the number of test passes. 3. Reordering detection for DAG-NOT-DAG had no obvious real benefit. We don't have evidence from real uses cases to help us debate conclusions #2 and #3, but #1 at least seems intuitive. Reviewed By: probinson Differential Revision: https://reviews.llvm.org/D48986 llvm-svn: 337605
* [FileCheck] Implement -v and -vv for tracing matchesJoel E. Denny2018-07-131-33/+137
| | | | | | | | | | | | | | | | | | -v prints all directive pattern matches. -vv additionally prints info that might be noise to users but that can be helpful to FileCheck developers. To maximize code reuse and to make diagnostics more consistent, this patch also adjusts and extends some of the existing diagnostics. CHECK-NOT failures now report variables uses. Many more diagnostics now report the check prefix and kind of directive. Reviewed By: probinson Differential Revision: https://reviews.llvm.org/D47114 llvm-svn: 336967
* [FileCheck] Don't permit overlapping CHECK-DAGJoel E. Denny2018-07-111-12/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | That is, make CHECK-DAG skip matches that overlap the matches of any preceding consecutive CHECK-DAG directives. This change makes CHECK-DAG more consistent with other directives, and there is evidence it makes CHECK-DAG more intuitive and less error-prone. See the RFC discussion starting at: http://lists.llvm.org/pipermail/llvm-dev/2018-May/123010.html Moreover, this behavior enables CHECK-DAG groups for unordered, non-unique strings or patterns. For example, it is useful for verifying output or logs from a parallel program, such as the OpenMP runtime. This patch also implements the command-line option -allow-deprecated-dag-overlap, which reverts CHECK-DAG to the old overlapping behavior. This option should not be used in new tests. It is meant only for the existing tests that are broken by this change and that need time to update. See the following bugzilla issue for tracking of such tests: https://bugs.llvm.org/show_bug.cgi?id=37532 Patches to add -allow-deprecated-dag-overlap to those tests will follow immediately. Reviewed By: probinson Differential Revision: https://reviews.llvm.org/D47106 llvm-svn: 336847
* Revert r336830: [FileCheck] Don't permit overlapping CHECK-DAGJoel E. Denny2018-07-111-53/+12
| | | | | | | Companion patches are failing to commit, and this patch alone breaks many tests. llvm-svn: 336833
* [FileCheck] Don't permit overlapping CHECK-DAGJoel E. Denny2018-07-111-12/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | That is, make CHECK-DAG skip matches that overlap the matches of any preceding consecutive CHECK-DAG directives. This change makes CHECK-DAG more consistent with other directives, and there is evidence it makes CHECK-DAG more intuitive and less error-prone. See the RFC discussion starting at: http://lists.llvm.org/pipermail/llvm-dev/2018-May/123010.html Moreover, this behavior enables CHECK-DAG groups for unordered, non-unique strings or patterns. For example, it is useful for verifying output or logs from a parallel program, such as the OpenMP runtime. This patch also implements the command-line option -allow-deprecated-dag-overlap, which reverts CHECK-DAG to the old overlapping behavior. This option should not be used in new tests. It is meant only for the existing tests that are broken by this change and that need time to update. See the following bugzilla issue for tracking of such tests: https://bugs.llvm.org/show_bug.cgi?id=37532 Patches to add -allow-deprecated-dag-overlap to those tests will follow immediately. Reviewed By: probinson Differential Revision: https://reviews.llvm.org/D47106 llvm-svn: 336830
* [FileCheck] Add CHECK-EMPTY directive for checking for blank linesJames Henderson2018-06-261-10/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Prior to this change, there was no clean way of getting FileCheck to check that a line is completely empty. The expected way of using "CHECK: {{^$}}" does not work because the '^' matches the end of the previous match (this behaviour may be desirable in certain instances). For the same reason, "CHECK-NEXT: {{^$}}" will fail when the previous match was at the end of the line, as the pattern will match there. Using the recommended [[:space:]] to match an explicit new line could also match a space, and thus is not always desired. Literal '\n' matches also do not work. A workaround was suggested in the review, but it is a little clunky. This change adds a new directive that behaves the same as CHECK-NEXT, except that it only matches against empty lines (nothing, not even whitespace, is allowed). As with CHECK-NEXT, it will fail if more than one newline occurs before the next blank line. Example usage: ; test.txt foo bar ; CHECK: foo ; CHECK-EMPTY: ; CHECK-NEXT: bar Differential Revision: https://reviews.llvm.org/D28896 Reviewed by: probinson llvm-svn: 335613
* Define InitLLVM to do common initialization all at once.Rui Ueyama2018-04-131-4/+2
| | | | | | | | | | | We have a few functions that virtually all command wants to run on process startup/shutdown. This patch adds InitLLVM class to do that all at once, so that we don't need to copy-n-paste boilerplate code to each llvm command's main() function. Differential Revision: https://reviews.llvm.org/D45602 llvm-svn: 330046
* [FileCheck] - Fix possible buffer out of bounds access when parsing ↵George Rimar2018-01-161-0/+3
| | | | | | | | | | | | | | | | | | | --check-prefix. FileCheck tool crashes when trying to parse --check-prefix argument if there is no any data after it. For example test like following would crash if there are no symbols and no EOL mark after `boom`: # REQUIRES: x86 # RUN: <skipped few lines> # RUN: llvm-readobj -t %t | FileCheck %s --check-prefix=boom Patch fixes the issue. Differential revision: https://reviews.llvm.org/D42057 llvm-svn: 322536
* [CMake] Use PRIVATE in target_link_libraries for executablesShoaib Meenai2017-12-051-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We currently use target_link_libraries without an explicit scope specifier (INTERFACE, PRIVATE or PUBLIC) when linking executables. Dependencies added in this way apply to both the target and its dependencies, i.e. they become part of the executable's link interface and are transitive. Transitive dependencies generally don't make sense for executables, since you wouldn't normally be linking against an executable. This also causes issues for generating install export files when using LLVM_DISTRIBUTION_COMPONENTS. For example, clang has a lot of LLVM library dependencies, which are currently added as interface dependencies. If clang is in the distribution components but the LLVM libraries it depends on aren't (which is a perfectly legitimate use case if the LLVM libraries are being built static and there are therefore no run-time dependencies on them), CMake will complain about the LLVM libraries not being in export set when attempting to generate the install export file for clang. This is reasonable behavior on CMake's part, and the right thing is for LLVM's build system to explicitly use PRIVATE dependencies for executables. Unfortunately, CMake doesn't allow you to mix and match the keyword and non-keyword target_link_libraries signatures for a single target; i.e., if a single call to target_link_libraries for a particular target uses one of the INTERFACE, PRIVATE, or PUBLIC keywords, all other calls must also be updated to use those keywords. This means we must do this change in a single shot. I also fully expect to have missed some instances; I tested by enabling all the projects in the monorepo (except dragonegg), and configuring both with and without shared libraries, on both Darwin and Linux, but I'm planning to rely on the buildbots for other configurations (since it should be pretty easy to fix those). Even after this change, we still have a lot of target_link_libraries calls that don't specify a scope keyword, mostly for shared libraries. I'm thinking about addressing those in a follow-up, but that's a separate change IMO. Differential Revision: https://reviews.llvm.org/D40823 llvm-svn: 319840
* Add a -D flag to FileCheck to define variablesAlexander Richardson2017-11-071-0/+7
| | | | | | | | | | | | | | | | Summary: This makes it very easy to test files that only differ in a constant value somewhere in the test case. Reviewers: jlebar, hfinkel, chandlerc, probinson Reviewed By: probinson Subscribers: probinson, llvm-commits Differential Revision: https://reviews.llvm.org/D39629 llvm-svn: 317572
* [FileCheck] Don't scan past the closing CHECK-DAG for CHECK-NOT inside CHECK-DAGBenjamin Kramer2017-06-071-1/+1
| | | | | | | If there's enough data in fron of it the skipped region would just become arbitrarily large, and we scan for the CHECK-NOT everywhere. llvm-svn: 304900
* Test commit.James Henderson2017-03-141-1/+1
| | | | llvm-svn: 297731
* [FileCheck] Added --enable-var-scope option to enable scope for regex variables.Artem Belevich2017-03-091-8/+33
| | | | | | | | | | | | | If `--enable-var-scope` is in effect, variables with names that start with `$` are considered to be global. All other variables are local. All local variables get undefined at the beginning of each CHECK-LABEL block. Global variables are not affected by CHECK-LABEL. This makes it easier to ensure that individual tests are not affected by variables set in preceding tests. Differential Revision: https://reviews.llvm.org/D30749 llvm-svn: 297396
* [FileCheck] Fix --strict-whitespace --match-full-linesTom de Vries2016-12-181-5/+7
| | | | | | | Make sure FileCheck --strict-whitespace --match-full-lines translates 'CHECK: bla ' into pattern '^ bla $' instead of pattern '^bla$'. llvm-svn: 290069
* [FileCheck] Fix comment in ReadCheckFileTom de Vries2016-12-181-1/+1
| | | | | | | The comment in ReadCheckFile claims that both leading and trailing whitespace are removed, but the associated statement only removes leading whitespace. llvm-svn: 290061
* [FileCheck] Re-implement the logic to find each check prefix in theChandler Carruth2016-12-111-93/+94
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | check file to not be unreasonably slow in the face of multiple check prefixes. The previous logic would repeatedly scan potentially large portions of the check file looking for alternative prefixes. In the worst case this would scan most of the file looking for a rare prefix between every single occurance of a common prefix. Even if we bounded the scan, this would do bad things if the order of the prefixes was "unlucky" and the distant prefix was scanned for first. None of this is necessary. It is straightforward to build a state machine that recognizes the first, longest of the set of alternative prefixes. That is in fact exactly whan a regular expression does. This patch builds a regular expression once for the set of prefixes and then uses it to search incrementally for the next prefix. This requires some threading of state but actually makes the code dramatically simpler. I've also added a big comment describing the algorithm as it was not at all obvious to me when I started. With this patch, several previously pathological test cases in test/CodeGen/X86 are 5x and more faster. Overall, running all tests under test/CodeGen/X86 uses 10% less CPU after this, and because all the slowest tests were hitting this, finishes in 40% less wall time on my system (going from just over 5.38s to just over 3.23s) on a release build! This patch substantially improves the time of all 7 X86 tests that were in the top 20 reported by --time-tests, 5 of them are completely off the list and the remaining 2 are much lower. (Sadly, the new tests on the list include 2 new X86 ones that are slow for unrelated reasons, so the count stays at 4 of the top 20.) It isn't clear how much this helps debug builds in aggregate in part because of the noise, but it again makes mane of the slowest x86 tests significantly faster (10% or more improvement). llvm-svn: 289382
* [FileCheck] Remove a parameter that was simply always set toChandler Carruth2016-12-111-9/+4
| | | | | | | | a commandline flag and test the flag directly. NFC. If we ever need this generality it can be added back. llvm-svn: 289381
* [FileCheck] Clean up doxygen comments throughout. NFC.Chandler Carruth2016-12-111-70/+62
| | | | llvm-svn: 289380
* [FileCheck] Run clang-format over this code. NFC.Chandler Carruth2016-12-111-118/+108
| | | | | | | | | | This fixes one formatting goof I left in my previous commit and *many* other inconsistencies. I'm planning to make substantial changes here and so wanted to get to a clean baseline. llvm-svn: 289379
* Refactor FileCheck some to reduce memory allocation and copying. AlsoChandler Carruth2016-12-111-87/+90
| | | | | | | | | | | | | | | | | | | | | | | | | | | | make some readability improvements. Both the check file and input file have to be fully buffered to normalize their whitespace. But previously this would be done in a stack SmallString and then copied into a heap allocated MemoryBuffer. That seems pretty wasteful, especially for something like FileCheck where there are only ever two such entities. This just rearranges the code so that we can keep the canonicalized buffers on the stack of the main function, use reasonably large stack buffers to reduce allocation. A rough estimate seems to show that about 80% of LLVM's .ll and .s files will fit into a 4k buffer, so this should completely avoid heap allocation for the buffer in those cases. My system's malloc is fast enough that the allocations don't directly show up in timings. However, on some very slow test cases, this saves 1% - 2% by avoiding the copy into the heap allocated buffer. This also splits out the code which checks the input into a helper much like the code to build the checks as that made the code much more readable to me. Nit picks and suggestions welcome here. It has really exposed a *bunch* of stuff that could be cleaned up though, so I'm probably going to go and spring clean all of this code as I have more changes coming to speed things up. llvm-svn: 289378
* FileCheck: Minor cleanup of the class PatternSaleem Abdulrasool2016-08-261-8/+6
| | | | | | | | | 1. Add the "explicit" specifier to the single-argument constructor of Pattern 2. Reorder the fields to remove excessive padding (8 bytes). Patch by Alexander Shaposhnikov! llvm-svn: 279832
OpenPOWER on IntegriCloud