summaryrefslogtreecommitdiffstats
path: root/llvm/test/FileCheck
Commit message (Collapse)AuthorAgeFilesLines
* [FileCheck] Given multiple -dump-input, prefer most verboseJoel E. Denny2019-12-031-6/+101
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: `FILECHECK_OPTS` was implemented so that a test runner, such as a bot, can specify FileCheck debugging options, such as `-dump-input=fail`. However, some existing test suites have FileCheck calls that already specify `-dump-input=fail` or `-dump-input=always`. Without this patch, such tests fail under such a test runner because FileCheck doesn't accept multiple occurrences of `-dump-input`. Solution: This patch permits multiple occurrences of `-dump-input` by assigning precedence to its values in the following descending order: `help`, `always`, `fail`, and `never`. That is, any occurrence of `help` always obtains help, and otherwise the behavior is similar to `-v` vs. `-vv` in that the option specifying the greatest verbosity has precedence. Rationale: My justification for the new behavior is as follows. I have not experienced use cases where, either as a test runner or as a test author, I want to **limit** the permitted debugging verbosity (except as a test author in FileCheck's or lit's test suites where the FileCheck debugging output itself is under test, but the solution there is `env FILECHECK_OPTS=`, and I imagine we should use the same solution anywhere else this need might occur). Of course, as either a test runner or test author, it is useful to **increase** debugging verbosity. Reviewed By: probinson Differential Revision: https://reviews.llvm.org/D70784
* [FileCheck] Make FILECHECK_OPTS useful for its test suiteJoel E. Denny2019-11-2129-114/+310
| | | | | | | | | | | | | | | | | | | | Without this patch, `FILECHECK_OPTS` isn't propagated to FileCheck's test suite so that `FILECHECK_OPTS` doesn't inadvertently affect test results by affecting the output of FileCheck calls under test. As a result, `FILECHECK_OPTS` is useless for debugging FileCheck's test suite. In `llvm/test/FileCheck/lit.local.cfg`, this patch provides a new subsitution, `%ProtectFileCheckOutput`, to address this problem for both `FILECHECK_OPTS` and the deprecated `FILECHECK_DUMP_INPUT_ON_FAILURE`. The rest of the patch uses `%ProtectFileCheckOutput` throughout the test suite Fixes PR40284. Reviewed By: probinson, thopre Differential Revision: https://reviews.llvm.org/D65121
* [FileCheck] Use lit's internal shell for the test suiteJoel E. Denny2019-11-191-0/+5
| | | | | | | | | | | | | | | | | | An advantage is that there are less portability concerns when writing tests. For example, `-u` is not supported by all implementations of `env`, but lit's internal shell provides its own `env` that supports `-u`. A disadvantage is that some shell constructs, such as parentheses, are not supported, but FileCheck's test suite currently doesn't require such constructs. For comparison, lit configures its test suite in the same manner. See `llvm/utils/lit/tests/lit.cfg`. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D70278
* [FileCheck] Implement --ignore-case option.Kai Nacke2019-10-111-0/+45
| | | | | | | | | | | | The FileCheck utility is enhanced to support a `--ignore-case` option. This is useful in cases where the output of Unix tools differs in case (e.g. case not specified by Posix). Reviewers: Bigcheese, jakehehrlich, rupprecht, espindola, alexshap, jhenderson, MaskRay Differential Revision: https://reviews.llvm.org/D68146 llvm-svn: 374538
* Revert "[FileCheck] Implement --ignore-case option."Dmitri Gribenko2019-10-101-45/+0
| | | | | | | This reverts commit r374339. It broke tests: http://lab.llvm.org:8011/builders/clang-x86_64-debian-fast/builds/19066 llvm-svn: 374359
* [FileCheck] Implement --ignore-case option.Kai Nacke2019-10-101-0/+45
| | | | | | | | | | | | The FileCheck utility is enhanced to support a `--ignore-case` option. This is useful in cases where the output of Unix tools differs in case (e.g. case not specified by Posix). Reviewers: Bigcheese, jakehehrlich, rupprecht, espindola, alexshap, jhenderson, MaskRay Differential Revision: https://reviews.llvm.org/D68146 llvm-svn: 374339
* [FileCheck] Forbid using var defined on same lineThomas Preud'homme2019-09-021-2/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Commit r366897 introduced the possibility to set a variable from an expression, such as [[#VAR2:VAR1+3]]. While introducing this feature, it introduced extra logic to allow using such a variable on the same line later on. Unfortunately that extra logic is flawed as it relies on a mapping from variable to expression defining it when the mapping is from variable definition to expression. This flaw causes among other issues PR42896. This commit avoids the problem by forbidding all use of a variable defined on the same line, and removes the now useless logic. Redesign will be done in a later commit because it will require some amount of refactoring first for the solution to be clean. One example is the need for some sort of transaction mechanism to set a variable temporarily and from an expression and rollback if the CHECK pattern does not match so that diagnostics show the right variable values. Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk Subscribers: JonChesterfield, rogfer01, hfinkel, kristina, rnk, tra, arichardson, grimar, dblaikie, probinson, llvm-commits, hiraditya Tags: #llvm Differential Revision: https://reviews.llvm.org/D66141 llvm-svn: 370663
* [FileCheck] Move -dump-input diagnostic to first lineJoel E. Denny2019-08-141-8/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Without this patch, `-dump-input` prints a diagnostic at the end of its marker range. For example: ``` 1: Start. check:1 ^~~~~~ 2: Bad. next:2 X~~~ 3: Many lines next:2 ~~~~~~~~~~ 4: of input. next:2 ~~~~~~~~~ 5: End. next:2 ~~~~ error: no match found ``` This patch moves it to the beginning like this: ``` 1: Start. check:1 ^~~~~~ 2: Bad. next:2 X~~~ error: no match found 3: Many lines next:2 ~~~~~~~~~~ 4: of input. next:2 ~~~~~~~~~ 5: End. next:2 ~~~~ ``` The former somehow looks nicer because the diagnostic doesn't appear to be somewhere within the marker range. However, the latter is more practical, especially when the marker range includes the remainder of a very long dump. First, in the case of an error, this patch enables me to search the dump for `error:` and usually immediately land where the detected error began. Second, when trying to follow FileCheck's logic, it's best to read top down, so this patch enables me to see each diagnostic as soon as I encounter its marker. Reviewed By: thopre Differential Revision: https://reviews.llvm.org/D65702 llvm-svn: 368786
* [FileCheck]] Canonicalize caret location testingThomas Preud'homme2019-07-245-80/+81
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Testing of caret location in diagnostic message is currently made with CHECK directive with the following general format: CHECK: {{^ \^$}} James Henderson suggested the following would be more readable: CHECK: {{^}} ^{{$}} and when whole lines can be matched (as is the case for command-line testing where error messages do not include path): CHECK: ^ using the option --match-full-lines. This commit implements these 2 changes on all existing caret position tests. It also aligns the caret to the character it is trying to match in the above line. Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk Subscribers: JonChesterfield, rogfer01, hfinkel, kristina, rnk, tra, arichardson, grimar, dblaikie, probinson, llvm-commits, hiraditya Tags: #llvm Differential Revision: https://reviews.llvm.org/D64923 > llvm-svn: 366861 llvm-svn: 366898
* FileCheck [8/12]: Define numeric var from exprThomas Preud'homme2019-07-243-36/+73
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch is part of a patch series to add support for FileCheck numeric expressions. This specific patch lift the restriction for a numeric expression to either be a variable definition or a numeric expression to try to match. This commit allows a numeric variable to be set to the result of the evaluation of a numeric expression after it has been matched successfully. When it happens, the variable is allowed to be used on the same line since its value is known at match time. It also makes use of this possibility to reuse the parsing code to parse a command-line definition by crafting a mirror string of the -D option with the equal sign replaced by a colon sign, e.g. for option '-D#NUMVAL=10' it creates the string '-D#NUMVAL=10 (parsed as [[#NUMVAL:10]])' where the numeric expression is parsed to define NUMVAL. This result in a few tests needing updating for the location diagnostics on top of the tests for the new feature. It also enables empty numeric expression which match any number without defining a variable. This is done here rather than in commit #5 of the patch series because it requires to dissociate automatic regex insertion in RegExStr from variable definition which would make commit #5 even bigger than it already is. Copyright: - Linaro (changes up to diff 183612 of revision D55940) - GraphCore (changes in later versions of revision D55940 and in new revision created off D55940) Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield Tags: #llvm Differential Revision: https://reviews.llvm.org/D60388 > llvm-svn: 366860 llvm-svn: 366897
* Revert "FileCheck [8/12]: Define numeric var from expr"Thomas Preud'homme2019-07-243-73/+36
| | | | | | This reverts commit 1b05977538d9487aa845ee2f3bec8b89c63c4f29. llvm-svn: 366872
* Revert "[FileCheck]] Canonicalize caret location testing"Thomas Preud'homme2019-07-245-81/+80
| | | | | | This reverts commit ba4e8d791342219c072a5c8de8a3f41fbb53c6be. llvm-svn: 366871
* [FileCheck]] Canonicalize caret location testingThomas Preud'homme2019-07-235-80/+81
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Testing of caret location in diagnostic message is currently made with CHECK directive with the following general format: CHECK: {{^ \^$}} James Henderson suggested the following would be more readable: CHECK: {{^}} ^{{$}} and when whole lines can be matched (as is the case for command-line testing where error messages do not include path): CHECK: ^ using the option --match-full-lines. This commit implements these 2 changes on all existing caret position tests. It also aligns the caret to the character it is trying to match in the above line. Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk Subscribers: JonChesterfield, rogfer01, hfinkel, kristina, rnk, tra, arichardson, grimar, dblaikie, probinson, llvm-commits, hiraditya Tags: #llvm Differential Revision: https://reviews.llvm.org/D64923 llvm-svn: 366861
* FileCheck [8/12]: Define numeric var from exprThomas Preud'homme2019-07-233-36/+73
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch is part of a patch series to add support for FileCheck numeric expressions. This specific patch lift the restriction for a numeric expression to either be a variable definition or a numeric expression to try to match. This commit allows a numeric variable to be set to the result of the evaluation of a numeric expression after it has been matched successfully. When it happens, the variable is allowed to be used on the same line since its value is known at match time. It also makes use of this possibility to reuse the parsing code to parse a command-line definition by crafting a mirror string of the -D option with the equal sign replaced by a colon sign, e.g. for option '-D#NUMVAL=10' it creates the string '-D#NUMVAL=10 (parsed as [[#NUMVAL:10]])' where the numeric expression is parsed to define NUMVAL. This result in a few tests needing updating for the location diagnostics on top of the tests for the new feature. It also enables empty numeric expression which match any number without defining a variable. This is done here rather than in commit #5 of the patch series because it requires to dissociate automatic regex insertion in RegExStr from variable definition which would make commit #5 even bigger than it already is. Copyright: - Linaro (changes up to diff 183612 of revision D55940) - GraphCore (changes in later versions of revision D55940 and in new revision created off D55940) Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield Tags: #llvm Differential Revision: https://reviews.llvm.org/D60388 llvm-svn: 366860
* [FileCheck] Fix numeric variable redefinitionThomas Preud'homme2019-07-182-10/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Commit r365249 changed usage of FileCheckNumericVariable to have one instance of that class per variable as opposed to one instance per definition of a given variable as was done before. However, it retained the safety check in setValue that it should only be called with the variable unset, even after r365625. However this causes assert failure when a non-pseudo variable is being redefined. And while redefinition of @LINE at each CHECK line work in the general case, it caused problem when a substitution failed (fixed in r365624) and still causes problem when a CHECK line does not match since @LINE's value is cleared after substitutions in match() happened but printSubstitutions also attempts a substitution. This commit solves the root of the problem by changing setValue to set a new value regardless of whether a value was set or not, thus fixing all the aforementioned issues. Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield Tags: #llvm Differential Revision: https://reviews.llvm.org/D64882 llvm-svn: 366434
* FileCheck [7/12]: Arbitrary long numeric expressionsThomas Preud'homme2019-07-133-9/+17
| | | | | | | | | | | | | | | | | | | | | | Summary: This patch is part of a patch series to add support for FileCheck numeric expressions. This specific patch extend numeric expression to support an arbitrary number of operands, either variable or literals. Copyright: - Linaro (changes up to diff 183612 of revision D55940) - GraphCore (changes in later versions of revision D55940 and in new revision created off D55940) Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield Tags: #llvm Differential Revision: https://reviews.llvm.org/D60387 llvm-svn: 366001
* [FileCheck] Don't diagnose undef vars at parse timeThomas Preud'homme2019-07-051-2/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Diagnosing use of undefined variables takes place in parseNumericVariableUse() and printSubstitutions() for numeric variables but only takes place in printSubstitutions() for string variables. The reason for the split location of diagnostics is that parsing is not aware of the clearing of variables due to --enable-var-scope and thus use of variables cleared in this way can only be catched by printSubstitutions(). Beyond the code level inconsistency, there is also a user facing inconsistency since diagnostics look different between the two functions. While the diagnostic in printSubstitutions is more verbose, doing the diagnostic there allows to diagnose all undefined variables rather than just the first one and error out. This patch create dummy variable definition when encountering a use of undefined variable so that parsing can proceed and be diagnosed by printSubstitutions() later. Tests that were testing whether parsing fails in such case are thus modified accordingly. Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk Subscribers: JonChesterfield, rogfer01, hfinkel, kristina, rnk, tra, arichardson, grimar, dblaikie, probinson, llvm-commits, hiraditya Tags: #llvm Differential Revision: https://reviews.llvm.org/D64228 llvm-svn: 365219
* [FileCheck] Factor some parsing checks outThomas Preud'homme2019-07-051-2/+2
| | | | | | | | | | | | | | | | | Summary: Both callers of parseNumericVariableDefinition() perform the same extra check that no character is found after the variable name. This patch factors out this check into parseNumericVariableDefinition(). Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk Subscribers: JonChesterfield, rogfer01, hfinkel, kristina, rnk, tra, arichardson, grimar, dblaikie, probinson, llvm-commits, hiraditya Tags: #llvm Differential Revision: https://reviews.llvm.org/D64226 llvm-svn: 365191
* [FileCheck] Stop qualifying expressions as numericThomas Preud'homme2019-06-192-4/+4
| | | | | | | | | | | | | | | | | Summary: Stop referring to "numeric expression", using simply the term "expression" instead. Likewise for numeric operation since operations are only used in numeric expressions. Reviewers: jhenderson, jdenny, probinson, arichardson Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D63500 llvm-svn: 363901
* FileCheck: Return parse error w/ Error & ExpectedThomas Preud'homme2019-06-192-1/+13
| | | | | | | | | | | | | | | | Summary: Make use of Error and Expected to bubble up diagnostics and force checking of errors in the callers. Reviewers: jhenderson, jdenny, probinson, arichardson Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D63125 llvm-svn: 363900
* FileCheck [6/12]: Introduce numeric variable definitionThomas Preud'homme2019-06-064-62/+97
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch is part of a patch series to add support for FileCheck numeric expressions. This specific patch introduces support for defining numeric variable in a CHECK directive. This commit introduces support for defining numeric variable from a litteral value in the input text. Numeric expressions can then use the variable provided it is on a later line. Copyright: - Linaro (changes up to diff 183612 of revision D55940) - GraphCore (changes in later versions of revision D55940 and in new revision created off D55940) Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield Tags: #llvm Differential Revision: https://reviews.llvm.org/D60386 llvm-svn: 362705
* [FileCheck] Introduce substitution subclassesThomas Preud'homme2019-05-233-6/+6
| | | | | | | | | | | | | | | | | | | | Summary: With now a clear distinction between string and numeric substitutions, this patch introduces separate classes to represent them with a parent class implementing the common interface. Diagnostics in printSubstitutions() are also adapted to not require knowing which substitution is being looked at since it does not hinder clarity and makes the implementation simpler. Reviewers: jhenderson, jdenny, probinson, arichardson Subscribers: llvm-commits, probinson, arichardson, hiraditya Tags: #llvm Differential Revision: https://reviews.llvm.org/D62241 llvm-svn: 361446
* FileCheck: Improve FileCheck variable terminologyThomas Preud'homme2019-05-234-13/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Terminology introduced by [[#]] blocks is confusing and does not integrate well with existing terminology. First, variables referred by [[]] blocks are called "pattern variables" while the text a CHECK directive needs to match is called a "CHECK pattern". This is inconsistent with variables in [[#]] blocks since [[#]] blocks are also found in CHECK pattern yet those variables are called "numeric variable". Second, the replacing of both [[]] and [[#]] blocks by the value of the variable or expression they contain is represented by a FileCheckPatternSubstitution class. The naming refers to being a substitution in a CHECK pattern but could be wrongly understood as being a substitution of a pattern variable. Third and lastly, comments use "numeric expression" to refer both to the [[#]] blocks as well as to the numeric expressions these blocks contain which get evaluated at match time. This patch solves these confusions by - calling variables in [[]] and [[#]] blocks as string and numeric variables respectively; - referring to [[]] and [[#]] as substitution *blocks*, with the former being a string substitution block and the latter a numeric substitution block; - calling [[]] and [[#]] blocks to be replaced by the value of a variable or expression they contain a substitution (as opposed to definition when these blocks are used to defined a variable), with the former being a string substitution and the latter a numeric substitution; - renaming the FileCheckPatternSubstitution as a FileCheckSubstitution class with FileCheckStringSubstitution and FileCheckNumericSubstitution subclasses; - restricting the use of "numeric expression" to refer to the expression that is evaluated in a numeric substitution. While numeric substitution blocks only support numeric substitutions of numeric expressions at the moment there are plans to augment numeric substitution blocks to support numeric definitions as well as both a numeric definition and numeric substitution in the same numeric substitution block. Reviewers: jhenderson, jdenny, probinson, arichardson Subscribers: hiraditya, arichardson, probinson, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62146 llvm-svn: 361445
* Reinstate "FileCheck [5/12]: Introduce regular numeric variables"Thomas Preud'homme2019-05-147-30/+225
| | | | | | | | | | | | | | | | | | | | | | | This reinstates r360578 (git e47362c1ec1ea31b626336cc05822035601c3e57), reverted in r360653 (git 004393681c25e34e921adccc69ae6378090dee54), with a fix for the list added in FileCheck.rst to build without error. Copyright: - Linaro (changes up to diff 183612 of revision D55940) - GraphCore (changes in later versions of revision D55940 and in new revision created off D55940) Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield Tags: #llvm Differential Revision: https://reviews.llvm.org/D60385 llvm-svn: 360665
* Revert "FileCheck [5/12]: Introduce regular numeric variables"Thomas Preud'homme2019-05-147-225/+30
| | | | | | | | This reverts r360578 (git e47362c1ec1ea31b626336cc05822035601c3e57) to solve the sphinx build failure on http://lab.llvm.org:8011/builders/llvm-sphinx-docs buildbot. llvm-svn: 360653
* FileCheck [5/12]: Introduce regular numeric variablesThomas Preud'homme2019-05-137-30/+225
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch is part of a patch series to add support for FileCheck numeric expressions. This specific patch introduces regular numeric variables which can be set on the command-line. This commit introduces regular numeric variable that can be set on the command-line with the -D option to a numeric value. They can then be used in CHECK patterns in numeric expression with the same shape as @LINE numeric expression, ie. VAR, VAR+offset or VAR-offset where offset is an integer literal. The commit also enable strict whitespace in the verbose.txt testcase to check that the position or the location diagnostics. It fixes one of the existing CHECK in the process which was not accurately testing a location diagnostic (ie. the diagnostic was correct, not the CHECK). Copyright: - Linaro (changes up to diff 183612 of revision D55940) - GraphCore (changes in later versions of revision D55940 and in new revision created off D55940) Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield Tags: #llvm Differential Revision: https://reviews.llvm.org/D60385 llvm-svn: 360578
* [FileCheck, NFC] Split defines.txt in twoThomas Preud'homme2019-05-083-48/+78
| | | | | | | | | | | | | | | | | Summary: Split defines.txt into diagnostics test and functionality test. Also add comments, remove the semicolon prefix and group RUN lines with their CHECK directives. Reviewers: jhenderson, probinson, arichardson Subscribers: llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D61679 llvm-svn: 360289
* [CommandLine] Change help output to prefix long options with `--` instead of ↵Don Hinton2019-05-031-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | `-`. NFC . Part 3 of 5 Summary: By default, `parseCommandLineOptions()` will accept either a `-` or `--` prefix for long options -- options with names longer than a single character. While this change does not affect behavior, it will be helpful with a subsequent change that requires long options use the `--` prefix. Reviewers: rnk, thopre Reviewed By: thopre Subscribers: thopre, cfe-commits, hiraditya, llvm-commits Tags: #llvm, #clang Differential Revision: https://reviews.llvm.org/D61269 llvm-svn: 359909
* [FileCheck] Fix line-count.txt testThomas Preud'homme2019-05-021-2/+2
| | | | | | | | | | | | | | Summary: Enable currently skipped diagnostic test and fix column number Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk Subscribers: JonChesterfield, rogfer01, hfinkel, kristina, rnk, tra, arichardson, grimar, dblaikie, probinson, llvm-commits, hiraditya Tags: #llvm Differential Revision: https://reviews.llvm.org/D61320 llvm-svn: 359742
* FileCheck [4/12]: Introduce @LINE numeric expressionsThomas Preud'homme2019-05-021-12/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch is part of a patch series to add support for FileCheck numeric expressions. This specific patch introduces the @LINE numeric expressions. This commit introduces a new syntax to express a relation a numeric value in the input text must have with the line number of a given CHECK pattern: [[#<@LINE numeric expression>]]. Further commits build on that to express relations between several numeric values in the input text. To help with naming, regular variables are renamed into pattern variables and old @LINE expression syntax is referred to as legacy numeric expression. Compared to existing @LINE expressions, this new syntax allow arbitrary spacing between the component of the expression. It offers otherwise the same functionality but the commit serves to introduce some of the data structure needed to support more general numeric expressions. Copyright: - Linaro (changes up to diff 183612 of revision D55940) - GraphCore (changes in later versions of revision D55940 and in new revision created off D55940) Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield Tags: #llvm Differential Revision: https://reviews.llvm.org/D60384 llvm-svn: 359741
* FileCheck [3/12]: Stricter parsing of @LINE expressionsThomas Preud'homme2019-04-291-14/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch is part of a patch series to add support for FileCheck numeric expressions. This specific patch gives earlier and better diagnostics for the @LINE expressions. Rather than detect parsing errors at matching time, this commit adds enhance parsing to detect issues with @LINE expressions at parse time and diagnose them more accurately. Copyright: - Linaro (changes up to diff 183612 of revision D55940) - GraphCore (changes in later versions of revision D55940 and in new revision created off D55940) Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield Tags: #llvm Differential Revision: https://reviews.llvm.org/D60383 llvm-svn: 359475
* FileCheck [2/12]: Stricter parsing of -D optionThomas Preud'homme2019-04-291-1/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch is part of a patch series to add support for FileCheck numeric expressions. This specific patch gives earlier and better diagnostics for the -D option. Prior to this change, parsing of -D option was very loose: it assumed that there is an equal sign (which to be fair is now checked by the FileCheck executable) and that the part on the left of the equal sign was a valid variable name. This commit adds logic to ensure that this is the case and gives diagnostic when it is not, making it clear that the issue came from a command-line option error. This is achieved by sharing the variable parsing code into a new function ParseVariable. Copyright: - Linaro (changes up to diff 183612 of revision D55940) - GraphCore (changes in later versions of revision D55940 and in new revision created off D55940) Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield Tags: #llvm Differential Revision: https://reviews.llvm.org/D60382 llvm-svn: 359447
* [FileCheck]Remove assertions that prevent matching an empty string at file ↵James Henderson2019-03-121-0/+16
| | | | | | | | | | | | | | | | | | | | | | start before CHECK-NEXT/SAME This patch removes two assertions that were preventing writing of a test that checked an empty line followed by some text. For example: CHECK: {{^$}} CHECK-NEXT: foo() The assertion was because the current location the CHECK-NEXT was scanning from was the start of the buffer. A similar issue occurred with CHECK-SAME. These assertions don't protect against anything, as there is already an error check that checks that CHECK-NEXT/EMPTY/SAME don't appear first in the checks, and the following code works fine if the pointer is at the start of the input. Reviewed by: probinson, thopre, jdenny Differential Revision: https://reviews.llvm.org/D58784 llvm-svn: 355928
* Fixup test on Windows with a case-insensitive filesystem due to path ↵Douglas Yung2019-02-051-1/+1
| | | | | | printing changes from r352704. llvm-svn: 353238
* Recommit: Detect incorrect FileCheck variable CLI definitionThomas Preud'homme2019-02-051-5/+21
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: While the backend code of FileCheck relies on definition of variable from the command-line to have an equal sign '=' and a variable name before that, the frontend does not actually enforce it. This leads to FileCheck crashing when invoked with invalid syntax for the -D option. This patch adds the missing validation in the frontend. It also makes the -D option an AlwaysPrefix option to be able to detect -D=FOO as being a define without variable and -D as missing its value. Copyright: - Linaro (changes in version 2 of revision D55940) - GraphCore (changes in later versions) Reviewers: jdenny Subscribers: JonChesterfield, hiraditya, kristina, probinson, llvm-commits Differential Revision: https://reviews.llvm.org/D55940 llvm-svn: 353173
* Fixup test after r352704 since it changes how paths may be emitted.Douglas Yung2019-01-312-2/+2
| | | | | | | | On Unix/Mac OS X, normpath() returns the path unchanged (FileCheck), but on case-insensitive filesystems (like NTFS on Windows), it converts the path to lowercase (filecheck) which was causing the test to fail. llvm-svn: 352735
* Revert "Detect incorrect FileCheck variable CLI definition"Thomas Preud'homme2019-01-271-21/+5
| | | | | | This reverts commit r351039. llvm-svn: 352309
* Revert "Fix defines.txt"Thomas Preud'homme2019-01-271-1/+1
| | | | | | This reverts commit r351042. llvm-svn: 352308
* [FileCheck] Suppress old -v/-vv diags if dumping inputJoel E. Denny2019-01-222-80/+196
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The old diagnostic form of the trace produced by -v and -vv looks like: ``` check1:1:8: remark: CHECK: expected string found in input CHECK: abc ^ <stdin>:1:3: note: found here ; abc def ^~~ ``` When dumping annotated input is requested (via -dump-input), I find that this old trace is not useful and is sometimes harmful: 1. The old trace is mostly redundant because the same basic information also appears in the input dump's annotations. 2. The old trace buries any error diagnostic between it and the input dump, but I find it useful to see any error diagnostic up front. 3. FILECHECK_OPTS=-dump-input=fail requests annotated input dumps only for failed FileCheck calls. However, I have to also add -v or -vv to get a full set of annotations, and that can produce massive output from all FileCheck calls in all tests. That's a real problem when I run this in the IDE I use, which grinds to a halt as it tries to capture all that output. When -dump-input=fail|always, this patch suppresses the old trace from -v or -vv. Error diagnostics still print as usual. If you want the old trace, perhaps to see variable expansions, you can set -dump-input=none (the default). Reviewed By: probinson Differential Revision: https://reviews.llvm.org/D55825 llvm-svn: 351881
* Fix defines.txtThomas Preud'homme2019-01-141-1/+1
| | | | | | | Support arbitrary suffix when matching FileCheck executable name in defines.txt to successfully match FileCheck.EXE on Microsoft Windows. llvm-svn: 351042
* Detect incorrect FileCheck variable CLI definitionThomas Preud'homme2019-01-141-5/+21
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: While the backend code of FileCheck relies on definition of variable from the command-line to have an equal sign '=' and a variable name before that, the frontend does not actually enforce it. This leads to FileCheck crashing when invoked with invalid syntax for the -D option. This patch adds the missing validation in the frontend. It also makes the -D option an AlwaysPrefix option to be able to detect -D=FOO as being a define without variable and -D as missing its value. Copyright: - Linaro (changes in version 2 of revision D55940) - GraphCore (changes in later versions) Reviewers: jdenny Subscribers: JonChesterfield, hiraditya, kristina, probinson, llvm-commits Differential Revision: https://reviews.llvm.org/D55940 llvm-svn: 351039
* [FileCheck] Don't propagate `FILECHECK_DUMP_INPUT_ON_FAILURE` andDan Liew2019-01-101-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | `FILECHECK_OPTS` into environment for FileCheck tests. Summary: This fixes the following FileCheck tests: * FileCheck/dump-input-enable.txt * FileCheck/match-full-lines.txt when `FILECHECK_DUMP_INPUT_ON_FAILURE` is set in the environment. By default llvm-lit propagates `FILECHECK_DUMP_INPUT_ON_FAILURE` and `FILECHECK_OPTS` from llvm-lit's environment into the test environment. Unfortunately this can break FileCheck's tests because they expect that these environment variables not to be set. rdar://problem/47176262 Reviewers: jdenny, probinson, george.karpenkov Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D56541 llvm-svn: 350850
* [FileCheck] Try to fix test on windows due to r349418Joel E. Denny2018-12-181-1/+1
| | | | llvm-svn: 349432
* [FileCheck] Annotate input dump (final tweaks)Joel E. Denny2018-12-181-0/+7
| | | | | | | | | | | | | | | | | | | Apply final suggestions from probinson for this patch series plus a few more tweaks: * Improve various docs, for MatchType in particular. * Rename some members of MatchType. The main problem was that the term "final match" became a misnomer when CHECK-COUNT-<N> was created. * Split InputStartLine, etc. declarations into multiple lines. Differential Revision: https://reviews.llvm.org/D55738 Reviewed By: probinson llvm-svn: 349425
* [FileCheck] Annotate input dump (7/7)Joel E. Denny2018-12-181-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch implements annotations for diagnostics reporting CHECK-NOT failed matches. These diagnostics are enabled by -vv. As for diagnostics reporting failed matches for other directives, these annotations mark the search ranges using `X~~`. The difference here is that failed matches for CHECK-NOT are successes not errors, so they are green not red when colors are enabled. For example: ``` $ FileCheck -dump-input=help The following description was requested by -dump-input=help to explain the input annotations printed by -dump-input=always and -dump-input=fail: - L: labels line number L of the input file - T:L labels the only match result for a pattern of type T from line L of the check file - T:L'N labels the Nth match result for a pattern of type T from line L of the check file - ^~~ marks good match (reported if -v) - !~~ marks bad match, such as: - CHECK-NEXT on same line as previous match (error) - CHECK-NOT found (error) - CHECK-DAG overlapping match (discarded, reported if -vv) - X~~ marks search range when no match is found, such as: - CHECK-NEXT not found (error) - CHECK-NOT not found (success, reported if -vv) - CHECK-DAG not found after discarded matches (error) - ? marks fuzzy match when no match is found - colors success, error, fuzzy match, discarded match, unmatched input If you are not seeing color above or in input dumps, try: -color $ FileCheck -vv -dump-input=always check5 < input5 |& sed -n '/^<<<</,$p' <<<<<< 1: abcdef check:1 ^~~ not:2 X~~ 2: ghijkl not:2 ~~~ check:3 ^~~ 3: mnopqr not:4 X~~~~~ 4: stuvwx not:4 ~~~~~~ 5: eof:4 ^ >>>>>> $ cat check5 CHECK: abc CHECK-NOT: foobar CHECK: jkl CHECK-NOT: foobar $ cat input5 abcdef ghijkl mnopqr stuvwx ``` Reviewed By: george.karpenkov, probinson Differential Revision: https://reviews.llvm.org/D53899 llvm-svn: 349424
* [FileCheck] Annotate input dump (6/7)Joel E. Denny2018-12-181-4/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch implements input annotations for diagnostics reporting CHECK-DAG discarded matches. These diagnostics are enabled by -vv. These annotations mark discarded match ranges using `!~~` because they are bad matches even though they are not errors. CHECK-DAG discarded matches create another case where there can be multiple match results for the same directive. For example: ``` $ FileCheck -dump-input=help The following description was requested by -dump-input=help to explain the input annotations printed by -dump-input=always and -dump-input=fail: - L: labels line number L of the input file - T:L labels the only match result for a pattern of type T from line L of the check file - T:L'N labels the Nth match result for a pattern of type T from line L of the check file - ^~~ marks good match (reported if -v) - !~~ marks bad match, such as: - CHECK-NEXT on same line as previous match (error) - CHECK-NOT found (error) - CHECK-DAG overlapping match (discarded, reported if -vv) - X~~ marks search range when no match is found, such as: - CHECK-NEXT not found (error) - CHECK-DAG not found after discarded matches (error) - ? marks fuzzy match when no match is found - colors success, error, fuzzy match, discarded match, unmatched input If you are not seeing color above or in input dumps, try: -color $ FileCheck -vv -dump-input=always check4 < input4 |& sed -n '/^<<<</,$p' <<<<<< 1: abcdef dag:1 ^~~~ dag:2'0 !~~~ discard: overlaps earlier match 2: cdefgh dag:2'1 ^~~~ check:3 X~ error: no match found >>>>>> $ cat check4 CHECK-DAG: abcd CHECK-DAG: cdef CHECK: efgh $ cat input4 abcdef cdefgh ``` This shows that the line 3 CHECK fails to match even though its pattern appears in the input because its search range starts after the line 2 CHECK-DAG's match range. The trouble might be that the line 2 CHECK-DAG's match range is later than expected because its first match range overlaps with the line 1 CHECK-DAG match range and thus is discarded. Because `!~~` for CHECK-DAG does not indicate an error, it is not colored red. Instead, when colors are enabled, it is colored cyan, which suggests a match that went cold. Reviewed By: george.karpenkov, probinson Differential Revision: https://reviews.llvm.org/D53898 llvm-svn: 349423
* [FileCheck] Annotate input dump (5/7)Joel E. Denny2018-12-182-25/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch implements input annotations for diagnostics enabled by -v, which report good matches for directives. These annotations mark match ranges using `^~~`. For example: ``` $ FileCheck -dump-input=help The following description was requested by -dump-input=help to explain the input annotations printed by -dump-input=always and -dump-input=fail: - L: labels line number L of the input file - T:L labels the only match result for a pattern of type T from line L of the check file - T:L'N labels the Nth match result for a pattern of type T from line L of the check file - ^~~ marks good match (reported if -v) - !~~ marks bad match, such as: - CHECK-NEXT on same line as previous match (error) - CHECK-NOT found (error) - X~~ marks search range when no match is found, such as: - CHECK-NEXT not found (error) - ? marks fuzzy match when no match is found - colors success, error, fuzzy match, unmatched input If you are not seeing color above or in input dumps, try: -color $ FileCheck -v -dump-input=always check3 < input3 |& sed -n '/^<<<</,$p' <<<<<< 1: abc foobar def check:1 ^~~ not:2 !~~~~~ error: no match expected check:3 ^~~ >>>>>> $ cat check3 CHECK: abc CHECK-NOT: foobar CHECK: def $ cat input3 abc foobar def ``` -vv enables these annotations for FileCheck's implicit EOF patterns as well. For an example where EOF patterns become relevant, see patch 7 in this series. If colors are enabled, `^~~` is green to suggest success. -v plus color enables highlighting of input text that has no final match for any expected pattern. The highlight uses a cyan background to suggest a cold section. This highlighting can make it easier to spot text that was intended to be matched but that failed to be matched in a long series of good matches. CHECK-COUNT-<num> good matches are another case where there can be multiple match results for the same directive. Reviewed By: george.karpenkov, probinson Differential Revision: https://reviews.llvm.org/D53897 llvm-svn: 349422
* [FileCheck] Annotate input dump (4/7)Joel E. Denny2018-12-181-0/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch implements input annotations for diagnostics that report unexpected matches for CHECK-NOT. Like wrong-line matches for CHECK-NEXT, CHECK-SAME, and CHECK-EMPTY, these annotations mark match ranges using red `!~~` to indicate bad matches that are errors. For example: ``` $ FileCheck -dump-input=help The following description was requested by -dump-input=help to explain the input annotations printed by -dump-input=always and -dump-input=fail: - L: labels line number L of the input file - T:L labels the only match result for a pattern of type T from line L of the check file - T:L'N labels the Nth match result for a pattern of type T from line L of the check file - !~~ marks bad match, such as: - CHECK-NEXT on same line as previous match (error) - CHECK-NOT found (error) - X~~ marks search range when no match is found, such as: - CHECK-NEXT not found (error) - ? marks fuzzy match when no match is found - colors error, fuzzy match If you are not seeing color above or in input dumps, try: -color $ FileCheck -v -dump-input=always check3 < input3 |& sed -n '/^<<<</,$p' <<<<<< 1: abc foobar def not:2 !~~~~~ error: no match expected >>>>>> $ cat check3 CHECK: abc CHECK-NOT: foobar CHECK: def $ cat input3 abc foobar def ``` Reviewed By: george.karpenkov, probinson Differential Revision: https://reviews.llvm.org/D53896 llvm-svn: 349421
* [FileCheck] Annotate input dump (3/7)Joel E. Denny2018-12-181-3/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch implements input annotations for diagnostics that report wrong-line matches for the directives CHECK-NEXT, CHECK-SAME, and CHECK-EMPTY. Instead of the usual `^~~`, which is used by later patches for good matches, these annotations use `!~~` to mark the bad match ranges so that this category of errors is visually distinct. Because such matches are errors, these annotates are red when colors are enabled. For example: ``` $ FileCheck -dump-input=help The following description was requested by -dump-input=help to explain the input annotations printed by -dump-input=always and -dump-input=fail: - L: labels line number L of the input file - T:L labels the only match result for a pattern of type T from line L of the check file - T:L'N labels the Nth match result for a pattern of type T from line L of the check file - !~~ marks bad match, such as: - CHECK-NEXT on same line as previous match (error) - X~~ marks search range when no match is found, such as: - CHECK-NEXT not found (error) - ? marks fuzzy match when no match is found - colors error, fuzzy match If you are not seeing color above or in input dumps, try: -color $ FileCheck -v -dump-input=always check2 < input2 |& sed -n '/^<<<</,$p' <<<<<< 1: foo bar next:2 !~~ error: match on wrong line >>>>>> $ cat check2 CHECK: foo CHECK-NEXT: bar $ cat input2 foo bar ``` Reviewed By: george.karpenkov, probinson Differential Revision: https://reviews.llvm.org/D53894 llvm-svn: 349420
* [FileCheck] Annotate input dump (2/7)Joel E. Denny2018-12-181-20/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch implements input annotations for diagnostics that suggest fuzzy matches for directives for which no matches were found. Instead of using the usual `^~~`, which is used by later patches for good matches, these annotations use `?` so that fuzzy matches are visually distinct. No tildes are included as these diagnostics (independently of this patch) currently identify only the start of the match. For example: ``` $ FileCheck -dump-input=help The following description was requested by -dump-input=help to explain the input annotations printed by -dump-input=always and -dump-input=fail: - L: labels line number L of the input file - T:L labels the only match result for a pattern of type T from line L of the check file - T:L'N labels the Nth match result for a pattern of type T from line L of the check file - X~~ marks search range when no match is found - ? marks fuzzy match when no match is found - colors error, fuzzy match If you are not seeing color above or in input dumps, try: -color $ FileCheck -v -dump-input=always check1 < input1 |& sed -n '/^<<<</,$p' <<<<<< 1: ; abc def 2: ; ghI jkl next:3'0 X~~~~~~~~ error: no match found next:3'1 ? possible intended match >>>>>> $ cat check1 CHECK: abc CHECK-SAME: def CHECK-NEXT: ghi CHECK-SAME: jkl $ cat input1 ; abc def ; ghI jkl ``` This patch introduces the concept of multiple "match results" per directive. In the above example, the first match result for the CHECK-NEXT directive is the failed match, for which the annotation shows the search range. The second match result is the fuzzy match. Later patches will introduce other cases of multiple match results per directive. When colors are enabled, `?` is colored magenta. That is, it doesn't indicate the actual error, which a red `X~~` marker indicates, but its color suggests it's closely related. Reviewed By: george.karpenkov, probinson Differential Revision: https://reviews.llvm.org/D53893 llvm-svn: 349419
OpenPOWER on IntegriCloud