[lit] Support custom parsers in parseIntegratedTestScript

Summary:
Libc++ frequently has the need to parse more than just the builtin *test keywords* (`RUN`, `REQUIRES`, `XFAIL`, ect). For example libc++ currently needs a new keyword `MODULES-DEFINES: macro list...`. Instead of re-implementing the script parsing in libc++ this patch allows `parseIntegratedTestScript` to take custom parsers.

This patch introduces a new class `IntegratedTestKeywordParser` which implements the logic to parse/process a test keyword. Parsing of various keyword "kinds" are supported out of the box, including 'TAG', 'COMMAND', and 'LIST', which parse keywords such as `END.`, `RUN:` and `XFAIL:` respectively.

As an example after this change libc++ can implement the `MODULES-DEFINES` simply using: 
```
mparser = IntegratedTestKeywordParser('MODULES-DEFINES:', ParserKind.LIST)
parseIntegratedTestScript(test, additional_parsers=[mparser])
macro_list = mparser.getValue()
```


Reviewers: ddunbar, modocache, rnk, danalbert, jroelofs

Subscribers: mgrang, llvm-commits, cfe-commits

Differential Revision: https://reviews.llvm.org/D27005

llvm-svn: 288694

2 files changed, 27 insertions, 0 deletions

diff --git a/llvm/utils/lit/tests/Inputs/testrunner-custom-parsers/lit.cfg b/llvm/utils/lit/tests/Inputs/testrunner-custom-parsers/lit.cfg
new file mode 100644
index 00000000000..cf46c1674e5
--- /dev/null
+++ b/llvm/utils/lit/tests/Inputs/testrunner-custom-parsers/lit.cfg
@@ -0,0 +1,14 @@
+import lit.formats
+import os
+import lit.Test
+
+class TestParserFormat(lit.formats.FileBasedTest):
+  def execute(self, test, lit_config):
+      return lit.Test.PASS, ''
+
+config.name = 'custom-parsers'
+config.suffixes = ['.txt']
+config.test_format = TestParserFormat()
+config.test_source_root = None
+config.test_exec_root = None
+config.target_triple = 'x86_64-unknown-unknown'
diff --git a/llvm/utils/lit/tests/Inputs/testrunner-custom-parsers/test.txt b/llvm/utils/lit/tests/Inputs/testrunner-custom-parsers/test.txt
new file mode 100644
index 00000000000..ed118f382fe
--- /dev/null
+++ b/llvm/utils/lit/tests/Inputs/testrunner-custom-parsers/test.txt
@@ -0,0 +1,13 @@
+
+// MY_TAG.
+// foo bar baz
+// MY_RUN: baz
+// MY_LIST: one, two
+// MY_LIST: three, four
+// MY_RUN: foo \
+// MY_RUN: bar
+//
+// MY_CUSTOM: a b c
+//
+// END.
+// MY_LIST: five