[lit] Report line number for failed RUN command

(Relands r333584, reverted in 333592.)

When debugging test failures with -vv (or -v in the case of the
internal shell), this makes it easier to locate the RUN line that
failed.  For example, clang's test/Driver/linux-ld.c has 892 total RUN
lines, and clang's test/Driver/arm-cortex-cpus.c has 424 RUN lines
after concatenation for line continuations.

When reading the generated shell script, this also makes it easier to
locate the RUN line that produced each command.

To support reporting RUN line numbers in the case of the internal
shell, this patch extends the internal shell to support the null
command, ":", except pipelines are not supported.

To support reporting RUN line numbers in the case of windows cmd.exe
as the external shell, this patch extends -vv to set "echo on" instead
of "echo off" in bat files.  (Support for windows cmd.exe as a lit
external shell will likely be dropped later, but I found out too
late.)

Reviewed By: delcypher,	asmith, stella.stamenova, jmorse, lebedev.ri, rnk

Differential Revision: https://reviews.llvm.org/D44598

llvm-svn: 333614

1 files changed, 70 insertions, 0 deletions

diff --git a/llvm/utils/lit/tests/shtest-run-at-line.py b/llvm/utils/lit/tests/shtest-run-at-line.py
new file mode 100644
index 00000000000..adb7af81812
--- /dev/null
+++ b/llvm/utils/lit/tests/shtest-run-at-line.py
@@ -0,0 +1,70 @@
+# Check that -vv makes the line number of the failing RUN command clear.
+# (-v is actually sufficient in the case of the internal shell.)
+#
+# RUN: not %{lit} -j 1 -vv %{inputs}/shtest-run-at-line > %t.out
+# RUN: FileCheck --input-file %t.out -Dpdbg0="%{pdbg0}" -Dpdbg1="%{pdbg1}" %s
+#
+# END.
+
+
+# CHECK: Testing: 4 tests
+
+
+# In the case of the external shell, we check for only RUN lines in stderr in
+# case some shell implementations format "set -x" output differently.
+
+# CHECK-LABEL: FAIL: shtest-run-at-line :: external-shell/basic.txt
+
+# CHECK:      Script:
+# CHECK:      [[pdbg0]]RUN: at line 4[[pdbg1]] &&  true
+# CHECK-NEXT: [[pdbg0]]RUN: at line 5[[pdbg1]] &&  false
+# CHECK-NEXT: [[pdbg0]]RUN: at line 6[[pdbg1]] &&  true
+
+# CHECK:     RUN: at line 4
+# CHECK:     RUN: at line 5
+# CHECK-NOT: RUN
+
+# CHECK-LABEL: FAIL: shtest-run-at-line :: external-shell/line-continuation.txt
+
+# CHECK:      Script:
+# CHECK:      [[pdbg0]]RUN: at line 4[[pdbg1]] &&  echo 'foo bar'  | FileCheck
+# CHECK-NEXT: [[pdbg0]]RUN: at line 6[[pdbg1]] &&  echo  'foo baz'  | FileCheck
+# CHECK-NEXT: [[pdbg0]]RUN: at line 9[[pdbg1]] &&  echo 'foo bar'  | FileCheck
+
+# CHECK:     RUN: at line 4
+# CHECK:     RUN: at line 6
+# CHECK-NOT: RUN
+
+
+# CHECK-LABEL: FAIL: shtest-run-at-line :: internal-shell/basic.txt
+
+# CHECK:      Script:
+# CHECK:      : 'RUN: at line 1' &&  true
+# CHECK-NEXT: : 'RUN: at line 2' &&  false
+# CHECK-NEXT: : 'RUN: at line 3' &&  true
+
+# CHECK:      Command Output (stdout)
+# CHECK:      $ ":" "RUN: at line 1"
+# CHECK-NEXT: $ "true"
+# CHECK-NEXT: $ ":" "RUN: at line 2"
+# CHECK-NEXT: $ "false"
+# CHECK-NOT:  RUN
+
+# CHECK-LABEL: FAIL: shtest-run-at-line :: internal-shell/line-continuation.txt
+
+# CHECK:      Script:
+# CHECK:      : 'RUN: at line 1' &&  : first line continued to second line
+# CHECK-NEXT: : 'RUN: at line 3' &&  echo 'foo bar'  | FileCheck
+# CHECK-NEXT: : 'RUN: at line 5' &&  echo  'foo baz'  | FileCheck
+# CHECK-NEXT: : 'RUN: at line 8' &&  echo 'foo bar'  | FileCheck
+
+# CHECK:      Command Output (stdout)
+# CHECK:      $ ":" "RUN: at line 1"
+# CHECK-NEXT: $ ":" "first" "line" "continued" "to" "second" "line"
+# CHECK-NEXT: $ ":" "RUN: at line 3"
+# CHECK-NEXT: $ "echo" "foo bar"
+# CHECK-NEXT: $ "FileCheck" "{{.*}}"
+# CHECK-NEXT: $ ":" "RUN: at line 5"
+# CHECK-NEXT: $ "echo" "foo baz"
+# CHECK-NEXT: $ "FileCheck" "{{.*}}"
+# CHECK-NOT:  RUN