Add `-verify-ignore-unexpected` option to ignore unexpected diagnostics in VerifyDiagnosticsConsumer

Summary:
The goal of this patch is to make `-verify` easier to use when testing libc++. The `notes` attached to compile error diagnostics are numerous and relatively unstable when they reference libc++ header internals. This patch allows libc++ to write stable compilation failure tests by allowing unexpected diagnostic messages to be ignored where they are not relevant.

This patch adds a new CC1 flag called `-verify-ignore-unexpected`. `-verify-ignore-unexpected` tells `VerifyDiagnosticsConsumer` to ignore *all* unexpected diagnostic messages. `-verify-ignore-unexpected=<LevelList>` can be used to only ignore certain diagnostic levels. `<LevelList>` is a comma separated list of diagnostic levels to ignore. The supported levels are `note`, `remark`, `warning` and `error`.



Reviewers: bogner, grosser, EricWF

Reviewed By: EricWF

Subscribers: cfe-commits

Differential Revision: http://reviews.llvm.org/D10138

llvm-svn: 239665

1 files changed, 81 insertions, 0 deletions

diff --git a/clang/test/Frontend/verify-ignore-unexpected.c b/clang/test/Frontend/verify-ignore-unexpected.c
new file mode 100644
index 00000000000..bc3e0d16e68
--- /dev/null
+++ b/clang/test/Frontend/verify-ignore-unexpected.c
@@ -0,0 +1,81 @@
+// RUN: not %clang_cc1 -DTEST_SWITCH -verify-ignore-unexpected=remark,aoeu,note -verify %s 2>&1 \
+// RUN:     | FileCheck -check-prefix=CHECK-BAD-SWITCH %s
+#ifdef TEST_SWITCH
+// expected-no-diagnostics
+#endif
+// CHECK-BAD-SWITCH: error: 'error' diagnostics seen but not expected:
+// CHECK-BAD-SWITCH-NEXT: (frontend): invalid value 'aoeu' in '-verify-ignore-unexpected='
+
+// RUN: %clang_cc1 -DTEST1 -verify %s
+// RUN: %clang_cc1 -DTEST1 -verify -verify-ignore-unexpected %s
+#ifdef TEST1
+#warning MyWarning1
+    // expected-warning@-1 {{MyWarning1}}
+int x; // expected-note {{previous definition is here}}
+float x; // expected-error {{redefinition of 'x'}}
+#endif
+
+// RUN: not %clang_cc1 -DTEST2 -verify %s 2>&1 \
+// RUN:     | FileCheck -check-prefix=CHECK-UNEXP %s
+// RUN: not %clang_cc1 -DTEST2 -verify -verify-ignore-unexpected= %s 2>&1 \
+// RUN:     | FileCheck -check-prefix=CHECK-UNEXP %s
+// RUN: not %clang_cc1 -DTEST2 -verify -verify-ignore-unexpected=note %s 2>&1 \
+// RUN:     | FileCheck -check-prefix=CHECK-NOTE %s
+// RUN: not %clang_cc1 -DTEST2 -verify -verify-ignore-unexpected=warning %s 2>&1 \
+// RUN:     | FileCheck -check-prefix=CHECK-WARN %s
+// RUN: not %clang_cc1 -DTEST2 -verify -verify-ignore-unexpected=error %s 2>&1 \
+// RUN:     | FileCheck -check-prefix=CHECK-ERR %s
+#ifdef TEST2
+#warning MyWarning2
+int x;
+float x;
+#endif
+// CHECK-UNEXP: no expected directives found
+// CHECK-UNEXP-NEXT: 'error' diagnostics seen but not expected
+// CHECK-UNEXP-NEXT: Line {{[0-9]+}}: redefinition of 'x'
+// CHECK-UNEXP-NEXT: 'warning' diagnostics seen but not expected
+// CHECK-UNEXP-NEXT: Line {{[0-9]+}}: MyWarning2
+// CHECK-UNEXP-NEXT: 'note' diagnostics seen but not expected
+// CHECK-UNEXP-NEXT: Line {{[0-9]+}}: previous definition is here
+// CHECK-UNEXP-NEXT: 4 errors generated.
+
+// CHECK-NOTE: no expected directives found
+// CHECK-NOTE-NEXT: 'error' diagnostics seen but not expected
+// CHECK-NOTE-NEXT: Line {{[0-9]+}}: redefinition of 'x'
+// CHECK-NOTE-NEXT: 'warning' diagnostics seen but not expected
+// CHECK-NOTE-NEXT: Line {{[0-9]+}}: MyWarning2
+// CHECK-NOTE-NEXT: 3 errors generated.
+
+// CHECK-WARN: no expected directives found
+// CHECK-WARN-NEXT: 'error' diagnostics seen but not expected
+// CHECK-WARN-NEXT: Line {{[0-9]+}}: redefinition of 'x'
+// CHECK-WARN-NEXT: 'note' diagnostics seen but not expected
+// CHECK-WARN-NEXT: Line {{[0-9]+}}: previous definition is here
+// CHECK-WARN-NEXT: 3 errors generated.
+
+// CHECK-ERR: no expected directives found
+// CHECK-ERR-NEXT: 'warning' diagnostics seen but not expected
+// CHECK-ERR-NEXT: Line {{[0-9]+}}: MyWarning2
+// CHECK-ERR-NEXT: 'note' diagnostics seen but not expected
+// CHECK-ERR-NEXT: Line {{[0-9]+}}: previous definition is here
+// CHECK-ERR-NEXT: 3 errors generated.
+
+// RUN: not %clang_cc1 -DTEST3 -verify -verify-ignore-unexpected %s 2>&1 \
+// RUN:     | FileCheck -check-prefix=CHECK-EXP %s
+#ifdef TEST3
+// expected-error {{test3}}
+#endif
+// CHECK-EXP: 'error' diagnostics expected but not seen
+// CHECK-EXP-NEXT: Line {{[0-9]+}}: test3
+
+// RUN: not %clang_cc1 -DTEST4 -verify -verify-ignore-unexpected %s 2>&1 \
+// RUN:     | FileCheck -check-prefix=CHECK-NOEXP %s
+// RUN: not %clang_cc1 -DTEST4 -verify -verify-ignore-unexpected=warning,error,note %s 2>&1 \
+// RUN:     | FileCheck -check-prefix=CHECK-NOEXP %s
+#ifdef TEST4
+#warning MyWarning4
+int x;
+float x;
+#endif
+// CHECK-NOEXP: error: no expected directives found
+// CHECK-NOEXP-NEXT: 1 error generated