[clang-tidy] check_clang_tidy.py: support CHECK-NOTES prefix

Summary:
Currently, there is two configured prefixes: `CHECK-FIXES` and `CHECK-MESSAGES`
`CHECK-MESSAGES` checks that there are no test output lines with `warning:|error:`, which are not explicitly handled in lit tests.
However there does not seem to be a nice way to enforce for all the `note:` to be checked.
This was useful for me when developing D36836.

Reviewers: alexfh, klimek, aaron.ballman, hokein

Reviewed By: alexfh, aaron.ballman

Subscribers: JonasToth, JDevlieghere, xazax.hun, cfe-commits

Tags: #clang-tools-extra

Differential Revision: https://reviews.llvm.org/D36892

llvm-svn: 339437

1 files changed, 18 insertions, 2 deletions

diff --git a/clang-tools-extra/test/clang-tidy/check_clang_tidy.py b/clang-tools-extra/test/clang-tidy/check_clang_tidy.py
index bb2c5a26052..dadb84d5790 100755
--- a/clang-tools-extra/test/clang-tidy/check_clang_tidy.py
+++ b/clang-tools-extra/test/clang-tidy/check_clang_tidy.py
@@ -78,6 +78,7 @@ def main():
   file_check_suffix = ('-' + args.check_suffix) if args.check_suffix else ''
   check_fixes_prefix = 'CHECK-FIXES' + file_check_suffix
   check_messages_prefix = 'CHECK-MESSAGES' + file_check_suffix
+  check_notes_prefix = 'CHECK-NOTES' + file_check_suffix
 
   # Tests should not rely on STL being available, and instead provide mock
   # implementations of relevant APIs.
@@ -91,9 +92,11 @@ def main():
 
   has_check_fixes = check_fixes_prefix in input_text
   has_check_messages = check_messages_prefix in input_text
+  has_check_notes = check_notes_prefix in input_text
 
-  if not has_check_fixes and not has_check_messages:
-    sys.exit('Neither %s nor %s found in the input' % (check_fixes_prefix, check_messages_prefix) )
+  if not has_check_fixes and not has_check_messages and not has_check_notes:
+    sys.exit('%s, %s or %s not found in the input' % (check_fixes_prefix,
+             check_messages_prefix, check_notes_prefix) )
 
   # Remove the contents of the CHECK lines to avoid CHECKs matching on
   # themselves.  We need to keep the comments to preserve line numbers while
@@ -156,5 +159,18 @@ def main():
       print('FileCheck failed:\n' + e.output.decode())
       raise
 
+  if has_check_notes:
+    notes_file = temp_file_name + '.notes'
+    write_file(notes_file, clang_tidy_output)
+    try:
+      subprocess.check_output(
+          ['FileCheck', '-input-file=' + notes_file, input_file_name,
+           '-check-prefix=' + check_notes_prefix,
+           '-implicit-check-not={{note|warning|error}}:'],
+          stderr=subprocess.STDOUT)
+    except subprocess.CalledProcessError as e:
+      print('FileCheck failed:\n' + e.output.decode())
+      raise
+
 if __name__ == '__main__':
   main()