[Syntax] Do not glue multiple empty PP expansions to a single mapping

Summary:
This change makes sure we have a single mapping for each macro expansion,
even if the result of expansion was empty.

To achieve that, we take information from PPCallbacks::MacroExpands into
account. Previously we relied only on source locations of expanded tokens.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D62953

llvm-svn: 364236

1 files changed, 4 insertions, 1 deletions

diff --git a/clang/unittests/Tooling/Syntax/TokensTest.cpp b/clang/unittests/Tooling/Syntax/TokensTest.cpp
index a812b844c85..24adba28a29 100644
--- a/clang/unittests/Tooling/Syntax/TokensTest.cpp
+++ b/clang/unittests/Tooling/Syntax/TokensTest.cpp
@@ -424,6 +424,7 @@ file './input.cpp'
        "    ['#'_0, 'int'_22) => ['int'_0, 'int'_0)\n"
        "    ['ADD'_25, ';'_46) => ['1'_3, ';'_12)\n"},
       // Empty macro replacement.
+      // FIXME: the #define directives should not be glued together.
       {R"cpp(
     #define EMPTY
     #define EMPTY_FUNC(X)
@@ -436,7 +437,9 @@ file './input.cpp'
   spelled tokens:
     # define EMPTY # define EMPTY_FUNC ( X ) EMPTY EMPTY_FUNC ( 1 + 2 + 3 )
   mappings:
-    ['#'_0, '<eof>'_18) => ['<eof>'_0, '<eof>'_0)
+    ['#'_0, 'EMPTY'_9) => ['<eof>'_0, '<eof>'_0)
+    ['EMPTY'_9, 'EMPTY_FUNC'_10) => ['<eof>'_0, '<eof>'_0)
+    ['EMPTY_FUNC'_10, '<eof>'_18) => ['<eof>'_0, '<eof>'_0)
 )"},
       // File ends with a macro replacement.
       {R"cpp(