Add support for retrieving the Doxygen comment associated with a given

declaration in the AST. 

The new ASTContext::getCommentForDecl function searches for a comment
that is attached to the given declaration, and returns that comment, 
which may be composed of several comment blocks.

Comments are always available in an AST. However, to avoid harming
performance, we don't actually parse the comments. Rather, we keep the
source ranges of all of the comments within a large, sorted vector,
then lazily extract comments via a binary search in that vector only
when needed (which never occurs in a "normal" compile).

Comments are written to a precompiled header/AST file as a blob of
source ranges. That blob is only lazily loaded when one requests a
comment for a declaration (this never occurs in a "normal" compile). 

The indexer testbed now supports comment extraction. When the
-point-at location points to a declaration with a Doxygen-style
comment, the indexer testbed prints the associated comment
block(s). See test/Index/comments.c for an example.

Some notes:
  - We don't actually attempt to parse the comment blocks themselves,
  beyond identifying them as Doxygen comment blocks to associate them
  with a declaration.
  - We won't find comment blocks that aren't adjacent to the
  declaration, because we start our search based on the location of
  the declaration.
  - We don't go through the necessary hops to find, for example,
  whether some redeclaration of a declaration has comments when our
  current declaration does not. Similarly, we don't attempt to
  associate a \param Foo marker in a function body comment with the
  parameter named Foo (although that is certainly possible).
  - Verification of my "no performance impact" claims is still "to be
  done".

llvm-svn: 74704

3 files changed, 32 insertions, 1 deletions

diff --git a/clang/lib/Lex/Lexer.cpp b/clang/lib/Lex/Lexer.cpp
index 81ea606f0d9..6f1043ae735 100644
--- a/clang/lib/Lex/Lexer.cpp
+++ b/clang/lib/Lex/Lexer.cpp
@@ -903,7 +903,10 @@ bool Lexer::SkipBCPLComment(Token &Result, const char *CurPtr) {
   } while (C != '\n' && C != '\r');
 
   // Found but did not consume the newline.
-    
+  if (PP)
+    PP->HandleComment(SourceRange(getSourceLocation(BufferPtr), 
+                                  getSourceLocation(CurPtr)));
+                   
   // If we are returning comments as tokens, return this comment as a token.
   if (inKeepCommentMode())
     return SaveBCPLComment(Result, CurPtr);
@@ -1146,6 +1149,10 @@ bool Lexer::SkipBlockComment(Token &Result, const char *CurPtr) {
     C = *CurPtr++;
   }
   
+  if (PP) 
+    PP->HandleComment(SourceRange(getSourceLocation(BufferPtr), 
+                                  getSourceLocation(CurPtr)));
+
   // If we are returning comments as tokens, return this comment as a token.
   if (inKeepCommentMode()) {
     FormTokenWithChars(Result, CurPtr, tok::comment);
diff --git a/clang/lib/Lex/Pragma.cpp b/clang/lib/Lex/Pragma.cpp
index 3227d1cf09c..bb0b71e2268 100644
--- a/clang/lib/Lex/Pragma.cpp
+++ b/clang/lib/Lex/Pragma.cpp
@@ -19,6 +19,7 @@
 #include "clang/Lex/LexDiagnostic.h"
 #include "clang/Basic/FileManager.h"
 #include "clang/Basic/SourceManager.h"
+#include <algorithm>
 using namespace clang;
 
 // Out-of-line destructor to provide a home for the class.
diff --git a/clang/lib/Lex/Preprocessor.cpp b/clang/lib/Lex/Preprocessor.cpp
index 24ee5d8a8f9..9f0c15f59e4 100644
--- a/clang/lib/Lex/Preprocessor.cpp
+++ b/clang/lib/Lex/Preprocessor.cpp
@@ -476,3 +476,26 @@ void Preprocessor::HandleIdentifier(Token &Identifier) {
   if (II.isExtensionToken() && !DisableMacroExpansion)
     Diag(Identifier, diag::ext_token_used);
 }
+
+void Preprocessor::AddCommentHandler(CommentHandler *Handler) {
+  assert(Handler && "NULL comment handler");
+  assert(std::find(CommentHandlers.begin(), CommentHandlers.end(), Handler) ==
+         CommentHandlers.end() && "Comment handler already registered");
+  CommentHandlers.push_back(Handler);
+}
+
+void Preprocessor::RemoveCommentHandler(CommentHandler *Handler) {
+  std::vector<CommentHandler *>::iterator Pos
+  = std::find(CommentHandlers.begin(), CommentHandlers.end(), Handler);
+  assert(Pos != CommentHandlers.end() && "Comment handler not registered");
+  CommentHandlers.erase(Pos);
+}
+
+void Preprocessor::HandleComment(SourceRange Comment) {
+  for (std::vector<CommentHandler *>::iterator H = CommentHandlers.begin(),
+       HEnd = CommentHandlers.end();
+       H != HEnd; ++H)
+    (*H)->HandleComment(*this, Comment);
+}
+
+CommentHandler::~CommentHandler() { }