diff options
author | John Thompson <John.Thompson.JTSoftware@gmail.com> | 2015-02-19 16:47:27 +0000 |
---|---|---|
committer | John Thompson <John.Thompson.JTSoftware@gmail.com> | 2015-02-19 16:47:27 +0000 |
commit | 8eb8d9367291c65b7f508bbc1555e7606f01b554 (patch) | |
tree | 3539d8a785f824c5a6dd57b203cac84bead6dbcc | |
parent | be5680f98553854eb637d5113374cd20772ab342 (diff) | |
download | bcm5719-llvm-8eb8d9367291c65b7f508bbc1555e7606f01b554.tar.gz bcm5719-llvm-8eb8d9367291c65b7f508bbc1555e7606f01b554.zip |
Added module map coverage support, extracted from module-map-checker.
llvm-svn: 229869
27 files changed, 852 insertions, 12 deletions
diff --git a/clang-tools-extra/docs/ModularizeUsage.rst b/clang-tools-extra/docs/ModularizeUsage.rst index c93a72b3a22..b43389e822c 100644 --- a/clang-tools-extra/docs/ModularizeUsage.rst +++ b/clang-tools-extra/docs/ModularizeUsage.rst @@ -41,6 +41,10 @@ Note that by default, the underlying Clang front end assumes .h files contain C source, so you might need to specify the ``-x c++`` Clang option to tell Clang that the header contains C++ definitions. +Note also that because modularize does not use the clang driver, +you will likely need to pass in additional compiler front-end +arguments to match those passed in by default by the driver. + Modularize Command Line Options =============================== @@ -66,3 +70,11 @@ Modularize Command Line Options check to only those headers explicitly listed in the header list. This is a work-around for avoiding error messages for private includes that purposefully get included inside blocks. + +.. option:: -no-coverage-check + + Don't do the coverage check for a module map. + +.. option:: -coverage-check-only + + Only do the coverage check for a module map. diff --git a/clang-tools-extra/docs/modularize.rst b/clang-tools-extra/docs/modularize.rst index 6e964dd337b..2612461aa1a 100644 --- a/clang-tools-extra/docs/modularize.rst +++ b/clang-tools-extra/docs/modularize.rst @@ -56,6 +56,8 @@ Modularize will check for the following: * Macro instances, 'defined(macro)', or #if, #elif, #ifdef, #ifndef conditions that evaluate differently in a header * #include directives inside 'extern "C/C++" {}' or 'namespace (name) {}' blocks +* Module map header coverage completeness (in the case of a module map input + only) Modularize will do normal C/C++ parsing, reporting normal errors and warnings, but will also report special error messages like the following:: @@ -107,6 +109,44 @@ and can produce error message like the following:: ^ The "extern "C" {}" block is here. +.. _module-map-coverage: + +Module Map Coverage Check +========================= + +The coverage check uses the Clang library to read and parse the +module map file. Starting at the module map file directory, or just the +include paths, if specified, it will collect the names of all the files it +considers headers (no extension, .h, or .inc--if you need more, modify the +isHeader function). It then compares the headers against those referenced +in the module map, either explicitly named, or implicitly named via an +umbrella directory or umbrella file, as parsed by the ModuleMap object. +If headers are found which are not referenced or covered by an umbrella +directory or file, warning messages will be produced, and this program +will return an error code of 1. If no problems are found, an error code of +0 is returned. + +Note that in the case of umbrella headers, this tool invokes the compiler +to preprocess the file, and uses a callback to collect the header files +included by the umbrella header or any of its nested includes. If any +front end options are needed for these compiler invocations, these +can be included on the command line after the module map file argument. + +Warning message have the form: + + warning: module.modulemap does not account for file: Level3A.h + +Note that for the case of the module map referencing a file that does +not exist, the module map parser in Clang will (at the time of this +writing) display an error message. + +To limit the checks :program:`modularize` does to just the module +map coverage check, use the ``-coverage-check-only option``. + +For example:: + + modularize -coverage-check-only module.modulemap + .. _module-map-generation: Module Map Generation diff --git a/clang-tools-extra/modularize/CMakeLists.txt b/clang-tools-extra/modularize/CMakeLists.txt index 64994fcd70f..7de809ce640 100644 --- a/clang-tools-extra/modularize/CMakeLists.txt +++ b/clang-tools-extra/modularize/CMakeLists.txt @@ -7,6 +7,7 @@ add_clang_executable(modularize Modularize.cpp ModuleAssistant.cpp ModularizeUtilities.cpp + CoverageChecker.cpp PreprocessorTracker.cpp ) diff --git a/clang-tools-extra/modularize/CoverageChecker.cpp b/clang-tools-extra/modularize/CoverageChecker.cpp new file mode 100644 index 00000000000..17ea10712b5 --- /dev/null +++ b/clang-tools-extra/modularize/CoverageChecker.cpp @@ -0,0 +1,415 @@ +//===--- extra/module-map-checker/CoverageChecker.cpp -------------------===// +// +// The LLVM Compiler Infrastructure +// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +//===----------------------------------------------------------------------===// +// +// This file implements a class that validates a module map by checking that +// all headers in the corresponding directories are accounted for. +// +// This class uses a previously loaded module map object. +// Starting at the module map file directory, or just the include +// paths, if specified, it will collect the names of all the files it +// considers headers (no extension, .h, or .inc--if you need more, modify the +// ModularizeUtilities::isHeader function). +// It then compares the headers against those referenced +// in the module map, either explicitly named, or implicitly named via an +// umbrella directory or umbrella file, as parsed by the ModuleMap object. +// If headers are found which are not referenced or covered by an umbrella +// directory or file, warning messages will be produced, and the doChecks +// function will return an error code of 1. Other errors result in an error +// code of 2. If no problems are found, an error code of 0 is returned. +// +// Note that in the case of umbrella headers, this tool invokes the compiler +// to preprocess the file, and uses a callback to collect the header files +// included by the umbrella header or any of its nested includes. If any +// front end options are needed for these compiler invocations, these are +// to be passed in via the CommandLine parameter. +// +// Warning message have the form: +// +// warning: module.modulemap does not account for file: Level3A.h +// +// Note that for the case of the module map referencing a file that does +// not exist, the module map parser in Clang will (at the time of this +// writing) display an error message. +// +// Potential problems with this program: +// +// 1. Might need a better header matching mechanism, or extensions to the +// canonical file format used. +// +// 2. It might need to support additional header file extensions. +// +// Future directions: +// +// 1. Add an option to fix the problems found, writing a new module map. +// Include an extra option to add unaccounted-for headers as excluded. +// +//===----------------------------------------------------------------------===// + +#include "ModularizeUtilities.h" +#include "clang/AST/ASTConsumer.h" +#include "CoverageChecker.h" +#include "clang/AST/ASTContext.h" +#include "clang/AST/RecursiveASTVisitor.h" +#include "clang/Basic/SourceManager.h" +#include "clang/Driver/Options.h" +#include "clang/Frontend/CompilerInstance.h" +#include "clang/Frontend/FrontendActions.h" +#include "clang/Lex/PPCallbacks.h" +#include "clang/Lex/Preprocessor.h" +#include "clang/Tooling/CompilationDatabase.h" +#include "clang/Tooling/Tooling.h" +#include "llvm/Option/Option.h" +#include "llvm/Support/CommandLine.h" +#include "llvm/Support/FileSystem.h" +#include "llvm/Support/Path.h" +#include "llvm/Support/raw_ostream.h" + +using namespace Modularize; +using namespace clang; +using namespace clang::driver; +using namespace clang::driver::options; +using namespace clang::tooling; +namespace cl = llvm::cl; +namespace sys = llvm::sys; + +// Preprocessor callbacks. +// We basically just collect include files. +class CoverageCheckerCallbacks : public PPCallbacks { +public: + CoverageCheckerCallbacks(CoverageChecker &Checker) : Checker(Checker) {} + ~CoverageCheckerCallbacks() {} + + // Include directive callback. + void InclusionDirective(SourceLocation HashLoc, const Token &IncludeTok, + StringRef FileName, bool IsAngled, + CharSourceRange FilenameRange, const FileEntry *File, + StringRef SearchPath, StringRef RelativePath, + const Module *Imported) { + Checker.collectUmbrellaHeaderHeader(File->getName()); + } + +private: + CoverageChecker &Checker; +}; + +// Frontend action stuff: + +// Consumer is responsible for setting up the callbacks. +class CoverageCheckerConsumer : public ASTConsumer { +public: + CoverageCheckerConsumer(CoverageChecker &Checker, Preprocessor &PP) { + // PP takes ownership. + PP.addPPCallbacks(llvm::make_unique<CoverageCheckerCallbacks>(Checker)); + } +}; + +class CoverageCheckerAction : public SyntaxOnlyAction { +public: + CoverageCheckerAction(CoverageChecker &Checker) : Checker(Checker) {} + +protected: + std::unique_ptr<ASTConsumer> CreateASTConsumer(CompilerInstance &CI, + StringRef InFile) override { + return llvm::make_unique<CoverageCheckerConsumer>(Checker, + CI.getPreprocessor()); + } + +private: + CoverageChecker &Checker; +}; + +class CoverageCheckerFrontendActionFactory : public FrontendActionFactory { +public: + CoverageCheckerFrontendActionFactory(CoverageChecker &Checker) + : Checker(Checker) {} + + virtual CoverageCheckerAction *create() { + return new CoverageCheckerAction(Checker); + } + +private: + CoverageChecker &Checker; +}; + +// CoverageChecker class implementation. + +// Constructor. +CoverageChecker::CoverageChecker(StringRef ModuleMapPath, + std::vector<std::string> &IncludePaths, + ArrayRef<std::string> CommandLine, + clang::ModuleMap *ModuleMap) + : ModuleMapPath(ModuleMapPath), IncludePaths(IncludePaths), + CommandLine(CommandLine), + ModMap(ModuleMap) {} + +// Create instance of CoverageChecker, to simplify setting up +// subordinate objects. +CoverageChecker *CoverageChecker::createCoverageChecker( + StringRef ModuleMapPath, std::vector<std::string> &IncludePaths, + ArrayRef<std::string> CommandLine, clang::ModuleMap *ModuleMap) { + + return new CoverageChecker(ModuleMapPath, IncludePaths, CommandLine, + ModuleMap); +} + +// Do checks. +// Starting from the directory of the module.modulemap file, +// Find all header files, optionally looking only at files +// covered by the include path options, and compare against +// the headers referenced by the module.modulemap file. +// Display warnings for unaccounted-for header files. +// Returns error_code of 0 if there were no errors or warnings, 1 if there +// were warnings, 2 if any other problem, such as if a bad +// module map path argument was specified. +std::error_code CoverageChecker::doChecks() { + std::error_code returnValue; + + // Collect the headers referenced in the modules. + collectModuleHeaders(); + + // Collect the file system headers. + if (!collectFileSystemHeaders()) + return std::error_code(2, std::generic_category()); + + // Do the checks. These save the problematic file names. + findUnaccountedForHeaders(); + + // Check for warnings. + if (!UnaccountedForHeaders.empty()) + returnValue = std::error_code(1, std::generic_category()); + + return returnValue; +} + +// The following functions are called by doChecks. + +// Collect module headers. +// Walks the modules and collects referenced headers into +// ModuleMapHeadersSet. +void CoverageChecker::collectModuleHeaders() { + for (ModuleMap::module_iterator I = ModMap->module_begin(), + E = ModMap->module_end(); + I != E; ++I) { + collectModuleHeaders(*I->second); + } +} + +// Collect referenced headers from one module. +// Collects the headers referenced in the given module into +// ModuleMapHeadersSet. +// FIXME: Doesn't collect files from umbrella header. +bool CoverageChecker::collectModuleHeaders(const Module &Mod) { + + if (const FileEntry *UmbrellaHeader = Mod.getUmbrellaHeader()) { + // Collect umbrella header. + ModuleMapHeadersSet.insert(ModularizeUtilities::getCanonicalPath( + UmbrellaHeader->getName())); + // Preprocess umbrella header and collect the headers it references. + if (!collectUmbrellaHeaderHeaders(UmbrellaHeader->getName())) + return false; + } + else if (const DirectoryEntry *UmbrellaDir = Mod.getUmbrellaDir()) { + // Collect headers in umbrella directory. + if (!collectUmbrellaHeaders(UmbrellaDir->getName())) + return false; + } + + for (auto &HeaderKind : Mod.Headers) + for (auto &Header : HeaderKind) + ModuleMapHeadersSet.insert(ModularizeUtilities::getCanonicalPath( + Header.Entry->getName())); + + for (Module::submodule_const_iterator MI = Mod.submodule_begin(), + MIEnd = Mod.submodule_end(); + MI != MIEnd; ++MI) + collectModuleHeaders(**MI); + + return true; +} + +// Collect headers from an umbrella directory. +bool CoverageChecker::collectUmbrellaHeaders(StringRef UmbrellaDirName) { + // Initialize directory name. + SmallString<256> Directory(ModuleMapDirectory); + if (UmbrellaDirName.size()) + sys::path::append(Directory, UmbrellaDirName); + if (Directory.size() == 0) + Directory = "."; + // Walk the directory. + std::error_code EC; + sys::fs::file_status Status; + for (sys::fs::directory_iterator I(Directory.str(), EC), E; I != E; + I.increment(EC)) { + if (EC) + return false; + std::string File(I->path()); + I->status(Status); + sys::fs::file_type Type = Status.type(); + // If the file is a directory, ignore the name and recurse. + if (Type == sys::fs::file_type::directory_file) { + if (!collectUmbrellaHeaders(File)) + return false; + continue; + } + // If the file does not have a common header extension, ignore it. + if (!ModularizeUtilities::isHeader(File)) + continue; + // Save header name. + ModuleMapHeadersSet.insert(ModularizeUtilities::getCanonicalPath(File)); + } + return true; +} + +// Collect headers rferenced from an umbrella file. +bool +CoverageChecker::collectUmbrellaHeaderHeaders(StringRef UmbrellaHeaderName) { + + SmallString<256> PathBuf(ModuleMapDirectory); + + // If directory is empty, it's the current directory. + if (ModuleMapDirectory.length() == 0) + sys::fs::current_path(PathBuf); + + // Create the compilation database. + std::unique_ptr<CompilationDatabase> Compilations; + Compilations.reset(new FixedCompilationDatabase(Twine(PathBuf), CommandLine)); + + std::vector<std::string> HeaderPath; + HeaderPath.push_back(UmbrellaHeaderName); + + // Create the tool and run the compilation. + ClangTool Tool(*Compilations, HeaderPath); + int HadErrors = Tool.run(new CoverageCheckerFrontendActionFactory(*this)); + + // If we had errors, exit early. + return HadErrors ? false : true; +} + +// Called from CoverageCheckerCallbacks to track a header included +// from an umbrella header. +void CoverageChecker::collectUmbrellaHeaderHeader(StringRef HeaderName) { + + SmallString<256> PathBuf(ModuleMapDirectory); + // If directory is empty, it's the current directory. + if (ModuleMapDirectory.length() == 0) + sys::fs::current_path(PathBuf); + // HeaderName will have an absolute path, so if it's the module map + // directory, we remove it, also skipping trailing separator. + if (HeaderName.startswith(PathBuf)) + HeaderName = HeaderName.substr(PathBuf.size() + 1); + // Save header name. + ModuleMapHeadersSet.insert(ModularizeUtilities::getCanonicalPath(HeaderName)); +} + +// Collect file system header files. +// This function scans the file system for header files, +// starting at the directory of the module.modulemap file, +// optionally filtering out all but the files covered by +// the include path options. +// Returns true if no errors. +bool CoverageChecker::collectFileSystemHeaders() { + + // Get directory containing the module.modulemap file. + // Might be relative to current directory, absolute, or empty. + ModuleMapDirectory = ModularizeUtilities::getDirectoryFromPath(ModuleMapPath); + + // If no include paths specified, we do the whole tree starting + // at the module.modulemap directory. + if (IncludePaths.size() == 0) { + if (!collectFileSystemHeaders(StringRef(""))) + return false; + } + else { + // Otherwise we only look at the sub-trees specified by the + // include paths. + for (std::vector<std::string>::const_iterator I = IncludePaths.begin(), + E = IncludePaths.end(); + I != E; ++I) { + if (!collectFileSystemHeaders(*I)) + return false; + } + } + + // Sort it, because different file systems might order the file differently. + std::sort(FileSystemHeaders.begin(), FileSystemHeaders.end()); + + return true; +} + +// Collect file system header files from the given path. +// This function scans the file system for header files, +// starting at the given directory, which is assumed to be +// relative to the directory of the module.modulemap file. +// \returns True if no errors. +bool CoverageChecker::collectFileSystemHeaders(StringRef IncludePath) { + + // Initialize directory name. + SmallString<256> Directory(ModuleMapDirectory); + if (IncludePath.size()) + sys::path::append(Directory, IncludePath); + if (Directory.size() == 0) + Directory = "."; + if (IncludePath.startswith("/") || IncludePath.startswith("\\") || + ((IncludePath.size() >= 2) && (IncludePath[1] == ':'))) { + llvm::errs() << "error: Include path \"" << IncludePath + << "\" is not relative to the module map file.\n"; + return false; + } + + // Recursively walk the directory tree. + std::error_code EC; + sys::fs::file_status Status; + int Count = 0; + for (sys::fs::recursive_directory_iterator I(Directory.str(), EC), E; I != E; + I.increment(EC)) { + if (EC) + return false; + std::string file(I->path()); + I->status(Status); + sys::fs::file_type type = Status.type(); + // If the file is a directory, ignore the name (but still recurses). + if (type == sys::fs::file_type::directory_file) + continue; + // If the file does not have a common header extension, ignore it. + if (!ModularizeUtilities::isHeader(file)) + continue; + // Save header name. + FileSystemHeaders.push_back(ModularizeUtilities::getCanonicalPath(file)); + Count++; + } + if (Count == 0) { + llvm::errs() << "warning: No headers found in include path: \"" + << IncludePath << "\"\n"; + } + return true; +} + +// Find headers unaccounted-for in module map. +// This function compares the list of collected header files +// against those referenced in the module map. Display +// warnings for unaccounted-for header files. +// Save unaccounted-for file list for possible. +// fixing action. +// FIXME: There probably needs to be some canonalization +// of file names so that header path can be correctly +// matched. Also, a map could be used for the headers +// referenced in the module, but +void CoverageChecker::findUnaccountedForHeaders() { + // Walk over file system headers. + for (std::vector<std::string>::const_iterator I = FileSystemHeaders.begin(), + E = FileSystemHeaders.end(); + I != E; ++I) { + // Look for header in module map. + if (ModuleMapHeadersSet.insert(*I).second) { + UnaccountedForHeaders.push_back(*I); + llvm::errs() << "warning: " << ModuleMapPath + << " does not account for file: " << *I << "\n"; + } + } +} diff --git a/clang-tools-extra/modularize/CoverageChecker.h b/clang-tools-extra/modularize/CoverageChecker.h new file mode 100644 index 00000000000..321dcae756f --- /dev/null +++ b/clang-tools-extra/modularize/CoverageChecker.h @@ -0,0 +1,172 @@ +//===-- CoverageChecker.h - Module map coverage checker -*- C++ -*-------===// +// +// The LLVM Compiler Infrastructure +// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +//===--------------------------------------------------------------------===// +/// +/// \file +/// \brief Definitions for CoverageChecker. +/// +//===--------------------------------------------------------------------===// + +#ifndef COVERAGECHECKER_H +#define COVERAGECHECKER_H + +#include "clang/Basic/Diagnostic.h" +#include "clang/Basic/FileManager.h" +#include "clang/Basic/LangOptions.h" +#include "clang/Basic/TargetInfo.h" +#include "clang/Basic/TargetOptions.h" +#include "clang/Frontend/TextDiagnosticPrinter.h" +#include "clang/Lex/HeaderSearch.h" +#include "clang/Lex/HeaderSearchOptions.h" +#include "clang/Lex/ModuleMap.h" +#include "clang/Lex/Preprocessor.h" +#include "llvm/ADT/StringSet.h" +#include "llvm/Support/Host.h" +#include <string> +#include <vector> + +namespace Modularize { + +/// Subclass TargetOptions so we can construct it inline with +/// the minimal option, the triple. +class ModuleMapTargetOptions : public clang::TargetOptions { +public: + ModuleMapTargetOptions() { Triple = llvm::sys::getDefaultTargetTriple(); } +}; + +/// Module map checker class. +/// This is the heart of the checker. +/// The doChecks function does the main work. +/// The data members store the options and internally collected data. +class CoverageChecker { + // Checker arguments. + + /// The module.modulemap file path. Can be relative or absolute. + llvm::StringRef ModuleMapPath; + /// The include paths to check for files. + /// (Note that other directories above these paths are ignored. + /// To expect all files to be accounted for from the module.modulemap + /// file directory on down, leave this empty.) + std::vector<std::string> IncludePaths; + /// The remaining arguments, to be passed to the front end. + llvm::ArrayRef<std::string> CommandLine; + /// The module map. + clang::ModuleMap *ModMap; + + // Internal data. + + /// Directory containing the module map. + /// Might be relative to the current directory, or absolute. + std::string ModuleMapDirectory; + /// Set of all the headers found in the module map. + llvm::StringSet<llvm::MallocAllocator> ModuleMapHeadersSet; + /// All the headers found in the file system starting at the + /// module map, or the union of those from the include paths. + std::vector<std::string> FileSystemHeaders; + /// Headers found in file system, but not in module map. + std::vector<std::string> UnaccountedForHeaders; + +public: + /// Constructor. + /// You can use the static createCoverageChecker to create an instance + /// of this object. + /// \param ModuleMapPath The module.modulemap file path. + /// Can be relative or absolute. + /// \param IncludePaths The include paths to check for files. + /// (Note that other directories above these paths are ignored. + /// To expect all files to be accounted for from the module.modulemap + /// file directory on down, leave this empty.) + /// \param CommandLine Compile command line arguments. + /// \param ModuleMap The module map to check. + CoverageChecker(llvm::StringRef ModuleMapPath, + std::vector<std::string> &IncludePaths, + llvm::ArrayRef<std::string> CommandLine, + clang::ModuleMap *ModuleMap); + + /// Create instance of CoverageChecker. + /// \param ModuleMapPath The module.modulemap file path. + /// Can be relative or absolute. + /// \param IncludePaths The include paths to check for files. + /// (Note that other directories above these paths are ignored. + /// To expect all files to be accounted for from the module.modulemap + /// file directory on down, leave this empty.) + /// \param CommandLine Compile command line arguments. + /// \param ModuleMap The module map to check. + /// \returns Initialized CoverageChecker object. + static CoverageChecker *createCoverageChecker( + llvm::StringRef ModuleMapPath, std::vector<std::string> &IncludePaths, + llvm::ArrayRef<std::string> CommandLine, + clang::ModuleMap *ModuleMap); + + /// Do checks. + /// Starting from the directory of the module.modulemap file, + /// Find all header files, optionally looking only at files + /// covered by the include path options, and compare against + /// the headers referenced by the module.modulemap file. + /// Display warnings for unaccounted-for header files. + /// \returns 0 if there were no errors or warnings, 1 if there + /// were warnings, 2 if any other problem, such as a bad + /// module map path argument was specified. + std::error_code doChecks(); + + // The following functions are called by doChecks. + + /// Collect module headers. + /// Walks the modules and collects referenced headers into + /// ModuleMapHeadersSet. + void collectModuleHeaders(); + + /// Collect referenced headers from one module. + /// Collects the headers referenced in the given module into + /// ModuleMapHeadersSet. + /// \param Mod The module reference. + /// \return True if no errors. + bool collectModuleHeaders(const clang::Module &Mod); + + /// Collect headers from an umbrella directory. + /// \param UmbrellaDirName The umbrella directory name. + /// \return True if no errors. + bool collectUmbrellaHeaders(llvm::StringRef UmbrellaDirName); + + /// Collect headers rferenced from an umbrella file. + /// \param UmbrellaHeaderName The umbrella file path. + /// \return True if no errors. + bool collectUmbrellaHeaderHeaders(llvm::StringRef UmbrellaHeaderName); + + /// Called from CoverageCheckerCallbacks to track a header included + /// from an umbrella header. + /// \param HeaderName The header file path. + void collectUmbrellaHeaderHeader(llvm::StringRef HeaderName); + + /// Collect file system header files. + /// This function scans the file system for header files, + /// starting at the directory of the module.modulemap file, + /// optionally filtering out all but the files covered by + /// the include path options. + /// \returns True if no errors. + bool collectFileSystemHeaders(); + + /// Collect file system header files from the given path. + /// This function scans the file system for header files, + /// starting at the given directory, which is assumed to be + /// relative to the directory of the module.modulemap file. + /// \returns True if no errors. + bool collectFileSystemHeaders(llvm::StringRef IncludePath); + + /// Find headers unaccounted-for in module map. + /// This function compares the list of collected header files + /// against those referenced in the module map. Display + /// warnings for unaccounted-for header files. + /// Save unaccounted-for file list for possible. + /// fixing action. + void findUnaccountedForHeaders(); +}; + +} // end namespace Modularize + +#endif // COVERAGECHECKER_H diff --git a/clang-tools-extra/modularize/Modularize.cpp b/clang-tools-extra/modularize/Modularize.cpp index 4578185efa5..f113398495c 100644 --- a/clang-tools-extra/modularize/Modularize.cpp +++ b/clang-tools-extra/modularize/Modularize.cpp @@ -7,6 +7,8 @@ // //===----------------------------------------------------------------------===// // +// Introduction +// // This file implements a tool that checks whether a set of headers provides // the consistent definitions required to use modules. It can also check an // existing module map for full coverage of the headers in a directory tree. @@ -62,17 +64,23 @@ // -block-check-header-list-only // Only warn if #include directives are inside extern or namespace // blocks if the included header is in the header list. +// -no-coverage-check +// Don't do the coverage check. +// -coverage-check-only +// Only do the coverage check. // -// Note that unless a "-prefix (header path)" option is specified, -// non-absolute file paths in the header list file will be relative -// to the header list file directory. Use -prefix to specify a different -// directory. +// Note that because modularize does not use the clang driver, +// you will likely need to pass in additional compiler front-end +// arguments to match those passed in by default by the driver. // // Note that by default, the underlying Clang front end assumes .h files // contain C source. If your .h files in the file list contain C++ source, // you should append the following to your command lines: -x c++ // -// Modularize will do normal parsing, reporting normal errors and warnings, +// Modularization Issue Checks +// +// In the process of checking headers for modularization issues, modularize +// will do normal parsing, reporting normal errors and warnings, // but will also report special error messages like the following: // // error: '(symbol)' defined at multiple locations: @@ -125,6 +133,36 @@ // // See PreprocessorTracker.cpp for additional details. // +// Module Map Coverage Check +// +// The coverage check uses the Clang ModuleMap class to read and parse the +// module map file. Starting at the module map file directory, or just the +// include paths, if specified, it will collect the names of all the files it +// considers headers (no extension, .h, or .inc--if you need more, modify the +// isHeader function). It then compares the headers against those referenced +// in the module map, either explicitly named, or implicitly named via an +// umbrella directory or umbrella file, as parsed by the ModuleMap object. +// If headers are found which are not referenced or covered by an umbrella +// directory or file, warning messages will be produced, and this program +// will return an error code of 1. Other errors result in an error code of 2. +// If no problems are found, an error code of 0 is returned. +// +// Note that in the case of umbrella headers, this tool invokes the compiler +// to preprocess the file, and uses a callback to collect the header files +// included by the umbrella header or any of its nested includes. If any +// front end options are needed for these compiler invocations, these +// can be included on the command line after the module map file argument. +// +// Warning message have the form: +// +// warning: module.modulemap does not account for file: Level3A.h +// +// Note that for the case of the module map referencing a file that does +// not exist, the module map parser in Clang will (at the time of this +// writing) display an error message. +// +// Module Map Assistant - Module Map Generation +// // Modularize also has an option ("-module-map-path=module.modulemap") that will // skip the checks, and instead act as a module.modulemap generation assistant, // generating a module map file based on the header list. An optional @@ -255,6 +293,21 @@ BlockCheckHeaderListOnly("block-check-header-list-only", cl::init(false), cl::desc("Only warn if #include directives are inside extern or namespace" " blocks if the included header is in the header list.")); +// Option for include paths for coverage check. +static cl::list<std::string> +IncludePaths("I", cl::desc("Include path for coverage check."), +cl::ZeroOrMore, cl::value_desc("path")); + +// Option for just doing the coverage check. +static cl::opt<bool> +NoCoverageCheck("no-coverage-check", cl::init(false), +cl::desc("Don't do the coverage check.")); + +// Option for just doing the coverage check. +static cl::opt<bool> +CoverageCheckOnly("coverage-check-only", cl::init(false), +cl::desc("Only do the coverage check.")); + // Save the program name for error messages. const char *Argv0; // Save the command line for comments. @@ -669,7 +722,8 @@ int main(int Argc, const char **Argv) { } std::unique_ptr<ModularizeUtilities> ModUtil; - + int HadErrors = 0; + ModUtil.reset( ModularizeUtilities::createModularizeUtilities( ListFileNames, HeaderPrefix)); @@ -686,6 +740,17 @@ int main(int Argc, const char **Argv) { return 0; // Success - Skip checks in assistant mode. } + // If we're doing module maps. + if (!NoCoverageCheck && ModUtil->HasModuleMap) { + // Do coverage check. + if (ModUtil->doCoverageCheck(IncludePaths, CommandLine)) + HadErrors = 1; + } + + // Bail early if only doing the coverage check. + if (CoverageCheckOnly) + return HadErrors; + // Create the compilation database. SmallString<256> PathBuf; sys::fs::current_path(PathBuf); @@ -702,7 +767,6 @@ int main(int Argc, const char **Argv) { EntityMap Entities; ClangTool Tool(*Compilations, ModUtil->HeaderFileNames); Tool.appendArgumentsAdjuster(getAddDependenciesAdjuster(ModUtil->Dependencies)); - int HadErrors = 0; ModularizeFrontendActionFactory Factory(Entities, *PPTracker, HadErrors); HadErrors |= Tool.run(&Factory); @@ -737,7 +801,7 @@ int main(int Argc, const char **Argv) { for (EntryBinArray::iterator DI = EntryBins.begin(), DE = EntryBins.end(); DI != DE; ++DI, ++KindIndex) { int ECount = DI->size(); - // If only 1 occurrence of this entity, skip it, as we only report duplicates. + // If only 1 occurrence of this entity, skip it, we only report duplicates. if (ECount <= 1) continue; LocationArray::iterator FI = DI->begin(); diff --git a/clang-tools-extra/modularize/ModularizeUtilities.cpp b/clang-tools-extra/modularize/ModularizeUtilities.cpp index afc03d49bd5..302e2526a17 100644 --- a/clang-tools-extra/modularize/ModularizeUtilities.cpp +++ b/clang-tools-extra/modularize/ModularizeUtilities.cpp @@ -13,16 +13,17 @@ // //===----------------------------------------------------------------------===// -#include "ModularizeUtilities.h" #include "clang/Basic/SourceManager.h" #include "clang/Driver/Options.h" #include "clang/Frontend/CompilerInstance.h" #include "clang/Frontend/FrontendActions.h" +#include "CoverageChecker.h" #include "llvm/ADT/SmallString.h"
#include "llvm/Support/FileUtilities.h"
#include "llvm/Support/MemoryBuffer.h"
#include "llvm/Support/Path.h"
#include "llvm/Support/raw_ostream.h"
+#include "ModularizeUtilities.h" using namespace clang; using namespace llvm; @@ -42,6 +43,7 @@ ModularizeUtilities::ModularizeUtilities(std::vector<std::string> &InputPaths, llvm::StringRef Prefix) : InputFilePaths(InputPaths), HeaderPrefix(Prefix), + HasModuleMap(false), // Init clang stuff needed for loading the module map and preprocessing. LangOpts(new LangOptions()), DiagIDs(new DiagnosticIDs()), DiagnosticOpts(new DiagnosticOptions()), @@ -88,7 +90,34 @@ std::error_code ModularizeUtilities::loadAllHeaderListsAndDependencies() { } return std::error_code(); } - + +// Do coverage checks. +// For each loaded module map, do header coverage check. +// Starting from the directory of the module.map file, +// Find all header files, optionally looking only at files +// covered by the include path options, and compare against +// the headers referenced by the module.map file. +// Display warnings for unaccounted-for header files. +// Returns 0 if there were no errors or warnings, 1 if there +// were warnings, 2 if any other problem, such as a bad +// module map path argument was specified. +std::error_code ModularizeUtilities::doCoverageCheck( + std::vector<std::string> &IncludePaths, + llvm::ArrayRef<std::string> CommandLine) { + int ModuleMapCount = ModuleMaps.size(); + int ModuleMapIndex; + std::error_code EC; + for (ModuleMapIndex = 0; ModuleMapIndex < ModuleMapCount; ++ModuleMapIndex) { + std::unique_ptr<clang::ModuleMap> &ModMap = ModuleMaps[ModuleMapIndex]; + CoverageChecker *Checker = CoverageChecker::createCoverageChecker( + InputFilePaths[ModuleMapIndex], IncludePaths, CommandLine, ModMap.get()); + std::error_code LocalEC = Checker->doChecks(); + if (LocalEC.value() > 0) + EC = LocalEC; + } + return EC; +} + // Load single header list and dependencies. std::error_code ModularizeUtilities::loadSingleHeaderListsAndDependencies( llvm::StringRef InputPath) { @@ -209,6 +238,9 @@ std::error_code ModularizeUtilities::loadModuleMap( // Save module map. ModuleMaps.push_back(std::move(ModMap)); + // Indicate we are using module maps. + HasModuleMap = true; + return std::error_code(); } @@ -338,3 +370,16 @@ bool ModularizeUtilities::isHeader(StringRef FileName) { return true; return false; } + +// Get directory path component from file path. +// \returns the component of the given path, which will be +// relative if the given path is relative, absolute if the +// given path is absolute, or "." if the path has no leading +// path component. +std::string ModularizeUtilities::getDirectoryFromPath(StringRef Path) { + SmallString<256> Directory(Path); + sys::path::remove_filename(Directory); + if (Directory.size() == 0) + return "."; + return Directory.str(); +} diff --git a/clang-tools-extra/modularize/ModularizeUtilities.h b/clang-tools-extra/modularize/ModularizeUtilities.h index 31f76d09ea8..61a97d8f22e 100644 --- a/clang-tools-extra/modularize/ModularizeUtilities.h +++ b/clang-tools-extra/modularize/ModularizeUtilities.h @@ -50,6 +50,8 @@ public: llvm::SmallVector<std::string, 32> HeaderFileNames; /// Map of top-level header file dependencies. DependencyMap Dependencies; + /// True if we have module maps. + bool HasModuleMap; // Functions. @@ -67,12 +69,30 @@ public: /// \returns Initialized ModularizeUtilities object. static ModularizeUtilities *createModularizeUtilities( std::vector<std::string> &InputPaths, - llvm::StringRef Prefix); + llvm::StringRef Prefix); /// Load header list and dependencies. /// \returns std::error_code. std::error_code loadAllHeaderListsAndDependencies(); + /// Do coverage checks. + /// For each loaded module map, do header coverage check. + /// Starting from the directory of the module.map file, + /// Find all header files, optionally looking only at files + /// covered by the include path options, and compare against + /// the headers referenced by the module.map file. + /// Display warnings for unaccounted-for header files. + /// \param IncludePaths The include paths to check for files. + /// (Note that other directories above these paths are ignored. + /// To expect all files to be accounted for from the module.modulemap + /// file directory on down, leave this empty.) + /// \param CommandLine Compile command line arguments. + /// \returns 0 if there were no errors or warnings, 1 if there + /// were warnings, 2 if any other problem, such as a bad + /// module map path argument was specified. + std::error_code doCoverageCheck(std::vector<std::string> &IncludePaths, + llvm::ArrayRef<std::string> CommandLine); + // Internal. protected: @@ -127,6 +147,13 @@ public: /// \returns true if it has a header extension or no extension. static bool isHeader(llvm::StringRef FileName); + /// Get directory path component from file path. + /// \returns the component of the given path, which will be + /// relative if the given path is relative, absolute if the + /// given path is absolute, or "." if the path has no leading + /// path component. + static std::string getDirectoryFromPath(llvm::StringRef Path); + // Internal data. /// Options controlling the language variant. diff --git a/clang-tools-extra/test/modularize/Inputs/CoverageNoProblems/Includes1/Level1A.h b/clang-tools-extra/test/modularize/Inputs/CoverageNoProblems/Includes1/Level1A.h new file mode 100644 index 00000000000..10eef6787e4 --- /dev/null +++ b/clang-tools-extra/test/modularize/Inputs/CoverageNoProblems/Includes1/Level1A.h @@ -0,0 +1 @@ +#define MACRO_1A 1 diff --git a/clang-tools-extra/test/modularize/Inputs/CoverageNoProblems/Includes2/Level2A.h b/clang-tools-extra/test/modularize/Inputs/CoverageNoProblems/Includes2/Level2A.h new file mode 100644 index 00000000000..9a355df06a8 --- /dev/null +++ b/clang-tools-extra/test/modularize/Inputs/CoverageNoProblems/Includes2/Level2A.h @@ -0,0 +1 @@ +#define MACRO_2A 1 diff --git a/clang-tools-extra/test/modularize/Inputs/CoverageNoProblems/NonIncludes/Level3A.h b/clang-tools-extra/test/modularize/Inputs/CoverageNoProblems/NonIncludes/Level3A.h new file mode 100644 index 00000000000..594e284cbf9 --- /dev/null +++ b/clang-tools-extra/test/modularize/Inputs/CoverageNoProblems/NonIncludes/Level3A.h @@ -0,0 +1 @@ +#define MACRO_3A 1 diff --git a/clang-tools-extra/test/modularize/Inputs/CoverageNoProblems/module.modulemap b/clang-tools-extra/test/modularize/Inputs/CoverageNoProblems/module.modulemap new file mode 100644 index 00000000000..97ad285cd23 --- /dev/null +++ b/clang-tools-extra/test/modularize/Inputs/CoverageNoProblems/module.modulemap @@ -0,0 +1,10 @@ +// module.map
+
+module Level1A {
+ header "Includes1/Level1A.h"
+ export *
+}
+module Level2A {
+ header "Includes2/Level2A.h"
+ export *
+}
diff --git a/clang-tools-extra/test/modularize/Inputs/CoverageProblems/Level1A.h b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/Level1A.h new file mode 100644 index 00000000000..165efc738ea --- /dev/null +++ b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/Level1A.h @@ -0,0 +1,2 @@ +#include "Level2A.h" +#define MACRO_1A 1 diff --git a/clang-tools-extra/test/modularize/Inputs/CoverageProblems/Level1B.h b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/Level1B.h new file mode 100644 index 00000000000..1e607987e63 --- /dev/null +++ b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/Level1B.h @@ -0,0 +1,2 @@ +#include "Level2B.h" +#define MACRO_1B 1 diff --git a/clang-tools-extra/test/modularize/Inputs/CoverageProblems/Level2A.h b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/Level2A.h new file mode 100644 index 00000000000..9a355df06a8 --- /dev/null +++ b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/Level2A.h @@ -0,0 +1 @@ +#define MACRO_2A 1 diff --git a/clang-tools-extra/test/modularize/Inputs/CoverageProblems/Level2B.h b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/Level2B.h new file mode 100644 index 00000000000..25b70179c93 --- /dev/null +++ b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/Level2B.h @@ -0,0 +1 @@ +#define MACRO_2B 1 diff --git a/clang-tools-extra/test/modularize/Inputs/CoverageProblems/Level3A.h b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/Level3A.h new file mode 100644 index 00000000000..1699061c0b7 --- /dev/null +++ b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/Level3A.h @@ -0,0 +1,2 @@ +#include "Sub/Level3B.h" +#define MACRO_3A 1 diff --git a/clang-tools-extra/test/modularize/Inputs/CoverageProblems/Sub/Level3B.h b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/Sub/Level3B.h new file mode 100644 index 00000000000..e1eee7ef8b5 --- /dev/null +++ b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/Sub/Level3B.h @@ -0,0 +1 @@ +#define MACRO_3B 1 diff --git a/clang-tools-extra/test/modularize/Inputs/CoverageProblems/UmbrellaFile.h b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/UmbrellaFile.h new file mode 100644 index 00000000000..5417a4272c1 --- /dev/null +++ b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/UmbrellaFile.h @@ -0,0 +1,3 @@ +#define UMBRELLA_HEADER 1 +#include "UmbrellaInclude1.h" +#include "UmbrellaInclude2.h" diff --git a/clang-tools-extra/test/modularize/Inputs/CoverageProblems/UmbrellaInclude1.h b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/UmbrellaInclude1.h new file mode 100644 index 00000000000..adf82cb507e --- /dev/null +++ b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/UmbrellaInclude1.h @@ -0,0 +1 @@ +#define UMBRELLA_INCLUDE_1 1 diff --git a/clang-tools-extra/test/modularize/Inputs/CoverageProblems/UmbrellaInclude2.h b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/UmbrellaInclude2.h new file mode 100644 index 00000000000..a3875ab39d0 --- /dev/null +++ b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/UmbrellaInclude2.h @@ -0,0 +1 @@ +#define UMBRELLA_INCLUDE_2 1 diff --git a/clang-tools-extra/test/modularize/Inputs/CoverageProblems/UmbrellaSub/Umbrell1.h b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/UmbrellaSub/Umbrell1.h new file mode 100644 index 00000000000..719c2237439 --- /dev/null +++ b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/UmbrellaSub/Umbrell1.h @@ -0,0 +1 @@ +#define UMBRELLA_1 1 diff --git a/clang-tools-extra/test/modularize/Inputs/CoverageProblems/UmbrellaSub/Umbrell2.h b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/UmbrellaSub/Umbrell2.h new file mode 100644 index 00000000000..8efec99b5f5 --- /dev/null +++ b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/UmbrellaSub/Umbrell2.h @@ -0,0 +1 @@ +#define UMBRELLA_2 1 diff --git a/clang-tools-extra/test/modularize/Inputs/CoverageProblems/module.modulemap b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/module.modulemap new file mode 100644 index 00000000000..af529aec281 --- /dev/null +++ b/clang-tools-extra/test/modularize/Inputs/CoverageProblems/module.modulemap @@ -0,0 +1,30 @@ +// module.map
+
+module Level1A {
+ header "Level1A.h"
+ export *
+}
+module Level1B {
+ header "Level1B.h"
+ export *
+ module Level2B {
+ header "Level2B.h"
+ export *
+ }
+}
+module Level2A {
+ header "Level2A.h"
+ export *
+}
+module UmbrellaDirectoryModule {
+ umbrella "UmbrellaSub"
+}
+module UmbrellaHeaderModule {
+ umbrella header "UmbrellaFile.h"
+}
+/*
+module NoHeader {
+ header "NoHeader.h"
+ export *
+}
+*/
diff --git a/clang-tools-extra/test/modularize/NoProblems.modularize b/clang-tools-extra/test/modularize/NoProblems.modularize index 48d3482910f..b3cc36b605a 100644 --- a/clang-tools-extra/test/modularize/NoProblems.modularize +++ b/clang-tools-extra/test/modularize/NoProblems.modularize @@ -1,6 +1,6 @@ # RUN: modularize %s -x c++ # RUN: modularize -prefix=%p %s -x c++ -# RUN: modularize %S/Inputs/NoProblems.modulemap -x c++ +# RUN: modularize -no-coverage-check %S/Inputs/NoProblems.modulemap -x c++ Inputs/SomeTypes.h Inputs/SomeDecls.h diff --git a/clang-tools-extra/test/modularize/NoProblemsCoverage.modularize b/clang-tools-extra/test/modularize/NoProblemsCoverage.modularize new file mode 100644 index 00000000000..bcb06f84b1f --- /dev/null +++ b/clang-tools-extra/test/modularize/NoProblemsCoverage.modularize @@ -0,0 +1 @@ +# RUN: modularize -I Includes1 -I Includes2 %S/Inputs/CoverageNoProblems/module.modulemap diff --git a/clang-tools-extra/test/modularize/ProblemsCoverage.modularize b/clang-tools-extra/test/modularize/ProblemsCoverage.modularize new file mode 100644 index 00000000000..68224e1c374 --- /dev/null +++ b/clang-tools-extra/test/modularize/ProblemsCoverage.modularize @@ -0,0 +1,4 @@ +# RUN: not modularize %S/Inputs/CoverageProblems/module.modulemap 2>&1 | FileCheck %s + +# CHECK: warning: {{.*}}{{[/\\]}}Inputs/CoverageProblems/module.modulemap does not account for file: {{.*}}{{[/\\]}}Inputs/CoverageProblems/Level3A.h +# CHECK-NEXT: warning: {{.*}}{{[/\\]}}Inputs/CoverageProblems/module.modulemap does not account for file: {{.*}}{{[/\\]}}Inputs/CoverageProblems/Sub/Level3B.h |