From 1731fc88d1fa1fa55edd056db73a339b415dd5d6 Mon Sep 17 00:00:00 2001 From: Bruno Cardoso Lopes Date: Tue, 15 Oct 2019 14:23:55 +0000 Subject: Reapply: [Modules][PCH] Hash input files content Summary: When files often get touched during builds, the mtime based validation leads to different problems in implicit modules builds, even when the content doesn't actually change: - Modules only: module invalidation due to out of date files. Usually causing rebuild traffic. - Modules + PCH: build failures because clang cannot rebuild a module if it comes from building a PCH. - PCH: build failures because clang cannot rebuild a PCH in case one of the input headers has different mtime. This patch proposes hashing the content of input files (headers and module maps), which is performed during serialization time. When looking at input files for validation, clang only computes the hash in case there's a mtime mismatch. I've tested a couple of different hash algorithms availble in LLVM in face of building modules+pch for `#import `: - `hash_code`: performace diff within the noise, total module cache increased by 0.07%. - `SHA1`: 5% slowdown. Haven't done real size measurements, but it'd be BLOCK_ID+20 bytes per input file, instead of BLOCK_ID+8 bytes from `hash_code`. - `MD5`: 3% slowdown. Like above, but BLOCK_ID+16 bytes per input file. Given the numbers above, the patch uses `hash_code`. The patch also improves invalidation error msgs to point out which type of problem the user is facing: "mtime", "size" or "content". rdar://problem/29320105 Reviewers: dexonsmith, arphaman, rsmith, aprantl Subscribers: jkorous, cfe-commits, ributzka Tags: #clang Differential Revision: https://reviews.llvm.org/D67249 > llvm-svn: 374841 llvm-svn: 374895 --- clang/lib/Frontend/CompilerInstance.cpp | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) (limited to 'clang/lib/Frontend/CompilerInstance.cpp') diff --git a/clang/lib/Frontend/CompilerInstance.cpp b/clang/lib/Frontend/CompilerInstance.cpp index a7b7114f791..97b92667416 100644 --- a/clang/lib/Frontend/CompilerInstance.cpp +++ b/clang/lib/Frontend/CompilerInstance.cpp @@ -510,7 +510,8 @@ IntrusiveRefCntPtr CompilerInstance::createPCHExternalASTSource( PP, ModuleCache, &Context, PCHContainerRdr, Extensions, Sysroot.empty() ? "" : Sysroot.data(), DisablePCHValidation, AllowPCHWithCompilerErrors, /*AllowConfigurationMismatch*/ false, - HSOpts.ModulesValidateSystemHeaders, UseGlobalModuleIndex)); + HSOpts.ModulesValidateSystemHeaders, HSOpts.ValidateASTInputFilesContent, + UseGlobalModuleIndex)); // We need the external source to be set up before we read the AST, because // eagerly-deserialized declarations may use it. @@ -1492,6 +1493,7 @@ void CompilerInstance::createModuleManager() { /*AllowASTWithCompilerErrors=*/false, /*AllowConfigurationMismatch=*/false, HSOpts.ModulesValidateSystemHeaders, + HSOpts.ValidateASTInputFilesContent, getFrontendOpts().UseGlobalModuleIndex, std::move(ReadTimer)); if (hasASTConsumer()) { ModuleManager->setDeserializationListener( -- cgit v1.2.3