summaryrefslogtreecommitdiffstats
path: root/clang/lib/Serialization/GeneratePCH.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Apply clang-tidy's performance-unnecessary-value-param to parts of clang.Benjamin Kramer2017-03-211-3/+3
| | | | | | No functionality change intended. llvm-svn: 298443
* Reapply "Modules: Cache PCMs in memory and avoid a use-after-free"Duncan P. N. Exon Smith2017-03-201-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit r298185, effectively reapplying r298165, after fixing the new unit tests (PR32338). The memory buffer generator doesn't null-terminate the MemoryBuffer it creates; this version of the commit informs getMemBuffer about that to avoid the assert. Original commit message follows: ---- Clang's internal build system for implicit modules uses lock files to ensure that after a process writes a PCM it will read the same one back in (without contention from other -cc1 commands). Since PCMs are read from disk repeatedly while invalidating, building, and importing, the lock is not released quickly. Furthermore, the LockFileManager is not robust in every environment. Other -cc1 commands can stall until timeout (after about eight minutes). This commit changes the lock file from being necessary for correctness to a (possibly dubious) performance hack. The remaining benefit is to reduce duplicate work in competing -cc1 commands which depend on the same module. Follow-up commits will change the internal build system to continue after a timeout, and reduce the timeout. Perhaps we should reconsider blocking at all. This also fixes a use-after-free, when one part of a compilation validates a PCM and starts using it, and another tries to swap out the PCM for something new. The PCMCache is a new type called MemoryBufferCache, which saves memory buffers based on their filename. Its ownership is shared by the CompilerInstance and ModuleManager. - The ModuleManager stores PCMs there that it loads from disk, never touching the disk if the cache is hot. - When modules fail to validate, they're removed from the cache. - When a CompilerInstance is spawned to build a new module, each already-loaded PCM is assumed to be valid, and is frozen to avoid the use-after-free. - Any newly-built module is written directly to the cache to avoid the round-trip to the filesystem, making lock files unnecessary for correctness. Original patch by Manman Ren; most testcases by Adrian Prantl! llvm-svn: 298278
* Revert "Modules: Cache PCMs in memory and avoid a use-after-free"Renato Golin2017-03-181-2/+1
| | | | | | This reverts commit r298165, as it broke the ARM builds. llvm-svn: 298185
* Modules: Cache PCMs in memory and avoid a use-after-freeDuncan P. N. Exon Smith2017-03-171-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Clang's internal build system for implicit modules uses lock files to ensure that after a process writes a PCM it will read the same one back in (without contention from other -cc1 commands). Since PCMs are read from disk repeatedly while invalidating, building, and importing, the lock is not released quickly. Furthermore, the LockFileManager is not robust in every environment. Other -cc1 commands can stall until timeout (after about eight minutes). This commit changes the lock file from being necessary for correctness to a (possibly dubious) performance hack. The remaining benefit is to reduce duplicate work in competing -cc1 commands which depend on the same module. Follow-up commits will change the internal build system to continue after a timeout, and reduce the timeout. Perhaps we should reconsider blocking at all. This also fixes a use-after-free, when one part of a compilation validates a PCM and starts using it, and another tries to swap out the PCM for something new. The PCMCache is a new type called MemoryBufferCache, which saves memory buffers based on their filename. Its ownership is shared by the CompilerInstance and ModuleManager. - The ModuleManager stores PCMs there that it loads from disk, never touching the disk if the cache is hot. - When modules fail to validate, they're removed from the cache. - When a CompilerInstance is spawned to build a new module, each already-loaded PCM is assumed to be valid, and is frozen to avoid the use-after-free. - Any newly-built module is written directly to the cache to avoid the round-trip to the filesystem, making lock files unnecessary for correctness. Original patch by Manman Ren; most testcases by Adrian Prantl! llvm-svn: 298165
* Modules: Use hash of PCM content for SIGNATUREDuncan P. N. Exon Smith2017-03-131-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Change ASTFileSignature from a random 32-bit number to the hash of the PCM content. - Move definition ASTFileSignature to Basic/Module.h so Module and ASTSourceDescriptor can use it. - Change the signature from uint64_t to std::array<uint32_t,5>. - Stop using (saving/reading) the size and modification time of PCM files when there is a valid SIGNATURE. - Add UNHASHED_CONTROL_BLOCK, and use it to store the SIGNATURE record and other records that shouldn't affect the hash. Because implicit modules reuses the same file for multiple levels of -Werror, this includes DIAGNOSTIC_OPTIONS and DIAG_PRAGMA_MAPPINGS. This helps to solve a PCH + implicit Modules dependency issue: PCH files are handled by the external build system, whereas implicit modules are handled by internal compiler build system. This prevents invalidating a PCH when the compiler overwrites a PCM file with the same content (modulo the diagnostic differences). Design and original patch by Manman Ren! llvm-svn: 297655
* Use shared_ptr instead of IntrusiveRefCntPtr for ModuleFileExtensionDavid Blaikie2017-01-051-1/+1
| | | | | | | The intrusiveness wasn't needed here, so this simplifies/clarifies the ownership model. llvm-svn: 291150
* C++ Modules TS: add frontend support for building pcm files from moduleRichard Smith2016-08-261-2/+5
| | | | | | | interface files. At the moment, all declarations (and no macros) are exported, and 'export' declarations are not supported yet. llvm-svn: 279794
* Refactor to remove the assumption that we know the name of the module we're ↵Richard Smith2016-08-251-6/+13
| | | | | | emitting at the point when we create a PCHGenerator (with the C++ modules TS, we find that out part way through parsing the input). llvm-svn: 279766
* [NFC] Header cleanupMehdi Amini2016-07-181-4/+1
| | | | | | | | | | Summary: Removed unused headers, replaced some headers with forward class declarations Patch by: Eugene <claprix@yandex.ru> Differential Revision: https://reviews.llvm.org/D20100 llvm-svn: 275882
* [PCH/preamble] Make sure that if the preamble/PCH was serialized with errors ↵Argyrios Kyrtzidis2016-07-131-1/+4
| | | | | | | | | | that we set diagnostic engine state appropriately. Otherwise there can be a crash with CFG analysis warnings doing work on invalid AST. Fixes crash of rdar://26224134 llvm-svn: 275313
* Introduce module file extensions to piggy-back data onto module files.Douglas Gregor2015-11-031-5/+7
| | | | | | | | | | | | | | | | | | | | | Introduce the notion of a module file extension, which introduces additional information into a module file at the time it is built that can then be queried when the module file is read. Module file extensions are identified by a block name (which must be unique to the extension) and can write any bitstream records into their own extension block within the module file. When a module file is loaded, any extension blocks are matched up with module file extension readers, that are per-module-file and are given access to the input bitstream. Note that module file extensions can only be introduced by programmatic clients that have access to the CompilerInvocation. There is only one such extension at the moment, which is used for testing the module file extension harness. As a future direction, one could imagine allowing the plugin mechanism to introduce new module file extensions. llvm-svn: 251955
* Roll-back r250822.Angel Garcia Gomez2015-10-201-1/+2
| | | | | | | | | | Summary: It breaks the build for the ASTMatchers Subscribers: klimek, cfe-commits Differential Revision: http://reviews.llvm.org/D13893 llvm-svn: 250827
* Apply modernize-use-default to clang.Angel Garcia Gomez2015-10-201-2/+1
| | | | | | | | | | | | Summary: Replace empty bodies of default constructors and destructors with '= default'. Reviewers: bkramer, klimek Subscribers: klimek, alexfh, cfe-commits Differential Revision: http://reviews.llvm.org/D13890 llvm-svn: 250822
* Module Debugging: Use the clang module signature as the module's dwo_idAdrian Prantl2015-09-221-1/+2
| | | | | | | | | | | when building a module. Clang already records the module signature when building a skeleton CU to reference a clang module. Matching the id in the skeleton with the one in the module allows a DWARF consumer to verify that they found the correct version of the module without them needing to know about the clang module format. llvm-svn: 248345
* [modules] When explicitly building a module file, don't include timestamps inRichard Smith2015-08-171-2/+3
| | | | | | | the produced pcm file for stable file creation across distributed build systems. llvm-svn: 245199
* Remove unnecessary include.Adrian Prantl2015-06-291-1/+0
| | | | llvm-svn: 240963
* Introduce a PCHContainerOperations interface (NFC).Adrian Prantl2015-06-201-22/+11
| | | | | | | | | | | | | | | | A PCHContainerOperations abstract interface provides operations for creating and unwrapping containers for serialized ASTs (precompiled headers and clang modules). The default implementation is RawPCHContainerOperations, which uses a flat file for the output. The main application for this interface will be an ObjectFilePCHContainerOperations implementation that uses LLVM to wrap the module in an ELF/Mach-O/COFF container to store debug info alongside the AST. rdar://problem/20091852 llvm-svn: 240225
* Revert "Wrap clang module files in a Mach-O, ELF, or COFF container."Adrian Prantl2015-02-251-6/+12
| | | | llvm-svn: 230454
* Wrap clang module files in a Mach-O, ELF, or COFF container.Adrian Prantl2015-02-251-12/+6
| | | | | | | | | | | | | | | | | This is a necessary prerequisite for debugging with modules. The .pcm files become containers that hold the serialized AST which allows us to store debug information in the module file that can be shared by all object files that were built importing the module. This reapplies r230044 with a fixed configure+make build and updated dependencies and testcase requirements. Over the last iteration this version adds - missing target requirements for testcases that specify an x86 triple, - a missing clangCodeGen.a dependency to libClang.a in the make build. rdar://problem/19104245 llvm-svn: 230423
* Revert "Wrap clang module files in a Mach-O, ELF, or COFF container."Adrian Prantl2015-02-241-6/+12
| | | | | | | This reverts commit r230305. Off to fix another round of missing dependencies on various platforms. llvm-svn: 230309
* Wrap clang module files in a Mach-O, ELF, or COFF container.Adrian Prantl2015-02-241-12/+6
| | | | | | | | | | | | | | This is a necessary prerequisite for debugging with modules. The .pcm files become containers that hold the serialized AST which allows us to store debug information in the module file that can be shared by all object files that were built importing the module. rdar://problem/19104245 This reapplies r230044 with a fixed configure+make build and updated dependencies. Take 3. llvm-svn: 230305
* Revert "Wrap clang module files in a Mach-O, ELF, or COFF container."Adrian Prantl2015-02-211-6/+12
| | | | | | | | This reverts commit 230099. The Linux configure+make build variant still needs some work. llvm-svn: 230103
* Wrap clang module files in a Mach-O, ELF, or COFF container.Adrian Prantl2015-02-201-12/+6
| | | | | | | | | | | | | | This is a necessary prerequisite for debugging with modules. The .pcm files become containers that hold the serialized AST which allows us to store debug information in the module file that can be shared by all object files that were built importing the module. rdar://problem/19104245 This reapplies r230044 with a fixed configure+make build and updated dependencies. Take 2. llvm-svn: 230089
* Revert "Wrap clang module files in a Mach-O, ELF, or COFF container."Adrian Prantl2015-02-201-6/+12
| | | | | | | | This reverts commit r230067. Investigating another batch of problems found by the bots. llvm-svn: 230073
* Wrap clang module files in a Mach-O, ELF, or COFF container.Adrian Prantl2015-02-201-12/+6
| | | | | | | | | | | | | | This is a necessary prerequisite for debugging with modules. The .pcm files become containers that hold the serialized AST which allows us to store debug information in the module file that can be shared by all object files that were built importing the module. rdar://problem/19104245 This reapplies r230044 with a fixed configure+make build and updated dependencies. llvm-svn: 230067
* Revert "Wrap clang module files in a Mach-O, ELF, or COFF container."Adrian Prantl2015-02-201-6/+12
| | | | | | | | | This reverts commit r230044 while dealing with buildbot breakage. Conflicts: test/Modules/module_container.m llvm-svn: 230052
* Wrap clang module files in a Mach-O, ELF, or COFF container.Adrian Prantl2015-02-201-12/+6
| | | | | | | | | | | This is a necessary prerequisite for debugging with modules. The .pcm files become containers that hold the serialized AST which allows us to store debug information in the module file that can be shared by all object files that were built importing the module. rdar://problem/19104245 llvm-svn: 230044
* [C++11] Use 'nullptr'. Serialization edition.Craig Topper2014-05-221-1/+1
| | | | llvm-svn: 209392
* [libclang] Allow building a precompiled preamble with compiler errorsArgyrios Kyrtzidis2013-06-111-4/+13
| | | | | | | | | | | A while ago we allowed libclang to build a PCH that had compiler errors; this was to retain the performance afforded by a PCH even if the user's code is in an intermediate state. Extend this for the precompiled preamble as well. rdar://14109828 llvm-svn: 183717
* [Preprocessor] Remove PPMutationListener.Argyrios Kyrtzidis2013-03-271-4/+0
| | | | | | It's not used anymore. llvm-svn: 178106
* Sort all of Clang's files under 'lib', and fix up the broken headersChandler Carruth2012-12-041-3/+3
| | | | | | | | | | | | | uncovered. This required manually correcting all of the incorrect main-module headers I could find, and running the new llvm/utils/sort_includes.py script over the files. I also manually added quite a few missing headers that were uncovered by shuffling the order or moving headers up to be main-module-headers. llvm-svn: 169237
* [PCH] Remove the stat cache from the PCH file.Argyrios Kyrtzidis2012-10-311-7/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The stat cache became essentially useless ever since we started validating all file entries in the PCH. But the motivating reason for removing it now is that it also affected correctness in this situation: -You have a header without include guards (using "#pragma once" or #import) -When creating the PCH: -The same header is referenced in an #include with different filename cases. -In the PCH, of course, we record only one file entry for the header file -But we cache in the PCH file the stat info for both filename cases -Then the source files are updated and the header file is updated in a way that its size and modification time are the same but its inode changes -When using the PCH: -We validate the headers, we check that header file and we create a file entry with its current inode -There's another #include with a filename with different case than the previously created file entry -In order to get its stat info we go through the cached stat info of the PCH and we receive the old inode -because of the different inodes, we think they are different files so we go ahead and include its contents. Removing the stat cache will potentially break clients that are attempting to use the stat cache as a way of avoiding having the actual input files available. If that use case is important, patches are welcome to bring it back in a way that will actually work correctly (i.e., emit a PCH that is self-contained, coping with literal strings, line/column computations, etc.). This fixes rdar://5502805 llvm-svn: 167172
* Rework the (de-)serialization of macros, as stored inDouglas Gregor2012-10-091-0/+4
| | | | | | | | | | | | | | | | MacroInfo*. Instead of simply dumping an offset into the current file, give each macro definition a proper ID with all of the standard modules-remapping facilities. Additionally, when a macro is modified in a subsequent AST file (e.g., #undef'ing a macro loaded from another module or from a precompiled header), provide a macro update record rather than rewriting the entire macro definition. This gives us greater consistency with the way we handle declarations, and ties together macro definitions much more cleanly. Note that we're still not actually deserializing macro history (we never were), but it's far easy to do properly now. llvm-svn: 165560
* Revert my patches which removed Diagnostic.h includes by moving some ↵Benjamin Kramer2012-02-071-1/+0
| | | | | | | | | | | | | | | | | | operator overloads out of line. This seems to negatively affect compile time onsome ObjC tests (which use a lot of partial diagnostics I assume). I have to come up with a way to keep them inline without including Diagnostic.h everywhere. Now adding a new diagnostic requires a full rebuild of e.g. the static analyzer which doesn't even use those diagnostics. This reverts commit 6496bd10dc3a6d5e3266348f08b6e35f8184bc99. This reverts commit 7af19b817ba964ac560b50c1ed6183235f699789. This reverts commit fdd15602a42bbe26185978ef1e17019f6d969aa7. This reverts commit 00bd44d5677783527d7517c1ffe45e4d75a0f56f. This reverts commit ef9b60ffed980864a8db26ad30344be429e58ff5. llvm-svn: 150006
* Remove Diagnostic.h include from Preprocessor.h.Benjamin Kramer2012-02-041-0/+1
| | | | | | | - Move the offending methods out of line and fix transitive includers. - This required changing an enum in the PPCallback API into an unsigned. llvm-svn: 149782
* Clean up the comments for this file now that its contents are totallyChandler Carruth2011-12-091-4/+3
| | | | | | | | different from what the comments indicated. Also drop a no longer used include that also violates the layering between Serialization and Frontend. llvm-svn: 146230
* Promote ModuleMap::Module to a namespace-scope class in the BasicDouglas Gregor2011-11-301-1/+1
| | | | | | | | | library, since modules cut across all of the libraries. Rename serialization::Module to serialization::ModuleFile to side-step the annoying naming conflict. Prune a bunch of ModuleMap.h includes that are no longer needed (most files only needed the Module type). llvm-svn: 145538
* When writing a module file, pass the module through to the ASTDouglas Gregor2011-11-301-3/+3
| | | | | | writer. No functionality change. llvm-svn: 145479
* In libclang, when visiting preprocessed entities in a source range, useArgyrios Kyrtzidis2011-09-191-7/+0
| | | | | | | | | | PreprocessingRecord's getPreprocessedEntitiesInRange. Also remove all the stuff that were added in ASTUnit that are unnecessary now that we do a binary search for preprocessed entities and deserialize only what is necessary. llvm-svn: 140063
* Modules hide macro definitions by default, so that silly things likeDouglas Gregor2011-09-011-2/+4
| | | | | | | | | | | include guards don't show up as macro definitions in every translation unit that imports a module. Macro definitions can, however, be exported with the intentionally-ugly #__export_macro__ directive. Implement this feature by not even bothering to serialize non-exported macros to a module, because clients of that module need not (should not) know that these macros even exist. llvm-svn: 138943
* Remove the Chaining argument from the PCH/module generator. It's no longer usedDouglas Gregor2011-08-251-9/+3
| | | | llvm-svn: 138596
* Introduce a -cc1 option "-emit-module", that creates a binary moduleDouglas Gregor2011-08-251-1/+1
| | | | | | | | | from the given source. -emit-module behaves similarly to -emit-pch, except that Sema is somewhat more strict about the contents of -emit-module. In the future, there are likely to be more interesting differences. llvm-svn: 138595
* remove unneeded llvm:: namespace qualifiers on some core types now that ↵Chris Lattner2011-07-231-1/+1
| | | | | | | | LLVM.h imports them into the clang namespace. llvm-svn: 135852
* Switch all of the "isysroot" const char*'s throughout the AST readerDouglas Gregor2011-07-221-9/+3
| | | | | | and writer to StringRef or std::string, as appropriate. llvm-svn: 135769
* Fix an embarrassing bug in relocatable PCH support, where we wereDouglas Gregor2011-07-221-1/+10
| | | | | | | passing a temporary const char* down as the "isysroot" parameter and then accessing it later. Fixes <rdar://problem/9035180>. llvm-svn: 135749
* Allow resolving headers from a PCH even after headers+PCH were moved to ↵Argyrios Kyrtzidis2011-02-151-2/+3
| | | | | | | | | | | another path. Store in PCH the directory that the PCH was originally created in. If a header file is not found at the path that we expect it to be and the PCH file was moved from its original location, try to resolve the file by assuming that header+PCH were moved together and the header is in the same place relative to the PCH. llvm-svn: 125576
* When using a precompiled preamble with detailed preprocessing records,Douglas Gregor2010-11-301-1/+7
| | | | | | | | | | trap the serialized preprocessing records (macro definitions, macro instantiations, macro definitions) from the generation of the precompiled preamble, then replay those when walking the list of preprocessed entities. This eliminates a bug where clang_getCursor() wasn't able to find preprocessed-entity cursors in the preamble. llvm-svn: 120396
* rework the stat cache, pulling it out of FileManager.h intoChris Lattner2010-11-231-1/+2
| | | | | | | its own header and giving it some more structure. No functionality change. llvm-svn: 120030
* Put the mechanism in place to track modifications in an AST entity that were ↵Argyrios Kyrtzidis2010-10-241-1/+7
| | | | | | | | | | | | committed after its initial creation/deserialization and store the changes in a chained PCH. The idea is that the AST entities call methods on the ASTMutationListener to give notifications of changes; the PCHWriter implements the ASTMutationListener interface and stores the incremental changes of the updated entity. WIP llvm-svn: 117235
* More PCH -> AST renaming.Sebastian Redl2010-08-181-1/+1
| | | | llvm-svn: 111472
OpenPOWER on IntegriCloud