summaryrefslogtreecommitdiffstats
path: root/clang-tools-extra/clangd/index/BackgroundIndexStorage.cpp
Commit message (Collapse)AuthorAgeFilesLines
* [Support] Add overload writeFileAtomically(std::function Writer)Jan Korous2019-09-131-31/+7
| | | | | | Differential Revision: https://reviews.llvm.org/D67424 llvm-svn: 371890
* [clang-tools-extra] Migrate llvm::make_unique to std::make_uniqueJonas Devlieghere2019-08-141-4/+4
| | | | | | | | | | Now that we've moved to C++14, we no longer need the llvm::make_unique implementation from STLExtras.h. This patch is a mechanical replacement of (hopefully) all the llvm::make_unique instances across the monorepo. Differential revision: https://reviews.llvm.org/D66259 llvm-svn: 368944
* Revert "Revert r366458, r366467 and r366468"Kadir Cetinkaya2019-07-191-8/+32
| | | | | | | | | | | | | | | | | | | | | | This reverts commit 9c377105da0be7c2c9a3c70035ce674c71b846af. [clangd][BackgroundIndexLoader] Directly store DependentTU while loading shard Summary: We were deferring the population of DependentTU field in LoadedShard until BackgroundIndexLoader was consumed. This actually triggers a use after free since the shards FileToTU was pointing at could've been moved while consuming the Loader. Reviewers: sammccall Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D64980 llvm-svn: 366559
* Revert r366458, r366467 and r366468Azharuddin Mohammed2019-07-191-32/+8
| | | | | | | | | | | r366458 is causing test failures. r366467 and r366468 had to be reverted as they were casuing conflict while reverting r366458. r366468 [clangd] Remove dead code from BackgroundIndex r366467 [clangd] BackgroundIndex stores shards to the closest project r366458 [clangd] Refactor background-index shard loading llvm-svn: 366551
* [clangd] BackgroundIndex stores shards to the closest projectKadir Cetinkaya2019-07-181-8/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Changes persistance logic to store shards at the directory of closest CDB. Previously we were storing all shards to directory of the CDB that triggered indexing, it had its downsides. For example, if you had two TUs coming from a different CDB but depending on the same header foo.h, we will store the foo.h only for the first CDB, and it would be missing for the second and we would never persist it since it was actually present in the memory and persisted before. This patch still stores only a single copy of a shard, but makes the directory a function of the file name. So that the shard place will be unique even with multiple CDBs accessing the file. This directory is determined as the first directory containing a CDB in the file's parent directories, if no such directory exists we make use of the home directory. Reviewers: sammccall Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D64745 llvm-svn: 366467
* [clangd] Use xxhash instead of SHA1 for background index file digests.Sam McCall2019-07-081-7/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Currently SHA1 is about 10% of our CPU, this patch reduces it to ~1%. xxhash is a well-defined (stable) non-cryptographic hash optimized for fast checksums (like crc32). Collisions shouldn't be a problem, despite the reduced length: - for actual file content (used to invalidate bg index shards), there are only two versions that can collide (new shard and old shard). - for file paths in bg index shard filenames, we would need 2^32 files with the same filename to expect a collision. Imperfect hashing may reduce this a bit but it's well beyond what's plausible. This will invalidate shards on disk (as usual; I bumped the version), but this time the filenames are changing so the old files will stick around :-( So this is more expensive than the usual bump, but would be good to land before the v9 branch when everyone will start using bg index. Reviewers: kadircet Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D64306 llvm-svn: 365311
* [clangd] Delete an unused declarationFangrui Song2019-05-011-3/+0
| | | | llvm-svn: 359674
* [clangd] Store index in '.clangd/index' instead of '.clangd-index'Ilya Biryukov2019-02-201-4/+4
| | | | | | | | | | | | | | | | Summary: To take up the .clangd folder for other potential uses in the future. Reviewers: kadircet, sammccall Reviewed By: kadircet Subscribers: ioeric, MaskRay, jkorous, arphaman, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D58440 llvm-svn: 354505
* Update the file headers across all of the LLVM projects in the monorepoChandler Carruth2019-01-191-4/+3
| | | | | | | | | | | | | | | | | to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
* [clangd] Change diskbackedstorage to be atomicKadir Cetinkaya2018-12-171-8/+33
| | | | | | | | | | | | | | | | Summary: There was a chance that multiple clangd instances could try to write same shard, in which case we would get a malformed file most likely. This patch changes the writing mechanism to first write to a temporary file and then rename it to fit real destination. Which is guaranteed to be atomic by POSIX. Reviewers: ilya-biryukov Subscribers: ioeric, MaskRay, jkorous, arphaman, jfb, cfe-commits Differential Revision: https://reviews.llvm.org/D55417 llvm-svn: 349348
* [clangd] Enable auto-index behind a flag.Sam McCall2018-11-261-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Ownership and configuration: The auto-index (background index) is maintained by ClangdServer, like Dynamic. (This means ClangdServer will be able to enqueue preamble indexing in future). For now it's enabled by a simple boolean flag in ClangdServer::Options, but we probably want to eventually allow injecting the storage strategy. New 'sync' command: In order to meaningfully test the integration (not just unit-test components) we need a way for tests to ensure the asynchronous index reads/writes occur before a certain point. Because these tests and assertions are few, I think exposing an explicit "sync" command for use in tests is simpler than allowing threading to be completely disabled in the background index (as we do for TUScheduler). Bugs: I fixed a couple of trivial bugs I found while testing, but there's one I can't. JSONCompilationDatabase::getAllFiles() may return relative paths, and currently we trigger an assertion that assumes they are absolute. There's no efficient way to resolve them (you have to retrieve the corresponding command and then resolve against its directory property). In general I think this behavior is broken and we should fix it in JSONCompilationDatabase and require CompilationDatabase::getAllFiles() to be absolute. Reviewers: kadircet Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, cfe-commits Differential Revision: https://reviews.llvm.org/D54894 llvm-svn: 347567
* [clangd] Auto-index watches global CDB for changes.Sam McCall2018-11-261-1/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Instead of receiving compilation commands, auto-index is triggered by just filenames to reindex, and gets commands from the global comp DB internally. This has advantages: - more of the work can be done asynchronously (fetching compilation commands upfront can be slow for large CDBs) - we get access to the CDB which can be used to retrieve interpolated commands for headers (useful in some cases where the original TU goes away) - fits nicely with the filename-only change observation from r347297 The interface to GlobalCompilationDatabase gets extended: when retrieving a compile command, the GCDB can optionally report the project the file belongs to. This naturally fits together with getCompileCommand: it's hard to implement one without the other. But because most callers don't care, I've ended up with an awkward optional-out-param-in-virtual method pattern - maybe there's a better one. This is the main missing integration point between ClangdServer and BackgroundIndex, after this we should be able to add an auto-index flag. Reviewers: ioeric, kadircet Subscribers: MaskRay, jkorous, arphaman, cfe-commits, ilya-biryukov Differential Revision: https://reviews.llvm.org/D54865 llvm-svn: 347538
* Introduce shard storage to auto-index.Kadir Cetinkaya2018-11-161-0/+112
Reviewers: sammccall, ioeric Reviewed By: sammccall Subscribers: llvm-commits, mgorny, Eugene.Zelenko, ilya-biryukov, jkorous, arphaman, cfe-commits Differential Revision: https://reviews.llvm.org/D54269 llvm-svn: 347038
OpenPOWER on IntegriCloud