summaryrefslogtreecommitdiffstats
path: root/clang-tools-extra/clangd/index
Commit message (Collapse)AuthorAgeFilesLines
...
* [clangd] Add priorities to background index queue, extract to separate classSam McCall2019-07-113-136/+198
| | | | | | | | | | | | Reviewers: kadircet Subscribers: mgorny, ilya-biryukov, MaskRay, jkorous, arphaman, jfb, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D64560 llvm-svn: 365773
* [clangd] Reland rL365634Kadir Cetinkaya2019-07-111-3/+7
| | | | | | | | | This was reverted in rL365678, the failure was due to YAML parsing of compile_commands.json. Converting backslashes to forward slashes to fix the issue in unittest. llvm-svn: 365748
* Revert Recommit "[CommandLine] Remove OptionCategory and SubCommand caches ↵Haojian Wu2019-07-111-1/+1
| | | | | | | | | | from the Option class." This reverts r365675 (git commit 43d75f977853c3ec891a440c362b2df183a211b5) The patch causes a crash in SupportTests (CommandLineTest.AliasesWithArguments). llvm-svn: 365742
* Revert "[clangd] Filter out non-governed files from broadcast"Matthew Voss2019-07-101-7/+3
| | | | | | | | | | This reverts commit d5214dfa7b5650745eaeb102857c9e90adb16137. It's causing failures, both in our local CI and the PS4 Windows bot. http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/builds/26872/steps/test/logs/stdio llvm-svn: 365678
* Recommit "[CommandLine] Remove OptionCategory and SubCommand caches from the ↵Don Hinton2019-07-101-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | Option class." Previously reverted in 364141 due to buildbot breakage, and fixed here by making GeneralCategory global a ManagedStatic. Summary: This change processes `OptionCategory`s and `SubCommand`s as they are seen instead of caching them in the Option class and processing them later. Doing so simplifies the work needed to be done by the Global parser and significantly reduces the size of the Option class to a mere 64 bytes. Removing the `OptionCategory` cache saved 24 bytes, and removing the `SubCommand` cache saved an additional 48 bytes, for a total of a 72 byte reduction. Reviewed By: serge-sans-paille Tags: #llvm, #clang Differential Revision: https://reviews.llvm.org/D62105 llvm-svn: 365675
* [clangd] Filter out non-governed files from broadcastKadir Cetinkaya2019-07-101-3/+7
| | | | | | | | | | | | | | | | Summary: This also turns off implicit discovery of additional compilation databases. Reviewers: sammccall Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D64247 llvm-svn: 365634
* [clangd] Rewrite of logic to rebuild the background index serving structures.Sam McCall2019-07-094-50/+255
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Previously it was rebuilding every 5s by default, which was much too frequent in the long run - the goal was to provide an early build. There were also some bugs. There were also some bugs, and a dedicated thread was used in production but not tested. - rebuilds are triggered by #TUs built, rather than time. This should scale more sensibly to fast vs slow machines. - there are two separate indexed-TU thresholds to trigger index build: 5 TUs for the first build, 100 for subsequent rebuilds. - rebuild is always done on the regular indexing threads, and is affected by blockUntilIdle. This means unit/lit tests run the production configuration. - fixed a bug where we'd rebuild after attempting to load shards, even if there were no shards. - the BackgroundIndexTests don't really test the subtleties of the rebuild policy (for determinism, we call blockUntilIdle, so rebuild-on-idle is enough to pass the tests). Instead, we expose the rebuilder as a separate class and have fine-grained tests for it. Reviewers: kadircet Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, jfb, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D64291 llvm-svn: 365531
* [clangd] Use xxhash instead of SHA1 for background index file digests.Sam McCall2019-07-084-10/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Currently SHA1 is about 10% of our CPU, this patch reduces it to ~1%. xxhash is a well-defined (stable) non-cryptographic hash optimized for fast checksums (like crc32). Collisions shouldn't be a problem, despite the reduced length: - for actual file content (used to invalidate bg index shards), there are only two versions that can collide (new shard and old shard). - for file paths in bg index shard filenames, we would need 2^32 files with the same filename to expect a collision. Imperfect hashing may reduce this a bit but it's well beyond what's plausible. This will invalidate shards on disk (as usual; I bumped the version), but this time the filenames are changing so the old files will stick around :-( So this is more expensive than the usual bump, but would be good to land before the v9 branch when everyone will start using bg index. Reviewers: kadircet Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D64306 llvm-svn: 365311
* [clangd] Encapsulate fields in dex token. NFCSam McCall2019-07-082-6/+7
| | | | llvm-svn: 365288
* [clangd] Fix breakage on gcc 5.4Kadir Cetinkaya2019-07-041-1/+1
| | | | llvm-svn: 365140
* [clangd] Make HadErrors part of background index's internal stateKadir Cetinkaya2019-07-042-56/+66
| | | | | | | | | | | | Reviewers: sammccall Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D64147 llvm-svn: 365123
* [clangd] Add HadErrors field into shardsKadir Cetinkaya2019-07-043-8/+15
| | | | | | | | | | | | Reviewers: sammccall Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D64133 llvm-svn: 365122
* [clangd] Store hash of command line in index shards.Kadir Cetinkaya2019-07-043-1/+73
| | | | | | | | | | | | | | Summary: This is to enable cache invalidation when command line flags changes. Reviewers: sammccall Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D64018 llvm-svn: 365121
* [clangd] Also cache failures while indexingKadir Cetinkaya2019-07-044-16/+33
| | | | | | | | | | | | | | | | Summary: Clangd currently doesn't cache any indexing failures, which results in retrying those failed files even if their contents haven't changed. Reviewers: sammccall Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D63986 llvm-svn: 365120
* [clangd] Collect the refs when the main file is header.Haojian Wu2019-07-022-6/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Previously, we only collect refs of the symbols which are declared in the preamble and referenced in the main file, it works well when the main file is .cpp file. However, when the main file is .h file (when opening a .h file in the editor), we don't collect refs of the symbol declared in this file, so we miss these refs in our dynamic index. A typical scenario: 1. Open Foo.h (which contains class Foo) 2. Open Foo.cpp, call find references for Foo And we only get refs from Foo.cpp. Reviewers: ilya-biryukov Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D63818 llvm-svn: 364893
* Cleanup: llvm::bsearch -> llvm::partition_point after r364719Fangrui Song2019-06-301-2/+2
| | | | llvm-svn: 364720
* [ADT] Implement llvm::bsearch() with std::partition_point()Fangrui Song2019-06-301-4/+4
| | | | | | | | | | | | | | | | | | | Summary: Delete the begin-end form because the standard std::partition_point can be easily used as a replacement. The ranges-style llvm::bsearch will be renamed to llvm::partition_point in the next clean-up patch. The name "bsearch" doesn't meet people's expectation because in C: > If two or more members compare equal, which member is returned is unspecified. Reviewed By: sammccall Differential Revision: https://reviews.llvm.org/D63718 llvm-svn: 364719
* BitStream reader: propagate errorsJF Bastien2019-06-261-3/+3
| | | | | | | | | | | | | | | | | | | | | | The bitstream reader handles errors poorly. This has two effects: * Bugs in file handling (especially modules) manifest as an "unexpected end of file" crash * Users of clang as a library end up aborting because the code unconditionally calls `report_fatal_error` The bitstream reader should be more resilient and return Expected / Error as soon as an error is encountered, not way late like it does now. This patch starts doing so and adopting the error handling where I think it makes sense. There's plenty more to do: this patch propagates errors to be minimally useful, and follow-ups will propagate them further and improve diagnostics. https://bugs.llvm.org/show_bug.cgi?id=42311 <rdar://problem/33159405> Differential Revision: https://reviews.llvm.org/D63518 llvm-svn: 364464
* [clangd] Add include-mapping for C symbols.Haojian Wu2019-06-213-7/+17
| | | | | | | | | | | | | | | | | | | Summary: This resolves the issue of introducing c++-style includes for C files. - refactor the gen_std.py, make it reusable for parsing C symbols. - add a language mode to the mapping method to use different mapping for C and C++ files. Reviewers: kadircet Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, jfb, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D63270 llvm-svn: 364044
* [clangd] Perform merge for main file symbols.Haojian Wu2019-06-171-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: Previously, we randomly pick one main file symbol in dynamic index, we may loose the ideal symbol (with definition location) in the index. It fixes the issue where sometimes we fail to go to the symbol definition, see: 1. call go-to-decl on Foo in Foo.cpp 2. jump to Foo.h, call go-to-def on Foo in Foo.h we can't go back to Foo.cpp -- because we open Foo.cpp, Foo.h in clangd, both files have Foo symbol (one with def&decl, one with decl only), we randomely choose one. Reviewers: kadircet Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D63425 llvm-svn: 363568
* Fix gcc-05.4 bot failures caused by in r363481 "[clangd] Index API and ↵Don Hinton2019-06-161-1/+2
| | | | | | | | | | implementations for relations" Use std::make_tuple instead of initializer list to make gcc-5.4 happy. See https://reviews.llvm.org/D62839 for details. llvm-svn: 363504
* [clangd] Index API and implementations for relationsNathan Ridge2019-06-1515-54/+265
| | | | | | | | | | | | | | | | | Summary: This builds on the relations support added in D59407, D62459, and D62471 to expose relations in SymbolIndex and its implementations. Reviewers: kadircet Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D62839 llvm-svn: 363481
* [clangd] SymbolCollector support for relationsNathan Ridge2019-06-042-4/+52
| | | | | | | | | | | | | | | | | | | Summary: The only relation currently collected is RelationBaseOf, because this is all we need for type hierarchy subtypes. Additional relations can be collected in the future as the need arises. This patch builds on D59407 and D62459. Reviewers: kadircet Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D62471 llvm-svn: 362467
* [clangd] Serialization support for RelationSlabNathan Ridge2019-06-033-4/+153
| | | | | | | | | | | | | | Summary: This builds on D59407 to provide YAML and RIFF serialization support. Reviewers: kadircet Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D62459 llvm-svn: 362353
* [clangd] Add RelationSlabNathan Ridge2019-06-033-0/+129
| | | | | | | | | | | | | | | | Summary: RelationSlab is a new index data structure that stores relations between symbols. Reviewers: kadircet Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D59407 llvm-svn: 362352
* [clangd] clang-format SymbolCollector.cppNathan Ridge2019-05-301-4/+3
| | | | llvm-svn: 362176
* [clangd] Fix buildbot error.Haojian Wu2019-05-291-0/+4
| | | | llvm-svn: 361960
* [clangd] Remove the whitelist std symbols in CanonicalIncludes.Haojian Wu2019-05-291-17/+0
| | | | | | | | | | | | | | Summary: These symbols have been included via StdSymbolMap.inc. Reviewers: sammccall Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D62527 llvm-svn: 361952
* [clangd] Bump index version and get rid of wrong assertionKadir Cetinkaya2019-05-092-2/+4
| | | | | | | | | | | | | | | | | | | | Summary: After rL360344, BackgroundIndex expects symbols with zero refcounts. Therefore existing index files are no longer valid. Assertion regarding finding target of a reference was wrong, since background-index might still be going on or we might've loaded only some part of the shards and might be missing the declaring shards for the symbol. Reviewers: ilya-biryukov Subscribers: MaskRay, jkorous, arphaman, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D61734 llvm-svn: 360349
* [clangd] Count number of references while merging RefSlabs inside FileIndexKadir Cetinkaya2019-05-094-19/+49
| | | | | | | | | | | | | | | | | | | | | | | Summary: For counting number of references clangd was relying on merging every duplication of a symbol. Unfortunately this does not apply to FileIndex(and one of its users' BackgroundIndex), since we get rid of duplication by simply dropping symbols coming from non-canonical locations. So only one or two(coming from canonical declaration header and defined source file, if exists) replications of the same symbol reaches merging step. This patch changes reference counting logic to rather count number of different RefSlabs a given SymbolID exists. Reviewers: ilya-biryukov Subscribers: ioeric, MaskRay, jkorous, mgrang, arphaman, jdoerfert, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D59481 llvm-svn: 360344
* [clangd] Use AsyncTaskRunner in BackgroundIndex instead of std::threadIlya Biryukov2019-05-092-6/+7
| | | | | | | | | | | | | | | | | | Summary: To unify the way we create threads in clangd. This should simplify landing D50993. Reviewers: kadircet Reviewed By: kadircet Subscribers: MaskRay, jkorous, arphaman, jfb, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D61724 llvm-svn: 360332
* [clangd] Fix header-guard check for include insertion, and don't index ↵Sam McCall2019-05-034-42/+62
| | | | | | | | | | | | | | | | | | | | | header guards. Summary: Both of these attempt to check whether a header guard exists while parsing the file. However the file is only marked as guarded once clang finishes processing it. We defer the checks and work until SymbolCollector::finish(). This is ugly and ad-hoc, deferring *all* work might be cleaner. Reviewers: kadircet Subscribers: ilya-biryukov, MaskRay, jkorous, mgrang, arphaman, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D61442 llvm-svn: 359880
* Reapply r359778: [clangd] Fix code completion of macros defined in the ↵Sam McCall2019-05-021-0/+1
| | | | | | | | | preamble region of the main file. The bad assert has been removed, and updateOutOfDateIdentifier has been guarded. This reverts commit r359796. llvm-svn: 359799
* Revert rL359778 : [clangd] Fix code completion of macros defined in the ↵Simon Pilgrim2019-05-021-1/+0
| | | | | | | | | | | | | | | | | | | | | | preamble region of the main file. Summary: This is a tricky case (we baked the assumption that symbols come from the preamble xor mainfile pretty deeply) and the fix is a bit of a hack: We look at the code to guess the macro names, and deserialize them from the preamble "by hand". Reviewers: ilya-biryukov Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D60937 ........ Fix buildbots http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast/builds/47684/ llvm-svn: 359796
* [clangd] Fix code completion of macros defined in the preamble region of the ↵Sam McCall2019-05-021-0/+1
| | | | | | | | | | | | | | | | | | | | main file. Summary: This is a tricky case (we baked the assumption that symbols come from the preamble xor mainfile pretty deeply) and the fix is a bit of a hack: We look at the code to guess the macro names, and deserialize them from the preamble "by hand". Reviewers: ilya-biryukov Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D60937 llvm-svn: 359778
* [clangd] Delete an unused declarationFangrui Song2019-05-011-3/+0
| | | | llvm-svn: 359674
* [clangd] Fix serialization logic for Origin and Flags.Kadir Cetinkaya2019-04-291-2/+2
| | | | llvm-svn: 359470
* [clangd] Optimize "don't include me" check.Sam McCall2019-04-252-7/+34
| | | | | | | | | | | | | | | | Summary: llvm::Regex is really slow, and regex evaluation during preamble indexing was showing up as 25% on a profile of clangd in a codebase with large preambles. Reviewers: ilya-biryukov Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D61120 llvm-svn: 359214
* [clangd] Use llvm::set_thread_priority in background-indexKadir Cetinkaya2019-04-182-14/+28
| | | | | | | | | | | | Reviewers: gribozavr Subscribers: krytarowski, ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, jfb, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D60865 llvm-svn: 358664
* [clangd] Recognize "don't include me directly" pattern, and suppress include ↵Sam McCall2019-04-172-47/+69
| | | | | | | | | | | | | | | | | | | | | | | | | | | insertion. Summary: Typically used with umbrella headers, e.g. GTK: #if !defined (__GTK_H_INSIDE__) && !defined (GTK_COMPILATION) #error "Only <gtk/gtk.h> can be included directly." #endif Heuristic is fairly conservative, a quick code search over github showed a fair number of hits and few/no false positives. (Not all were umbrella headers, but I'd be happy avoiding include insertion for all of them). We may want to relax the heuristic later to catch more cases. Reviewers: ioeric Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D60815 llvm-svn: 358605
* [clangd] Include insertion: require header guards, drop other heuristics, ↵Sam McCall2019-04-173-51/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | treat .def like .inc. Summary: We do have some reports of include insertion behaving badly in some codebases. Requiring header guards both makes sense in principle, and is likely to disable this "nice-to-have" feature in codebases where headers don't follow the expected pattern. With this we can drop some other heuristics, such as looking at file extensions to detect known non-headers - implementation files have no guards. One wrinkle here is #import - objc headers may not have guards because they're intended to be used via #import. If the header is the main file or is #included, we won't collect locations - merge should take care of this if we see the file #imported somewhere. Seems likely to be OK. Headers which have a canonicalization (stdlib, IWYU) are exempt from this check. *.inc files continue to be handled by looking up to the including file. This patch also adds *.def here - tablegen wants this pattern too. In terms of code structure, the division between SymbolCollector and CanonicalIncludes has shifted: SymbolCollector is responsible for more. This is because SymbolCollector has all the SourceManager/HeaderSearch access needed for checking for guards, and we interleave these checks with the *.def checks in a loop (potentially). We could hand all the info into CanonicalIncludes and put the logic there if that's preferable. Reviewers: ioeric Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D60316 llvm-svn: 358571
* [clangd] lower_bound -> bsearch, NFCSam McCall2019-04-172-8/+8
| | | | llvm-svn: 358561
* [clangd] Fallback to OrigD when SLoc is invalidKadir Cetinkaya2019-04-151-0/+6
| | | | | | | | | | | | | | | | | Summary: Some implicit/built-in decls lack the source location information. Fallback to OrigD that we've seen in the source code instead of the canonical one in those cases. Reviewers: sammccall Subscribers: cfe-commits, arphaman, jkorous, MaskRay, ioeric, ilya-biryukov Tags: #clang Differential Revision: https://reviews.llvm.org/D60689 llvm-svn: 358413
* [clangd] Bump clangd-index version for TemplateArgument changesKadir Cetinkaya2019-04-151-1/+1
| | | | llvm-svn: 358383
* [clangd] Show template argument list in workspacesymbols and documentsymbols ↵Kadir Cetinkaya2019-04-122-18/+0
| | | | | | | | | | | | | | | | | | responses Summary: Last part of re-landing rC356541. Puts TemplateArgumentsList into responses of the above mentioned two requests. Reviewers: ioeric, ilya-biryukov Subscribers: MaskRay, jkorous, arphaman, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D59641 llvm-svn: 358274
* [clangd] Add TemplateArgumentList into SymbolKadir Cetinkaya2019-04-124-1/+12
| | | | | | | | | | | | | | | | | | | | | Summary: Part of re-landing rC356541 with D59599. Changes the way we store template arguments, previous patch was storing them inside Name field of Symbol. Which was violating the assumption: ```Symbol::Scope+Symbol::Name == clang::clangd::printQualifiedName``` which was made in multiple places inside codebase. This patch instead moves those arguments into their own field. Currently the field is meant to be human-readable, can be made structured if need be. Reviewers: ioeric, ilya-biryukov, gribozavr Subscribers: MaskRay, jkorous, arphaman, jdoerfert, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D59640 llvm-svn: 358273
* [clangd] Use identifiers in file as completion candidates when build is not ↵Eric Liu2019-04-112-5/+6
| | | | | | | | | | | | | | | | | | | | | | ready. Summary: o Lex the code to get the identifiers and put them into a "symbol" index. o Adds a new completion mode without compilation/sema into code completion workflow. o Make IncludeInserter work even when no compile command is present, by avoiding inserting non-verbatim headers. Reviewers: sammccall Reviewed By: sammccall Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, jdoerfert, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D60126 llvm-svn: 358159
* [clangd] Fix non-indexing of builtin functions like printf when the TU is CSam McCall2019-04-101-3/+3
| | | | llvm-svn: 358098
* [clangd] Stop passing around PCHContainerOperations, just create it in ↵Sam McCall2019-04-041-3/+2
| | | | | | place. NFC llvm-svn: 357689
* [clangd] Use capacity() instead of size() in RefSlab::bytes()Ilya Biryukov2019-04-021-1/+1
| | | | | | | | | | | | | | | | Patch by Nathan Ridge. Reviewers: gribozavr Reviewed By: gribozavr Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D60040 llvm-svn: 357454
OpenPOWER on IntegriCloud