summaryrefslogtreecommitdiffstats
path: root/clang-tools-extra/clangd/index
Commit message (Collapse)AuthorAgeFilesLines
...
* [clangd] Get rid of Decls parameter in indexMainDecls. NFCEric Liu2018-09-182-12/+10
| | | | | | It's already available in ParsedAST. llvm-svn: 342473
* [clangd] Merge ClangdServer::DynamicIndex into FileIndex. NFC.Eric Liu2018-09-182-55/+99
| | | | | | | | | | | | | | | | | | | | Summary: FileIndex now provides explicit interfaces for preamble and main file updates. This avoids growing parameter list when preamble and main symbols diverge further (e.g. D52078). This also gets rid of the hack in `indexAST` that inferred main file index based on `TopLevelDecls`. Also separate `indexMainDecls` from `indexAST`. Reviewers: sammccall Reviewed By: sammccall Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits Differential Revision: https://reviews.llvm.org/D52222 llvm-svn: 342460
* [clangd] dexp tool uses llvm::cl to parse its flags.Sam McCall2018-09-181-70/+135
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: We can use cl::ResetCommandLineParser() to support different types of command-lines, as long as we're careful about option lifetimes. (I tried using subcommands, but the error messages were bad) I found a mostly-reasonable pattern to isolate the fiddly parts. Added -scope and -limit flags to the `find` command to demonstrate. (Note that scope support seems to be broken in dex?) Fixed symbol lookup to parse symbol IDs. Caveats: - with command help (e.g. `find -help`), you also get some spam about required arguments. This is a bug in llvm::cl, which prints these to errs() rather than the designated stream. Reviewers: kbobyrev Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits Differential Revision: https://reviews.llvm.org/D51989 llvm-svn: 342456
* [clangd] Adapt API change after 342451.Eric Liu2018-09-181-1/+1
| | | | llvm-svn: 342452
* [clangd] Get rid of AST matchers in SymbolCollector. NFCEric Liu2018-09-171-13/+36
| | | | | | | | | | Reviewers: ilya-biryukov, kadircet Subscribers: MaskRay, jkorous, arphaman, cfe-commits Differential Revision: https://reviews.llvm.org/D52089 llvm-svn: 342362
* [clangd] Introduce PostingList interfaceKirill Bobyrev2018-09-136-92/+149
| | | | | | | | | | | | | | | | This patch abstracts `PostingList` interface and reuses existing implementation. It will be used later to test different `PostingList` representations. No functionality change is introduced, this patch is mostly refactoring so that the following patches could focus on functionality while not being too hard to review. Reviewed By: sammccall, ioeric Differential Revision: https://reviews.llvm.org/D51982 llvm-svn: 342155
* [clangd] Fix Dexp buildKirill Bobyrev2018-09-131-1/+1
| | | | | | %s/MaxCandidateCount/Limit/g after rL342138. llvm-svn: 342143
* [clangd] Cleanup FuzzyFindRequest filtering limit semanticsKirill Bobyrev2018-09-134-16/+16
| | | | | | | | | | | As discussed during D51860 review, it is better to use `llvm::Optional` here as it has clear semantics which reflect intended behavior. Reviewed By: sammccall Differential Revision: https://reviews.llvm.org/D52028 llvm-svn: 342138
* [clangd] Don't create child AND and OR iterators with one posting listKirill Bobyrev2018-09-131-3/+7
| | | | | | | | | | | | | | `AND( AND( Child ) ... )` -> `AND( Child ... )` `AND( OR( Child ) ... )` -> `AND( Child ... )` This simple optimization results in 5-6% performance improvement in the benchmark with 2000 serialized `FuzzyFindRequest`s. Reviewed By: ilya-biryukov Differential Revision: https://reviews.llvm.org/D52016 llvm-svn: 342124
* [clangd] Add missing clangBasic target_link_librariesHeejin Ahn2018-09-121-0/+1
| | | | | | Without this, builds with `-DSHARED_LIB=ON` fail. llvm-svn: 342037
* [clangd] Implement a Proof-of-Concept tool for symbol index explorationKirill Bobyrev2018-09-122-0/+176
| | | | | | | | Reviewed By: sammccall, ilya-biryukov Differential Revision: https://reviews.llvm.org/D51628 llvm-svn: 342025
* [clangd] NFC: Use uint32_t for FuzzyFindRequest limitsKirill Bobyrev2018-09-112-12/+10
| | | | | | | | Reviewed By: ioeric Differential Revision: https://reviews.llvm.org/D51860 llvm-svn: 341921
* [clangd] Unbreak buildbots after r341802Kirill Bobyrev2018-09-101-1/+1
| | | | | Solution: use std::move when returning result from toJSON(...). llvm-svn: 341832
* [clangd] Implement FuzzyFindRequest JSON (de)serializationKirill Bobyrev2018-09-102-1/+32
| | | | | | | | | | | | JSON (de)serialization of `FuzzyFindRequest` might be useful for both D51090 and D51628. Also, this allows precise logging of the fuzzy find requests. Reviewed By: sammccall Differential Revision: https://reviews.llvm.org/D51852 llvm-svn: 341802
* [clangd] Add symbol slab size to index memory consumption estimatesKirill Bobyrev2018-09-105-14/+32
| | | | | | | | | | | | | Currently, `SymbolIndex::estimateMemoryUsage()` returns the "overhead" estimate, i.e. the estimate of the Index data structure excluding backing data (such as Symbol Slab and Reference Slab). This patch propagates information about paired data size where necessary. Reviewed By: ioeric, sammccall Differential Revision: https://reviews.llvm.org/D51539 llvm-svn: 341800
* [clangd] NFC: Rename DexIndex to DexKirill Bobyrev2018-09-103-24/+23
| | | | | | | | | | Also, cleanup some redundant includes. Reviewed By: sammccall Differential Revision: https://reviews.llvm.org/D51774 llvm-svn: 341784
* [clangd] Make advanceTo() faster on Posting ListsKirill Bobyrev2018-09-101-1/+4
| | | | | | | | | | | | If the current element is already beyond advanceTo()'s DocID, just return instead of doing binary search. This simple optimization saves up to 6-7% performance, Reviewed By: ilya-biryukov Differential Revision: https://reviews.llvm.org/D51802 llvm-svn: 341781
* [clangd] Canonicalize include paths in clangd.Eric Liu2018-09-0712-24/+24
| | | | | | Get rid of "../" and "../../". llvm-svn: 341645
* [clangd] Add "Deprecated" field to Symbol and CodeCompletion.Eric Liu2018-09-067-12/+59
| | | | | | | | | | | | | | Summary: Also set "deprecated" field in LSP CompletionItem. Reviewers: sammccall, kadircet Reviewed By: sammccall Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits Differential Revision: https://reviews.llvm.org/D51724 llvm-svn: 341576
* [clangd] Fix Dex initializationKirill Bobyrev2018-09-062-3/+11
| | | | | | | | | | | | This patch sets URI schemes of Dex to SymbolCollector's default schemes in case callers tried to pass empty list of schemes. This was the case for initialization in Clangd main and was a reason of incorrect behavior. Also, it fixes a bug with missed `continue;` after spotting invalid URI scheme conversion. llvm-svn: 341552
* [clangd] NFC: Use TopN instead of std::priority_queueKirill Bobyrev2018-09-061-11/+7
| | | | | | | | | | | | | | Quality.cpp defines a structure for convenient storage of Top N items, it should be used instead of the `std::priority_queue` with slightly obscure semantics. This patch does not affect functionality. Reviewed By: sammccall Differential Revision: https://reviews.llvm.org/D51676 llvm-svn: 341544
* [clangd] NFC: mark single-parameter constructors explicitKirill Bobyrev2018-09-061-4/+4
| | | | | | | | | | Code health: prevent implicit conversions to user-defined types. Reviewed By: sammccall Differential Revision: https://reviews.llvm.org/D51690 llvm-svn: 341543
* [clangd] Implement proximity path boosting for DexKirill Bobyrev2018-09-065-42/+171
| | | | | | | | | | | | | | | This patch introduces `PathURI` Search Token kind and utilizes it to uprank symbols which are defined in files with small distance to the directory where the fuzzy find request is coming from (e.g. files user is editing). Reviewed By: ioeric Reviewers: ioeric, sammccall Differential Revision: https://reviews.llvm.org/D51481 llvm-svn: 341542
* [clangd] Set SymbolID for sema macros so that they can be merged with index ↵Eric Liu2018-09-061-10/+6
| | | | | | | | | | | | macros. Reviewers: sammccall Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits Differential Revision: https://reviews.llvm.org/D51688 llvm-svn: 341534
* [clangd] make zlib compression optional for binary formatSam McCall2018-09-051-11/+29
| | | | llvm-svn: 341465
* [clangd] Fix buildbot failures on older compilers from r341375Sam McCall2018-09-051-3/+3
| | | | llvm-svn: 341451
* [clangd] Load static index asynchronously, add tracing.Sam McCall2018-09-042-4/+11
| | | | | | | | | | | | | | Summary: Like D51475 but simplified based on recent patches. While here, clarify that loadIndex() takes a filename, not file content. Reviewers: ioeric Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits Differential Revision: https://reviews.llvm.org/D51638 llvm-svn: 341376
* [clangd] Define a compact binary serialization fomat for symbol slab/index.Sam McCall2018-09-045-46/+472
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This is intended to replace the current YAML format for general use. It's ~10x more compact than YAML, and ~40% more compact than gzipped YAML: llvmidx.riff = 20M, llvmidx.yaml = 272M, llvmidx.yaml.gz = 32M It's also simpler/faster to read and write. The format is a RIFF container (chunks of (type, size, data)) with: - a compressed string table - simple binary encoding of symbols (with varints for compactness) It can be extended to include occurrences, Dex posting lists, etc. There's no rich backwards-compatibility scheme, but a version number is included so we can detect incompatible files and do ad-hoc back-compat. Alternatives considered: - compressed YAML or JSON: bulky and slow to load - llvm bitstream: confusing model and libraries are hard to use. My attempt produced slightly larger files, and the code was longer and slower. - protobuf or similar: would be really nice (esp for back-compat) but the dependency is a big hassle - ad-hoc binary format without a container: it seems clear we're going to add posting lists and occurrences here, and that they will benefit from sharing a string table. The container makes it easy to debug these pieces in isolation, and make them optional. Reviewers: ioeric Subscribers: mgorny, ilya-biryukov, MaskRay, jkorous, mgrang, arphaman, kadircet, cfe-commits Differential Revision: https://reviews.llvm.org/D51585 llvm-svn: 341375
* [clangd] NFC: Change quality type to floatKirill Bobyrev2018-09-042-2/+2
| | | | | | | | Reviewed by: sammccall Differential Revision: https://reviews.llvm.org/D51636 llvm-svn: 341374
* [clangd] Move buildStaticIndex() to SymbolYAMLKirill Bobyrev2018-09-042-4/+24
| | | | | | | | | | | | | | `buildStaticIndex()` is used by two other tools that I'm building, now it's useful outside of `tool/ClangdMain.cpp`. Also, slightly refactor the code while moving it to the different source file. Reviewed By: sammccall Differential Revision: https://reviews.llvm.org/D51626 llvm-svn: 341369
* [clangd] SymbolOccurrences -> Refs and cleanupSam McCall2018-09-0412-250/+212
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: A few things that I noticed while merging the SwapIndex patch: - SymbolOccurrences and particularly SymbolOccurrenceSlab are unwieldy names, and these names appear *a lot*. Ref, RefSlab, etc seem clear enough and read/format much better. - The asymmetry between SymbolSlab and RefSlab (build() vs freeze()) is confusing and irritating, and doesn't even save much code. Avoiding RefSlab::Builder was my idea, but it was a bad one; add it. - DenseMap<SymbolID, ArrayRef<Ref>> seems like a reasonable compromise for constructing MemIndex - and means many less wasted allocations than the current DenseMap<SymbolID, vector<Ref*>> for FileIndex, and none for slabs. - RefSlab::find() is not actually used for anything, so we can throw away the DenseMap and keep the representation much more compact. - A few naming/consistency fixes: e.g. Slabs,Refs -> Symbols,Refs. Reviewers: ioeric Subscribers: ilya-biryukov, MaskRay, jkorous, mgrang, arphaman, kadircet, cfe-commits Differential Revision: https://reviews.llvm.org/D51605 llvm-svn: 341368
* [clangd] Fix index-twice regression from r341242Sam McCall2018-09-031-1/+0
| | | | llvm-svn: 341337
* [clangd] Factor out the data-swapping functionality from MemIndex/DexIndex.Sam McCall2018-09-038-299/+233
| | | | | | | | | | | | | | | | | | | | | | | Summary: This is now handled by a wrapper class SwapIndex, so MemIndex/DexIndex can be immutable and focus on their job. Old and busted: I have a MemIndex, which holds a shared_ptr<vector<Symbol*>>, which keeps the symbol slab alive. I update by calling build(shared_ptr<vector<Symbol*>>). New hotness: I have a SwapIndex, which holds a unique_ptr<SymbolIndex>, which holds a MemIndex, which holds a shared_ptr<void>, which keeps backing data alive. I update by building a new MemIndex and calling SwapIndex::reset(). Reviewers: kbobyrev, ioeric Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, mgrang, arphaman, kadircet, cfe-commits Differential Revision: https://reviews.llvm.org/D51422 llvm-svn: 341318
* [clangd] Support multiple #include headers in one symbol.Eric Liu2018-09-035-14/+64
| | | | | | | | | | | | | | | | | | | | Summary: Currently, a symbol can have only one #include header attached, which might not work well if the symbol can be imported via different #includes depending on where it's used. This patch stores multiple #include headers (with # references) for each symbol, so that CodeCompletion can decide which include to insert. In this patch, code completion simply picks the most popular include as the default inserted header. We also return all possible includes and their edits in the `CodeCompletion` results. Reviewers: sammccall Reviewed By: sammccall Subscribers: mgrang, ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits Differential Revision: https://reviews.llvm.org/D51291 llvm-svn: 341304
* [clangd] Fix many typos. NFCFangrui Song2018-09-018-9/+9
| | | | llvm-svn: 341273
* [clangd] Implement findOccurrences interface in dynamic index.Haojian Wu2018-08-317-29/+167
| | | | | | | | | | | | | | | | | | | | Summary: Implement the interface in - FileIndex - MemIndex - MergeIndex Depends on https://reviews.llvm.org/D50385. Reviewers: sammccall, ilya-biryukov Reviewed By: sammccall Subscribers: mgrang, ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits Differential Revision: https://reviews.llvm.org/D51279 llvm-svn: 341242
* [clangd] Flatten out Symbol::Details. It was ill-conceived, sorry.Sam McCall2018-08-317-107/+37
| | | | | | | | | | Reviewers: ioeric Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits Differential Revision: https://reviews.llvm.org/D51504 llvm-svn: 341211
* [clangd] Collect symbol occurrences in SymbolCollector.Haojian Wu2018-08-314-22/+192
| | | | | | | | | | SymbolCollector will be used for two cases: - collect Symbol type only, used for indexing preamble AST. - collect Symbol and SymbolOccurrences, used for indexing main AST. For finding local references from the AST, we will implement it in other ways. llvm-svn: 341208
* [NFC] Cleanup DexKirill Bobyrev2018-08-314-30/+33
| | | | | | | | | | | | | * Use consistent assertion messages in iterators implementations * Silence a bunch of clang-tidy warnings: use `emplace_back` instead of `push_back` where possible, make sure arguments have the same name in header and implementation file, use for loop over ranges where possible Reviewed by: ioeric Differential Revision: https://reviews.llvm.org/D51528 llvm-svn: 341190
* [clangd] Remove UB introduced in rL341057Kirill Bobyrev2018-08-301-1/+0
| | | | llvm-svn: 341066
* [clangd] Implement iterator costKirill Bobyrev2018-08-302-3/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch introduces iterator cost concept to improve the performance of Dex query iterators (mainly, AND iterator). Benchmarks show that the queries become ~10% faster. Before ``` ------------------------------------------------------- Benchmark Time CPU Iteration ------------------------------------------------------- DexAdHocQueries 5883074 ns 5883018 ns 117 DexRealQ 959904457 ns 959898507 ns 1 ``` After ``` ------------------------------------------------------- Benchmark Time CPU Iteration ------------------------------------------------------- DexAdHocQueries 5238403 ns 5238361 ns 130 DexRealQ 873275207 ns 873269453 ns 1 ``` Reviewed by: sammccall Differential Revision: https://reviews.llvm.org/D51310 llvm-svn: 341057
* [clangd] Use TRUE iterator instead of complete posting listKirill Bobyrev2018-08-271-4/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Stop using `$$$` (empty) trigram and generating a posting list with all items. Since TRUE iterator is already implemented and correctly inserted when there are no real trigram posting lists, this is a valid transformation. Benchmarks show that this simple change allows ~30% speedup on dataset of real completion queries. Before ``` ------------------------------------------------------- Benchmark Time CPU Iterations ------------------------------------------------------- DexAdHocQueries 5640321 ns 5640265 ns 120 DexRealQ 939835603 ns 939830296 ns 1 ``` After ``` ------------------------------------------------------- Benchmark Time CPU Iterations ------------------------------------------------------- DexAdHocQueries 3452014 ns 3451987 ns 203 DexRealQ 667455912 ns 667455750 ns 1 ``` Reviewed by: ilya-biryukov Differential Revision: https://reviews.llvm.org/D51287 llvm-svn: 340729
* [clangd] Implement LIMIT iteratorKirill Bobyrev2018-08-243-36/+79
| | | | | | | | | | | | | This patch introduces LIMIT iterator, which is very important for improving the quality of search query. LIMIT iterators can be applied on top of BOOST iterators to prevent populating query request with a huge number of low-quality symbols. Reviewed by: sammccall Differential Revision: https://reviews.llvm.org/D51029 llvm-svn: 340605
* [clangd] Speculative code completion index request before Sema is run.Eric Liu2018-08-241-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | Summary: For index-based code completion, send an asynchronous speculative index request, based on the index request for the last code completion on the same file and the filter text typed before the cursor, before sema code completion is invoked. This can reduce the code completion latency (by roughly latency of sema code completion) if the speculative request is the same as the one generated for the ongoing code completion from sema. As a sequence of code completions often have the same scopes and proximity paths etc, this should be effective for a number of code completions. Trace with speculative index request:{F6997544} Reviewers: hokein, ilya-biryukov Reviewed By: ilya-biryukov Subscribers: javed.absar, jfb, MaskRay, jkorous, arphaman, kadircet, cfe-commits Differential Revision: https://reviews.llvm.org/D50962 llvm-svn: 340604
* [clangd] Log memory usage of DexIndex and MemIndexKirill Bobyrev2018-08-248-0/+48
| | | | | | | | | | | | This patch prints information about built index size estimation to verbose logs. This is useful for optimizing memory usage of DexIndex and comparisons with MemIndex. Reviewed by: sammccall Differential Revision: https://reviews.llvm.org/D51154 llvm-svn: 340601
* [clangd] Get rid of regexes in CanonicalIncludesIlya Biryukov2018-08-222-679/+687
| | | | | | | | | | | | | | Summary: Replace them with suffix mappings. Reviewers: ioeric, kbobyrev Reviewed By: ioeric Subscribers: MaskRay, jkorous, arphaman, jfb, kadircet, cfe-commits Differential Revision: https://reviews.llvm.org/D51088 llvm-svn: 340410
* [clangd] Implement BOOST iteratorKirill Bobyrev2018-08-223-13/+113
| | | | | | | | | | | | | | | This patch introduces BOOST iterator - a substantial block for efficient and high-quality symbol retrieval. The concept of boosting allows performing computationally inexpensive scoring on the query side so that the final (expensive) scoring can only be applied on the items with the highest preliminary score while eliminating the need to score too many items. Reviewed by: ilya-biryukov Differential Revision: https://reviews.llvm.org/D50970 llvm-svn: 340409
* [clangd] Make FileIndex aware of the main fileIlya Biryukov2018-08-222-10/+24
| | | | | | | | | | | | | | | | | | | | | Summary: It was previously only indexing the preamble decls. The new implementation will index both the preamble and the main AST and report both sets of symbols, preferring the ones from the main AST whenever the symbol is present in both. The symbols in the main AST slab always store all information available in the preamble symbols, possibly adding more, e.g. definition locations. Reviewers: hokein, ioeric Reviewed By: ioeric Subscribers: kadircet, MaskRay, jkorous, arphaman, cfe-commits Differential Revision: https://reviews.llvm.org/D50889 llvm-svn: 340404
* [clangd] Cleanup after D50897Kirill Bobyrev2018-08-221-1/+1
| | | | | | | The wrong diff that was uploaded to Phabricator was building the wrong index. llvm-svn: 340388
* [clangd] Allow using experimental Dex indexKirill Bobyrev2018-08-214-12/+30
| | | | | | | | | | | This patch adds hidden Clangd flag ("use-dex-index") which replaces (currently) default `MemIndex` with `DexIndex` for the static index. Reviewed by: ioeric Differential Revision: https://reviews.llvm.org/D50897 llvm-svn: 340262
OpenPOWER on IntegriCloud