summaryrefslogtreecommitdiffstats
path: root/clang-tools-extra/clangd/index
Commit message (Collapse)AuthorAgeFilesLines
...
* [clangd] DexIndex implementation prototypeKirill Bobyrev2018-08-203-1/+244
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch is a proof-of-concept Dex index implementation. It has several flaws, which don't allow replacing static MemIndex yet, such as: * Not being able to handle queries of small size (less than 3 symbols); a way to solve this is generating trigrams of smaller size and having such incomplete trigrams in the index structure. * Speed measurements: while manually editing files in Vim and requesting autocompletion gives an impression that the performance is at least comparable with the current static index, having actual numbers is important because we don't want to hurt the users and roll out slow code. Eric (@ioeric) suggested that we should only replace MemIndex as soon as we have the evidence that this is not a regression in terms of performance. An approach which is likely to be successful here is to wait until we have benchmark library in the LLVM core repository, which is something I have suggested in the LLVM mailing lists, received positive feedback on and started working on. I will add a dependency as soon as the suggested patch is out for a review (currently there's at least one complication which is being addressed by https://github.com/google/benchmark/pull/649). Key performance improvements for iterators are sorting by cost and the limit iterator. * Quality measurements: currently, boosting iterator and two-phase lookup stage are not implemented, without these the quality is likely to be worse than the current implementation can yield. Measuring quality is tricky, but another suggestion in the offline discussion was that the drop-in replacement should only happen after Boosting iterators implementation (and subsequent query enhancement). The proposed changes do not affect Clangd functionality or performance, `DexIndex` is only used in unit tests and not in production code. Reviewed by: ioeric Differential Revision: https://reviews.llvm.org/D50337 llvm-svn: 340175
* [clangd] Simplify the code using UniqueStringSaver, NFC.Haojian Wu2018-08-202-12/+9
| | | | llvm-svn: 340161
* [clangd] NFC: Cleanup Dex Iterator comments and simplify testsKirill Bobyrev2018-08-202-9/+9
| | | | | | | | | | | | | | | | | | | | | Proposed changes: * Cleanup comments in `clangd/index/dex/Iterator.h`: Vim's `gq` formatting added redundant spaces instead of newlines in few places * Few comments in `OrIterator` are wrong * Use `EXPECT_TRUE(Condition)` instead of `EXPECT_THAT(Condition, true)` (same with `EXPECT_FALSE`) * Don't expose `dump()` method to the public by misplacing `private:` This patch does not affect functionality. Reviewed by: ioeric Differential Revision: https://reviews.llvm.org/D50956 llvm-svn: 340157
* [clangd] Add missing lock in the lookup.Haojian Wu2018-08-201-0/+1
| | | | | | | | | | Reviewers: ioeric Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits Differential Revision: https://reviews.llvm.org/D50960 llvm-svn: 340156
* [clangd] Implement TRUE IteratorKirill Bobyrev2018-08-202-0/+43
| | | | | | | | | | | This patch introduces TRUE Iterator which efficiently handles posting lists containing all items within `[0, Size)` range. Reviewed by: ioeric Differential Revision: https://reviews.llvm.org/D50955 llvm-svn: 340155
* [clangd] NFC: Improve Dex Iterators debugging traitsKirill Bobyrev2018-08-162-3/+14
| | | | | | | | | | | | This patch improves `dex::Iterator` string representation by incorporating the information about the element which is currently being pointed to by the `DocumentIterator`. Reviewed by: ioeric Differential Revision: https://reviews.llvm.org/D50689 llvm-svn: 339877
* NFC: Enforce good formatting across multiple clang-tools-extra filesKirill Bobyrev2018-08-149-13/+22
| | | | | | | | | | | This patch improves readability of multiple files in clang-tools-extra and enforces LLVM Coding Guidelines. Reviewed by: ioeric Differential Revision: https://reviews.llvm.org/D50707 llvm-svn: 339687
* Fix MSVC 'std::min: no matching overloaded function found' error.Simon Pilgrim2018-08-131-1/+2
| | | | llvm-svn: 339557
* [clangd] Generate incomplete trigrams for the Dex indexKirill Bobyrev2018-08-133-34/+82
| | | | | | | | | | | | | | This patch handles trigram generation "short" identifiers and queries. Trigram generator produces incomplete trigrams for short names so that the same query iterator API can be used to match symbols which don't have enough symbols to form a trigram and correctly handle queries which also are not sufficient for generating a full trigram. Reviewed by: ioeric Differential revision: https://reviews.llvm.org/D50517 llvm-svn: 339548
* [clangd] Allow consuming limited number of itemsKirill Bobyrev2018-08-102-5/+7
| | | | | | | | | | | | | | | | | This patch modifies `consume` function to allow retrieval of limited number of symbols. This is the "cheap" implementation of top-level limiting iterator. In the future we would like to have a complete limit iterator implementation to insert it into the query subtrees, but in the meantime this version would be enough for a fully-functional proof-of-concept Dex implementation. Reviewers: ioeric, ilya-biryukov Reviewed by: ioeric Differential Revision: https://reviews.llvm.org/D50500 llvm-svn: 339426
* [clangd] Share getSymbolID implementation.Haojian Wu2018-08-071-9/+8
| | | | | | | | | | | | Summary: And remove all duplicated implementation. Reviewers: ioeric Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits Differential Revision: https://reviews.llvm.org/D50375 llvm-svn: 339116
* [clangd] Index Interfaces for XrefsHaojian Wu2018-08-066-3/+77
| | | | | | | | | | | | | | Summary: This is the first step of implementing Xrefs in clangd: - add index interfaces, and related data structures. Reviewers: sammccall Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, cfe-commits Differential Revision: https://reviews.llvm.org/D49658 llvm-svn: 339011
* [clangd] Make SymbolLocation => bool conversion explicitly.Haojian Wu2018-08-011-1/+8
| | | | | | | | | | | | | | | Summary: The implicit bool conversion could happen superisingly, e.g. when checking `if (Loc1 == Loc2)`, the compiler will convert SymbolLocation to bool before comparing (because we don't define operator `==` for SymbolLocation). Reviewers: sammccall Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, cfe-commits Differential Revision: https://reviews.llvm.org/D49657 llvm-svn: 338517
* [clangd] Return Dex IteratorsKirill Bobyrev2018-07-272-0/+396
| | | | | | | | | | | | | | | The original Dex Iterators patch (https://reviews.llvm.org/rL338017) caused problems for Clang 3.6 and Clang 3.7 due to the compiler bug which prevented inferring template parameter (`Size`) in create(And|Or)? functions. It was reverted in https://reviews.llvm.org/rL338054. In this revision the mentioned helper functions were replaced with variadic templated versions. Proposed changes were tested on multiple compiler versions, including Clang 3.6 which originally caused the failure. llvm-svn: 338116
* Revert Clangd Dex Iterators patchKirill Bobyrev2018-07-262-387/+0
| | | | | | | | | | | | This reverts two revisions: * https://reviews.llvm.org/rL338017 * https://reviews.llvm.org/rL338028 They caused crash for Clang 3.6 & Clang 3.7 buildbots, it was reported by Jeremy Morse. llvm-svn: 338054
* [clangd] Fix (most) naming warnings from clang-tidy. NFCIlya Biryukov2018-07-263-3/+3
| | | | llvm-svn: 338021
* [clangd] Proof-of-concept query iterators for Dex symbol indexKirill Bobyrev2018-07-262-0/+387
| | | | | | | | | | | | | | | | | | | | | | | | | | This patch introduces three essential types of query iterators: `DocumentIterator`, `AndIterator`, `OrIterator`. It provides a convenient API for query tree generation and serves as a building block for the next generation symbol index - Dex. Currently, many optimizations are missed to improve code readability and to serve as the reference implementation. Potential improvements are briefly mentioned in `FIXME`s and will be addressed in the following patches. Dex RFC in the mailing list: http://lists.llvm.org/pipermail/clangd-dev/2018-July/000022.html Iterators, their applications and potential extensions are explained in detail in the design proposal: https://docs.google.com/document/d/1C-A6PGT6TynyaX4PXyExNMiGmJ2jL1UwV91Kyx11gOI/edit#heading=h.903u1zon9nkj Reviewers: ioeric, sammccall, ilya-biryukov Subscribers: cfe-commits, klimek, jfb, mgrang, mgorny, MaskRay, jkorous, arphaman Differential Revision: https://reviews.llvm.org/D49546 llvm-svn: 338017
* [clangd] Introduce Dex symbol index search tokensKirill Bobyrev2018-07-253-0/+306
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch introduces the core building block of the next-generation Clangd symbol index - Dex. Search tokens are the keys in the inverted index and represent a characteristic of a specific symbol: examples of search token types (Token Namespaces) are * Trigrams - these are essential for unqualified symbol name fuzzy search * Scopes for filtering the symbols by the namespace * Paths, e.g. these can be used to uprank symbols defined close to the edited file This patch outlines the generic for such token namespaces, but only implements trigram generation. The intuition behind trigram generation algorithm is that each extracted trigram is a valid sequence for Fuzzy Matcher jumps, proposed implementation utilize existing FuzzyMatcher API for segmentation and trigram extraction. However, trigrams generation algorithm for the query string is different from the previous one: it simply yields sequences of 3 consecutive lowercased valid characters (letters, digits). Dex RFC in the mailing list: http://lists.llvm.org/pipermail/clangd-dev/2018-July/000022.html The trigram generation techniques are described in detail in the proposal: https://docs.google.com/document/d/1C-A6PGT6TynyaX4PXyExNMiGmJ2jL1UwV91Kyx11gOI/edit#heading=h.903u1zon9nkj Reviewers: sammccall, ioeric, ilya-biryukovA Subscribers: cfe-commits, klimek, mgorny, MaskRay, jkorous, arphaman Differential Revision: https://reviews.llvm.org/D49591 llvm-svn: 337901
* [clangd] Upgrade logging facilities with levels and formatv.Sam McCall2018-07-111-3/+2
| | | | | | | | | | | | | | | | | | | | | | Summary: log() is split into four functions: - elog()/log()/vlog() have different severity levels, allowing filtering - dlog() is a lazy macro which uses LLVM_DEBUG - it logs to the logger, but conditionally based on -debug-only flag and is omitted in release builds All logging functions use formatv-style format strings now, e.g: log("Could not resolve URI {0}: {1}", URI, Result.takeError()); Existing log sites have been split between elog/log/vlog by best guess. This includes a workaround for passing Error to formatv that can be simplified when D49170 or similar lands. Subscribers: ilya-biryukov, javed.absar, ioeric, MaskRay, jkorous, cfe-commits Differential Revision: https://reviews.llvm.org/D49008 llvm-svn: 336785
* [clangd] Make sure macro information exists before increasing usage count.Eric Liu2018-07-091-6/+4
| | | | llvm-svn: 336581
* [clangd] Support indexing MACROs.Eric Liu2018-07-092-25/+121
| | | | | | | | | | | | | | Summary: This is not enabled in the global-symbol-builder or dynamic index yet. Reviewers: sammccall Reviewed By: sammccall Subscribers: ilya-biryukov, MaskRay, jkorous, cfe-commits Differential Revision: https://reviews.llvm.org/D49028 llvm-svn: 336553
* [clangd] Make SymbolOrigin an enum class, rather than a plain enum.Sam McCall2018-07-063-5/+15
| | | | | | | I never intended to define namespace pollution like clangd::AST, clangd::Unknown etc. Oops! llvm-svn: 336431
* [clangd] Track origins of symbols (various indexes, Sema).Sam McCall2018-07-056-0/+32
| | | | | | | | | | | | Summary: Surface it in the completion items C++ API, and when a flag is set. Reviewers: ioeric Subscribers: ilya-biryukov, MaskRay, jkorous, cfe-commits Differential Revision: https://reviews.llvm.org/D48938 llvm-svn: 336309
* [clangd] Always remove dots before converting paths to URIs in symbol collector.Eric Liu2018-06-251-1/+2
| | | | llvm-svn: 335458
* [clangd] More precise representation of symbol names/labels in the index.Sam McCall2018-06-225-43/+34
| | | | | | | | | | | | | | | | | | | | | | Summary: Previously, the strings matched LSP completion pretty closely. The completion label was a single string, for instance. This made implementing completion itself easy but makes it hard to use the names in other way, e.g. pretty-printed name in synthesized documentation/hover. It also limits our introspection into completion items, which can only be as precise as the indexed symbols. This change is a prerequisite to improvements to overload bundling which need to inspect e.g. signature structure. Reviewers: ioeric Subscribers: ilya-biryukov, MaskRay, jkorous, cfe-commits Differential Revision: https://reviews.llvm.org/D48475 llvm-svn: 335360
* [clangd] Expose qualified symbol names in CompletionItem (C++ structure ↵Eric Liu2018-06-221-12/+1
| | | | | | | | | | | | | | | | | | | | only, no json). Summary: The qualified name can be used to match a completion item to its corresponding symbol. This can be useful for tools that measure code completion quality. Qualified names are not precise for identifying symbols; we need to figure out a better way to identify completion items. Reviewers: sammccall Reviewed By: sammccall Subscribers: ilya-biryukov, MaskRay, jkorous, cfe-commits Differential Revision: https://reviews.llvm.org/D48425 llvm-svn: 335334
* [clangd] Remove FilterText from the index.Sam McCall2018-06-225-10/+2
| | | | | | | | | | | | | | | | | Summary: It's almost always identical to Name, and in fact we never used it (we used name instead). The only case where they differ is objc method selectors (foo: vs foo:bar:). We can live with the latter for both name and filterText, so I've made that change too. Reviewers: ioeric Subscribers: ilya-biryukov, MaskRay, jkorous, cfe-commits Differential Revision: https://reviews.llvm.org/D48375 llvm-svn: 335321
* [clangd] Expose 'shouldCollectSymbol' helper from SymbolCollector.Eric Liu2018-06-212-47/+53
| | | | | | | | | | | | | | Summary: This allows tools to examine symbols that would be collected in a symbol index. For example, a tool that measures index-based completion quality would be interested in references to symbols that are collected in the index. Reviewers: sammccall Reviewed By: sammccall Subscribers: ilya-biryukov, MaskRay, jkorous, cfe-commits Differential Revision: https://reviews.llvm.org/D48418 llvm-svn: 335218
* [clangd] Customizable URI schemes for dynamic index.Eric Liu2018-06-152-3/+16
| | | | | | | | | | | | | | | | | Summary: This allows dynamic index to have consistent URI schemes with the static index which can have customized URI schemes, which would make file proximity scoring based on URIs easier. Reviewers: sammccall Reviewed By: sammccall Subscribers: ilya-biryukov, MaskRay, jkorous, cfe-commits Differential Revision: https://reviews.llvm.org/D47931 llvm-svn: 334809
* [clangd] Support proximity paths in index fuzzy find.Eric Liu2018-06-121-0/+3
| | | | | | | | | | Reviewers: sammccall Subscribers: ilya-biryukov, MaskRay, jkorous, cfe-commits Differential Revision: https://reviews.llvm.org/D47937 llvm-svn: 334485
* [clangd] Add "member" symbols to the indexMarc-Andre Laperle2018-06-055-20/+35
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: This adds more symbols to the index: - member variables and functions - enum constants in scoped enums The code completion behavior should remain intact but workspace symbols should now provide much more useful symbols. Other symbols should be considered such as the ones in "main files" (files not being included) but this can be done separately as this introduces its fair share of problems. Signed-off-by: Marc-Andre Laperle <marc-andre.laperle@ericsson.com> Reviewers: ioeric, sammccall Reviewed By: ioeric, sammccall Subscribers: hokein, sammccall, jkorous, klimek, ilya-biryukov, jkorous-apple, ioeric, MaskRay, cfe-commits Differential Revision: https://reviews.llvm.org/D44954 llvm-svn: 334017
* [clangd] Avoid indexing decls associated with friend decls.Eric Liu2018-06-042-0/+19
| | | | | | | | | | | | | | | | | | | | Summary: These decls are sometime used as the canonical declarations (e.g. for go-to-def), which seems to be bad. - friend decls that are not definitions should be ignored for indexing purposes - this means they should never be selected as canonical decl - if the friend decl is the only decl, then the symbol should not be indexed Reviewers: sammccall Reviewed By: sammccall Subscribers: mgorny, klimek, ilya-biryukov, MaskRay, jkorous, cfe-commits Differential Revision: https://reviews.llvm.org/D47623 llvm-svn: 333885
* [clangd] Remove the dead offset fields in Symbol.Haojian Wu2018-06-041-6/+0
| | | | | | | | | | Reviewers: sammccall Subscribers: klimek, ilya-biryukov, ioeric, MaskRay, jkorous, cfe-commits Differential Revision: https://reviews.llvm.org/D47699 llvm-svn: 333882
* [clangd] clang-format the source code. NFCIlya Biryukov2018-05-303-23/+23
| | | | llvm-svn: 333537
* [clangd] Build index on preamble changes instead of the AST changesIlya Biryukov2018-05-242-10/+18
| | | | | | | | | | | | | | | | | | | | | Summary: This is more efficient and avoids data races when reading files that come from the preamble. The staleness can occur when reading a file from disk that changed after the preamble was built. This can lead to crashes, e.g. when parsing comments. We do not to rely on symbols from the main file anyway, since any info that those provide can always be taken from the AST. Reviewers: ioeric, sammccall Reviewed By: ioeric Subscribers: malaperle, klimek, javed.absar, MaskRay, jkorous, cfe-commits Differential Revision: https://reviews.llvm.org/D47272 llvm-svn: 333196
* [clangd] Serve comments for headers decls from dynamic index onlyIlya Biryukov2018-05-241-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: To fix a crash in code completion that occurrs when reading doc comments from files that were updated after the preamble was computed. In that case, the files on disk could've been changed and we can't rely on finding the comment text with the same range anymore. The current workaround is to not provide comments from the headers at all and rely on the dynamic index instead. A more principled solution would be to store contents of the files read inside the preamble, but it is way harder to implement properly, given that it would definitely increase the sizes of the preamble. Together with D47272, this should fix all preamble-related crashes we're aware of. Reviewers: sammccall Reviewed By: sammccall Subscribers: klimek, ioeric, MaskRay, jkorous, cfe-commits Differential Revision: https://reviews.llvm.org/D47274 llvm-svn: 333189
* [clangd] Skip .inc headers when canonicalizing header #include.Eric Liu2018-05-243-18/+49
| | | | | | | | | | | | | | | | Summary: This assumes that .inc files are supposed to be included via headers that include them. Reviewers: sammccall Reviewed By: sammccall Subscribers: klimek, ilya-biryukov, MaskRay, jkorous, cfe-commits Differential Revision: https://reviews.llvm.org/D47187 llvm-svn: 333188
* [clangd] Correctly handle IWYU prama with verbatim #include header.Eric Liu2018-05-221-2/+3
| | | | llvm-svn: 332959
* [clangd] Retrieve minimally formatted comment text in completion.Ilya Biryukov2018-05-161-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Previous implementation used to extract brief text from doxygen comments. Brief text parsing slows down completion and is not suited for non-doxygen comments. This commit switches to providing comments that mimic the ones originally written in the source code, doing minimal reindenting and removing the comments markers to make the output more user-friendly. It means we lose support for doxygen-specific features, e.g. extracting brief text, but provide useful results for non-doxygen comments. Switching the doxygen support back is an option, but I suggest to see whether the current approach gives more useful results. Reviewers: sammccall, hokein, ioeric Reviewed By: sammccall Subscribers: klimek, MaskRay, jkorous, cfe-commits Differential Revision: https://reviews.llvm.org/D45999 llvm-svn: 332459
* [clangd] Filter out private proto symbols in SymbolCollector.Eric Liu2018-05-161-0/+43
| | | | | | | | | | | | | | | | | Summary: This uses heuristics to identify private proto symbols. For example, top-level symbols whose name contains "_" are considered private. These symbols are not expected to be used by users. Reviewers: ilya-biryukov, malaperle Reviewed By: ilya-biryukov Subscribers: sammccall, klimek, MaskRay, jkorous, cfe-commits Differential Revision: https://reviews.llvm.org/D46751 llvm-svn: 332456
* [clangd] Log error message instead write to errs(). NFCEric Liu2018-05-151-2/+1
| | | | llvm-svn: 332366
* [clangd] Incorporate #occurrences in scoring code complete results.Sam McCall2018-05-033-1/+15
| | | | | | | | | | | | Summary: needs tests Reviewers: ilya-biryukov Subscribers: klimek, ioeric, MaskRay, jkorous, cfe-commits Differential Revision: https://reviews.llvm.org/D46183 llvm-svn: 331457
* [clangd] Using index for GoToDefinition.Haojian Wu2018-04-302-14/+12
| | | | | | | | | | | | | | | | | | | Summary: This patch adds index support for GoToDefinition -- when we don't get the definition from local AST, we query our index (Static&Dynamic) index to get it. Since we currently collect top-level symbol in the index, it doesn't support all cases (e.g. class members), we will extend the index to include more symbols in the future. Reviewers: sammccall Subscribers: klimek, ilya-biryukov, jkorous-apple, ioeric, MaskRay, cfe-commits Differential Revision: https://reviews.llvm.org/D45717 llvm-svn: 331189
* [clangd] Also use UTF-16 in index position.Haojian Wu2018-04-301-7/+3
| | | | | | | | | | Reviewers: sammccall Subscribers: klimek, ilya-biryukov, ioeric, MaskRay, jkorous, cfe-commits Differential Revision: https://reviews.llvm.org/D46258 llvm-svn: 331168
* [clangd] Add "str()" method to SymbolID.Haojian Wu2018-04-252-0/+10
| | | | | | | | | | | | | | Summary: This is a convenient function when we try to get std::string of SymbolID. Reviewers: ioeric Subscribers: klimek, ilya-biryukov, MaskRay, jkorous, cfe-commits Differential Revision: https://reviews.llvm.org/D46065 llvm-svn: 330835
* [clangd] Minor fixes for C++ standard library header mapping.Eric Liu2018-04-251-7/+24
| | | | llvm-svn: 330803
* [clangd] Implementation of workspace/symbol requestMarc-Andre Laperle2018-04-231-10/+2
| | | | | | | | | | | | | | | | | | | | | | | Summary: This is a basic implementation of the "workspace/symbol" request which is used to find symbols by a string query. Since this is similar to code completion in terms of result, this implementation reuses the "fuzzyFind" in order to get matches. For now, the scoring algorithm is the same as code completion and improvements could be done in the future. The index model doesn't contain quite enough symbols for this to cover common symbols like methods, enum class enumerators, functions in unamed namespaces, etc. The index model will be augmented separately to achieve this. Reviewers: sammccall, ilya-biryukov Reviewed By: sammccall Subscribers: jkorous, hokein, simark, sammccall, klimek, mgorny, ilya-biryukov, mgrang, jkorous-apple, ioeric, MaskRay, cfe-commits Differential Revision: https://reviews.llvm.org/D44882 llvm-svn: 330637
* [index] Fix methods that take a shared_ptr to just take a reference.Benjamin Kramer2018-04-231-3/+3
| | | | | | | There is no ownership here, passing a shared_ptr just adds confusion. No functionality change intended. llvm-svn: 330595
* [clangd] Fix label and snippet for funcs in the indexIlya Biryukov2018-04-141-8/+3
| | | | | | | This is a follow-up to r330004 to fix functions with required template args, e.g. std::make_shared. llvm-svn: 330087
* [clangd] Match AST and Index label for template SymbolsIlya Biryukov2018-04-131-1/+16
| | | | | | | | | | | | | | | | Summary: Previsouly, class completions items from the index were missing template parameters in both the snippet and the label. Reviewers: sammccall, hokein Reviewed By: sammccall Subscribers: klimek, jkorous-apple, ioeric, MaskRay, cfe-commits Differential Revision: https://reviews.llvm.org/D45482 llvm-svn: 330004
OpenPOWER on IntegriCloud