summaryrefslogtreecommitdiffstats
path: root/clang/unittests/Tooling/Syntax
Commit message (Collapse)AuthorAgeFilesLines
* [Syntax] Merge overlapping top-level macros in TokenBufferSam McCall2020-06-101-4/+21
| | | | | | | | | | | | | | | | | | | Summary: Our previous definition of "top-level" was too informal, and didn't allow for overlapping macros that each directly produce expanded tokens. See D77507 for previous discussion. Fixes http://bugs.llvm.org/show_bug.cgi?id=45428 Reviewers: kadircet, vabridgers Subscribers: cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D77615 (cherry picked from commit d66afd6dde542dc373f87e07fe764c071fe20d76)
* [Syntax] Unset IsOriginal flag on nodes removed from the treeIlya Biryukov2020-01-141-0/+3
| | | | | | And add a corresponding test. Only nodes inside the TranslationUnit subtree can be marked as original, computeReplacements() relies on this.
* [Syntax] Mark synthesized nodes as modifiableIlya Biryukov2020-01-141-0/+18
| | | | | This was an oversight in the original patch. Also add corresponding tests.
* [Syntax] Build spanning SimpleDecalration for classes, structs, etcIlya Biryukov2020-01-031-18/+63
| | | | | | | When they are free-standing, e.g. `struct X;` or `struct X {};`. Although this complicates the common case (of free-standing class declarations), this ensures the less common case (e.g. `struct X {} a;`) are handled uniformly and produce similar syntax trees.
* [Syntax] Use a hash table to search for tokens by their locationIlya Biryukov2019-12-181-0/+40
| | | | | | | | This is both more efficient and avoids corner cases in `SourceManager::isBeforeInTranslationUnit`. The change is trivial and clearly a performance improvement on the hot path of building the syntax tree, so sending without review.
* [Syntax] Uppercase the first letter of the test name. NFCIlya Biryukov2019-12-181-1/+1
| | | | To match naming style of other tests.
* [Syntax] Allow to mutate syntax treesIlya Biryukov2019-12-182-5/+121
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch adds facilities to mutate the syntax trees and produce corresponding text replacements. The public interface of the syntax library now includes facilities to: 1. perform type-safe modifications of syntax trees, 2. compute textual replacements to apply the modifications, 3. create syntax trees not backed by the source code. For each of the three, we only add a few example transformations in this patch to illustrate the idea, support for more kinds of nodes and transformations will be done in follow-up patches. The high-level mutation operations are implemented on top of operations that allow to arbitrarily change the trees. They are considered to be implementation details and are not available to the users of the library. Reviewers: sammccall, gribozavr2 Reviewed By: gribozavr2 Subscribers: merge_guards_bot, mgorny, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D64573
* [Tooling/Syntax] Helpers to find spelled tokens touching a location.Sam McCall2019-12-121-0/+41
| | | | | | | | | | | | Summary: Useful when positions are used to target nodes, with before/after ambiguity. Reviewers: ilya-biryukov, kbobyrev Subscribers: cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D71356
* [Syntax] Build nodes for simple cases of top level declarationsIlya Biryukov2019-12-121-1/+184
| | | | | | | | | | | | | | | | Summary: More complicated nodes (e.g. template declarations) will be implemented in the follow-up patches. Reviewers: gribozavr2 Reviewed By: gribozavr2 Subscribers: merge_guards_bot, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D70856
* [Syntax] Show input code on tests failures. NFCIlya Biryukov2019-12-051-0/+2
|
* Reland [clangd] Rethink how SelectionTree deals with macros and #includes.Sam McCall2019-12-031-0/+15
| | | | | | This reverts commit 905b002c139f039a32ab9bf1fad63d745d12423f. Avoid tricky (and invalid) comparator for std::set.
* Revert "[clangd] Rethink how SelectionTree deals with macros and #includes."Sam McCall2019-11-291-15/+0
| | | | | | | | This reverts commit 19daa21f841ad45290c923689ee3d25198651a4c. It causes a bunch of failures on a bot that I've been unable to reproduce so far: http://45.33.8.238/mac/3308/step_7.txt
* [clangd] Rethink how SelectionTree deals with macros and #includes.Sam McCall2019-11-291-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The exclusive-claim model is successful at resolving conflicts over tokens between parent/child or siblings. However claims at the spelled-token level do the wrong thing for macro expansions, where siblings can be equally associated with the macro invocation. Moreover, any model that only uses the endpoints in a range can fail when a macro invocation occurs inside the node. To address this, we use the existing TokenBuffer in more depth. Claims are expressed in terms of expanded tokens, so there is no need to worry about macros, includes etc. Once we know which expanded tokens were claimed, they are mapped onto spelled tokens for hit-testing. This mapping is fairly flexible, currently the handling of macros is pretty simple (map macro args onto spellings, other macro expansions onto the macro name token). This mapping is in principle token-by-token for correctness (though there's some batching for performance). The aggregation of the selection enum is now more principled as we need to be able to aggregate several hit-test results together. For simplicity i removed the ability to determine selectedness of TUDecl. (That was originally implemented in 90a5bf92ff97b1, but doesn't seem to be very important or worth the complexity any longer). The expandedTokens(SourceLocation) helper could be added locally, but seems to make sense on TokenBuffer. Fixes https://github.com/clangd/clangd/issues/202 Fixes https://github.com/clangd/clangd/issues/126 Reviewers: hokein Subscribers: MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits, ilya-biryukov Tags: #clang Differential Revision: https://reviews.llvm.org/D70512
* [Syntax] Build SimpleDeclaration node that groups multiple declaratorsIlya Biryukov2019-11-291-25/+93
| | | | | | | | | | | | | | | | | | | | Summary: Also remove the temporary TopLevelDeclaration node and add UnknownDeclaration to represent other unknown nodes. See the follow-up change for building more top-level declarations. Adding declarators is also pretty involved and will be done in another follow-up patch. Reviewers: gribozavr2 Reviewed By: gribozavr2 Subscribers: merge_guards_bot, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D70787
* [Syntax] Add nodes for most common statementsIlya Biryukov2019-11-061-8/+305
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Most of the statements mirror the ones provided by clang AST. Major differences are: - expressions are wrapped into 'ExpressionStatement' instead of being a subclass of statement, - semicolons are always consumed by the leaf expressions (return, expression satement, etc), - some clang statements are not handled yet, we wrap those into an UnknownStatement class, which is not present in clang. We also define an 'Expression' and 'UnknownExpression' classes in order to produce 'ExpressionStatement' where needed. The actual implementation of expressions is not yet ready, it will follow later. Reviewers: sammccall Reviewed By: sammccall Subscribers: cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D63835
* [Syntax] Added function to get macro expansion tokens to TokenBuffer.Johan Vikstrom2019-08-201-0/+23
| | | | | | | | | | | | | | | | Summary: Returns the first token in every mapping where the token is an identifier. This API is required to be able to highlight macro expansions in clangd. Reviewers: hokein, ilya-biryukov Subscribers: kadircet, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D66470 llvm-svn: 369385
* [Clang] Migrate llvm::make_unique to std::make_uniqueJonas Devlieghere2019-08-142-4/+4
| | | | | | | | | | Now that we've moved to C++14, we no longer need the llvm::make_unique implementation from STLExtras.h. This patch is a mechanical replacement of (hopefully) all the llvm::make_unique instances across the monorepo. Differential revision: https://reviews.llvm.org/D66259 llvm-svn: 368942
* [Syntax] Do not add a node for 'eof' into the treeIlya Biryukov2019-08-061-9/+8
| | | | | | | | | | | | | | | | | | Summary: While useful as a sentinel value when iterating over tokens, having 'eof' in the tree, seems to do more harm than good. Reviewers: sammccall Reviewed By: sammccall Subscribers: javed.absar, kristof.beyls, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D64576 llvm-svn: 368062
* [Preprocessor] Always discard body of #define if we failed to parse itIlya Biryukov2019-08-011-0/+15
| | | | | | | | | | | | | | | | | | | | Summary: Preivously we would only discard it if we failed to parse parameter lists. If we do not consume the body, parser sees tokens inside directive. In turn, this leads to spurious diagnostics and a crash in TokenBuffer, see the added tests. Reviewers: sammccall Reviewed By: sammccall Subscribers: cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D65517 llvm-svn: 367530
* [Syntax] Move roles into a separate enumIlya Biryukov2019-07-091-4/+4
| | | | | | To align with reviewer's suggestions. llvm-svn: 365479
* Reland r365355: [Syntax] Introduce syntax treesIlya Biryukov2019-07-092-0/+161
| | | | | | With a fix to a PS4 buildbot crash. llvm-svn: 365466
* Revert rL365355 : [Syntax] Introduce syntax treesSimon Pilgrim2019-07-092-161/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: A tooling-focused alternative to the AST. This commit focuses on the memory-management strategy and the structure of the AST. More to follow later: - Operations to mutate the syntax trees and corresponding textual replacements. - Mapping between clang AST nodes and syntax tree nodes. - More node types corresponding to the language constructs. Reviewers: sammccall Reviewed By: sammccall Subscribers: llvm-commits, mgorny, cfe-commits Tags: #clang, #llvm Differential Revision: https://reviews.llvm.org/D61637 ........ Fixes buildbots which were crashing on SyntaxTests.exe llvm-svn: 365465
* [Syntax] Introduce syntax treesIlya Biryukov2019-07-082-0/+161
| | | | | | | | | | | | | | | | | | | | | | | | Summary: A tooling-focused alternative to the AST. This commit focuses on the memory-management strategy and the structure of the AST. More to follow later: - Operations to mutate the syntax trees and corresponding textual replacements. - Mapping between clang AST nodes and syntax tree nodes. - More node types corresponding to the language constructs. Reviewers: sammccall Reviewed By: sammccall Subscribers: llvm-commits, mgorny, cfe-commits Tags: #clang, #llvm Differential Revision: https://reviews.llvm.org/D61637 llvm-svn: 365355
* cmake: Add CLANG_LINK_CLANG_DYLIB optionTom Stellard2019-07-031-2/+6
| | | | | | | | | | | | | | | | Summary: Setting CLANG_LINK_CLANG_DYLIB=ON causes clang tools to link against libclang_shared.so instead of the individual component libraries. Reviewers: mgorny, beanz, smeenai, phosek, sylvestre.ledru Subscribers: arphaman, cfe-commits, llvm-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D63503 llvm-svn: 365092
* [Syntax] Do not glue multiple empty PP expansions to a single mappingIlya Biryukov2019-06-241-1/+4
| | | | | | | | | | | | | | | | | | | | | Summary: This change makes sure we have a single mapping for each macro expansion, even if the result of expansion was empty. To achieve that, we take information from PPCallbacks::MacroExpands into account. Previously we relied only on source locations of expanded tokens. Reviewers: sammccall Reviewed By: sammccall Subscribers: cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D62953 llvm-svn: 364236
* [Syntax] Fix a crash when dumping empty token bufferIlya Biryukov2019-06-191-0/+8
| | | | llvm-svn: 363801
* [Syntax] Add a helper to find expansion by its first spelled tokenIlya Biryukov2019-06-181-0/+78
| | | | | | | | | | | | | | | | Summary: Used in clangd for a code tweak that expands a macro. Reviewers: sammccall Reviewed By: sammccall Subscribers: kadircet, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D62954 llvm-svn: 363698
* [Syntax] Do not depend on llvm targets for Syntax tests. NFCIlya Biryukov2019-06-041-1/+0
| | | | | | They are not required and only slow down the build. llvm-svn: 362530
* Add missing newline at end of fileDuncan P. N. Exon Smith2019-05-251-1/+1
| | | | llvm-svn: 361708
* Reland r361148 with a fix to the buildbot failure.Ilya Biryukov2019-05-222-0/+674
| | | | | | | Reverted in r361377. Also reland the '.gn' files (reverted in r361389). llvm-svn: 361391
* Revert r361148 "[Syntax] Introduce TokenBuffer, start clangToolingSyntax ↵Russell Gallop2019-05-222-674/+0
| | | | | | | | | | | | library" Also reverted r361264 "[Syntax] Rename TokensTest to SyntaxTests. NFC" which built on it. This is because there were hitting an assert on bots http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast llvm-svn: 361377
* [Syntax] Rename TokensTest to SyntaxTests. NFCIlya Biryukov2019-05-211-2/+2
| | | | | | | To be more consistent with conventions used in the codebase. The new name will be a better fit when more bits of the syntax library land. llvm-svn: 361264
* [Syntax] Introduce TokenBuffer, start clangToolingSyntax libraryIlya Biryukov2019-05-202-0/+674
Summary: TokenBuffer stores the list of tokens for a file obtained after preprocessing. This is a base building block for syntax trees, see [1] for the full proposal on syntax trees. This commits also starts a new sub-library of ClangTooling, which would be the home for the syntax trees and syntax-tree-based refactoring utilities. [1]: https://lists.llvm.org/pipermail/cfe-dev/2019-February/061414.html Reviewers: gribozavr, sammccall Reviewed By: sammccall Subscribers: mgrang, riccibruno, Eugene.Zelenko, mgorny, jdoerfert, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D59887 llvm-svn: 361148
OpenPOWER on IntegriCloud