summaryrefslogtreecommitdiffstats
path: root/clang/lib/Lex/Lexer.cpp
Commit message (Collapse)AuthorAgeFilesLines
...
* use 'features' instead of 'PP->getLangOptions'.Chris Lattner2010-08-301-8/+8
| | | | llvm-svn: 112490
* In Microsoft compatibility mode, don't parse the exponent as part ofDouglas Gregor2010-08-301-1/+10
| | | | | | | the pp-number in a hexadecimal floating point literal, from Francois Pichet! Fixes PR7968. llvm-svn: 112481
* Implement C++0x user-defined string literals.Alexis Hunt2010-08-291-2/+29
| | | | | | | | | | The extra data stored on user-defined literal Tokens is stored in extra allocated memory, which is managed by the PreprocessorLexer because there isn't a better place to put it that makes sure it gets deallocated, but only after it's used up. My testing has shown no significant slowdown as a result, but independent testing would be appreciated. llvm-svn: 112458
* Introduce a preprocessor code-completion hook for contexts where weDouglas Gregor2010-08-251-6/+22
| | | | | | | expect "natural" language and should not provide any completions, e.g., comments, string literals, #error. llvm-svn: 112054
* Introduce basic code-completion support for preprocessor directives,Douglas Gregor2010-08-241-18/+18
| | | | | | e.g., after a "#" we'll suggest #if, #ifdef, etc. llvm-svn: 111943
* Don't emit end-of-file diagnostics like "unterminated conditional" orDouglas Gregor2010-08-121-4/+7
| | | | | | "unterminated string" when we're performing code completion. llvm-svn: 110933
* Random temporary string cleanup.Benjamin Kramer2010-08-111-1/+1
| | | | llvm-svn: 110807
* Use precompiled preambles for in-process code completion.Douglas Gregor2010-08-091-2/+16
| | | | llvm-svn: 110596
* Introduce basic support for loading a precompiled preamble whileDouglas Gregor2010-07-261-2/+13
| | | | | | | | | | | | | | | | | | | | | | | reparsing an ASTUnit. When saving a preamble, create a buffer larger than the actual file we're working with but fill everything from the end of the preamble to the end of the file with spaces (so the lexer will quickly skip them). When we load the file, create a buffer of the same size, filling it with the file and then spaces. Then, instruct the lexer to start lexing after the preamble, therefore continuing the parse from the spot where the preamble left off. It's now possible to perform a simple preamble build + parse (+ reparse) with ASTUnit. However, one has to disable a bunch of checking in the PCH reader to do so. That part isn't committed; it will likely be handled with some other kind of flag (e.g., -fno-validate-pch). As part of this, fix some issues with null termination of the memory buffers created for the preamble; we were trying to explicitly NULL-terminate them, even though they were also getting implicitly NULL terminated, leading to excess warnings about NULL characters in source files. llvm-svn: 109445
* Improve performance during cursor traversal when a region of interestDouglas Gregor2010-07-221-0/+53
| | | | | | | | | | | | | | | | | | | | | is present. Rather than using clang_getCursorExtent(), which requires us to lex the token at the ending position to determine its length. Then, we'd be comparing [a, b) source ranges that cover the characters in the range rather than the normal behavior for Clang's source ranges, which covers the tokens in the range. However, relexing causes us to read the source file (which may come from a precompiled header), which is rather unfortunate and affects performance. In the new scheme, we only use Clang-style source ranges that cover the tokens in the range. At the entry points where this matters (clang_annotateTokens, clang_getCursor), we make sure to move source locations to the start of the token. Addresses most of <rdar://problem/8049381>. llvm-svn: 109134
* Introduce a new lexer function to compute the "preamble" of a file,Douglas Gregor2010-07-201-0/+125
| | | | | | | | | which is the part of the file that contains all of the initial comments, includes, and preprocessor directives that occur before any of the actual code. Added a new -print-preamble cc1 action that is only used for testing. llvm-svn: 108913
* fix PR4499, patch by Kyle Dean!Chris Lattner2010-07-071-24/+16
| | | | llvm-svn: 107836
* simpler fix for rdar://8044135 - escaped newlines have alreadyChris Lattner2010-05-301-10/+7
| | | | | | been processed, so they don't have to be tip-toed around. llvm-svn: 105182
* Improve our handling of NULL after an escaping '\' in a stringDouglas Gregor2010-05-301-2/+7
| | | | | | literal. Fixes <rdar://problem/8044135>. llvm-svn: 105181
* Improve code completion in failure cases in two ways:Douglas Gregor2010-05-251-0/+3
| | | | | | | | | | | 1) Suppress diagnostics as soon as we form the code-completion token, so we don't get any error/warning spew from the early end-of-file. 2) If we consume a code-completion token when we weren't expecting one, go into a code-completion recovery path that produces the best results it can based on the context that the parser is in. llvm-svn: 104585
* robustify the conflict marker stuff. Don't add 7 twice, which wouldChris Lattner2010-05-171-1/+2
| | | | | | | | | | | | | | | | | | | | | make it miss (invalid) things like: <<<<<<< >>>>>>> and crash if <<<<<<< was at the end of the line. When we find a >>>>>>> that is not at the end of the line, make sure to reset Pos so we don't crash on something like: <<<<<<< >>>>>>> This isn't worth making testcases for, since each would require a new file. rdar://7987078 - signal 11 compiling "<<<<<<<<<<" llvm-svn: 103968
* when code completing inside a C-style block comment, don't emit errors aboutChris Lattner2010-05-161-2/+3
| | | | | | | | a missing */ since we truncated the file. This fixes rdar://7948776 llvm-svn: 103913
* fix a minor bug I noticed while work with Jordy's patch for PR6101,Chris Lattner2010-04-121-2/+4
| | | | | | | | | | | | | | | | | in an input file like this: # 42 int x; we were emitting: # <something> int x; (with a space before the int) because we weren't clearing the leading whitespace flag properly after the \n from the directive was handled. llvm-svn: 101084
* Reinstate my CodeModificationHint -> FixItHint renaming patch, withoutDouglas Gregor2010-03-311-2/+1
| | | | | | the C-only "optimization". llvm-svn: 100022
* Revert r100008, which inexplicably breaks the clang-i686-darwin10 builderDouglas Gregor2010-03-311-1/+2
| | | | llvm-svn: 100018
* Rename CodeModificationHint to FixItHint, since we've been using theDouglas Gregor2010-03-311-2/+1
| | | | | | | term "fix-it" everywhere and even *I* get tired of long names sometimes. No functionality change. llvm-svn: 100008
* Remove unused variableDouglas Gregor2010-03-161-1/+0
| | | | llvm-svn: 98691
* Audit all Preprocessor::getSpelling() callers, improving failureDouglas Gregor2010-03-161-1/+5
| | | | | | recovery for those that need it. llvm-svn: 98689
* Audit all callers of SourceManager::getCharacterData(); update some ofDouglas Gregor2010-03-161-0/+1
| | | | | | them to recover more gracefully on failure. llvm-svn: 98672
* Let SourceManager::getBufferData return StringRef instead of a pair of two ↵Benjamin Kramer2010-03-161-5/+4
| | | | | | const char*. llvm-svn: 98630
* Give SourceManager a Diagnostic object with which to report errors,Douglas Gregor2010-03-161-5/+3
| | | | | | and start simplifying the interfaces in SourceManager that can fail. llvm-svn: 98594
* Introduce a new BufferResult class to act as the return type ofDouglas Gregor2010-03-151-1/+8
| | | | | | | | | | | | | | SourceManager's getBuffer() (and similar) operations. This abstract can be used to force callers to cope with errors in getBuffer(), such as missing files and changed files. Fix a bunch of callers to use the new interface. Add some very basic checks for file consistency (file size, modification time) into ContentCache::getBuffer(), although these checks don't help much until we've updated the main callers (e.g., SourceManager::getSpelling()). llvm-svn: 98585
* don't inform comment handlers about comments in #if 0 blocks,Chris Lattner2010-02-031-7/+9
| | | | | | | | | doing so invalidates the file guard optimization and is not in the spirit of "#if 0" because it is supposed to completely skip everything, even if it isn't lexically valid. Patch by Abramo Bagnara! llvm-svn: 95253
* Teach CIndex's cursor visitor to restrict its traversal to a specificDouglas Gregor2010-01-221-0/+4
| | | | | | | | | | | | | | region of interest (if provided). Implement clang_getCursor() in terms of this traversal rather than using the Index library; the unified cursor visitor is more complete, and will be The Way Forward. Minor other tweaks needed to make this work: - Extend Preprocessor::getLocForEndOfToken() to accept an offset from the end, making it easy to move to the last character in the token (rather than just past the end of the token). - In Lexer::MeasureTokenLength(), the length of whitespace is zero. llvm-svn: 94200
* allow the HandlerComment callback to push tokens into theChris Lattner2010-01-181-14/+24
| | | | | | | preprocessor. This could be used by an OpenMP implementation or something. Patch by Abramo Bagnara! llvm-svn: 93795
* add a TODO for a perf improvement in LexIdentifier.Chris Lattner2010-01-111-2/+5
| | | | llvm-svn: 93141
* Do not parse hexadecimal floating point literals in C++0x mode because they areAlexis Hunt2010-01-101-1/+2
| | | | | | | | | | | | | | | | | | incompatible with user-defined literals, specifically with the following form: 0x1p+1 The preprocessing-number token extends only as far as the 'p'; the '+' is not included. Previously we could get away with this extension as p was an invalid suffix, but now with user-defined literals, 'p' might well be a valid suffix and we are forced to consider it as such. This patch also adds a warning in non-0x C++ modes telling the user that this extension is incompatible with C++0x that is enabled by default (previously and with other languages, we warn only with a compliance option such as -pedantic). llvm-svn: 93135
* reimplement r90860, fixing a couple of problems:Chris Lattner2009-12-171-9/+21
| | | | | | | | | 1. Don't make a copy of LangOptions every time a lexer is created. 2. Don't make CharInfo global mutable state. 3. Fix the implementation to properly treat ^Z as EOF instead of as horizontal whitespace, which matches the semantic implemented by VC++. llvm-svn: 91586
* teach clang to recover gracefully from conflict markers left in sourceChris Lattner2009-12-141-14/+133
| | | | | | files: PR5238. llvm-svn: 91270
* Integrate the following from the 'objective-rewrite' branch:Steve Naroff2009-12-081-4/+9
| | | | | | http://llvm.org/viewvc/llvm-project?view=rev&revision=80043 llvm-svn: 90860
* Extend the source manager with the ability to override the contents ofDouglas Gregor2009-12-021-22/+9
| | | | | | | | | | files with the contents of an arbitrary memory buffer. Use this new functionality to drastically clean up the way in which we handle file truncation for code-completion: all of the truncation/completion logic is now encapsulated in the preprocessor where it belongs (<rdar://problem/7434737>). llvm-svn: 90300
* Fix PR5633 by making the preprocessor handle the case where we canChris Lattner2009-11-301-6/+5
| | | | | | | | | | stat a file but where mmaping it fails. In this case, we emit an error like: t.c:1:10: fatal error: error opening file '../../foo.h' instead of "cannot find file". llvm-svn: 90110
* Move DISABLE_INLINE to the front of the decl so MSVC can parse it. Patch by ↵Benjamin Kramer2009-11-141-4/+4
| | | | | | Amine Khaldi! llvm-svn: 88797
* Teach Lexer::MeasureTokenLength to be able to measure theChris Lattner2009-10-141-0/+1
| | | | | | length of comment tokens. Patch by Abramo Bagnara! llvm-svn: 84100
* Replace the -code-completion-dump option with Douglas Gregor2009-09-221-9/+21
| | | | | | | | | | | -code-completion-at=filename:line:column which performs code completion at the specified location by truncating the file at that position and enabling code completion. This approach makes it possible to run multiple tests from a single test file, and gives a more natural command-line interface. llvm-svn: 82571
* Refactor and simplify the CodeCompleteConsumer, so that all of theDouglas Gregor2009-09-211-12/+13
| | | | | | | real work is performed within Sema. Addresses Chris's comments, but still retains the heavyweight list-of-multimaps data structure. llvm-svn: 82459
* Initial implementation of a code-completion interface in Clang. InDouglas Gregor2009-09-171-1/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | essence, code completion is triggered by a magic "code completion" token produced by the lexer [*], which the parser recognizes at certain points in the grammar. The parser then calls into the Action object with the appropriate CodeCompletionXXX action. Sema implements the CodeCompletionXXX callbacks by performing minimal translation, then forwarding them to a CodeCompletionConsumer subclass, which uses the results of semantic analysis to provide code-completion results. At present, only a single, "printing" code completion consumer is available, for regression testing and debugging. However, the design is meant to permit other code-completion consumers. This initial commit contains two code-completion actions: one for member access, e.g., "x." or "p->", and one for nested-name-specifiers, e.g., "std::". More code-completion actions will follow, along with improved gathering of code-completion results for the various contexts. [*] In the current -code-completion-dump testing/debugging mode, the file is truncated at the completion point and EOF is translated into "code completion". llvm-svn: 82166
* Remove tabs, and whitespace cleanups.Mike Stump2009-09-091-148/+148
| | | | llvm-svn: 81346
* Convert the CharInfo table to be statically initialized, instead of ↵Chris Lattner2009-07-071-15/+86
| | | | | | dynamically initialized. Patch by Ryan Flynn! llvm-svn: 74919
* fix an out-of-date comment.Chris Lattner2009-07-071-5/+3
| | | | llvm-svn: 74894
* Add support for retrieving the Doxygen comment associated with a givenDouglas Gregor2009-07-021-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | declaration in the AST. The new ASTContext::getCommentForDecl function searches for a comment that is attached to the given declaration, and returns that comment, which may be composed of several comment blocks. Comments are always available in an AST. However, to avoid harming performance, we don't actually parse the comments. Rather, we keep the source ranges of all of the comments within a large, sorted vector, then lazily extract comments via a binary search in that vector only when needed (which never occurs in a "normal" compile). Comments are written to a precompiled header/AST file as a blob of source ranges. That blob is only lazily loaded when one requests a comment for a declaration (this never occurs in a "normal" compile). The indexer testbed now supports comment extraction. When the -point-at location points to a declaration with a Doxygen-style comment, the indexer testbed prints the associated comment block(s). See test/Index/comments.c for an example. Some notes: - We don't actually attempt to parse the comment blocks themselves, beyond identifying them as Doxygen comment blocks to associate them with a declaration. - We won't find comment blocks that aren't adjacent to the declaration, because we start our search based on the location of the declaration. - We don't go through the necessary hops to find, for example, whether some redeclaration of a declaration has comments when our current declaration does not. Similarly, we don't attempt to associate a \param Foo marker in a function body comment with the parameter named Foo (although that is certainly possible). - Verification of my "no performance impact" claims is still "to be done". llvm-svn: 74704
* Fix our check for "random whitespace between a \ and newline" to workChris Lattner2009-06-231-2/+3
| | | | | | | | | | | | | | | with dos style newlines. I have a trivial test for this: // RUN: clang-cc %s -verify #define test(x, y) \ x ## y but I don't know how to get svn to not change newlines and testrunner doesn't work with dos style newlines either, so "not worth it". :) rdar://6994000 llvm-svn: 73945
* Fix rdar://6880630 - # in _Pragma does not start a preprocessor directive.Chris Lattner2009-05-131-2/+2
| | | | llvm-svn: 71643
* Get rid of some useless uses of NoExtensions. The philosophy here is Eli Friedman2009-04-281-2/+1
| | | | | | | | | | | | that if we're going to print an extension warning anyway, there's no point to changing behavior based on NoExtensions: it will only make error recovery worse. Note that this doesn't cause any behavior change because NoExtensions isn't used by the current front-end. I'm still considering what to do about the remaining use of NoExtensions in IdentifierTable.cpp. llvm-svn: 70273
* fix rdar://6816766 - Crash with function-like macro test at end of directive.Chris Lattner2009-04-241-0/+2
| | | | llvm-svn: 69964
OpenPOWER on IntegriCloud