summaryrefslogtreecommitdiffstats
path: root/clang/lib/Lex/Lexer.cpp
Commit message (Collapse)AuthorAgeFilesLines
...
* -Wc++98-compat warnings for the lexer.Richard Smith2011-10-151-0/+15
| | | | | | | | This also adds a -Wc++98-compat-pedantic for warning on constructs which would be diagnosed by -std=c++98 -pedantic (that is, it warns even on C++11 features which we enable by default, with no warning, in C++98 mode). llvm-svn: 142034
* We do parse hexfloats in C++11; make it actually work.Douglas Gregor2011-10-121-2/+1
| | | | llvm-svn: 141798
* Handle Perforce-style conflict markers like normal conflict markers. PerforceRichard Smith2011-10-121-24/+38
| | | | | | | swaps over the <<<< and >>>> markers, and uses shorter markers than traditional tools. llvm-svn: 141751
* Fixed exapnsion range for # and ##.Abramo Bagnara2011-10-031-1/+1
| | | | llvm-svn: 141012
* Rename SourceLocation::getFileLocWithOffset -> getLocWithOffset.Argyrios Kyrtzidis2011-09-191-10/+10
| | | | | | It already works (and is useful with) macro locs as well. llvm-svn: 140057
* Rename LangOptions::Microsoft to LangOptions::MicrosoftExt to make it clear ↵Francois Pichet2011-09-171-4/+4
| | | | | | | | that this flag must be used only for Microsoft extensions and not emulation; to avoid confusion with the new LangOptions::MicrosoftMode flag. Many of the code now under LangOptions::MicrosoftExt will eventually be moved under the LangOptions::MicrosoftMode flag. llvm-svn: 139987
* Speed up BCPL comment lexing by looking aggressively for newlines and then ↵Benjamin Kramer2011-09-051-9/+24
| | | | | | | | scannig backwards to see if the newline is escaped. 3% speedup in preprocessing all of clang with -Eonly. Also includes a small testcase for coverage. llvm-svn: 139116
* Use the Lexer's definition of whitespace here.Benjamin Kramer2011-09-051-3/+2
| | | | llvm-svn: 139115
* Support code-completion for C++ inline methods and ObjC buffering methods.Argyrios Kyrtzidis2011-09-041-36/+62
| | | | | | | | | | | | | | Previously we would cut off the source file buffer at the code-completion point; this impeded code-completion inside C++ inline methods and, recently, with buffering ObjC methods. Have the code-completion inserted into the source buffer so that it can be buffered along with a method body. When we actually hit the code-completion point the cut-off lexing or parsing. Fixes rdar://10056932&8319466 llvm-svn: 139086
* Fix Lexer::ComputePreamble when MaxLines parameter is non-zero.Argyrios Kyrtzidis2011-09-041-3/+18
| | | | | | | | The function was only counting lines that included tokens and not empty lines, but MaxLines (mainly initiated to the line where the code-completion point resides) is a count of overall lines (even empty ones). llvm-svn: 139085
* Introduce support for a simple module import declaration, whichDouglas Gregor2011-08-261-0/+1
| | | | | | | | | | | | | | | | | | | | | | loads the named module. The syntax itself is intentionally hideous and will be replaced at some later point with something more palatable. For now, we're focusing on the semantics: - Module imports are handled first by the preprocessor (to get macro definitions) and then the same tokens are also handled by the parser (to get declarations). If both happen (as in normal compilation), the second one is redundant, because we currently have no way to hide macros or declarations when loading a module. Chris gets credit for this mad-but-workable scheme. - The Preprocessor now holds on to a reference to a module loader, which is responsible for loading named modules. CompilerInstance is the only important module loader: it now knows how to create and wire up an AST reader on demand to actually perform the module load. - We search for modules in the include path, using the module name with the suffix ".pcm" (precompiled module) for the file name. This is a temporary hack; we hope to improve the situation in the future. llvm-svn: 138679
* Make Lexer::ComputePreamble accept a LangOptions parameter, otherwise it may beArgyrios Kyrtzidis2011-08-251-3/+3
| | | | | | out-of-sync how a file is compiled. Patch by Matthias Kleine! llvm-svn: 138580
* In Lexer::isAtEndOfMacroExpansion use SourceManager::isInFileID and avoidArgyrios Kyrtzidis2011-08-231-5/+3
| | | | | | the extra SourceManager::getFileID call. llvm-svn: 138376
* Make Lexer::GetBeginningOfToken able to handle macro arg expansion locations.Argyrios Kyrtzidis2011-08-171-3/+23
| | | | llvm-svn: 137795
* Add support for C++0x raw string literals.Craig Topper2011-08-111-19/+156
| | | | llvm-svn: 137298
* Add a utility function to the Lexer, which makes it easier to find a token ↵Anna Zaks2011-07-271-0/+59
| | | | | | after the given location. (It is a generalized version of trans::findLocationAfterSemi from ArcMigrate, which will be changed to use the Lexer utility). llvm-svn: 136268
* Add support for C++0x unicode string and character literals, from Craig Topper!Douglas Gregor2011-07-271-13/+64
| | | | llvm-svn: 136210
* Migrate 'Instantiation' data and API bits of SLocEntry to 'Expansion'Chandler Carruth2011-07-261-3/+2
| | | | | | | | etc. With this I think essentially all of the SourceManager APIs are converted. Comments and random other bits of cleanup should be all thats left. llvm-svn: 136057
* Convert InstantiationInfo and much of the related code to ExpansionInfoChandler Carruth2011-07-261-2/+2
| | | | | | | | | and various other 'expansion' based terms. I've tried to reformat where appropriate and catch as many references in comments but I'm going to do several more passes. Also I've tried to expand parameter names to be more clear where appropriate. llvm-svn: 136056
* Rename create(MacroArg)InstantiationLoc to create(MacroArg)ExpansionLoc.Chandler Carruth2011-07-261-4/+4
| | | | llvm-svn: 136054
* Rename SourceManager::getImmediateInstantiationRange toChandler Carruth2011-07-251-1/+1
| | | | | | getImmediateExpansionRange. llvm-svn: 135960
* Rename SourceManager::getInstantiationRange to getExpansionRange.Chandler Carruth2011-07-251-1/+1
| | | | llvm-svn: 135915
* Mechanically rename SourceManager::getInstantiationLoc andChandler Carruth2011-07-251-1/+1
| | | | | | | | FullSourceLoc::getInstantiationLoc to ...::getExpansionLoc. This is part of the API and documentation update from 'instantiation' as the term for macros to 'expansion'. llvm-svn: 135914
* remove unneeded llvm:: namespace qualifiers on some core types now that ↵Chris Lattner2011-07-231-15/+15
| | | | | | | | LLVM.h imports them into the clang namespace. llvm-svn: 135852
* SpellingJoerg Sonnenberger2011-07-201-1/+1
| | | | llvm-svn: 135545
* Revamp the SourceManager to separate the representation of parsedDouglas Gregor2011-07-191-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | source locations from source locations loaded from an AST/PCH file. Previously, loading an AST/PCH file involved carefully pre-allocating space at the beginning of the source manager for the source locations and FileIDs that correspond to the prefix, and then appending the source locations/FileIDs used for parsing the remaining translation unit. This design forced us into loading PCH files early, as a prefix, whic has become a rather significant limitation. This patch splits the SourceManager space into two parts: for source location "addresses", the lower values (growing upward) are used to describe parsed code, while upper values (growing downward) are used for source locations loaded from AST/PCH files. Similarly, positive FileIDs are used to describe parsed code while negative FileIDs are used to file/macro locations loaded from AST/PCH files. As a result, we can load PCH/AST files even during parsing, making various improvemnts in the future possible, e.g., teaching #include <foo.h> to look for and load <foo.h.gch> if it happens to be already available. This patch was originally written by Sebastian Redl, then brought forward to the modern age by Jonathan Turner, and finally polished/finished by me to be committed. llvm-svn: 135484
* Convert terminology in the Lexer from 'instantiate' and variants toChandler Carruth2011-07-141-26/+27
| | | | | | | | | 'expand'. Also update the public API it provides to the new term, and propagate that update to the various clients. No functionality changed. llvm-svn: 135138
* Move SourceManager::isAt[Start/End]OfMacroInstantiation functions to the ↵Argyrios Kyrtzidis2011-07-071-1/+52
| | | | | | Lexer, since they depend on it now. llvm-svn: 134644
* Make the Preprocessor more memory efficient and improve macro instantiation ↵Argyrios Kyrtzidis2011-07-071-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | diagnostics. When a macro instantiation occurs, reserve a SLocEntry chunk with length the full length of the macro definition source. Set the spelling location of this chunk to point to the start of the macro definition and any tokens that are lexed directly from the macro definition will get a location from this chunk with the appropriate offset. For any tokens that come from argument expansion, '##' paste operator, etc. have their instantiation location point at the appropriate place in the instantiated macro definition (the argument identifier and the '##' token respectively). This improves macro instantiation diagnostics: Before: t.c:5:9: error: invalid operands to binary expression ('struct S' and 'int') int y = M(/); ^~~~ t.c:5:11: note: instantiated from: int y = M(/); ^ After: t.c:5:9: error: invalid operands to binary expression ('struct S' and 'int') int y = M(/); ^~~~ t.c:3:20: note: instantiated from: \#define M(op) (foo op 3); ~~~ ^ ~ t.c:5:11: note: instantiated from: int y = M(/); ^ The memory savings for a candidate boost library that abuses the preprocessor are: - 32% less SLocEntries (37M -> 25M) - 30% reduction in PCH file size (900M -> 635M) - 50% reduction in memory usage for the SLocEntry table (1.6G -> 800M) llvm-svn: 134587
* Allow Lexer::getLocForEndOfToken to return the location just passed the ↵Argyrios Kyrtzidis2011-06-241-2/+10
| | | | | | | | | | macro instantiation if the location given points at the last token of the macro instantiation. Fixes rdar://9045701. llvm-svn: 133804
* Don't strlen() every file before parsing it.Eli Friedman2011-05-101-1/+2
| | | | llvm-svn: 131132
* fix a bunch of comment typos found by codespell. Patch byChris Lattner2011-04-151-1/+1
| | | | | | Luis Felipe Strano Moraes! llvm-svn: 129559
* Implement C++0x [lex.pptoken]p3's handling of <::.Richard Smith2011-04-141-0/+15
| | | | llvm-svn: 129525
* Eat the UTF-8 BOM at the beginning of a file since it's ignored anyhow.Eric Christopher2011-04-091-1/+14
| | | | | | | | Nom Nom Nom. Patch by Anton Korobeynikov! llvm-svn: 129174
* Fix getLocForEndOfToken to not double-count spurious internal characters John McCall2011-04-061-1/+1
| | | | | | | within a token, like trigraphs and escaped newlines. Patch by Marcin Kowalczyk! llvm-svn: 128978
* Lexer: Add extremely limited support for -traditional-cpp, ignoring BCPLDaniel Dunbar2011-03-181-3/+5
| | | | | | comments. llvm-svn: 127910
* Fix my earlier commit to work with escaped newlines and leave breadcrumbsJohn McCall2011-03-081-0/+48
| | | | | | | in case we want to make a world where we can check intermediate instantiations for this kind of breadcrumb. llvm-svn: 127221
* Rename tok::eom to tok::eod.Peter Collingbourne2011-02-281-7/+7
| | | | | | | | The previous name was inaccurate as this token in fact appears at the end of every preprocessing directive, not just macro definitions. No functionality change, except for a diagnostic tweak. llvm-svn: 126631
* Warn for missing terminating " or ' instead of error for gcc compatibility. ↵Argyrios Kyrtzidis2011-02-151-2/+2
| | | | | | Fixed rdar://8914293. llvm-svn: 125616
* Lexer: add CUDA kernel call tokensPeter Collingbourne2011-02-091-0/+8
| | | | llvm-svn: 125218
* Harden Lexer::GetBeginningOfToken() against bogus source locations andDouglas Gregor2011-01-311-0/+6
| | | | | | the disappearance/alteration of files. llvm-svn: 124616
* Introduced raw_identifier token kind.Abramo Bagnara2010-12-221-25/+24
| | | | llvm-svn: 122394
* move getSpelling from Preprocessor to Lexer, which it is more conceptually ↵Chris Lattner2010-11-171-0/+101
| | | | | | related to. llvm-svn: 119479
* move AdvanceToTokenCharacter and getLocForEndOfToken fromChris Lattner2010-11-171-0/+77
| | | | | | Preprocessor to Lexer where they make more sense. llvm-svn: 119474
* Update remaining attribute macros to new style.Chandler Carruth2010-10-231-4/+2
| | | | llvm-svn: 117204
* In MeasureTokenLength, the FileLoc supplied to the lexer must point to the ↵Sebastian Redl2010-09-301-1/+2
| | | | | | start of the buffer, or we risk overflow. llvm-svn: 115117
* improve isHexaLiteral to work with escaped newlines and trigraphs,Chris Lattner2010-08-311-7/+8
| | | | | | patch by Francois Pichet! llvm-svn: 112602
* silence a warningChris Lattner2010-08-301-1/+1
| | | | llvm-svn: 112549
* Revert my user-defined literal commits - r1124{58,60,67} pendingAlexis Hunt2010-08-301-29/+2
| | | | | | some issues being sorted out. llvm-svn: 112493
* add a fixme.Chris Lattner2010-08-301-1/+5
| | | | llvm-svn: 112491
OpenPOWER on IntegriCloud