summaryrefslogtreecommitdiffstats
path: root/clang/lib/Lex
Commit message (Collapse)AuthorAgeFilesLines
...
* Always allow "#pragma region".Matt Davis2018-01-271-2/+4
| | | | | | | | | | | | | | | | | | | | | Summary: Both MS and PS4 targets are capable of recognizing the existence of: #pragma region, #pragma endregion. Since this pragma is only a hint for certain editors, and has no logic, it seems helpful to permit this pragma in all cases, not just MS compatibility mode. Reviewers: rnk, rsmith, majnemer Reviewed By: majnemer Subscribers: Quuxplusone, probinson, majnemer, cfe-commits Differential Revision: https://reviews.llvm.org/D42248 llvm-svn: 323577
* [CodeComplete] Fix completion in the middle of idents in macro callsIlya Biryukov2018-01-221-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch removes IdentifierInfo from completion token after remembering the identifier in the preprocessor. Prior to this patch, completion token had the IdentifierInfo set to null when completing at the start of identifier and to the II for completion prefix when in the middle of identifier. This patch unifies how code completion token is handled when it is insterted before the identifier and in the middle of the identifier. The actual IdentifierInfo can still be obtained from the Preprocessor. Reviewers: bkramer, arphaman Reviewed By: bkramer Subscribers: cfe-commits Differential Revision: https://reviews.llvm.org/D42241 llvm-svn: 323133
* [Lex] Fix crash on code completion in comment in included file.Volodymyr Sapsai2018-01-192-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This fixes PR32732 by updating CurLexerKind to reflect available lexers. We were hitting null pointer in Preprocessor::Lex because CurLexerKind was CLK_Lexer but CurLexer was null. And we set it to null in Preprocessor::HandleEndOfFile when exiting a file with code completion point. To reproduce the crash it is important for a comment to be inside a class specifier. In this case in Parser::ParseClassSpecifier we improve error recovery by pushing a semicolon token back into the preprocessor and later on try to lex a token because we haven't reached the end of file. Also clang crashes only on code completion in included file, i.e. when IncludeMacroStack is not empty. Though we reset CurLexer even if include stack is empty. The difference is that during pushing back a semicolon token, preprocessor calls EnterCachingLexMode which decides it is already in caching mode because various lexers are null and IncludeMacroStack is not empty. As the result, CurLexerKind remains CLK_Lexer instead of updating to CLK_CachingLexer. rdar://problem/34787685 Reviewers: akyrtzi, doug.gregor, arphaman Reviewed By: arphaman Subscribers: cfe-commits, kfunk, arphaman, nemanjai, kbarton Differential Revision: https://reviews.llvm.org/D41688 llvm-svn: 323008
* [PCH] Serialize skipped preprocessor rangesCameron Desrochers2018-01-152-2/+23
| | | | | | | | The skipped preprocessor ranges are now serialized in the AST PCH file. This fixes, for example, libclang's clang_getSkippedRanges() returning zero ranges after reparsing a translation unit. Differential Revision: https://reviews.llvm.org/D20124 llvm-svn: 322503
* [Lex] Avoid out-of-bounds dereference in LexAngledStringLiteral.Volodymyr Sapsai2018-01-121-8/+11
| | | | | | | | | | | | | | | | | | | | | | Fix makes the loop in LexAngledStringLiteral more like the loops in LexStringLiteral, LexCharConstant. When we skip a character after backslash, we need to check if we reached the end of the file instead of reading the next character unconditionally. Discovered by OSS-Fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3832 rdar://problem/35572754 Reviewers: arphaman, kcc, rsmith, dexonsmith Reviewed By: rsmith, dexonsmith Subscribers: cfe-commits, rsmith, dexonsmith Differential Revision: https://reviews.llvm.org/D41423 llvm-svn: 322390
* [Lex] Use WritableMemoryBuffer in ScratchBuffer.cppPavel Labath2018-01-111-4/+4
| | | | | | | | This avoids the need to const_cast the buffer contents to write to it. NFCI. llvm-svn: 322268
* Preserve unknown STDC pragma through preprocessorSteven Wu2018-01-051-55/+0
| | | | | | | | | | | | | | | | | | | Summary: #pragma STDC FP_CONTRACT handler is only registered in parser so we should keep the unknown STDC pragma through preprocessor and we also should not emit warning for unknown STDC pragma during preprocessor. rdar://problem/35724351 Reviewers: efriedma, rsmith, arphaman Reviewed By: efriedma Subscribers: cfe-commits Differential Revision: https://reviews.llvm.org/D41780 llvm-svn: 321909
* Track shadow modules with a generation counter.Bruno Cardoso Lopes2018-01-052-36/+25
| | | | | | | | | | This is a follow up to r321855, closing the gap between our internal shadow modules implementation and upstream. It has been tested for longer and provides a better approach for tracking shadow modules. Mostly NFCI. rdar://problem/23612102 llvm-svn: 321906
* Reapply r321781: [Modules] Allow modules specified by -fmodule-map-file to ↵Bruno Cardoso Lopes2018-01-053-42/+98
| | | | | | | | | | | | | | | | | | | | | | | | | | | | shadow implicitly found ones When modules come from module map files explicitly specified by -fmodule-map-file= arguments, allow those to override/shadow modules with the same name that are found implicitly by header search. If such a module is looked up by name (e.g. @import), we will always find the one from -fmodule-map-file. If we try to use a shadowed module by including one of its headers report an error. This enables developers to force use of a specific copy of their module to be used if there are multiple copies that would otherwise be visible, for example if they develop modules that are installed in the default search paths. Patch originally by Ben Langmuir, http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20151116/143425.html Based on cfe-dev discussion: http://lists.llvm.org/pipermail/cfe-dev/2015-November/046164.html Differential Revision: https://reviews.llvm.org/D31269 rdar://problem/23612102 llvm-svn: 321855
* Revert "[Modules] Allow modules specified by -fmodule-map-file to shadow ↵Bruno Cardoso Lopes2018-01-043-95/+42
| | | | | | | | | | | implicitly found ones" This reverts r321781 until I fix the leaks pointed out by bots: http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/12146 http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/builds/3741 llvm-svn: 321786
* [Modules] Allow modules specified by -fmodule-map-file to shadow implicitly ↵Bruno Cardoso Lopes2018-01-043-42/+95
| | | | | | | | | | | | | | | | | | | | | | | | | | | | found ones When modules come from module map files explicitly specified by -fmodule-map-file= arguments, allow those to override/shadow modules with the same name that are found implicitly by header search. If such a module is looked up by name (e.g. @import), we will always find the one from -fmodule-map-file. If we try to use a shadowed module by including one of its headers report an error. This enables developers to force use of a specific copy of their module to be used if there are multiple copies that would otherwise be visible, for example if they develop modules that are installed in the default search paths. Patch originally by Ben Langmuir, http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20151116/143425.html Based on cfe-dev discussion: http://lists.llvm.org/pipermail/cfe-dev/2015-November/046164.html Differential Revision: https://reviews.llvm.org/D31269 rdar://problem/23612102 llvm-svn: 321781
* [Modules] Change private modules rules and warningsBruno Cardoso Lopes2017-12-222-40/+71
| | | | | | | | | | | | | | | | We used to advertise private modules to be declared as submodules (Foo.Private). This has proven to not scale well since private headers might carry several dependencies, introducing unwanted content into the main module and often causing dep cycles. Change the canonical way to name it to Foo_Private, forcing private modules as top level ones, and provide warnings under -Wprivate-module to suggest fixes for other private naming. Update documentation to reflect that. rdar://problem/31173501 llvm-svn: 321337
* Remove llvm::MemoryBuffer const_castsPavel Labath2017-12-201-4/+3
| | | | | | | | | | | | | | | Summary: llvm has grown a WritableMemoryBuffer class, which is convertible (inherits from) a MemoryBuffer. We can use it to avoid conts_casting the buffer contents when we want to write to it. Reviewers: dblaikie, rsmith Subscribers: cfe-commits Differential Revision: https://reviews.llvm.org/D41387 llvm-svn: 321167
* __is_target_environment: Check the environment after parsing itAlex Lorenz2017-12-151-4/+3
| | | | | | | This ensures that target triples with environment versions can still work with __is_target_environment. llvm-svn: 320854
* __is_target_arch: Check the arch and subarch instead of the arch nameAlex Lorenz2017-12-151-3/+3
| | | | | | | | | This ensures that when compiling for "arm64" __is_target_arch will succeed for both "arm64" and "aarch64". Thanks to Bob Wilson who pointed this out! llvm-svn: 320853
* [Preprocessor] Implement __is_target_{arch|vendor|os|environment} function-likeAlex Lorenz2017-12-141-0/+88
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | builtin macros This patch implements the __is_target_arch, __is_target_vendor, __is_target_os, and __is_target_environment Clang preprocessor extensions that were proposed by @compnerd in Bob's cfe-dev post: http://lists.llvm.org/pipermail/cfe-dev/2017-November/056166.html. These macros can be used to examine the components of the target triple at compile time. A has_builtin(is_target_???) preprocessor check can be used to check for their availability. __is_target_arch allows you to check if an arch is specified without worring about a specific subarch, e.g. __is_target_arch(arm) returns 1 for the target arch "armv7" __is_target_arch(armv7) returns 1 for the target arch "armv7" __is_target_arch(armv6) returns 0 for the target arch "armv7" __is_target_vendor and __is_target_environment match the specific vendor or environment. __is_target_os matches the specific OS, but __is_target_os(darwin) will match any Darwin-based OS. "Unknown" can be used to test if the triple's component is specified. rdar://35753116 Differential Revision: https://reviews.llvm.org/D41087 llvm-svn: 320734
* Warn if we find a Unicode homoglyph for a symbol in an identifier.Richard Smith2017-12-141-1/+78
| | | | | | | | | | | | | | | | Specifically, warn if: * we find a character that the language standard says we must treat as an identifier, and * that character is not reasonably an identifier character (it's a punctuation character or similar), and * it renders identically to a valid non-identifier character in common fixed-width fonts. Some tools "helpfully" substitute the surprising characters for the expected characters, and replacing semicolons with Greek question marks is a common "prank". llvm-svn: 320697
* Hardware-assisted AddressSanitizer (clang part).Evgeniy Stepanov2017-12-091-1/+4
| | | | | | | | | | | | | | Summary: Driver, frontend and LLVM codegen for HWASan. A clone of ASan, basically. Reviewers: kcc, pcc, alekseyshl Subscribers: srhines, javed.absar, cfe-commits Differential Revision: https://reviews.llvm.org/D40936 llvm-svn: 320232
* [Lex] Fix some Clang-tidy modernize and Include What You Use warnings; other ↵Eugene Zelenko2017-12-084-90/+124
| | | | | | minor fixes (NFC). llvm-svn: 320207
* Fix more line endings changed in r320089. NFC.Ahmed Bougacha2017-12-071-1918/+1918
| | | | llvm-svn: 320114
* Add support for the __has_c_attribute builtin preprocessor macro.Aaron Ballman2017-12-071-1913/+1918
| | | | | | This behaves similar to the __has_cpp_attribute builtin macro in that it allows users to detect whether an attribute is supported with the [[]] spelling syntax, which can be enabled in C with -fdouble-square-bracket-attributes. llvm-svn: 320088
* [Lex] Fix some Clang-tidy modernize and Include What You Use warnings; other ↵Eugene Zelenko2017-12-063-97/+141
| | | | | | minor fixes (NFC). llvm-svn: 319986
* Stringizing raw string literals containing newlineTaewook Oh2017-12-061-56/+65
| | | | | | | | | | | | | | Summary: This patch implements 4.3 of http://open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4220.pdf. If a raw string contains a newline character, replace each newline character with the \n escape code. Without this patch, included test case (macro_raw_string.cpp) results compilation failure. Reviewers: rsmith, doug.gregor, jkorous-apple Reviewed By: jkorous-apple Subscribers: jkorous-apple, vsapsai, cfe-commits Differential Revision: https://reviews.llvm.org/D39279 llvm-svn: 319904
* [Lex] Fix some Clang-tidy modernize and Include What You Use warnings; other ↵Eugene Zelenko2017-12-043-55/+50
| | | | | | minor fixes (NFC). llvm-svn: 319714
* Now that C++17 is official (https://www.iso.org/standard/68564.html), start ↵Aaron Ballman2017-12-043-5/+5
| | | | | | changing the C++1z terminology over to C++17. NFC intended, these are all mechanical changes. llvm-svn: 319688
* [c++2a] P0515R3: lexer support for new <=> token.Richard Smith2017-12-012-0/+24
| | | | llvm-svn: 319509
* [refactor][extract] insert semicolons into extracted/inserted codeAlex Lorenz2017-11-031-16/+20
| | | | | | | | | | | | | | | | | | when needed This commit implements the semicolon insertion logic into the extract refactoring. The following rules are used: - extracting expression: add terminating ';' to the extracted function. - extracting statements that don't require terminating ';' (e.g. switch): add terminating ';' to the callee. - extracting statements with ';': move (if possible) the original ';' from the callee and add terminating ';'. - otherwise, add ';' to both places. Differential Revision: https://reviews.llvm.org/D39441 llvm-svn: 317343
* [Driver] Add Scudo as a possible -fsanitize= optionKostya Kortchinsky2017-11-031-0/+1
| | | | | | | | | | | | | | | | | | | | | | Summary: This change adds Scudo as a possible Sanitizer option via -fsanitize=. This allows for easier static & shared linking of the Scudo library, it allows us to enforce PIE (otherwise the security of the allocator is moot), and check for incompatible Sanitizers combo. In its current form, Scudo is not compatible with any other Sanitizer, but the plan is to make it work in conjunction with UBsan (-fsanitize=scudo,undefined), which will require additional work outside of the scope of this change. Reviewers: eugenis, kcc, alekseyshl Reviewed By: eugenis, alekseyshl Subscribers: llvm-commits, srhines Differential Revision: https://reviews.llvm.org/D39334 llvm-svn: 317337
* [preamble] Also record the "skipping" state of the preprocessorErik Verbruggen2017-11-032-11/+27
| | | | | | | | | | When a preamble ends in a conditional preprocessor block that is being skipped, the preprocessor needs to continue skipping that block when the preamble is used. This fixes PR34570. llvm-svn: 317308
* Add -f[no-]double-square-bracket-attributes as new driver options to control ↵Aaron Ballman2017-10-151-1/+3
| | | | | | use of [[]] attributes in all language modes. This is the initial implementation of WG14 N2165, which is a proposal to add [[]] attributes to C2x, but also allows you to enable these attributes in C++98, or disable them in C++11 or later. llvm-svn: 315856
* [Lex] Remove unused variables. No functionality change.Benjamin Kramer2017-10-151-6/+0
| | | | llvm-svn: 315845
* [c++2a] Implement P0306 __VA_OPT__ (Comma omission and comma deletion)Faisal Vali2017-10-154-21/+251
| | | | | | | | | | | | | | | | | | | | | | | | | This patch implements an extension to the preprocessor: __VA_OPT__(contents) --> which expands into its contents if variadic arguments are supplied to the parent macro, or behaves as an empty token if none. - Currently this feature is only enabled for C++2a (this could be enabled, with some careful tweaks, for other dialects with the appropriate extension or compatibility warnings) - The patch was reviewed here: https://reviews.llvm.org/D35782 and asides from the above (and moving some of the definition and expansion recognition logic into the corresponding state machines), I believe I incorporated all of Richard's suggestions. A few technicalities (most of which were clarified through private correspondence between rsmith, hubert and thomas) are worth mentioning. Given: #define F(a,...) a #__VA_OPT__(a ## a) a ## __VA_OPT__(__VA_ARGS__) - The call F(,) Does not supply any tokens for the variadic arguments and hence VA_OPT behaves as a placeholder. - When expanding VA_OPT (for e.g. F(,1) token pasting occurs eagerly within its contents if the contents need to be stringified. - A hash or a hashhash prior to VA_OPT does not inhibit expansion of arguments if they are the first token within VA_OPT. - When a variadic argument is supplied, argument substitution occurs within the contents as does stringification - and these resulting tokens are inserted back into the macro expansions token stream just prior to the entire stream being rescanned and concatenated. See wg21.link/P0306 for further details on the feature. Acknowledgment: This patch would have been poorer if not for Richard Smith's usual thoughtful analysis and feedback. llvm-svn: 315840
* [Lex] Avoid out-of-bounds dereference in SkipLineCommentAlex Lorenz2017-10-141-1/+2
| | | | | | | | | Credit to OSS-Fuzz for discovery: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3145 rdar://34526482 llvm-svn: 315785
* Fix warnings. [-Wdocumentation]NAKAMURA Takumi2017-10-121-2/+1
| | | | llvm-svn: 315573
* A '<' with a trigraph '#' is not a valid editor placeholderAlex Lorenz2017-10-111-1/+2
| | | | | | | | | Credit to OSS-Fuzz for discovery: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3137#c5 rdar://34923985 llvm-svn: 315398
* Add parens around the boolean condition of one of the added asserts in ↵Faisal Vali2017-10-031-2/+2
| | | | | | | | | | | | | | | r314747 ... ... in the hopes of teaching the bots the gift of silence ;) For quick reference: https://reviews.llvm.org/rL314747 llvm-svn: 314753
* Remove an assertion I added from the refactoring of pasteTokens ↵Faisal Vali2017-10-031-4/+0
| | | | | | | | | | | | (https://reviews.llvm.org/rL314747). - it made the bots v angry! I'm not exactly sure why the assertion doesn't hold - if anyone has any insight - would appreciate it. Thanks! llvm-svn: 314748
* [NFC] Refactor PasteTokens so that it can be passed the Token Stream and ↵Faisal Vali2017-10-031-38/+58
| | | | | | | | | | | | | | Index to start concatenating at. In passing: - change the name of the function to pasteTokens c/w coding standards - rename CurToken to CurTokenIdx (since it is not the token, but the index) - add doxygen comments to document some of pasteTokens' functionality - use parameter names different from the data member names. This will be useful for implementing __VA_OPT__ (https://reviews.llvm.org/D35782#inline-322587) llvm-svn: 314747
* [NFC] Add assertion that we assume a valid macro argument index.Faisal Vali2017-09-301-0/+3
| | | | llvm-svn: 314600
* [NFC] Remove superfluous parameter Faisal Vali2017-09-302-7/+6
| | | | | | - MacroArgs already knows the maximum number of arguments that can be supplied to the macro. No need to pass MacroInfo (information about the macro definition) to the call to getPreExpArgument (which by the way might benefit from being called getExpandedArgument() ?) for it to compute the number of arguments. llvm-svn: 314593
* [NFC] Replace 'arguments' with 'parameters' in comments relating to lexing a ↵Faisal Vali2017-09-291-7/+7
| | | | | | macro definition. llvm-svn: 314484
* [NFC] Rename variable 'Arguments' to 'Parameters' when lexing the Macro ↵Faisal Vali2017-09-291-9/+9
| | | | | | Definition. llvm-svn: 314483
* Use std::is_trivial instead of is_trivially_copyable.Benjamin Kramer2017-09-281-2/+2
| | | | | | | | The oldest versions of GCC we support (before 5) didn't support that trait. is_trivial is stronger superset that clang::Token fulfills, so just use that instead. llvm-svn: 314391
* [NFC] Don't use C++17 standard lib variable template helper traits, instead ↵Faisal Vali2017-09-281-2/+2
| | | | | | use ::value. llvm-svn: 314373
* [NFC] Modernize MacroArgs using TrailingObjectsFaisal Vali2017-09-281-10/+15
| | | | | | Refactor MacroArgs to use TrailingObjects when creating a variably sized object on the heap to store the unexpanded tokens immediately after the MacroArgs object. llvm-svn: 314372
* [Preprocessor] Preserve #pragma clang assume_nonnull in preprocessed outputEli Friedman2017-09-271-0/+5
| | | | | | | | Patch by Zbigniew Sarbinowski! Differential Revision: https://reviews.llvm.org/D37861 llvm-svn: 314364
* Delete trailing whitespace.Nico Weber2017-09-261-1/+1
| | | | llvm-svn: 314232
* Fixed unused variable warning introduced in r313796 causing build failureCameron Desrochers2017-09-201-3/+0
| | | | llvm-svn: 313802
* [PCH] Fixed preamble breaking with BOM presence (and particularly, ↵Cameron Desrochers2017-09-202-10/+10
| | | | | | | | | | | | | | fluctuating BOM presence) This patch fixes broken preamble-skipping when the preamble region includes a byte order mark (BOM). Previously, parsing would fail if preamble PCH generation was enabled and a BOM was present. This also fixes preamble invalidation when a BOM appears or disappears. This may seem to be an obscure edge case, but it happens regularly with IDEs that pass buffer overrides that never (or always) have a BOM, yet the underlying file from the initial parse that generated a PCH might (or might not) have a BOM. I've included a test case for these scenarios. Differential Revision: https://reviews.llvm.org/D37491 llvm-svn: 313796
* Another attempt to fix warning discovered by r313487. [-Wunused-lambda-capture]Vitaly Buka2017-09-181-1/+1
| | | | llvm-svn: 313521
OpenPOWER on IntegriCloud