aboutsummaryrefslogtreecommitdiff
path: root/lib/Lex/Lexer.cpp
AgeCommit message (Collapse)Author
2012-06-28Add warning flag -Winvalid-pp-token for preprocessing-tokens which haveRichard Smith
undefined behaviour, and move the diagnostic for '' from an Error into an ExtWarn in this group. This is important for some users of the preprocessor, and is necessary for gcc compatibility. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@159335 91177308-0d34-0410-b5e6-96231b3b80d8
2012-06-17Documentation cleanup:James Dennett
* Removed docs for Lexer::makeFileCharRange from Lexer.cpp, as they're in the header file; * Reworked the documentation for SkipBlockComment so that it doesn't confuse Doxygen's comment parsing; * Added another summary with \brief markup. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@158618 91177308-0d34-0410-b5e6-96231b3b80d8
2012-06-15[-E] Emit a rewritten _Pragma on its own line.Jordan Rose
1. Teach Lexer that pragma lexers are like macro expansions at EOF. 2. Treat pragmas like #define/#undef when printing. 3. If we just printed a directive, add a newline before any more tokens. (4. Miscellaneous cleanup in PrintPreprocessedOutput.cpp) PR10594 and <rdar://problem/11562490> (two separate related problems) git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@158571 91177308-0d34-0410-b5e6-96231b3b80d8
2012-06-15Documentation cleanup: escape backslashes in Doxygen comments.James Dennett
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@158552 91177308-0d34-0410-b5e6-96231b3b80d8
2012-06-15PR12717: Clang supports hexadecimal floating-point literals in all languageRichard Smith
modes. For languages other than C99/C11, this isn't quite a conforming extension, and for C++11, it breaks some reasonable code containing user-defined literals. In languages which don't officially have hexfloats, pare back this extension to only apply in cases where the token starts 0x and does not contain an underscore. The extension is still not quite conforming, but it's a lot closer now. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@158487 91177308-0d34-0410-b5e6-96231b3b80d8
2012-06-15Fix PR13065.David Blaikie
This condition (added in r158093) was overly conservative. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@158483 91177308-0d34-0410-b5e6-96231b3b80d8
2012-06-08Correct method name in comment: from LexRawToken to LexFromRawLexer, accordingDmitri Gribenko
to a change done long ago in r57393. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@158243 91177308-0d34-0410-b5e6-96231b3b80d8
2012-06-07Insert a space if necessary when suggesting CFBridgingRetain/Release.Jordan Rose
This was a problem for people who write 'return(result);' Also fix ARCMT's corresponding code, though there's no test case for this because implicit casts like this are rejected by the migrator for being ambiguous, and explicit casts have no problem. <rdar://problem/11577346> git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@158130 91177308-0d34-0410-b5e6-96231b3b80d8
2012-06-06Add a -rewrite-includes option, which is similar to -rewrite-macros, but ↵David Blaikie
only expands #include directives. Patch contributed by Lubos Lunak (l.lunax@suse.cz). Review by Matt Beaumont-Gay (matthewbg@google.com). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@158093 91177308-0d34-0410-b5e6-96231b3b80d8
2012-06-06Escape \n and \r in doxycomment.David Blaikie
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@158091 91177308-0d34-0410-b5e6-96231b3b80d8
2012-05-18Lexer::ReadToEndOfLine: Only build the string if it's actually used and do ↵Benjamin Kramer
so in a less malloc-intensive way. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@157064 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-13Support -Wc++98-compat-pedantic as requested:Seth Cantrell
http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20120409/056126.html git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@154655 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-13C++11 no longer requires files to end with a newlineSeth Cantrell
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@154643 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-07ext_reserved_user_defined_literal must not default to Error in ↵Francois Pichet
MicrosoftMode. Hence create ext_ms_reserved_user_defined_literal that doesn't default to Error; otherwise MSVC headers won't parse. Fixes PR12383. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@154273 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-11Unify naming of LangOptions variable/get function across the Clang stack ↵David Blaikie
(Lex to AST). The member variable is always "LangOpts" and the member function is always "getLangOpts". Reviewed by Chris Lattner git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@152536 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-08Implement C++11 [lex.ext]p10 for string and character literals: a ud-suffix notRichard Smith
starting with an underscore is ill-formed. Since this rule rejects programs that were using <inttypes.h>'s macros, recover from this error by treating the ud-suffix as a separate preprocessing-token, with a DefaultError ExtWarn. The approach of treating such cases as two tokens is under discussion for standardization, but is in any case a conforming extension and allows existing codebases to keep building while the committee makes up its mind. Reword the warning on the definition of literal operators not starting with underscores (which are, strangely, legal) to more explicitly state that such operators can't be called by literals. Remove the special-case diagnostic for hexfloats, since it was both triggering in the wrong cases and incorrect. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@152287 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-07Add -Wc++11-compat warning for string and character literals followed byRichard Smith
identifiers, in cases where those identifiers would be treated as user-defined literal suffixes in C++11. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@152198 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-06User-defined literals: reject string and character UDLs in all places where theRichard Smith
grammar requires a string-literal and not a user-defined-string-literal. The two constructs are still represented by the same TokenKind, in order to prevent a combinatorial explosion of different kinds of token. A flag on Token tracks whether a ud-suffix is present, in order to prevent clients from needing to look at the token's spelling. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@152098 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-05Lexing support for user-defined literals. Currently these lex as the same tokenRichard Smith
kinds as the underlying string literals, and we silently drop the ud-suffix; those issues will be fixed by subsequent patches. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@152012 91177308-0d34-0410-b5e6-96231b3b80d8
2012-02-03Change Lexer::makeFileCharRange() to have it accept a CharSourceRangeArgyrios Kyrtzidis
instead of a SourceRange, and handle the case where the range is a char (not token) range. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@149677 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-23Improve Lexer::getImmediateMacroName to take into account inner macrosArgyrios Kyrtzidis
of macro arguments. For "MAC1( MAC2(foo) )" and location of 'foo' token it would return "MAC1" instead of "MAC2". git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@148704 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-20Enhance Lexer::makeFileCharRange to check for ranges inside a macro argumentArgyrios Kyrtzidis
expansion, in which case it returns a file range in the location where the argument was spelled. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@148551 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-19Introduce Lexer::getSourceText() that returns a string for the sourceArgyrios Kyrtzidis
that the given source range encompasses. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@148481 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-19Introduce Lexer::makeFileCharRange() that accepts a token source rangeArgyrios Kyrtzidis
and returns a character range with file locations. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@148480 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-19For Lexer's isAt[Start/End]OfMacroExpansion add an out parameter for the macroArgyrios Kyrtzidis
start/end location. It is commonly needed after calling the function; with this way we avoid recalculating it. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@148479 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-18Refactor: Pull getImmediateMacroName() out of DiagnosticRenderer andAnna Zaks
into Lexer and Preprocessor; making it widely available. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@148410 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-15Two variables had been added for an assert, but their values wereChandler Carruth
re-computed rather than the variables be re-used just after the assert. Just use the variables since we have them already. Fixes an unused variable warning. Also fix an 80-column violation. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@148212 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-22In Lexer::getCharAndSizeSlow[NoWarn] if we come up againstArgyrios Kyrtzidis
\<newline><newline> don't consume the second newline. Thanks to David Blaikie for pointing out the crash! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@147138 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-21In Lexer::getCharAndSizeSlow[NoWarn] make sure we don't go over the end of ↵Argyrios Kyrtzidis
the buffer when the end of the buffer is immediately after an escaped newline. Fixes http://llvm.org/PR10153. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@147091 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-20Unweaken vtables as per ↵David Blaikie
http://llvm.org/docs/CodingStandards.html#ll_virtual_anch git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@146959 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-22Remove assert from hot code path and add a clarifying comment.Benjamin Kramer
The assert wasn't adding much value but slowed down Release+Asserts builds. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@145082 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-22Lexer: Don't throw away the hard work SSE did to find a slash.Benjamin Kramer
We can reuse the information and avoid looping over all the bytes again. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@145070 91177308-0d34-0410-b5e6-96231b3b80d8
2011-10-17Move about 20 random diagnostics under -W flags. Patch by Ahmed Charles!Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@142284 91177308-0d34-0410-b5e6-96231b3b80d8
2011-10-15-Wc++98-compat warnings for the lexer.Richard Smith
This also adds a -Wc++98-compat-pedantic for warning on constructs which would be diagnosed by -std=c++98 -pedantic (that is, it warns even on C++11 features which we enable by default, with no warning, in C++98 mode). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@142034 91177308-0d34-0410-b5e6-96231b3b80d8
2011-10-12We do parse hexfloats in C++11; make it actually work.Douglas Gregor
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@141798 91177308-0d34-0410-b5e6-96231b3b80d8
2011-10-12Handle Perforce-style conflict markers like normal conflict markers. PerforceRichard Smith
swaps over the <<<< and >>>> markers, and uses shorter markers than traditional tools. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@141751 91177308-0d34-0410-b5e6-96231b3b80d8
2011-10-03Fixed exapnsion range for # and ##.Abramo Bagnara
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@141012 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-19Rename SourceLocation::getFileLocWithOffset -> getLocWithOffset.Argyrios Kyrtzidis
It already works (and is useful with) macro locs as well. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@140057 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-17Rename LangOptions::Microsoft to LangOptions::MicrosoftExt to make it clear ↵Francois Pichet
that this flag must be used only for Microsoft extensions and not emulation; to avoid confusion with the new LangOptions::MicrosoftMode flag. Many of the code now under LangOptions::MicrosoftExt will eventually be moved under the LangOptions::MicrosoftMode flag. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139987 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-05Speed up BCPL comment lexing by looking aggressively for newlines and then ↵Benjamin Kramer
scannig backwards to see if the newline is escaped. 3% speedup in preprocessing all of clang with -Eonly. Also includes a small testcase for coverage. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139116 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-05Use the Lexer's definition of whitespace here.Benjamin Kramer
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139115 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-04Support code-completion for C++ inline methods and ObjC buffering methods.Argyrios Kyrtzidis
Previously we would cut off the source file buffer at the code-completion point; this impeded code-completion inside C++ inline methods and, recently, with buffering ObjC methods. Have the code-completion inserted into the source buffer so that it can be buffered along with a method body. When we actually hit the code-completion point the cut-off lexing or parsing. Fixes rdar://10056932&8319466 git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139086 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-04Fix Lexer::ComputePreamble when MaxLines parameter is non-zero.Argyrios Kyrtzidis
The function was only counting lines that included tokens and not empty lines, but MaxLines (mainly initiated to the line where the code-completion point resides) is a count of overall lines (even empty ones). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139085 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-26Introduce support for a simple module import declaration, whichDouglas Gregor
loads the named module. The syntax itself is intentionally hideous and will be replaced at some later point with something more palatable. For now, we're focusing on the semantics: - Module imports are handled first by the preprocessor (to get macro definitions) and then the same tokens are also handled by the parser (to get declarations). If both happen (as in normal compilation), the second one is redundant, because we currently have no way to hide macros or declarations when loading a module. Chris gets credit for this mad-but-workable scheme. - The Preprocessor now holds on to a reference to a module loader, which is responsible for loading named modules. CompilerInstance is the only important module loader: it now knows how to create and wire up an AST reader on demand to actually perform the module load. - We search for modules in the include path, using the module name with the suffix ".pcm" (precompiled module) for the file name. This is a temporary hack; we hope to improve the situation in the future. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138679 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-25Make Lexer::ComputePreamble accept a LangOptions parameter, otherwise it may beArgyrios Kyrtzidis
out-of-sync how a file is compiled. Patch by Matthias Kleine! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138580 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-23In Lexer::isAtEndOfMacroExpansion use SourceManager::isInFileID and avoidArgyrios Kyrtzidis
the extra SourceManager::getFileID call. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138376 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-17Make Lexer::GetBeginningOfToken able to handle macro arg expansion locations.Argyrios Kyrtzidis
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@137795 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-11Add support for C++0x raw string literals.Craig Topper
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@137298 91177308-0d34-0410-b5e6-96231b3b80d8
2011-07-27Add a utility function to the Lexer, which makes it easier to find a token ↵Anna Zaks
after the given location. (It is a generalized version of trans::findLocationAfterSemi from ArcMigrate, which will be changed to use the Lexer utility). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@136268 91177308-0d34-0410-b5e6-96231b3b80d8
2011-07-27Add support for C++0x unicode string and character literals, from Craig Topper!Douglas Gregor
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@136210 91177308-0d34-0410-b5e6-96231b3b80d8
generated by cgit v1.2.3-18-g5258 (git 2.32.0) at 2025-07-17 20:25:05 +0000