aboutsummaryrefslogtreecommitdiff
path: root/include/clang/Lex
AgeCommit message (Collapse)Author
2008-11-12Move LexIncludeFilename from Lexer to PreprocessorLexer.Ted Kremenek
PreprocessorLexer now has a virtual method "IndirectLex" which allows it to call the lex method of its subclasses. This is not for performance intensive operations. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59185 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-12Unbreak last commit.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59180 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-12Add Preprocessor::PushIncludeMacroStack() and ↵Ted Kremenek
Preprocessor::PopIncludeMacroStack(), two utility methods for manipulating the Preprocessor stack. These will be used to remove manually manipulation of IncludeMacroStack from the rest of the Preprocessor implementation. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59179 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-12Add skeleton for PTH lexer.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59169 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-12Move pieces of Lexer that the Preprocessor mutates to a new base class ↵Ted Kremenek
'PreprocessorLexer'. This will also be the base class of the new Preprocessed-Token-Header (PTH) lexer. No functionality change. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59168 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-08Introduce annotation tokens, a special kind of token, created and used only ↵Argyrios Kyrtzidis
by the parser to replace a group of tokens with a single token encoding semantic information. Will be fully utilized later for C++ nested-name-specifiers. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@58911 91177308-0d34-0410-b5e6-96231b3b80d8
2008-10-28clarify comment, rename argument to avoid a subtle conflictChris Lattner
with an ivar that wasn't a bug but was confusing. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@58311 91177308-0d34-0410-b5e6-96231b3b80d8
2008-10-27Rename Characteristic_t to CharacteristicKindChris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@58224 91177308-0d34-0410-b5e6-96231b3b80d8
2008-10-21Added method to access the raw flags of Token.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57877 91177308-0d34-0410-b5e6-96231b3b80d8
2008-10-12Change FormTokenWithChars to take the token kind to form, since all clientsChris Lattner
were setting a kind and then forming it. This is just a minor API cleanup, no functionality change. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57404 91177308-0d34-0410-b5e6-96231b3b80d8
2008-10-12Add a new mode to the lexer which enables it to return all characters,Chris Lattner
even whitespace, as tokens from the file. This is enabled with L->SetKeepWhitespaceMode(true) on a raw lexer. In this mode, you too can use clang as a really complex version of 'cat' with code like this: Lexer RawLex(SourceLocation::getFileLoc(SM.getMainFileID(), 0), PP.getLangOptions(), File.first, File.second); RawLex.SetKeepWhitespaceMode(true); Token RawTok; RawLex.LexFromRawLexer(RawTok); while (RawTok.isNot(tok::eof)) { std::cout << PP.getSpelling(RawTok); RawLex.LexFromRawLexer(RawTok); } This will emit exactly the input file, with no canonicalization or other translation. Realistic clients actually do something with the tokens of course :) git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57401 91177308-0d34-0410-b5e6-96231b3b80d8
2008-10-12add a new inKeepCommentMode() accessor to abstract the KeepCommentModeChris Lattner
ivar. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57397 91177308-0d34-0410-b5e6-96231b3b80d8
2008-10-12Simplify raw mode lexing by treating an unterminate /**/ comment theChris Lattner
same we we do an unterminated string or character literal. This makes it so we can guarantee that the lexer never calls into the preprocessor (which would be suicide for a raw lexer). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57395 91177308-0d34-0410-b5e6-96231b3b80d8
2008-10-12Change how raw lexers are handled: instead of creating them and thenChris Lattner
using LexRawToken, create one and use LexFromRawLexer. This avoids twiddling the RawLexer flag around and simplifies some code (even speeding raw lexing up a tiny bit). This change also improves the token paster to use a Lexer on the stack instead of new/deleting it. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57393 91177308-0d34-0410-b5e6-96231b3b80d8
2008-10-04Add Preprocessor::RemovePragmaHandler.Daniel Dunbar
- No functionality change. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57065 91177308-0d34-0410-b5e6-96231b3b80d8
2008-09-26clean up a bunch of fixme's I added, by moving Chris Lattner
DirectoryLookup::DirType into SourceManager.h git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@56692 91177308-0d34-0410-b5e6-96231b3b80d8
2008-09-26Fix the rest of rdar://6243860 hopefully. This requires changing FileIDInfoChris Lattner
to whether the fileid is a 'extern c system header' in addition to whether it is a system header, most of this is spreading plumbing around. Once we have that, PPLexerChange bases its "file enter/exit" notifications to PPCallbacks to base the system header state on FileIDInfo instead of HeaderSearch. Finally, in Preprocessor::HandleIncludeDirective, mirror logic in GCC: the system headerness of a file being entered can be set due to the #includer or the #includee. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@56688 91177308-0d34-0410-b5e6-96231b3b80d8
2008-09-26Comment fix.Daniel Dunbar
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@56634 91177308-0d34-0410-b5e6-96231b3b80d8
2008-08-24Rename Preprocessor::DisableBacktrack -> Preprocessor::CommitBacktrackedTokens.Argyrios Kyrtzidis
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@55281 91177308-0d34-0410-b5e6-96231b3b80d8
2008-08-24Comment tweak.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@55272 91177308-0d34-0410-b5e6-96231b3b80d8
2008-08-22Allow nested backtracks.Argyrios Kyrtzidis
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@55204 91177308-0d34-0410-b5e6-96231b3b80d8
2008-08-14Change Parser & Sema to use interned "super" for comparions.Daniel Dunbar
- Added as private members for each because it is not clear where to put the common definition. Perhaps the IdentifierInfos all of these "pseudo-keywords" should be collected into one place (this would KnownFunctionIDs and Objective-C property IDs, for example). Remove Token::isNamedIdentifier. - There isn't a good reason to use strcmp when we have interned strings, and there isn't a good reason to encourage clients to do so. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@54794 91177308-0d34-0410-b5e6-96231b3b80d8
2008-08-10* Remove isInSystemHeader() from DiagClient, move it to SourceManagerNico Weber
* Move FormatError() from TextDiagnostic up to DiagClient, remove now empty class TextDiagnostic * Make DiagClient optional for Diagnostic This fixes the following problems: * -html-diags (and probably others) does now output the same set of warnings as console clang does * nothing crashes if one forgets to call setHeaderSearch() on TextDiagnostic * some code duplication is removed git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@54620 91177308-0d34-0410-b5e6-96231b3b80d8
2008-08-10Allow the preprocessor to cache the lexed tokens, so that we can do ↵Argyrios Kyrtzidis
efficient lookahead and backtracking. 1) New public methods added: -EnableBacktrackAtThisPos -DisableBacktrack -Backtrack -isBacktrackEnabled 2) LookAhead() implementation is replaced with a more efficient one. 3) LookNext() is removed. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@54611 91177308-0d34-0410-b5e6-96231b3b80d8
2008-07-10Add an accessor, patch by Csaba Hruska.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@53391 91177308-0d34-0410-b5e6-96231b3b80d8
2008-07-09Add Preprocessor::LookNext method, which implements an efficient way to ↵Argyrios Kyrtzidis
'take a peek' at the next token without consuming it. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@53375 91177308-0d34-0410-b5e6-96231b3b80d8
2008-06-30refactor some code out into a new method.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@52889 91177308-0d34-0410-b5e6-96231b3b80d8
2008-05-05Neil pointed out that clang doesn't generate ranges from diagnosticsChris Lattner
related to pp-expressions. Doing so is pretty simple and this patch implements it, yielding nice diagnostics like: t.c:2:7: error: division by zero in preprocessor expression #if 1 / (0 + 0) ~ ^ ~~~~~~~ t.c:5:14: error: expected ')' in preprocessor expression #if (412 + 42 ~~~~~~~~^ t.c:5:5: error: to match this '(' #if (412 + 42 ^ t.c:10:10: warning: left side of operator converted from negative value to unsigned: -42 to 18446744073709551574 #if (-42 + 0U) / -2 ~~~ ^ ~~ t.c:10:16: warning: right side of operator converted from negative value to unsigned: -2 to 18446744073709551614 #if (-42 + 0U) / -2 ~~~~~~~~~~ ^ ~~ 5 diagnostics generated. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@50638 91177308-0d34-0410-b5e6-96231b3b80d8
2008-04-19simplify ownership of the predefines buffer.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@49973 91177308-0d34-0410-b5e6-96231b3b80d8
2008-04-17class Preprocessor: Now owns the "predefines" char*; it deletes [] it in its ↵Ted Kremenek
dstor. clang.cpp: InitializePreprocessor now makes a copy of the contents of PredefinesBuffer and passes it to the preprocessor object. clang.cpp: DriverPreprocessorFactory now calls "InitializePreprocessor" instead of this being done in main(). html::HighlightMacros() now takes a PreprocessorFactory, allowing it to conjure up a new Preprocessor to highlight macros. class HTMLDiagnostics now takes a PreprocessorFactory* that it can use for html::HighlightMacros(). Updated clients of HTMLDiagnostics to use this new interface. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@49875 91177308-0d34-0410-b5e6-96231b3b80d8
2008-04-17Added "PreprocessorFactory", an interface for lazily creating Preprocessor ↵Ted Kremenek
objects on-demand. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@49868 91177308-0d34-0410-b5e6-96231b3b80d8
2008-04-16reenable highlighting of (the first line of) commentsChris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@49816 91177308-0d34-0410-b5e6-96231b3b80d8
2008-03-14Make the preprocessor own its PPCallbacks, fixing a memory leak.Chris Lattner
Patch by Sam Bishop! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@48357 91177308-0d34-0410-b5e6-96231b3b80d8
2008-03-10implement simple support for arbitrary token lookahead. Change the Chris Lattner
objc @try parser to use it, fixing a FIXME. Update the objc-try-catch-1.m file to pass now that we get more reasonable errors. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@48129 91177308-0d34-0410-b5e6-96231b3b80d8
2008-03-09rename HandleEndOfMacro -> HandleEndOfTokenLexerChris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@48076 91177308-0d34-0410-b5e6-96231b3b80d8
2008-03-09split the MacroArgs class out of TokenLexer.cpp/h intoChris Lattner
MacroArgs.cpp/h git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@48075 91177308-0d34-0410-b5e6-96231b3b80d8
2008-03-09rename some MacroExpander-related ivars to TokenLexer.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@48073 91177308-0d34-0410-b5e6-96231b3b80d8
2008-03-09Rename MacroExpander.cpp/h -> TokenLexer.cpp/hChris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@48072 91177308-0d34-0410-b5e6-96231b3b80d8
2008-03-09rename the MacroExpander class to TokenLexer. It handles bothChris Lattner
token streams and macro lexing, so a more generic name is useful. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@48071 91177308-0d34-0410-b5e6-96231b3b80d8
2008-03-09rename MacroTokens -> Tokens. When this is a token stream, there is no macroChris Lattner
involved. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@48070 91177308-0d34-0410-b5e6-96231b3b80d8
2008-03-05Remove the first layer of support for "portability" warnings. This is Chris Lattner
theoretically useful, but not useful in practice. It adds a bunch of complexity, and not much value. It's best to nuke it. One big advantage is that it means the target interfaces will soon lose their SLoc arguments and target queries can never emit diagnostics anymore (yay). Removing this also simplifies some of the core preprocessor which should make it slightly faster. Ted, I didn't simplify TripleProcessor, which can now have at most one triple, and can probably just be removed. Please poke at it when you have time. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@47930 91177308-0d34-0410-b5e6-96231b3b80d8
2008-02-25Fix PR2086.Lauro Ramos Venancio
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@47551 91177308-0d34-0410-b5e6-96231b3b80d8
2008-02-24Two more Windows-related fixes:Ted Kremenek
- More enum signeness bitfield fixes (MSVC treats enums as signed). - Fixed in Lex/HeaderSearch.cpp an unsafe copy between two HeaderSearch::PerFileInfo entries in a common vector. The copy involved two calls to getFileInfo() within the assignment; these calls could have side-effects that enlarged the internal vector, and with MSVC this would invalidate one of the values in the assignment. Patch by Argiris Kirtzidis! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@47536 91177308-0d34-0410-b5e6-96231b3b80d8
2008-02-23Change encoding of TokenKind in IdentifierTable to be of type "unsigned"Ted Kremenek
instead of TokenKind because of signedness issues with MSVC and enums. Patch from Argiris Kirtzidis. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@47515 91177308-0d34-0410-b5e6-96231b3b80d8
2008-02-07Implement support for the extremely atrocious MS /##/ extension, Chris Lattner
which pastes together a comment. This is only enabled with -fms-extensions of course. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@46845 91177308-0d34-0410-b5e6-96231b3b80d8
2008-02-02Cleanup previous patch (based on feedback from Ted).Steve Naroff
Since this behavior is useful for most classes, we might consider adding a simple 3 method class that implements the behavior. Ted said that Boost has such a class. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@46654 91177308-0d34-0410-b5e6-96231b3b80d8
2008-02-01Make sure SourceManager/HeaderSearch don't support default copy constructors ↵Steve Naroff
(since they result in bad runtime behavior). I'm sure there are other classes that might need this "guard", however I was bitten by these 2 recently (so I thought I'd fix them). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@46653 91177308-0d34-0410-b5e6-96231b3b80d8
2008-01-29fix comment typoChris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@46505 91177308-0d34-0410-b5e6-96231b3b80d8
2008-01-08readability improvement suggested by Sam Bishop, thanks!Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@45735 91177308-0d34-0410-b5e6-96231b3b80d8
2008-01-07Fix a nasty corner case that Neil noticed in PR1900, where we would Chris Lattner
incorrectly apply the multiple include optimization to files with guards like: #if !defined(x) MACRO where MACRO could expand to different things in different contexts. Thanks Neil! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@45716 91177308-0d34-0410-b5e6-96231b3b80d8
generated by cgit v1.2.3-18-g5258 (git 2.32.0) at 2025-07-12 23:26:41 +0000