aboutsummaryrefslogtreecommitdiff
path: root/lib/Lex
AgeCommit message (Collapse)Author
2008-12-23PTH:Ted Kremenek
- In PTHLexer::Lex read all of the token data from PTH file before constructing the token. The idea is to enhance locality. - Do not use Read8/Read32 in PTHLexer::Lex. Inline these operations manually. - Change PTHManager::ReadIdentifierInfo() to PTHManager::GetIdentifierInfo(). They are functionally the same except that PTHLexer::Lex() reads the persistent id. These changes result in a 3.3% speedup for PTH on Cocoa.h (-Eonly). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61363 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-23PTH:Ted Kremenek
- Embed 'eom' tokens in PTH file. - Use embedded 'eom' tokens to not lazily generate them in the PTHLexer. This means that PTHLexer can always advance to the next token after reading a token (instead of buffering tokens using a copy). - Moved logic of 'ReadToken' into Lex. GetToken & ReadToken no longer exist. - These changes result in a 3.3% speedup (-Eonly) on Cocoa.h. - The code is a little gross. Many cleanups are possible and should be done. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61360 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-18Don't define __STDC__ when compiling with -fms-extensionsSteve Naroff
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61223 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-18Use '&' to test StartOfLine flag.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61205 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-17Rewrite PTHLexer::DiscardToEndOfLine() to not use GetToken and instead only ↵Ted Kremenek
read the bytes needed to determine if a token is not at the start of the line. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61172 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-17Change PTHLexer::getSourceLocation() to not call GetToken() and instead just ↵Ted Kremenek
read the file offset in the token data buffer directly. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61170 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-17add a dropped word backChris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61152 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-17Shadow CurPtr with a local variable in ReadToken.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61145 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-15Preprocessor: Allocate MacroInfo objects using a BumpPtrAllocator instead ↵Ted Kremenek
using new/delete. This speeds up -Eonly on Cocoa.h using the regular lexer by 1.8% and the PTHLexer by 3%. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61042 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-13eliminate the isCXXNamedOperator function and some string compares andChris Lattner
use identifierinfo instead. Patch by Chris Goller! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60992 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-12PTH: Added minor 'sibling jumping' optimization for iterating over the side ↵Ted Kremenek
table used for fast preprocessor block skipping. This has a minor performance improvement when preprocessing Cocoa.h, but can have some wins in pathologic cases. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60966 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-12Added PTH optimization to not process entire blocks of tokens that appear in ↵Ted Kremenek
skipped preprocessor blocks. This improves PTH speed by 6%. The code for this optimization itself is not very optimized, and will get cleaned up. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60956 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-12rdar://6060752 - don't warn about trigraphs in bcpl-style commentsChris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60942 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-12fix thought-oChris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60937 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-11PTH:Ted Kremenek
- Added a side-table per each token-cached file with the preprocessor conditional stack. This tracks what #if's are matched with what #endifs and where their respective tokens are in the PTH file. This will allow for quick skipping of excluded conditional branches in the Preprocessor. - Performance testing shows the addition of this information (without actually utilizing it) leads to no performance regressions. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60911 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-11PTHLexer: Keep track of the location of the last '#' token and provide the ↵Ted Kremenek
means to jump ahead in the token stream. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60905 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-11Remove unused ivar CurTokenIdx.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60896 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-10PreprocessorLexer (and subclasses):Ted Kremenek
- Added virtual method 'getSourceLocation()' (no arguments) that gets the location of the next "observable" location (e.g., next character, next token). PPLexerChange.cpp: - Implemented FIXME by using PreprocessorLexer::getSourceLocation() to get the location in the file we are returning to after lexing a #included file. This appears to be slightly faster than having the branch (i.e., 'if(CurLexer)'). It's also not a really hot part of the Preprocessor. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60860 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-10Declare PerIDCache as IdentifierInfo** instead of void*. This is just ↵Ted Kremenek
cleaner. No performance change. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60843 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-04Remove unneeded assertion.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60559 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-04Use 'free' to release PerIDCache since it was allocated using calloc().Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60556 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-03PTH:Ted Kremenek
Use an array instead of a DenseMap to cache persistent IDs -> IdentifierInfo*. This leads to a 4% speedup at -fsyntax-only using PTH. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60452 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-03- Remove PTHManager.cpp. Move all of its functions to PTHLexer.cpp since ↵Ted Kremenek
some of the internal methods are used by PTHLexer (their implementations are intertwined.) This enables some important inlining opportunities at -O3. - Don't construct an std::vector<Token> prior to feeding PTH tokens to the Preprocessor. Stream them off the PTH file directly. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60447 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-02Preprocessor:Ted Kremenek
- Added method "setPTHManager" that will be called by the driver to install a PTHManager for the Preprocessor. - Fixed some comments. - Added EnterSourceFileWithPTH to mirror EnterSourceFileWithLexer. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60437 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-02Added PTHManager, a utility class that will be used by Preprocessor to ↵Ted Kremenek
lazily create PTHLexer objects for pre-tokenized files. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60436 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-01Objective-C keywords are not always identifiers. Some are also C++ keywordsDouglas Gregor
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60373 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-01Add LangOptions marker for assembler-with-cpp mode and use to defineDaniel Dunbar
__ASSEMBLER__ properly. Patch from Roman Divacky (with minor formatting changes). Thanks! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60362 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-27PTHLexer now owns the Token vector.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60136 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-25Comment fix.Daniel Dunbar
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59997 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-23make the 'to match this' diagnostic a note.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59921 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-23Convert IdentifierInfo's to be printed the same as DeclarationNames Chris Lattner
with implicit quotes around them. This has a bunch of follow-on effects and requires tweaking to a whole lot of code. This causes a regression in two tests (xfailed) by causing it to emit things like: Line 10: duplicate interface declaration for category 'MyClass1' ('Category1') instead of: Line 10: duplicate interface declaration for category 'MyClass1(Category1)' I will fix this in a follow-up commit. As part of this, I had to start switching stuff to use ->getDeclName() instead of Decl::getName() for consistency. This is good, but I was planning to do this as an independent patch. There will be several follow-on patches to clean up some of the mess, but this patch is already too big. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59917 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-22Fix a weird inconsistency with hex floats. Previously the lexer Chris Lattner
would not eat the "-1" in "0x0p-1", but LiteralSupport would accept it when extensions are on. This caused strangeness and failures when hexfloats were properly treated as an extension (not error) in LiteralSupport. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59865 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-22remove the NumericLiteralParser::Diag helper method, inlining it intoChris Lattner
its call sites. This makes it more explicit when the hasError flag is getting set and removes a confusing difference in behavior between PP.Diag and Diag in this code. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59863 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-22Move the Preprocessor::Diag methods inline. This has the interestingChris Lattner
(and carefully calculated) effect of allowing the compiler to reason about the aliasing properties of DiagnosticBuilder object better, allowing the whole thing to be promoted to registers instead of resulting in a ton of stack traffic. While I'm not very concerned about the performance of the Diag() method invocations, I *am* more concerned about their code size and impact on the non-diagnostic code. This patch shrinks the clang executable (in release-asserts mode with gcc-4.2) from 14523980 to 14519816 bytes. This isn't much, but it shrinks the lexer from 38192 to 37776, PPDirectives.o from 31116 to 28868 bytes, etc. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59862 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-22inline a method into its only two call sites.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59860 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-22actually, this version isn't really needed.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59859 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-22remove a sneaky version of Diag hiding in PreprocessorLexer.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59858 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-22Change the Lexer::Diag method to not magically silence warnings,Chris Lattner
force the caller to check instead. This eliminates the need (and the risk!) of weird null DiagnosticBuilder's floating around. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59856 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-22Split the DiagnosticInfo class into two disjoint classes:Chris Lattner
one for building up the diagnostic that is in flight (DiagnosticBuilder) and one for pulling structured information out of the diagnostic when formatting and presenting it. There is no functionality change with this patch. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59849 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-21In PTHLexer::DiscardToEndOfLine() use Lex() instead of AdvanceToken(). This ↵Ted Kremenek
handles transitions in the preprocessor state. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59845 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-21Reenable the default lexer.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59843 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-21When creating the raw tokens for PTHLexer, make sure the token representing ↵Ted Kremenek
the file to include is checked for being an identifier. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59842 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-21When creating raw tokens for the PTHLexer specially handle angled strings ↵Ted Kremenek
for #include directives. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59840 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-21PTHLexer:Ted Kremenek
- Move out logic for handling the end-of-file to LexEndOfFile (to match the Lexer) class. The logic now mirrors the Lexer class more, which allows us to pass most of the Preprocessor test cases. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59768 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-20PTHLexer:Ted Kremenek
- Move PTHLexer::GetToken() to be inside PTHLexer.cpp. - When lexing in raw mode, null out identifiers. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59744 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-20Handle another case where we should use PTHLexer as an alternative to the ↵Ted Kremenek
normal Lexer. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59736 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-20PTHLexer:Ted Kremenek
- Rename 'CurToken' and 'LastToken' to 'CurTokenIdx' and 'LastTokenIdx' respectively. - Add helper methods GetToken(), AdvanceToken(), AtLastToken() to abstract away details of the token stream. This also allows us to easily replace their implementation later. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59733 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-20Rename IsNonPragmaNonMacroLexer to IsFileLexer.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59731 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-20Rewrote PTHLexer::Lex by digging through the sources of Lexer again. Now we ↵Ted Kremenek
can do basic macro expansion using the PTHLexer. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59724 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-20Add ugly "test harness" for PTHLexer that is not enabled by default. TheTed Kremenek
(temporary hack) to test the PTHLexer is that whenever we would create a Lexer object we instead raw lex a memory buffer first and then use the PTHLexer. This logic exists only to driver the PTHLexer and will be removed/changed in the future. Note that the regular path using normal Lexer objects is what is used by default. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59723 91177308-0d34-0410-b5e6-96231b3b80d8
generated by cgit v1.2.3-18-g5258 (git 2.32.0) at 2025-07-17 08:18:56 +0000