aboutsummaryrefslogtreecommitdiff
path: root/lib/Lex/PTHLexer.cpp
AgeCommit message (Collapse)Author
2009-01-15PTH: Fix termination condition in binary search.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62277 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-15IdentifierInfo:Ted Kremenek
- IdentifierInfo can now (optionally) have its string data not be co-located with itself. This is for use with PTH. This aspect is a little gross, as getName() and getLength() now make assumptions about a possible alternate representation of IdentifierInfo. Perhaps we should make IdentifierInfo have virtual methods? IdentifierTable: - Added class "IdentifierInfoLookup" that can be used by IdentifierTable to perform "string -> IdentifierInfo" lookups using an auxilliary data structure. This is used by PTH. - Perform tests show that IdentifierTable::get() does not slow down because of the extra check for the IdentiferInfoLookup object (the regular StringMap lookup does enough work to mitigate the impact of an extra null pointer check). - The upshot is that now that some IdentifierInfo objects might be owned by the IdentiferInfoLookup object. This should be reviewed. PTH: - Modified PTHManager::GetIdentifierInfo to *not* insert entries in IdentifierTable's string map, and instead create IdentifierInfo objects on the fly when mapping from persistent IDs to IdentifierInfos. This saves a ton of work with string copies, hashing, and StringMap lookup and resizing. This change was motivated because when processing source files in the PTH cache we don't need to do any string -> IdentifierInfo lookups. - PTHManager now subclasses IdentifierInfoLookup, allowing clients of IdentifierTable to transparently use IdentifierInfo objects managed by the PTH file. PTHManager resolves "string -> IdentifierInfo" queries by doing a binary search over a sorted table of identifier strings in the PTH file (the exact algorithm we use can be changed as needed). These changes lead to the following performance changes when using PTH on Cocoa.h: - fsyntax-only: 10% performance improvement - Eonly: 30% performance improvement git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62273 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-15PTH: Embed a persistentID side-table in the PTH file that is sorted in theTed Kremenek
lexical order of the corresponding identifier strings. This will be used for a forthcoming optimization. This slows down PTH generation time by 7%. We can revert this change if the optimization proves to not be valuable. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62248 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-13PTH:Ted Kremenek
- Use canonical FileID when using getSpelling() caching. This addresses some cache misses we were seeing with -fsyntax-only on Cocoa.h - Added Preprocessor::getPhysicalCharacterAt() utility method for clients to grab the first character at a specified sourcelocation. This uses the PTH spelling cache. - Modified Sema::ActOnNumericConstant() to use Preprocessor::getPhysicalCharacterAt() instead of SourceManager::getCharacterData() (to get PTH hits). These changes cause -fsyntax-only to not page in any sources from Cocoa.h. We see a speedup of 27%. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62193 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-13Fix corner cases in PTH getSpelling() binary search.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62187 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-13PTH: Fix remaining cases where the spelling cache in the PTH file was being ↵Ted Kremenek
missed when it shouldn't. This shaves another 7% off PTH time for -Eonly on Cocoa.h git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62186 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-09Enhance PTH 'getSpelling' caching:Ted Kremenek
- Refactor caching logic into a helper class PTHSpellingSearch - Allow "random accesses" in the spelling cache, thus catching the remaining cases where 'getSpelling' wasn't hitting the PTH cache For -Eonly, PTH, Cocoa.h: - This reduces wall time by 3% (user time unchanged, sys time reduced) - This reduces the amount of paged source by 1112K. The remaining 1112K still being paged in is from somewhere else (investigating). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62009 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-09Invert assertion condition.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61961 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-08PTH: Hook up getSpelling() caching in PTHLexer. This results in a niceTed Kremenek
performance gain. Here's what we see for -Eonly on Cocoa.h (using PTH): - wall time decreases by 21% (26% speedup overall) - system time decreases by 35% - user time decreases by 6% These reductions are due to not paging source files just to get spellings for literals. The solution in place doesn't appear to be 100% yet, as we still see some of the pages for source files getting mapped in. Using -print-stats, we see that SourceManager maps in 7179K less bytes of source text (reduction of 75%). Will investigate why the remaining 25% are getting paged in. With these changes, here's how PTH compares to non-PTH on Cocoa.h: -Eonly: PTH takes 64% of the time as non-PTH (54% speedup) -fsyntax-only: PTH takes 89% of the time as non-PTH (11% speedup) git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61913 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-08PTH:Ted Kremenek
- Added stub PTHLexer::getSpelling() that will be used for fetching cached spellings from the PTH file. This doesn't do anything yet. - Added a hook in Preprocessor::getSpelling() to call PTHLexer::getSpelling() when using a PTHLexer. - Updated PTHLexer to read the offsets of spelling tables in the PTH file. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61911 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-23PTH: Remove some methods and simplify some conditions in PTHLexer::Lex(). ↵Ted Kremenek
No big functionality change. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61381 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-23PTH: Use 3 bytes instead of 4 bytes to encode the persistent ID for a token.Ted Kremenek
- This reduces the PTH size for Cocoa.h by 7%. - The increases PTH -Eonly speed for Cocoa.h by 0.8%. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61377 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-23Cosmetics: rename a variable and tighten spacing. No functionality change.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61375 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-23PTH:Ted Kremenek
- Encode the token length with 2 bytes instead of 4. - This reduces the size of the .pth file for Cocoa.h by 12%. - This speeds up PTH time (-Eonly) on Cocoa.h by 1.6%. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61364 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-23PTH:Ted Kremenek
- In PTHLexer::Lex read all of the token data from PTH file before constructing the token. The idea is to enhance locality. - Do not use Read8/Read32 in PTHLexer::Lex. Inline these operations manually. - Change PTHManager::ReadIdentifierInfo() to PTHManager::GetIdentifierInfo(). They are functionally the same except that PTHLexer::Lex() reads the persistent id. These changes result in a 3.3% speedup for PTH on Cocoa.h (-Eonly). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61363 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-23PTH:Ted Kremenek
- Embed 'eom' tokens in PTH file. - Use embedded 'eom' tokens to not lazily generate them in the PTHLexer. This means that PTHLexer can always advance to the next token after reading a token (instead of buffering tokens using a copy). - Moved logic of 'ReadToken' into Lex. GetToken & ReadToken no longer exist. - These changes result in a 3.3% speedup (-Eonly) on Cocoa.h. - The code is a little gross. Many cleanups are possible and should be done. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61360 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-18Use '&' to test StartOfLine flag.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61205 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-17Rewrite PTHLexer::DiscardToEndOfLine() to not use GetToken and instead only ↵Ted Kremenek
read the bytes needed to determine if a token is not at the start of the line. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61172 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-17Change PTHLexer::getSourceLocation() to not call GetToken() and instead just ↵Ted Kremenek
read the file offset in the token data buffer directly. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61170 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-17add a dropped word backChris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61152 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-17Shadow CurPtr with a local variable in ReadToken.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61145 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-12PTH: Added minor 'sibling jumping' optimization for iterating over the side ↵Ted Kremenek
table used for fast preprocessor block skipping. This has a minor performance improvement when preprocessing Cocoa.h, but can have some wins in pathologic cases. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60966 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-12Added PTH optimization to not process entire blocks of tokens that appear in ↵Ted Kremenek
skipped preprocessor blocks. This improves PTH speed by 6%. The code for this optimization itself is not very optimized, and will get cleaned up. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60956 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-11PTH:Ted Kremenek
- Added a side-table per each token-cached file with the preprocessor conditional stack. This tracks what #if's are matched with what #endifs and where their respective tokens are in the PTH file. This will allow for quick skipping of excluded conditional branches in the Preprocessor. - Performance testing shows the addition of this information (without actually utilizing it) leads to no performance regressions. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60911 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-11PTHLexer: Keep track of the location of the last '#' token and provide the ↵Ted Kremenek
means to jump ahead in the token stream. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60905 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-11Remove unused ivar CurTokenIdx.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60896 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-10Declare PerIDCache as IdentifierInfo** instead of void*. This is just ↵Ted Kremenek
cleaner. No performance change. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60843 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-04Remove unneeded assertion.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60559 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-04Use 'free' to release PerIDCache since it was allocated using calloc().Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60556 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-03PTH:Ted Kremenek
Use an array instead of a DenseMap to cache persistent IDs -> IdentifierInfo*. This leads to a 4% speedup at -fsyntax-only using PTH. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60452 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-03- Remove PTHManager.cpp. Move all of its functions to PTHLexer.cpp since ↵Ted Kremenek
some of the internal methods are used by PTHLexer (their implementations are intertwined.) This enables some important inlining opportunities at -O3. - Don't construct an std::vector<Token> prior to feeding PTH tokens to the Preprocessor. Stream them off the PTH file directly. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60447 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-27PTHLexer now owns the Token vector.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60136 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-21In PTHLexer::DiscardToEndOfLine() use Lex() instead of AdvanceToken(). This ↵Ted Kremenek
handles transitions in the preprocessor state. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59845 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-21PTHLexer:Ted Kremenek
- Move out logic for handling the end-of-file to LexEndOfFile (to match the Lexer) class. The logic now mirrors the Lexer class more, which allows us to pass most of the Preprocessor test cases. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59768 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-20PTHLexer:Ted Kremenek
- Move PTHLexer::GetToken() to be inside PTHLexer.cpp. - When lexing in raw mode, null out identifiers. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59744 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-20PTHLexer:Ted Kremenek
- Rename 'CurToken' and 'LastToken' to 'CurTokenIdx' and 'LastTokenIdx' respectively. - Add helper methods GetToken(), AdvanceToken(), AtLastToken() to abstract away details of the token stream. This also allows us to easily replace their implementation later. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59733 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-20Rewrote PTHLexer::Lex by digging through the sources of Lexer again. Now we ↵Ted Kremenek
can do basic macro expansion using the PTHLexer. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59724 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-20- Default initialize ParsingPreprocessorDirective, ParsingFilename, andTed Kremenek
LexingRawMode in the ctor of PreprocessorLexer. - PTHLexer: Use "LastToken" instead of "NumToken" git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59690 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-20Add (untested) implementation of PTHLexer::isNextPPTokenLParen() and ↵Ted Kremenek
PTHLexer::DiscardToEndOfLine(). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59687 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-19Add stub for PTHLexer::isNextPPTokenLParen().Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59670 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-19When using a PTHLexer, use DiscardToEndOfLine() instead of ReadToEndOfLine().Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59668 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-19- Move static function IsNonPragmaNonMacroLexer into Preprocessor.h.Ted Kremenek
- Add variants of IsNonPragmaNonMacroLexer to accept an IncludeMacroStack entry (simplifies some uses). - Use IsNonPragmaNonMacroLexer in Preprocessor::LookupFile. - Add 'FileID' to PreprocessorLexer, and have Preprocessor query this fileid when looking up the FileEntry for a file Performance testing of -Eonly on Cocoa.h shows no performance regression because of this patch. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59666 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-16Trivial tidyingChris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59424 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-12Move some diagnostic handling to PreprocessorLexer.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59191 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-12Add skeleton for PTH lexer.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59169 91177308-0d34-0410-b5e6-96231b3b80d8
generated by cgit v1.2.3-18-g5258 (git 2.32.0) at 2025-05-04 01:14:54 +0000