aboutsummaryrefslogtreecommitdiff
path: root/lib/Lex/PTHLexer.cpp
AgeCommit message (Collapse)Author
2009-04-20Move the on-disk hash table code into its own header. No functionality change.Douglas Gregor
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69580 91177308-0d34-0410-b5e6-96231b3b80d8
2009-03-22PTHManager::Create():Ted Kremenek
- Make the Diagnostic::Level for PTH errors to be specified by the caller clang (driver): - Set the PTHManager diagnostic level to "Diagnostic::Error" for -include-pth (a hard error) and Diagnostic::Warning for -token-cache (we can still proceed). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@67462 91177308-0d34-0410-b5e6-96231b3b80d8
2009-03-21Add back warning about a PTH file not containing any identifiers, but don't makeTed Kremenek
it a hard error. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@67424 91177308-0d34-0410-b5e6-96231b3b80d8
2009-03-21Allow PTH files with no identifiers.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@67423 91177308-0d34-0410-b5e6-96231b3b80d8
2009-03-20PTHManager::Create() now creates a PTHManager even if the PTH file contains noTed Kremenek
cached tokens. This is for use with -include-pth. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@67385 91177308-0d34-0410-b5e6-96231b3b80d8
2009-03-19Add PTHManager::getOriginalSourceFile(), a method that returns the name of theTed Kremenek
original source file (if any) that was used to generate the PTH cache. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@67343 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-23Move PTHStatCache within the anonymous namespace.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@65348 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-19Fix another PTH warning that should not be a note.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@65072 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-19Make PTH warnings actual warnings instead of 'notes'.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@65071 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-13PTH: Cache directory and negative 'stat' calls. This gives us a 1% ↵Ted Kremenek
performance improvement on Cocoa.h (fsyntax-only+PTH). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64490 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-13Add some boilerplate to the PTH file to prepare for the caching of stats for ↵Ted Kremenek
directories (and negative stats too). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64477 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-13Fix gcc warning: gcc correctly notes that const-qualifying the return Eli Friedman
type doesn't do anything. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64424 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-12Fix assertion when input is an empty string.Daniel Dunbar
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64397 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-12Re-enable PTH stat caching. All tests pass now.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64356 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-12Fix bad reading of bytes in ReadUnalignedLE64() (copy-paste error).Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64355 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-12Temporarily disable PTH stat caching as it appears to be failing on some ↵Ted Kremenek
machines. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64354 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-12PTH: Cache stat information for files in the PTH file. Hook up FileManagerTed Kremenek
to use this stat information in the PTH file using a 'StatSysCallCache' object. Performance impact (Cocoa.h, PTH): - number of stat calls reduces from 1230 to 425 - fsyntax-only: time improves by 4.2% We can reduce the number of stat calls to almost zero by caching negative stat calls and directory stat calls in the PTH file as well. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64353 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-11PTH: Have meta data be at the beginning of the PTH file, not the end.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64338 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-11PTH: Replace string identifier to persistent ID lookup with a hashtable. This isTed Kremenek
actually *slightly* slower than the binary search. Since this is algorithmically better, further performance tuning should be able to make this faster. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64326 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-11PTH: Don't emit the PTH offset of the IdentifierInfo string data as that data isTed Kremenek
referenced by other tables. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64304 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-10PTH: Replace ad hoc 'file name' -> 'PTH data' lookup table in the PTH file ↵Ted Kremenek
with an on-disk chained hash table. This data structure is implemented using templates, and will be used to replace similar data structures. This change leads to no visibile performance impact on Cocoa.h, but now we only pay a price for the table on order with the number of files accessed and not the number in the PTH file. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64245 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-28Add more PTH diagnostics for invalid PTH files, etc.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63232 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-28Enhance PTHManager::Create() to take an optional Diagnostic* argument that ↵Ted Kremenek
can be used to report issues such as a missing PTH file. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63231 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-27PTH: Use Token::setLiteralData() to directly store a pointer to cached ↵Ted Kremenek
spelling data in the PTH file. This removes a ton of code for looking up spellings using sourcelocations in the PTH file. This simplifies both PTH-generation and reading. Performance impact for -fsyntax-only on Cocoa.h (with Cocoa.h in the PTH file): - PTH generation time improves by 5% - PTH reading improves by 0.3%. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63072 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-26Silence warning.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63054 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-26Add version number checking to PTH files.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63047 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-26Embed the offset of the PTH table inside the prologue of the PTH file. This ↵Ted Kremenek
will help improve gradual versioning of PTH files instead of relying that the PTH table is at a fixed offset. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63045 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-26Check in the long promised SourceLocation rewrite. This lays theChris Lattner
ground work for implementing #line, and fixes the "out of macro ID's" problem. There is nothing particularly tricky about the code, other than the very performance sensitive SourceManager::getFileID() method. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62978 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-23This is a follow-up to r62675:Chris Lattner
Refactor how the preprocessor changes a token from being an tok::identifier to a keyword (e.g. tok::kw_for). Instead of doing this in HandleIdentifier, hoist this common case out into the caller, so that every keyword doesn't have to go through HandleIdentifier. This drops time in HandleIdentifier from 1.25ms to .62ms, and speeds up clang -Eonly with PTH by about 1%. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62855 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-23Update comment.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62819 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-22remove my gross #ifdef's, using portable abstractions now that the 32-bitChris Lattner
load is always aligned. I verified that the bswap doesn't occur in the assembly code on x86. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62815 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-22remove Read8/Read24, which are dead. Rename Read16/Read32 to be moreChris Lattner
descriptive. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62775 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-21Fix <rdar://problem/6512717> by correctly reading the right offset in the ↵Ted Kremenek
token data in PTHLexer::getSourceLocation(). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62725 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-21merge two checks for identifiers in the pth loop into one.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62677 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-21Add a bit to IdentifierInfo that acts as a simple predicate whichChris Lattner
tells us whether Preprocessor::HandleIdentifier needs to be called. Because this method is only rarely needed, this saves a call and a bunch of random checks. This drops the time in HandleIdentifier from 3.52ms to .98ms on cocoa.h on my machine. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62675 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-21Don't crash on empty PTH files. This fixes <rdar://problem/6512714>.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62673 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-21really we only need on Read24!Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62672 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-21revert my previous patch, it assumed endianness.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62671 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-21minor cleanups: now that tokens are 4-byte aligned in a PTH Chris Lattner
file, just load them directly as ints. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62668 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-20Fix: <rdar://problem/6510344> [pth] PTH slows down regular lexer ↵Ted Kremenek
considerably (when it has substantial work) Changes to IdentifierTable: - High-level summary: StringMap never owns IdentifierInfos. It just references them. - The string map now has StringMapEntry<IdentifierInfo*> instead of StringMapEntry<IdentifierInfo>. The IdentifierInfo object is allocated using the same bump pointer allocator as used by the StringMap. Changes to IdentifierInfo: - Added an extra pointer to point to the StringMapEntry<IdentifierInfo*> in the string map. This pointer will be null if the IdentifierInfo* is *only* used by the PTHLexer (that is it isn't in the StringMap). Algorithmic changes: - Non-PTH case: IdentifierInfo::get() will always consult the StringMap first to see if we have an IdentifierInfo object. If that StringMapEntry references a null pointer, we allocate a new one from the BumpPtrAllocator and update the reference in the StringMapEntry. - PTH case: We do the same lookup as with the non-PTH case, but if we don't get a hit in the StringMap we do a secondary lookup in the PTHManager for the IdentifierInfo. If we don't find an IdentifierInfo we create a new one as in the non-PTH case. If we do find and IdentifierInfo in the PTHManager, we update the StringMapEntry to refer to it so that the IdentifierInfo will be found on the next StringMap lookup. This way we only do a binary search in the PTH file at most once for a given IdentifierInfo. This greatly speeds things up for source files containing a non-trivial amount of code. Performance impact: While these changes do add some extra indirection in IdentifierTable to access an IdentifierInfo*, I saw speedups even in the non-PTH case as well. Non-PTH: For -fsyntax-only on Cocoa.h, we see a 6% speedup. PTH (with Cocoa.h in token cache): 11% speedup. I also did an experiment where we did -fsyntax-only on a source file including a large header and Cocoa.h, but the token cache did not contain the larger header. For this file, we were seeing a performance *regression* when using PTH of 3% over non-PTH. Now we are seeing a performance improvement of 9%! Tests: The serialization tests are now failing. I looked at this extensively, and I my belief is that this change is unmasking a bug rather than introducing a new one. I have disabled the serialization tests for now. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62636 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-19PTH: Emitted tokens now consist of 12 bytes that are loaded used 3 32-bit ↵Ted Kremenek
loads. This reduces user time but increases system time because of the slightly larger PTH file. Although there is no performance win on Cocoa.h and -Eonly, overall this seems like a good step. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62542 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-18rearrange GetIdentifierInfo so that the fast path can be partially inlined ↵Chris Lattner
into PTHLexer::Lex. This speeds up the user time of PTH -Eonly by another 2ms (4.4%) git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62454 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-18rename some variables, only set a tokens identifierinfo if non-null.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62450 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-18On i386 and x86-64, just do unaligned loads Chris Lattner
instead of assembling from bytes. This speeds up -Eonly PTH reading of cocoa.h by about 2ms, which is 4.2%. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62447 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-18switch PTHLexer to use Read32 and friends instead of lots of inlinedChris Lattner
copies. I verified that this causes no performance change in PTH. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62445 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-18switch PTH lexer from using "const char*"s to "const unsigned char*"s Chris Lattner
internally. This is just a cleanup that reduces the need to cast to unsigned char before assembling a larger integer. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62442 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-17simplify PTHManager::CreateLexerChris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62424 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-17suck the call to "getSpellingLoc" that all clients do intoChris Lattner
the implementation of PTHManager::getSpelling. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62408 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-17this massive patch introduces a simple new abstraction: it makesChris Lattner
"FileID" a concept that is now enforced by the compiler's type checker instead of yet-another-random-unsigned floating around. This is an important distinction from the "FileID" currently tracked by SourceLocation. *That* FileID may refer to the start of a file or to a chunk within it. The new FileID *only* refers to the file (and its #include stack and eventually #line data), it cannot refer to a chunk. FileID is a completely opaque datatype to all clients, only SourceManager is allowed to poke and prod it. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62407 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-16Change some terminology in SourceLocation: instead of referring to Chris Lattner
the "physical" location of tokens, refer to the "spelling" location. This is more concrete and useful, tokens aren't really physical objects! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62309 91177308-0d34-0410-b5e6-96231b3b80d8
generated by cgit v1.2.3-18-g5258 (git 2.32.0) at 2025-05-03 09:10:01 +0000