aboutsummaryrefslogtreecommitdiff
path: root/Driver/CacheTokens.cpp
AgeCommit message (Collapse)Author
2009-03-24Move <root>/Driver into <root>/tools/clang-cc.Daniel Dunbar
Again, I tried to update cmake but it is untested. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@67605 91177308-0d34-0410-b5e6-96231b3b80d8
2009-03-19Store the name of the original file used to generate the PTH file in the PTHTed Kremenek
file itself. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@67340 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-24PTH: Cache *un-cleaned* spellings for literals instead of cleaned spellings.Ted Kremenek
This allows the PTH file to stay 100% in fidelity with the source code and defines away some weird cosmetic bugs for operations such as '-E' where maintaining knowledge of the original literal representation is useful. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@65361 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-24PTH: When emitting tokens for literals with cached spellings, change the tokenTed Kremenek
size to that of the *cleaned* spelling. This way 'getSpelling()' for literals in the Preprocessor just works and doesn't read beyond the bounds of the cached spelling buffer. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@65354 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-20PTH generation: Clear the cleaning bit for literals (whose spellings are ↵Ted Kremenek
cached). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@65148 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-14Fix the build on win32.Cedric Venet
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64556 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-13PTH: Cache directory and negative 'stat' calls. This gives us a 1% ↵Ted Kremenek
performance improvement on Cocoa.h (fsyntax-only+PTH). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64490 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-13Add some boilerplate to the PTH file to prepare for the caching of stats for ↵Ted Kremenek
directories (and negative stats too). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64477 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-12PTH: Cache stat information for files in the PTH file. Hook up FileManagerTed Kremenek
to use this stat information in the PTH file using a 'StatSysCallCache' object. Performance impact (Cocoa.h, PTH): - number of stat calls reduces from 1230 to 425 - fsyntax-only: time improves by 4.2% We can reduce the number of stat calls to almost zero by caching negative stat calls and directory stat calls in the PTH file as well. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64353 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-11PTH: Have meta data be at the beginning of the PTH file, not the end.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64338 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-11PTH: Replace string identifier to persistent ID lookup with a hashtable. This isTed Kremenek
actually *slightly* slower than the binary search. Since this is algorithmically better, further performance tuning should be able to make this faster. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64326 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-11PTH: Don't emit the PTH offset of the IdentifierInfo string data as that data isTed Kremenek
referenced by other tables. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64304 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-10PTH generation: Discard tokens that appear after and on the same line as ↵Ted Kremenek
'#endif'. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64250 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-10PTH generation: Don't call 'EmitToken' in the loop condition. This is ↵Ted Kremenek
preparing for other changes within the loop. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64247 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-10PTH: Replace ad hoc 'file name' -> 'PTH data' lookup table in the PTH file ↵Ted Kremenek
with an on-disk chained hash table. This data structure is implemented using templates, and will be used to replace similar data structures. This change leads to no visibile performance impact on Cocoa.h, but now we only pay a price for the table on order with the number of files accessed and not the number in the PTH file. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64245 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-10Rearrange code. No functionality change.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64193 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-10Fix potential padding error in PTH file and add stub code for emitting an ↵Ted Kremenek
on-disk chained hash table. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64192 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-03switch SourceManager from using an std::map and std::list of Chris Lattner
ContentCache objects to using a densemap and list, and allocating the ContentCache objects from a bump pointer. This does not speed up or slow down things substantially, but gives us control over their alignment. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63628 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-27rename getFullFilePos -> getFileOffset.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63097 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-27PTH: Use Token::setLiteralData() to directly store a pointer to cached ↵Ted Kremenek
spelling data in the PTH file. This removes a ton of code for looking up spellings using sourcelocations in the PTH file. This simplifies both PTH-generation and reading. Performance impact for -fsyntax-only on Cocoa.h (with Cocoa.h in the PTH file): - PTH generation time improves by 5% - PTH reading improves by 0.3%. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63072 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-26Add version number checking to PTH files.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63047 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-26Embed the offset of the PTH table inside the prologue of the PTH file. This ↵Ted Kremenek
will help improve gradual versioning of PTH files instead of relying that the PTH table is at a fixed offset. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63045 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-26This change refactors some of the low-level lexer interfaces a bit.Chris Lattner
Token now has a class of kinds for "literals", which include numeric constants, strings, etc. These tokens can optionally have a pointer to the start of the token in the lexer buffer. This makes it faster to get spelling and do other gymnastics, because we don't have to go through source locations. This change is performance neutral, but will make other changes more feasible down the road. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63028 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-19PTH: Emitted tokens now consist of 12 bytes that are loaded used 3 32-bit ↵Ted Kremenek
loads. This reduces user time but increases system time because of the slightly larger PTH file. Although there is no performance win on Cocoa.h and -Eonly, overall this seems like a good step. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62542 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-17add a simplified lexer ctor that sets up the lexer to raw-lex anChris Lattner
entire file. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62414 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-17this massive patch introduces a simple new abstraction: it makesChris Lattner
"FileID" a concept that is now enforced by the compiler's type checker instead of yet-another-random-unsigned floating around. This is an important distinction from the "FileID" currently tracked by SourceLocation. *That* FileID may refer to the start of a file or to a chunk within it. The new FileID *only* refers to the file (and its #include stack and eventually #line data), it cannot refer to a chunk. FileID is a completely opaque datatype to all clients, only SourceManager is allowed to poke and prod it. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62407 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-17no need to check this: content cache is already 1-1 map with fileentries.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62402 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-17Instead of iterating over FileID's, have PTH generation iterate over theChris Lattner
content cache directly. Content cache has a 1-1 mapping with fileentries, whereas multiple FileIDs can be the same FileEntry. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62401 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-15IdentifierInfo:Ted Kremenek
- IdentifierInfo can now (optionally) have its string data not be co-located with itself. This is for use with PTH. This aspect is a little gross, as getName() and getLength() now make assumptions about a possible alternate representation of IdentifierInfo. Perhaps we should make IdentifierInfo have virtual methods? IdentifierTable: - Added class "IdentifierInfoLookup" that can be used by IdentifierTable to perform "string -> IdentifierInfo" lookups using an auxilliary data structure. This is used by PTH. - Perform tests show that IdentifierTable::get() does not slow down because of the extra check for the IdentiferInfoLookup object (the regular StringMap lookup does enough work to mitigate the impact of an extra null pointer check). - The upshot is that now that some IdentifierInfo objects might be owned by the IdentiferInfoLookup object. This should be reviewed. PTH: - Modified PTHManager::GetIdentifierInfo to *not* insert entries in IdentifierTable's string map, and instead create IdentifierInfo objects on the fly when mapping from persistent IDs to IdentifierInfos. This saves a ton of work with string copies, hashing, and StringMap lookup and resizing. This change was motivated because when processing source files in the PTH cache we don't need to do any string -> IdentifierInfo lookups. - PTHManager now subclasses IdentifierInfoLookup, allowing clients of IdentifierTable to transparently use IdentifierInfo objects managed by the PTH file. PTHManager resolves "string -> IdentifierInfo" queries by doing a binary search over a sorted table of identifier strings in the PTH file (the exact algorithm we use can be changed as needed). These changes lead to the following performance changes when using PTH on Cocoa.h: - fsyntax-only: 10% performance improvement - Eonly: 30% performance improvement git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62273 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-15PTH: Embed a persistentID side-table in the PTH file that is sorted in theTed Kremenek
lexical order of the corresponding identifier strings. This will be used for a forthcoming optimization. This slows down PTH generation time by 7%. We can revert this change if the optimization proves to not be valuable. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62248 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-09Simpler solution to LiteralSupport compatibility: just add one whitespace ↵Ted Kremenek
character after each cached string. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61962 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-08PTH: For the cached spellings of literals, store one whitespace character ↵Ted Kremenek
after the spelling to accomodate sanity checking in LiteralSuppoert.cpp. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61956 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-08Remove debugging variable I forgot to remove in my last commit.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61910 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-08Cache the "spellings" of string, character, and numeric literals in the PTHTed Kremenek
file. For Cocoa.h, this enlarges the PTH file by 310K (4%). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61909 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-08Refactor CacheTokens to use a PTHWriter class that creates and manages most ↵Ted Kremenek
of the PTH generation data structures. No functionality change. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61902 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-06use getBuffer() to fix compile error. Ted, please review.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61786 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-23PTH: Use 3 bytes instead of 4 bytes to encode the persistent ID for a token.Ted Kremenek
- This reduces the PTH size for Cocoa.h by 7%. - The increases PTH -Eonly speed for Cocoa.h by 0.8%. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61377 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-23PTH:Ted Kremenek
- Encode the token length with 2 bytes instead of 4. - This reduces the size of the .pth file for Cocoa.h by 12%. - This speeds up PTH time (-Eonly) on Cocoa.h by 1.6%. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61364 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-23PTH:Ted Kremenek
- Embed 'eom' tokens in PTH file. - Use embedded 'eom' tokens to not lazily generate them in the PTHLexer. This means that PTHLexer can always advance to the next token after reading a token (instead of buffering tokens using a copy). - Moved logic of 'ReadToken' into Lex. GetToken & ReadToken no longer exist. - These changes result in a 3.3% speedup (-Eonly) on Cocoa.h. - The code is a little gross. Many cleanups are possible and should be done. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61360 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-12Enhance PTH preprocessor-condition-block side table to track ↵Ted Kremenek
#elseinformation as well. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60955 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-11PTH:Ted Kremenek
- Added a side-table per each token-cached file with the preprocessor conditional stack. This tracks what #if's are matched with what #endifs and where their respective tokens are in the PTH file. This will allow for quick skipping of excluded conditional branches in the Preprocessor. - Performance testing shows the addition of this information (without actually utilizing it) leads to no performance regressions. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60911 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-04Remove unneeded assertion. We already know that FE->getName() is an ↵Ted Kremenek
absolute path. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60558 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-03PTH:Ted Kremenek
Use an array instead of a DenseMap to cache persistent IDs -> IdentifierInfo*. This leads to a 4% speedup at -fsyntax-only using PTH. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60452 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-02PTH emission:Ted Kremenek
- Output 32 bit integers using bit-shifting + write of individual bytes. This is motivated because we aren't guaranteed to load 32-bit ints of the mmaped PTH file at 4-byte offsets. - Don't emit flags for IdentifierInfos. These are lazily populated by the Preprocessor/Parser. - Only write out tokens for files with absolute paths. This is potentially temporary, but simplifies things for now. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60435 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-26- Enhance PTH generation to write out IdentifierInfo table in two parts:Ted Kremenek
- a table including the IdentifierInfo data - an index from persistent IdentifierInfo IDs to indices within this file. - Enhance PTH generation to write out file map information, mapping inodes to tokens. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60132 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-26Re-apply r60071 now that raw_fd_ostream::tell has been committed.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60086 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-26Revert 60071, depends on uncommitted LLVM changes.Daniel Dunbar
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60077 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-26Migrate token-cache generation logic from dummy harness in PPLexerChange.cpp ↵Ted Kremenek
to CacheTokens.cpp. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60071 91177308-0d34-0410-b5e6-96231b3b80d8
2008-11-13[LLVM up] Update for raw_fd_ostream change. This fixes a FIXME thatDaniel Dunbar
the Backend output should be done in binary mode. - I'd appreciate it if someone who has a Windows build could verify this. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@59221 91177308-0d34-0410-b5e6-96231b3b80d8
2008-10-21Added the start of a prototype implementation of PCH based on token caching.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@57863 91177308-0d34-0410-b5e6-96231b3b80d8
generated by cgit v1.2.3-18-g5258 (git 2.32.0) at 2025-05-02 07:03:46 +0000