aboutsummaryrefslogtreecommitdiff
path: root/lib/Lex/Preprocessor.cpp
AgeCommit message (Collapse)Author
2011-12-02When we treat an #include or #import as a module import, create anDouglas Gregor
implicit ImportDecl in the translation unit to record the presence of the import. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@145727 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-02Implementing parsing and resolution of module export declarationsDouglas Gregor
within module maps, which will (eventually) be used to re-export a module from another module. There are still some pieces missing, however. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@145665 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-01Introduce the notion of name visibility into modules. For a givenDouglas Gregor
(sub)module, all of the names may be hidden, just the macro names may be exposed (for example, after the preprocessor has seen the import of the module but the parser has not), or all of the names may be exposed. Importing a module makes its names, and the names in any of its non-explicit submodules, visible to name lookup (transitively). This commit only introduces the notion of name visible and marks modules and submodules as visible when they are imported. The actual name-hiding logic in the AST reader will follow (along with test cases). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@145586 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-30Teach the preprocessor how to handle module import declarations thatDouglas Gregor
involve submodules (e.g., importing std.vector), rather than always importing the top-level module. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@145478 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-30Switch the module-loading interfaces and parser from a simpleDouglas Gregor
top-level module name to a module path (e.g., std.vector). We're still missing a number of pieces for this actually to do something. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@145462 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-23Remove an assertion that is not valid if we cancel parsing.Argyrios Kyrtzidis
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@145108 91177308-0d34-0410-b5e6-96231b3b80d8
2011-10-27Make the loading of information attached to an IdentifierInfo from anDouglas Gregor
AST file more lazy, so that we don't eagerly load that information for all known identifiers each time a new AST file is loaded. The eager reloading made some sense in the context of precompiled headers, since very few identifiers were defined before PCH load time. With modules, however, a huge amount of code can get parsed before we see an @import, so laziness becomes important here. The approach taken to make this information lazy is fairly simple: when we load a new AST file, we mark all of the existing identifiers as being out-of-date. Whenever we want to access information that may come from an AST (e.g., whether the identifier has a macro definition, or what top-level declarations have that name), we check the out-of-date bit and, if it's set, ask the AST reader to update the IdentifierInfo from the AST files. The update is a merge, and we now take care to merge declarations before/after imports with declarations from multiple imports. The results of this optimization are fairly dramatic. On a small application that brings in 14 non-trivial modules, this takes modules from being > 3x slower than a "perfect" PCH file down to 30% slower for a full rebuild. A partial rebuild (where the PCH file or modules can be re-used) is down to 7% slower. Making the PCH file just a little imperfect (e.g., adding two smallish modules used by a bunch of .m files that aren't in the PCH file) tips the scales in favor of the modules approach, with 24% faster partial rebuilds. This is just a first step; the lazy scheme could possibly be improved by adding versioning, so we don't search into modules we already searched. Moreover, we'll need similar lazy schemes for all of the other lookup data structures, such as DeclContexts. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@143100 91177308-0d34-0410-b5e6-96231b3b80d8
2011-10-11Add a -Wc++0x-compat warning for C++11 keywords used as identifiers when inRichard Smith
C++98 mode. Only the first occurrence of each keyword will produce a warning. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@141700 91177308-0d34-0410-b5e6-96231b3b80d8
2011-10-03Fixed exapnsion range for # and ##.Abramo Bagnara
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@141012 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-26Properly initialize Preprocessor::CurLexerKind to avoid use of uninitialized ↵Zhongxing Xu
variable. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@140514 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-25Rename Diagnostic to DiagnosticsEngine as per issue 5397David Blaikie
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@140478 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-19Introduce PreprocessingRecord::getPreprocessedEntitiesInRange()Argyrios Kyrtzidis
which will do a binary search and return a pair of iterators for preprocessed entities in the given source range. Source ranges of preprocessed entities are stored twice currently in the PCH/Module file but this will be fixed in a subsequent commit. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@140058 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-15Add an experimental flag -fauto-module-import that automatically turnsDouglas Gregor
#include or #import direcctives of framework headers into module imports of the corresponding framework module. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139860 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-07Optimize the preprocessor's handling of the __import_module__Douglas Gregor
keyword. We now handle this keyword in HandleIdentifier, making a note for ourselves when we've seen the __import_module__ keyword so that the next lexed token can trigger a module import (if needed). This greatly simplifies Preprocessor::Lex(), and completely erases the 5.5% -Eonly slowdown Argiris noted when I originally implemented __import_module__. Big thanks to Argiris for noting that horrible regression! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139265 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-04Use const_cast to avoid warnings.Benjamin Kramer
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139104 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-04Support code-completion for C++ inline methods and ObjC buffering methods.Argyrios Kyrtzidis
Previously we would cut off the source file buffer at the code-completion point; this impeded code-completion inside C++ inline methods and, recently, with buffering ObjC methods. Have the code-completion inserted into the source buffer so that it can be buffered along with a method body. When we actually hit the code-completion point the cut-off lexing or parsing. Fixes rdar://10056932&8319466 git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139086 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-01Allow the preprocessor to be constructed without performing target-Douglas Gregor
and language-specific initialization. Use this to allow ASTUnit to create a preprocessor object *before* loading the AST file. No actual functionality change. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138983 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-01Teach ASTContext and Preprocessor to hold on to references to the sameDouglas Gregor
LangOptions, rather than making distinct copies of LangOptions. Granted, LangOptions doesn't actually get modified, but this will eventually make it easier to construct ASTContext and Preprocessor before we know all of the LangOptions. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138959 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-31Make sure to initialize field. Hopefully this will fix some test failures ↵Eli Friedman
on Windows. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138880 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-31Switch __import__ over to __import_module__, so we don't conflict withDouglas Gregor
existing practice with Python extension modules. Not that Python extension modules should be using a double-underscored identifier anyway, but... git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138870 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-27Take an entirely different approach to handling the "parsing" ofDouglas Gregor
__import__ within the preprocessor, since the prior one foolishly assumed that Preprocessor::Lex() was re-entrant. We now handle __import__ at the top level (only), after macro expansion. This should fix the buildbot failures. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138704 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-26Introduce support for a simple module import declaration, whichDouglas Gregor
loads the named module. The syntax itself is intentionally hideous and will be replaced at some later point with something more palatable. For now, we're focusing on the semantics: - Module imports are handled first by the preprocessor (to get macro definitions) and then the same tokens are also handled by the parser (to get declarations). If both happen (as in normal compilation), the second one is redundant, because we currently have no way to hide macros or declarations when loading a module. Chris gets credit for this mad-but-workable scheme. - The Preprocessor now holds on to a reference to a module loader, which is responsible for loading named modules. CompilerInstance is the only important module loader: it now knows how to create and wire up an AST reader on demand to actually perform the module load. - We search for modules in the include path, using the module name with the suffix ".pcm" (precompiled module) for the file name. This is a temporary hack; we hope to improve the situation in the future. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@138679 91177308-0d34-0410-b5e6-96231b3b80d8
2011-07-27Change Preprocessor::getTotalMemory() to use llvm::capacity_in_bytes().Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@136239 91177308-0d34-0410-b5e6-96231b3b80d8
2011-07-26Report more memory using in Preprocessor::getTotalMemory() and ↵Ted Kremenek
PreprocessingRecord::getTotalMemory(). Most of the memory was already reported; but now we report more memory from side data structures. Fixes <rdar://problem/9379717>. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@136150 91177308-0d34-0410-b5e6-96231b3b80d8
2011-07-26Rename create(MacroArg)InstantiationLoc to create(MacroArg)ExpansionLoc.Chandler Carruth
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@136054 91177308-0d34-0410-b5e6-96231b3b80d8
2011-07-23remove unneeded llvm:: namespace qualifiers on some core types now that ↵Chris Lattner
LLVM.h imports them into the clang namespace. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@135852 91177308-0d34-0410-b5e6-96231b3b80d8
2011-07-14Move the rest of the preprocessor terminology from 'instantiate' andChandler Carruth
variants to 'expand'. This changed a couple of public APIs, including one public type "MacroInstantiation" which is now "MacroExpansion". The rest of the codebase was updated to reflect this, especially the libclang code. Two of the C++ (and thus easily changed) libclang APIs were updated as well because they pertained directly to the old MacroInstantiation class. No functionality changed. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@135139 91177308-0d34-0410-b5e6-96231b3b80d8
2011-06-29Introduce a caching mechanism for macro expanded tokens.Argyrios Kyrtzidis
Previously macro expanded tokens were added to Preprocessor's bump allocator and never released, even after the TokenLexer that were lexing them was finished, thus they were wasting memory. A very "useful" boost library was causing clang to eat 1 GB just for the expanded macro tokens. Introduce a special cache that works like a stack; a TokenLexer can add the macro expanded tokens in the cache, and when it finishes, the tokens are removed from the end of the cache. Now consumed memory by expanded tokens for that library is ~ 1.5 MB. Part of rdar://9327049. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@134105 91177308-0d34-0410-b5e6-96231b3b80d8
2011-06-29Introduce Preprocessor::getTotalMemory() and use it in CIndex.cpp, no ↵Argyrios Kyrtzidis
functionality change. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@134103 91177308-0d34-0410-b5e6-96231b3b80d8
2011-05-06Introduce a new libclang parsing flag,Douglas Gregor
CXTranslationUnit_NestedMacroInstantiations, which indicates whether we want to see "nested" macro instantiations (e.g., those that occur inside other macro instantiations) within the detailed preprocessing record. Many clients (e.g., those that only care about visible tokens) don't care about this information, and in code that uses preprocessor metaprogramming, this information can have a very high cost. Addresses <rdar://problem/9389320>. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@130990 91177308-0d34-0410-b5e6-96231b3b80d8
2011-04-28Parsing/AST support for Structured Exception HandlingJohn Wiegley
Patch authored by Sohail Somani. Provide parsing and AST support for Windows structured exception handling. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@130366 91177308-0d34-0410-b5e6-96231b3b80d8
2011-03-08Fix my earlier commit to work with escaped newlines and leave breadcrumbsJohn McCall
in case we want to make a world where we can check intermediate instantiations for this kind of breadcrumb. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@127221 91177308-0d34-0410-b5e6-96231b3b80d8
2011-03-08Add an API call to retrieve the spelling data of a token from its ↵John McCall
SourceLocation. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@127216 91177308-0d34-0410-b5e6-96231b3b80d8
2010-12-22Introduced raw_identifier token kind.Abramo Bagnara
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@122394 91177308-0d34-0410-b5e6-96231b3b80d8
2010-11-23now the FileManager has a FileSystemOpts ivar, stop threadingChris Lattner
FileSystemOpts through a ton of apis, simplifying a lot of code. This also fixes a latent bug in ASTUnit where it would invoke methods on FileManager without creating one in some code paths in cindextext. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@120010 91177308-0d34-0410-b5e6-96231b3b80d8
2010-11-17move getSpelling from Preprocessor to Lexer, which it is more conceptually ↵Chris Lattner
related to. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@119479 91177308-0d34-0410-b5e6-96231b3b80d8
2010-11-17move AdvanceToTokenCharacter and getLocForEndOfToken fromChris Lattner
Preprocessor to Lexer where they make more sense. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@119474 91177308-0d34-0410-b5e6-96231b3b80d8
2010-11-17add a static version of PP::AdvanceToTokenCharacter.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@119472 91177308-0d34-0410-b5e6-96231b3b80d8
2010-11-17add a static form of the efficient PP::getSpelling method.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@119469 91177308-0d34-0410-b5e6-96231b3b80d8
2010-11-03Implement -working-directory.Argyrios Kyrtzidis
When -working-directory is passed in command line, file paths are resolved relative to the specified directory. This helps both when using libclang (where we can't require the user to actually change the working directory) and to help reproduce test cases when the reproduction work comes along. --FileSystemOptions is introduced which controls how file system operations are performed (currently it just contains the working directory value if set). --FileSystemOptions are passed around to various interfaces that perform file operations. --Opening & reading the content of files should be done only through FileManager. This is useful in general since file operations will be abstracted in the future for the reproduction mechanism. FileSystemOptions is independent of FileManager so that we can have multiple translation units sharing the same FileManager but with different FileSystemOptions. Addresses rdar://8583824. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@118203 91177308-0d34-0410-b5e6-96231b3b80d8
2010-10-19Really^2 fix <rdar://problem/8361834>, this time without crashing.Ted Kremenek
Now MICache is a linked list (per the FIXME), where we tradeoff between MacroInfo objects being in MICache and MIChainHead. MacroInfo objects in the MICache chain are already "Destroy()'ed", so they can be reused. When inserting into MICache, we need to remove them from the regular linked list so that they aren't destroyed more than once. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@116869 91177308-0d34-0410-b5e6-96231b3b80d8
2010-10-19Simplify loop. No functionality change.Ted Kremenek
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@116861 91177308-0d34-0410-b5e6-96231b3b80d8
2010-10-19Simplify lifetime management of MacroInfo objects in Preprocessor by having ↵Ted Kremenek
the Preprocessor maintain them in a linked list of allocated MacroInfos. This requires only 1 extra pointer per MacroInfo object, and allows us to blow them away in one place. This fixes an elusive memory leak with MacroInfos (whose exact location I couldn't still figure out despite substantial digging). Fixes <rdar://problem/8361834>. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@116842 91177308-0d34-0410-b5e6-96231b3b80d8
2010-10-19In ~Preprocessor(), also cleanup the MacroInfo objects left-over from stray ↵Ted Kremenek
"#pragma push_macro" uses. This fixes a potential memory leak. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@116826 91177308-0d34-0410-b5e6-96231b3b80d8
2010-09-03Use getSpelling to get original text of theFariborz Jahanian
c++ operator token. (radar 8328250). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@112977 91177308-0d34-0410-b5e6-96231b3b80d8
2010-09-03Patch to allow alternative representation of c++Fariborz Jahanian
operators (and, or, etc.) to be used as selectors to match g++'s behavior. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@112935 91177308-0d34-0410-b5e6-96231b3b80d8
2010-08-30Revert my user-defined literal commits - r1124{58,60,67} pendingSean Hunt
some issues being sorted out. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@112493 91177308-0d34-0410-b5e6-96231b3b80d8
2010-08-29Implement C++0x user-defined string literals.Sean Hunt
The extra data stored on user-defined literal Tokens is stored in extra allocated memory, which is managed by the PreprocessorLexer because there isn't a better place to put it that makes sure it gets deallocated, but only after it's used up. My testing has shown no significant slowdown as a result, but independent testing would be appreciated. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@112458 91177308-0d34-0410-b5e6-96231b3b80d8
2010-08-26Tweak wording in an assertion, from dawn@burble.org.Douglas Gregor
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@112182 91177308-0d34-0410-b5e6-96231b3b80d8
2010-08-25Introduce a preprocessor code-completion hook for contexts where weDouglas Gregor
expect "natural" language and should not provide any completions, e.g., comments, string literals, #error. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@112054 91177308-0d34-0410-b5e6-96231b3b80d8
generated by cgit v1.2.3-18-g5258 (git 2.32.0) at 2025-05-05 14:39:42 +0000