aboutsummaryrefslogtreecommitdiff
path: root/lib/Lex/Lexer.cpp
AgeCommit message (Collapse)Author
2009-11-14Move DISABLE_INLINE to the front of the decl so MSVC can parse it. Patch by ↵Benjamin Kramer
Amine Khaldi! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@88797 91177308-0d34-0410-b5e6-96231b3b80d8
2009-10-14Teach Lexer::MeasureTokenLength to be able to measure theChris Lattner
length of comment tokens. Patch by Abramo Bagnara! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@84100 91177308-0d34-0410-b5e6-96231b3b80d8
2009-09-22Replace the -code-completion-dump option with Douglas Gregor
-code-completion-at=filename:line:column which performs code completion at the specified location by truncating the file at that position and enabling code completion. This approach makes it possible to run multiple tests from a single test file, and gives a more natural command-line interface. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@82571 91177308-0d34-0410-b5e6-96231b3b80d8
2009-09-21Refactor and simplify the CodeCompleteConsumer, so that all of theDouglas Gregor
real work is performed within Sema. Addresses Chris's comments, but still retains the heavyweight list-of-multimaps data structure. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@82459 91177308-0d34-0410-b5e6-96231b3b80d8
2009-09-17Initial implementation of a code-completion interface in Clang. InDouglas Gregor
essence, code completion is triggered by a magic "code completion" token produced by the lexer [*], which the parser recognizes at certain points in the grammar. The parser then calls into the Action object with the appropriate CodeCompletionXXX action. Sema implements the CodeCompletionXXX callbacks by performing minimal translation, then forwarding them to a CodeCompletionConsumer subclass, which uses the results of semantic analysis to provide code-completion results. At present, only a single, "printing" code completion consumer is available, for regression testing and debugging. However, the design is meant to permit other code-completion consumers. This initial commit contains two code-completion actions: one for member access, e.g., "x." or "p->", and one for nested-name-specifiers, e.g., "std::". More code-completion actions will follow, along with improved gathering of code-completion results for the various contexts. [*] In the current -code-completion-dump testing/debugging mode, the file is truncated at the completion point and EOF is translated into "code completion". git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@82166 91177308-0d34-0410-b5e6-96231b3b80d8
2009-09-09Remove tabs, and whitespace cleanups.Mike Stump
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@81346 91177308-0d34-0410-b5e6-96231b3b80d8
2009-07-07Convert the CharInfo table to be statically initialized, instead of ↵Chris Lattner
dynamically initialized. Patch by Ryan Flynn! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@74919 91177308-0d34-0410-b5e6-96231b3b80d8
2009-07-07fix an out-of-date comment.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@74894 91177308-0d34-0410-b5e6-96231b3b80d8
2009-07-02Add support for retrieving the Doxygen comment associated with a givenDouglas Gregor
declaration in the AST. The new ASTContext::getCommentForDecl function searches for a comment that is attached to the given declaration, and returns that comment, which may be composed of several comment blocks. Comments are always available in an AST. However, to avoid harming performance, we don't actually parse the comments. Rather, we keep the source ranges of all of the comments within a large, sorted vector, then lazily extract comments via a binary search in that vector only when needed (which never occurs in a "normal" compile). Comments are written to a precompiled header/AST file as a blob of source ranges. That blob is only lazily loaded when one requests a comment for a declaration (this never occurs in a "normal" compile). The indexer testbed now supports comment extraction. When the -point-at location points to a declaration with a Doxygen-style comment, the indexer testbed prints the associated comment block(s). See test/Index/comments.c for an example. Some notes: - We don't actually attempt to parse the comment blocks themselves, beyond identifying them as Doxygen comment blocks to associate them with a declaration. - We won't find comment blocks that aren't adjacent to the declaration, because we start our search based on the location of the declaration. - We don't go through the necessary hops to find, for example, whether some redeclaration of a declaration has comments when our current declaration does not. Similarly, we don't attempt to associate a \param Foo marker in a function body comment with the parameter named Foo (although that is certainly possible). - Verification of my "no performance impact" claims is still "to be done". git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@74704 91177308-0d34-0410-b5e6-96231b3b80d8
2009-06-23Fix our check for "random whitespace between a \ and newline" to workChris Lattner
with dos style newlines. I have a trivial test for this: // RUN: clang-cc %s -verify #define test(x, y) \ x ## y but I don't know how to get svn to not change newlines and testrunner doesn't work with dos style newlines either, so "not worth it". :) rdar://6994000 git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@73945 91177308-0d34-0410-b5e6-96231b3b80d8
2009-05-13Fix rdar://6880630 - # in _Pragma does not start a preprocessor directive.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@71643 91177308-0d34-0410-b5e6-96231b3b80d8
2009-04-28Get rid of some useless uses of NoExtensions. The philosophy here is Eli Friedman
that if we're going to print an extension warning anyway, there's no point to changing behavior based on NoExtensions: it will only make error recovery worse. Note that this doesn't cause any behavior change because NoExtensions isn't used by the current front-end. I'm still considering what to do about the remaining use of NoExtensions in IdentifierTable.cpp. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@70273 91177308-0d34-0410-b5e6-96231b3b80d8
2009-04-24fix rdar://6816766 - Crash with function-like macro test at end of directive.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69964 91177308-0d34-0410-b5e6-96231b3b80d8
2009-04-18add a new Lexer::SkipEscapedNewLines method.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69483 91177308-0d34-0410-b5e6-96231b3b80d8
2009-04-18factor escape newline measuring out into its own helper function.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69482 91177308-0d34-0410-b5e6-96231b3b80d8
2009-04-18remove unneeded scopes.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69481 91177308-0d34-0410-b5e6-96231b3b80d8
2009-04-17Fix two problems from PR3916, and one problem I noticed while hackingChris Lattner
on the code. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69404 91177308-0d34-0410-b5e6-96231b3b80d8
2009-04-14Change Lexer::MeasureTokenLength to take a LangOptions reference.Chris Lattner
This allows it to accurately measure tokens, so that we get: t.cpp:8:13: error: unknown type name 'X' static foo::X P; ~~~~~^ instead of the woefully inferior: t.cpp:8:13: error: unknown type name 'X' static foo::X P; ~~~~ ^ Most of this is just plumbing to push the reference around. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69099 91177308-0d34-0410-b5e6-96231b3b80d8
2009-04-05fix rdar://6757323, where an escaped newline in a // commentChris Lattner
was causing the char after the newline to get eaten. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@68430 91177308-0d34-0410-b5e6-96231b3b80d8
2009-04-02A code modification hint for files that don't end in a newline.Mike Stump
Eventually, would be nice to be able to run these modifications even when we don't want the warning or errors for the actual diagnostic. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@68272 91177308-0d34-0410-b5e6-96231b3b80d8
2009-03-18silence some errors that should not apply to .S files on code like:Chris Lattner
'' ' ' git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@67237 91177308-0d34-0410-b5e6-96231b3b80d8
2009-03-18properly form a full token for # before calling HandleDirective.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@67235 91177308-0d34-0410-b5e6-96231b3b80d8
2009-03-08simplify some logic by making ScratchBuffer handle the application of trailingChris Lattner
\0's to created tokens instead of making all clients do it. No functionality change. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@66373 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-19fix PR3609, emit:Chris Lattner
t.c:1:10: error: missing terminating '>' character #include <stdio.h ^ instead of: t.c:1:10: error: missing terminating " character #include <stdio.h ^ git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@65052 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-15track "just a little more" location information for macro instantiations.Chris Lattner
Now instead of just tracking the expansion history, also track the full range of the macro that got replaced. For object-like macros, this doesn't change anything. For _Pragma and function-like macros, this means we track the locations of the ')'. This is required for PR3579 because apparently GCC uses the line of the ')' of a function-like macro as the location to expand __LINE__ to. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64601 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-29move library-specific diagnostic headers into library private dirs. ReduceChris Lattner
redundant #includes. Patch by Anders Johnsen! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63271 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-27Split the single monolithic DiagnosticKinds.def file into oneChris Lattner
.def file for each library. This means that adding a diagnostic to sema doesn't require all the other libraries to be rebuilt. Patch by Anders Johnsen! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63111 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-26Fix a bug I introduced in my changes, which caused MeasureTokenLengthChris Lattner
to crash when given an instantiation location. Thanks to Fariborz for the testcase. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63057 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-26allow _Pragmas formed from #defines to keep their full instantiationChris Lattner
history git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63035 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-26This change refactors some of the low-level lexer interfaces a bit.Chris Lattner
Token now has a class of kinds for "literals", which include numeric constants, strings, etc. These tokens can optionally have a pointer to the start of the token in the lexer buffer. This makes it faster to get spelling and do other gymnastics, because we don't have to go through source locations. This change is performance neutral, but will make other changes more feasible down the road. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63028 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-26Check in the long promised SourceLocation rewrite. This lays theChris Lattner
ground work for implementing #line, and fixes the "out of macro ID's" problem. There is nothing particularly tricky about the code, other than the very performance sensitive SourceManager::getFileID() method. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62978 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-23This is a follow-up to r62675:Chris Lattner
Refactor how the preprocessor changes a token from being an tok::identifier to a keyword (e.g. tok::kw_for). Instead of doing this in HandleIdentifier, hoist this common case out into the caller, so that every keyword doesn't have to go through HandleIdentifier. This drops time in HandleIdentifier from 1.25ms to .62ms, and speeds up clang -Eonly with PTH by about 1%. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62855 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-21a trivial micro optimization to save a load.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62676 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-21Add a bit to IdentifierInfo that acts as a simple predicate whichChris Lattner
tells us whether Preprocessor::HandleIdentifier needs to be called. Because this method is only rarely needed, this saves a call and a bunch of random checks. This drops the time in HandleIdentifier from 3.52ms to .98ms on cocoa.h on my machine. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62675 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-19Rename SourceManager::getCanonicalFileID -> getFileID. There isChris Lattner
no longer such thing as a non-canonical FileID. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62499 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-19Make SourceLocation::getFileLoc private to reduce the API exposure of Chris Lattner
SourceLocation. This requires making some cleanups to token pasting and _Pragma expansion. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62490 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-17Rename SourceLocation::getFileID to getChunkID, because it returnsChris Lattner
the chunk ID not the file ID. This exposes problems in TextDiagnosticPrinter where it should have been using the canonical file ID but wasn't. Fix these along the way. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62427 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-17simplify some lookups.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62426 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-17Change the Lexer ctor used to lex _Pragma directives into a static factoryChris Lattner
method. This lets us clean up the interface and make it more obvious that this method is *really really* _Pragma specific. Note that _Pragma handling uglifies the Lexer in the critical path. It would be very interesting to consider making _Pragma remapping be a new special lexer class of its own. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62425 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-17Change the Lexer ctor used in the non _Pragma case to take a FileID insteadChris Lattner
of a SourceLocation. This should speed it up and definitely simplifies it. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62422 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-17More simplifications to the lexer ctors.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62419 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-17make the verbose raw-lexer ctor fully explicit instead of havingChris Lattner
embedded magic. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62417 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-17add a simplified lexer ctor that sets up the lexer to raw-lex anChris Lattner
entire file. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62414 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-17refactor some common initialization code out of the two lexer ctors.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62411 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-17this massive patch introduces a simple new abstraction: it makesChris Lattner
"FileID" a concept that is now enforced by the compiler's type checker instead of yet-another-random-unsigned floating around. This is an important distinction from the "FileID" currently tracked by SourceLocation. *That* FileID may refer to the start of a file or to a chunk within it. The new FileID *only* refers to the file (and its #include stack and eventually #line data), it cannot refer to a chunk. FileID is a completely opaque datatype to all clients, only SourceManager is allowed to poke and prod it. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62407 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-17Instead of iterating over FileID's, have PTH generation iterate over theChris Lattner
content cache directly. Content cache has a 1-1 mapping with fileentries, whereas multiple FileIDs can be the same FileEntry. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62401 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-16Fix PR2477 - clang misparses "//*" in C89 modeChris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62368 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-16more SourceLocation lexicon change: instead of referring to theChris Lattner
"logical" location, refer to the "instantiation" location. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62316 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-16Change some terminology in SourceLocation: instead of referring to Chris Lattner
the "physical" location of tokens, refer to the "spelling" location. This is more concrete and useful, tokens aren't really physical objects! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62309 91177308-0d34-0410-b5e6-96231b3b80d8
2008-12-12rdar://6060752 - don't warn about trigraphs in bcpl-style commentsChris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60942 91177308-0d34-0410-b5e6-96231b3b80d8
generated by cgit v1.2.3-18-g5258 (git 2.32.0) at 2025-05-03 08:13:21 +0000