aboutsummaryrefslogtreecommitdiff
path: root/lib/Lex/Lexer.cpp
AgeCommit message (Collapse)Author
2010-04-12fix a minor bug I noticed while work with Jordy's patch for PR6101,Chris Lattner
in an input file like this: # 42 int x; we were emitting: # <something> int x; (with a space before the int) because we weren't clearing the leading whitespace flag properly after the \n from the directive was handled. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@101084 91177308-0d34-0410-b5e6-96231b3b80d8
2010-03-31Reinstate my CodeModificationHint -> FixItHint renaming patch, withoutDouglas Gregor
the C-only "optimization". git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@100022 91177308-0d34-0410-b5e6-96231b3b80d8
2010-03-31Revert r100008, which inexplicably breaks the clang-i686-darwin10 builderDouglas Gregor
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@100018 91177308-0d34-0410-b5e6-96231b3b80d8
2010-03-31Rename CodeModificationHint to FixItHint, since we've been using theDouglas Gregor
term "fix-it" everywhere and even *I* get tired of long names sometimes. No functionality change. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@100008 91177308-0d34-0410-b5e6-96231b3b80d8
2010-03-16Remove unused variableDouglas Gregor
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98691 91177308-0d34-0410-b5e6-96231b3b80d8
2010-03-16Audit all Preprocessor::getSpelling() callers, improving failureDouglas Gregor
recovery for those that need it. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98689 91177308-0d34-0410-b5e6-96231b3b80d8
2010-03-16Audit all callers of SourceManager::getCharacterData(); update some ofDouglas Gregor
them to recover more gracefully on failure. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98672 91177308-0d34-0410-b5e6-96231b3b80d8
2010-03-16Let SourceManager::getBufferData return StringRef instead of a pair of two ↵Benjamin Kramer
const char*. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98630 91177308-0d34-0410-b5e6-96231b3b80d8
2010-03-16Give SourceManager a Diagnostic object with which to report errors,Douglas Gregor
and start simplifying the interfaces in SourceManager that can fail. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98594 91177308-0d34-0410-b5e6-96231b3b80d8
2010-03-15Introduce a new BufferResult class to act as the return type ofDouglas Gregor
SourceManager's getBuffer() (and similar) operations. This abstract can be used to force callers to cope with errors in getBuffer(), such as missing files and changed files. Fix a bunch of callers to use the new interface. Add some very basic checks for file consistency (file size, modification time) into ContentCache::getBuffer(), although these checks don't help much until we've updated the main callers (e.g., SourceManager::getSpelling()). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98585 91177308-0d34-0410-b5e6-96231b3b80d8
2010-02-03don't inform comment handlers about comments in #if 0 blocks,Chris Lattner
doing so invalidates the file guard optimization and is not in the spirit of "#if 0" because it is supposed to completely skip everything, even if it isn't lexically valid. Patch by Abramo Bagnara! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@95253 91177308-0d34-0410-b5e6-96231b3b80d8
2010-01-22Teach CIndex's cursor visitor to restrict its traversal to a specificDouglas Gregor
region of interest (if provided). Implement clang_getCursor() in terms of this traversal rather than using the Index library; the unified cursor visitor is more complete, and will be The Way Forward. Minor other tweaks needed to make this work: - Extend Preprocessor::getLocForEndOfToken() to accept an offset from the end, making it easy to move to the last character in the token (rather than just past the end of the token). - In Lexer::MeasureTokenLength(), the length of whitespace is zero. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@94200 91177308-0d34-0410-b5e6-96231b3b80d8
2010-01-18allow the HandlerComment callback to push tokens into theChris Lattner
preprocessor. This could be used by an OpenMP implementation or something. Patch by Abramo Bagnara! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@93795 91177308-0d34-0410-b5e6-96231b3b80d8
2010-01-11add a TODO for a perf improvement in LexIdentifier.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@93141 91177308-0d34-0410-b5e6-96231b3b80d8
2010-01-10Do not parse hexadecimal floating point literals in C++0x mode because they areSean Hunt
incompatible with user-defined literals, specifically with the following form: 0x1p+1 The preprocessing-number token extends only as far as the 'p'; the '+' is not included. Previously we could get away with this extension as p was an invalid suffix, but now with user-defined literals, 'p' might well be a valid suffix and we are forced to consider it as such. This patch also adds a warning in non-0x C++ modes telling the user that this extension is incompatible with C++0x that is enabled by default (previously and with other languages, we warn only with a compliance option such as -pedantic). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@93135 91177308-0d34-0410-b5e6-96231b3b80d8
2009-12-17reimplement r90860, fixing a couple of problems:Chris Lattner
1. Don't make a copy of LangOptions every time a lexer is created. 2. Don't make CharInfo global mutable state. 3. Fix the implementation to properly treat ^Z as EOF instead of as horizontal whitespace, which matches the semantic implemented by VC++. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@91586 91177308-0d34-0410-b5e6-96231b3b80d8
2009-12-14teach clang to recover gracefully from conflict markers left in sourceChris Lattner
files: PR5238. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@91270 91177308-0d34-0410-b5e6-96231b3b80d8
2009-12-08Integrate the following from the 'objective-rewrite' branch:Steve Naroff
http://llvm.org/viewvc/llvm-project?view=rev&revision=80043 git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@90860 91177308-0d34-0410-b5e6-96231b3b80d8
2009-12-02Extend the source manager with the ability to override the contents ofDouglas Gregor
files with the contents of an arbitrary memory buffer. Use this new functionality to drastically clean up the way in which we handle file truncation for code-completion: all of the truncation/completion logic is now encapsulated in the preprocessor where it belongs (<rdar://problem/7434737>). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@90300 91177308-0d34-0410-b5e6-96231b3b80d8
2009-11-30Fix PR5633 by making the preprocessor handle the case where we canChris Lattner
stat a file but where mmaping it fails. In this case, we emit an error like: t.c:1:10: fatal error: error opening file '../../foo.h' instead of "cannot find file". git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@90110 91177308-0d34-0410-b5e6-96231b3b80d8
2009-11-14Move DISABLE_INLINE to the front of the decl so MSVC can parse it. Patch by ↵Benjamin Kramer
Amine Khaldi! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@88797 91177308-0d34-0410-b5e6-96231b3b80d8
2009-10-14Teach Lexer::MeasureTokenLength to be able to measure theChris Lattner
length of comment tokens. Patch by Abramo Bagnara! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@84100 91177308-0d34-0410-b5e6-96231b3b80d8
2009-09-22Replace the -code-completion-dump option with Douglas Gregor
-code-completion-at=filename:line:column which performs code completion at the specified location by truncating the file at that position and enabling code completion. This approach makes it possible to run multiple tests from a single test file, and gives a more natural command-line interface. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@82571 91177308-0d34-0410-b5e6-96231b3b80d8
2009-09-21Refactor and simplify the CodeCompleteConsumer, so that all of theDouglas Gregor
real work is performed within Sema. Addresses Chris's comments, but still retains the heavyweight list-of-multimaps data structure. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@82459 91177308-0d34-0410-b5e6-96231b3b80d8
2009-09-17Initial implementation of a code-completion interface in Clang. InDouglas Gregor
essence, code completion is triggered by a magic "code completion" token produced by the lexer [*], which the parser recognizes at certain points in the grammar. The parser then calls into the Action object with the appropriate CodeCompletionXXX action. Sema implements the CodeCompletionXXX callbacks by performing minimal translation, then forwarding them to a CodeCompletionConsumer subclass, which uses the results of semantic analysis to provide code-completion results. At present, only a single, "printing" code completion consumer is available, for regression testing and debugging. However, the design is meant to permit other code-completion consumers. This initial commit contains two code-completion actions: one for member access, e.g., "x." or "p->", and one for nested-name-specifiers, e.g., "std::". More code-completion actions will follow, along with improved gathering of code-completion results for the various contexts. [*] In the current -code-completion-dump testing/debugging mode, the file is truncated at the completion point and EOF is translated into "code completion". git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@82166 91177308-0d34-0410-b5e6-96231b3b80d8
2009-09-09Remove tabs, and whitespace cleanups.Mike Stump
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@81346 91177308-0d34-0410-b5e6-96231b3b80d8
2009-07-07Convert the CharInfo table to be statically initialized, instead of ↵Chris Lattner
dynamically initialized. Patch by Ryan Flynn! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@74919 91177308-0d34-0410-b5e6-96231b3b80d8
2009-07-07fix an out-of-date comment.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@74894 91177308-0d34-0410-b5e6-96231b3b80d8
2009-07-02Add support for retrieving the Doxygen comment associated with a givenDouglas Gregor
declaration in the AST. The new ASTContext::getCommentForDecl function searches for a comment that is attached to the given declaration, and returns that comment, which may be composed of several comment blocks. Comments are always available in an AST. However, to avoid harming performance, we don't actually parse the comments. Rather, we keep the source ranges of all of the comments within a large, sorted vector, then lazily extract comments via a binary search in that vector only when needed (which never occurs in a "normal" compile). Comments are written to a precompiled header/AST file as a blob of source ranges. That blob is only lazily loaded when one requests a comment for a declaration (this never occurs in a "normal" compile). The indexer testbed now supports comment extraction. When the -point-at location points to a declaration with a Doxygen-style comment, the indexer testbed prints the associated comment block(s). See test/Index/comments.c for an example. Some notes: - We don't actually attempt to parse the comment blocks themselves, beyond identifying them as Doxygen comment blocks to associate them with a declaration. - We won't find comment blocks that aren't adjacent to the declaration, because we start our search based on the location of the declaration. - We don't go through the necessary hops to find, for example, whether some redeclaration of a declaration has comments when our current declaration does not. Similarly, we don't attempt to associate a \param Foo marker in a function body comment with the parameter named Foo (although that is certainly possible). - Verification of my "no performance impact" claims is still "to be done". git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@74704 91177308-0d34-0410-b5e6-96231b3b80d8
2009-06-23Fix our check for "random whitespace between a \ and newline" to workChris Lattner
with dos style newlines. I have a trivial test for this: // RUN: clang-cc %s -verify #define test(x, y) \ x ## y but I don't know how to get svn to not change newlines and testrunner doesn't work with dos style newlines either, so "not worth it". :) rdar://6994000 git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@73945 91177308-0d34-0410-b5e6-96231b3b80d8
2009-05-13Fix rdar://6880630 - # in _Pragma does not start a preprocessor directive.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@71643 91177308-0d34-0410-b5e6-96231b3b80d8
2009-04-28Get rid of some useless uses of NoExtensions. The philosophy here is Eli Friedman
that if we're going to print an extension warning anyway, there's no point to changing behavior based on NoExtensions: it will only make error recovery worse. Note that this doesn't cause any behavior change because NoExtensions isn't used by the current front-end. I'm still considering what to do about the remaining use of NoExtensions in IdentifierTable.cpp. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@70273 91177308-0d34-0410-b5e6-96231b3b80d8
2009-04-24fix rdar://6816766 - Crash with function-like macro test at end of directive.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69964 91177308-0d34-0410-b5e6-96231b3b80d8
2009-04-18add a new Lexer::SkipEscapedNewLines method.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69483 91177308-0d34-0410-b5e6-96231b3b80d8
2009-04-18factor escape newline measuring out into its own helper function.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69482 91177308-0d34-0410-b5e6-96231b3b80d8
2009-04-18remove unneeded scopes.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69481 91177308-0d34-0410-b5e6-96231b3b80d8
2009-04-17Fix two problems from PR3916, and one problem I noticed while hackingChris Lattner
on the code. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69404 91177308-0d34-0410-b5e6-96231b3b80d8
2009-04-14Change Lexer::MeasureTokenLength to take a LangOptions reference.Chris Lattner
This allows it to accurately measure tokens, so that we get: t.cpp:8:13: error: unknown type name 'X' static foo::X P; ~~~~~^ instead of the woefully inferior: t.cpp:8:13: error: unknown type name 'X' static foo::X P; ~~~~ ^ Most of this is just plumbing to push the reference around. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@69099 91177308-0d34-0410-b5e6-96231b3b80d8
2009-04-05fix rdar://6757323, where an escaped newline in a // commentChris Lattner
was causing the char after the newline to get eaten. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@68430 91177308-0d34-0410-b5e6-96231b3b80d8
2009-04-02A code modification hint for files that don't end in a newline.Mike Stump
Eventually, would be nice to be able to run these modifications even when we don't want the warning or errors for the actual diagnostic. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@68272 91177308-0d34-0410-b5e6-96231b3b80d8
2009-03-18silence some errors that should not apply to .S files on code like:Chris Lattner
'' ' ' git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@67237 91177308-0d34-0410-b5e6-96231b3b80d8
2009-03-18properly form a full token for # before calling HandleDirective.Chris Lattner
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@67235 91177308-0d34-0410-b5e6-96231b3b80d8
2009-03-08simplify some logic by making ScratchBuffer handle the application of trailingChris Lattner
\0's to created tokens instead of making all clients do it. No functionality change. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@66373 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-19fix PR3609, emit:Chris Lattner
t.c:1:10: error: missing terminating '>' character #include <stdio.h ^ instead of: t.c:1:10: error: missing terminating " character #include <stdio.h ^ git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@65052 91177308-0d34-0410-b5e6-96231b3b80d8
2009-02-15track "just a little more" location information for macro instantiations.Chris Lattner
Now instead of just tracking the expansion history, also track the full range of the macro that got replaced. For object-like macros, this doesn't change anything. For _Pragma and function-like macros, this means we track the locations of the ')'. This is required for PR3579 because apparently GCC uses the line of the ')' of a function-like macro as the location to expand __LINE__ to. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64601 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-29move library-specific diagnostic headers into library private dirs. ReduceChris Lattner
redundant #includes. Patch by Anders Johnsen! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63271 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-27Split the single monolithic DiagnosticKinds.def file into oneChris Lattner
.def file for each library. This means that adding a diagnostic to sema doesn't require all the other libraries to be rebuilt. Patch by Anders Johnsen! git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63111 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-26Fix a bug I introduced in my changes, which caused MeasureTokenLengthChris Lattner
to crash when given an instantiation location. Thanks to Fariborz for the testcase. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63057 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-26allow _Pragmas formed from #defines to keep their full instantiationChris Lattner
history git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63035 91177308-0d34-0410-b5e6-96231b3b80d8
2009-01-26This change refactors some of the low-level lexer interfaces a bit.Chris Lattner
Token now has a class of kinds for "literals", which include numeric constants, strings, etc. These tokens can optionally have a pointer to the start of the token in the lexer buffer. This makes it faster to get spelling and do other gymnastics, because we don't have to go through source locations. This change is performance neutral, but will make other changes more feasible down the road. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63028 91177308-0d34-0410-b5e6-96231b3b80d8
generated by cgit v1.2.3-18-g5258 (git 2.32.0) at 2025-05-12 09:00:53 +0000