aboutsummaryrefslogtreecommitdiff
path: root/lib/AST/CommentLexer.cpp
AgeCommit message (Collapse)Author
2013-03-04Comment parsing: refactor handling of command markers in ASTDmitri Gribenko
* Use the term 'command marker', because the semantics of 'backslash' and 'at' commands are the same. (Talking about 'at commands' makes them look like a special entity.) * Sink the flag down into bitfields, reducing the size of AST nodes. * Change the flag into an enum for clarity. Boolean function parameters are not very clear. * Add unittests for new tok::at_command tokens. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@176461 91177308-0d34-0410-b5e6-96231b3b80d8
2013-03-02Some refactoring in my patch on documentFariborz Jahanian
command source fidelity. // rdar://13066276 git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@176401 91177308-0d34-0410-b5e6-96231b3b80d8
2013-03-01comment parsing. Keep the original command format Fariborz Jahanian
in AST for source fidelity and use it in diagnostics to refer to the original format. // rdar://13066276 git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@176387 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-10Use static functions instead of an unnamed namespaceDmitri Gribenko
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@174835 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-09Comment parsing: use CharInfo.hDmitri Gribenko
This also gives us 0.2% speedup on '-fsyntax-only -Wdocumentation' time for a testcase that consists of all Clang headers. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@174810 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-30Comment parsing: resolve more named character referencesDmitri Gribenko
This reimplements r173850 with a better approach: (1) use a TableGen-generated matcher instead of doing a linear search; (2) avoid allocations for new strings by converting code points to string iterals with TableGen. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@173931 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-30Move UTF conversion routines from clang/lib/Basic to llvm/lib/SupportDmitri Gribenko
This is required to use them in TableGen. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@173924 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-30Removed couple of html named character references inFariborz Jahanian
my last patch. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@173856 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-29[Doc parsing] Patch to parse Doxygen-supported HTML character Fariborz Jahanian
references to their UTIF-8 encoding. Reviewed offline by Doug. // rdar://12392215 git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@173850 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-19Use llvm::hexDigitValue in comment lexerDmitri Gribenko
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@172924 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-30Comment lexing: replace manual comparison with StringRef::find_first_ofDmitri Gribenko
This gives an about 1.8% improvement on Clang bootstrap with -Wdocumentation git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@171262 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-14Comment parsing: don't parse comment marker followed by a digit as a commandDmitri Gribenko
since no Doxygen command starts with a digit. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@163909 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-10Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp.Dmitri Gribenko
Now we have a list of all commands. This is a good thing in itself, but it also enables us to easily implement typo correction for command names. With this change we have objects that contain information about each command, so it makes sense to resolve command name just once during lexing (currently we store command names as strings and do a linear search every time some property value is needed). Thus comment token and AST nodes were changed to contain a command ID -- index into a tables of builtin and registered commands. Unknown commands are registered during parsing and thus are also uniformly assigned an ID. Using an ID instead of a StringRef is also a nice memory optimization since ID is a small integer that fits into a common bitfield in Comment class. This change implies that to get any information about a command (even a command name) we need a CommandTraits object to resolve the command ID to CommandInfo*. Currently a fresh temporary CommandTraits object is created whenever it is needed since it does not have any state. But with this change it has state -- new commands can be registered, so a CommandTraits object was added to ASTContext. Also, in libclang CXComment has to be expanded to include a CXTranslationUnit so that all functions working on comment AST nodes can get a CommandTraits object. This breaks binary compatibility of CXComment APIs. Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't need TU parameter anymore, so it was removed. This is a source-incompatible change for this C API. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@163540 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-31Remove the useless CommentOptions class.Dmitri Gribenko
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@162986 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-31Comment HTML tag name machers: move from StringSwitch to an efficientDmitri Gribenko
TableGen-generated string matcher. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@162969 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-22Comment parsing: parse "<blah" as an HTML tag only if "blah" is a known tagDmitri Gribenko
name. This should reduce the amount of warning false positives about bad HTML in comments when the comment author intended to put a reference to a template. This change will also enable us parse the comment as intended in these cases. Fixes part 1 of PR13374. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@162407 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-09Comment parsing: extract TableGen'able pieces into new CommandTraits class.Dmitri Gribenko
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@161548 91177308-0d34-0410-b5e6-96231b3b80d8
2012-07-27Implement resolving of HTML character references (named: &amp;, decimal: &#42;,Dmitri Gribenko
hex: &#x1a;) during comment parsing. Now internal representation of plain text in comment AST does not contain character references, but the characters themselves. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@160891 91177308-0d34-0410-b5e6-96231b3b80d8
2012-07-18Comment parsing: don't parse whitespace before \endverbatim as a separate ↵Dmitri Gribenko
line of whitespace. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@160464 91177308-0d34-0410-b5e6-96231b3b80d8
2012-07-13Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML ↵Dmitri Gribenko
start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@160153 91177308-0d34-0410-b5e6-96231b3b80d8
2012-07-11Enable comment parsing and semantic analysis to emit diagnostics. A fewDmitri Gribenko
diagnostics implemented -- see testcases. I created a new TableGen file for comment diagnostics, DiagnosticCommentKinds.td, because comment diagnostics don't logically fit into AST diagnostics file. But I don't feel strongly about it. This also implements support for self-closing HTML tags in comment lexer and parser (for example, <br />). In order to issue precise diagnostics CommentSema needs to know the declaration the comment is attached to. There is no easy way to find a decl by comment, so we match comments and decls in lockstep: after parsing one declgroup we check if we have any new, not yet attached comments. If we do -- then we do the usual comment-finding process. It is interesting that this automatically handles trailing comments. We pick up not only comments that precede the declaration, but also comments that *follow* the declaration -- thanks to the lookahead in the lexer: after parsing the declgroup we've consumed the semicolon and looked ahead through comments. Added -Wdocumentation-html flag for semantic HTML errors to allow the user to disable only HTML warnings (but not HTML parse errors, which we emit as warnings in -Wdocumentation). git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@160078 91177308-0d34-0410-b5e6-96231b3b80d8
2012-07-09Comment lexing: fix lexing to actually work in non-error cases.Dmitri Gribenko
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@159963 91177308-0d34-0410-b5e6-96231b3b80d8
2012-07-06Implement AST classes for comments, a real parser for Doxygen comments and aDmitri Gribenko
very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@159790 91177308-0d34-0410-b5e6-96231b3b80d8
2012-06-27Fix an infinite loop in comment lexer: we were not advancing in the input ↵Dmitri Gribenko
character stream when we saw a '<' that is not a start of an HTML tag. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@159303 91177308-0d34-0410-b5e6-96231b3b80d8
2012-06-27Remove unsigned and a pointer from a comment token (so that each token can ↵Dmitri Gribenko
have only one semantic string value attached to it), at a cost of adding an additional token. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@159270 91177308-0d34-0410-b5e6-96231b3b80d8
2012-06-27Comment lexer: counting backwards from token end is thought to be confusing. ↵Dmitri Gribenko
We already have a pointer to the beginning of the token, so use it to extract the text instead. git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@159269 91177308-0d34-0410-b5e6-96231b3b80d8
2012-06-26Implement a lexer for structured comments.Dmitri Gribenko
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@159223 91177308-0d34-0410-b5e6-96231b3b80d8
generated by cgit v1.2.3-18-g5258 (git 2.32.0) at 2025-05-04 21:40:15 +0000