Keep history of macro definitions and #undefs

Summary:
Summary: Keep history of macro definitions and #undefs with corresponding source locations, so that we can later find out all macros active in a specified source location. We don't save the history in PCH (no need currently). Memory overhead is about sizeof(void*)*3*<number of macro definitions and #undefs>+<in-memory size of all #undef'd macros>

I've run a test on a file composed of 109 .h files from boost 1.49 on x86-64 linux.
Stats before this patch:
*** Preprocessor Stats:
73222 directives found:
  19171 #define.
  4345 #undef.
  #include/#include_next/#import:
    5233 source files entered.
    27 max include stack depth
  19210 #if/#ifndef/#ifdef.
  2384 #else/#elif.
  6891 #endif.
  408 #pragma.
14466 #if/#ifndef#ifdef regions skipped
80023/451669/1270 obj/fn/builtin macros expanded, 85724 on the fast path.
127145 token paste (##) operations performed, 11008 on the fast path.

Preprocessor Memory: 5874615B total
  BumpPtr: 4399104
  Macro Expanded Tokens: 417768
  Predefines Buffer: 8135
  Macros: 1048576
  #pragma push_macro Info: 0
  Poison Reasons: 1024
  Comment Handlers: 8

Stats with this patch:
...
Preprocessor Memory: 7541687B total
  BumpPtr: 6066176
  Macro Expanded Tokens: 417768
  Predefines Buffer: 8135
  Macros: 1048576
  #pragma push_macro Info: 0
  Poison Reasons: 1024
  Comment Handlers: 8

In my test increase in memory usage is about 1.7Mb, which is ~28% of initial preprocessor's memory usage and about 0.8% of clang's total VMM allocation.

As for CPU overhead, it should only be noticeable when iterating over all macros, and should mostly consist of couple extra dereferences and one comparison per macro + skipping of #undef'd macros. It's less trivial to measure, though, as the preprocessor consumes a very small fraction of compilation time.


Reviewers: doug.gregor, klimek, rsmith, djasper

Reviewed By: doug.gregor

CC: cfe-commits, chandlerc

Differential Revision: http://llvm-reviews.chandlerc.com/D28

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@162810 91177308-0d34-0410-b5e6-96231b3b80d8

1 files changed, 38 insertions, 36 deletions

diff --git a/lib/Lex/MacroInfo.cpp b/lib/Lex/MacroInfo.cpp
index 3d0c9a1c2b..7a4964f7d2 100644
--- a/lib/Lex/MacroInfo.cpp
+++ b/lib/Lex/MacroInfo.cpp
@@ -15,44 +15,46 @@
 #include "clang/Lex/Preprocessor.h"
 using namespace clang;
 
-MacroInfo::MacroInfo(SourceLocation DefLoc) : Location(DefLoc) {
-  IsFunctionLike = false;
-  IsC99Varargs = false;
-  IsGNUVarargs = false;
-  IsBuiltinMacro = false;
-  IsFromAST = false;
-  ChangedAfterLoad = false;
-  IsDisabled = false;
-  IsUsed = false;
-  IsAllowRedefinitionsWithoutWarning = false;
-  IsWarnIfUnused = false;
-  IsDefinitionLengthCached = false;
-  IsPublic = true;
-  
-  ArgumentList = 0;
-  NumArguments = 0;
+MacroInfo::MacroInfo(SourceLocation DefLoc)
+  : Location(DefLoc),
+    PreviousDefinition(0),
+    ArgumentList(0),
+    NumArguments(0),
+    IsDefinitionLengthCached(false),
+    IsFunctionLike(false),
+    IsC99Varargs(false),
+    IsGNUVarargs(false),
+    IsBuiltinMacro(false),
+    IsFromAST(false),
+    ChangedAfterLoad(false),
+    IsDisabled(false),
+    IsUsed(false),
+    IsAllowRedefinitionsWithoutWarning(false),
+    IsWarnIfUnused(false),
+    IsPublic(true) {
 }
 
-MacroInfo::MacroInfo(const MacroInfo &MI, llvm::BumpPtrAllocator &PPAllocator) {
-  Location = MI.Location;
-  EndLocation = MI.EndLocation;
-  ReplacementTokens = MI.ReplacementTokens;
-  IsFunctionLike = MI.IsFunctionLike;
-  IsC99Varargs = MI.IsC99Varargs;
-  IsGNUVarargs = MI.IsGNUVarargs;
-  IsBuiltinMacro = MI.IsBuiltinMacro;
-  IsFromAST = MI.IsFromAST;
-  ChangedAfterLoad = MI.ChangedAfterLoad;
-  IsDisabled = MI.IsDisabled;
-  IsUsed = MI.IsUsed;
-  IsAllowRedefinitionsWithoutWarning = MI.IsAllowRedefinitionsWithoutWarning;
-  IsWarnIfUnused = MI.IsWarnIfUnused;
-  IsDefinitionLengthCached = MI.IsDefinitionLengthCached;
-  DefinitionLength = MI.DefinitionLength;
-  IsPublic = MI.IsPublic;
-  
-  ArgumentList = 0;
-  NumArguments = 0;
+MacroInfo::MacroInfo(const MacroInfo &MI, llvm::BumpPtrAllocator &PPAllocator)
+  : Location(MI.Location),
+    EndLocation(MI.EndLocation),
+    UndefLocation(MI.UndefLocation),
+    PreviousDefinition(0),
+    ArgumentList(0),
+    NumArguments(0),
+    ReplacementTokens(MI.ReplacementTokens),
+    DefinitionLength(MI.DefinitionLength),
+    IsDefinitionLengthCached(MI.IsDefinitionLengthCached),
+    IsFunctionLike(MI.IsFunctionLike),
+    IsC99Varargs(MI.IsC99Varargs),
+    IsGNUVarargs(MI.IsGNUVarargs),
+    IsBuiltinMacro(MI.IsBuiltinMacro),
+    IsFromAST(MI.IsFromAST),
+    ChangedAfterLoad(MI.ChangedAfterLoad),
+    IsDisabled(MI.IsDisabled),
+    IsUsed(MI.IsUsed),
+    IsAllowRedefinitionsWithoutWarning(MI.IsAllowRedefinitionsWithoutWarning),
+    IsWarnIfUnused(MI.IsWarnIfUnused),
+    IsPublic(MI.IsPublic) {
   setArgumentList(MI.ArgumentList, MI.NumArguments, PPAllocator);
 }