diff options
| -rw-r--r-- | docs/paper.tex | 51 | ||||
| -rw-r--r-- | src/jsifier.js | 2 | ||||
| -rw-r--r-- | src/preamble.js | 2 | ||||
| -rw-r--r-- | tests/runner.py | 128 | ||||
| -rw-r--r-- | third_party/CppHeaderParser/CppHeaderParser/CppHeaderParser.py | 1356 | ||||
| -rw-r--r-- | third_party/CppHeaderParser/CppHeaderParser/CppHeaderParser3.py | 669 | ||||
| -rw-r--r-- | third_party/CppHeaderParser/CppHeaderParser/__init__.py | 7 | ||||
| -rw-r--r-- | third_party/CppHeaderParser/CppHeaderParser/examples/SampleClass.h | 85 | ||||
| -rwxr-xr-x | third_party/CppHeaderParser/CppHeaderParser/examples/readSampleClass.py | 25 | ||||
| -rw-r--r-- | tools/bindings_generator.py | 69 |
10 files changed, 1529 insertions, 865 deletions
diff --git a/docs/paper.tex b/docs/paper.tex index 453e1d66..82b99336 100644 --- a/docs/paper.tex +++ b/docs/paper.tex @@ -28,9 +28,9 @@ %\maketitle \begin{abstract} -We present Emscripten, an LLVM-to-JavaScript compiler. Emscripten compiles -LLVM (Low Level Virtual Machine) assembly code into standard JavaScript, which opens up two avenues for running code written -in languages other than JavaScript on the web: (1) Compile code directly into LLVM bitcode, and +We present Emscripten, a compiler from LLVM (Low Level Virtual Machine) assembly to JavaScript. This +opens up two avenues for running code written +in languages other than JavaScript on the web: (1) Compile code directly into LLVM assemby, and then compile that into JavaScript using Emscripten, or (2) Compile a language's entire runtime into LLVM and then JavaScript, as in the previous approach, and then use the compiled runtime to run code written in that language. For example, the @@ -107,8 +107,8 @@ not Python, so for example division of integers can yield unexpected results but in JavaScript and in Pyjamas a floating-point number can be generated). In this paper we present another project along those lines: \textbf{Emscripten}, -which compiles LLVM assembly code into JavaScript. LLVM (the Low Level Virtual -Machine\footnote{\url{http://llvm.org/}}) is a compiler project primarily focused on C, C++ and +which compiles LLVM (Low Level Virtual Machine\footnote{\url{http://llvm.org/}}) assembly into JavaScript. +LLVM is a compiler project primarily focused on C, C++ and Objective-C. It compiles those languages through a \emph{frontend} (the main ones of which are Clang and LLVM-GCC) into the LLVM intermediary representation (which can be machine-readable @@ -683,6 +683,40 @@ improve the speed as well, as are improvements to LLVM, the Closure Compiler, and JavaScript engines themselves; see further discussion in the Summary. +\subsection{Limitations} + +Emscripten's compilation approach, as has been described in this Section so far, +is to generate `natural' JavaScript, as close as possible to normal JavaScript +on the web, so that modern JavaScript engines perform well on it. In particular, +we try to generate `normal' JavaScript operations, like regular addition and +multiplication and so forth. This is a very +different approach than, say, emulating a CPU on a low level, or for the case +of LLVM, writing an LLVM bitcode interpreter in JavaScript. The latter approach +has the benefit of being able to run virtually any compiled code, at the cost +of speed, whereas Emscripten makes a tradeoff in the other direction. We will +now give a summary of some of the limitations of Emscripten's approach. + +\begin{itemize} +\item \textbf{64-bit Integers}: JavaScript numbers are all 64-bit doubles, with engines + typically implementing them as 32-bit integers where possible for speed. + A consequence of this is that it is impossible to directly implement + 64-bit integers in JavaScript, as integer values larger than 32 bits will become doubles, + with only 53 bits for the significand. Thus, when Emscripten uses normal + JavaScript addition and so forth for 64-bit integers, it runs the risk of + rounding effects. This could be solved by emulating 64-bit integers, + but it would be much slower than native code. +\item \textbf{Multithreading}: JavaScript has Web Workers, which are additional + threads (or processes) that communicate via message passing. There is no + shared state in this model, which means that it is not directly possible + to compile multithreaded code in C++ into JavaScript. A partial solution + could be to emulate threads, without Workers, by manually controlling + which blocks of code run (a variation on the switch in a loop construction + mentioned earlier) and manually switching between threads every so often. + However, in that case there would not be any utilization + of additional CPU cores, and furthermore performance would be slow due to not + using normal JavaScript loops. +\end{itemize} + \section{Emscripten's Architecture} \label{sec:emarch} @@ -997,12 +1031,7 @@ things, compile real-world C and C++ code and run that on the web. In addition, by compiling the runtimes of languages which are implemented in C and C++, we can run them on the web as well, for example Python and Lua. -Important future tasks for Emscripten are to broaden its -standard library and to support additional types of code, such as -multithreaded code (JavaScript Web Workers do not support shared state, -so this is not possible directly, but it can be emulated in various ways). - -But perhaps the largest future goal of Emscripten is to improve the performance of +Perhaps the largest future goal of Emscripten is to improve the performance of the generated code. As we have seen, speeds of around $1/10$th that of GCC are possible, which is already good enough for many purposes, but can be improved much more. The code Emscripten generates will become faster diff --git a/src/jsifier.js b/src/jsifier.js index 486eeb02..2babaa05 100644 --- a/src/jsifier.js +++ b/src/jsifier.js @@ -681,7 +681,7 @@ function JSify(data, functionsOnly, givenFunctions, givenGlobalVariables) { + makeFunctionCall(item.ident, item.params, item.funcData) + ' ' + '} catch(e) { ' + 'if (ABORT) throw e; __THREW__ = true; ' - + (EXCEPTION_DEBUG ? 'print("Exception: " + e + " : " + e.stack + ", currently at: " + (new Error().stack)); ' : '') + + (EXCEPTION_DEBUG ? 'print("Exception: " + e + ", currently at: " + (new Error().stack)); ' : '') + 'return null } })(); if (!__THREW__) { ' + makeBranch(item.toLabel, item.currLabelId) + ' } else { ' + makeBranch(item.unwindLabel, item.currLabelId) + ' }'; return ret; diff --git a/src/preamble.js b/src/preamble.js index 408ef6aa..5903a5be 100644 --- a/src/preamble.js +++ b/src/preamble.js @@ -482,7 +482,7 @@ function __shutdownRuntime__() { if (typeof func === 'number') { func = FUNCTION_TABLE[func]; } - func(atexit.arg); + func(atexit.arg || null); } // allow browser to GC, set heaps to null? diff --git a/tests/runner.py b/tests/runner.py index 28abbfde..8c699330 100644 --- a/tests/runner.py +++ b/tests/runner.py @@ -2421,8 +2421,9 @@ if 'benchmark' not in sys.argv: 'hello python world!\n[0, 2, 4, 6]\n5\n22\n5.470000', args=['-S', '-c' '''print "hello python world!"; print [x*2 for x in range(4)]; t=2; print 10-3-t; print (lambda x: x*2)(11); print '%f' % 5.47''']) - ### Test cases in separate files - + # Test cases in separate files. Note that these files may contain invalid .ll! + # They are only valid enough for us to read for test purposes, not for llvm-as + # to process. def test_cases(self): global CHECK_OVERFLOWS; CHECK_OVERFLOWS = 0 if LLVM_OPTS: return self.skip() # Our code is not exactly 'normal' llvm assembly @@ -2500,6 +2501,7 @@ if 'benchmark' not in sys.argv: ### Integration tests def test_scriptaclass(self): + header_filename = os.path.join(self.get_dir(), 'header.h') header = ''' struct ScriptMe { int value; @@ -2510,8 +2512,9 @@ if 'benchmark' not in sys.argv: void mulVal(int mul); }; ''' - header_filename = os.path.join(self.get_dir(), 'header.h') - open(header_filename, 'w').write(header) + h = open(header_filename, 'w') + h.write(header) + h.close() src = ''' #include "header.h" @@ -2543,23 +2546,100 @@ if 'benchmark' not in sys.argv: # Way 2: use CppHeaderParser + header = ''' + #include <stdio.h> + + class Parent { + protected: + int value; + public: + Parent(int val); + int getVal() { return value; }; // inline should work just fine here, unlike Way 1 before + void mulVal(int mul); + }; + + class Child1 : public Parent { + public: + Child1() : Parent(7) { printf("Child1:%d\\n", value); }; + Child1(int val) : Parent(val*2) { value -= 1; printf("Child1:%d\\n", value); }; + int getValSqr() { return value*value; } + int getValSqr(int more) { return value*value*more; } + }; + + class Child2 : Parent { + public: + Child2() : Parent(9) { printf("Child2:%d\\n", value); }; + int getValCube() { return value*value*value; } + private: + void doSomethingSecret() { printf("security breached!\\n"); }; // we should not be able to do this + }; + ''' + open(header_filename, 'w').write(header) + basename = os.path.join(self.get_dir(), 'bindingtest') - Popen(['python', BINDINGS_GENERATOR, basename, header_filename], stdout=PIPE, stderr=STDOUT).communicate()[0] + output = Popen(['python', BINDINGS_GENERATOR, basename, header_filename], stdout=PIPE, stderr=STDOUT).communicate()[0] + assert 'Traceback' not in output, 'Failure in binding generation: ' + output src = ''' #include "header.h" - ScriptMe::ScriptMe(int val) : value(val) { } - int ScriptMe::getVal() { return value; } - void ScriptMe::mulVal(int mul) { value *= mul; } + Parent::Parent(int val) : value(val) { printf("Parent:%d\\n", val); } + void Parent::mulVal(int mul) { value *= mul; } #include "bindingtest.c" ''' script_src_2 = ''' - var sme = new ScriptMe(83); + var sme = new Parent(42); sme.mulVal(2); - print('*' + sme.getVal() + '*'); + print('*') + print(sme.getVal()); + + print('c1'); + + var c1 = new Child1(); + print(c1.getVal()); + c1.mulVal(2); + print(c1.getVal()); + print(c1.getValSqr()); + print(c1.getValSqr_2(3)); + + print('c1 v2'); + + c1 = new Child1_2(8); + print(c1.getVal()); + c1.mulVal(2); + print(c1.getVal()); + print(c1.getValSqr()); + print(c1.getValSqr_2(3)); + + print('c2') + + var c2 = new Child2(); + print(c2.getVal()); + c2.mulVal(2); + print(c2.getVal()); + print(c2.getValCube()); + var succeeded; + try { + succeeded = 0; + print(c2.doSomethingSecret()); // should fail since private + succeeded = 1; + } catch(e) {} + print(succeeded); + try { + succeeded = 0; + print(c2.getValSqr()); // function from the other class + succeeded = 1; + } catch(e) {} + print(succeeded); + try { + succeeded = 0; + c2.getValCube(); // sanity + succeeded = 1; + } catch(e) {} + print(succeeded); + print('*ok*'); ''' @@ -2570,7 +2650,33 @@ if 'benchmark' not in sys.argv: '// {{MODULE_ADDITIONS}' ) open(filename, 'w').write(src) - self.do_test(src, '*166*\n*ok*', post_build=post2) + self.do_test(src, '''* +84 +c1 +Parent:7 +Child1:7 +7 +14 +196 +588 +c1 v2 +Parent:16 +Child1:15 +15 +30 +900 +2700 +c2 +Parent:9 +Child2:9 +9 +18 +5832 +0 +0 +1 +*ok* +''', post_build=post2) ### Tests for tools diff --git a/third_party/CppHeaderParser/CppHeaderParser/CppHeaderParser.py b/third_party/CppHeaderParser/CppHeaderParser/CppHeaderParser.py index cc3755f7..675b74d8 100644 --- a/third_party/CppHeaderParser/CppHeaderParser/CppHeaderParser.py +++ b/third_party/CppHeaderParser/CppHeaderParser/CppHeaderParser.py @@ -41,8 +41,56 @@ # # http://www.opensource.org/licenses/bsd-license.php # -"""Parse C++ header files and generate a data structure -representing the class +""" +CppHeaderParser2.0: April 2011 - July 2011 + by HartsAntler + bhartsho@yahoo.com + http://pyppet.blogspot.com + + Quick Start - User API: + h = CppHeaderParser.CppHeader("someheader.h") + for name in h.classes: + c = h.classes[name] + for method in c['methods']['public']: + print method['name'] + print dir(method) # view the rest of the API here. + + ... TODO document more ... + + + + New Features by Hart: + should be able to parse all c++ files, not just headers + parsing global typedefs with resolution + parsing global structs + fixes nested struct in class changes accessor type + parsing if class is abstract + parsing more info about variables + save ordering of classes, structs, and typedefs + handle forward decl of class in a class + handle mutable, static, and other variable types + handle 1D arrays + + + Internal Developer Notes: + + 1. in method['rtnType'] is not recommended, instead use: 'returns', 'returns_pointer', 'returns_reference', 'returns_fundamental' + (camelCase may become deprecated in the future for the dict lookup keys) + + 2. double name stacks: + . the main stack is self.nameStack, this stack is simpler and easy to get hints from + . the secondary stack is self.stack is the full name stack, required for parsing somethings + . each stack maybe cleared at different points, since they are used to detect different things + . it looks ugly but it works :) + + 3. Tabs vs Spaces: + This file now contains tabs and spaces, the tabb'ed code is the new stuff, + in the future this should be made consistent, one way or the other. + + 4. Had to make the __repr__ methods simple because some of these dicts are interlinked. + For nice printing, call something.show() + + """ import ply.lex as lex @@ -57,8 +105,8 @@ def lineno(): return inspect.currentframe().f_back.f_lineno -__version__ = "1.9" -version = "1.9" +version = __version__ = "1.9.9" + tokens = [ 'NUMBER', 'NAME', @@ -85,7 +133,7 @@ tokens = [ 'NEW_LINE', ] -t_ignore = " \t\r[].|!?%@" +t_ignore = " \t\r[].|!?%@'^\\" # harts hack (litteral backslash is a bad idea?) - old version: " \t\r[].|!?%@" t_NUMBER = r'[0-9][0-9XxA-Fa-f]*' t_NAME = r'[<>A-Za-z_~][A-Za-z0-9_]*' t_OPERATOR_DIVIDE_OVERLOAD = r'/=' @@ -133,7 +181,7 @@ def t_NEWLINE(t): t.lexer.lineno += len(t.value) def t_error(v): - print "Lex error: ", v + print( "Lex error: ", v ) lex.lex() debug = 0 @@ -181,6 +229,8 @@ class CppClass(dict): and values are lists of CppVariable's self['enums'] - Dictionary where keys are from supportedAccessSpecifier and values are lists of CppEnum's + self['structs'] - Dictionary where keys are from supportedAccessSpecifier and + values are lists of nested Struct's An example of how this could look is as follows: #self = @@ -207,10 +257,45 @@ class CppClass(dict): } } """ + def __repr__( self ): return self['name'] + + def get_all_methods(self): + r = [] + for typ in 'public protected private'.split(): r += self['methods'][typ] + return r + + def get_all_method_names( self ): + r = [] + for typ in 'public protected private'.split(): r += self.get_method_names(typ) # returns list + return r + + def get_all_pure_virtual_methods( self ): + r = {} + for typ in 'public protected private'.split(): r.update(self.get_pure_virtual_methods(typ)) # returns dict + return r + + + def get_method_names( self, type='public' ): return [ meth['name'] for meth in self['methods'][ type ] ] + + def get_pure_virtual_methods( self, type='public' ): + r = {} + for meth in self['methods'][ type ]: + if meth['pure_virtual']: r[ meth['name'] ] = meth + return r + def __init__(self, nameStack): - if (debug): print "Class: ", nameStack + self['nested_classes'] = [] + self['parent'] = None + self['abstract'] = False + self._public_enums = {} + self._public_structs = {} + self._public_typedefs = {} + self._public_forward_declares = [] + self['namespace'] = "" + + if (debug): print( "Class: ", nameStack ) if (len(nameStack) < 2): - print "Error detecting class" + print( "Error detecting class" ) return global doxygenCommentCache if len(doxygenCommentCache): @@ -218,11 +303,15 @@ class CppClass(dict): doxygenCommentCache = "" self["name"] = nameStack[1] inheritList = [] + if ":" in nameStack: + self['name'] = nameStack[ nameStack.index(':') - 1 ] # harts hack - fixes: class __something__ classname + + if nameStack.count(':') == 1: nameStack = nameStack[nameStack.index(":") + 1:] while len(nameStack): tmpStack = [] - tmpInheritClass = {"access":"private"} + tmpInheritClass = {"access":"private"} # shouldn't public be default? if "," in nameStack: tmpStack = nameStack[:nameStack.index(",")] nameStack = nameStack[nameStack.index(",") + 1:] @@ -237,58 +326,89 @@ class CppClass(dict): tmpInheritClass["access"] = tmpStack[0] tmpInheritClass["class"] = tmpStack[1] else: - print "Warning: Cant figure out class inheriting %s\n"%(" ".join(tmpStack)) - continue - inheritList.append(tmpInheritClass) + print( "Warning: can not parse inheriting class %s"%(" ".join(tmpStack))) + if '>' in tmpStack: pass # allow skip templates for now + else: raise NotImplemented + + if 'class' in tmpInheritClass: inheritList.append(tmpInheritClass) + + elif nameStack.count(':') == 2: self['parent'] = self['name']; self['name'] = nameStack[-1] + + elif nameStack.count(':') > 2: print('ERROR can not parse', nameStack) # TODO + + self['inherits'] = inheritList + methodAccessSpecificList = {} propertyAccessSpecificList = {} enumAccessSpecificList = {} + structAccessSpecificList = {} + typedefAccessSpecificList = {} + forwardAccessSpecificList = {} for accessSpecifier in supportedAccessSpecifier: methodAccessSpecificList[accessSpecifier] = [] propertyAccessSpecificList[accessSpecifier] = [] enumAccessSpecificList[accessSpecifier] = [] - self['inherits'] = inheritList + structAccessSpecificList[accessSpecifier] = [] + typedefAccessSpecificList[accessSpecifier] = [] + forwardAccessSpecificList[accessSpecifier] = [] + self['methods'] = methodAccessSpecificList self['properties'] = propertyAccessSpecificList self['enums'] = enumAccessSpecificList - self['namespace'] = "" + self['structs'] = structAccessSpecificList + self['typedefs'] = typedefAccessSpecificList + self['forward_declares'] = forwardAccessSpecificList - def __repr__(self): + + def show(self): """Convert class to a string""" namespace_prefix = "" if self["namespace"]: namespace_prefix = self["namespace"] + "::" - rtn = "class %s\n"%(namespace_prefix + self["name"]) - try: - print self["doxygen"], - except: pass + rtn = "class %s"%(namespace_prefix + self["name"]) + if self['abstract']: rtn += ' (abstract)\n' + else: rtn += '\n' + + if 'doxygen' in self.keys(): rtn += self["doxygen"] + '\n' + if 'parent' in self.keys() and self['parent']: rtn += 'parent class:' + self['parent'] + '\n' + if "inherits" in self.keys(): - rtn += "Inherits: " + rtn += " Inherits: " for inheritClass in self["inherits"]: rtn += "%s %s, "%(inheritClass["access"], inheritClass["class"]) rtn += "\n" - rtn += "{\n" + rtn += " {\n" for accessSpecifier in supportedAccessSpecifier: - rtn += "%s\n"%(accessSpecifier) + rtn += " %s\n"%(accessSpecifier) #Enums if (len(self["enums"][accessSpecifier])): - rtn += " // Enums\n" + rtn += " <Enums>\n" for enum in self["enums"][accessSpecifier]: - rtn += " %s\n"%(repr(enum)) + rtn += " %s\n"%(repr(enum)) #Properties if (len(self["properties"][accessSpecifier])): - rtn += " // Properties\n" + rtn += " <Properties>\n" for property in self["properties"][accessSpecifier]: - rtn += " %s\n"%(repr(property)) + rtn += " %s\n"%(repr(property)) #Methods if (len(self["methods"][accessSpecifier])): - rtn += " // Method\n" + rtn += " <Methods>\n" for method in self["methods"][accessSpecifier]: - rtn += " %s\n"%(repr(method)) - rtn += "}\n" - return rtn + rtn += "\t\t" + method.show() + '\n' + rtn += " }\n" + print( rtn ) + +class _CppMethod( dict ): + def _params_helper( self, params ): + for p in params: + p['method'] = self # save reference in variable to parent method + if '::' in p['type']: + ns = p['type'].split('::')[0] + if ns not in Resolver.NAMESPACES and ns in Resolver.CLASSES: + p['type'] = self['namespace'] + p['type'] + else: p['namespace'] = self[ 'namespace' ] -class CppMethod(dict): +class CppMethod( _CppMethod ): """Takes a name stack and turns it into a method Contains the following Keys: @@ -297,8 +417,17 @@ class CppMethod(dict): self['doxygen'] - Doxygen comments associated with the method if they exist self['parameters'] - List of CppVariables """ - def __init__(self, nameStack, curClass): - if (debug): print "Method: ", nameStack + def show(self): + r = ['method name: %s (%s)' %(self['name'],self['debug']) ] + if self['returns']: r.append( 'returns: %s'%self['returns'] ) + if self['parameters']: r.append( 'number arguments: %s' %len(self['parameters'])) + if self['pure_virtual']: r.append( 'pure virtual: %s'%self['pure_virtual'] ) + if self['constructor']: r.append( 'constructor' ) + if self['destructor']: r.append( 'destructor' ) + return '\n\t\t '.join( r ) + + def __init__(self, nameStack, curClass, methinfo): + if (debug): print( "Method: ", nameStack ) global doxygenCommentCache if len(doxygenCommentCache): self["doxygen"] = doxygenCommentCache @@ -311,11 +440,16 @@ class CppMethod(dict): self["name"] = " ".join(nameStack[nameStack.index('(') - 1:nameStack.index('(')]) if len(self["rtnType"]) == 0 or self["name"] == curClass: self["rtnType"] = "void" + + self.update( methinfo ) # harts hack + paramsStack = nameStack[nameStack.index('(') + 1: ] + if paramsStack: paramsStack = paramsStack[ : paramsStack.index(')') ] # fixed april 7th #Remove things from the stack till we hit the last paren, this helps handle abstract and normal methods - while (paramsStack[-1] != ")"): - paramsStack.pop() - paramsStack.pop() + #if paramsStack: # harts hacks - this is a bug caused by change to line 88? + # while paramsStack[-1] != ")": paramsStack.pop() + # paramsStack.pop() + params = [] #See if there is a doxygen comment for the variable doxyVarDesc = {} @@ -348,17 +482,45 @@ class CppMethod(dict): #Create the variable now while (len(paramsStack)): if (',' in paramsStack): - params.append(CppVariable(paramsStack[0:paramsStack.index(',')], doxyVarDesc=doxyVarDesc)) + param = CppVariable(paramsStack[0:paramsStack.index(',')], doxyVarDesc=doxyVarDesc) + if len(param.keys()): params.append(param) paramsStack = paramsStack[paramsStack.index(',') + 1:] else: param = CppVariable(paramsStack, doxyVarDesc=doxyVarDesc) - if len(param.keys()): - params.append(param) + if len(param.keys()): params.append(param) break + + self["parameters"] = params + self._params_helper( params ) + + +class _CppVariable(dict): + def _name_stack_helper( self, stack ): + stack = list(stack) + if '=' not in stack: # TODO refactor me + # check for array[n] and deal with funny array syntax: "int myvar:99" + array = [] + while stack and stack[-1].isdigit(): array.append( stack.pop() ) + if array: array.reverse(); self['array'] = int(''.join(array)) + if stack and stack[-1].endswith(':'): stack[-1] = stack[-1][:-1] # fixed June 23, BulletPhysics + + while stack and not stack[-1]: stack.pop() # can be empty + return stack + def init(self): + #assert self['name'] # allow unnamed variables, methods like this: "void func(void);" + a = [] + self['aliases'] = []; self['parent'] = None; self['typedef'] = None + for key in 'constant reference pointer static typedefs class fundamental unresolved'.split(): + self[ key ] = 0 + for b in self['type'].split(): + if b == '__const__': b = 'const' + a.append( b ) + self['type'] = ' '.join( a ) -class CppVariable(dict): + +class CppVariable( _CppVariable ): """Takes a name stack and turns it into a method Contains the following Keys: @@ -370,10 +532,12 @@ class CppVariable(dict): self['defaltValue'] - Default value of the variable, this key will only exist if there is a default value """ + Vars = [] def __init__(self, nameStack, **kwargs): - if (debug): print "Variable: ", nameStack - if (len(nameStack) < 2): - return + nameStack = self._name_stack_helper( nameStack ) + if (debug): print( "Variable: ", nameStack ) + if (len(nameStack) < 2): return + global doxygenCommentCache if len(doxygenCommentCache): self["doxygen"] = doxygenCommentCache @@ -381,10 +545,17 @@ class CppVariable(dict): if ("=" in nameStack): self["type"] = " ".join(nameStack[:nameStack.index("=") - 1]) self["name"] = nameStack[nameStack.index("=") - 1] - self["defaltValue"] = " ".join(nameStack[nameStack.index("=") + 1:]) - else: + self["defaltValue"] = " ".join(nameStack[nameStack.index("=") + 1:]) # deprecate camelCase in dicts + self['default'] = " ".join(nameStack[nameStack.index("=") + 1:]) + + elif nameStack[-1] == '*': # rare case - function param is an unnamed pointer: "void somemethod( SomeObject* )" + self['type'] = ' '.join(nameStack) + self['name'] = '' + + else: # common case self["type"] = " ".join(nameStack[:-1]) self["name"] = nameStack[-1] + self["type"] = self["type"].replace(" :",":") self["type"] = self["type"].replace(": ",":") self["type"] = self["type"].replace(" <","<") @@ -394,7 +565,30 @@ class CppVariable(dict): self["desc"] = kwargs["doxyVarDesc"][self["name"]] except: pass -class CppEnum(dict): + self.init() + CppVariable.Vars.append( self ) # save and resolve later + +class _CppEnum(dict): + def resolve_enum_values( self, values ): + t = int; i = 0 + names = [ v['name'] for v in values ] + for v in values: + if 'value' in v: + a = v['value'].strip() + if a.isdigit(): i = a = int( a ) + elif a in names: + for other in values: + if other['name'] == a: + v['value'] = other['value'] + break + + elif '"' in a or "'" in a: t = str # only if there are quotes it this a string enum + + else: v['value'] = i + i += 1 + return t + +class CppEnum(_CppEnum): """Takes a name stack and turns it into an Enum Contains the following Keys: @@ -425,17 +619,21 @@ class CppEnum(dict): else: tmpStack = valueStack valueStack = [] - if len(tmpStack) == 1: - valueList.append({"name": tmpStack[0]}) + d = {} + if len(tmpStack) == 1: d["name"] = tmpStack[0] elif len(tmpStack) >= 3 and tmpStack[1] == "=": - valueList.append({"name": tmpStack[0], "value": " ".join(tmpStack[2:])}) + d["name"] = tmpStack[0]; d["value"] = " ".join(tmpStack[2:]) elif len(tmpStack) == 2 and tmpStack[1] == "=": - if (debug): print "Missed value for %s"%tmpStack[0] - valueList.append({"name": tmpStack[0]}) + if (debug): print( "WARN-enum: parser missed value for %s"%tmpStack[0] ) + d["name"] = tmpStack[0] + + if d: valueList.append( d ) + if len(valueList): + self['type'] = self.resolve_enum_values( valueList ) # returns int for standard enum self["values"] = valueList else: - #An enum without any values is useless, dont bother existing + print( 'WARN-enum: empty enum', nameStack ) return #Figure out if it has a name preBraceStack = nameStack[:nameStack.index("{")] @@ -444,6 +642,7 @@ class CppEnum(dict): self["name"] = preBraceStack[1] elif len(postBraceStack) and "typedef" in nameStack: self["name"] = " ".join(postBraceStack) + else: print( 'WARN-enum: nameless enum', nameStack ) #See if there are instances of this if "typedef" not in nameStack and len(postBraceStack): self["instances"] = [] @@ -453,20 +652,873 @@ class CppEnum(dict): self["instances"].append(var) self["namespace"] = "" -class CppHeader: +def is_fundamental(s): + for a in s.split(): + if a not in 'size_t struct union unsigned signed bool char short int float double long void *': return False + return True + +def is_method_namestack(stack): + r = False + if '(' not in stack: r = False + #elif '=' in stack and stack.index('=') < stack.index('(') and stack[stack.index('=')-1] != 'operator': r = False #disabled July6th - allow all operators + elif 'operator' in stack: r = True # allow all operators + elif '{' in stack and stack.index('{') < stack.index('('): r = False # struct that looks like a method/class + elif '(' in stack and ')' in stack: + if '{' in stack and '}' in stack: r = True + elif stack[-1] == ';': r = True + elif '{' in stack: r = True # ideally we catch both braces... TODO + else: r = False + return r + + +class CppStruct(dict): + Structs = [] + def __init__(self, nameStack): + if len(nameStack) >= 2: self['type'] = nameStack[1] + else: self['type'] = None + self['fields'] = [] + self.Structs.append( self ) + +C99_NONSTANDARD = { + 'int8' : 'signed char', + 'int16' : 'short int', + 'int32' : 'int', + 'int64' : 'int64_t', # this can be: long int (64bit), or long long int (32bit) + 'uint' : 'unsigned int', + 'uint8' : 'unsigned char', + 'uint16' : 'unsigned short int', + 'uint32' : 'unsigned int', + 'uint64' : 'uint64_t', # depends on host bits +} + + +def standardize_fundamental( s ): + if s in C99_NONSTANDARD: return C99_NONSTANDARD[ s ] + else: return s + + +class Resolver(object): + C_FUNDAMENTAL = 'size_t unsigned signed bool char wchar short int float double long void'.split() + C_FUNDAMENTAL += 'struct union enum'.split() + + + SubTypedefs = {} # TODO deprecate? + NAMESPACES = [] + CLASSES = {} + STRUCTS = {} + + def initextra(self): + self.typedefs = {} + self.typedefs_order = [] + self.classes_order = [] + self.structs = Resolver.STRUCTS + self.structs_order = [] + self.namespaces = Resolver.NAMESPACES # save all namespaces + self.curStruct = None + self.stack = [] # full name stack, good idea to keep both stacks? (simple stack and full stack) + self._classes_brace_level = {} # class name : level + self._structs_brace_level = {} # struct type : level + self._method_body = None + self._forward_decls = [] + self._template_typenames = [] # template<typename XXX> + + def current_namespace(self): return self.cur_namespace(True) + + def cur_namespace(self, add_double_colon=False): + rtn = "" + i = 0 + while i < len(self.nameSpaces): + rtn += self.nameSpaces[i] + if add_double_colon or i < len(self.nameSpaces) - 1: rtn += "::" + i+=1 + return rtn + + + def guess_ctypes_type( self, string ): + pointers = string.count('*') + string = string.replace('*','') + + a = string.split() + if 'unsigned' in a: u = 'u' + else: u = '' + if 'long' in a and 'double' in a: b = 'longdouble' # there is no ctypes.c_ulongdouble (this is a 64bit float?) + elif a.count('long') == 2 and 'int' in a: b = '%sint64' %u + elif a.count('long') == 2: b = '%slonglong' %u + elif 'long' in a: b = '%slong' %u + elif 'double' in a: b = 'double' # no udouble in ctypes + elif 'short' in a: b = '%sshort' %u + elif 'char' in a: b = '%schar' %u + elif 'wchar' in a: b = 'wchar' + elif 'bool' in a: b = 'bool' + elif 'float' in a: b = 'float' + + elif 'int' in a: b = '%sint' %u + elif 'int8' in a: b = 'int8' + elif 'int16' in a: b = 'int16' + elif 'int32' in a: b = 'int32' + elif 'int64' in a: b = 'int64' + + elif 'uint' in a: b = 'uint' + elif 'uint8' in a: b = 'uint8' + elif 'uint16' in a: b = 'uint16' + elif 'uint32' in a: b = 'uint32' + elif 'uint64' in a: b = 'uint64' + + elif 'size_t' in a: b = 'size_t' + elif 'void' in a: b = 'void_p' + + elif string in 'struct union'.split(): b = 'void_p' # what should be done here? don't trust struct, it could be a class, no need to expose via ctypes + else: b = 'void_p' + + if not pointers: return 'ctypes.c_%s' %b + else: + x = '' + for i in range(pointers): x += 'ctypes.POINTER(' + x += 'ctypes.c_%s' %b + x += ')' * pointers + return x + + def resolve_type( self, string, result ): # recursive + ''' + keeps track of useful things like: how many pointers, number of typedefs, is fundamental or a class, etc... + ''' + ## be careful with templates, what is inside <something*> can be a pointer but the overall type is not a pointer + ## these come before a template + s = string.split('<')[0] + result[ 'constant' ] += s.split().count('const') + result[ 'static' ] |