The Shogun toolbox saves its meta examples in .sg files written in a meta-language, as described in the docs. Can you provide any resources on the syntax of this meta-language? I suspect that the necessary information lies in the "parse.py" and "translate.py" files under "examples/meta/generator", but that doesn't help with my particular problem, math operators.
What would the correct syntax of the following line be?
bool result = ((5/2) > 3)
Make currently complains about / and >.
There are examples for basic meta language usage in examples/meta/src/meta_api/*.sg.
The meta language currently does not support basic math operators. This is in fact on purpose as the listings are just supposed to be used to illustrate Shogun's API usage.
Related
when i read through source files of opensource projects i often come across some weird phrases in the comments
/*
#brief ......
#usage.....
#remarks....
#par....
*/
questions
1.What are they?(were not mentioned when i was learning c++)
2.Do they have any documentation(where)
They are just comments and as such have no special meaning in C++. They are probably to allow a documentation generator (For example Doxygen) to extract the data from the comments.
Those are for some flavour of automatic documentation generator. Another program runs through the code looking for comments of like you see there. The #... keywords identify how the documentation should be laid out, and that program generates pretty HTML or printed documentation directly from the source code. It's a way to keep the docs up-to-date with the code more easily.
I'm on Windows, using MSVC to compile my project, but I need clang for its neat AST parser, which allow me to write a little code generator.
Problem is, clang cannot parse MSVC headers (a very-well known and understandable problem).
I tried two options :
I include MSVC header folder, parsing the built-in headers included in my code will end-up leading to a fatal error at some point, preventing me from parsing the parts I want correctly.
What I did before is simply not provide any built-in headers and forward declare the types I needed. It worked fine and somehow it doesn't anymore with latest Clang. I don't really know if the parser policy on missing header changed, but it is causing complete failure every time something like <string> is included and not much get parsed.
I am using the python bindings (libclang), but I would consider switching to C/C++ API if there would be a solution there.
Is there anyway I can alter this behavior and make clang continue parsing even when some headers are not found ?
Use SetSuppressIncludeNotFoundError. Took me an hour to find! You can imagine how glad I was to find it!
https://clang.llvm.org/doxygen/classclang_1_1Preprocessor.html#ac7bafe67fc32e41460855b39d20ff6af
One way to ignore the errors due to missing headers is to set SetSuppressIncludeNotFoundError to true in your definition of ASTFrontendAction. An example for the same is given below.
{
public:
virtual std::unique_ptr<clang::ASTConsumer> CreateASTConsumer(
clang::CompilerInstance &Compiler, llvm::StringRef InFile)
{
Compiler.getPreprocessor().SetSuppressIncludeNotFoundError(true);
return std::unique_ptr<clang::ASTConsumer>(
new CustomASTConsumer(&Compiler.getASTContext()));
}
};
For a complete example using ASTFrontendAction, please visit at https://clang.llvm.org/docs/RAVFrontendAction.html
So you want to process C++ code that uses MS headers, and you want access to ASTs so that you can generate code. And Clang won't handle MS headers.
So Clang can't be the answer unless it gets a radical upgrade.
You asked for "any solution that can make this work".
Our DMS Software Reengineering Tookit with its C++14 Front End can do this.
DMS provides general parsing, AST construction/inspection/transformation/generation, and inverse parsing (conversion of ASTs back into compilable code), parameterized by language definitions.
The C++ front end provides a full C++14 parser, preprocessor handling, AST construction, and full name and type resolution. It has been tested with GCC and MS VS 2013 header files; we're testing with 2015 header files now.
(It also handles MS VS 2013 syntax, too).
It handles the tough parsing cases completely, including the C++ famous "most vexing parse". You can see parse trees at get human readable AST from c++ code.
DMS does not provide Python bindings, nor a direct C++ interface. Rather, it is a standalone tool designed to support the construction of custom tools (e.g., your "little code generator"). It has its own very extensive set of internal APIs, coded in metaprogramming language PARLANSE, which is LISP-like. Other aspects of DMS are managed by using DSLs for lexers, grammars, and transformations. See below.
A word of caution: any tool that can process C++ is gauranteed to be complex. DMS is correspondingly complex, and it takes a while to learn to use it, so you're not going to get instant answers. The good news here
is that some things are easier to do. Your code generation problem
is likely "read a skeleton file, and then replace key entries in it with problem specific code". If that's the case, a DMS tool with the following code (simplified for presentation here) will likely do the trick:
...
(= myAST (Registry:ParseFile (. filename) (. `CppVisualStudio2013') ...)
(Registry:ApplyTransforms myAST (. `MyTransforms.rsl'))
(Registry:PrettyPrint myAST (concat filename `.modified'))
...
with a transforms file MyTransforms.rsl containing source-to-source surface-syntax (e.g, C++ syntax) transformation rules of the conceptual form
rule rulename if_you_see THIS then replace_by ("-->") THAT
An actual C++ rule might look like (making this up because I don't
know your actual code generation goals)
rule replace_abstraction(s: STRING_LITERAL):
" abstraction_place_holder(\s) "
-> " my_DSL_library(\s,17); "
The ApplyTransforms call above will apply all the rules in this file until none apply any further.
Writing surface syntax transforms, where you can do it, is way easier than making calls on a procedure library (which, like Clang, DMS offers) that hack at the tree.
You can write more complex metaprograms using PARLANSE to apply some rules in one place, other rules someplace else, and you can mix source-to-source transforms with procedural transforms that hack directly at the tree if you want.
If you want more details on what transforms look like, ask and I'll provide a link.
I am using cppcheck for static analysis. To accelerate review process I want to set up cppcheck to look for some custom rules,
for example to check if geter functions defined as a const.
If anyone has experience in writing custom rules for cppcheck please can you provide some example to write custom rules?.
P.S I have done some research to find a tool which will allow me to write custom rules and make review process faster.
I have find these links about this topic
What open source C++ static analysis tools are available?
C++ static code analysis tool on Windows
A free tool to check C/C++ source code against a set of coding standards?
I am a Cppcheck developer.
You can perhaps use the --rule and --rule-file options to add such rules. Maybe you can use a regular expression such as:
\sget[A-Za-z]+\(\)\s+{\s+return
It depends on your code base.
If you can write a regular expression then this is the most direct and simple way to create a custom rule.
For more information, read the "Writing rules" articles here:
http://sourceforge.net/projects/cppcheck/files/Articles/
But maybe you want to write more sophisticated rules that search for such getter methods by using the Cppcheck SymbolDatabase, tokenlist and syntax tree. You can't use --rule and --rule-file then. You have these choices then:
Use --dump and write your own custom scripts that read the output data (xml).
Write C++ code and compile it into Cppcheck. This is relatively straightforward imho but requires that you compile Cppcheck yourself.
I'm about to parse a lot of files, with a precise hierarchy for the XML ones and a precise syntax for the others. I would like to abstract these files at "token level" to simplify my code and logic. I also need UTF-8 support.
Is there a library, or perhaps a library only formed by several headers, that can do this in C++?
EDIT:
supposing that my file is something like that
COLOR=red Language=en
COLOR=blue Language=se
COLOR=green Language=fr
with token level i mean that i can access this values after parsing in this way:
Object.getValue(color, 1)
and this should return red.
Well, there is boost::spirit, that I think may be what you are looking for. You can use it to create rules for parsing input. I personally have never used it but heard good comments for it. Hope it helps.
I want to programmatically parse and edit C++ source files. I need to change/add code in certain sections of code (i.e. in functions, class blocks, etc). I would also (preferably) be able to get comments as well.
Part of what I want to do can be explained by the following piece of code:
CPlusPlusSourceParser cp = new CPlusPlusSourceParser(“x.cpp”); // Create C++ Source Parser Object
CPlusPlusSourceFunction[] funcs = cp.getFunctions(); // Get all the functions
for (int i = 0; i < funcs.length; i++) { // Loop through all functions
funcs[i].append(/* … code I want to append …*/); // Append some code to function
}
cp.save(); // Save new source
cp.close(); // Close file
How can I do that?
I’d like to be able to do this preferably in Java, C++, Perl, Python or C#. However, I am open to other language API’s.
This is similar to AST from C code
If your comfortable with Java antlr can easily parser your code into an abstract syntax tree, and then apply transformation to that tree. A default AST transform is to simply print out the original source.
You can use any parser generator tool to generate a c++ parser for you, but first you have to get the CFG (context free grammar) for C++ , check Antlr
Edit:
Also Antlr supports a lot of target languages
You need a working grammar and parser for C++ which is, however, not too easy as this can't be constructed with most parser generators out there. But once you have a parser you can actually take the abstract syntax tree of the program and alter it in nearly any way you want.
The Mozilla project has a tool that does this.
The Clang static analyzer is now somewhat famous for doing a good job analyzing and rewriting C++. Stroustrup wrote a paper about a research project at Texas A&M, but I don't think it's been released.
A robust C++ parser is available with our DMS Software Reengineering Toolkit. It parses a variety of C++ dialects including ANSI, GNU 3/4, MSVS6 and MSVisual Studio 2005 and managaged C++.
It builds ASTs and symbol tables (the latter is way harder than you might think). You can navigate the ASTs, transform into different valid C++ programs, and regenerate code including comments.
In a C# -- or general .net -- approach, you might be able to get some use out of the C++/CLI CodeDOM provider -- having not used the C++ version of this type, I don't know how well it would handle code that is template heavy.
have a look at the doxygen project, its a open source project, to parse and document several programming languages, C++ included. I believe using this project's lexer will get you more than half the way