How to visit a serialized clang abstract syntax tree (AST) - c++

I've been able to implement an ASTFrontendAction to create an ASTConsumer, which uses a RescursiveASTVisitor to traverse a translation unit decl, thereby visiting all the nodes of an AST for a given source file. I do this by implementing a ToolAction which is passed to ClangTool::run(ToolAction *action). This tool overrides the ToolAction::runInvocation member function to do some processing between each call to my ASTFrontendAction. So far so good, everything is working as expected, and my custom clang tool is helping me better explore a rather large, over 15 years old code base.
However, every time I want to run my tool, I need to do a full blown parse of the ASTs. As I mentioned, this is a rather large code base, so it takes a while to do a single run. I have gathered, by looking through the code, that it is possible to create and traverse an AST from a saved file, rather than perform the parse. Googling around confirmed that it's possible to save an AST, and by looking at the ClangTool and ASTUnit API, it seems it's pretty easy to do.
While it seems straightforward to save the AST, I have no idea how to make use of a saved AST when running my custom clang tool. Looking at the code path for running a tool, I see there is a point where the AST is created either by parsing a source file or by reading it from a file. What I would like to do is have all my ASTs available in a file(s), so that each run of my tool will create the ASTs from a file, and not need to perform a full parse (which, I assume would be much faster).
Could someone please help? Thanks in advance!

This worked for me:
clang::CompilerInstance CI;
CI.createDiagnostics();
std::shared_ptr<clang::TargetOptions> TO = std::make_shared<clang::TargetOptions>();
TO->Triple = "x86_64-pc-win32"; // see clang -v
CI.setTarget(clang::TargetInfo::CreateTargetInfo(CI.getDiagnostics(), TO));
std::unique_ptr<ASTUnit> ast = ASTUnit::LoadFromASTFile("TheAstFile.ast",
CI.getPCHContainerReader(),
ASTUnit::LoadEverything,
&CI.getDiagnostics(),
CI.getFileSystemOpts());
MyAstConsumer consumer(ast->getASTContext());
consumer.HandleTranslationUnit(ast->getASTContext());

Related

Get runtime ColdFusion syntax trees?

Is it possible to get access to / modify ColdFusion syntax trees at run time?
I'd wager not, and a 10 minute google search didn't find anything. Fiddling with closures and writing metadata dumps, we can see stringified versions of objects like [runtime expression], for example in the following:
function x(a=b+1) {}
WriteDump(getMetaData(x).parameters[1]["default"]);
Does it allow us to go no deeper than this, or perhaps someone knows how to keep digging and start walking trees?
Default UDF parameter expressions aren't available in function metadata as you've found. Other libraries that have implemented some form of CFML parser are
CFLint (written in Java and using ANTLR)
https://github.com/cflint/CFLint
CFFormat (also uses a binary compiled from Rust)
https://www.forgebox.io/view/commandbox-cfformat
Function LineNums (pure CFML)
https://www.forgebox.io/view/funclinenums
There is also a function callStackGet() docs: https://cfdocs.org/callstackget which might be useful to whatever you are trying to do.
And another CFML parser (written in CFML) here: https://github.com/foundeo/cfmlparser

How to change back to code from html created by doxygen

I batch create the documents from the code by using doxygen. However, I lost code and I didn't lose the document. I want to convert the code back from the documents. Is there any option in doxygen to do this? Thank you very much.
By the way, the documents are all html files
Doxygen is a documentation generator; it's job is to go from code to documentation. As such, it has no functionality for reversing this process. Especially since the generated HTML can change from version to version.
Documentation conversion is also an inherently lossy process. Unless you outputted all of your source code into the documentation, you're not going to be able to reconstruct everything. The best you might do is rebuild most aspects of some headers, but even then, anything that goes undocumented (like header include files and such) won't be in the HTML.

generate C/C++ command line argument parsing code from XML (or similar)

Is there a tool that generates C/C++ source code from XML (or something similar) to create command line argument parsing functionality?
Now a longer explanation of the question:
I have up til now used gengetopt for command line argument parsing. It is a nice tool that generates C source code from its own configuration format (a text file). For instance the gengetopt configuration line
option "max-threads" m "max number of threads" int default="1" optional
among other things generates a variable
int max_threads_arg;
that I later can use.
But gengetopt doesn't provide me with this functionality:
A way to generate Unix man pages from the gengetopt configuration format
A way to generate DocBook or HTML documentation from the gengetopt configuration format
A way to reuse C/C++ source code and to reuse gengetopt configuration lines when I have multiple programs that share some common command line options
Of course gengetopt can provide me with a documentation text by running
command --help
but I am searching for marked up documentation (e.g. HTML, DocBook, Unix man pages).
Do you know if there is any C/C++ command line argument tool/library with a liberal open source license that would suite my needs?
I guess that such a tool would use XML to specify the command line arguments. That would make it easy to generate documentation in different formats (e.g. man pages). The XML file should only be needed at build time to generate the C/C++ source code.
I know it is possible to use some other command line argument parsing library to read a configuration file in XML at runtime but I am looking for a tool that generate C/C++ source code from XML (or something similar) at build time.
Update 1
I would like to do as much as possible of the computations at compile time and as less as possible at run time. So I would like to avoid libraries that give you a map of the command line options, like for instance boost::program_options::variables_map ( tutorial ).
I other words, I prefer args_info.iterations_arg to vm["iterations"].as<int>()
User tsug303 suggested the library TCLAP. It looks quite nice. It would fit my needs to divide the options into groups so that I could reuse code when multiple programs share some common options. Although it doesn't generate out the source code from a configuration file format in XML, I almost marked that answer as the accepted answer.
But none of the suggested libraries fullfilled all of my requirements so I started thinking about writing my own library. A sketch: A new tool that would take as input a custom XML format and that would generate both C++ code and an XML schema. Some other C++ code is generated from the XML schema with the tool CodeSynthesis XSD. The two chunks of C++ code are combined into a library. One extra benefit is that we get an XML Schema for the command line options and that we get a way to serialize all of them into a binary format (in CDR format generated from CodeSynthesis XSD). I will see if I get the time to write such a library. Better of course is to find a libraray that has already been implemented.
Today I read about user Nore's suggested alternative. It looks promising and I will be eager to try it out when the planned C++ code generation has been implemented. The suggestion from Nore looks to be the closest thing to what I have been looking for.
Maybe this TCLAP library would fit your needs ?
May I suggest you look at this project. It is something I am currently working on: A XSD Schema to describe command line arguments in XML. I made XSLT transformations to create bash and Python code, XUL frontend interface and HTML documentation.
Unfortunately, I do not generate C/C++ code yet (it is planed).
Edit: a first working version of the C parser is now available. Hope it helps
I will add yet another project called protoargs. It generates C++ argument parser code out of protobuf proto file, using cxxopts.
Unfortunately it does not satisfy all author needs. No documentation generated. no compile time computation. However someone may find it useful.
UPD: As mentioned in comments, I must specify that this is my own project

In a SConstruct, how do you get what 'scons --tree=all' would print out?

This stackoverflow question shows how to get a list of all the dependencies for one or more targets. How do you get the same thing from within a SConstruct file?
Edit:
That is, as an Object or something. You could always recursively call scons -tree=all and grab stdout from that process, but I don't want a hack. I'd like access to the dependency object scons is using.
The best way to get this information is to just look at the source for SCons. You're looking for the file SCons/Script/Main.py, and in particular for the TreePrinter class. The basic idea is that all SCons nodes contain within themselves all the relevant details for reconstructing the dependancy graph.
The simplest test case I could think of is:
import os
env = Environment(ENV = os.environ)
app = env.Program('dummy', 'main.cpp')
for item in app:
for child in item.all_children():
print child
The code for doing this isn't really that complicated, but I certainly wouldn't want to rely on it not changing (it isn't part of the public interface for SCons).
I do not think that is possible from within the code. The code in the SConstruct file is the code that generates the tree. So the tree cannot be provided before all the code in the SConstruct file is executed. On the other hand I think it should be possible to get the object you mention within python, if you have the patience to look through the Scons code to see how they operate their --tree option. But you will have to wait until the tree is generated before you access it.

xsd-based code generator to build xml?

I have a schema (xsd), and I want to create xml files that conform to it.
I've found code generators that generate classes which can be loaded from an xml file (CodeSynthesis). But I'm looking to go the other direction.
I want to generate code that will let me build an object which can easily be written out as an xml file. In C++. I might be able to use Java for this, but C++ would be preferable. I'm on solaris, so a VisualStudio plugin won't help me (such as xsd2code).
Is there a code generator that lets me do this?
To close this out: I did wind up using CodeSynthesis. It worked very well, as long as I used a single xsd as its source. Since I actually had two xsds (one imported the other), I had to manually merge them (they did some weird inheritance that needed manual massaging).
But yes, Code Synthesis was the way to go.