LLVM generally gives Control Flow Graphs (CFGs) for its intermediate representation (IR) language. You can also get high-level source-code-based CFGs with little effort. I want to get CFGs at the level of Machine Code. Is there any way to get this?
I did a little bit of digging around. In LLVM's back-end code generation phase, there's a stage called SSA-based Machine Code Optimizations. There's not much information on this stage. However, I guess LLVM generates a SSA-based machine code in some intermediate stage. If such a stage exists, then we can have Basic Blocks based on the code at that stage. With those Basic Blocks, a CFG could be created on that stage. Can anybody give any clue on the source-file that I have to look in the LLVM source tree (possibly in lib\CodeGen) to find any information regarding this? Or the class that would give me SSA-based Machine Code walk-through and Basic Blocks? I would appreciate any pointer.
I figured it out.
You need to write MachineFunctionPass for some target in lib\Target\<target architecture> folder.
Then in the runOnMachineFunction(MachineFunction &MF) function, you can view a CFG by calling the MF.viewCFG() function(in debug mode or with some tweaking inside the viewCFG to get CFG in Release mode as well).
You can access MachineBasicBlock and MachineInstr through the iterator over MF. Following is an example:
int i = 0;
for (auto &MBB : MF) {
errs() << "Basic Block: " << i++ << "\n\n";
for (auto &MI : MBB) {
MI.print(errs(), true, false);
errs() << "\n";
}
}
Related
I have a project where I want to dynamically build a graph of specific function calls. For example if I have 2 template classes, A and B, where A have a tracked method (saved as graph node) and B has 3 methods (non-tracked method, tracked method and a tracked method which calls A's tracked method), then I want to be able to only register the tracked method calls into the graph object as nodes. The graph object could be a singleton.
template <class TA>
class A
{
public:
void runTracked()
{
// do stuff
}
};
template <class TB>
class B
{
public:
void runNonTracked()
{
// do stuff
}
void runTracked()
{
// do stuff
}
void callATracked()
{
auto a = A<TB>();
a.runTracked();
// do stuff
}
};
void root()
{
auto b1 = B<int>();
auto b2 = B<double>();
b1.runTracked();
b2.runNonTracked();
b2.callATracked();
}
int main()
{
auto b = B<int>();
b.runTracked()
root();
return 0;
}
This should output a similar graph object to the below:
root()
\-- B<int>::runTracked()
\-- B<double>::callATracked()
\-- A<double>::runTracked()
The tracked functions should be adjustable. If the root would be adjustable (as in the above example) that would be the best.
Is there an easy way to achieve this?
I was thinking about introducing a macro for the tracked functionalities and a Singleton graph object which would register the tracked functions as nodes. However, I'm not sure how to determine which is the last tracked function in the callstack, or (from the graphs perspective) which graph node should be the parent when I want to add a new node.
In general, you have 2 strategies:
Instrument your application with some sort of logging/tracing framework, and then try to replicate some sort of tracing mixin-like functionality to apply global/local tracing depending on which parts of code you apply the mixins.
Recompile your code with some sort of tracing instrumentation feature enabled for your compiler or runtime, and then use the associated tracing compiler/runtime-specific tools/frameworks to transform/sift through the data.
For 1, this will require you to manually insert more code or something like _penter/_pexit for MSVC manually or create some sort of ScopedLogger that would (hopefully!) log async to some external file/stream/process. This is not necessarily a bad thing, as having a separate process control the trace tracking would probably be better in the case where the traced process crashes. Regardless, you'd probably have to refactor your code since C++ does not have great first-class support for metaprogramming to refactor/instrument code at a module/global level. However, this is not an uncommon pattern anyways for larger applications; for example, AWS X-Ray is an example of a commercial tracing service (though, typically, I believe it fits the use case of tracing network calls and RPC calls rather than in-process function calls).
For 2, you can try something like utrace or something compiler-specific: MSVC has various tools like Performance Explorer, LLVM has XRay, GCC has gprof. You essentially compile in a sort of "debug++" mode or there is some special OS/hardware/compiler magic to automatically insert tracing instructions or markers that help the runtime trace your desired code. These tracing-enabled programs/runtimes typically emit to some sort of unique tracing format that must then be read by a unique tracing format reader.
Finally, to dynamically build the graph in memory is a a similar story. Like the tracing strategies above, there are a variety of application and runtime-level libraries to help trace your code that you can interact with programmatically. Even the simplest version of creating ScopedTracer objects that log to a tracing file can then be fitted with a consumer thread that owns and updates the trace graph with whatever desired latency and data durability requirements you have.
Edit: If you would like, OpenTelemetry/Jaeger may be a good place to start visualizing traces once you have extracted the data (and you can also report directly to it if you want), although it prefers a tree presentation format: Jaeger documentation for Trace Detail View
I have a protobuf message with non-consecutive enum values something like this:
message Information {
enum Versions {
version1 = 0;
version2 = 1;
version3 = 10;
version4 = 20;
version5 = 30;
}
}
I want to have a C++ function GetNextVersion() which takes in one enum version and gives the next version as output. For eg: GetNextVersion(Information::version4) should give Information::version5 as output. Is there any inbuilt and easy method to do this?
You can use protobuf's reflection to achieve the goal:
Information::Versions GetNextVersion(Information::Versions ver) {
const auto *desc = Information::Versions_descriptor();
auto cur_idx = desc->FindValueByNumber(ver)->index();
if (cur_idx >= desc->value_count() - 1) {
throw runtime_error("no next enum");
}
auto next_idx = cur_idx + 1;
return Information::Versions(desc->value(next_idx)->number());
}
int main() {
try {
auto ver = Information::version1;
while (true) {
cout << ver << endl;
ver = GetNextVersion(ver);
}
} catch (const runtime_error &e) {
cout << e.what() << endl;
}
return 0;
}
Is there any inbuilt and easy method to do this?
I see no easy method to get that.
But I can suggest metaprogramming approaches (at least on Linux) with C++ code generation.
You could, assuming you have access to the source code of protobuf-c :
write some GNU gawk script to parse that C++ code and generate the C++ code of GetNextVersion
perhaps write some GNU sed (or a Python one) script doing the same.
write some GCC plugin and use it to parse that C++ code and generate the C++ code of GetNextVersion
write some GNU emacs code doing the same.
wait a few months and (in spring 2021) use Bismon. I am developing it, so contact me by email
extend and adapt the Clang static analyzer for your needs.
extend and adapt the SWIG tool for your needs.
extend and adapt the RPGGEN tool for your needs.
use GNU bison or ANTLR to parse C++ code, or design your domain specific language with some documented EBNF syntax and write some code generator with them.
You could also keep the description of enum Versions in some database (sqlite, PostGreSQL, etc...) or some JSON file or some CSV file (or an XML one, using XSLT or libexpat) and emit it (for protobuf) and the source code of GetNextVersion using some Python script, or GNU m4, or GPP.
You could write a GNU guile script or some rules for CLIPS generating some C++ code with your protobuf description.
In a few months (spring 2021), the RefPerSys system might be helpful. Before that, you could contribute and extend it and reuse it for your needs.
A pragmatic approach could be to add a comment in your protobuf declaration to remind you of editing another file when you need to change the protobuf message and protocol.
No, there isn't.
You define your own data type, so you also must define the operators for it.
So, your GetNextVersion()method contains that knowledge how to increment the version number. If you had decided to use an integer, then the compiler knows already how to increment that, but you wanted something special and that is the price you have to pay for it.
I have an IP camera that receives commands using POST HTTP requests(for example to call PTZ commands or set various camera settings). The standard way of controlling it is through it's own web interface which is partially an ActiveX plugin and partially standard html+js. Of course because of the ActiveX part it only works in IE under Windows.
I'm attempting to change that by figuring out all the commands and writing a small python or javascript code to do the same, so that it is more cross platform.
I have one major problem. Each POST request contains a calculated "cc" field which I assume is a checksum. The JS code in the cam interface points out that it is calculated by calling a function inside the plugin:
tt = new Date().Format("yyyyMMddhhmmss");
jo_header["tt"] = tt;
if (getCpPlugin() != null && getCpPlugin().valid) {
jo_header["cc"] = getCpPlugin().nsstpGetCC(tt, session_id);
}
nsstpGetCC function obviously calculates the checksum from two parameters the timestamp and session_id. Real example(captured with Wireshark):
tt = "20171018231918"
session_id = "30303532646561302D623434612D3131"
cc = "849e586524385e1071caa4023a3df75401e5bb82"
Checksum seems to be 160bit. I tried both sha-1 and ripemd-160 and all combinations of concatenating tt and session_id I could think of. But I can't seem to get the same hash as the one the original plugin gets. The plugin dll seems to be written in c++. And I have almost no experience with decompilation to dive into this problem from that angle.
So my question basically is can someone figure out how they calculated that cc, or at least give me an idea in which direction to research further. Maybe I'm looking at wrong hash algorithms or something... Or give me some idea how I could somehow figure out what the original ActiveX function nsstpGetCC is doing for example by decompilation or maybe by monitoring it's operation in memory while running. What tools should I use?
I'm new to LLVM. I am using the clang c++ API to compile multiple stub files (in c) to IR, and then stick them together using IR builder (after linking them) to eventually run via JIT.
All this works great, unless I add a functionInlining pass to my optimizations, at which point one of these function calls made in IR builder will trigger the following exception when the pass manager is run:
Assertion failed: (New->getType() == getType() && "replaceAllUses of value with new value of different type!"), function replaceAllUsesWith, file /Users/mike/Development/llvm/llvm/lib/IR/Value.cpp, line 356.
This is how I make the call instruction (pretty straight forward):
Function *kernelFunc = mModule->getFunction( (kernel->Name() + StringRef("_") + StringRef(funcName)).str());
if (kernelFunc){
CallInst* newInst = builder.CreateCall(kernelFunc, args);
}
Later the module is optimized:
legacy::PassManager passMan;
PassManagerBuilder Builder;
Builder.OptLevel = 3;
//Builder.Inliner = llvm::createFunctionInliningPass(); //commenting this back in trigger the exception
Builder.populateModulePassManager(passMan);
passMan.run( *mModule ); //exception occurs before this call returns
Any ideas what to look for?
Try running llvm::verifyModule on your module to see if it's correct. You might have an error and have been getting lucky beforehand but it tripped something up in the inliner.
In general assertions check a subset of things that can be wrong with your module but verify checks a lot more.
It could be a bug in LLVM but more than likely it's a bad module, it's easy to happen.
So I finally setup my dev environment so I could inspect the assertion call in the debugger. I turns out the basic block being replaced had a different context set than the one it was being replaced with. going back and making sure IRBuilder was using the same context as the IR parsers solved the problem.
I'm working on a new language using the LLVM C++ API and would like to take advantage of optimization passes. (Note: I'm currently using the latest from source LLVM which I believe equates to 3.8)
I have yet to find any examples that use the new PassManager and even Clang is still utilizing the LegacyPassManager.
I have come across posts such as this that are several years old now that mention the new PassManager, but they all still use the legacy system.
Is there any examples/tutorials on how to use this new(ish) PassManager? Should new LLVM projects prefer PassManager to LegacyPassManager? Does Clang plan on migrating or is this why the Legacy system has stuck around?
From what I've gathered with help from the #llvm IRC:
FunctionPassManager FPM;
//Use the PassInfoMixin types
FPM.addPass(InstCombinePass());
//Register any analysis passes that the transform passes might need
FunctionAnalysisManager FAM;
//Use the AnalysisInfoMixin types
FAM.registerPass([&] { return AssumptionAnalysis(); });
FAM.registerPass([&] { return DominatorTreeAnalysis(); });
FAM.registerPass([&] { return BasicAA(); });
FAM.registerPass([&] { return TargetLibraryAnalysis(); });
FPM.run(*myFunction, FAM);
But to avoid the hassle of manually registering each pass you can use PassBuilder to register the analysis passes
FunctionPassManager FPM;
FPM.addPass(InstCombinePass());
FunctionAnalysisManager FAM;
PassBuilder PB;
PB.registerFunctionAnalyses(FAM);
FPM.run(*myFunction, FAM);
Extending Lukes answer, with PassBuilder you can build predefined "out of box" simplification pipelines with different optimization levels:
llvm::FunctionAnalysisManager FAManager;
llvm::PassBuilder passBuilder;
passBuilder.registerFunctionAnalyses(FAManager);
passBuilder.buildFunctionSimplificationPipeline(
llvm::PassBuilder::OptimizationLevel::O2,
llvm::PassBuilder::ThinLTOPhase::None);
which will add a bunch of passes to FunctionAnalysisManager. This may simplify your life. The best place to see the full set of passes added for each OptimizationLevel is the original sources.