Workflow of LLVM and clang - llvm

I am just a beginner in LLVM, and (https://www.cs.cornell.edu/~asampson/blog/llvm.html) webpage along with the stack overflow, and my fellow researcher has helped me a lot.
I would first like to illustrate what I am trying to work on (the problem) and then I will describe the approach that I have taken to work on the problem.
Then, I need your advice and guidance if I am missing anything.
Work Problem
My input is a C program and output is its SSA form in prefix representation printed in an output File.
For eg, if the C code segment is :
x=4;
x++;
z=x+7;
The output SSA form in prefix representation is :
( = x0 4)
( = x1 (+ x0 1) )
( = z (x1 + 7) )
Please ignore the actual IR instruction for now, just assume that I am able to read the IR and convert it to this form, with some extra statements (which I am not presenting here for readability).
My ignorant Approach of using LLVM (Please find the complete program below)
using namespace llvm;
namespace {
struct TestPass: public ModulePass {
IRssa::ptr ir_ssa = IRssa::ptr(new IRssa());
static char ID;
typedef std::list<std::pair<std::string, std::list<Instruction *> > > funcDump;
TestPass() : ModulePass(ID) { }
std::map<std::string, funcDump> workingList;
bool runOnModule(Module &M) {
std::string funcName, bkName;
for (Function &F : M) { //Found a new Function
if (isa<Function>(F) && !(F.getName().startswith("llvm."))) {
funcName = F.getName();
std::pair<std::string, std::list<Instruction *> > funcBlockList;
std::list<std::pair<std::string, std::list<Instruction *> > > wholeFuncBlocks;
for (BasicBlock &B : F) { //Blocks of the Function
if (isa<BasicBlock>(B)) {
bkName = B.getName();
}
std::list<Instruction *> listInst;
for (auto &I : B) {
Instruction *ins;
ins = &I;
listInst.push_back(ins);
}
funcBlockList.first = bkName;
funcBlockList.second = listInst;
wholeFuncBlocks.push_back(funcBlockList);
}
workingList[funcName] = wholeFuncBlocks;//Mapping of the functions
}
}
ir_ssa->setFunctionDump(workingList);
funcDump funcData;
funcData = workingList["start_program"]; //Starting from the start_program function
convertFunctionToSSA(funcData, ir_ssa);
std::ofstream outFile;
outFile.open("Out.ssa");
printSSA_toFile(outFile, ir_ssa);
return false;
}
};
}
char TestPass::ID = 0;
static RegisterPass<TestPass> X("testPass", "Testing A Pass");
static void registerTestPass(const PassManagerBuilder &, legacy::PassManagerBase &PM) {
PM.add(new TestPass());
}
static RegisterStandardPasses RegisterMyPass(PassManagerBuilder::EP_ModuleOptimizerEarly, registerTestPass);
static RegisterStandardPasses RegisterMyPass0(PassManagerBuilder::EP_EnabledOnOptLevel0, registerTestPass);
//Automatically enable the pass (http://adriansampson.net/blog/clangpass.html)
Description:
As shown above I am calling a runOnModule() and collecting all the IR Instructions of all the blocks for each function in the program into a workingList data structure (a std::map in this case). After all the functions in the given program is finished reading, I then do my required task of reading IR instructions one at a time, function by function and block by block (in the user defined function convertFunctionToSSA(funcData, ir_ssa) taking the whole function IR as argument and the result of processing these IR is returned in the argument ir_ssa). I am also printing the resulted value from ir_ssa onto the output file outFile.
Now How do I Run (I type the following)
clang -O1 -g -Xclang -emit-llvm -c someProgram.c -o test.bc
opt -O1 -instnamer -mem2reg -simplifycfg -loops -lcssa -loop-simplify -loop-rotate -loop-unroll -unroll-count=15 -unroll-allow-partial -load src/libTestPass.so -testPass test.bc -o test
My Expectation
I assume (as per my understanding) that the above two commands does the following.
First clang takes the program someProgram.c and generates IR as an output file "test.bc".
The next command opt, takes the file "test.bc" and then applies all the above passes one by one till the last pass "-unroll-allow-partial" and it also links my library libTestPass.so (this .so file is generated on compiling the above ModulePass program) then, finally the pass "-testPass" which I think is the pass where I am doing my process (of converting to SSA prefix representation).
Your Advice and Comments
I am not sure if LLVM is actually running in the sequence as I am assuming (My Expectation). Kindly comment if I am missing anything or if my assumption is not correct. Also please feel free to ask more details if necessary.
Current Problem Faced
I am able to successfully convert most of the C programs but on a specific program I stuck with some error. Debugging this error lead me to think that I am missing somethink or my assumption about this working of LLVM in regards to the calling order of clang and opt is not correct.
Your help is highly appreciated.

Related

Get filename and location from Function

I have an LLVM pass that iterates over LLVM IR code and I would like to get a directory and a filename for both the functions and basic blocks for the original code. I know that when I have an Instruction Pointer, I can easily get the information using the code below: Thanks to #hailinzeng
(How to get the filename and directory from a LLVM Instruction?)
const llvm::DebugLoc &location = i_iter->getDebugLoc();
if (location && debugLocationInfoOn) {
std::string dbgInfo;
llvm::raw_string_ostream rso(dbgInfo);
location.print(rso);
std::cout << rso.str();
}
However, since the class Function and BasicBlock do not have a member function getDebugLoc(), this doesn't work. I saw another post here using the metadata but I do not know how to get to the DILocation or DIScope from the metadata. Using
MDNode *n = inst->getMetadata("dbg");
DILocation loc(n); `
Gives the error below
/usr/lib/llvm-3.9/include/llvm/IR/Metadata.def:83:42: note: forward declaration of 'llvm::DILocation'
HANDLE_SPECIALIZED_MDNODE_LEAF_UNIQUABLE(DILocation)
I'm using llvm 3.9.
UPDATE ::
Thanks Stanislav Pankevich. I wasn't including the right headers but now I have a new issue. DILocation requires LLVMContext, StorageType, and unsigned Line. How do I get the line number and and storage type from a function pointer?
DILocation(LLVMContext &C, StorageType Storage, unsigned Line,
For those working with a similar issue, you can get LLVMContext using
llvm::MDNode * testmd = F.getMetadata("dbg");
F.getContext ()
If you look at the .ll file for your code, you would see that every function has DINode associated with it, something like !<some_number>. That's the metadata node number that is has info about that function. The type of that node is DISubprogram You can access it like this:
SmallVector<std::pair<unsigned, MDNode *>, 4> MDs;
F.getAllMetadata(MDs);
for (auto &MD : MDs) {
if (MDNode *N = MD.second) {
if (auto *subProgram = dyn_cast<DISubprogram>(N)) {
errs() << subProgram->getLine();
}
}
}
You can use all the information that is there in the debug node.
How about if we want column details, which is not possible with DISubprogram.
I tried this:
DILocation *debugLocation = dyn_cast<DILocation>(N);
debugLocation->getLine();
The sample.ll file does contain these lines:
!10 = !DILocation(line: 1, column: 1, scope: !1)
However, it gives core dumped at run time. Any suggestions please how to get it working.

Getting a function name (__func__) from a class T and a pointer to member function void(T::*pmf)()

Is it possible to write some f() template function that takes a type T and a pointer to member function of signature void(T::*pmf)() as (template and/or function) arguments and returns a const char* that points to the member function's __func__ variable (or to the mangled function name)?
EDIT: I am asked to explain my use-case. I am trying to write a unit-test library (I know there is a Boost Test library for this purpose). And my aim is not to use any macros at all:
struct my_test_case : public unit_test::test {
void some_test()
{
assert_test(false, "test failed.");
}
};
My test suite runner will call my_test_case::some_test() and if its assertion fails, I want it log:
ASSERTION FAILED (&my_test_case::some_test()): test failed.
I can use <typeinfo> to get the name of the class but the pointer-to-member-function is just an offset, which gives no clue to the user about the test function being called.
It seems like what you are trying to achieve, is to get the name of the calling function in assert_test(). With gcc you can use
backtace to do that. Here is a naive example:
#include <iostream>
#include <execinfo.h>
#include <cxxabi.h>
namespace unit_test
{
struct test {};
}
std::string get_my_caller()
{
std::string caller("???");
void *bt[3]; // backtrace
char **bts; // backtrace symbols
size_t size = sizeof(bt)/sizeof(*bt);
int ret = -4;
/* get backtrace symbols */
size = backtrace(bt, size);
bts = backtrace_symbols(bt, size);
if (size >= 3) {
caller = bts[2];
/* demangle function name*/
char *name;
size_t pos = caller.find('(') + 1;
size_t len = caller.find('+') - pos;
name = abi::__cxa_demangle(caller.substr(pos, len).c_str(), NULL, NULL, &ret);
if (ret == 0)
caller = name;
free(name);
}
free(bts);
return caller;
}
void assert_test(bool expression, const std::string& message)
{
if (!expression)
std::cout << "ASSERTION FAILED " << get_my_caller() << ": " << message << std::endl;
}
struct my_test_case : public unit_test::test
{
void some_test()
{
assert_test(false, "test failed.");
}
};
int main()
{
my_test_case tc;
tc.some_test();
return 0;
}
Compiled with:
g++ -std=c++11 -rdynamic main.cpp -o main
Output:
ASSERTION FAILED my_test_case::some_test(): test failed.
Note: This is a gcc (linux, ...) solution, which might be difficult to port to other platforms!
TL;DR: It is not possible to do this in a reasonably portable way, other than using macros. Using debug symbols is really a hard solution, which will introduce a maintenance and architecture problem in the future, and a bad solution.
The names of functions, in any form, is not guaranteed to be stored in the binary [or anywhere else for that matter]. Static free functions certainly won't have to expose their name to the rest of the world, and there is no real need for virtual member functions to have their names exposed either (except when the vtable is formed in A.c and the member function is in B.c).
It is also entirely permissible for the linker to remove ALL names of functions and variables. Names MAY be used by shared libraries to find functions not present in the binary, but the "ordinal" way can avoid that too, if the system is using that method.
I can't see any other solution than making assert_test a macro - and this is actually a GOOD use-case of macros. [Well, you could of course pass __func__ as a an argument, but that's certainly NOT better than using macros in this limited case].
Something like:
#define assert_test(x, y) do_assert_test(x, y, __func__)
and then implment do_assert_test to do what your original assert_test would do [less the impossible bit of figuring out the name of the function].
If it's unit tests, and you can be sure that you will always do this with debug symbols, you could solve it in a very non-portable way by building with debug symbols and then using the debug interface to find the name of the function you are currently in. The reason I say it's non-portable is that the debug API for a given OS is not standard - Windows does it one way, Linux another, and I'm not sure how it works in MacOS - and to make matters worse, my quick search on the subject seems to indicate that reading debug symbols doesn't have an API as such - there is a debug API that allows you to inspect the current process and figure out where you are, what the registers contain, etc, but not to find out what the name of the function is. So that's definitely a harder solution than "convince whoever needs to be convinced that this is a valid use of a macro".

LLVM Optimization Using C++ API

I'm trying to figure out how to perform all optimizations on an LLVM Module (e.g., all -O3 optimizations). I've tried the following but I'm not sure that all possible optimizations are being applied (e.g., inlining).
//take string "llvm" (LLVM IR) and return "output_llvm" (optimized LLVM IR)
static string optimize(string llvm) {
LLVMContext &ctx = getGlobalContext();
SMDiagnostic err;
Module *ir = ParseIR(MemoryBuffer::getMemBuffer(llvm), err, ctx);
PassManager *pm = new PassManager();
PassManagerBuilder builder;
builder.OptLevel = 3;
builder.populateModulePassManager(*pm);
pm->run(*ir);
delete pm;
string output_llvm;
raw_string_ostream buff(output_llvm);
ir->print(buff, NULL);
return output_llvm;
}
Is there anything else I can do to improve the performance of the output LLVM IR?
EDIT: I have tried to add all of the optimizations from the AddOptimizationPasses() function in opt.cpp, as shown below:
PassManager *pm = new PassManager();
int optLevel = 3;
int sizeLevel = 0;
PassManagerBuilder builder;
builder.OptLevel = optLevel;
builder.SizeLevel = sizeLevel;
builder.Inliner = createFunctionInliningPass(optLevel, sizeLevel);
builder.DisableUnitAtATime = false;
builder.DisableUnrollLoops = false;
builder.LoopVectorize = true;
builder.SLPVectorize = true;
builder.populateModulePassManager(*pm);
pm->run(*module);
Also, I create a FunctionPassManager before I create the PassManager and add several passes like so:
FunctionPassManager *fpm = new FunctionPassManager(module);
// add several passes
fpm->doInitialization();
for (Function &f : *ir)
fpm->run(f);
fpm->doFinalization();
However, the performance is the same as running on the command line with -O1 whereas I can get much better performance on the command line using -O3. Any suggestions?
Follow the logic in the function AddOptimizationPasses in opt.cpp. This is the source of truth.
While looking into LLVM optimization I found this information on pass ordering and I think it's potentially telling for why someone might encounter this situation.
Depending on your language and the optimizations you're expecting, you may need to specifically tune your optimizing passes to your use-cases. In particular, the ordering of those passes may be important. For example, if your better -O3 code was optimizing completely un-optimized code or code that was already partially optimized by your program, it may just be that you need to re-order or duplicate some passes in order to the expected final result.
Given the specific wording here and the fact that Eli's answer was accepted I'm not 100% sure if this is what the OP was seeing but this knowledge may be helpful for others with similar questions who find this answer like I did.

Dynamic Cast C++ Fail

I am having a dynamic cast fail on a g++ compiler (Redhat 5.5 gcc version 3.4.6) that works fine on a Windows Visual studio 2003, 2005, and 2010 compiler. To understand what I am seeing I will try to quickly break the problem down. We have a process that loads in numerous "plugins" from a directory and dynamically loads these plugins (which are dynamically linked libraries). The process is supposed to compare different rules and data types to return an answer. To make the process modulure we made the process understand about a "BaseDataType" but not about the actual specific types (so we could keep the process generic). The flow of the program goes something like this:
All our "SpecifcObject" types inherit from "BaseDataType" like this
class SpecificObject : public virtual BaseDataType {
... Class Items ...
}
This is the code from the process looks like:
// Receive raw data
void receive_data(void *buff, int size,DataTypeEnum type)
{
// Get the plugin associated with this data
ProcessPlugin *plugin = m_plugins[type];
// Since we need to cast the data properly into its actual type and not its
// base data type we need the plugin to cast it for us (so we can keep the
// process generic)
BaseDataType *data = plugin->getDataObject(buff);
if(data)
{
// Cast worked just fine
.... Other things happen (but object isn't modified) ....
// Now compare our rule
RuleObject obj = getRule();
ResultObject *result = plugin->CompareData(obj,data);
if(result)
... Success Case ...
else
... Error Case ...
}
}
Now this is (generically) what a plugin would look like
BaseDataType* ProcessPluginOne::getDataObject(unsigned char *buff)
{
// SpecificObject inherits from BaseDataType using a "virtual" inheritance
SpecificObject *obj = reinterpret_cast<SpecificObject*>(buff);
if(obj)
return (BaseDataType*)obj;
else
return NULL;
}
ResultObject* ProcessPluginOne::CompareData(RuleObject rule, BaseDataType *data)
{
ResultObject *obj = NULL;
// This method checks out fine
if(data->GetSomeBaseMethod())
{
// This cast below FAILS every time in gcc but passes in Visual Studio
SpecificObject *obj = dynamic_cast<SpecificObject*>(data);
if(obj)
{
... Do Something ...
}
}
return result;
}
Again all of this works under Visual Studio but not under GCC. To debug the program I started adding some code to different sections. I finally got it to work once I did the following in the main process (see added code below):
// In process with Modification
void receive_data(void *buff, int size,DataTypeEnum type)
{
// Get the plugin associated with this data
ProcessPlugin *plugin = m_plugins[type];
// Since we need to cast the data properly into its actual type and not its
// base data type we need the plugin to cast it for us (so we can keep the
// process generic)
BaseDataType *data = plugin->getDataObject(buff);
if(data)
{
// Cast worked just fine
.... Other things happen (but object isn't modified) ....
// Now compare our rule
RuleObject obj = getRule();
/** I included the specific data types in as headers for debugging and linked in
* all the specific classes and added the following code
*/
SpecificObject *test_obj = dynamic_cast<SpecificObject*>(data);
if(test_obj)
cout << "Our was Data was casted correctly!" << endl;
/// THE CODE ABOVE FIXES THE BAD CAST IN MY PLUGIN EVEN THOUGH
/// THE CODE ABOVE IS ALL I DO
ResultObject *result = plugin->CompareData(obj,data);
if(result)
... Success Case ...
else
... Error Case ...
}
}
Significant Process Compile Options:
Compile: -m64 -fPIC -wno-non-template-friend -DNDEGBUG -I <Includes>
Link: -Wl -z muldefs -m64
Significant Plugin Compile Options
Compile: -Wall -wno-non-template-friend -O -O2
Link: -Wl -Bstatic -Bdynamic -z muldefs -shared -m64
Since I am not modifying the object "data" I have no idea why the rest of the program would suddenly start working. The only thing I can think of is that the virtual table is getting stripped off somewhere in the process and the "extra" dynamic cast forces the main process to keep the table (which still doesn't make a lot of sense).
I have tried taking out all optimization settings in gcc and its still the same. Any thoughts as to what is going on here?
Thanks.
The two most likely scenarios are that BaseDataType is not a polymorphic type, or that the compiler doesn't see the relationship between BaseDataType and SpecificObject at some point in the code (for example, in getDataObject the compiler might generate different code depending on knowledge of the parent-child relationship, since you use C-cast from child to parent. This is super easy to check: Change the C-cast to static_cast. If it fails to compile you're missing a critical include.

GCC plugin with method was declared inside class

I am developing an GCC plugin to process AST in SSA form.
I create a callback run everytime after SSA form of function was compiled.
Here is my code
char* get_name_node(tree node) {
// return string represent node name
}
void execute_plugin_pass() {
printf("%s\n", get_name_node(cfun->decl));
}
struct opt_pass plugin_pass =
{
GIMPLE_PASS,
"plugin_pass",
0,
execute_plugin_pass,
NULL,
NULL,
0,
TV_PLUGIN_RUN,
PROP_gimple_any,
0,
0,
0,
0
};
extern "C" int
plugin_init(plugin_name_args* info, plugin_gcc_version* ver)
{
struct register_pass_info pass_info;
pass_info.reference_pass_name = where;
pass_info.pass = pass;
pass_info.ref_pass_instance_number = 0;
pass_info.pos_op = PASS_POS_INSERT_AFTER;
register_callback("plugin", PLUGIN_PASS_MANAGER_SETUP, NULL, &pass_info);
return 0;
}
But above code not run for method of class which was declare inside class declaration
For example, with this code
class A {
void method1();
void method2() {
// run some code here
}
};
void A::method1() {
// run some code here
}
My plugin only run for method1, but dont run for method2
At begining, i think this problems is because method2() will be considered as inline function, so i add option -fno-inline when run plugin. But it doesn't work
Can anyone help me?
Probably, the issue is where to insert your pass in the executed passes. Remember that the set and order of executed GCC passes vary with the optimization (so is different with -O0, -O1, -O2, ...). I don't understand where you are inserting it.
I also sometimes have the same issue of choosing where should I insert my pass. I often have a trial and error approach. Look into file gcc/passes.c of the GCC source tree.
If you need the SSA form of Gimple, consider inserting after the "ssa" pass or maybe after "phiopt".
If you don't need SSA, consider inserting after "cfg".
Out of curiosity, could you explain what your GCC plugin is doing?
Did you consider using GCC MELT (MELT is a high level domain specific language to code GCC extensions) for your work?