LLVM JIT-compiled program cannot find external functions - c++

My program which JIT compiles a LLVM IR module and calls a function foo defined therein fails at runtime if foo uses an externally-defined function:
LLVM ERROR: Program used external function 'glutInit' which could not be resolved!
My program:
// foo1.cpp
#include <GL/glut.h>
extern "C" void foo()
{
glutInit(0,0);
}
// foo2.cpp
#include <iostream>
#include <fstream>
#include <string>
#include <llvm/Support/raw_ostream.h>
#include <llvm/LLVMContext.h>
#include <llvm/Support/TargetSelect.h>
#include <llvm/Support/IRReader.h>
#include <llvm/ExecutionEngine/ExecutionEngine.h>
#include <llvm/Support/MemoryBuffer.h>
#include <llvm/ExecutionEngine/JIT.h>
#include <llvm/ExecutionEngine/RuntimeDyld.h>
int main(int argc, char **argv)
{
using namespace llvm;
InitializeNativeTarget();
LLVMContext context;
SMDiagnostic error;
std::ifstream ir_file("foo1.s");
std::string ir((std::istreambuf_iterator<char>(ir_file)),
(std::istreambuf_iterator<char>()));
Module *m = ParseIR(MemoryBuffer::getMemBuffer(StringRef(ir)), error, context);
if(!m)
{
error.print(argv[0], errs());
}
ExecutionEngine *ee = ExecutionEngine::create(m);
Function *func = ee->FindFunctionNamed("foo");
if(func == 0)
{
std::cerr << "Couldn't find Function foo" << std::endl;
std::exit(-1);
}
typedef void (*fcn_ptr)();
fcn_ptr foo = reinterpret_cast<fcn_ptr>(ee->getPointerToFunction(func));
foo();
delete ee;
return 0;
}
Here's how I build my program:
$ clang -S -emit-llvm foo1.cpp
$ g++ -rdynamic foo2.cpp `llvm-config --cxxflags` `llvm-config --libs` `llvm-config --ldflags` -lglut
The output:
$ ./a.out
LLVM ERROR: Program used external function 'glutInit' which could not be resolved!
It fails with a similar error any time I try to use an externally-defined function which is not in the C++ standard library (e.g., printf, malloc, & free are no problem). What am I doing wrong?

Make sure that glutInit was linked into a.out. If your host code (the thing executing the JIT) didn't call it, it could have been nixed by the linker. If that's the case, you have to add a dummy reference to it or use linker scripts / flags.

Adding the command line option -Wl,-no-as-needed immediately before -lglut will prevent the linker from dropping the glut library, which it otherwise thinks is not needed:
$ g++ -rdynamic foo2.cpp `llvm-config --cxxflags` `llvm-config --libs` `llvm-config --ldflags` -Wl,-no-as-needed -lglut

Related

Force alle functions in shared library to be defined

I want to write a shared library and I want to get a compiler/linker error if I forgot to implement some functions.
Consider the following case:
test.h
class Test {
public:
Test();
};
test.cpp
#include "test.h"
main.cpp
#include "test.h"
int main() {
new Test();
}
If I create a library with this command gcc -c -fpic test.cpp && g++ -shared -o libtest.so -Wl,--no-undefined -Wl,--no-allow-shlib-undefined test.o there is no error message, but the library is broken. Is there a way to force the creation of a not broken library?
Edit: adding additional flag, but doesn't change result
These codes have been modified:
test.h :
class Test {
public:
Test();
};
test.cpp :
#include "test.h"
Test::Test(){} // you must implement the constructor
You must have to implement the constructor, and if not, you get an error "undefined reference to `Test::Test()'".
main.cpp :
#include <iostream>
#include "test.h"
using namespace std;
int main(void)
{
Test* t = new Test(); // you must define a pointer
cout << "test* was created: " << t << endl;
delete t;
t = nullptr;
return 0;
}
Now all the code is OK. Then we create a shared-library with the following command:
g++ -shared -o test.so -fPIC test.cpp
Finally, we compile the main.cpp file at the same time as referring to the test.so shared-library and get the exe output, by the command below:
g++ -g main.cpp test.so -o test.exe

g++ link error: 'undefined reference to 'main'

I have 3 files; main.cpp (which contains main()), FileWriter.h, and FileWriter.cpp. I'm using g++ (version Debian 4.9.2-10) on Debian Jessie. My project contains .cpp files in '/root/dev/Practice/src/', and a single header (FileWriter.h) in '/root/dev/Practice/include/'. The compilation of the two object files works, but the linking to an executable complains about undefined reference to main(), although I do indeed have a seemingly valid one defined in 'main.cpp'.
Here's the output of my make file (which is in the root '/root/dev/Practice/' directory):
g++ -c -g -Wall -o src/FileWriter.o src/FileWriter.cpp
g++ -c -g -Wall -o src/main.o src/FileWriter.cpp
g++ src/FileWriter.o src/main.o -o bin/Practice
/usr/lib/gcc/i586-linux-gnu/4.9/../../../i386-linux-gnu/crt1.o: In function '_start'"
/build/glibc-J1NNmk/glibc-2.19/csu/../sysdeps/i386/start.S:111: undefined reference to 'main'
collect2: error: ls returned 1 exit status
Makefile:10: recipe for target 'bin/Practice' failed
Here's the contents of my main.cpp file:
#include <string>
#include <iostream>
#include "/root/dev/Practice/include/FileWriter.h"
int main() {
std::cout << "Hello!" << std::endl;
FileWriter * fw = new FileWriter("foofile");
fw->AddLine("CRAP!");
fw->AddLine("NO!");
return 0;
}
My FileWriter.h:
#ifndef FILEWRITER_H_
#define FILEWRITER_H_
#include <string>
#include <iostream>
class FileWriter{
public:
FileWriter(std::string);
~FileWriter();
void AddLine(std::string);
private:
std::string fileLocation;
std::ofstream *filestream;
};
#endif /* FILEWRITER_H_ */
...and my FileWriter.cpp:
#include "/root/dev/Practice/include/FileWriter.h"
#include <fstream>
// g++ linker error if 'inline' not included - why?
inline FileWriter::FileWriter(std::string fileName)
{
this->fileLocation = fileName;
const char * x = this->fileLocation.c_str();
this->filestream = new std::ofstream();
this->filestream->open(x, std::ios::out | std::ios::app);
}
inline FileWriter::~FileWriter()
{
this->filestream->close();
}
inline void FileWriter::AddLine(std::string line)
{
*this->filestream << line << std::endl;
}
This line:
g++ -c -g -Wall -o src/main.o src/FileWriter.cpp
should be:
g++ -c -g -Wall -o src/main.o src/main.cpp
I don't have access to this compiler, but in the past if you had main() in a C++ file you needed to "decorate" it with __cdecl
int __cdecl main() {
Try that? Or:
extern "C" int main() {

How to compile a cpp and then link it to a shared library

I want to have the functions which are defined in another .cpp file become available in another simulation tool.
I found the following code in this question: -finstrument-functions doesn't work with dynamically loaded g++ shared objects (.so)
Trace.cpp
#include <stdio.h>
#ifdef __cplusplus
extern "C"
{
void __cyg_profile_func_enter(void *this_fn, void *call_site)
__attribute__((no_instrument_function));
void __cyg_profile_func_exit(void *this_fn, void *call_site)
__attribute__((no_instrument_function));
}
#endif
void __cyg_profile_func_enter(void* this_fn, void* call_site)
{
printf("entering %p\n", (int*)this_fn);
}
void __cyg_profile_func_exit(void* this_fn, void* call_site)
{
printf("exiting %p\n", (int*)this_fn);
}
Trace.cpp is compiled by doing:
g++ -g -finstrument-functions -Wall -Wl,-soname,libMyLib.so.0 -shared -fPIC -rdynamic MyLib.cpp MyLibStub.cpp Trace.cpp -o libMyLib.so.0.0
ln -s libMyLib.so.0.0 libMyLib.so.0
ln -s libMyLib.so.0.0 libMyLib.so
g++ MainStatic.cpp -g -Wall -lMyLib -L./ -o MainStatic
g++ MainDynamic.cpp -g -Wall -ldl -o MainDynamic
Note that I don't need: MyLib.cpp and MyLibStub.cpp.
Instead compiled Trace.cpp doing:
g++ -g -finstrument-functions -Wall -Wl,-soname,libMyLib.so.0 -shared -fPIC -rdynamic Trace.cpp -o libMyLib.so.0.0
What I've tried:
The shared object where I want to have Trace.cpp is obtained by:
opp_makemake -f --deep --no-deep-includes --make-so -I . -o veins -O out -I../../inet/src/util/headerserializers/sctp/headers -L../../inet/src -linet
I added -L and -l:
opp_makemake -f --deep --no-deep-includes --make-so -I . -o veins -L /home/user/Desktop/test/ -lMyLib -O out -I../../inet/src/util/headerserializers/sctp/headers -L../../inet/src -linet
and got:
/usr/bin/ld: cannot find -lMyLib
I also tried:
opp_makemake -f --deep --no-deep-includes --make-so -I . -o veins /home/user/Desktop/test/libMyLib.so.0.0 -O out -I../../inet/src/util/headerserializers/sctp/headers -L../../inet/src -linet
which compiled successfully but the application crashed:
Error during startup: Cannot load library
'../../src//libveins.so': libMyLib.so.0: cannot open shared object
file: No such file or directory.
Question:
How to compile Trace.cpp correctly?
How to link it with the rest of the shared library?
As you might notice I am not very experienced with compiling, linking and similar. So, any extra explanation is very welcome!
As #Flexo restates what #EmployedRussian said in the linked question, the main point is to get your implementation of __cyg_profile_func_*** before the one provided by libc.so.6.
One method to do this, is to use the LD_PRELOAD environment variable. Here you can read what LD_PRELOAD does and how it works.
To use the LD_PRELOAD trick you will need to compile your implementation of the above-mentioned functions as a shared library.
You can do this by doing:
g++ -shared -fPIC myImp.cc -o myImp.so -ldl
Once you get the .so file, navigate to the directory where your executable is located and do:
LD_PRELOAD=<path/to/myImp.so>- ./<myExecutable>
For shared libraries, dynamic linking is used. Meaning:
resolving of some undefined symbols (is postponed) until a program is run.
By using LD_PRELOAD you resolve the symbols of your interest before letting the linked do that.
Here you have an implementation of myImp.cc, which I took from: https://groups.google.com/forum/#!topic/gnu.gcc.help/a-hvguqe10I
The current version lacks proper implementation for __cyg_profile_func_exit, and I have not been able to demangle the function names.
#ifdef __cplusplus
extern "C"
{
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <dlfcn.h>
void __cyg_profile_func_enter(void *this_fn, void *call_site)__attribute__((no_instrument_function));
void __cyg_profile_func_exit(void *this_fn, void *call_site)__attribute__((no_instrument_function));
}
#endif
static FILE *fp;
int call_level=0;
void * last_fn;
void __cyg_profile_func_enter(void *this_fn, void *call_site)
{
Dl_info di;
if (fp == NULL) fp = fopen( "trace.txt", "w" );
if (fp == NULL) exit(-1);
if ( this_fn!=last_fn) ++call_level;
for (int i=0;i<=call_level;i++)
{
fprintf(fp,"\t");
}
//fprintf(fp, "entering %p\n", (int *)this_fn);
fprintf(fp, "entering %p", (int *)this_fn);
if (dladdr(this_fn, &di)) {
fprintf(fp, " %s (%s)", di.dli_sname ? di.dli_sname : "<unknown>", di.dli_fname);
}
fputs("\n", fp);
(void)call_site;
last_fn = this_fn;
}
void __cyg_profile_func_exit(void *this_fn, void *call_site)
{
--call_level;
for (int i=0;i<=call_level;i++) fprintf(fp,"\t");
fprintf(fp, "exiting %p\n", (int *)this_fn);
(void)call_site;
}
Another option for function tracing which uses LD_PRELOAD is used by LTTng, in the section Function Tracing, but I have never used it...

How to pass arguments to a method loaded from a static library in CPP

I'm trying to write a program to use a static library of a C++ code into another C++ code. The first C++ code is hello.cpp:
#include <iostream>
#include <string.h>
using namespace std;
extern "C" void say_hello(const char* name) {
cout << "Hello " << name << "!\n";
}
int main(){
return 0;
}
The I made a static library from this code, hello.a, using this command:
g++ -o hello.a -static -fPIC hello.cpp -ldl
Here's the second C++ code to use the library, say_hello.cpp:
#include <iostream>
#include <string>
#include <dlfcn.h>
using namespace std;
int main(){
void* handle = dlopen("./hello.a", RTLD_LAZY);
cout<<handle<<"\n";
if (!handle) {
cerr<<"Cannot open library: "<<dlerror()<<'\n';
return 1;
}
typedef void (*hello_t)();
dlerror(); // reset errors
hello_t say_hello = (hello_t) dlsym(handle, "say_hello");
const char *dlsym_error = dlerror();
if (dlsym_error) {
cerr<<"Cannot load symbol 'say_hello': "<<dlsym_error<<'\n';
dlclose(handle);
return 1;
}
say_hello("World");
dlclose(handle);
return 0;
}
Then I compiled say_hello.cpp using:
g++ -W -ldl say_hello.cpp -o say_hello
and ran ./say_hello in the command line. I expected to get Hello World! as output, but I got this instead:
0x8ea4020
Hello ▒▒▒▒!
What is the problem? Is there any trick to make compatibility for method's argument like what we use in ctypes or what?
If it helps I use a lenny.
EDIT 1:
I have changed the code and used a dynamic library, 'hello.so', which I've created using this command:
g++ -o hello.so -shared -fPIC hello.cpp -ldl
The 6th line of the code changed to:
void* handle = dlopen("./hello.so", RTLD_LAZY);
When I tried to compile say_hello.cpp, I got this error:
say_hello.cpp: In function ‘int main()’:
say_hello.cpp:21: error: too many arguments to function
I also tried to compile it using this line:
g++ -Wall -rdynamic say_hello.cpp -ldl -o say_hello
But same error raised. So I removed the argument "World" and the it has been compiled with no error; but when I run the executable, I get the same output like I have mentioned before.
EDIT 2:
Based on #Basile Starynkevitch 's suggestions, I changed my say_hello.cpp code to this:
#include <iostream>
#include <string>
#include <dlfcn.h>
using namespace std;
int main(){
void* handle = dlopen("./hello.so", RTLD_LAZY);
cout<<handle<<"\n";
if (!handle) {
cerr<<"Cannot open library: "<<dlerror()<<'\n';
return 1;
}
typedef void hello_sig(const char *);
void* hello_ad = dlsym(handle, "say_hello");
if (!hello_ad){
cerr<<"dlsym failed:"<<dlerror()<<endl;
return 1;
}
hello_sig* fun = reinterpret_cast<hello_sig*>(hello_ad);
fun("from main");
fun = NULL;
hello_ad = NULL;
dlclose(handle);
return 0;
}
Before that, I used below line to make a .so file:
g++ -Wall -fPIC -g -shared hello.cpp -o hello.so
Then I compiled say_hello.cpp wth this command:
g++ -Wall -rdynamic -g say_hello.cc -ldl -o say_hello
And then ran it using ./say_hello. Now everything is going right. Thanks to #Basile Starynkevitch for being patient about my problem.
Functions never have null addresses, so dlsym on a function name (or actually on any name defined in C++ or C) cannot be NULL without failing:
hello_t say_hello = (hello_t) dlsym(handle, "say_hello");
if (!say_hello) {
cerr<<"Cannot load symbol 'say_hello': "<<dlerror()<<endl;
exit(EXIT_FAILURE);
};
And dlopen(3) is documented to dynamically load only dynamic libraries (not static ones!). This implies shared objects (*.so) in ELF format. Read Drepper's paper How To Use Shared Libraries
I believe you might have found a bug in dlopen (see also its POSIX dlopen specification); it should fail for a static library hello.a; it is always used on position independent shared libraries (like hello.so).
You should dlopen only position independent code shared objects compiled with
g++ -Wall -O -shared -fPIC hello.cpp -o hello.so
or if you have several C++ source files:
g++ -Wall -O -fPIC src1.cc -c -o src1.pic.o
g++ -Wall -O -fPIC src2.cc -c -o src2.pic.o
g++ -shared src1.pic.o src2.pic.o -o yourdynlib.so
you could remove the -O optimization flag or add -g for debugging or replace it with -O2 if you want.
and this works extremely well: my MELT project (a domain specific language to extend GCC) is using this a lot (generating C++ code, forking a compilation like above on the fly, then dlopen-ing the resulting shared object). And my manydl.c example demonstrates that you can dlopen a big lot of (different) shared objects on Linux (typically millions, and hundred of thousands at least). Actually the limitation is the address space.
BTW, you should not dlopen something having a main function, since main is by definition defined in the main program calling (perhaps indirectly) dlopen.
Also, order of arguments to g++ matters a lot; you should compile the main program with
g++ -Wall -rdynamic say_hello.cpp -ldl -o say_hello
The -rdynamic flag is required to let the loaded plugin (hello.so) call functions from inside your say_hello program.
For debugging purposes always pass -Wall -g to g++ above.
BTW, you could in principle dlopen a shared object which don't have PIC (i.e. was not compiled with -fPIC); but it is much better to dlopen some PIC shared object.
Read also the Program Library HowTo and the C++ dlopen mini-howto (because of name mangling).
example
File helloshared.cc (my tiny plugin source code in C++) is
#include <iostream>
#include <string.h>
using namespace std;
extern "C" void say_hello(const char* name) {
cout << __FILE__ << ":" << __LINE__ << " hello "
<< name << "!" << endl;
}
and I am compiling it with:
g++ -Wall -fPIC -g -shared helloshared.cc -o hello.so
The main program is in file mainhello.cc :
#include <iostream>
#include <string>
#include <dlfcn.h>
#include <stdlib.h>
using namespace std;
int main() {
cout << __FILE__ << ":" << __LINE__ << " starting." << endl;
void* handle = dlopen("./hello.so", RTLD_LAZY);
if (!handle) {
cerr << "dlopen failed:" << dlerror() << endl;
exit(EXIT_FAILURE);
};
// signature of loaded function
typedef void hello_sig_t(const char*);
void* hello_ad = dlsym(handle,"say_hello");
if (!hello_ad) {
cerr << "dlsym failed:" << dlerror() << endl;
exit(EXIT_FAILURE);
}
hello_sig_t* fun = reinterpret_cast<hello_sig_t*>(hello_ad);
fun("from main");
fun = NULL; hello_ad = NULL;
dlclose(handle);
cout << __FILE__ << ":" << __LINE__ << " ended." << endl;
return 0;
}
which I compile with
g++ -Wall -rdynamic -g mainhello.cc -ldl -o mainhello
Then I am running ./mainhello with the expected output:
mainhello.cc:7 starting.
helloshared.cc:5 hello from main!
mainhello.cc:24 ended.
Please notice that the signature hello_sig_t in mainhello.cc should be compatible (homomorphic, i.e. the same as) with the function say_hello of the helloshared.cc plugin, otherwise it is undefined behavior (and you probably would have a SIGSEGV crash).

C++ Undefined Reference (Even with Include)

I cannot get this simple piece of code to compile without including the TestClass.cpp file explicitly in my main.cpp file. What am I doing wrong? Thanks in advance!
Here is the code:
TestClass.h
#ifndef TESTCLASS_H_
#define TESTCLASS_H_
class TestClass
{
public:
static int foo();
};
#endif
TestClass.cpp
#include "TestClass.h"
int TestClass::foo() { return 42; }
main.cpp
#include <iostream>
#include "TestClass.h"
using namespace std;
int main()
{
cout << TestClass::foo() << endl;
return 0;
}
Here is the error:
g++ main.cpp -o main.app
/tmp/ccCjOhpy.o: In function `main':
main.cpp:(.text+0x18e): undefined reference to `TestClass::foo()'
collect2: ld returned 1 exit status
Include TestClass.cpp into the commandline, so the linker can find the function definition:
g++ main.cpp TestClass.cpp -o main.app
Alternatively, compile each to their own object file, then tell the compiler to link them together (it will forward them to the linker)
g++ -c main.cpp -o main.o
g++ -c TestClass.cpp -o TestClass.o
g++ main.o TestClass.o -o main.app
You're not compiling and linking against TestClass.cpp (where the implementation of foo() is). The compiler is thus complaining that your trying to use an undefined function.