LLVM JIT Symbols not found - c++

Question

I am trying to write a program to JIT some code. The JITTed code needs to make calls back into the running application for run-time support and the run-time support symbols are not found when the function is materialized.
I tried to follow the Kaleidoscope tutorial. I need to call a function in the run-time from some IR generated code. For example, I want to call this function from some llvm IR.
extern "C" void* llvmNewVector() {
return new vector<int>();
}
According to the Kaleidoscope tutorial it should be declared extern "C" and in the run-time of the application. Within the LLVM IR I have created a function prototype and the IR is correctly generated (no errors after checking the function I am jitting).
It would seem to me that there would be something more to do to link this function to the jitted code, but the Kaleidoscope tutorial doesn't seem to do that.
My problem is that the jitted code fails to materialize because the external symbols are not resolved.
The following code prints "made it here" but gets no further.
cerr << "made it here." << endl;
auto Sym = ExitOnErr(TheJIT->lookup(name));
NativeCodePtr FP = (NativeCodePtr)Sym.getAddress();
assert(FP && "Failed to find function ");
cerr << "returning jitted function " << name << endl;
return FP;
I am sure I am doing something wrong or missing some step, but I have not been able to find it.
The output I get is:
made it here.
JIT session error: Symbols not found: { llvmNewVector }
Failed to materialize symbols: { my_test }
The code was compiled using LLVM-9 with the following flags:
clang++ -I. -g -I../include/ -std=c++11 -fexceptions -fvisibility=hidden -fno-rtti -D_DEBUG -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.cpp
For linking the following was used:
llvm-config --libs

Answer 1

I ran into this same issue and could solve it the following way:
The following lines of code in the tutorial, whose goal is to resolve symbols in the host process, does not seem to work.
ES.getMainJITDylib().setGenerator(
cantFail(DynamicLibrarySearchGenerator::GetForCurrentProcess(DL)));
So instead, I manually registered the symbols that I wanted linked like this:
SymbolMap M;
// Register every symbol that can be accessed from the JIT'ed code.
M[Mangle("llvmNewVector")] = JITEvaluatedSymbol(
pointerToJITTargetAddress(&llvmNewVector), JITSymbolFlags());
}
cantFail(ES.getMainJITDylib().define(absoluteSymbols(M)));
I added this code right after the two lines of code that I mentioned above, from the tutorial.

Answer 2

How about adding -Xlinker --export-dynamic option for clang?
I met the similar problem in the tutorial.
In my environment (Ubuntu 20.04), sin and cos can be resolved, but printd or putchard (the functions defined in the source code of Kaleidoscope processor) cannot.
After compilation, can you see the function name in the dynamic symbol table of the program?
objdump -T program | grep llvmNewVector
If there are no -T option in objdump (e.g., Mac), it might not be the case.
In my case, printd nor putchard do not appear in the dynamic symbol table (but appear in symbol table).
To add these function names into the dynamic symbol table, you need to pass -Xlinker --export-dynamic option for clang (actually, the option is passed to ld), for example (this is one for the tutorial),
clang++ -Xlinker --export-dynamic -g toy.cpp `llvm-config --ldflags --system-libs --libs all` -O3 -o toy
After compilation, the function names appear in dynamic symbol table, and the examples of the tutorial work well.

Answer 3

It depends on which llvm version you use. LLVM 10 has LLJIT class and it was working for me the following way
auto J = ExitOnErr(LLJITBuilder().create());
auto M = createDemoModule();
auto &dl = J->getDataLayout();
MangleAndInterner Mangle(J->getExecutionSession(), dl);
auto &jd = J->getMainJITDylib();
auto s = absoluteSymbols({{ Mangle("printd"), JITEvaluatedSymbol(pointerToJITTargetAddress(&printd), JITSymbolFlags::Exported)}});
jd.define(s);
the printd function was defined in the same file
extern "C" int32_t printd() {
std::cout << "calling " << __FUNCTION__ << "...\n";
return 11;
}