I am trying to compile a C++ library to wasm32 wasi for use inside my rust application. However I am running into this issue.
Error: failed to run main module `target/wasm32-wasi/release/so.wasm`
Caused by:
0: failed to instantiate "target/wasm32-wasi/release/so.wasm"
1: unknown import: `env::_ZdlPv` has not been defined
This error relates to line 5 in my mylib.cpp. Which just declares a variable
#include "OpenXLSX/OpenXLSX.hpp"
extern "C" int test() {
std::string i = "";
OpenXLSX::XLXmlData s;
return 0;
}
my main.rs
#[link(name = "mylib")]
extern "C" {
pub fn test() -> i32;
}
pub fn main() {
let res = unsafe { test() };
println!("test code: {}", res);
}
and my build.rs, where I assume the error is
use std::env;
fn main() {
cc::Build::new()
.cpp_link_stdlib(None)
.cpp(true)
.flag("-std=c++17")
.archiver("llvm-ar")
.include("OpenXLSX/external/pugixml")
.include("OpenXLSX/external/nowide")
.include("OpenXLSX/external/zippy")
.include("OpenXLSX/headers")
.include("OpenXLSX")
.flag("--sysroot=/opt/wasi-sysroot")
.flag("-fvisibility=default")
.file("OpenXLSX/sources/XLCell.cpp")
.file("OpenXLSX/sources/XLCellIterator.cpp")
.file("OpenXLSX/sources/XLCellRange.cpp")
.file("OpenXLSX/sources/XLCellReference.cpp")
.file("OpenXLSX/sources/XLCellValue.cpp")
.file("OpenXLSX/sources/XLColor.cpp")
.file("OpenXLSX/sources/XLColumn.cpp")
.file("OpenXLSX/sources/XLContentTypes.cpp")
.file("OpenXLSX/sources/XLDateTime.cpp")
.file("OpenXLSX/sources/XLDocument.cpp")
.file("OpenXLSX/sources/XLFormula.cpp")
.file("OpenXLSX/sources/XLProperties.cpp")
.file("OpenXLSX/sources/XLRelationships.cpp")
.file("OpenXLSX/sources/XLRow.cpp")
.file("OpenXLSX/sources/XLRowData.cpp")
.file("OpenXLSX/sources/XLSharedStrings.cpp")
.file("OpenXLSX/sources/XLSheet.cpp")
.file("OpenXLSX/sources/XLWorkbook.cpp")
.file("OpenXLSX/sources/XLXmlData.cpp")
.file("OpenXLSX/sources/XLXmlFile.cpp")
.file("OpenXLSX/sources/XLZipArchive.cpp")
.file("OpenXLSX/external/pugixml/pugixml.cpp")
.compile("OpenXLSX");
cc::Build::new()
.archiver("llvm-ar")
.cpp_link_stdlib(None)
.cpp(true)
.flag("-fvisibility=default")
.flag("-std=c++17")
.include("OpenXLSX/external/pugixml")
.include("OpenXLSX/headers")
.include("OpenXLSX")
.flag("--sysroot=/opt/wasi-sysroot")
.file("mylib.cpp")
.compile("libmylib.a");
}
In a separate attempt, Instead of step 1 in my build.rs where I try to link the OpenXLSX files, I have also used cmake to generate a single.a file compiled for wasm32-wasi with wasi sysroot, that I tried loading.
let src_dir = env::var("CARGO_MANIFEST_DIR").unwrap();
println!("cargo:rustc-link-lib=static=OpenXLSX");
println!("cargo:rustc-link-search=native={}/wasm-libs", src_dir);
And I tried to generate the bindings with Bindgen, it gave me duplicate definition errors. Which is the main reason I am writing my code in a c++ function and trying to call that from rust. I assume there would still be a linking issue,even if I get bindgen to work.
In short, your c++ code is using operator delete(void*) but you are not linking the c++ standard library that usually provides the implementation for it.
In a bit more detail:
_ZdlPv is the mangled name of operator delete(void*). That's the standard C++ delete operator that's used to free objects allocated on the heap with new. Check this SO answer. You can also check the name on Demangler.com.
On line 5 OpenXLSX::XLXmlData s; you are creating an XLXmlData instance. XLXmlData has a std::unique_ptr as a member. So, when a XLXmlData object goes out of scope (at the end of test()) its destructor ~XLXmlData() is called and in it, the compiler generates code to clean-up all members, including the unique_ptr. This code, contains a call to the delete operator.Check this (abridged) godbolt.org version of XLXmlData to see what code gets generated. The call to delete is on line 38.
Even though delete is a C++ keyword, under the hood, the compiler emits just a regular function call (under the mangled _ZdlPv name, as we already saw), so something needs to provide the implementation for that symbol. This "something" typically is the C++ standard library.
However, your build.rs passes None to .cpp_link_stdlib(). According to the docs this disables automatic linking, so the linker will not look for the implementation of delete in the standard library. As nothing else implements delete, this ultimately leads to the unknown import error you are seeing.
To recap, you should link with the standard library. A lot of C++ stuff simply will not work without it.
Related
I created a test library called libmathClass.so which I will be loading from the code below. This shared object has a class and library call is created to return object of this class.
How can I call the methods of this object from the main code displayed below. I get undefined reference error from ld(linker) as it is not aware of the definition of methods.
void* handle;
handle=dlopen("math1/libmathClass.so", RTLD_LAZY);
if(!handle)
{
cout<<"error loading library: "<<dlerror()<<endl;
exit(2);
}
else
{
cout<<"***libmathClass.so library load successful!"<<endl;
}
void* (*mathInit) ();
mathInit = (void* (*)())dlsym(handle, "CreateMathOperationInstance");
if(!mathInit)
{
cout<<"error loading instance method: "<<dlerror()<<endl;
exit(3);
}
else
{
cout<<"***method load successful!"<<endl;
}
mathOperationClass *mathInstance;
auto obj = (*mathInit)();
if(!obj)
{
cout<<"object is not created"<<endl;
exit(4);
}
else
{
cout<<"object created!!!"<<endl;
mathInstance = reinterpret_cast<mathOperationClass *>(obj);
}
int num1 = atoi(argv[1]);
int num2 = atoi(argv[2]);
cout<< mathInstance->AddInt(num1, num2)<<endl;
Command I used to compile - g++ --std=c++11 -g -o dynamicTest dynamicMain.cpp -ldl
Error message:
dynamicMain.cpp:54: undefined reference to `mathOperationClass::AddInt(int, int)'
collect2: error: ld returned 1 exit status
mathInit = (void* (*)())dlsym(handle, "CreateMathOperationInstance");
Here you are using dlsym() to find this symbol in the shared library. This must be a function with C linkage, since the symbol name is not mangled. This is important, and keep this in mind, while you're staring at this line:
cout<< mathInstance->AddInt(num1, num2)<<endl;
Here, AddInt is a method of the class pointed to by mathInstance. A class method is just another function, except that it always takes a hidden this pointer as an extra argument. That's what a class method is, in so many words, and this actually turns out to be the case with a typical C++ implementation. C++ technically does not actually require this to be the case. A C++ implementation is free to implement methods in whatever manner produces the results that are compliant with the C++ specification. But, in practical terms, in a typical C++ implementation this is what a class method actually is. A function with an extra parameter that's referenced as this.
Therefore, in a manner of speaking, the above line is basically equivalent to:
cout<< mathOperationClass::AddInt(mathInstance, num1, num2)<<endl;
This basically is what's going on here, speaking very loosely.
This mathOperationClass::AddInt method/function is, presumably, in the same shared library that you dlopen-ed; and because you dlopen-ed it and you did not actually link to it, you have a reference to this symbol, and this reference cannot be resolved at runtime, hence your runtime undefined symbol error.
The only way you can even the slightest hope of invoking this class method -- if it can be invoked at all in this manner -- is by also using dlsym(). But in order to have even the slightest prayer of actually being be able to pull this off, a whole bunch of things need to happen just right.
First, you have to figure out the actual mangled C++ symbol name. Using my Linux x86_64 g++ compiler as a reference, the mangled name for this method would be "_ZN18mathOperationClass6AddIntEii". With that in hand, you can use dlsym to find this symbol (or whatever your C++ implementation's actual mangled symbol name for this method is) in your shared library.
And once you have this symbol, what now? Well, let's hope that your C++ implementation does, indeed, have a hackable C++ ABI where you can invoke a class method by explicitly passing it an extra this parameter, something like this:
int (*addInt)(mathOperationClass *, int, int)=
reinterpret_cast<int (*)(mathOperationClass *, int, int)>
(dlsym(handle, "_ZN18mathOperationClass6AddIntEii"));
cout << (*addInt)(mathInstance, num1, num2) << endl;
This entire house of cards will collapse unless it can be confirmed that C++ methods can be invoked this hackish way, in your C++ implementation's ABI. Since you're already using dlopen() you're already in non-portable territory, using your C++ implementation-specific resources, so you might as well and figure out whether your C++ methods can be called this way. If not, you'll have to figure out how they can be called, using a plain pointer.
And now for something completely different...
Having said all of the above:
There is one way you can likely avoid dealing with this mess: by making this class method a virtual class method. Virtual class methods are dispatched via an internal virtual function table. So, just try declaring this AddInt method as a virtual class method, and calling it, as is. It's very likely to work in your C++ implementation, since the compiler will not emit, in this instance, an explicit symbol reference for mathOperationClass::AddInt. It will find the method via the virtual function table that's quietly attached to every instance of the object.
Of course, you also need to keep in mind what virtual functions are, and the implications of them. But, in nearly all cases this is a pretty cheap way to call methods of classes that are dynamically loaded from a shared library.
vim docs state that I've got to use C calling conventions for all my functions.
This in mind I wrote a bare minimum dll just to see if everything is ok.
#include <string>
std::string _declspec(dllexport) Meow() {
std::string Meow = "Meow!";
return Meow;
}
For compiling I wrote a makefile
test.dll: test.cpp
cl /LD test.cpp
clean:
del *.obj
del *.dll
del *.exp
del *.lib
Compiled without any issues and copied the dll into my vim directory.
In my understanding calling the function via
:call libcall("test.dll","Meow",0)<cr>
should work. But I keep getting Error 364: Library call failed for "Meow()".
Changing the .dll name inside libcall to something that doesnt exist results in the same error, thus I came to the conclusion something is wrong with my dll.
But then again my dll is compiled without any problems,
which leaves me puzzled.
I see at least two problems with your code:
In C++ names typically get mangled when exported so your Meow function will become something like ?Meow##YA?AV?$basic_string#DU?$char_traits#D#std##V?$allocator#D#2##std##XZ.
Vim docs seem to state that function called with libcall must match some rather restrictive conditions:
The function must take exactly one parameter, either a character pointer or a long integer, and must return a character pointer or NULL. The character pointer returned must point to memory that will remain valid after the function has returned (e.g. in static data in the DLL).
The following should fix your example:
extern "C"
{
static char null_terminated_string[2048];
char* _declspec(dllexport) Meow(char *arg)
{
strncpy(null_terminated_string, arg, std::min(sizeof(null_terminated_string), strlen(arg));
return null_terminated_string;
}
}
I'm trying to experiment with llvm right now. I'd like to use languages that can be compiled to llvm bitcode for scripting. I've managed so far to load an llvm bitcode module and call a function defined in it from my 'internal' c++ code. I've next tried to expose a c++ function from my internal code to the jit'd code - so far in this effort I haven't managed to get anything but SEGFAULT.
My code is as follows. I've tried to create a Function and a global mapping in my execution engine that points to a function I'd like to call.
extern "C" void externGuy()
{
cout << "I'm the extern guy" << endl;
}
void ExposeFunction()
{
std::vector<Type*> NoArgs(0);
FunctionType* FT = FunctionType::get(Type::getVoidTy(getGlobalContext()), NoArgs, false);
Function* fnc = Function::Create(FT, Function::ExternalLinkage, "externGuy", StartModule);
JIT->addGlobalMapping(fnc, (void*)externGuy);
}
// ... Create module, create execution engine
ExposeFunction();
Is the problem that I can't add a function to the module after its been loaded from bitcode file?
Update:
I've refactored my code so that it reads like so instead:
// ... Create module, create execution engine
std::vector<Type*> NoArgs(0);
FunctionType* FT = FunctionType::get(Type::getVoidTy(getGlobalContext()), NoArgs, false);
Function* fnc = Function::Create(FT, Function::ExternalLinkage, "externGuy", m);
fnc->dump();
JIT->addGlobalMapping(fnc, (void*)externGuy);
So instead of segfault I get:
Program used external function 'externGuy' which could not be resolved
Also, the result of dump() prints:
declare void #externGuy1()
If I change my c++ script bitcode thing to call externGuy1() instead of externGuy() it will suggest to me that I meant to use the externGuy. The addGlobalMapping just doesn't seem to be working for me. I'm not sure what I'm missing here. I also added -fPIC to my compilation command like I saw suggested in another question - I'm honestly not sure if its helped anything but no harm in trying.
I finally got this to work. My suspicion is to maybe creating the function, as well as defining it in the script is causing perhaps more than 1 declaration of the functions name and the mapping just wasn't working out. What I did was define the function in the script and then use getFunction to use for mapping instead. Using the dump() method on the function output:
declare void #externGuy() #1
Which is why I think that had something to do with the mapping not working originally. I also remember the Kaleidoscope tutorial saying that getPointerToFunction would do the JIT compiling when its called if it isn't already done so I thought that I would have to do the mapping before this call.
So altogether to get this whole thing working it was as follows:
// Get the function and map it
Function* extrn = m->getFunction("externGuy");
extrn->dump();
JIT->addGlobalMapping(extrn, (void*)&::externGuy);
// Get a pointer to the jit compiled function
Function* mane = m->getFunction("hello");
void* fptr = JIT->getPointerToFunction(mane);
// Make a call to the jit compiled function which contains a call to externGuy
void (*FP)() = (void(*)())(intptr_t)fptr;
FP();
I have a program which is to be used with a shared library.
I have one library (1) that is compiled with the methods full of code i.e.
class A
{
public:
int* funcA(int a, int b)
{
int* pInt = new int;
*pInt = a + (b * 20);
return pInt;
}
};
Then I have another library (2) with exactly the same name and interface with nothing in the methods i.e. a dummy class
class A
{
public:
int* funcA(int a, int b)
{
return 0;
}
};
(note: code just used to illustrate my problem)
If I compile against library 1 and then use library 1 at runtime, everything works as expected.
If I compile against library 2 and then use library 1 at runtime, the first called to funcA dies.
If I used nm -D libMy.so and look at the offset of funcA at runtime, it is different. Is this included in the binary?
Ive read various manuals and tutorials but am none the wiser as to how the compilation and runtime aspect causes this failure. I would have thought the interface is the same so the method would succeed.
Thanks.
The reason this is failing is that you have linked against a different library and thus (as you have seen) the function offsets are different. The linker has placed the offsets into your compiled binary and so it will only run against that library. In order to accomplish what you are attempting here you will need to use dynamic library loading see this SO question for more info.
EDIT:
With a little further reading, I came across this PDF, you may find it helpful.
(I don't have enough rep to just make a comment below your question)
This might be because the program is prelinked (Linux) or prebinded (MacOS) although I am not 100% sure. Some basic info about it on wikipedia below. Have you encountered this on your searches through the manuals?
http://en.wikipedia.org/wiki/Prelink
http://en.wikipedia.org/wiki/Prebinding
Did you forget a -fPIC option while compiling libraries? Please, add compilation commands.
I have an application a part of which uses shared libraries. These libraries are linked at compile time.
At Runtime the loader expects the shared object to be in the LD_LIBRARY_PATH , if not found the entire application crashes with error "unable to load shared libraries".Note that there is no guarantee that client would be having the library, in that case I want the application to leave a suitable error message also the independent part should work correctly.
For this purpose I am using dlsym() and dlopen() to use the API in the shared library. The problem with this is if I have a lot of functions in the API, i have to access them Individually using dlsym() and ptrs which in my case are leading to memory corruption and code crashes.
Are there any alternatives for this?
The common solution to your problem is to declare a table of function pointers, to do a single dlsym() to find it, and then call all the other functions through a pointer to that table. Example (untested):
// libfoo.h
struct APIs {
void (*api1)(void);
void *(*api2)(int);
long (*api3)(int, void *);
};
// libfoo.cc
void fn1(void) { ... }
void *fn2(int) { ... }
long fn3(int, void *) { ... }
APIs api_table = { fn1, fn2, fn3 };
// client.cc
#include "libfoo.h"
...
void *foo_handle = dlopen("libfoo.so", RTLD_LAZY);
if (!foo_handle) {
return false; // library not present
}
APIs *table = dlsym(foo_handle, "api_table");
table->api1(); // calls fn1
void *p = table->api2(42); // calls fn2
long x = table->api3(1, p); // calls fn3
P.S. Accessing your API functions individually using dlsym and pointers does not in itself lead to memory corruption and crashes. Most likely you just have bugs.
EDIT:
You can use this exact same technique with a 3rd-party library. Create a libdrmaa_wrapper.so and put the api_table into it. Link the wrapper directly against libdrmaa.so.
In the main executable, dlopen("libdrmaa_wrapper.so", RTLD_NOW). This dlopen will succeed if (and only if) libdrmaa.so is present at runtime and provides all API functions you used in the api_table. If it does succeed, a single dlsym call will give you access to the entire API.
You can wrap your application with another one which first checks for all the required libraries, and if something is missing it errors out nicely, but if everything is allright it execs the real application.
Use below type of code
Class DynLib
{
/* All your functions */
void fun1() {};
void fun2() {};
.
.
.
}
DynLib* getDynLibPointer()
{
DynLib* x = new Dynlib;
return x;
}
use dlopen() for loading this library at runtime.
and use dlsym() and call getDynLibPointer() which returns DynLib object.
from this object you can access all your functions jst as obj.fun1().....
This is ofcource a C++ style of struct method proposed earlier.
You are probably looking for some form of delay library load on Linux. It's not available out-of-the-box but you can easily mimic it by creating a small static stub library that would try to dlopen needed library on first call to any of it's functions (emitting diagnostic message and terminating if dlopen failed) and then forwarding all calls to it.
Such stub libraries can be written by hand, generated by project/library-specific script or generated by universal tool Implib.so:
$ implib-gen.py libxyz.so
$ gcc myapp.c libxyz.tramp.S libxyz.init.c ...
Your problem is that the resolution of unresolved symbols is done very early on - on Linux I believe the data symbols are resolved at process startup, and the function symbols are done lazily. Therefore depending on what symbols you have unresolved, and on what sort of static initialization you have going on - you may not get a chance to get in with your code.
My suggestion would be to have a wrapper application that traps the return code/error string "unable to load shared libraries", and then converts this into something more meaningful. If this is generic, it will not need to be updated every time you add a new shared library.
Alternatively you could have your wrapper script run ldd and then parse the output, ldd will report all libraries that are not found for your particular application.