In my project, I am emitting LLVM IR which makes calls to external functions in dynamic libraries.
I declare my external functions like:
declare %"my_type"* #"my_function"()
In the external library, functions are declared like:
extern "C" {
my_type* my_function();
}
When I compile the IR and run it, the process immediately crashes. The same behavior happens if I declare and call a nonsense function that I know doesn't exist, so I assume what's happening is that the external function is not being found/linked. (I don't think that the function itself is crashing).
I am using Python's llvmlite library for this task, and within the same process where I JIT and invoke my LLVM IR, I have another python library imported which requires the external dynamic library; so I assume that library is loaded and in-memory.
The procedure I'm using to compile and execute my LLVM code is basically the same as what's in this document, except that the IR declares and invokes an external function. I have tried invoking cos(), as in the Kaleidoscope tutorial, and this succeeds, so I am not sure what is different about my own library functions.
I have tried adding an underscore to the beginning of the function name, but I get the same result. (Do I need to add the underscore in the LLVM function declaration?)
How do I verify my assumption that the process is crashing because the named function isn't found?
How do I diagnose why the function isn't being found?
What do I need to do in order to make use of external functions in a dynamic library, from LLVM code?
Edit: It seems there is indeed trouble getting the function pointers to my external function. If I try to merely print the function address by replacing my calls with %"foo" = ptrtoint %"my_type"* ()* #"my_function" to i64 and return/print the result, it still segfaults. Merely trying to obtain the pointer is enough to cause a crash! Why is this, and how do I fix it?
Edit: Also forgot to mention— this is on Ubuntu (in a Docker container, on OSX).
I figured it out— I was missing that I need to call llvmlite.binding.load_library_permanently(filename) in order for the external symbols to become available. Even though the library is already in memory, the function still needs to be invoked. (This corresponds to the LLVM native function llvm::sys::DynamicLibrary::LoadLibraryPermanently()).
From this answer, it appears calling the above function with nullptr will import all the symbols available to the process.
Oddly, on OSX, I found that the external symbols were available even though load_library_permanently() had not been explicitly called— I'm not sure why this is (perhaps the OSX build of llvmlite itself happened to call the function with nullptr, as above?).
Related
I exported a function from a DLL, using a .def file, similar to this:
void __stdcall remove(FlagFile* flagP)
{
if (flagP == nullptr) {
return;
}
delete flagP;
}
Trying to call the function, I received crashes I couldn't place, until I saw in the disassembler that instead of calling the code above, the linker had exported a different function from the CRT instead, and then that function was called in place of the function I expected to be called. This resulted in a stack corruption, likely because the function above has void result, as opposed to CRT's remove(), which returns int. The solution was to simply rename the exported function to resolve the name conflict. Unfortunately, MSVC does not give a warning about the name conflict.
(Just putting this here so people can object if my analysis is wrong. This post hopefully gives others a clue who run into a similar problem.)
This was observed with MSVC 2019 under Windows 11. Not sure if it applies to other compilers/operating systems (shared libraries in that case).
So the question would be: aside from giving all functions a prefix (like OpenGL's gl), what are your strategies for dealing with this problem, if it ever was a problem?
I am developing code for an embedded system (specifically, the PSoC 5, using PSoC Creator), and writing in C++.
While I've overcome most hurdles with using C++ , first off compiling in C++ using the compiler flag -x c++, defining the new and delete operators, making sure exceptions aren't thrown with the compiler flag -fno-exception, I've come to a brick wall when it comes to using virtual functions.
If I try and declare a virtual function, the compiler gives me the error undefined reference to "vtable for __cxxabiv1::__class_type_info". The only way to get around this is to use the compiler flag -fno-rtti, which prevents the error and makes it compile successfully. However, if I do that, the embedded program crashes when trying to run the overloaded virtual function, and I'm thinking this is because the vtable does not exist.
I don't see why you shouldn't be able to implement vtables on an embedded platform, since all it is a extra space in memory before or after member objects (depending on the exact compiler).
The reason I am trying to use virtual functions is because I am wanting to use FreeRTOS with C++, and other people have implemented this by using virtual functions (see http://www.freertos.org/FreeRTOS_Support_Forum_Archive/July_2010/freertos_Is_it_possible_create_freertos_task_in_c_3778071.html for the discussion, and https://github.com/yuriykulikov/Event-driven_Framework_for_Embedded_Systems for a well written embedded C++ FreeRTOS framework)
The fact that the error message refers to a class named __cxxabiv1 suggests that you are not linking against the correct C++ runtime for your platform. I don't know anything about PSoC, but on more "normal" platforms, this sort of error could happen if you used the gcc (resp. clang) command at link-time instead of g++ (resp. clang++); or under handwavey circumstances if you used -lc++ without -stdlib=libc++ or -lstdc++ without -stdlib=libstdc++.
Use the -v option to examine your linker command line, and try to find out exactly which C++ runtime library it's pulling in. It'll probably be named something like libcxxabi or libcxxrt.
This guy here gives step-by-step instructions for compiling C++ in PSoC Creator; but he never figured out how to link with a C++ runtime library, so all his tips are focused on how to remove C++isms from your code (-fno-rtti, -fno-exceptions,...). I agree that there doesn't seem to be any information online about how to actually use C++ with PSoC.
For this specific error, you could always try defining the missing symbol yourself:
// file "fix-link-errors.cpp"
namespace __cxxabiv1 {
class __class_type_info {
virtual void dummy();
};
void __class_type_info::dummy() { } // causes the vtable to get created here
};
Or many linkers have the ability to define undefined symbols as 0x0 through command-line options such as -C or --defsym. However, that's not only a Bad Idea but also inconvenient, because you'd have to figure out what the actual (mangled) name of the vtable object is, and the linker didn't tell you that. (This being GCC, it's probably something like __ZTVN10__cxxabiv117__class_type_infoE.)
Either of those "solutions" would result in horrible crashes if the program ever tried to do anything with the vtable; but they'd shut the linker up, if that's all you cared about and you knew the program would never actually use RTTI. But in that case, it should be sufficient to use -fno-rtti consistently on your entire project.
What, specifically, goes wrong when you use -fno-rtti?
I created c++ dll (using mingw) from code I wrote on linux (gcc), but somehow have difficulties using it in VC++. The dll basically exposes just one class, I created pure virtual interface for it and also factory function which creates the object (the only export) which looks like this:
extern "C" __declspec(dllexport) DeviceDriverApi* GetX5Driver();
I added extern "C" to prevent name mangling, dllexport is replaced by dllimport in actual code where I want to use the dll, DeviceDriverApi is the pure virtual interface.
Now I wrote simple code in VC++ which just call the factory function and then just tries to delete the pointer. It compiles without any problems but when I try to run it I get access violation error. If I try to call any method of the object I get access violation again.
When I compile the same code in MinGW (gcc) and use the same library, it runs without any problems. So there must be something (hehe, I guess many differences actually :)) between how VC++ code uses the library and gcc code.
Any ideas what?
Cheers,
Tom
Edit:
The code is:
DeviceDriverApi* x5Driver = GetX5Driver();
if (x5Driver->isConnected())
Console::WriteLine(L"Hello World");
delete x5Driver;
It's crashing when I try to call the method and when I try to delete the pointer as well. The object is created correctly though (the first line). There are some debug outputs when the object is created and I can see them before I get the access violation error.
You're using one compiler (mingw) for the DLL, and another (VC++) for the calling code.
You're calling a 'C' function, but returning a pointer to a C++ Object.
That will never work, because VTable layouts are almost guranteed to be incompatible. And, the DLL and app are probably using different memory managers, so you're doing new() with one and delete() with the other. Again, it just won't work.
For this to work the two compilers need to both support a standard ABI (Application Binary Interface). I don't think such a thing exists for Windows.
The best option is to expose all you DLL object methods and properties via C functions (including one to delete the object). You can the re-wrap into a C++ object on the calling end.
The two different compilers may be using different calling conventions. Try putting _cdecl before the function name in both the client code and the DLL code and recompiling both.
More info on calling conventions here: http://en.wikipedia.org/wiki/X86_calling_conventions
EDIT: The question was updated with more detail and it looks likely the problem is what Adrien Plisson describes at the end of his answer. You're creating an object in one module and freeing it in another, which is wrong.
(1) I suspect a calling covnention problem as well, though the simple suggestion by Leo doesn't seem to have helped.
Is isConnected virtual? It is possible that MinGW and VC++ use different implementations for a VTable, in which case, well, tough luck.
Try to see how far you get with the debugger: does it crash at the call, or the return? Do you arrive at invalid code? (If you know to read assembly, that usually helps a lot with these problems.)
Alternatively, add trace statements to the various methods, to see how far you get.
(2) For a public DLL interface, never free memory in the caller that was allocated by a callee (or vice versa). The DLL likely runs with a completely different heap, so the pointer is not known.
If you want to rely on that behavior, you need to make sure:
Caller and Callee (i.e. DLL and main program, in your case) are compiled with the same version of the sam compiler
for all supported compilers, you have configured the compile options to ensure caller and callee use the same shared runtime library state.
So the best way is to change your API to:
extern "C" __declspec(dllexport) DeviceDriverApi* GetX5Driver();
extern "C" __declspec(dllexport) void FreeDeviceDriver(DeviceDriverApi* driver);
and, at caller site, wrap in some way (e.g. in a boost::intrusive_ptr).
try looking at the imported libraries from both your DLL and your client executable. (you can use the Dependency Viewer or dumpbin or any other tool you like). verify that both the DLL and the client code are using the same C++ runtime.
if it is not the case, you can indeed run into some issues since the way the memory is managed may be different between the 2, leading to a crash when freeing from one runtime a pointer allocated from another runtime.
if this is really your problem, try not destroying the pointer in your client executable, but rather declare and export a function in your DLL which will take care of destroying the pointer.
Whilst compiling with avr-gcc I have encountered linker errors such as the following:
undefined reference to `__cxa_pure_virtual'
I've found this document which states:
The __cxa_pure_virtual function is an error handler that is invoked when a pure virtual function is called.
If you are writing a C++ application that has pure virtual functions you must supply your own __cxa_pure_virtual error handler function. For example:
extern "C" void __cxa_pure_virtual() { while (1); }
Defining this function as suggested fixes the errors but I'd like to know:
what the purpose of this function is,
why I should need to define it myself and
why it is acceptable to code it as an infinite loop?
If anywhere in the runtime of your program an object is created with a virtual function pointer not filled in, and when the corresponding function is called, you will be calling a 'pure virtual function'.
The handler you describe should be defined in the default libraries that come with your development environment. If you happen to omit the default libraries, you will find this handler undefined: the linker sees a declaration, but no definition. That's when you need to provide your own version.
The infinite loop is acceptable because it's a 'loud' error: users of your software will immediately notice it. Any other 'loud' implementation is acceptable, too.
1) What's the purpose of the function __cxa_pure_virtual()?
Pure virtual functions can get called during object construction/destruction. If that happens, __cxa_pure_virtual() gets called to report the error. See Where do "pure virtual function call" crashes come from?
2) Why might you need to define it yourself?
Normally this function is provided by libstdc++ (e.g. on Linux), but avr-gcc and the Arduino toolchain don't provide a libstdc++.
The Arduino IDE manages to avoid the linker error when building some programs because it compiles with the options "-ffunction-sections -fdata-sections" and links with "-Wl,--gc-sections", which drops some references to unused symbols.
3) Why is it acceptable to code __cxa_pure_virtual() as an infinite loop?
Well, this is at least safe; it does something predictable. It would be more useful to abort the program and report the error. An infinite loop would be awkward to debug, though, unless you have a debugger that can interrupt execution and give a stack backtrace.
I would like to call a method from an dll, but i don't have the source neither the header file. I tried to use the dumpbin /exports to see the name of the method, but i can found the methods signature?
Is there any way to call this method?
Thanks,
If the function is a C++ one, you may be able to derive the function signature from the mangled name. Dependency Walker is one tool that will do this for you. However, if the DLL was created with C linkage (Dependency Walker will tell you this), then you are out of luck.
The C++ language does not know anything about dlls.
Is this on Windows? One way would be to:
open the dll up in depends.exe shipped with (Visual Studio)
verify the signature of the function you want to call
use LoadLibrary() to get load this dll (be careful about the path)
use GetProcAddress() to get a pointer to the function you want to call
use this pointer-to-function to make a call with valid arguments
use FreeLibrary() to release the handle
BTW: This method is also commonly referred to as runtime dynamic linking as opposed to compile-time dynamic linking where you compile your sources with the associated lib file.
There exists some similar mechanism for *nixes with dlopen, but my memory starts to fail after that. Something called objdump or nm should get you started with inspecting the function(s).
As you have found, the exports list in a DLL only stores names, not signatures. If your DLL exports C functions, you will probably have to disassemble and reverse engineer the functions to determine method signatures. However, C++ encodes the method signature in the export name. This process of combining the method name and signature is called "name mangling". This Stackoverflow question has a reference for determining the method signature from the mangled export name.
Try the free "Dependency Walker" (a.k.a. "depends") utility. The "Undecorate C++ Functions" option should determine the signature of a C++ method.
It is possible to figure out a C function signature by analysing beginnig of its disassembly. The function arguments will be on the stack and the function will do some "pops" to read them in reverse order. You will not find the argument names, but you should be able to find out their number and the types. Things may get more difficult with return value - it may be via 'eax' register or via a special pointer passed to the function as the last pseudo-argument (on the top of the stack).
If you indeed know or strongly suspect the function is there, you can dynamically load the DLL with loadLibrary and get a pointer to the function with getProcAddress. See MSDN
Note that this is a manual, dynamic way to load the library; you'll still have to know the correct function signature to map to the function pointer in order to use it. AFAIK there is no way to use the dll in a load-time capability and use the functions without a header file.
Calling non-external functions is a great way to have your program break whenever the 3rd party DLL is updated.
That said, the undname utility may also be helpful.