I'm trying to use Ruby's FFI library to link functions from the bitcoin-core secp256k1 library.
To make the secp256k1_ecdsa_sign function accessible, I built libsecp256k1 using autotools (as directed in README.md). Then I created a shared object to use in FFI by running g++ -shared secp256k1/src/.libs/libsecp256k1_la-secp256k1.o. Importing this into my Ruby file using FFI let me use the function and everything worked perfectly.
I'm trying to do the exact same with the secp256k1_ecdsa_sign_recoverable function, which is in the same C project, just a different header file. However, I get the error Function 'secp256k1_ecdsa_sign_recoverable' not found in [bin/secp256k1.so] (FFI::NotFoundError).
I think this is either because I'm not creating the shared object properly (created with the aforementioned g++ command), or because this function is for some reason not a public-facing one in the C project (though I don't know enough about C to know how to figure out if this is the case).
If someone could help me figure out how to use this function it would be greatly appreciated.
It ends up the secp256k1_recovery.h file is only included if you specify that when building the libsecp256k1 library. Specifically, I needed to run ./configure --enable-module-recovery instead of ./configure.
Related
I am trying to write a plugin for a popular program whose code and compilation process I do not have control over. The program is written in C. However, I have written parts of my plugin in C++, since I use the QT5 library for graphics capabilities. The functions that the C program calls are written in C.
When the C program tries to load the plugin (shared library), it produces this error:
dlopen('build/libfoo.so') failed: build/libfoo.so: undefined symbol: _ZTV13JoystickPanel
JoystickPanel is a class in the C++ part of the program.
I've tried rewriting parts of the program in C, but the error was unaffected. I know that I could rewrite the entire program in C, but I'd rather not have to switch to another, more C-friendly GUI framework. I've also opened up libfoo.so in a text editor and search for JoystickPanel, but it appears to be mangled as _ZN13JoystickPanel.
Are there any compiler options or solutions that I'm missing?
I have no idea what _ZN13JoystickPanel means, since it's not apparently a valid mangled C++ name. It should perhaps be _ZN13JoystickPanelE, which would translate to JoystickPanel. That'd be symbol name for sure, but without much meaning anyway. You must have truncated something: I tried quite a bit and just can't generate an object file that would include _ZN13JoystickPanel as the complete symbol. It's just a prefix, there should be a "second half" attached to it - was there?
But _ZTV13JoystickPanel is the vtable for the JoystickPanel class. It's missing because you didn't provide implementations for all the virtual methods of the JoystickPanel class. Most likely, you didn't invoke moc properly, or forgot to compile and link its output.
You need to show a complete build script for your plugin at the very least (the .pro file, or CMakeLists.txt). You'll also need to provide a github link to your project (I presume it's open source).
The symbols you want to find in the compiled output are at least _ZTV13JoystickPanelD#Ev - virtual destructors, where # is a digit, _ZTV13JoystickPanel - the virtual method table,
Those symbols may be absent when compiled with optimization and/or LTCG, but also absent will be references to them.
You may wish to delete the build folder and rebuild your project, just to be sure. qmake is bad at dependency generation for the makefiles it produces, so if you use it, I suggest switching to cmake + ninja.
Apparently, I'd forgetten to put the #include "moc_controller.cpp" line at the bottom of a file that needed it.
For anyone else chasing this issue while using Qt on CMake, consider making sure that the proper lines are added.
Working on a code that enables Python to call a C++ code compiled into shared library file (.so file) via Python's ctypes module, using the standard ctypes.CDLL method. The C++ code performs the numerical calculations and the Python code mainly serves as a controller and performs data analysis. Everything worked fine. However, as soon as I included jsoncpp library in C++ code, Python started to complain about undefined symbol errors such as undefined symbol: _ZN4Json5ValueaSES0_ etc. I used jsoncpp library in C++ mainly to export data to a JSON file. Having searched the internet for a while, it appears this hasn't been discussed that much. Any idea how to handle this?
Well, looks like the solution is to also include -ljsoncpp flag while compiling towards the shared .so file:
$(CC) -shared -o testcode.so testcode.o -ljsoncpp
Lets say my executable file, MyApp, links dynamically against an so file, boost-system.so. I compile and everything works fine for a while; my calls to boost-system funcs do what they are supposed to do.
After a while i realize i need to link to another so file, SomeAPI.so. However, SomeAPI.so have statically linked against boost-system.a, but an older, buggy version, which has functions with exactly the same names as "my" boost-system (ofcourse). Now, ALL calls (both from MyApp and SomeAPI) will go to the version SomeAPI statically linked (or my version of boost-system, depending on link order, both are bad for me).
I would like my calls from MyApp to go to my version of boost-system, and SomeAPI to use its statically linked boost-system.a functions. On Windows this is how it works.
I realize why this is not happening, but is there any way around this except renaming the namespaces in "my" boost-system to something local?
Use LD version scripts to hide function names that you don't want other .so files to see.
I hit this issue about two years ago when I first implemented our SWIG bindings. As soon as we exposed a large amount of code we got to the point where SWIG would output C++ files so large the compiler could not handle them. The only way I could get around the issue was to split up the interfaces into multiple modules and to compile them separately.
This has several downsides:
• Each module must know about dependencies in other modules. I have a script to generate the interface files which handles this side of things, but it adds extra complexity.
• Each additional module increases the time that the dynamic linker requires to load in the code. I have added an init.py file that imports all the submodules, so that the fact that the code is split up is transparent to the user, but what is always visible is the long load times.
I'm currently reviewing our build scripts / build process and I wanted to see if I could find a solution to this issue that was better than what I have now. Ideally, I'd have one shared library containing all the wrapper code.
Does anyone know how I can acheive this with SWIG? I've seen some custom code written in Ruby for a specific project, where the output is post-processed to make this possible, but when I looked at the feasibility for Python wrappers it does not look so easy.
I just did equivalent hack for TCL library: I use several SWIG modules, generating several .cpp files that are compiled in several .o files but compile them all in a single .so file that is loaded by a single TCL "load" command.
The idea is to creates a top swig module (Top) that calls initialization functions of all sub-modules (Sub1 and Sub2):
%module Top
%header %{
extern "C" {
SWIGEXPORT int Sub1_Init(Tcl_Interp *);
SWIGEXPORT int Sub2_Init(Tcl_Interp *);
}
%}
%init %{
if (Sub1_Init(interp) != TCL_OK) {return TCL_ERROR;}
if (Sub2_Init(interp) != TCL_OK) {return TCL_ERROR;}
%}
There's nothing special in the submodules files.
I end up with file Top.so that I load from TCL with command "load ./Top.so"
I don't know python but's likely to be similar. You may need to understand how the python extensions are loaded, though.
If split properly, the modules don't necessarily need to have the same dependencies as the others - just what's necessary to do compilation. If you break things up appropriately, you can have libraries without cyclic dependencies. The issue with using multiple libraries is that by default, SWIG declares its runtime code statically, and as a result, as problems passing objects from one module to another. You need to enable a shared version of the SWIG runtime code.
From the documentation (SWIG web page documentation link is broken):
The runtime functions are private to
each SWIG-generated module. That is,
the runtime functions are declared
with "static" linkage and are visible
only to the wrapper functions defined
in that module. The only problem with
this approach is that when more than
one SWIG module is used in the same
application, those modules often need
to share type information. This is
especially true for C++ programs where
SWIG must collect and share
information about inheritance
relationships that cross module
boundaries.
Check out that section in your downloaded documentation (section 16.2 The SWIG runtime code), and it'll give you details on how to enable this so that objects can be properly handled when passed from one module to the other.
FWIW, I've not worked with Python SWIG, but have done Tcl SWIG.
So, I have an interesting issue. I am working with a proprietary set of dlls that I ,obviously, don't have the source for. The goal is to write an intermediate dll that groups together a large series of funnction calls from the proprietary dlls. The problem I am having, when compiling with g++, is that I get errors for the original dlls along the lines of:
cannot export libname_NULL_THUNK_DATA. Symbol not found.
If I add a main and just compile to an executable everything works as expected. I'm using mingw for compilation. Thanks for any help.
In response to the first reply: Either I'm confused about what you're saying or I didn't word my question very well. I'm not explicitly trying to export anything from my wrapper I am just calling functions from their dlls. The problem is that I get errors that it can't export these specific symbols from the dll to my wrapper. The issue is that I'm not even entirely sure what these _NULL_THUNK_DATA symbols are for. I did a search and read somewhere that they shouldn't be exported because they're internal symbols that windows uses. I have tried using the --exclude-symbols directive to the linker but it didn't seem to do anything. I apologize if I'm completely misunderstanding what you're trying to say.
So, I think my issue was related to this. When just compiling a standard executable that uses a dll I was able to include the headers and directly call the functions for example:
#include :3rdparty.h
int main(){
dostuff(); // a function in the 3rdparty.dll
}
this would compile and run fine. I just needed to link the libraries in the g++ command.
When linking with the -shared flag I would get these errors (with main removed of course). I think it has something to do with the fact that by default g++ attempts to import all symbols from the dll. What I didn't understand is why this happens in the dll vs in an executable. I will try doing it using GetProcAddress(). Thank you!
it should be as easy as you think it should be.
eg:
your dll code needs:
void doStuff()
{
3rdparty.login();
3rdparty.dostuff();
3rdparty.logoff();
};
so far - so good, you've included the right headers .... (if you have them, if you don't then you need to import the library using LoadLibrary(), then create a function pointer to each exported dll entrypoint using GetProcAddress() and then call that function pointer)
You then link with the 3rd party lib and that's it. Occasionally you will have to wrap the definitions with 'extern "C"' in order to get the linkage name mangling correct.
As you say you're using g++, you can't be getting confused with __declspec(dllimport) which is a MS VC extension.
"Compiling" tells me that you're approaching this from the wrong end. Your DLL should not export its own wrapper functions, but directly refer to exports from other DLLs.
E.g. in a Windows Kernel32.DEF file, the following forward exists:
EXPORTS
...
HeapAlloc = NTDLL.RtlAllocHeap
There's no code for the HeapAlloc function.