How to link shared library to another shared library - c++

I am trying to create shared library which is implementing JNI(Java Native Interface). My shared library is using another shared library which is named libPosAPI.so. But my shared library not correctly linking shared functions of libPosAPI.so.
In an implementation cpp, i am trying to use function vatps::PosAPI::sendData() of libPosAPI.so. Here is my build command:
g++ -fPIC -I"$JAVA_HOME/include" -I"$JAVA_HOME/include/linux" -I. -shared -o libPosAPIJni.so main_ubp_pos_PosAPIJni.cpp
It compiles successfully. Even it does not ask to provide libPosAPI.so with linker. But when use output shared library(libPosAPIJni.so), it gives following error undefined symbol: _ZN5vatps6PosAPI8sendDataB5cxx11Ev. I also provided libPosAPI.so with -L -l options. The result is same.
Here is my header file. That is a result of javac -h PosAPIJni.java.
main_ubp_pos_PosAPIJni.h
/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class main_ubp_pos_PosAPIJni */
#ifndef _Included_main_ubp_pos_PosAPIJni
#define _Included_main_ubp_pos_PosAPIJni
#ifdef __cplusplus
extern "C" {
#endif
/*
* Class: main_ubp_pos_PosAPIJni
* Method: sendData
* Signature: ()Ljava/lang/String;
*/
JNIEXPORT jstring JNICALL Java_main_ubp_pos_PosAPIJni_sendData
(JNIEnv *, jclass);
#ifdef __cplusplus
}
#endif
#endif
main_ubp_pos_PosAPIJni.cpp
#include "main_ubp_pos_PosAPIJni.h"
#include "PosAPI.h" // header file of libPosAPI.so
#include <iostream>
using namespace std;
inline string jstring_to_string(JNIEnv* env, jstring string) {
string value;
....
}
// IMPLEMENTATIONS
JNIEXPORT jstring JNICALL Java_main_ubp_pos_PosAPIJni_sendData(JNIEnv* env,
jclass cls) {
string res_sendData = vatps::PosAPI::sendData(); // The PROBLEM IS HERE!!. trying to use function of libPosAPI.so. declared in PosAPI.h
return string_to_jstring(env, res_sendData);
}
The output of nm command of two shared libraries:
nm -D libPosAPI.so | grep sendData :
000000000000c3e0 T sendData
000000000000a8dc T _ZN5vatps6PosAPI8sendDataEv
nm -u libPosAPIJni.so | grep sendData:
U _ZN5vatps6PosAPI8sendDataB5cxx11Ev
Please direct me right way :)

Let's decode your symbol:
$ c++filt _ZN5vatps6PosAPI8sendDataB5cxx11Ev
vatps::PosAPI::sendData[abi:cxx11]()
So, your code expects sendData with std::string with C++11 ABI, whereas libPosAPI.so provides sendData with pre-C++11 ABI std::string.
abi:cxx11 hints to GCC5 and the C++11 ABI:
Users that depend on third-party libraries or plugin interfaces that still use the old ABI can build their code with -D_GLIBCXX_USE_CXX11_ABI=0 and everything should work fine. In most cases, it will be obvious when this flag is needed because of errors from the linker complaining about unresolved symbols involving __cxx11.

Related

link cython module in a c++ program

Is it possible to build a cython module with some cdef functions and link the resulting shared library into a C++ program?
I tried a proof of concept:
cymod.pyx:
# distutils: language=c++
from libcpp.string cimport string
cdef public string simple_echo(string test_string):
return test_string
cpp_test.cpp:
#define PyMODINIT_FUNC void
#include <iostream>
#include "cymod.h"
int main(int argc, char const *argv[])
{
std::cout << simple_echo("test") << std::endl;
return 0;
}
setup.py:
from setuptools import setup, Extension
from Cython.Build import cythonize
setup(
name='cymod',
ext_modules=cythonize(
Extension(
"cymod", ["cymod.pyx"],
),
)
)
The cython module builds fine, but when I try to build the c++ code that will use the cython function I get:
$ g++ -L. -l:cymod.so cpp_test.cpp -o cpp_test
/tmp/cc48Vc2z.o: In function `main':
cpp_test.cpp:(.text+0x51): undefined reference to `simple_echo'
collect2: error: ld returned 1 exit status
Which is odd. The generated header file has it:
cymod.h:
/* Generated by Cython 0.29.1 */
#ifndef __PYX_HAVE__cymod
#define __PYX_HAVE__cymod
#ifndef __PYX_HAVE_API__cymod
#ifndef __PYX_EXTERN_C
#ifdef __cplusplus
#define __PYX_EXTERN_C extern "C"
#else
#define __PYX_EXTERN_C extern
#endif
#endif
#ifndef DL_IMPORT
#define DL_IMPORT(_T) _T
#endif
__PYX_EXTERN_C std::string simple_echo(std::string);
#endif /* !__PYX_HAVE_API__cymod */
/* WARNING: the interface of the module init function changed in CPython 3.5. */
/* It now returns a PyModuleDef instance instead of a PyModule instance. */
#if PY_MAJOR_VERSION < 3
PyMODINIT_FUNC initcymod(void);
#else
PyMODINIT_FUNC PyInit_cymod(void);
#endif
#endif /* !__PYX_HAVE__cymod */
and I see my function in cymod.so:
nm cymod.so| grep simple_echo
0000000000001e50 T simple_echo
NOTE: I realize that to actually get this working I'll need to link against the python libraries and initialize the interpreter etc. I left that out to make this a tad shorter and I get the same error either way.
The short answer is that I was putting the -l argument too early in the compilation command. It is also important to handle the library lookup path. The simplest way is to use rpath. I set the rpath to the directory that the executable is in, i.e., .
Additionally, it is necessary to link against the python libraries and set the include and library paths. These can be determined at compile time by using the output of the python-config utility. Here is the compilation command that ultimately did the trick:
g++ cpp_test.cpp -o cpp_test -L. -l:cymod.so $(python-config --libs) $(python-config --includes) $(python-config --cflags) -Wl,-rpath,"\$ORIGIN"
I also updated the c++ file to #include "Python.h" and added calls to Py_Initialize(), Py_Finalize(), and initcymod():
#include <iostream>
#include "Python.h"
#include "cymod.h"
int main(int argc, char *argv[])
{
Py_Initialize();
initcymod();
std::cout << simple_echo("test") << std::endl;
Py_Finalize();
return 0;
}
NOTE: the call to initcymod() is necessary, but python2 specific. On python3 you should call PyImport_AppendInittab("cymod", PyInit_cymod); prior to Py_Initialize(). The cymod part is the module name, substitute your module name.
Thanks to #ead for the informative link to the docs on this topic https://cython.readthedocs.io/en/latest/src/userguide/external_C_code.html#using-cython-declarations-from-c and his answer to a related question https://stackoverflow.com/a/45424720/2069572
While reading the linked docs, I came across this:
Note On some operating systems like Linux, it is also possible to first build the Cython extension in the usual way and then link against the resulting .so file like a dynamic library. Beware that this is not portable, so it should be avoided.
So it turns out that you should not do what I was trying to do.
Instead, what I should have done was run:
cython --cplus cymod.pyx
And then compiled cpp_test.cpp with the generated cymod.cpp file.
No need to link the cython shared library, and it turns out that it is not a good idea to do so.

Using Eigen in a C Project

I am working on a C project I got from the Internet, and I'm trying to add some functions to the project that involve linear algebra. In my previous works in C++, I usually rely on Eigen for linear algebra.
Is there a way to use Eigen for a C project? If yes, what should I do to make that work? (Simply adding Eigen header files is not enough since for example the standard C++ files do not get included automatically)
Eigen is a library which heavily uses C++ features which are not present in C. As such, it cannot be directly used from a C translation unit.
However, you can wrap the parts using Eigen in a separate shared library and expose a C interface. Here is a small example how one could write such a library.
Library interface
/* foo.h */
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
void foo(int arg);
#ifdef __cplusplus
} /* extern "C" */
#endif /* __cplusplus */
By default, C++ uses different mangling rules than C for names of exported functions. We use extern "C" to instruct the C++ compiler to use the C rules. Because the interface file will be seen by both the C++ and the C compiler, we wrap the extern declaration in #ifdefs which will only trigger for the C++ compiler.
Library implementation
/* foo.cpp */
#include "foo.h"
#include <iostream>
extern "C" {
void foo(int arg) {
std::cout << arg << std::endl;
}
} /* extern "C" */
We also need to define C linkage in the definition of the interface. Other than that, you can use any C++ features you like in the implementation (including Eigen).
C project
/* main.c */
#include "foo.h"
int main() {
foo(42);
return 0;
}
Include the interface header and use it like any other C library.
Building
$ g++ foo.cpp -shared -fPIC -o libfoo.so
$ gcc main.c -L. -lfoo -o main
Use a C++ compiler to build the shared library libfoo.so. Use a C compiler to build the main program, linking to the shared library. The exact build steps may vary for your compiler/platform.

How to link C++ to use variable defined in static C library (symbol not found error)

I have C static library that has a struct global variable defined within it. I wish to access that variable within C++ code that will link to that C library. At the moment, I am getting "ld: symbol(s) not found for architecture x86_64" errors.
I am trying the following (simplified struct for the sake of the question):
// library.h
#ifdef __cplusplus
extern "C" {
#endif
typedef struct {
int simpleVariableA;
int simpleVariableB;
} GlobalStruct;
extern GlobalStruct gs;
#ifdef __cplusplus
}
#endif
// library.c
#include "library.h"
GlobalStruct gs;
Library.c and library.h are compiled into library.a, a statically-linked library. I then want to link a C++ file to the library, but get errors.
// main.cpp
#include "library.h"
int main(int argc, char** argv) {
gs.simpleVariableA = 1;
gs.simpleVariableB = 1;
}
What am I doing wrong? I have run "nm -g library.a" and get "0000000000000008 C _gs" back, so I think this means that the symbol is being exported, no? Is it an issue with C++ not finding C code?
I compile the files like this:
gcc -c library.c -o library.o
ar rcs liblibrary.a library.o
g++ main.cpp -L. -llibrary
BTW, I get an error with the ar command:
warning: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: warning for library: liblibrary.a the table of contents is empty (no object file members in the library define global symbols)
However, this error goes away if I define a function f() in library.h and library.c and compile this into the library.
Each of your mentions of gs is merely a declaration (that is it indicates that there is a thing by this name somewhere), but none of them is a definition (which is something which additionally sets aside space for the named thing).
If you initialise gs in library.c, your problem goes away:
GlobalStruct gs = { 0, 0 };
The function of the extern keyword in your library.h is specifically to stop this being a (tentative) definition. It's possibly redundant in this case, but probably best to include it, and so make sure that there's only one definition of gs, in library.c.
Including the initialiser in library.c also stops the warning message, because that initialiser is what creates the thing that goes in the library (that is, you were correct: the warning was significant for your problem).

How to link app with static library + why this is not working

I have a problem. I wrote example code and I want to build it without the error:
main.cpp(.text+0x5): undefined reference to `test()'
Library
test1.c
#include <stdlib.h>
void test()
{
puts("Działa");
}
test1.h
#ifndef TEST1_H
#define TEST1_H
extern void test();
#endif
makefile
all:
gcc -c ./src/test1.c -o ./lib/test1.o
ar rcs ./lib/libtest1.a ./lib/test1.o
Program
main.cpp
#include <test1.h>
int main()
{
test();
return 0;
}
makefile
all:
g++ -static -I../test1/include -L../test1/lib ./src/main.cpp -o ./build/MyApp -ltest1
What am I doing wrong?
You are compiling a C code function, but you are expecting to link a C++ function.
Because of 'type safe linkage', the function you provide is not the function that the C++ code calls.
Either in test1.h use:
#ifdef __cplusplus
extern "C" {
#endif
extern void test1(void);
#ifdef __cplusplus
}
#endif
Or:
Compile the function with the C++ compiler.
The C++ compiler will mangle the symbol names to provide type-safe linkage (a term which you should be able to search for via your preferred search engine).
The 'compiler' - actually the linker - is looking for a function with a C++ mangled name representing the C++ function with the signature 'void test1(void);'.
For example (but remember - different compilers deliberately mangle things differently), G++ 4.2.1 on MacOS X 10.6.2 generates a symbol '__Z5test1v' for the function; GCC generates a symbol '_test1'. Clearly, when the linker is looking for '__Z5test1v', the symbol '_test1' is not going to be used - it is not spelled the same. This is a good thing.
You can use 'nm -g' on the object file for the main program to see what it is looking for, and on the object file in the library to see what it is providing. And, given that the spellings are different, that is why the loader does not pick up the library function - it is looking for something with a different name.
You are calling a C function from a C++ function. The naming between the two is different (C++ mangles names to include parameter information).
Change the header file to look like this:
#ifdef __cplusplus
extern "C" {
#endif
extern void test();
#ifdef __cplusplus
}
#endif
This will tell the compiler that the function follows the C naming/calling convention.

Why do we need extern "C"{ #include <foo.h> } in C++?

Why do we need to use:
extern "C" {
#include <foo.h>
}
Specifically:
When should we use it?
What is happening at the compiler/linker level that requires us to use it?
How in terms of compilation/linking does this solve the problems which require us to use it?
C and C++ are superficially similar, but each compiles into a very different set of code. When you include a header file with a C++ compiler, the compiler is expecting C++ code. If, however, it is a C header, then the compiler expects the data contained in the header file to be compiled to a certain format—the C++ 'ABI', or 'Application Binary Interface', so the linker chokes up. This is preferable to passing C++ data to a function expecting C data.
(To get into the really nitty-gritty, C++'s ABI generally 'mangles' the names of their functions/methods, so calling printf() without flagging the prototype as a C function, the C++ will actually generate code calling _Zprintf, plus extra crap at the end.)
So: use extern "C" {...} when including a c header—it's that simple. Otherwise, you'll have a mismatch in compiled code, and the linker will choke. For most headers, however, you won't even need the extern because most system C headers will already account for the fact that they might be included by C++ code and already extern "C" their code.
extern "C" determines how symbols in the generated object file should be named. If a function is declared without extern "C", the symbol name in the object file will use C++ name mangling. Here's an example.
Given test.C like so:
void foo() { }
Compiling and listing symbols in the object file gives:
$ g++ -c test.C
$ nm test.o
0000000000000000 T _Z3foov
U __gxx_personality_v0
The foo function is actually called "_Z3foov". This string contains type information for the return type and parameters, among other things. If you instead write test.C like this:
extern "C" {
void foo() { }
}
Then compile and look at symbols:
$ g++ -c test.C
$ nm test.o
U __gxx_personality_v0
0000000000000000 T foo
You get C linkage. The name of the "foo" function in the object file is just "foo", and it doesn't have all the fancy type info that comes from name mangling.
You generally include a header within extern "C" {} if the code that goes with it was compiled with a C compiler but you're trying to call it from C++. When you do this, you're telling the compiler that all the declarations in the header will use C linkage. When you link your code, your .o files will contain references to "foo", not "_Z3fooblah", which hopefully matches whatever is in the library you're linking against.
Most modern libraries will put guards around such headers so that symbols are declared with the right linkage. e.g. in a lot of the standard headers you'll find:
#ifdef __cplusplus
extern "C" {
#endif
... declarations ...
#ifdef __cplusplus
}
#endif
This makes sure that when C++ code includes the header, the symbols in your object file match what's in the C library. You should only have to put extern "C" {} around your C header if it's old and doesn't have these guards already.
In C++, you can have different entities that share a name. For example here is a list of functions all named foo:
A::foo()
B::foo()
C::foo(int)
C::foo(std::string)
In order to differentiate between them all, the C++ compiler will create unique names for each in a process called name-mangling or decorating. C compilers do not do this. Furthermore, each C++ compiler may do this is a different way.
extern "C" tells the C++ compiler not to perform any name-mangling on the code within the braces. This allows you to call C functions from within C++.
It has to do with the way the different compilers perform name-mangling. A C++ compiler will mangle the name of a symbol exported from the header file in a completely different way than a C compiler would, so when you try to link, you would get a linker error saying there were missing symbols.
To resolve this, we tell the C++ compiler to run in "C" mode, so it performs name mangling in the same way the C compiler would. Having done so, the linker errors are fixed.
C and C++ have different rules about names of symbols. Symbols are how the linker knows that the call to function "openBankAccount" in one object file produced by the compiler is a reference to that function you called "openBankAccount" in another object file produced from a different source file by the same (or compatible) compiler. This allows you to make a program out of more than one source file, which is a relief when working on a large project.
In C the rule is very simple, symbols are all in a single name space anyway. So the integer "socks" is stored as "socks" and the function count_socks is stored as "count_socks".
Linkers were built for C and other languages like C with this simple symbol naming rule. So symbols in the linker are just simple strings.
But in C++ the language lets you have namespaces, and polymorphism and various other things that conflict with such a simple rule. All six of your polymorphic functions called "add" need to have different symbols, or the wrong one will be used by other object files. This is done by "mangling" (that's a technical term) the names of symbols.
When linking C++ code to C libraries or code, you need extern "C" anything written in C, such as header files for the C libraries, to tell your C++ compiler that these symbol names aren't to be mangled, while the rest of your C++ code of course must be mangled or it won't work.
When should we use it?
When you are linking C libaries into C++ object files
What is happening at the
compiler/linker level that requires us
to use it?
C and C++ use different schemes for symbol naming. This tells the linker to use C's scheme when linking in the given library.
How in terms of compilation/linking
does this solve the problems which
require us to use it?
Using the C naming scheme allows you to reference C-style symbols. Otherwise the linker would try C++-style symbols which wouldn't work.
You should use extern "C" anytime that you include a header defining functions residing in a file compiled by a C compiler, used in a C++ file. (Many standard C libraries may include this check in their headers to make it simpler for the developer)
For example, if you have a project with 3 files, util.c, util.h, and main.cpp and both the .c and .cpp files are compiled with the C++ compiler (g++, cc, etc) then it isn't really needed, and may even cause linker errors. If your build process uses a regular C compiler for util.c, then you will need to use extern "C" when including util.h.
What is happening is that C++ encodes the parameters of the function in its name. This is how function overloading works. All that tends to happen to a C function is the addition of an underscore ("_") to the beginning of the name. Without using extern "C" the linker will be looking for a function named DoSomething##int#float() when the function's actual name is _DoSomething() or just DoSomething().
Using extern "C" solves the above problem by telling the C++ compiler that it should look for a function that follows the C naming convention instead of the C++ one.
The C++ compiler creates symbol names differently than the C compiler. So, if you are trying to make a call to a function that resides in a C file, compiled as C code, you need to tell the C++ compiler that the symbol names that it is trying to resolve look different than it defaults to; otherwise the link step will fail.
The extern "C" {} construct instructs the compiler not to perform mangling on names declared within the braces. Normally, the C++ compiler "enhances" function names so that they encode type information about arguments and the return value; this is called the mangled name. The extern "C" construct prevents the mangling.
It is typically used when C++ code needs to call a C-language library. It may also be used when exposing a C++ function (from a DLL, for example) to C clients.
This is used to resolve name mangling issues. extern C means that the functions are in a "flat" C-style API.
Decompile a g++ generated binary to see what is going on
To understand why extern is necessary, the best thing to do is to understand what is going on in detail in the object files with an example:
main.cpp
void f() {}
void g();
extern "C" {
void ef() {}
void eg();
}
/* Prevent g and eg from being optimized away. */
void h() { g(); eg(); }
Compile with GCC 4.8 Linux ELF output:
g++ -c main.cpp
Decompile the symbol table:
readelf -s main.o
The output contains:
Num: Value Size Type Bind Vis Ndx Name
8: 0000000000000000 6 FUNC GLOBAL DEFAULT 1 _Z1fv
9: 0000000000000006 6 FUNC GLOBAL DEFAULT 1 ef
10: 000000000000000c 16 FUNC GLOBAL DEFAULT 1 _Z1hv
11: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND _Z1gv
12: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND eg
Interpretation
We see that:
ef and eg were stored in symbols with the same name as in the code
the other symbols were mangled. Let's unmangle them:
$ c++filt _Z1fv
f()
$ c++filt _Z1hv
h()
$ c++filt _Z1gv
g()
Conclusion: both of the following symbol types were not mangled:
defined
declared but undefined (Ndx = UND), to be provided at link or run time from another object file
So you will need extern "C" both when calling:
C from C++: tell g++ to expect unmangled symbols produced by gcc
C++ from C: tell g++ to generate unmangled symbols for gcc to use
Things that do not work in extern C
It becomes obvious that any C++ feature that requires name mangling will not work inside extern C:
extern "C" {
// Overloading.
// error: declaration of C function ‘void f(int)’ conflicts with
void f();
void f(int i);
// Templates.
// error: template with C linkage
template <class C> void f(C i) { }
}
Minimal runnable C from C++ example
For the sake of completeness and for the newbs out there, see also: How to use C source files in a C++ project?
Calling C from C++ is pretty easy: each C function only has one possible non-mangled symbol, so no extra work is required.
main.cpp
#include <cassert>
#include "c.h"
int main() {
assert(f() == 1);
}
c.h
#ifndef C_H
#define C_H
/* This ifdef allows the header to be used from both C and C++. */
#ifdef __cplusplus
extern "C" {
#endif
int f();
#ifdef __cplusplus
}
#endif
#endif
c.c
#include "c.h"
int f(void) { return 1; }
Run:
g++ -c -o main.o -std=c++98 main.cpp
gcc -c -o c.o -std=c89 c.c
g++ -o main.out main.o c.o
./main.out
Without extern "C" the link fails with:
main.cpp:6: undefined reference to `f()'
because g++ expects to find a mangled f, which gcc did not produce.
Example on GitHub.
Minimal runnable C++ from C example
Calling C++ from is a bit harder: we have to manually create non-mangled versions of each function we want to expose.
Here we illustrate how to expose C++ function overloads to C.
main.c
#include <assert.h>
#include "cpp.h"
int main(void) {
assert(f_int(1) == 2);
assert(f_float(1.0) == 3);
return 0;
}
cpp.h
#ifndef CPP_H
#define CPP_H
#ifdef __cplusplus
// C cannot see these overloaded prototypes, or else it would get confused.
int f(int i);
int f(float i);
extern "C" {
#endif
int f_int(int i);
int f_float(float i);
#ifdef __cplusplus
}
#endif
#endif
cpp.cpp
#include "cpp.h"
int f(int i) {
return i + 1;
}
int f(float i) {
return i + 2;
}
int f_int(int i) {
return f(i);
}
int f_float(float i) {
return f(i);
}
Run:
gcc -c -o main.o -std=c89 -Wextra main.c
g++ -c -o cpp.o -std=c++98 cpp.cpp
g++ -o main.out main.o cpp.o
./main.out
Without extern "C" it fails with:
main.c:6: undefined reference to `f_int'
main.c:7: undefined reference to `f_float'
because g++ generated mangled symbols which gcc cannot find.
Example on GitHub.
Tested in Ubuntu 18.04.