Having troubles with mixing library types (static vs. dynamic) - c++

After battling my makefile woes I'm now onto problems with how the two libraries are supposed to interact. So, this is on Linux (CentOS 6.2 - 6.4, not that that seems to make much difference in terms of tools). The project, on the whole, is to be deployed in two flavors
A C++ Static library which is used for linking with other C++ appications (a *.a file)
A shared *.so which is deployed through python using Boost.python
I have everything compiling, but I'm not linking correctly somehow. In the case of the static library, it is built in two ways:
If meant to be linked with other C++ code, -fPIC is not used
If meant to be linked into the python module, use -fPIC
I control this through passing parameters to the make program. Presently, I'm trying to build the boost python module because the static stuff compiles just fine. So, I have dependencies on the boost libraries and zlib. The final linking command looks like this:
g++ -o pythonmod.so -L/boost/boost_libs -L/zlibs -lz -lboost_python -lboost_thread -lboost_regex -lboost_system /path/to/static.a -fPIC -shared [many_objects]
The "many_objects" comes from the various wrappers, and other code, that wraps the "pure" C++ from the boost.python layer in the code. Each of these object files are compiled with -fPIC as well. Their compiled with:
g++ -I/boost/boost_1_47 -I/usr/include/python2.6 -D _linux -MMD -std=c++0x -c -m32 -fPIC <input> -o <output>
The lines compiling the object files for the archive file look quite similar to the above only they do not include the python include directory.
I've found other links here to similar problems and tried the solutions but to no avail thus far. For example, this link uses -Wl,--whole-archive ... -Wl,--no-whole-archive. I tried this solution when I was trying to link in the static library archive before I was compiling it with -fPIC. Now that I'm doing that, I've tried this solution but also to no avail. In each case, every time I load up python and import the module, I get some sort of undefined symbol error --> something went wrong during linking.
How should I mix these libraries together to make the python module work?

After joining the gcc-help mailing list I received the pointer I needed to resolve the issue. The problem turned out to the ordering of the libraries used for linking on the build command line. Basically, the object files generated during the build for the *.so needed to be placed first. Then, the references to the boost and other libraries. Reordering the object files to reference the objects built for wrapping the static library before the other libraries was the key. No longer am I seeing weird "unresolved objects" when loading my python module.
After 4 or 5 years of using Visual Studio, my gcc knowledge had become sufficiently rusty that I'd forgotten the importance of ordering when it comes to linking.

Related

Error when linking shared library after compiling objects

I compiled the object files, and then tried to compile the executable, which failed by saying that there are undefined references to functions in "theLib".
g++ -I./theLib/src -c -o obj/main.o src/main.cpp
(works so far)
g++ -L./theLib -Wl,-rpath=./theLib obj/main.o -ltheLib -o exe
(error: libtheLib.so: undefined reference to 'some_function')
I checked for answers everywhere, and they all just suggest moving the -ltheLib part after the dependencies (which I did). But it still doesn't work. What really boggles my mind is that the same library compiles just fine for an example in a different directory.
The library is in C. Can that mess up trying to compile C++? Or am I just missing something with compiling the .o files first?
Turns out the library depended on functions that I had to implement and provide in my own source code. I never knew that some libraries did that, but lesson learned. Once I implemented the functions that were causing the errors and added those source files, it worked.

NVCC attempting to link unnecessary objects

I have a project that I'm working on making run with CUDA. For various reasons, it needs to compile an executable either with or without GTK support, without recompiling all of the associated files. Under C, I accomplished this by compiling a base version of the objects to *.o and a GTK version of the objects to *.gtk.o. Thus, I can link to that library and if it needs to use GTK it will pull in those functions (and their requirements); if it doesn't it won't touch those objects.
Converting to nvcc has caused some issues: it works in either always or never GTK mode; but if I compile the libraries with the additional GTK objects, it refuses to ignore them and link a GTKless executable. (It fails with errors about being unable to find the cairo functions I call.)
I'm guessing that nvcc is linking to (at least one of) its helper functions embedded in the object, which is causing the linker to resolve the entire object.
Running ar d <lib> <objects.gtk.o> to manually strip them from the library will "fix" the problem, so there isn't a real dependency there.
I'm compiling/linking with
/usr/local/cuda/bin/nvcc --compiler-options -Wall --compiler-options -pipe
-rdc=true -O0 -g -G -I inc -I inc/ext -arch compute_20 -o program
program.cu obs/external.o libs/base.a libs/extra.a libs/core.a -lm
How can I get nvcc to ignore the unneeded objects?
How can I get nvcc to ignore the unneeded objects?
Before you can achieve that, you need to understand which symbol is causing the *.gtk.o objects to be pulled in from the library when they shouldn't be.
The way to do that is to run link with -Wl,--print-map, and look for linker messages such as:
Archive member included because of file (symbol)
libfoo.a(foo.o) main.o (foo)
Above, main.o referenced foo, which is defined in libfoo.a(foo.o), which caused foo.o to be pulled in into the main binary.
Once you know which symbols cause xxxx.gtk.o to be pulled into the link, searching the web and/or NVidia documentation may reveal a way to get rid of them.

Static linking to libcrypto++, with g++

I am trying to compile a program on my system with Debian Wheezy and g++4.7. I want it to be able to run on another system with Debian Squeeze (and no recent g++). I can't compile the program on the Squeeze, because I use certain C++11 features the old g++ does not support, as well as a new Boost version and libcrypto++9.
As far as I understand the usual way to get around this problem is to static link the libraries not supported at the other system, in my case libstdc, boost and crypto++.
My (linking) compiler call right now is
g++-4.7 .obj/btcmirco.o -Wl,-Bstatic -lboost_program_options -lboost_system -lcrypto++ -Wl,-Bdynamic -lcurl -static-libgcc -std=c++11 -o MyProgram
However I seem to have missed something, because it throws a lot of undefined reference errors. It works fine if I dynamic link to crypto++ (and only static link libstdc and boost).
Can anyone tell me whats wrong, or if there is a fundamental error in my approach?
The linker errors I get are (shorted):
`.text._ZN8CryptoPP22BufferedTransformationD2Ev' referenced in section `.text._ZN8CryptoPP22BufferedTransformationD1Ev[_ZN8CryptoPP22BufferedTransformationD1Ev]' of /usr/lib/gcc/x86_64-linux-gnu/4.7/../../../../lib/libcrypto++.a(cryptlib.o): defined in discarded section `.text._ZN8CryptoPP22BufferedTransformationD2Ev[_ZN8CryptoPP22BufferedTransformationD5Ev]' of /usr/lib/gcc/x86_64-linux-gnu/4.7/../../../../lib/libcrypto++.a(cryptlib.o)
`.text._ZN8CryptoPP25MessageAuthenticationCodeD2Ev' referenced in section `.text._ZN8CryptoPP25MessageAuthenticationCodeD1Ev[_ZN8CryptoPP25MessageAuthenticationCodeD1Ev]' of /usr/lib/gcc/x86_64-linux-gnu/4.7/../../../../lib/libcrypto++.a(cryptlib.o): defined in discarded section `.text._ZN8CryptoPP25MessageAuthenticationCodeD2Ev[_ZN8CryptoPP25MessageAuthenticationCodeD5Ev]' of /usr/lib/gcc/x86_64-linux-gnu/4.7/../../../../lib/libcrypto++.a(cryptlib.o)
I experienced the same problem and this has to do with the fact that you are trying to mix code generated by g++-4.7 (your program) with code generated by a previous version of g++ (cryptopp library).
The reason behind this is that when you execute compile the library executing make command, it uses the default version of g++ set up for your system, usually the one that comes with the OS.
In order to solve the issue what you should do is compile cryptopp library with g++-4.7.
For that, compile the library by executing make CXX=g++-4.7. The resulting static library shouldn't give you the error when being linked with your code.

Correctly linking GLX library in Ubuntu

I'm trying to compile one of these mixes of X11 + OpenGL, but I'm not having luck with the compiler. In particular, I get:
undefined symbol: glXMakeCurrent
I have tried
-lX11 -lGLU -lGL -lXext
as arguments to the linker, and some permutations of them, with no luck so far.
I'm running Ubuntu 12.04, and I have installed all the development packages related to opengl that I had a fuzzy idea could be related. I'm also developing in C++, something that could cause problems if the opengl headers are not prepared for it... but they are right?
I even looked for the symbol explicitly with an fgrep in /usr/lib/x86_64-linux-gnu/, but it is not there, and furthermore, `nm' says that there are no symbols.
So, what's the correct way of linking with glx?
EDIT: It is linking problem, the error is produced when python tries to load the compiled (and incorrectly linked) module. Not at compilation time.
EDIT: Here is the compilation log
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
g++ -o build/debug/objects/alve/layouter/flowing_data.os -c -std=c++0x -g -I/usr /include/python2.7 -fPIC -I/opt/cairo_new/include/cairo/ -I/opt/boost_1_48_0/include -DMIC_RT_SPEED_BACKS -Icsrc csrc/alve/layouter/flowing_data.cpp
g++ -o build/debug/objects/alve/layouter/liblayouter.so -L/opt/cairo_new/lib -L/opt/boost_1_48_0/lib -shared build/debug/objects/alve/layouter/flowing_data.os build/debug/objects/alve/layouter/show_network.os -Lbuild/debug/lib -Llibdeps
Install file: "build/debug/objects/alve/layouter/liblayouter.so" as "build/debug/lib/liblayouter.so"
g++ -o build/debug/objects/alve/layouter/liblayouter_mod.so -L/opt/cairo_new/lib -L/opt/boost_1_48_0/lib -shared build/debug/objects/alve/layouter/module.os Lbuild/debug/lib -Llibdeps -lboost_python build/debug/objects/alve/layouter/liblayouter.so -lcairo -lX11 -lGL -lGLU -lXext
scons: done building targets.
and here is how the function is called:
glXMakeCurrent (dpy, win, ctx);
The message "undefined symbol" indicates that it's not a linker, but a compilation unit problem: The compiler does not know the symbol glXMakeCurrent because it has been neither declared, nor defined, but you use it.
Probably the GLX header has not been included.
Add
#include <GL/glx.h>
As it turns out OPs problem was related to the fact, that the build consisted of cascading shared objects forming a Python module. One shared object implements the actual OpenGL operations, while the other does the interfacing to the Python interpreter.
Now shared objects (.so) are fully qualified ELF binaries themself, each with their own import and export symbol table. A shared object can be configured to expose all the symbols of other shared object it links to. However a shared object will not see any symbols of the compilation units they're linked into (if you think about it, this is to be expected, as a shared object can not and should not make any assumptions about the environment it's going to be linked into).
Hence, when compiling and linking multiple shared object in a larger build it's important to individually link each shared object to any libraries it will need at runtime.
As #datenwolf ways, special precautions are needed for linking. Which are them is a mystery for me, but using ldd helps. So basically what I did was to use ldd in both the final and the intermediate shared objects. Despite the command line arguments, my library didn't get linked with libGL and its dependencies until I also included the '-lGL -lGLU -lX11' in the intermediate step (production of 'liblayouter.so').

Boost::Thread linking error on OSX?

So I'm going nuts trying to figure this one out. Here's my basic setup:
I'm compiling a shared library with a bunch of core functionality that uses a lot of boost stuff. We'll call this library libpf_core.so. It's linked with the boost static libraries, specifically the python, system, filesystem, thread, and program_options libraries. This all goes swimmingly.
Now, I have a little test program called test_socketio which is compiled into a shared library (it's loaded as a plugin at runtime). It uses some boost stuff like boost::bind and boost::thread, and it's linked again libpf_core.so (which has the boost libraries included remember).
When I go to compile test_socketio though, out of all my plugins it gives me a linking error:
[ Building test_socketio ]
g++ -c -pg -g -O0 -I/usr/local/include -I../include test_socketio.cc -o test_socketio.o
g++ -shared test_socketio.o -lpy_core -o test_socketio.so
Undefined symbols:
"boost::lock_error::lock_error()", referenced from:
boost::unique_lock<boost::mutex>::lock() in test_socketio.o
ld: symbol(s) not found
collect2: ld returned 1 exit status
And I'm going crazy trying to figure out why this is. I've tried explicitly linking boost::thread into the plugin to no avail, tried ensuring that I'm using the boost headers associated with the libraries linked into libpf_core.so in case there was a conflict there.
Is there something OSX specific regarding boost that I'm missing? In my searching on google I've seen a number of other people get this error but no one seems to have come up with a satisfactory solution.
Edit: Figured it out, OSX comes with boost 1.40 in /usr/local/include. Needed to put the headers for my version of boost somewhere and make sure that my plugins sees those first.
You need to link to libboost_thread. Add the -lboost_thread switch.
When you link libpf_core.so against the static boost libraries, it's only going to get copies of the functions it actually uses. The linker doesn't bother to pull in functions that aren't referenced by the code in your library.
You need to link your test program against the boost libraries as well. You can't reliably "chain" the linkages.