I am working on a collection of reusable libraries that need to be made available both as static libraries (.a & .lib) and as dynamic libraries (.so & .dll).
I want dependency management for the dynamic libraries to be as simple as possible (you only take one dynamic library for each bit of functionality that you need), so all of the functional dependencies that each dynamic library has are actually statically linked into it. Thus, the dynamic libraries offer their functionality to downstream clients dynamically, but their upstream dependencies are satisfied statically.
The upshot of all this is that all of my static libraries need to be compiled with -fPIC so that their code is suitable for linkage into a shared library. The same goes for any third-party library that we use. It has to be a static library, compiled with -fPIC.
(I could, I suppose, build both PIC and non-PIC variants of my libraries - but I really do not want to compile the libraries a third time for each target platform -- twice is quite (more than) enough!).
So, here is my problem:
I have been trying to compile boost_system as a static library with -fPIC, but I am not sure if I am succeeding:
/b2 --build-type=complete variant=release link=static threading=multi runtime-link=static --layout=versioned --cxxflags=-fPIC
This build produces .a files as output, as expected. However, when I try to link the boost static library into one of my shared libraries, I start getting an error message that indicates that boost_system is not Position Independent Code:
.../dependencies/external/boost/1_54_0/stage/lib/linux_x86_64/libboost_system-gcc46-s-1_54.a(error_code.o):
relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
However, I have (attempted) to build boost with -fPIC. Is there any test that I can use to determine if libboost_system is actually PIC code? I.e. if the problem is with building boost - or with linking it to my application.
I believe your problem can be solved by removing the command line option "runtime-link=static" which enables static linking of the C++ runtime libraries. Since you are building a dynamic shared library object, you want to avoid this behavior, especially if clients are to link to your library from different Linux OS configurations. However, the option "link=static" is ok and should remain.
Related
I have built a shared library that depends on a bunch of other libraries. But when loading that shared library, it needs the dependent libraries to be present in order to run properly. I want this shared library to be portable and hence want it to contain all the dependencies in itself. Is there a way in cmake to do this or what is the best solution here?
Actually this is not related with CMake, but with concepts of linking. You should link with static version of all your dependent libraries.
I have a Qt application that uses the boost logger library. I want to make it a standalone. However, after I managed the libraries for static linking, the application is still dependent on boost libraries.
The libraries that I included are:
..../boost_1_61_0_b1/stage/lib/libboost_regex.a
..../boost_1_61_0_b1/stage/lib/libboost_log_setup.a
..../boost_1_61_0_b1/stage/lib/libboost_thread.a
..../boost_1_61_0_b1/stage/lib/libboost_log.a
..../boost_1_61_0_b1/stage/lib/libboost_system.a
..../boost_1_61_0_b1/stage/lib/libboost_filesystem.a
The application compiles( after countless attempts). However, when I use ldd tool, it shows boost libraries on the dependency list.
Note: I have to define BOOST_ALL_DYN_LINK. Otherwise, it doesn't link.
Is there any way not to use this macro and overcome the dependency problem ? If not, what solutions do you suggest to circumvent this problem?
By default on modern UNIX-like systems gcc links with shared libraries by default. In order to force static linking you can either add -static to your linking command line (see the docs) or make sure gcc doesn't find the shared libraries but only finds the static libraries (e.g. move the shared libraries to a separate directory while you're linking your project). Note that -static will make all libraries linked statically, including libstdc++.
Alternatively, you can specify the static libraries directly, without the -l switch. You will have to use the full path to the libraries though, so instead of
gcc ... -lboost_log ...
you would write
gcc ... ..../boost_1_61_0_b1/stage/lib/libboost_log.a ...
In any case, you should not define BOOST_ALL_DYN_LINK because this macro means exactly the opposite - that you intend to link with Boost shared libraries.
I have a very basic client/server project that uses boost::asio. It generates two executables, a client and a server.
When I run the client, I get the following:
./client: error while loading shared libraries:
libboost_system.so.1.55.0: cannot open shared object
file: No such file or directory
This means that the program requires the boost_system binary to be loaded dynamically at run-time. This makes sense, as one dependency of boost_asio is boost_system.
What does this mean for the ease of distributing my application to end-users?
1) Do I simply pop my development version of the boost_system binary on my system, which in this case is libboost_system.so.1.55.0? How do I ensure that when the user runs the client, it will find the dynamic archive? Obviously, on my system, even with my boost install it still didn't find the archive.
2) I am building on Linux and thus I have .so binaries. How will #1 change if I try to cross-compile my app for Windows with mingw-w64?
I am brand-spanking new to distributing C++ programs and working with dynamic/shared libraries.
When I compile statically, I get the following warning:
Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
Suggestion:
1) If you use shared libraries, you'll definitely need to include those libraries your program actually uses alone with your executable.
2) Here is a list of the Boost libraries. Your program will require just a subset:
http://www.boost.org/doc/libs/1_49_0/more/getting_started/unix-variants.html
The only Boost libraries that must be built separately are:
Boost.Filesystem
Boost.GraphParallel
Boost.IOStreams
Boost.MPI
Boost.ProgramOptions
Boost.Python (see the Boost.Python build documentation before building and installing it)
Boost.Regex
Boost.Serialization
Boost.Signals
Boost.System
Boost.Thread
Boost.Wave
A few libraries have optional separately-compiled binaries:
Boost.DateTime has a binary component that is only needed if you're using its to_string/from_string or serialization features, or if you're targeting Visual C++ 6.x or Borland.
Boost.Graph also has a binary component that is only needed if you intend to parse GraphViz files. * Boost.Math has binary components for the TR1 and C99 cmath functions.
Boost.Random has a binary component which is only needed if you're using random_device.
Boost.Test can be used in “header-only” or “separately compiled” mode, although separate compilation is recommended for serious use.
Alternatively, you can link your program with static (.a) Boost libraries instead of shared (.so), in which case there will be NO runtime dependencies.
Or you can mix/match shared/statis as you wish.
The choice is yours.
Look at the Boost documentation: b2 Static and Shared libraries
As said, you need to compile boost with the static option, for example
bjam install --toolset=msvc variant=release link=static threading=multi runtime-link=static
You can have more information in this Thread
Do i have static or dynamic boost libraries?
Something to notice, if you do an ldd on your executable, you'll probably notice some runtime dependencies on gcc/libc libraries, even if you compile it in static mode.
That means your client platform has to have those libraries installed. 90% of the time they're there, but it might be more complicated when you compile with the latest version of the compiler and the client has an older one.
I have a bunch of static libraries (*.a), and I want to build a shared library (*.so) to link against those static libraries (*.a). How can I do so in gcc/g++?
You can (just extract all the .o files and link them with -shared to make a .so), but whether it works, and how well it works, depends on the platform and whether the static library was compiled as position-independent code (PIC). On some platforms (e.g. x86_64), non-PIC code is not valid in shared libraries and will not work (actually I think the linker will refuse to make the .so). On other platforms, non-PIC code will work in shared libraries, but the in-memory copy of the library is not sharable between different programs using it or even different instances of the same program, so it will result in HUGE memory bloat.
I can't see why you couldn't just build the files of your dynamic library to .o files and link with;
gcc -shared *.o -lstaticlib1 -lstaticlib2 -o mylib.so
I have a software library and I used to create .a files, so that people can install them and link against them: g++ foo.o -L/path/to -llibrary
But now I often encounter third-party libraries where only .so files are available (instead of .a), and you just link against them without the -l switch, e.g. g++ foo.o /path/to/liblibrary.so.
What are the differences between these solutions? Should I prefer creating .so files for the users of my library?
Typically, libfoo.a is a static library, and libfoo.so is a shared library. You can use the same -L/-l linker options against either a static or shared. Or you can name the full path to the lib with static or shared. Often libraries are built both static and shared to provide application developers the choice of which they want.
All the code needed from a static lib is part of the final executable. This obviously makes it bigger, but it also means it's self-contained. Once it is compiled, you can run your app without the lib.
Code from a shared lib is not part of the executable. There are just some hooks in place to make the executable aware of the name of the lib it needs. In order to run your app, the shared lib has to be present in the lib search path (e.g. $LD_LIBRARY_PATH).
If you have two apps that share the same code, they can each link against a shared lib to keep the binary size down. If you want to upgrade parts of the app without rebuilding the whole thing, shared libs are good for that too.
Good overview of static, shared dynamic and loadable libraries at
http://www.yolinux.com/TUTORIALS/LibraryArchives-StaticAndDynamic.html
Some features that aren't really called out from comments I've seen so far.
Static linkage (.a/.lib)
Sharing memory between these compilation units is generally ok because they should(?will) all be using the same runtime.
Static linkage means you avoid 'dll hell' but the cost is recompilation to make use of any change at all. static linkage into Shared libraries (.so) can lead to strange results if you have more than 1 such shared library used by the final executable - global variables may exist multiple times and which one is used and when they are initialised can cause an entirely different hell.
The library will be part of the shipped product but obfuscated and not directly usable.
Shared/Dynamic libraries (.so/.dll)
Sharing memory between these compilation units can be hazardous as they may choose to use different runtime. This can mean you provide different Shared/Dynamic libraries based on the debug/release or single/multi threaded or...
Shared libraries (.so) are less prone to 'dll hell' then Dynamic libraries (.dll) as they include options for quite specific versioning.
Compiling against a .so will capture version information internal to the file (hard to fake) so that you get quite specific .so usage. Compiling against the .lib/.dll only gives a basic file name, any versioning is done managed by the developer (using naming or manually loading the library and checking version details by hand)
The library will have to ship with the final product (somebody else can pick it up and use it)
But now I often encounter third-party libraries where only .so files are available [...] and you just link against them without the -l switch, e.g. g++ foo.o /path/to/liblibrary.so.
JFYI, if you link to a shared library which does not have a SONAME set (compare with readelf -a liblibrary.so), you will end up putting the specified path of liblibrary.so into your target object (executable or another shared library), and which is usually undesired, for users have their own ideas of where to put a program and its associated files. The preferred way is to use -L/path/to -llibrary, perhaps together with -Wl,-rpath,/whatever/path/to if this is the final path (such pathing decisions are made by Linux distributions for example).
Should I prefer creating .so files for the users of my library?
If you distribute source code, the user will make the particular choice.