I was working with SFML, I compiled a little test program and added the linkage option -lsfml-audio. Then, I used ldd ./program to see the dynamic libraries it was linking to. Surprisingly, there were a lot, none of them had I manually selected in my makefile, nor using pkg-config --libs.
I started reading about shared libraries, and made a little example to solve my doubts. However, I have this question:
why some libraries need you to add the dependencies in your makefile
(either manually or using a script like pkg-config) and other
libraries automatically link their dependencies?
When you're creating your dynamic library, is just as easy as adding the proper -ldependency options in the g++ -shared ... command to avoid the user the hassle of manually adding the dependencies later on. Why many of the available libraries don't do that?
I guess it must be related to the ability of fine tuning which libraries get linked and such.
Shared libraries will generally link in their dependencies. However, static libraries are not capable of doing so. pkg-config --libs often includes all dependencies (direct and indirect) so that you can switch to static compilation by simply adding -static without having to add additional library dependencies as well.
Note that these excess direct dependencies are considered unwanted in some cases (eg, debian tries to avoid them in packaged binaries, as they make library soname transitions more traumatic than necessary). You can instruct the linker to strip direct dependencies from the final executable that aren't needed with the -Wl,--as-needed flag.
Related
I have a Qt application that uses the boost logger library. I want to make it a standalone. However, after I managed the libraries for static linking, the application is still dependent on boost libraries.
The libraries that I included are:
..../boost_1_61_0_b1/stage/lib/libboost_regex.a
..../boost_1_61_0_b1/stage/lib/libboost_log_setup.a
..../boost_1_61_0_b1/stage/lib/libboost_thread.a
..../boost_1_61_0_b1/stage/lib/libboost_log.a
..../boost_1_61_0_b1/stage/lib/libboost_system.a
..../boost_1_61_0_b1/stage/lib/libboost_filesystem.a
The application compiles( after countless attempts). However, when I use ldd tool, it shows boost libraries on the dependency list.
Note: I have to define BOOST_ALL_DYN_LINK. Otherwise, it doesn't link.
Is there any way not to use this macro and overcome the dependency problem ? If not, what solutions do you suggest to circumvent this problem?
By default on modern UNIX-like systems gcc links with shared libraries by default. In order to force static linking you can either add -static to your linking command line (see the docs) or make sure gcc doesn't find the shared libraries but only finds the static libraries (e.g. move the shared libraries to a separate directory while you're linking your project). Note that -static will make all libraries linked statically, including libstdc++.
Alternatively, you can specify the static libraries directly, without the -l switch. You will have to use the full path to the libraries though, so instead of
gcc ... -lboost_log ...
you would write
gcc ... ..../boost_1_61_0_b1/stage/lib/libboost_log.a ...
In any case, you should not define BOOST_ALL_DYN_LINK because this macro means exactly the opposite - that you intend to link with Boost shared libraries.
I have a C++ file a.cpp with the library dependency in the path /home/name/lib and the name of the library abc.so.
I do the compilation as follows:
g++ a.cpp -L/home/name/lib -labc
This compiles the program with no errors.
However while running the program, I get the ERROR:
./a.out: error while loading shared libraries: libabc.so.0: cannot open shared object file: No such file or directory
However if before running the program, I add the library path as
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/name/lib;
and compile and run now, it works fine.
Why am I not able to link the library by giving it from the g++ command?
Because shared object libraries are linked at runtime - you either need to add the location to the library search path (as you've done), place it somewhere within an already existing path (like /usr/lib), or add a symbolic link to an existing library path that links to the location where it exists.
If you want to compile the library in so there is no runtime dependency, you'll need a static library (which would be abc.a) - note that this has numerous downsides, however (if the library is updated, you'll need to recompile your executable to incorporate that update).
Why am I not able to link the library by giving it from the g++ command?
You are able to link, and you did link the library succesfully. Otherwise you would not be able to build executable (in your case a.out). The problem you mixed 2 different things: linking with shared libraries and loading them at runtime. Loading shared libraries is a pretty complex concept and described pretty well here Program-Library-HOWTO read from 3.2.
You are linking dynamically, is the default behavior with GCC. LD_LIBRARY_PATH is used to specify directories where to look for libraries (is a way of enforce using an specific library), read: Program-Library-HOWTO for more info. There is also an ld option -rpath to specify libraries search path for the binary being compiled (this info is written in the binary and only used for that binary, the LD_LIBRARY_PATH affect other apps using the same library, probably expecting a new or old version).
Linking statically is possible (but a little tricky) and no dependency would be required (but sometimes is not recommended, because prevent the update of the dependent libraries, for example for security reason, in static linking your always are using the versions of the libraries you have when compiled the binary).
I'm trying to setup an automake project that uses a mix of libtool libraries and exectuables, and I'm having a hard time grokking the automake documentation, esp. as relates to telling the compiler to link against.
So can someone explain the differences between LDADD and LIBADD?
Things like:
when is one used over the other,
which one takes -lname_of_library style values vs. direct filenames,
etc.
Whenever I try to read the relevant documentation, it seems like it assumes that I know things that I don't.
Use the LIBADD primary for libraries, and LDADD for executables. If you were building a libtool library libfoo.la, that depended on another library libbar.la, you would use:
libfoo_la_LIBADD = libbar.la
If you had other non-libtool libraries, you would also add these with -L and -l options:
libfoo_la_LIBADD = libbar.la -L/opt/local/lib -lpng
Typically, you would use the configure script to find these extra libraries, and use AC_SUBST to pass them with:
libfoo_la_LIBADD = libbar.la $(EXTRA_FOO_LIBS)
For a program, just use LDADD :
myprog_LDADD = libfoo.la # links libfoo, libbar, and libpng to myprog.
Sometimes the boundaries are a bit vague. $(EXTRA_FOO_LIBS) could have been added to myprog_LDADD. Adding dependencies to a libtool (.la) library, and using libtool do all the platform-specific linker magic, is usually the best approach. It keeps all the linker metadata in the one place.
As mentioned in one of the books, LDADD is ADDitional linker(LD) items - i.e. items that are added when performing linking. This would be, for example, when producing programs. LDADD can specify:
libtool files e.g. lib/libfudge.la
libraries e.g. -lm, or
full paths to libraries e.g. /lib/libmagicalwonderland.a
link flags & libraries e.g. -L/opt/lib -lmagical
They are in order of preference - Using the last two is just asking for trouble as they're pointing at things that may or may not be present.
LIBADD is to specify ADDitional LIBraries to use. This is used when building a library to specify that additional libraries are needed in order to build or make use of the library. You'll see it specified as something like libfred_la_LIBADD =. It can be used to specify libtool libraries, or system libraries and will place these libraries into the resulting libtool .la for the library so when it comes to linking against the library you get all the appropriate libraries brought along.
You should only specify libraries to link, so, for example, my library libfred.la depends on some math routines i.e. it depends on libm. When I'm specifying the additional libraries for the library, I state:
libfred_la_LIBADD = -lm
This dependency is encoded when I build the library, and gets passed on to consumers of the library as well.
The rule of thumb is:
LIBADD for things that are being added to a library
LDADD for things that are being added to a program
I have a software library and I used to create .a files, so that people can install them and link against them: g++ foo.o -L/path/to -llibrary
But now I often encounter third-party libraries where only .so files are available (instead of .a), and you just link against them without the -l switch, e.g. g++ foo.o /path/to/liblibrary.so.
What are the differences between these solutions? Should I prefer creating .so files for the users of my library?
Typically, libfoo.a is a static library, and libfoo.so is a shared library. You can use the same -L/-l linker options against either a static or shared. Or you can name the full path to the lib with static or shared. Often libraries are built both static and shared to provide application developers the choice of which they want.
All the code needed from a static lib is part of the final executable. This obviously makes it bigger, but it also means it's self-contained. Once it is compiled, you can run your app without the lib.
Code from a shared lib is not part of the executable. There are just some hooks in place to make the executable aware of the name of the lib it needs. In order to run your app, the shared lib has to be present in the lib search path (e.g. $LD_LIBRARY_PATH).
If you have two apps that share the same code, they can each link against a shared lib to keep the binary size down. If you want to upgrade parts of the app without rebuilding the whole thing, shared libs are good for that too.
Good overview of static, shared dynamic and loadable libraries at
http://www.yolinux.com/TUTORIALS/LibraryArchives-StaticAndDynamic.html
Some features that aren't really called out from comments I've seen so far.
Static linkage (.a/.lib)
Sharing memory between these compilation units is generally ok because they should(?will) all be using the same runtime.
Static linkage means you avoid 'dll hell' but the cost is recompilation to make use of any change at all. static linkage into Shared libraries (.so) can lead to strange results if you have more than 1 such shared library used by the final executable - global variables may exist multiple times and which one is used and when they are initialised can cause an entirely different hell.
The library will be part of the shipped product but obfuscated and not directly usable.
Shared/Dynamic libraries (.so/.dll)
Sharing memory between these compilation units can be hazardous as they may choose to use different runtime. This can mean you provide different Shared/Dynamic libraries based on the debug/release or single/multi threaded or...
Shared libraries (.so) are less prone to 'dll hell' then Dynamic libraries (.dll) as they include options for quite specific versioning.
Compiling against a .so will capture version information internal to the file (hard to fake) so that you get quite specific .so usage. Compiling against the .lib/.dll only gives a basic file name, any versioning is done managed by the developer (using naming or manually loading the library and checking version details by hand)
The library will have to ship with the final product (somebody else can pick it up and use it)
But now I often encounter third-party libraries where only .so files are available [...] and you just link against them without the -l switch, e.g. g++ foo.o /path/to/liblibrary.so.
JFYI, if you link to a shared library which does not have a SONAME set (compare with readelf -a liblibrary.so), you will end up putting the specified path of liblibrary.so into your target object (executable or another shared library), and which is usually undesired, for users have their own ideas of where to put a program and its associated files. The preferred way is to use -L/path/to -llibrary, perhaps together with -Wl,-rpath,/whatever/path/to if this is the final path (such pathing decisions are made by Linux distributions for example).
Should I prefer creating .so files for the users of my library?
If you distribute source code, the user will make the particular choice.
In my project there is a dependency on a static library (just called libsomething from now on) from a 3rd party. Recently, libsomething has become available in another version. My task is to provide my software with support for the old and the new version. Only one version of libsomething is used at run-time at any given time, but which version this is should be configurable between program runs.
I am using MSVC2005 on WinXP, a secondary objective is to become prepared to switch over to Linux and GCC.
Since both versions of libsomething are using the same symbols, linking them both into my executable is out of the question as the symbols of both versions are going to clash all over at link-time.
While I could create two executables (one linking against the old version, the other one using the new version), I cannot implement a decision on which executable to call in the final deployment environment (legacy reasons).
I came up with the idea of creating a dynamic library wrapper for each version of libsomething and linking them at run-time depending on some config file. With MSCV, this would mean going down the road of using LoadLibrary(), GetProcAddress(), etc., while on Linux I would have to use dlopen() and dlsym().
I understand that using libtool (i.e., libtldl) is wrapping this platform-dependency for using shared libraries. Is this an appropriate path to follow? Are there better (or, at least, different) ways? Do alternatives for libtldl exist as open-source?
I know you said you couldn't use two executables due to the decision of which to execute, but couldn't you exec back and forth between executables depending on which version is selected at configuration?
On Linux it would be easier for you to link to shared library and use symlinks to correct version - IMO it's much easier than using dlopen() + dlsym().
Thus you would create shared libraries for the old and new versions of your library:
g++ -shared -o libshared.so.1.1 -Wl,-soname=libshared.so.1 -L. -Wl,--whole-archive libstatic.a.old -Wl,-no-whole-archive
and
g++ -shared -o libshared.so.1.2 -Wl,-soname=libshared.so.1 -L. -Wl,--whole-archive libstatic.a.new -Wl,-no-whole-archive
Create the symlinks:
ln -s libshared.so.1.1 libshared.so.1
ln -s libshared.so.1 libshared.so
Build your application, linking it to the old version of the library. I suppose both versions are binary compatible (ABI not broken), but the new one could have some new symbols.
g++ -o myapp myapp.cpp -L. -lshared
Since shared library's SONAME is libshared.so.1 your application will depend on it and will search libshared.so.1 in paths from /etc/ld.so.conf or LD_LIBRARY_PATH
Before you run your application you may set the libshared.so.1 symlink to point to libshared.so.1.2 or libshared.so.1.1.
Little info about the linker options used here:
--whole-archive
For each archive mentioned on the command line after the --whole-archive option, include every object file in the archive in the
link, rather than searching the archive for the required object files. This is normally used to turn an archive file into a shared
library, forcing every object to be included in the resulting shared library. This option may be used more than once.
Two notes when using this option from gcc: First, gcc doesn't know about this option, so you have to use -Wl,-whole-archive.
Second, don't forget to use -Wl,-no-whole-archive after your list of archives, because gcc will add its own list of archives to your
link and you may not want this flag to affect those as well.
-soname=name
When creating an ELF shared object, set the internal DT_SONAME field to the specified name. When an executable is linked with a
shared object which has a DT_SONAME field, then when the executable is run the dynamic linker will attempt to load the shared object
specified by the DT_SONAME field rather than the using the file name given to the linker.
It has been few years now but I'd like to mention another solution for completeness. Instead of manual dlopen and dlsym you could generate simple stubs for all necessary functions and on first call (or at program startup) decide which library version is needed, load it and resolve the addresses.
You could write a script specifically tailored for your project or use Implib.so tool:
# This will generate mylib.so.init.c and mylib.so.tramp.S
# which implement stubs. These need to be linked to your
# executable.
$ implib-gen.py mylib.so
Implib.so is Linux-only atm but should be easily adaptable to Windows.