First, I don't know if there is a solution to my problem at all.
I have the following situation:
I have developed a framework library that depends on several other libraries for specific hardware access, etc.
Until now this framework library was only statically linked against.
For the executable that uses the framework library only the dependencies of code that is actually used by the executable have to be linked. (If I don't access a specific hardware at all I don't have to depend on its associated libraries.)
Now I need to also make a shared object of the framework library. Also the dependencies are available as shared libraries, so there is no need for any static linking.
The problem I have now:
When building an application that links dynamically to the framework library I have to either link all dependencies dynamically to the framework library or the application. (Otherwise I get undefined references complaints from ld)
My questions:
Is there any way to ignore certain shared object dependencies if I know that my application will not use any code of the framework library that depends on this shared object?
Is there any way to do this without or with minimal code changes? (linker / compiler switches)
I also need the static linking as described in the original situation to still work.
Additional Info:
Operating system: Linux (Debian Lenny)
Compiler: gcc-4.3
You can, but you basically have to do all of the dynamic library handling yourself. i.e. dlopen the library, and then look up the symbols you need directly with dlsym.
It will make your code more complicated, how much depends on the interface you've got into the libraries.
From man ld
--as-needed
--no-as-needed
This option affects ELF DT_NEEDED tags for dynamic libraries mentioned on the command line after the --as-needed option. Normally,
the linker will add a DT_NEEDED tag for each dynamic library mentioned on the command line, regardless of whether the library is
actually needed. --as-needed causes a DT_NEEDED tag to only be emitted for a library that satisfies a symbol reference from regular
objects which is undefined at the point that the library was linked, or, if the library is not found in the DT_NEEDED lists of other
libraries linked up to that point, a reference from another dynamic library. --no-as-needed restores the default behaviour.
I haven't used it myself but sounds like what you're looking for.
g++ -o your_app -Wl,--as-needed -lframework -la -lb -lc -Wl,--no-as-needed
Edit (suggested by Hanno)
--warn-unresolved-symbols
If the linker is going to report an unresolved symbol (see the option --unresolved-symbols) it will normally generate an error.
This option makes it generate a warning instead.
Related
I've recently learnt that static linking and implicit linking are basically the same thing, just different nomenclature. My understanding is that when you statically (implicitely) link to a binary, you are by definition linking against a *.lib (windows) or *.a (linux) file, often using target_link_libraries in cmake. On the other hand when you explicitely link (using LoadLibrary on windows) you are by definition linking to a *.dll file (or *.so on linux) (and there is no corresponding cmake command because all the work is done inside the actual code).
However, in multiple places I've read people referring to statically/implicitely linking to a dll file, which has confused me. Clearly there is a hole in my knowledge somewhere and I was hoping somebody here could plug it.
Edit
Its been pointed out that this question refers mainly to windows, which it does. However, I am currently trying to produce cross platform code so I am still interested on how (or if) these concepts generalise to other platforms.
There are actually 3 different kinds of linking, not 2.
For UNIX:
Link against archive (aka static) library:
gcc main.o libfoo.a
link against dynamic (aka shared) library:
gcc main.o libfoo.so
Link against libdl, which allows you to dlopen arbitrary other shared libraries (which don't need to exist at the time of the link):
gcc main.o -ldl
Both 2 and 3 involve dynamic linker (and are using shared libraries), but to a different extent.
An equivalent exists on Windows: when you link against foo.lib, you are using either 1 or 2, depending on whether foo.lib contains actual code, or refers to foo.dll.
When you use LoadLibrary, you are in case 3.
I have a C++ file a.cpp with the library dependency in the path /home/name/lib and the name of the library abc.so.
I do the compilation as follows:
g++ a.cpp -L/home/name/lib -labc
This compiles the program with no errors.
However while running the program, I get the ERROR:
./a.out: error while loading shared libraries: libabc.so.0: cannot open shared object file: No such file or directory
However if before running the program, I add the library path as
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/name/lib;
and compile and run now, it works fine.
Why am I not able to link the library by giving it from the g++ command?
Because shared object libraries are linked at runtime - you either need to add the location to the library search path (as you've done), place it somewhere within an already existing path (like /usr/lib), or add a symbolic link to an existing library path that links to the location where it exists.
If you want to compile the library in so there is no runtime dependency, you'll need a static library (which would be abc.a) - note that this has numerous downsides, however (if the library is updated, you'll need to recompile your executable to incorporate that update).
Why am I not able to link the library by giving it from the g++ command?
You are able to link, and you did link the library succesfully. Otherwise you would not be able to build executable (in your case a.out). The problem you mixed 2 different things: linking with shared libraries and loading them at runtime. Loading shared libraries is a pretty complex concept and described pretty well here Program-Library-HOWTO read from 3.2.
You are linking dynamically, is the default behavior with GCC. LD_LIBRARY_PATH is used to specify directories where to look for libraries (is a way of enforce using an specific library), read: Program-Library-HOWTO for more info. There is also an ld option -rpath to specify libraries search path for the binary being compiled (this info is written in the binary and only used for that binary, the LD_LIBRARY_PATH affect other apps using the same library, probably expecting a new or old version).
Linking statically is possible (but a little tricky) and no dependency would be required (but sometimes is not recommended, because prevent the update of the dependent libraries, for example for security reason, in static linking your always are using the versions of the libraries you have when compiled the binary).
I have a limited knowledge of dynamic libraries and I usually have problems related to libraries that I do not understand.
I recently learned of libraries from google search and especially from the following links:
Difference between shared objects (.so), static libraries (.a), and DLL's (.so)?.
http://www.ibm.com/developerworks/library/l-dynamic-libraries/. That article was very useful in understanding the dynamic libraries and their usage:
If I understood well (correct me if I am wrong), there are two possible usages of shared objects:
dynamic linking: the shared object is automatically loaded by the dynamic linker when the program starts.
dynamic loading: the share object is loaded and used under the program control at runtime through the dynamic loading API (dlopen, dlerror, dlsym and dlclose). That option is useful for plugins.
If I got everything right, in the case of dynamic linking, all the symbols are verified at compilation time. This allows the compiler/linker to know exactly which shared object is effectively used by the program and which one is not used.
Now, it happens that the dynamic linker is always invoked at runtime even if the shared object is not used. It can be verified by linking an empty program against libraries that are not in locations searchable at runtime, and the execution will fail. Linking a program against library that is not actually used in the program can happen when there are updates and the use of a library is no longer necessary. It also happen when one isolates a part of the program for debugging, and link against all the libraries of the main program.
My question is: is there an option to ask the compiler/linker to not include reference to shared objects that do not have symbols referred to in the program?
Is there any issue that prevent the compiler from doing that?
The following posts share some similarities with the present question, but none of them has an accepted answer, nor an answer that satisfies my curiosity:
https://stackoverflow.com/questions/22617744/how-to-disable-the-runtime-checking-of-shared-object-if-they-are-not-used
Delay-Load equivalent in unix based systems
If you happen to use g++/ld there are a few suggestions spelled out on How to remove unused C/C++ symbols with GCC and ld?
For example:
gcc -Os -fdata-sections -ffunction-sections test.cpp -o test.o -Wl,--gc-sections
-dead_strip
-dead_strip_dylibs
However I'm actually not sure it's possible for the compiler to do this in the general case. Consider a dependent shared library that has a weak reference to the library that you want to remove from your link line: How would the compiler know that it's safe to remove the library and/or symbols at that point?
I was working with SFML, I compiled a little test program and added the linkage option -lsfml-audio. Then, I used ldd ./program to see the dynamic libraries it was linking to. Surprisingly, there were a lot, none of them had I manually selected in my makefile, nor using pkg-config --libs.
I started reading about shared libraries, and made a little example to solve my doubts. However, I have this question:
why some libraries need you to add the dependencies in your makefile
(either manually or using a script like pkg-config) and other
libraries automatically link their dependencies?
When you're creating your dynamic library, is just as easy as adding the proper -ldependency options in the g++ -shared ... command to avoid the user the hassle of manually adding the dependencies later on. Why many of the available libraries don't do that?
I guess it must be related to the ability of fine tuning which libraries get linked and such.
Shared libraries will generally link in their dependencies. However, static libraries are not capable of doing so. pkg-config --libs often includes all dependencies (direct and indirect) so that you can switch to static compilation by simply adding -static without having to add additional library dependencies as well.
Note that these excess direct dependencies are considered unwanted in some cases (eg, debian tries to avoid them in packaged binaries, as they make library soname transitions more traumatic than necessary). You can instruct the linker to strip direct dependencies from the final executable that aren't needed with the -Wl,--as-needed flag.
In my project there is a dependency on a static library (just called libsomething from now on) from a 3rd party. Recently, libsomething has become available in another version. My task is to provide my software with support for the old and the new version. Only one version of libsomething is used at run-time at any given time, but which version this is should be configurable between program runs.
I am using MSVC2005 on WinXP, a secondary objective is to become prepared to switch over to Linux and GCC.
Since both versions of libsomething are using the same symbols, linking them both into my executable is out of the question as the symbols of both versions are going to clash all over at link-time.
While I could create two executables (one linking against the old version, the other one using the new version), I cannot implement a decision on which executable to call in the final deployment environment (legacy reasons).
I came up with the idea of creating a dynamic library wrapper for each version of libsomething and linking them at run-time depending on some config file. With MSCV, this would mean going down the road of using LoadLibrary(), GetProcAddress(), etc., while on Linux I would have to use dlopen() and dlsym().
I understand that using libtool (i.e., libtldl) is wrapping this platform-dependency for using shared libraries. Is this an appropriate path to follow? Are there better (or, at least, different) ways? Do alternatives for libtldl exist as open-source?
I know you said you couldn't use two executables due to the decision of which to execute, but couldn't you exec back and forth between executables depending on which version is selected at configuration?
On Linux it would be easier for you to link to shared library and use symlinks to correct version - IMO it's much easier than using dlopen() + dlsym().
Thus you would create shared libraries for the old and new versions of your library:
g++ -shared -o libshared.so.1.1 -Wl,-soname=libshared.so.1 -L. -Wl,--whole-archive libstatic.a.old -Wl,-no-whole-archive
and
g++ -shared -o libshared.so.1.2 -Wl,-soname=libshared.so.1 -L. -Wl,--whole-archive libstatic.a.new -Wl,-no-whole-archive
Create the symlinks:
ln -s libshared.so.1.1 libshared.so.1
ln -s libshared.so.1 libshared.so
Build your application, linking it to the old version of the library. I suppose both versions are binary compatible (ABI not broken), but the new one could have some new symbols.
g++ -o myapp myapp.cpp -L. -lshared
Since shared library's SONAME is libshared.so.1 your application will depend on it and will search libshared.so.1 in paths from /etc/ld.so.conf or LD_LIBRARY_PATH
Before you run your application you may set the libshared.so.1 symlink to point to libshared.so.1.2 or libshared.so.1.1.
Little info about the linker options used here:
--whole-archive
For each archive mentioned on the command line after the --whole-archive option, include every object file in the archive in the
link, rather than searching the archive for the required object files. This is normally used to turn an archive file into a shared
library, forcing every object to be included in the resulting shared library. This option may be used more than once.
Two notes when using this option from gcc: First, gcc doesn't know about this option, so you have to use -Wl,-whole-archive.
Second, don't forget to use -Wl,-no-whole-archive after your list of archives, because gcc will add its own list of archives to your
link and you may not want this flag to affect those as well.
-soname=name
When creating an ELF shared object, set the internal DT_SONAME field to the specified name. When an executable is linked with a
shared object which has a DT_SONAME field, then when the executable is run the dynamic linker will attempt to load the shared object
specified by the DT_SONAME field rather than the using the file name given to the linker.
It has been few years now but I'd like to mention another solution for completeness. Instead of manual dlopen and dlsym you could generate simple stubs for all necessary functions and on first call (or at program startup) decide which library version is needed, load it and resolve the addresses.
You could write a script specifically tailored for your project or use Implib.so tool:
# This will generate mylib.so.init.c and mylib.so.tramp.S
# which implement stubs. These need to be linked to your
# executable.
$ implib-gen.py mylib.so
Implib.so is Linux-only atm but should be easily adaptable to Windows.