I have a Qt application that uses the boost logger library. I want to make it a standalone. However, after I managed the libraries for static linking, the application is still dependent on boost libraries.
The libraries that I included are:
..../boost_1_61_0_b1/stage/lib/libboost_regex.a
..../boost_1_61_0_b1/stage/lib/libboost_log_setup.a
..../boost_1_61_0_b1/stage/lib/libboost_thread.a
..../boost_1_61_0_b1/stage/lib/libboost_log.a
..../boost_1_61_0_b1/stage/lib/libboost_system.a
..../boost_1_61_0_b1/stage/lib/libboost_filesystem.a
The application compiles( after countless attempts). However, when I use ldd tool, it shows boost libraries on the dependency list.
Note: I have to define BOOST_ALL_DYN_LINK. Otherwise, it doesn't link.
Is there any way not to use this macro and overcome the dependency problem ? If not, what solutions do you suggest to circumvent this problem?
By default on modern UNIX-like systems gcc links with shared libraries by default. In order to force static linking you can either add -static to your linking command line (see the docs) or make sure gcc doesn't find the shared libraries but only finds the static libraries (e.g. move the shared libraries to a separate directory while you're linking your project). Note that -static will make all libraries linked statically, including libstdc++.
Alternatively, you can specify the static libraries directly, without the -l switch. You will have to use the full path to the libraries though, so instead of
gcc ... -lboost_log ...
you would write
gcc ... ..../boost_1_61_0_b1/stage/lib/libboost_log.a ...
In any case, you should not define BOOST_ALL_DYN_LINK because this macro means exactly the opposite - that you intend to link with Boost shared libraries.
Related
I am working on a little project to understand the chain of compiler and linker better.
Suppose that i have the libraries libfoo.a and libbar.a. I want to create a library libmy.a, that acts like a wrapper or top level API to both libraries. The target is, that only libmy.a should be required to build an executable, that uses my defined wrapper functions. I created a cmake project and set up the library
cmake_minimum_required(VERSION 3.14)
project(Wrapper)
set(CMAKE_CXX_STANDARD 11)
add_library(my STATIC ${SOME_SRC_FILES})
#set up the lib/inc paths and libs to link
target_include_directories(my PUBLIC /path/to/Foo/inc/ /path/to/Bar/inc/)
target_link_directories(my PUBLIC /path/to/Foo/lib/ /path/to/Bar/lib)
target_link_libraries(my PUBLIC foo bar)
That works fine and there is no problem in compilation. However, if I try to reference the object from an external project, it tells me, that I have undefined references to the functions in libfoo.a and libbar.a. As far as I understand the problem, the linker only creates a declaration in the libmy.a, without including its definition from the external library. I checked this by opening libmy.a with the nm libmy.a command, where the used functions of the external libraries are declared, but undefined.
I came across one solution that used ar to combine multiple library files. However I would like to avoid such methods, because if it is not a single library, but a bunch of, say 10 libraries, it is not suitable to search each library for a definition and copy it into libmy.a. Just throwing all libraries together isn't a solution either, because the file will get too big.
Is it importand to note, that one of these library packages is CUDA?
I am sure there is a solution, but I was not able to find one. Any help would be appreciated
The target is, that only libmy.a should be required to build an executable
This is already an unconventional goal for static libraries.
Static libraries normally only contain the object code built from the source code for that library. Users of that library must also link to the libraries that your library requires, because the definitions have not been copied in to your library when it was built.
Tools like ar can be used to combine multiple static libraries together, since they are just archives of object code. The tool can not predict which object code the end-user will use though, so it will bundle entire libraries. Otherwise, the end user may be looking for a definition that you left out and then need to link in a 2nd copy of the dependency lib anyways.
If you want to provide a library that has everything the end-user needs, cut down to what your wrapper actually uses, you can build a shared library. Shared libraries are considered executable, so the compiler knows that any unreferenced object code is not going to be used, and it will not be included in the shared library.
You can force the entire static libraries to be included in shared libraries though.
On GCC you can use the linker argument: --whole-archive to ensure that all of the object code from the following libraries is included.
On MSVC, you can use the /WHOLEARCHIVE:<library file name> argument to do the same.
I have a C++ file a.cpp with the library dependency in the path /home/name/lib and the name of the library abc.so.
I do the compilation as follows:
g++ a.cpp -L/home/name/lib -labc
This compiles the program with no errors.
However while running the program, I get the ERROR:
./a.out: error while loading shared libraries: libabc.so.0: cannot open shared object file: No such file or directory
However if before running the program, I add the library path as
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/name/lib;
and compile and run now, it works fine.
Why am I not able to link the library by giving it from the g++ command?
Because shared object libraries are linked at runtime - you either need to add the location to the library search path (as you've done), place it somewhere within an already existing path (like /usr/lib), or add a symbolic link to an existing library path that links to the location where it exists.
If you want to compile the library in so there is no runtime dependency, you'll need a static library (which would be abc.a) - note that this has numerous downsides, however (if the library is updated, you'll need to recompile your executable to incorporate that update).
Why am I not able to link the library by giving it from the g++ command?
You are able to link, and you did link the library succesfully. Otherwise you would not be able to build executable (in your case a.out). The problem you mixed 2 different things: linking with shared libraries and loading them at runtime. Loading shared libraries is a pretty complex concept and described pretty well here Program-Library-HOWTO read from 3.2.
You are linking dynamically, is the default behavior with GCC. LD_LIBRARY_PATH is used to specify directories where to look for libraries (is a way of enforce using an specific library), read: Program-Library-HOWTO for more info. There is also an ld option -rpath to specify libraries search path for the binary being compiled (this info is written in the binary and only used for that binary, the LD_LIBRARY_PATH affect other apps using the same library, probably expecting a new or old version).
Linking statically is possible (but a little tricky) and no dependency would be required (but sometimes is not recommended, because prevent the update of the dependent libraries, for example for security reason, in static linking your always are using the versions of the libraries you have when compiled the binary).
I am working on a collection of reusable libraries that need to be made available both as static libraries (.a & .lib) and as dynamic libraries (.so & .dll).
I want dependency management for the dynamic libraries to be as simple as possible (you only take one dynamic library for each bit of functionality that you need), so all of the functional dependencies that each dynamic library has are actually statically linked into it. Thus, the dynamic libraries offer their functionality to downstream clients dynamically, but their upstream dependencies are satisfied statically.
The upshot of all this is that all of my static libraries need to be compiled with -fPIC so that their code is suitable for linkage into a shared library. The same goes for any third-party library that we use. It has to be a static library, compiled with -fPIC.
(I could, I suppose, build both PIC and non-PIC variants of my libraries - but I really do not want to compile the libraries a third time for each target platform -- twice is quite (more than) enough!).
So, here is my problem:
I have been trying to compile boost_system as a static library with -fPIC, but I am not sure if I am succeeding:
/b2 --build-type=complete variant=release link=static threading=multi runtime-link=static --layout=versioned --cxxflags=-fPIC
This build produces .a files as output, as expected. However, when I try to link the boost static library into one of my shared libraries, I start getting an error message that indicates that boost_system is not Position Independent Code:
.../dependencies/external/boost/1_54_0/stage/lib/linux_x86_64/libboost_system-gcc46-s-1_54.a(error_code.o):
relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
However, I have (attempted) to build boost with -fPIC. Is there any test that I can use to determine if libboost_system is actually PIC code? I.e. if the problem is with building boost - or with linking it to my application.
I believe your problem can be solved by removing the command line option "runtime-link=static" which enables static linking of the C++ runtime libraries. Since you are building a dynamic shared library object, you want to avoid this behavior, especially if clients are to link to your library from different Linux OS configurations. However, the option "link=static" is ok and should remain.
I have mostly worked in Windows, and recently I started working in Linux. I have a doubt. I have used Visual Studio as IDE in Windows and used Makefile in Linux.
There are two types of libraries in Windows (VC++), static library (.lib) and DLL. It is quite obvious (isn't it?) if I link with lib file I am using static linking else dynamic linking.
Now when I use g++ compiler, why I need to explicitly mention -Bstatic/-static or Bdynamic/-dynamic flags. Because if file is .a file then I must be using static -linking and if file is .so I am using dynamic linking.
There are times when you want to "force" the compiler to do what it normally wouldn't. In particularly -static is useful when you are building against a library that may not be installed [or not have the same version installed] on another machine where you want your code to run.
the -Bdynamic is useful if you want to have ONE library linked statically, but not EVERY library that your code uses.
e.g. gcc -o myprog myprog.o -Wl,-Bstatic -lspecial -Wl,-Bdynamic
will link myprog using static linking for libspecial (which may be something that isn't widely distributed, for example something you have built yourself)
For general, local development, you don't need either.
In the GNU toolchain, and other Unix compilers, you generally don't specify the full path to a library; you give the linker a flag such as
-lfoo
and let it figure out whether it should link with libfoo.a, libfoo.so, and where these files are located. When only a static library is available, it will link with that.
So, you don't really need to specify -Bstatic except in a few specific cases. Dynamic linking is the default and if you don't want a custom library to be linked dynamically, then just don't build a .so out of it.
Besides, you can explicitly link with a static library by giving the full filename to the linker
gcc -o some_binary main.o libfoo.a
to get a dynamically linked binary with libfoo statically linked in.
I think you've got it backwards. Whether I'm linking with
a dynamic library or with a static one, under Windows, I need
a .lib file; they just aren't generated in the same manner,
and don't contain the same thing. Under Unix, when I link with
a shared object, I link directly against the .so file, and not
against some additional file generated at the same time.
The options -Bstatic and -Bdynamic only come into play when
both a .so and a .a are present, in the same directory; they
tell the linker which one to choose for libraries specified
using the -l option. In Windows, this situation cannot occur;
since you need a .lib for the .dll, you cannot have a .lib
to be statically linked in the same directory.
Because if file is .a file then I must be using static -linking and if file is .so I am using dynamic linking.
These filename "extensions" are only human conventions — there is no reason that they have to be .a and .so respectively. Quite rightly does GCC not force us into that convention.
I was working with SFML, I compiled a little test program and added the linkage option -lsfml-audio. Then, I used ldd ./program to see the dynamic libraries it was linking to. Surprisingly, there were a lot, none of them had I manually selected in my makefile, nor using pkg-config --libs.
I started reading about shared libraries, and made a little example to solve my doubts. However, I have this question:
why some libraries need you to add the dependencies in your makefile
(either manually or using a script like pkg-config) and other
libraries automatically link their dependencies?
When you're creating your dynamic library, is just as easy as adding the proper -ldependency options in the g++ -shared ... command to avoid the user the hassle of manually adding the dependencies later on. Why many of the available libraries don't do that?
I guess it must be related to the ability of fine tuning which libraries get linked and such.
Shared libraries will generally link in their dependencies. However, static libraries are not capable of doing so. pkg-config --libs often includes all dependencies (direct and indirect) so that you can switch to static compilation by simply adding -static without having to add additional library dependencies as well.
Note that these excess direct dependencies are considered unwanted in some cases (eg, debian tries to avoid them in packaged binaries, as they make library soname transitions more traumatic than necessary). You can instruct the linker to strip direct dependencies from the final executable that aren't needed with the -Wl,--as-needed flag.