I don't quite understand how the compilation of a library that uses another library work. From what I understand the is 4 cases :
1) Compiling a static library (A) that uses a static library (B) : The compilation would work, but linking against library A would not work because a static library is just an archive containing the .o files resulting from the compilation of A, and not from B. Is it true that when creating a static library, the compilation phase searches for function definition and find them in the header. At the linking phase, the compiler searches for function implementations in the library B and the compilation is successful if it finds them but it doesn't actually put the function implementations in the static library A. And that's why when linking against A it doesn't work
2) Compiling a static library that uses a dynamic library : I think both the creation of A and the use of A would work, but why is that? What does the compiler actually put in the dlls/so?
3) Compiling a dynamic library that uses a static library : Would this work?
4) Compiling a dynamic library that uses another dynamic library : Would this work too?
Thanks for your time.
If you create a library or program in C or C++, the build process consists of two steps:
Compiling each C/C++ file into an object file
Linking the object files and creating library or executable (If you create a static library, it isn't really linking; but let's use this word for simplicity.)
Compliation
If you compile code that makes use of a library, you need header files (.h) for the library. They declare the public functions and classes in the library. You don't need to binary files for compilation.
Linking
For the linking step, you then need the binary file of the static or dynamic library (.lib or .a for static libraries, .dll or .so for dynamic libraries).
Dependencies between libraries
If you create a static library, all your object files will be put into a new library file. Nothing from the consumed library (static or dynamic) will be included. But it will be needed when somebody uses your library. So your library isn't self contained.
If you create a dynamic library and consume a static library, the necessary code from the static library will be included into your dynamic library. The new dynamic library will be self contained as far as the consumed library is concerned.
If you create a dynamic library and consume a dynamic library, only a reference to the consumed library will be included. So to run the final product, both the new and the consumed library will need to be available.
Related
I have a static library in Linux that depends on some dynamic libraries (GL and SDL2). At link time, I want those to link automatically without needing to pass more arguments to the linker (gcc in my case). I saw a solution that adds a special file into the library named __.LIBDEP by passing -l to ar but it's not an elegant solution as it adds more arguments when linking (hint the dynamic library dependencies in the archive file).
I have a C++ project called testlib.pro (using Qtcreator) which will create a static library libtest.a.
The project is also included staticlib.a (example) and staticlib1.a. i.e using someother static libraries im creating one static library. After creating static library, Im creating C wrapper (testApi.c) to use the C++ code using the static library. I am compiling the testApi.c using the below option
gcc -o demo testApi.c -L ./testlib -ltest
But it is giving linker errors which stats that it requires the static libraries which i used to link the libtest.a. So i recompile the program with below comment and it works fine
gcc -o demo testApi.c -L ./testlib -ltest -lstatuclib -lstaticlib1
My understanding is If I ship the libtest.a to someother machine and try to compile testApi.c file it may requires staticlib.a and staticlib1.a in that machine. But I would like to use only newly created static library libtest.a. Am i missing any?
NOTE: I have included staticlib.a, staticlib1.a using -l option in my testlib.pro
If your library uses other static libraries then they will also need to be provided at link time.
There is an ugly way around it (on Linux). You can unpack existing static library (or several of them) and repack them into a new library. So you could, if you felt particularly frisky, unpack those libraries that yours depends on, then pack their contents along with your own stuff into a new library. Ugly, confusing, possibly causing all sorts of other problems, but if that is the way you want to go...
The static library concept is archaic in nature. When a program had a lot of modules, it was sometimes impossible to put them all in the command line for the linker to add them to the program. Also, for libraries, as the .o modules where all being included in the final executable, there had to be some mechanism to allow the linker select only the needed modules and not to include all of them in the final executable, or the executables will grow a lot including modules that the program will not use. Both things where solved with the introduction of dynamic shared objects, so using .a files is somewhat deprecated today and it is only used for statically linking programs.
Anyway, the algorithm to select the object modules in the linker is not recursive, so when it opens a .a library to search for dependent files to be included in the final executable, it searches only for .o (and probably .so, but I have not tested this), and it will ignore any .a file it finds in there. Many systems include an index file in the archive that has a mapping between provided identifiers, and the name of the module that provides them, so in one pass the compiler knows which archived objects need to be extracted. That index file should be appended (and rebuilt) in case a library (with its own index) where included in the file, so this justifies not using recursion at all in the library search.
The solution for this problem, is to link all those libraries you need to make the final executable, or as you have already been told, to extract the .o files in the library and put them in another library. There is still a third solution, that is: The linker allows you to specify a file that has options (and you can specify library names, and .o files you want it to scan) and it will read that file to check the set of libraries you want it to scan.
Another point is that the linker never includes a library as such. A library is just an archive (like a .tar or .zip file) in which the linker explores and extracts the files it needs, so there's no need to make the search algorithm recursive at all. And there's no difference between an archived file in a library and that same file out of the archive.
I am working on a little project to understand the chain of compiler and linker better.
Suppose that i have the libraries libfoo.a and libbar.a. I want to create a library libmy.a, that acts like a wrapper or top level API to both libraries. The target is, that only libmy.a should be required to build an executable, that uses my defined wrapper functions. I created a cmake project and set up the library
cmake_minimum_required(VERSION 3.14)
project(Wrapper)
set(CMAKE_CXX_STANDARD 11)
add_library(my STATIC ${SOME_SRC_FILES})
#set up the lib/inc paths and libs to link
target_include_directories(my PUBLIC /path/to/Foo/inc/ /path/to/Bar/inc/)
target_link_directories(my PUBLIC /path/to/Foo/lib/ /path/to/Bar/lib)
target_link_libraries(my PUBLIC foo bar)
That works fine and there is no problem in compilation. However, if I try to reference the object from an external project, it tells me, that I have undefined references to the functions in libfoo.a and libbar.a. As far as I understand the problem, the linker only creates a declaration in the libmy.a, without including its definition from the external library. I checked this by opening libmy.a with the nm libmy.a command, where the used functions of the external libraries are declared, but undefined.
I came across one solution that used ar to combine multiple library files. However I would like to avoid such methods, because if it is not a single library, but a bunch of, say 10 libraries, it is not suitable to search each library for a definition and copy it into libmy.a. Just throwing all libraries together isn't a solution either, because the file will get too big.
Is it importand to note, that one of these library packages is CUDA?
I am sure there is a solution, but I was not able to find one. Any help would be appreciated
The target is, that only libmy.a should be required to build an executable
This is already an unconventional goal for static libraries.
Static libraries normally only contain the object code built from the source code for that library. Users of that library must also link to the libraries that your library requires, because the definitions have not been copied in to your library when it was built.
Tools like ar can be used to combine multiple static libraries together, since they are just archives of object code. The tool can not predict which object code the end-user will use though, so it will bundle entire libraries. Otherwise, the end user may be looking for a definition that you left out and then need to link in a 2nd copy of the dependency lib anyways.
If you want to provide a library that has everything the end-user needs, cut down to what your wrapper actually uses, you can build a shared library. Shared libraries are considered executable, so the compiler knows that any unreferenced object code is not going to be used, and it will not be included in the shared library.
You can force the entire static libraries to be included in shared libraries though.
On GCC you can use the linker argument: --whole-archive to ensure that all of the object code from the following libraries is included.
On MSVC, you can use the /WHOLEARCHIVE:<library file name> argument to do the same.
I need to provide an SDK with a static library. Let's call it "libsdk.a".
This library should hopefully be standalone, meaning a simple example "example.cpp" could link against it without any other library, except the system ones.
Here my configuration :
cmake for all my 10 dependencies libraries. There is a static library (.a) generated for each of my module. These libraries contain only object files .o of the given module. Dependency tree is not flat, some of them depends on others.
a simple example "example.cpp", with a cmake, which compiles and works. At this level, cmakes generates a very complex link command to deal with deps tree.
external dependencies such as boost (some static libraries too)
At the moment, I tried this :
make an archive of the different .a generated but it doesn't work because linking against this lib tells me the archive a no index (even after ranlib). Yet, I remembered I could add .a libraries inside .a files without problems.
extract all .o object (with ar -x) files from all *.a files and recreated a "libsdk.a" with all these object files. It doesn't work either (unresolved references). Moreover, it includes all objects even those which are not needed... My working example takes 3.7M. This library is around 35M.
make a .so shared library. It seems to work but I would prefer having a static library.
compile all statically but then linker complains it doesn't find -lgcc_s. Ok I want to compile in static but not that far, and just my own libs together !
So my final question : is there any way I can generate my static lib containing all my other libs, and not system ones ?
BTW, another interesting topic on that :
Combining static libraries
Thank you for any piece of advice to open my mind !
What you are trying to do by hand is the job of the linker. While it's feasible, you shouldn't bother with it.
When you compile libsdk.a, make sure that all of its dependencies are linked statically. If you do this, libsdk.a should be standalone. Static linking means copying the code to the right places in the final executable, so anything that is linked statically will not need to be provided in an external file.
See this post on CMake mailing list. libutils.cmake attached to the message has MERGE_STATIC_LIBS() macro that does the job. On Linux (and all other Unixes except OSX) it uses ar to pack and unpack objects files.
I have a bunch of projects that all could share a "common" static library of classes.
What confuses me is if I make a static library out of these classes and link against it in my projects that I still need the headers of the classes in the static library in my main projects.
What is the benefit of the static library then?
How do companies like Adobe deal with this?
Static libraries allow you to create a library and use that library in many projects.
The need for header files:
Since the project using the library is programmed and compiled independent of the library, that program needs to know the declaration of the things you're using. Otherwise how would your compiler know you're writing valid code?
A compiler only takes source code as input and produces output. It does not deal with compiled object files or static libraries on input.
The need for linking in the library:
So having the headers allows you to write valid code in your project, but when it comes to link time you'll need to provide the definition which is contained inside the static library.
The linker takes all object files (compiled code) and also all static libraries and produces an executable or binary.
More info about static libraries (benefits, comparing dynamic, etc...):
Amongst other things, it is nice to separate your project into libraries so that you don't end up with 1 huge monolithic project.
You do not need to distribute the source code (typically in the .cpp files) this way.
If you were to simply include all the .cpp files in every project that used the common library then you would have to compile the .cpp files each time.
An advantage of static libraries over dynamic libraries is that you can always be sure that your programs will be self contained and that they are using the correct version of the library (since they are compiled into the executable itself). You will also have a slight speed advantage over dynamic linking.
Disadvantages of static libraries over dynamic libraries include that your file sizes will be bigger because each executable needs their own copy, and that you can't swap out a different version of the library since it's not dynamically loaded.
Re your question: How do companies deal with this:
A typical company will make use of both static and dynamic libraries extensively.
The typical way you make use of a static library is to have a target in your Makefile (or whatever build system you use) that installs the headers into an appropriate location at the same time that it installs the library.
So, your static library ends up in /usr/local/lib, and the headers go into /usr/local/include or wherever.
Also, when compared with linking against object files, linking against static library may result is a smaller final executable. The reason for this is, if you don't call any of the functions from a particular object file (included in the static library), the linker will not include the code for those functions in you final executable. See Extraneous Library Linkage