I have a following question about linking on Linux:
Suppose I have a class Foo that uses Qt. To build this class I'd have to use qmake to generate Makefile.
Later on I want to use this class Foo for a Perl module, which is a shared library. However, to build it I have to use Perl's MakeMaker to generate Makefile of it's own.
The way I'm doing it right now is that I build class Foo as a static library, and when building Perl module's shared library I'm linking it against Foo's static library.
The problem is that when building Perl module's shared library I have to link it against all those Qt libraries that Foo's static library is linked against.
So the question is:
Does this approach even make sense?!
Is it possible to build Foo's static library in a way that I wouldn't have to specify all it's dependencies when building the Perl module's shared library? (Because it is somewhat hard to add all those dependencies to module's Makefile)
Would it be any different if Foo's library was shared, not static?
1) You can't link a static library against a shared library.
2) You would not need to explicitly link against Foo's dependencies if Foo itself was a DSO
3) You can easily modify the Makefile.PL LIBS section to add extra linker dependencies.
4) Qt is a downright pain to link statically anyway. You're better off just doing the whole dynamic shebang unless you have specific version dependencies and OS/platform limitations. Hint: Even if you do have a 'static' build of Qt, this 'static' build won't include things which it might decides need to be present as loadable modules. Been there.
5) I believe there's a CPAN module (somewhat recent) which provides Qt4 bindings. I've never used it and don't know its status, but it may be worth checking out.
But your best bet is to make Foo a dynamic library.. everyone's happy then.
1) It depends on what your objective is.
2) If it's about minimizing the hassle building it, you could include Qt's static libraries as static to your Foo lib. All that matters is that the symbols you reference in Foo can be found at runtime. That way you won't need to include Qt libs in your PerlMake.
3) If it's about minimizing executable size you'd have to go for shared libraries. Then you should build everything as a shared-lib.
Building statically has the advantage to be independent of installed shared libraries on the target platform, while having the disadvantage of executable-size bloat and non re-usability of libraries (more memory needed to load two same executables).
Linking against shared has the advantage of smaller code size but at the same time the proper shared libs have to be installed on the target platform.
Related
I am working on a little project to understand the chain of compiler and linker better.
Suppose that i have the libraries libfoo.a and libbar.a. I want to create a library libmy.a, that acts like a wrapper or top level API to both libraries. The target is, that only libmy.a should be required to build an executable, that uses my defined wrapper functions. I created a cmake project and set up the library
cmake_minimum_required(VERSION 3.14)
project(Wrapper)
set(CMAKE_CXX_STANDARD 11)
add_library(my STATIC ${SOME_SRC_FILES})
#set up the lib/inc paths and libs to link
target_include_directories(my PUBLIC /path/to/Foo/inc/ /path/to/Bar/inc/)
target_link_directories(my PUBLIC /path/to/Foo/lib/ /path/to/Bar/lib)
target_link_libraries(my PUBLIC foo bar)
That works fine and there is no problem in compilation. However, if I try to reference the object from an external project, it tells me, that I have undefined references to the functions in libfoo.a and libbar.a. As far as I understand the problem, the linker only creates a declaration in the libmy.a, without including its definition from the external library. I checked this by opening libmy.a with the nm libmy.a command, where the used functions of the external libraries are declared, but undefined.
I came across one solution that used ar to combine multiple library files. However I would like to avoid such methods, because if it is not a single library, but a bunch of, say 10 libraries, it is not suitable to search each library for a definition and copy it into libmy.a. Just throwing all libraries together isn't a solution either, because the file will get too big.
Is it importand to note, that one of these library packages is CUDA?
I am sure there is a solution, but I was not able to find one. Any help would be appreciated
The target is, that only libmy.a should be required to build an executable
This is already an unconventional goal for static libraries.
Static libraries normally only contain the object code built from the source code for that library. Users of that library must also link to the libraries that your library requires, because the definitions have not been copied in to your library when it was built.
Tools like ar can be used to combine multiple static libraries together, since they are just archives of object code. The tool can not predict which object code the end-user will use though, so it will bundle entire libraries. Otherwise, the end user may be looking for a definition that you left out and then need to link in a 2nd copy of the dependency lib anyways.
If you want to provide a library that has everything the end-user needs, cut down to what your wrapper actually uses, you can build a shared library. Shared libraries are considered executable, so the compiler knows that any unreferenced object code is not going to be used, and it will not be included in the shared library.
You can force the entire static libraries to be included in shared libraries though.
On GCC you can use the linker argument: --whole-archive to ensure that all of the object code from the following libraries is included.
On MSVC, you can use the /WHOLEARCHIVE:<library file name> argument to do the same.
I have built a shared library that depends on a bunch of other libraries. But when loading that shared library, it needs the dependent libraries to be present in order to run properly. I want this shared library to be portable and hence want it to contain all the dependencies in itself. Is there a way in cmake to do this or what is the best solution here?
Actually this is not related with CMake, but with concepts of linking. You should link with static version of all your dependent libraries.
I've got a huge application here called HugeApp, it needs different libraries (which I've coded) and some of these libraries might need dependencies (other libs coming from the internet or ad hoc lib developped here).
I was wondering, if it was feasible and/or a good idea to hide some of these dependencies from HugeApp.
Let's say that you make a library in charge of doing the encrypted communication on a system, does the top application care and/or needs to know that there is some encryption libraries that are needed for this part (comms) of the system? It might be implementation specific... or not...
Thank you
There's no need for it to know, if you build those libraries as external DLL's then the external libraries are the only thing that care about the dependency. If you add a reference to the pre-built DLL then HugeApp doesn't need to know about the dependencies of the library (as long as they are either present in the library or the appropriate DLL or lib file is present so that your dll can make use of it). If anything your library can be another project altogether and you can include a reference to that in which case the project of your HugeApp only cares about that main reference and the other project will handle everything else.
If you enable /OPT:REF in linker optimizations it you will list which (if any) libraries that have no functions or data used by the project during link time. You can then remove them from the dependency list and link line in project settings. This will reduce the chance of removing a static library that is a dependency of another static library if any are present/used in your VS solution.
I have a software library and I used to create .a files, so that people can install them and link against them: g++ foo.o -L/path/to -llibrary
But now I often encounter third-party libraries where only .so files are available (instead of .a), and you just link against them without the -l switch, e.g. g++ foo.o /path/to/liblibrary.so.
What are the differences between these solutions? Should I prefer creating .so files for the users of my library?
Typically, libfoo.a is a static library, and libfoo.so is a shared library. You can use the same -L/-l linker options against either a static or shared. Or you can name the full path to the lib with static or shared. Often libraries are built both static and shared to provide application developers the choice of which they want.
All the code needed from a static lib is part of the final executable. This obviously makes it bigger, but it also means it's self-contained. Once it is compiled, you can run your app without the lib.
Code from a shared lib is not part of the executable. There are just some hooks in place to make the executable aware of the name of the lib it needs. In order to run your app, the shared lib has to be present in the lib search path (e.g. $LD_LIBRARY_PATH).
If you have two apps that share the same code, they can each link against a shared lib to keep the binary size down. If you want to upgrade parts of the app without rebuilding the whole thing, shared libs are good for that too.
Good overview of static, shared dynamic and loadable libraries at
http://www.yolinux.com/TUTORIALS/LibraryArchives-StaticAndDynamic.html
Some features that aren't really called out from comments I've seen so far.
Static linkage (.a/.lib)
Sharing memory between these compilation units is generally ok because they should(?will) all be using the same runtime.
Static linkage means you avoid 'dll hell' but the cost is recompilation to make use of any change at all. static linkage into Shared libraries (.so) can lead to strange results if you have more than 1 such shared library used by the final executable - global variables may exist multiple times and which one is used and when they are initialised can cause an entirely different hell.
The library will be part of the shipped product but obfuscated and not directly usable.
Shared/Dynamic libraries (.so/.dll)
Sharing memory between these compilation units can be hazardous as they may choose to use different runtime. This can mean you provide different Shared/Dynamic libraries based on the debug/release or single/multi threaded or...
Shared libraries (.so) are less prone to 'dll hell' then Dynamic libraries (.dll) as they include options for quite specific versioning.
Compiling against a .so will capture version information internal to the file (hard to fake) so that you get quite specific .so usage. Compiling against the .lib/.dll only gives a basic file name, any versioning is done managed by the developer (using naming or manually loading the library and checking version details by hand)
The library will have to ship with the final product (somebody else can pick it up and use it)
But now I often encounter third-party libraries where only .so files are available [...] and you just link against them without the -l switch, e.g. g++ foo.o /path/to/liblibrary.so.
JFYI, if you link to a shared library which does not have a SONAME set (compare with readelf -a liblibrary.so), you will end up putting the specified path of liblibrary.so into your target object (executable or another shared library), and which is usually undesired, for users have their own ideas of where to put a program and its associated files. The preferred way is to use -L/path/to -llibrary, perhaps together with -Wl,-rpath,/whatever/path/to if this is the final path (such pathing decisions are made by Linux distributions for example).
Should I prefer creating .so files for the users of my library?
If you distribute source code, the user will make the particular choice.
I have a bunch of projects that all could share a "common" static library of classes.
What confuses me is if I make a static library out of these classes and link against it in my projects that I still need the headers of the classes in the static library in my main projects.
What is the benefit of the static library then?
How do companies like Adobe deal with this?
Static libraries allow you to create a library and use that library in many projects.
The need for header files:
Since the project using the library is programmed and compiled independent of the library, that program needs to know the declaration of the things you're using. Otherwise how would your compiler know you're writing valid code?
A compiler only takes source code as input and produces output. It does not deal with compiled object files or static libraries on input.
The need for linking in the library:
So having the headers allows you to write valid code in your project, but when it comes to link time you'll need to provide the definition which is contained inside the static library.
The linker takes all object files (compiled code) and also all static libraries and produces an executable or binary.
More info about static libraries (benefits, comparing dynamic, etc...):
Amongst other things, it is nice to separate your project into libraries so that you don't end up with 1 huge monolithic project.
You do not need to distribute the source code (typically in the .cpp files) this way.
If you were to simply include all the .cpp files in every project that used the common library then you would have to compile the .cpp files each time.
An advantage of static libraries over dynamic libraries is that you can always be sure that your programs will be self contained and that they are using the correct version of the library (since they are compiled into the executable itself). You will also have a slight speed advantage over dynamic linking.
Disadvantages of static libraries over dynamic libraries include that your file sizes will be bigger because each executable needs their own copy, and that you can't swap out a different version of the library since it's not dynamically loaded.
Re your question: How do companies deal with this:
A typical company will make use of both static and dynamic libraries extensively.
The typical way you make use of a static library is to have a target in your Makefile (or whatever build system you use) that installs the headers into an appropriate location at the same time that it installs the library.
So, your static library ends up in /usr/local/lib, and the headers go into /usr/local/include or wherever.
Also, when compared with linking against object files, linking against static library may result is a smaller final executable. The reason for this is, if you don't call any of the functions from a particular object file (included in the static library), the linker will not include the code for those functions in you final executable. See Extraneous Library Linkage