Let's describe the following scenario:
I intend to create an application for the Linux platform
The application will contain a core shared/dynamic library and the executable. The library will act like an engine, providing common and essential classes and functions.
Now, I know that both the library and executable may change over time and I want to make sure that each executable knows what library is for it and in case the library version is different it will show a message about the library being two old (this could be a constant, abiding by the ABI rules, that will always contain the version and will be checked against the executable's version at the beginning of the entry point).
I have read the ABI specifications and rules and I don't like them (too strict). I don't want to have a single library on the user's system, which I update and try to keep compatible with older executables. I prefer, because I don't know how the interface will change and I want to reserve myself the freedom of doing any change later, to have each executable with its own library version on the user's machine. For example:
Today the user has downloaded the executable "MyApp1" and the version 1.0.0 of the library "MyCoreLib".
One week later I created a new application based on a new version of the library. The user downloads the version 1.1.0 of the library "MyCoreLib" and a new executable called "MyApp2". So, we have 2 different executables and two different versions of the same library (the same name).
"MyApp1" should still be linked to the version 1.0.0 of "MyCoreLib" while "MyApp2" should be linked to the new version 1.1.0.
One week later I create a new application based on the older version 1.0.0 of the library. As the library already exists on the user's computer then we will use that.
Now we have two executables using the first version of the library and one executable using the newer version, and so on.
In Linux, we have the version numbers at the end of the .so libraries. Does this give us the possibility to have multiple versions of the same library and each executable linked to its own library version?
This should be a entry point:
http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html
Yes. Read up on SONAMEs and RPATH.
Read Drepper's paper: How To Write Shared Libraries
The version 1.0 of your library should generally change only for incompatible API changes.
Related
Say I have a C++ binary that was compiled with a version of gcc say 4.4.x, which is used on a client linux box.
If I want to upgrade my compiler to use a newer one, say 4.9.3 (because I want to use C++11):
What kind of things would need to be upgraded on the client box to run this new binary? (e.g. .so libraries)
And how would one find this out?
What kind of things would need to be upgraded on the client box to run this new binary?
You will need to ship two shared libraries with your application: libgcc_s and libstdc++. See https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html for more details.
You can ship these libraries in the same directory with your executables if you link using $ORIGIN. See https://stackoverflow.com/a/4742034/412080 for more details.
And how would one find this out?
Run ldd and readelf -d on your executables to see what libraries they need.
I'm guessing from the fact that you mention GCC 3.4 that your client system is running RedHat/CentOS/Scientific Linux 4 which is so old that even Red Hat ended support for it three years ago. If you were running any newer version then you would have been able to take advantage of the Developer Toolsets which include a modified version of GCC that statically links newer parts of the standard library into your binary so that it can run on legacy systems without newer glibc/libstdc++ runtimes.
There are two mechanisms to test compatibility of shared libraries:
The SONAME: a canonical name for the library that is used by the linker to reference the library. You can query the list of required libraries for every ELF object (executable or library) with the ldd command, and you need to do this recursively for each referenced library to get a full list of libraries needed.
The symbol version information. This is an additional constraint that allows adding functionality to existing libraries by introducing version requirements per symbol used -- a program only using symbols that have existed for ages will require a lower minimum version of the library than one that uses new functionality.
Both of these need to be fulfilled in order for the program to run.
The typical approach of Linux distributions is to keep a mapping of SONAME to package name (as multiple versions differing in SONAME can be installed concurrently), and a table of versioned symbols to the package revision these were introduced. The appropriate package development tools for your distribution should be able to create a list of dependency specifications that matches your program's requirements; if they fail to do so because symbols are unknown, it is likely that this version cannot be supported on that release of the distribution.
An alternative could be to link statically all libraries (notably libstdc++) other than the libc (and libm and libdl if you use them); then the produced ELF executable depends only of the libc; however, with ancient enough kernels or libc (on the target machine) even that might fail to work...
For details, read Drepper's paper: How To Write Shared Libraries
I am compiling a shared library using g++ on SUse Linux with cmake that depends on libicuuc.so and friends.
Suse has libicuuc.so, libicuuc.so.46 and libicuuc.so.46.1 in /usr/lib.
Now when I use ldd to list the dependencies of my library it tells me that it depends on libicuuc.so.46.
Since I want to distribute my library in binary form (it takes about 45 minutes to compile on a fast PC) this dependency is a problem. The target PCs have different versions of libicuuc.so.
What can I do so that my library depends on libicuuc.so and not libicuuc.so.46?
I tried to remove the so.46 versions in my /usr/lib folder before compiling but libicuuc.so depends on libicudata.so.46 so I keep that dependency on a 46 version what I try to avoid.
Read about external library versioning here.
What can I do so that my library depends on libicuuc.so and not libicuuc.so.46?
You can't do anything about that. The libicuuc.so that you have has SONAME set to libicuuc.so.46, and the linker dutifully records that dependency (as it should).
If developers release libicuuc.so.47, they would do so because the new library is not ABI-compatible with the old one (at least that's what they should do if they are not clueless).
If your library loaded libcuuc.so.47 (as you want it to), it would most likely crash due to the ABI incompatibility. Or worse: corrupt your end user's data. So achieving your desired result would get you into worse trouble than what you have now (not running is better than randomly crashing or corrupting data).
Update:
The libicuuc.so documentation explicitly states that "Binary compatibility is for versions that share the same major+minor number."
That means: you can't link a library compiled with version 4.6 (SONAME libicuuc.so.46) and expect it work with version 4.7.
You must either rebuild your library for each version of ICUUC, or distribute matching libicuuc.so.NN with your library (and hope that the user is not already using some other version of libicuuc).
Another possible alternative: statically link libicuuc.a into your library, and hide all of libicuuc.a symbols so they don't conflict with anything else. Note: this has licensing implications.
I'm writing an application that uses both Intel's TBB library, and an API from a company called Maplink, which also uses TBB. The problem is that both my application and the Maplink API want to load TBB.dll from the directory containing my application's binary. The version of TBB.dll that Maplink provided with their API differs from the one my application requires, and they can't both co-exist in the application's executable directory. Do I have any option here other than statically linking TBB into my application so that it doesn't try to load the wrong version of TBB.dll that the Maplink API is using?
In the real world, it is a bad idea to mix different versions of the same DLL. You should really try and get your platform aligned. It is not called package hell for nothing.
That being said, it is very much up to the TBB.dll if it allows for multiple versions at once. You might be able to statically link your code against your version of TBB, but in doing so you will need to make sure the statically linked-in symbols are not dynamically visible (a compiler collection dependant linker option). The code that you have that depends on TBB must probably also be linked in a separate linker step from the one that includes linking to maplink. And the application will need to be linked without relinking against TBB.dll.
At least that is how it could work for so files in Linux.
As mentioned in the comments, you may put the newer version of tbb.dll into your application directory, and it should work properly for both the application and the 3rd party library it uses. For example, the recent version - TBB 4.2 - is binary compatible with old versions back to TBB 2.0.
We have been porting our Java API to C++, and creating a library file (.a file) for Linux. Our API has dependency on the boost framework and also on log4cxx. (We discussed with the first user for this, and they are okay with those dependencies).
When I compile a sample application which uses our library file, it is linking against .so files for the dependencies (libboost_system.so, libboost_thread.so, and liblog4cxx.so). When the application runs, it needs to find the specific versioned file names for each of those .so files, e.g. liblog4cxx.so.10.
If the user is using a newer version of boost (for example), would s/he be able to link against their local version and run with that newer version (assuming bacwards compatibility)? Is there some other way to deal with versions/dependencies like these (i.e. do you try to link in the external referenced libraries to your own library)?
Yes, the program can link to newer libraries with a compatible interface. The numbers in the filename encode the library Application Binary Interface (ABI).
FreeBSD's handbook has more information about library versioning. The basic rules are:
Start from 1.0
If there is a change that is backwards compatible, bump minor number (note that ELF systems ignore the minor number)
If there is an incompatible change, bump major number
The linker will sort out the details to use the newest library version with a compatible interface. There is also more information available here on stackoverflow.
I'm writing a cross-platform application which is not GNU GPL compatible. The major problem I'm currently facing is that the application is linked dynamically with glibc and libstdc++, and almost every new major update to the libraries are not backwards compatible. Hence, random crashes are seen in my application.
As a workaround, I distribute binaries of my application compiled on several different systems (with different C/C++ runtime versions). But I want to do without this. So my question is, keeping licensing and everything in mind, can I link against glibc and libstdc++ statically? Also, will this cause issues with rtld?
You don't need to.
Copy the original libraries you linked against to a directory (../lib in this example) in your application folder.
Like:
my_app_install_path
.bin
lib
documentation
Rename you app for something like app.bin. Substitute your app for a little shell script that sets the enviroment variable LD_LIBRARY_PATH to the library path (and concatenate the previous LD_LIBRARY_PATH contents, if any). Now ld should be able to find the dynamic libraries you linked against and you don't need to compile them statically to your executable.
Remember to comply with the LGPL adding the given attribution to the libraries and pointing in the documentation where the source can be downloaded.
glibc is under the LGPL. Under section 6. of LGPL 2.1, you can distribute your program linked to the library provided you comply with one of five options. The first is to provide the source code of the library, along with the object code (source is optional, not required) of your own program, so it can be relinked with the library. You can alternatively provide a written offer of the same. Your own code does not have to be under the LGPL, and you don't have to release source.
libstdc++ is under the GPL, but with a major exception. You can basically just distribute under the license of your choice without providing source for either your own code or libstdc++. The only condition is that you compile normally, without e.g. proprietary modifications or plugins to GCC.
IANAL, and you should consider consulting one if you need real legal advice.
Specifying the option -static-libgcc to the linker would cause it to link against a static version of the C library, if available on the system. Otherwise it is ignored.
I must question what the heck you are doing with the poor library functions?
I have some cross platform software as well. It runs fine on Linux systems of all sorts. Build with the oldest version of software that you want to support. The glibc and libstdc++ libraries are really very backward compatible.
I have built on CentOS 4 and run it on RHEL 6 beta. No problems.
I can build on stable Debian and run it on testing.
Now, I do sometimes have trouble with some libraries if I try to build on, say old Debian and try to run it on CentOS 5.4. That is usually due to distribution configuration choices that are different, like choosing threading or non-threading.