I am compiling a shared library using g++ on SUse Linux with cmake that depends on libicuuc.so and friends.
Suse has libicuuc.so, libicuuc.so.46 and libicuuc.so.46.1 in /usr/lib.
Now when I use ldd to list the dependencies of my library it tells me that it depends on libicuuc.so.46.
Since I want to distribute my library in binary form (it takes about 45 minutes to compile on a fast PC) this dependency is a problem. The target PCs have different versions of libicuuc.so.
What can I do so that my library depends on libicuuc.so and not libicuuc.so.46?
I tried to remove the so.46 versions in my /usr/lib folder before compiling but libicuuc.so depends on libicudata.so.46 so I keep that dependency on a 46 version what I try to avoid.
Read about external library versioning here.
What can I do so that my library depends on libicuuc.so and not libicuuc.so.46?
You can't do anything about that. The libicuuc.so that you have has SONAME set to libicuuc.so.46, and the linker dutifully records that dependency (as it should).
If developers release libicuuc.so.47, they would do so because the new library is not ABI-compatible with the old one (at least that's what they should do if they are not clueless).
If your library loaded libcuuc.so.47 (as you want it to), it would most likely crash due to the ABI incompatibility. Or worse: corrupt your end user's data. So achieving your desired result would get you into worse trouble than what you have now (not running is better than randomly crashing or corrupting data).
Update:
The libicuuc.so documentation explicitly states that "Binary compatibility is for versions that share the same major+minor number."
That means: you can't link a library compiled with version 4.6 (SONAME libicuuc.so.46) and expect it work with version 4.7.
You must either rebuild your library for each version of ICUUC, or distribute matching libicuuc.so.NN with your library (and hope that the user is not already using some other version of libicuuc).
Another possible alternative: statically link libicuuc.a into your library, and hide all of libicuuc.a symbols so they don't conflict with anything else. Note: this has licensing implications.
Related
I am trying to compile various programs such as MariaDB with a musl toolchain. That is, I don't want any dependencies on glibc or GNU's linker after compilation has finished.
Thus far, I have been using musl's GCC wrapper, musl-gcc to compile things. But, with larger programs like MariaDB I struggle to get all the necessary libraries and headers and symlinking or adding environment variables for the compilation doesn't really help.
I see mention of building a cross-compiler targeting musl libc with additional documentation and code at this GitHub repo. From the documentation on the cross-compiler:
This gives you a full, relocatable musl-targeting toolchain, with C++ support and its own library paths you can install third-party libraries into.
It sounds like this could help me, but I am not sure how this is very different from musl's GCC wrapper, which as I understand, just alters where GCC looks for libraries and headers etc.
Ultimately, I am unsure how different this cross-compiler really is from the GCC wrapper and if it would be useful in my case. Why would I need my own library paths to install third-party libraries into when I can just symlink to existing libraries and use the GCC wrapper? Is the cross-compiler the way I should be compiling things, especially bigger code bases?
All the wrapper does is remove the default library and include paths and add replacement ones. Otherwise, it relies on the compiler's view of the ABI being sufficiently matched that a GCC that thinks it's targeting glibc (*-linux-gnu) works with a musl (*-linux-musl) target.
A full GCC toolchain has a number of "target libraries" - libraries that are linked into the output program to provide some functionality the compiler offers. This includes libgcc (software multiply or divide, software floating point, etc. according to whether the target arch needs these things, and unwinding support for exception handling), libstd++ (the C++ standard library), and a number of other things like libgomp (GNU OpenMP runtime implementation for use with #pragma OMP ...). In theory all of these possibly depend on the specific target ABI, including the libc, and potentially link to symbols from libc. In practice, for the most part libgcc doesn't, and is "safely" reusable with a different libc as long as the basic type definitions and a few other ABI things match.
For the other libraries with more complex dependencies on libc, it's generally not expected that a build against one libc would work with a different one. musl libc provides some degree of ABI-compat for using glibc-linked libraries, so there's some hope that they'd work anyway, but you'd need to copy them to the new library path. However, GCC's C++ standard library headers also seem to have some heavy dependency on the (libc) system headers for the target, and when setup for glibc do not seem to work with musl. There is probably some way to make this work (i.e. to add C++ support to the wrapper), but it's an open problem exactly how to do it.
Even if this could be made to work, though, the deeper you go, the more reasons you're likely to find that you'd rather just have a proper cross-compiler toolchain that knows it's targeting musl. And nowadays these are easy enough to build. But if you find you're needing additonal third party libraries too, and don't want to build them yourself, it probably makes more sense to just use a Docker image (or other container) with a distro that provides library binaries for you.
Say I have a C++ binary that was compiled with a version of gcc say 4.4.x, which is used on a client linux box.
If I want to upgrade my compiler to use a newer one, say 4.9.3 (because I want to use C++11):
What kind of things would need to be upgraded on the client box to run this new binary? (e.g. .so libraries)
And how would one find this out?
What kind of things would need to be upgraded on the client box to run this new binary?
You will need to ship two shared libraries with your application: libgcc_s and libstdc++. See https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html for more details.
You can ship these libraries in the same directory with your executables if you link using $ORIGIN. See https://stackoverflow.com/a/4742034/412080 for more details.
And how would one find this out?
Run ldd and readelf -d on your executables to see what libraries they need.
I'm guessing from the fact that you mention GCC 3.4 that your client system is running RedHat/CentOS/Scientific Linux 4 which is so old that even Red Hat ended support for it three years ago. If you were running any newer version then you would have been able to take advantage of the Developer Toolsets which include a modified version of GCC that statically links newer parts of the standard library into your binary so that it can run on legacy systems without newer glibc/libstdc++ runtimes.
There are two mechanisms to test compatibility of shared libraries:
The SONAME: a canonical name for the library that is used by the linker to reference the library. You can query the list of required libraries for every ELF object (executable or library) with the ldd command, and you need to do this recursively for each referenced library to get a full list of libraries needed.
The symbol version information. This is an additional constraint that allows adding functionality to existing libraries by introducing version requirements per symbol used -- a program only using symbols that have existed for ages will require a lower minimum version of the library than one that uses new functionality.
Both of these need to be fulfilled in order for the program to run.
The typical approach of Linux distributions is to keep a mapping of SONAME to package name (as multiple versions differing in SONAME can be installed concurrently), and a table of versioned symbols to the package revision these were introduced. The appropriate package development tools for your distribution should be able to create a list of dependency specifications that matches your program's requirements; if they fail to do so because symbols are unknown, it is likely that this version cannot be supported on that release of the distribution.
An alternative could be to link statically all libraries (notably libstdc++) other than the libc (and libm and libdl if you use them); then the produced ELF executable depends only of the libc; however, with ancient enough kernels or libc (on the target machine) even that might fail to work...
For details, read Drepper's paper: How To Write Shared Libraries
We have been porting our Java API to C++, and creating a library file (.a file) for Linux. Our API has dependency on the boost framework and also on log4cxx. (We discussed with the first user for this, and they are okay with those dependencies).
When I compile a sample application which uses our library file, it is linking against .so files for the dependencies (libboost_system.so, libboost_thread.so, and liblog4cxx.so). When the application runs, it needs to find the specific versioned file names for each of those .so files, e.g. liblog4cxx.so.10.
If the user is using a newer version of boost (for example), would s/he be able to link against their local version and run with that newer version (assuming bacwards compatibility)? Is there some other way to deal with versions/dependencies like these (i.e. do you try to link in the external referenced libraries to your own library)?
Yes, the program can link to newer libraries with a compatible interface. The numbers in the filename encode the library Application Binary Interface (ABI).
FreeBSD's handbook has more information about library versioning. The basic rules are:
Start from 1.0
If there is a change that is backwards compatible, bump minor number (note that ELF systems ignore the minor number)
If there is an incompatible change, bump major number
The linker will sort out the details to use the newest library version with a compatible interface. There is also more information available here on stackoverflow.
I've been having a lot of conceptual issues with Microsoft's CRT. For any project you have to compile all required libraries to link against the same version of the CRT.
The first problem is when your project statically links against the CRT (/MT). Then all the dependant libraries must also link their own CRT statically. So each library has its own version of - for example - malloc(). If you compiled one of the libraries last year on system A, that CRT version may be different than the one you're currently using on another system B with service pack 3+. So if you're freeing objects allocated by the library you may run into problems.
So it seems dynamically linked CRT is the way to go (/MD). With dlls all the libraries would get the current implementation of the CRT on the system. Except that with Microsoft's Side by Side mechanism that's not what happens. Instead you get the CRT version that's stamped on the library you compiled and that version of the DLL is supplied to that library. So exactly the same problem I described before can occur. You compile a library on system A a year ago against one CRT. A year later there's a new version with upgrade. Your main program gets the DLL with one version of the CRT, the library gets the DLL with another version of CRT, same problem can occur.
So what do you do? I realize cross library memory allocation is frowned upon. But you can ignore the malloc example and come up with another one. Do you have every developer recompile every dependant library on their machine to make sure everything does use the same CRT? Then for the release you recompile every library again?
How does this work on Linux? That's my main interest. Is there a CRT supplied with GCC or the Linux system itself comes with CRT libraries? I've never seen the CRT linked explicitly in Makefils.
On Linux, what CRT do dynamic libraries link against? The most current one on the machine, or is it more "side by side" mechanism.
On the Linux side I think there are two basic parts of the standard library that are at issue: We have the C-runtime part, which should pretty much be ABI compatible forever. Effectively whichever version links at final link time should be fine, and you can redistribute any needed shared library with your binary if it's an older version needed for compatibility. Usually the libraries just sit side-by-side on *NIX systems.
Secondly, you have the C++ libraries. These are pretty much guaranteed to not be ABI compatible in any way, so you must rebuild every single component of a final binary against the same version of the C++ library. There's just no way around it unfortunately because otherwise you could wind up with a variety of mismatches. This is why many open source libraries don't even bother with premade library binaries: Everyone needs to build their own copy to make sure that it will properly link into their final application code.
I'm writing a cross-platform application which is not GNU GPL compatible. The major problem I'm currently facing is that the application is linked dynamically with glibc and libstdc++, and almost every new major update to the libraries are not backwards compatible. Hence, random crashes are seen in my application.
As a workaround, I distribute binaries of my application compiled on several different systems (with different C/C++ runtime versions). But I want to do without this. So my question is, keeping licensing and everything in mind, can I link against glibc and libstdc++ statically? Also, will this cause issues with rtld?
You don't need to.
Copy the original libraries you linked against to a directory (../lib in this example) in your application folder.
Like:
my_app_install_path
.bin
lib
documentation
Rename you app for something like app.bin. Substitute your app for a little shell script that sets the enviroment variable LD_LIBRARY_PATH to the library path (and concatenate the previous LD_LIBRARY_PATH contents, if any). Now ld should be able to find the dynamic libraries you linked against and you don't need to compile them statically to your executable.
Remember to comply with the LGPL adding the given attribution to the libraries and pointing in the documentation where the source can be downloaded.
glibc is under the LGPL. Under section 6. of LGPL 2.1, you can distribute your program linked to the library provided you comply with one of five options. The first is to provide the source code of the library, along with the object code (source is optional, not required) of your own program, so it can be relinked with the library. You can alternatively provide a written offer of the same. Your own code does not have to be under the LGPL, and you don't have to release source.
libstdc++ is under the GPL, but with a major exception. You can basically just distribute under the license of your choice without providing source for either your own code or libstdc++. The only condition is that you compile normally, without e.g. proprietary modifications or plugins to GCC.
IANAL, and you should consider consulting one if you need real legal advice.
Specifying the option -static-libgcc to the linker would cause it to link against a static version of the C library, if available on the system. Otherwise it is ignored.
I must question what the heck you are doing with the poor library functions?
I have some cross platform software as well. It runs fine on Linux systems of all sorts. Build with the oldest version of software that you want to support. The glibc and libstdc++ libraries are really very backward compatible.
I have built on CentOS 4 and run it on RHEL 6 beta. No problems.
I can build on stable Debian and run it on testing.
Now, I do sometimes have trouble with some libraries if I try to build on, say old Debian and try to run it on CentOS 5.4. That is usually due to distribution configuration choices that are different, like choosing threading or non-threading.