I am doing some research whether to compile C++ libraries static or dynamic for a new project. (I have read this answer, also). I saw that on OpenBSD and FreeBSD the system libraries are a lot bigger (5-7 MB) than on Linux (1.5 MB) because they are not stripped. So I have 2 questions:
The OpenBSD libestdc++.so is any different than libstdc++.so? I didn't find any info on it, google just corrects me, removing the "e" letter from the word.
Why these libraries are so big on BSD? If I would like to deploy or statically link them they will be huge. Is there a workaround for this?
Thank you.
On OpenBSD, libstdc++ is the base c++ library (GCC 4.2), libestdc++ is installed from ports (GCC 4.9 or 6). The libraries are installed with symbols on OpenBSD, you can strip the symbols with strip -s libwhatever.so.
Related
Say I have a C++ binary that was compiled with a version of gcc say 4.4.x, which is used on a client linux box.
If I want to upgrade my compiler to use a newer one, say 4.9.3 (because I want to use C++11):
What kind of things would need to be upgraded on the client box to run this new binary? (e.g. .so libraries)
And how would one find this out?
What kind of things would need to be upgraded on the client box to run this new binary?
You will need to ship two shared libraries with your application: libgcc_s and libstdc++. See https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html for more details.
You can ship these libraries in the same directory with your executables if you link using $ORIGIN. See https://stackoverflow.com/a/4742034/412080 for more details.
And how would one find this out?
Run ldd and readelf -d on your executables to see what libraries they need.
I'm guessing from the fact that you mention GCC 3.4 that your client system is running RedHat/CentOS/Scientific Linux 4 which is so old that even Red Hat ended support for it three years ago. If you were running any newer version then you would have been able to take advantage of the Developer Toolsets which include a modified version of GCC that statically links newer parts of the standard library into your binary so that it can run on legacy systems without newer glibc/libstdc++ runtimes.
There are two mechanisms to test compatibility of shared libraries:
The SONAME: a canonical name for the library that is used by the linker to reference the library. You can query the list of required libraries for every ELF object (executable or library) with the ldd command, and you need to do this recursively for each referenced library to get a full list of libraries needed.
The symbol version information. This is an additional constraint that allows adding functionality to existing libraries by introducing version requirements per symbol used -- a program only using symbols that have existed for ages will require a lower minimum version of the library than one that uses new functionality.
Both of these need to be fulfilled in order for the program to run.
The typical approach of Linux distributions is to keep a mapping of SONAME to package name (as multiple versions differing in SONAME can be installed concurrently), and a table of versioned symbols to the package revision these were introduced. The appropriate package development tools for your distribution should be able to create a list of dependency specifications that matches your program's requirements; if they fail to do so because symbols are unknown, it is likely that this version cannot be supported on that release of the distribution.
An alternative could be to link statically all libraries (notably libstdc++) other than the libc (and libm and libdl if you use them); then the produced ELF executable depends only of the libc; however, with ancient enough kernels or libc (on the target machine) even that might fail to work...
For details, read Drepper's paper: How To Write Shared Libraries
According to the gcc ABI policy, gcc 4.4.7 should depend on libstdc++ 6.0.13. As far as I understood, the compiler version and the libstdc++ version are deeply interrelated and can't be swapped, so it came to me as a surprise to discover the following facts:
CentOS 5.8 somehow manages to have a gcc44 package that links against 6.0.8, apparently coming with the default system (which is based on gcc-4.1.2)
that the libstdc++.so in the compiler directory (/usr/lib/gcc/x86_64-redhat-linux6E/4.4.7, where I expected to find a libstdc++-6.0.13) is not a link to a shared object of any sort, but a text file containing INPUT ( -lstdc++_nonshared /usr/lib64/libstdc++.so.6 )
What kind of magic is going on here?
Specifically:
How could they provide a gcc 4.4.7 that links against an older version of libstdc++? I thought it was not possible.
what is this stdc++_nonshared library?
I didn't know a .so file could contain that text. Who parses it (dynamic linker I guess) and what are its specifications and consequences?
How far can this magic go? Can I use gcc4.7 with libstdc++ 6.0.3? What is the spectrum of compatibility
What kind of magic is going on here?
This is mostly the magic of ELF symbol versioning. The best reference I've seen on this is How to Write Shared Libraries by Ulrich Drepper, specifically section 3.
How could they provide a gcc 4.4.7 that links against an older version of libstdc++? I thought it was not possible.
The libstdc++ developers are careful about ABI compatibility. This makes it fairly straightforward to see what are the new symbols introduced between two different versions of libstdc++.
Looking at libstdc++'s ABI compatibility page, you can see that GCC 4.1's libstdc++ has GLIBCXX_3.4.8 and the stock (FSF release) of GCC 4.4 has GLIBCXX_3.4.13. It's then possible to look at the libstdc++ version scripts to see which symbols were added between these two versions.
A more empirical route for looking at symbol versions is to use a tool like readelf. E.g. on my Ubuntu system readelf --dyn-syms --wide /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | c++filt shows output like this (just a couple of selected shortish lines):
223: 000000000006dbe0 2121 FUNC GLOBAL DEFAULT 12 std::ios_base::Init::Init()##GLIBCXX_3.4
...
226: 00000000002ed920 1 OBJECT GLOBAL DEFAULT 27 std::adopt_lock##GLIBCXX_3.4.11
Note the symbol versions at the end of the symbol name, after the ##.
what is this stdc++_nonshared library?
Joining the dots from the above paragraph, stdc++_nonshared is a static library which contains the symbols which would be in the stock libstdc++.so.6.0.13 but not in libstdc++.so.6.0.8.
I didn't know a .so file could contain that text. Who parses it (dynamic linker I guess) and what are its specifications and consequences?
The regular linker parses this, not the runtime linker. See the binutils documentation for INPUT in a linker script. As a consequence, linking with this special libstdc++.so.6.0.13 will cause the linker to dynamically link against libstdc++.so.6.0.8, and statically link against the library which contains the extra symbols.
The downside of this is that an application compiled this way will be a little more bloated than the same application which is dynamically linked to only a stock libstdc++.so.6.0.13 which contains all the necessary symbols. The upside is that features from the newer standard library can be used on standard installations of older systems, without independent software vendors needing to ship their own copies of libstdc++.
How far can this magic go? Can I use gcc4.7 with libstdc++ 6.0.3? What is the spectrum of compatibility
Now that you know what the magic is, probably you can appreciate that this version of GCC 4.4 you have is a special version made just for Centos 5 (well, originally Red Hat 5). So probably you can create a GCC 4.7 which will produce binaries that only require libstdc++ 6.0.3, but it will require a non-trivial amount of work patching your GCC tree to put the correct symbols into stdc++_nonshared.a. You can at least look at the sources for the gcc44 RPM to see how Red Hat did it there.
Red Hat seems to have continued this approach with their Red Hat Developer Toolset. At the time of writing, GCC 4.8.1 is the newest compiler which can build binaries for EL5 which still only require libstdc++.so.6.0.8 to be installed on the target system. This is excellent for developers who want to write in C++11, but need to be able to deploy on EL5 or EL6.
I want to build my program with LSB C++ Compiler from the Linux Standard Base http://www.linuxfoundation.org/collaborate/workgroups/lsb. Program depends on the Boost library, built with gcc 4.4 version. Compilation fails. Is it possible to build the Boost library with LSB C++ Compiler? Alternatively, is it possible to build the Boost library with some old gcc version, what version is recommended? My final goal is to get my executable and third-party Boost libraries running on most Linux distributions.
Generally, what can be done to get better binary compatibility for Linux distributions, developing C++ closed-source application depending on the Boost library?
I've recently had call to do this, in the event that it's of use to anyone else these are the steps I followed:
Download and install the LSB SDK
Download a version of boost and extract to /opt/boost/boost_<version> (I used 1.43)
Make sure libbz2-dev is installed.
Bootstrap with
cd /opt/boost/boost_<version>
./bootstrap.sh --prefix=/opt/boost --without-libraries=python,mpi --without-icu
Edit /opt/boost/boost_<version>/project-config.jam and add the line
using gcc : : /opt/lsb/bin/lsbc++ : <cflags>-U_GNU_SOURCE <cxxflags>-U_GNU_SOURCE ;
near the top of the file. Note that this will fail if you have a using declaration in one of the other files bjam reads its configuration from, you can pass --debug-configuration to get an idea of which files it's reading.
Run
./bjam cflags=-fPIC cxxflags=-fPIC linkflags=-fPIC install
I didn't try to get either the python or MPI libraries working, nor did I try to get ICU working with the boost.regex library. The latter is probably a case of building static versions of the ICU libraries with the LSB toolset.
The -fPIC isn't strictly necessary for 32-bit Linux, but is required if you want to link static libraries into a shared library for 64-bit Linux.
The end result should be binaries in /opt/boost/lib and headers in /opt/boost/include, obviously you can modify the prefix to suit your own preferences. I still have a fair amount of work to do before I've ported all our code to the LSB, so I can't report on how well the certification process goes yet.
LSB C++ compiler is not actually a compiler. The lsbc++ executable is a wrapper around GCC compiler that is installed on your system (the actual compiler can be controlled via --lsb-cxx option). You will most likely hack into boost build system for it to call LSB wrapper instead of native gcc compiler.
So the issues that may arise are most likely not that LSB compiler can't compile the language constructs, but instead, that there are some linking issues.
For example, LSB compiler by default discards any shared libraries the code is linked against, unless they belong to LSB. This may lead to linking errors if BOOST relies on such libraries. This can be controlled via LSBCC_SHAREDLIBS environment variable, but you should make sure you ship these libs along with your product.
Another issue is that LSB falls behind GCC compiler releases (and BOOST may crawl into all dark corners of compilers). As far as I know, GCC 4.4 is not tested sufficiently, so you'd better try it with 4.3 compiler.
And Google doesn't seem to find anything related to building boost with LSBCC, so if you manage to do it, please, share your experience, for example, as your own answer to your question.
According to the LSB scanner, my binary is supposedly incompatible with a specific version of Linux because it uses GBLICXX_3.4.9 symbols. But when I tried to run the binary myself on that version, everything seems to work fine...
Can a binary even start on a Linux distro if that distro is missing the runtime libraries containing the required symbols?
I don't know if I've understood well the question but as far as I know even though you have compiled your program with a modern glibc does not necessarily mean that you won't be able to execute into an older version. The next Linux command:
objdump -T "your exe or lib file" | grep GLIB
will show you which version of the glibc the symbols of your program belong to.
For further information there is a paper called How to write shared libraries by Ulrich Drepper that explains a lot of things of how symbols work in linux not only for shared libraries but also for executables
I suspect they're warning you that you're using symbols that, even if they are available on your test system, may not be available on all LSB-compliant systems.
I have compiled a shared library on my Ubuntu 9.10 desktop. I want to send the shared lib to a co-developer who has a Red Hat Enterprise 5 box.
Can he use my shared lib on his machine?
First point: all of the answers regarding compiler version seem misguided. What's important are the linkages (and the architecture, of course).
If you copy the .so file over to the start system (into its own /usr/local/* or /opt/* directory, for example) then try to run the intended executable using an LD_PRELOAD environment settings. If the linker (ld-linux.so) manages to resolve all the symbols between the two then the program should load and run.
So it should be possible, and reasonably safe (so long as you're not over-writing any of the existing system libraries and just using LD_* /etc/ld.so.preload (in a chroot?) magic to link the target executables to this library.
However, I think it's a bad idea. You have a package management issue. Both Ubuntu and Red Hat have fine package management tools. Use them! (Note the proper place to ask questions about package management would be ServerFault or SuperUser, definitely not SO).
Unlikely: you wouldn't have asked this question if it just worked, would you?
According to DistroWatch, Ubuntu 9.10 uses glibc-2.10.1, while RHEL-5.4 uses glibc-2.5. This means that if your library references any symbols with versions GLIBC_2.6 and above, it will not work on RHEL-5.
You can tell whether you use any such symbols (and which ones) with:
readelf -s /path/to/your/library.so | egrep 'GLIBC_2.([6-9]|10)'
If the output is non-empty, then the library will not work on RHEL-5.
You might be able to build a library compatible with RHEL-5 by using autopackage.
I join to Xinus. IMHO compiler, in case of Ubuntu and RHEL, it will be gcc, is tightly coupled with glibc. So if on both machines it's same, than most probably it caa run.
But why guessing, do a small test drive (main with couple of lines) and if it's running than there's a good chance that a bigger program can run on a "hostile" environment :)
The best solution is to give your code to your co developper,
and let's compile it !!!!
You have several solution
Upgrading his gcc to the same version as your
Install his gcc version on your computer and compile it
You must check if you both work on the same architecture 32 bits or 64 bits.
My opinion is that you can have some problems , because you probably do not use the same glibc
.
Yes, it is possible. Provide a static library to your partner and keep your gcc to the same or compatible version. you can check my post here: https://zqfan.github.io/2021/07/01/cpp-static-library/