I have a target system Astra Linux Smolensk, which has access to very outdated packages with GCC 6 among them. It does not support C++17, which is required for some very useful 3rd party libraries and language features. At some point I am considering a possibility of using concepts from 20 standard. But there is no easy way to achieve that.
Basically, I see 2 ways:
Compile needed version of GCC by myself and have a custom version of Astra Linux for building with additional packages (which is not a good option since we have restrictions for system modification). I am about to try out this option, but it is not the subject of this question.
Cross-compile with latest GCC on Ubuntu using a toolchain.
So, what do I need in order to create a toolchain for a custom version of Linux? The only thing I am sure of is the Linux Core version. Can I use an existing Linux toolchain or do I have to export system libraries and create a custom toolchain?
Here I found out some tools that seem to be helpful. For instance:
Buildroot
Buildroot is a complete build system based on the Linux kernel configuration system and supports a wide range of target architectures. It generates root file system images ready to be written to flash. In addition to having a huge number of packages which can be compiled into the image, it also generates a cross toolchain to build those packages from source. Even if you don't want to use buildroot for your root filesystem, it is a useful tool for generating a toolchain. Buildroot supports uClibc-ng, glibc and musl.
I wonder if it does what I need and can I use a latest GCC compiler with those generated toolchains.
I found similar questions:
How to build C++17 application in old linux distro with old stdlib and libc?
https://askubuntu.com/questions/162465/are-gcc-versions-tied-to-kernel-versions
How can I link to a specific glibc version?
Some clarification is needed:
The project relies heavily on a lot of 3rd party dependencies from the target linux's package repository. Moreover, I use dynamic .so modules that may be loaded both implicitly and explicitly.
Today with docker and modern CI/CD pipelines based on container, we don't rely on process compile very often as old days.
With the help of musl, we can even create universal Linux binaries with static linkage: We use all static libraries rather than dynamic libraries. Then we ship one executable file.
According to musl's doc, it needs
Linux kernel >=2.6.39
This is a very old version released around 2011, so even old Linux distro's can run our binaries.
Musl is widely used in many projects, especially in Rust projects, we provide Musl builds for users as conveniences.
Note that we may need to fix our codebase when using Musl, there are very slight differences with GNU libc, which we should be aware.
Related
When I develop across different OS's I find that a program built on one Linux system can not be run on another system, due to the different libc version.
How can I build in all the shared libraries just like golang did in c/c++?
Including libc and libcxx
If you want to run on multiple Linux systems, all you really need is to build using the oldest glibc from any of them. The easiest way is to simply download a virtual machine image from an old system like CentOS 5 and build there. You don't need to worry about static linking, just building against an old version will mean you are mostly compatible with newer versions.
Say I have a C++ binary that was compiled with a version of gcc say 4.4.x, which is used on a client linux box.
If I want to upgrade my compiler to use a newer one, say 4.9.3 (because I want to use C++11):
What kind of things would need to be upgraded on the client box to run this new binary? (e.g. .so libraries)
And how would one find this out?
What kind of things would need to be upgraded on the client box to run this new binary?
You will need to ship two shared libraries with your application: libgcc_s and libstdc++. See https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html for more details.
You can ship these libraries in the same directory with your executables if you link using $ORIGIN. See https://stackoverflow.com/a/4742034/412080 for more details.
And how would one find this out?
Run ldd and readelf -d on your executables to see what libraries they need.
I'm guessing from the fact that you mention GCC 3.4 that your client system is running RedHat/CentOS/Scientific Linux 4 which is so old that even Red Hat ended support for it three years ago. If you were running any newer version then you would have been able to take advantage of the Developer Toolsets which include a modified version of GCC that statically links newer parts of the standard library into your binary so that it can run on legacy systems without newer glibc/libstdc++ runtimes.
There are two mechanisms to test compatibility of shared libraries:
The SONAME: a canonical name for the library that is used by the linker to reference the library. You can query the list of required libraries for every ELF object (executable or library) with the ldd command, and you need to do this recursively for each referenced library to get a full list of libraries needed.
The symbol version information. This is an additional constraint that allows adding functionality to existing libraries by introducing version requirements per symbol used -- a program only using symbols that have existed for ages will require a lower minimum version of the library than one that uses new functionality.
Both of these need to be fulfilled in order for the program to run.
The typical approach of Linux distributions is to keep a mapping of SONAME to package name (as multiple versions differing in SONAME can be installed concurrently), and a table of versioned symbols to the package revision these were introduced. The appropriate package development tools for your distribution should be able to create a list of dependency specifications that matches your program's requirements; if they fail to do so because symbols are unknown, it is likely that this version cannot be supported on that release of the distribution.
An alternative could be to link statically all libraries (notably libstdc++) other than the libc (and libm and libdl if you use them); then the produced ELF executable depends only of the libc; however, with ancient enough kernels or libc (on the target machine) even that might fail to work...
For details, read Drepper's paper: How To Write Shared Libraries
My c++ app depends on GCC, MongoDB C++ driver and Boost. My current way is to keep the OS consistent. I develop C++ on Ubuntu 12.04 64bit Desktop, and deploy it on Ubuntu 12.04 64bit server. Also I should install the same version of dependencies on target server.
But If I want to develop my C++ app on Ubuntu 13.04 and use newest Boost, MongoDB driver and GCC 4.8.1, which way is easy to deploy C++ app on Ubuntu 12.04 server.
static linking
Dynamic linking, also deploy all dependencies to target server?
Which way is simple? Sometimes, I cannot compile libraries on target server.
If the dependencies are small easiest way is to compile everything statically. It is fairly easy during the build step, and nothing fancy is needed. However, with bigger libraries, and a bigger project this might get inconvenient.
I think that best practice would be to compile dependencies into shared objects ship them along the binaries and execute stuff in a way that ld will look for your stuff 1st. I think it's possible by for example using LD_LIBRARY_PATH e.g. LD_LIBRARY_PATH=/where/did/i/ship/lib:$LD_LIBRARY_PATH my_binary.
It can be somewhat cumbersome as you need to set up your build system to compile stuff as shared objects and properly pack everything.
I'm pretty sure some of the pre-compiled programs that are shipped for linux work this way. Strangely, I can't find any custom pre-compiled app under my hand at the moment.
It depends on your application. If your application consists of only one specific binary, then static linking of all C++ libraries is in order. You can safely link dynamically all C libraries, as the C ABI is unchanging; this just leaves you with version dependencies. However in most cases the major SO-Name verions are mostly compatible and libraries of differing major SO-Name can be installed in parallel. So I'd rely on the package manager to install those. C++ libraries are tricky due to lack of a common ABI. Even a mere compiler version bump can make them incompatible *sigh*.
Clang has a binary distro, of sorts, but there isn't any README file or anything to tell you what's in the tarball or what to do with it.
It appears that I need to separately download and install libc++. I may need to copy only the clang binary and maybe a few others, but not all the llvm-* stuff. This is just judging by the lack of any C++ headers in the binary distro (although some environment-specific headers are included), and the lack of llvm-as and such on my existing LLVM 3.2 installation from Xcode.
I just want to run the compiler, not develop with libclang or assemble LLVM assembly files. Is there an instruction page somewhere?
The LLVM project doesn't actually expect many people to use the binary distribution they put out. LLVM does releases for the periodic validation, but it's expected that most users will get LLVM through their OS distro or will build the version they want from source.
See this email thread where clang developers are discussing how the binaries distrbution is used.
That said, you can use their distribution if you want. What to install depends on what you want to do:
Use clang as a static compiler.
Build clang based tools.
Use LLVM as a backend for your custom language compiler.
I may need to copy only the clang binary and maybe a few others, but not all the llvm-* stuff.
If all you want to do is compile C/C++/Obj-C, then I believe all you need is the clang binary (and the 'clang++' symbolic link), the 'built-in' headers, and the runtime libraries. You'll find those headers and libs in /lib/clang/<version>/. (The clang compiler typically finds its built-in parts by their location relative to the binary.)
If you want to use LLVM as a backend, you'll need either the LLVM headers and libraries to build and link against, or you'll need some of the ll* binaries to process output of your frontend.
If you want to build clang based tools you'll need the clang headers and libraries to build and link against, either the stable C API or the unstable C++ API.
Note that the libraries are built with RTTI and exceptions disabled. This changes the ABI and so you can't link these with code built with RTTI or exceptions enabled.
It appears that I need to separately download and install libc++.
Correct, libc++ is not included as part of LLVM's distribution. Many of the nominal LLVM subprojects aren't included. LLDB is another example.
Nor does LLVM include a standard C library or the basic Objective-C frameworks.
For Ubuntu/Debian (incuding Linux Mint) based Linux distributions, there are also pre-built .deb files from http://llvm.org/apt/
This has the advantage that it is easer to uninstall at a later point, and also provides Clang 3.4 nightly builds (the 3.3 version is also provided). Simply add one line to your sources.list (or use a GUI package manager to do so) and you're set.
I'm writing a cross-platform application which is not GNU GPL compatible. The major problem I'm currently facing is that the application is linked dynamically with glibc and libstdc++, and almost every new major update to the libraries are not backwards compatible. Hence, random crashes are seen in my application.
As a workaround, I distribute binaries of my application compiled on several different systems (with different C/C++ runtime versions). But I want to do without this. So my question is, keeping licensing and everything in mind, can I link against glibc and libstdc++ statically? Also, will this cause issues with rtld?
You don't need to.
Copy the original libraries you linked against to a directory (../lib in this example) in your application folder.
Like:
my_app_install_path
.bin
lib
documentation
Rename you app for something like app.bin. Substitute your app for a little shell script that sets the enviroment variable LD_LIBRARY_PATH to the library path (and concatenate the previous LD_LIBRARY_PATH contents, if any). Now ld should be able to find the dynamic libraries you linked against and you don't need to compile them statically to your executable.
Remember to comply with the LGPL adding the given attribution to the libraries and pointing in the documentation where the source can be downloaded.
glibc is under the LGPL. Under section 6. of LGPL 2.1, you can distribute your program linked to the library provided you comply with one of five options. The first is to provide the source code of the library, along with the object code (source is optional, not required) of your own program, so it can be relinked with the library. You can alternatively provide a written offer of the same. Your own code does not have to be under the LGPL, and you don't have to release source.
libstdc++ is under the GPL, but with a major exception. You can basically just distribute under the license of your choice without providing source for either your own code or libstdc++. The only condition is that you compile normally, without e.g. proprietary modifications or plugins to GCC.
IANAL, and you should consider consulting one if you need real legal advice.
Specifying the option -static-libgcc to the linker would cause it to link against a static version of the C library, if available on the system. Otherwise it is ignored.
I must question what the heck you are doing with the poor library functions?
I have some cross platform software as well. It runs fine on Linux systems of all sorts. Build with the oldest version of software that you want to support. The glibc and libstdc++ libraries are really very backward compatible.
I have built on CentOS 4 and run it on RHEL 6 beta. No problems.
I can build on stable Debian and run it on testing.
Now, I do sometimes have trouble with some libraries if I try to build on, say old Debian and try to run it on CentOS 5.4. That is usually due to distribution configuration choices that are different, like choosing threading or non-threading.