I have an application with STL objects used as part of the C++ interface for plug-in writers.
I know the best option for compatibility would have been to use a C interface instead, but that's not currently feasible.
I know that everything from GCC 3.4 to 4.8 in libstdc++ has been highly compatible in terms of ABI.
So for example, if I compile with GCC 4.1, and a plug-in vendor writes code compiled with GCC 4.7, then barring corner cases all will be well on a platform with a libstdc++ version corresponding to GCC 4.7 or later, provided STL useage is internal to the .so only, and that the external .so interface is using pure C, which sadly is not the case for me.
So, I'm curious about what the case will be with respect to the STL classes used as part of the plug-in interface. Can I safely pass STL objects between shared objects that weren't compiled with the same compiler version (e.g. 4.1 and 4.8)? And is there anything I need to be mindful of with regards to how templates are compiled and resolved if people use different compiler options?
I suspect it'll be problematic. However, there's a chance the symbol versioning magic done by the GCC folks might somehow make this work.
For this question, I am only interested in pre-C++11 compilation and linking. I am also only interested in Linux and Mac OS X, using GCC.
I already answered this on the mailing list but as Marc said, it will just work.
It doesn't make any difference whether you use the library internally to your DSO or in the interface, the library doesn't care and is backwards compatible back to GCC 3.4 either way.
Related
My question specifies the version numbers of my scenario, but I'm interested in a general to the question.
I'd like to use gcc 11 on Alma Linux/Red Hat 8 (identical ABI), which come with gcc 8, in order to use C++20/23 features in my own programs. The programs I compile with it (all userspace applications) would run on the same system.
I'm thinking of compiling gcc11 from source, installing it in /usr/local/gcc11/, and calling it when I need it. I don't want to replace/remove the system gcc, since various tools that support a given distro will be expecting that compiler. I would just call a wrapper script to use gcc11 when I need it.
I expect gcc11 will compile just fine, and so will programs I compile with it. But any non-trivial program tends to link against libc, libm, libdl, libpthread, libgcc_s, libstdc++, and so on. I would be using gcc11 with relatively old versions of these system libraries. These are all system libraries which I've never had to deal with directly.
The situation that worries me is if some new dependency I use was written around libpthread version Z but my system has version X, and despite the ABI being unchanged (which allows my program to link successfully against the system libpthread), the behavior is different due to bugfixes, and I end up with subtle runtime bugs.
Is this a valid worry? Or am I golden as soon as my application compiles/launches successfully?
The only existing discussion on this that I could find ( Is there any issue in upgrading GCC version to other than the ones come with distro? ) is light on info, and the warnings given don't apply to me since I don't intend to touch the system gcc/libs.
It depends which features you're compiling against. This article from cppreference.com has more details on which c++ features for each language version are supported by which c++ runtime. https://en.cppreference.com/w/cpp/compiler_support
Since you're talking about compiling with gcc, you're going to want to look at the libstdc++ runtime library. It looks like not even every feature of c++20 is even supported by version 8 of the runtime library. But it is most things.
To work around this you could
install the runtime version of libstdc++ you want
package and distribute the libstdc++ runtime library along with your code
statically link against libstdc++, which I don't recommend
Keep in mind that there is theoretically an option to compile against libstdc++ statically, but you're not guaranteed to get those symbols at runtime depending on your situation. It also makes a difference if you're compiling an application vs a library. Loading a library my cause your libraries symbols to conflict with what's already been loaded (probably by libstdc++). This post does a good job explaining some things to look out for https://stackoverflow.com/a/14082540/1196033.
(Also, Android is weird https://developer.android.com/ndk/guides/cpp-support)
The common explanation for not fixing some issues with C++ is that it would break the ABI and require recompilation, but on the other hand I encounter statements like this:
Honestly, this is true for pretty much all C++ non-POD types, not just exceptions. It is possible to use C++ objects across library boundaries but generally only so long as all of the code is compiled and linked using the same tools and standard libraries. This is why, for example, there are boost binaries for all of the major versions of MSVC.
(from this SO answer)
So does C++ have a stable ABI or not?
If it does, can I mix and match executables and libraries compiled with different toolsets on the same platform (for example VC++ and GCC on Windows)? And if it does not, is there any way to do that?
And more importantly, if there is no stable ABI in C++, why are people so concerned about breaking it?
Although the C++ Standard doesn't prescribe any ABI, some actual implementations try hard to preserve ABI compatibility between versions of the toolchain. E.g. with GCC 4.x, it was possible to use a library linked against an older version of libstdc++, from a program that's compiled by a newer toolchain with a newer libstdc++. The older versions of the symbols expected by the library are provided by the newer libstdc++.so, and layouts of the classes defined in the C++ Standard Library are the same.
But when C++11 introduced the new requirements to std::string and std::list, these couldn't be implemented in libstdc++ without changing the layout of these classes. This means that, if you don't use the _GLIBCXX_USE_CXX11_ABI=0 kludge with GCC 5 and higher, you can't pass e.g. std::string objects between a GCC4-compiled library and a GCC5-compiled program. So the ABI was broken.
Some C++ implementations don't try that hard to have compatible ABI: e.g. MSVC++ doesn't provide such compatibility between major compiler releases (see this question), so one has to provide different versions of library to use with different versions of MSVC++.
So, in general, you can't mix and match libraries and executables compiled with different versions even of the same toolchain.
C++ does not have an ABI standard as of yet. They are attempts to have it in the standard; You can read following it explains it in details:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p2028r0.pdf
Suppose we have a shared library which accepts or returns some kind of std class:
//lib.h
#include <vector>
std::vector<int> returnSomeInts();
//lib.cpp
#include "lib.cpp"
std::vector<int> returnSomeInts() {
return {1, 3, 5};
}
So, obviously, when compiling the shared library lib.so, this code had to be compiled against a particular version of the standard library, for instance with -std=c++11.
Now imagine we have an application which will be using our shared library, but it will be compiled against a newer std library, for example -std=c++2a
//app.cpp
#include <lib.h>
int main()
auto v = returnSomeInts();
//Process v
}
As the standard library defines inline classes, if the layout of the class members changes, ABI compatibility gets broken, so the code above would not work properly.
My questions are: Is there any guarantee for ABI stability with the common implementations of the the std library when compiling against the same header using different c++ standards? And when compiling against different header versions (libstdc++-8 and libstdc++-9 for example)?
PD: The code above is just an example, I am not referring specifically to std::vector
ABIs in practice are not linked to the standard, for example consider this following code compiled with gcc 4.9.4 and gcc 5.1
using the same flags:
-std=c++11 -O2
#include <string>
int main(){
return sizeof (std::string);
}
gcc 4.9.4 returns 8 from main, gcc 5.1 returns 32.
As for guarantees: it is complicated:
Nothing is guaranteed by the standard.
Practically MSVC used to break ABI compatability, they stopped (v140,v141,v142 use the same ABI), clang/gcc have a stable ABI for a long time.
For those interested in learning more:
For a broad discussion of ABI/C++ standard that is not directly related to this question you an look at this blog post.
Your example with std::vector is not a problem. As other answer points out compilers maintain compatibility and std::vector is quite old.
Problem can be new library features of newer C++ standard is used.
For example I have some product which uses C++17 languages features. My product supports MacOS 10.13. I can build my project using C++17 (language features), but I can't use for example std::optional since some methods are throwing std::bad_optional_access which is part dynamic library and this is not supported/present in MacOS 10.13. Clang warns me about that (reports an error).
Same applies on other system (clang nicely controls that). So when you use some library features you need make sure that libraries on deployed system supports that (on Linux package managers handle that nicely if system doesn't have access to required package version installation will fail). On Windows 10 AFAIK Windows update keeps newest version of msvc redistributable, older versions of Windows need manual updates.
Note that many templates are becoming part of your executable and do not have dependencies to shared standard library. Those will not create problems.
These are the kinds of question that the standard doesn't answer, it is down to individual implementers how long they keep compatibility for and which (if any) compiler flags affect compatibility.
In practice libstdc++ on Linux has tried very hard to keep a stable ABI for quite a long time now. The soversion has remained at 6 for around 15 years now.
However there have been some changes, most notable were those introduced with g++-5.1 related to std::list and std::string. New requirements in c++11 meant that to be compliant libstdc++ needed to make breaking changes to the ABI of those classes.
g++'s solution was to define two versions of those classes. To allow both types to coexist in the same shared library they are defined with different names, so the old std::list is defined as std::list while the new std::list is defined as std::__cxx11::list and then aliased as std::list. Which definition is used in a given source file is controlled by a preprocessor macro.
Note that the setting of said preprocessor macro is independent of the -std= setting, you can use the compiler in c++11 mode with the old string/list implementation (in which case it won't be fully compliant with the standard) or you can use the compiler in c++03 mode with the new string/list implementations.
The default value of the macro (if not set explicitly by the user) depends on compiler version and can also be overridden by distribution maintainers.
The trouble is while this solves the problem for libstdc++ it doesn't solve it for all the other libraries. If a program was built with the new implementations while a library was built with the old ones or vice-versa and the library used std::string or std::list in it's API, then the likely outcome was a link failure (it was also possible but less-likely for the linking to succeed but the program to be silently broken). In Debian at least this lead to a massive set of library transitions (it would not surprise me if it was the largest in the project's entire history).
I've written a fairly substantial C++11 library, and I'm planning to allow download of pre-compiled versions from my website.
So I've set up an automated build which compiles the library with clang and make it available for download, but this has exposed a problem: if I try to use the clang-compiled library with GCC I get undefined references (mainly related to std::string). I think this is related to the GCC dual-ABI changes in GCC 5.1 but I'm not sure how to fix it.
My question is, what flags should I set, or practices should I follow to make a C++ library compatible with both clang and GCC?
Or should I give up and compile two separate libraries?
As already mentioned in several places (eg. here) libc++ is not fully binary compatible with libstdc++.
There are several options, but some of them are somewhat not-so-straightforward.
Compile two separate libraries - always working solution.
Remove incompatible containers from your interface (eg. std::string) - but this might be lot of work and sometimes not a good idea.
Instruct your library GCC users that you link with libc++ and that they need to do the same, basic steps here. But I guess most GCC users do not want to do this.
Use clang with libstdc++ using -stdlib=libstdc++ flag to be compatible with libstdc++ (as suggested in other answer). This solution might be harder to setup on some platforms though.
I would suggest as already mentioned in comments to go with option 1.
There are several options:
Don't distribute it in binary form. Instead, make it easy to build everywhere (e.g. by using CMake, or autotools or ...)
Make it header only. This is by far the simplest solution but might not be what you want. It only really makes sense for templated code, and incurs a heavy impact on compile-time performance of your library.
Tell people to link with libstdc++ when using Clang and your library. Suboptimal solution (I for one like to check my code against libc++ as well as libstdc++), but (virtually) every Linux user has libstdc++ installed anyway. Make sure to pick a slightly older version (the one shipped in the latest Debian Stable distro is a good choice), because newer versions might introduce new symbols olders versions are missing. New versions should be ABI compatible anyway.
Note the situation for Visual Studio users is even worse, where every single compiler release mandates a new binary because they guarantee absolutely nothing with respect to the C++ libraries' or compiler's ABI.
Another option is for your shared library to not expose any C++ standard library types in its interface. And have a header file supplied with your shared library that converts std::string to types consumed by your library, such as struct my_string_span { char const *begin, *end; }; and other standard containers as necessary.
Having gone through a number of similar questions (see below), I don't think any of them cover this case.
Problem
Is it possible to dlopen and use a C++11 shared library compiled using gcc 4.9 from a C++03 application compiled with gcc 4.1? Both the library and application use libstdc++, including on the API (but not std::list, which is apparently a problem). I can't change the way the application is compiled (sadly), and must be certain that existing code doesn't break (e.g. if it ends up dynamically linking to a new version of libstdc++.so).
As far as I understand it (which is not very), on Linux the dynamic linker uses a flat namespace, meaning that is not possible to have the same symbol defined in two libraries. Does statically linking the library against the newer libstdc++ help at all here, or is there perhaps some other way?
Some similar questions that don't seem to answer this
C++11 backwards compatibility
C++03 library with C++11 source code
C++11 compatibility with existing libraries/frameworks
Can a compiled C++11 library (lib,dll,etc.) be linked in older C++ compilers? [softwareengineering.stackexchange.com]
If you do that, you definitely need to use the newer libstdc++.so.6, which should be compatible with the system libstdc++.so.6 based on GCC 4.1 (in the sense that GCC upstream intends to preserve ABI compatibility for the library). It would be a very good idea to use the libstdc++.so.6 library that came with the GCC 4.9 compiler.
Once you do that, it is supposed to work, unless you hit the few of the compatibility gotchas you already listed, and as long as the C++ part of the library interface actually sticks to the C++98 subset and does not use any constructs which are not expressible in the language subset.
All this assumes that the library was actually compiled on the the system which uses GCC 4.1, which probably has something like glibc 2.5. If the library was compiled on a completely different, newer system, then you will likely run into library compatibility issues beyond libstdc++.so.6, and these libraries tend to be even harder to upgrade.
(There is also the possibility that the library was explicitly compiled for use with the 4.1-based system libstdc++.so.6 and everything just works, just as if by magic, but then you wouldn't be asking here, I suppose.)
If you still have problems there is the option to use a statically linked C-API library as border between the two, application and your library, see this post.
Also you may be able to explicitely demand an old version of a symbol, see this post