I am wondering if it is possible to link a c++ program compiled with gcc4.2 with a shared c++ library that is compiled in a later version like gcc4.5.
I've tried to do this, but have run into some different kind of problems.
When compiling the shared library gcc5.3 I get a message saying:
*"malloc: error for object 0x7fff707d2500: pointer being freed was not allocated
set a breakpoint in malloc_error_break to debug"*.
If I try to compile the shared library with gcc4.6 I get really strange behaviour. The std::stringstream class is not working correctly. The resulting string is empty after writing to the stream.
Is it possible to do this? Or am I trying something that is impossible? I was hoping that this was possible since I'm linking the lib dynamically. Btw I'm running on MacOSX.
BR
Beginning with gcc 3.0, g++ follows the Itanium ABI, so in theory there should be no problem. However, g++ 4.2 has CXXABI_1.3.1 whereas g++ 4.5 has CXXABI_1.3.4 (see here). Therefore I'd be careful. One does not bump up revision numbers if there are no differences.
Further, the glibc++ has gone through 5 revisions between those versions, which may be one reason why you see std::stringstream do funny things.
Lastly, there exist many config options (such as for example making strings fully dynamic or not) which affect the behaviour and compatibility of the standard library directly. Given two (random, unknown) builds, you cannot even know that they have the same config options.
In my experience the ABI compatibility means that C++ libraries can link to each-other without problems.
However, because C++ uses so many inline functions this doesn't mean much.
If the Standard C++ Library used all inline functions or used all library functions then you could use code compiled with older versions of GCC with newer versions.
But it doesn't. The library mixes inline and external library code. This means that if something is changed in std::string, or std::vector or locales or whatever, then the inlined code from the old GCC is out of sync with the library code linked from the new GCC.
Related
There are dynamic libraries provided by others, which are out of my control. What I know is that most of the aforementioned libraries are compiled
by g++4.9.4 & g++4.8.4 with C++11 support, and few libraries are
compiled by g++11.1.0 with C++14 support. And I have to use newer g++ which supports C++17 to compile my project. I intend to use g++11.3.0 on Ubuntu20.4.
Is there any potential problem I should be aware of when linking to dynamic libraries compiled by the said old version g++?
TL;DR: the dynamic library linkage you describe should "just work", provided you link the executable using the newer version of g++.
Both versions of GNU g++ you mention use libstdc++ (specifically libstdc++.so.6.0.*) to implement the support libraries needed for C++11/C++17.
For the most part, newer versions of libstdc++ maintain what their ABI policy manual calls "forward compatibility":
Versioning gives subsequent releases of library binaries the ability to add new symbols and add functionality, all the while retaining compatibility with the previous releases in the series. Thus, program binaries linked with the initial release of a library binary will still run correctly if the library binary is replaced by carefully-managed subsequent library binaries. This is called forward compatibility.
There are a few caveats, for unusual cases (which are probably irrelevant to your situation):
This assumes all the objects are built with compatible CPU ABI/architecture flags (i.e. -march, -mcpu, -m64, etc. if any), which is a general requirement for correctly linking libraries together (regardless of compiler version). The compiler defaults for these options should select compatible settings for your system, so this challenge usually only arises with cross-compilation scenarios where you've deliberately changed to a non-default ABI.
The internal representation of some C++ library classes (eg std::string, std::list) have changed over time, which can lead to problems if two libraries are sharing access to the same C++ object. This should only be a problem if you pass C++ library objects or references to them across the library interface where the g++ version differs.
Versions of gcc before 5.1 did not fully support C++11. They could not possibly. The ABI needed to support C++11 was not ready.
See this article for more information.
If you have object files compiled with gcc <5.1 and object files compiled with gcc >=5.1 (on default ABI settings), and they somehow share data that contain std::string and/or std::list objects, then they cannot work together. You need to recompile at least some of them.
In many cases you will get linker errors, and in some other (rare) cases the program will compile, link, and have mysterious crashes at run time. You probably should not worry about these cases as they are unlikely to occur naturally with normal libraries.
If there are no shared std::string or std::list involved, then you might be able to get away with it. However there are corner cases.
If you have a shared library dynamically linked with libstdc++.5.x. and another one dynamically linked to libstdc++.6.x, the final executable will be indirectly linked with both. This could be a problem in some cases. Try LD_PRELOADing one or the other to establish precedence.
We have a lot of prebuilt libraries (via CMake mostly), built using Visual Studio 2017 v141. When we try to use these against a project using Visual STudio 2019 v142 we see errors like:
Error C1047 The object or library file
‘boost_chrono-vc141-mt-gd-x32-1_68.lib’ was created by a different
version of the compiler than other objects...
On the other hand, we also use pre-compiled .libs from 3rd-party vendors which are over a decade old and these have worked just fine when linked against our codebase.
What determines whether a library needs to be rebuilt, and why can some ancient libraries still be used when others that are only one version behind cannot?
ABI incompatibilities could cause some issues. Even though the C++ standard calls for objects such as std::vector and std::mutex and that they need to have specific public/protected members, how these classes are made is left to the implementation.
In practice, it means that nothing prevents the GNU standard library from having their data fields in another orders than the LLVM standard library, or having completely different private members.
As such, if you try to use a function from a library built with the LLVM libc++ by sending it a GNU libstdc++ vector it causes UB. Even on the same standard library, different versions could have changed something and that could be a problem.
To avoid these issues, popular C++ libraries only use C data structures in their ABIs since (at least for now) every compiler produces the same memory layout for a char*, an int or a struct.
These ABI issues can appears in two places:
When you use dynamic libraries (.so and .dll files) your compiler probably won't say anything and you'll get undefined behavior when you call a function of the library using incompatible C++ objects.
When you use static libraries (.a and .lib files) I'm not really sure, I'm guessing it could either print an error if it sees there's gonna be a problem or successfully compile some Frankenstein monster of a binary that will behave like the above point
I will try to answer some integral parts, but be aware this answer could be incomplete. With more information from peers we will maybe be able to construct a full answer!
The simples kind of linking is linking towards a C library. Since there is no concept of classes and overloading function names, the compiler creators are able to create entry points to functions by their pure name. This seems to be pretty much quasi-standardized since, I myself, haven't encountered a pure C library not at least linkable to my projects. You can select this behaviour in C++ code by prepending a function declaration with extern "C". (This also makes it easy to link against a library from C# code) Here is a detailed explanation about extern "C". But as far as I am aware this behaviour is not standardized; it is just so simple - it seems - there is just one sane solution.
Going into C++ we start to encounter function, variable and struct names repeating. Lets just talk about overloaded functions here. For that compiler creators have to come up with some kind of mapping between void a(); void a(int x); void a(char x); ... and their respective library representation. Since this process also is not standardized (see this thread) and this process is far more complex than the 1 to 1 mapping of C, the ABIs of different compilers or even compiler versions can differ in any way.
Now given two compilers (or linkers I couldn't find a resource wich specifies wich one exactly is responsible for the mangling but since this process is not standardized it could be also outsourced to cthulhu) with different name mangling schemes, create following function entry points (simplified):
compiler1
_a_
_a_int_
_a_char_
compiler2
_a_NULL_
_a_++INT++_
_a_++CHAR++_
Different linkers will not understand the output of your particular process; linker1 will try to search for _a_int_ in a library containing only _a_++INT++_. Since linkers can't use fuzzy string comparison (this could lead to a apocalypse imho) it won't find your function in the library. Also don't be fooled by the simplicity of this example: For every feature like namespace, class, method etc. there has to be a method implemented to map a function name to a entry point or memory structure.
Given your example you are lucky you use libraries of the same publisher who coded some logic to detect old libraries. Usually you will get something along the lines of <something> could not be resolved or some other convoluted, irritating and/or unhelpful error message.
Some info and experience dump regarding Visual Studio and libraries in general:
In general the Visual C++ suite doesn't support crosslinked libs between different versions but you could be lucky and it works. Don't rely on it.
Since VC++ 2015 the ABI of the libraries is guaranteed by microsoft to be compatible as drescherjm commented: link to microsoft documentation
In general when using libraries from different suites you should always be cautious as n. 1.8e9-where's-my-share m. commented here (here is your share btw) about dependencies to other libraries and runtimes. In general in general not having the control over how libraries are built is a huge pita
Edit addressing memory layout incompatibilities in addition to Tzigs answer: different name mangling schemes seem to be partially intentional to protect users against linkage against incompatible libraries. This answer goes into detail about it. The relevant passage from gcc docs:
G++ does not do name mangling in the same way as other C++ compilers. This means that object files compiled with one compiler cannot be used with another.
This effect is intentional [...].
Error C1047
This is caused by /GL Global optimization or /LTGC Link Time Code Generation
These use information in the .obj, to perform global optimizations. When present, VS looks at the compiler which generated the original .lib, and if they are different emits the error. These compilation switches are for code from a single compiler, and not intended for cross version usage.
The other builds which work, don't have the switches, so are compatible.
Visual studio has started to use a new #pragma detect_mismatch
This causes an old build to identify it is incompatible with a new build, by detecting the version change.
Very old builds didn't have / support the pragma, so had no checking.
When you build a lib, its dependencies are loaded and satisified by the linker, but this is not a guarantee of working. The one-definition-rule signs the developer up to a contract, that within a compiled binary, all implementations of the same named function are the same. If this came from different compilers, that may not be true, and so the linker can choose any, causing latent bugs, where mixtures of old and new code are linkeded into the binary.
If the definition or implementation of std::string has changed, it may link, but have code which is flawed.
This new compiler check, causes a fail early, which I thoroughly approve of.
Are there any differences in the linking process between gcc and g++?
I have a big C project and I just switched part of the code to C++. The code isn't using std C++ library yet, so -llibstdc++ isn't needed for now.
The main difference is that (assuming the files are detected as C++) g++ sets up the flags needed for linking with the C++ standard library. It may also set up exception handling. I wouldn't rely on the fact that just because your application doesn't use the standard library that it isn't needed when compiled as C++ (for example the default exception handler).
EDIT: As pointed out in comments you'll have trouble with any constructors (that do work) for static objects as well as not getting virtual function tables (so if you're using those features of C++ you still need to link that library).
EDIT2: Unless you're using C99 specific code in your C project I would actually just switch to compiling the whole thing as C++ as the first step in your migration process.
gcc and g++ are both just driver programs that don't do anything other than calling other programs, so you can use the -v option to see exactly what they do -- what other programs they invoke with what args. So you can see exactly what the difference is between linking with gcc and g++ for the specific version and architecture of gcc that you happen to have installed. You can't rely on that staying the same if you want portability, however.
Depending on what you are doing, you might also be interested in the -### argument
I think that the g++ linker will look for the CPP mangled function names, and it is different from the C ones. I'm not sure gcc can cope with that. (Provided you can explicitly use the C version rather than the C++ one).
Edit:
It should work if you have
extern "C" {
<declarations of stuff that uses C linkage>
}
in your code and the object file has been compiled with g++ -c. But I won't bet on this.
For example could I mix a set of libraries that have been compiled in say GCC-4.6 with GCC-4.9.
I'm aware different compilers "breeds" such as VS cannot be with MinGW but can different generations of the same compiler? Are issues likely to occur? If so what?
Different generations of the same compiler sometimes can be compatible with each other, but not always. For example, GCC 4.7.0 changed its C/C++ ABI, meaning libraries compiled with 4.7.0+ and 4.7.0- are not likely to be compatible with each other (so in your example, the library compiled with 4.6 will not be compatible with the library compiled with 4.9). There can also be ABI bugs within a given compiler release, as happened in GCC 4.7.0/4.7.1:
GCC versions 4.7.0 and 4.7.1 had changes to the C++ standard library which affected the ABI in C++11 mode: a data member was added to std::list changing its size and altering the definitions of some member functions, and std::pair's move constructor was non-trivial which altered the calling convention for functions with std::pair arguments or return types. The ABI incompatibilities have been fixed for GCC version 4.7.2 but as a result C++11 code compiled with GCC 4.7.0 or 4.7.1 may be incompatible with C++11 code compiled with different GCC versions and with C++98/C++03 code compiled with any version.
The GCC ABI Policy and Guidelines page indicates they try to maintain forward compatibility, but not backward compatibility:
Versioning gives subsequent releases of library binaries the ability to add new symbols and add functionality, all the while retaining compatibility with the previous releases in the series. Thus, program binaries linked with the initial release of a library binary will still run correctly if the library binary is replaced by carefully-managed subsequent library binaries. This is called forward compatibility.
The reverse (backwards compatibility) is not true. It is not possible to take program binaries linked with the latest version of a library binary in a release series (with additional symbols added), substitute in the initial release of the library binary, and remain link compatible.
That page also has some fairly lengthy explanations on the versioning system GCC uses to mark different versions of given components, as well as an explanation for the versioning behind GCC itself:
Allowed Changes
The following will cause the library minor version number to increase, say from "libstdc++.so.3.0.4" to "libstdc++.so.3.0.5".
Adding an exported global or static data member
Adding an exported function, static or non-virtual member function
Adding an exported symbol or symbols by additional instantiations
Other allowed changes are possible.
Prohibited Changes
The following non-exhaustive list will cause the library major version number to increase, say from "libstdc++.so.3.0.4" to "libstdc++.so.4.0.0".
Changes in the gcc/g++ compiler ABI
Changing size of an exported symbol
Changing alignment of an exported symbol
Changing the layout of an exported symbol
Changing mangling on an exported symbol
Deleting an exported symbol
Changing the inheritance properties of a type by adding or removing base classes
Changing the size, alignment, or layout of types specified in the C++ standard. These may not necessarily be instantiated or otherwise exported in the library binary, and include all the required locale facets, as well as things like std::basic_streambuf, et al.
Adding an explicit copy constructor or destructor to a class that would otherwise have implicit versions. This will change the way the compiler deals with this class in by-value return statements or parameters: instead of passing instances of this class in registers, the compiler will be forced to use memory. See the section on Function Calling Conventions and APIs of the C++ ABI documentation for further details.
Note the bolded bit. In a perfect world, GCC versions with the same major release number would be binary-compatible. This isn't a perfect world, so test very very carefully before you go mixing compiler versions like this, but in general you'll probably be okay.
You can only mix generated binary files from different compilers or different versions of the same compiler if they are ABI (Application Binary Interface) compatible.
Things like:
Calling procedure
Name mangling
Thread local storage handling
are all part of the ABI.
If one of these things change, you will find that you either get linker errors, crashes or other forms of unexpected behaviour.
As a general rule, compiler vendors will often try maintaining at least backwards compatibility with older version, but there is no guarantee of this. As other have said you must either read the documentation or just recompile everything.
Are there any differences in the linking process between gcc and g++?
I have a big C project and I just switched part of the code to C++. The code isn't using std C++ library yet, so -llibstdc++ isn't needed for now.
The main difference is that (assuming the files are detected as C++) g++ sets up the flags needed for linking with the C++ standard library. It may also set up exception handling. I wouldn't rely on the fact that just because your application doesn't use the standard library that it isn't needed when compiled as C++ (for example the default exception handler).
EDIT: As pointed out in comments you'll have trouble with any constructors (that do work) for static objects as well as not getting virtual function tables (so if you're using those features of C++ you still need to link that library).
EDIT2: Unless you're using C99 specific code in your C project I would actually just switch to compiling the whole thing as C++ as the first step in your migration process.
gcc and g++ are both just driver programs that don't do anything other than calling other programs, so you can use the -v option to see exactly what they do -- what other programs they invoke with what args. So you can see exactly what the difference is between linking with gcc and g++ for the specific version and architecture of gcc that you happen to have installed. You can't rely on that staying the same if you want portability, however.
Depending on what you are doing, you might also be interested in the -### argument
I think that the g++ linker will look for the CPP mangled function names, and it is different from the C ones. I'm not sure gcc can cope with that. (Provided you can explicitly use the C version rather than the C++ one).
Edit:
It should work if you have
extern "C" {
<declarations of stuff that uses C linkage>
}
in your code and the object file has been compiled with g++ -c. But I won't bet on this.