We have a lot of compiled C++ tools including some statically linked C++ libraries. We had extended some libs and of course recompiled it -> now the newer dll's are not compatible with the older C++ tools!
error:
TransceiveDataRaw#CCommonUtils#RegTestAPI##QAEIPAEH0AAHPBD2#Z could
not be located in the dynamic link library IFX_CC_RegTestBase.dll
Is there any way to ignore this name mangling without recompiling our tools (there are hundreds of them)?
Probably not. Different C++ compilers (and sometimes different versions of the same compiler) use different name mangling schemes, and in a way this is a safety mechanism. Along with a name mangling system, there are runtime mechanisms unique to a given compiler, and these won't be compatible either: virtual method dispatch, for example. The fact that these won't link is probably saving you from building an executable that would just crash and burn.
Related
We have a lot of prebuilt libraries (via CMake mostly), built using Visual Studio 2017 v141. When we try to use these against a project using Visual STudio 2019 v142 we see errors like:
Error C1047 The object or library file
‘boost_chrono-vc141-mt-gd-x32-1_68.lib’ was created by a different
version of the compiler than other objects...
On the other hand, we also use pre-compiled .libs from 3rd-party vendors which are over a decade old and these have worked just fine when linked against our codebase.
What determines whether a library needs to be rebuilt, and why can some ancient libraries still be used when others that are only one version behind cannot?
ABI incompatibilities could cause some issues. Even though the C++ standard calls for objects such as std::vector and std::mutex and that they need to have specific public/protected members, how these classes are made is left to the implementation.
In practice, it means that nothing prevents the GNU standard library from having their data fields in another orders than the LLVM standard library, or having completely different private members.
As such, if you try to use a function from a library built with the LLVM libc++ by sending it a GNU libstdc++ vector it causes UB. Even on the same standard library, different versions could have changed something and that could be a problem.
To avoid these issues, popular C++ libraries only use C data structures in their ABIs since (at least for now) every compiler produces the same memory layout for a char*, an int or a struct.
These ABI issues can appears in two places:
When you use dynamic libraries (.so and .dll files) your compiler probably won't say anything and you'll get undefined behavior when you call a function of the library using incompatible C++ objects.
When you use static libraries (.a and .lib files) I'm not really sure, I'm guessing it could either print an error if it sees there's gonna be a problem or successfully compile some Frankenstein monster of a binary that will behave like the above point
I will try to answer some integral parts, but be aware this answer could be incomplete. With more information from peers we will maybe be able to construct a full answer!
The simples kind of linking is linking towards a C library. Since there is no concept of classes and overloading function names, the compiler creators are able to create entry points to functions by their pure name. This seems to be pretty much quasi-standardized since, I myself, haven't encountered a pure C library not at least linkable to my projects. You can select this behaviour in C++ code by prepending a function declaration with extern "C". (This also makes it easy to link against a library from C# code) Here is a detailed explanation about extern "C". But as far as I am aware this behaviour is not standardized; it is just so simple - it seems - there is just one sane solution.
Going into C++ we start to encounter function, variable and struct names repeating. Lets just talk about overloaded functions here. For that compiler creators have to come up with some kind of mapping between void a(); void a(int x); void a(char x); ... and their respective library representation. Since this process also is not standardized (see this thread) and this process is far more complex than the 1 to 1 mapping of C, the ABIs of different compilers or even compiler versions can differ in any way.
Now given two compilers (or linkers I couldn't find a resource wich specifies wich one exactly is responsible for the mangling but since this process is not standardized it could be also outsourced to cthulhu) with different name mangling schemes, create following function entry points (simplified):
compiler1
_a_
_a_int_
_a_char_
compiler2
_a_NULL_
_a_++INT++_
_a_++CHAR++_
Different linkers will not understand the output of your particular process; linker1 will try to search for _a_int_ in a library containing only _a_++INT++_. Since linkers can't use fuzzy string comparison (this could lead to a apocalypse imho) it won't find your function in the library. Also don't be fooled by the simplicity of this example: For every feature like namespace, class, method etc. there has to be a method implemented to map a function name to a entry point or memory structure.
Given your example you are lucky you use libraries of the same publisher who coded some logic to detect old libraries. Usually you will get something along the lines of <something> could not be resolved or some other convoluted, irritating and/or unhelpful error message.
Some info and experience dump regarding Visual Studio and libraries in general:
In general the Visual C++ suite doesn't support crosslinked libs between different versions but you could be lucky and it works. Don't rely on it.
Since VC++ 2015 the ABI of the libraries is guaranteed by microsoft to be compatible as drescherjm commented: link to microsoft documentation
In general when using libraries from different suites you should always be cautious as n. 1.8e9-where's-my-share m. commented here (here is your share btw) about dependencies to other libraries and runtimes. In general in general not having the control over how libraries are built is a huge pita
Edit addressing memory layout incompatibilities in addition to Tzigs answer: different name mangling schemes seem to be partially intentional to protect users against linkage against incompatible libraries. This answer goes into detail about it. The relevant passage from gcc docs:
G++ does not do name mangling in the same way as other C++ compilers. This means that object files compiled with one compiler cannot be used with another.
This effect is intentional [...].
Error C1047
This is caused by /GL Global optimization or /LTGC Link Time Code Generation
These use information in the .obj, to perform global optimizations. When present, VS looks at the compiler which generated the original .lib, and if they are different emits the error. These compilation switches are for code from a single compiler, and not intended for cross version usage.
The other builds which work, don't have the switches, so are compatible.
Visual studio has started to use a new #pragma detect_mismatch
This causes an old build to identify it is incompatible with a new build, by detecting the version change.
Very old builds didn't have / support the pragma, so had no checking.
When you build a lib, its dependencies are loaded and satisified by the linker, but this is not a guarantee of working. The one-definition-rule signs the developer up to a contract, that within a compiled binary, all implementations of the same named function are the same. If this came from different compilers, that may not be true, and so the linker can choose any, causing latent bugs, where mixtures of old and new code are linkeded into the binary.
If the definition or implementation of std::string has changed, it may link, but have code which is flawed.
This new compiler check, causes a fail early, which I thoroughly approve of.
Qt library states binary compatibility across major releases. But what about compatibility between different compilers?
For example, I have an application and 2 dlls that use Qt (dynamically linked). But one of them is built with MSVC and the other with MinGW. Thus they are linked to different builds of Qt library (of the same version).
The question is: is it possible for these dlls to work together using one shared Qt dll?
If not then what workarounds are possible considering that changing the compiler is not an option?
I looked at Qt dlls with dependency walker and I see that there are dozens of exported functions which have compiler-scpecific name mangling. So it seems that it is not possible to make them work together.
C++ does not have standard ABI. That means, binaries including DLL files aren't usually compatible between compilers (they may not even be compatible between different build options on same compiler...). This is is the case with MSVC vs. MinGW, no DLL compatibility.
Plain C libraries have a defined ABI on Windows, so they may be used with any compiler (you may still run into problems with libraries having incompatible dependencies, even if ABI is compatible).
If recompiling one library is not an option, then the only workaround I know is to have separate processes, which then communicate using some IPC mechanism. If both libraries offer GUI elements, and you want to mix them in same GUI, that's not possible (or at least it is hard, you need to open window of other app in the window of another, sort of like an overlay).
I have an old library (in C++) that I'm only able to build on MinGW+MSYS32 on Windows. From that I can produce a .a static library file generated from GNU libtool. My primary development is done in Visual Studio 2008. If I try to take the MinGW produced library and link it in Visual Studio, it complains about missing externals. This is most likely due to the C++ symbol mangling that is done, and that it doesn't match whats in the .a file. Is there any know way to convert a static .a file to VC++ library format?
If symbols are mangled differently, the compilers are using a different ABI and you won't be able to "convert" or whatever the compiled libraries: the reason names are mangled in different ways is intentional to avoid users from ever succeeding with building an executable from object files using different ABIs. The ABI defines how arguments are passed, how virtual function tables are represented, how exceptions are thrown, the size of basic data types, and quite a number of other things. The name mangling is also part of this and carefully chosen to be efficient and different from other ABIs on the same platform. The easiest way to see if the name mangling is indeed different is to compiler a file with a couple of functions using only basic types with both compilers and have a look at the symbols (on UNIX I would use nm for this; I'm not a Windows programmer and thus I don't know what tool to use there).
Based on Wikipedia it seems that MinGW also uses a different run-time library: the layout and the implementation of the standard C++ library used by MinGW (libstdc++) and Visual Studio (Dinkumware) is not compatible. Even if the ABI between the two compilers is the same, the standard C++ library used isn't. If the ABI is the same you would need to replace set things up so that one compiler uses the standard library of the respective other compiler. There is no way to do this with the already compiled library. I don't know if this is doable.
Personally I wouldn't bother with trying to link things between these two different compiler because it is bound not to work at all. Instead, you'd need to port the implementation to a common compiler and go from there.
Search for the def file and run the command e.g. lib /machine:i386 /def:test.def it will generate
an import lib file.
I have a C++\MFC application written in Visual Studio 2003 SP1 links to an external static library "SomeExtStaticLib.lib". I also include the header files provided with "SomeExtStaticLib.lib" to create the objects in my application.
SomeExtStaticLib.lib is a static library built with VC6.
Now, I am migrating my application to Visual Studio 2008.
I have very basic question.
Should I also migrate the "SomeExtStaticLib.lib" to VS2008 compiled one?
When I tried to use this VC6 compiled "SomeExtStaticLib.lib" in my VC9 compiled application it did not give any linker errors. I was expecting at least some name mangling issues.
Are static libraries eliminating the name mangling issues?
The issue isn't one of static vs. dynamic linking, nor really of name
mangling. The issue is one of binary compatibility for everything used
in the interface. Thus, for example, unless I'm badly mistaken, the
definition of std::string changed between VC6 and VC9, with a
different layout. So if any of the code uses std::string, you'll have
to recompile, or get strange and unexplicable errors at runtime.
In general, it's best to assume no binary compatibility as soon as
different versions of the compiler, or even different compilation
options are involved, unless the vendor guarantees otherwise. (Although
some common sense is in order: you can freely mix options which only
control warnings, for example. But beware of /Ds which cause added
debugging code to be generated or not.)
If the application is unchanged it needs the same set of symbols from the library. And hence perhaps you can link to the library compiled with VC6.0. Name mangling is not the concern at all unless both the application and library are the same as the compatible(working) ones in VC6.0.
Should I also migrate the SomeExtStaticLib.lib to VS2008 compiled one?
There are compatibility problems between VC6.0 and visual 2008. So YES you should rebuild yor library with Visual 2008.
Just because you can link to the library as is doesn't mean it will work correctly.
Are static libraries eliminating the name mangling issues?
Not really. They are not doing anything special at all.
Static libraries have nothing to do with or without name mangling.... if your code is C++ there is mangling, and if its C (or extern "C" in C++) then there is no mangling. As long as the library and the code that link it agree, there's no problem to link in the library.
Over the months I've written some nice generic enough functionality that I want to build as a library and link dynamically against rather than importing 50-odd header/source files.
The project is maintained in Xcode and Dev-C++ (I do understand that I might have to go command line to do what I want) and have to link against OpenGL and SDL (dynamically in SDL's case). Target platforms are Windows and OS X.
What am I looking at at all?
What will be the entry point of my
library if it needs one?
What do I have to change in my code?
(calling conventions?)
How do I release it? My understanding
is that headers and the compiled
library (.dll, .dylib(, .framework),
whatever it'll be) need to be
available for the project -
especially as template functionality
can not be included in the library by
nature.
What else I need to be aware of?
I'd recommend building as a statc library rather than a DLL. A lot of the issues of exporting C++ functions and classes go away if you do this, provided you only intend to link with code produced by the same compiler you built the library with.
Building a static library is very easy as it is just an collection of .o/.obj files - a bit like a ZIP file but without compression. There is no need to export anything - just include the library in the list of files that your application links with. To access specific functions or classes, just include the relevant header file. Note you can't get rid of header files - the C++ compilation model, particularly for templates, depends on them.
It can be problematic to export a C++ class library from a dynamic library, but it is possible.
You need to mark each function to be exported from the DLL (syntax depends on the compiler). I'm poking around to see if I can find how to do this from xcode. In VC it's __declspec(dllexport) and in CodeWarrior it's #pragma export on/#pragma export off.
This is perfectly reasonable if you are only using your binary in-house. However, one issue is that C++ methods are named differently by different compilers. This means that nobody who uses a different compiler will be able to use your DLL, unless you are only exporting C functions.
Also, you need to make sure the calling conventions match in the DLL and the DLL's client. This either means you should have the same default calling convention flag passed to the compiler for both the DLL or the client, or better, explicitly set the calling convention on each exported function in the DLL, so that it won't matter what the default is for the client.
This article explains the naming issue:
http://en.wikipedia.org/wiki/Name_decoration
The C++ standard doesn't define a standard ABI, and that's bad news for people trying to build C++ libraries. This means that you get different behavior from your compiled code depending on which flags were used to compile it, and that can lead to mysterious bugs in code that compiles and links just fine.
This extends beyond just different calling conventions - C++ code can be compiled to support or not support RTTI, exception handling, and with various optimizations that can affect the the memory layout of class instances, which C++ code relies on.
So, what can you do? I would build C++ libraries inside my source tree, and make sure that they're built as part of my project's build, and that all the libraries and the code that links to them use the same compiler flags.
Note that name mangling, which was supposed to at least prevent you from linking object files that were compiled with different compilers/compiler flags only mostly works, and there are certain things you can do, especially with GCC, that will result in code that links just fine and fails at runtime.
You have to be extra careful with vendor supplied dynamic C++ libraries (QT on most Linux distributions, for example.) I've seen instances of vendor supplied libraries that were compiled in ways that prevented certain things from working properly. For example, some Redhat Linux releases (maybe all of them) disabled exceptions in QT, which made it impossible to catch exceptions in main() if the exceptions were thrown in a QT callback. Fun.