I wanted to write a unicode version of std::exception and std::runtime_error.
So I thought what better way then to just take implementations from the C++ Standard Library and alter them to support unicode.
So I pulled up the exception and stdexcept headers in Visual C++, copied the code, made my changes.
The thing is I couldn't get it to link unless I removed the _CRTIMP_PURE.
I also removed the _EXCEPTION_INLINE __CLR_OR_THIS_CALL prefix from all the member functions.
It's working but I'm very curious what all those things did.
_EXCEPTION_INLINE it literally defined right above it as #define _EXCEPTION_INLINE, and my googling skills can't find any documentation on what they do.
So, does anyone know what these are meant to do? And why it wound't link until I removed the _CRTIMP_PURE prefix from the class?
These aren't really anything mysterious (but it might be a bit pf a pain to track down where they're defined - but only a little). They're defined in headers that are part of the library, and they take on different definitions depending on how the compiler is configured for the current run. In particular, these macros seem to be concerned mostly with whether or not the current run is configured for /clr:pure.
_CRTIMP_PURE is defined to __declspec(dllimport) if you're linking against the DLL version of the C runtime (and not building with /clr:pure), and defined to nothing otherwise.
If your library isn't a DLL (or if it won't necessarily be a DLL whenever the DLL runtime is configured), then you shouldn't use it. You probably shouldn't use it anyway, because you'd need to define it differently when building your library than when your library is being used (that's what Microsoft does when they build the C runtime libraries).
__CLR_OR_THIS_CALL is used by Microsoft's libraries to declare a function with __clrcall if you're building with /clr:pure (indicating these functions will only be called by managed code - the compiler can perform certain optimizations in that case it seems).
Finally, _EXCEPTION_INLINE is used to make the member functions of class exception inline if building with /clr:pure.
So the bottom line is, don't use __CLR_OR_THIS_CALL or _EXCEPTION_INLINE unless you plan to support /clr for your library, and you probably shouldn't use _CRTIMP_PURE in your implementation, but probably should use something similar of your own making and under your own control.
Standard library implementation code often uses compiler-specific extensions for performance/documentation/reliability reasons. You should not use these extensions in your own code, because they can break in subsequent versions of the compiler.
There's no problem in duplicating the interface of std::exception and friends, and you can look at Visual Studio's implementation for inspiration. But your implementation should only use publically documented language/library features.
Related
We have a lot of prebuilt libraries (via CMake mostly), built using Visual Studio 2017 v141. When we try to use these against a project using Visual STudio 2019 v142 we see errors like:
Error C1047 The object or library file
‘boost_chrono-vc141-mt-gd-x32-1_68.lib’ was created by a different
version of the compiler than other objects...
On the other hand, we also use pre-compiled .libs from 3rd-party vendors which are over a decade old and these have worked just fine when linked against our codebase.
What determines whether a library needs to be rebuilt, and why can some ancient libraries still be used when others that are only one version behind cannot?
ABI incompatibilities could cause some issues. Even though the C++ standard calls for objects such as std::vector and std::mutex and that they need to have specific public/protected members, how these classes are made is left to the implementation.
In practice, it means that nothing prevents the GNU standard library from having their data fields in another orders than the LLVM standard library, or having completely different private members.
As such, if you try to use a function from a library built with the LLVM libc++ by sending it a GNU libstdc++ vector it causes UB. Even on the same standard library, different versions could have changed something and that could be a problem.
To avoid these issues, popular C++ libraries only use C data structures in their ABIs since (at least for now) every compiler produces the same memory layout for a char*, an int or a struct.
These ABI issues can appears in two places:
When you use dynamic libraries (.so and .dll files) your compiler probably won't say anything and you'll get undefined behavior when you call a function of the library using incompatible C++ objects.
When you use static libraries (.a and .lib files) I'm not really sure, I'm guessing it could either print an error if it sees there's gonna be a problem or successfully compile some Frankenstein monster of a binary that will behave like the above point
I will try to answer some integral parts, but be aware this answer could be incomplete. With more information from peers we will maybe be able to construct a full answer!
The simples kind of linking is linking towards a C library. Since there is no concept of classes and overloading function names, the compiler creators are able to create entry points to functions by their pure name. This seems to be pretty much quasi-standardized since, I myself, haven't encountered a pure C library not at least linkable to my projects. You can select this behaviour in C++ code by prepending a function declaration with extern "C". (This also makes it easy to link against a library from C# code) Here is a detailed explanation about extern "C". But as far as I am aware this behaviour is not standardized; it is just so simple - it seems - there is just one sane solution.
Going into C++ we start to encounter function, variable and struct names repeating. Lets just talk about overloaded functions here. For that compiler creators have to come up with some kind of mapping between void a(); void a(int x); void a(char x); ... and their respective library representation. Since this process also is not standardized (see this thread) and this process is far more complex than the 1 to 1 mapping of C, the ABIs of different compilers or even compiler versions can differ in any way.
Now given two compilers (or linkers I couldn't find a resource wich specifies wich one exactly is responsible for the mangling but since this process is not standardized it could be also outsourced to cthulhu) with different name mangling schemes, create following function entry points (simplified):
compiler1
_a_
_a_int_
_a_char_
compiler2
_a_NULL_
_a_++INT++_
_a_++CHAR++_
Different linkers will not understand the output of your particular process; linker1 will try to search for _a_int_ in a library containing only _a_++INT++_. Since linkers can't use fuzzy string comparison (this could lead to a apocalypse imho) it won't find your function in the library. Also don't be fooled by the simplicity of this example: For every feature like namespace, class, method etc. there has to be a method implemented to map a function name to a entry point or memory structure.
Given your example you are lucky you use libraries of the same publisher who coded some logic to detect old libraries. Usually you will get something along the lines of <something> could not be resolved or some other convoluted, irritating and/or unhelpful error message.
Some info and experience dump regarding Visual Studio and libraries in general:
In general the Visual C++ suite doesn't support crosslinked libs between different versions but you could be lucky and it works. Don't rely on it.
Since VC++ 2015 the ABI of the libraries is guaranteed by microsoft to be compatible as drescherjm commented: link to microsoft documentation
In general when using libraries from different suites you should always be cautious as n. 1.8e9-where's-my-share m. commented here (here is your share btw) about dependencies to other libraries and runtimes. In general in general not having the control over how libraries are built is a huge pita
Edit addressing memory layout incompatibilities in addition to Tzigs answer: different name mangling schemes seem to be partially intentional to protect users against linkage against incompatible libraries. This answer goes into detail about it. The relevant passage from gcc docs:
G++ does not do name mangling in the same way as other C++ compilers. This means that object files compiled with one compiler cannot be used with another.
This effect is intentional [...].
Error C1047
This is caused by /GL Global optimization or /LTGC Link Time Code Generation
These use information in the .obj, to perform global optimizations. When present, VS looks at the compiler which generated the original .lib, and if they are different emits the error. These compilation switches are for code from a single compiler, and not intended for cross version usage.
The other builds which work, don't have the switches, so are compatible.
Visual studio has started to use a new #pragma detect_mismatch
This causes an old build to identify it is incompatible with a new build, by detecting the version change.
Very old builds didn't have / support the pragma, so had no checking.
When you build a lib, its dependencies are loaded and satisified by the linker, but this is not a guarantee of working. The one-definition-rule signs the developer up to a contract, that within a compiled binary, all implementations of the same named function are the same. If this came from different compilers, that may not be true, and so the linker can choose any, causing latent bugs, where mixtures of old and new code are linkeded into the binary.
If the definition or implementation of std::string has changed, it may link, but have code which is flawed.
This new compiler check, causes a fail early, which I thoroughly approve of.
I've been looking into some of the features of the "newer" C++ standards (C++11 and C++14), and that got me thinking about something. I'm currently using the VC++2008 compiler for my projects (for various reasons), which means that the newest standard I have access to is C++03, plus TR1. TR1 has some nice things in it, but there are features in C++11 and C++14 that would be nice to have.
My question is this: Would there be any way that I could build some code using a newer compiler (say MSVC2012 or 2013) to build libraries or DLLs using the newer C++11 and C++14 functionality and then link that into my project that's running the '08 compiler?
The only thing that I could think of that wouldn't work would be anywhere I had to have a C++11 or C++14 feature in a header included by my '08 compiler project. However as long as everything "new" were hidden behind my interface, shouldn't this work?
Yes but its going to get ugly.. since the ABI is not compatible you'll have to go down to the "extern "C" {}" ABIness.
That means you can't pass C++ objects at all.. like I said, painful. It also means it must be a DLL since you won't be able to link in a static lib with another ABI.
Its up to you if its worth wrapping up a DLL in a C API just to use a couple of new features or not, I would recommend just upgraded the whole project though.
I almost forgot, you probably can't link the import lib either, so you'll have to have some code that uses LoadLibrary, GetProcAddress and FreeLibrary (did I mention this is ugly/painful?).
Unfortunately, what you're trying to do is not possible with MSVC. They intentionally break binary compatibility with every major release as stated in MSDN documentation:
To enable new optimizations and debugging checks, the Visual Studio implementation of the C++ Standard Library intentionally breaks binary compatibility from one version to the next. Therefore, when the C++ Standard Library is used, object files and static libraries that are compiled by using different versions can't be mixed in one binary (EXE or DLL), and C++ Standard Library objects can't be passed between binaries that are compiled by using different versions. Such mixing emits linker errors about _MSC_VER mismatches. (_MSC_VER is the macro that contains the compiler's major version—for example, 1800 for Visual C++ in Visual Studio 2013.) This check cannot detect DLL mixing, and cannot detect mixing that involves Visual C++ 2008 or earlier.
Your options are to then only pass around POD types, or implement COM interfaces to interop between the DLLs compiled using different version of the VC compiler, neither of which is particularly palatable.
My advice would be, if you must stick with VS2008 for certain legacy applications, suck it up and deal with the feature set it supports (at least you have TR1). For newer projects, try and talk your team into using newer versions of VC.
Assume that I compile the code in C++11 (I'll use Lambdas) to ".o" or library ".a".
And I have a program, where I will include previous library and header file, that I can't compile with C++11, but old one ( C++98 ).
Will it compile and work fine?
It will work fine if:
the (public) header doesn't use any C++11 features
the ABI hasn't changed
consult your platform/compiler on this one
no common dependency has changed
as per the GCC document linked by Vaughn Cato, this includes the standard library. Anything that generates different code or object layouts when compiled with C++11, and is used by both library and client may be a problem ... even if it isn't used in the interface itself.
If point 3 is your only issue, you may be able to get around it by compiling a dynamic library (depending on platform a .so, or a .dynlib, or a DLL as Adrian suggests) with all dependencies statically linked internally and not exported. It's a bit hairy though.
Probably not. The reason that name mangling (why the ABI changes) in c++ exists is because incompatibility differences between c++ versions can make code unstable if it will work at all.
If you have code that doesn't compile against c++11, you'll probably have to refactor one of your programs to compile against the other compiler. (Most likely get your old code to compile with the new compiler)
If this isn't an option, you can try and make the c++11 lib a DLL with a C interface or with a COM object interface, but exceptions would stop at that boundary, and if you go the DLL route, you would more than likely want to write a wrapper class to access the c++11 object, so that it acts like an object on your pre c++11 side of the boundary.
One common approach is to provide a C version of the API (extern "C" functions) with objects passed around using opaque pointers. This is more likely to be compatible between languages and compilers.
I'm working on a certain C++ library (or more framework). I want to make it backward
compatible with previous versions preserving not only API compatibility but also ABI (like the great job Qt does).
I use lots of functionality of Boost and it seems for me that this makes backward compatibility just impossible, unless I force a user to have exactly the same (sometimes old) version of Boost.
Is there any way (without rewriting 1/2 of Boost) to make some "prefix" around its namespace/rename it in order to prevent it from interfering with a user version of Boost?
For example my libXYZ uses Boost 1.33 and it has class boost::foo. In version 1.35 boost::foo was upgraded and new member was added, so, boost::foo from 1.33 and 1.35 are
not ABI compatible. So, a user of libXYZ must use Boost 1.33 or recompile libXYZ with
Boost 1.35 (that may be already had broken some API in a way XYZ would not compile).
Note: I'm talking about UNIX/Linux OS with ELF where dynamic linking is similar to static linking, so you can't link with two different versions of libraries because symbols would interfere.
One suitable solution I may think of is putting Boost in some other private namespace. So, libXYZ would use ::XYZ::boost::foo instead of ::boost::foo. This would prevent collision with other version of Boost that user may use.
So, the libXYZ would continue to work with Boost 1.33 statically or dynamically linked with it withing other namespace, assuming, that it:
Would not expose Boost API outside.
Would keep stable private version of exposed API.
Is there any way to do such things with Boost?
Edit: Finally I decided to create a script that would rename all boost symbols in the source to some custom symbol.
Rationale: simplification of build process, independent of compiler visibility support, also, it visibility works on dynamic libraries only, for static this does not work, so I need separate builds and dependencies for each type of libraries.
The script is available there: http://art-blog.no-ip.info/files/rename.py
Edit 2: Latest version of Boost BCP supports namespace renaming.
Basically, just make sure the public interface to your library does not expose Boost. You can always use it however much you want internally. Generally, having the interface of a library depend on another library is bad (unless it depends on a standard library like STL). Boost almost fits into the "standard" library category, but its ABI changes so much that that your interface shouldn't use it.
To make sure you don't expose the Boost symbols, there are a few things you could do:
A. compile with -fvisibility=hidden and mark all public symbols with __attribute__((visibility("default"))). you could use a macro to make this easier:
#define ABI __attribute__((visibility("default")))
B. do something like this:
#pragma GCC visibility push(hidden)
#include <boost/whatever.hpp>
#pragma GCC visibility pop
You should also wrap this around all other internal symbols that you don't want exported, or declare this with __attribute__((visibility("hidden"))). Again, you could use a macro to make this easier:
#define INTERNAL __attribute__((visibility("hidden")))
Of these options, I like A better, because it makes you explicitly think about which symbols are exported, so you don't accidentally export things you don't want.
By the way, you can find a lot more information about making DSOs in Ulrich Drepper's How to Write Shared Libraries.
In general you can't rely on any type of ABI in C++ beyond standard C bindings. But Depending on how many assumptions you make, you can use more and more of C++ in your interface.
I found this great article on steps to make your API's turn into a stable ABI. For instance, never pass Standard C++ Library (or Boost) datatypes across your interface; it might break even with a small bug fix to the library.
Some examples of the issues to watch out for when publishing an ABI compatible API are:
Windows Debug heap. You have to be sure that all allocations and deallocations are on the same side of a "module" (that is, executable or DLL).
Fragile Binary Interface problem. Even if both sides of your system consistently use the same compiler and libraries, you have to be careful in C++ about what you publish in your .h files, and where allocations happen.
If you follow the linked article, you will find solutions for these and other issues.
Edit:
I also found an interesting article published by Microsoft which describes how COM interfaces work, by taking a C++ project and turning it into COM. It's my belief that one of the primary reasons that Microsoft developed COM was to solve the Fragile Binary Interface problem that C++ has, so they can ship DLLs with publish object oriented APIs.
Consider using abi-compliance-checker tool to maintain a stable API/ABI interface.
You should be able to do something like this:
namespace XYZ
{
#include <boost/my_library.hpp>
}
And it should dump the boost headers into namespace XYZ. Note, however, that this will only work with the header-only libraries.
Over the months I've written some nice generic enough functionality that I want to build as a library and link dynamically against rather than importing 50-odd header/source files.
The project is maintained in Xcode and Dev-C++ (I do understand that I might have to go command line to do what I want) and have to link against OpenGL and SDL (dynamically in SDL's case). Target platforms are Windows and OS X.
What am I looking at at all?
What will be the entry point of my
library if it needs one?
What do I have to change in my code?
(calling conventions?)
How do I release it? My understanding
is that headers and the compiled
library (.dll, .dylib(, .framework),
whatever it'll be) need to be
available for the project -
especially as template functionality
can not be included in the library by
nature.
What else I need to be aware of?
I'd recommend building as a statc library rather than a DLL. A lot of the issues of exporting C++ functions and classes go away if you do this, provided you only intend to link with code produced by the same compiler you built the library with.
Building a static library is very easy as it is just an collection of .o/.obj files - a bit like a ZIP file but without compression. There is no need to export anything - just include the library in the list of files that your application links with. To access specific functions or classes, just include the relevant header file. Note you can't get rid of header files - the C++ compilation model, particularly for templates, depends on them.
It can be problematic to export a C++ class library from a dynamic library, but it is possible.
You need to mark each function to be exported from the DLL (syntax depends on the compiler). I'm poking around to see if I can find how to do this from xcode. In VC it's __declspec(dllexport) and in CodeWarrior it's #pragma export on/#pragma export off.
This is perfectly reasonable if you are only using your binary in-house. However, one issue is that C++ methods are named differently by different compilers. This means that nobody who uses a different compiler will be able to use your DLL, unless you are only exporting C functions.
Also, you need to make sure the calling conventions match in the DLL and the DLL's client. This either means you should have the same default calling convention flag passed to the compiler for both the DLL or the client, or better, explicitly set the calling convention on each exported function in the DLL, so that it won't matter what the default is for the client.
This article explains the naming issue:
http://en.wikipedia.org/wiki/Name_decoration
The C++ standard doesn't define a standard ABI, and that's bad news for people trying to build C++ libraries. This means that you get different behavior from your compiled code depending on which flags were used to compile it, and that can lead to mysterious bugs in code that compiles and links just fine.
This extends beyond just different calling conventions - C++ code can be compiled to support or not support RTTI, exception handling, and with various optimizations that can affect the the memory layout of class instances, which C++ code relies on.
So, what can you do? I would build C++ libraries inside my source tree, and make sure that they're built as part of my project's build, and that all the libraries and the code that links to them use the same compiler flags.
Note that name mangling, which was supposed to at least prevent you from linking object files that were compiled with different compilers/compiler flags only mostly works, and there are certain things you can do, especially with GCC, that will result in code that links just fine and fails at runtime.
You have to be extra careful with vendor supplied dynamic C++ libraries (QT on most Linux distributions, for example.) I've seen instances of vendor supplied libraries that were compiled in ways that prevented certain things from working properly. For example, some Redhat Linux releases (maybe all of them) disabled exceptions in QT, which made it impossible to catch exceptions in main() if the exceptions were thrown in a QT callback. Fun.