C++ Plugins ABI issues on Linux - c++

Question

I am working on a plugin system to replace shared libs.
I am aware of ABI issues when designing an API for shared libs and entry points in the libs, such as exported classes, should be carefully design.
For example, adding, removing or reordering private member variables of an exported class may lead to different memory layout and runtime errors (from my understanding, that's why the Pimpl pattern might be useful). Of course there are many other pitfalls to avoid when modifying exported classes.
I have built a small example here to illustrate my question.
First, i provide the following header for the plugin developer :
// character.h
#ifndef CHARACTER_H
#define CHARACTER_H
#include <iostream>
class Character
{
public:
virtual std::string name() = 0;
virtual ~Character() = 0;
};
inline Character::~Character() {}
#endif
Then the plugin is built as a shared lib "libcharacter.so" :
#include "character.h"
#include <iostream>
class Wizard : public Character
{
public:
virtual std::string name() {
return "wizard";
}
};
extern "C"
{
Wizard *createCharacter()
{
return new Wizard;
}
}
And finally the main application that uses the plugin :
#include "character.h"
#include <iostream>
#include <dlfcn.h>
int main(int argc, char *argv[])
{
(void)argc, (void)argv;
using namespace std;
Character *(*creator)();
void *handle = dlopen("../character/libcharacter.so", RTLD_NOW);
if (handle == nullptr) {
cerr << dlerror() << endl;
exit(1);
}
void *f = dlsym(handle, "createCharacter");
creator = (Character *(*)())f;
Character *character = creator();
cout << character->name() << endl;
dlclose(handle);
return 0;
}
Is it sufficient to define an abstract class to get rid of all ABI issues?

Answer 1

Is it sufficient to define an abstract class to get rid of all ABI issues?
Short answer:
No.
I wouldn't recommend using C++ for a plugin API (see longer answer below), but if you do decide to stick with C++ then:
Don't use any standard library types in your plugin API.
For instance, Character::name() returns a std::string. If the implementation of std::string ever changes (and it has in the past in GCC) then that will result in Undefined Behavior. Really, anything that you don't control (any third-party library) shouldn't be used in the API.
Don't use exceptions or RTTI across the plugin boundary. On Linux exceptions and RTTI might work if you load the plugin with RTLD_GLOBAL (not a good idea for plugins) and both the host and the plugin use the same runtime. But in general you either won't be able to catch exceptions from another module, or they might even cause heap corruption (if they are allocated by different runtimes).
Only add functions to the end of your abstract classes, or everything will silently break because of the vtable layout changing (and that can be really hard to diagnose).
Always allocate and deallocate an object from the same module. I noticed you don't have a destroyCharacter() function (main() actually leaks the character but that's another question). Always provide symmetric create and destroy functions for resources created by a different module (shared library or plugin).
I believe on Linux with GCC the host application's operator new and operator delete get correctly propagated to the loaded plugin (through weak symbols), but if you want it to work on Windows then don't assume that operator new and operator delete in the host application and the plugin are the same. A statically linked runtime, especially built with LTO, might also mess with this.
Longer answer:
There are a lot of possible issues when exporting a C++ API from a plugin.
Generally speaking, there are no guarantees about it working if anything about the toolchains used to build the host application and the plugin differs. This can include (but is not limited to) compilers, versions of the language, compiler flags, preprocessor definitions, etc.
The common wisdom regarding plugins is to use a pure C89 API, because the C ABI on all common platforms is very stable.
Keeping to the common subset of C89 and C++ will mean that the host and plugin can use different language standards, standard libraries, etc. Unless the host or the plugin are built with some weird (and probably non-standard-conforming) APIs, this should be reasonably safe. Obviously, you still have to be careful with data structure layouts.
You can then provide a rich C++ header-only wrapper for the C API that handles lifetime and error code/exception conversions, etc.
As a nice bonus, C APIs are producible and consumable by most languages, which could allow the plugin authors to use not just C++.
There are actually quite a few pitfalls even in a C API. If we're being pedantic then the only safe things are functions with fixed-size arguments and return types (pointers, size_t, [u]intN_t) - not even necessarily built-in types (short, int, long, ...), or enums. E.g. in GCC: -fshort-enums can change the size of enums, -fpack-struct[=n] can change the padding within structs.
So, if you really want to be safe then don't use enums and either pack all your structs or don't expose them directly (instead expose accessor functions).
Other considerations:
These aren't strictly related to the question but should definitely be considered before committing to a specific style of API.
Error handling: Whether or not you use C++, you'll need an alternative to exceptions.
This will probably be some form of error code. std::error_code in C++ can be then used to wrap the raw enum/int as soon as you're in C++ land, and if the API uses C++ then a std::expected-like or Boost.Outcome-like type with a stable ABI could be used.
Loading the plugin and importing symbols: With abstract classes it's easy - a simple factory function is all you need. With a traditional C API you might end up needing to import hundreds of symbols. One way of dealing with that would be to emulate a vtable in C. Make each object that has associated functions start with a pointer to a dispatch table, e.g.
typedef struct game_string_view { const char *data; size_t size; } game_string_view;
typedef enum game_plugin_error_code { game_plugin_success = 0, /* ... */ } game_plugin_error_code;
typedef struct game_plugin_character_impl *GamePluginCharacter; // handle to a Character
typedef struct game_plugin_character_dispatch_table { // basically a vtable
void (*destroy)(GamePluginCharacter character); // you could even put destroy() here
game_string_view (*name)(GamePluginCharacter character);
void (*update)(GamePluginCharacter character, /*...*/, game_plugin_error_code *ec); // might fail
} game_plugin_character_dispatch_table;
typedef struct game_plugin_character_impl {
// every call goes through this table and takes GamePluginCharacter as it's first argument
const game_plugin_character_dispatch_table *dispatch;
} game_plugin_character_impl;
Future extensibility and compatibility: You should design the API, knowing that you'll want to change it in the future and keep compatibility. IMO, a C API lends itself well to this because it forces you to be very precise in what is exposed. The plugin should be able to expose it's API version to the host in a way that is forward and backward compatible.
It's a good idea to think about extensibility when designing each function signature. E.g. if a struct is passed by pointer (instead of by value), then it's size can be extended without breaking compatibility (so long as at run time both the caller and the callee agree on it's size).
Visibility: Maybe look into visibility on Linux and other platforms. This isn't really a question of API design, just helps clean up the symbols exported from a shared library.
All of the above is by no means extensive.
I would suggest the talk "Hourglass Interfaces for C++ APIs" as further "reading".
And of course there other good talks and articles on the matter (that I can't remember of the top of my head).