I have a library which has a basic class which is used extensively by the particular library in question (say library_1).
namespace library_1 {
class some_class {
}
}
I have want this library to use instead another version of this class that I am defining.
namespace my_own {
class some_class {
}
}
my_own::some_class and library_1::some_class are going to have the same public interface (but different data members so they are not exactly dynamic castable). So I want to be able to compile this library replacing just this one class. This is doable.
The complication in this whole process, however, is that I have a second library (whose source code I do not have access to, call it library_2) which makes use of the first library (including accesses to some_class).
My main executable needs to access both library_2 (which is compiled against the original library) and a different version of the library_1 with this some_class replaced.
I know this is a complicated situation but what is the best way to achieve this (from a code perspective and about how to maintain this in version control)?
What you can do is to expose just the API for the part where you need to use your replace class and compile the corresponding portion into a dynamically linked library, statically resolving all symbols to the library you are meddling with. Obviously, the meddled with objects shall not escape this interface. With this, you program can effectively use two conflicting implementations of the same library although they won't be able to shared objects. Essentially, this is how COM exposes its interface but this technique works on other platforms than Windows although I wouldn't be come up with the needed steps in creating a shared library doing this for a UNIX system.
Related
I am trying to limit the inheritance of an c++ class to within the same library, while allowing it to be instantiated in other libraries.
The use case is that I have some code that needs to be real-time capable (compiled with special flags and poisoned code) and that needs to be used/interfaced to non-RT code. However, I need to make absolutely sure that no non-RT code can ever be called inside the RT code. Therefore I have to libraries: one that is RT capable and one that isn't (which depends on the RT library and may use code from it).
Now, I have some abstract classes, which I want to be only inherited from inside of the RT library. Is it possible to prohibit the inheritance of those ABCs from classes defined outside of the RT library?
What I came up so far (without it working) is defining a macro that makes the classes final outside of RT code and a templated base class that uses std::conditional
class BaseA REALTIME_FINAL
{
virtual void foo() = 0;
}
template <bool allow = REALTIME_TRUE>
class BaseB : : virtual public std::conditional<allow, std::true_t, std::nullptr_t>::type
{
virtual void foo() = 0;
}
While both of these methods prohibit the inheritance from the abstract base, it also makes it impossible in the non-RT library to call or instantiate (or even include the header) any classes derived from it in the RT lib.
You can solve this problem much more simply, by moving your realtime code into its own library with its own header files, and building it without any dependency on your non-realtime library.
Put your realtime and non-realtime headers into separate directories, and if your realtime code is built as a shared library, use the linker option to prohibit undefined symbols in that library.
Then all you have to do is remember not to add your system's equivalent of -Inon-realtime or -lnon-realtime in the realtime library's build configuration.
One thing you need to think hard is - you can solve this problem today but tomorrow an intern makes changes that sneak into your code which compiles but will break in the hands of your client. So I am quite dramatic on adopting a solution that does not leave anything to imagination.
My go-to approach is always to separate libraries with a C API. This guarantees that no C++ features, which compilers are frequently guilty of manipulating, can be tampered.
More often I use this to encapsulate code that was compiled with the old ABI (pre-C++11) which is unfortunately still common from certain vendors.
This also guarantees that I can access these features from a CD/CI perspective from other scripting languages like Python, Java, Rust more easily.
In file gperftools-2.2.1/src/gperftools/malloc_extension.h, it reads:
// Extra extensions exported by some malloc implementations. These
// extensions are accessed through a virtual base class so an
// application can link against a malloc that does not implement these
// extensions, and it will get default versions that do nothing.
//
// NOTE FOR C USERS: If you wish to use this functionality from within
// a C program, see malloc_extension_c.h.
My question is how exactly can I access these extensions through a virtual base class?
Usually to load a class from a dynamic library, I would need to write a base class which allows me to get an instance of the wanted class and its functions through polymorphism, as described here.
However to do so there must be some class factory functions available in the API, but there are no such functions in any tcmalloc files. Moreover I would also need to load the tcmalloc library with dlopen(), which is not recommended according to the install note:
...loading a malloc-replacement library via dlopen is
asking for trouble in any case: some data will be allocated with one malloc, some with another.
So clearly accessing the extensions through the typical way as mentioned above is not an option. I can get away with using the C versions as declared in malloc_extensions_c.h but just wonder if there is any better solution.
I managed to load the malloc extensions via some 'hack', which is not as clean as I would prefer, but it gets the job done. Here is the (temporary) solution for those who are interested in.
First, most of these malloc extension functions are similar to static functions in a way that they are mostly called on the current instance only, e.g. to call the GetMemoryReleaseRate() function on the current process you just call MallocExtension::instance()->GetMemoryReleaseRate(). Therefore we don't need to create a base class and get an instance of MallocExtension class to call these functions.
For the example above, I'd just create a standalone function getMemoryReleaseRate() which simply calls the required function when it gets called, as below:
getMemoryReleaseRate()
{
MallocExtension::instance()->GetMemoryReleaseRate();
}
This function can be inserted directly to a source file, e.g. tcmalloc.cc, or, if you prefer not to edit the tcmalloc source every time there is a new version, added to your makefile, to be attached to the source file when it is compiled.
Now in your code, you can call the MallocExtension function via the 'facade' function you have created via dlsym(), e.g. as below:
typedef void (*getMemoryReleaseRate)();
((getMemoryReleaseRate)dlsym(RTLD_DEFAULT, "getMemoryReleaseRate"))();
Simply including this header and doing MallocExtension::instance()->GetMemoryReleaseRate(); would work too. No need to modify tcmalloc for that.
I am searching for a simple, light-weight solution for interface-based runtime object composition in C++. I want to be able to specify interfaces (methods declarations), and objects (creatable through factory pattern) implementing these. At runtime I want mechanisms to instantiate these objects and interconnect these based on interface-connectors. The method calls at runtime should remain fairly cheap, i.e. only several more instructions per call, comparable to functor patterns.
The whole thing needs to be platform independent (at least MS Windows and Linux). And the solution needs to be licensed liberally, like open source LGPL or (even better) BSD or something, especially allowing use commercial products.
What I do not want are heavy things like networking, inter-process-communication, extra compiler steps (one-time code generation is ok though), or dependencies to some heavy libraries (like Qt).
The concrete scenario is: I have such a mechanism in a larger software, but the mechanism is not very well implemented. Interfaces are realized by base classes exported by Dlls. These Dlls also export factory functions to instantiate the implementing objects, based on hand-written class ids.
Before I now start to redesign and implement something better myself, I want to know if there is something out there which would be even better.
Edit: The solution also needs to support multi-threading environments. Additionally, as everything will happen inside the same process, I do not need data serialization mechanisms of any kind.
Edit: I know how such mechanisms work, and I know that several teaching books contain corresponding examples. I do not want to write it myself. The aim of my question is: Is there some sort of "industry standard" lib for this? It is a small problem (within a single process) and I am really only searching for a small solution.
Edit: I got the suggestion to add a pseudo-code example of what I really want to do. So here it is:
Somewhere I want to define interfaces. I do not care if it's C-Headers or some language and code generation.
class interface1 {
public:
virtual void do_stuff(void) = 0;
};
class interface2 {
public:
virtual void do_more_stuff(void) = 0;
};
Then I want to provide (multiple) implementations. These may even be placed in Dll-based plugins. Especially, these two classes my be implemented in two different Dlls not knowing each other at compile time.
class A : public interface1 {
public:
virtual void do_stuff(void) {
// I even need to call further interfaces here
// This call should, however, not require anything heavy, like data serialization or something.
this->con->do_more_stuff();
}
// Interface connectors of some kind. Here I use something like a template
some_connector<interface2> con;
};
class B : public interface2 {
public:
virtual void do_more_stuff() {
// finally doing some stuff
}
};
Finally, I may application main code I want to be able to compose my application logic at runtime (e.g. based on user input):
void main(void) {
// first I create my objects through a factory
some_object a = some_factory::create(some_guid<A>);
some_object b = some_factory::create(some_guid<B>);
// Then I want to connect the interface-connector 'con' of object 'a' to the instance of object 'b'
some_thing::connect(a, some_guid<A::con>, b);
// finally I want to call an interface-method.
interface1 *ia = a.some_cast<interface1>();
ia->do_stuff();
}
I am perfectly able to write such a solution myself (including all pitfalls). What I am searching for is a solution (e.g. a library) which is used and maintained by a wide user base.
While not widely used, I wrote a library several years ago that does this.
You can see it on GitHub zen-core library, and it's also available on Google Code
The GitHub version only contains the core libraries, which is really all the you need. The Google Code version contains a LOT of extra libraries, primarily for game development, but it does provide a lot of good examples on how to use it.
The implementation was inspired by Eclipse's plugin system, using a plugin.xml file that indicates a list of available plugins, and a config.xml file that indicates which plugins you would like to load. I'd also like to change it so that it doesn't depend on libxml2 and allow you to be able to specify plugins using other methods.
The documentation has been destroyed thanks to some hackers, but if you think this would be useful then I can write enough documentation to get you started.
A co-worker gave me two further tips:
The loki library (originating from the modern c++ book):
http://loki-lib.sourceforge.net/
A boost-like library:
http://kifri.fri.uniza.sk/~chochlik/mirror-lib/html/
I still have not looked at all the ideas I got.
Here is what I am trying to do:
I am developing a cross-platform IDE (Linux and Windows) that supports plug-ins. I need to support extensibility using an adapter framework similar to the one that Eclipse provides. See here for more details, but basically I need the following:
Let Adaptee and Adapted be completely unrelated classes which already exist and which we are not allowed to change in any way. I want to create an AdapterManager class which has a method
template <class Adaptee, class Adapted> Adapted* adapt( Adaptee* object);
which will create an instance of Adapted given an instance of Adaptee. How exactly the instance is created depends on an adapter function which will have to be registered with AdapterManager. Each new plug-in should be able to contribute adapter functions for arbitrary types.
Here are my thoughts about a possible solution and why it does not work:
C++11's RTTI functions and the type_info class provide a hash_code() method which returns a unique integer for each type in the program. See here. Thus AdapterManager could simply contain a map that given the hash codes for the Adaptee and Adapter classes returns a function pointer to the adapter function. This makes the implementation of the adapt() function above trivial:
template <class Adaptee, class Adapted> Adapted* AdapterManager::adapt( Adaptee* object)
{
AdapterMapKey mk( typeid(Adapted).hash_code(), typeid(Adaptee).hash_code());
AdapterFunction af = adapterMap.get(mk);
if (!af) return nullptr;
return (Adapted*) af(object);
}
Any plug-in can easily extend the framework by simply inserting an additional function into the map. Also note that any plug-in can try to adapt any class to any other class and succeed if there exists a corresponding adapter function registered with AdapterManager regardless of who registered it.
A problem with this is the combination of templates and plug-ins (shared objects / DLLs). Since two plug-ins can instantiate a template class with the same parameters, this could potentially lead to two separate instances of the corresponding type_info structures and potentially different hash_code() results, which will break the mechanism above. Adapter functions registered from one plug-in might not always work in another plug-in.
In Linux, the dynamic linker seems to be able to deal with multiple declarations of types in different shared libraries under some conditions according to this (point 4.2). However the real problem is in Windows, where it seems that each DLL will get its own version of a template instantiation regardless of whether it is also defined in other loaded DLLs or the main executable. The dynamic linker seems quite inflexible compared to the one used in Linux.
I have considered using explicit template instantiations which seems to reduce the problem, but still does not solve it as two different plug-ins might still instantiate the same template in the same way.
Questions:
Does anyone know of a way to achieve this in Windows? If you were allowed to modify existing classes, would this help?
Do you know of another approach to achieve this functionality in C++, while still preserving all the desired properties: no change to existing classes, works with templates, supports plug-ins and is cross-platform?
Update 1:
This project uses the Qt framework for many things including the plug-in infrastructure. Qt really helps with cross platform development. If you know of a Qt specific solution to the problem, that's also welcome.
Update 2:
n.m.'s comment made me realize that I only know about the problem in theory and have not actually tested it. So I did some testing in both Windows and Linux using the following definition:
template <class T>
class TypeIdTest {
public:
virtual ~TypeIdTest() {};
static int data;
};
template <class T> int TypeIdTest<T>::data;
This class is instantiated in two different shared libraries/DLLs with T=int. Both libraries are explicitly loaded at run-time. Here is what I found:
In Linux everything just works:
The two instantiations used the same vtable.
The object returned by typeid was at the same address.
Even the static data member was the same.
So the fact that the template was instantiated in multiple dynamically loaded shared libraries made absolutely no difference. The linker seems to simply use the first loaded instantiation and ignore the rest.
In Windows the two instantiations are 'somewhat' distinct:
The typeid for the different instances returns type_info objects at different addresses. These objects however are equal when tested with ==. The corresponding hash codes are also equal. It seems like on Windows equality between types is established using the type's name - which makes sense. So far so good.
However the vtables for the two instances were different. I'm not sure how much of a problem this is. In my tests I was able to use dynamic_cast to downcast an instance of TypeIdTest to a derived type across shared library boundaries.
What's also a problem is that each instantiation used its own copy of the static field data. That can cause a lot of problems and basically disallows static fields in template classes.
Overall, it seems that even in Windows things are not as bad as I thought, but I'm still reluctant to use this approach given that template instantiations still use distinct vtables and static storage. Does anyone know how to avoid this problem? I did not find any solution.
I think Boost Extension deals with exactly this problem domain:
http://boost-extension.redshoelace.com/docs/boost/extension/index.html
(in preparation for this library's submission to Boost for review)
In particular you'd be interested in what the author wrote in this blog post: "Resource Management Across DLL Boundaries:
RTTI does not always function as expected across DLL boundaries. Check out the type_info classes to see how I deal with that.
I'm not sure whether his solution is actually robust, but he sure gave this thought, before. In fact, there are some samples using Boost Extensions that you can give a go, you might want to use it.
This question is related to "How to make consistent dll binaries across VS versions ?"
We have applications and DLLs built
with VC6 and a new application built
with VC9. The VC9-app has to use
DLLs compiled with VC6, most of
which are written in C and one in
C++.
The C++ lib is problematic due to
name decoration/mangling issues.
Compiling everything with VC9 is
currently not an option as there
appear to be some side effects.
Resolving these would be quite time
consuming.
I can modify the C++ library, however it must be compiled with VC6.
The C++ lib is essentially an OO-wrapper for another C library. The VC9-app uses some static functions as well as some non-static.
While the static functions can be handled with something like
// Header file
class DLL_API Foo
{
int init();
}
extern "C"
{
int DLL_API Foo_init();
}
// Implementation file
int Foo_init()
{
return Foo::init();
}
it's not that easy with the non-static methods.
As I understand it, Chris Becke's suggestion of using a COM-like interface won't help me because the interface member names will still be decorated and thus inaccessible from a binary created with a different compiler. Am I right there?
Would the only solution be to write a C-style DLL interface using handlers to the objects or am I missing something?
In that case, I guess, I would probably have less effort with directly using the wrapped C-library.
The biggest problem to consider when using a DLL compiled with a different C++ compiler than the calling EXE is memory allocation and object lifetime.
I'm assuming that you can get past the name mangling (and calling convention), which isn't difficult if you use a compiler with compatible mangling (I think VC6 is broadly compatible with VS2008), or if you use extern "C".
Where you'll run into problems is when you allocate something using new (or malloc) from the DLL, and then you return this to the caller. The caller's delete (or free) will attempt to free the object from a different heap. This will go horribly wrong.
You can either do a COM-style IFoo::Release thing, or a MyDllFree() thing. Both of these, because they call back into the DLL, will use the correct implementation of delete (or free()), so they'll delete the correct object.
Or, you can make sure that you use LocalAlloc (for example), so that the EXE and the DLL are using the same heap.
Interface member names will not be decorated -- they're just offsets in a vtable. You can define an interface (using a C struct, rather than a COM "interface") in a header file, thusly:
struct IFoo {
int Init() = 0;
};
Then, you can export a function from the DLL, with no mangling:
class CFoo : public IFoo { /* ... */ };
extern "C" IFoo * __stdcall GetFoo() { return new CFoo(); }
This will work fine, provided that you're using a compiler that generates compatible vtables. Microsoft C++ has generated the same format vtable since (at least, I think) MSVC6.1 for DOS, where the vtable is a simple list of pointers to functions (with thunking in the multiple-inheritance case). GNU C++ (if I recall correctly) generates vtables with function pointers and relative offsets. These are not compatible with each other.
Well, I think Chris Becke's suggestion is just fine. I would not use Roger's first solution, which uses an interface in name only and, as he mentions, can run into problems of incompatible compiler-handling of abstract classes and virtual methods. Roger points to the attractive COM-consistent case in his follow-on.
The pain point: You need to learn to make COM interface requests and deal properly with IUnknown, relying on at least IUnknown:AddRef and IUnknown:Release. If the implementations of interfaces can support more than one interface or if methods can also return interfaces, you may also need to become comfortable with IUnknown:QueryInterface.
Here's the key idea. All of the programs that use the implementation of the interface (but don't implement it) use a common #include "*.h" file that defines the interface as a struct (C) or a C/C++ class (VC++) or struct (non VC++ but C++). The *.h file automatically adapts appropriately depending on whether you are compiling a C Language program or a C++ language program. You don't have to know about that part simply to use the *.h file. What the *.h file does is define the Interface struct or type, lets say, IFoo, with its virtual member functions (and only functions, no direct visibility to data members in this approach).
The header file is constructed to honor the COM binary standard in a way that works for C and that works for C++ regardless of the C++ compiler that is used. (The Java JNI folk figured this one out.) This means that it works between separately-compiled modules of any origin so long as a struct consisting entirely of function-entry pointers (a vtable) is mapped to memory the same by all of them (so they have to be all x86 32-bit, or all x64, for example).
In the DLL that implements the the COM interface via a wrapper class of some sort, you only need a factory entry point. Something like an
extern "C" HRESULT MkIFooImplementation(void **ppv);
which returns an HRESULT (you'll need to learn about those too) and will also return a *pv in a location you provide for receiving the IFoo interface pointer. (I am skimming and there are more careful details that you'll need here. Don't trust my syntax) The actual function stereotype that you use for this is also declared in the *.h file.
The point is that the factory entry, which is always an undecorated extern "C" does all of the necessary wrapper class creation and then delivers an Ifoo interface pointer to the location that you specify. This means that all memory management for creation of the class, and all memory management for finalizing it, etc., will happen in the DLL where you build the wrapper. This is the only place where you have to deal with those details.
When you get an OK result from the factory function, you have been issued an interface pointer and it has already been reserved for you (there is an implicit IFoo:Addref operation already performed on behalf of the interface pointer you were delivered).
When you are done with the interface, you release it with a call on the IFoo:Release method of the interface. It is the final release implementation (in case you made more AddRef'd copies) that will tear down the class and its interface support in the factory DLL. This is what gets you correct reliance on a consistent dynamic stoorage allocation and release behind the interface, whether or not the DLL containing the factory function uses the same libraries as the calling code.
You should probably implement IUnknown:QueryInterface (as method IFoo:QueryInterface) too, even if it always fails. If you want to be more sophisticated with using the COM binary interface model as you have more experience, you can learn to provide full QueryInterface implementations.
This is probably too much information, but I wanted to point out that a lot of the problems you are facing about heterogeneous implementations of DLLs are resolved in the definition of the COM binary interface and even if you don't need all of it, the fact that it provides worked solutions is valuable. In my experience, once you get the hang of this, you will never forget how powerful this can be in C++ and C++ interop situations.
I haven't sketched the resources you might need to consult for examples and what you have to learn in order to make *.h files and to actually implement factory-function wrappers of the libraries you want to share. If you want to dig deeper, holler.
There are other things you need to consider too, such as which run-times are being used by the various libraries. If no objects are being shared that's fine, but that seems quite unlikely at first glance.
Chris Becker's suggestions are pretty accurate - using an actual COM interface may help you get the binary compatibility you need. Your mileage may vary :)
not fun, man. you are in for a lot of frustration, you should probably give this:
Would the only solution be to write a
C-style DLL interface using handlers
to the objects or am I missing
something? In that case, I guess, I
would probably have less effort with
directly using the wrapped C-library.
a really close look. good luck.