I'm planning on doing a C++ plugin interface ala How to create some class from dll(constructor in dll)?(с++)
but concerns have been raised that if the interface is used to create DLLs via MinGW or Borland and the DLL loader is compiled with MSVC++, there might be problems. Since the only exported function is declared extern "C" I fail to see why it would not work ?
Ideas ?
If you want to be compatible across compilers (and Release / Debug) and use C++, you need a little more effort.
Basically - you are allowed to pass basic datatypes, and pointer to pure virtual classes. These classes must not contain any data member, their destructor must not be public and they should not have overloaded functions.
Memory must not be allocated in one dll and released in another. This means no exceptions and you need some kind of reference counting or returning mechanism.
All methods inside pure virtual class (aka "Interface") must be marked with a call convention (I'd prefer stdcall).
Dynamic casts are not possible as well, so you might need some functionality in all your interfaces to do the trick (like QueryInterface in COM).
This works because most compiler on win32 try to be COM compatible and solve the same problems in a COM compatible way. For getting the first interface, you need a plain C function that is exported from the dll.
If you just use C functions and C data types, everything will work as well. But then you are limited to C without classes & inheritance.
I hope that helps.
Name mangling is not a problem:
1st: if you use C functions with C data types, everything is defined, there's no name mangling (exception: in VS with STDCALL, you need to remap the name to the "normal" C name via Linker directive)
2nd: Methods inside classes are not exported and thus not mangled. You call methods via pointer to pure virtual classes (aka "Interfaces"). This uses an offset and no name. You still can't use the destructor, as the position of the destructor inside the vtbl is not fixed as far as I know.
If you pass structs to functions / methods, be sure to fix the alignment. It is not defined across different compilers.
An idea could be using standard extern C function to create a ClassFactory. Have some convenction about a fixed ( or more ) entry point the dll must expose to be a valid plugin.
Related
I've read several times that passing STL objects like vector and string outside of a DLL boundary is bad practice because different compiler versions can generate different code for STL objects. Therefore, you should design a C-style interface and not pass STL objects at all. However, there are still some things unclear to me:
1. What is the 'boundary' of a DLL?
Is it right to say, that the boundary is where code is beeing compiled on DLL side? What if I define a .h file inside a DLL (f.e. to write a factory class) and use that header file in a different project? Is that .h file inside or outside the boundary of the DLL and why?
2. What is contained in a DLL?
Let' say I have a class Foo:
class Foo
{
public:
__declspec(dllexport) void f1(); //instantiates v1 inside function
private:
unique_ptr<vector<int>> v1 = nullptr;
}
If I only mark the function f1() with __declspec(dllexport), only this function should be contained in the DLL. How does the code inside f1() know what v1 is if v1 isn't contained in the DLL?
3. Passing objects out of a DLL-boundary using unique_ptr
I'm using unique_ptr almost everytime in my project. From what I understand, returning a unique_ptr from a DLL would be bad practice because unique_ptr is an STL object. How can I instantiate an object inside the DLL and return a unique_ptr to it?
4. Why does defining interfaces or using PIMPL help to define an DLL interface?
I still have to convert my STL classes to C-style objects. And in the project using the DLL, I would have to somehow wrap the C-style objects inside STL classes again. I don't see any advantage of using interfaces or PIMPL in this case.
Also, if I define an interface (class with pure virtual functions), wouldn't this have the same effect as just declaring the functions in my class with __declspec(dllexport)?
class IFoo
{
public:
virtual ~IFoo() = 0 {};
virtual void f1() = 0;
}
class Foo : public IFoo
{
public:
void f1();
//__declspec(dllexport) void f1(); //why use an interface if I can just declare the functions like this?
}
How is the DLL-STL problematic solved in modern C++ 11/14 libraries? Are there any modern open-source libraries that I can have a look at?
Unfortunately STL types aren't consistent across compilers. Even different versions of Visual Studio have differences.
The boundary is where the code is compiled. If you have an implementation in a header file in your library, then the compiler used to compile the EXE will compile the code. This is potentially very bad because what the code in the EXE thinks is the data is different to what the code in the DLL thinks is the data. (You need to look out for differences like this especially if you have #ifs in a struct definition and you need to be explicit about packing).
The only way to be sure is to define all your own types (being careful of packing) and not use STL. This is what DLL libraries usually do.
Interfaces can enable the user to dynamically link to the library. Using __declspec(dllexport) requires a static linking; that is the EXE has to link to the .lib generated when you compiled the DLL to be able to access all the functions. This means amongst other things you can't update the DLL without the EXE having to be recompiled (probably - you can get away with this in some circumstances, but it's not a good idea).
By dynamically linking you can update the DLL or add functionality to the DLL without relinking the EXE as long as you don't change your interfaces. The EXE might call LoadLibrary() on the DLL and GetProcAddress() to access one function that returns an interface. Everything else including data types passed as parameters are interfaces (i.e. contain only pure virtual functions) or simple structs. This is how the basic level of COM works.
To answer question 2, when you declare something as __declspec(dllexport) you are stating that this is part of the interface to the DLL - something that is accessible to the component that loads the DLL. Anything declared without __declspec(dllexport) should be present within the DLL but will not be available to be called/used by an external component.
I'm looking to implement a custom implementation of COM in C++ on a UNIX type platform to allow me to dynamically load and link object oriented code. I'm thinking this would be based on a similar set of functionality that POSIX provides to load and call dll's ie dlopen, dlsym and dlclose.
I understand that the general idea of COM is that you link to a few functions ie QueryInterface, AddRef and Release in a common dll (Kernel32.dll) which then allows you to access interfaces which are just a table of function pointers encapsulated with a pointer to the object for which the function pointers should be called with. These functions are exposed through IUnknown which you must inherit off of.
So how does this all work? Is there a better way to dynamically link and load to object oriented code? How does inheritance from a dll work - does every call to the base class have to be to an exposed member function i.e private/protected/public is simply ignored?
I'm quite well versed in C++ and template meta-programming and already have a fully reflective C++ system i.e member properties, member functions and global/static functions that uses boost.
A couple of things to keep in mind:
The power of COM comes largely from the IDL and the midl compiler. It allows a verry succint definition of the objects and interfaces with all the C/C++ boilerplate generated for you.
COM registration. On Windows the class IDs (CLSID) are recorded in the registry where they are associated with the executable. You must provide similar functionality in the UNIX environment.
The whole IUnknown implementation is fairly trivial, except for QueryInterface which works when implemented in C (i.e. no RTTI).
A whole another aspect of COM is IDispatch - i.e. late bound method invocation and discovery (read only reflection).
Have a look at XPCOM as it is a multi-platform COM like environment. This is really one of those things you are better off leveraging other technologies. It can suck up a lot of the time better spent elsewhere.
I'm looking to implement a custom implementation of COM in C++ on a UNIX type platform to allow me to dynamically load and link object oriented code. I'm thinking this would be based on a similar set of functionality that POSIX provides to load and call dll's ie dlopen, dlsym and dlclose.
At its simplest level, COM is implemented with interfaces. In c++, if you are comfortable with the idea of pure virtual, or abstract base classes, then you already know how to define an interface in c++
struct IMyInterface {
void Method1() =0;
void Method2() =0;
};
The COM runtime provides a lot of extra services that apply to the windows environment but arn't really needed when implementing "mini" COM in a single application as a means to dynamically link to a more OO interface than traditionally allowed by dlopen, dlsym, etc.
COM objects are implemented in .dll, .so or .dylib files depending on your platform. These files need to export at least one function that is standardized: DllGetClassObject
In your own environment you can prototype it however you want but to interop with the COM runtime on windows obviously the name and parameters need to conform to the com standard.
The basic idea is, this is passed a pointer to a GUID - 16 bytes that uniquely are assigned to a particular object, and it creates (based on the GUID) and returns the IClassFactory* of a factory object.
The factory object is then used, by the COM runtime, to create instances of the object when the IClassFactory::CreateInstance method is called.
So, so far you have
a dynamic library exporting at least one symbol, named "DllGetClassObject" (or some variant thereof)
A DllGetClassObject method that checks the passed in GUID to see if and which object is being requested, and then performs a "new CSomeObjectClassFactory"
A CSomeObjectClassFactory implementation that implements (derives from) IClassFactory, and implements the CreateInstance method to "new" instances of CSupportedObject.
CSomeSupportedObject that implements a custom, or COM defined interface that derives from IUnknown. This is important because IClassFactory::CreateInstance is passed an IID (again, a 16byte unique id defining an interface this time) that it will need to QueryInterface on the object for.
I understand that the general idea of COM is that you link to a few functions ie QueryInterface, AddRef and Release in a common dll (Kernel32.dll) which then allows you to access interfaces which are just a table of function pointers encapsulated with a pointer to the object for which the function pointers should be called with. These functions are exposed through IUnknown which you must inherit off of.
Actually, COM is implemented by OLE32.dll which exposes a "c" api called CoCreateInstance. The app passed CoCreateInstance a GUID, which it looks up in the windows registry - which has a DB of GUID -> "path to dll" mappings. OLE/COM then loads (dlopen) the dll, calls its DllGetClassObject (dlsym) method, passing in the GUID again, presuming that succeeds, OLE/COM then calls the CreateInstance and returns the resulting interface to app.
So how does this all work? Is there a better way to dynamically link and load to object oriented code? How does inheritance from a dll work - does every call to the base class have to be to an exposed member function i.e private/protected/public is simply ignored?
implicit inheritance of c++ code from a dll/so/dylib works by exporting every method in the class as a "decorated" symbol. The method name is decorated with the class, and type of every parameter. This is the same way the symbols are exported from static libraries (.a or .lib files iirc). Static or dynamic libraries, "private, protected etc." are always enforced by the compiler, parsing the header files, never the linker.
I'm quite well versed in C++ and template meta-programming and already have a fully reflective C++ system i.e member properties, member functions and global/static functions that uses boost.
c++ classes can typically only be exported from dlls with static linkage - dlls that are loaded at load, not via dlopen at runtime. COM allows c++ interfaces to be dynamically loaded by ensuring that all datatypes used in COM are either pod types, or are pure virtual interfaces. If you break this rule, by defining an interface that tries to pass a boost or any other type of object you will quickly get into a situation where the compiler/linker will need more than just the header file to figure out whats going on and your carefully prepared "com" dll will have to be statically or implicitly linked in order to function.
The other rule of COM is, never pass ownership of an object accross a dynamic library boundary. i.e. never return an interface or data from a dll, and require the app to delete it. Interfaces all need to implement IUnknown, or at least a Release() method, that allows the object to perform a delete this. Any returned data types likewise must have a well known de-allocator - if you have an interface with a method called "CreateBlob", there should probably be a buddy method called "DeleteBlob".
To really understand how COM works, I suggest reading "Essential COM" by Don Box.
Look at the CORBA documentation, at System.ComponentModel in the sscli, the XPCOM parts of the Mozilla codebase. Miguel de Icaza implemented something like OLE in GNOME called Bonobo which might be useful as well.
Depending on what you're doing with C++ though, you might want to look at plugin frameworks for C++ like Yehia. I believe Boost also has something similar.
Edit: pugg seems better maintained than Yehia at the moment. I have not tried it though.
The basic design of COM is pretty simple.
All COM objects expose their functionality through one or more interfaces
All interfaces are derived from the IUnknown interface, thus all interfaces have
QueryInterface, AddRef & Release methods as the first 3 methods of their virtual
function table in a known order
All objects implement IUnknown
Any interface that an object supports can be queried from any other interface.
Interfaces are identified by Globally Unique Identifiers, these are IIDs GUIDs or CLSIDs, but they are all really the same thing. http://en.wikipedia.org/wiki/Globally_Unique_Identifier
Where COM gets complex is in how it deals with allowing interfaces to be called from outside the process where the object resides. COM marshalling is a nasty, hairy, beast. Made even more so by the fact that COM supports both single threaded and multi-threaded programming models.
The Windows implementaion of COM allows objects to be registered (the original use of the Windows registry was for COM). At a minimum the COM registry contains the mapping between the unique GUID for a COM object, and the library (dll) that contains it's code.
For this to work. DLLs that implement COM objects must have a ClassFactory - an entry point in the DLL with a standard name that can be called to create one of the COM objects the DLL implements. (In practice, Windows COM gets an IClassFactory object from this entry point, and uses that to create other COM objects).
so that's the 10 cent tour, but to really understand this, you need to read Essential COM by Don Box.
You may be interested in the (not-yet)Boost.Extension library.
Application is written in delphi 2010 and the underlying dll is a C++ dll.
In ideal case, when your application is in C++; The dll makes a callback to an application when an event occurs. The callback is implemented through an interface. Application developers implements the abstract c++ class and pass the object to the dll. The dll will then make a callback to a member function of your implemented class. A classic callback pattern it is.
But how do I pass a delphi object to the dll for it to make a callback.
I wouldn't really call that ideal. It is selfish and short-sighted to make a DLL that requires its consumers to use the same compiler as the DLL used. (Class layout is implementation-defined, and since both modules need to have the same notion of what a class is, they need to use the same compiler.)
Now, that doesn't mean other consumers of the DLL can't fake it. It just won't be as easy for them as the DLL's designer intended.
When you say the callback is implemented through an interface, do you mean a COM-style interface, where the C++ class has nothing but pure virtual methods, including AddRef, Release, and QueryInterface, and they all use the stdcall calling convention? If that's the case, then you can simply write a Delphi class that implements the same interface. There are many examples of that in the Delphi source code and other literature.
If you mean you have a non-COM interface, where the C++ class has only pure virtual methods, but not the three COM functions, then you can write a Delphi class with the same layout. Duplicate the method order, and make sure all the methods are virtual. The Delphi VMT has the same layout as most C++ vtables on Windows implementations, at least as far as the function-pointer order is concerned. (The Delphi VMT has a lot of non-method data as well, but that doesn't interfere with the method addresses.) Just be sure you maintain clear ownership boundaries. The DLL must never attempt to destroy the object; it won't have a C++-callable destructor that the delete operator could invoke.
If you mean that you have an arbitrary C++ class that could include data members, constructors, or non-pure methods, then your task is considerably more difficult. Follow up if this is the case; otherwise, I'd rather not address it right now.
Overall, I'll echo Mason's advice that the DLL should use plain C-style callback functions. A good rule of thumb is that if you stick to techniques you see in the Windows API, you'll be OK. If you're not in control of how to interact with the DLL, then so be it. But if you can make the DLL's external interface more C-like, that would be best. And that doesn't mean you need to abandon the C++-style interface; you could provide two interfaces, where the C-style interface serves as a wrapper for your already-working C++style interface.
You can't pass a Delphi object to C++, at least not without a very good understanding of how the object model works at the binary level. If you need callbacks, do them using C types only and plain functions and procedures (no methods) and you should be fine.
This question is related to "How to make consistent dll binaries across VS versions ?"
We have applications and DLLs built
with VC6 and a new application built
with VC9. The VC9-app has to use
DLLs compiled with VC6, most of
which are written in C and one in
C++.
The C++ lib is problematic due to
name decoration/mangling issues.
Compiling everything with VC9 is
currently not an option as there
appear to be some side effects.
Resolving these would be quite time
consuming.
I can modify the C++ library, however it must be compiled with VC6.
The C++ lib is essentially an OO-wrapper for another C library. The VC9-app uses some static functions as well as some non-static.
While the static functions can be handled with something like
// Header file
class DLL_API Foo
{
int init();
}
extern "C"
{
int DLL_API Foo_init();
}
// Implementation file
int Foo_init()
{
return Foo::init();
}
it's not that easy with the non-static methods.
As I understand it, Chris Becke's suggestion of using a COM-like interface won't help me because the interface member names will still be decorated and thus inaccessible from a binary created with a different compiler. Am I right there?
Would the only solution be to write a C-style DLL interface using handlers to the objects or am I missing something?
In that case, I guess, I would probably have less effort with directly using the wrapped C-library.
The biggest problem to consider when using a DLL compiled with a different C++ compiler than the calling EXE is memory allocation and object lifetime.
I'm assuming that you can get past the name mangling (and calling convention), which isn't difficult if you use a compiler with compatible mangling (I think VC6 is broadly compatible with VS2008), or if you use extern "C".
Where you'll run into problems is when you allocate something using new (or malloc) from the DLL, and then you return this to the caller. The caller's delete (or free) will attempt to free the object from a different heap. This will go horribly wrong.
You can either do a COM-style IFoo::Release thing, or a MyDllFree() thing. Both of these, because they call back into the DLL, will use the correct implementation of delete (or free()), so they'll delete the correct object.
Or, you can make sure that you use LocalAlloc (for example), so that the EXE and the DLL are using the same heap.
Interface member names will not be decorated -- they're just offsets in a vtable. You can define an interface (using a C struct, rather than a COM "interface") in a header file, thusly:
struct IFoo {
int Init() = 0;
};
Then, you can export a function from the DLL, with no mangling:
class CFoo : public IFoo { /* ... */ };
extern "C" IFoo * __stdcall GetFoo() { return new CFoo(); }
This will work fine, provided that you're using a compiler that generates compatible vtables. Microsoft C++ has generated the same format vtable since (at least, I think) MSVC6.1 for DOS, where the vtable is a simple list of pointers to functions (with thunking in the multiple-inheritance case). GNU C++ (if I recall correctly) generates vtables with function pointers and relative offsets. These are not compatible with each other.
Well, I think Chris Becke's suggestion is just fine. I would not use Roger's first solution, which uses an interface in name only and, as he mentions, can run into problems of incompatible compiler-handling of abstract classes and virtual methods. Roger points to the attractive COM-consistent case in his follow-on.
The pain point: You need to learn to make COM interface requests and deal properly with IUnknown, relying on at least IUnknown:AddRef and IUnknown:Release. If the implementations of interfaces can support more than one interface or if methods can also return interfaces, you may also need to become comfortable with IUnknown:QueryInterface.
Here's the key idea. All of the programs that use the implementation of the interface (but don't implement it) use a common #include "*.h" file that defines the interface as a struct (C) or a C/C++ class (VC++) or struct (non VC++ but C++). The *.h file automatically adapts appropriately depending on whether you are compiling a C Language program or a C++ language program. You don't have to know about that part simply to use the *.h file. What the *.h file does is define the Interface struct or type, lets say, IFoo, with its virtual member functions (and only functions, no direct visibility to data members in this approach).
The header file is constructed to honor the COM binary standard in a way that works for C and that works for C++ regardless of the C++ compiler that is used. (The Java JNI folk figured this one out.) This means that it works between separately-compiled modules of any origin so long as a struct consisting entirely of function-entry pointers (a vtable) is mapped to memory the same by all of them (so they have to be all x86 32-bit, or all x64, for example).
In the DLL that implements the the COM interface via a wrapper class of some sort, you only need a factory entry point. Something like an
extern "C" HRESULT MkIFooImplementation(void **ppv);
which returns an HRESULT (you'll need to learn about those too) and will also return a *pv in a location you provide for receiving the IFoo interface pointer. (I am skimming and there are more careful details that you'll need here. Don't trust my syntax) The actual function stereotype that you use for this is also declared in the *.h file.
The point is that the factory entry, which is always an undecorated extern "C" does all of the necessary wrapper class creation and then delivers an Ifoo interface pointer to the location that you specify. This means that all memory management for creation of the class, and all memory management for finalizing it, etc., will happen in the DLL where you build the wrapper. This is the only place where you have to deal with those details.
When you get an OK result from the factory function, you have been issued an interface pointer and it has already been reserved for you (there is an implicit IFoo:Addref operation already performed on behalf of the interface pointer you were delivered).
When you are done with the interface, you release it with a call on the IFoo:Release method of the interface. It is the final release implementation (in case you made more AddRef'd copies) that will tear down the class and its interface support in the factory DLL. This is what gets you correct reliance on a consistent dynamic stoorage allocation and release behind the interface, whether or not the DLL containing the factory function uses the same libraries as the calling code.
You should probably implement IUnknown:QueryInterface (as method IFoo:QueryInterface) too, even if it always fails. If you want to be more sophisticated with using the COM binary interface model as you have more experience, you can learn to provide full QueryInterface implementations.
This is probably too much information, but I wanted to point out that a lot of the problems you are facing about heterogeneous implementations of DLLs are resolved in the definition of the COM binary interface and even if you don't need all of it, the fact that it provides worked solutions is valuable. In my experience, once you get the hang of this, you will never forget how powerful this can be in C++ and C++ interop situations.
I haven't sketched the resources you might need to consult for examples and what you have to learn in order to make *.h files and to actually implement factory-function wrappers of the libraries you want to share. If you want to dig deeper, holler.
There are other things you need to consider too, such as which run-times are being used by the various libraries. If no objects are being shared that's fine, but that seems quite unlikely at first glance.
Chris Becker's suggestions are pretty accurate - using an actual COM interface may help you get the binary compatibility you need. Your mileage may vary :)
not fun, man. you are in for a lot of frustration, you should probably give this:
Would the only solution be to write a
C-style DLL interface using handlers
to the objects or am I missing
something? In that case, I guess, I
would probably have less effort with
directly using the wrapped C-library.
a really close look. good luck.
I want to use implicit linking in my project , and nmake really wants a .def file . The problem is , that this is a class , and I don't know what to write in the exports section .
Could anyone point me in the right direction ?
The error message is the following :
NMAKE : U1073: don't know how to make 'DLLCLASS.def'
P.S: I'm trying to build using Windows CE Platform Builder .
If I recall correctly, you can use __declspec(dllexport) on the class, and VC++ will automatically create exports for all the symbols related to the class (constructors/destructor, methods, vtable, typeinfo, etc).
Microsoft has more information on this here.
You can always find the decorated name for the member function by using dumpbin /symbols myclass.obj
in my case
class A {
public:
A( int ){}
};
the dumpbin dump showed the symbol ??0A##QAE#H#Z (public: __thiscall A::A(int))
Putting this symbol in the .def file causes the linker to create the A::A(int) symbol in the export symbols.
BUT! as #paercebal states in his comment: the manual entry of decorated (mangled) names is a chore - error prone, and sadly enough, not guarenteed to be portable across compiler versions.
I've found the best route to be an abstract factory.
Start by defining a purely virtual base class. This is a class with no implementation, a purely virtual interface class.
You can export this virtual base "abstract interface" class, but there's no real reason to do so. When the caller uses it, they will be using it through a pointer (PImpl, or Pointer to Implementation) so all the caller knows about is a simple memory address. A Def file, while a teensy bit more work to keep up with, provides benefits beyond what __declspec(dllexport) can attain. What benefits, you ask? We'll get to that, you just wait.
Have your real class publicly inherit from the virtual base. Now create a factory method to construct your object and a "release"ish callable destructor to perform cleanup. Name these methods something like "ConstructMyClass" and "ReleaseMyClass". Please, please replace "MyClass" :-)
Those factory / release methods should take only POD types if they need any paramaters (plain-old-data: integer, char, etc.). The return type should be your virtual abstract interface base class -- or rather, a pointer to it.
IMyClass* CreateAnObjectOfTypeIMyClass();
Perhaps it's now obvious why we need the virtual base class? Since the virtual interface class has no implementation, it's essentially all POD types (sort-of) so the "datatype" of the class can be understood by most callers like Visual Basic, C or vastly different C++ compilers.
If you're sufficiently fancy, you can get around the need for a "manual release" method (sorry, had to do it). How? Manage your own resources in the class through smart-pointers and a pImpl type of architecture so when the object dies, it will clean up after itself. Doing so means your class is, in the immortal words of our saint and saviour Scott Meyers, "easy to use correctly and hard to use incorrectly" by letting the caller disregard the need to clean up. Let those among us who have never forgotten to call ".close" cast the first stone.
Maybe this architecture sounds familiar? It should, it's basically a micromachine version of COM. Well, sort-of, at least the concept for interface, factory construction and release.
In the end you've exported the interface for your class, made (and exported) Create and Destroy methods, and now callers can invoke your PleaseConstructMyClass factory function to have your DLL return a fully constructed, fully implemented and fully baked object under the guise of its interface. They can call down to all the public methods of your class (at least the ones in your abstract virtual interface) and do all the fun stuff.
When they are finished with the object which was returned by the factory function, then they can call a "ReleaseMyClass" function to ask your DLL to clean up the object's resources, or you can help them along by making your class clean up itself, making the "ReleaseMyClass" method redundant and useless.
If anyone's interested in the specific gains & tradeoffs for using the Def file and the interface (besides my blind say-so), please pipe up and we can dig deeper.
Don't you just love this stuff?
The solution is the following :
since a class is exported,you also need to add the exported methods in the .def file
I didn't find out how to export a constructor , so I went with using a factory method ( static ) , which will return new instances of an object
the other functions will be exported by adding normal exporting declaration in the .def file
Hope someone will benefit from this information .