I'm using the Boost.DLL library for my project in order to offload its functionality into a suite of pluggable modules. The core application provides headers defining various structs and classes that will be used both within the core application, and which will (in some cases) be passed to plugins when they need to perform various tasks.
One of the initial things I need to do is pass a class ServiceCatalog to each plugin's initialization method, to allow that plugin to register various capabilities with the core application. I've grouped the definitions I'd like to share with my plugins into a header called core.h, and I'm referencing this header both in more application and in my plugin projects.
ServiceCatalog has a number of larger functions that, to me, make sense to offload into a cpp file, rather than trying to bundle them all into the header file. I'd still like the plugin to know the "shape" of the class though, because, while I don't need it to instantiate the class, I need it to be able to call instance methods.
However because I'm referencing the header file in the plugin, but not the cpp file containing the implementation, I get "unresolved external symbol" errors when linking. This makes sense on a basic level of course (i.e. what would happen if the plugin tried to instantiate something it doesn't have access to the implementation for?) but there doesn't seem to be any way (that I can find) to declare that the class is, I guess, "extern" or something?
Do I literally need to use a base abstract class to achieve this? Such a solution feels a little convoluted. I guess I could replace my class with a struct containing some function pointers instead.
What approach do you recommend? In a nutshell, I want my core application to be able to pass instances of things to my plugins, where each thing has a number of related methods that can be called.
// core.h, "main.exe"
class ServiceCatalog
{
public:
void complex_method_with_external_implementation();
}
//plugin.cpp, "plugin.dll"
#include "core.h"
void register_plugin(ServiceCatalog* catalog)
{
catalog->complex_method_with_external_implementation();
}
If I'm understanding this right, the ServiceCatalog object is only created by the "main.exe" program, and not in any of the plugins. If that's the case, you can declare the functions that can be called by the plugins as virtual.
class ServiceCatalog
{
public:
virtual void complex_method_with_external_implementation();
}
Then all of the reference to them will be in the vtable, referenced by the constructor. The plugins will access the functions thru the vtable so they don't need any sort of export.
There are issues with versioning, as adding new 'exports' needs to be done in such a way not to mess up the existing layout of the vtable (you need to add them after all existing virtual functions, not insert them in between two existing virtual functions), and you can't remove any, either, without replacing them with something else.
Related
Pretty much like the title says, I want to split some parts of my Qt application into plugins, so I
can add new functionalities at runtime. Ideally, plugins would be compiled separately and put into a
dedicated path for plugins; when the application launches, installed extensions are automatically
loaded, or can be reloaded at the user request at any time.
I should mention that the objects I want to put into plugins are not QObjects, but if it can make
the solution simpler it's acceptable that they inherit from QObject.
How can I do that? I want the simplest solution that's portable and doesn't require anything else
than Qt (no external dependencies).
Although I answer my own question, I'm more than interested to hear others'!
For a start, you need to have a common interface among your plugins. Here's an example:
class MyPlugin
{
public:
virtual ~MyPlugin() {} // Needs to be virtual. Important!
// Put here your method(s)
virtual void frobnicate() = 0;
};
Do not name your interface like this, though. If your plugins represent video codecs, name it
"VideoCodec", for example. Some prefer to put an "I" before interfaces' name (e.g. IVideoCodec).
Also, some people would tell you to have public methods calling protected virtuals, but that's not
strictly necessary there.
Why an interface? That's because it's the only way the application can use plugins without knowing
the classes themselves beforehand. This means that because the application doesn't know the
classes, the plugin must allow creating the plugin component via a factory. In fact, the only
required function to declare is a factory function that creates a fresh instance of the "plugin".
This factory function could be declared as such:
extern "C" std::unique_ptr<MyPlugin> MyPlugin_new();
(You need extern "C", otherwise you'll get trouble with QLibrary because of C++ name mangling ―
see below)
The factory function need not be without parameters, but the parameters must make sense for all types
of plugins. This could be a hashtable or a file containing general configuration information, or
even better, an interface for a configuration object, for instance.
Now the loading part. The easiest way is to use a QDirIterator initialized to the plugin
directory, iterate through all files and try to load them. Something along the lines of...
void load_plugins_from_path(const QString &plugin_dir)
{
QDirIterator it(plugin_dir, QDir::Files, QDir::Readable);
while (it.hasNext()) {
try_load_plugin(it.next());
}
}
(it's written like it's a function, but it should be a method)
Do not try in any way to filter the files by extension or by using the QDir::Executable flag: this
will needlessly reduce the portability of the program―each OSes have their own file extensions, and QDir::Executable only work on unices (probably because there's no exec bit on Windows).
Here, the method load_plugins_from_path just loads plugins from one given path; the caller may
invoke that method over the elements of a list containing all the paths to search for plugins, for
example. try_load_plugin may be defined like this:
void try_load_plugin(const QString &filename)
{
QLibrary lib(filename);
auto factory = reinterpret_cast<decltype (MyPlugin_new) *>(lib.resolve("MyPlugin_new"));
if (factory) {
std::unique_ptr<MyPlugin> plugin(factory());
// Do something with "plugin", e.g. store in a std::vector
}
}
decltype is used on MyPlugin_new so we doesn't have to specify its type
(std::unique_ptr<MyPlugin> (*)()) and using it with auto will save you the trouble of changing
the code more than it needs to be, should you change the signature of MyPlugin_new.
This method just tries to load a file as a library (whether it's a valid library file or not!) and
attempts to resolve the required function, returning nullptr if either we're not dealing with a
valid library file or the requested symbol (our function) didn't exist. Note that because we do the
search directly in a dynamic library, we must know the exact name of the entity in that library.
Because C++ mangles names, and that mangling is dependent on the implementation, the only sensible
thing is to use extern "C" functions. Don't worry though: extern "C" will only prevent
overloading of that function, but otherwise all C++ can be used inside of that function. Also, even
though the factory function is not inside any namespace, it won't collide with other factory
functions in other libraries, because we use explicit linking; that way, we can have
MyPlugin_new from plugin A and MyPlugin_new from plugin B, and they will live at separate
addresses.
Finally, if your set of plugins is too diverse to be expressed by one interface, one solution is to
simply define (possibly) multiple factories inside of your plugins, each returning a pointer to a
different kind of interface.
Qt already has a class called QPluginLoader that does what you're trying to achieve.
I want to make a class-library using pimpl idiom, so that I can hide my implementation details for the user of the library.
Is it possible, to make a class, where some methods are public and callable from users perspective, while having methods that are only callable from the internals.
Right now I only see a solution with the friend keyword and declaring the internal methods private.
For example:
MyPartiallyVisibleClass: Class containing a mixture of methods accessible to the user, and methods only accessible to the internals of library.
InternalClass: Class internally in the library. The user will never know this excist.
// MyPartiallyVisibleClass.h: Will be included by the user.
class MyPartiallyVisibleClass
{
private:
class Impl; // Forward declare the implementation
Impl* pimpl;
InternalMethod(); // Can only be called from within the library-internals.
public:
UserMethod(); // Will be visible and callable from users perspective.
}
// MyPartiallyVisibleClass.cpp
class MyPartiallyVisibleClass::Impl
{
private:
InternalMethod();
public:
UserMethod();
friend class InternalClass;
}
// Internal class that will not be included into users application.
class InternalClass
{
public:
InternalMethod()
{
MyPartiallyVisibleClass pvc;
pvc.InternalMethod();
}
}
Is there a better way of doing this?
There are pros and cons.
By a source stand point, if you distribute only headers and binaries, everything in in cpp files will not be seen by the source user.
So, also MyPartiallyVisibleClass::Impl::Usermethod is not visible, but being public, callable everywhere in the cpp file it is declared.
Zero way, one way or two way friendship between external and internal class can be required if you don't want to repeat external methods internally. It can seem an encapsulation break, but it is not, since the "capsule" here, is the external class. Creating complex hierarchy of intenal privacy (public external, private external, public internal private internal public even more internal ... etc.) can become clueless if everything goes under your same responsibility. Unless the internal part is so large to be assigned to different developers, so that another level of interface & implementation is needed.
By a binary user standpoint, however, every function that is not inlined exist, and -having external linkage- its name is available into the library, and hence it is "callable" by writing another header that makes it publicly available to other sources. I will just be an undocumented feature.
The concept of public/private etc. is for code safety (avoid function you don't want to promise to maintain with always the same "contract" to be available, thus making external code more stable, eliminating unwanted dependency) not for "security" (avoid who wants to call to find a way to call).
There is then also another drawback: templates cannot be hidden into sources, since they have to be expanded into the users's source code space (not the developer binary space). And the growing of generic programming, functional static polimorphism etc. makes the pimpl idiom less and less attractive.
There are today many programs made by a single cpp file that instantiate a single "manager object" whose entire functionality reside as is made of header only libraries. Believing or not, this way to program makes code even more portable between compilers, since it doesn't have to exist in a different binary form for every possible client compiler. And does not necessarily make build time longer: precompiled header can be generated once for code that don't change too often. Who cares is the user ca see the code? If he wants to use it and be supported is not his interest to change it improperly. If he wants to hack or stole, he will find anyway another way to do it.
I have a game that consists of few modules.
One of them is database module.
I want to make it something like that:
Database{
public:
save(&Object); //all my classes in the all modules inherit from Object
load(&Object);
};
What would be the best way to make that module independent from other modules (other modules will store data in Database using save and load functions)?
I consider few solutions:
All objects have something like serialize() method that is inherited from Object class (analogy to Java). Database use that method to get the string and save it. Obvious disadvantages are: all objects have to implement new method and it won't be optimum to save strings (not knowing about the classes' structure).
Make 'manifests' for all the classes (in e.g. text file that will be send to Database). That manifests will describe what the structure of class is (e.g. one string, two double, one rare use int). Disadvantage is flexibility - changing the classes in other modules will have affect on manifests.
All classes has own save and load methods and Database use them. I don't want it, because all classes would have to know about database type and save and load should be in Database class, not distributed in the whole code (it's a main point to make such a module).
Database knows about all other modules (and will know how to save all objects). Bad thing here is a lot of dependencies. Changes in any of modules will affect the Database.
Which way will be good? Or maybe there's a better option?
One solution I've come across is to have all Object subclasses implement a virtual void serialize(ISerializer& serializer) method.
ISerializer would have pure virtual methods like void onInt(int value), void onString(const char* string) etc to be called by the Object subclass inside its serialize()-method. Your Database module could implement ISerializer in two separate classes, DatabaseReader and DatabaseWriter. Later on you could add ObjectInspectionFileDumper, OnScreenObjectStateDebugger or NetworkWriter that also implement ISerializer, but in other modules. Each object only needs to implement the serialize()-method once to gain all those possibilities for extension.
Pros:
Reading and writing is pretty much guaranteed to match up as long as you don't read data for an old version of an object without some kind of versioning scheme on top.
This is an orthogonal design, where the number of Object types and Serializer types can grow independently of each other.
Cons:
Mainly some virtual function overhead, if that is an issue for your project. This isn't something you will typically be doing much during regular gameplay though.
Later, you might come across things you want to call Objects which you don't want to serialize, then it could make sense to separate that out into an ISerializable interface class, only containing the pure virtual serialize()-method. To accommodate serializers where it matters (like debug serializers), you might want to change to void onInt(const char* name, int value) etc instead.
HTH
Run into a bit of an issue, and I'm looking for the best solution concept/theory.
I have a system that needs to use objects. Each object that the system uses has a known interface, likely implemented as an abstract class. The interfaces are known at build time, and will not change. The exact implementation to be used will vary and I have no idea ahead of time what module will be providing it. The only guarantee is that they will provide the interface. The class name and module (DLL) come from a config file or may be changed programmatically.
Now, I have all that set up at the moment using a relatively simple system, set up something like so (rewritten pseudo-code, just to show the basics):
struct ClassID
{
Module * module;
int number;
};
class Module
{
HMODULE module;
function<void * (int)> * createfunc;
static Module * Load(String filename);
IObject * CreateClass(int number)
{
return createfunc(number);
}
};
class ModuleManager
{
bool LoadModule(String filename);
IObject * CreateClass(String classname)
{
ClassID class = AvailableClasses.find(classname);
return class.module->CreateObject(class.number);
}
vector<Module*> LoadedModules;
map<String, ClassID> AvailableClasses;
};
Modules have a few exported functions to give the number of classes they provide and the names/IDs of those, which are then stored. All classes derive from IObject, which has a virtual destructor, stores the source module and has some methods to get the class' ID, what interface it implements and such.
The only issue with this is each module has to be manually loaded somewhere (listed in the config file, at the moment). I would like to avoid doing this explicitly (outside of the ModuleManager, inside that I'm not really concerned as to how it's implemented).
I would like to have a similar system without having to handle loading the modules, just create an object and (once it's all set up) it magically appears.
I believe this is similar to what COM is intended to do, in some ways. I looked into the COM system briefly, but it appears to be overkill beyond belief. I only need the classes known within my system and don't need all the other features it handles, just implementations of interfaces coming from somewhere.
My other idea is to use the registry and keep a key with all the known/registered classes and their source modules and numbers, so I can just look them up and it will appear that Manager::CreateClass finds and makes the object magically. This seems like a viable solution, but I'm not sure if it's optimal or if I'm reinventing something.
So, after all that, my question is: How to handle this? Is there an existing technology, if not, how best to set it up myself? Are there any gotchas that I should be looking out for?
COM very likely is what you want. It is very broad but you don't need to use all the functionality. For example, you don't need to require participants to register GUIDs, you can define your own mechanism for creating instances of interfaces. There are a number of templates and other mechanisms to make it easy to create COM interfaces. What's more, since it is a standard, it is easy to document the requirements.
One very important thing to bear in mind is that importing/exporting C++ objects requires all participants to be using the same compiler. If you think that ever could be a problem to you then you should use COM. If you are happy to accept that restriction then you can carry on as you are.
I don't know if any technology exists to do this.
I do know that I worked with a system very similar to this. We used XML files to describe the various classes that different modules made available. Our equivalent of ModuleManager would parse the xml files to determine what to create for the user at run time based on the class name they provided and the configuration of the system. (Requesting an object that implemented interface 'I' could give back any of objects 'A', 'B' or 'C' depending on how the system was configured.)
The big gotcha we found was that the system was very brittle and at times hard to debug/understand. Just reading through the code, it was often near impossible to see what concrete class was being instantiated. We also found that maintaining the XML created more bugs and overhead than expected.
If I was to do this again, I would keep the design pattern of exposing classes from DLL's through interfaces, but I would not try to build a central registry of classes, nor would I derive everything from a base class such as IObject.
I would instead make each module responsible for exposing its own factory functions(s) to instantiate objects.
I have a code base where many of the classes I implement derive from classes that are provided by other divisions of my company. Working with these other devisions often have the working relationship as though they are third party middle ware vendors.
I'm trying to write test code without modifying these base classes. However, there are issues with creating meaningful test
objects due to the lack of interfaces:
//ACommonClass.h
#include "globalthermonuclearwar.h" //which contains deep #include dependencies...
#include "tictactoe.h" //...and need to exist at compile time to get into test...
class Something //which may or may not inherit from another class similar to this...
{
public:
virtual void fxn1(void); //which often calls into many other classes, similar to this
//...
int data1; //will be the only thing I can test against, but is often meaningless without fxn1 implemented
//...
};
I'd normally extract an interface and work from there, but as these are "Third Party", I can't commit these changes.
Currently, I've created a separate file that holds fake implementations for functions that are defined in the third-party supplied base class headers on a need to know basis, as has been described in the book "Working with Legacy Code".
My plan was to continue to use these definitions and provide alternative test implementations for each third party class that I needed:
//SomethingRequiredImplementations.cpp
#include "ACommonClass.h"
void CGlobalThermoNuclearWar::Simulate(void) {}; // fake this and all other required functions...
// fake implementations for otherwise undefined functions in globalthermonuclearwar.h's #include files...
void Something::fxn1(void) { data1 = blah(); } //test specific functionality.
But before I start doing that I was wondering if any one has tried providing actual objects on a code base similar to mine, which would allow creating new test specific classes to use in place of actual third-party classes.
Note all code bases in question are written in C++.
Mock objects are suitable for this kind of task. They allow you to simulate the existence of other components without needing them to be present. You simply define the expected input and output in your tests.
Google have a good mocking framework for C++.
I'm running into a very similar problem at the moment. I don't want to add a bunch of interfaces that are only there for the purpose of testing, so I can't use any of the existing mock object libraries. To get around this I do the same thing, creating a different file with fake implementations, and having my tests link the fake behaviour, and production code links the real behaviour.
What I wish I could do at this point, is take the internals of another mock framework, and use it inside my fake objects. It would look a little something like this:
Production.h
class ConcreteProductionClass { // regular everyday class
protected:
ConcreteProductionClass(); // I've found the 0 arg constructor useful
public:
void regularFunction(); // regular function that I want to mock
}
Mock.h
class MockProductionClass
: public ConcreteProductionClass
, public ClassThatLetsMeSetExpectations
{
friend class ConcreteProductionClass;
MockTypes membersNeededToSetExpectations;
public:
MockClass() : ConcreteProductionClass() {}
}
ConcreteProductionClass::regularFunction() {
membersNeededToSetExpectations.PassOrFailTheTest();
}
ProductionCode.cpp
void doSomething(ConcreteProductionClass c) {
c.regularFunction();
}
Test.cpp
TEST(myTest) {
MockProductionClass m;
m.SetExpectationsAndReturnValues();
doSomething(m);
ASSERT(m.verify());
}
The most painful part of all this is that the other mock frameworks are so close to this, but don't do it exactly, and the macros are so convoluted that it's not trivial to adapt them. I've begun looking into this on my spare time, but it's not moving along very quickly. Even if I got my method working the way I want, and had the expectation setting code in place, this method still has a couple drawbacks, one of them being that your build commands can get to be kind of long if you have to link against a lot of .o files rather than one .a, but that's manageable. It's also impossible to fall through to the default implementation, since we're not linking it. Anyway, I know this doesn't answer the question, or really even tell you anything you don't already know, but it shows how close the C++ community is to being able to mock classes that don't have a pure virtual interface.
You might want to consider mocking instead of faking as a potential solution. In some cases you may need to write wrapper classes that are mockable if the original classes aren't. I've done this with framework classes in C#/.Net, but not C++ so YMMV.
If I have a class that I need under test that derives from something I can't (or don't want to) run under test I'll:
Make a new logic-only class.
Move the code-i-wanna-test to the logic class.
Use an interface to talk back to the real class to interact with the base class and/or things I can't or won't put in the logic.
Define a test class using that same interface. This test class could have nothing but noops or fancy code that simulates the real classes.
If I have a class that I just need to use in testing, but using the real class is a problem (dependencies or unwanted behaviors):
I'll define a new interface that looks like all of the public methods I need to call.
I'll create a mock version of the object that supports that interface for testing.
I'll create another class that is constructed with a "real" version of that class. It also supports that interface. All interface calls a forwarded to the real object methods.
I'll only do this for methods I actually call - not ALL the public methods. I'll add to these classes as I write more tests.
For example, I wrap MFC's GDI classes like this to test Windows GDI drawing code. Templates can make some of this easier - but we often end up not doing that for various technical reasons (stuff with Windows DLL class exporting...).
I'm sure all this is in Feather's Working with Legacy Code book - and what I'm describing has actual terms. Just don't make me pull the book off the shelf...
One thing you did not indicate in your question is the reason why your classes derive from base classes from the other division. Is the relationship really a IS-A relationshiop ?
Unless your classes needs to be used by a framework, you could consider favoring delegation over inheritance. Then you can use dependency injection to provide your class with a mock of their class in the unit tests.
Otherwise, an idea would be to write a script to extract and create the interface your need from the header they provide, and integrate this to the compilation process so your unit test can ve checked in.