I want to use the pimpl idiom to avoid having users of my library need our external dependencies (like boost, etc) however when my class is templated that seems to be impossible because the methods must be in the header. Is there something I can do instead?
If the class is templated, your users essentially need to compile it (and this is literally true in the most widely-used C++ implementations) and so they need your external dependencies.
The simplest solution is to put the bulk of your class's implementation in a non-template base class (or encapsulated member object of some class). Solve the module-hiding problem there.
And then write the template derived (or enclosing) class to add type safety to it.
For example, suppose you have a template that provides the amazing ability to allocate on first access (omitting the necessary copy constructor, assignment, destructor):
template <class T>
class MyContainer
{
T *instance_;
public:
MyContainer() : instance_(0) {}
T &access()
{
if (instance_ == 0)
instance_ = new T();
return *instance_;
}
};
If you wanted the "logic" to be separated into a non-template base class, you'd have to parameterise the behaviour in the non-template way, which is to say, use virtual functions:
class MyBase
{
void *instance_;
virtual void *allocate() = 0;
public:
MyBase() : instance_(0) {}
void *access()
{
if (instance_ == 0)
instance_ = allocate();
return instance_;
}
};
Then you can add the type-awareness in the outer layer:
template <class T>
class MyContainer : MyBase
{
virtual void *allocate()
{ return new T(); }
public:
T &access()
{ return *(reinterpret_cast<T *>(MyBase::access())); }
};
i.e. You use virtual functions to allow the template to "fill in" the type-dependent operations. Obviously this pattern would only really make sense if you have some business logic that is worth the effort of hiding.
You can explicitly instantiate templates in the source file, but that is possible only if you know what the template type is going to be. Otherwise, do not use pimpl idiom for templates.
Something like this :
header.hpp :
#ifndef HEADER_HPP
#define HEADER_HPP
template< typename T >
class A
{
// constructor+methods + pimpl
};
#endif
source.cpp :
#include "header.hpp"
// implementation
// explicitly instantiate for types that will be used
template class A< int >;
template class A< float >;
// etc...
There are two general solutions:
while the interface depends on some type T, it defers to a more weakly typed implementation (e.g. one using void* pointers directly or trough type erasure), or
you support only a specific and quite limited number of types.
The second solution is relevant for e.g. char/wchar_t-dependent stuff.
The first solution was quite common in the early days of C++ templates, because at that time compilers were not good at recognizing commonalities in the generated machine code, and would introduce so called “code bloat”. Today, much to the surprise of any novice who tries it out, a templated solution can often have smaller machine code footprint than a solution relying on runtime polymorphism. Of course, YMMV.
Cheers & hth.,
Related
I'm trying to build a wrapper around a managed class so I can call it from native code.
Here is the managed function :
void MyManagedFunction(MyStruct iStruct)
{
// Code according to what are the types of the 2 MyStruct members
}
struct MyStruct
{
public MyStruct(Object iValue1, Object iValue2) : this()
{
Value1 = iValue1; // Often contains DateTime::Now
Value2 = iValue2;
}
public Object Value1;
public Object Value2;
}
In my case, Value1 will almost always be System::DateTime::Now and the Value2 will almost always be a common data type (int, double, float, string, bool). I thought of making two templated function in the wrapper.
In the wrapper's .h I have the following :
#ifdef MYWRAPPER_EXPORTS
# define MYWRAPPER __declspec(dllexport)
#else
# define MYWRAPPER __declspec(dllimport)
#endif
class MYWRAPPER MyWrapper
{
public:
MyWrapper();
~MyWrapper();
template <class T> void MyNativeFunction(T iParam1)
{
MyStruct^ wStruct = gcnew MyStruct(System::DateTime::Now, iParam1);
//The class containing the managed function is a singleton
MyManagedClass::Instance->MyManagedFunction(wStruct);
}
template <class T, class W> void MyNativeFunction(T iParam1, W iParam2)
{
MyStruct^ wStruct = gcnew MyStruct(iParam1, iParam2);
//The class containing the managed function is a singleton
MyManagedClass::Instance->MyManagedFunction(wStruct);
}
};
This wrapper compiled without problem. The problem obviously occurred when I included the .h in the purely native code. Since I can't hide the content of the templated function, I have managed stuff visible on the native side which prevent the native code from compiling.
I was wondering if there was a workaround in order to achieve this. I don't mind if I'm limited into using only primitive types as parameters for the function. The best thing is if I was able to simply hide the content of the templated function in the native code so it only knows about the signature
Here's what I've tried/considered so far :
Converting the parameters to void* and call a function in which would call the managed one. By doing so, I can't cast the void* back to an object since I lose its type and using typeid to get the 'T' or 'W' type doesn't help since it vary from a compiler to another.
Overloading the function for every types I want to use. This is what I'll most likely use if I doesn't find a better solution. The problem is it means alot of overloading (especially for the 2 parameters function considering the number of combination)
If you know all the types your template will take, you can force instantiate it with those variables and thus put the code for the template functions in the source file instead of a header.
You can look at the example provided in Storing C++ template function definitions in a .CPP file
As he says you can do as below (copy-paste alert):
.h file
class foo
{
public:
template <typename T>
void do(const T& t);
};
.cpp file
template <typename T>
void foo::do(const T& t)
{
// Do something with t
}
template void foo::do<int>(const int&);
template void foo::do<std::string>(const std::string&);
After asking this question and reading up a lot on templates, I am wondering whether the following setup for a class template makes sense.
I have a class template called ResourceManager that will only be loading a few specific resources like ResourceManager<sf::Image>, ResourceManager<sf::Music>, etc. Obviously I define the class template in ResourceManager.h . However, since there are only a few explicit instantiations, would it be appropriate to do something like...
// ResourceManager.cpp
template class ResourceManager<sf::Image>;
template class ResourceManager<sf::Music>;
...
// Define methods in ResourceManager, including explicit specializations
In short, I'm trying to find the cleanest way to handle declaring and defining a template class and its methods, some of which may be explicit specializations. This is a special case, in which I know that there will only be a few explicit instantiations used.
Yes.
This is perfectly legittamate.
You may want to hide the fact that it is templatised behind a typedef (like std::basic_string does) then put a comment in the header not to use the template explicitly.
ResourceManager.h
template<typename T>
class ResourceManager
{
T& getType();
};
// Do not use ResourceManager<T> directly.
// Use one of the following types explicitly
typedef ResourceManager<sf::Image> ImageResourceManager;
typedef ResourceManager<sf::Music> MusicResourceManager;
ResourceManager.cpp
#include "ResourceManager.h"
// Code for resource Manager
template<typename T>
T& ResourceManager::getType()
{
T newValue;
return newValue;
}
// Make sure only explicit instanciations are valid.
template class ResourceManager<sf::Image>;
template class ResourceManager<sf::Music>;
If you require different implementations of the functions, depending on the type, I'd recommend using inheritance instead of templates.
class ResourceManager {
// Virtual and non-virtual functions.
}
class ImageManager : public ResourceManager {
// Implement virtual functions.
}
class MusicManager : public ResourceManager {
// Implement virtual functions.
}
Consider the following:
PImpl.hpp
class Impl;
class PImpl
{
Impl* pimpl;
PImpl() : pimpl(new Impl) { }
~PImpl() { delete pimpl; }
void DoSomething();
};
PImpl.cpp
#include "PImpl.hpp"
#include "Impl.hpp"
void PImpl::DoSomething() { pimpl->DoSomething(); }
Impl.hpp
class Impl
{
int data;
public:
void DoSomething() {}
}
client.cpp
#include "Pimpl.hpp"
int main()
{
PImpl unitUnderTest;
unitUnderTest.DoSomething();
}
The idea behind this pattern is that Impl's interface can change, yet clients do not have to be recompiled. Yet, I fail to see how this can truly be the case. Let's say I wanted to add a method to this class -- clients would still have to recompile.
Basically, the only kinds of changes like this that I can see ever needing to change the header file for a class for are things for which the interface of the class changes. And when that happens, pimpl or no pimpl, clients have to recompile.
What kinds of editing here give us benefits in terms of not recompiling client code?
The main advantage is that the clients of the interface aren't forced to include the headers for all your class's internal dependencies. So any changes to those headers don't cascade into a recompile of most of your project. Plus general idealism about implementation-hiding.
Also, you wouldn't necessarily put your impl class in its own header. Just make it a struct inside the single cpp and make your outer class reference its data members directly.
Edit: Example
SomeClass.h
struct SomeClassImpl;
class SomeClass {
SomeClassImpl * pImpl;
public:
SomeClass();
~SomeClass();
int DoSomething();
};
SomeClass.cpp
#include "SomeClass.h"
#include "OtherClass.h"
#include <vector>
struct SomeClassImpl {
int foo;
std::vector<OtherClass> otherClassVec; //users of SomeClass don't need to know anything about OtherClass, or include its header.
};
SomeClass::SomeClass() { pImpl = new SomeClassImpl; }
SomeClass::~SomeClass() { delete pImpl; }
int SomeClass::DoSomething() {
pImpl->otherClassVec.push_back(0);
return pImpl->otherClassVec.size();
}
There has been a number of answers... but no correct implementation so far. I am somewhat saddened that examples are incorrect since people are likely to use them...
The "Pimpl" idiom is short for "Pointer to Implementation" and is also referred to as "Compilation Firewall". And now, let's dive in.
1. When is an include necessary ?
When you use a class, you need its full definition only if:
you need its size (attribute of your class)
you need to access one of its method
If you only reference it or have a pointer to it, then since the size of a reference or pointer does not depend on the type referenced / pointed to you need only declare the identifier (forward declaration).
Example:
#include "a.h"
#include "b.h"
#include "c.h"
#include "d.h"
#include "e.h"
#include "f.h"
struct Foo
{
Foo();
A a;
B* b;
C& c;
static D d;
friend class E;
void bar(F f);
};
In the above example, which includes are "convenience" includes and could be removed without affecting the correctness ? Most surprisingly: all but "a.h".
2. Implementing Pimpl
Therefore, the idea of Pimpl is to use a pointer to the implementation class, so as not to need to include any header:
thus isolating the client from the dependencies
thus preventing compilation ripple effect
An additional benefit: the ABI of the library is preserved.
For ease of use, the Pimpl idiom can be used with a "smart pointer" management style:
// From Ben Voigt's remark
// information at:
// http://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Checked_delete
template<class T>
inline void checked_delete(T * x)
{
typedef char type_must_be_complete[ sizeof(T)? 1: -1 ];
(void) sizeof(type_must_be_complete);
delete x;
}
template <typename T>
class pimpl
{
public:
pimpl(): m(new T()) {}
pimpl(T* t): m(t) { assert(t && "Null Pointer Unauthorized"); }
pimpl(pimpl const& rhs): m(new T(*rhs.m)) {}
pimpl& operator=(pimpl const& rhs)
{
std::auto_ptr<T> tmp(new T(*rhs.m)); // copy may throw: Strong Guarantee
checked_delete(m);
m = tmp.release();
return *this;
}
~pimpl() { checked_delete(m); }
void swap(pimpl& rhs) { std::swap(m, rhs.m); }
T* operator->() { return m; }
T const* operator->() const { return m; }
T& operator*() { return *m; }
T const& operator*() const { return *m; }
T* get() { return m; }
T const* get() const { return m; }
private:
T* m;
};
template <typename T> class pimpl<T*> {};
template <typename T> class pimpl<T&> {};
template <typename T>
void swap(pimpl<T>& lhs, pimpl<T>& rhs) { lhs.swap(rhs); }
What does it have that the others didn't ?
It simply obeys the Rule of Three: defining the Copy Constructor, Copy Assignment Operator and Destructor.
It does so implementing the Strong Guarantee: if the copy throws during an assignment, then the object is left unchanged. Note that the destructor of T should not throw... but then, that is a very common requirement ;)
Building on this, we can now define Pimpl'ed classes somewhat easily:
class Foo
{
public:
private:
struct Impl;
pimpl<Impl> mImpl;
}; // class Foo
Note: the compiler cannot generate a correct constructor, copy assignment operator or destructor here, because doing so would require access to Impl definition. Therefore, despite the pimpl helper, you will need to define manually those 4. However, thanks to the pimpl helper the compilation will fail, instead of dragging you into the land of undefined behavior.
3. Going Further
It should be noted that the presence of virtual functions is often seen as an implementation detail, one of the advantages of Pimpl is that we have the correct framework in place to leverage the power of the Strategy Pattern.
Doing so requires that the "copy" of pimpl be changed:
// pimpl.h
template <typename T>
pimpl<T>::pimpl(pimpl<T> const& rhs): m(rhs.m->clone()) {}
template <typename T>
pimpl<T>& pimpl<T>::operator=(pimpl<T> const& rhs)
{
std::auto_ptr<T> tmp(rhs.m->clone()); // copy may throw: Strong Guarantee
checked_delete(m);
m = tmp.release();
return *this;
}
And then we can define our Foo like so
// foo.h
#include "pimpl.h"
namespace detail { class FooBase; }
class Foo
{
public:
enum Mode {
Easy,
Normal,
Hard,
God
};
Foo(Mode mode);
// Others
private:
pimpl<detail::FooBase> mImpl;
};
// Foo.cpp
#include "foo.h"
#include "detail/fooEasy.h"
#include "detail/fooNormal.h"
#include "detail/fooHard.h"
#include "detail/fooGod.h"
Foo::Foo(Mode m): mImpl(FooFactory::Get(m)) {}
Note that the ABI of Foo is completely unconcerned by the various changes that may occur:
there is no virtual method in Foo
the size of mImpl is that of a simple pointer, whatever what it points to
Therefore your client need not worry about a particular patch that would add either a method or an attribute and you need not worry about the memory layout etc... it just naturally works.
With the PIMPL idiom, if the internal implementation details of the IMPL class changes, the clients do not have to be rebuilt. Any change in the interface of the IMPL (and hence header file) class obviously would require the PIMPL class to change.
BTW,
In the code shown, there is a strong coupling between IMPL and PIMPL. So any change in class implementation of IMPL also would cause a need to rebuild.
Consider something more realistic and the benefits become more notable. Most of the time that I have used this for compiler firewalling and implementation hiding, I define the implementation class within the same compilation unit that visible class is in. In your example, I wouldn't have Impl.h or Impl.cpp and Pimpl.cpp would look something like:
#include <iostream>
#include <boost/thread.hpp>
class Impl {
public:
Impl(): data(0) {}
void setData(int d) {
boost::lock_guard l(lock);
data = d;
}
int getData() {
boost::lock_guard l(lock);
return data;
}
void doSomething() {
int d = getData();
std::cout << getData() << std::endl;
}
private:
int data;
boost::mutex lock;
};
Pimpl::Pimpl(): pimpl(new Impl) {
}
void Pimpl::doSomething() {
pimpl->doSomething();
}
Now no one needs to know about our dependency on boost. This gets more powerful when mixed together with policies. Details like threading policies (e.g., single vs multi) can be hidden by using variant implementations of Impl behind the scenes. Also notice that there are a number of additional methods available in Impl that aren't exposed. This also makes this technique good for layering your implementation.
In your example, you can change the implementation of data without having to recompile the clients. This would not be the case without the PImpl intermediary. Likewise, you could change the signature or name of Imlp::DoSomething (to a point), and the clients wouldn't have to know.
In general, anything that can be declared private (the default) or protected in Impl can be changed without recompiling the clients.
In non-Pimpl class headers the .hpp file defines the public and private components of your class all in one big bucket.
Privates are closely coupled to your implementation, so this means your .hpp file really can give away a lot about your internal implementation.
Consider something like the threading library you choose to use privately inside the class. Without using Pimpl, the threading classes and types might be encountered as private members or parameters on private methods. Ok, a thread library might be a bad example but you get the idea: The private parts of your class definition should be hidden away from those who include your header.
That's where Pimpl comes in. Since the public class header no longer defines the "private parts" but instead has a Pointer to Implementation, your private world remains hidden from logic which "#include"s your public class header.
When you change your private methods (the implementation), you are changing the stuff hidden beneath the Pimpl and therefore clients of your class don't need to recompile because from their perspective nothing has changed: They no longer see the private implementation members.
http://www.gotw.ca/gotw/028.htm
Not all classes benefit from p-impl. Your example has only primitive types in its internal state which explains why there's no obvious benefit.
If any of the members had complex types declared in another header, you can see that p-impl moves the inclusion of that header from your class's public header to the implementation file, since you form a raw pointer to an incomplete type (but not an embedded field nor a smart pointer). You could just use raw pointers to all your member variables individually, but using a single pointer to all the state makes memory management easier and improves data locality (well, there's not much locality if all those types use p-impl in turn).
This code:
template <typename T>
struct A
{
T t;
void DoSomething()
{
t.SomeFunction();
}
};
struct B
{
};
A<B> a;
is easily compiled without any complaints, as long as I never call a.DoSomething().
However, if I define DoSomething as a virtual function, I will get a compile error saying that B doesn't declare SomeFunction. I can somewhat see why it happens (DoSomething should now have an entry in the vtable), but I can't help feeling that it's not really obligated. Plus it sucks.
Is there any way to overcome this?
EDIT 2: Okay. I hope this time it makes sence:
Let's say I am doing intrusive ref count, so all entities must inherit from base class Object. How can I suuport primitive types too? I can define:
template <typename T>
class Primitive : public Object
{
T value;
public:
Primitive(const T &value=T());
operator T() const;
Primitive<T> &operator =(const T &value);
Primitive<T> &operator +=(const T &value);
Primitive<T> &operator %=(const T &value);
// And so on...
};
so I can use Primitive<int>, Primitive<char>...
But how about Primitive<float>? It seems like a problem, because floats don't have a %= operator. But actually, it isn't, since I'll never call operator %= on Primitive<float>.
That's one of the deliberate features of templates.
If, for some reason, I would define operator %= as virtual. Or, if i'll pre-export Primitive<float> from a dll to avoid link errors, the compiler will complain even if I never call operator %= on a Primitive<float>. If it would just have fill in a dummy value for operator %= in Primitive<float>'s vtable (that raises an exception?), everything would have been fine.
Put the virtuals into selectable base classes...
struct Jumper
{
virtual void Jump =0;
};
struct Crawler
{
virtual void Crawl() =0;
};
struct JumperCrawler:
public Jumper,
public Crawler
{
};
template<typename T, typename Methods>
class ICanBoostJumpingAndCrawling :
public Methods
{
T t;
};
Now you can use ICanBoostJumpingAndCrawling with Jumper,Crawler or JumperCrawler supplied as the Methods template parameter; realizing that you need to be derived from it so that you can implement Jumping and or Crawling in a subclass.
FYI, This makes the name "ICanBoostJumpingAndCrawling" completely misleading because it may or may not be able to do that; which means it should be renamed to something like "Booster".
That's not a bug, it's a feature -- seriously. At one time, most compilers would NOT compile the code, for exactly the reason you give. They've since been updated to compile it, partly because the standard requires it.
There was feature called "Concepts" in the C++ 0x standard for quite a while that would have allowed you to specify that T needed a member named 'SomeFunction', including its return type, argument type(s), etc.
Sadly, at the last meeting of the standard committee, they decided finishing up Concepts would delay the standard longer that most people wanted to wait, so they removed them.
Though it's not nearly as good, Boost does have a Concept Checking library that can do what you're asking about.
One way to overcome this is to specialize A for template argument B and not declare DoSomething()
template <>
struct A<struct B>
{
T t;
};
Ofcourse this means that you now have to implement the entire A struct from scratch.
So the compiler should be able to work out within the compilation unit just what is in use. As soon as you start to involve multiple compilation units it no longer has the limited scope and takes the necessary step to ensure that all of the class can compile.
For exporting from a library don't force the pre export, you can ignore the warnings about not exporting the templates as long as you compile all the code with the same compiler the template will be compiled the same in all locations, only compile what is necessary within each compilation unit.
To work around the problem with virtual, then the best you can do is to defer the issue to some other class - don't put virtual in the template.
Perhaps
addition of a 'traits' portion to your template would allow a cheap way out.
use multiple inheritance to define the composite template, like using shims
ie.
template <typename T>
class Additive
{
public:
Primitive<T> &operator =(const T &value);
Primitive<T> &operator +=(const T &value);
};
template <typename T>
class Multiplicative
{
public:
Primitive<T> &operator *=(const T &value);
Primitive<T> &operator /=(const T &value);
};
template <typename T>
class Integers : public Additive<T>, public Multiplicative<T>;
I'd really go back and ask if your abstracting the right level of information for making the template.
How do I inherit from a virtual template class, in this code:
// test.h
class Base {
public:
virtual std::string Foo() = 0;
virtual std::string Bar() = 0;
};
template <typename T>
class Derived : public Base {
public:
Derived(const T& data) : data_(data) { }
virtual std::string Foo();
virtual std::string Bar();
T data() {
return data_;
}
private:
T data_;
};
typedef Derived<std::string> DStr;
typedef Derived<int> DInt;
// test.cpp
template<typename T>
std::string Derived<T>::Foo() { ... }
template<typename T>
std::string Derived<T>::Bar() { ... }
When I try to use the DStr or DInt, the linker complain that there are unresolved externals, which are Derived<std::string>::Foo() and Derived<std::string>::Bar(), and the same for Derived<int>.
Did I miss something in my code?
EDIT:
Thanks all. It's pretty clear now.
You need to define template<typename T> std::string Derived<T>::Foo() { ... } and template<typename T>
std::string Derived<T>::Bar() { ... } in the header file. When the compiler is compiling test.cpp it doesn't know all the possible values of T that you might use in other parts of the program.
I think there are some compilers that have connections between the compiling and linking steps that notice references to missing template instantiations and go instantiate them from the .cpp file where they are declared. But I don't know which they are, and the functionality is exceedingly rare to find.
If you define them in the header file most compilers will emit them as a 'weak' symbol into every compilation unit in which they're referenced. And the linker will toss out all except for one definition of the weak symbol. This causes extra compilation time.
Alternately, there are syntaxes for explicitly instantiating templates and forcing the compiler to emit definitions right there. But that requires you to be aware of all the values T could possibly have and you will inevitably miss some.
You have to ensure that the member functions are instantiate for all the required types somewhere.
Usually this is accomplished by defining template functions inline in the header file where they are declared so that any use of the functions will cause them to be instantiated.
As an alternative, you can use explicit instantiations in the source file where you define them but this does require you to know in advance all the types that your template will ever be instantiated for.
This doesn't really have much to do with derivation. It's just a general rule with templates: with most compilers (anything but Comeau) you have to put the full implementation of a template where it's visible in every translation unit that will instantiate that template -- normally in the header.
Even with Comeau, you have to use the export keyword to make things work right. Since They're the only ones that implement export, chances are pretty good that you don't care much though.