Pimpl idiom without using dynamic memory allocation - c++

we want to use pimpl idiom for certain parts of our project. These parts of the project also happen to be parts where dynamic memory allocation is forbidden and this decision is not in our control.
So what i am asking is, is there a clean and nice way of implementing pimpl idiom without dynamic memory allocation?
Edit
Here are some other limitations: Embedded platform, Standard C++98, no external libraries, no templates.

Warning: the code here only showcases the storage aspect, it is a skeleton, no dynamic aspect (construction, copy, move, destruction) has been taken into account.
I would suggest an approach using the C++0x new class aligned_storage, which is precisely meant for having raw storage.
// header
class Foo
{
public:
private:
struct Impl;
Impl& impl() { return reinterpret_cast<Impl&>(_storage); }
Impl const& impl() const { return reinterpret_cast<Impl const&>(_storage); }
static const size_t StorageSize = XXX;
static const size_t StorageAlign = YYY;
std::aligned_storage<StorageSize, StorageAlign>::type _storage;
};
In the source, you then implement a check:
struct Foo::Impl { ... };
Foo::Foo()
{
// 10% tolerance margin
static_assert(sizeof(Impl) <= StorageSize && StorageSize <= sizeof(Impl) * 1.1,
"Foo::StorageSize need be changed");
static_assert(StorageAlign == alignof(Impl),
"Foo::StorageAlign need be changed");
/// anything
}
This way, while you'll have to change the alignment immediately (if necessary) the size will only change if the object changes too much.
And obviously, since the check is at compilation time, you just cannot miss it :)
If you do not have access to C++0x features, there are equivalents in the TR1 namespace for aligned_storage and alignof and there are macros implementations of static_assert.

pimpl bases on pointers and you can set them to any place where your objects are allocated. This can also be a static table of objects declared in the cpp file. The main point of pimpl is to keep the interfaces stable and hide the implementation (and its used types).

See The Fast Pimpl Idiom and The Joy of Pimpls about using a fixed allocator along with the pimpl idiom.

If you can use boost, consider boost::optional<>. This avoids the cost of dynamic allocation, but at the same time, your object will not be constructed until you deem necessary.

One way would be to have a char[] array in your class. Make it large enough for your Impl to fit, and in your constructor, instantiate your Impl in place in your array, with a placement new: new (&array[0]) Impl(...).
You should also ensure you don't have any alignment problems, probably by having your char[] array a member of an union. This:
union {
char array[xxx];
int i;
double d;
char *p;
};
for instance, will make sure the alignment of array[0] will be suitable for an int, double or a pointer.

The point of using pimpl is to hide the implementation of your object. This includes the size of the true implementation object. However this also makes it awkward to avoid dynamic allocation - in order to reserve sufficient stack space for the object, you need to know how big the object is.
The typical solution is indeed to use dynamic allocation, and pass the responsibility for allocating sufficient space to the (hidden) implementation. However, this isn't possible in your case, so we'll need another option.
One such option is using alloca(). This little-known function allocates memory on the stack; the memory will be automatically freed when the function exits its scope. This is not portable C++, however many C++ implementations support it (or a variation on this idea).
Note that you must allocate your pimpl'd objects using a macro; alloca() must be invoked to obtain the necessary memory directly from the owning function. Example:
// Foo.h
class Foo {
void *pImpl;
public:
void bar();
static const size_t implsz_;
Foo(void *);
~Foo();
};
#define DECLARE_FOO(name) \
Foo name(alloca(Foo::implsz_));
// Foo.cpp
class FooImpl {
void bar() {
std::cout << "Bar!\n";
}
};
Foo::Foo(void *pImpl) {
this->pImpl = pImpl;
new(this->pImpl) FooImpl;
}
Foo::~Foo() {
((FooImpl*)pImpl)->~FooImpl();
}
void Foo::Bar() {
((FooImpl*)pImpl)->Bar();
}
// Baz.cpp
void callFoo() {
DECLARE_FOO(x);
x.bar();
}
This, as you can see, makes the syntax rather awkward, but it does accomplish a pimpl analogue.
If you can hardcode the size of the object in the header, there's also the option of using a char array:
class Foo {
private:
enum { IMPL_SIZE = 123; };
union {
char implbuf[IMPL_SIZE];
double aligndummy; // make this the type with strictest alignment on your platform
} impl;
// ...
}
This is less pure than the above approach, as you must change the headers whenever the implementation size changes. However, it allows you to use normal syntax for initialization.
You could also implement a shadow stack - that is, a secondary stack separate from the normal C++ stack, specifically to hold pImpl'd objects. This requires very careful management, but, properly wrapped, it should work. This sort of is in the grey zone between dynamic and static allocation.
// One instance per thread; TLS is left as an exercise for the reader
class ShadowStack {
char stack[4096];
ssize_t ptr;
public:
ShadowStack() {
ptr = sizeof(stack);
}
~ShadowStack() {
assert(ptr == sizeof(stack));
}
void *alloc(size_t sz) {
if (sz % 8) // replace 8 with max alignment for your platform
sz += 8 - (sz % 8);
if (ptr < sz) return NULL;
ptr -= sz;
return &stack[ptr];
}
void free(void *p, size_t sz) {
assert(p == stack[ptr]);
ptr += sz;
assert(ptr < sizeof(stack));
}
};
ShadowStack theStack;
Foo::Foo(ShadowStack *ss = NULL) {
this->ss = ss;
if (ss)
pImpl = ss->alloc(sizeof(FooImpl));
else
pImpl = new FooImpl();
}
Foo::~Foo() {
if (ss)
ss->free(pImpl, sizeof(FooImpl));
else
delete ss;
}
void callFoo() {
Foo x(&theStack);
x.Foo();
}
With this approach it is critical to ensure that you do NOT use the shadow stack for objects where the wrapper object is on the heap; this would violate the assumption that objects are always destroyed in reverse order of creation.

One technique I've used is a non-owning pImpl wrapper. This is a very niche option and isn't as safe as traditional pimpl, but it can help if performance is a concern. It may require some re-architecture to more functional like apis.
You can create a non-owning pimpl class, as long as you can (somewhat) guarantee the stack pimpl object will outlive the wrapper.
For ex.
/* header */
struct MyClassPimpl;
struct MyClass {
MyClass(MyClassPimpl& stack_object); // Initialize wrapper with stack object.
private:
MyClassPimpl* mImpl; // You could use a ref too.
};
/* in your implementation code somewhere */
void func(const std::function<void()>& callback) {
MyClassPimpl p; // Initialize pimpl on stack.
MyClass obj(p); // Create wrapper.
callback(obj); // Call user code with MyClass obj.
}
The danger here, like most wrappers, is the user stores the wrapper in a scope that will outlive the stack allocation. Use at your own risk.

Related

Why destructors does not accept parameters in C++?

Recently, I came across an example that my destructor needs to take a parameter.
I am working on a C package that manages memory internally and keeps track of allocating and freeing with parameters of its own. I do not want to break that.
I wrote a C code, that initializes my own data structures and frees them at the end. When I decided to migrate to C++ I realized that allocating data structures and freeing them should be placed in constructor and destructors. So I changed those functions to constructor and destructor.
My problem now is that I need to use a parameter to pass to destructor to free allocated data; that parameter is not in my own code and is C and I do not want to mess with that.
My question is twofold:
Why C++ decided not to accept parameters in destructors in the first place?
and
What are my options?
(I can save a pointer to that parameter or somehow call that function in my destructor, but it seems not a good practice of C++ programming)
Update: adding some code
Here is my imaginary class:
class paru_symbolic/* paru_symbolic*/
{
public:
paru_symbolic ( cholmod_sparse *A, cholmod_common *cc ); // constructor
~paru_symbolic (cholmod_common *cc ); // destructor
// -------------------------------------------------------------------------
// row-form of the input matrix and its permutations
// -----------------------------------------------------------------------
//--- other stuff
...
};
Here is my current C constructor:
#include "Parallel_LU.hpp"
paru_symbolic *paru_sym_analyse
(
// inputs, not modified
cholmod_sparse *A,
// workspace and parameters
cholmod_common *cc ){
DEBUGLEVEL(0);
...
aParent = (Int*) paru_alloc (m+nf, sizeof(Int),cc);
...
}
and destructor:
void paru_freesym (paru_symbolic **LUsym_handle,
// workspace and parameters
cholmod_common *cc
){
DEBUGLEVEL (0);
if (LUsym_handle == NULL || *LUsym_handle == NULL){
// nothing to do; caller probably ran out of memory
return;
}
paru_symbolic *LUsym;
LUsym = *LUsym_handle;
Int m, n, anz, nf, rjsize;
...
cholmod_l_free (m+nf, sizeof (Int), LUsym->aParent, cc);
...
cholmod_l_free (1, sizeof (paru_symbolic), LUsym, cc);
*LUsym_handle = NULL;
}
The parameter cc is used inside the SuiteSparse package for tracking allocations and freeing data. It has been used all over in SuiteSparse package and is a useful tool for tracking memory.
Some people mentioned that who wants to pass a parameter to the destructor.That is a fair point, but we could have the same parameters that we have in the constructor as a default.
Parameters in destructors do not make sense as destructors are automatically invoked when an object goes out of scope. How could you pass a parameter to that?
{
Foo x;
} // `~Foo()` is automatically called here
What you probably want is to store the resource in your class. Here's an unrealistic example with a dynamically-allocated pointer:
struct RAIIPointer
{
Foo* ptr;
// Initialize the resource
RAIIPointer() : ptr{new Foo{/*...*/}}
{
}
RAIIPointer(const RAIIPointer&) = delete;
RAIIPointer& operator=(const RAIIPointer&) = delete;
RAIIPointer(RAIIPointer&&) { /* implement appropriately */ }
RAIIPointer& operator=(RAIIPointer&&) { /* implement appropriately */ }
// Release the resource
~RAIIPointer()
{
delete ptr;
}
};
Note that in a real-world scenario you would use std::unique_ptr, and if you were implementing your own RAII resource class you would need to implement proper copy/move operations.
#Vittorio's answer has the general approach. I'll just tailor it to your added sample code.
The way you presented the issue is as follows:
You use a utility type cholmod_common to do some useful work for your type.
The instance of that type can be passed to several paru_symbolic objects you create.
No single instance of paru_symbolic owns it uniquely, but they all depend on it at the start and end of their lifetime. It must exist for the same duration as all of those objects.
That screams std::shared_ptr to me.
class paru_symbolic
{
std::shared_ptr<cholmod_common> _cc;
public:
paru_symbolic ( cholmod_sparse *A, std::shared_ptr<cholmod_common> cc)
: _cc(cc) {
// constructor
}
~paru_symbolic () {
// use _cc , it's guaranteed to live until the closing brace at the least.
}
};
That's it. The ownership is now shared. The last object that depends on the instance of cholmod_common will be the one to clean it up with no extra work from you.
In C++, this sort of thing is done via an allocator, see also std::allocator. So, you could create your own allocator with a cholmod_common* data member.
Alternatively, you can use the deleter parameter for memory management via smart pointers. For example
class foo
{
struct bar_deleter {
cholmod_common *cc;
bar_deleter(cholmod_common *c) : cc(c) {}
void destroy(bar*) { cc->free(bar); }
};
std::unique_ptr<bar,bar_deleter> data;
public:
foo(some_type const&input, cholmod_common *cc)
: data(cc->create<bar>(input), bar_deleter(cc)) {}
};
Which, of course, also keeps a cholmod_common* with the data.
The name "cholmod_common" suggests that this is a common resource, i.e. there is only one cholmod_common object at any given time. Iff this is so, then you could use a stateless allocator, which keeps the common resource as a static data member (or obtains it via a singleton).

Bypassing constructor in C++

I am attempting to use C++ for AVR programming using gcc-avr. The main issue is that there is no libc++ available and the implementation does not define any of the new or delete operators. Similarly there is no header to include using placement new is not an option.
When attempting to allocate a new dynamic object I am tempted to just do this:
Class* p = reinterpret_cast<Class*>(malloc(sizeof(Class)));
p->Init();
where Init() manually initializes all internal variables. But is this safe or even possible?
I have read that object construction in C++ is somewhat complex but without new or delete how do I initialize a dynamically allocated object?
To expand on the above question.
Using standard g++ and placement new it is possible to subvert constructors in two ways, assuming that C++ uses the same straightforward ways of alligning memory as C (code example below).
Using placement new to initialize any allocated memory.
Initialize allocated memory directly using class methods.
Of course this only holds if the assumptions are true that:
Memory layout of an object is fixed at compile time.
Memory allocation is only concerned with class variables and observers normal C rules (allocated in order of declaration aligned to memory boundary).
If the above holds could I not just allocated memory using malloc and use a reinterpret_cast to convert to the correct class and initialize it manually? Of course this is both non-portable and hack-ish but the only other way I can see is to work around the problem and not use dynamically allocated memory at all.
Example:
Class A {
int i;
long l;
public:
A() : i(1), l(2) {}
int get_i() { return i; }
void set_i(int x) { i = x; }
long get_l() { return l; }
void set_l(long y) { l = y; }
};
Class B {
/* Identical to Class A, except constructor. */
public B() : i(3), l(4) {}
};
int main() {
A* a = (A*) ::operator new(sizeof(A));
B* b = (B*) ::operator new(sizeof(B));
/* Allocating A using B's constructor. */
a = (A*) new (a) B();
cout << a->get_i() << endl; // prints 3
cout << a->get_l() << endl; // prints 4
/* Setting b directly without constructing */
b->set_i(5);
b->set_l(6);
cout << b->get_i() << endl; // prints 5
cout << b->get_l() << endl; // prints 6
If your allegedly C++ compiler does not support operator new, you should be able to simply provide your own, either in the class or as a global definition. Here's a simple one from an article discussing operator new, slightly modified (and the same can be found in many other places, too):
void* operator new(size_t sz) {
void* mem = malloc(sz);
if (mem)
return mem;
else
throw std::bad_alloc();
}
void operator delete(void* ptr) {
free(ptr);
}
A longer discussion of operator new, in particular for class-specific definitions, can also be found here.
From the comments, it seems that given such a definition, your compiler then happily supports the standard object-on-heap creations like these:
auto a = std::make_shared<A>();
A *pa = new A{};
The problem with using Init methods as shown in the code snippet in your question is that it can be a pain to get that to work properly with inheritance, especially multiple or virtual inheritance, at least when something during object construction might throw. The C++ language has elaborate rules to make sure something useful and predictable happens in that situation with constructors; duplicating that with ordinary functions probably will get tricky rather fast.
Whether you can get away with your malloc()-reinterprete_cast<>-init() approach depends on whether you have virtual functions/inheritance. If there is nothing virtual in your class (it's a plain old datatype), your approach will work.
However, if there is anything virtual in it, your approach will fail miserably: In these cases, C++ adds a v-table to the data layout of your class which you cannot access directly without going very deep into undefined behavior. This v-table pointer is usually set when the constructor is run. Since you can't safely mimic the behavior of the constructor in this regard, you must actually call a constructor. That is, you must use placement-new at the very least.
Providing a classless operator new() as Christopher Creutzig suggests, is the easiest way to provide full C++ functionality. It is the function that is used internally by new expressions to provide the memory on which the constructors can be called to provide a fully initialized object.
One last point of assurance: as long as you do not use a variable length array at the end of a struct like this
typedef struct foo {
size_t arraySize;
int array[];
} foo;
the size of any class/struct is entirely a compile time constant.

preventing unaligned data on the heap

I am building a class hierarchy that uses SSE intrinsics functions and thus some of the members of the class need to be 16-byte aligned. For stack instances I can use __declspec(align(#)), like so:
typedef __declspec(align(16)) float Vector[4];
class MyClass{
...
private:
Vector v;
};
Now, since __declspec(align(#)) is a compilation directive, the following code may result in an unaligned instance of Vector on the heap:
MyClass *myclass = new MyClass;
This too, I know I can easily solve by overloading the new and delete operators to use _aligned_malloc and _aligned_free accordingly. Like so:
//inside MyClass:
public:
void* operator new (size_t size) throw (std::bad_alloc){
void * p = _aligned_malloc(size, 16);
if (p == 0) throw std::bad_alloc()
return p;
}
void operator delete (void *p){
MyClass* pc = static_cast<MyClass*>(p);
_aligned_free(p);
}
...
So far so good.. but here is my problem. Consider the following code:
class NotMyClass{ //Not my code, which I have little or no influence over
...
MyClass myclass;
...
};
int main(){
...
NotMyClass *nmc = new NotMyClass;
...
}
Since the myclass instance of MyClass is created statically on a dynamic instance of NotMyClass, myclass WILL be 16-byte aligned relatively to the beginning of nmc because of Vector's __declspec(align(16)) directive. But this is worthless, since nmc is dynamically allocated on the heap with NotMyClass's new operator, which doesn't nesessarily ensure (and definitely probably NOT) 16-byte alignment.
So far, I can only think of 2 approaches on how to deal with this problem:
Preventing MyClass users from being able to compile the following code:
MyClass myclass;
meaning, instances of MyClass can only be created dynamically, using the new operator, thus ensuring that all instances of MyClass are truly dynamically allocatted with MyClass's overloaded new. I have consulted on another thread on how to accomplish this and got a few great answers:
C++, preventing class instance from being created on the stack (during compiltaion)
Revert from having Vector members in my Class and only have pointers to Vector as members, which I will allocate and deallocate using _aligned_malloc and _aligned_free in the ctor and dtor respectively. This methos seems crude and prone to error, since I am not the only programmer writing these Classes (MyClass derives from a Base class and many of these classes use SSE).
However, since both solutions have been frowned upon in my team, I come to you for suggestions of a different solution.
If you're set against heap allocation, another idea is to over allocate on the stack and manually align (manual alignment is discussed in this SO post). The idea is to allocate byte data (unsigned char) with a size guaranteed to contain an aligned region of the necessary size (+15), then find the aligned position by rounding down from the most-shifted region (x+15 - (x+15) % 16, or x+15 & ~0x0F). I posted a working example of this approach with vector operations on codepad (for g++ -O2 -msse2). Here are the important bits:
class MyClass{
...
unsigned char dPtr[sizeof(float)*4+15]; //over-allocated data
float* vPtr; //float ptr to be aligned
public:
MyClass(void) :
vPtr( reinterpret_cast<float*>(
(reinterpret_cast<uintptr_t>(dPtr)+15) & ~ 0x0F
) )
{}
...
};
...
The constructor ensures that vPtr is aligned (note the order of members in the class declaration is important).
This approach works (heap/stack allocation of containing classes is irrelevant to alignment), is portabl-ish (I think most compilers provide a pointer sized uint uintptr_t), and will not leak memory. But its not particularly safe (being sure to keep the aligned pointer valid under copy, etc), wastes (nearly) as much memory as it uses, and some may find the reinterpret_casts distasteful.
The risks of aligned operation/unaligned data problems could be mostly eliminated by encapsulating this logic in a Vector object, thereby controlling access to the aligned pointer and ensuring that it gets aligned at construction and stays valid.
You can use "placement new."
void* operator new(size_t, void* p) { return p; }
int main() {
void* p = aligned_alloc(sizeof(NotMyClass));
NotMyClass* nmc = new (p) NotMyClass;
// ...
nmc->~NotMyClass();
aligned_free(p);
}
Of course you need to take care when destroying the object, by calling the destructor and then releasing the space. You can't just call delete. You could use shared_ptr<> with a different function to deal with that automatically; it depends if the overhead of dealing with a shared_ptr (or other wrapper of the pointer) is a problem to you.
The upcoming C++0x standard proposes facilities for dealing with raw memory. They are already incorporated in VC++2010 (within the tr1 namespace).
std::tr1::alignment_of // get the alignment
std::tr1::aligned_storage // get aligned storage of required dimension
Those are types, you can use them like so:
static const floatalign = std::tr1::alignment_of<float>::value; // demo only
typedef std::tr1::aligned_storage<sizeof(float)*4, 16>::type raw_vector;
// first parameter is size, second is desired alignment
Then you can declare your class:
class MyClass
{
public:
private:
raw_vector mVector; // alignment guaranteed
};
Finally, you need some cast to manipulate it (it's raw memory until now):
float* MyClass::AccessVector()
{
return reinterpret_cast<float*>((void*)&mVector));
}

c++ Garbage collection and calling destructors

Per-frame I need to allocate some data that needs to stick around until the end of the frame.
Currently, I'm allocating the data off a different memory pool that allows me to mark it with the frame count. At the end of the frame, I walk the memory pool and delete the memory that was allocated in a particular frame.
The problem I'm running into is that in order to keep a hold on the data, I have to place it in a structure thusly:
struct FrameMemory
{
uint32 frameIndex;
bool allocatedType; //0 = new(), 1 = new[]
void* pMemPtr;
}
So later, when i get around to freeing the memory, it looks something like this:
{
for(all blocks)
if(block[i].frameIndex == targetIndex)
if(block[i].allocatedType == 0)
delete block[i].pMemPtr;
else if (block[i].allocatedType ==1)
delete[] block[i].pMemPtr;
}
The issue is that, because I have to overload the pointer to the memory as a void*, the DELETE operator doesn't properly DELETE the memory as its' native base type. IE the destructor NEVER gets called for the object.
I've attempted to find ways to use smart-pointer templated objects for the solution, but in order to do that, I have to overload the templated class to a non-templated base-type, which makes deletion even more difficult.
Does anyone have a solution to a problem like this?
If you don't want to force all the objects to inherit from Destructible, you can store a pointer to a deleter function (or functor) along with the pointer to the data itself. The client code is responsible for providing a function that knows how to delete the data correctly, typically something like:
void xxx_deleter(void *data) {
xxx *ptr = static_cast<xxx *>(data);
delete ptr;
}
Though the deleter will usually be a lot like the above, this also gives the client the option of storing complex data structures and still getting them deleted correctly.
class Destructable
{
public:
virtual ~Destructable() {}
};
Instead of void *, store Destructable * in the pool. Make objects allocated from the pool inherit from Destructable.
Alternatively, override the operators new and delete for the relevant classes. Make them use the pool. Delete the object as soon as the CPU is done with it, in the regular way, in the code that owns it and hence knows its correct type; since the pool will not reuse the memory until it sees the appropriate frame end, whatever asynchronous hardware required you to lag garbage collection in this way can still do its thing.
The only way I could think of to do that would be to add a type entry to the FrameMemory struct, then use that to properly cast the memory for the delete. For example, if you have memory of type foo, you could have something like:
if (block[i].BlockType == BLOCKTYPE_FOO)
{
foo *theMemory = (foo *)block[i].pMemPtr;
delete theMemory;
}
Note that this can be an ****extremely**** dangerous operation if you do it wrong.
If you are mean stack frame (i.e. inside function)
you can try to use alloca()
First thing that I can think of is using boost::shared_ptr<void> (for the non-array version, some work may be required to adapt it for the array version) as the pointer type. And I think that should take care of mostly every detail. Whenever the frame is destroyed the memory will be appropriately deleted:
struct FrameMemory
{
uint32 frameIndex;
// bool allocatedType; //0 = new(), 1 = new[] only works with instances, not arrays
boost::shared_ptr<void> pMemPtr;
};
If you want to implement something similar manually, you can use a 'deleter' function pointer to handle the deletion of the objects, instead of calling delete directly. Here is a rough approach to how you could modify your code:
// helper deleter functions
template <typename T>
void object_deleter( void *p ) {
delete static_cast<T*>(p);
}
template <typename T>
void array_deleter( void *p ) {
delete [] static_cast<T*>(p);
}
class FrameMemory
{
public:
const uint32 frameIndex;
void* pMemPtr;
private:
void (*deleter)(void*);
public:
template <typename T>
FrameMemory( uint32 frame, T* memory, bool isarray = false )
: frameIndex(frame), pMemPtr(memory),
deleter( isarray? array_deleter<T> : object_deleter<T> )
{}
void delete() {
deleter(pMemPtr)
}
};
struct X;
void usage()
{
{
FrameMemory f( 1, new X );
f.delete();
}
{
FrameMemory f( 1, new x[10], true );
f.delete();
}
}
I would modify it further so that instead of having to call FrameMemory::delete() that was performed in the destructor, but that would take more time than I have right now to do correctly (that is, deciding how copies are to be handled and so on...
I would do something like this:
struct FrameMemoryBase
{
uint32 frameIndex;
bool allocatedType; //0 = new(), 1 = new[]
virtual void Free() = 0;
};
template <typename T>
struct FrameMemory : public FrameMemoryBase
{
void Free()
{
if(allocatedType == 0)
delete pMemPtr;
else if (allocatedType ==1)
delete[] pMemPtr;
}
T *pMemPtr;
};
which you would use via:
{
for(all blocks)
if(block[i].frameIndex == targetIndex)
block[i].Free();
}
If you also free the FrameMemory struct you could just change Free to a virtual destructor. I'm not sure this is what you're looking for since I don't understand what "I've attempted to find ways to use smart-pointer templated objects for the solution, but in order to do that, I have to overload the templated class to a non-templated base-type, which makes deletion even more difficult." means, but I hope this is helpful.
This requires that the memory management code somehow have access to the declarations of what you wish to free, but I don't think there's any way around that, assuming you want destructors called, which you explicitly do.

Variable Sized Struct C++

Is this the best way to make a variable sized struct in C++? I don't want to use vector because the length doesn't change after initialization.
struct Packet
{
unsigned int bytelength;
unsigned int data[];
};
Packet* CreatePacket(unsigned int length)
{
Packet *output = (Packet*) malloc((length+1)*sizeof(unsigned int));
output->bytelength = length;
return output;
}
Edit: renamed variable names and changed code to be more correct.
Some thoughts on what you're doing:
Using the C-style variable length struct idiom allows you to perform one free store allocation per packet, which is half as many as would be required if struct Packet contained a std::vector. If you are allocating a very large number of packets, then performing half as many free store allocations/deallocations may very well be significant. If you are also doing network accesses, then the time spent waiting for the network will probably be more significant.
This structure represents a packet. Are you planning to read/write from a socket directly into a struct Packet? If so, you probably need to consider byte order. Are you going to have to convert from host to network byte order when sending packets, and vice versa when receiving packets? If so, then you could byte-swap the data in place in your variable length struct. If you converted this to use a vector, it would make sense to write methods for serializing / deserializing the packet. These methods would transfer it to/from a contiguous buffer, taking byte order into account.
Likewise, you may need to take alignment and packing into account.
You can never subclass Packet. If you did, then the subclass's member variables would overlap with the array.
Instead of malloc and free, you could use Packet* p = ::operator new(size) and ::operator delete(p), since struct Packet is a POD type and does not currently benefit from having its default constructor and its destructor called. The (potential) benefit of doing so is that the global operator new handles errors using the global new-handler and/or exceptions, if that matters to you.
It is possible to make the variable length struct idiom work with the new and delete operators, but not well. You could create a custom operator new that takes an array length by implementing static void* operator new(size_t size, unsigned int bitlength), but you would still have to set the bitlength member variable. If you did this with a constructor, you could use the slightly redundant expression Packet* p = new(len) Packet(len) to allocate a packet. The only benefit I see compared to using global operator new and operator delete would be that clients of your code could just call delete p instead of ::operator delete(p). Wrapping the allocation/deallocation in separate functions (instead of calling delete p directly) is fine as long as they get called correctly.
If you never add a constructor/destructor, assignment operators or virtual functions to your structure using malloc/free for allocation is safe.
It's frowned upon in c++ circles, but I consider the usage of it okay if you document it in the code.
Some comments to your code:
struct Packet
{
unsigned int bitlength;
unsigned int data[];
};
If I remember right declaring an array without a length is non-standard. It works on most compilers but may give you a warning. If you want to be compliant declare your array of length 1.
Packet* CreatePacket(unsigned int length)
{
Packet *output = (Packet*) malloc((length+1)*sizeof(unsigned int));
output->bitlength = length;
return output;
}
This works, but you don't take the size of the structure into account. The code will break once you add new members to your structure. Better do it this way:
Packet* CreatePacket(unsigned int length)
{
size_t s = sizeof (Packed) - sizeof (Packed.data);
Packet *output = (Packet*) malloc(s + length * sizeof(unsigned int));
output->bitlength = length;
return output;
}
And write a comment into your packet structure definition that data must be the last member.
Btw - allocating the structure and the data with a single allocation is a good thing. You halve the number of allocations that way, and you improve the locality of data as well. This can improve the performance quite a bit if you allocate lots of packages.
Unfortunately c++ does not provide a good mechanism to do this, so you often end up with such malloc/free hacks in real world applications.
This is OK (and was standard practice for C).
But this is not a good idea for C++.
This is because the compiler generates a whole set of other methods automatically for you around the class. These methods do not understand that you have cheated.
For Example:
void copyRHSToLeft(Packet& lhs,Packet& rhs)
{
lhs = rhs; // The compiler generated code for assignement kicks in here.
// Are your objects going to cope correctly??
}
Packet* a = CreatePacket(3);
Packet* b = CreatePacket(5);
copyRHSToLeft(*a,*b);
Use the std::vector<> it is much safer and works correctly.
I would also bet it is just as efficient as your implementation after the optimizer kicks in.
Alternatively boost contains a fixed size array:
http://www.boost.org/doc/libs/1_38_0/doc/html/array.html
You can use the "C" method if you want but for safety make it so the compiler won't try to copy it:
struct Packet
{
unsigned int bytelength;
unsigned int data[];
private:
// Will cause compiler error if you misuse this struct
void Packet(const Packet&);
void operator=(const Packet&);
};
I'd probably just stick with using a vector<> unless the minimal extra overhead (probably a single extra word or pointer over your implementation) is really posing a problem. There's nothing that says you have to resize() a vector once it's been constructed.
However, there are several The advantages of going with vector<>:
it already handles copy, assignment & destruction properly - if you roll your own you need to ensure you handle these correctly
all the iterator support is there - again, you don't have to roll your own.
everybody already knows how to use it
If you really want to prevent the array from growing once constructed, you might want to consider having your own class that inherits from vector<> privately or has a vector<> member and only expose via methods that just thunk to the vector methods those bits of vector that you want clients to be able to use. That should help get you going quickly with pretty good assurance that leaks and what not are not there. If you do this and find that the small overhead of vector is not working for you, you can reimplement that class without the help of vector and your client code shouldn't need to change.
There are already many good thoughts mentioned here. But one is missing. Flexible Arrays are part of C99 and thus aren't part of C++, although some C++ compiler may provide this functionality there is no guarantee for that. If you find a way to use them in C++ in an acceptable way, but you have a compiler that doesn't support it, you perhaps can fallback to the "classical" way
If you are truly doing C++, there is no practical difference between a class and a struct except the default member visibility - classes have private visibility by default while structs have public visibility by default. The following are equivalent:
struct PacketStruct
{
unsigned int bitlength;
unsigned int data[];
};
class PacketClass
{
public:
unsigned int bitlength;
unsigned int data[];
};
The point is, you don't need the CreatePacket(). You can simply initialize the struct object with a constructor.
struct Packet
{
unsigned long bytelength;
unsigned char data[];
Packet(unsigned long length = 256) // default constructor replaces CreatePacket()
: bytelength(length),
data(new unsigned char[length])
{
}
~Packet() // destructor to avoid memory leak
{
delete [] data;
}
};
A few things to note. In C++, use new instead of malloc. I've taken some liberty and changed bitlength to bytelength. If this class represents a network packet, you'll be much better off dealing with bytes instead of bits (in my opinion). The data array is an array of unsigned char, not unsigned int. Again, this is based on my assumption that this class represents a network packet. The constructor allows you to create a Packet like this:
Packet p; // default packet with 256-byte data array
Packet p(1024); // packet with 1024-byte data array
The destructor is called automatically when the Packet instance goes out of scope and prevents a memory leak.
You probably want something lighter than a vector for high performances. You also want to be very specific about the size of your packet to be cross-platform. But you don't want to bother about memory leaks either.
Fortunately the boost library did most of the hard part:
struct packet
{
boost::uint32_t _size;
boost::scoped_array<unsigned char> _data;
packet() : _size(0) {}
explicit packet(packet boost::uint32_t s) : _size(s), _data(new unsigned char [s]) {}
explicit packet(const void * const d, boost::uint32_t s) : _size(s), _data(new unsigned char [s])
{
std::memcpy(_data, static_cast<const unsigned char * const>(d), _size);
}
};
typedef boost::shared_ptr<packet> packet_ptr;
packet_ptr build_packet(const void const * data, boost::uint32_t s)
{
return packet_ptr(new packet(data, s));
}
There's nothing whatsoever wrong with using vector for arrays of unknown size that will be fixed after initialization. IMHO, that's exactly what vectors are for. Once you have it initialized, you can pretend the thing is an array, and it should behave the same (including time behavior).
Disclaimer: I wrote a small library to explore this concept: https://github.com/ppetr/refcounted-var-sized-class
We want to allocate a single block of memory for a data structure of type T and an array of elements of type A. In most cases A will be just char.
For this let's define a RAII class to allocate and deallocate such a memory block. This poses several difficulties:
C++ allocators don't provide such API. Therefore we need to allocate plain chars and place the structure in the block ourselves. For this std::aligned_storage will be helpful.
The memory block must be properly aligned. Because in C++11 there doesn't seem to be API for allocating an aligned block, we need to slightly over-allocate by alignof(T) - 1 bytes and then use std::align.
// Owns a block of memory large enough to store a properly aligned instance of
// `T` and additional `size` number of elements of type `A`.
template <typename T, typename A = char>
class Placement {
public:
// Allocates memory for a properly aligned instance of `T`, plus additional
// array of `size` elements of `A`.
explicit Placement(size_t size)
: size_(size),
allocation_(std::allocator<char>().allocate(AllocatedBytes())) {
static_assert(std::is_trivial<Placeholder>::value);
}
Placement(Placement const&) = delete;
Placement(Placement&& other) {
allocation_ = other.allocation_;
size_ = other.size_;
other.allocation_ = nullptr;
}
~Placement() {
if (allocation_) {
std::allocator<char>().deallocate(allocation_, AllocatedBytes());
}
}
// Returns a pointer to an uninitialized memory area available for an
// instance of `T`.
T* Node() const { return reinterpret_cast<T*>(&AsPlaceholder()->node); }
// Returns a pointer to an uninitialized memory area available for
// holding `size` (specified in the constructor) elements of `A`.
A* Array() const { return reinterpret_cast<A*>(&AsPlaceholder()->array); }
size_t Size() { return size_; }
private:
// Holds a properly aligned instance of `T` and an array of length 1 of `A`.
struct Placeholder {
typename std::aligned_storage<sizeof(T), alignof(T)>::type node;
// The array type must be the last one in the struct.
typename std::aligned_storage<sizeof(A[1]), alignof(A[1])>::type array;
};
Placeholder* AsPlaceholder() const {
void* ptr = allocation_;
size_t space = sizeof(Placeholder) + alignof(Placeholder) - 1;
ptr = std::align(alignof(Placeholder), sizeof(Placeholder), ptr, space);
assert(ptr != nullptr);
return reinterpret_cast<Placeholder*>(ptr);
}
size_t AllocatedBytes() {
// We might need to shift the placement of for up to `alignof(Placeholder) - 1` bytes.
// Therefore allocate this many additional bytes.
return sizeof(Placeholder) + alignof(Placeholder) - 1 +
(size_ - 1) * sizeof(A);
}
size_t size_;
char* allocation_;
};
Once we've dealt with the problem of memory allocation, we can define a wrapper class that initializes T and an array of A in an allocated memory block.
template <typename T, typename A = char,
typename std::enable_if<!std::is_destructible<A>{} ||
std::is_trivially_destructible<A>{},
bool>::type = true>
class VarSized {
public:
// Initializes an instance of `T` with an array of `A` in a memory block
// provided by `placement`. Callings a constructor of `T`, providing a
// pointer to `A*` and its length as the first two arguments, and then
// passing `args` as additional arguments.
template <typename... Arg>
VarSized(Placement<T, A> placement, Arg&&... args)
: placement_(std::move(placement)) {
auto [aligned, array] = placement_.Addresses();
array = new (array) char[placement_.Size()];
new (aligned) T(array, placement_.Size(), std::forward<Arg>(args)...);
}
// Same as above, with initializing a `Placement` for `size` elements of `A`.
template <typename... Arg>
VarSized(size_t size, Arg&&... args)
: VarSized(Placement<T, A>(size), std::forward<Arg>(args)...) {}
~VarSized() { std::move(*this).Delete(); }
// Destroys this instance and returns the `Placement`, which can then be
// reused or destroyed as well (deallocating the memory block).
Placement<T, A> Delete() && {
// By moving out `placement_` before invoking `~T()` we ensure that it's
// destroyed even if `~T()` throws an exception.
Placement<T, A> placement(std::move(placement_));
(*this)->~T();
return placement;
}
T& operator*() const { return *placement_.Node(); }
const T* operator->() const { return &**this; }
private:
Placement<T, A> placement_;
};
This type is moveable, but obviously not copyable. We could provide a function to convert it into a shared_ptr with a custom deleter. But this will need to internally allocate another small block of memory for a reference counter (see also How is the std::tr1::shared_ptr implemented?).
This can be solved by introducing a specialized data type that will hold our Placement, a reference counter and a field with the actual data type in a single structure. For more details see my refcount_struct.h.
You should declare a pointer, not an array with an unspecified length.