So I am making a Memory Stack Allocator that is capable of allocating any instance of any type onto the heap in a continuous fashion.
In order to do this i have added a 'AllocationHeader' class directly before each allocation which contains a void* to the instance address itself and a AllocationHeader* to the previous header.
So far this has worked perfectly and i'm able to allocate and deallocate etc.
But i've ran into 1 (possibly 2) problems.
In order to deallocate properly I also need to call the destructor of the object in question and then remove it from the stack.
This is fine when i'm removing an instance that i have a pointer to as i can just pass it to the deallocate function and it knows what type it is and does its thing.
The problem lies when I want to create functions like deallocateAll() or deallocateAllAboveObject(T* obj) as I would need to know the types of each allocation without explicitly passing their pointer, so i could go down each allocation in the stack calling the deconstructor on each as I go.
What I would like is to be able to create an AllocationHeader which can store a pointer of any type, and then the type of the pointer can be retrieved at a later date without already knowing the type.
I'm really not sure how to do this as all suggestions that i've seen so far involve comparing the object with a specific type and seeing if they're the same. However I can't just check against each possible class in the program as there could be 1000's as I continue to build on this.
Alternatively if you have any suggestions for an alternative approach to a stack allocator that can deal with any type, that would be great as well.
Any help at all would be greatly appreciated.
Due to C++'s static type system, I can only see a couple of solutions to the problem
Make every type you use with the allocator derive from a (consistent) type with a virtual distructor e.g. struct destructible { virtual ~destructible() { } }, however this will potentially enlarge & alter the layout of any types you alter to derive from this.
Or uppon allocation store a function object that does the destruction, e.g. using the following template
template<typename T> void destroy(void* p) { reinterpret_cast<T*>(p)->T::~T(); }
struct AllocationHeader
{
template<typename T> AllocationHeader(AllocationHeader* previouse, void* data)
: previouse(previouse), data(data), destructor(&destroy<T>) { }
AllocationHeader* previouse;
void* data;
void (*destructor)(void*);
}
void deallocateAll(AllocationHeader& start)
{
for (AllocationHeader* a = &start; a != nullptr; a = start->previouse;)
{
a->destructor(a->data);
delete a->data;
}
}
(Without providing your code for the AllocationHeader type it is difficuilt to help you.)
Note: My compiler/IDE is currently reinstalling so I am unable to test the above code, I am pretty sure about most of it, except I may need to put a typename in the destructor call syntax reinterpret_cast<T*>(p).T::~T();
EDIT Using a template constructor, where the template arguments cannot be inferred is a bad idea, the following constructor should be used instead
AllocationHeader(AllocationHeader* previouse, void* data, void(*destructor)(void*))
: previouse(previouse), data(data), destructor(destructor) { }
Just pass &destroy<T> as the 3rd argument to it.
Related
Say I have two classes inheriting from a common base, such as
class Thing{
public:
virtual void f()=0;
};
class Thing_variant_a: public Thing{
public:
void f(){
std::cout<<"I am (a)"<<std::endl;
}
};
class Thing_variant_b: public Thing{
public:
void f(){
std::cout<<"I am (b)"<<std::endl;
}
};
And a function taking a reference to a Thing object as an argument.
void function(Thing& t){
t.f();
}
Depending on conditions I would like to call function with either a thing_a or thing_b (and possibly extend this at some point adding another possibility of thing_c)
I know I can do this using a pointer
Thing *t = nullptr;
if(condition_a){
t = new Thing_variant_a();
} else if(condition_b){
t = new Thing_variant_b();
}
function(*t);
However, I would like to know if there is a better way, that
does not allocate heap memory
does not require me to take care of deleting t at some point (probably smart pointers, but I don't know much about those)
ensures I always pass a valid Thing reference to function (there might be more conditionals in a complicated structure than in this minimal example) I could do if(t){ function(*t);}else{/*handle error*/}), but it seems like there should be a more elegant solution.
If not all of the above are possible any combination of those?
This sounds very much like an XY problem. There is probably a different solution to your problem entirely.
C++ is a statically-typed language; that means types used in a given code path are fixed at compile-time. Dynamic types (types known at run time) are normally allocated via the heap or all-at-once and then selected at run time.
So not much is possible in your case as you've noticed..
You could for example just have two different code paths:
if (condition_a) {
Thing_variant_a a;
function(a);
} else if (condition_b) {
Thing_variant_a b;
function(b);
}
Preallocate the types:
Thing_variant_a a;
Thing_variant_a b;
if (condition_a) {
function(a);
} else if (condition_b) {
function(b);
}
Or use a template:
template<typename T>
void do_something() {
T t;
function(t);
}
// somewhere else in the code ...
do_something<Thing_variant_a>();
// or ...
do_something<Thing_variant_b>();
Here's a way using dynamic memory and unique_ptr:
std::unique_ptr<Thing> t;
if (condition_a) {
t = std::make_unique<Thing_variant_a>();
} else if (condition_b) {
t = std::make_unique<Thing_variant_b>();
}
function(*t);
// t is delete'd automatically at end of scope...
And by the way, a function like int f(){...} should return some int value.
Here is a way to do it without using the heap or pointers:
Thing_variant_a thingA;
Thing_variant_b thingB;
if(condition_a){
function(thingA);
} else if(condition_b){
function(thingB);
}
If you want, you reduce it to a single call via the ternary operator:
Thing_variant_a thingA;
Thing_variant_b thingB;
function(condition_a ? static_cast<Thing &>(thingA) : static_cast<Thing &>(thingB));
As far as references go, references in C++ are required to be always be non-NULL -- so if you try to dereference a NULL pointer (e.g. by calling function(*t) when t==NULL) you've already invoked undefined behavior and are doomed; there is nothing the code inside function() can do to save you. So if there is any change that your pointer is NULL, you must check for that before dereferencing it.
I'll try to answer each of your questions
does not allocate heap memory
Unfortunately c++ only supports polymorphism using pointers. I guess the problem you would face here is fragmented memory (meaning that your pointers are everywhere in the heap). The best way to handle that is to allocate the memory using a memory pool.
You could use an std::variant but you will still need to test for the currently available type in the variant.
does not require me to take care of deleting t at some point (probably smart pointers, but I don't know much about those)
You could use a std::unique_ptr which will basically called the destructor when no one holds that pointer anymore.
ensures I always pass a valid Thing reference to function (there might be more conditionals in a complicated structure than in this minimal example) I could do if(t){ function(*t);}else{/handle error/}), but it seems like there should be a more elegant solution.
If you use pointers your could just check for the nullptr as you are doing right now. I'm not sure what you are meaning by valid reference as a reference always points toward something and cannot be empty.
have a whole list of C wrappers for OpenCV C++ functions like the one below. And all of them return a "new". I can't change them because they are becoming part of OpenCV and it would make my library perfect to have a consistently updated skeleton to wrap around.
Mat* cv_create_Mat() {
return new Mat();
}
I can't rewrite the C wrapper for the C++ function so I wrote a delete wrapper like the one below,The memory I'm trying to free is a Mat*, Mat is an OpenCV c++ class...and the delete wrapper below works. There is absolutely no memory leakage at all.
I have a lot of other C wrappers for OpenCV C++ functions, though, that return a new pointer...there is at least 10 or 15 and my intention is to not have to write a separate delete wrapper for all of them. If you can show me how to write one delete wrapper that would free any pointer after having it not have to be told which type to free and fast too that would be awesome.
Those are my intentions and I know you great programmers can help me with that solution:)...in a nutshell...I have CvSVMParams*, Brisk*, RotatedRect*, CVANN_MLP* pointers there are a few others as well that all need to be free'd with one wrapper...one go to wrapper for C++'s delete that would free anything...Any help at this is greatly valued.
void delete_ptr(void* ptr) {
delete (Mat*)ptr;
}
Edit: I'd need one of the two of you who I sent the messages to, to tell me exactly how to run your posted code...The registry version doesn't work when I place in Emacs g++ above the main and run with Free(cv_create_Mat); a new Mat* creator and stub gets 5 error message running the same way. I need exact compile instructions. My intention is to be able to compile this to .so file You have really a lot of attention to this post though and I do appreciate it..Thank you
How about this, and then let the compiler deal with all the specializations:
template <typename T>
void delete_ptr(T *ptr) {
delete ptr;
}
The delete operator doesn't just free memory, it also calls destructors, and it has to be called on a typed pointer (not void*) so that it knows which class's destructor to call. You'll need a separate wrapper for each type.
For POD types that don't have destructors, you can allocate with malloc() instead of new, so that the caller can just use free().
I would advise against having a generic delete_ptr function.
Since creation and deletion come in pairs, I would create one for creation and for deletion of specific types.
Mat* cv_create_Mat();
void cv_delete_Mat(Mat*);
If you do this, there will be less ambiguity about the kind of object you are dealing with. Plus, the implementation of cv_delete_Mat(Mat*) will be less error prone and has to assume less.
void cv_delete_Mat(Mat* m)
{
delete m;
}
Generic operations like this can only be implemented in C by removing type information (void*) or by individually ensuring all of the wrapper functions exist.
C's ABI doesn't allow function overloading, and the C++ delete keyword is exactly the sort of generic wrapper you are asking for.
That said, there are things you can do, but none of them are any simpler than what you are already proposing. Any generic C++ code you write will be uncallable from C.
You could add members to your objects which know how to destroy themselves, e.g.
class CVersionOfFoo : public Foo {
...
static void deleter(CVersionOfFoo* p) { delete p; }
};
But that's not accessible from C.
Your last option is to set up some form of manual registry, where objects register their pointer along with a delete function. But that's going to be more work and harder to debug than just writing wrappers.
--- EDIT ---
Registry example; if you have C++11:
#include <functional>
#include <map>
/* not thread safe */
typedef std::map<void*, std::function<void(void*)>> ObjectMap;
static ObjectMap s_objectMap;
struct Foo {
int i;
};
struct Bar {
char x[30];
};
template<typename T>
T* AllocateAndRegister() {
T* t = new T();
s_objectMap[t] = [](void* ptr) { delete reinterpret_cast<T*>(ptr); };
return t;
}
Foo* AllocateFoo() {
return AllocateAndRegister<Foo>();
}
Bar* AllocateBar() {
return AllocateAndRegister<Bar>();
}
void Free(void* ptr) {
auto it = s_objectMap.find(ptr);
if (it != s_objectMap.end()) {
it->second(ptr); // call the lambda
s_objectMap.erase(it);
}
}
If you don't have C++11... You'll have to create a delete function.
As I said, it's more work than the wrappers you were creating.
It's not a case of C++ can't do this - C++ is perfectly capable of doing this, but you're trying to do this in C and C does not provide facilities for doing this automatically.
The core problem is that delete in C++ requires a type, and passing a pointer through a C interface loses that type. The question is how to recover that type in a generic way. Here are some choices.
Bear in mind that delete does two things: call the destructor and free the memory.
Separate functions for each type. Last resort, what you're trying to avoid.
For types that have a trivial destructor, you can cast your void pointer to anything you like because all it does it free the memory. That reduces the number of functions. [This is undefined behaviour, but it should work.]
Use run-time type information to recover the type_info of the pointer, and then dynamic cast it to the proper type to delete.
Modify your create functions to store the pointer in a dictionary with its type_info. On delete, retrieve the type and use it with dynamic cast to delete the pointer.
For all that I would probably use option 1 unless there were hundreds of the things. You could write a C++ template with explicit instantiation to reduce the amount of code needed, or a macro with token pasting to generate unique names. Here is an example (edited):
#define stub(T) T* cv_create_ ## T() { return new T(); } \
void cv_delete_ ## T(void *p) { delete (T*)p; }
stub(Mat);
stub(Brisk);
One nice thing about the dictionary approach is for debugging. You can track new and delete at run-time and make sure they match correctly. I would pick this option if the debugging was really important, but it takes more code to do.
I encountered a question when I was reading the item28 in More Effective C++. In this item, the author shows to us that we can use member template in SmartPtr such that the SmartPtr<Cassette> can be converted to SmartPtr<MusicProduct>.
The following code is not the same as in the book, but has the same effect.
#include <iostream>
class Base {};
class Derived : public Base {};
template<typename T>
class smart {
public:
smart(T* ptr)
: ptr(ptr)
{}
template<typename U>
operator smart<U>()
{
return smart<U>(ptr);
}
~smart()
{
delete ptr;
}
private:
T* ptr;
};
void test(const smart<Base>& ) {}
int main()
{
smart<Derived> sd(new Derived);
test(sd);
return 0;
}
It indeed can be compiled without compilation error. But when I ran the executable file, I got a core dump. I think that's because the member function of the conversion operator makes a temporary smart, which has a pointer to the same ptr in sd (its type is smart<Derived>). So the delete directive operates twice. What's more, after calling test, we can never use sd any more, since ptr in sd has already been delete.
Now my questions are :
Is my thought right? Or my code is not the same as the original code in the book?
If my thought is right, is there any method to do this?
Thanks very much for your help.
Yes, you've described the problem with your code fairly accurately.
As far as how to make it work: just about like the usual when you run into problems from a shallow copy: do a deep copy instead. That is, instead of just creating another pointer to the same data, you'd need to clone the data, and have the second object point to the clone of the data instead of the original data.
Alternatively, use a reference counted pointer, and increment the reference count when you do a copy, and decrement it when a copy is destroyed. When the count reaches zero (and not before) delete the pointee data.
Generally speaking: avoid doing all of this. Assuming you're using a relatively up-to-date compiler, the standard library should already contain a shared_ptr and a unique_ptr that can handle a lot of your smart pointer needs.
Your interpretation is correct, the conversion operator will create a different object that holds a pointer to the same underlying object. Once it goes out of scope it will be destroyed, and it will in turn call delete.
Not sure I understand the last question, if what you ask is whether this can be useful or not, it can be useful if implemented correctly. For example, if instead of a raw pointer and manually allocating/deleting the memory you were using a std::shared_ptr then it would work just fine. In other cases there might not even be a dynamically allocated object... This is just a tool, use it where it makes sense.
I encounter a confused question when I go through the source code of firebreath (src/ScriptingCore/Variant.h)
// function pointer table
struct fxn_ptr_table {
const std::type_info& (*get_type)();
void (*static_delete)(void**);
void (*clone)(void* const*, void**);
void (*move)(void* const*,void**);
bool (*less)(void* const*, void* const*);
};
// static functions for small value-types
template<bool is_small>
struct fxns
{
template<typename T>
struct type {
static const std::type_info& get_type() {
return typeid(T);
}
static void static_delete(void** x) {
reinterpret_cast<T*>(x)->~T();
}
static void clone(void* const* src, void** dest) {
new(dest) T(*reinterpret_cast<T const*>(src));
}
static void move(void* const* src, void** dest) {
reinterpret_cast<T*>(dest)->~T();
*reinterpret_cast<T*>(dest) = *reinterpret_cast<T const*>(src);
}
static bool lessthan(void* const* left, void* const* right) {
T l(*reinterpret_cast<T const*>(left));
T r(*reinterpret_cast<T const*>(right));
return l < r;
}
};
};
// static functions for big value-types (bigger than a void*)
template<>
struct fxns<false>
{
template<typename T>
struct type {
static const std::type_info& get_type() {
return typeid(T);
}
static void static_delete(void** x) {
delete(*reinterpret_cast<T**>(x));
}
static void clone(void* const* src, void** dest) {
*dest = new T(**reinterpret_cast<T* const*>(src));
}
static void move(void* const* src, void** dest) {
(*reinterpret_cast<T**>(dest))->~T();
**reinterpret_cast<T**>(dest) = **reinterpret_cast<T* const*>(src);
}
static bool lessthan(void* const* left, void* const* right) {
return **reinterpret_cast<T* const*>(left) < **reinterpret_cast<T* const*>(right);
}
};
};
template<typename T>
struct get_table
{
static const bool is_small = sizeof(T) <= sizeof(void*);
static fxn_ptr_table* get()
{
static fxn_ptr_table static_table = {
fxns<is_small>::template type<T>::get_type
, fxns<is_small>::template type<T>::static_delete
, fxns<is_small>::template type<T>::clone
, fxns<is_small>::template type<T>::move
, fxns<is_small>::template type<T>::lessthan
};
return &static_table;
}
};
The question is why the implementation of static functions for the big value-types (bigger than void*) is different from the small ones.
For example, static_delete for small value-type is just to invoke destructor on T instance, while that for big value-type is to use 'delete'.
Is there some trick? Thanks in advance.
It looks like Firebreath uses a dedicated memory pool for its small objects, while large objects are allocated normally in the heap. Hence the different behaviour. Notice the placement new in clone() for small objects, for instance: this creates the new object in a specified memory location without allocating it. When you create an object using placement new, you must explicitly call the destructor on it before deallocating memory, and this is what static_delete() does.
Memory is not actually deallocated because, as I say, it looks like a dedicated memory pool is in use. Memory management must be performed somewhere else. This kind of memory pool is a common optimisation for small objects.
What does the internal documentation say? If the author hasn't
documented it, he probably doesn't know himself.
Judging from the code, the interface to small objects is different than
that to large objects; the pointer you pass for a small object is a
pointer to the object itself, where as the one you pass for a large
object is a pointer to a pointer to the object.
The author, however, doesn't seem to know C++ very well (and I would
avoid using any code like this). For example, in move, he explicitly
destructs the object, then assigns to it: this is guaranteed undefined
behavior, and probably won't work reliably for anything but the simplest
built-in types. Also the distinction small vs. large objects is largely
irrelevant; some “small” objects can be quite expensive to
copy. And of course, given that everything here is a template anyway,
there's absolutely no reason to use void* for anything.
I have edited your question to include a link to the original source file, since obviously most of those answering here have not read it to see what is actually going on. I admit that this is probably one of the most confusing pieces of code in FireBreath; at the time, I was trying to avoid using boost and this has worked really well.
Since then I've considered switching to boost::any (for those itching to suggest it, no, boost::variant wouldn't work and I'm not going to explain why here; ask another question if you really care) but we have customized this class a fair amount to make it exactly what we need and boost::any would be difficult to customize in a similar manner. More than anything, we've been following the old axim: if it ain't broke, don't fix it!
First of all, you should know that several C++ experts have gone over this code; yes, it uses some practices that many consider dubious, but they are very carefully considered and they are consistent and reliable on the compilers supported by FireBreath. We have done extensive testing with valgrind, visual leak detector, LeakFinder, and Rational Purify and have never found any leaks in this code. It is more than a bit confusing; it's amazing to me that people who don't understand code assume the author doesn't know C++. In this case, Christopher Diggins (who wrote the code you quoted and the original cdiggins::any class that this is taken from) seems to know C++ extremely well as evidenced by the fact that he was able to write this code. The code is used internally and is highly optimized -- perhaps more than FireBreath needs, in fact. However, it has served us well.
I will try to explain the answer to your question as best I remember; keep in mind that I don't have a lot of time and it's been awhile since I really dug in deep with this. The main reason for "small" types using a different static class is that "small" types are pretty much built-in types; an int, a char, a long, etc. Anything bigger than void* is assumed to be an object of some sort. This is an optimization to allow it to reuse memory whenever possible rather than deleting and reallocating it.
If you look at the code side-by-side it's a lot clearer. If you look at delete and clone you'll see that on "large" objects it's dynamically allocating the memory; it calls "delete" in delete and in clone it uses a regular "new". In the "small" version it just stores the memory internally and reuses it; it never "delete"s the memory, it just calls the destructor or the constructor of the correct type on the memory that it has internally. Again, this is just done for the sake of efficiency. In move on both types it calls the destructor of the old object and then assigns the new object data.
The object itself is stored as a void* because we don't actually know what type the object will be; to get the object back out you have to specify the type, in fact. This is part of what allows the container to hold absolutely any type of data. That is the reason there are so many reinterpret_cast calls there -- many people see that and say "oh, no! The author must be clueless!" However, when you have a void* that you need to dereference, that's exactly the operator that you would use.
Anyway, all of that said, cdiggins has actually put out a new version of his any class this year; I'll need to take a look at it and probably will try to pull it in to replace the current one. The trick is that I have customized the current one (primarily to add a comparison operator so it can be put in a STL container and to add convert_cast) so I need to make sure I understand the new version well enough to do that safely.
Hope that helps; the article I got it from is here: http://www.codeproject.com/KB/cpp/dynamic_typing.aspx
Note that the article has been updated and it doesn't seem to be possible to get to the old one with the original anymore.
EDIT
Since I wrote this we have confirmed some issues with the old variant class and it has been updated and replaced with one that utilizes boost::any. Thanks to dougma for most of the work on this. FireBreath 1.7 (current master branch as of the time of this writing) contains that fix.
I've got a generic class that manages resources of all kinds of types, but since I don't want to create an instance of ResourceManager for every T there is (thus having one resource manager for each type T), I have to make the type of T unknown to the ResourceManager class.
I do this by saving a map of void* pointers and converting them back to the required format if someone requests a certain type out of a templated Load() method;
template <typename T>
T* Load(const std::string &location)
{
//do some stuff here
//everybody take cover!!!
return static_cast<T*>(m_resources[location]);
}
I use template specialization to introduce different Loaders to the class:
template<>
AwesomeType* Load(const std::string &location)
{
//etc.
return static_cast<AwesomeType*>(m_resources[location]);
}
I am aware that this is ugly, but there is no way around it right now. I could introduce static maps in the inside of the specialized Load methods, but that way I can't bind the lifetime of the resources to the lifetime of an ResourceManager object, which is an essential feature.
But since this is somewhat dangerous (since those void* pointers can be anything), I'd like to at least check at runtime if the conversion is going to work, so I can react to it without having the application crash.
How can I do this?
There is no way to check what you can cast void* to, unless you store additional information that indicates the actual type with each pointer.
A more "C++ way" to do what you want is to derive each resource class from an abstract base class Resource, and store a map of pointers to Resource in your resource manager. Then you can use dynamic_cast<T*> to convert to the required type, and this will return NULL if the pointer is to an object of the wrong type. Or (depending on what you want to do) you can simply return a Resource* pointer and use virtual functions to implement the functionality of each resource.
You can easily do this, if you extend your saved value type - make it a struct that also saves a type_info object:
#include <type_info>
struct ResourceInfo
{
std::type_info const& info;
void* ptr;
};
// ...
// just to give you the general idea
template<class Res>
void CacheResource(std::string const& location, Res* res)
{
ResourceInfo ri = { typeid(Res), res };
m_resources.insert(std::make_pair(location, ri));
}
template<class Res>
Res* Load(std::string const& location)
{
map_type::const_iterator res_it = m_resources.find(location);
if(res_it != m_resources.end())
{
if(typeid(Res) != res_it->second.info)
{
throw SorryBuddyWrongResourceType(some_info_here);
}
return static_cast<Res*>(res_it->second.ptr);
}
}
This is similar to how I do it, but I use a shared_ptr<void> to save the resources.
(I'm sure this is already answered by many other questions about void pointers, but here we go ...)
But since this is somewhat dangerous
(since those void* pointers can be
anything), I'd like to at least check
at runtime if the conversion is going
to work, so I can react to it without
having the application crash.
You cannot check. This is the thing about void* pointers. You don't have a clue what they are pointing to, and you cannot (are not allowed to) inspect the memory they point to without knowing its type.
If you have a void* you simply must know beforehand what it is really pointing to and then cast appropriately.