I think this question might be a duplicate, but I don't know how to search for it.
I'm trying to overload operator new so that I can allow for a variable-length buffer after my class. Does my current design work as intended, or is it undefined behavior?
If the answer differs in C++98, C++03, and C++11 then please explain the differences.
struct POD { /* ...other POD members here... */ };
struct BufferedPOD : POD
{
size_t n;
BufferedPOD()
// Assume n is already initialized...
{
}
static void *operator new(size_t size)
{
return ::operator new(size);
}
static void *operator new(size_t size, size_t additional_size)
{
void *const p = operator new(size + additional_size);
static_cast<BufferedPOD *>(p)->n = additional_size;
return p;
}
static void operator delete(void *p)
{
return ::operator delete(p);
}
static void operator delete(void *p, size_t)
{
return operator delete(p);
}
};
int main()
{
std::auto_ptr<BufferedPOD> p(new (1000) BufferedPOD());
foo(p.get()); // do something with buffer
return 0;
}
First off, you are relying on undefined behavior, the memory is indeterminate upon calling the constructor.
In debug builds, it will often be filled with some marker-pattern for easier debugging, in release builds this freedom is generally just used to speed up the construction.
In both, reading indeterminate objects gets you UB, however that will play out in detail.
Anyway, you are going at it the wrong way (let's ignore violation of the "rule of three" for the time being:
Just declare matching overloads for ::operator new and ::operator delete, and a factory-function (which should be the only code with access to the only ctor you leave usable) which uses that and passes the extra-space on:
void* operator new(size_t a, size_t b) {
if(a+b< a || a+b < b)
throw new std::bad_alloc("::operator new(size_t, size_t) too big");
return operator new(a+b);
}
void operator delete(void* p, size_t a, size_t b) {return operator delete(p/*, a+b*/);}
struct Buffered : POD { // Not a pod due to inheritance
Buffered* make(size_t extra) {return new(extra) Buffered(extra);}
private:
size_t n;
Buffered(size_t extra) : n(extra) {}
Buffered(Buffered&) = delete;
void operator=(Buffered&) = delete;
};
Related
The following code is abstracted from the book << Hands-On Design Patterns with C++ >> by Fedor G. Pikus published by Packt.
Some confusions have been bugging me for weeks.
(1) How the char array mem_ is initialized?
(2) Is allocate used to allocate memory? How?
(3) Why does mem_ == p ? How was the memory delocated?
// 02_scoped_ptr.C
// Version 01 with deletion policy.
#include <cstdlib>
#include <cassert>
#include <iostream>
template <typename T, typename DeletionPolicy>
class SmartPtr {
public:
explicit SmartPtr(T* p = nullptr,
const DeletionPolicy& deletion_policy = DeletionPolicy() )
: p_(p), deletion_policy_(deletion_policy) {}
SmartPtr(const SmartPtr&) = delete;
SmartPtr& operator=(const SmartPtr&) = delete;
~SmartPtr() { deletion_policy_(p_); }
void release() { p_ = NULL; }
T* operator->() { return p_; }
const T* operator->() const { return p_; }
T& operator*() { return *p_; }
const T& operator*() const { return *p_; }
private:
T* p_;
DeletionPolicy deletion_policy_;
};
class SmallHeap {
public:
SmallHeap() {}
SmallHeap(const SmallHeap &) = delete;
SmallHeap &operator=(const SmallHeap &) = delete;
~SmallHeap() {}
void * allocate(size_t s) {
assert(s <= size_);
return mem_; // ------------------ is allocate used to allocate memory? how?
}
void deallocate(void *p) {
assert(mem_ == p); // ------------------- why does mem_ == p ? How was the memory delocated?
}
private:
static constexpr size_t size_ = 1024;
char mem_[size_]; // ------------------- how mem_ is initialized?
};
void * operator new(size_t s, SmallHeap *h)
{
return h->allocate(s);
}
template<typename T>
struct DeleteSmallHeap {
explicit DeleteSmallHeap(SmallHeap &heap)
: heap_(heap) {}
void operator()(T *p) const {
p->~T();
heap_.deallocate(p);
}
private:
SmallHeap &heap_;
};
int main() {
SmallHeap a_sh_obj;
SmartPtr<int, DeleteSmallHeap<int>> sh_sp{new(&a_sh_obj) int(42), DeleteSmallHeap<int>(a_sh_obj)};
std::cout << *sh_sp << std::endl;
}
------------------ Update 1 : how is char related to memory? --------------------
Thanks for the helpful explanations, and I need some time to them, especially the custom allocator.
But one thing that is really strange to me is that:
we are talking about memory stuff, but why do we need a char array here?
This code demonstrates a custom allocator which has a static fixed size of size (1024). There is no allocation, but it can be used as an allocator on a STL container on the assumption that you will never need more than 1024 bytes.
If you do need more, boom.
char mem_[size_];
This line initializes the static size and allocate() simply returns that without any call to new.
For the deallocation it uses a simple assert to ensure that the memory that is to be 'deleted' is the same than the one that was 'created'.
All these practises are practically non existant. If you do need a vector of a static size, use a std::array. If you need a vector of an unknown size, use the reserve() vector function to preallocate. If your vector's size is unknown but expected to be small, it's okay to leave it as it is for, in Windows (and I assume in other OSes), it eventually calls HeapAlloc and HeapFree which, for small allocations, are probably cheap, especially if the vector is within a limited scope.
If you need some flexible combination of stack/heap vector, you can use https://github.com/thelink2012/SmallVector.
How the char array mem_ is initialized?
mem_ is not initialized as in filled with values until the use of the custom new operator in new(&a_sh_obj) int(42). This only initializes a small portion of the memory though. Space is allocated on the stack however when you create the local SmallHeap a_sh_obj; variable in main.
Is allocate used to allocate memory? How?
Yes, it is used. The expression new(&a_sh_obj) int(42) uses
void * operator new(size_t s, SmallHeap *h)
{
return h->allocate(s);
}
which gets sizeof(int) passed as first parameter and &a_sh_obj as second parameter.
Why does mem_ == p? How was the memory delocated?
On destruction of sh_sp the DeleteSmallHeap<int> object is used to get rid of the object. The assert is just verification that the memory "freed" is actually the one expected. This doesn't actually deallocate anything, since the memory is still owned by a_sh_obj. It's leaving the main function that in fact releases the memory during when cleaning up a_sh_obj.
I was trying to understand the c++17 pmr.
So I did this and it is not working as I thought, what could go wrong?
template <typename T>
class Memory : public std::experimental::pmr::memory_resource {
public:
Memory() { this->memory = allocate(sizeof(T), alignof(T)); }
void *getMemory() { return this->memory; }
~Memory() { deallocate(this->memory, sizeof(T), alignof(T)); }
private:
void *do_allocate(std::size_t bytes, std::size_t alignment)
{
memory = ::operator new(bytes);
}
void do_deallocate(void *p, std::size_t bytes, std::size_t alignment)
{
::operator delete(memory);
}
bool do_is_equal(
const std::experimental::pmr::memory_resource& other) const noexcept
{
}
void *memory;
};
what can be going wrong with my implementation?
This is the client..
Memory<std::string> mem;
std::string * st = (std::string*)mem.getMemory();
st->assign("Pius");
std::cout << *st;
The polymorphic resource allocators allocate memory; that's all they do. Unlike container Allocators, they don't create objects. That's why they return void*s.
Memory resources are not meant to be used by themselves. That's why std::polymorphic_allocator<T> exists. You can also do the object creation/destruction yourself, using placement-new and manual destructor calls.
Also, your memory_resource implementation makes no sense. do_allocate should return the allocated memory, not store it internally. Your function provokes undefined behavior by returning nothing (which your compiler should have warned about).
have created an STL-style, allocator aware class which I'm trying to use with a custom CUDA allocator. The CUDA allocator works fine for allocating the data storage in unified memory, but in order for the this pointer to be accessible on both host and device, I need to make sure that whenever the data is allocated in unified memory, that the class is as well.
To solve this, I thought a simple tag dispatch would be appropriate. If the allocater is a cudaAllocator, new should create the class in unified memory, and if not, it should just return the regular new output. Unfortunately I think I'm missing something with how tag-dispatch works. Here's the relevant part of the class:
#ifdef __CUDACC__
public:
using cudaAllocator_tag = std::true_type;
using hostAllocator_tag = std::false_type;
void *operator new(size_t len)
{
return new(len, std::is_same<Alloc, cudaAllocator<value_type>>());
}
void operator delete(void *ptr)
{
return delete(ptr, std::is_same<Alloc, cudaAllocator<value_type>>());
}
void *operator new(size_t len, cudaAllocator_tag)
{
void *ptr;
cudaMallocManaged(&ptr, len);
cudaDeviceSynchronize();
return ptr;
}
void operator delete(void *ptr, cudaAllocator_tag)
{
cudaDeviceSynchronize();
cudaFree(ptr);
}
void *operator new(size_t len, hostAllocator_tag)
{
return ::new(len);
}
void operator delete(void *ptr, hostAllocator_tag)
{
::delete(ptr);
}
#endif // __CUDACC__
but the (NVCC) compiler throws up with the following errors:
2> error : expected a type specifier
2>
2> detected during instantiation of "void *CircularQueue<T, Alloc>::operator new(size_t) [with T=float, Alloc=cudaAllocator<float>]"
2>
2> main.cu(21): here
2>
2>
2>
2> error : no instance of overloaded "operator new" matches the argument list
2>
2> argument types are: (unsigned long long, size_t, std::is_same<cudaAllocator<float>, cudaAllocator<float>>)
2>
2> detected during instantiation of "void *CircularQueue<T, Alloc>::operator new(size_t) [with T=float, Alloc=cudaAllocator<float>]"
2>
2> main.cu(21): here
Any ideas on what I'm doing wrong?
There are several problems here:
new is missing the identifier of the object to create. Assuming your container is called myContainer It should be:
static void *operator new(size_t len)
{
return new myContainer(len, std::is_same<Alloc, cudaAllocator<value_type>>());
}
The arguments of operator delete cannot be overloaded like they can for new. You can get around this by having delete invoke a custom destroy function, using inline and tag-dispatch to avoid any run-time penalties.
static void operator delete(void *ptr)
{
destroy(ptr, std::is_same<Alloc, cudaAllocator<value_type>>());
}
to avoid confusion /infinite recursion, it's probably best to do with with new as well.
Complete Solution:
#ifdef __CUDACC__
public:
using cudaAllocator_tag = std::true_type;
using hostAllocator_tag = std::false_type;
using isCudaAllocator = typename std::is_same<Alloc, cudaAllocator<value_type>>;
static void *operator new(size_t len)
{
return create(len, isCudaAllocator());
}
static void operator delete(void *ptr)
{
destroy(ptr, isCudaAllocator());
}
protected:
static inline void *create(size_t len, cudaAllocator_tag)
{
void *ptr;
cudaMallocManaged(&ptr, len);
cudaDeviceSynchronize();
return ptr;
}
static inline void destroy(void *ptr, cudaAllocator_tag)
{
cudaDeviceSynchronize();
cudaFree(ptr);
}
static inline void *create(size_t len, hostAllocator_tag)
{
return ::new CircularQueue(len);
}
static inline void destroy(void *ptr, hostAllocator_tag)
{
::delete(static_cast<CircularQueue*>(ptr));
}
#endif // __CUDACC__
I am using boost shared_ptr with my own memory manager like this (stripped down example, I hope there are no errors in it):
class MemoryManager
{
public:
/** Allocate some memory.*/
inline void* allocate(size_t nbytes)
{
return malloc(nbytes);
}
/** Remove memory agian.*/
inline void deallocate(void* p)
{
free(p);
}
};
MemoryManager globalMM;
// New operators
inline void* operator new(size_t nbytes, ogl2d::MemoryManagerImpl& mm)
{
return globalMM.allocate(nbytes);
}
// Corresponding delete operators
inline void operator delete(void *p, ogl2d::MemoryManagerImpl& mm)
{
globalMM.deallocate(p);
}
/** Class for smart pointers, to ensure
* correct deletion by the memory manger.*/
class Deleter
{
public:
void operator()(void *p) {
globalMM.deallocate(p);
}
};
And I am using it like this:
shared_ptr<Object>(new(globalMM) Object, Deleter);
But now I am realizing. If the shared_ptr deletes my onject, it calls Deleter::operator() and the objects gets deleted. But the destructor does not get called ...
How can I change this?
Because deleter should destroy the object:
class Deleter
{
public:
void operator()(Object *p) {
p->~Object();
globalMM.deallocate(p);
}
};
Edit: I was wrong in my deleter, fixed
You can explicitly call the destructor (which means that the Deleter should probably receive a T * instead of a void *). Note that the code you provided doesn't actually use placement new/delete, so my answer is only meaningful for this particular example.
I have implemented a custom allocator (to be used by STL containers within my memory debugging utility, without them using my overridden new operator). Within the memory debugger I use an instance of the same allocator class to allocate the objects I need to keep track of 'normal' memory allocations. It's all working fine, but I'm not sure if the way I'm using the allocator interface is correct. Here are the utility methods as they currently stand (correct initialization parameters for the entry will be added soon):
iidebug::CMemoryDebugger::CEntry* iidebug::CMemoryDebugger::NewEntry()
{
CEntry* pEntry = m_entryAllocator.allocate(1);
if (0 != pEntry)
{
new(pEntry) CEntry(0, 0, __FILE__, 0, 0, 0);
}
return pEntry;
}
void iidebug::CMemoryDebugger::DeleteEntry( iidebug::CMemoryDebugger::CEntry* pEntry )
{
if (0 != pEntry)
{
destruct(pEntry);
m_entryAllocator.deallocate(pEntry, 1);
}
}
This just seems very messy, but I can't see how I can improve it.
You can actually overload new and delete to take an allocator parameter, like so:
inline void *operator new(size_t sizeT, Allocator *&a) {
return a->allocate(sizeT);
}
inline void operator delete(void * mem, Allocator *&a) {
a->release(mem);
}
int main()
{
Allocator * a = new Allocator;
C *c = new(a) C;
c->~C();
operator delete(c, a);
return 0;
}
See the wikipedia article for more detail. It's still a bit messy because you have to be sure not to call the regular delete operator if your allocator does something special.
Just for reference in case anyone struggles to use Drew's code, it needed a few tweaks. The following is what I ended up using:
template <typename T>
void* operator new(SizeT iSize, SimpleAllocator<T>& rAllocator)
{
return rAllocator.allocate(1);
}
template <typename T>
void operator delete(void* pvMemory, SimpleAllocator<T>& rAllocator)
{
((T*)pvMemory)->~T();
rAllocator.deallocate(pvMemory, 1);
}
And actually using it is as simple as:
// SimpleAllocator<CFoo> fooAllocator = ...
CFoo* pFoo = new(fooAllocator) CFoo(param1, param2, ...);
// Do stuff with pFoo...
operator delete(pFoo, fooAllocator);
What is destruct? I suppose it should be:
void iidebug::CMemoryDebugger::DeleteEntry( iidebug::CMemoryDebugger::CEntry* pEntry )
{
if (0 != pEntry)
{
pEntry->~CEntry();
m_entryAllocator.deallocate(pEntry, 1);
}
}