Custom allocator leaking memory - c++

So the implementation of my custom allocator has a base class with 2 static variables, one to track the instances of allocators and one is the memory-pool.
template <typename T>
class Allocator : public Base_Allocator
{
public:
// Required types
typedef T value_type;
typedef size_t size_type;
typedef ptrdiff_t difference_type;
typedef T* pointer;
typedef const T* const_pointer;
typedef T& reference;
typedef const T& const_reference;
template <typename U>
struct rebind
{
typedef Allocator<U> other;
};
// Required Opeartions
explicit Allocator(void) : Base_Allocator()
{
}
~Allocator(void) { }
Allocator(const Allocator& a) : Base_Allocator()
{
} // copy constructor
pointer address(reference value) const { return &value; }
const_pointer address(const_reference value) const { return &value; }
size_type max_size(void) const { size_type m = 4096; return m; }
pointer allocate(size_type n)
{
return static_cast<value_type*>( Base_Allocator::m_pMemMngr->Alloc( sizeof(value_type) * n) );
}
void deallocate(pointer p, size_type n) {
Base_Allocator::m_pMemMngr->Free(p);
}
void construct(pointer p, const T& value) {
new((T*)p) T(value);
}
void destroy(pointer p) {
p->~T();
}
bool operator == (const Allocator& right) const { return true; }
bool operator != (const Allocator& right) const { return false; }
};
and here is the baseclass...
class Base_Allocator
{
public:
static int m_icount;
static MemoryManager* m_pMemMngr;
public:
Base_Allocator(void)
{
m_icount++;
if(!m_pMemMngr)
{
NEW(m_pMemMngr);
m_pMemMngr->Init();
}
}
~Base_Allocator(void)
{
m_icount--;
if(m_icount<0)
{
SAFE_DELETE(m_pMemMngr);
}
}
};
Here is the definition of the static members
#include "Base.h"
int Base_Allocator::m_icount = 0;
MemoryManager* Base_Allocator::m_pMemMngr = nullptr;
my thing here is that the memory is never being released. I'm passing it to a forward_list and this forward list creates 3 allocators but it also deletes 3. That's the reason I have the base class only free the memory once it is less than 0. But things haven't really worked out too well. -1 is never reached so I'm never releasing the memory in the memorypools. Any ideas would be appreciated.

from effective STL
Make your allocator a template, with the template parameter T representing the type of objects for which you are allocating memory.
Satisfied
Provide the typedefs pointer and reference, but always have pointer be T* and reference be T&.
Satisfied
Never give your allocators per-object state. In general, allocators should have no nonstatic data members.
Not Satisfied, you have private member in ur allocator!
Remember that an allocator's allocate member functions are passed the
number of objects for which memory is required, not the number of
bytes needed. Also remember that these functions return T* pointers Ma
the pointer typedef), even though no T objects have yet been
constructed.
Satisfied
Be sure to provide the nested rebind template on which standard
containers depend.
Satisfied
so remove ur private member...

Related

Replacing global operator new / delete and allocating memory within these, leading to heap corruption

#include <cstdlib>
#include <memory>
#include <unordered_map>
template <class T>
struct allocator {
typedef size_t size_type;
typedef ptrdiff_t difference_type;
typedef T* pointer;
typedef const T* const_pointer;
typedef T& reference;
typedef const T& const_reference;
typedef T value_type;
allocator() = default;
template <class U>
allocator(const allocator<U>&) {}
T* allocate(std::size_t n) const { return (T*)malloc(n); } // debugger breaks here
void deallocate(T* p, std::size_t) const { free(p); }
};
using allocations_map =
std::unordered_map<void*, std::size_t, std::hash<void*>,
std::equal_to<void*>,
allocator<std::pair<void* const, std::size_t>>>;
allocations_map allocations; // heap corruption in the constructor
void* operator new(std::size_t n) {
auto p = malloc(n);
allocations.emplace(p, n);
return p;
}
void operator delete(void* p) noexcept {
allocations.erase(p);
free(p);
}
int main() { std::vector<int> v(5); }
Why do i corrupt the heap in the constructor of allocations_map? The debugger detects the first heap corruption in a malloc call of the custom allocator, called inside the constructor.
Is there a more elegant solution then to write a non-logging custom allocator for allocations_map? The container shall obviously not log its own allocations.
I also tried two singleton approaches, as suggested in the comments, without success:
allocations_map& get_allocations_map()
{
static allocations_map* allocations_ptr = nullptr;
if (allocations_ptr == nullptr)
{
allocations_ptr = (allocations_map*) malloc(sizeof(allocations_map));
allocations_ptr = new(allocations_ptr)allocations_map;
}
return *allocations_ptr;
}
allocations_map& get_allocations_map()
{
static allocations_map allocations;
return allocations;
}
From std::allocator::allocate allocator allocates n "things" not n bytes. You should change:
T* allocate(std::size_t n) const { return (T*)malloc(n); }
to:
T* allocate(std::size_t n) const { return (T*)malloc(sizeof(T) * n); }
Why do i corrupt the heap in the constructor of allocations_map?
Because the constructor of elements stored in that map access allocated memory out-of-bounds.

Custom pool allocator std::list [duplicate]

This question already has answers here:
How can I write a stateful allocator in C++11, given requirements on copy construction?
(3 answers)
Closed 7 years ago.
I'm writing a custom allocator to be used with std::list. The list size will always be bounded to a small number and the list elements will be allocated and deallocated very frequently within a constraint tree search algorithm and thus I think a custom pool allocator (that allocates elements out of the stack) ought to improve performance.
My question is how does std::list allocated the data structures used to store the links/nodes. It will use the custom allocator to allocate the elements but then that wouldn't really help much if the nodes are still allocated from the heap.
This is the custom allocator I'm implementing:
#include <algorithm>
#include <cassert>
template <class Tp, std::size_t N>
class PoolStackAllocator {
public:
typedef Tp value_type;
typedef value_type* pointer;
typedef const value_type* const_pointer;
typedef value_type& reference;
typedef const value_type& const_reference;
typedef std::size_t size_type;
typedef std::ptrdiff_t difference_type;
template<typename U>
struct rebind {
typedef PoolStackAllocator<U, N> other;
};
inline explicit PoolStackAllocator() : data_size_(0) {}
template <class U, std::size_t M>
inline explicit PoolStackAllocator(const PoolStackAllocator<U, M>& other) : data_size_(other.data_size_) {
typename PoolStackAllocator<U>::const_pointer i = other.data_;
typename PoolStackAllocator<U>::const_pointer end = other.data_ + data_size_;
pointer j = data_;
while (i != end){
*j++ = Tp(*i++);
}
j = data_ + data_size_;
pointer* k = free_;
pointer end_ = data_ + 25;
while (j != end_){
*k++ = j++;
}
}
inline pointer address(reference r) { return &r; }
inline const_pointer address(const_reference r) { return &r; }
inline pointer allocate(size_type n){
assert(n == 1);
assert(data_size_ < N);
return free_[data_size_++];
}
inline void deallocate(Tp* p, size_type n){
assert(n == 1);
free_[--data_size_] = p;
}
inline size_type max_size(){
return 1;
}
private:
size_type data_size_;
value_type* free_[N];
value_type data_[N];
};
template <class T, class U, std::size_t N>
inline bool operator==(const PoolStackAllocator<T, N>& a, const PoolStackAllocator<U, N>& b){
return &a == &b;
}
template <class T, class U, std::size_t N>
inline bool operator!=(const PoolStackAllocator<T, N>& a, const PoolStackAllocator<U, N>& b){
return &a != &b;
}
And this is an example of how I intend to use it.
typedef std::forward_list<Alien, PoolStackAllocator<Alien, 25>> Aliens;
Edit 1:
I got an answer on how std::list allocates the data structures:
In short, given allocator, we can simply do allocator::rebind::other.allocate(1) to be allocating memory large enough to hold an object U. This is the magic required for std::list to work properly, since given std::list(allocator()), std::list actually needs to allocate memory for Node, and not int. Thus, they need to rebind to allocator()::rebind >::other instead.
http://www.codeproject.com/Articles/4795/C-Standard-Allocator-An-Introduction-and-Implement
But now I'm still puzzled with a new question. How to implement the copy constructor.
When I create a list like:
std::forward_list<int, PoolStackAllocator<int>> ints;
This copy constructor gets called:
template <class Tp, std::size_t N>
class PoolStackAllocator {
...
template <class U, std::size_t M>
inline explicit PoolStackAllocator(const PoolStackAllocator<U, M>& other);
...
}
with
U = int
Tp = std::_Fwd_list_node<int>
I don't know what to do in this copy constructor. From the example allocaters I've seen it seems that nothing must be done there. But why? Why does it get called then?
Edit 2
Got the answer from:
How can I write a stateful allocator in C++11, given requirements on copy construction?
the container uses the same allocator to allocate nodes and memory for the stored objects.

Bizarre error message in VC12 when attempting to create pool allocator

I am attempting to create a simple pool allocator that can work with containers like list and map. First, I have the class free_list which inherits from the node type that is used by the container and uses a specialized forward linked list (with all its memory already allocated--allocation occurs when the first item is allocated by the allocator) to give and take back memory requested in the allocator's allocate and deallocate functions. Then I have the class pool_alloc which implements the allocator itself and returns and accepts in its allocation and deallocation functions free_list<T> object types.
The problem child is this statement: m_next->give_back(ptr); under pool_alloc::deallocate, which returns VC12 error C2664: It says I cannot convert from type 'free_list<T>*' to type 'free_list<free_list<T>>*'.
I do not understand why free_list<T>::give_back expects type free_list<free_list<T>>* when one would expect it to expect free_list<T>*.
Is there any way to fix this?
The full source code is below:
#include <algorithm>
template<class T>
class free_list : public T {
public:
free_list* init(std::size_t num_elements) {
m_next = this;
free_list* temp = m_next + 1;
free_list* runner = m_next;
for (std::size_t s = 1; s < num_elements; ++s) {
runner->m_next = temp;
runner = temp;
temp = runner + 1;
}
runner->m_next = nullptr;
return m_next;
}
free_list* obtain() {
free_list* head = m_next;
m_next = m_next->m_next;
return head;
}
void give_back(free_list* ptr) {
ptr->m_next = m_next;
m_next = ptr;
}
free_list* m_next;
};
template<class T>
class pool_alloc {
typedef pool_alloc<T> myt;
public:
typedef std::size_t size_type;
typedef std::ptrdiff_t difference_type;
typedef free_list<T> value_type;
typedef T& reference;
typedef const T& const_reference;
typedef free_list<T>* pointer;
typedef const free_list<T>* const_pointer;
template<class U>
struct rebind {
typedef pool_alloc<U> other;
};
pool_alloc() : data(nullptr), m_next(nullptr), capacity(4096), allocated(0) {}
pool_alloc(std::size_t capacity) : data(nullptr), m_next(nullptr), capacity(capacity), allocated(0) {}
T* address(reference ref) {
return &ref;
}
const T* address(const_reference ref) const {
return &ref;
}
std::size_t max_size() const {
return capacity;
}
pointer allocate(std::size_t) {
if (allocated >= capacity) throw(std::bad_alloc());
if (allocated == 0) {
data = (void*) new char[capacity * sizeof(free_list<T>)];
m_next = static_cast<free_list<value_type>*>(data);
m_next->init(capacity);
}
return m_next->obtain();
}
void deallocate(pointer ptr, std::size_t) {
m_next->give_back(ptr);
--allocated;
if (allocated == 0) {
delete[](char*)(data);
data = nullptr;
m_next = nullptr;
}
}
template<class T, class... Args>
void construct(T* ptr, Args&&... args) {
::new ((void*) ptr) T(std::forward<Args>(args)...);
}
template <class T>
void destroy(T* ptr) {
ptr->~T();
}
bool operator==(const myt& other) {
return (char)(data) == (char)(other.data);
}
bool operator!=(const myt& other) {
return !operator==(other);
}
private:
void* data;
free_list<value_type>* m_next;
std::size_t capacity;
std::size_t allocated;
};
It's not that weird: you've defined value_type (incorrectly) to be free_list<T> instead of T, so m_next has type free_list<free_list<T>>. pointer and const_pointer have the same issue.
Other issues:
pool_alloc doesn't have proper copy/move/assignment operators defined.
pool_alloc doesn't have a templated constructor that accepts const pool_alloc<U>&, so you won't be able to initialize
pool_alloc::allocate will do horrible things if anyone ever tries to make a std::vector<T, pool_alloc<T>>. You should probably either assert that the argument is 1 or fallback to ::operator new(n * sizeof(T)) if the argument is greater than 1.
operator== and operator!= should probably be const.
construct and destroy could also be const, or even static.
It's not necessary to keep both data and m_next, since they're different typed aliases for the same value - just cast m_next to void* whenever you need data.
Dumping the memory chunk in deallocate instead of in ~pool_alloc could have pathological performance for e.g std::stack<T, std::list<T, pool_alloc<T>>.
The "list" initialization in free_list<T>::init is pointing each node's m_next pointer at itself instead of building a list. Should be:
void init(std::size_t num_elements) {
for (std::size_t i = 0; i < num_elements - 1; ++i) {
this[i]->m_next = &this[i + 1];
}
this[num_elements - 1]->m_next = nullptr;
}

Fixed Allocator with stack - how to implement allocate and deallocate?

I've started to code FixedAllocator class that allocates memory by chunks of fixed size and works as stack, so that it works in constant time to allocate/deallocate. Actually, I'll need this class to use it with std::vector, so that I have to implement all std::allocator methods.
Everything here is for learning purposes so that - I don't need any complete implementations or headers - the real ones have a lot of code over my problem.
And I got stuck on allocate/deallocate methods - I understand that I should somehow reserve some memory pool - for example using vector, I understand that I should use static_cast to convert char type into T-type, but I don't completely understand how to rebuild this two ideas into list. Deallocate takes pointer as argument, not TNode - that's maybe the main problem.
If someone already wrote this kind of allocator - answer with code will be perfect.
Any suggestions, links and other source of knowledge are welcome. Thank you.
Here is the skeleton of code:
template <typename T, unsigned int nodeSize>
class FixedAllocator : public std::allocator<T>{
private:
static size_t Used;
static const size_t MAX_SIZE = 100000;
struct TNode {
TNode* next;
char data[nodeSize];
};
TNode* head;
public:
typedef T* pointer;
typedef const T* const_pointer;
typedef T & reference;
typedef const T & const_reference;
typedef T value_type;
template <typename U> struct rebind { typedef allocator<U> other; };
FixedAllocator() {
if (Pool.empty()) {
Pool.resize(MAX_SIZE * sizeof(T));
Used = 0;
}
}
FixedAllocator(const FixedAllocator &) {
}
template<typename U>
FixedAllocator(const FixedAllocator<U> &) {
if (Pool.empty()) {
Pool.resize(MAX_SIZE * sizeof(T));
Used = 0;
}
}
pointer address(reference x) const {
return &x;
}
const_pointer address(const_reference x) const {
return &x;
}
pointer allocate(size_t n, FixedAllocator<void>::const_pointer = 0) {}
void deallocate(pointer, size_t) {}
size_t max_size() const throw() {
return MAX_SIZE - size;
}
void construct(pointer p, const_reference val) {
new (static_cast<void*>(p)) value_type(val);
}
void destroy(pointer p) {
p->~value_type();
}
};

Why doesn't this C++ STL allocator allocate?

I'm trying to write a custom STL allocator that is derived from std::allocator, but somehow all calls to allocate() go to the base class. I have narrowed it down to this code:
template <typename T> class a : public std::allocator<T> {
public:
T* allocate(size_t n, const void* hint = 0) const {
cout << "yo!";
return 0;
}
};
int main()
{
vector<int, a<int>> v(1000, 42);
return 0;
}
I expect "Yo!" to get printed, followed by some horrible error because I don't actually allocate anything. Instead, the program runs fine and prints nothing. What am I doing wrong?
I get the same results in gcc and VS2008.
You will need to provide a rebind member template and the other stuff that is listed in the allocator requirements in the C++ Standard. For example, you need a template copy constructor which accepts not only allocator<T> but also allocator<U>. For example, one code might do, which a std::list for example is likely to do
template<typename Allocator>
void alloc1chunk(Allocator const& alloc) {
typename Allocator::template rebind<
wrapper<typename Allocator::value_type>
>::other ot(alloc);
// ...
}
The code will fail if there either exist no correct rebind template, or there exist no corresponding copy constructor. You will get nowhere useful with guessing what the requirements are. Sooner or later you will have to do with code that relies on one part of those allocator requirements, and the code will fail because your allocator violates them. I recommend you take a look at them in some working draft your your copy of the Standard in 20.1.5.
In this case, the problem is that I didn't override the rebind member of the allocator. This version works (in VS2008):
template <typename T> class a : public std::allocator<T> {
public:
T* allocate(size_t n, const void* hint = 0) const {
cout << "yo!";
return 0;
}
template <typename U> struct rebind
{
typedef a<U> other;
};
};
int main() {
vector<int, a<int>> v(1000, 42);
return 0;
}
I found this by debugging through the STL headers.
Whether this works or not will be completely dependent on the STL implementation though, so I think that ultimately, Klaim is right in that this shouldn't be done this way.
I have two templates for creating customized allocators; the first works automagically if it is used on a custom type:
template<>
class std::allocator<MY_TYPE>
{
public:
typedef size_t size_type;
typedef ptrdiff_t difference_type;
typedef MY_TYPE* pointer;
typedef const MY_TYPE* const_pointer;
typedef MY_TYPE& reference;
typedef const MY_TYPE& const_reference;
typedef MY_TYPE value_type;
template <class U>
struct rebind
{
typedef std::allocator<U> other;
};
pointer allocate(size_type n, std::allocator<void>::const_pointer hint = 0)
{
return reinterpret_cast<pointer>(ALLOC_FUNC(n * sizeof(T)));
}
void construct(pointer p, const_reference val)
{
::new(p) T(val);
}
void destroy(pointer p)
{
p->~T();
}
void deallocate(pointer p, size_type n)
{
FREE_FUNC(p);
}
size_type max_size() const throw()
{
// return ~size_type(0); -- Error, fixed according to Constantin's comment
return std::numeric_limits<size_t>::max()/sizeof(MY_TYPE);
}
};
The second is used when we want to have our own allocator for a predefined type with a standard allocator, for instance char, wchar_t, std::string, etc.:
namespace MY_NAMESPACE
{
template <class T> class allocator;
// specialize for void:
template <>
class allocator<void>
{
public:
typedef void* pointer;
typedef const void* const_pointer;
// reference to void members are impossible.
typedef void value_type;
template <class U>
struct rebind
{
typedef allocator<U> other;
};
};
template <class T>
class allocator
{
public:
typedef size_t size_type;
typedef ptrdiff_t difference_type;
typedef T* pointer;
typedef const T* const_pointer;
typedef T& reference;
typedef const T& const_reference;
typedef T value_type;
template <class U>
struct rebind
{
typedef allocator<U> other;
};
allocator() throw()
{
}
template <class U>
allocator(const allocator<U>& u) throw()
{
}
~allocator() throw()
{
}
pointer address(reference r) const
{
return &r;
}
const_pointer address(const_reference r) const
{
return &r;
}
size_type max_size() const throw()
{
// return ~size_type(0); -- Error, fixed according to Constantin's comment
return std::numeric_limits<size_t>::max()/sizeof(T);
}
pointer allocate(size_type n, allocator<void>::const_pointer hint = 0)
{
return reinterpret_cast<pointer>(ALLOC_FUNC(n * sizeof(T)));
}
void deallocate(pointer p, size_type n)
{
FREE_FUNC(p);
}
void construct(pointer p, const_reference val)
{
::new(p) T(val);
}
void destroy(pointer p)
{
p->~T();
}
};
template <class T1, class T2>
inline
bool operator==(const allocator<T1>& a1, const allocator<T2>& a2) throw()
{
return true;
}
template <class T1, class T2>
inline
bool operator!=(const allocator<T1>& a1, const allocator<T2>& a2) throw()
{
return false;
}
}
The first template above, for your own defined type, does not require any further handling but is used automatically by the standard container classes. The second template requires further work when used on a standard type. For std::string, for example, one have to use the following construct when declaring variables of that type (it is simplest with a typedef):
std::basic_string<char>, std::char_traits<char>, MY_NAMESPACE::allocator<char> >
The following code prints "yo" as expected - what you were seeing was our old friend "undefined behaviour".
#include <iostream>
#include <vector>
using namespace std;
template <typename T> class a : public std::allocator<T> {
public:
T* allocate(size_t n, const void* hint = 0) const {
cout << "yo!";
return new T[10000];
}
};
int main()
{
vector<int, a<int> > v(1000, 42);
return 0;
}
Edit: I just checked out the C++ Standard regarding the default allocator. There is no prohibition on inheriting from it. In fact, as far as I'm aware, there is no such prohibition in any part of the Standard.