malloc/free based STL allocator - c++

Is there a malloc/free based allocator in the STL? If not, does anyone know of a simple copy/paste one? I need it for a map that must not call new/delete.

First, I'd note that changing the allocator for the map itself won't change the allocation used by the objects stored in the map. For example, if you do something like:
std::map<std::string, int, my_allocator<std::pair<const std::string, int> > m;
The map itself will allocate memory using the specified allocator, but when the std::strings in the map allocate memory, they'll still use the default allocator (which will use new and delete. So, if you need to avoid new and delete in general, you have to ensure that not only the map itself uses the right allocator, but that any objects it stores do the same (I know that's probably stating the obvious, but I've overlooked it, so maybe it's worth mentioning).
With that proviso, on with the code:
#ifndef ALLOCATOR_H_INC_
#define ALLOCATOR_H_INC_
#include <stdlib.h>
#include <new>
#include <limits>
namespace JVC {
template <class T>
struct allocator {
typedef size_t size_type;
typedef ptrdiff_t difference_type;
typedef T* pointer;
typedef const T* const_pointer;
typedef T& reference;
typedef const T& const_reference;
typedef T value_type;
template <class U> struct rebind { typedef allocator<U> other; };
allocator() throw() {}
allocator(const allocator&) throw() {}
template <class U> allocator(const allocator<U>&) throw(){}
~allocator() throw() {}
pointer address(reference x) const { return &x; }
const_pointer address(const_reference x) const { return &x; }
pointer allocate(size_type s, void const * = 0) {
if (0 == s)
return NULL;
pointer temp = (pointer)malloc(s * sizeof(T));
if (temp == NULL)
throw std::bad_alloc();
return temp;
}
void deallocate(pointer p, size_type) {
free(p);
}
size_type max_size() const throw() {
return std::numeric_limits<size_t>::max() / sizeof(T);
}
void construct(pointer p, const T& val) {
new((void *)p) T(val);
}
void destroy(pointer p) {
p->~T();
}
};
}
#endif
And, a little test code:
#include <map>
#include <vector>
#include <iostream>
#include <string>
#include <iterator>
#include "allocator.h"
// Technically this isn't allowed, but it's only demo code, so we'll live with it.
namespace std {
std::ostream &operator<<(std::ostream &os, std::pair<std::string, int> const &c) {
return os << c.first << ": " << c.second;
}
}
int main() {
std::map<std::string, int, std::less<std::string>,
JVC::allocator<std::pair<const std::string, int> > > stuff;
stuff["string 1"] = 1;
stuff["string 2"] = 2;
stuff["string 3"] = 3;
std::copy(stuff.begin(), stuff.end(),
std::ostream_iterator<std::pair<std::string, int> >(std::cout, "\n"));
return 0;
}

Indeed, as #MichaelBurr suggests, Stephen J, Lavavej's 'mallocator' is what you're looking for. I just got updated and prettied-up code for it in this answer by #Arnaud today, do have a look.

Related

Replacing global operator new / delete and allocating memory within these, leading to heap corruption

#include <cstdlib>
#include <memory>
#include <unordered_map>
template <class T>
struct allocator {
typedef size_t size_type;
typedef ptrdiff_t difference_type;
typedef T* pointer;
typedef const T* const_pointer;
typedef T& reference;
typedef const T& const_reference;
typedef T value_type;
allocator() = default;
template <class U>
allocator(const allocator<U>&) {}
T* allocate(std::size_t n) const { return (T*)malloc(n); } // debugger breaks here
void deallocate(T* p, std::size_t) const { free(p); }
};
using allocations_map =
std::unordered_map<void*, std::size_t, std::hash<void*>,
std::equal_to<void*>,
allocator<std::pair<void* const, std::size_t>>>;
allocations_map allocations; // heap corruption in the constructor
void* operator new(std::size_t n) {
auto p = malloc(n);
allocations.emplace(p, n);
return p;
}
void operator delete(void* p) noexcept {
allocations.erase(p);
free(p);
}
int main() { std::vector<int> v(5); }
Why do i corrupt the heap in the constructor of allocations_map? The debugger detects the first heap corruption in a malloc call of the custom allocator, called inside the constructor.
Is there a more elegant solution then to write a non-logging custom allocator for allocations_map? The container shall obviously not log its own allocations.
I also tried two singleton approaches, as suggested in the comments, without success:
allocations_map& get_allocations_map()
{
static allocations_map* allocations_ptr = nullptr;
if (allocations_ptr == nullptr)
{
allocations_ptr = (allocations_map*) malloc(sizeof(allocations_map));
allocations_ptr = new(allocations_ptr)allocations_map;
}
return *allocations_ptr;
}
allocations_map& get_allocations_map()
{
static allocations_map allocations;
return allocations;
}
From std::allocator::allocate allocator allocates n "things" not n bytes. You should change:
T* allocate(std::size_t n) const { return (T*)malloc(n); }
to:
T* allocate(std::size_t n) const { return (T*)malloc(sizeof(T) * n); }
Why do i corrupt the heap in the constructor of allocations_map?
Because the constructor of elements stored in that map access allocated memory out-of-bounds.

custom STL allocator causes first character to be dropped

Per suggestion from #BenVoigt in response to my question regarding stack allocated stringstream storage, I designed a stack_allocator (code follows below), and declared a basic_ostringstream type using it.
I am experiencing a strange bug though.
The first character I place into the stream is omitted when I print the resulting string!
Here is an example:
template<typename T, size_t capacity, size_t arr_size>
__thread bool stack_allocator<T, capacity, arr_size>::_used[arr_size] = {};
template<typename T, size_t capacity, size_t arr_size>
__thread T stack_allocator<T, capacity, arr_size>::_buf[capacity][arr_size] = {};
typedef std::basic_ostringstream<char,
std::char_traits<char>,
stack_allocator<char, 1024, 5> > stack_ostringstream;
int main()
{
stack_ostringstream _os;
_os << "hello world";
std::cout << _os.str() << std::endl;
return 0;
}
The resulting output is:
ello world
Can anyone elaborate on what is happening to the first character?
The stack_allocator impl follows: It's pretty simplistic, and I'm sure has lots of room for improvement (not withstanding fixing the bug!)
#include <cstddef>
#include <limits>
#include <bits/allocator.h>
template<typename T, size_t capacity = 1024, size_t arr_size = 5>
class stack_allocator
{
public:
typedef T value_type;
typedef value_type* pointer;
typedef const value_type* const_pointer;
typedef value_type& reference;
typedef const value_type& const_reference;
typedef std::size_t size_type;
typedef std::ptrdiff_t difference_type;
inline explicit stack_allocator() { }
template<typename U>
inline explicit stack_allocator(const stack_allocator<U, capacity, arr_size>& that) { }
inline ~stack_allocator() {}
template<typename U>
struct rebind
{
typedef stack_allocator<U, capacity, arr_size> other;
};
inline pointer allocate(size_type cnt, typename std::allocator<void>::const_pointer = 0)
{
if (cnt > capacity)
return reinterpret_cast<pointer>(::operator new(cnt * sizeof (T)));
for (size_t i = 0; i < arr_size; ++i)
{
if (!_used[i])
{
_used[i] = true;
return reinterpret_cast<pointer>(_buf[i]);
}
}
}
inline void deallocate(pointer p, size_type)
{
for (size_t i = 0; i < arr_size; ++i)
{
if (p != _buf[i])
continue;
_used[i] = false;
return;
}
::operator delete(p);
}
inline pointer address(reference r) { return &r; }
inline const_pointer address(const_reference r) { return &r; }
inline size_type max_size() const
{
return std::numeric_limits<size_type>::max() / sizeof(T);
}
inline void construct(pointer p, const T& t) { new(p) T(t); }
inline void destroy(pointer p) { p->~T(); }
inline bool operator==(const stack_allocator&) const { return true; }
inline bool operator!=(const stack_allocator& a) const { return !operator==(a); }
private:
static __thread bool _used[arr_size];
static __thread T _buf[capacity][arr_size];
};
Your allocate function can fall off the end if you allocate more than arr_size items. If you use g++ -Wall it will warn you about those sorts of things.
The other problem is that your _buf array indexes are backwards. It should be static T _buf[arr_size][capacity]; which has the arr_size as the row, not the other order that you have it in the original code which makes the capacity be the first index.
Also as a side note, just avoid identifiers that start with leading _ because some such identifiers are reserved for the implementation and it's easier to never use them than to remember the precise rules. Finally, never include the bits/ headers directly, just use the real headers. In this case, memory. I also had to add includes for <iostream> and <sstream> to get it to compile.

Container that doesn't require its elements to be default and copy constructible

I'm looking for a C++ container-like class that wraps a typed array of objects that are not necessarily initialized and don't have to be default-constructible or copy-constructible. This would be interesting for RAII objects that have no well-defined copy semantics. Such a container-like class seems to be fairly easy to write (using an allocator to allocate uninitialized memory and placement new). Is there something like this in Boost that I have just overlooked? I'm not looking for std::vector (which requires its elements to be copy-constructible) or a pointer container, but for something like this:
#include <cstddef>
#include <memory>
#include <vector>
#include <algorithm>
#include <iostream>
template< typename T, typename Alloc = std::allocator<T> >
class FixedVector {
public:
typedef typename Alloc::value_type value_type;
typedef typename Alloc::pointer pointer;
typedef typename Alloc::reference reference;
typedef typename Alloc::const_pointer const_pointer;
typedef typename Alloc::const_reference const_reference;
typedef typename Alloc::size_type size_type;
typedef typename Alloc::difference_type difference_type;
typedef pointer iterator;
typedef const_pointer const_iterator;
explicit FixedVector(size_type size, const Alloc& allocator = Alloc()):
m_alloc(allocator),
m_size(size),
m_data(m_alloc.allocate(size)),
m_constructed(size) { }
FixedVector(const FixedVector& other):
m_alloc(other.m_alloc),
m_size(other.m_size),
m_data(m_alloc.allocate(m_size)),
m_constructed(other.m_constructed) {
for (size_type i = 0; i != m_size; ++i) {
if (m_constructed[i]) m_alloc.construct(m_alloc.address(m_data[i]), other[i]);
}
}
~FixedVector() {
for (size_type i = 0; i != m_size; ++i) {
if (m_constructed[i]) m_alloc.destroy(m_alloc.address(m_data[i]));
}
m_alloc.deallocate(m_data, m_size);
}
FixedVector& operator=(FixedVector other) {
other.swap(*this);
return *this;
}
// operator[] and other unimportant stuff
void swap(FixedVector& other) {
std::swap(m_alloc, other.m_alloc);
std::swap(m_size, other.m_size);
std::swap(m_data, other.m_data);
std::swap(m_constructed, other.m_constructed);
}
void construct(size_type index) {
new (m_alloc.address(m_data[index])) T();
m_constructed[index] = true;
}
template<typename U>
void construct(size_type index, U& val) {
new (m_alloc.address(m_data[index])) T(val);
m_constructed[index] = true;
}
template<typename U>
void construct(size_type index, const U& val) {
new (m_alloc.address(m_data[index])) T(val);
m_constructed[index] = true;
}
private:
Alloc m_alloc;
size_type m_size;
pointer m_data;
std::vector<bool> m_constructed;
};
template<typename T, typename Alloc>
void swap(FixedVector<T, Alloc>& first, FixedVector<T, Alloc>& second) {
first.swap(second);
}
namespace std {
template<typename T, typename Alloc>
void swap(FixedVector<T, Alloc>& first, FixedVector<T, Alloc>& second) {
first.swap(second);
}
}
class Test {
public:
explicit Test(int val): m_val(val) {
std::cout << "Test::Test(" << val << ')' << std::endl;
}
~Test() {
std::cout << "Test::~Test() [with m_val = " << m_val << ']' << std::endl;
}
int val() const {
return m_val;
}
private:
int m_val;
Test(const Test&);
Test& operator=(const Test&);
};
template<typename Char, typename Traits>
std::basic_ostream<Char, Traits>& operator<<(std::basic_ostream<Char, Traits>& stream, const Test& object) {
return stream << object.val();
}
int main() {
typedef FixedVector<Test> FVT;
FVT w(10);
w.construct(7, 7);
w.construct(2, 2);
std::cout << "w[2] = " << w[2] << std::endl;
}
The solution should work in C++03 (e.g. no move semantics allowed). The question is a bit academical—I'm just wondering why such a class doesn't seem to exist in Boost.
Such a container-like class seems to
be fairly easy to write (using an
allocator to allocate uninitialized
memory and placement new).
And that is exactly what std::vector does. To use placement new, you would have to make a copy.
void store(const T& value)
{
new (storage) T(value); //<-- invokes copy constructor
}
Perhaps boost::ptr_vector would work for non-copyable types (you'd give it pointers).
#include <boost/noncopyable.hpp>
#include <boost/ptr_container/ptr_vector.hpp>
#include <iostream>
struct X: boost::noncopyable
{
X(int x): x(x) {}
int x;
};
int main()
{
boost::ptr_vector<X> vec;
for (int i = 1; i < 10; ++i) {
vec.push_back(new X(i));
}
for (size_t i = 0; i != vec.size(); ++i) {
std::cout << vec[i].x << '\n';
}
}
And in C++0x, containers will accept non-copyable types as long as they are movable (which should normally be implementable for non-copyable types).
In C++0x, the elements of a std::vector don't have to be copy-constructible as long as they're movable.

Why doesn't this C++ STL allocator allocate?

I'm trying to write a custom STL allocator that is derived from std::allocator, but somehow all calls to allocate() go to the base class. I have narrowed it down to this code:
template <typename T> class a : public std::allocator<T> {
public:
T* allocate(size_t n, const void* hint = 0) const {
cout << "yo!";
return 0;
}
};
int main()
{
vector<int, a<int>> v(1000, 42);
return 0;
}
I expect "Yo!" to get printed, followed by some horrible error because I don't actually allocate anything. Instead, the program runs fine and prints nothing. What am I doing wrong?
I get the same results in gcc and VS2008.
You will need to provide a rebind member template and the other stuff that is listed in the allocator requirements in the C++ Standard. For example, you need a template copy constructor which accepts not only allocator<T> but also allocator<U>. For example, one code might do, which a std::list for example is likely to do
template<typename Allocator>
void alloc1chunk(Allocator const& alloc) {
typename Allocator::template rebind<
wrapper<typename Allocator::value_type>
>::other ot(alloc);
// ...
}
The code will fail if there either exist no correct rebind template, or there exist no corresponding copy constructor. You will get nowhere useful with guessing what the requirements are. Sooner or later you will have to do with code that relies on one part of those allocator requirements, and the code will fail because your allocator violates them. I recommend you take a look at them in some working draft your your copy of the Standard in 20.1.5.
In this case, the problem is that I didn't override the rebind member of the allocator. This version works (in VS2008):
template <typename T> class a : public std::allocator<T> {
public:
T* allocate(size_t n, const void* hint = 0) const {
cout << "yo!";
return 0;
}
template <typename U> struct rebind
{
typedef a<U> other;
};
};
int main() {
vector<int, a<int>> v(1000, 42);
return 0;
}
I found this by debugging through the STL headers.
Whether this works or not will be completely dependent on the STL implementation though, so I think that ultimately, Klaim is right in that this shouldn't be done this way.
I have two templates for creating customized allocators; the first works automagically if it is used on a custom type:
template<>
class std::allocator<MY_TYPE>
{
public:
typedef size_t size_type;
typedef ptrdiff_t difference_type;
typedef MY_TYPE* pointer;
typedef const MY_TYPE* const_pointer;
typedef MY_TYPE& reference;
typedef const MY_TYPE& const_reference;
typedef MY_TYPE value_type;
template <class U>
struct rebind
{
typedef std::allocator<U> other;
};
pointer allocate(size_type n, std::allocator<void>::const_pointer hint = 0)
{
return reinterpret_cast<pointer>(ALLOC_FUNC(n * sizeof(T)));
}
void construct(pointer p, const_reference val)
{
::new(p) T(val);
}
void destroy(pointer p)
{
p->~T();
}
void deallocate(pointer p, size_type n)
{
FREE_FUNC(p);
}
size_type max_size() const throw()
{
// return ~size_type(0); -- Error, fixed according to Constantin's comment
return std::numeric_limits<size_t>::max()/sizeof(MY_TYPE);
}
};
The second is used when we want to have our own allocator for a predefined type with a standard allocator, for instance char, wchar_t, std::string, etc.:
namespace MY_NAMESPACE
{
template <class T> class allocator;
// specialize for void:
template <>
class allocator<void>
{
public:
typedef void* pointer;
typedef const void* const_pointer;
// reference to void members are impossible.
typedef void value_type;
template <class U>
struct rebind
{
typedef allocator<U> other;
};
};
template <class T>
class allocator
{
public:
typedef size_t size_type;
typedef ptrdiff_t difference_type;
typedef T* pointer;
typedef const T* const_pointer;
typedef T& reference;
typedef const T& const_reference;
typedef T value_type;
template <class U>
struct rebind
{
typedef allocator<U> other;
};
allocator() throw()
{
}
template <class U>
allocator(const allocator<U>& u) throw()
{
}
~allocator() throw()
{
}
pointer address(reference r) const
{
return &r;
}
const_pointer address(const_reference r) const
{
return &r;
}
size_type max_size() const throw()
{
// return ~size_type(0); -- Error, fixed according to Constantin's comment
return std::numeric_limits<size_t>::max()/sizeof(T);
}
pointer allocate(size_type n, allocator<void>::const_pointer hint = 0)
{
return reinterpret_cast<pointer>(ALLOC_FUNC(n * sizeof(T)));
}
void deallocate(pointer p, size_type n)
{
FREE_FUNC(p);
}
void construct(pointer p, const_reference val)
{
::new(p) T(val);
}
void destroy(pointer p)
{
p->~T();
}
};
template <class T1, class T2>
inline
bool operator==(const allocator<T1>& a1, const allocator<T2>& a2) throw()
{
return true;
}
template <class T1, class T2>
inline
bool operator!=(const allocator<T1>& a1, const allocator<T2>& a2) throw()
{
return false;
}
}
The first template above, for your own defined type, does not require any further handling but is used automatically by the standard container classes. The second template requires further work when used on a standard type. For std::string, for example, one have to use the following construct when declaring variables of that type (it is simplest with a typedef):
std::basic_string<char>, std::char_traits<char>, MY_NAMESPACE::allocator<char> >
The following code prints "yo" as expected - what you were seeing was our old friend "undefined behaviour".
#include <iostream>
#include <vector>
using namespace std;
template <typename T> class a : public std::allocator<T> {
public:
T* allocate(size_t n, const void* hint = 0) const {
cout << "yo!";
return new T[10000];
}
};
int main()
{
vector<int, a<int> > v(1000, 42);
return 0;
}
Edit: I just checked out the C++ Standard regarding the default allocator. There is no prohibition on inheriting from it. In fact, as far as I'm aware, there is no such prohibition in any part of the Standard.

Is there a dereference_iterator in the STL?

I was wondering if there is an iterator in the STL that dereferences the object pointed before returning it. This could be very useful when manipulating containers aggregating pointers. Here's an example of what I would like to be able to do:
#include <vector>
#include <iterator>
#include <algorithm>
using namespace std;
int main()
{
vector<int*> vec;
int i = 1;
int j = 2;
int k = 3;
vec.push_back(&i);
vec.push_back(&j);
vec.push_back(&k);
copy(deref_iterator(vec.begin()),
deref_iterator(vec.end()),
ostream_iterator<int>(cout, " ")); // prints "1 2 3"
return 0;
}
Try Boost's indirect_iterator.
An indirect_iterator has the same category as the iterator it is wrapping. For example, an indirect_iterator<int**> is a random access iterator.
Assuming your actual use case is a bit more complex than a container of integer pointers!
You could check out the boost ptr containers
http://www.boost.org/doc/libs/1_35_0/libs/ptr_container/doc/reference.html
The containers contain dynamically allocated objects (ie pointers).
But all access to the objects (direct or via iterator) returns a reference to the object.
#include <boost/ptr_container/ptr_vector.hpp>
#include <iostream>
#include <iterator>
#include <algorithm>
using namespace std;
int main()
{
boost::ptr_vector<int> vec;
vec.push_back(new int(1));
vec.push_back(new int(2));
vec.push_back(new int(3));
copy(vec.begin(),vec.end(),
ostream_iterator<int>(std::cout, " ")); // prints "1 2 3 "
return 0;
}
If it is impossible using Boost, writing a custom iterator is not that hard. Here is an example of a "dereference iterator" that meets the InputIterator requirements :
#include <iterator>
template <typename T>
struct PointedType;
template <typename T>
struct PointedType<T*>
{
typedef T value_type;
};
template <typename InputIterator>
struct DerefIterator
{
typedef input_iterator_tag iterator_category;
typedef typename PointedType<
typename iterator_traits<InputIterator>::value_type>::value_type
value_type;
typedef typename iterator_traits<InputIterator>::difference_type
difference_type;
typedef value_type* pointer;
typedef value_type& reference;
public:
explicit DerefIterator(const InputIterator& ii)
: it(ii) {}
// Returns the object pointed by the object referenced by it
reference operator*() const { return **it; }
pointer operator->() const { return *it; }
DerefIterator& operator++()
{
++it;
return *this;
}
DerefIterator operator++(int)
{
DerefIterator tmp = *this;
++it;
return tmp;
}
bool equals(const DerefIterator<InputIterator> & di) const
{
return di.it == it;
}
private:
InputIterator it;
};
// Equality functions
template <typename InputIterator>
inline bool operator==(const DerefIterator<InputIterator>& di1,
const DerefIterator<InputIterator>& di2)
{
return di1.equals(di2);
}
template <typename InputIterator>
inline bool operator!=(const DerefIterator<InputIterator>& di1,
const DerefIterator<InputIterator>& di2)
{
return ! (di1 == di2);
}
//Helper function
template <typename InputIterator>
DerefIterator<InputIterator> deref_iterator(const InputIterator& ii)
{
return DerefIterator<InputIterator>(ii);
}