I'm trying to write a custom allocator, which preallocates space for a fixed number of elements. However, I have some problems with understanding the requirements.
allocator.h
#pragma once
#ifndef _ALLOCATOR_H
#define _ALLOCATOR_H
template<typename T>
class Allocator
{
public:
// typedefs
typedef T value_type;
typedef value_type* pointer;
typedef const value_type* const_pointer;
typedef value_type& reference;
typedef const value_type& const_reference;
typedef std::size_t size_type;
typedef std::ptrdiff_t difference_type;
public:
// convert an allocator<T> to allocator<U>
template<typename U>
struct rebind
{
typedef Allocator<U> other;
};
public:
explicit Allocator(void)
{
mCurElement = 0;
mMaxElements = 650000000;
mBase = reinterpret_cast<pointer>(::operator new(mMaxElements * sizeof(T)));
}
virtual ~Allocator(void)
{
::operator delete(mBase);
}
explicit Allocator(Allocator const &oOther)
{
mCurElement = oOther.mCurElement;
mMaxElements = oOther.mMaxElements;
mBase = oOther.mBase;
}
template<typename U>
explicit Allocator(Allocator<U> const &oOther)
{
mCurElement = 0;
mMaxElements = 650000000;
mBase = oOther.mBase;
}
// address
pointer address(reference r) { return &r; }
const_pointer address(const_reference r) { return &r; }
// memory allocation
pointer allocate(size_type nElements, typename std::allocator<void>::const_pointer = 0)
{
if (mCurElement > mMaxElements)
return NULL;
//pointer p = reinterpret_cast<pointer>(::operator new(cnt * sizeof(T)));
pointer p = &mBase[mCurElement];
mCurElement += nElements;
return p;
}
void deallocate(pointer pAddress, size_type)
{
//::operator delete(pAddress);
mCurElement--;
}
// size
size_type max_size() const
{
return std::numeric_limits<size_type>::max() / sizeof(T);
}
// construction/destruction
void construct(pointer pAddress, const T& oObject)
{
new(pAddress) T(oObject);
}
void destroy(pointer pAddress)
{
pAddress->~T();
}
bool operator==(Allocator const&) { return true; }
bool operator!=(Allocator const& oAllocator) { return !operator==(oAllocator); }
public:
T *getBase(void) const { return mBase; }
private:
static usize_t mId;
T *mBase;
usize_t mMaxElements;
usize_t mCurElement;
};
#endif // _ALLOCATOR_H
allocator.cpp
#define _CRT_SECURE_NO_WARNINGS
#include <iostream>
#include <sstream>
#include <set>
#include <ctime>
#include "allocator.h"
typedef unsigned int uint_t;
typedef unsigned long long usize_t;
usize_t Allocator<usize_t>::mId;
void testStdAllocator(usize_t nIterations, usize_t nMaxValue)
{
std::set<usize_t, std::less<usize_t>, Allocator<usize_t>> st;
std::string id = "Standard Set";
clock_t start = clock();
for (usize_t i = 0; i < nIterations; i++)
{
usize_t val = (usize_t)(rand() % nMaxValue) + 1;
if (i % 1000000 == 0)
std::cout << id << " testing ... " << i << "/" << nIterations << "\r";
st.insert(val);
}
std::cout << id << " Elapsed: " << clock() - start << std::endl;
}
int main(int argc, char *argv[])
{
usize_t iterations = 650000000;
usize_t val = 6500000;
std::cout << "Allocator" << std::endl;
testStdAllocator(iterations, val);
return 0;
}
The problem I have is:
Why do I need the template <typename U> ...? (I found an example and adpated it)
When I made it compilable and tested it the std::set apparently creates copies of the allocator, so I would have to pass around the pointer. I can use an std::shared_ptr for that, but I don't really see why this should be needed in the first place.
Apparently there is something about proxied containers where the template <typename U> is needed for, but this again creates the additional problem of passing the pointer around for an (apparently) different allocator type.
So I would appreciate some pointers where I'm going wrong.
When you pass an allocator to std::set<T, C A> it is meant to have an allocate() function allcoating space for T objects. However, the std::set<T, C, A> will not allocate any T object. It will, instead, allocate _Node<T> objects where _Node is some tree node representation capable of holding T objects but also containing suitable pointers to other nodes.
To allocate an object of _Node<T> an allocator based on A is needed. This allocator's type is obtained from A::rebind<_Node<T>>::other and initialized appropriately by passing the original allocator object (or an object created from that) as constructor argument.
Of course, using stateful allocators does assume that you use the C++11 allocator model. Prior to C++11 allocators did not appropriately construct other allocators and they were essentially stateless. In case you need to use code prior to C++11 but want to deal with allocators, you might want to use the containers from BSL: these are allocator aware and do compile with C++03 compilers.
Related
I have a pointer-like struct that goes in the place of a pointer.
The difference with a pointer is that it has extra information that the (also special) allocator can use to deallocate the memory.
This pointer-like structure works well for all basic uses.
I can allocate and deallocate memory, dereferrence, increment,->, etc.
Now I want to use this pointers to be managed by a STL-like container.
Early on, I realized that STL vector basically cannot handle non-raw pointers.
T* is too hard coded, and the standard basically rules out anything that is not a pointer.
Inspired by Boost.Interprocess' offset_ptr<T> I decided to use Boost.Container vector, which is very customizable and in principle can manage anything, the allocator passed to the boost::container::vector can handle anything that is pointer-like.
Now the class boost::container::vector<T, myallocator_with_special_pointer<T>> can do anything... except resize()!!
Looking at the code in boost/container/vector.hpp it seems that the process of resizing (which is basically and allocation, followed by a copy (or move) and deallocation) involves raw pointers.
The offending line is:
[line 2729:] T * const new_buf = container_detail::to_raw_pointer
(allocator_traits_type::allocate(this->m_holder.alloc(), new_cap, this->m_holder.m_start));
Which is later followed by
[line 3022:] this->m_holder.start(new_start); // new_start is the same as new_buf above.
// member ::start(pointer&) will need to convert a raw pointer to the pointer typedef.
Both lines absolutely kill the possibility of using anything that is not a raw_pointer. Even if I have a conversion operator to a raw pointer, other information about the special pointer will be lost.
It seems pretty silly that this small detail forbids the use of non-raw pointers. Given all the effort for the container to be general (e.g. defining the pointer typedef), why this portion of the code uses T* just for resizing?
In other words, why Boost Container doesn't use this line instead
[alternative] pointer const new_buf =
allocator_traits_type::allocate(this->m_holder.alloc(), new_cap, this->m_holder.m_start);
Is there a workaround or an alternative way to use Boost Container vector to handle non-raw pointers?
Boost.Container says in its manual page http://www.boost.org/doc/libs/1_64_0/doc/html/container/history_and_reasons.html#container.history_and_reasons.Why_boost_container
Boost.Container is a product of a long development effort that started
in 2004 with the experimental Shmem library, which pioneered the use
of standard containers in shared memory. Shmem included modified SGI
STL container code tweaked to support non-raw allocator::pointer types
and stateful allocators. Once reviewed, Shmem was accepted as
Boost.Interprocess and this library continued to refine and improve
those containers.
The current implementation (in the context of resize) goes against this design goal.
I asked a less specific question here, about other traits of the allocators: Is it still possible to customize STL vector's "reference" type?
For reference the allocator that specifies the special pointer (which is propagated to the container) is something like this,
template<class T>
struct allocator{
using value_type = T;
using pointer = array_ptr<T>; // simulates T*
using const_pointer = array_ptr<T const>; // simulates T const*
using void_pointer = array_ptr<void>; // simulates void*
using const_void_pointer = array_ptr<void const>; // simulates void const*
some_managed_shared_memory& msm_;
allocator(some_managed_shared_memory& msm) : msm_(msm){}
array_ptr<T> allocate(mpi3::size_t n){
auto ret = msm_.allocate(n*sizeof(T));
return static_cast<array_ptr<T>>(ret);
}
void deallocate(array_ptr<T> ptr, mpi3::size_t = 0){
msm_.deallocate(ptr);
}
};
Full working code http://coliru.stacked-crooked.com/a/f43b6096f9464cbf
#include<iostream>
#include <boost/container/vector.hpp>
template<typename T>
struct array_ptr;
template<>
struct array_ptr<void> {
using T = void;
T* p;
int i; //some additional information
// T& operator*() const { return *p; }
T* operator->() const { return p; }
// operator T*() const { return p; }
template<class TT>
operator array_ptr<TT>() const{return array_ptr<TT>((TT*)p, i);}
operator bool() const{return p;}
array_ptr(){}
array_ptr(std::nullptr_t) : p(nullptr){}
array_ptr(T* ptr, int _i) : p(ptr), i(_i){}
template<class Other>
array_ptr(array_ptr<Other> other) : p(other.p), i(other.i){}
};
template<>
struct array_ptr<void const> {
using T = void const;
T* p;
int i; //some additional information
// T& operator*() const { return *p; }
T* operator->() const { return p; }
operator T*() const { return p; }
array_ptr(){}
array_ptr(std::nullptr_t) : p(nullptr){}
array_ptr(T* ptr, int _i) : p(ptr), i(_i){}
template<class Other>
array_ptr(array_ptr<Other> other) : p(other.p), i(other.i){}
};
template<typename T>
struct array_ptr {
T* p;
int i; //some additional information
T& operator*() const { return *p; }
T* operator->() const { return p; }
T& operator[](std::size_t n) const{
assert(i == 99);
return *(p + n);
}
bool operator==(array_ptr const& other) const{return p == other.p and i == other.i;}
bool operator!=(array_ptr const& other) const{return not((*this)==other);}
// operator T*() const { return p; }
array_ptr& operator++(){++p; return *this;}
array_ptr& operator+=(std::ptrdiff_t n){p+=n; return *this;}
array_ptr& operator-=(std::ptrdiff_t n){p-=n; return *this;}
array_ptr operator+(std::size_t n) const{array_ptr ret(*this); ret+=n; return ret;}
std::ptrdiff_t operator-(array_ptr const& other) const{return p - other.p;}
array_ptr(){}
array_ptr(std::nullptr_t) : p(nullptr), i(0){}
operator bool() const{return p;}
array_ptr(T* ptr, int _i) : p(ptr), i(_i){}
array_ptr(T* ptr) : p(ptr), i(0){}
array_ptr(int) : p(nullptr), i(0){}
array_ptr(array_ptr<void> const& other) : p(static_cast<T*>(other.p)), i(other.i){}
};
struct some_managed_shared_memory {
array_ptr<void> allocate(size_t n) { return array_ptr<void>(::malloc(n), 99); }
void deallocate(array_ptr<void> ptr) { if (ptr) ::free(ptr.p); }
};
template<typename T>
struct allocator{
using value_type = T;
using pointer = array_ptr<T>; // simulates T*
using const_pointer = array_ptr<T const>; // simulates T const*
using void_pointer = array_ptr<void>; // simulates void*
using const_void_pointer = array_ptr<void const>; // simulates void const*
some_managed_shared_memory& msm_;
allocator(some_managed_shared_memory& msm) : msm_(msm){}
array_ptr<T> allocate(size_t n){
auto ret = msm_.allocate(n*sizeof(T));
return static_cast<array_ptr<T>>(ret);
}
void deallocate(array_ptr<T> ptr, std::size_t = 0){
msm_.deallocate(ptr);
}
};
int main() {
some_managed_shared_memory realm;
boost::container::vector<int, allocator<int> > v(10, realm);
assert( v[4] == 0 );
v[4] = 1;
assert( v[4] == 1 );
for(std::size_t i = 0; i != v.size(); ++i) std::cout << v[i] << std::endl;
for(auto it = v.begin(); it != v.end(); ++it) std::cout << *it << std::endl;
// none of these compile:
v.push_back(8);
assert(v.size() == 11);
v.resize(100);
std::cout << v[89] << std::endl; // will fail an assert because the allocator information is lost
//v.assign({1,2,3,4,5});
}
I looked into things.
The TL;DR seems to be: non-raw pointers are supported, but they need a implicit conversion from raw in some operations. Whether or not this is by design, I don't know, but it doesn't seem to contradict the design goal.
In fact this is very analogous to the history of allocator support: STL containers had support for custom allocators, but not for stateful allocators (meaning, non-default-constructible allocator types).
Allocator Versions
At first I tried some of the allocator versions:
using version = boost::container::version_0; // seems unsupported, really
using version = boost::container::version_1;
using version = boost::container::version_2; // does different operations
But it had no (decisive) effect. Maybe the documentation has clues.
Pointer Arithmetic
After that I looked into the specific errors. Looking at the cited line/error it dawned on me that the raw-pointer might have been an accident. Looking at the output of these:
std::cout << boost::container::container_detail::impl::version<allocator<int> >::value << "\n";
array_ptr<int> p;
auto rawp = boost::container::container_detail::to_raw_pointer(p);
std::cout << typeid(rawp).name() << "\n";
std::cout << typeid(p).name() << "\n";
std::cout << typeid(p + 5).name() << "\n";
std::cout << typeid(p - 5).name() << "\n";
Shows something like¹
1
int*
array_ptr<int>
int*
int*
¹ prettified with the help of c++filt -t
This lead me to define pointer arithmetic:
template <typename T, typename N>
array_ptr<T> operator+(array_ptr<T> const& p, N n) { return array_ptr<T>(p.p+n, p.i); }
template <typename T>
array_ptr<T>& operator++(array_ptr<T>& p) { return ++p.p, p; }
template <typename T>
array_ptr<T> operator++(array_ptr<T>& p, int) { auto q = p.p++; return array_ptr<T>(q, p.i); }
template <typename T, typename N>
array_ptr<T> operator-(array_ptr<T> const& p, N n) { return array_ptr<T>(p.p-n, p.i); }
template <typename T>
ptrdiff_t operator-(array_ptr<T> const& a, array_ptr<T> const& b) { return a.p - b.p; }
Now the output becomes
1
int*
array_ptr<int>
array_ptr<int>
array_ptr<int>
Many more use cases compile successfully with these definitions. Assuming that the "annotation" data inside the array_pointer is valid after increment, it should not lose any allocator information
The Real Culprit
With that out of the way, some things still don't compile. Specifically, in some spots the allocator's pointer type is constructed back from a raw-pointer. This fails because there's no suitable "default" conversion constructor. If you declare the constructors with the data value optional, everything compiles, but you could argue that this loses information as there is a path from
array_pointer<T> p;
auto* rawp = to_raw_pointer(p);
array_pointer<T> clone(rawp); // oops lost the extra info in p
OBSERVATION
Note that, as you apparently realized (judging from the commented operators), adding the default constructor argument removes the need for the arithmetic operations (except pre-increment).
However, adding them makes sure that the lossy conversion path is taken less often, which could be important to your use case.
DEMO TIME
Live On Coliru
#if COMPILATION_INSTRUCTIONS
clang++ -std=c++14 -Wall -Wfatal-errors $0 -o $0x.x && $0x.x $# && rm -f $0x.x; exit
#endif
#define DEFAULT_DATA = 0
#define DEFINE_ARITHMETIC_OPERATIONS
#include <iostream>
#include <boost/container/vector.hpp>
#include <typeinfo>
template<typename T>
struct array_ptr {
T* p;
int i; //some additional information
T& operator*() const { return *p; }
T* operator->() const { return p; }
operator T*() const { return p; }
array_ptr(){}
//array_ptr(std::nullptr_t) : p(nullptr), i(0){}
array_ptr(T* ptr, int _i DEFAULT_DATA) : p(ptr), i(_i){}
};
template<>
struct array_ptr<void> {
using T = void;
T* p;
int i; //some additional information
// T& operator*() const { return *p; }
T* operator->() const { return p; }
operator T*() const { return p; }
template<class T>
operator array_ptr<T>() const{return array_ptr<T>((T*)p, i);}
// array_ptr& operator++(){++p; return *this;}
array_ptr(){}
array_ptr(std::nullptr_t) : p(nullptr){}
array_ptr(T* ptr, int _i DEFAULT_DATA) : p(ptr), i(_i){}
template<class Other>
array_ptr(array_ptr<Other> other) : p(other.p), i(other.i){}
};
template<>
struct array_ptr<void const> {
using T = void const;
T* p;
int i; //some additional information
// T& operator*() const { return *p; }
T* operator->() const { return p; }
operator T*() const { return p; }
// array_ptr& operator++(){++p; return *this;}
// template<class Other> array_ptr(array_ptr<Other> const& other) : p(other.p), i(other.i){}
array_ptr(){}
array_ptr(std::nullptr_t) : p(nullptr){}
array_ptr(T* ptr, int _i DEFAULT_DATA) : p(ptr), i(_i){}
template<class Other>
array_ptr(array_ptr<Other> other) : p(other.p), i(other.i){}
};
struct some_managed_shared_memory {
array_ptr<void> allocate(size_t n) { return array_ptr<void>(::malloc(n), 99); }
void deallocate(array_ptr<void> ptr) { if (ptr) ::free(ptr.p); }
};
template<typename T>
struct allocator{
using version = boost::container::version_1;
using value_type = T;
using pointer = array_ptr<T>; // simulates T*
using const_pointer = array_ptr<T const>; // simulates T const*
using void_pointer = array_ptr<void>; // simulates void*
using const_void_pointer = array_ptr<void const>; // simulates void const*
some_managed_shared_memory& msm_;
allocator(some_managed_shared_memory& msm) : msm_(msm){}
array_ptr<T> allocate(size_t n){
auto ret = msm_.allocate(n*sizeof(T));
return static_cast<array_ptr<T>>(ret);
}
void deallocate(array_ptr<T> ptr, std::size_t = 0){
msm_.deallocate(ptr);
}
};
#ifdef DEFINE_ARITHMETIC_OPERATIONS
template <typename T, typename N>
array_ptr<T> operator+(array_ptr<T> const& p, N n) { return array_ptr<T>(p.p+n, p.i); }
template <typename T>
array_ptr<T>& operator++(array_ptr<T>& p) { return ++p.p, p; }
template <typename T>
array_ptr<T> operator++(array_ptr<T>& p, int) { auto q = p.p++; return array_ptr<T>(q, p.i); }
template <typename T, typename N>
array_ptr<T> operator-(array_ptr<T> const& p, N n) { return array_ptr<T>(p.p-n, p.i); }
template <typename T>
ptrdiff_t operator-(array_ptr<T> const& a, array_ptr<T> const& b) { return a.p - b.p; }
#endif
int main() {
std::cout << boost::container::container_detail::impl::version<allocator<int> >::value << "\n";
if (1) { // some diagnostics
array_ptr<int> p;
auto rawp = boost::container::container_detail::to_raw_pointer(p);
std::cout << typeid(rawp).name() << "\n";
std::cout << typeid(p).name() << "\n";
std::cout << typeid(p + 5).name() << "\n";
std::cout << typeid(p - 5).name() << "\n";
}
some_managed_shared_memory realm;
boost::container::vector<int, allocator<int> > v(10, realm);
assert( v[4] == 0 );
v[4] = 1;
assert( v[4] == 1 );
for(std::size_t i = 0; i != v.size(); ++i) std::cout << v[i] << std::endl;
// these compile:
v.push_back(12);
v.resize(100);
v.assign({1,2,3,4,5});
}
Prints
1
Pi
9array_ptrIiE
9array_ptrIiE
9array_ptrIiE
0
0
0
0
1
0
0
0
0
0
I wrote a custom allocator. However, the Clang/LLVM compiler started complaining about mismatching a constructor for initialization of my allocator when I added list.sort() into main().
Though the code is a bit long, this is the minimum workable snippet:
#include <iostream>
#include <ctime>
#include <list>
#include <limits>
template<typename T, int start = 16, int ratio = 2, int thrsh = 65536>
class Allocator
{
private:
T *avsp;
int used, vcnt;
struct _block
{ struct _block *next;
T nodes[1];
} *pool, *pblock;
public :
// typedefs
typedef T value_type;
typedef value_type* pointer;
typedef const value_type* const_pointer;
typedef value_type& reference;
typedef const value_type& const_reference;
typedef std::size_t size_type;
typedef std::ptrdiff_t difference_type;
public :
// convert an allocator<T> to allocator<U>
template<typename U>
struct rebind
{
typedef Allocator<U, start, ratio, thrsh> other;
};
public :
explicit Allocator()
{
avsp = NULL;
used = 0;
vcnt = 0;
pool = NULL;
pblock = NULL;
}
~Allocator() {}
explicit Allocator(Allocator const&) {}
template<typename U>
explicit Allocator(Allocator<U, start, ratio, thrsh> const&) {}
// address
pointer address(reference r)
{
return &r;
}
const_pointer address(const_reference r)
{
return &r;
}
// memory allocation
pointer allocate(size_type cnt = 1, // SHOULD ALWAYS BE ONE
typename std::allocator<void>::const_pointer = 0)
{
(void)cnt;
if (avsp == NULL)
{
if (vcnt == 0)
{
pblock = pool;
pool = NULL;
if (used == 0)
vcnt = (used = start);
else
vcnt = (used < thrsh) ? (used *= ratio) : (used = thrsh);
if (pool != NULL)
std::cerr << "Potential Memory Leak." << std::endl; // Compatibility Purpose Only
pool = static_cast<struct _block*>(malloc((sizeof(*pblock)) +
(sizeof(pblock->nodes)) * (size_t)(used - 1)));
if (pool == NULL)
std::cerr << "Memory Allocation Failure." << std::endl; // Compatibility Purpose Only
pool->next = pblock;
}
return &(pool->nodes[--vcnt]);
}
else
{
// NOT IMPL: AVSP
exit(EXIT_FAILURE);
}
// NEVER REACH !!
exit(EXIT_FAILURE);
}
void deallocate(pointer p, size_type)
{
// NOT IMPL: AVSP
(void)p;
}
// size
size_type max_size() const
{
return std::numeric_limits<size_type>::max() / sizeof(T);
}
// construction/destruction
void construct(pointer p, const T& t)
{
new(p) T(t);
}
void destroy(pointer p)
{
p->~T();
}
template<typename U>
bool operator==(const Allocator<U, start, ratio, thrsh>&) const
{
return true;
}
template<typename U>
bool operator!=(const Allocator<U, start, ratio, thrsh>&) const
{
return false;
}
};
int main (void)
{
std::list<uint32_t, Allocator<uint32_t>> list;
for (int cnt = 0; cnt < 1 << 27; cnt++)
list.push_back(rand());
list.sort(); // <-- Problems Here
return 0;
}
This is the error message:
/usr/include/c++/4.6/bits/move.h:127
error: no matching constructor for initialization of 'Allocator<std::_List_node<unsigned int>, 16, 2, 65536>'
/usr/include/c++/4.6/bits/allocator.h:163:4:
in instantiation of function template specialization 'std::swap<Allocator<std::_List_node<unsigned int>, 16, 2, 65536> >' requested here
/usr/include/c++/4.6/bits/stl_list.h:1185:4:
in instantiation of member function 'std::__alloc_swap<Allocator<std::_List_node<unsigned int>, 16, 2, 65536>, false>::_S_do_it' requested here
/usr/include/c++/4.6/bits/list.tcc:375:11:
in instantiation of member function 'std::list<unsigned int, Allocator<unsigned int, 16, 2, 65536> >::swap' requested here
<this-file>.cpp:??:10:
in instantiation of member function 'std::list<unsigned int, Allocator<unsigned int, 16, 2, 65536> >::sort' requested here
Just simply remove explicit and everything works like a charm.
Types with explicit copy constructors do not meet the CopyConstructible requirements, and allocators must be CopyConstructible. This was clarified by https://wg21.link/lwg2081
So you need to remove the explicit from your copy constructor. To be safe (and ensure portability to all standard library implementations) you should probably also remove explicit from the converting constructor template.
The following code yields warning in G++:
#include <iostream>
#include <cstdint>
template <typename T, typename P, typename Q>
Q T::*pointer_to(P T::*p, Q P::*q)
{
typedef Q T::* output_ptr;
// warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
size_t tmp = reinterpret_cast<const size_t&>(p) + reinterpret_cast<const size_t&>(q);
return reinterpret_cast<const output_ptr&>(tmp);
}
struct A { int x; };
struct B { A a; };
int main()
{
B b = B();
b.*pointer_to(&B::a, &A::x) = 1;
std::cout << b.a.x << std::endl;
}
It works properly anyway, but that makes me worry.
What is your opinion, are these "sub-member" pointers susceptible to extra strict aliasing issues than plain member pointers?
I would recommend against doing it this way.
You stated in your comments that you tried using a nested std::bind, but there's an issue with the compiler version you're using. Rather than resort to the hack, I would roll my own repeated pointer to member class.
#include <iostream>
#include <cstdint>
#include <type_traits>
#include <utility>
template<typename Ptr1, typename... Rest>
class pointer_to_sub;
template<typename ObjType, typename Class>
class pointer_to_sub<ObjType Class::* >
{
typedef ObjType Class::* ptr_type;
public:
typedef ObjType value_type;
typedef Class input_type;
pointer_to_sub(ptr_type input) : ptr(input)
{
}
value_type& operator()(input_type& from) const
{
return from.*ptr;
}
value_type const& operator()(input_type const& from) const
{
return from.*ptr;
}
value_type& operator()(input_type* from) const
{
return from->*ptr;
}
value_type const& operator()(input_type const* from) const
{
return from->*ptr;
}
private:
ptr_type ptr;
};
template<typename ObjType, typename Class, typename... Rest >
class pointer_to_sub<ObjType Class::*, Rest...> : private pointer_to_sub<Rest...>
{
typedef ObjType Class::* ptr_type;
typedef pointer_to_sub<Rest...> base_type;
public:
typedef typename base_type::value_type value_type;
typedef Class input_type;
pointer_to_sub(ptr_type input, Rest... args) : base_type(args...), ptr(input)
{
}
value_type& operator()(input_type& from) const
{
return base_type::operator()(from.*ptr);
}
value_type const& operator()(input_type const& from) const
{
return base_type::operator()(from.*ptr);
}
value_type& operator()(input_type* from) const
{
return base_type::operator()(from->*ptr);
}
value_type const& operator()(input_type const* from) const
{
return base_type::operator()(from->*ptr);
}
private:
ptr_type ptr;
};
template<typename T, typename... Args>
pointer_to_sub<T, Args...> make_pointer_to_sub(T t1, Args... args)
{
return pointer_to_sub<T, Args...>(t1, args...);
}
The above basically provides a make_pointer_to_sub which takes a list of member object pointers. It accepts as its input a reference or a pointer that's convertible to the first type, and then dereferences each of the pointers in turn. It could be improved to accept unique_ptr or shared_ptr, but that's for later. You use it as seen below.
struct A { int x; double y;};
struct B { A a; };
int main()
{
auto ptr = make_pointer_to_sub(&B::a, &A::x);
B b = B();
ptr(b) = 1;
// b.*pointer_to(&B::a, &A::x) = 1;
std::cout << b.a.x << std::endl;
ptr(&b) = 2;
std::cout << b.a.x << std::endl;
}
If you needed to, this could be assigned to a std::function with the appropriate arguments.
std::list uses linked lists in its implementation, how large are each of the elements in the list (minus payload)?
By testing, using mingw (not mingw-64) on a windows 7 machine each element takes up 24 bytes for each element of an int.
A pointer to the left and a pointer to the right is only 4+4=8 bytes though! and an int is only 4 bytes (as determined by sizeof(void*) and sizeof(int)), so I'm curious, wheres the extra space going?
(test involved making many elements, seeing the size of the program, making more elements and again seeing the size of the program, taking the difference)
When having memory question about STL containers... remember that all memory they obtain come from the allocator you pass (which defaults to std::allocator).
So, just instrumenting the allocator shall answer most questions. The live demo is at liveworkspace, the output is presented here for std::list<int, MyAllocator>:
allocation of 1 elements of 24 bytes each at 0x1bfe0c0
deallocation of 1 elements of 24 bytes each at 0x1bfe0c0
So, 24 bytes in this case, which on a 64 bits platform is to be expected: two pointers for next and previous, the 4 bytes payload and 4 bytes of padding.
The full code listing is:
#include <iostream>
#include <limits>
#include <list>
#include <memory>
template <typename T>
struct MyAllocator {
typedef T value_type;
typedef T* pointer;
typedef T& reference;
typedef T const* const_pointer;
typedef T const& const_reference;
typedef std::size_t size_type;
typedef std::ptrdiff_t difference_type;
template <typename U>
struct rebind {
typedef MyAllocator<U> other;
};
MyAllocator() = default;
MyAllocator(MyAllocator&&) = default;
MyAllocator(MyAllocator const&) = default;
MyAllocator& operator=(MyAllocator&&) = default;
MyAllocator& operator=(MyAllocator const&) = default;
template <typename U>
MyAllocator(MyAllocator<U> const&) {}
pointer address(reference x) const { return &x; }
const_pointer address(const_reference x) const { return &x; }
pointer allocate(size_type n, void const* = 0) {
pointer p = reinterpret_cast<pointer>(malloc(n * sizeof(value_type)));
std::cout << "allocation of " << n << " elements of " << sizeof(value_type) << " bytes each at " << (void const*)p << "\n";
return p;
}
void deallocate(pointer p, size_type n) {
std::cout << "deallocation of " <<n << " elements of " << sizeof(value_type) << " bytes each at " << (void const*)p << "\n";
free(p);
}
size_type max_size() const throw() { return std::numeric_limits<size_type>::max() / sizeof(value_type); }
template <typename U, typename... Args>
void construct(U* p, Args&&... args) { ::new ((void*)p) U (std::forward<Args>(args)...); }
template <typename U>
void destroy(U* p) { p->~U(); }
};
template <typename T>
using MyList = std::list<T, MyAllocator<T>>;
int main() {
MyList<int> l;
l.push_back(1);
}
In the code below is the function make_vector(). It creates a vector and returns it to the caller. I want to be able to specify an allocator for the vector to use, but use the default std::allocator by default. This is because under some circumstances the default allocator is all I need, but other times I need to allocate from some pre-defined memory pools.
The closest I've come is the make_vector2() function template. It works with the std::allocator, but I don't know how to pass the 'arena' argument into my custom allocator.
Hopefull this working c++11 example will explain it better:
#include <malloc.h>
#include <cinttypes>
#include <cstddef>
#include <iostream>
#include <limits>
#include <stdexcept>
#include <vector>
namespace mem
{
// Memory arena to allocate from.
enum class ARENA
{
TEXTURES,
FONTS,
SCRIPTS
};
// Allocate block from specific arena.
void *malloc( const std::size_t size, const ARENA arena )
{
return std::malloc( size /*, arena */ );
}
// Free block from specific arena.
void free( void *ptr, const ARENA arena )
{
std::free( ptr /*, arena */ );
}
// The allocator - forward declaration.
// Not derived from std::allocator - should it be?
// Based on code from here:
// http://drdobbs.com/184403759?pgno=2
template<typename T> class allocator;
// Specialised for void.
template<> class allocator<void>
{
public:
typedef std::size_t size_type;
typedef ptrdiff_t difference_type;
typedef void* pointer;
typedef const void* const_pointer;
typedef void value_type;
template<typename U> struct rebind
{
typedef allocator<U> other;
};
};
template<typename T> class allocator
{
public:
typedef std::size_t size_type;
typedef std::ptrdiff_t difference_type;
typedef T* pointer;
typedef const T* const_pointer;
typedef T& reference;
typedef const T& const_reference;
typedef T value_type;
template<typename U> struct rebind
{
typedef allocator<U> other;
};
allocator( ARENA arena ) noexcept :
arena_( arena )
{}
~allocator() noexcept
{}
pointer address( reference x ) const
{
return &x;
}
const_pointer address( const_reference x ) const
{
return &x;
}
pointer allocate( size_type n, allocator<void>::const_pointer hint = 0 )
{
void *p = mem::malloc( n * sizeof( T ), arena_ );
if ( p == nullptr )
{
throw std::bad_alloc();
}
return static_cast<pointer>( p );
}
void deallocate( pointer p, size_type n )
{
mem::free( p, arena_ );
}
size_type max_size() const noexcept
{
return std::numeric_limits<std::size_t>::max() / sizeof( T );
}
void construct( pointer p, const T& val )
{
new (p) T(val);
}
void destroy( pointer p )
{
p->~T();
}
allocator( const allocator& src ) noexcept
{
arena_ = src.arena_;
}
ARENA arena_;
};
} // namespace mem
template<class T1, class T2> bool operator==( const mem::allocator<T1> &alloc1, const mem::allocator<T2> &alloc2 ) noexcept
{
return alloc1.arena_ == alloc2.arena_;
}
template<class T1, class T2> bool operator!=( const mem::allocator<T1> &alloc1, const mem::allocator<T2> &alloc2 ) noexcept
{
if alloc1.arena_ != alloc2.arena_;
}
// How do I allow the custom allocator to be passed? Function parameter? Template?
std::vector<uint8_t> make_vector()
{
std::vector<uint8_t> vec;
// Do stuff with the vector
return vec;
}
// This template function seems to work with std::allocator
template< typename T > std::vector<uint8_t,T> make_vector2()
{
std::vector<uint8_t,T> vec;
// Do stuff with the vector.
return vec;
}
int main( int argc, char **argv )
{
// vec1 - Allocates from TEXTURES arena
// See the C++11 FAQ by Bjarne Stroustrup here:
// http://www2.research.att.com/~bs/C++0xFAQ.html#scoped-allocator
std::vector<uint8_t, mem::allocator<uint8_t>> vec1( mem::allocator<uint8_t>{mem::ARENA::TEXTURES} );
// vec2 - Make the vector using the default allocator.
auto vec2 = make_vector2< std::allocator<uint8_t> >();
return 0;
}
In main() vec1 is created to use the TEXTURES arena for allocation. The arena to use is passed into the constructor of the allocator. Vec2 is created by the make_vector2() templated function and uses the std::allocator.
Q: How can I define the make_vector() function so it can create a vector that uses the std::allocator or the custom pool allocator above?
In C++11, function templates can have default template arguments:
template<class T, class Alloc = std::allocator<T>>
std::vector<T, Alloc> make_vector(Alloc const& al = Alloc()){
std::vector<T, Alloc> v(al);
// ...
return v;
}
Live example on Ideone.
In C++03 (or with compilers that don't support that feature), it's a bit more cumbersome, but you can overload based on the template parameters:
template<class T>
std::vector<T> make_vector(){
std::vector<T> v;
// ...
return v;
}
template<class T, class Alloc>
std::vector<T, Alloc> make_vector(Alloc const& al = Alloc()){
std::vector<T, Alloc> v(al);
// ...
return v;
}
Live example on Ideone.
You can provide two different function overloads:
template<typename T, typename Alloc>
std::vector<T, Alloc> func(Alloc a)
{
return std::vector<T, Alloc>();
}
template<typename T>
std::vector<T, std::allocator<T> > func()
{
return func<T>(std::allocator<T>());
}
The problem is that the type of allocator is part of the type of the vector, so you are basically asking for a function whose return value type depends on an argument. You can do this by returning a boost::variant, but I'm not sure this is a good idea. The client code still needs to know the type in order to use the vector.