Is this simple implementation of an allocator acceptable
template<typename T>
class Allocator
{
public:
T * allocate(int n); //allocate space for n objects of type T
void deallocate(T* p, int n); //deallocate n objects of type T starting at p
void construct(T* p, const T& v); //construct a T with the value v in p
void destroy(T* p); //destroy the T in p
};
template<typename T>
T* Allocator<T>::allocate(int n)
{
T* new_mem = (T*)malloc(n * sizeof(T));
return new_mem;
}
template<typename T>
void Allocator<T>::construct(T* p, const T& v)
{
T* constructed_object = new(p) T{ v };
}
template<typename T>
void Allocator<T>::deallocate(T* p, int n)
{
for (int i = 0; i < n; ++i)
{
free(&p[i]);
}
}
template<typename T>
void Allocator<T>::destroy(T* p)
{
p->~T();
}
I'm to use it in a vector to implement function that reserves exra space as follows:
template<typename T, typename A>
void vector<T, A>::reserve(int newalloc)
{
if (newalloc <= space)return;
T* p = alloc.allocate(newalloc);
for (int i = 0; i < sz; ++i)alloc.construct(&p[i], elem[i]);
for (int i = 0; i < sz; ++i)alloc.destroy(&elem[i]);
elem = p;
space = newalloc;
}
where typename A = Allocator<T> and alloc is of type A.
Will the allocator class I implemented be functional enough to work?
(I feel the deallocate function is suspect)
Your deallocation function is indeed incorrect. The rules for free and malloc is simple: you must pass exactly the pointers you got from malloc to free.
template<typename T>
void Allocator<T>::deallocate(T* p, size_t)
{
free(p);
}
Note you should in general pass the same pointer type to deallocation functions as well, but in this case since free only takes void* as parameter, the implicit conversion will take care of that.
No, this isn't going to work.
You ought to have a rebind method.
You need to provide comparison operators (== and !=)
Your allocate method allocates too much. By the time it gets to you, the sizeof(T) has already happened.
Your deallocate method is just wrong. You get a single pointer. The second parameter is a hint to the size of the allocation.
The construct and destruct methods are optional. If you don't provide them, they will be synthesized for you by allocator_traits.
See https://github.com/llvm-mirror/libcxx/blob/5a424a985612452d4f7a3f02422207456d54a53e/test/support/min_allocator.h#L21 for an example of a minimal allocator (albeit w/o a rebind )
Related
template<typename T>
struct raster {
std::vector<T> v;
template<typename U, typename = std::enable_if_t<sizeof(T) == sizeof(U)>>
raster(raster<U>&& other){
// What code goes here?
}
}
Suppose we have raster<uint32_t> r such that r.v.size() is in the millions, and a guarantee that all its elements are within int32_t's range. Is it possible for raster<int32_t>::raster(raster<uint32_t>&& other) to avoid copying the memory backing other.v?
Or should I just do something like *reinterpret_cast<raster<int32_t>*>(&r) instead of calling that constructor?
There is no legal way to do this in C++; you can only move buffers from a std::vector to another std::vector of the exact same type.
There are a variety of ways you can hack this. The most illegal and evil would be
std::vector<uint32_t> evil_steal_memory( std::vector<int32_t>&& in ) {
return reinterpret_cast< std::vector<uint32_t>&& >(in);
}
or something similar.
A less evil way would be to forget it is a std::vector at all.
template<class T>
struct buffer {
template<class A>
buffer( std::vector<T,A> vec ):
m_begin( vec.data() ),
m_end( m_begin + vec.size() )
{
m_state = std::unique_ptr<void, void(*)(void*)>(
new std::vector<T,A>( std::move(vec) ),
[](void* ptr){
delete static_cast<std::vector<T,A>*>(ptr);
}
);
}
buffer(buffer&&)=default;
buffer& operator=(buffer&&)=default;
~buffer() = default;
T* begin() const { return m_begin; }
T* end() const { return m_end; }
std::size_t size() const { return begin()==end(); }
bool empty() const { return size()==0; }
T* data() const { return m_begin; }
T& operator[](std::size_t i) const {
return data()[i];
}
explicit operator bool() const { return (bool)m_state; }
template<class U>
using is_buffer_compatible = std::integral_constant<bool,
sizeof(U)==sizeof(T)
&& alignof(U)==alignof(T)
&& !std::is_pointer<T>{}
>;
template<class U,
std::enable_if_t< is_buffer_compatible<U>{}, bool > = true
>
buffer reinterpret( buffer<U> o ) {
return {std::move(o.m_state), reinterpret_cast<T*>(o.m_begin()),reinterpret_cast<T*>(o.m_end())};
}
private:
buffer(std::unique_ptr<void, void(*)(void*)> state, T* b, T* e):
m_state(std::move(state)),
m_begin(begin),
m_end(end)
{}
std::unique_ptr<void, void(*)(void*)> m_state;
T* m_begin = 0;
T* m_end = 0;
};
live example: this type erases a buffer of T.
template<class T>
struct raster {
buffer<T> v;
template<typename U, typename = std::enable_if_t<sizeof(T) == sizeof(U)>>
raster(raster<U>&& other):
v( buffer<T>::reinterpret( std::move(other).v ) )
{}
};
note that my buffer has a memory allocation in it; compared to millions of elements that is cheap. It is also move-only.
The memory allocation can be eliminated through careful use of the small buffer optimization.
I'd leave it move-only (who wants to accidentally copy a million elements?) and maybe write
buffer clone() const;
which creates a new buffer with the same contents.
Note that instead of a const buffer<int> you should use a buffer<const int> under the above design. You can change that by duplicating the begin() const methods to have const and non-const versions.
This solution relies on your belief that reinterpreting a buffer of int32_ts as a buffer of uint32_ts (or vice versa) doesn't mess with anything. Your compiler may provide this guarantee, but the C++ standard does not.
The problem is that the implementation of the vector template itself may be specialised for the type. For some weird whacky reason we don't understand today, the top-level vector might need an extra member not provided in vector, so simply reinterpret casting will not work safely.
Another evil approach might be to look at the Allocator of your two vectors.
If this was
a custom allocator you had written and both were a derived class from vector
and you wrote an overload in the class for swap and =&&
within which you created wrapped tmp vectors to swap with
and detected that the Allocator was being called within those temporary objects constructors/destructors, and on the same thread
to release and reallocate the same sized array
Then, perhaps you could legitimately pass the memory buffer between them without resetting the content.
But that is a lot of fragile work!
As the title says, I wonder if this is a proper way to prevent deallocation in vector<T> a when moving vector<T> a tovector<T> b.
Vector header:
template<class T, class A = std::allocator<T>>
class vector
{
typedef typename A::size_type size_type;
A alloc;
T* start;
T* end;
public:
explicit vector(size_type n, const T& val = T(), const A& =A());
vector(vector&&);
~vector();
};
Constructor:
Note: allocate can throw std::bad_alloc.
template<class T, class A>
vector<T,A>::vector(size_type n, const T& val, const A& a)
: alloc(a)
{
start = alloc.allocate(n); //allocate memory for n elements.
end = start + n;
for(auto p = start; p!=end; p++)
alloc.construct(p, val); //construct val at allocated memory.
}
Move constructor:
Question: Is this a proper way to move vector v?
template<class T, class A>
vector<T,A>::vector(vector&& v)
:alloc(v.alloc) // copy the allocator in v.
{
start = std::move(v.start); // move the pointer previously returned by allocator.
end = std::move(v.end); // same boundary value.
v.start = v.end = nullptr; // nullptr to prevent deallocation when v is destroyed.
}
Destructor:
Question: According to cppreference, allocator::deallocate(p, n) deallocates memory from a pointer previously returned by allocator and n must be equal to the number of elements allocated for. What if this is not the case? My answer would be that allocator::deallocate does nothing if the pointer or the number of elements, n, is not equal to previous call to allocator::allocate(n). Is this true?
template<class T, class A>
vector<T, A>::~vector()
{
for(auto p = start; p!=end; p++)
alloc.destroy(p); //Destroy objects pointed to by alloc.
alloc.deallocate(start, end-start); //Deallocate memory.
}
Is this a proper way to move vector v?
Seems fine, but std::move is redundant with pointers. Also, you don't deallocate end, so you don't need to set it to null.
What if this is not the case? My answer would be that allocator::deallocate does nothing if the pointer or the number of elements, n, is not equal to previous call to allocator::allocate(n). Is this true?
No, the behaviour is undefined. You must pass the same n.
I've got a few functions, some of which are overloaded and some are templates, ex.:
void fun1 (vector<string> &a, int b);
void fun1 (vector<double> &a, int b);
template <typename t>
void fun2 (t a[], int b);
and so on. Every function I use has a version for 2 data types, one is an array and the other one is a vector (a vector of strings, a vector of doubles, an array of strings and an array of doubles). The question is, can I create a template for an array of pointers? Is there any way of doing it, apart from:
void (*strvecfun[])(vector <string>&, long, long)={fun1, fun2};
void (*dblvecfun[])(vector <double>&, long, long)={fun1, fun2};
void (*strarrfun[])(string [], long, long)={fun1, fun2};
and so on?
You can use
template<typename Func>
struct func_array {
static Func *const data[];
};
template<typename Func>
Func *const func_array<Func>::data[] = { fun1, fun2 };
And later call
func_array<void(std::vector<double>&, long, long)>::data[0](dvec, l1, l2);
func_array<void(std::vector<string>&, long, long)>::data[1](svec, l1, l2);
// ...and so forth
This requires that there are matching overloads for all the signatures you're going to use of all functions you put into the list, naturally.
Instead of having 2 implementations, have one. Have your data take an array_view<double>:
template<class T>
struct array_view {
// can make this private:
T* b = 0; T* e = 0;
// core methods:
T* begin() const { return b; }
T* end() const { return e; }
// utility methods:
size_t size() const { return end()-begin(); }
T& front() const { return *begin(); }
T& back() const { return *std::prev(end()); }
bool empty() const { return begin()==end(); }
// core ctors:
array_view(T* s, T* f):b(s),e(f) {};
array_view()=default;
array_view(array_view const&)=default;
// derived ctors:
array-view(T* s, size_t l):array_view(s, s+l) {};
template<size_t N>
array_view( T(&arr)[N] ):array_view(arr, N) {};
template<size_t N>
array_view( std::array<T,N>&arr ):array_view(arr.data(), N) {};
template<class A>
array_view( std::vector<T,A>& v ):array_view(v.data(), v.size()) {};
// extra ctors that fail to compile if `T` is not const, but
// are mostly harmless even if `T` is not const, and useful if
// `T` is const. We could do fancy work to disable them, but
// I am lazy:
using non_const_T = std::remove_const_t<T>;
template<class A>
array_view( std::vector<non_const_T,A>const& v ):array_view(v.data(), v.size()) {};
template<size_t N>
array_view( std::array<non_const_T,N>const&arr ):array_view(arr.data(), N) {};
array_view( std::initializer_list<non_const_T> il ):array_view(il.data(), il.size()) {};
};
array_view acts like a view into a container, and can be implicitly converted from a number of std containers as well as raw arrays.
void fun1 (array_view<std::string> a);
a.size() tells you how long it is, and it can be iterated over in a for(:) loop even.
std::vector<T>& is far more powerful than what you need. By using array_view, we only expose what you need (access to elements), and thus are able to take both an array and a container.
If you pass in a "real" C style array, it will auto-deduce the length. If you instead pass in a pointer (or a [] really-a-pointer array), you also have to pass the length like:
fun1( {ptr, length} );
Create a function called max that will return the value of the largest element in an array.
The arguments to the function should be the address of the array size.
Make this function into a template so it will work with an array of any numerical types.
Recently i have been looking for a pool/allocator mechanism.
Boost Pool seems to provide the solution, but there is still things, which it have not been able to deduce from the documentation.
What need to be allocated
Several small classes (~30 chars)
std::map (i want to ensure it do not perform dynamic allocator by itself)
allocation within pugi::xml
std::strings
How to control of address space for allocation (or just amount)
The object_pool seem the to provide a good way for allocating need 1)
However it would like to set a fixed size for the allocator to use. By default it grabs memory be ifself.
If possible i would like to give it the address space it can play within.
char * mem_for_class[1024*1024];
boost::object_pool<my_class,mem_for_class> q;
or:
const int max_no_objs=1024;
boost::object_pool<my_class,max_no_objs> q;
Although the UserAllocator is available in Boost::Pool; it seem to defeat the point. I am afraid the control needed would make it too inefficient... and it would be better to start from scratch.
It it possible to set a fixed area for pool_allocator ?
The question is a bit similar to the first.
Do boost pool provide any way of limiting how much / where there is allocated memory when giving boost::pool_allocator to a std-type-class (e.g. map)
My scenario
Embedded linux programming. The system must keep running for..ever. So we can not risk any memory segmentation. Currently i mostly either static allocation (stack), but also a few raw "new"s.
I would like an allocation scheme that ensure i use the same memory area each time the program loops.
Speed /space is important, but safety is still top priority.
I hope StackOverflow is the place to ask. I tried contacting the author of Boost::Pool "Stephen" without luck. I have not found any Boost-specific forum.
You can always create an allocator that works with STL. If it works with STL, it should work with boost as you are able to pass boost allocators to STL containers..
Considering the above, an allocator that can allocate at a specified memory address AND has a size limitation specified by you can be written as follows:
#include <iostream>
#include <vector>
template<typename T>
class CAllocator
{
private:
std::size_t size;
T* data = nullptr;
public:
typedef T* pointer;
typedef const T* const_pointer;
typedef T& reference;
typedef const T& const_reference;
typedef std::size_t size_type;
typedef std::ptrdiff_t difference_type;
typedef T value_type;
CAllocator() {}
CAllocator(pointer data_ptr, size_type max_size) noexcept : size(max_size), data(data_ptr) {};
template<typename U>
CAllocator(const CAllocator<U>& other) noexcept {};
CAllocator(const CAllocator &other) : size(other.size), data(other.data) {}
template<typename U>
struct rebind {typedef CAllocator<U> other;};
pointer allocate(size_type n, const void* hint = 0) {return &data[0];}
void deallocate(void* ptr, size_type n) {}
size_type max_size() const {return size;}
};
template <typename T, typename U>
inline bool operator == (const CAllocator<T>&, const CAllocator<U>&) {return true;}
template <typename T, typename U>
inline bool operator != (const CAllocator<T>& a, const CAllocator<U>& b) {return !(a == b);}
int main()
{
const int size = 1024 / 4;
int ptr[size];
std::vector<int, CAllocator<int>> vec(CAllocator<int>(&ptr[0], size));
int ptr2[size];
std::vector<int, CAllocator<int>> vec2(CAllocator<int>(&ptr2[0], size));
vec.push_back(10);
vec.push_back(20);
vec2.push_back(30);
vec2.push_back(40);
for (std::size_t i = 0; i < vec2.size(); ++i)
{
int* val = &ptr2[i];
std::cout<<*val<<"\n";
}
std::cout<<"\n\n";
vec2 = vec;
for (std::size_t i = 0; i < vec2.size(); ++i)
{
int* val = &ptr2[i];
std::cout<<*val<<"\n";
}
std::cout<<"\n\n";
vec2.clear();
vec2.push_back(100);
vec2.push_back(200);
for (std::size_t i = 0; i < vec2.size(); ++i)
{
int* val = &ptr2[i];
std::cout<<*val<<"\n";
}
}
This allocator makes sure that all memory is allocated at a specified address. No more than the amount you specify can be allocated with the freedom to allocate were you want whether it is on the stack or the heap.
You may create your own pool or use a std::unique_ptr as the pool for a single container.
EDIT: For strings, you need an offset of sizeof(_Rep_base). See: Why std::string allocating twice?
and http://ideone.com/QWtxWg
It is defined as:
struct _Rep_base
{
std::size_t _M_length;
std::size_t _M_capacity;
_Atomic_word _M_refcount;
};
So the example becomes:
struct Repbase
{
std::size_t length;
std::size_t capacity;
std::int16_t refcount;
};
int main()
{
typedef std::basic_string<char, std::char_traits<char>, CAllocator<char>> CAString;
const int size = 1024;
char ptr[size] = {0};
CAString str(CAllocator<char>(&ptr[0], size));
str = "Hello";
std::cout<<&ptr[sizeof(Repbase)];
}
I'm working with a low-level API that accepts a char* and numeric value to represent a string and its length, respectively. My code uses std::basic_string and calls into these methods with the appropriate translation. Unfortunately, many of these methods accept string lengths of varying size (i.e. max(unsigned char), max(short), etc...) and I'm stuck writing code to make sure that my string instances do not exceed the maximum length prescribed by the low-level API.
By default, the maximum length of an std::basic_string instance is bound by the maximum value of size_t (either max(unsigned int) or max(__int64)). Is there a way to manipulate the traits and allocator implementations of a std::basic_string implementation so that I may specify my own type to use in place of size_t? By doing so, I am hoping to leverage any existing bounds checks within the std::basic_string implementation so I don't have to do so when performing the translation.
My initial investigation suggests that this is not possible without writing my own string class, but I'm hoping that I overlooked something :)
you can pass a custom allocator to std::basic_string which has a max size of whatever you want. This should be sufficient. Perhaps something like this:
template <class T>
class my_allocator {
public:
typedef T value_type;
typedef std::size_t size_type;
typedef std::ptrdiff_t difference_type;
typedef T* pointer;
typedef const T* const_pointer;
typedef T& reference;
typedef const T& const_reference;
pointer address(reference r) const { return &r; }
const_pointer address(const_reference r) const { return &r; }
my_allocator() throw() {}
template <class U>
my_allocator(const my_allocator<U>&) throw() {}
~my_allocator() throw() {}
pointer allocate(size_type n, void * = 0) {
// fail if we try to allocate too much
if((n * sizeof(T))> max_size()) { throw std::bad_alloc(); }
return static_cast<T *>(::operator new(n * sizeof(T)));
}
void deallocate(pointer p, size_type) {
return ::operator delete(p);
}
void construct(pointer p, const T& val) { new(p) T(val); }
void destroy(pointer p) { p->~T(); }
// max out at about 64k
size_type max_size() const throw() { return 0xffff; }
template <class U>
struct rebind { typedef my_allocator<U> other; };
template <class U>
my_allocator& operator=(const my_allocator<U> &rhs) {
(void)rhs;
return *this;
}
};
Then you can probably do this:
typedef std::basic_string<char, std::char_traits<char>, my_allocator<char> > limited_string;
EDIT: I've just done a test to make sure this works as expected. The following code tests it.
int main() {
limited_string s;
s = "AAAA";
s += s;
s += s;
s += s;
s += s;
s += s;
s += s;
s += s; // 512 chars...
s += s;
s += s;
s += s;
s += s;
s += s;
s += s; // 32768 chars...
s += s; // this will throw std::bad_alloc
std::cout << s.max_size() << std::endl;
std::cout << s.size() << std::endl;
}
That last s += s will put it over the top and cause a std::bad_alloc exception, (since my limit is just short of 64k). Unfortunately gcc's std::basic_string::max_size() implementation does not base its result on the allocator you use, so it will still claim to be able to allocate more. (I'm not sure if this is a bug or not...).
But this will definitely allow you impose hard limits on the sizes of strings in a simple way. You could even make the max size a template parameter so you only have to write the code for the allocator once.
I agree with Evan Teran about his solution. This is just a modification of his solution no more:
template <typename Type, typename std::allocator<Type>::size_type maxSize>
struct myalloc : std::allocator<Type>
{
// hide std::allocator[ max_size() & allocate(...) ]
std::allocator<Type>::size_type max_size() const throw()
{
return maxSize;
}
std::allocator<Type>::pointer allocate
(std::allocator<Type>::size_type n, void * = 0)
{
// fail if we try to allocate too much
if((n * sizeof(Type))> max_size()) { throw std::bad_alloc(); }
return static_cast<Type *>(::operator new(n * sizeof(Type)));
}
};
Be aware you should not use polymorphism at all with myalloc. So this is disastrous:
// std::allocator doesn't have a virtual destructor
std::allocator<char>* alloc = new myalloc<char>;
You just use it as if it is a separate type, it is safe in following case:
myalloc<char, 1024> alloc; // max size == 1024
Can't you create a class with std::string as parent and override c_str()?
Or define your own c_str16(), c_str32(), etc and implement translation there?