C++ allocators (as used by std::vector) are tricky to. I understand that they changed a lot to allow stateful allocators and PMR, leading to some of the cruftiness. My core question is: if allocators are intended to replace new and delete, why do they only provide an API like malloc and free? I understand why, e.g., std::vector needs a malloc interface since it needs to allocate a buffer without calling constructors, but in general, it seems like we are missing these functions:
#include <cassert>
#include <iostream>
#include <memory>
//! Rebind alloc to type T
template <typename T, typename Alloc>
auto rebound_allocator(const Alloc& alloc) {
return typename std::allocator_traits<Alloc>::template rebind_alloc<T>{alloc};
}
//! Like operator delete but for a single T allocated by rebound_allocator<T>(alloc).
template <typename T, typename Alloc>
void allocator_delete(const Alloc& alloc, T* ptr) {
assert(ptr);
auto a = rebound_allocator<T>(alloc);
using traits_t = std::allocator_traits<decltype(a)>;
// Should we try/catch around destroy and always deallocate?
traits_t::destroy(a, ptr);
traits_t::deallocate(a, ptr, 1);
}
//! Returned memory must be freed with, e.g., allocator_delete(alloc, ptr).
template <typename T, typename Alloc, typename... Args>
[[nodiscard]] T* allocator_new(const Alloc& alloc, Args&&... args) {
auto a = rebound_allocator<T>(alloc);
using traits_t = std::allocator_traits<decltype(a)>;
auto deallocate = [&a](T* ptr) { traits_t::deallocate(a, ptr, 1); };
// Hold in a unique_ptr to deallocate if construction throws.
auto buf = std::unique_ptr<T, decltype(deallocate)>(traits_t::allocate(a, 1), deallocate);
traits_t::construct(a, buf.get(), std::forward<Args>(args)...);
return buf.release();
}
//! Like make_unique. Beware: The allocator is is referenced by the deleter!
template <typename T, typename Alloc, typename... Args>
[[nodiscard]] auto allocator_make_unique(const Alloc& alloc, Args&&... args) {
auto dtor = [&alloc](T* ptr) { allocator_delete<T>(alloc, ptr); };
return std::unique_ptr<T, decltype(dtor)>(allocator_new<T>(alloc, std::forward<Args>(args)...),
dtor);
}
struct S {
float x;
S(float x) : x(x) { std::cout << "S::S()" << std::endl; }
~S() { std::cout << "S::~S()" << std::endl; }
};
int main() {
std::allocator<int> alloc;
auto ptr = allocator_make_unique<S>(alloc, 42.5f);
assert(ptr);
std::cout << ptr->x << std::endl;
}
Output:
S::S()
42.5
S::~S()
https://godbolt.org/z/sheec6br3
Am I missing something? Is this the right way to implement essentially new and delete and make_unique using allocators? If so, is this really not provided by the standard library?
Edit:
I think (but am not sure?) that if T is allocator-aware, traits_t::construct(a, ptr, n) will propagate itself into the created object?
Edit:
Here's a cleaned-up version: https://godbolt.org/z/47Tdzf4W7
Edit:
Original version: https://godbolt.org/z/dGW7hzdc1
My core question is: if allocators are intended to replace new and delete, why do they only provide an API like malloc and free?
The API of allocators are such as it is because an important point of allocators is that memory allocation and object creation must be separated. This is necessary for example to implement a container such as std::vector. Allocators are a generalisation of operator new / delete, not generalisation of the new expression.
is this really not provided by the standard library?
No, these functions aren't provided by the standard library.
Is this the right way to implement essentially new and delete and make_unique using allocators?
auto dtor = [&alloc](T* ptr) { destruct_and_deallocate(alloc, ptr); };
^
Capturing allocator by reference into the deleter seems bad. It should be copied to be safe.
Other suggestions:
In the default case of std::allocator, we would like to avoid paying for the overhead of the deleter. Consider adding a specialisation that delegates to std::make_unique when std::allocator is used.
You could avoid the try-catch by using an intermediary unique pointer with deleter that only deallocates:
T* ptr = traits_t::allocate(rebound_alloc, 1);
auto dealloc = [&](T* ptr) { traits_t::deallocate(rebound_alloc, ptr, 1); };
std::unique_ptr<T, decltype(dealloc)> storage(ptr, dealloc);
traits_t::construct(rebound_alloc, storage.get(), std::forward<Args>(args)...);
auto dtor = [alloc](T* ptr) { destruct_and_deallocate(alloc, ptr); };
return std::unique_ptr<T, decltype(dtor)>(storage.release(), dtor);
Edit: I think (but am not sure?) that if T is allocator-aware, traits_t::construct(a, ptr, n) will propagate itself into the created object?
No, object's have no knowledge of the allocator that creates them or allocates their memory. "Allocator aware" containers are simply generic tempaltes that allow the user to provide a custom allocator, and avoid allocating memory through other means.
Related
I'd like to build a simple boost::lockfree::queue of functions that take no arguments and return no values.
It appears that boost::lockfree::queue requires the item type to be trivially assignable and destructible, requirements that boost::function<void ()> unfortunately doesn' meet.
In the spirit of https://stackoverflow.com/a/21406186/393756, I'm now trying to achieve this by a boost::lockfree::queue of plain function pointers:
boost::lockfree::queue<void (*)()> queue;
Can I push a boost::function<void ()> into this queue? If so, how?
Can I push a boost::function<void()> into this queue?
Not directly, as boost::function<void()> is an heavyweight owning type-erased wrapper that is not implicitly convertible to a function pointer and also stores some data.
If you need a trivially-assignable an trivially-destructible type that can refer to any function object, you could implement a function_view class that points to some function object without owning it. If you're careful with lifetimes and guarantee that function_view always points to "live objects" you can safely store instances of that in your queue.
Conceptually, function_view is a pair of pointers. I have an implementation in my "passing functions to functions" article, which I'm pasting below:
template <typename TReturn, typename... TArgs>
class function_view<TReturn(TArgs...)> final
{
private:
using signature_type = TReturn(void*, TArgs...);
void* _ptr;
TReturn (*_erased_fn)(void*, TArgs...);
public:
template <typename T, typename = std::enable_if_t<
std::is_callable<T&(TArgs...)>{} &&
!std::is_same<std::decay_t<T>, function_view>{}>>
function_view(T&& x) noexcept : _ptr{(void*)std::addressof(x)}
{
_erased_fn = [](void* ptr, TArgs... xs) -> TReturn {
return (*reinterpret_cast<std::add_pointer_t<T>>(ptr))(
std::forward<TArgs>(xs)...);
};
}
decltype(auto) operator()(TArgs... xs) const
noexcept(noexcept(_erased_fn(_ptr, std::forward<TArgs>(xs)...)))
{
return _erased_fn(_ptr, std::forward<TArgs>(xs)...);
}
};
This class passes the following tests:
using type = function_view<void()>;
static_assert(is_trivially_assignable<type, type>{});
static_assert(is_trivially_destructible<type>{});
live example on wandbox
no, but you can use dynamic memory allocation + type erasure for that cause:
struct callback_back{
virtual void execute() = 0;
~callback_base() = default;
};
template<class F>
class callback{
private:
F m_function;
public:
callback(F&& function) : m_function(std::forward<F>(function)){}
virtual void execute() {
m_function();
}
}
template<class F>
std::unique_ptr<callback_base> make_callback(F&& f){
return std::unique_ptr<callback_base>(
new callback<F>(std::forward<F>(f));
);
}
use callback_base as a noexcept-movable type (aka boost::lockfree::queue<std::unique_ptr<callback_base>>).
The only way I found so far is to make a raw pointer of the function object
boost::lockfree::queue<std::function<void(void)> *> tasks_; // the queue
// let f = stack allocated std::function<T(T)> instance
tasks_.push(new std::function<void(void)>(f));
// pop
std::function<void(void)> * f;
tasks_.pop(f);
// execute in try/catch make sure to delete in case of exception?
(*f)();
// you should delete f here if you are done with it
// in the destructor of the class that owns tasks_ you should delete the remaining std::function instances
The challenge here is to when to delete this instance with exception safety in mind
I was trying to implement an std::unique_ptr factory that I could use like this:
auto fd = my_make_unique<fclose>(fopen("filename", "r"));
I.e., pass the deleter function as a template argument.
My best attempt in C++11 was:
template<typename D, D deleter, typename P>
struct Deleter {
void operator()(P* ptr) {
deleter(ptr);
}
};
template<typename D, D deleter, typename P>
std::unique_ptr<P, Deleter<D, deleter, P>> my_make_unique(P* ptr)
{
return std::unique_ptr<P, Deleter<D, deleter, P>>(ptr);
}
In C++14 it is much cleaner:
template<typename D, D deleter, typename P>
auto my_make_unique(P* ptr)
{
struct Deleter {
void operator()(P* ptr) {
deleter(ptr);
}
};
return std::unique_ptr<P, Deleter>(ptr);
}
But both solutuions would require me to pass the type of &fclose before fclose itself as template argument:
auto fd = my_make_unique<decltype(&fclose), fclose>(fopen("filename", "r"));
Is it possible to get rid of decltype(&fclose) template argument in C++11? What about in C++14?
EDIT: Why this question is not a duplicate of RAII and smart pointers in C++: referenced question is about general RAII techniques in C++, and one of the answers estates that std::unique_ptr can be used for this purpose. I am already familiar with RAII pattern and how std::unique_ptr is a solution, but I am concerned with present question on how to build an easier to use abstraction to this frequent case I encounter when interacting with C libraries.
Is it possible to get rid of decltype(&fclose) template argument in C++11? What about in C++14?
No, not until C++17 can you get rid of the type of that parameter. Template non-type parameters need a type, which you can't deduce - because it has to be a template non-type parameter. That's one problem.
Additionally, you have the problem that taking the address of functions in the standard library is unspecified. The standard library is always allowed to provide additional overloads, for instance, so &fclose may be invalid. The only truly portable way of doing this is to provide a lambda or write your own wrapper function:
auto my_fclose_lam = [](std::FILE* f) { std::fclose(f); }
void my_fclose_fun(std::FILE* f) { std::fclose(f); }
And with either of those, with C++14 at best you can introduce a macro like:
#define DECL(v) decltype(v), v
auto fd = my_make_unique<DECL(my_fclose_lam)>(fopen("filename", "r"));
C++17 allows you to at least lift your custom function into a template parameter (though not yet a lambda) via template auto:
template <auto deleter, typename P>
auto my_make_unique(P* ptr)
{
struct Deleter {
void operator()(P* ptr) {
deleter(ptr);
}
};
return std::unique_ptr<P, Deleter>(ptr);
}
my_make_unique<my_fclose_fun>(fopen(...));
C++20 will finally allow you to stick a lambda into there:
my_make_unique<[](std::FILE* f){ std::fclose(f); }>(fopen(...));
Old incorrect answer:
So the best you can do is introduce a macro like:
#define DECL(v) decltype(v), v
auto fd = my_make_unique<DECL(&fclose)>(fopen("filename", "r"));
Whether or not you think that's a good idea is probably up to your coworkers.
In C++17, with template auto, you could just be able to write my_make_unique<fclose>, which is great:
template <auto deleter, typename P>
auto my_make_unique(P* ptr)
{
struct Deleter {
void operator()(P* ptr) {
deleter(ptr);
}
};
return std::unique_ptr<P, Deleter>(ptr);
}
Pragmatic approach: Make the deleter a runtime parameter.
template<typename P, typename D>
auto my_make_unique(P* ptr, D deleter)
{
return std::unique_ptr<P, D>(ptr, deleter);
}
int main()
{
auto fd = my_make_unique(fopen("filename", "r"), fclose);
}
Another workaround is use exactly signature of function:
template<typename T, int (*P)(T*)> //for `fclose`
auto my_make_unique(T*) { ... }
template<typename T, void (*P)(T*)> //for other function signatures
auto my_make_unique(T*) { ... }
//etc.
auto uniq = my_make_unique<File, fclose>(fopen("filename", "r"));
This is not universal solution but in 95% cases will work.
The typical way to create a std::unique_ptr for a FILE* pointer is:
auto fd = std::unique_ptr<FILE, decltype(fclose)>(fopen(...), fclose);
You could wrap that in a macro:
#define my_make_unique(ptr, deleter) \
std::unique_ptr<std::remove_pointer<decltype<ptr>>::type, d>(ptr, deleter)
And then use it like this:
auto fd = my_make_unique(fopen(...), fclose);
Is there something wrong with the code below?
#include <iostream>
#include <type_traits>
template <typename T>
void assign_lambda(T&& f)
{
typedef typename std::remove_reference<T>::type functor_type;
typedef typename std::aligned_storage<sizeof(functor_type),
std::alignment_of<functor_type>::value>::type buffer_type;
static char store[sizeof(buffer_type)];
auto const p(new (store) functor_type(std::forward<T>(f)));
(*p)();
}
int main()
{
for (int i(0); i != 5; ++i)
{
assign_lambda([i](){ std::cout << i << std::endl; });
}
return 0;
}
I worry though that this might be non-standard and/or dangerous to do.
EDIT:
Why initialize into a char array you ask? One might allocate a block of size sizeof(buffer_type) from the heap and reuse for repeated assignments (i.e. avoid repeated memory allocations), if the block should prove large enough.
void*operator new(std::size_t size);
Effects: The allocation function (3.7.4.1) called by a new-expression (5.3.4) to allocate size bytes of storage suitably aligned to represent any object of that size.
I suppose if I allocate from the heap the alignment issues will go away.
You'll have to make sure that store has the proper alignment for functor_type. Apart from that, I don't see any problems regarding standard conformance. However, you can easily address the multithreading issue by making the array nonstatic, because sizeof gives a compiletime constant.
The alignment is demanded by §5.3.4,14:
[ Note: when the allocation function returns a value other than null, it must be a pointer to a block of storage in which space for the object has been reserved. The block of storage is assumed to be appropriately aligned and of the requested size. [...] -end note ]
There is another paragraph, §3.7.4.1 about alignment, but that one does explicitly not apply to placement new (§18.6.1.3,1).
To get the alignment right, you can do the following:
template <typename T>
void assign_lambda(T&& f)
{
typedef typename std::remove_reference<T>::type functor_type;
//alignas(functor_type) char store[sizeof(functor_type)];
std::aligned_storage<sizeof(functor_type),
std::alignment_of<functor_type>::value>::type store;
auto const p(new (&store) functor_type(std::forward<T>(f)));
(*p)();
//"placement delete"
p->~functor_type();
}
Update:
The approach shown above is not different from using just a normal variable:
template <typename T>
void assign_lambda(T&& f)
{
typedef typename std::remove_reference<T>::type functor_type;
functor_type func{std::forward<T>(f)};
func();
}
If it has to be a static variable inside the function you will need an RAII wrapper for functors that are not assignable. Just placement-newing is not sufficient since the functors will not get destroyed properly and ressources they possess (e.g. via captured smartpointers) will not get released.
template <typename F>
struct RAIIFunctor {
typedef typename std::remove_reference<F>::type functor_type;
std::aligned_storage<sizeof(functor_type),
std::alignment_of<functor_type>::value>::type store;
functor_type* f;
RAIIFunctor() : f{nullptr} {}
~RAIIFunctor() { destroy(); }
template <class T>
void assign(T&& t) {
destroy();
f = new(&store) functor_type {std::forward<T>(t)};
}
void destroy() {
if (f)
f->~functor_type();
f = nullptr;
}
void operator() {
(*f)();
}
};
template <typename T>
void assign_lambda(T&& f)
{
static RAIIFunctor<T> func;
func.assign(std::forward<T>(f));
func();
}
You can see the code in action here
I don't get it. Why would one use aligned_storage merely to get some size to create uninitialised storage, instead of... using the aligned storage it provides? It's almost like travelling from Berlin to Lisbon by taking a Berlin -> Lisbon flight followed by a Lisbon -> Moscow flight.
typedef typename std::remove_reference<T>::type functor_type;
typedef typename std::aligned_storage<sizeof(functor_type),
std::alignment_of<functor_type>::value>::type buffer_type;
static buffer_type store;
auto const p(new (&store) functor_type(std::forward<T>(f)));
In addition to the alignment issue already mentioned, you are creating a copy of the lambda through placement new but you are not destroying the copy.
The following code illustrates the problem:
// This class plays the role of the OP's lambdas
struct Probe {
Probe() { std::cout << "Ctr" << '\n'; }
Probe(const Probe&) { std::cout << "Cpy-ctr" << '\n'; }
~Probe() { std::cout << "Dtr" << '\n'; }
};
// This plays the role of the OP's assign_lambda
void f(const Probe& p) {
typedef typename std::aligned_storage<sizeof(Probe),
std::alignment_of<Probe>::value>::type buffer_type;
static buffer_type store;
new (&store) Probe(p);
}
int main() {
Probe p;
// This plays the role of the loop
f(p);
f(p);
f(p);
}
The output is:
Ctr
Cpy-ctr
Cpy-ctr
Cpy-ctr
Dtr
Therefore, 4 objects are constructed and only one is destroyed.
In addition, in the OP's code the store is static and this means that one lambda is repeatedly constructed on top of the other as if the latter was just raw memory.
I'm interested in building an uninitialized_vector container, which will be semantically identical to std::vector with the caveat that new elements which otherwise would be created with a no-argument constructor will instead be created without initialization. I'm primarily interested in avoiding initializing POD to 0. As far as I can tell, there's no way to accomplish this by combining std::vector with a special kind of allocator.
I'd like to build my container in the same vein as std::stack, which adapts a user-provided container (in my case, std::vector). In other words, I'd like to avoid reimplementing the entirety of std::vector and instead provide a "facade" around it.
Is there a simple way to control default construction from the "outside" of std::vector?
Here's the solution I arrived at, which was inspired Kerrek's answer:
#include <iostream>
#include <vector>
#include <memory>
#include <algorithm>
#include <cassert>
// uninitialized_allocator adapts a given base allocator
// uninitialized_allocator's behavior is equivalent to the base
// except for its no-argument construct function, which is a no-op
template<typename T, typename BaseAllocator = std::allocator<T>>
struct uninitialized_allocator
: BaseAllocator::template rebind<T>::other
{
typedef typename BaseAllocator::template rebind<T>::other super_t;
template<typename U>
struct rebind
{
typedef uninitialized_allocator<U, BaseAllocator> other;
};
// XXX for testing purposes
typename super_t::pointer allocate(typename super_t::size_type n)
{
auto result = super_t::allocate(n);
// fill result with 13 so we can check afterwards that
// the result was not default-constructed
std::fill(result, result + n, 13);
return result;
}
// catch default-construction
void construct(T *p)
{
// no-op
}
// forward everything else with at least one argument to the base
template<typename Arg1, typename... Args>
void construct(T* p, Arg1 &&arg1, Args&&... args)
{
super_t::construct(p, std::forward<Arg1>(arg1), std::forward<Args>(args)...);
}
};
namespace std
{
// XXX specialize allocator_traits
// this shouldn't be necessary, but clang++ 2.7 + libc++ has trouble
// recognizing that uninitialized_allocator<T> has a well-formed
// construct function
template<typename T>
struct allocator_traits<uninitialized_allocator<T> >
: std::allocator_traits<std::allocator<T>>
{
typedef uninitialized_allocator<T> allocator_type;
// for testing purposes, forward allocate through
static typename allocator_type::pointer allocate(allocator_type &a, typename allocator_type::size_type n)
{
return a.allocate(n);
}
template<typename... Args>
static void construct(allocator_type &a, T* ptr, Args&&... args)
{
a.construct(ptr, std::forward<Args>(args)...);
};
};
}
// uninitialized_vector is implemented by adapting an allocator and
// inheriting from std::vector
// a template alias would be another possiblity
// XXX does not compile with clang++ 2.9
//template<typename T, typename BaseAllocator>
//using uninitialized_vector = std::vector<T, uninitialized_allocator<T,BaseAllocator>>;
template<typename T, typename BaseAllocator = std::allocator<T>>
struct uninitialized_vector
: std::vector<T, uninitialized_allocator<T,BaseAllocator>>
{};
int main()
{
uninitialized_vector<int> vec;
vec.resize(10);
// everything should be 13
assert(std::count(vec.begin(), vec.end(), 13) == vec.size());
// copy construction should be preserved
vec.push_back(7);
assert(7 == vec.back());
return 0;
}
This solution will work depending on how closely a particular vendor's compiler & STL's std::vector implementation conforms to c++11.
Instead of using a wrapper around the container, consider using a wrapper around the element type:
template <typename T>
struct uninitialized
{
uninitialized() { }
T value;
};
I think the problem boils down to the type of initialization that the container performs on elements. Compare:
T * p1 = new T; // default-initalization
T * p2 = new T(); // value-initialization
The problem with the standard containers is that they take the default argument to be value initialized, as in resize(size_t, T = T()). This means that there's no elegant way to avoid value-initialization or copying. (Similarly for the constructor.)
Even using the standard allocators doesn't work, because their central construct() function takes an argument that becomes value-initialized. What you would rather need is a construct() that uses default-initialization:
template <typename T>
void definit_construct(void * addr)
{
new (addr) T; // default-initialization
}
Such a thing wouldn't be a conforming standard allocator any more, but you could build your own container around that idea.
I don't believe this is possible by wrapping a vector (that works with every type), unless you resize the vector on every add and remove operation.
If you could give up wrapping STL containers, you could do this by keeping an array of char on the heap and using placement new for each of the objects you want to construct. This way you could control exactly when the constructors and destructors of objects were called, one by one.
I am trying to write a generic allocator class that does not really release an object's memory when it is free()'d but holds it in a queue and returns a previously allocated object if a new one is requested. Now, what I can't wrap my head around is how to pass arguments to the object's constructor when using my allocator (at least without resorting to variadic templates, that is). The alloc() function i came up with looks like this:
template <typename... T>
inline T *alloc(const &T... args) {
T *p;
if (_free.empty()) {
p = new T(args...);
} else {
p = _free.front();
_free.pop();
// to call the ctor of T, we need to first call its DTor
p->~T();
p = new( p ) T(args...);
}
return p;
}
Still, I need the code to be compatible with today's C++ (and older versions of GCC that do not support variadic templates). Is there any other way to go about passing an arbitrary amount of arguments to the objects constructor?
When you need to target pre-C++0x compilers you need to provide pseudo-variadic templates, i.e. you need to provide a template function for every needed arity:
template<class T>
T* alloc() {
/* ... */
}
template<class T, class A0>
T* alloc(const A0& a0) {
/* ... */
}
/* ... */
You can use preprocessor metaprogramming though to handle the repititions, e.g. by using Boost.Preprocessor or by simply generating the functions using a simple script.
Following is a simple example using Boost.PP:
#include <boost/preprocessor/arithmetic/inc.hpp>
#include <boost/preprocessor/repetition/enum_binary_params.hpp>
#include <boost/preprocessor/repetition/enum_params.hpp>
template<class T>
T* alloc() {
return new T;
}
#define FUNCTION_ALLOC(z, N, _) \
template<class T, BOOST_PP_ENUM_PARAMS_Z(z, BOOST_PP_INC(N), class T)> \
T* alloc(BOOST_PP_ENUM_BINARY_PARAMS_Z(z, BOOST_PP_INC(N), const T, &p)) { \
return new T( \
BOOST_PP_ENUM_PARAMS_Z(z, BOOST_PP_INC(N), p) \
); \
}
BOOST_PP_REPEAT(10, FUNCTION_ALLOC, ~)
#undef FUNCTION_ALLOC
This generates you alloc() template functions for up to 10 arguments.
The pre-C++11 solution to the problem is to provide only one simple alloc function which constructs a copy of its argument. This is the way C++03 allocators and all the containers worked, for more than 20 years. Applying it to your code it becomes:
template <typename T>
inline T *alloc(const &T arg) {
T *p;
if (_free.empty()) {
p = new T(arg);
} else {
p = _free.front();
_free.pop();
// to call the ctor of T, we need to first call its DTor
p->~T();
p = new( p ) T(arg);
}
return p;
}
And then you call it as:
// copy construct T into the allocator's memory:
instance_of_your_allocator.alloc(T(1, 2, 3));
The downside of this approach is that it requires a copy-constructor to be available, and its potentially a costly operation.
One more example:
vector<T> vec;
vec.push_back(T(1, 2, 3)); // C++03 way, uses move cons-tor in C++11 if possible.
vec.emplace_back(1, 2, 3); // C++11 way, constructs in-place