For a project of mine, I am writing some STL containers from scratch (I have my reasons). Since I am mimicking the functionality and interfaces of the STL so closely I am doing my best to keep with the policy "if it has the same name as a standard construct, it will conform to the standard as much as possible."
So, of course my containers take allocators as a template parameter, which is very nice as it allows for some custom allocation schemes. On to my question.
The std::allocator interface separates memory allocation from object construction. Likewise it separates deallocation from destruction. This makes sense since where you get memory from is more or less irrelevant to properly constructing an object in c++.
So there are two construction/deallocation functions which look like this for the default implementation (lifted straight from a book):
void construct(pointer p, const T& val) { new(p) T(val); }
void destroy(pointer p) { p->~T(); }
as you can see construct simply calls placement new and destroy simply calls the destructor.
Is there any reason to use these over just using placement new and destructor syntax? can a "correct" allocator implement these in another way? Or am I guaranteed that all allocator implementations which conform to the standard will have there construct/destroy methods implemented in this way?
More to the point, is it safe to say that I can always use the std::uninitialized_copy and std::uninitialized_fill for constructing the elements of my containers?
Thanks.
The allocator could add logging statements before and after construction/destruction, or any other side effects it cared to do.
Of course the actual construction has to occur by calling placement new and the destructor, but it doesn't say in the rulebook that nothing else must happen in the construct/destroy functions
This is just to hide the details of allocation in a method. i.e., we are providing the APIs for construction and destruction, in the future we can change the implementation. Now we are using placement new to allocate the memory, in the future if we want to change the allocation we just have to change these two methods.
Related
I was reading Top 10 dumb mistakes to avoid with C++11 smart pointer.
Number #5 reads:
Mistake # 5 : Not assigning an object(raw pointer) to a shared_ptr as
soon as it is created !
int main()
{
Aircraft* myAircraft = new Aircraft("F-16");
shared_ptr<aircraft> pAircraft(myAircraft);
...
shared_ptr<aircraft> p2(myAircraft);
// will do a double delete and possibly crash
}
and the recommendation is something like:
Use make_shared or new and immediately construct the pointer with
it.
Ok, no doubt about it the problem and the recommendation.
However I have a question about the design of shared_ptr.
This is a very easy mistake to make and the whole "safe" design of shared_ptr could be thrown away by very easy-to-detect missuses.
Now the question is, could this be easily been fixed with an alternative design of shared_ptr in which the only constructor from raw pointer would be that from a r-value reference?
template<class T>
struct shared_ptr{
shared_ptr(T*&& t){...basically current implementation...}
shared_ptr(T* t) = delete; // this is to...
shared_ptr(T* const& t) = delete; // ... illustrate the point.
shared_ptr(T*& t) = delete;
...
};
In this way shared_ptr could be only initialized from the result of new or some factory function.
Is this an underexploitation of the C++ language in the library?
or What is the point of having a constructor from raw pointer (l-value) reference if this is going to be most likely a misuse?
Is this a historical accident? (e.g. shared_ptr was proposed before r-value references were introduced, etc) Backwards compatibility?
(Of course one could say std::shared_ptr<type>(std::move(ptr)); that that is easier to catch and also a work around if this is really necessary.)
Am I missing something?
Pointers are very easy to copy. Even if you restrict to r-value reference you can sill easily make copies (like when you pass a pointer as a function parameter) which will invalidate the safety setup. Moreover you will run into problems in templates where you can easily have T* const or T*& as a type and you get type mismatches.
So you are proposing to create more restrictions without significant safety gains, which is likely why it was not in the standard to begin with.
The point of make_shared is to atomize the construction of a shared pointer. Say you have f(shared_ptr<int>(new int(5)), throw_some_exception()). The order of parameter invokation is not guaranteed by the standard. The compiler is allowed to create a new int, run throw_some_exception and then construct the shared_ptr which means that you could leak the int (if throw_some_exception actually throws an exception). make_shared just creates the object and the shared pointer inside itself, which doesn't allow the compiler to change the order, so it becomes safe.
I do not have any special insight into the design of shared_ptr, but I think the most likely explanation is that the timelines involved made this impossible:
The shared_ptr was introduced at the same time as rvalue-references, in C++11. The shared_ptr already had a working reference implementation in boost, so it could be expected to be added to standard libraries relatively quickly.
If the constructor for shared_ptr had only supported construction from rvalue references, it would have been unusable until the compiler had also implemented support for rvalue references.
And at that time, compiler and standards development was much more asynchronous, so it could have taken years until all compiler had implemented support, if at all. (export templates were still fresh on peoples minds in 2011)
Additionally, I assume the standards committee would have felt uncomfortable standardizing an API that did not have a reference implementation, and could not even get one until after the standard was published.
There's a number of cases in which you may not be able to call make_shared(). For example, your code may not be responsible for allocating and constructing the class in question. The following paradigm (private constructors + factory functions) is used in some C++ code bases for a variety of reasons:
struct A {
private:
A();
};
A* A_factory();
In this case, if you wanted to stick the A* you get from A_factory() into a shared_ptr<>, you'd have to use the constructor which takes a raw pointer instead of make_shared().
Off the top of my head, some other examples:
You want to get aligned memory for your type using posix_memalign() and then store it in a shared_ptr<> with a custom deleter that calls free() (this use case will go away soon when we add aligned allocation to the language!).
You want to stick a pointer to a memory-mapped region created with mmap() into a shared_ptr<> with a custom deleter that calls munmap() (this use case will go away when we get a standardized facility for shmem, something I'm hoping to work on in the next few months).
You want to stick a pointer allocated by into a shared_ptr<> with a custom deleter.
I'm developing a container-like class and I would like to make use of the standard allocator infrastructure much like the standard containers. On the net I find a lot of material about how to use the std::allocator class alone, or how to define a custom allocator for standard containers, but the material about how to make generically use of an standard conforming allocator is very rare, in particular in the context of C++11, where things seem to be much easier from the point of view of who writes a custom allocator, but more complex from the container's point of view.
So my question is about how to correctly make use of a standard conforming allocator in the most generic way, specifically:
First of all, when should I design a custom container in this way? Is there a sensible performance overhead (including missing optimization opportunities) in using the default allocator instead of plain new/delete?
Do I have to explicitly call contained objects' destructors?
How do I discriminate between stateful and stateless allocators?
How to handle stateful allocators?
When (if ever) are two instances interchangeable (when can I destroy with one instance the memory allocated with another one)?
They have to be copied when the container is copied?
They can/have to be moved when the container is moved?
In the container's move constructor and move assignment operator, when can I move the pointer to allocated memory, and when do I have to allocate different memory and move the elements instead?
Are there issues about exception safety in this context?
I'm specifically interested in an answer about the C++11 world (does it change anything in C++14?)
In all answers below, I'm assuming you want to follow the rules for C++11 std-defined containers. The standard does not require you to write your custom containers this way.
First of all, when should I design a custom container in this way? Is there a sensible performance overhead (including missing
optimization opportunities) in using the default allocator instead of
plain new/delete?
One of the most common and effective uses for custom allocators is to have it allocate off of the stack, for performance reasons. If your custom container can not accept such an allocator, then your clients will not be able to perform such an optimization.
Do I have to explicitly call contained objects' destructors?
You have to explicitly call allocator_traits<allocator_type>::destroy(alloc, ptr), which in turn will either directly call the value_type's destructor, or will call the destroy member of the allocator_type.
How do I discriminate between stateful and stateless allocators?
I would not bother. Just assume the allocator is stateful.
How to handle stateful allocators?
Follow the rules laid out in C++11 very carefully. Especially those for allocator-aware containers specified in [container.requirements.general]. The rules are too numerous to list here. However I'm happy to answer specific questions on any of those rules. But step one is get a copy of the standard, and read it, at least the container requirements sections. I recommend the latest C++14 working draft for this purpose.
When (if ever) are two instances interchangeable (when can I destroy with one instance the memory allocated with another one)?
If two allocators compare equal, then either can deallocate pointers allocated from the other. Copies (either by copy construction or copy assignment) are required to compare equal.
They have to be copied when the container is copied?
Search the standard for propagate_on and select_on_container_copy_construction for the nitty gritty details. The nutshell answer is "it depends."
They can/have to be moved when the container is moved?
Have to be for move construction. Move assignment depends on propagate_on_container_move_assignment.
In the container's move constructor and move assignment operator, when can I move the pointer to allocated memory, and when do I have to allocate different memory
and move the elements instead?
The newly move constructed container should have gotten its allocator by move constructing the rhs's allocator. These two allocators are required to compare equal. So you can transfer memory ownership for all allocated memory for which your container has a valid state for that pointer being nullptr in the rhs.
The move assignment operator is arguably the most complicated: The behavior depends on propagate_on_container_move_assignment and whether or not the two allocators compare equal. A more complete description is below in my "allocator cheat sheet."
Are there issues about exception safety in this context?
Yes, tons. [allocator.requirements] lists the allocator requirements, which the container can depend on. This includes which operations can and can not throw.
You will also need to deal with the possibility that the allocator's pointer is not actually a value_type*. [allocator.requirements] is also the place to look for these details.
Good luck. This is not a beginner project. If you have more specific questions, post them to SO. To get started, go straight to the standard. I am not aware of any other authoritative source on the subject.
Here is a cheat-sheet I made for myself which describes allocator behavior, and the container's special members. It is written in English, not standard-eze. If you find any discrepancies between my cheat sheet, and the C++14 working draft, trust the working draft. One known discrepancy is that I've added noexcept specs in ways the standard has not.
Allocator behavior:
C() noexcept(is_nothrow_default_constructible<allocator_type>::value);
C(const C& c);
Gets allocator from alloc_traits::select_on_container_copy_construction(c).
C(const C& c, const allocator_type& a);
Gets allocator from a.
C(C&& c)
noexcept(is_nothrow_move_constructible<allocator_type>::value && ...);
Gets allocator from move(c.get_allocator()), transfers resources.
C(C&& c, const allocator_type& a);
Gets allocator from a. Transfers resources if a == c.get_allocator().
Move constructs from each c[i] if a != c.get_allocator().
C& operator=(const C& c);
If alloc_traits::propagate_on_container_copy_assignment::value is true,
copy assigns allocators. In this case, if allocators are not equal prior
to assignment, dumps all resources from *this.
C& operator=(C&& c)
noexcept(
allocator_type::propagate_on_container_move_assignment::value &&
is_nothrow_move_assignable<allocator_type>::value);
If alloc_traits::propagate_on_container_move_assignment::value is true,
dumps resources, move assigns allocators, and transfers resources from c.
If alloc_traits::propagate_on_container_move_assignment::value is false
and get_allocator() == c.get_allocator(), dumps resources, and transfers
resources from c.
If alloc_traits::propagate_on_container_move_assignment::value is false
and get_allocator() != c.get_allocator(), move assigns each c[i].
void swap(C& c)
noexcept(!allocator_type::propagate_on_container_swap::value ||
__is_nothrow_swappable<allocator_type>::value);
If alloc_traits::propagate_on_container_swap::value is true, swaps
allocators. Regardless, swaps resources. Undefined behavior if the
allocators are unequal and propagate_on_container_swap::value is false.
Custom allocators, and specifically the ability to customise the allocate/deallocate methods are very useful - allowing for control over the low-level memory allocation strategies used in certain performance critical areas. Pooled allocators for node-based containers, and fixed-size allocators are a few examples that come to mind.
But when is it useful to define custom construct/destroy methods?
As I understand it, the standard mandates that these methods have the same effect as placement new and explicit destruction, i.e:
//construct should give the same effect as:
construct (_Type *_ptr, _Type const&_val)
{
new(_ptr) _Type(_val);
}
//destroy should give the same effect as:
destroy (_Type *_ptr)
{
_ptr->~_Type();
}
Is there any way to implement these methods in another way - not using placement new and explicit destruction? If not, why are they part of the allocator object? Wouldn't it be easier if everyone just used placement new and explicit destruction directly?
For an allocator that substitutes a wrapper type around the template argument, and operates on that instead, construct and destroy should call the wrapper constructor and destructor rather than the allocated type.
In C++11, construct forwards any number of arguments to the constructor. A custom allocator could insert arguments at the beginning or end of the list, or otherwise intercept and modify arguments. This would only work when the user uses in-place construction, though.
By the way, you should never use _Type as an identifier; identifiers with leading underscore followed by capital are reserved for the implementation.
I understand that there are 3 general ways to modify the behaviour of new and delete in C++:
Replacing the default new/delete and new[]/delete[]
Overriding or overloading the placement versions (overriding the one with a memory location passed to it, overloading when creating versions which pass other types or numbers of arguments)
Overloading class specific versions.
What are the restrictions for performing these modifications to the behaviour of new/delete?
In particular are there limitations on the signatures that new and delete can be used with?
It makes sense if any replacement versions must have the same signature (otherwise they wouldn't be replacement or would break other code, like the STL for example), but is it permissible to have global placement or class specific versions return smart pointers or some custom handle for example?
First off, don't confuse the new/delete expression with the operator new() function.
The expression is a language construct that performs construction and destruction. The operator is an ordinary function that performs memory (de)allocation.
Only the default operators (operator new(size_t) and operator delete(void *) can be used with the default new and delete expressions. All other forms are summarily called "placement" forms, and for those you can only use new, but you have to destroy objects manually by invoking the destructor. Placement forms are of rather limited and specialised need. By far the most useful placement form is global placement-new, ::new (addr) T, but the behavior of that cannot even be changed (which is presumably why it's the only popular one).
All new operators must return void *. These allocation functions are far more low-level than you might appreciate, so basically you "will know when you need to mess with them".
To repeat: C++ separates the notions of object construction and memory allocation. All you can do is provide alternative implementations for the latter.
When you overload new and delete within a class you are effectively modifying the way the memory is allocated and released for the class, asking for it to give you this control.
This may be done when a class wants to use some kind of pool to allocate its instances, either for optimisation or for tracking purposes.
Restrictions, as with pretty much any operator overload, is the parameter list you may pass, and the behaviour it is expected to adhere to.
I'm currently learning about concurrency in C++ and came across using a vector of threads, which I believe will be possible in C++0x. However, my current compiler doesn't appear to have an implementation of move-aware containers and so I get errors generated because std::thread::thread(const std::thread&) is deleted, ie I can only use the move constructor/move assignment with std::thread.
Am I correct in thinking I could circumvent this issue by writing a custom allocator using
void MyAllocator::construct (pointer p, reference val)
/* should be non-const reference to val because using move constructor? */
{
new ((void*)p) T (std::move(val));
}
rather than
void allocator::construct (pointer p, const_reference val)
{
new ((void*)p) T (val);
}
? Or some other variation on this theme (possibly using an overload of MyAllocator::construct).
NB: This is mainly intended to be a short-term educational exercise and well enough performing work around to play around with threads in containers. I'd only be using MyAllocator in this context. However, please also point me at any libraries that may have this implemented so I can have a poke around the source.
If your compiler doesn't provide a move-aware std::vector then you'll have to write your own specialization of std::vector<std::thread> rather than just provide a custom allocator. The whole C++03 vector interface relies on copying: push_back() copies elements in; resize() initializes the empty elements with a copy of the element passed as the second parameter (even if that is the default value of T()); resize(), reserve(), insert(), erase() and push_back() will copy elements if the vector needs reallocating, or elements otherwise need moving around, and so forth.
This is such a common problem that I've included such a specialization with my (commercial) just::thread implementation of std::thread.
The easiest way to circumvent the problem would be to allocate the threads on the heap and manipulate pointers to them.
Check the Boost Pointer Container library: boost::ptr_vector<std::thread> seems to me what you are looking for.
The requirement that std containers only take copyable objects has more to do with the C++03 container interfaces than it does with the allocator implementation.
For example
vector<T> b(100);
vector<T> a;
a=b;
assert(a==b);
The standard assures us a==b is true. However, if T were not copyable, then in the best case a=b will not compile, in the worst a=b is undefined. Furthermore,
a.push_back(T());
may cause a to allocate new space, and under the hood there are copies made to the new underlying storage from the old.
Furthermore, there is nothing in the C++03 standard that says an implementation actually has to call allocator.construct, and in fact many (gcc for example) do not.
The C++0x standard adds new member functions to the container interface for moveable types, and clarifies how things like operator= behave in their presence.
See www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2486.pdf