I'm writing a custom container template class that, as many (if not all) the containers in the stl, can use a specified allocator type.
To implement the range insert function, I need to move some of the elements in the container a number of spaces forward, where the memory is still uninitialized. To do this I want to use some (nonexistent) version of std::uninitialized_move() that uses the allocator in the container.
The other option is to do the move-construction of the objects using the allocator in a for loop and destroy the constructed objects in case of an exception. That's basically re-implementing std::uninitialized_move() with an extra feature.
The standard library implementation for my compiler (GCC) has exactly the functions I need (std::__uninitialized_move_a(), std::__uninitialized_copy_a(), etc), and are in fact used in the implementation of std containers, but I think those are rather compiler-specific.
Should I use this functions (portability)? Or is there other, more practical, option?
Maybe there is something in the standard library that I'm missing.
Should I use this functions (portability)?
You shouldn't use the GCC internal functions.
No, there doesn't seem to be standard equivalents. You can write your own versions of those functions. If you do, note that CustomAlloc::construct is an optional function (for example, std::allocator doesn't have this function since C++20), so it should be use through std::allocator_traits<CustomAlloc>::construct. This has additional benefit of being constexpr since C++20.
Or is there other, more practical, option?
One option is to ignore the possibility that CustomAlloc::construct has been implemented to do something other than direct placement new, and thereby simply use the standard std::uninitialized_move.
This technically limits the allocators that your container supports in theory, but on the other hand I've never seen such custom allocator used in practice. This may be reasonable limitation if the container is for internal use at least. If you do this, document the behaviour carefully.
Related
I'm learning the c++ STL and it came to my attention that while most functionalities supported by std::vector and std::array (contiguous storage) are marked with constexpr, that's not the case for std::deque and other non-contiguous storages.
So I spent some time doing some research, I found a proposal in 2019, Making std::deque constexpr, and std::deque still has not implemented constexpr for its methods.
My confusion is that std::array guarantees that its elements are stored on the stack; just like a normal C-style array, so it should be computed at compile time, but std::vector allocates memory on the heap so if it's evaluated at compile time, so is deque, right?
Thanks!
According to https://github.com/cplusplus/papers/issues/665 which keeps a log of the progression of the proposal through the standards committee process, there seem to have been some doubts whether a constexpr std::deque can be implemented without core language changes.
Unfortunately it doesn't say what the specific concern is. Probably some common implementation makes use of some language construct that specifically is not allowed in constant expressions or the implementation relies on some construct that is undefined behavior according to the standard. The latter is usually not a problem for the standard library, since it is not bound by the language rules and can make assumptions about the particular compiler's behavior. However in constant expressions core language undefined behavior is always a hard error and therefore such constructs might often not be usable in constant expression contexts without introducing magic compiler workarounds.
As mentioned in the linked github issue, there seem to also be some library facilities which need to have constexpr added to make this work as well.
Aside from such issues, generally, I don't think there is any reason to not make all containers and container adaptors constexpr-friendly now that std::allocator can be used in constant expressions. They probably just want to make sure that they can be properly implemented as constexpr first. My guess is that for the same reason only std::string and std::vector were done with C++20, since these are the simplest and most important allocator-aware containers to apply constexpr to. (std::array has been constexpr for longer since it doesn't require any dynamic allocations.)
Although, looking at the date of the last entry in the issue (as well as the accompanying issues for std::list, std::priority_queue, etc.) it seems to not have progressed in the last two years, maybe because the author of the proposal did not pursue it further, but I can't really tell.
In either case, when we say that std::vector (or other allocator-aware containers) are constexpr-friendly, this means something else than it does for e.g. std::array. You can declare a constexpr std::array variable and use it as you would use a const std::array variables, but you can't declare a constexpr std::vector variable (generally at all) and use it as you would a const std::vector.
What you can do however is use a std::vector variable in e.g. a constexpr function to do some calculations, as long as the variable is created after the evaluation of the constant expression starts and is destroyed before it ends. Currently this is not possible with e.g. a std::list which is not constexpr-friendly.
The reason for this is that the compiler will not actually allocate memory at compile-time for the container and then transfer it in some way into a runtime allocation (whether static or dynamic). Instead dynamic allocations at compile-time are separate from runtime ones and must be deallocated before the constant expression evaluation in which they were allocated ends.
In most places where the C++ standard library allocates memory, the user is able to customise this by providing a class which meets the Allocator requirements. For example, almost all containers take an allocator template argument, and std::allocate_shared returns a shared_ptr whose contained element and control block are both allocated via a provided Allocator.
However, there are a few places where the standard library can (potentially) allocate memory, but no Allocator support is provided. The ones I can think of are:
std::make_unique() (no corresponding allocate_unique())
std::any
std::function (allocator support will be removed in C++17)
std::valarray
std::basic_filebuf (although std::basic_stringbuf does use an Allocator)
std::inplace_merge()
Questions:
I'm sure this list is incomplete, but what else have I missed?
Of the non-Allocator classes and functions, are they specified to use global ::operator new, plain new, or is the memory source unspecified?
If anybody knows, what are the reasons for not providing allocator support in any, and removing it from function?
Not an exhaustive list.
Everything in <stdexcept>, which needs to allocate memory to store the message string.
The standard pool/monotonic memory resource classes obviously allocate memory, but they do it from another memory resource, not an allocator, so they technically fit your description.
boyer_moore_searcher and boyer_moore_horspool_searcher.
Several <algorithm> algorithms attempt to obtain additional memory (e.g., stable_partition, stable_sort) and all parallel algorithms (regardless of header) may need additional memory.
Many things in the filesystem library:
path; that one used to use the default allocator internally, but looks like the newest draft removed that requirement, although it seems to be the intent still.
directory_iterator and recursive_directory_iterator.
Some filesystem operations that can construct temporary paths .
basic_regex.
thread and async.
packaged_task, allocator support removed in C++17.
promise will need somewhere to put the shared state.
iostreams.
Many things are hard-coded to use the default allocator:
error_code::message(), error_condition::message(), error_category::message(): these return a string, so default allocator only.
bitset's stream insertion operator notionally calls to_string with the default allocator, though no high-quality implementation would do that in practice.
The to_string and to_wstring free functions return std::string/std::wstring respectively; no chance to specify your own allocator. Obviously the user-defined literals for string (e.g., "foo"s) also have no custom allocator support.
There are several facets in <locale> that have member functions returning either std::string or std::basic_string<charT>, i.e., using the default allocator (e.g., numpunct), or accepting only basic_string<charT> (e.g., money_get).
Various types in <random> uses a vector with the default allocator.
If anybody knows, what are the reasons for not providing allocator
support in any, and removing it from function?
any's allocator support is unimplementable. function's allocator support is poorly specified and plagued with issues.
The c++17 specification deprecates the construct and destroy members of the std::allocator object. The working group provided rationale for deprecating other member functions here, under the heading "Deprecate the redundant members of std::allocator".
However they don't mention specifically why those two members are deprecated or what the recommendation is for replacing that functionality. I'm assuming the implication is to use std::allocator_traits::construct instead.
I'm a bit confused about whether implementing construct may actually still be necessary in some cases though because of this comment about std::allocator_traits::construct
Because this function provides the automatic fall back to placement new, the member function construct() is an optional Allocator requirement since C++11.
For custom allocators (e.g. for page-aligned memory using memalign), will falling back to placement new always produce the correct behavior?
The allocator requirements table says that construct(c, args), if provided, must "construct an object of type C at c".
It says absolutely nothing about 1) what arguments are to be passed to C's constructor or 2) how these arguments are to be passed. That's the allocator's choice, and in fact two allocators in the standard do mess with the arguments before passing them to C's constructor: std::scoped_allocator_adaptor and std::pmr::polymorphic_allocator. When constructing a std::pair, in particular, the arguments they pass to pair's constructor may not even resemble the ones they received.
There's no requirement to perfectly forward, either; a C++03-style construct(T*, const T&) is conforming if inefficient.
std::allocator's construct and destroy are deprecated because they are useless: no good C++11 and later code should ever call them directly, and they add nothing over the default.
Handling memory alignment should be the task of allocate, not construct.
The functions were removed along with others from the paper D0174R0 Deprecating Vestigial Library Parts in C++17. If we look at the relevant section we have
Many members of std::allocator redundantly duplicate behavior that is otherwise produced by std::allocator_traits<allocator<T>>, and could safely be removed to simplify this class. Further, addressof as a free function supersedes std::allocator<T>::address which requires an allocator object of the right type. Finally, the reference type aliases were initially provided as an expected means for extension with other allocators, but turned out to not serve a useful purpose when we specified the allocator requirements (17.6.3.5 [allocator.requirements]).
While we cannot remove these members without breaking backwards compatibility with code that explicitly used this allocator type, we should not be recommending their continued use. If a type wants to support generic allocators, it should access the allocator's functionality through allocator_traits rather than directly accessing the allocator's members - otherwise it will not properly support allocators that rely on the traits to synthesize the default behaviors. Similarly, if a user does not intend to support generic allocators, then it is much simpler to directly invoke new, delete, and assume the other properties of std::allocator such as pointer-types directly.
Emphasis mine
So the rational was we do not need to duplicate all of the code in allocator since we have the allocator traits. If we look at std::allocator_traits we will see that it does have
allocate
deallocate
construct
destroy
max_size
static functions so we can use those instead of the ones in the allocator.
So I know the point of iterators is to abstract the underlying container so you don't have to worry about what it is.
But say you wanted to write an optimized version of merge sort and wanted to do an in place sort if the underlying container was a linked list, since you can run merge sort in-place on a linked list without the need for extra container allocations.
Is there any way to get this information to know whether you're operating on a linked structure and/or access the pointers for Standard Library and other containers?
I'm assuming there is a way since that is what std::sort does? How?
I have also always wished for a way to do this, but the answer is kind-of, but not really. This isn't possible in general, because of the template deduction rules.
Each iterator may be a member of a class, but since multiple classes may have the same iterators, if a function receives these iterators, it is literally and theoretically impossible to deduce which container it came from. That hinders things somewhat.
If you don't need the container type and can work with the iterator type alone, then yes, std::sort could optimize based on the underlying container. But no, std::sort doesn't have a special algorithm for node based containers in any C++ standard library that I know of.
The containers themselves sometimes have specialized versions, see std::list<T>::sort, but the generic std::sort can't make use of that since it works on any arbitrary range, and std::list<T>::sort only works on the entire container.
I really wish they would. Also a specialization of std::lower_bound and similar when called on the tree-based containers would be awesome.
A library implementor can do a certain set of things that are not portable and depend on the details of the implementation. While in the standard the iterator types are referred as nested types in the container, nested types are not deducible, the implementation can decide to implement those as types with namespace scope, which are deducible, and provide a nested typedef. The implementation could then, within the limits of the requirements placed in the standard, optimize the operations.
I don't know of any implementation that implements std::sort as merge sort for lists, but there are other tricks that are used in implementations, for example std::copy when the iterators are into vectors holding POD types is implemented in some libraries by calling std::memmove rather than copying one element at a time.
As I mentioned before, while this can be done by the implementor, writing your own code attempting to do the same would be non-portable and brittle, as it would depend on details of one implementation that might change in the next version of the library or even by changing compiler flags.
I was doing some hacking today and found out that std::priority_queue does not have a clear() member function. Are there any technical reasons as to why the standards committee may have left this out?
To be clear, I am aware that it is easy to work around this via assignment:
oldPQ = std::priority_queue<int>{};
This solution is less desirable because:
It requires you to repeat the type - this does not continue to work under maintenance. As #chris pointed out below, you can simplify this if you're using the default constructor, but if you have a custom comparator, this may not be possible.
std::priority_queue cannot be used in a templated function that expects a clear() member function.
It is generally undesirable that it does not meet the common interface provided by the other containers. In particular, everything from std::forward_list to std::unordered_map to std::string has clear(). The only other exceptions I note are std::array, for which the semantics would not make sense, and std::stack and std::queue, for which the semantics are more questionable when std::deque works without any extra effort.
One item that looks like an issue, but in practice needn't be:
Because the internal container used for std::priority_queue is templated and may not have a clear() member function of its own, this creates an interesting problem, in particular it raises the question of backward compatibility. This is a non-issue because:
For internal containers that do not provide clear(), as long as nobody attempts to invoke std::priority_queue::clear(), the code will continue to compile.
It may still be possible with SFINAE to provide the new interface (the clear member) by calling clear() on the internal container when it's available and by repeatedly popping if it is not.
It is my opinion that this is a defect in the C++ standard. Assuming a technical discussion does not provide a strong case for why this method is omitted, I intend to pursue the creation of a standards proposal.
Edit:
Seems this is being handled in-committee (note the last post): https://groups.google.com/a/isocpp.org/forum/?fromgroups#!searchin/std-discussion/clear/std-discussion/_mYobAFBOrM/ty-2347w1T4J
http://wg21.cmeerw.net/lwg/issue2194
The specification of container adaptors is known to be overly pedantic: since the "abstract" spec of the corresponding data structure (from some book on abstract algorithms and data structures) does not include operation clear for canonical priority queues or stacks, it is not provided in the adaptor. This indeed often makes it quite inconvenient to use these adaptors in practice.
The good news though is that the inner container member is declared inside the adaptor as a protected member of the adapter, named c. This is probably done specifically for you to be able to easily implement your own version of the adaptor: by inheriting from the standard adaptor and adding whatever member functions you want to add, including clear.
As for comparing these adaptors' interfaces with standard container interfaces... I don't think it is a valid comparison. These adaptors have never been intended to be compatible with containers in terms of interface. Quite the opposite, the purpose of these adaptors was largely to restrict the public interface of the data structure and force it into the narrow bounds of what is allowed by its canonical abstract definition.
For example, you are not allowed to iterate over the canonical stack. Stack, by definition, is not "iterable". The fact that stack adaptor disables iteration interface is a good thing. But the absence of clear certainly feels too pedantic, since it has a great practical value without looking like a big violation of the canonical interface.