What is "Per-object Data in Allocators". I can't seem to find what this means. Anyone have a good explanation or link for what this means in terms of the C++ Language?
CLARIFICATION
Section 19.4.2 "The C++ Programming Language (Special Edition)" pg. 573
"Similarly, if allocators were allowed to be perfectly general, the rebind mechanism that allows an allocator to allocate elements of arbitrary types would have to be more elaborate. Consequently, a standard allocator is assumed to hold no per-object data and an implementation of a standard container may take advantage of that."
Per-object data or local state refers to any non-static data members in the allocator class.
The issue is that currently (in c++03) there is no support for allocators with so-called local state. This is often considered to be a flaw with the allocator model in present-day c++.
Have a read through this article that details the design for a custom allocator. A paragraph under Design specifically addresses some of the pitfalls of allocators that incorporate local state.
Briefly, some operations in the standard library currently require that objects of a particular type be safely allocated by one instance of an allocator and deallocated by another instance of the allocator (both allocators are of the same type - of course!). This can be the case when implementing list::splice for instance. If allocators are allowed to have local state this can get tricky...
In the upcoming c++0x revision, it appears that allocators will be allowed to incorporate local state, check out the scoped allocator section here.
Hope this helps.
It simply means that std::allocator<T> does not contain any per-instance data members ... it is mainly a wrapper around memory allocation and deallocation functions, and also contains definitions of certain required typedefs, as well as mechanisms for rebinding an existing allocator so that it can allocate types that were not part of the original allocator template instantiation. So basically what is being stated is that if there were actual private data-members that had to be managed, especially in light of the requirement for a STL allocator to allow rebinding, that could greatly complicate the implementation of a generic allocator depending on what those per-instance data-members represented.
Related
I am reading Item 10 in Effective STL by Scott Meyers on allocators in C++.
Standard says that an implementation of the STL is permitted to assume
that all allocator objects of the same type are equivalent and always
compare equal.
That's all well and good, but the more you think about it. the more
you'll realize just how draconian a restriction it is that STL
implementations may assume that allocators of the same type are
equivalent. It means that portable allocator objects — allocators that
will function correctly under different STL implementations — may not
have state. Let's be explicit about this: it means that portable
allocators may not have any nonstatic data members, at least not any
that affect their behavior. None. Nada. That means, for example, you
can't have one SpecialAllocator that allocates from one heap and
a different SpecialAllocator that allocates from a different
heap. Such allocators wouldn't be equivalent, and STL implementations
exist where attempts to use both allocators could lead to corrupt
runtime data structures.
In fairness to the Standardization Committee, I should point out that
it included the following statement immediately after the text that
permits STL implementers to assume that allocators of the same type
are equivalent:
Implementors are encouraged to supply libraries that ... support
non-equal instances. In such implementations. ... the semantics of
containers and algorithms when allocator instances compare non-equal
are implementation-defined.
This is a lovely sentiment, but as a user of the STL who is
considering the development of a custom allocator with state, it
offers you next to nothing. You can take advantage of this statement
only if (1) you know that the STL implementations you are using
support inequivalent allocators, (2) you are willing to delve into
their documentation to determine whether the implementation-defined
behavior of "non-equal" allocators is acceptable to you, and
(3) you're not concerned about porting your code to STL
implementations that may take advantage of the latitude expressly
extended to them by the Standard. In short, this paragraph — paragraph
5 of section 20.1.5. for those who insist on knowing — is the
Standard's "1 have a dream" speech for allocators. Until that dream
becomes common reality, programmers concerned about portability will
limit themselves to custom allocators with no state.
My question on above paragraph are
What does author mean by inequivalent or non-equal allocators?
What does last paragraph in above text i.e, point 3 mean in simple terms?
That information is out of date. C++11 and later versions support stateful allocators.
The quotes you have posted from Effective C++ are only of concern if you are writing a C++ library which requires custom allocators, does not require C++11, and which supports building against unknown/unspecified standard libraries. To a first approximation, nobody is doing this anymore. The people who were doing it before often had their own "enhanced" standard library implementations to support stateful allocators, such as EASTL or BDESTL.
Two allocators should compare equal if memory allocated by one can be freed by the other. So, for example, an allocator object that allocates from a pool that it holds can allocate and free memory from that pool, but a different allocator object that has a different pool can't (without a great deal of extra bookkeeping) free memory allocated by the first object.
Making that work right was beyond what the standards committee wanted to take on when allocators were first introduced, which is why the words were so squishy. And that license has been revoked in more recent versions of the standard.
The last paragraph means that if you write an allocator whose objects rely on internal state (e.g., the pool allocator that I mentioned above), and the library you're using respects the equality operator (as in, it won't try to pass pointers around among allocators that don't compare equal), your code will break when you try to use it with a different implementation that doesn't pay attention to the equality operator.
In most places where the C++ standard library allocates memory, the user is able to customise this by providing a class which meets the Allocator requirements. For example, almost all containers take an allocator template argument, and std::allocate_shared returns a shared_ptr whose contained element and control block are both allocated via a provided Allocator.
However, there are a few places where the standard library can (potentially) allocate memory, but no Allocator support is provided. The ones I can think of are:
std::make_unique() (no corresponding allocate_unique())
std::any
std::function (allocator support will be removed in C++17)
std::valarray
std::basic_filebuf (although std::basic_stringbuf does use an Allocator)
std::inplace_merge()
Questions:
I'm sure this list is incomplete, but what else have I missed?
Of the non-Allocator classes and functions, are they specified to use global ::operator new, plain new, or is the memory source unspecified?
If anybody knows, what are the reasons for not providing allocator support in any, and removing it from function?
Not an exhaustive list.
Everything in <stdexcept>, which needs to allocate memory to store the message string.
The standard pool/monotonic memory resource classes obviously allocate memory, but they do it from another memory resource, not an allocator, so they technically fit your description.
boyer_moore_searcher and boyer_moore_horspool_searcher.
Several <algorithm> algorithms attempt to obtain additional memory (e.g., stable_partition, stable_sort) and all parallel algorithms (regardless of header) may need additional memory.
Many things in the filesystem library:
path; that one used to use the default allocator internally, but looks like the newest draft removed that requirement, although it seems to be the intent still.
directory_iterator and recursive_directory_iterator.
Some filesystem operations that can construct temporary paths .
basic_regex.
thread and async.
packaged_task, allocator support removed in C++17.
promise will need somewhere to put the shared state.
iostreams.
Many things are hard-coded to use the default allocator:
error_code::message(), error_condition::message(), error_category::message(): these return a string, so default allocator only.
bitset's stream insertion operator notionally calls to_string with the default allocator, though no high-quality implementation would do that in practice.
The to_string and to_wstring free functions return std::string/std::wstring respectively; no chance to specify your own allocator. Obviously the user-defined literals for string (e.g., "foo"s) also have no custom allocator support.
There are several facets in <locale> that have member functions returning either std::string or std::basic_string<charT>, i.e., using the default allocator (e.g., numpunct), or accepting only basic_string<charT> (e.g., money_get).
Various types in <random> uses a vector with the default allocator.
If anybody knows, what are the reasons for not providing allocator
support in any, and removing it from function?
any's allocator support is unimplementable. function's allocator support is poorly specified and plagued with issues.
The c++17 specification deprecates the construct and destroy members of the std::allocator object. The working group provided rationale for deprecating other member functions here, under the heading "Deprecate the redundant members of std::allocator".
However they don't mention specifically why those two members are deprecated or what the recommendation is for replacing that functionality. I'm assuming the implication is to use std::allocator_traits::construct instead.
I'm a bit confused about whether implementing construct may actually still be necessary in some cases though because of this comment about std::allocator_traits::construct
Because this function provides the automatic fall back to placement new, the member function construct() is an optional Allocator requirement since C++11.
For custom allocators (e.g. for page-aligned memory using memalign), will falling back to placement new always produce the correct behavior?
The allocator requirements table says that construct(c, args), if provided, must "construct an object of type C at c".
It says absolutely nothing about 1) what arguments are to be passed to C's constructor or 2) how these arguments are to be passed. That's the allocator's choice, and in fact two allocators in the standard do mess with the arguments before passing them to C's constructor: std::scoped_allocator_adaptor and std::pmr::polymorphic_allocator. When constructing a std::pair, in particular, the arguments they pass to pair's constructor may not even resemble the ones they received.
There's no requirement to perfectly forward, either; a C++03-style construct(T*, const T&) is conforming if inefficient.
std::allocator's construct and destroy are deprecated because they are useless: no good C++11 and later code should ever call them directly, and they add nothing over the default.
Handling memory alignment should be the task of allocate, not construct.
The functions were removed along with others from the paper D0174R0 Deprecating Vestigial Library Parts in C++17. If we look at the relevant section we have
Many members of std::allocator redundantly duplicate behavior that is otherwise produced by std::allocator_traits<allocator<T>>, and could safely be removed to simplify this class. Further, addressof as a free function supersedes std::allocator<T>::address which requires an allocator object of the right type. Finally, the reference type aliases were initially provided as an expected means for extension with other allocators, but turned out to not serve a useful purpose when we specified the allocator requirements (17.6.3.5 [allocator.requirements]).
While we cannot remove these members without breaking backwards compatibility with code that explicitly used this allocator type, we should not be recommending their continued use. If a type wants to support generic allocators, it should access the allocator's functionality through allocator_traits rather than directly accessing the allocator's members - otherwise it will not properly support allocators that rely on the traits to synthesize the default behaviors. Similarly, if a user does not intend to support generic allocators, then it is much simpler to directly invoke new, delete, and assume the other properties of std::allocator such as pointer-types directly.
Emphasis mine
So the rational was we do not need to duplicate all of the code in allocator since we have the allocator traits. If we look at std::allocator_traits we will see that it does have
allocate
deallocate
construct
destroy
max_size
static functions so we can use those instead of the ones in the allocator.
First of all, I don't think it is. But, I've observed such a behavior with MSVC 10.0 in Debug mode. I'm using a custom allocator class which relies on the user to pass only pointers allocated on the same instance to deallocate. However, in Release mode, my code is working.
Is this a bug or am I mistaken?
The standard requires that any allocator be able to deallocate memory produced by any other allocator of the same type, even if it's a totally different instance. This is required to get list::splice working correctly. It's largely considered a design flaw in the C++ spec, and in C++0x they're introducing a set of fixups to allocators to rememdy this. In the meantime, any allocator you use in the STL containers must not have its own local state.
EDIT: For those of you who want the original language on this, here's §20.1.5/4 of the C++ ISO spec:
Implementations of containers described in this International Standard are permitted to assume that their
Allocator template parameter meets the following two additional requirements beyond those in Table 32.
— All instances of a given allocator type are required to be interchangeable and always compare equal to
each other.
In the latest ISO draft of the C++0x standard, this requirement is no longer present. The default std::allocator will still maintain this invariant as required, but it doesn't look like you'll have to constrain yourself this way in the future.
Why isn't standard C++03 interface for querying member types for allocators used in C++0x? What are the use cases where member types are not sufficient?
To explain allocator_traits in terms of design pattern, its an Adapter to wrap your custom Allocator that meets far less implementation requirement (no need for construct, destroy, all those typedefs ...) and turns it into FlyWeight object that completes the rest of the Allocator implementation requirement for you with static members and types.
With allocator_traits, you just need to provide minimum of 10 lines of code for your custom allocator, as per page 3 of open-std doc Scoped Allocator Model (thx to #icecrime for mentioning).
I think of allocator_traits and allocator as a nice real-world example of turning non-FlyWeight object into FlyWeight in order to relieve the burden of implementaion details. Its a good API design practice for turning a class into FlyWeight that should've been FlyWeight in the first place.
To Java Programmers, in terms of design pattern, std::allocator_traits is like package private classes CharacterDataLatin1, ChracterData00, CharacterData0E, 01, 02 ... that inherits from java.lang.CharacterData to provide static Unicode definition and helpers that are to be shared by every instance of Character (std::allocator) classes.
Edit:
Another advantage of indirectly calling your custom allocator through allocator_traits is that it guarantees foward compatibility (page 3 of Scoped Allocator Model).
Number of requirements may grow in the future, and even if your ignorant on implementing new requirements to your allocator, those requirements will already be there in allocator_traits implemented by the compiler manufacturer. Knowing that C++ containers call allocator indirectly by allocator_traits, the STL containers using your custom allocator will be benefiting from the new requirements without the need for you to change your code.
I'm not familiar with this kind of things (at all), but this document seems like a good starting point to get a grasp on the rationale behind allocator_traits :
The keystone of this proposal is the
definition of an allocator_traits
template containing types and static
member functions for using allocators,
effectively replacing the Allocator concept that was lost in
Frankfurt.