I'm reading the VC11 Blog on VC11's C++11 features when I've come up to the SCARY iterators topic.
What are SCARY iterators and how does this affect my C++ coding experience?
If you're using them, there's no need to get SCAREd... just ignore their SCARY-ness.
If you're making them, that means you have to make your iterators independent of the container's allocator type, and of other generic parameters to the container that don't affect the iterators.
From the linked PDF, at http://www.open-std.org/jtc1/sc22/WG21/docs/papers/2009/n2911.pdf
The acronym SCARY describes assignments and initializations that are Seemingly erroneous (appearing Constrained by conflicting generic parameters), but Actually work with the Right implementation (unconstrained bY the conflict due to minimized dependencies).
Related
Why is std::range::sort (and other range-based algorithms) implemented in the range namespace? Why isn't it defined as an overload of std::sort taking a range?
It's to avoid disrupting existing code bases. Eric Niebler, Sean Parent and Andrew Sutton discussed different approaches in their design paper D4128.
3.3.6 Algorithm Return Types are Changed to Accommodate Sentinels
... In similar fashion, most algorithm get new return types when they
are generalized to support sentinels. This is a source-breaking change
in many cases. In some cases, like for_each, the change is unlikely to
be very disruptive. In other cases it may be more so. Merely accepting
the breakage is clearly not acceptable. We can imagine three ways to
mitigate the problem:
Only change the return type when the types of the iterator and the sentinel differ. This leads to a slightly more complicated interface
that may confuse users. It also greatly complicates generic code,
which would need metaprogramming logic just to use the result of
calling some algorithms. For this reason, this possibility is not
explored here.
Make the new return type of the algorithms implicitly convertible to the old return type. Consider copy, which currently returns the
ending position of the output iterator. When changed to accommodate
sentinels, the return type would be changed to something like pair<I, O>; that is, a pair of the input and output iterators. Instead of
returning a pair, we could return a kind of pair that is implicitly
convertible to its second argument. This avoids breakage in some, but
not all, scenarios. This subterfuge is unlikely to go completely
unnoticed.
Deliver the new standard library in a separate namespace that users must opt into. In that case, no code is broken until the user
explicitly ports their code. The user would have to accommodate the
changed return types then. An automated upgrade tool similar to clang
modernize can greatly help here.
We, the authors, prefer (3).
Ultimately, it was to be the least disruptive to existing code bases that move onto building using C++20 enabled compilers. It's the approach they themselves preferred, and seems like the rest is history.
It's a well-known fact that the C++ standard library containers, in general, cannot be instantiated with incomplete types. The result of doing so is UB, although in practice a given implementation will either accept the code without issues or issue a compilation error. Discussion about this restriction can be found here: Why C++ containers don't allow incomplete types?
However, in C++17, there are three containers that explicitly allow incomplete types: std::forward_list (26.3.9.1/4), std::list (26.3.10.1/4), and std::vector (26.3.11.1/4).
This is the result of N4510. The paper notes that "based on the discussion on the Issaquah meeting" the decision was made to, at least at first, limit such support to those three containers. But why?
Because we know how to implement those containers to deal with incomplete types, without breaking the ABI.
std::array, on the other hand, needs to know how big an element is (for example).
But why?
The reason incomplete types weren't allowed in the standard containers was that some containers can work with them, but some don't. They didn't want to think too much about this issue at the time and made a blanket prohibition of incomplete types in all standard containers.
Matt Austern documented that in his great article "The Standard Librarian: Containers of Incomplete Types", which is no longer available, but there are still quotes from it in Boost Containers of Incomplete Types.
This C++17 change does justice by undoing the harm inflicted by that blanket prohibition.
Issue
In C++17, associative containers in standard library will have insert_or_assign member function, which will do what its name suggests. Unfortunately, it seems like it doesn't have iterator based interface for bulk insert/assign. I even tried to compile small example, and from the compiler error, compiler couldn't find suitable overload, and neither candidate was reasonably close to iterator based interface.
Question
Why C++17 didn't include iterator based insert_or_assign for operations in bulk? Was there any technical reasons? Design issues?
My assumptions and ideas
I don't see any technical reason to not add iterator based bulk insertion/addition. It seems quite doable. It needs to lookup the key anyway, so I don't see any violations of "Don't pay for what you don't use".
Actually not having the overload makes standard library less consistent, and kind of going against itself. Plain insert supports it, so I'd expect insert_or_assign to support that too. I don't think that not having the overload will make it "Easier to use correctly, and harder to use incorrectly".
The only clue left is this notice from cppreference:
insert_or_assign returns more information than operator[] and does not require default-constructibility of the mapped type.
I'm not sure why this might be a limitation, as the associative container has access to all of the internals and doesn't need to deal with operator[] at all.
Binary compatibility is not applicable here, if I didn't forget anything. Modification will be in a header, and everything will need to recompile anyway.
I couldn't find any associated paper either. Splicing maps and sets doesn't seem to mention it. The feature looks like a phantom.
Standard could at least include insert_assign_iterator, so that one could write std::copy(input_first, input_last, insert_assign_iterator{map});, but standard includes neither.
insert_or_emplace is intended to be a better form of doing some_map[key] = value;. The latter requires that the mapped_type is default constructible, while insert_or_emplace does not.
What you're talking about is similar, but different. You have some range of key-value pairs, and you want to stick all of those values into the map, whether they have equivalent keys or not. This was not the problem that these functions were created to solve, as evidenced by the original proposal for them.
It's not that they're necessarily a bad idea. It simply wasn't the problem the function was added to solve.
std::set_union and its kin take two pairs of iterators for the sets to be operated on. That's great in that it's the most flexible thing to do. However they very easily could have made an additional convenience functions which would be more elegant for 80% of typical uses.
For instance:
template<typename ContainerType, typename OutputIterator>
OutputIterator set_union( const ContainerType & container1,
const ContainerType & container2,
OutputIterator & result )
{
return std::set_union( container1.begin(), container1.end(),
container2.begin(), container2.end(),
result );
}
would turn:
std::set_union( mathStudents.begin(), mathStudents.end(),
physicsStudents.begin(), physicsStudents.end(),
students.begin() );
into:
std::set_union( mathStudents, physicsStudents, students.begin() );
So:
Are there convenience functions like this hiding somewhere that I just haven't found?
If not, can anyone thing of a reason why it would be left out of STL?
Is there perhaps a more full featured set library in boost? (I can't find one)
I can of course always put my implementations in a utility library somewhere, but it's hard to keep such things organized so that they're used across all projects, but not conglomerated improperly.
Are there convenience functions like this hiding somewhere that I just haven't found?
Not in the standard library.
If not, can anyone thing of a reason why it would be left out of STL?
The general idea with algorithms is that they work with iterators, not containers. Containers can be modified, altered, and poked at; iterators cannot. Therefore, you know that, after executing an algorithm, it has not altered the container itself, only potentially the container's contents.
Is there perhaps a more full featured set library in boost?
Boost.Range does this. Granted, Boost.Range does more than this. It's algorithms don't take "containers"; they take iterator ranges, which STL containers happen to satisfy the conditions for. They also have lazy evaluation, which can be nice for performance.
One reason for working with iterators is of course that it is more general and works on ranges that are not containers, or just a part of a container.
Another reason is that the signatures would be mixed up. Many algorithms, like std::sort have more than one signature already:
sort(Begin, End);
sort(Begin, End, Compare);
Where the second one is for using a custom Compare when sorting on other than standard less-than.
If we now add a set of sort for containers, we get these new functions
sort(Container);
sort(Container, Compare);
Now we have the two signatures sort(Begin, End) and sort(Container, Compare) which both take two template parameters, and the compiler will have problems resolving the call.
If we change the name of one of the functions to resolve this (sort_range, sort_container?) it will not be as convenient anymore.
I agree, STL should take containers instead of iterators-pairs for the following reasons;
Simpler code
Algorithms could be overloaded for specified containers, ie, could use the map::find algorithm instead of std::find -> More general code
A subrange could easily be wrapped into a container, as is done in boost::range
#Bo Persson has pointed to a problem with ambiguity, and I think that's quite valid.
I think there's a historical reason that probably prevented that from ever really even being considered though.
The STL was introduced into C++ relatively late in the standardization process. Shortly after it was accepted, the committee voted against even considering any more new features for addition into C++98 (maybe even at the same meeting). By the time most people had wrapped their head around the existing STL to the point of recognizing how much convenience you could get from something like ranges instead of individual iterators, it was too late to even be considered.
Even if the committee was still considering new features, and somebody had written a proposals to allow passing containers instead of discrete iterators, and had dealt acceptably with the potential for ambiguity, I suspect the proposal would have been rejected. Many (especially the C-oriented people) saw the STL as a huge addition to the standard library anyway. I'm reasonably certain quite a few people would have considered it completely unacceptable to add (lots) more functions/overloads/specializations just to allowing passing one parameter in place of two.
Using the begin & end elements for iteration allows one to use non-container types as inputs. For example:
ContainerType students[10];
vector<ContainerType> physicsStudents;
std::set_union(physicsStudents.begin(), physicsStudents.end(),
&students[0], &students[10],
physicsStudents.begin());
Since they are such simple implementations, I think it makes sense not to add them to the std library and allow authors to add their own. Especially given that they are templates, thus potentially increasing the lib size of the code and adding convenience functions across std would lead to code bloat.
Can/Should i inherit from STL iterator to implement my own iterator class? If no, why not?
Short answer
Many consider that the class std::iterator does not offer much compared to regular type aliases, and even obfuscates them a bit by not explicitly providing the names and relying on the order of the template parameters instead. It is deprecated in C++17 and is likely to be gone in a few years.
This means that you shouldn't use std::iterator anymore. You can read the whole post below if you're interested in the full story (there's a bit of redundancy since it has been started before the deprecation proposal).
Legacy answer
You can ignore everything below if you're not interested in history. The following fragments even contradict themselves several times.
As of today (C++11/C++14), the standard seems to imply that it isn't a good idea anymore to inherit from std::iterator to implement custom iterators. Here is a brief explanation, from N3931:
Although the Standard has made this mistake almost a dozen times, I recommend not depicting directory_iterator and recursive_directory_iterator as deriving from std::iterator, since that's a binding requirement on implementations. Instead they should be depicted as having the appropriate typedefs, and leave it up to implementers to decide how to provide them. (The difference is observable to users with is_base_of, not that they should be asking that question.)
[2014-02-08 Daniel comments and provides wording]
This issue is basically similar to the kind of solution that had been used to remove the requirement to derive from unary_function and friends as described by N3198 and I'm strongly in favour to follow that spirit here as well. I'd like to add that basically all "newer" iterator types (such as the regex related iterator) don't derive from std::iterator either.
The paper cites N3198 which itself states that it follows the deprecation discussed in N3145. The reasons for deprecating the classes that only exist to provide typedefs are given as such:
Our experience with concepts gives us confidence that it is rarely necessary to depend on specific base class-derived class relations, if availability of types and functions is sufficient. The new language tools allow us even in the absence of language-supported concepts to deduce the existence of typenames in class types, which would introduce a much weaker coupling among them. Another advantage of replacing inheritance by associated types is the fact, that this will reduce the number of cases, where ambiguities arise: This can easily happen, if a type would inherit both from unary_function and binary_function (This makes sense, if a functor is both an unary and a binary function object).
tl;dr: classes which only provide typedefs are now deemed useless. Moreover, they increase coupling when it is not needed, are more verbose, and can have unwanted side effects in some corner cases (see the previous quotation).
Update: issue 2438 from N4245 seems to actually contradict what I asserted earlier:
For LWG convenience, nine STL iterators are depicted as deriving from std::iterator to get their iterator_category/etc. typedefs. Unfortunately (and unintentionally), this also mandates the inheritance, which is observable (not just through is_base_of, but also overload resolution). This is unfortunate because it confuses users, who can be misled into thinking that their own iterators must derive from std::iterator, or that overloading functions to take std::iterator is somehow meaningful. This is also unintentional because the STL's most important iterators, the container iterators, aren't required to derive from std::iterator. (Some are even allowed to be raw pointers.) Finally, this unnecessarily constrains implementers, who may not want to derive from std::iterator. (For example, to simplify debugger views.)
To sum up, I was wrong, #aschepler was right: it can be used, but it is certainely not required - it isn't discouraged either. The whole "let's remove std::iterator" thing exists for the standard not to constrain the standard library implementers.
Round 3: P0174R0 proposes to deprecate std::iterator for a possible removal in the future. The proposal is already pretty good at explaining why it should be deprecated, so here we go:
The long sequence of void arguments is much less clear to the reader than simply providing the expected typedefs in the class definition itself, which is the approach taken by the current working draft, following the pattern set in C++14 where we deprecated the derivation throughout the library of functors from unary_function and binary_function.
In addition to the reduced clarity, the iterator template also lays a trap for the unwary, as in typical usage it will be a dependent base class, which means it will not be looking into during name lookup from within the class or its member functions. This leads to surprised users trying to understand why the following simple usage does not work:
#include <iterator>
template <typename T>
struct MyIterator : std::iterator<std::random_access_iterator_tag, T> {
value_type data; // Error: value_type is not found by name lookup
// ... implementations details elided ...
};
The reason of clarity alone was sufficient to persuade the LWG to update the standard library specification to no longer mandate the standard iterator adapators as deriving from std::iterator, so there is no further use of this template within the standard itself. Therefore, it looks like a strong candidate for deprecation.
This is becoming a bit tiring and not everyone seems to agree, so I will let you draw your own conclusions. If the committee eventually decides that std::iterator should be deprecated, then it will make it pretty clear that you shouldn't use it anymore. Note that the follow-up paper highlights a great support for the removal of std::iterator:
Update from Jacksonville, 2016:
Poll: Deprecate iterator for C++17??
SF F N A SA
6 10 1 0 0
In the above poll results, SF, F, N, A and SA stand for Strongly For, For, Neutral, Against and Strongly Against.
Update from Oulu, 2016:
Poll: Still want to deprecate std::iterator?
SF F N A SA
3 6 3 2 0
P0619R1 proposes to remove std::iterator, possibly as soon as C++20, and also proposes to enhance std::iterator_traits so that it can automatically deduce the types difference_type, pointer and reference the way std::iterator does when they're not explicitly provided.
If you mean std::iterator: yes, that's what it's for.
If you mean anything else: no, because none of the STL iterators have virtual destructors. They're not meant for inheritance and a class inheriting from them might not clean up properly.
No one should not because of the potential problems that might be encountered. Probably you are better off using Composition rather than Inheritance with STL Iterators.
Undefined Behavior due to absence of virtual destructors:
STL containers & iterators are not meant to act as base classes as they do not have virtual destructors.
For classes with no virtual destructors being used as Base class, the problem arises when deallocating through a pointer to the base class (delete, delete[] etc). Since the classes don't have virtual destructors, they cannot be cleaned up properly and results in Undefined Behavior.
One might argue that there would not be a need to delete the iterator polymorphically & hence nothing wrong to go ahead with deriving from STL iterators, well there might be some other problems like:
Inheritance maynot be possible at all:
All iterator types in the standard container are Implementation defined.
For e.g: std::vector<T>::iterator might be just a T*. In this case, you simply cannot inherit from it.
The C++ standard has no provisions demanding that say std::vector<T>::iterator does not
use inheritance inhibiting techniques to prevent derivation. Thus, if you are deriving from a STL iterator you are relying on a feature of your STL that happens to allow derivation. That makes such an implementation non portable.
Buggy behaviors if not implemented properly:
Consider that you are deriving from vector iterator class like:
class yourIterator : std::vector<T>::iterator { ... };
There might be a function which operates on the vector iterators,
For ex:
void doSomething(std::vector<T>::iterator to, std::vector<T>::iterator from);
Since yourIterator is a std::vector<T>::iterator you can call doSomething() on your container class but you will be facing the ugly problem of Object Slicing. The doSomething() has to be implemented in a proper templated manner, to avoid the
problem.
Problems while using Standard Library Algorithms:
Consider you are using the derivation from vector iterator, and then you use a Standard library algorithm like std::transform()
For Ex:
yourIterator a;
yourIterator b;
...
std::transform( a++, b--, ... );
The postfix operator ++ returns a std::vector<T>::iterator and not a
yourIterator resulting in a wrong template being chosen.
Thus, Inheriting from STL Iterators is possible indeed but if you ready to dig out all such and many other potential problems and address them, Personally I won't give it the time and the effort to do so.
If you're talking about std::iterator template, then yes, you should, but I hope you do understand that it has no functionality, just a bunch of typedefs. The pro of this decision is that your iterator can be fed to iterator_traits template.
If, on the other hand, you're talking about some specific STL iterator, like vector<T>::iterator or other, then the answer is a resounding NO. Let alone everything else, you don't know for sure that it's actually a class (e.g. the same vector<T>::iterator can be just typedefed as T*)