1.
In the current standard draft the specification of std::stable_partition is:
template<class BidirectionalIterator, class Predicate>
BidirectionalIterator stable_partition(
BidirectionalIterator first, BidirectionalIterator last, Predicate pred);
I didn't find the requirement that BidirectionalIterator should be a bidirectional iterator, but the name suggests so. (See below)
2.
In the SGI STL, the specification is:
template <class ForwardIterator, class Predicate>
ForwardIterator stable_partition(
ForwardIterator first, ForwardIterator last, Predicate pred);
Requirements on types: ForwardIterator is a model of Forward Iterator.
The specification of complexity is the same for both, current standard and SGI, versions: at most N log N swaps, only O(N) swaps if there is enough extra memory and exactly N applications of the predicate and projection.
3.
The declaration in libstdc++ and libc++ looks like:
template<typename ForwardIterator, typename Predicate>
ForwardIterator stable_partition(
ForwardIterator first, ForwardIterator last, Predicate pred);
In GCC and Clang std::stable_partition indeed works with forward iterators. For example:
int main() {
std::forward_list<int> list{1, 4, 5, 2, 3, 0};
std::stable_partition(list.begin(), list.end(), [](int i) { return i < 3;});
for (auto v : list)
std::cout << v << ' ';
}
compiles and produces the correct output. Microsoft's compiler fails to compile this code (no -- operator). Intel's one succeeds.
I have two related questions:
Does std::stable_partition accepts at least bidirectional iterators by the standard or the name BidirectionalIterator is misleading?
If it indeed accepts only bidirectional iterators, why the support of forward iterators has been dropped?
Edit.
Found this clause:
If an algorithm's template parameter is named BidirectionalIterator, BidirectionalIterator1, or BidirectionalIterator2, the template argument shall meet the Cpp17BidirectionalIterator requirements.
So, only the second question remains.
First of all, no support has been dropped, std::stable_partition has always required BidirectionalIterator by the standard. Which does not mean that the implementors of the library are disallowed to give less restrictions on input arguments (if it continues to comply with other parts of the standard ofc). And so Gcc, Clang and Intel use their rights and made the code even more generic. You can think of it as a compiler extension of standard library.
Having said that one may ask why standard requires BidirectionalIterator here. I suppose it is possible because the authors of the standard didn't see a way to comply with the complexity requirements without this requirement. It is possible that the authors of gcc found a way to do it better than anticipated by the standard. Looking at the gcc source code kinda confirms this. https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-v3/include/bits/stl_algo.h#L1613
EDIT:
I have dug through GCC library implementation and I think I get it. Implementation for ForwardIterator, BidirectionalIterator and RandomAccessIterator are different for std::stable_partition. This is due to different implementations of std::rotate which is used by the partitioning algorithm. And so, for forward iterator number of swaps is greater and can exceed (last - first) * log(last - first).
Look here and here.
The problem seems to have historical and not mathematical reasons. Looking through Alexander Stepanov's papers, I found this one: "Partition and Related Functions".
It contains the following passage:
Remark: It is interesting that this excellent algorithm is not in the C++ standard which requires bidirectional iterators for partition. I had known, implemented, and taught this algorithm for quite some time – since I first read about it in Bentley’s column in CACM in the mid-eighties. But my original STL proposal does, somehow, specify bidirectional iterators for both partition and stable_partition. Both of them were corrected in SGI STL, but most vendors are still behind. This little thing has been bothering me for over 10 years now; the most bothersome part being the fact of omission. How did it happen? I suspect that the explanation is quite simple: while in the early 90ties I already understood the idea of reducing every algorithms to its minimal requirements, and I also knew that the same operation could be implemented using better algorithms when we know more about the data to which they are applied, I was not yet fully aware of the need to provide an algorithm for the weakest case, if such an algorithm is available. It took several more years to understand the importance of “filling the algorithmic space.”
The following simple algorithm
template <typename I, // I models Forward Iterator
typename N, // N models Integer
typename P> // P models Unary Predicate
pair<I, I> stable_partition_inplace_n(I f, N n, P p)
{
if (n == 0) return make_pair(f, f);
if (n == 1) {
I l = successor(f);
if (p(*f)) l = f;
return make_pair(f, l);
}
pair<I, I> i = stable_partition_inplace_n(f, n/2, p);
pair<I, I> j = stable_partition_inplace_n(i.second, n – n/2, p);
return make_pair(rotate(i.first, i.second, j.first), j.second);
}
is given in that paper. It works with forward iterators and performs O(N log N) swaps in the worst case.
Related
This program:
#include <ranges>
#include <numeric>
#include <iostream>
int main() {
auto rng = std::ranges::istream_view<int>(std::cin);
std::cout << std::accumulate(std::ranges::begin(rng), std::ranges::end(rng), 0);
}
is supposed to sum up all integers appearing as text on the standard input stream. But - it doesn't compile. I know std::ranges::begin() and std::ranges::end() exist, so what's going on? The compiler tells me it can't find a suitable candidate; why?
From inception up through C++17, everything in <algorithm> is based on iterator pairs: you have one iterator referring to the beginning of a range and one iterator referring to the end of the range, always having the same type.
In C++20, this was generalized. A range is now denoted by an iterator and a sentinel for that iterator - where the sentinel itself need not actually be an iterator of any kind, it just needs to be a type that can compare equal to its corresponding iterator (this is the sentinel_for concept).
C++17 ranges tend to be† valid C++20 ranges, but not necessarily in the opposite direction. One reason is the ability to have a distinct sentinel type, but there are others, which also play into this question (see below).
To go along with the new model, C++20 added a large amount of algorithms into the std::ranges namespace that take an iterator and a sentinel, rather than two iterators. So for instance, while we've always had:
template<class InputIterator, class T>
constexpr InputIterator find(InputIterator first, InputIterator last,
const T& value);
we now also have:
namespace ranges {
template<input_iterator I, sentinel_for<I> S, class T, class Proj = identity>
requires indirect_binary_predicate<ranges::equal_to, projected<I, Proj>, const T*>
constexpr I find(I first, S last, const T& value, Proj proj = {});
template<input_range R, class T, class Proj = identity>
requires indirect_binary_predicate<ranges::equal_to,
projected<iterator_t<R>, Proj>, const T*>
constexpr borrowed_iterator_t<R>
find(R&& r, const T& value, Proj proj = {});
}
The first overload here takes an iterator/sentinel pair and the second takes a range instead.
While a lot of algorithms added corresponding overloads into std::ranges, the ones in <numeric> were left out. There is a std::accumulate but there is no std::ranges::accumulate. As such, the only version we have available at the moment is one that takes an iterator-pair. Otherwise, you could just write:
auto rng = std::ranges::istream_view<int>(std::cin);
std::cout << std::ranges::accumulate(rng, 0);
Unfortunately, std::ranges::istream_view is one of the new, C++20 ranges whose sentinel type differs from its iterator type, so you cannot pass rng.begin() and rng.end() into std::accumulate either.
This leaves you with two options generally (three, if you include waiting for C++23, which will hopefully have a std::ranges::fold):
Write your own range-based and iterator-sentinel-based algorithms. Which for fold is very easy to do.
Or
There is a utility to wrap a C++20 range into a C++17-compatible one: views::common. So you could this:
auto rng = std::ranges::istream_view<int>(ints) | std::views::common;
std::cout << std::accumulate(rng.begin(), rng.end(), 0);
Except not in this specific case.
istream_view's iterators aren't copyable, and in C++17 all iterators must be. So there isn't really a way to provide C++17-compatible iterators based on istream_view. You need proper C++20-range support. The future std::ranges::fold will support move-only views and move-only iterators, but std::accumulate never can.
Which in this case, just leaves option 1.
†A C++20 iterator needs to be default-constructible, which was not a requirement of C++17 iterators. So a C++17 range with non-default-constructible iterators would not be a valid C++20 range.
The problem is that the end of a C++ range is not, in the general case, an iterator, but rather, a sentinel. A sentinel can have a different type than an iterator and admit fewer operations - as, generally speaking, you mostly need to compare against it to know you've reached the end of the range, and may not be allowed to just work with it like any iterator. For more about this distinction, read:
What's the difference between a sentinel and an end iterator?
Now, standard-library algorithms (including the ones in <numeric>) take pairs of iterators of the same type. In your example:
template< class InputIt, class T >
constexpr T accumulate( InputIt first, InputIt last, T init );
see? InputIt must be the type of both the beginning and end of the range. This can (probably) not even be "fixed" for the istream_view range, because the end of the standard input really isn't an iterator per se. (Although maybe you could bludgeon it into being an iterator and throw exceptions when doing irrelevant things with it.)
So, we would need either a new variant of std::accumulate or an std::ranges::accumulate, which we currently don't have. Or, of course, you could write that yourself, which should not be too difficult.
Edit: One last option, suggested by #RemyLebeau, is to use an std::istream_iterator instead:
std::cout << std::accumulate(
std::istream_iterator<int>(std::cin),
std::istream_iterator<int>(),
0);
Consider I had an std::vector such that its contents were std::strings and {"a","b","c"}. If I were to perform std::find() looking for "a", would it stop once it iterated over "a" (ie. short-circuit) or continue to the end?
std::vector<std::string> vec;
vec.insert(vec.begin(), "a");
vec.insert(vec.begin(), "b");
vec.insert(vec.begin(), "c");
Which is faster?
std::find(vec.begin(), vec.end(), "a");
std::find(vec.begin(), vec.end(), "c");
See the possible implimenation of std::find
template<class InputIt, class T>
constexpr InputIt find(InputIt first, InputIt last, const T& value)
{
for (; first != last; ++first) {
if (*first == value) { // if the condition met
return first; // ---> here returns the iterator
}
}
return last;
}
It will stop iterating, once it finds the match.
Based on description here, yes it does.
Returns the first element in the range [first, last) that satisfies
specific criteria.
Complexity: At most last - first applications of the predicate
And by taking a look at its possible implementations it is stated that std::find uses short circuit
The C++17 standard draft defines the behavior of std::find in [alg.find] (only part relevant to overload used in question cited):
template<class InputIterator, class T>
InputIterator find(InputIterator first, InputIterator last,
const T& value);
[...]
Returns: The first iterator i in the range [first, last) for which the following corresponding conditions hold: *i == value, [...]. Returns last if no such iterator is found.
Complexity: At most last - first applications of the corresponding predicate.
Previous standard versions, including C++03, contain basically the same wording.
Nothing in this guarantees that the elements are searched in any specific order at all or that std::find must stop testing the predicate once it found a match.
However, since the function must return the first iterator in the range satisfying the condition, it makes no sense to test out-of-order, because if a match was found, all previous iterators would need to be tested for an earlier match anyway.
Once a match is found it is also pointless to continue applying the predicate and the standard only requires the predicate to be applied "at most" as often as there are elements in the range.
Therefore any reasonable sequential implementation of std::find will search the iterator range in-order and return when a match is found. If an implementation did not do that, users would complain about it as soon as they noticed. Standard library implementors want their code to be fast where possible and they have no benefit letting their code do more work than necessary.
I suppose though that an implementation could make use of parallelization if it knows that this will not cause data races for the given types and in that case it might happen that the search examines iterators beyond the first match. Whether something like this is implemented in any standard library for the non-parallel std::find overload from the question, I don't know.
So simple question.
template<class InputIt, class Size, class OutputIt>
OutputIt copy_n(InputIt first, Size count, OutputIt result);
Why does std::copy_n take a type for the number of elements it's going to copy instead of simply std::size_t? I just can't think of a reason.
template<class InputIt, class OutputIt>
OutputIt copy_n(InputIt first, std::size_t count, OutputIt result);
Speculating about original rationales is mostly futile in such a case, but with this design copy_n can be invoked with a negative count of e.g. int or ptrdiff_t type, in which case it simply does nothing, and that must surely have been clear to the standardization committee's members, who are pretty competent folks.
One additional advantage is that with special iterators, such as input and output iterators, the size might be larger than any possible pointer difference, and hence possibly larger than size_t can represent. E.g. this is so for files larger than 4GB in 32-bit Windows. The definition of copy_n is expressed in terms of apparent pointer/iterator arithmetic, “For each non-negative integer i < n, performs *(result + i) = *(first + i)”, which would seem to relegate this advantage to very special cases indeed, but the notation accommodates pure input and output iterators as explained in
C++11 §25.1/12:
” In the description of the algorithms operators + and - are used for some of the iterator categories for which
they do not have to be defined. In these cases the semantics of a+n is the same as that of
X tmp = a;
advance(tmp, n);
return tmp;
and that of b-a is the same as of return distance(a, b);
The genericity of the design has no inherent advantage, rather it's a disadvantage in itself in that it's more verbose and generates less easily understood diagnostics for incorrect usage code. Its advantages include the two ones listed above. Evidently the committee felt that these advantages, and perhaps others (?), outweighed the inherent disadvantages of having Size as a template parameter.
I would guess it is because of genericity.
Containers size_type in C++ are usually size_t. But if you use a custom container, or use custom allocators, that might not be the case hence the template parameter.
For a custom container, size_type does not have to be a typedef to size_t (it has to be an unsigned integer large enough to represent all positive values of difference_type).
For STL containers, size_type is a typedef to allocator::size_type which is size_t for the default allocator. But the type can be different if you specify a custom allocator.
The C++11 algorithms std::is_sorted and std::is_sorted_until both require ForwardIterators. However, the Boost.Range version boost::is_sorted only requires SinglePassRanges which corresponds to InputIterators. In particular, it delegates to an an iterator-based implementation like this:
template<class Iterator, class Comp>
inline Iterator is_sorted_until (Iterator first, Iterator last, Comp c) {
if (first == last)
return last;
Iterator it = first; ++it;
for (; it != last; first = it, ++it)
if (c(*it, *first))
return it;
return it;
}
Here we see a *first iterator dereference that happens after the ++it iterator increment. This means that Iterator should have ForwardIterator as its required category. Why? Because the Standard says so in
24.2.3 Input iterators [input.iterators]/p2 (see table 107 with the line about ++r)
post: any copies of the previous value of r are no longer required
either to be dereferenceable or to be in the domain of ==.
Note: this is not intended to be "a proof by single example", but it seems that any comparison based algorithm (e.g. adjacent_find) would necessarily require forward iterators in order to be able to make a comparison between two iterators.
Question: why doesn't the Boost.Range version of is_sorted require the stronger concept of ForwardRange (and ForwardIterator for its low-level routines) that is required by std::is_sorted? Is it a bug in Boost.Range?
It looks like the iterator versions in boost.algorithm correctly require ForwardIterators. And believe it or not, there are also range-based versions in boost.algorithm. Code duplication at its best. The documentation is lagging behind the source according to Ticket #9367, Changeset #86741 corrects the rest of the documentation to state that all flavors of the sort-checking algorithms require ForwardIterators.
I would prefer the implementations in <boost/algorithm/cxx11/is_sorted.hpp> over those in <boost/range/algorithm_ext/is_sorted.hpp> which seem to be bit-rotting since 2010.
EDIT: Digging around, it appears that the Boost.Range implementations did require ForwardIterator, but this commit broke them in 2010?!?.
ISO C++11 24.3:
template <class InputIterator, class Distance>
void advance(InputIterator& i, Distance n);
// ...
template <class ForwardIterator>
ForwardIterator next
(
ForwardIterator x,
typename std::iterator_traits<ForwardIterator>::difference_type n = 1
);
Why std::next does not accept InputIterators?
One of legal use cases I am thinking about is:
first = find(next(first, x), last, 11); // ...
I have found appropriate DR:
next/prev return an incremented iterator without changing the value of the original iterator. However, even this may invalidate an InputIterator. A ForwardIterator is required to guarantee the 'multipass' property.
But I don't understand how multipass/invalidation is related to that. Using same multipass/invalidation reasoning, we can even ban std::find for InputIterators:
template<class InputIterator, class T>
InputIterator find(InputIterator first, InputIterator last, const T& value);
There is nothing special about std::next in compare to std::find or std::vector::insert(pos, first, last) which have perfectly legal use cases for InputIterators
Moreover std::next(it, n) can be used in generic code, which operates not only on InputIterators.
In effect, input iterators cannot be usefully copied, because once an input iterator is incremented, any copy left lying around is invalidated.
std::next takes an iterator and returns another iterator which has been advanced n times. You can't do that with an input iterator without invalidating the original iterator, which makes std::next pointless. By constrast, std::advance advances the specified iterator n times, which is fine with an input iterator.
std::next is the iterator generalization of operator+(T*, size_t). std::advance is the iterator generalization of operator+=(T*&, size_t). It may well be that std::advance, like operator+=, should return a reference instead of void.
It's true that there is a similar issue with std::find (and related functions); they, too, will invalidate any copy of the specified input iterators. But it is quite possible that the committee found that issue less serious.