I was told here that:
The order of generate is not guaranteed => depending on the implementation
I have looked up gcc's implementation of generate:
for (; __first != __last; ++__first)
*__first = __gen();
And Visual Studio implements it identically to that. This is a relief to me as using a lambda in generate that reads and writes to a capture could have undeterministic results:
int foo[] = {1, 0, 13};
vector<int> bar(3);
generate(bar.begin(), bar.end(), [&]() {
static auto i = 0;
static auto total = 0;
total += foo[i];
return foo[i] / total;
});
I expect bar to contain {1, 0, 0}.
If I am allowed to execute out of order this could even cause a divide by 0 error.
So I'd sleep easier if this question could be answered with proof that generate is required to execute sequentially.
As a note here, I know that experimental::parallel::generate will not be sequential. I'm just asking about generate.
I was intrigued by this so have done some research.
At the bottom of this answer is a copy of the relevant section from the standard as of 2011.
You will see from the template declaration of std::generate<> that the iterator parameters must conform to the concept of a ForwardIterator and OutputIterator respectively.
A ForwardIterator does not support random access. it can only read or write sequentially. An OutputIterator is even more restrictive - its operator* implicitly includes the effect of an operator++. This is an explicit feature of the concept.
Therefore, by implication, the implementation of this function must access elements sequentially (and therefore generate values sequentially) since to not do so would break the contract implicit in the interface.
Therefore the standard does (implicitly and quietly) guarantee that std::generate initialise its elements sequentially. It would be impossible to write a well-formed implementation of std::generate that did not.
QED
25.3.7 Generate [alg.generate]
template<class ForwardIterator, class Generator>
void generate(ForwardIterator first, ForwardIterator last,
Generator gen);
template<class OutputIterator, class Size, class Generator>
OutputIterator generate_n(OutputIterator first, Size n, Generator gen);
1 Effects: The first algorithm invokes the function object gen and
assigns the return value of gen through all the iterators in the range
[first,last). The second algorithm invokes the function object gen and
assigns the return value of gen through all the iterators in the range
[first,first + n) if n is positive, otherwise it does nothing.
2
Requires: gen takes no arguments, Size shall be convertible to an
integral type (4.7, 12.3).
3 Returns: generate_n returns first + n for
non-negative values of n and first for negative values.
4 Complexity:
Exactly last - first, n, or 0 invocations of gen and assignments,
respectively.
Here is all the standard says about it (25.7.3/1):
template<class ForwardIterator, class Generator>
void generate(ForwardIterator first, ForwardIterator last, Generator gen);
template<class OutputIterator, class Size, class Generator>
OutputIterator generate_n(OutputIterator first, Size n, Generator gen);
The first algorithm invokes the function object
gen
and assigns the return value of
gen
through
all the iterators in the range
[first,last)
. The second algorithm invokes the function object
gen
and assigns the return value of
gen
through all the iterators in the range
[first,first + n)
if
n
is
positive, otherwise it does nothing.
As you can see, it is not explicitly stated that the iterator assignments must be done sequentially.
Related
This program:
#include <ranges>
#include <numeric>
#include <iostream>
int main() {
auto rng = std::ranges::istream_view<int>(std::cin);
std::cout << std::accumulate(std::ranges::begin(rng), std::ranges::end(rng), 0);
}
is supposed to sum up all integers appearing as text on the standard input stream. But - it doesn't compile. I know std::ranges::begin() and std::ranges::end() exist, so what's going on? The compiler tells me it can't find a suitable candidate; why?
From inception up through C++17, everything in <algorithm> is based on iterator pairs: you have one iterator referring to the beginning of a range and one iterator referring to the end of the range, always having the same type.
In C++20, this was generalized. A range is now denoted by an iterator and a sentinel for that iterator - where the sentinel itself need not actually be an iterator of any kind, it just needs to be a type that can compare equal to its corresponding iterator (this is the sentinel_for concept).
C++17 ranges tend to be† valid C++20 ranges, but not necessarily in the opposite direction. One reason is the ability to have a distinct sentinel type, but there are others, which also play into this question (see below).
To go along with the new model, C++20 added a large amount of algorithms into the std::ranges namespace that take an iterator and a sentinel, rather than two iterators. So for instance, while we've always had:
template<class InputIterator, class T>
constexpr InputIterator find(InputIterator first, InputIterator last,
const T& value);
we now also have:
namespace ranges {
template<input_iterator I, sentinel_for<I> S, class T, class Proj = identity>
requires indirect_binary_predicate<ranges::equal_to, projected<I, Proj>, const T*>
constexpr I find(I first, S last, const T& value, Proj proj = {});
template<input_range R, class T, class Proj = identity>
requires indirect_binary_predicate<ranges::equal_to,
projected<iterator_t<R>, Proj>, const T*>
constexpr borrowed_iterator_t<R>
find(R&& r, const T& value, Proj proj = {});
}
The first overload here takes an iterator/sentinel pair and the second takes a range instead.
While a lot of algorithms added corresponding overloads into std::ranges, the ones in <numeric> were left out. There is a std::accumulate but there is no std::ranges::accumulate. As such, the only version we have available at the moment is one that takes an iterator-pair. Otherwise, you could just write:
auto rng = std::ranges::istream_view<int>(std::cin);
std::cout << std::ranges::accumulate(rng, 0);
Unfortunately, std::ranges::istream_view is one of the new, C++20 ranges whose sentinel type differs from its iterator type, so you cannot pass rng.begin() and rng.end() into std::accumulate either.
This leaves you with two options generally (three, if you include waiting for C++23, which will hopefully have a std::ranges::fold):
Write your own range-based and iterator-sentinel-based algorithms. Which for fold is very easy to do.
Or
There is a utility to wrap a C++20 range into a C++17-compatible one: views::common. So you could this:
auto rng = std::ranges::istream_view<int>(ints) | std::views::common;
std::cout << std::accumulate(rng.begin(), rng.end(), 0);
Except not in this specific case.
istream_view's iterators aren't copyable, and in C++17 all iterators must be. So there isn't really a way to provide C++17-compatible iterators based on istream_view. You need proper C++20-range support. The future std::ranges::fold will support move-only views and move-only iterators, but std::accumulate never can.
Which in this case, just leaves option 1.
†A C++20 iterator needs to be default-constructible, which was not a requirement of C++17 iterators. So a C++17 range with non-default-constructible iterators would not be a valid C++20 range.
The problem is that the end of a C++ range is not, in the general case, an iterator, but rather, a sentinel. A sentinel can have a different type than an iterator and admit fewer operations - as, generally speaking, you mostly need to compare against it to know you've reached the end of the range, and may not be allowed to just work with it like any iterator. For more about this distinction, read:
What's the difference between a sentinel and an end iterator?
Now, standard-library algorithms (including the ones in <numeric>) take pairs of iterators of the same type. In your example:
template< class InputIt, class T >
constexpr T accumulate( InputIt first, InputIt last, T init );
see? InputIt must be the type of both the beginning and end of the range. This can (probably) not even be "fixed" for the istream_view range, because the end of the standard input really isn't an iterator per se. (Although maybe you could bludgeon it into being an iterator and throw exceptions when doing irrelevant things with it.)
So, we would need either a new variant of std::accumulate or an std::ranges::accumulate, which we currently don't have. Or, of course, you could write that yourself, which should not be too difficult.
Edit: One last option, suggested by #RemyLebeau, is to use an std::istream_iterator instead:
std::cout << std::accumulate(
std::istream_iterator<int>(std::cin),
std::istream_iterator<int>(),
0);
I have written a fancy "zip iterator" that already fulfils many roles (can be used in for_each, copy loops, container iterator range constructors etc...).
Under all the template code to work around the pairs/tuples involved, it comes down to the dereference operator of the iterator returning a tuple/pair of references and not a reference to a tuple/pair.
I want my iterator to work with std::sort, so I need to be able to do swap(*iter1, *iter2) and have the underlying values switched in the original containers being iterated over.
The code and a small demo can be viewed here (it's quite a bit to get through): http://coliru.stacked-crooked.com/a/4fe23b4458d2e692
Although libstdc++'s sort uses std::iter_swap which calls swap, e.g. libc++'s does not, and it just calls swap directly, so I would like a solution involving swap as the customization point.
What I have tried (and gotten oooooh so close to working) is instead of returning std::pair/std::tuple from the operator* as I am doing now, is returning a simple wrapper type instead. The intent is to have the wrapper behave as if it were a std::pair/std::tuple, and allow me to write a swap function for it.
It looked like this:
template<typename... ValueTypes>
struct TupleWrapper : public PairOrTuple_t<ValueTypes...>
{
using PairOrTuple_t<ValueTypes...>::operator=;
template<typename... TupleValueTypes>
operator PairOrTuple_t<TupleValueTypes...>() const
{
return static_cast<PairOrTuple_t<ValueTypes...>>(*this);
}
};
template<std::size_t Index, typename... ValueTypes>
decltype(auto) get(TupleWrapper<ValueTypes...>& tupleWrapper)
{
return std::get<Index>(tupleWrapper);
}
template<std::size_t Index, typename... ValueTypes>
decltype(auto) get(TupleWrapper<ValueTypes...>&& tupleWrapper)
{
return std::get<Index>(std::forward<TupleWrapper<ValueTypes...>>(tupleWrapper));
}
template<typename... ValueTypes,
std::size_t... Indices>
void swap(TupleWrapper<ValueTypes...> left,
TupleWrapper<ValueTypes...> right,
const std::index_sequence<Indices...>&)
{
(std::swap(std::get<Indices>(left), std::get<Indices>(right)), ...);
}
template<typename... ValueTypes>
void swap(TupleWrapper<ValueTypes...> left,
TupleWrapper<ValueTypes...> right)
{
swap(left, right, std::make_index_sequence<sizeof...(ValueTypes)>());
}
namespace std
{
template<typename... ValueTypes>
class tuple_size<utility::implementation::TupleWrapper<ValueTypes...>> : public tuple_size<utility::implementation::PairOrTuple_t<ValueTypes...>> {};
template<std::size_t Index, typename... ValueTypes>
class tuple_element<Index, utility::implementation::TupleWrapper<ValueTypes...>> : public tuple_element<Index, utility::implementation::PairOrTuple_t<ValueTypes...>> {};
}
Full code here: http://coliru.stacked-crooked.com/a/951cd639d95af130.
Returning this wrapper in operator* seems to compile (at least on GCC) but produces garbage.
On Clang's libc++, the std::tie fails to compile.
Two questions:
How can I get this to compile with libc++ (the magic seems to lie in the conversion operator of TupleWrapper?)
Why is the result wrong and what did I do wrong?
I know it's a lot of code, but well, I can't get it any shorter as all the tiny examples of swapping tuple wrappers worked fine for me.
1st problem
One of the issues is that the ZipIterator class does not satisfy the requirements of RandomAccessIterator.
std::sort requires RandomAccessIterators as its parameters
RandomAccessIterators must be BidirectionalIterators
BidirectionalIterators must be ForwardIterators
ForwardIterators have the condition that ::reference must be value_type& / const value_type&:
The type std::iterator_traits<It>::reference must be exactly
T& if It satisfies OutputIterator (It is mutable)
const T& otherwise (It is constant)
(where T is the type denoted by std::iterator_traits<It>::value_type)
which ZipIterator currently doesn't implement.
It works fine with std::for_each and similar functions that only require the iterator to satisfy the requirements of InputIterator / OutputIterator.
The reference type for an input iterator that is not also a LegacyForwardIterator does not have to be a reference type: dereferencing an input iterator may return a proxy object or value_type itself by value (as in the case of std::istreambuf_iterator).
tl;dr: ZipIterator can be used as an InputIterator / OutputIterator, but not as a ForwardIterator, which std::sort requires.
2nd problem
As #T.C. pointed out in their comment std::sort is allowed to move values out of the container and then later move them back in.
The type of dereferenced RandomIt must meet the requirements of MoveAssignable and MoveConstructible.
which ZipIterator currently can't handle (it never copies / moves the referenced objects), so something like this doesn't work as expected:
std::vector<std::string> vector_of_strings{"one", "two", "three", "four"};
std::vector<int> vector_of_ints{1, 2, 3, 4};
auto first = zipBegin(vector_of_strings, vector_of_ints);
auto second = first + 1;
// swap two values via a temporary
auto temp = std::move(*first);
*first = std::move(*second);
*second = std::move(temp);
// Result:
/*
two, 2
two, 2
three, 3
four, 4
*/
(test on Godbolt)
Result
Unfortunately it is not possible to create an iterator that produces elements on the fly and can by used as a ForwardIterator with the current standard (for example this question)
You could of course write your own algorithms that only require InputIterators / OutputIterators (or handle your ZipIterator differently)
For example a simple bubble sort: (Godbolt)
template<class It>
void bubble_sort(It begin, It end) {
using std::swap;
int n = std::distance(begin, end);
for (int i = 0; i < n-1; i++) {
for (int j = 0; j < n-i-1; j++) {
if (*(begin+j) > *(begin+j+1))
swap(*(begin+j), *(begin+j+1));
}
}
}
Or change the ZipIterator class to satisfy RandomAccessIterator.
I unfortunately can't think of a way that would be possible without putting the tuples into a dynamically allocated structure like an array (which you're probably trying to avoid)
Given the following code,
std::vector<int> numbers = {1, 2, 3, 4, 5};
std::any_of(std::begin(numbers), std::end(numbers),
[](int number) { return number > 3; } );
is std::any_of required (by the standard) to return as soon as it reaches 4?
The standard itself doesn't place any such hard requirement. But one may infer it is indirectly encouraged ([alg.any_of]):
template <class InputIterator, class Predicate>
bool any_of(InputIterator first, InputIterator last, Predicate pred);
template <class ExecutionPolicy, class ForwardIterator, class Predicate>
bool any_of(ExecutionPolicy&& exec, ForwardIterator first, ForwardIterator last,
Predicate pred);
Returns: false if [first, last) is empty or if there is no iterator i in the range [first, last) such that pred(*i) is true, and true
otherwise.
Complexity: At most last - first applications of the predicate.
While a perfectly conforming implementation may apply the predicate exactly last-first times, the wording to me sounds like it would be encouraged to exit as soon as possible.
Note that it's virtually impossible to ask the same of the overload that accepts an ExecutionPolicy. Since then the order of evaluation is not known.
On a less formal note, any implementation of the sequential version that does not exit the moment the predicate is true, puts the credentials of its author into question.
I've recently stumbled upon this wording from cppreference:
Unlike the rest of the algorithms, for_each is not allowed to make copies of the elements in the sequence even if they are trivially copyable.
Is the statement correct? I haven't found any grounds in the standard. Do I understand well, that it would btw. imply the following example from the same page invalid?
struct Sum
{
Sum(): sum{0} { }
void operator()(int n) { sum += n; }
int sum;
};
int main()
{
std::vector<int> nums{3, 4, 2, 8, 15, 267};
// ...
// calls Sum::operator() for each number
Sum s = std::for_each(nums.begin(), nums.end(), Sum());
// ...
}
That sentence on cppreference.com comes from the wording added to C++17 about the new support for parallel algorithms.
[algorithms.parallel.exec]/3 says:
Unless otherwise stated, implementations may make arbitrary copies of elements (with type T) from sequences where is_trivially_copy_constructible_v<T> and is_trivially_destructible_v<T> are true.
Note this is in a context discussing parallel algorithms, defined as standard library function templates which have a template parameter named ExecutionPolicy.
But then [alg.foreach]/9 says of for_each(ExecutionPolicy&&, ForwardIterator, ForwardIterator, Function):
Implementations do not have the freedom granted under [algorithms.parallel.exec] to make arbitrary copies of elements from the input sequence.
Presumably some parallelization techniques can be more efficient for trivial types by making copies of the elements. (Maybe to make them contiguous in memory? I'm just guessing.)
So none of this applies to the older non-parallel for_each(InputIterator first, InputIterator last, Function f). For that algorithm, it's simply the case that since the effects are specified as "Applies f to the result of dereferencing every iterator in the range [first, last)...", the functor argument must be e.g. *first and not a copy of *first.
The wording is indeed confusing, as it is also mentionned that
The signature of the function should be equivalent to the following:
void fun(const Type &a);
The signature does not need to have const &.
I think you need to see the fact that for_each is not allowed to make copies of the elements as a guarantee given by the standard to the user rather than a limitation on the user predicate function applied to each element..
Is there more beyond advance takes negative numbers?
std::advance
modifies its argument
returns nothing
works on input iterators or better (or bi-directional iterators if a negative distance is given)
std::next
leaves its argument unmodified
returns a copy of the argument, advanced by the specified amount
works on forward iterators or better (or bi-directional iterators if a negative distance is given))
Perhaps the biggest practical difference is that std::next() is only available from C++11.
std::next() will advance by one by default, whereas std::advance() requires a distance.
And then there are the return values:
std::advance(): (none) (the iterator passed in is modified)
std::next(): The n th successor.
std::next() takes negative numbers just like std::advance, and in that case requires that the iterator must be bidirectional. std::prev() would be more readable when the intent is specifically to move backwards.
std::advance
The function advance() increments the position of an iterator passed as the argument. Thus, the function lets the iterator step forward (or backward) more than one element:
#include <iterator>
void advance (InputIterator& pos, Dist n)
Lets the input iterator pos step n elements forward (or backward).
For bidirectional and random-access iterators, n may be negative to step backward.
Dist is a template type. Normally, it must be an integral type because operations such as <, ++, --, and comparisons with 0 are
called.
Note that advance() does not check whether it crosses the end() of a sequence (it can’t check because iterators in general do not know the
containers on which they operate). Thus, calling this function might
result in undefined behavior because calling operator ++ for the end
of a sequence is not defined.
std::next(and std::prev new in C++11)
#include <iterator>
ForwardIterator next (ForwardIterator pos)
ForwardIterator next (ForwardIterator pos, Dist n)
Yields the position the forward iterator pos would have if moved forward 1 or n positions.
For bidirectional and random-access iterators, n may be negative to yield previous ositions.
Dist is type std::iterator_traits::difference_type.
Calls advance(pos,n) for an internal temporary object.
Note that next() does not check whether it crosses the end() of a sequence. Thus, it is up to the caller to ensure that the result is
valid.
cite from The C++ Standard Library Second Edition
They're pretty much the same, except that std::next returns a copy and std::advance modifies its argument. Note that the standard requires std::next to behave like std::advance:
24.4.4 Iterator operations [iterator.operations]
template <class InputIterator, class Distance>
void advance(InputIterator& i [remark: reference], Distance n);
2. Requires: n shall be negative only for bidirectional and random access iterators
3. Effects: Increments (or decrements for negative n) iterator reference i by n.
[...]
template <class ForwardIterator>
ForwardIterator next(ForwardIterator x, [remark: copy]
typename std::iterator_traits<ForwardIterator>::difference_type n = 1);
6. Effects: Equivalent to advance(x, n); return x;
Note that both actually support negative values if the iterator is an input iterator. Also note that std::next requires the iterator to meet the conditions of an ForwardIterator, while std::advance only needs an Input Iterator (if you don't use negative distances).