I have written a fancy "zip iterator" that already fulfils many roles (can be used in for_each, copy loops, container iterator range constructors etc...).
Under all the template code to work around the pairs/tuples involved, it comes down to the dereference operator of the iterator returning a tuple/pair of references and not a reference to a tuple/pair.
I want my iterator to work with std::sort, so I need to be able to do swap(*iter1, *iter2) and have the underlying values switched in the original containers being iterated over.
The code and a small demo can be viewed here (it's quite a bit to get through): http://coliru.stacked-crooked.com/a/4fe23b4458d2e692
Although libstdc++'s sort uses std::iter_swap which calls swap, e.g. libc++'s does not, and it just calls swap directly, so I would like a solution involving swap as the customization point.
What I have tried (and gotten oooooh so close to working) is instead of returning std::pair/std::tuple from the operator* as I am doing now, is returning a simple wrapper type instead. The intent is to have the wrapper behave as if it were a std::pair/std::tuple, and allow me to write a swap function for it.
It looked like this:
template<typename... ValueTypes>
struct TupleWrapper : public PairOrTuple_t<ValueTypes...>
{
using PairOrTuple_t<ValueTypes...>::operator=;
template<typename... TupleValueTypes>
operator PairOrTuple_t<TupleValueTypes...>() const
{
return static_cast<PairOrTuple_t<ValueTypes...>>(*this);
}
};
template<std::size_t Index, typename... ValueTypes>
decltype(auto) get(TupleWrapper<ValueTypes...>& tupleWrapper)
{
return std::get<Index>(tupleWrapper);
}
template<std::size_t Index, typename... ValueTypes>
decltype(auto) get(TupleWrapper<ValueTypes...>&& tupleWrapper)
{
return std::get<Index>(std::forward<TupleWrapper<ValueTypes...>>(tupleWrapper));
}
template<typename... ValueTypes,
std::size_t... Indices>
void swap(TupleWrapper<ValueTypes...> left,
TupleWrapper<ValueTypes...> right,
const std::index_sequence<Indices...>&)
{
(std::swap(std::get<Indices>(left), std::get<Indices>(right)), ...);
}
template<typename... ValueTypes>
void swap(TupleWrapper<ValueTypes...> left,
TupleWrapper<ValueTypes...> right)
{
swap(left, right, std::make_index_sequence<sizeof...(ValueTypes)>());
}
namespace std
{
template<typename... ValueTypes>
class tuple_size<utility::implementation::TupleWrapper<ValueTypes...>> : public tuple_size<utility::implementation::PairOrTuple_t<ValueTypes...>> {};
template<std::size_t Index, typename... ValueTypes>
class tuple_element<Index, utility::implementation::TupleWrapper<ValueTypes...>> : public tuple_element<Index, utility::implementation::PairOrTuple_t<ValueTypes...>> {};
}
Full code here: http://coliru.stacked-crooked.com/a/951cd639d95af130.
Returning this wrapper in operator* seems to compile (at least on GCC) but produces garbage.
On Clang's libc++, the std::tie fails to compile.
Two questions:
How can I get this to compile with libc++ (the magic seems to lie in the conversion operator of TupleWrapper?)
Why is the result wrong and what did I do wrong?
I know it's a lot of code, but well, I can't get it any shorter as all the tiny examples of swapping tuple wrappers worked fine for me.
1st problem
One of the issues is that the ZipIterator class does not satisfy the requirements of RandomAccessIterator.
std::sort requires RandomAccessIterators as its parameters
RandomAccessIterators must be BidirectionalIterators
BidirectionalIterators must be ForwardIterators
ForwardIterators have the condition that ::reference must be value_type& / const value_type&:
The type std::iterator_traits<It>::reference must be exactly
T& if It satisfies OutputIterator (It is mutable)
const T& otherwise (It is constant)
(where T is the type denoted by std::iterator_traits<It>::value_type)
which ZipIterator currently doesn't implement.
It works fine with std::for_each and similar functions that only require the iterator to satisfy the requirements of InputIterator / OutputIterator.
The reference type for an input iterator that is not also a LegacyForwardIterator does not have to be a reference type: dereferencing an input iterator may return a proxy object or value_type itself by value (as in the case of std::istreambuf_iterator).
tl;dr: ZipIterator can be used as an InputIterator / OutputIterator, but not as a ForwardIterator, which std::sort requires.
2nd problem
As #T.C. pointed out in their comment std::sort is allowed to move values out of the container and then later move them back in.
The type of dereferenced RandomIt must meet the requirements of MoveAssignable and MoveConstructible.
which ZipIterator currently can't handle (it never copies / moves the referenced objects), so something like this doesn't work as expected:
std::vector<std::string> vector_of_strings{"one", "two", "three", "four"};
std::vector<int> vector_of_ints{1, 2, 3, 4};
auto first = zipBegin(vector_of_strings, vector_of_ints);
auto second = first + 1;
// swap two values via a temporary
auto temp = std::move(*first);
*first = std::move(*second);
*second = std::move(temp);
// Result:
/*
two, 2
two, 2
three, 3
four, 4
*/
(test on Godbolt)
Result
Unfortunately it is not possible to create an iterator that produces elements on the fly and can by used as a ForwardIterator with the current standard (for example this question)
You could of course write your own algorithms that only require InputIterators / OutputIterators (or handle your ZipIterator differently)
For example a simple bubble sort: (Godbolt)
template<class It>
void bubble_sort(It begin, It end) {
using std::swap;
int n = std::distance(begin, end);
for (int i = 0; i < n-1; i++) {
for (int j = 0; j < n-i-1; j++) {
if (*(begin+j) > *(begin+j+1))
swap(*(begin+j), *(begin+j+1));
}
}
}
Or change the ZipIterator class to satisfy RandomAccessIterator.
I unfortunately can't think of a way that would be possible without putting the tuples into a dynamically allocated structure like an array (which you're probably trying to avoid)
Related
Here is my (simplified) attempt to implement a ranges::min_element version that would work for both lvalue and rvalue arguments:
#include <iterator>
#include <algorithm>
#include <type_traits>
#include <utility>
namespace better_std_ranges
{
template<typename Range>
constexpr auto min_element(Range& range)
{
using std::begin;
using std::end;
return std::min_element(begin(range), end(range));
}
template<typename Range>
constexpr auto min_element(Range&& range)
{
static_assert(!std::is_reference_v<Range>, "wrong overload chosen");
class _result_iterator_type // todo: inherit from some crtp base that will provide lacking operators depending on _underlying_iterator_type::iterator_category
{
using _underlying_iterator_type = std::decay_t<decltype(std::begin(std::declval<Range&>()))>;
public:
explicit constexpr _result_iterator_type(Range&& range) noexcept(std::is_nothrow_move_constructible_v<Range>)
: _underlying_range{std::move(range)}
, _underlying_iterator(::better_std_ranges::min_element(_underlying_range))
{
}
using difference_type = typename _underlying_iterator_type::difference_type;
using value_type = typename _underlying_iterator_type::value_type;
using pointer = typename _underlying_iterator_type::pointer;
using reference = typename _underlying_iterator_type::reference;
using iterator_category = typename _underlying_iterator_type::iterator_category;
constexpr decltype(auto) operator*() const
{
return *_underlying_iterator;
}
// todo: define other member functions that were not provided by the inheritance above
private:
Range _underlying_range;
_underlying_iterator_type _underlying_iterator;
};
return _result_iterator_type{std::move(range)};
}
}
#include <vector>
#include <iostream>
auto make_vector()
{
return std::vector{100, 200, 42, 500, 1000};
}
int main()
{
auto lvalue_vector = make_vector();
auto lvalue_vector_min_element_iterator = better_std_ranges::min_element(lvalue_vector);
std::cout << *lvalue_vector_min_element_iterator << '\n';
auto rvalue_vector_min_element_iterator = better_std_ranges::min_element(make_vector());
std::cout << *rvalue_vector_min_element_iterator << '\n';
}
The output is
42
42
Surely it lacks some implementation details, but the idea must be clear: if an input range is an rvalue, the return value can store a moved copy of it.
Thus it must be totally possible for std::ranges algorithms to work with rvalue arguments.
My question is: why does the standard go the opposite way and just prohibit using rvalue ranges with its algorithms by introducing that strange std::ranges::dangling placeholder?
There are two problems with this approach.
First, it breaks the semantics of the algorithm. The point of min_element (and any other algorithm that returns an iterator) is to return an iterator into the range. You're not doing that - you're returning an iterator into a different range. That really confuses the notion of what the return even means in this case. What would you even compare this iterator to? There's no corresponding .end()?
Second, the iterator model in C++ is based very strongly around the notion that iterators are cheap to copy. Every algorithm takes iterators by value and copies them around freely. Iterators are assumed to be light-weight and, importantly, non-owning. For forward iterators, copies of an iterator are assumed to be interchangeable.
Everything about this breaks if you suddenly have an iterator that has member std::vector<T> that it refers into. Copying iterators becomes very expensive. And now each distinct iterator copy is actually an iterator into a completely different range?
You can do a little bit better by having the iterator have a member std::shared_ptr<std::vector<T>> instead of a std::vector<T>. This way copies are much cheaper and no longer independent, so you have something closer to a legitimate iterator. But now you have to do an extra allocation (to create the shared pointer), you still have the issue where the iterator you're returning is into a different range than the algorithm was given, and you have the issue where the algorithm has very different semantics based on whether you provide an lvalue or rvalue range.
Basically, min_element on an rvalue range needs to either:
just return an iterator into the range, even if it will dangle
return some kind of wrapper around such a potentially-dangling iterator (this was the original Ranges design, dangling<I> could still let you get at the underlying I)
return some kind of type indicating that this doesn't work (the current design)
fail to compile entirely if usage would lead to dangling (what Rust would allow for)
I don't think there's another option here, really.
This program:
#include <ranges>
#include <numeric>
#include <iostream>
int main() {
auto rng = std::ranges::istream_view<int>(std::cin);
std::cout << std::accumulate(std::ranges::begin(rng), std::ranges::end(rng), 0);
}
is supposed to sum up all integers appearing as text on the standard input stream. But - it doesn't compile. I know std::ranges::begin() and std::ranges::end() exist, so what's going on? The compiler tells me it can't find a suitable candidate; why?
From inception up through C++17, everything in <algorithm> is based on iterator pairs: you have one iterator referring to the beginning of a range and one iterator referring to the end of the range, always having the same type.
In C++20, this was generalized. A range is now denoted by an iterator and a sentinel for that iterator - where the sentinel itself need not actually be an iterator of any kind, it just needs to be a type that can compare equal to its corresponding iterator (this is the sentinel_for concept).
C++17 ranges tend to be† valid C++20 ranges, but not necessarily in the opposite direction. One reason is the ability to have a distinct sentinel type, but there are others, which also play into this question (see below).
To go along with the new model, C++20 added a large amount of algorithms into the std::ranges namespace that take an iterator and a sentinel, rather than two iterators. So for instance, while we've always had:
template<class InputIterator, class T>
constexpr InputIterator find(InputIterator first, InputIterator last,
const T& value);
we now also have:
namespace ranges {
template<input_iterator I, sentinel_for<I> S, class T, class Proj = identity>
requires indirect_binary_predicate<ranges::equal_to, projected<I, Proj>, const T*>
constexpr I find(I first, S last, const T& value, Proj proj = {});
template<input_range R, class T, class Proj = identity>
requires indirect_binary_predicate<ranges::equal_to,
projected<iterator_t<R>, Proj>, const T*>
constexpr borrowed_iterator_t<R>
find(R&& r, const T& value, Proj proj = {});
}
The first overload here takes an iterator/sentinel pair and the second takes a range instead.
While a lot of algorithms added corresponding overloads into std::ranges, the ones in <numeric> were left out. There is a std::accumulate but there is no std::ranges::accumulate. As such, the only version we have available at the moment is one that takes an iterator-pair. Otherwise, you could just write:
auto rng = std::ranges::istream_view<int>(std::cin);
std::cout << std::ranges::accumulate(rng, 0);
Unfortunately, std::ranges::istream_view is one of the new, C++20 ranges whose sentinel type differs from its iterator type, so you cannot pass rng.begin() and rng.end() into std::accumulate either.
This leaves you with two options generally (three, if you include waiting for C++23, which will hopefully have a std::ranges::fold):
Write your own range-based and iterator-sentinel-based algorithms. Which for fold is very easy to do.
Or
There is a utility to wrap a C++20 range into a C++17-compatible one: views::common. So you could this:
auto rng = std::ranges::istream_view<int>(ints) | std::views::common;
std::cout << std::accumulate(rng.begin(), rng.end(), 0);
Except not in this specific case.
istream_view's iterators aren't copyable, and in C++17 all iterators must be. So there isn't really a way to provide C++17-compatible iterators based on istream_view. You need proper C++20-range support. The future std::ranges::fold will support move-only views and move-only iterators, but std::accumulate never can.
Which in this case, just leaves option 1.
†A C++20 iterator needs to be default-constructible, which was not a requirement of C++17 iterators. So a C++17 range with non-default-constructible iterators would not be a valid C++20 range.
The problem is that the end of a C++ range is not, in the general case, an iterator, but rather, a sentinel. A sentinel can have a different type than an iterator and admit fewer operations - as, generally speaking, you mostly need to compare against it to know you've reached the end of the range, and may not be allowed to just work with it like any iterator. For more about this distinction, read:
What's the difference between a sentinel and an end iterator?
Now, standard-library algorithms (including the ones in <numeric>) take pairs of iterators of the same type. In your example:
template< class InputIt, class T >
constexpr T accumulate( InputIt first, InputIt last, T init );
see? InputIt must be the type of both the beginning and end of the range. This can (probably) not even be "fixed" for the istream_view range, because the end of the standard input really isn't an iterator per se. (Although maybe you could bludgeon it into being an iterator and throw exceptions when doing irrelevant things with it.)
So, we would need either a new variant of std::accumulate or an std::ranges::accumulate, which we currently don't have. Or, of course, you could write that yourself, which should not be too difficult.
Edit: One last option, suggested by #RemyLebeau, is to use an std::istream_iterator instead:
std::cout << std::accumulate(
std::istream_iterator<int>(std::cin),
std::istream_iterator<int>(),
0);
Intuitively to check whecker pointer p lies in [a,b) one will do
a<=p && p<b
However, comparing pointers from two arrays results in unspecified behavior and thus we cannot safely say p is in [a,b) from this comparison.
Is there any way one can check for this with certainty?
(It would be better if it can be done for std::vector<T>::const_iterator, but I don't think it's feasible.)
Here's a partial solution. You can leverage the fact that the comparison would invoke unspecified behavior, and the fact that a core-constant-expression can't perform this operation:
template<typename T>
constexpr bool check(T *p, T *a, T *b)
{
return a <= p and p < b;
}
Now this function can be used like this:
int main()
{
int arr[5];
int arr_2[5];
constexpr bool b1 = check(arr + 1, arr, arr + 3); // ok
constexpr bool b2 = check(arr_2 + 1, arr, arr + 3); // error
}
Here's a demo.
This obviously works only if the pointer values are known at compile time. At run-time, there is no efficient way of doing this check.
The solution for pointers is to use the comparison objects defined in <functional>, like less/less_equal, etc.
From §20.8.5/8 of the c++17 standard1:
For templates greater, less, greater_equal, and less_equal, the specializations for any pointer type yield a total order, even if the built-in operators <, >, <=, >= do not.
So the solution for pointers would be:
template<typename T>
bool check(T *p, T *a, T *b)
{
return std::less_equal<T*>{}(a,p) && std::less<T*>{}(p,b);
}
Here's a working example using pointers.
There is no such strict guarantee for iterators; however this can be worked around in c++20, since it provides std::to_address which can convert pointable objects to pointers. Note, however, that the behavior of doing this for the purpose of comparisons is only really well defined for contiguous iterators.
Since we know that std::vector iterators cover a contiguous range, we can use this to retrieve the underlying pointer (note: not dereference it, as this would be undefined behavior for the past-the-end pointer).
So for a std::vector<T>::iterator, a solution might look like:
template <typename T>
bool check(const std::vector<T>::const_iterator p, std;:vector<T>::const_iterator a, std::vector<T>::const_iterator b)
{
// Delegate to the pointer check version defined above, for brevity
return check(std::to_address(p), std::to_address(a), std::to_address(b));
}
Here's a working example using iterators.
1 This same note exists all the way back to c++11 under §23.14.7/2, with similar wording.
If I understand you correctly, you want to check if vector iterator is between two other vector iterators.
Then you may use std::distance to compute distance between vector.begin and a, p and b and then simply compare itegers you get from distance return value.
std::distance(first, last) from C++17 can be used for both, but result is undefined if last is unreachable from first (e.g. different range or invalid iterator)
I want to have a wide template that 'does whatever it needs to' except for when I have explicitly specified the case.
Specifically, I am overloading operator() to use it for a matrix index of a multidimensional matrix. I also want to allow specifying an arbitrary number of indices using an iterator. Ideally I'd have the following signatures:
operator()(size_t);
operator()(size_t,size_t);
operator()(size_t,size_t,size_t);
...
template<class Iterator> operator()(Iterator,Iterator);
The problem is that operator()(size_t,size_t) is never reached, because the compiler is also able to template template<class Iterator> operator()(Iterator,Iterator). How can I avoid this?
An obvious solution is to use std::vector<size_t>::iterator instead of Iterator. I've tried this, but this narrows the usage elsewhere.
A minimal example:
#include <iostream>
class Foo
{
private:
double data[9];
public:
Foo(){};
double& operator()(size_t i, size_t j)
{
std::cout << "operator()(size_t i, size_t j)" << std::endl;
return data[0];
}
template<class Iterator>
double& operator()(Iterator first, Iterator last)
{
std::cout << "operator()(Iterator first, Iterator last)" << std::endl;
return data[0];
}
};
int main()
{
Foo bar;
bar(0,1);
}
Outputs:
operator()(Iterator first, Iterator last)
whereas I want this case to output
operator()(size_t i, size_t j)
It's almost a certainty that the reason your template gets selected by overload resolution is because the two parameters you're passing in are not really size_t. They're probably ints, or something else. If they were truly size_ts, then I would expect your non-template overload to be picked. Cleaning that up should make things work, as is, but it's simple enough to make this work in any case.
The usual approach in this kind of a situation is to use SFINAE to exclude the template from participating in overload resolution when the passed-in parameter is size_t. Something along the lines of (using C++17):
template<class Iterator,
typename=std::enable_if_t<
std::negation_v<std::is_integral_v<Iterator>>>> operator()(Iterator,Iterator)
{
// ...
}
This is your starting point. It's tempting to use std::is_same_v<Iterator,size_t>, but you'll quickly discover that this only works if you're passing in exactly a size_t. It's very easy for an int to slip in there, if you're not careful, and this is going to fall apart in this case. So you'll probably need to use std::is_integral_v. Hopefully you're not passing in floating point values anywhere, and rely on them being truncated to the next nearest integer value. If you do, you'll have to tweak this further.
The std::is_integral_v and std::enable_if_t shortcuts are available only in C++17 (as well as std::void_t), but it's simple enough to reinvent that wheel in earlier standards, if necessary.
You can also try using SFINAE in the opposite direction: have this template participate in overload resolution only if Iterator resolved to something that std::iterator_traits recognizes. The best approach depends on your specific class's requirements.
What I mean is the following. I want a template function that takes two vector iterators (or two pointers to array of double) and returns a double that is somehow related to the vector iterators or array pointers that I pass. However, I want this to work for double or int, or any arithmetic type.
I think I'm not allowed to say:
template <class T>
T* func(T Begin, T End)
T new_variable = Begin + 5;
return (*new_variable);
}
because the compiler won't understand what T* means. A solution I thought of is to take what I'm trying to return and make it a third argument:
template <class T>
void func(T Begin, T End, T* new_variable)
new_variable = Begin + 5;
return (*new_variable);
}
Will this work? Even if so, is there another way of doing what I'm trying to do? (Sorry if I haven't been clear enough.)
If you want to return a double (i.e the type that you would get when dereferencing), you can use the iterator traits:
template<typename RandomAccessIterator>
typename std::iterator_traits<RandomAccessIterator>::value_type
func(RandomAccessIterator a, RandomAccessIterator b) {
typedef typename std::iterator_traits<RandomAccessIterator>::value_type
value_type;
// use value_type now, when you want to save some temporary
// value into a local variable, for instance
value_type t = value_type();
for(; a != b; ++a) t += *a;
return t;
}
These work for all iterators, including pointers:
int main() {
int d[3] = { 1, 2, 3 };
assert(func(d, d + 3) == 6);
}
Well, your code seems to contradict what you described in the text. If T is the iterator type, then the result of the iterator dereference (as you said in the text) will not have type T * (as you seem to believe in the code). T * is a completely opposite thing: it is something you'd get if you took the address of your iterator, not dereferenced it.
In fact, there's no way to express the "dereferenced type" using C++ core language features (maybe decltype will do it in the future, as in decltype(*T())). The only way to describe the result of dereference of type T is to use a library-based solution: iterator traits, as Johannes explained in his answer.