How to write an infinite sequence compatible with std::ranges? - c++

I would like to write a struct which will generate an infinite sequence of fibonacci numbers in a way compatible with std::ranges and range adaptors.
So if I wanted the first 5 even fibonacci numbers, I would write something like this:
#include <iostream>
#include <ranges>
using namespace std;
int main() {
auto is_even = [](auto a) { return a % 2 == 0; };
for (auto v :
something | ranges::views::filter(is_even) | ranges::views::take(5))
std::cout << v << std::endl;
return 0;
}
What "something" needs to be ?
It seems to me that it has to be a class with a forward iterator, but I can't find any example.

Edit: As it was pointed out by 康桓瑋 in the comments, there has been a better, cleaner solution presented at Cppcon, link, by Tristan Brindle.
I think this could serve as a quick reference to make custom iterator-based generators.
By your requirements, something must be a std::ranges::view, meaning it must be a moveable std::ranges::range deriving from std::ranges::view_interface<something>.
We can tackle all three with the following:
#include <ranges>
template<typename T>
class fib : public std::ranges::view_interface<fib<T>>{
public:
struct iterator;
auto begin() const { return iterator{}; }
auto end() const { return std::unreachable_sentinel; }
};
Notice std::unreachable_sentinel which makes creating sequences without end really simple.
We still have to define the iterator which does the actual work. In your case we want fib to be "the source" of the values, so our iterator should actually be std::input_iterator. There's some boiler plate code needed for that but it's basically just a type which can be incremented and dereferenced to yield its current value.
Something like this will do:
#include <iterator>
template<typename T>
struct fib<T>::iterator {
using iterator_category = std::input_iterator_tag;
using value_type = T;
using difference_type = std::ptrdiff_t;
using pointer = T*;
using reference = T;
constexpr iterator() noexcept = default;
iterator& operator++() {
auto old_next = next;
next = next + current;
current = old_next;
return *this;
}
iterator operator++(int) {
iterator current{*this};
++(*this);
return current;
}
value_type operator*() const {
return current;
}
bool operator==(const iterator& other) const { return current == other.current && next==other.next; }
private:
T current= {};
T next = T{} + 1; // Could perhaps be fancier.
};
The increment operator does the computation itself, it's the simple iterating algorithm.
That's it, here's a working example:
#include <cstdint>
#include <iostream>
int main() {
auto is_even = [](auto a) { return a % 2 == 0; };
for (auto v :
fib<std::uint64_t>{} | std::ranges::views::filter(is_even) | std::ranges::views::take(10))
std::cout << v << std::endl;
return 0;
}
which outputs:
0
2
8
34
144
610
2584
10946
46368
196418
Of course you won't get very far even with std::uint64_t. But T can be anything numeric enough.
One can easily generalize the iterator to hold a stateful functor, likely passed from the range itself, call it during each increment and store the yielded value for dereferencing later. This would be very crude, but simple, way how to at least simulate "yield-based" generators.

Related

Can I use std::lower_bound on function calls instead iterators?

Simple use case: I want to find with binary search the minimal index i for which f(i)>=t, where t is some threshold and f is a monotonically increasing function over integer values.
A simple way to go would be to just call this function on every possible input, save it to a container and then use lower_bound, this would be useful in a scenario where I want to do multiple searches on the same function with different thresholds.
However, in my scenario, function evaluations are runtime expensive and I have multiple different functions/lambdas on which I only perform a single binary search.
So I guess, what I need, is either a lower_bound function taking a function and a value range instead of start and end iterators, or I need a mechanism to hide the function calls inside an iterator structure. I know the first solution is easy to implement, but I hoped for a solution that would avoid implementing binary search from scratch.
To me, this seemed like a common use case but somehow I failed to find anything on this particular problem on the web. I'd appreciate any tips, tricks, and links.
EDIT
I found the two first provided solutions very intriguing. The given solution using the comp argument is very elegant, however I forget to mention a reason why this approach does not work for me. In addition to long runtimes for a single function evaluation I also have the problem of having a large search space (e.g. over 10**15 integers) which makes it impractical to allocate a dummy vector for this purpose. I do not know how it would work with the boost increment iterators but I need something to work with std only anyway.
The second solution using a custom iterator is quite verbose though. I tested it with the large number I mentioned (and changing the ints to long longs) but this seems to be also to slow. It seems that lower_bound actually calls the operator++ several times to go from one location to the other, so the implementation of the std::lower_bound might already be the killer for my approach here (see below modified code example and output) and there is no way around a custom implementation (which I already have, no need here).
Thanks, however, for the insights, both answers showed me something new. And maybe someone can shine some more light on the points mentioned above, as I am definitely no expert on iterators or the implementation of lower_bound, maybe I used it wrong or it is something about the code example given by #idclev that makes it iterate this inefficiently through the numbers which I did not recognize.
Modified code example
#include <iostream>
#include <unordered_map>
#include <algorithm>
long long foo(long long i){ std::cout << "function evaluation:\t" << i << std::endl; return i;}
using function_type = long long(*)(long long);
template <function_type F>
struct fun_iterator {
using difference_type = size_t;
using value_type = long long;
using pointer = int*;
using reference = int&;
using iterator_category = std::forward_iterator_tag;
static std::unordered_map<long long,long long> m;
long long index;
fun_iterator(long long index) : index(index) {}
fun_iterator& operator++() {
std::cout << "operator++:\t" << index << std::endl;
++index;
return *this;
}
fun_iterator operator++(int x) {
fun_iterator it = *this;
++index;
return it;
}
int operator*() {
auto it = m.find(index);
if (it != m.end()) return it->second;
auto res = F(index);
m[index] = res;
return res;
}
bool operator!=(const fun_iterator& other){
return index != other.index;
}
bool operator==(const fun_iterator& other){
return index == other.index;
}
bool operator<(const fun_iterator& other){
return index < other.index;
}
};
template <function_type F>
std::unordered_map<long long,long long> fun_iterator<F>::m;
template <function_type F>
std::pair<fun_iterator<F>,fun_iterator<F>> make_begin_and_end(long long begin,long long end){
return {{begin},{end}};
}
int main() {
auto x = make_begin_and_end<foo>(0,10L);
auto it = std::lower_bound(x.first,x.second,4L);
// auto x = make_begin_and_end<foo>(0,1000000000000L);
// auto it = std::lower_bound(x.first,x.second,400000000000L);
std::cout << it.index << std::endl;
}
and the output:
operator++: 0
operator++: 1
operator++: 2
operator++: 3
operator++: 4
operator++: 5
operator++: 6
operator++: 7
operator++: 8
operator++: 9
operator++: 0
operator++: 1
operator++: 2
operator++: 3
operator++: 4
function evaluation: 5
operator++: 0
operator++: 1
function evaluation: 2
operator++: 2
operator++: 3
function evaluation: 4
function evaluation: 3
operator++: 3
4
As #KamilCuk suggested, write your own iterator, or alternatively
You can take any container of natural numbers (if you don't have ranges at hand, simply create an std::vector<int> and populate it with monotonically growing numbers — provided you at least know the boundaries of expected interval your expected value lies within). Next, std::lower_bound accepts a comp argument:
std::vector<int> args(1000);
std::iota(args.begin(), args.end(), 0);
root = std::lower_bound(args.cbegin(), args.cend(), t,
[](int x, int t){ return f(x) < t; });
(As a sanity check, check if root is args's begin — then 0 can be higher than your desired root — or is args's end — then the root is higher than the estimated right boundary.)
You can write your own iterator and to avoid unnessesary evaluations of the function, you can use memoization. The iterator can have a static map to cache results of function calls. To make iterators for different functions a different type, I parametrized the iterator on a function pointer:
#include <iostream>
#include <unordered_map>
#include <algorithm>
double foo(int i){ return i;}
using function_type = double(*)(int);
template <function_type F>
struct fun_iterator {
using difference_type = size_t;
using value_type = int;
using pointer = int*;
using reference = int&;
using iterator_category = std::forward_iterator_tag;
static std::unordered_map<int,double> m;
int index;
fun_iterator(int index) : index(index) {}
fun_iterator& operator++() {
++index;
return *this;
}
fun_iterator operator++(int x) {
fun_iterator it = *this;
++index;
return it;
}
int operator*() {
auto it = m.find(index);
if (it != m.end()) return it->second;
auto res = F(index);
m[index] = res;
return res;
}
bool operator!=(const fun_iterator& other){
return index != other.index;
}
bool operator==(const fun_iterator& other){
return index == other.index;
}
bool operator<(const fun_iterator& other){
return index < other.index;
}
};
template <function_type F>
std::unordered_map<int,double> fun_iterator<F>::m;
template <function_type F>
std::pair<fun_iterator<F>,fun_iterator<F>> make_begin_and_end(int begin,int end){
return {{begin},{end}};
}
int main() {
auto x = make_begin_and_end<foo>(0,100);
auto it = std::lower_bound(x.first,x.second,50);
std::cout << it.index;
}
The map will be reused also for later instances of fun_iterator for the same function. And because it is parametrized on the function pointer, a fun_iterator for a different function uses a different map to store the results.

`c++17` ranged for loop over std::vectors with value and index [duplicate]

Is there an equivalent to the range-based enumerate loop from python in C++?
I would imagine something like this.
enumerateLoop (auto counter, auto el, container) {
charges.at(counter) = el[0];
aa.at(counter) = el[1];
}
Can this be done with templates or macros?
I'm aware that I can just use an old school for-loop and iterate until I reach container.size(). But I'm interested how this would be solved using templates or macros.
EDIT
I played a bit with boost iterators after the hint in the comments. I got another working solution using C++14.
template <typename... T>
auto zip(const T &... containers) -> boost::iterator_range<boost::zip_iterator<
decltype(boost::make_tuple(std::begin(containers)...))>> {
auto zip_begin =
boost::make_zip_iterator(boost::make_tuple(std::begin(containers)...));
auto zip_end =
boost::make_zip_iterator(boost::make_tuple(std::end(containers)...));
return boost::make_iterator_range(zip_begin, zip_end);
}
template <typename T>
auto enumerate(const T &container) {
return zip(boost::counting_range(0, static_cast<int>(container.size())),
container);
}
https://gist.github.com/kain88-de/fef962dc1c15437457a8
Enumeration of multiple variables has been an idiom since C. The only complication is that you can't declare both variables in the initializer of the for loop.
int index;
for (auto p = container.begin(), index = 0; p != container.end(); ++p, ++index)
I don't think it gets any simpler (or more powerful) than that.
There is a pre C++11 solution in boost to this: boost.range.indexed.
Unfortunately it doesn't work with C++11 range based for-loops, only old style verbose loops. However with C++17 it should be become (almost) as easy as in python using structured bindings
Then it should be possible implement something that works like this:
for (auto& [n,x] : enumerate(vec)) x = n;
So, a bit of waiting still ;)
I wrote something for this a while back.
Essentially, you need to wrap an iterator and give it pair semantics.
AFAIK, there's nothing like this built into the language. And I don't think boost has it either. You pretty much have to roll your own.
// Wraps a forward-iterator to produce {value, index} pairs, similar to
// python's enumerate()
template <typename Iterator>
struct EnumerateIterator {
private:
Iterator current;
Iterator last;
size_t index;
bool atEnd;
public:
typedef decltype(*std::declval<Iterator>()) IteratorValue;
typedef pair<IteratorValue const&, size_t> value_type;
EnumerateIterator()
: index(0), atEnd(true) {}
EnumerateIterator(Iterator begin, Iterator end)
: current(begin), last(end), index(0) {
atEnd = current == last;
}
EnumerateIterator begin() const {
return *this;
}
EnumerateIterator end() const {
return EnumerateIterator();
}
EnumerateIterator operator++() {
if (!atEnd) {
++current;
++index;
atEnd = current == last;
}
return *this;
}
value_type operator*() const {
return {*current, index};
}
bool operator==(EnumerateIterator const& rhs) const {
return
(atEnd && rhs.atEnd) ||
(!atEnd && !rhs.atEnd && current == rhs.current && last == rhs.last);
}
bool operator!=(EnumerateIterator const& rhs) const {
return !(*this == rhs);
}
explicit operator bool() const {
return !atEnd;
}
};
template<typename Iterable>
EnumerateIterator<decltype(std::declval<Iterable>().begin())> enumerateIterator(Iterable& list) {
return EnumerateIterator<decltype(std::declval<Iterable>().begin())>(list.begin(), list.end());
}
template<typename ResultContainer, typename Iterable>
ResultContainer enumerateConstruct(Iterable&& list) {
ResultContainer res;
for (auto el : enumerateIterator(list))
res.push_back(move(el));
return res;
}
C++17 and structured bindings makes this look OK - certainly better than some ugly mutable lambda with a local [i = 0](Element&) mutable or whatever I've done before admitting that probably not everything should be shoehorned into for_each() et al. - and than other solutions that require a counter with scope outside the for loop.
for (auto [it, end, i] = std::tuple{container.cbegin(), container.cend(), 0};
it != end; ++it, ++i)
{
// something that needs both `it` and `i`ndex
}
You could make this generic, if you use this pattern often enough:
template <typename Container>
auto
its_and_idx(Container&& container)
{
using std::begin, std::end;
return std::tuple{begin(container), end(container), 0};
}
// ...
for (auto [it, end, i] = its_and_idx(foo); it != end; ++it, ++i)
{
// something
}
C++ Standard proposal P2164 proposes to add views::enumerate, which would provide a view of a range giving both reference-to-element and index-of-element to a user iterating it.
We propose a view enumerate whose value type is a struct with 2 members index and value representing respectively the position and value of the elements in the adapted range.
[ . . .]
This feature exists in some form in Python, Rust, Go (backed into the language), and in many C++ libraries: ranges-v3, folly, boost::ranges (indexed).
The existence of this feature or lack thereof is the subject of recurring stackoverflow questions.
Hey, look! We're famous.
You can also more elegantly use the auto ranges available since C++11:
int i = 0;
for (auto& el : container){
charges.at(counter) = el[0];
aa.at(counter) = el[1];
++i;
}
You still have to count the i up by hand, though.
Here's a macro-based solution that probably beats most others on simplicity, compile time, and code generation quality:
#include <iostream>
#define fori(i, ...) if(size_t i = -1) for(__VA_ARGS__) if(i++, true)
int main() {
fori(i, auto const & x : {"hello", "world", "!"}) {
std::cout << i << " " << x << std::endl;
}
}
Result:
$ g++ -o enumerate enumerate.cpp -std=c++11 && ./enumerate
0 hello
1 world
2 !
Tobias Widlund wrote a nice MIT licensed Python style header only enumerate (C++17 though):
GitHub
Blog Post
Really nice to use:
std::vector<int> my_vector {1,3,3,7};
for(auto [i, my_element] : en::enumerate(my_vector))
{
// do stuff
}
Boost::Range supports this as of 1.56.
#include <boost/range/adaptor/indexed.hpp>
#include <boost/assign.hpp>
#include <iterator>
#include <iostream>
#include <vector>
int main(int argc, const char* argv[])
{
using namespace boost::assign;
using namespace boost::adaptors;
std::vector<int> input;
input += 10,20,30,40,50,60,70,80,90;
// for (const auto& element : index(input, 0)) // function version
for (const auto& element : input | indexed(0))
{
std::cout << "Element = " << element.value()
<< " Index = " << element.index()
<< std::endl;
}
return 0;
}

Reversing range-based for loop on a custom container class

I'm trying to improve my C++ skills by porting the major examples in Algorithms, 4th Edition by Sedgewick and Wayne. I wrote a generic stack implementation based on their Java example.
My stack works fine, but I'd like to make a performance improvement and got stuck trying to write a reverse iterator.
template<typename T> class ResizingArrayStack {
public:
T* begin() { return &array_ptr[0]; }
T* end() { return &array_ptr[N]; }
...
// Here we're iterating forward through the array, with an unused variable `i`.
// It would be nice performance-wise to iterate in reverse without calling pop(), and without triggering a resize.
for ( auto& i : lifo_stack ) {
cout << "Current loop iteration has i = " << i << endl;
}
// // Alternatively, pop from the stack N times.
// cout << "Popped an item from the stack: " << lifo_stack.pop() << endl;
I tried switching the begin and end member functions above, but found that the expanded for-loop always increments with ++__begin, even if __end is at a lower memory address. How can we get i to loop in reverse (LIFO with respect to the stack)?
Please feel free to comment on my code style if there are egregious errors or aspects that look out of date. I want stay in-line with good 'modern' C++.
If you want to use the range-for loop with reverse iterators, you can use a wrapper class Reverse that stores a range and returns the reverse_iterators corresponding to begin and end
#include <iostream>
#include <iterator>
#include <vector>
template<class Rng>
class Reverse
{
Rng const& rng;
public:
Reverse(Rng const& r) noexcept
:
rng(r)
{}
auto begin() const noexcept { using std::end; return std::make_reverse_iterator(end(rng)); }
auto end() const noexcept { using std::begin; return std::make_reverse_iterator(begin(rng)); }
};
int main()
{
std::vector<int> my_stack;
my_stack.push_back(1);
my_stack.push_back(2);
my_stack.push_back(3);
// prints 3,2,1
for (auto const& elem : Reverse(my_stack)) {
std::cout << elem << ',';
}
}
Live Example
Note that this uses C++1z template deduction, only supported by g++ 7.0 SVN and clang 5.0 SVN. For earlier compilers you could add a helper function
template<class Rng>
auto MakeReverse(Rng const& rng) { return Reverse<Rng>(rng); }
for (auto const& elem : MakeReverse(my_stack)) {
std::cout << elem << ',';
}
Live Example (works as of gcc 5.1 or clang 3.5)
Alternatively, you can use the Boost.Range library and simply do (will work any C++11 compiler)
#include <iostream>
#include <vector>
#include <boost/range/adaptor/reversed.hpp>
int main()
{
std::vector<int> my_stack;
my_stack.push_back(1);
my_stack.push_back(2);
my_stack.push_back(3);
for (auto const& elem : boost::adaptors::reverse(my_stack)) {
std::cout << elem << ',';
}
}
Live Example
Note that you have to be careful about passing temporary variables to such adaptors, both mine and the Boost adaptor do not work when passing e.g. a raw std::vector<int>{3,2,1}, as pointed out by #Pixelchemist in the comments.
Here a scratch for your problem. Don't consider this as working code. Use it to just get an idea of how reverse iterator MAY be implemented (just one many possible ways).
template<typename T> class ResizingArrayStack {
public:
class reverse_iterator
{
ResizingArrayStack & _storage;
int _pointer;
public:
inline reverse_iterator(ResizingArrayStack & storage,
int pointer)
: _storage(storage)
, _pointer(pointer)
{}
inline reverse_iterator & operator++() // prefix
{
--_pointer;
return *this;
}
inline reverse_iterator operator++() // postfix
{
reverse_iterator tmp(*this);
--_pointer;
return tmp;
}
inline T & operator*()
{
return _storage.getByIndex(_pointer);
}
// TODO: == != etc
};
reverse_iterator rbegin() { return reverse_iterator(*this, N - 1); }
reverse_iterator rend() { return reverse_iterator(*this, -1); }
// ... //
};
once you have functioning (regular) iterators,
implement reverse iterators using the standard library
helper class template std::reverse_iterator
#include <iterator>
class XX {
// your code
typedef std::reverse_iterator<iterator> reverse_iterator;
reverse_iterator rbegin() { return reverse_iterator{end()}; }
reverse_iterator rend() { return reverse_iterator{begin()}; }
Looking at your full codelifo_stack.pop() invalidates your iterators, so it cannot be used inside a ranged for. You have Undefined Behavior
Moreover it doesn't make much sense to use a range for for a stack. If you can iterate over its elements then it's not a stack now isn't it? A stack has the property that you can only access the most recent inserted element.
Based on your comment:
Consider the case where you add items slowly and individually, but
wish to dump them out of the stack as quickly as possible. I don't
want the overhead of copying and resizing arrays which pop() would
trigger at that moment.
I still think that ranged-for does not make sense for a stack.
Here is how I see your problem solved:
lifo_stack.disable_resizing(); // prevents resizing
while (!lifo_stack.is_empty()
{
lifo_stack.pop(); // maybe use the poped element
}
lifo_stack.enable_resizing(); // re-enables resizing and triggers a resize
If you don't need the popped elements and just want to emtpy the stack, there is a faster way (based on your class implementation):
// empties the stack
void clear()
{
delete[] array_ptr;
array_ptr = new T[1];;
max_size = 1;
N = 0;
}
One last final though if you want to use modern C++ then use unique_ptr instead of manual new and delete. It is easier but most importantly safer. And read on the rule of 0/3/5.
This solution does not introduce unnecessary copies and does not exhibit incorrect forwarding as suggested by some comments. Explanation below.
You can use some wrapper which has begin and end functions that actually
return reverse iterators.
template<class T>
struct revert_wrapper
{
T o;
revert_wrapper(T&& i) : o(std::forward<T>(i)) {}
};
template<class T>
auto begin(revert_wrapper<T>& r)
{
using std::end;
return std::make_reverse_iterator(end(r.o));
}
template<class T>
auto end(revert_wrapper<T>& r)
{
using std::begin;
return std::make_reverse_iterator(begin(r.o));
}
template<class T>
auto begin(revert_wrapper<T> const& r)
{
using std::end;
return std::make_reverse_iterator(end(r.o));
}
template<class T>
auto end(revert_wrapper<T> const& r)
{
using std::begin;
return std::make_reverse_iterator(begin(r.o));
}
template<class T>
auto reverse(T&& ob)
{
return revert_wrapper<T>{ std::forward<T>(ob) };
}
Used like this:
std::vector<int> v{1, 2, 3, 4};
for (auto i : reverse(v))
{
std::cout << i << "\n";
}
or in your case
for ( auto& i : reverse(lifo_stack) ) {
cout << "Current loop iteration has i = " << i << endl;
cout << "Popped an item from the stack: " << lifo_stack.pop() << endl;
}
Since fowarding is not an easy topic and there is misconception around I'll further explain some details. I'll use std::vector<int> as an example for the "to be reversed" type T.
1. The function template reverse.
1.1 Passing an lvalue std::vector<int>:
std::vector<int> v{1, 2, 3, 4};
auto&& x = reverse(v);
The compiler created instance of reverse in this case would look like:
template<>
auto reverse<std::vector<int>&>(std::vector<int>& ob)
{
return revert_wrapper<std::vector<int>&>{ std::forward<std::vector<int>&>(ob) };
}
We see two things here:
The T of revert_wrapper will be std::vector<int>&, so no copy involved.
we're forwarding an lvalue as an lvalue to the constructor of revert_wrapper
1.2 Passing an rvalue std::vector<int>
std::vector<int> foo();
auto&& x = reverse(foo());
We look again at the instantiation of the function template:
template<>
auto reverse<std::vector<int>>(std::vector<int>&& ob)
{
return revert_wrapper<std::vector<int>>{ std::forward<std::vector<int>>(ob) };
}
And can again note two things:
The T of revert_wrapper will be std::vector<int>, thus copy the vector, preventing the rvalue from going out of scope before any range based loop can run
an rvalue std::vector<int>&& will be forwarded to the constructor of revert_wrapper
2. The class template revert_wrapper and its constructor
2.1 The revert_wrapper created by reverse in case of an lvalue std::vector<int>&
template<>
struct revert_wrapper<std::vector<int>&>
{
std::vector<int>& o;
revert_wrapper(std::vector<int>& i) :
o(std::forward<std::vector<int>&>(i)) {}
};
As noted above: No copies involved as we store a reference.
The forward also seems familiar and indeed it is just the same as above within reverse: We forward an lvalue as lvalue reference.
2.2 The revert_wrapper created by reverse in case of an rvalue std::vector<int>&&
template<>
struct revert_wrapper<std::vector<int>>
{
std::vector<int> o;
revert_wrapper(std::vector<int>&& i) :
o(std::forward<std::vector<int>>(i)) {}
};
This time we have the object stored by value to prevent a dangling reference.
Also the forwarding is fine: We forwarded the rvalue reference from reverse to the revert_wrapper constructor and we forward it on to the std::vector constructor. We could've used static_cast<T&&>(i) in the same way but we're not (std::)mov(e)ing from an lvalue, we're forwarding:
lvalues as lvalues and
rvalues as rvalues.
We can also see one more thing here:
The only available constructor of the revert_wrapper instance that stores by value takes an rvalue. Therefore, we can't (easily) trick this class to make unnecessary copies.
Note that replacing std::forward with std::move inside the initializer of o in the revert_wrapper constructor would actually be wrong.
Please see an excellent answer from TemplateRex here. I was able to solve the problem without a wrapper class, so I'll give a shot at answering my own question.
Here is the most helpful example I found on implementing iterators at http://en.cppreference.com, and you can find my updated ResizingArrayStack code at the same GitHub link as found the question.
template<typename T> class ResizingArrayStack {
public:
//----- Begin reversed iteration section -----//
// Please see the example here, (http://en.cppreference.com/w/cpp/iterator/iterator).
// Member typedefs inherit from std::iterator.
class stackIterator: public std::iterator<
std::input_iterator_tag, // iterator_category
T, // value_type
T, // difference_type
const T*, // pointer
T // reference
>{
int index = 0;
T* it_ptr = nullptr;
public:
// Prefix ++, equal, unequal, and dereference operators are the minimum required for range based for-loops.
stackIterator(int _index = 0, T* _it_ptr = nullptr) { index = _index; it_ptr = _it_ptr; }
// Here is where we reverse the sequence.
stackIterator& operator++() { --index; return *this; }
bool operator==(stackIterator other) { return index == other.index; }
bool operator!=(stackIterator other) { return !( *this == other ); }
T operator*() { return it_ptr[index-1]; }
};
stackIterator begin() { return stackIterator(N, array_ptr); }
stackIterator end() {
N = 0; // 'Empty' the array.
max_size = 1; // Don't waste time calling resize() now.
return stackIterator(0, array_ptr);
}
//----- End reversed iteration section -----//
private:
// Allocate space for a traditional array on the heap.
T* array_ptr = new T[1];
// Keep track of the space allocated for the array, max_size * sizeof(T).
int max_size = 1;
// Keep track of the current number of items on the stack.
int N = 0;
Calling code where the range based for-loop iterates in reversed (or LIFO) order by default.
// It's nice performance-wise to iterate in reverse without calling pop() or triggering a resize.
for ( auto i : lifo_stack) {
cout << "Current loop iteration has i = " << i << endl;
}

Finding minimum element based on a transformed value

Here is the task came to me from a code review. I want to select a minimum value from a set, based on a special kind of compare predicate. Like this:
struct Complex { ... };
float calcReduction(Complex elem);
Complex findMinValueWithPredicates(const std::vector<Complex>& values)
{
auto it = std::min_element(values.begin(), values.end(),
[](const Complex& a, const Complex& b) {
return calcReduction(a) < calcReduction(b);
});
if (it == values.end()) throw std::runtime_error("");
return *it;
}
Here I find the minimum element based on a predicate. This predicate computes a reduction of both values to float and then compares those floats. Works fine, looks neat.
Can you see the problem? Yes, for a set of N elements calcReduction() is called 2N times, while it is enough to compute it only N times - once for each element.
One way to solve this problem is to write explicit computations:
Complex findMinValueExplicit(const std::vector<Complex>& values)
{
float minReduction = std::numeric_limits<float>::max();
Complex minValue;
for (Complex value : values)
{
float reduction = calcReduction(value);
if (reduction < minReduction)
{
minReduction = reduction;
minValue = value;
}
}
if (minReduction == std::numeric_limits<float>::max()) throw std::runtime_error("");
return minValue;
}
It works fine and we only have N calls to calcReduction(). However, it looks too verbose and the intent is not such clear, as compared to explicit call of min_element. Because when you call min_element it is really easy to guess you are going to find a minimum element, you know.
The only idea I have for now is to create my own algorithm like min_element_with_reduction, accepting a range and a reduction function. Sounds reasonable, but I wonder whether there are any ready solutions.
Any ideas on how to solve this task with clear intent and some ready solutions? Boost is welcomed. C++17 and ranges are interesting to see.
You could use boost::range library.
auto reductionLambda = [](const Complex& a) { return calcReduction(a); };
auto it = boost::range::min_element(values | boost::adaptors::transformed(
std::ref(reductionLambda));
Ranges themselves should be coming to the standard C++ with C++17 as well.
Edit
As we figured out in comments, this would also make the conversion twice.
So here's something fun:
#include <boost/iterator/iterator_adaptor.hpp>
#include <boost/assign.hpp>
#include <algorithm>
#include <iostream>
#include <vector>
#include <functional>
template <class Iterator, class UnaryFunction>
class memoizing_transform_iterator
: public boost::iterator_adaptor<
memoizing_transform_iterator<Iterator, UnaryFunction> // Derived
, Iterator // Base
, std::decay_t<decltype(std::declval<UnaryFunction>()(std::declval<typename Iterator::value_type>()))> // Value
, boost::forward_traversal_tag // CategoryOrTraversal
>
{
public:
memoizing_transform_iterator() {}
explicit memoizing_transform_iterator(Iterator iter, UnaryFunction f)
: memoizing_transform_iterator::iterator_adaptor_(iter), fun(f) {}
static int total;
private:
friend class boost::iterator_core_access;
void increment() { ++this->base_reference(); memoized = false; }
using MemoType = std::decay_t<decltype(std::declval<UnaryFunction>()(std::declval<typename Iterator::value_type>()))>;
MemoType& dereference() const
{
if (!memoized) {
++total;
memoized = true;
memo = fun(*this->base());
}
return memo;
}
UnaryFunction fun;
mutable bool memoized = false;
mutable MemoType memo;
};
template <class Iterator, class UnaryFunction>
auto make_memoizing_transform_iterator(Iterator i, UnaryFunction&& f)
{
return memoizing_transform_iterator<Iterator, UnaryFunction>(i, f);
}
template<class I, class U>
int memoizing_transform_iterator<I, U>::total = 0;
// THIS IS COPIED FROM LIBSTDC++
template<typename _ForwardIterator>
_ForwardIterator
min_el(_ForwardIterator __first, _ForwardIterator __last)
{
if (__first == __last)
return __first;
_ForwardIterator __result = __first;
while (++__first != __last)
if (*__first < *__result)
__result = __first;
return __result;
}
int main(int argc, const char* argv[])
{
using namespace boost::assign;
std::vector<int> input;
input += 2,3,4,1,5,6,7,8,9,10;
auto transformLambda = [](const int& a) { return a*2; };
auto begin_it = make_memoizing_transform_iterator(input.begin(), std::ref(transformLambda));
auto end_it = make_memoizing_transform_iterator(input.end(), std::ref(transformLambda));
std::cout << *min_el(begin_it, end_it).base() << "\n";
std::cout <<begin_it.total;
return 0;
}
Basically I implemented an iterator that memoizes the result of calling the transformation functor. The weird part though is that at least in online compilers, the iterators are copied before their dereferenced values are compared (thus defeating the purpose of memoizing). However when I simply copied the implementation from libstdc++, it works as expected. Perhaps you could try it out on a real machine? The live example is here.
Small edit:
I tested on VS2015 and it works as expected with std::min_element.
Here's a solution using (already works with the range-v3 library, the implementation by the author of the upcoming Ranges TS)
#include <range/v3/all.hpp>
#include <iostream>
#include <limits>
using namespace ranges::v3;
int main()
{
auto const expensive = [](auto x) { static int n; std::cout << n++ << " "; return x; };
auto const v = view::closed_iota(1,10) | view::transform(expensive);
auto const m1 = *min_element(v);
std::cout << "\n" << m1 << "\n";
auto const inf = std::numeric_limits<int>::max();
auto const min = [](auto x, auto y) { return std::min(x, y); };
auto const m2 = accumulate(v, inf, min);
std::cout << "\n" << m2 << "\n";
}
Live On Coliru with output:
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
1
19 20 21 22 23 24 25 26 27 28
1
As you can see, using min_element takes 2N comparisons, but using accumulate only N.
The only thing missing is the meta-iterator.
A meta-iterator takes an iterator, and creates an iterator that contains a copy of it. It passes all operations through to the contained iterator, except when dereferenced returns a copy of the contained iterator instead.
With any care, the code used for this also works to create an iterator over size_t or int or similar torsor-likes.
template<class It, class R>
struct reduced_t {
It it;
R r;
friend bool operator<( reduced_t const& lhs, reduced_t const& rhs ) {
return lhs.r < rhs.r;
}
};
template<class It, class F>
reduced_t<It, std::result_of_t<F(typename std::iterator_traits<It>::reference)>>
reducer( It it, F&& f ) {
return {it, std::forward<F>(f)(*it)};
}
template<class It, class F>
It reduce( It begin, It end, F&& f ) {
if (begin==end)
return begin;
return std::accumulate(
meta_iterator(std::next(begin)), meta_iterator(end),
reducer(begin, f),
[&](
auto&& reduced, // reduced_t<blah...> in C++11
It i
) {
auto r2 = reducer( i, f );
return (std::min)(reduced, r2);
}
).it;
};
f(*it) is called exactly once per iterator.
I wouldn't call this ... obvious. The trick is that we use accumulate and meta-iterators to implement min_element, then we can have accumulate operate on transformed elements (which gets called once, and returned).
You could rewrite it in stack-based programming style using primitives, but there are lots of primitives to write. Maybe post ranges-v3.
At this point, I'm imagining having some high-powered compositional programming library. If I did, we could do the following:
reducer( X, f ) can be rewritten graph( deref |then| f )(X) using order_by( get_n_t<1> ) for ordering.
The accumulate call could read accumulate( skip_first(range), g(begin(range)), get_least( order_by( get_n_t<1> ) ) ).
Not sure if that is any clearer.
If you take a minElem as a lambda parameter you could use min_element
Complex findMinValueWithPredicates(const std::vector<Complex>& values)
{
float minElem = std::numeric_limits<float>::max();
auto it = std::min_element(values.begin(), values.end(),
[&minElem](const Complex& a, const Complex& b) {
float tmp = calcReduction(a);
if (tmp < minElem) {
minElem = tmp;
return true;
}
return false;
});
if (it == values.end()) throw std::runtime_error("");
return *it;
}
Edit:
Why does this work when bis not used?
25.4.7.21 min_element
21 Returns: The first iterator i in the range [first,last) such that
for every iterator j in the range [first,last) the following
corresponding conditions hold: !(*j < *i) or comp(*j, *i) == false.
Returns last if first == last.
because b should have been named smallestYet (code from cplusplus.com)
template <class ForwardIterator>
ForwardIterator min_element ( ForwardIterator first, ForwardIterator last )
{
if (first==last) return last;
ForwardIterator smallest = first;
while (++first!=last)
if (*first<*smallest) // or: if (comp(*first,*smallest)) for version (2)
smallest=first;
return smallest;
}
Which lead me to a new favourite quote:
"There are only 10 hard problems in Computer Science:
cache invalidation, naming things and off-by-one errors."
one commented on that we might be off-by-one as we don't use b.
I worried that the minElem cached might not be correct.
And I realized that the name b should have been more meaningful as it is "dereferenced iterator to smallest element yet" or smallestYet.
Finally that not all understand binary when its not written with a ´b´ at the end.
Here is another option, but it is still effectively your second solution. To be honest it doesn't look clear, but someone might like it. (I use std::pair<float, Complex> to store reduction result and the value that was reduced.)
std::pair<float, Complex> result{std::numeric_limits<float>::max(), {}};
auto output_function = [&result](std::pair<float, Complex> candidate) {
if (candidate.first < result.first)
result = candidate;
};
std::transform(values.begin(), values.end(),
boost::make_function_output_iterator(output_function),
[](Complex x) { return std::make_pair(calcReduction(x), x); });
P.S. If your calcReduction costs a lot, have you considered caching results in Complex objects? It will lead to a slightly more complicated implementation, but you'll be able to use plain std::min_element which makes your intentions clear.

itertools.tee equivalent in Boost::Range?

Python's itertools has tee for n-plicating iterables:
def tee(iterable, n=2):
it = iter(iterable)
deques = [collections.deque() for i in range(n)]
def gen(mydeque):
while True:
if not mydeque: # when the local deque is empty
newval = next(it) # fetch a new value and
for d in deques: # load it to all the deques
d.append(newval)
yield mydeque.popleft()
return tuple(gen(d) for d in deques)
I couldn't find the equivalent in Boost::Range. Am I missing something or should I just roll my own?
itertools.tee is well-suited for single pass iterables which are ubiquitous in Python. For instance Generators are single pass, and they are often used.
But if you already have list/deque you will not use itertools.tee for it, because it would involve superfluous duplication - you can just iterate over original list/deque over and over again.
C++ also has concept of single pass ranges, for instance Input Iterator, but they are not so ubiquitous. It is consequence of another set of aims of typical C++ program - give to user maximum as possible maintaining best performance. It is another mindset if you wish.
To illustrate this let's compare boost::transformed and itertools.imap (or generator expressions):
They both provide view of input sequence via given "prism". itertools.imap returns single pass iterable, while boost::transformed returns range view which has same category as input range - i.e., if you would pass Random Access Range as input you would get Random Access Range as the result.
Another fact is that C++ employs value semantics by default, while python has pointer semantics. It means that if copy iterator in C++, and "bump" it several times - original iterator will not be changed (though it can be invalidated if it is single pass range, but it is not the point).
But, sometimes you do want to accumulate values from single pass range and look at them several times. In such case, most common solution is to accumulate values to some container explicitly, by hands. For instance:
vector<int> cache(first,last);
However, tee-like wrappers are still possible in C++, here is proof-of-concept. Usage is:
auto forward_range = tee_range(first,last);
tee_range takes single pass range as argument and returns forward range (which is multi-pass) (there is also make_tee_iterator, which works at iterator level). So, you can take copies of that range and iterate it several times:
auto r = forward_range;
auto out = ostream_iterator<int>(cout," ");
copy(forward_range,out);
copy(forward_range,out);
copy(r,out);
Therer is also improvenment over itertools.tee - internally, only one deque is used to cache values.
live demo:
#include <boost/range/adaptor/transformed.hpp>
#include <boost/iterator/iterator_facade.hpp>
#include <boost/smart_ptr/make_shared.hpp>
#include <boost/range/iterator_range.hpp>
#include <boost/smart_ptr/shared_ptr.hpp>
#include <boost/container/vector.hpp>
#include <boost/container/deque.hpp>
#include <boost/range/algorithm.hpp>
#include <algorithm>
#include <iterator>
#include <cassert>
#include <limits>
template<typename InputIterator>
class tee_iterator : public boost::iterator_facade
<
tee_iterator<InputIterator>,
const typename std::iterator_traits<InputIterator>::value_type,
boost::forward_traversal_tag
>
{
typedef typename std::iterator_traits<InputIterator>::value_type Value;
typedef unsigned Index;
struct Data
{
boost::container::deque<Value> values;
boost::container::vector<tee_iterator*> iterators;
InputIterator current,end;
Index min_index, current_index;
Index poped_from_front;
//
Data(InputIterator first,InputIterator last)
: current(first), end(last), min_index(0), current_index(0), poped_from_front(0)
{}
~Data()
{
assert(iterators.empty());
}
};
boost::shared_ptr<Data> shared_data;
Index index;
static Index get_index(tee_iterator *p)
{
return p->index;
}
public:
tee_iterator()
: index(std::numeric_limits<Index>::max())
{}
tee_iterator(InputIterator first,InputIterator last)
: shared_data(boost::make_shared<Data>(first,last)), index(0)
{
shared_data->iterators.push_back(this);
}
tee_iterator(const tee_iterator &x)
: shared_data(x.shared_data), index(x.index)
{
if(shared_data)
shared_data->iterators.push_back(this);
}
friend void swap(tee_iterator &l,tee_iterator &r)
{
using std::swap;
*boost::find(l.shared_data->iterators,&l) = &r;
*boost::find(r.shared_data->iterators,&r) = &l;
swap(l.shared_data,r.shared_data);
swap(l.index,r.index);
}
tee_iterator &operator=(tee_iterator x)
{
swap(x,*this);
}
~tee_iterator()
{
if(shared_data)
{
erase_from_iterators();
if(!shared_data->iterators.empty())
{
using boost::adaptors::transformed;
shared_data->min_index = *boost::min_element(shared_data->iterators | transformed(&get_index));
Index to_pop = shared_data->min_index - shared_data->poped_from_front;
if(to_pop>0)
{
shared_data->values.erase(shared_data->values.begin(), shared_data->values.begin()+to_pop);
shared_data->poped_from_front += to_pop;
}
}
}
}
private:
friend class boost::iterator_core_access;
void erase_from_iterators()
{
shared_data->iterators.erase(boost::find(shared_data->iterators,this));
}
bool last_min_index() const
{
return boost::count
(
shared_data->iterators | boost::adaptors::transformed(&get_index),
shared_data->min_index
)==1;
}
Index obtained() const
{
return Index(shared_data->poped_from_front + shared_data->values.size());
}
void increment()
{
if((shared_data->min_index == index) && last_min_index())
{
shared_data->values.pop_front();
++shared_data->min_index;
++shared_data->poped_from_front;
}
++index;
if(obtained() <= index)
{
++shared_data->current;
if(shared_data->current != shared_data->end)
{
shared_data->values.push_back(*shared_data->current);
}
else
{
erase_from_iterators();
index=std::numeric_limits<Index>::max();
shared_data.reset();
}
}
}
bool equal(const tee_iterator& other) const
{
return (shared_data.get()==other.shared_data.get()) && (index == other.index);
}
const Value &dereference() const
{
if((index==0) && (obtained() <= index))
{
shared_data->values.push_back(*(shared_data->current));
}
assert( (index-shared_data->poped_from_front) < shared_data->values.size());
return shared_data->values[index-shared_data->poped_from_front];
}
};
template<typename InputIterator>
tee_iterator<InputIterator> make_tee_iterator(InputIterator first,InputIterator last)
{
return tee_iterator<InputIterator>(first,last);
}
template<typename InputIterator>
boost::iterator_range< tee_iterator<InputIterator> > tee_range(InputIterator first,InputIterator last)
{
return boost::iterator_range< tee_iterator<InputIterator> >
(
tee_iterator<InputIterator>(first,last),
tee_iterator<InputIterator>()
);
}
// _______________________________________________________ //
#include <iostream>
#include <ostream>
#include <sstream>
int main()
{
using namespace std;
stringstream ss;
ss << "1 2 3 4 5";
istream_iterator<int> first(ss /*cin*/ ),last;
typedef boost::iterator_range< tee_iterator< istream_iterator<int> > > Range; // C++98
Range r1 = tee_range(first,last);
Range r2 = r1, r3 = r1;
boost::copy(r1,ostream_iterator<int>(cout," "));
cout << endl;
boost::copy(r2,ostream_iterator<int>(cout," "));
cout << endl;
boost::copy(r2,ostream_iterator<int>(cout," "));
}
Output is:
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
Boost.Spirit has Multi Pass iterator which has similar aims.
The multi_pass iterator will convert any input iterator into a forward iterator suitable for use with Spirit.Qi. multi_pass will buffer data when needed and will discard the buffer when its contents is not needed anymore. This happens either if only one copy of the iterator exists or if no backtracking can occur.