Using range-based for loop with a third-party container - c++

I'm currently using a third-party library which contains a class that only provides indexed lookup, i.e. operator[].
I'd like to do a range-based for loop on this class's contents. However, having never written an iterator or iterator adapter, I'm quite lost. It seems that writing iterators is not a straightforward task.
My desired usage is:
for(auto element : container)
{
...
}
Instead of having to write:
for(int i = 0; i < container.size(); ++i)
{
auto element = container[i];
...
}
How can this be achieved? Does Boost provide this functionality?

Writing iterators is actually a rather straightforward task, but it gets extremely tedious. Since your container supports indexing by an integer, I assume its iterators would fall into the random access iterator category (if they existed). That needs a lot of boilerplate!
However, to support the range-based for loop, all you'll need is a forward iterator. We'll write a simple wrapper for the container that implements the forward iterator requirements, and then write two functions Iterator begin(Container&) and Iterator end(Container&) that enable the container to be used in the range-based for loop.
This Iterator will contain a reference to the container, the size of the container, and the current index within that container:
template<template<typename> class C, typename T>
class indexer : public std::iterator<std::forward_iterator, T>
{
public:
indexer(C<T>& c, std::size_t i, std::size_t end)
: c_(std::addressof(c)), i_(i), end_(end) {}
T& operator*() const {
return c_[i_];
}
indexer<C, T>& operator++() {
++i_;
return *this;
}
indexer<C, T> operator++(int) {
auto&& tmp = *this;
operator++();
return tmp;
}
bool operator==(indexer<C, T> const& other) const {
return i_ == other.i_;
}
bool operator!=(indexer<C, T> const& other) const {
return !(*this == other);
}
private:
C<T>* c_;
std::size_t i_, end_;
};
Inheriting from std::iterator conveniently declares the appropriate typedefs for use with std::iterator_traits.
Then, you would define begin and end as follows:
template<typename T>
indexer<Container, T> begin(Container<T>& c) {
return indexer<Container, T>(c, 0, c.size());
}
template<typename T>
indexer<Container, T> end(Container<T>& c) {
auto size = c.size();
return indexer<Container, T>(c, size, size);
}
Switch out Container for whatever the type of container is in your example, and with that, your desired usage works!
The requirements and behavior of all the various kinds of iterators can be found in the tables of section 24.2.2 of the standard, which are mirrored at cppreference.com here.
A random-access iterator implementation of the above, along with a demo of usage with a simple vector_view class can be found live on Coliru or ideone.com.

You can do the following:
1) define your own iterator It that contains, inside, a ref ref to your container container and an index i. When the iterator is dereferenced, it returns ref[i] by reference. You can write it yourself or you can use boost for help, it has an iterator library to easily define your own iterators. Constructor should accept a container& and a size_t. You can make also the const_iterator if this concept applies.
2) When operator++ is invoked on one iterator, it simply does ++i on the internal member. operator== and operator!= should simply compare i. You can assert, for security, that both iterators are coherent, that means their refs point to the same object.
3) add begin and end to your container class or, if this is not possible, define a global begin and end that accept your container& c. begin should simply return It(c, 0). end should return It(c, c.size()).
There could be a problem copying the iterators as they contain a reference and some other minor details, but I hope the overall idea is clear and correct.

Related

Designing a hierarchy of iterators

Let's suppose that I am coding a library, and I want to create a hierarchy of iterators in the same way that they are specified in the standard.
input iterator <- forward iterator <- random access iterator <- bidirectional... <- contiguous...
|| output iterator
I would like to offer a set of "generic iterators", that could be easily used for contiguous containers, like Array, to help the client to reduce the code that needs to write for an iterator.
So, first of all, I replicated std::iterator in my library.
export module iterator:legacy_iterator;
import std;
export namespace zero::iterator::legacy {
template<
iter_concepts::std_iterator_category Category,
typename T,
typename pointer_type = T*,
typename reference_type = T&
> struct iterator {
using value_type = T;
using iterator_category = Category;
using pointer = pointer_type;
using reference = reference_type;
using difference_type = std::ptrdiff;
};
}
the iterator_category concept is just a concept that makes sure that the client is passing one of the iterator category tags defined in the standard.
So, I want to offer a generic implementation of an input iterator, there we go:
export module iterator:legacy_input_iterator;
import :legacy_iterator;
import std;
export namespace zero::iterator::legacy {
template <typename T>
struct input_iter: iterator<std::input_iterator_tag, T> {
using base_it = iterator<std::input_iterator_tag, T>;
private:
typename base_it::pointer _ptr;
public:
input_iter<T>() = default;
explicit input_iter<T>(typename base_it::pointer ptr = nullptr)
: _ptr { ptr } {}
~input_iter<T>() = default;
input_iter<T>(const input_iter<T>& other) = default;
input_iter<T>(input_iter<T>&& other) noexcept = default;
auto operator=(typename base_it::pointer ptr) -> input_iter<T>& { _ptr = ptr; return *this; }
auto operator=(const input_iter<T>&) -> input_iter<T>& = default;
auto operator=(input_iter<T>&&) noexcept -> input_iter<T>& = default;
[[nodiscard]]
auto operator->() const noexcept -> typename base_it::pointer {
return _ptr;
}
[[nodiscard]]
auto operator*() const noexcept -> typename base_it::reference {
return *_ptr;
}
auto operator++() noexcept -> input_iter& {
++this-> _ptr;
return *this;
}
void operator++(int) noexcept {
++(*this);
}
[[nodiscard]]
friend auto operator==(input_iter& self, input_iter& rhs) noexcept -> bool {
return self._ptr == rhs._ptr;
}
[[nodiscard]]
friend auto operator!=(input_iter& self, input_iter& rhs) noexcept -> bool {
return self._ptr != rhs._ptr;
}
};
}
So, now, as a client, I notice that the input iterator in the library, could fit my needs, so I just create my own container, and I try to use it.
template<typename T, size_t N>
class Array {
private:
T array[N];
// code goes on here...
public:
using iterator = zero::iterator::legacy::input_iter<T>;
using const_iterator = zero::iterator::legacy::input_iter<const T>;
};
Everything is fine! I can iterate my array in range for loops, retrieve the begin and end iterators...
Now, going back to the origin, as the library creator, I want to offer at least one generic implementation for every generic category. So, I will try to implement a forward iterator, but I noticed that they must implement the same operators! But with subtle differences...
For an input iterator, after the use of operator++() any copies of the previous value are no longer validity guarantee, but a forward iterator must ensure the oposite
But also, a forward iterator:
Must satisfy legacy input iterator.
Default constructible.
Must be Multipass Guarantee changes to another copy of an iterator like incrementing/assignment (if mutable), can’t change the value read from other copies, unlike Input/Output iterators.
Operators: operator++(int) – returns iterator copy, and operator*() returns a reference to the iterator value.
And now is where I am getting totally lost. In order to follow principles like the DRY principle, or the KISS one, I got tempted to write this:
template <typename T>
struct forward_iterator: public input_iter<T> {
}
So, since the same operations and operators are required, I could use static binding (totally avoiding dynamic dispatch) and get rid out of all the code duplication, and just by changing the private T* ptr member of my input_iterator to protected, everything will look fine. But no...
Is my input iterator even implemented correctly?
Is the container or the datastructure the one who may take care about the iterator specifications? I mean, is the container the responsible for ensuring that the input iterator must be only single pass, and the forward iterator must be multipass, for example? (along with the other requirements)? or should that behaviour must be coded in the iterator?
If the inheritance approach is correct (I think that is a really bad idea), how can I express the specific requirements (if they must be encoded in the operators and operators defined by the iterator)?
How can I then, express the differences between the iterators that has the same operator overloads, without code duplication? (I have a completly different version using CRTP, that solves those problems, but I wanted to know if there's a way with the classical inheritance approach)
Disclaimer: Everything done here is for learning purposes, but as you may see, I am lost without probably basic concepts in C++, like the real implementation of an iterator based on some sort of category defined by the standard, and I just to like to have some nice help guiding me in the architechtural design (and having help with the implementation too would be really kind)

Do custom iterators always need to explicitly specify value_type?

Since iterator is deprecated, I began converting iterators in my code base to use non-deprecated constructs. I could not seem to make my indirect iterator compliant with the std::forward_iterator concept unless I explicitly specified value_type. I would like to know if this is expected.
Based on the definition of iter_value_t and indirectly_readible_traits, it seems like there is no automatic inference of std::iter_value_t. Naively, I would have expected std::iter_value_t<Itr> to be defined as std::remove_cvref_t<std::iter_reference_t<Itr>> if no definition for value_type is present (which is checked via has-member-value-type in indirectly_readible_traits).
#include <vector>
template <std::forward_iterator Itr>
class IndirectItr {
public:
using value_type = std::iter_value_t<Itr>; // **do I need this?**
explicit IndirectItr(Itr itr = {}) : m_itr{itr} {}
bool operator==(const IndirectItr& rhs) const { return m_itr == rhs.m_itr; }
bool operator!=(const IndirectItr& rhs) const { return m_itr != rhs.m_itr; }
typename std::iter_reference_t<Itr> operator *() const { return *m_itr; }
IndirectItr& operator++() { ++m_itr; return *this; }
IndirectItr operator++(int) { auto ret = *this; ++(*this); return ret; }
typename std::iter_difference_t<Itr> operator-(const IndirectItr& rhs) const { return m_itr - rhs.m_itr; }
private:
Itr m_itr;
};
using Base = std::vector<int>::iterator;
static_assert(std::forward_iterator<IndirectItr<Base>>);
static_assert(std::same_as<std::iter_value_t<Base>, std::remove_cvref_t<std::iter_reference_t<Base>>>);
P.S. I have several indirect iterator definitions that wrap other iterators. The example above is representative of a custom indirect iterator. I don't have this exact class in my code.
You don't have to have a member value_type on your iterator. But your only alternative is to specialize iterator_traits<T> for your iterator type and provide a value_type alias there. So you may as well make it a member of the iterator.
The value_type cannot be computed from something else, as it may have no obvious relation to reference or any other operation on the iterator. This is one of the things that allows for proxy iterators, which pre-C++20 concepts did not.
std::forward_iterator includes std::input_iterator, which includes std::indirectly_readable, which contains:
requires(const In in) {
typename std::iter_value_t<In>;
typename std::iter_reference_t<In>;
typename std::iter_rvalue_reference_t<In>;
{ *in } -> std::same_as<std::iter_reference_t<In>>;
{ ranges::iter_move(in) } -> std::same_as<std::iter_rvalue_reference_t<In>>;
}
(where In is std::remove_cvref_t<IndirectItr<Base>>).
That typename std::iter_value_t<In>; line requires you to declare a value_type or to specialize std::iterator_traits<IndirectItr<Base>> (and provide value_type there), as explained here.
You cannot specialize std::iterator_traits<IndirectItr<T>> for all T (see also), so you can either pick the first and very reasonable option, or fully specialize for each IndirectItr you intend to use.

Which C++20 standard library containers have a .at member access function?

Which C++20 standard library containers have a .at function? For example, at least std::map, std::unordered_map and std::vector do. What others are there?
Is there some way to work this out without going through the 2000 page standard by hand?
From comments, it seems you want something like:
template <typename Container, typename T>
typename Container::pointer my_at(Container&, const T&)
requires (requires(Container c, const T& key) { c.at(key); })
{
// ...
}
Demo
I don't think a pointer is the appropriate type to return for this operation, instead it should be an iterator. At which point the definition splits into two cases:
Random access containers:
iterator at_(size_type index) noexcept { return index < size() ? begin() + index : end(); }
const_iterator at_(size_type index) const noexcept { return index < size() ? begin() + index : end(); }
map and unordered_map:
template<typename T>
iterator at_(T && key) noexcept { return find(std::forward<T>(key)); }
template<typename T>
const_iterator at_(T && key) const noexcept { return find(std::forward<T>(key)); }
And then rather than testing against nullptr, you test against the container's end().
It's also a reasonable question as to whether (unordered_)map needs an alias for an existing member to match your nomenclature
I ended up grepping libstdc++ include directory for \bat( and that has given me:
std::basic_string
std::basic_string_view
std::array
std::vector
std::vector<bool>
std::unordered_map
std::map
std::dequeue
I'm not sure if this is exhaustive.
There is also rope and vstring but I don't think these are standard.

arithmetic of pointers in c++ due to allocation

I created a class which contains 3 elements of the same type (something like std::pair<T,std::pair<T,T>>). Moreover I prepared it for use for(auto&var:t)
But I don't know if I did it safely.
template <class T>
class Troika
{
public:
Troika(const T& f, const T& s, const T& t)
:first(f), second(s), third(t) {}
Troika(){}
T first;
T second;
T third;
typedef T* iterator;
typedef const T* const_iterator;
iterator begin()
{
return &first;
}
const_iterator begin() const
{
return &first;
}
iterator end()
{
return &third+1;
}
const_iterator end() const
{
return &third + 1;
}
};
I mean about this part: return &third + 1;. I know that if somebody have not huge knowledge about arithmetic of pointers in c++, it is really dangerous. Are these elements alocated one by one near each other?
Are these elements alocated one by one near each other?
They likely are, since they have the exact same accessibility. But it still doesn't make your implementation safe. Pointer arithmetic is only valid for elements in the same array object. Those three are not in the same array, there is no array in sight. Any attempt to use those iterators will result in undefined behavior.
I'm not sure of your intention in defining this class, but for a container of three elements of the same type, I'd offer the more idiomatic:
template<typename T>
using Troika = std::array<T, 3>;
You lose the first, second and third named members, but gain indexing (which is as good I think) and iteration for free.

How to achieve O(1) erasure from a std::list

The question is what is the recommended way to use std::list to achieve O(1) erasure of list items?
Usually, when I choose a doubly linked list, I want to be able to remove an element from a list in O(1) time, and then move it to a different list in O(1) time. If the element has its own prev and next pointers, there is no real trick to getting the job done. If the list is a doubly linked circular list, then removal doesn't necessarily require knowing the list that contains the item.
As per Iterator invalidation rules, std::list iterators are very durable. So, it seems to me to get the behavior I desire when using std::list on my own item is to stash an iterator within my class, and the containing list.
class Item {
typedef std::shared_ptr<Item> Ptr;
struct Ref {
std::list<Ptr>::iterator iter_;
std::list<Ptr> *list_;
};
Ref ref_;
//...
};
This has the downside that I will need to create my own decorated version of std::list that knows to update the ref_ whenever the item is added to a list. I can't think of a way that doesn't require the embedded iterator, since not having one would mean erasure would incur a O(n) find operation first.
What is the recommended way to get O(1) erasure with std::list? Or, is there a better approach to achieving the objective?
In the past, I have accomplished this by implementing my own list data structure, where the item placed in the list has its own next and prev pointers. Managing these pointers is natural, since they are inherent to the list operations themselves (the API to my list implementation adjusts the pointers). If I want to use the STL instead, what would be the best way to accomplish this? I offer the straw-man proposal of embedding the iterator. Are there better approaches?
If a concrete use-case is desired, consider a timer implementation. When a timer is created, it is placed into an appropriate list. If it is canceled, it is desirable to efficiently remove it. (This particular example can be solved via marking instead of removal, but it is a valid way to implement cancellation.) Additional use-cases are available upon request.
Another alternative I explored was to fuse a std::list with a std::unordered_map to create a specialized list for pointer types. This is more heavyweight (because of the hash table), but provides a container that is pretty close to the standard containers at the interface level, and gives me O(1) erasure of list elements. The only feature missing from the straw-man proposal is a pointer to the list which currently contains the item. I have put up the current implementation at CodeReview to solicit comment.
std::list::erase is guaranteed to be O(1).
There is not an awful lot of other ways to erase elements from a standard list. (std::list::remove and friends don't do quite the same thing so they don't count).
If you want to erase from a standard list, you need an iterator and the list itself. That's what you seem to already have. There is not very much freedom of doing it differently. I would keep the list containment separate from the objects, unlike what you have done, because why make an object that can be in only one list at a time? Seems like an unnecessary artificial restriction to me. But whatever powers your design.
Maybe you could redesign your interface to hand out iterators instead of raw objects? In the case of your timers example:
class Timer {
// ...
};
typedef std::list<Timer>::iterator TimerRef;
class Timers {
public:
TimerRef createTimer(long time);
void cancelTimer(TimerRef ref);
private:
std::list<Timer> timers;
};
Of course, instead of
timer.cancel();
users of the class now have to say
timers.cancelTimer(timerRef);
but depending on your use case, that might not be a problem.
Update: moving timers between lists:
class Timers {
public:
Timer removeTimer(TimerRef ref);
void addTimer(Timer const &timer);
// ...
};
Usage:
timers2.addTimer(timers1.removeTimer(timerRef));
Admittedly it's a bit cumbersome, but so are the alternatives.
There is no way to have O(1) erasure from std::list.
you may want to consider using an intrusive list, where list nodes are directly imbedded into the structures, like you have already done.
you can use boost::intrusive or roll your own, also check out this
Here's a "complete" solution using an embedded iterator. Some private traits are used to help reduce clutter in the class:
template <typename T> class List;
template <typename T>
class ListTraits {
protected:
typedef std::list<std::shared_ptr<T>> Impl;
typedef typename Impl::iterator Iterator;
typedef typename Impl::const_iterator ConstIterator;
typedef typename Impl::reverse_iterator Rotareti;
typedef typename Impl::const_reverse_iterator ConstRotareti;
typedef std::map<const List<T> *, typename Impl::iterator> Ref;
};
As shown, the list implementation will be using std::list, but the underlying value type will be a std::shared_ptr. What I am after is allowing an instance of T to efficiently derive its own iterator, to achieve O(1) erasure. This is done by using a Ref to memoize the iterator of the item after it is inserted into the list.
template <typename T>
class List : public ListTraits<T> {
template <typename ITER> class IteratorT;
typedef ListTraits<T> Traits;
typename Traits::Impl impl_;
public:
typedef typename Traits::Impl::size_type size_type;
typedef typename Traits::Impl::value_type pointer;
typedef pointer value_type;
typedef IteratorT<typename Traits::Iterator> iterator;
typedef IteratorT<typename Traits::ConstIterator> const_iterator;
typedef IteratorT<typename Traits::Rotareti> reverse_iterator;
typedef IteratorT<typename Traits::ConstRotareti> const_reverse_iterator;
class Item;
~List () { while (!empty()) pop_front(); }
size_type size () const { return impl_.size(); }
bool empty () const { return impl_.empty(); }
iterator begin () { return impl_.begin(); }
iterator end () { return impl_.end(); }
const_iterator begin () const { return impl_.begin(); }
const_iterator end () const { return impl_.end(); }
reverse_iterator rbegin () { return impl_.rbegin(); }
reverse_iterator rend () { return impl_.rend(); }
const_reverse_iterator rbegin () const { return impl_.rbegin(); }
const_reverse_iterator rend () const { return impl_.rend(); }
pointer front () const { return !empty() ? impl_.front() : pointer(); }
pointer back () const { return !empty() ? impl_.back() : pointer(); }
void push_front (const pointer &e);
void pop_front ();
void push_back (const pointer &e);
void pop_back ();
void erase (const pointer &e);
bool contains (const pointer &e) const;
};
This List follows mostly a queue like interface. But, an item may be removed from any position in the list. The simple functions mostly just delegate to the underlying std::list. But the push_*() and pop_*() methods also memoize the iterator.
template <typename T>
template <typename ITER>
class List<T>::IteratorT : public ITER {
friend class List<T>;
ITER iter_;
IteratorT (ITER i) : iter_(i) {}
public:
IteratorT () : iter_() {}
IteratorT & operator ++ () { ++iter_; return *this; }
IteratorT & operator -- () { --iter_; return *this; }
IteratorT operator ++ (int) { return iter_++; }
IteratorT operator -- (int) { return iter_--; }
bool operator == (const IteratorT &x) const { return iter_ == x.iter_; }
bool operator != (const IteratorT &x) const { return iter_ != x.iter_; }
T & operator * () const { return **iter_; }
pointer operator -> () const { return *iter_; }
};
This is the implementation of the helper template used to define the iterator types for List. What it does differently is that the * and -> operators are defined in a way that makes the iterator behave like it is a T * rather than a std::shared_ptr<T> * (which is what the underlying iterator would normally do).
template <typename T>
class List<T>::Item {
friend class List<T>;
mutable typename Traits::Ref ref_;
};
A type T that is derived from List<T>::Item can be added to a List<T>. This base class contains the Ref instance used to memoize the iterator when the item is added to a list.
template <typename T>
inline void List<T>::push_front (const pointer &e) {
const Item &item = *e;
typename Traits::Ref::iterator i = item.ref_.find(this);
if (i == item.ref_.end()) {
item.ref_[this] = impl_.insert(impl_.begin(), e);
} else if (front() != e) {
impl_.erase(i->second);
i->second = impl_.insert(impl_.begin(), e);
}
}
template <typename T>
inline void List<T>::pop_front () {
if (!empty()) {
const Item &item = *front();
item.ref_.erase(this);
impl_.pop_front();
}
}
This code illustrates how the memoization is performed. When doing push_front(), the item is first checked to see if it is already contained. If it is not, it is inserted, and the resulting iterator is added to the ref_ object. Otherwise, if it is not already the front, then the item is removed and reinserted at the front, and the memoized iterator is updated. pop_front() removes the memoized iterator, and then calls pop_front() on the the std::list.
template <typename T>
inline void List<T>::push_back (const pointer &e) {
const Item &item = *e;
typename Traits::Ref::iterator i = item.ref_.find(this);
if (i == item.ref_.end()) {
item.ref_[this] = impl_.insert(impl_.end(), e);
} else if (back() != e) {
impl_.erase(i->second);
i->second = impl_.insert(impl_.end(), e);
}
}
template <typename T>
inline void List<T>::pop_back () {
if (!empty()) {
const Item &item = *back();
item.ref_.erase(this);
impl_.pop_back();
}
}
push_back() and pop_back() are similar to push_front() and pop_front().
template <typename T>
inline void List<T>::erase (const pointer &e) {
const Item &item = *e;
typename Traits::Ref::iterator i = item.ref_.find(this);
if (i != item.ref_.end()) {
item.ref_.erase(i);
impl_.erase(i->second);
}
}
The erase() routine retrieves the memoized iterator, and uses it to perform the erasure.
template <typename T>
inline bool List<T>::contains (const pointer &e) const {
const Item &item = *e;
typename Traits::Ref::iterator i = item.ref_.find(this);
return i != item.ref_.end();
}
Since an item is its own iterator in many respects, a find() method should not be needed in this version of List. But, in lieu of that is this contains() method which is used to see if the element is a member of the list.
Now, the presented solution uses a std::map to associate list instances to iterators. This maintains the spirit of O(1) if the number of lists an item is a member of simultaneously is relatively small.
I will try my hand at a boost::intrusive version next.
Awful truth: while the linked list is powerful structure, the std::list can't fully exploit it's capabilities.
You can't erase the object from std::list using only iterators, because list has to deallocate the node, and you have to know where is the allocator that allocated the memory (hint: it's in a list).
Intrusive containers have many advantages over standard linked list, like self-awareness ;-), ability to store polymorphic objects by value and they make list tricks (like having single objects in multiple lists) feasible. Since you don't use std::list directly anyway, you can drop usage of std::list altogether and use third-party container, or roll your own.
(Also, your solution is also intrusive, since your type have to be derived from List<T>::Item, which places certain requirements on the type that std::list doesn't)