Related
I made a collection for which I want to provide an STL-style, random-access iterator. I was searching around for an example implementation of an iterator but I didn't find any. I know about the need for const overloads of [] and * operators. What are the requirements for an iterator to be "STL-style" and what are some other pitfalls to avoid (if any)?
Additional context: This is for a library and I don't want to introduce any dependency on it unless I really need to. I write my own collection to be able to provide binary compatibility between C++03 and C++11 with the same compiler (so no STL which would probably break).
https://cplusplus.com/reference/iterator/ has a handy chart that details the specs of § 24.2.2 of the C++11 standard. Basically, the iterators have tags that describe the valid operations, and the tags have a hierarchy. Below is purely symbolic, these classes don't actually exist as such.
iterator {
iterator(const iterator&);
~iterator();
iterator& operator=(const iterator&);
iterator& operator++(); //prefix increment
reference operator*() const;
friend void swap(iterator& lhs, iterator& rhs); //C++11 I think
};
input_iterator : public virtual iterator {
iterator operator++(int); //postfix increment
value_type operator*() const;
pointer operator->() const;
friend bool operator==(const iterator&, const iterator&);
friend bool operator!=(const iterator&, const iterator&);
};
//once an input iterator has been dereferenced, it is
//undefined to dereference one before that.
output_iterator : public virtual iterator {
reference operator*() const;
iterator operator++(int); //postfix increment
};
//dereferences may only be on the left side of an assignment
//once an output iterator has been dereferenced, it is
//undefined to dereference one before that.
forward_iterator : input_iterator, output_iterator {
forward_iterator();
};
//multiple passes allowed
bidirectional_iterator : forward_iterator {
iterator& operator--(); //prefix decrement
iterator operator--(int); //postfix decrement
};
random_access_iterator : bidirectional_iterator {
friend bool operator<(const iterator&, const iterator&);
friend bool operator>(const iterator&, const iterator&);
friend bool operator<=(const iterator&, const iterator&);
friend bool operator>=(const iterator&, const iterator&);
iterator& operator+=(size_type);
friend iterator operator+(const iterator&, size_type);
friend iterator operator+(size_type, const iterator&);
iterator& operator-=(size_type);
friend iterator operator-(const iterator&, size_type);
friend difference_type operator-(iterator, iterator);
reference operator[](size_type) const;
};
contiguous_iterator : random_access_iterator { //C++17
}; //elements are stored contiguously in memory.
You can either specialize std::iterator_traits<youriterator>, or put the same typedefs in the iterator itself, or inherit from std::iterator (which has these typedefs). I prefer the second option, to avoid changing things in the std namespace, and for readability, but most people inherit from std::iterator.
struct std::iterator_traits<youriterator> {
typedef ???? difference_type; //almost always ptrdiff_t
typedef ???? value_type; //almost always T
typedef ???? reference; //almost always T& or const T&
typedef ???? pointer; //almost always T* or const T*
typedef ???? iterator_category; //usually std::forward_iterator_tag or similar
};
Note the iterator_category should be one of std::input_iterator_tag, std::output_iterator_tag, std::forward_iterator_tag, std::bidirectional_iterator_tag, or std::random_access_iterator_tag, depending on which requirements your iterator satisfies. Depending on your iterator, you may choose to specialize std::next, std::prev, std::advance, and std::distance as well, but this is rarely needed. In extremely rare cases you may wish to specialize std::begin and std::end.
Your container should probably also have a const_iterator, which is a (possibly mutable) iterator to constant data that is similar to your iterator except it should be implicitly constructable from a iterator and users should be unable to modify the data. It is common for its internal pointer to be a pointer to non-constant data, and have iterator inherit from const_iterator so as to minimize code duplication.
My post at Writing your own STL Container has a more complete container/iterator prototype.
The iterator_facade documentation from Boost.Iterator provides what looks like a nice tutorial on implementing iterators for a linked list. Could you use that as a starting point for building a random-access iterator over your container?
If nothing else, you can take a look at the member functions and typedefs provided by iterator_facade and use it as a starting point for building your own.
Here is sample of raw pointer iterator.
You shouldn't use iterator class to work with raw pointers!
#include <iostream>
#include <vector>
#include <list>
#include <iterator>
#include <assert.h>
template<typename T>
class ptr_iterator
: public std::iterator<std::forward_iterator_tag, T>
{
typedef ptr_iterator<T> iterator;
pointer pos_;
public:
ptr_iterator() : pos_(nullptr) {}
ptr_iterator(T* v) : pos_(v) {}
~ptr_iterator() {}
iterator operator++(int) /* postfix */ { return pos_++; }
iterator& operator++() /* prefix */ { ++pos_; return *this; }
reference operator* () const { return *pos_; }
pointer operator->() const { return pos_; }
iterator operator+ (difference_type v) const { return pos_ + v; }
bool operator==(const iterator& rhs) const { return pos_ == rhs.pos_; }
bool operator!=(const iterator& rhs) const { return pos_ != rhs.pos_; }
};
template<typename T>
ptr_iterator<T> begin(T *val) { return ptr_iterator<T>(val); }
template<typename T, typename Tsize>
ptr_iterator<T> end(T *val, Tsize size) { return ptr_iterator<T>(val) + size; }
Raw pointer range based loop workaround. Please, correct me, if there is better way to make range based loop from raw pointer.
template<typename T>
class ptr_range
{
T* begin_;
T* end_;
public:
ptr_range(T* ptr, size_t length) : begin_(ptr), end_(ptr + length) { assert(begin_ <= end_); }
T* begin() const { return begin_; }
T* end() const { return end_; }
};
template<typename T>
ptr_range<T> range(T* ptr, size_t length) { return ptr_range<T>(ptr, length); }
And simple test
void DoIteratorTest()
{
const static size_t size = 10;
uint8_t *data = new uint8_t[size];
{
// Only for iterator test
uint8_t n = '0';
auto first = begin(data);
auto last = end(data, size);
for (auto it = first; it != last; ++it)
{
*it = n++;
}
// It's prefer to use the following way:
for (const auto& n : range(data, size))
{
std::cout << " char: " << static_cast<char>(n) << std::endl;
}
}
{
// Only for iterator test
ptr_iterator<uint8_t> first(data);
ptr_iterator<uint8_t> last(first + size);
std::vector<uint8_t> v1(first, last);
// It's prefer to use the following way:
std::vector<uint8_t> v2(data, data + size);
}
{
std::list<std::vector<uint8_t>> queue_;
queue_.emplace_back(begin(data), end(data, size));
queue_.emplace_back(data, data + size);
}
}
Thomas Becker wrote a useful article on the subject here.
There was also this (perhaps simpler) approach that appeared previously on SO: How to correctly implement custom iterators and const_iterators?
First of all you can look here for a list of the various operations the individual iterator types need to support.
Next, when you have made your iterator class you need to either specialize std::iterator_traits for it and provide some necessary typedefs (like iterator_category or value_type) or alternatively derive it from std::iterator, which defines the needed typedefs for you and can therefore be used with the default std::iterator_traits.
disclaimer: I know some people don't like cplusplus.com that much, but they provide some really useful information on this.
I was/am in the same boat as you for different reasons (partly educational, partly constraints). I had to re-write all the containers of the standard library and the containers had to conform to the standard. That means, if I swap out my container with the stl version, the code would work the same. Which also meant that I had to re-write the iterators.
Anyway, I looked at EASTL. Apart from learning a ton about containers that I never learned all this time using the stl containers or through my undergraduate courses. The main reason is that EASTL is more readable than the stl counterpart (I found this is simply because of the lack of all the macros and straight forward coding style). There are some icky things in there (like #ifdefs for exceptions) but nothing to overwhelm you.
As others mentioned, look at cplusplus.com's reference on iterators and containers.
I was trying to solve the problem of being able to iterate over several different text arrays all of which are stored within a memory resident database that is a large struct.
The following was worked out using Visual Studio 2017 Community Edition on an MFC test application. I am including this as an example as this posting was one of several that I ran across that provided some help yet were still insufficient for my needs.
The struct containing the memory resident data looked something like the following. I have removed most of the elements for the sake of brevity and have also not included the Preprocessor defines used (the SDK in use is for C as well as C++ and is old).
What I was interested in doing is having iterators for the various WCHAR two dimensional arrays which contained text strings for mnemonics.
typedef struct tagUNINTRAM {
// stuff deleted ...
WCHAR ParaTransMnemo[MAX_TRANSM_NO][PARA_TRANSMNEMO_LEN]; /* prog #20 */
WCHAR ParaLeadThru[MAX_LEAD_NO][PARA_LEADTHRU_LEN]; /* prog #21 */
WCHAR ParaReportName[MAX_REPO_NO][PARA_REPORTNAME_LEN]; /* prog #22 */
WCHAR ParaSpeMnemo[MAX_SPEM_NO][PARA_SPEMNEMO_LEN]; /* prog #23 */
WCHAR ParaPCIF[MAX_PCIF_SIZE]; /* prog #39 */
WCHAR ParaAdjMnemo[MAX_ADJM_NO][PARA_ADJMNEMO_LEN]; /* prog #46 */
WCHAR ParaPrtModi[MAX_PRTMODI_NO][PARA_PRTMODI_LEN]; /* prog #47 */
WCHAR ParaMajorDEPT[MAX_MDEPT_NO][PARA_MAJORDEPT_LEN]; /* prog #48 */
// ... stuff deleted
} UNINIRAM;
The current approach is to use a template to define a proxy class for each of the arrays and then to have a single iterator class that can be used to iterate over a particular array by using a proxy object representing the array.
A copy of the memory resident data is stored in an object that handles reading and writing the memory resident data from/to disk. This class, CFilePara contains the templated proxy class (MnemonicIteratorDimSize and the sub class from which is it is derived, MnemonicIteratorDimSizeBase) and the iterator class, MnemonicIterator.
The created proxy object is attached to an iterator object which accesses the necessary information through an interface described by a base class from which all of the proxy classes are derived. The result is to have a single type of iterator class which can be used with several different proxy classes because the different proxy classes all expose the same interface, the interface of the proxy base class.
The first thing was to create a set of identifiers which would be provided to a class factory to generate the specific proxy object for that type of mnemonic. These identifiers are used as part of the user interface to identify the particular provisioning data the user is interested in seeing and possibly modifying.
const static DWORD_PTR dwId_TransactionMnemonic = 1;
const static DWORD_PTR dwId_ReportMnemonic = 2;
const static DWORD_PTR dwId_SpecialMnemonic = 3;
const static DWORD_PTR dwId_LeadThroughMnemonic = 4;
The Proxy Class
The templated proxy class and its base class are as follows. I needed to accommodate several different kinds of wchar_t text string arrays. The two dimensional arrays had different numbers of mnemonics, depending on the type (purpose) of the mnemonic and the different types of mnemonics were of different maximum lengths, varying between five text characters and twenty text characters. Templates for the derived proxy class was a natural fit with the template requiring the maximum number of characters in each mnemonic. After the proxy object is created, we then use the SetRange() method to specify the actual mnemonic array and its range.
// proxy object which represents a particular subsection of the
// memory resident database each of which is an array of wchar_t
// text arrays though the number of array elements may vary.
class MnemonicIteratorDimSizeBase
{
DWORD_PTR m_Type;
public:
MnemonicIteratorDimSizeBase(DWORD_PTR x) { }
virtual ~MnemonicIteratorDimSizeBase() { }
virtual wchar_t *begin() = 0;
virtual wchar_t *end() = 0;
virtual wchar_t *get(int i) = 0;
virtual int ItemSize() = 0;
virtual int ItemCount() = 0;
virtual DWORD_PTR ItemType() { return m_Type; }
};
template <size_t sDimSize>
class MnemonicIteratorDimSize : public MnemonicIteratorDimSizeBase
{
wchar_t (*m_begin)[sDimSize];
wchar_t (*m_end)[sDimSize];
public:
MnemonicIteratorDimSize(DWORD_PTR x) : MnemonicIteratorDimSizeBase(x), m_begin(0), m_end(0) { }
virtual ~MnemonicIteratorDimSize() { }
virtual wchar_t *begin() { return m_begin[0]; }
virtual wchar_t *end() { return m_end[0]; }
virtual wchar_t *get(int i) { return m_begin[i]; }
virtual int ItemSize() { return sDimSize; }
virtual int ItemCount() { return m_end - m_begin; }
void SetRange(wchar_t (*begin)[sDimSize], wchar_t (*end)[sDimSize]) {
m_begin = begin; m_end = end;
}
};
The Iterator Class
The iterator class itself is as follows. This class provides just basic forward iterator functionality which is all that is needed at this time. However I expect that this will change or be extended when I need something additional from it.
class MnemonicIterator
{
private:
MnemonicIteratorDimSizeBase *m_p; // we do not own this pointer. we just use it to access current item.
int m_index; // zero based index of item.
wchar_t *m_item; // value to be returned.
public:
MnemonicIterator(MnemonicIteratorDimSizeBase *p) : m_p(p) { }
~MnemonicIterator() { }
// a ranged for needs begin() and end() to determine the range.
// the range is up to but not including what end() returns.
MnemonicIterator & begin() { m_item = m_p->get(m_index = 0); return *this; } // begining of range of values for ranged for. first item
MnemonicIterator & end() { m_item = m_p->get(m_index = m_p->ItemCount()); return *this; } // end of range of values for ranged for. item after last item.
MnemonicIterator & operator ++ () { m_item = m_p->get(++m_index); return *this; } // prefix increment, ++p
MnemonicIterator & operator ++ (int i) { m_item = m_p->get(m_index++); return *this; } // postfix increment, p++
bool operator != (MnemonicIterator &p) { return **this != *p; } // minimum logical operator is not equal to
wchar_t * operator *() const { return m_item; } // dereference iterator to get what is pointed to
};
The proxy object factory determines which object to created based on the mnemonic identifier. The proxy object is created and the pointer returned is the standard base class type so as to have a uniform interface regardless of which of the different mnemonic sections are being accessed. The SetRange() method is used to specify to the proxy object the specific array elements the proxy represents and the range of the array elements.
CFilePara::MnemonicIteratorDimSizeBase * CFilePara::MakeIterator(DWORD_PTR x)
{
CFilePara::MnemonicIteratorDimSizeBase *mi = nullptr;
switch (x) {
case dwId_TransactionMnemonic:
{
CFilePara::MnemonicIteratorDimSize<PARA_TRANSMNEMO_LEN> *mk = new CFilePara::MnemonicIteratorDimSize<PARA_TRANSMNEMO_LEN>(x);
mk->SetRange(&m_Para.ParaTransMnemo[0], &m_Para.ParaTransMnemo[MAX_TRANSM_NO]);
mi = mk;
}
break;
case dwId_ReportMnemonic:
{
CFilePara::MnemonicIteratorDimSize<PARA_REPORTNAME_LEN> *mk = new CFilePara::MnemonicIteratorDimSize<PARA_REPORTNAME_LEN>(x);
mk->SetRange(&m_Para.ParaReportName[0], &m_Para.ParaReportName[MAX_REPO_NO]);
mi = mk;
}
break;
case dwId_SpecialMnemonic:
{
CFilePara::MnemonicIteratorDimSize<PARA_SPEMNEMO_LEN> *mk = new CFilePara::MnemonicIteratorDimSize<PARA_SPEMNEMO_LEN>(x);
mk->SetRange(&m_Para.ParaSpeMnemo[0], &m_Para.ParaSpeMnemo[MAX_SPEM_NO]);
mi = mk;
}
break;
case dwId_LeadThroughMnemonic:
{
CFilePara::MnemonicIteratorDimSize<PARA_LEADTHRU_LEN> *mk = new CFilePara::MnemonicIteratorDimSize<PARA_LEADTHRU_LEN>(x);
mk->SetRange(&m_Para.ParaLeadThru[0], &m_Para.ParaLeadThru[MAX_LEAD_NO]);
mi = mk;
}
break;
}
return mi;
}
Using the Proxy Class and Iterator
The proxy class and its iterator are used as shown in the following loop to fill in a CListCtrl object with a list of mnemonics. I am using std::unique_ptr so that when the proxy class i not longer needed and the std::unique_ptr goes out of scope, the memory will be cleaned up.
What this source code does is to create a proxy object for the array within the struct which corresponds to the specified mnemonic identifier. It then creates an iterator for that object, uses a ranged for to fill in the CListCtrl control and then cleans up. These are all raw wchar_t text strings which may be exactly the number of array elements so we copy the string into a temporary buffer in order to ensure that the text is zero terminated.
std::unique_ptr<CFilePara::MnemonicIteratorDimSizeBase> pObj(pFile->MakeIterator(m_IteratorType));
CFilePara::MnemonicIterator pIter(pObj.get()); // provide the raw pointer to the iterator who doesn't own it.
int i = 0; // CListCtrl index for zero based position to insert mnemonic.
for (auto x : pIter)
{
WCHAR szText[32] = { 0 }; // Temporary buffer.
wcsncpy_s(szText, 32, x, pObj->ItemSize());
m_mnemonicList.InsertItem(i, szText); i++;
}
And now a keys iterator for range-based for loop.
template<typename C>
class keys_it
{
typename C::const_iterator it_;
public:
using key_type = typename C::key_type;
using pointer = typename C::key_type*;
using difference_type = std::ptrdiff_t;
keys_it(const typename C::const_iterator & it) : it_(it) {}
keys_it operator++(int ) /* postfix */ { return it_++ ; }
keys_it& operator++( ) /* prefix */ { ++it_; return *this ; }
const key_type& operator* ( ) const { return it_->first ; }
const key_type& operator->( ) const { return it_->first ; }
keys_it operator+ (difference_type v ) const { return it_ + v ; }
bool operator==(const keys_it& rhs) const { return it_ == rhs.it_; }
bool operator!=(const keys_it& rhs) const { return it_ != rhs.it_; }
};
template<typename C>
class keys_impl
{
const C & c;
public:
keys_impl(const C & container) : c(container) {}
const keys_it<C> begin() const { return keys_it<C>(std::begin(c)); }
const keys_it<C> end () const { return keys_it<C>(std::end (c)); }
};
template<typename C>
keys_impl<C> keys(const C & container) { return keys_impl<C>(container); }
Usage:
std::map<std::string,int> my_map;
// fill my_map
for (const std::string & k : keys(my_map))
{
// do things
}
That's what i was looking for. But nobody had it, it seems.
You get my OCD code alignment as a bonus.
As an exercise, write your own for values(my_map)
This is a code example using std::reverse_iterator:
template<typename T, size_t SIZE>
class Stack {
T arr[SIZE];
size_t pos = 0;
public:
T pop() {
return arr[--pos];
}
Stack& push(const T& t) {
arr[pos++] = t;
return *this;
}
auto begin() {
return std::reverse_iterator(arr+pos);
}
auto end() {
return std::reverse_iterator(arr);
// ^ does reverse_iterator take this `one back`? how?
}
};
int main() {
Stack<int, 4> s;
s.push(5).push(15).push(25).push(35);
for(int val: s) {
std::cout << val << ' ';
}
}
// output is as expected: 35 25 15 5
When using std::reverse_iterator as an adaptor for another iterator, the newly adapted end shall be one before the original begin. However calling std::prev on begin is UB.
How does std::reverse_iterator hold one before begin?
Initialization of std::reverse_iterator from an iterator does not decrease the iterator upon initialization, as it would then be UB when sending begin to it (one cannot assume that std::prev(begin) is a valid call).
The trick is simple, std::reverse_iterator holds the original iterator passed to it, without modifying it. Only when it is being dereferenced it peeks back to the actual value. So in a way the iterator is pointing inside to the next element, from which it can get the current.
It would look something like:
// partial possible implementation of reverse_iterator for demo purpose
template<typename Itr>
class reverse_iterator {
Itr itr;
public:
constexpr explicit reverse_iterator(Itr itr): itr(itr) {}
constexpr auto& operator*() {
return *std::prev(itr); // <== only here we peek back
}
constexpr auto& operator++() {
--itr;
return *this;
}
friend bool operator!=(reverse_iterator<Itr> a, reverse_iterator<Itr> b) {
return a.itr != b.itr;
}
};
This is however an internal implementation detail (and can be in fact implemented in other similar manners). The user of std::reverse_iterator shall not be concerned with how it is implemented.
Sometimes I need to pass a C string to a function using the common C++ iterator range interface [first, last). Is there a standard C++ iterator class for those cases, or a standard way of doing it without having to copy the string or call strlen()?
EDIT:
I know I can use a pointer as an iterator, but I would have to know where the string ends, what would require me to call strlen().
EDIT2:
While I didn't know if such iterator is standardized, I certainly know it is possible. Responding to the sarcastic answers and comments, this is the stub (incomplete, untested):
class CStringIterator
{
public:
CStringIterator(char *str=nullptr):
ptr(str)
{}
bool operator==(const CStringIterator& other) const
{
if(other.ptr) {
return ptr == other.ptr;
} else {
return !*ptr;
}
}
/* ... operator++ and other iterator stuff */
private:
char *ptr;
};
EDIT3:
Specifically, I am interested in a forward iterator, because I want to avoid to iterate over the sring twice, when I know the algorithm will only have to do it once.
There isn't any explicit iterator class, but regular raw pointers are valid iterators as well. Problem with C-strings, though, is that they do not come with a native end iterator, which makes them unusable in range based for loops – directly at least...
You might like to try the following template, though:
template <typename T>
class Range
{
T* b;
public:
class Sentinel
{
friend class Range;
Sentinel() { }
friend bool operator!=(T* t, Sentinel) { return *t; }
public:
Sentinel(Sentinel const& o) { }
};
Range(T* begin)
: b(begin)
{ }
T* begin() { return b; }
Sentinel end() { return Sentinel(); }
};
Usage:
for(auto c : Range<char const>("hello world"))
{
std::cout << c << std::endl;
}
It originally was designed to iterate over null-terminated argv of main, but works with any pointer to null terminated array – which a C-string is as well...
Secret is comparing against the sentinel, which actually does a totally different comparison (current pointer pointing the terminating null (pointer))...
Edit: Pre-C++17 variant:
template <typename T>
class Range
{
T* b;
public:
class Wrapper
{
friend class Range;
T* t;
Wrapper(T* t) : t(t) { }
public:
Wrapper(Wrapper const& o) : t(o.t) { }
Wrapper operator++() { ++t; return *this; }
bool operator!=(Wrapper const& o) const { return *t; }
T operator*() { return *t; }
};
Range(T* begin)
: b(begin)
{ }
Wrapper begin() { return Wrapper(b); }
Wrapper end() { return Wrapper(nullptr); }
};
Actually, yes - sort of. In c++17.
C++17 introduces std::string_view which can be constructed from a c-style string.
std::string_view is a random access (proxy) container which of course fully supports iterators.
Note that although constructing a string_view from a const char* will theoretically call std::strlen, the compiler is allowed to (and gcc certainly does) elide the call when it knows the length of the string at compile time.
Example:
#include <string_view>
#include <iostream>
template<class Pointer>
struct pointer_span
{
using iterator = Pointer;
pointer_span(iterator first, std::size_t size)
: begin_(first)
, end_(first + size)
{
}
iterator begin() const { return begin_; }
iterator end() const { return end_; }
iterator begin_, end_;
};
int main(int argc, char** argv)
{
for(auto&& ztr : pointer_span(argv, argc))
{
const char* sep = "";
for (auto ch : std::string_view(ztr))
{
std::cout << sep << ch;
sep = " ";
}
std::cout << std::endl;
}
}
See the example output here
Is there a standard C++ iterator for C strings?
Yes. A pointer is an iterator for an array. C strings are (null terminated) arrays of char. Therefore char* is an iterator for a C string.
... using the common C++ iterator range interface [first, last)
Just like with all other iterators, to have a range, you need to have an end iterator.
If you know or can assume that an array fully contains the string and nothing more, then you can get the iterator range in constant time using std::begin(arr) (std::begin is redundant for C arrays which decay to the pointer anyway, but nice for symmetry) and std::end(arr) - 1. Otherwise you can use pointer arithmetic with offsets within the array.
A little bit of care must be taken to account for the null terminator. One must remember that the full range of the array contains the null terminator of the string. If you want the iterator range to represent the string without the terminator, then subtract one from the end iterator of the array, which explains the subtraction in the previous paragraph.
If you don't have an array, but only a pointer - the begin iterator - you can get the end iterator by advancing the beginning by the length of the string. This advancement is a constant operation, because pointers are random access iterators. If you don't know the length, you can call std::strlen to find out (which isn't a constant operation).
Example, std::sort accepts a range of iterators. You can sort a C string like this:
char str[] = "Hello World!";
std::sort(std::begin(str), std::end(str) - 1);
for(char c : "test"); // range-for-loops work as well, but this includes NUL
In the case you don't know the length of the string:
char *str = get_me_some_string();
std::sort(str, str + std::strlen(str));
Specifically, I am interested in a forward iterator
A pointer is a random access iterator. All random access iterators are also forward iterators. A pointer meets all of the requirements listed in the linked iterator concept.
It is possible to write such iterator, something like this should work:
struct csforward_iterator :
std::iterator<std::bidirectional_iterator_tag, const char, void> {
csforward_iterator( pointer ptr = nullptr ) : p( ptr ) {}
csforward_iterator& operator++() { ++p; return *this; }
csforward_iterator operator++(int) { auto t = *this; ++p; return t; }
csforward_iterator& operator--() { --p; return *this; }
csforward_iterator operator--(int) { auto t = *this; --p; return t; }
bool operator==( csforward_iterator o ) {
return p == o.p or ( p ? not ( o.p or *p ) : not *o.p );
}
bool operator!=( csforward_iterator o ) { return not operator==( o ); }
void swap( csforward_iterator &o ) { std::swap( p, o.p ); }
reference operator*() const { return *p; }
pointer operator->() const { return p; }
private:
pointer p;
};
live example
though unfortunately standard one is not provided and it probably would be template over char type (like std::string ).
I'm afraid not, for last you'll need a pointer to the end of the string for which you'll need to call strlen.
If you have a string literal, you can get the end iterator without using std::strlen. If you have only a char*, you'll have to write your own iterator class or rely on std::strlen to get the end iterator.
Demonstrative code for string literals:
#include <iostream>
#include <utility>
template <typename T, size_t N>
std::pair<T*, T*> array_iterators(T (&a)[N]) { return std::make_pair(&a[0], &a[0]+N); }
int main()
{
auto iterators = array_iterators("This is a string.");
// The second of the iterators points one character past the terminating
// null character. To iterate over the characters of the string, we need to
// stop at the terminating null character.
for ( auto it = iterators.first; it != iterators.second-1; ++it )
{
std::cout << *it << std::endl;
}
}
For ultimate safety and flexibility, you end up wrapping the iterator, and it has to carry some state.
Issues include:
random access - which can be addressed in a wrapped pointer by limiting its overloads to block random access, or by making it strlen() on need
multiple iterators - when comparing with each other, not end
decrementing end - which you could again "fix" by limiting the overloads
begin() and end() need to be same type - in c++11 and some api calls.
a non-const iterator could add or remove content
Note that it is "not the iterator's problem" if it is randomly seeked outside the range of the container, and it can legally seek past a string_view.end(). It is also fairly standard that such a broken iterator could not then increment to end() any more.
The most painful of these conditions is that end can be decremented, or subtracted, and dereferenced (usually you can't, but for string it is a null character). This means the end object needs a flag that it is the end, and the address of the start, so that it can find the actual end using strlen() if either of these operations occurs.
Is there a standard C++ iterator class for those cases, or a standard way of doing it without having to copy the string
Iterators are a generalization of pointers. Specifically, they're designed so that pointers are valid iterators.
Note the pointer specializations of std::iterator_traits.
I know I can use a pointer as an iterator, but I would have to know where the string ends
Unless you have some other way to know where the string ends, calling strlen is the best you can do. If there were a magic iterator wrapper, it would also have to call strlen.
Sorry, an iterator is something that is normally obtained from an iterable instance. As char * is a basic type and not a class anymore. How do you think something like .begin() or .end(), can be achieved.
By the way, if you need to iterate a char *p knowing it is nul terminated. you just can do the following.
for( char *p = your_string; *p; ++p ) {
...
}
but the thing is that you cannot use iterators as they are defined in C++, because char * is a basic type, has no constructor, has no destructor or methods associated.
Accelerated C++, exercise 14.5 involves reimplementing a split function (which turns text input into a vector of strings). One must use store input in a std::string - like class (Str) & use the split function to return a Vec<Str>, Vec being a std::vector - like container. The Str class manages a custom pointer (Ptr) to the underlying Vec<char> data.
The Str class provides a constructor Str(i,j), in Str.h below, which constructs a Ptr to the underlying Vec
The problem arises when I try to create substrings by calling str(i,j)
I've detailed in the code where the the issues arise.
Here is a whittled-down version of the Str class (can post more code if needed):
Str.h
#include "Ptr.h"
#include "Vec.h"
class Str {
friend std::istream& operator>>(std::istream&, Str&);
public:
// define iterators
typedef char* iterator;
typedef char* const_iterator;
iterator begin() { return data->begin(); }
const_iterator begin() const { return data->begin(); }
iterator end() { return data->end(); }
const_iterator end() const { return data->end(); }
//** This is where we define a constructor for `Ptr`s to substrings **
template<class In> Str(In i, In j): data(new Vec<char>) {
std::copy(i, j, std::back_inserter(*data));
}
private:
// store a Ptr to a Vec
Ptr< Vec<char> > data;
};
Split.h
Vec<Str> split(const Str& str) {
typedef Str::const_iterator iter;
Vec<Str> ret;
iter i = str.begin();
while (i != str.end()) {
// ignore leading blanks
i = find_if(i, str.end(), not_space);
// find end of next word
iter j = find_if(i, str.end(), space);
// copy the characters in `[i,' `j)'
if (i != str.end())
ret.push_back(**substring**); // Need to create substrings here
// call to str(i,j) gives error, detailed below
i = j;
}
return ret;
}
My first thought was to use this constructor to create (pointers to) the required substrings. Calling str(i,j) here gives the error message
type 'const Str' does not provide a call operator
It appears as if one cannot simply call str(i,j) here. Why not?
Could a solution be to write a Str member function which is similar to substr?
I made a collection for which I want to provide an STL-style, random-access iterator. I was searching around for an example implementation of an iterator but I didn't find any. I know about the need for const overloads of [] and * operators. What are the requirements for an iterator to be "STL-style" and what are some other pitfalls to avoid (if any)?
Additional context: This is for a library and I don't want to introduce any dependency on it unless I really need to. I write my own collection to be able to provide binary compatibility between C++03 and C++11 with the same compiler (so no STL which would probably break).
https://cplusplus.com/reference/iterator/ has a handy chart that details the specs of § 24.2.2 of the C++11 standard. Basically, the iterators have tags that describe the valid operations, and the tags have a hierarchy. Below is purely symbolic, these classes don't actually exist as such.
iterator {
iterator(const iterator&);
~iterator();
iterator& operator=(const iterator&);
iterator& operator++(); //prefix increment
reference operator*() const;
friend void swap(iterator& lhs, iterator& rhs); //C++11 I think
};
input_iterator : public virtual iterator {
iterator operator++(int); //postfix increment
value_type operator*() const;
pointer operator->() const;
friend bool operator==(const iterator&, const iterator&);
friend bool operator!=(const iterator&, const iterator&);
};
//once an input iterator has been dereferenced, it is
//undefined to dereference one before that.
output_iterator : public virtual iterator {
reference operator*() const;
iterator operator++(int); //postfix increment
};
//dereferences may only be on the left side of an assignment
//once an output iterator has been dereferenced, it is
//undefined to dereference one before that.
forward_iterator : input_iterator, output_iterator {
forward_iterator();
};
//multiple passes allowed
bidirectional_iterator : forward_iterator {
iterator& operator--(); //prefix decrement
iterator operator--(int); //postfix decrement
};
random_access_iterator : bidirectional_iterator {
friend bool operator<(const iterator&, const iterator&);
friend bool operator>(const iterator&, const iterator&);
friend bool operator<=(const iterator&, const iterator&);
friend bool operator>=(const iterator&, const iterator&);
iterator& operator+=(size_type);
friend iterator operator+(const iterator&, size_type);
friend iterator operator+(size_type, const iterator&);
iterator& operator-=(size_type);
friend iterator operator-(const iterator&, size_type);
friend difference_type operator-(iterator, iterator);
reference operator[](size_type) const;
};
contiguous_iterator : random_access_iterator { //C++17
}; //elements are stored contiguously in memory.
You can either specialize std::iterator_traits<youriterator>, or put the same typedefs in the iterator itself, or inherit from std::iterator (which has these typedefs). I prefer the second option, to avoid changing things in the std namespace, and for readability, but most people inherit from std::iterator.
struct std::iterator_traits<youriterator> {
typedef ???? difference_type; //almost always ptrdiff_t
typedef ???? value_type; //almost always T
typedef ???? reference; //almost always T& or const T&
typedef ???? pointer; //almost always T* or const T*
typedef ???? iterator_category; //usually std::forward_iterator_tag or similar
};
Note the iterator_category should be one of std::input_iterator_tag, std::output_iterator_tag, std::forward_iterator_tag, std::bidirectional_iterator_tag, or std::random_access_iterator_tag, depending on which requirements your iterator satisfies. Depending on your iterator, you may choose to specialize std::next, std::prev, std::advance, and std::distance as well, but this is rarely needed. In extremely rare cases you may wish to specialize std::begin and std::end.
Your container should probably also have a const_iterator, which is a (possibly mutable) iterator to constant data that is similar to your iterator except it should be implicitly constructable from a iterator and users should be unable to modify the data. It is common for its internal pointer to be a pointer to non-constant data, and have iterator inherit from const_iterator so as to minimize code duplication.
My post at Writing your own STL Container has a more complete container/iterator prototype.
The iterator_facade documentation from Boost.Iterator provides what looks like a nice tutorial on implementing iterators for a linked list. Could you use that as a starting point for building a random-access iterator over your container?
If nothing else, you can take a look at the member functions and typedefs provided by iterator_facade and use it as a starting point for building your own.
Here is sample of raw pointer iterator.
You shouldn't use iterator class to work with raw pointers!
#include <iostream>
#include <vector>
#include <list>
#include <iterator>
#include <assert.h>
template<typename T>
class ptr_iterator
: public std::iterator<std::forward_iterator_tag, T>
{
typedef ptr_iterator<T> iterator;
pointer pos_;
public:
ptr_iterator() : pos_(nullptr) {}
ptr_iterator(T* v) : pos_(v) {}
~ptr_iterator() {}
iterator operator++(int) /* postfix */ { return pos_++; }
iterator& operator++() /* prefix */ { ++pos_; return *this; }
reference operator* () const { return *pos_; }
pointer operator->() const { return pos_; }
iterator operator+ (difference_type v) const { return pos_ + v; }
bool operator==(const iterator& rhs) const { return pos_ == rhs.pos_; }
bool operator!=(const iterator& rhs) const { return pos_ != rhs.pos_; }
};
template<typename T>
ptr_iterator<T> begin(T *val) { return ptr_iterator<T>(val); }
template<typename T, typename Tsize>
ptr_iterator<T> end(T *val, Tsize size) { return ptr_iterator<T>(val) + size; }
Raw pointer range based loop workaround. Please, correct me, if there is better way to make range based loop from raw pointer.
template<typename T>
class ptr_range
{
T* begin_;
T* end_;
public:
ptr_range(T* ptr, size_t length) : begin_(ptr), end_(ptr + length) { assert(begin_ <= end_); }
T* begin() const { return begin_; }
T* end() const { return end_; }
};
template<typename T>
ptr_range<T> range(T* ptr, size_t length) { return ptr_range<T>(ptr, length); }
And simple test
void DoIteratorTest()
{
const static size_t size = 10;
uint8_t *data = new uint8_t[size];
{
// Only for iterator test
uint8_t n = '0';
auto first = begin(data);
auto last = end(data, size);
for (auto it = first; it != last; ++it)
{
*it = n++;
}
// It's prefer to use the following way:
for (const auto& n : range(data, size))
{
std::cout << " char: " << static_cast<char>(n) << std::endl;
}
}
{
// Only for iterator test
ptr_iterator<uint8_t> first(data);
ptr_iterator<uint8_t> last(first + size);
std::vector<uint8_t> v1(first, last);
// It's prefer to use the following way:
std::vector<uint8_t> v2(data, data + size);
}
{
std::list<std::vector<uint8_t>> queue_;
queue_.emplace_back(begin(data), end(data, size));
queue_.emplace_back(data, data + size);
}
}
Thomas Becker wrote a useful article on the subject here.
There was also this (perhaps simpler) approach that appeared previously on SO: How to correctly implement custom iterators and const_iterators?
First of all you can look here for a list of the various operations the individual iterator types need to support.
Next, when you have made your iterator class you need to either specialize std::iterator_traits for it and provide some necessary typedefs (like iterator_category or value_type) or alternatively derive it from std::iterator, which defines the needed typedefs for you and can therefore be used with the default std::iterator_traits.
disclaimer: I know some people don't like cplusplus.com that much, but they provide some really useful information on this.
I was/am in the same boat as you for different reasons (partly educational, partly constraints). I had to re-write all the containers of the standard library and the containers had to conform to the standard. That means, if I swap out my container with the stl version, the code would work the same. Which also meant that I had to re-write the iterators.
Anyway, I looked at EASTL. Apart from learning a ton about containers that I never learned all this time using the stl containers or through my undergraduate courses. The main reason is that EASTL is more readable than the stl counterpart (I found this is simply because of the lack of all the macros and straight forward coding style). There are some icky things in there (like #ifdefs for exceptions) but nothing to overwhelm you.
As others mentioned, look at cplusplus.com's reference on iterators and containers.
I was trying to solve the problem of being able to iterate over several different text arrays all of which are stored within a memory resident database that is a large struct.
The following was worked out using Visual Studio 2017 Community Edition on an MFC test application. I am including this as an example as this posting was one of several that I ran across that provided some help yet were still insufficient for my needs.
The struct containing the memory resident data looked something like the following. I have removed most of the elements for the sake of brevity and have also not included the Preprocessor defines used (the SDK in use is for C as well as C++ and is old).
What I was interested in doing is having iterators for the various WCHAR two dimensional arrays which contained text strings for mnemonics.
typedef struct tagUNINTRAM {
// stuff deleted ...
WCHAR ParaTransMnemo[MAX_TRANSM_NO][PARA_TRANSMNEMO_LEN]; /* prog #20 */
WCHAR ParaLeadThru[MAX_LEAD_NO][PARA_LEADTHRU_LEN]; /* prog #21 */
WCHAR ParaReportName[MAX_REPO_NO][PARA_REPORTNAME_LEN]; /* prog #22 */
WCHAR ParaSpeMnemo[MAX_SPEM_NO][PARA_SPEMNEMO_LEN]; /* prog #23 */
WCHAR ParaPCIF[MAX_PCIF_SIZE]; /* prog #39 */
WCHAR ParaAdjMnemo[MAX_ADJM_NO][PARA_ADJMNEMO_LEN]; /* prog #46 */
WCHAR ParaPrtModi[MAX_PRTMODI_NO][PARA_PRTMODI_LEN]; /* prog #47 */
WCHAR ParaMajorDEPT[MAX_MDEPT_NO][PARA_MAJORDEPT_LEN]; /* prog #48 */
// ... stuff deleted
} UNINIRAM;
The current approach is to use a template to define a proxy class for each of the arrays and then to have a single iterator class that can be used to iterate over a particular array by using a proxy object representing the array.
A copy of the memory resident data is stored in an object that handles reading and writing the memory resident data from/to disk. This class, CFilePara contains the templated proxy class (MnemonicIteratorDimSize and the sub class from which is it is derived, MnemonicIteratorDimSizeBase) and the iterator class, MnemonicIterator.
The created proxy object is attached to an iterator object which accesses the necessary information through an interface described by a base class from which all of the proxy classes are derived. The result is to have a single type of iterator class which can be used with several different proxy classes because the different proxy classes all expose the same interface, the interface of the proxy base class.
The first thing was to create a set of identifiers which would be provided to a class factory to generate the specific proxy object for that type of mnemonic. These identifiers are used as part of the user interface to identify the particular provisioning data the user is interested in seeing and possibly modifying.
const static DWORD_PTR dwId_TransactionMnemonic = 1;
const static DWORD_PTR dwId_ReportMnemonic = 2;
const static DWORD_PTR dwId_SpecialMnemonic = 3;
const static DWORD_PTR dwId_LeadThroughMnemonic = 4;
The Proxy Class
The templated proxy class and its base class are as follows. I needed to accommodate several different kinds of wchar_t text string arrays. The two dimensional arrays had different numbers of mnemonics, depending on the type (purpose) of the mnemonic and the different types of mnemonics were of different maximum lengths, varying between five text characters and twenty text characters. Templates for the derived proxy class was a natural fit with the template requiring the maximum number of characters in each mnemonic. After the proxy object is created, we then use the SetRange() method to specify the actual mnemonic array and its range.
// proxy object which represents a particular subsection of the
// memory resident database each of which is an array of wchar_t
// text arrays though the number of array elements may vary.
class MnemonicIteratorDimSizeBase
{
DWORD_PTR m_Type;
public:
MnemonicIteratorDimSizeBase(DWORD_PTR x) { }
virtual ~MnemonicIteratorDimSizeBase() { }
virtual wchar_t *begin() = 0;
virtual wchar_t *end() = 0;
virtual wchar_t *get(int i) = 0;
virtual int ItemSize() = 0;
virtual int ItemCount() = 0;
virtual DWORD_PTR ItemType() { return m_Type; }
};
template <size_t sDimSize>
class MnemonicIteratorDimSize : public MnemonicIteratorDimSizeBase
{
wchar_t (*m_begin)[sDimSize];
wchar_t (*m_end)[sDimSize];
public:
MnemonicIteratorDimSize(DWORD_PTR x) : MnemonicIteratorDimSizeBase(x), m_begin(0), m_end(0) { }
virtual ~MnemonicIteratorDimSize() { }
virtual wchar_t *begin() { return m_begin[0]; }
virtual wchar_t *end() { return m_end[0]; }
virtual wchar_t *get(int i) { return m_begin[i]; }
virtual int ItemSize() { return sDimSize; }
virtual int ItemCount() { return m_end - m_begin; }
void SetRange(wchar_t (*begin)[sDimSize], wchar_t (*end)[sDimSize]) {
m_begin = begin; m_end = end;
}
};
The Iterator Class
The iterator class itself is as follows. This class provides just basic forward iterator functionality which is all that is needed at this time. However I expect that this will change or be extended when I need something additional from it.
class MnemonicIterator
{
private:
MnemonicIteratorDimSizeBase *m_p; // we do not own this pointer. we just use it to access current item.
int m_index; // zero based index of item.
wchar_t *m_item; // value to be returned.
public:
MnemonicIterator(MnemonicIteratorDimSizeBase *p) : m_p(p) { }
~MnemonicIterator() { }
// a ranged for needs begin() and end() to determine the range.
// the range is up to but not including what end() returns.
MnemonicIterator & begin() { m_item = m_p->get(m_index = 0); return *this; } // begining of range of values for ranged for. first item
MnemonicIterator & end() { m_item = m_p->get(m_index = m_p->ItemCount()); return *this; } // end of range of values for ranged for. item after last item.
MnemonicIterator & operator ++ () { m_item = m_p->get(++m_index); return *this; } // prefix increment, ++p
MnemonicIterator & operator ++ (int i) { m_item = m_p->get(m_index++); return *this; } // postfix increment, p++
bool operator != (MnemonicIterator &p) { return **this != *p; } // minimum logical operator is not equal to
wchar_t * operator *() const { return m_item; } // dereference iterator to get what is pointed to
};
The proxy object factory determines which object to created based on the mnemonic identifier. The proxy object is created and the pointer returned is the standard base class type so as to have a uniform interface regardless of which of the different mnemonic sections are being accessed. The SetRange() method is used to specify to the proxy object the specific array elements the proxy represents and the range of the array elements.
CFilePara::MnemonicIteratorDimSizeBase * CFilePara::MakeIterator(DWORD_PTR x)
{
CFilePara::MnemonicIteratorDimSizeBase *mi = nullptr;
switch (x) {
case dwId_TransactionMnemonic:
{
CFilePara::MnemonicIteratorDimSize<PARA_TRANSMNEMO_LEN> *mk = new CFilePara::MnemonicIteratorDimSize<PARA_TRANSMNEMO_LEN>(x);
mk->SetRange(&m_Para.ParaTransMnemo[0], &m_Para.ParaTransMnemo[MAX_TRANSM_NO]);
mi = mk;
}
break;
case dwId_ReportMnemonic:
{
CFilePara::MnemonicIteratorDimSize<PARA_REPORTNAME_LEN> *mk = new CFilePara::MnemonicIteratorDimSize<PARA_REPORTNAME_LEN>(x);
mk->SetRange(&m_Para.ParaReportName[0], &m_Para.ParaReportName[MAX_REPO_NO]);
mi = mk;
}
break;
case dwId_SpecialMnemonic:
{
CFilePara::MnemonicIteratorDimSize<PARA_SPEMNEMO_LEN> *mk = new CFilePara::MnemonicIteratorDimSize<PARA_SPEMNEMO_LEN>(x);
mk->SetRange(&m_Para.ParaSpeMnemo[0], &m_Para.ParaSpeMnemo[MAX_SPEM_NO]);
mi = mk;
}
break;
case dwId_LeadThroughMnemonic:
{
CFilePara::MnemonicIteratorDimSize<PARA_LEADTHRU_LEN> *mk = new CFilePara::MnemonicIteratorDimSize<PARA_LEADTHRU_LEN>(x);
mk->SetRange(&m_Para.ParaLeadThru[0], &m_Para.ParaLeadThru[MAX_LEAD_NO]);
mi = mk;
}
break;
}
return mi;
}
Using the Proxy Class and Iterator
The proxy class and its iterator are used as shown in the following loop to fill in a CListCtrl object with a list of mnemonics. I am using std::unique_ptr so that when the proxy class i not longer needed and the std::unique_ptr goes out of scope, the memory will be cleaned up.
What this source code does is to create a proxy object for the array within the struct which corresponds to the specified mnemonic identifier. It then creates an iterator for that object, uses a ranged for to fill in the CListCtrl control and then cleans up. These are all raw wchar_t text strings which may be exactly the number of array elements so we copy the string into a temporary buffer in order to ensure that the text is zero terminated.
std::unique_ptr<CFilePara::MnemonicIteratorDimSizeBase> pObj(pFile->MakeIterator(m_IteratorType));
CFilePara::MnemonicIterator pIter(pObj.get()); // provide the raw pointer to the iterator who doesn't own it.
int i = 0; // CListCtrl index for zero based position to insert mnemonic.
for (auto x : pIter)
{
WCHAR szText[32] = { 0 }; // Temporary buffer.
wcsncpy_s(szText, 32, x, pObj->ItemSize());
m_mnemonicList.InsertItem(i, szText); i++;
}
And now a keys iterator for range-based for loop.
template<typename C>
class keys_it
{
typename C::const_iterator it_;
public:
using key_type = typename C::key_type;
using pointer = typename C::key_type*;
using difference_type = std::ptrdiff_t;
keys_it(const typename C::const_iterator & it) : it_(it) {}
keys_it operator++(int ) /* postfix */ { return it_++ ; }
keys_it& operator++( ) /* prefix */ { ++it_; return *this ; }
const key_type& operator* ( ) const { return it_->first ; }
const key_type& operator->( ) const { return it_->first ; }
keys_it operator+ (difference_type v ) const { return it_ + v ; }
bool operator==(const keys_it& rhs) const { return it_ == rhs.it_; }
bool operator!=(const keys_it& rhs) const { return it_ != rhs.it_; }
};
template<typename C>
class keys_impl
{
const C & c;
public:
keys_impl(const C & container) : c(container) {}
const keys_it<C> begin() const { return keys_it<C>(std::begin(c)); }
const keys_it<C> end () const { return keys_it<C>(std::end (c)); }
};
template<typename C>
keys_impl<C> keys(const C & container) { return keys_impl<C>(container); }
Usage:
std::map<std::string,int> my_map;
// fill my_map
for (const std::string & k : keys(my_map))
{
// do things
}
That's what i was looking for. But nobody had it, it seems.
You get my OCD code alignment as a bonus.
As an exercise, write your own for values(my_map)