I have this Container class with begin() and end() functions for use with c++11 foreach loops:
class Element
{
//content doesn't matter
};
class Container
{
Element* elements;
int size;
/* constructor, destructor, operators, methods, etc.. */
Element* begin() { return elements; };
Element* end() { return elements + size; };
};
This is now a valid c++11 foreach loop:
Container container;
for (Element& e : container)
{
//do something
}
But now consider this foreach loop:
Container container;
for (Element* e : container)
{
//do something
}
Is it possible to have a foreach loop with Element* instead of Element& like this?
This would also have the great advantage of preventing one from typing for (Element e : container) which would copy the element each time.
In that case begin() and end() would have to return Element** as far as I know.
But sizeof(Element) is not guaranteed to be sizeof(Element*) and in most cases they don't match. Incrementing a pointer increments by the base type size which is sizeof(Element) for incrementing Element* and sizeof(Element*) for incrementing Element**.
So the prefix operator++() will offset the pointer by a false value and things get crappy. Any ideas how to get this to work?
I agree with LRiO that what you have right now is probably the best solution. It additionally lines up with how the standard containers operate, and taking the path of least surprise for your users is always the best path to take (barring compelling reasons to diverge).
That said, you can certainly get the behavior you want:
class Container
{
// ...
struct iterator {
Element* e;
// this is the important one
Element* operator*() { return e; }
// the rest are just boilerplate
iterator& operator++() { ++e; return *this; }
iterator operator++(int) {
iterator tmp{e};
++*this;
return tmp;
}
bool operator==(iterator rhs) const { return e == rhs.e; }
bool operator!=(iterator rhs) const { return e != rhs.e; }
};
iterator begin() { return {elements}; };
iterator end() { return {elements + size}; };
};
You could consider inheriting from std::iterator to get the typedefs right, or using boost::iterator_facade. But this'll at least give you the functionality.
I can see why you'd want to do this, since an Element*→Element typo would be caught as an error right away, whereas an Element&→Element typo is usually a silent bug. However, in the grand scheme of things, I don't think it's worth transforming your entire container.
You could try to create a container adaptor that maintains the current behaviour of its iterators, except in the value type they expose… but I'm not even sure it's possible to do that without breaking various preconditions.
Personally, I just wouldn't.
Related
I tried to simplify the situation as good as possible.
Basically I have a class that represents a dynamic Array (here shown as class Array with size 4) and secondly a class which is a HashMap, and has an Array of Arrays as a member (here represented by class Bar).
Both classes implement begin() and end() so you can use a foreach loop to iterate over all elements.
The iterator type of Array is simply T* and const T* for the const variant.
For Bar there is a special class Iterator, which correctly iterates through all members of the Array<Array<T>>.
You should now have a look at the classes I provided, so you know what exactly I'm talking about.
But now there is a problem when I call bar.begin() on a const Bar object
The Iterator class determines the ArrayIterator and ElementIterator types automatically (via decltype of T::begin()), because in my real application almost everything is templated and that's why I don't know the exact type in advance.
I figured out the problem is that decltype(((T*)nullptr)->begin()) always chooses the non-const begin() function of T, which absolutely makes sense since I haven't written (const T*)nullptr.
If I now call it from a const context, it will fail to assign a const T* like data.last().end() to the internal T* from the decltype which actually should be a const T*.
I can workaround the problem by declaring a second class ConstIterator which does everything exactly like the non-const one, but uses (const Array<T>*)nullptr and (const T*)nullptr inside the decltype statements.
So what can I do without copying the whole Bar::Iterator class?
Simplified code:
template<class T>
class Array
{
T data[4];
T last() { return data[3]; }
T* begin() { return data; };
T* end() { return data + 4; };
const T* begin() const { return data; };
const T* end() const { return data + 4; };
}
template<class T>
class Bar
{
class Iterator
{
using ArrayIterator = decltype(((Array<T>*)nullptr)->begin());
using ElementIterator = decltype(((T*)nullptr)->begin());
Iterator(const ArrayIterator& beg, const ArrayIterator& end)
{
//initialize the iterator to the first element of the first array
//(and rembember end)
};
Iterator(const ElementIterator& cur)
{
//initialize the iterator to the current element
};
//++ will iterate go to next element and eventually jump to the next array.
//== returns true if the current element is the same
};
Array<T> data;
Iterator begin() { return Iterator(data.begin(), data.end()); };
Iterator end() { return Iterator(data.last().end()); };
Iterator begin() const { return Iterator(data.begin(), data.end()); };
Iterator end() const { return Iterator(data.last().end()); };
};
Actually, you will have to implement ConstIterator class. See, for example, the standard containers like vector or list, you have both iterator and const_iterator there, and they're different classes. Const correctness is hard sometimes. However, you can get away with code duplication by using templates or const_cast (see, for example, Scott Myers Effective C++)
I have a C++ class that acts like a container: it has size() and operator[] member functions. The values stored "in" the container are std::tuple objects. However, the container doesn't actually hold the tuples in memory; instead, it constructs them on-demand based on underlying data stored in a different form.
std::tuple<int, int, int>
MyContainer::operator[](std::size_t n) const {
// Example: draw corresponding elements from parallel arrays
return { underlying_data_a[n], underlying_data_b[n], underlying_data_c[n] };
}
Hence, the return type of operator[] is a temporary object, not a reference. (This means it's not an lvalue, so the container is read-only; that's OK.)
Now I'm writing an iterator class that can be used to traverse the tuples in this container. I'd like to model RandomAccessIterator, which depends on InputIterator, but InputIterator requires support for the expression i->m (where i is an iterator instance), and as far as I can tell, an operator-> function is required to return a pointer.
Naturally, I can't return a pointer to a temporary tuple that's constructed on-demand. One possibility that comes to mind is to put a tuple instance into the iterator as a member variable, and use it to store a copy of whichever value the iterator is currently positioned on:
class Iterator {
private:
MyContainer *container;
std::size_t current_index;
// Copy of (*container)[current_index]
std::tuple<int, int, int> current_value;
// ...
};
However, updating the stored value will require the iterator to check whether its current index is less than the container's size, so that a past-the-end iterator doesn't cause undefined behavior by accessing past the end of the underlying arrays. That adds (a small amount of) runtime overhead — not enough to make the solution impractical, of course, but it feels a little inelegant. The iterator shouldn't really need to store anything but a pointer to the container it's iterating and the current position within it.
Is there a clean, well-established way to support operator-> for iterator types that construct their values on-demand? How would other developers do this sort of thing?
(Note that I don't really need to support operator-> at all — I'm implementing the iterator mainly so that the container can be traversed with a C++11 "range for" loop, and std::tuple doesn't have any members that one would typically want to access via -> anyway. But I'd like to model the iterator concepts properly nonetheless; it feels like I'm cutting corners otherwise. Or should I just not bother?)
template<class T>
struct pseudo_ptr {
T t;
T operator*()&&{return t;}
T* operator->(){ return &t; }
};
then
struct bar { int x,y; };
struct bar_iterator:std::iterator< blah, blah >{
// ...
pseudo_ptr<bar> operator->() const { return {**this}; }
// ...
};
This relies on how -> works.
ptr->b for pointer ptr is simply (*ptr).b.
Otherwise it is defined as (ptr.operator->())->b. This evaluates recursively if operator-> does not return a pointer.
The pseudo_ptr<T> above gives you a wrapper around a copy of T.
Note, however, that lifetime extension doesn't really work. The result is fragile.
Here's an example relying on the fact that operator-> is applied repeatedly until a pointer is returned. We make Iterator::operator-> return the Contained object as a temporary. This causes the compiler to reapply operator->. We then make Contained::operator-> simply return a pointer to itself. Note that if we don't want to put operator-> in the Contained on-the-fly object, we can wrap it in a helper object that returns a pointer to the internal Contained object.
#include <cstddef>
#include <iostream>
class Contained {
public:
Contained(int a_, int b_) : a(a_), b(b_) {}
const Contained *operator->() {
return this;
}
const int a, b;
};
class MyContainer {
public:
class Iterator {
friend class MyContainer;
public:
friend bool operator!=(const Iterator &it1, const Iterator &it2) {
return it1.current_index != it2.current_index;
}
private:
Iterator(const MyContainer *c, std::size_t ind) : container(c), current_index(ind) {}
public:
Iterator &operator++() {
++current_index;
return *this;
}
// -> is reapplied, since this returns a non-pointer.
Contained operator->() {
return Contained(container->underlying_data_a[current_index], container->underlying_data_b[current_index]);
}
Contained operator*() {
return Contained(container->underlying_data_a[current_index], container->underlying_data_b[current_index]);
}
private:
const MyContainer *const container;
std::size_t current_index;
};
public:
MyContainer() {
for (int i = 0; i < 10; i++) {
underlying_data_a[i] = underlying_data_b[i] = i;
}
}
Iterator begin() const {
return Iterator(this, 0);
}
Iterator end() const {
return Iterator(this, 10);
}
private:
int underlying_data_a[10];
int underlying_data_b[10];
};
int
main() {
MyContainer c;
for (const auto &e : c) {
std::cout << e.a << ", " << e.b << std::endl;
}
}
I have created a MemoryManager<T> class which is basically a wrapper around two vectors of pointers that manage lifetime of heap-allocated objects.
One vector stores the "alive" objects, the other one stores the object that will be added on next MemoryManager<T>::refresh.
This design was chosen to avoid iterator invalidation when looping over the MemoryManager<T>, as adding a new object directly to the MemoryManager<T>::alive vector can invalidate existing iterators (if it grows in size).
template<typename T> struct MemoryManager {
std::vector<std::unique_ptr<T>> alive;
std::vector<T*> toAdd;
T& create() {
auto r(new T);
toAdd.push_back(r);
return *r;
}
T& refresh() {
// Use erase-remove idiom on dead objects
eraseRemoveIf(alive, [](const std::unique_ptr<T>& p){ return p->alive; });
// Add all "toAdd" objects and clear the "toAdd" vector
for(auto i : toAdd) alive.emplace_back(i);
toAdd.clear();
}
void kill(T& mItem) { mItem.alive = false; }
IteratorType begin() { return alive.begin(); }
IteratorType end() { return alive.end(); }
}
I use it in my game engine to store entities, and update every "alive" entity every frame:
void game() {
MemoryManager<Entity> mm;
while(gameLoop) {
mm.refresh();
for(auto e : mm) processEntity(e);
auto& newEntity = mm.create();
// do something with newEntity
}
}
This has allowed me to constantly create/kill entities without having to worry about their lifetime too much.
However, I've recently come to the conclusion that using two std::vector is unnecessary. I could simply use a single vector and store an iterator to the "last alive object", adding the newly create objects immediately after the aforementioned iterator:
The idea, in my mind, works fine... but I cannot actually use a iterator type for end (as shown in the diagram), as it could get invalidated after the addition of some new elements to the vector. I've tested it, and this happens often, causing a crash.
The other solution I can think of is using an index instead of an iterator. This would solve the crashing, but I wouldn't be able to use the cool C++11 for(x : y) foreach loop because MemoryManager<T>::begin and MemoryManager<T>::end need to return an iterator.
Is there a way to achieve the current behavior with a single vector and still maintain a clear interface that can be used with C++11 for-each loops?
One of the simplest ways to get stable iterators (and references) is to use std::list<T>. And unless you are needing T to be a pointer to a polymorphic base class, it is better to use std::list<T>, as opposed to std::list<std::unique_ptr<T>>.
If on the other hand, your Entity is a polymorphic base, then consider using std::vector<std::unique_ptr<T>>. Although you can not depend upon iterators remaining valid, you can depend upon pointers and references to Entity remaining valid with std::vector<std::unique_ptr<T>>.
In your game() example, you never take advantage of stable iterators or pointers. You could just as easily (and more simply) do:
void game() {
std::vector<Entity> mm;
while(gameLoop) {
mm.erase(std::remove_if(mm.begin(), mm.end(), [](const Entity& e)
{ return e.alive; }),
mm.end());
for(auto e : mm) processEntity(e);
mm.push_back(create());
auto& newEntity = mm.back();
// do something with newEntity
}
}
During the processEntity loop, there is no way to invalidate iterators. If you did, you had better not use the range-based-for as the end iterator is only evaluated once, at the beginning of the loop.
But if you really do need stable iterators/references, substituting in std::list<Entity> would be very easy. I would change the erase/remove to use list's member remove_if instead. It will be more efficient.
If you do this, and performance testing (not guessing) indicates you've suffered a performance hit over your existing MemoryManager, you can optimize list by using a "stack allocator" such as the one demonstrated here:
http://howardhinnant.github.io/stack_alloc.html
This allows you to preallocate space (could be on the stack, could be on the heap), and have your container allocate from that. This will be both high performance and cache-friendly until the pre-allocated space is exhausted. And you've still got your iterator/pointer/reference stability.
In summary:
Find out / tell us if unique_ptr<Entity> is actually necessary because Entity is a base class. Prefer container<Entity> over container<unique_ptr<Entity>>.
Do you actually need iterator/pointer/reference stability? Your sample code does not. If you don't actually need it, don't pay for it. Use vector<Entity> (or vector<unique_ptr<Entity>> if you must).
If you actually need container<unique_ptr<Entity>>, can you get away with pointer/reference stability while sacrificing iterator stability? If yes, vector<unique_ptr<Entity>> is the way to go.
If you actually need iterator stability, strongly consider using std::list.
If you use std::list and discover via testing it has performance problems, optimize it with an allocator tuned to your needs.
If all of the above fails, then start designing your own data structure. If you get this far, know that this is the most difficult route, and everything will need to be backed up by both correctness and performance tests.
You can implement your own iterator class.
Something like the following may help.
template <typename T, typename... Ts>
class IndexIterator : public std::iterator<std::random_access_iterator_tag, T>
{
public:
IndexIterator(std::vector<T, Ts...>& v, std::size_t index) : v(&v), index(index) {}
// if needed.
typename std::vector<T, Ts...>::iterator getRegularIterator() const { return v->begin() + index; }
T& operator *() const { return v->at(index); }
T* operator ->() const { return &v->at(index); }
IndexIterator& operator ++() { ++index; return *this;}
IndexIterator& operator ++(int) { IndexIterator old(*this); ++*this; return old;}
IndexIterator& operator +=(std::ptrdiff_t offset) { index += offset; return *this;}
IndexIterator operator +(std::ptrdiff_t offset) const { IndexIterator res (*this); res += offset; return res;}
IndexIterator& operator --() { --index; return *this;}
IndexIterator& operator --(int) { IndexIterator old(*this); --*this; return old;}
IndexIterator& operator -=(std::ptrdiff_t offset) { index -= offset; return *this;}
IndexIterator operator -(std::ptrdiff_t offset) const { IndexIterator res (*this); res -= offset; return res;}
std::ptrdiff_t operator -(const IndexIterator& rhs) const { assert(v == rhs.v); return index - rhs.index; }
bool operator == (const IndexIterator& rhs) const { assert(v == rhs.v); return index == rhs.index; }
bool operator != (const IndexIterator& rhs) const { return !(*this == rhs); }
private:
std::vector<T, Ts...>* v;
std::size_t index;
};
template <typename T, typename... Ts>
IndexIterator<T, Ts...> IndexIteratorBegin(std::vector<T, Ts...>& v)
{
return IndexIterator<T, Ts...>(v, 0);
}
template <typename T, typename... Ts>
IndexIterator<T, Ts...> IndexIteratorEnd(std::vector<T, Ts...>& v)
{
return IndexIterator<T, Ts...>(v, v.size());
}
You could avoid moving elements of the container by maintaining a free-list (see http://www.memorymanagement.org/glossary/f.html#free.list).
To avoid invalidation of references to elements you can use a std::deque if you do not insert or erase in the middle.
To avoid invalidation of iterators you can use a std::list.
(Thanks Howard Hinnant)
You can implement your own iterator class which handles things the way you prefer. Then your begin() and end() can return instances of that class. For example, your custom iterator could store an integer index and a pointer to the vector itself, thereby making the iterator remain valid even in the face of reallocation.
I have a vector< Object > myvec which I use in my code to hold a list of objects in memory. I keep a pointer to the current object in that vector in the "normal" C fashion by using
Object* pObj = &myvec[index];
This all works fine if... myvec doesn't grow big enough that it is moved around during a push_back at which time pObj becomes invalid - vectors guarantee data is sequential, hence they make no effort to keep the vector at the same memory location.
I can reserve enough space for myvec to prevent this, but I dnt' like that solution.
I could keep the index of the selected myvec position and when I need to use it just access it directly, but it's a costly modification to my code.
I'm wondering if iterators keep the their references intact as a vector is reallocated/moved and if so can I just replace
Object* pObj = &myvec[index];
by something like
vector<Object>::iterator = myvec.begin()+index;
What are the implication of this?
Is this doable?
What is the standard pattern to save pointers to vector positions?
Cheers
No... using an iterator you would have the same exact problem. If a vector reallocation is performed then all iterators are invalidated and using them is Undefined Behavior.
The only solution that is reallocation-resistant with an std::vector is using the integer index.
Using for example std::list things are different, but also the are different efficiency compromises, so it really depends on what you need to do.
Another option would be to create your own "smart index" class, that stores a reference to the vector and the index. This way you could keep just passing around one "pointer" (and you could implement pointer semantic for it) but the code wouldn't suffer from reallocation risks.
Iterators are (potentially) invalidated by anything that could resize the vector (e.g., push_back).
You could, however, create your own iterator class that stored the vector and an index, which would be stable across operations that resized the vector:
#include <iterator>
#include <algorithm>
#include <iostream>
#include <vector>
namespace stable {
template <class T, class Dist=ptrdiff_t, class Ptr = T*, class Ref = T&>
class iterator : public std::iterator<std::random_access_iterator_tag, T, Dist, Ptr, Ref>
{
T &container_;
size_t index_;
public:
iterator(T &container, size_t index) : container_(container), index_(index) {}
iterator operator++() { ++index_; return *this; }
iterator operator++(int) { iterator temp(*this); ++index_; return temp; }
iterator operator--() { --index_; return *this; }
iterator operator--(int) { stable_itertor temp(*this); --index_; return temp; }
iterator operator+(Dist offset) { return iterator(container_, index_ + offset); }
iterator operator-(Dist offset) { return iterator(container_, index_ - offset); }
bool operator!=(iterator const &other) const { return index_ != other.index_; }
bool operator==(iterator const &other) const { return index_ == other.index_; }
bool operator<(iterator const &other) const { return index_ < other.index_; }
bool operator>(iterator const &other) const { return index_ > other.index_; }
typename T::value_type &operator *() { return container_[index_]; }
typename T::value_type &operator[](size_t index) { return container_[index_ + index]; }
};
template <class T>
iterator<T> begin(T &container) { return iterator<T>(container, 0); }
template <class T>
iterator<T> end(T &container) { return iterator<T>(container, container.size()); }
}
#ifdef TEST
int main() {
std::vector<int> data;
// add some data to the container:
for (int i=0; i<10; i++)
data.push_back(i);
// get iterators to the beginning/end:
stable::iterator<std::vector<int> > b = stable::begin(data);
stable::iterator<std::vector<int> > e = stable::end(data);
// add enough more data that the container will (probably) be resized:
for (int i=10; i<10000; i++)
data.push_back(i);
// Use the previously-obtained iterators:
std::copy(b, e, std::ostream_iterator<int>(std::cout, "\n"));
// These iterators also support most pointer-like operations:
std::cout << *(b+125) << "\n";
std::cout << b[150] << "\n";
return 0;
}
#endif
Since we can't embed this as a nested class inside of the container like a normal iterator class, this requires a slightly different syntax to declare/define an object of this type; instead of the usual std::vector<int>::iterator whatever;, we have to use stable::iterator<std::vector<int> > whatever;. Likewise, to obtain the beginning of a container, we use stable::begin(container).
There is one point that may be a bit surprising (at least at first): when you obtain a stable::end(container), that gets you the end of the container at that time. As shown in the test code above, if you later add more items to the container, the iterator your obtained previously is not adjusted to reflect the new end of the container -- it retains the position it had when you obtained it (i.e., the position that was the end of the container at that time, but isn't any more).
No, iterators are invalidated after vector growth.
The way to get around this problem is to keep the index to the item, not a pointer or iterator to it. This is because the item stays at its index, even if the vector grows, assuming of course that you don't insert any items before it (thus changing its index).
I'm writing some C++ code that manipulates a bunch of vectors that are changing in size and are thus being reallocated constantly.
I would like to get a "pointer" into these vectors that remains valid even after reallocation of the vector. More specifically, I just want these "pointers" to remember which vector they point into and the index to which they point. When I dereference them using the standard (*ptr) syntax, I just want them to do the obvious lookup.
Obviously, actual pointers will not be valid after reallocation, and my understanding is that iterators aren't valid after reallocation either. Note also that I don't care if elements are inserted before my objects, so these "pointers" really don't have to remember anything but a vector and an index.
Now, I could easily write such a class myself. Has anyone (Boost? STL?) done it for me already?
Edit: The answers don't address my question. I asked if this functionality is any standard library. I take the responses as a "no"?
Try a std::pair< vector*, int>, as neither the position of the vector nor the index of the element changes.
Or, as a class:
template<class T> class VectorElementPointer
{
vector<T>& vectorref;
typename vector<T>::size_type index;
public:
VectorElementPointer(vector<T>& vref, typename vector<T>::size_type index):vectorref(vref),index(index){}
T& operator*() const {return vectorref[index];}
T* operator->() const {return &vectorref[index];}
};
This is the easiest solution that comes to my mind, as neither the STL nor Boost contains anything to do it easier.
An article on persistent iterators, complete with implementation.
To summarize some ideas. Here is the minimalist wrapper that tries to mimic iterators but stay valid as opposite to vector's ones.
void print(const std::string& i)
{
std::cout << "<" << i << "> ";
}
int main()
{
typedef std::vector<std::string> Vector;
Vector v;
v.push_back("H");
v.push_back("E");
v.push_back("W");
StrongIterator<Vector> it0(v, 0);
StrongIterator<Vector> it3(v, v.end());
std::for_each(it0.it(), it3.it(), print);
std::cout << std::endl;
v.push_back("O");
std::for_each(it0.it(), it3.it(), print);
std::cout << *it0;
std::cout << it0->c_str();
return 0;
}
And the iterator itself.
template <typename TVector>
class StrongIterator
{
public:
typedef typename TVector::iterator iterator;
typedef typename TVector::size_type size_type;
typedef typename TVector::value_type value_type;
StrongIterator(TVector& vector,
size_type index):
vector_(vector),
index_(index)
{}
StrongIterator(TVector& vector,
iterator it):
vector_(vector),
index_(std::distance(vector.begin(), it))
{}
iterator it()
{
iterator it = vector_.begin();
std::advance(it, index_);
return it;
}
value_type& operator*()
{
return vector_[index_];
}
value_type* operator->()
{
return &vector_[index_];
}
private:
TVector& vector_;
size_type index_;
};
Unfortunately, once you modify the vector, the iterators that would "point" to an element of the vector are no longer guaranteed to be valid. The only STL structure that I know of which will keep the iterators valid even as the structure is changing is the list<>. If you only want sequential iteration of your structures than you can use std::list<> otherwise I do not know of any other library that can help you; that doesn't mean there isn't one.
Here's some clear documentation on std::list : http://www.cplusplus.com/reference/stl/list/
Using boost::iterator_facade :
// Warning: Untested, not even compiled
template<class VectorT>
class VectorIndex :
public boost::iterator_facade<VectorIndex, typename VectorT::reference, boost::random_access_traversal_tag>
{
public:
VectorIndex(VectorT& Vec, typename VectorT::size_type Index)
: m_Vec(Vec), m_Index(Index)
{
}
private:
friend class boost::iterator_core_access;
void increment()
{
++m_Index;
}
void decrement()
{
--m_Index;
}
void advance(difference_type N)
{
m_Index += N;
}
difference_type distance_to(const VectorIndex& Other)
{
assert(&this->m_Vec == &Other.m_Vec);
return Other.m_Index = this->m_Index;
}
bool equal(const VectorIndex& Other)const
{
return (this->m_Vec == Other.m_Vec)
&& (this->m_Index == Other.m_Index);
}
VectorT::reference dereference() const
{
return m_Vec[m_Index];
}
VectorT m_Vec;
VectorT::size_type m_Index;
};
Unless you write your own version of vector and smart pointer there is no way that a pointer will be valid after a reallocation. If you had your own implementations the smart vector could send notifications to your smart pointers.
However I think that the whole scenario is a bad design and you might be better of redesigning your scenario so that you don't have a requirement like that.
Depending on your use pattern, a std::deque may fufil your requirements. Pointers into a deque are only invalidated if you insert or delete items not at the beginning or end - in pother words push_front() and push_back() don't invalidate pointers into the deque, but other changes do. You get basically the same interface as a vector, but of course the underlying storage is not contiguous.