pointer to vector at index Vs iterator - c++

I have a vector< Object > myvec which I use in my code to hold a list of objects in memory. I keep a pointer to the current object in that vector in the "normal" C fashion by using
Object* pObj = &myvec[index];
This all works fine if... myvec doesn't grow big enough that it is moved around during a push_back at which time pObj becomes invalid - vectors guarantee data is sequential, hence they make no effort to keep the vector at the same memory location.
I can reserve enough space for myvec to prevent this, but I dnt' like that solution.
I could keep the index of the selected myvec position and when I need to use it just access it directly, but it's a costly modification to my code.
I'm wondering if iterators keep the their references intact as a vector is reallocated/moved and if so can I just replace
Object* pObj = &myvec[index];
by something like
vector<Object>::iterator = myvec.begin()+index;
What are the implication of this?
Is this doable?
What is the standard pattern to save pointers to vector positions?
Cheers

No... using an iterator you would have the same exact problem. If a vector reallocation is performed then all iterators are invalidated and using them is Undefined Behavior.
The only solution that is reallocation-resistant with an std::vector is using the integer index.
Using for example std::list things are different, but also the are different efficiency compromises, so it really depends on what you need to do.
Another option would be to create your own "smart index" class, that stores a reference to the vector and the index. This way you could keep just passing around one "pointer" (and you could implement pointer semantic for it) but the code wouldn't suffer from reallocation risks.

Iterators are (potentially) invalidated by anything that could resize the vector (e.g., push_back).
You could, however, create your own iterator class that stored the vector and an index, which would be stable across operations that resized the vector:
#include <iterator>
#include <algorithm>
#include <iostream>
#include <vector>
namespace stable {
template <class T, class Dist=ptrdiff_t, class Ptr = T*, class Ref = T&>
class iterator : public std::iterator<std::random_access_iterator_tag, T, Dist, Ptr, Ref>
{
T &container_;
size_t index_;
public:
iterator(T &container, size_t index) : container_(container), index_(index) {}
iterator operator++() { ++index_; return *this; }
iterator operator++(int) { iterator temp(*this); ++index_; return temp; }
iterator operator--() { --index_; return *this; }
iterator operator--(int) { stable_itertor temp(*this); --index_; return temp; }
iterator operator+(Dist offset) { return iterator(container_, index_ + offset); }
iterator operator-(Dist offset) { return iterator(container_, index_ - offset); }
bool operator!=(iterator const &other) const { return index_ != other.index_; }
bool operator==(iterator const &other) const { return index_ == other.index_; }
bool operator<(iterator const &other) const { return index_ < other.index_; }
bool operator>(iterator const &other) const { return index_ > other.index_; }
typename T::value_type &operator *() { return container_[index_]; }
typename T::value_type &operator[](size_t index) { return container_[index_ + index]; }
};
template <class T>
iterator<T> begin(T &container) { return iterator<T>(container, 0); }
template <class T>
iterator<T> end(T &container) { return iterator<T>(container, container.size()); }
}
#ifdef TEST
int main() {
std::vector<int> data;
// add some data to the container:
for (int i=0; i<10; i++)
data.push_back(i);
// get iterators to the beginning/end:
stable::iterator<std::vector<int> > b = stable::begin(data);
stable::iterator<std::vector<int> > e = stable::end(data);
// add enough more data that the container will (probably) be resized:
for (int i=10; i<10000; i++)
data.push_back(i);
// Use the previously-obtained iterators:
std::copy(b, e, std::ostream_iterator<int>(std::cout, "\n"));
// These iterators also support most pointer-like operations:
std::cout << *(b+125) << "\n";
std::cout << b[150] << "\n";
return 0;
}
#endif
Since we can't embed this as a nested class inside of the container like a normal iterator class, this requires a slightly different syntax to declare/define an object of this type; instead of the usual std::vector<int>::iterator whatever;, we have to use stable::iterator<std::vector<int> > whatever;. Likewise, to obtain the beginning of a container, we use stable::begin(container).
There is one point that may be a bit surprising (at least at first): when you obtain a stable::end(container), that gets you the end of the container at that time. As shown in the test code above, if you later add more items to the container, the iterator your obtained previously is not adjusted to reflect the new end of the container -- it retains the position it had when you obtained it (i.e., the position that was the end of the container at that time, but isn't any more).

No, iterators are invalidated after vector growth.
The way to get around this problem is to keep the index to the item, not a pointer or iterator to it. This is because the item stays at its index, even if the vector grows, assuming of course that you don't insert any items before it (thus changing its index).

Related

C++ "periodic" iterator over the custom container

I have a class that contains the vector of elements of the specific class. The main idea is to generate periodic sequence of the elements, based on the one period of the sequence (elems_) and the number of the periods (nperiod_) so I do not need to store all elements, but just one period.
class PeriodicContainer
{
private:
std::vector<Class> elems_; // elements
size_t nperiod_; // period of repetition of elems_
public:
PeriodicContainer();
PeriodicContainer(const std::vector<Class>& elems, size_t nperiod);
/*...*/
}
Is it possible to implement custom iterator for the PeriodicContainer so that I can do things like (semi-pseudo-code):
PeriodicContainer container({Class(1), Class(2)}, 4);
for (auto it : container)
std::cout << it << '\n';
and the output will be
Class(1)
Class(2)
Class(1)
Class(2)
Class(1)
Class(2)
Class(1)
Class(2)
If you can use range-v3, you can do:
namespace rv = ranges::views;
std::vector<Class> Container { Class(1), Class(2) };
for (auto it : rv::repeat_n(Container, 4) | rv::join)
std::cout << it;
and not have to write any additional code yourself. This will also work for any contiguous container, not just std::vector.
Here's a demo.
If your underlying container is simply a std::vector, then you know that it's a contiguous container -- which actually makes this quite easy.
You can form an iterator from the following:
A pointer (or reference) to the container being iterated, and
The current iteration count (note: not 'index'). This will be used as "index" into the underlying container's operator[] after wrapping around the container's size().
The behavior of this iterator would be simply:
Each increment just increments the current count
Each dereference returns (*elems_)[current_ % elems_->size()], which will account for the loop-around for the "period".
The begin() would simply return an iterator with a 0 count, and
The end() would return an iterator with a count of elems_.size() * nperiod_
An example of what this could look like as a LegacyForwardIterator is the following:
template <typename T>
class PeriodicContainerIterator
{
public:
using value_type = T;
using reference = T&;
using pointer = T*;
using difference_type = std::ptrdiff_t;
using iterator_category = std::forward_iterator_tag;
PeriodicContainerIterator(std::vector<T>* elems, std::size_t current)
: elems_{elems},
current_{current}
{}
reference operator*() {
return (*elems_)[current_ % elems_->size()]
}
pointer operator->() {
return &(*elems_)[current_ % elems_->size()];
}
PeriodicContainerIterator& operator++() const {
++current_;
return (*this);
}
PeriodicContainerIterator operator++(int) const {
auto copy = (*this);
++current_;
return copy;
}
bool operator==(const PeriodicContainerIterator& other) const {
return current_ == other.current_;
}
bool operator!=(const PeriodicContainerIterator& other) const {
return current_ != other.current_;
}
private:
std::vector<T>* elems_;
std::size_t current_;
};
The container would then define begin() and end() as:
PeriodicContainerIterator<Class> begin() {
return PeriodicContainerIterator<Class>{&elems_, 0};
}
PeriodicContainerIterator<Class> end() {
return PeriodicContainerIterator<Class>{&elems_, elems_->size() * nperiod_};
}
You could easily make this all the way up to a LegacyRandomAccessIterator, but this requires a lot of extra functions which will bulk this answer.
If you don't specifically need this as an iterator but just want a simple way to visit each element in the periodic sequence, it might be easier to read / understand if you were to make this into a for_each-like call that expects a callback instead. For example:
template <typename Fn>
void forEach(Fn&& fn)
{
for (auto i = 0; i < nperiod_; ++i) {
for (auto& e : elems_) {
fn(e);
}
}
}
Which allows for use like:
container.forEach([&](auto& e){
// 'e' is each visited element
});

Recasting a container of void pointers

Short version
Can I reinterpret_cast a std::vector<void*>* to a std::vector<double*>*?
What about with other STL containers?
Long version
I have a function to recast a vector of void pointers to a datatype specified by a template argument:
template <typename T>
std::vector<T*> recastPtrs(std::vector<void*> const& x) {
std::vector<T*> y(x.size());
std::transform(x.begin(), x.end(), y.begin(),
[](void *a) { return static_cast<T*>(a); } );
return y;
}
But I was thinking that copying the vector contents isn't really necessary, since we're really just reinterpreting what's being pointed to.
After some tinkering, I came up with this:
template <typename T>
std::vector<T*> recastPtrs(std::vector<void*>&& x) {
auto xPtr = reinterpret_cast<std::vector<T*>*>(&x);
return std::vector<T*>(std::move(*xPtr));
}
So my questions are:
Is it safe to reinterpret_cast an entire vector like this?
What if it was a different kind of container (like a std::list or std::map)? To be clear, I mean casting a std::list<void*> to std::list<T*>, not casting between STL container types.
I'm still trying to wrap my head around move semantics. Am I doing it right?
And one follow-up question: What would be the best way to generate a const version without code duplication? i.e. to define
std::vector<T const*> recastPtrs(std::vector<void const*> const&);
std::vector<T const*> recastPtrs(std::vector<void const*>&&);
MWE
#include <vector>
#include <algorithm>
#include <iostream>
template <typename T>
std::vector<T*> recastPtrs(std::vector<void*> const& x) {
std::vector<T*> y(x.size());
std::transform(x.begin(), x.end(), y.begin(),
[](void *a) { return static_cast<T*>(a); } );
return y;
}
template <typename T>
std::vector<T*> recastPtrs(std::vector<void*>&& x) {
auto xPtr = reinterpret_cast<std::vector<T*>*>(&x);
return std::vector<T*>(std::move(*xPtr));
}
template <typename T>
void printVectorAddr(std::vector<T> const& vec) {
std::cout<<" vector object at "<<&vec<<", data()="<<vec.data()<<std::endl;
}
int main(void) {
std::cout<<"Original void pointers"<<std::endl;
std::vector<void*> voidPtrs(100);
printVectorAddr(voidPtrs);
std::cout<<"Elementwise static_cast"<<std::endl;
auto dblPtrs = recastPtrs<double>(voidPtrs);
printVectorAddr(dblPtrs);
std::cout<<"reintepret_cast entire vector, then move ctor"<<std::endl;
auto dblPtrs2 = recastPtrs<double>(std::move(voidPtrs));
printVectorAddr(dblPtrs2);
}
Example output:
Original void pointers
vector object at 0x7ffe230b1cb0, data()=0x21de030
Elementwise static_cast
vector object at 0x7ffe230b1cd0, data()=0x21de360
reintepret_cast entire vector, then move ctor
vector object at 0x7ffe230b1cf0, data()=0x21de030
Note that the reinterpret_cast version reuses the underlying data structure.
Previously-asked questions that didn't seem relevant
These are the questions that come up when I tried to search this:
reinterpret_cast vector of class A to vector of class B
reinterpret_cast vector of derived class to vector of base class
reinterpret_cast-ing vector of one type to a vector of another type which is of the same type
And the answer to these was a unanimous NO, with reference to the strict aliasing rule. But I figure that doesn't apply to my case, since the vector being recast is an rvalue, so there's no opportunity for aliasing.
Why I'm trying to do this
I'm interfacing with a MATLAB library that gives me data pointers as void* along with a variable indicating the datatype. I have one function that validates the inputs and collects these pointers into a vector:
void parseInputs(int argc, mxArray* inputs[], std::vector<void*> &dataPtrs, mxClassID &numericType);
I can't templatize this part since the type is not known until runtime. On the other side, I have numeric routines to operate on vectors of a known datatype:
template <typename T>
void processData(std::vector<T*> const& dataPtrs);
So I'm just trying to connect one to the other:
void processData(std::vector<void*>&& voidPtrs, mxClassID numericType) {
switch (numericType) {
case mxDOUBLE_CLASS:
processData(recastPtrs<double>(std::move(voidPtrs)));
break;
case mxSINGLE_CLASS:
processData(recastPtrs<float>(std::move(voidPtrs)));
break;
default:
assert(0 && "Unsupported datatype");
break;
}
}
Given the comment that you're receiving the void * from a C library (something like malloc), it seems like we can probably narrow the problem down quite a bit.
In particular, I'd guess you're really dealing with something that's more like an array_view than a vector. That is, you want something that lets you access some data cleanly. You might change individual items in that collection, but you'll never change the collection as a whole (e.g., you won't try to do a push_back that could need to expand the memory allocation).
For such a case, you can pretty easily create a wrapper of your own that gives you vector-like access to the data--defines an iterator type, has a begin() and end() (and if you want, the others like rbegin()/rend(), cbegin()/cend() and crbegin()/crend()), as well as an at() that does range-checked indexing, and so on.
So a fairly minimal version could look something like this:
#pragma once
#include <cstddef>
#include <stdexcept>
#include <cstdlib>
#include <iterator>
template <class T> // note: no allocator, since we don't do allocation
class array_view {
T *data;
std::size_t size_;
public:
array_view(void *data, std::size_t size_) : data(reinterpret_cast<T *>(data)), size_(size_) {}
T &operator[](std::size_t index) { return data[index]; }
T &at(std::size_t index) {
if (index > size_) throw std::out_of_range("Index out of range");
return data[index];
}
std::size_t size() const { return size_; }
typedef T *iterator;
typedef T const &const_iterator;
typedef T value_type;
typedef T &reference;
iterator begin() { return data; }
iterator end() { return data + size_; }
const_iterator cbegin() { return data; }
const_iterator cend() { return data + size_; }
class reverse_iterator {
T *it;
public:
reverse_iterator(T *it) : it(it) {}
using iterator_category = std::random_access_iterator_tag;
using difference_type = std::ptrdiff_t;
using value_type = T;
using pointer = T *;
using reference = T &;
reverse_iterator &operator++() {
--it;
return *this;
}
reverse_iterator &operator--() {
++it;
return *this;
}
reverse_iterator operator+(size_t size) const {
return reverse_iterator(it - size);
}
reverse_iterator operator-(size_t size) const {
return reverse_iterator(it + size);
}
difference_type operator-(reverse_iterator const &r) const {
return it - r.it;
}
bool operator==(reverse_iterator const &r) const { return it == r.it; }
bool operator!=(reverse_iterator const &r) const { return it != r.it; }
bool operator<(reverse_iterator const &r) const { return std::less<T*>(r.it, it); }
bool operator>(reverse_iterator const &r) const { return std::less<T*>(it, r.it); }
T &operator *() { return *(it-1); }
};
reverse_iterator rbegin() { return data + size_; }
reverse_iterator rend() { return data; }
};
I've tried to show enough that it should be fairly apparent how to add most of the missing functionality (e.g., crbegin()/crend()), but I haven't worked really hard at including everything here, since much of what's left is more repetitive and tedious than educational.
This is enough to use the array_view in most of the typical vector-like ways. For example:
#include "array_view"
#include <iostream>
#include <iterator>
int main() {
void *raw = malloc(16 * sizeof(int));
array_view<int> data(raw, 16);
std::cout << "Range based:\n";
for (auto & i : data)
i = rand();
for (auto const &i : data)
std::cout << i << '\n';
std::cout << "\niterator-based, reverse:\n";
auto end = data.rend();
for (auto d = data.rbegin(); d != end; ++d)
std::cout << *d << '\n';
std::cout << "Forward, counted:\n";
for (int i=0; i<data.size(); i++) {
data[i] += 10;
std::cout << data[i] << '\n';
}
}
Note that this doesn't attempt to deal with copy/move construction at all, nor with destruction. At least as I've formulated it, the array_view is a non-owning view into some existing data. It's up to you (or at least something outside of the array_view) to destroy the data when appropriate. Since we're not destroying the data, we can use the compiler-generated copy and move constructors without any problem. We won't get a double-delete from doing a shallow copy of the pointer, because we don't do any delete when the array_view is destroyed.
No, you cannot do anything like this in Standard C++.
The strict aliasing rule says that to access an object of type T, you must use an expression of type T; with a very short list of exceptions to that.
Accessing a double * via a void * expression is not such an exception; let alone a vector of each. Nor is it an exception if you accessed the object of type T via an rvalue.

How to implement operator-> for an iterator that constructs its values on-demand?

I have a C++ class that acts like a container: it has size() and operator[] member functions. The values stored "in" the container are std::tuple objects. However, the container doesn't actually hold the tuples in memory; instead, it constructs them on-demand based on underlying data stored in a different form.
std::tuple<int, int, int>
MyContainer::operator[](std::size_t n) const {
// Example: draw corresponding elements from parallel arrays
return { underlying_data_a[n], underlying_data_b[n], underlying_data_c[n] };
}
Hence, the return type of operator[] is a temporary object, not a reference. (This means it's not an lvalue, so the container is read-only; that's OK.)
Now I'm writing an iterator class that can be used to traverse the tuples in this container. I'd like to model RandomAccessIterator, which depends on InputIterator, but InputIterator requires support for the expression i->m (where i is an iterator instance), and as far as I can tell, an operator-> function is required to return a pointer.
Naturally, I can't return a pointer to a temporary tuple that's constructed on-demand. One possibility that comes to mind is to put a tuple instance into the iterator as a member variable, and use it to store a copy of whichever value the iterator is currently positioned on:
class Iterator {
private:
MyContainer *container;
std::size_t current_index;
// Copy of (*container)[current_index]
std::tuple<int, int, int> current_value;
// ...
};
However, updating the stored value will require the iterator to check whether its current index is less than the container's size, so that a past-the-end iterator doesn't cause undefined behavior by accessing past the end of the underlying arrays. That adds (a small amount of) runtime overhead — not enough to make the solution impractical, of course, but it feels a little inelegant. The iterator shouldn't really need to store anything but a pointer to the container it's iterating and the current position within it.
Is there a clean, well-established way to support operator-> for iterator types that construct their values on-demand? How would other developers do this sort of thing?
(Note that I don't really need to support operator-> at all — I'm implementing the iterator mainly so that the container can be traversed with a C++11 "range for" loop, and std::tuple doesn't have any members that one would typically want to access via -> anyway. But I'd like to model the iterator concepts properly nonetheless; it feels like I'm cutting corners otherwise. Or should I just not bother?)
template<class T>
struct pseudo_ptr {
T t;
T operator*()&&{return t;}
T* operator->(){ return &t; }
};
then
struct bar { int x,y; };
struct bar_iterator:std::iterator< blah, blah >{
// ...
pseudo_ptr<bar> operator->() const { return {**this}; }
// ...
};
This relies on how -> works.
ptr->b for pointer ptr is simply (*ptr).b.
Otherwise it is defined as (ptr.operator->())->b. This evaluates recursively if operator-> does not return a pointer.
The pseudo_ptr<T> above gives you a wrapper around a copy of T.
Note, however, that lifetime extension doesn't really work. The result is fragile.
Here's an example relying on the fact that operator-> is applied repeatedly until a pointer is returned. We make Iterator::operator-> return the Contained object as a temporary. This causes the compiler to reapply operator->. We then make Contained::operator-> simply return a pointer to itself. Note that if we don't want to put operator-> in the Contained on-the-fly object, we can wrap it in a helper object that returns a pointer to the internal Contained object.
#include <cstddef>
#include <iostream>
class Contained {
public:
Contained(int a_, int b_) : a(a_), b(b_) {}
const Contained *operator->() {
return this;
}
const int a, b;
};
class MyContainer {
public:
class Iterator {
friend class MyContainer;
public:
friend bool operator!=(const Iterator &it1, const Iterator &it2) {
return it1.current_index != it2.current_index;
}
private:
Iterator(const MyContainer *c, std::size_t ind) : container(c), current_index(ind) {}
public:
Iterator &operator++() {
++current_index;
return *this;
}
// -> is reapplied, since this returns a non-pointer.
Contained operator->() {
return Contained(container->underlying_data_a[current_index], container->underlying_data_b[current_index]);
}
Contained operator*() {
return Contained(container->underlying_data_a[current_index], container->underlying_data_b[current_index]);
}
private:
const MyContainer *const container;
std::size_t current_index;
};
public:
MyContainer() {
for (int i = 0; i < 10; i++) {
underlying_data_a[i] = underlying_data_b[i] = i;
}
}
Iterator begin() const {
return Iterator(this, 0);
}
Iterator end() const {
return Iterator(this, 10);
}
private:
int underlying_data_a[10];
int underlying_data_b[10];
};
int
main() {
MyContainer c;
for (const auto &e : c) {
std::cout << e.a << ", " << e.b << std::endl;
}
}

Avoiding iterator invalidation using indices, maintaining clean interface

I have created a MemoryManager<T> class which is basically a wrapper around two vectors of pointers that manage lifetime of heap-allocated objects.
One vector stores the "alive" objects, the other one stores the object that will be added on next MemoryManager<T>::refresh.
This design was chosen to avoid iterator invalidation when looping over the MemoryManager<T>, as adding a new object directly to the MemoryManager<T>::alive vector can invalidate existing iterators (if it grows in size).
template<typename T> struct MemoryManager {
std::vector<std::unique_ptr<T>> alive;
std::vector<T*> toAdd;
T& create() {
auto r(new T);
toAdd.push_back(r);
return *r;
}
T& refresh() {
// Use erase-remove idiom on dead objects
eraseRemoveIf(alive, [](const std::unique_ptr<T>& p){ return p->alive; });
// Add all "toAdd" objects and clear the "toAdd" vector
for(auto i : toAdd) alive.emplace_back(i);
toAdd.clear();
}
void kill(T& mItem) { mItem.alive = false; }
IteratorType begin() { return alive.begin(); }
IteratorType end() { return alive.end(); }
}
I use it in my game engine to store entities, and update every "alive" entity every frame:
void game() {
MemoryManager<Entity> mm;
while(gameLoop) {
mm.refresh();
for(auto e : mm) processEntity(e);
auto& newEntity = mm.create();
// do something with newEntity
}
}
This has allowed me to constantly create/kill entities without having to worry about their lifetime too much.
However, I've recently come to the conclusion that using two std::vector is unnecessary. I could simply use a single vector and store an iterator to the "last alive object", adding the newly create objects immediately after the aforementioned iterator:
The idea, in my mind, works fine... but I cannot actually use a iterator type for end (as shown in the diagram), as it could get invalidated after the addition of some new elements to the vector. I've tested it, and this happens often, causing a crash.
The other solution I can think of is using an index instead of an iterator. This would solve the crashing, but I wouldn't be able to use the cool C++11 for(x : y) foreach loop because MemoryManager<T>::begin and MemoryManager<T>::end need to return an iterator.
Is there a way to achieve the current behavior with a single vector and still maintain a clear interface that can be used with C++11 for-each loops?
One of the simplest ways to get stable iterators (and references) is to use std::list<T>. And unless you are needing T to be a pointer to a polymorphic base class, it is better to use std::list<T>, as opposed to std::list<std::unique_ptr<T>>.
If on the other hand, your Entity is a polymorphic base, then consider using std::vector<std::unique_ptr<T>>. Although you can not depend upon iterators remaining valid, you can depend upon pointers and references to Entity remaining valid with std::vector<std::unique_ptr<T>>.
In your game() example, you never take advantage of stable iterators or pointers. You could just as easily (and more simply) do:
void game() {
std::vector<Entity> mm;
while(gameLoop) {
mm.erase(std::remove_if(mm.begin(), mm.end(), [](const Entity& e)
{ return e.alive; }),
mm.end());
for(auto e : mm) processEntity(e);
mm.push_back(create());
auto& newEntity = mm.back();
// do something with newEntity
}
}
During the processEntity loop, there is no way to invalidate iterators. If you did, you had better not use the range-based-for as the end iterator is only evaluated once, at the beginning of the loop.
But if you really do need stable iterators/references, substituting in std::list<Entity> would be very easy. I would change the erase/remove to use list's member remove_if instead. It will be more efficient.
If you do this, and performance testing (not guessing) indicates you've suffered a performance hit over your existing MemoryManager, you can optimize list by using a "stack allocator" such as the one demonstrated here:
http://howardhinnant.github.io/stack_alloc.html
This allows you to preallocate space (could be on the stack, could be on the heap), and have your container allocate from that. This will be both high performance and cache-friendly until the pre-allocated space is exhausted. And you've still got your iterator/pointer/reference stability.
In summary:
Find out / tell us if unique_ptr<Entity> is actually necessary because Entity is a base class. Prefer container<Entity> over container<unique_ptr<Entity>>.
Do you actually need iterator/pointer/reference stability? Your sample code does not. If you don't actually need it, don't pay for it. Use vector<Entity> (or vector<unique_ptr<Entity>> if you must).
If you actually need container<unique_ptr<Entity>>, can you get away with pointer/reference stability while sacrificing iterator stability? If yes, vector<unique_ptr<Entity>> is the way to go.
If you actually need iterator stability, strongly consider using std::list.
If you use std::list and discover via testing it has performance problems, optimize it with an allocator tuned to your needs.
If all of the above fails, then start designing your own data structure. If you get this far, know that this is the most difficult route, and everything will need to be backed up by both correctness and performance tests.
You can implement your own iterator class.
Something like the following may help.
template <typename T, typename... Ts>
class IndexIterator : public std::iterator<std::random_access_iterator_tag, T>
{
public:
IndexIterator(std::vector<T, Ts...>& v, std::size_t index) : v(&v), index(index) {}
// if needed.
typename std::vector<T, Ts...>::iterator getRegularIterator() const { return v->begin() + index; }
T& operator *() const { return v->at(index); }
T* operator ->() const { return &v->at(index); }
IndexIterator& operator ++() { ++index; return *this;}
IndexIterator& operator ++(int) { IndexIterator old(*this); ++*this; return old;}
IndexIterator& operator +=(std::ptrdiff_t offset) { index += offset; return *this;}
IndexIterator operator +(std::ptrdiff_t offset) const { IndexIterator res (*this); res += offset; return res;}
IndexIterator& operator --() { --index; return *this;}
IndexIterator& operator --(int) { IndexIterator old(*this); --*this; return old;}
IndexIterator& operator -=(std::ptrdiff_t offset) { index -= offset; return *this;}
IndexIterator operator -(std::ptrdiff_t offset) const { IndexIterator res (*this); res -= offset; return res;}
std::ptrdiff_t operator -(const IndexIterator& rhs) const { assert(v == rhs.v); return index - rhs.index; }
bool operator == (const IndexIterator& rhs) const { assert(v == rhs.v); return index == rhs.index; }
bool operator != (const IndexIterator& rhs) const { return !(*this == rhs); }
private:
std::vector<T, Ts...>* v;
std::size_t index;
};
template <typename T, typename... Ts>
IndexIterator<T, Ts...> IndexIteratorBegin(std::vector<T, Ts...>& v)
{
return IndexIterator<T, Ts...>(v, 0);
}
template <typename T, typename... Ts>
IndexIterator<T, Ts...> IndexIteratorEnd(std::vector<T, Ts...>& v)
{
return IndexIterator<T, Ts...>(v, v.size());
}
You could avoid moving elements of the container by maintaining a free-list (see http://www.memorymanagement.org/glossary/f.html#free.list).
To avoid invalidation of references to elements you can use a std::deque if you do not insert or erase in the middle.
To avoid invalidation of iterators you can use a std::list.
(Thanks Howard Hinnant)
You can implement your own iterator class which handles things the way you prefer. Then your begin() and end() can return instances of that class. For example, your custom iterator could store an integer index and a pointer to the vector itself, thereby making the iterator remain valid even in the face of reallocation.

In C++, how can I get a pointer into a vector?

I'm writing some C++ code that manipulates a bunch of vectors that are changing in size and are thus being reallocated constantly.
I would like to get a "pointer" into these vectors that remains valid even after reallocation of the vector. More specifically, I just want these "pointers" to remember which vector they point into and the index to which they point. When I dereference them using the standard (*ptr) syntax, I just want them to do the obvious lookup.
Obviously, actual pointers will not be valid after reallocation, and my understanding is that iterators aren't valid after reallocation either. Note also that I don't care if elements are inserted before my objects, so these "pointers" really don't have to remember anything but a vector and an index.
Now, I could easily write such a class myself. Has anyone (Boost? STL?) done it for me already?
Edit: The answers don't address my question. I asked if this functionality is any standard library. I take the responses as a "no"?
Try a std::pair< vector*, int>, as neither the position of the vector nor the index of the element changes.
Or, as a class:
template<class T> class VectorElementPointer
{
vector<T>& vectorref;
typename vector<T>::size_type index;
public:
VectorElementPointer(vector<T>& vref, typename vector<T>::size_type index):vectorref(vref),index(index){}
T& operator*() const {return vectorref[index];}
T* operator->() const {return &vectorref[index];}
};
This is the easiest solution that comes to my mind, as neither the STL nor Boost contains anything to do it easier.
An article on persistent iterators, complete with implementation.
To summarize some ideas. Here is the minimalist wrapper that tries to mimic iterators but stay valid as opposite to vector's ones.
void print(const std::string& i)
{
std::cout << "<" << i << "> ";
}
int main()
{
typedef std::vector<std::string> Vector;
Vector v;
v.push_back("H");
v.push_back("E");
v.push_back("W");
StrongIterator<Vector> it0(v, 0);
StrongIterator<Vector> it3(v, v.end());
std::for_each(it0.it(), it3.it(), print);
std::cout << std::endl;
v.push_back("O");
std::for_each(it0.it(), it3.it(), print);
std::cout << *it0;
std::cout << it0->c_str();
return 0;
}
And the iterator itself.
template <typename TVector>
class StrongIterator
{
public:
typedef typename TVector::iterator iterator;
typedef typename TVector::size_type size_type;
typedef typename TVector::value_type value_type;
StrongIterator(TVector& vector,
size_type index):
vector_(vector),
index_(index)
{}
StrongIterator(TVector& vector,
iterator it):
vector_(vector),
index_(std::distance(vector.begin(), it))
{}
iterator it()
{
iterator it = vector_.begin();
std::advance(it, index_);
return it;
}
value_type& operator*()
{
return vector_[index_];
}
value_type* operator->()
{
return &vector_[index_];
}
private:
TVector& vector_;
size_type index_;
};
Unfortunately, once you modify the vector, the iterators that would "point" to an element of the vector are no longer guaranteed to be valid. The only STL structure that I know of which will keep the iterators valid even as the structure is changing is the list<>. If you only want sequential iteration of your structures than you can use std::list<> otherwise I do not know of any other library that can help you; that doesn't mean there isn't one.
Here's some clear documentation on std::list : http://www.cplusplus.com/reference/stl/list/
Using boost::iterator_facade :
// Warning: Untested, not even compiled
template<class VectorT>
class VectorIndex :
public boost::iterator_facade<VectorIndex, typename VectorT::reference, boost::random_access_traversal_tag>
{
public:
VectorIndex(VectorT& Vec, typename VectorT::size_type Index)
: m_Vec(Vec), m_Index(Index)
{
}
private:
friend class boost::iterator_core_access;
void increment()
{
++m_Index;
}
void decrement()
{
--m_Index;
}
void advance(difference_type N)
{
m_Index += N;
}
difference_type distance_to(const VectorIndex& Other)
{
assert(&this->m_Vec == &Other.m_Vec);
return Other.m_Index = this->m_Index;
}
bool equal(const VectorIndex& Other)const
{
return (this->m_Vec == Other.m_Vec)
&& (this->m_Index == Other.m_Index);
}
VectorT::reference dereference() const
{
return m_Vec[m_Index];
}
VectorT m_Vec;
VectorT::size_type m_Index;
};
Unless you write your own version of vector and smart pointer there is no way that a pointer will be valid after a reallocation. If you had your own implementations the smart vector could send notifications to your smart pointers.
However I think that the whole scenario is a bad design and you might be better of redesigning your scenario so that you don't have a requirement like that.
Depending on your use pattern, a std::deque may fufil your requirements. Pointers into a deque are only invalidated if you insert or delete items not at the beginning or end - in pother words push_front() and push_back() don't invalidate pointers into the deque, but other changes do. You get basically the same interface as a vector, but of course the underlying storage is not contiguous.