I'm looking for a container that preserves it's contained objects' positions in memory (it's pointers remain valid)
The container will grow and shrink constantly. Elements in the middle may be erased, but there's no insertions in the middle; all elements are pushed onto the back of the container. Iterator validity isn't important in this case, my only concern is that the pointers remain valid.
Is std::deque a safe and efficient option in this situation? I was previously using list, but it is allocating far too many times to be useful in this instance.
No. std::deque is necessarily implemented using chunks. Erasing in the middle of a chunk would at the very least invalidate the addresses of all subsequent elements in that chunk.
Note that iterator invalidation and pointer invalidation are generally closely connected. An iterator often is a pointer to the (node holding the) element, with proper iteration semantics added. Such iterators get invalidated because the pointer they contain is invalidated.
Inserting or removing elements in the middle invalidates all iterators and references.
Inserting elements at the beginning/end invalidates iterators, but does not affect references.
Removing elements at the beginning/end does not affect iterators or references, except ones pointing to the removed element, and possibly the past-the-end iterator.
http://en.cppreference.com/w/cpp/container/deque/erase
All iterators and references are invalidated, unless the erased elements are at the end or the beginning of the container, in which case only the iterators and references to the erased elements are invalidated.
http://en.cppreference.com/w/cpp/container/deque/push_back
All iterators, including the past-the-end iterator, are invalidated. No references are invalidated.
(other methods that operate on front or back elements have similar notes).
NO ! As opposed to std::vector, the elements of a deque are not stored contiguously: typical implementations use a sequence of individually allocated fixed-size arrays.
The storage of a deque is automatically expanded and contracted as needed. Expansion of a deque is cheaper than the expansion of a std::vector because it does not involve copying of the existing elements to a new memory location
Related
I did some search on the implementation of deque. According to this post, deque uses vector of vectors. I know pushing at the begin and end should be both in constant time, and also random access is required. I think circular buffer meets all these requirements, and is much simpler. So why not use circular buffer?
I also found the boost circular buffer. How it compares to deque?
Edit:
Ok, so it has something to do with the Iterator Invalidation Rules.It states:
all iterators and references are invalidated, unless the inserted member is at an end (front or back) of the deque (in which case all iterators are invalidated, but references to elements are unaffected)
My understanding is to overload operator like iter++, the iterators has to own a pointer to the node map and a pointer to the chunk, so if the node map is reallocated, the iterator is invalidated. But since the data never moves, the references are still valid.
A circular buffer cannot be expanded indefinitely. Eventually, you need to allocate a new one and copy all items over.
Deque can simply allocate a new vector in front of the previous ones and add a "middle" element there. I think of it as a a list of vectors rather.
Is it possible that inserting and/or erasing elements can invalidate iterators to existing elements.
Thank you.
Yes. The documentation for boost::ptr_vector<T> states:
A ptr_vector<T> is a pointer container that uses an underlying std::vector<void*> to store the pointers.
And inserting elements into or erasing elements from a std::vector can cause re-allocation and therefore existing iterators to be invalidated.
Specifically, §23.3.6.5/3 of C++11 states about erase():
(3) Effects: Invalidates iterators and references at or after the point of the erase.
and about insert() and push_back():
(1) Remarks: Causes reallocation if the new size is greater than the old capacity. If no reallocation happens, all the iterators and references before the insertion point remain valid.
Therefore, to prevent iterator invalidation in the case of element insertion, you may use the reserve() function to increase the vector's capacity before obtaining any iterators from it. Those iterators will then remain valid until the size() of the vector exceeds the number of elements space was reserved for.
i erase elements from middle of my deque frequently. memory address of elements in deque are used in some other places. does erasing middle element from deque invalided all pointers to deque like it does with vectors? should i switch to list?(i iterate whole deque anyways)
i tried to read deque implementation but it is complex, i don't understand if it acts as a list or vector.
I'm using visualc++ std implementation of deque.
Yes. An insertion in the middle invalidates all iterators and references to elements (and thus pointers to elements). An insertion at either end invalidates all iterators, but not references or pointers. And you don't read the implementation to find out such things; you read the documentation. (The implementation may actually allow operations which aren't officially supported. Until the next bug fix.)
does erasing middle element from deque invalided all pointers to deque like it does with vectors?
In case of a deque all iterators and references to the deque are invalidated, unless the erased members are at an end (front or back) of the deque.
For a vector all iterators and references before the point of insertion are unaffected, unless the new container size is greater than the previous capacity.
I would like to track a pointer to a list element for the next read access. This pointer would be advanced every time the list is read. Would it be bad practice to cache an iterator to the list as a member variable, assuming I take proper precautions when deleting list elements?
An iterator to a list element remains valid until the element it refers to is removed. This is guaranteed by the standard.
There is no problem with caching an iterator as long as you make sure you don't remove elements from the list or refresh the cached iterator when you do.
Iterators are meant to be used. They aren't just for looping over each element in a for-loop. All you have to take into account is the rules for iterator invalidation.
std::vector iterators can be invalidated by any operation that inserts elements into the list. The iterators for all elements beyond the point of insertion are invalidated, and all iterators are invalidated if the insertion operation causes an increase in capacity. An operation that removes an element from the vector will invalidate any iterator after the point of removal.
std::deque iterators are invalidated by any operation that adds or removes elements from anywhere in the deque. So it's probably not a good idea to keep these around very long.
std::list, std::set, and std::map iterators are only invalidated by the specific removal of the particular element that the iterator refers to. These are the longest-lived of the iterator types.
As long as you keep these rules in mind, feel free to store these iterators all you want. It certainly isn't bad form to store std::list iterators, as long as you can be sure that that particular element isn't going anywhere.
The only way you're going to be able to properly advance though an STL std::list<T> in a platform and compiler independent way is through the use of a std::list<T>::iterator or std::list<T>::const_iterator, so that's really your only option unless you're planning on implementing your own linked-list. Per the standard, as others here have posted, an iterator to a std::list element will remain valid until that element is removed from the list.
Let's say I have some kind of collection and I obtained an iterator for the beginning of it. Now let's say I modified the collection. Can I still use the iterator safely, regardless of the type of the collection or the iterator?
To avoid confusion, here is the order of operations I talk about:
Get an iterator of the collection.
Modify the collection (obviously
not an element in it, but the collection itself).
Use the iterator obtained at step 1. Is it stil valid according to STL standard?!
Depends on the container. e.g. if it's a vector, after modifying the container all iterators can be invalidated. However, if it's a list, the iterators irrelevant to the modified place will remain valid.
A vector's iterators are invalidated when its memory is reallocated. Additionally, inserting or deleting an element in the middle of a vector invalidates all iterators that point to elements following the insertion or deletion point. It follows that you can prevent a vector's iterators from being invalidated if you use reserve() to preallocate as much memory as the vector will ever use, and if all insertions and deletions are at the vector's end. [1]
The semantics of iterator invalidation for deque is as follows. Insert (including push_front and push_back) invalidates all iterators that refer to a deque. Erase in the middle of a deque invalidates all iterators that refer to the deque. Erase at the beginning or end of a deque (including pop_front and pop_back) invalidates an iterator only if it points to the erased element. [2]
Lists have the important property that insertion and splicing do not invalidate iterators to list elements, and that even removal invalidates only the iterators that point to the elements that are removed. [3]
Map has the important property that inserting a new element into a map does not invalidate iterators that point to existing elements. Erasing an element from a map also does not invalidate any iterators, except, of course, for iterators that actually point to the element that is being erased. [4] (same for set, multiset and multimap)
That depends on the collection in question. Just for example, modifying a std::vector (e.g., adding an element somewhere) can invalidate all iterators into that vector. By contrast, with a std::list, iterators remain valid when you add another element to the list. In some cases, the rules are even more complex (e.g., if memory serves, with a std::deque, adding to the beginning or end leaves existing iterators valid, but adding anywhere else can invalidate them -- but my memory is sufficiently poor that you should check before depending on that).
No, iterators are only good while the iterated container is unchanged. If a collection is modified, the iterator should be obtained anew.