std::vector<string> names;
std::vector<string>::iterator start = names.begin();
std::vector<string>::iterator end = names.end();
sort (start,end);
//are my start and end valid at this point?
//or they do not point to front and tail resp?
According to the C++ Standard §23.1/11:
Unless otherwise specified (either explicitly or by defining a function in terms of other functions), invoking
a container member function or passing a container as an argument to a library function shall not invalidate
iterators to, or change the values of, objects within that container.
§25.3 "Sorting and related operations" doesn't specify that iterators will be invalidated, so iterators in the question should stay valid.
They still point to the beginning and end. The values in those slots of the vector have probably changed, but the storage location in which each resides remains the same.
std::sort will not invalidate iterators to a vector. The sort template uses the * operator on the iterators to access and modify the contents of the vector, and modifying a vector element though an iterator to an element already in the vector will not invalidate any iterators.
In summary,
your existing iterators will not be invalidated
however, the elements they point to may have been modified
In addition to the support for the standard provided by Kirill V. Lyadvinsky (Does a vector sort invalidate iterators?):
25/5 "Algorithms library"
If an algorithm’s Effects section says
that a value pointed to by any
iterator passed as an argument is
modified, then that algorithm has an
additional type requirement: The type
of that argument shall satisfy the
requirements of a mutable iterator
(24.1).
24.1/4 "Iterator requirements"
Besides its category, a forward,
bidirectional, or random access
iterator can also be mutable or
constant depending on whether the
result of the expression *i behaves as
a reference or as a reference to a
constant.
std::vector keeps its elements in contiguous memory. std::sort takes arguments (iterators) by value and re-arranges the sequence between them. The net result is your local variables start and end are still pointing to first and one-past-the-last elements of the vector.
Related
If I have an iterator pointing to an element in an STL container, and I moved the element with the iterator, does the standard guarantee that the iterator is still valid? Can I use it with container's method, e.g. container::erase?
Also does it matter, if the container is a continuous one, e.g. vector, or non-continuous one, e.g. list?
std::list<std::string> l{"a", "b", "c"};
auto iter = l.begin();
auto s = std::move(*iter);
l.erase(iter); // <----- is it valid to erase it, whose underlying element has been removed?
Yes, you've modified the object in the container. You've not modified the container itself so the iterator is still valid
"Moving" an underlying element may not be the best name to use in this context. The name of this operation express the intention behind it but not how it really works.
In fact, the move operation is a form of copy operation with one difference: it is allowed to change the state of the "copied" object if it speeds up the execution. In case of the std::string this means that the internal buffer containing characters may be not deep-copied but just copied by address. The original object has to be then set to an empty state, to tell it to not use this buffer anymore. (Emptying the source string is not guaranteed. Optimizations of std::string are more complicated than I described.)
The important thing is that after the move operation, the original object is still there. It's just not guaranteed to have any specific state.
In this particular case you've done nothing to the iterator, but much rather to the object within it, so yes: The iterator remains valid.
But if you look at std::list::erase, it sports a line such as "References and iterators to the erased elements are invalidated. Other references and iterators are not affected."
So if you tried to do *iter after erase, it would cause your program to fail.
This may seem obvious for erase, but there are other operations where it is not as obvious.
For std::list for example, the reference page says:
Adding, removing and moving the elements within the list or across several lists does not invalidate the iterators or references. An iterator is invalidated only when the corresponding element is deleted.
For std::vector on the other hand, the reference for the push_back method says:
If the new size() is greater than capacity() then all iterators and references (including the past-the-end iterator) are invalidated. Otherwise only the past-the-end iterator is invalidated.
That means, unlike with std::list, it is not generally safe to keep an iterator to an element around, if the vector grows (because the underlying storage location of the item changes).
C++03 Standard § 23.2.4.3/3 describes std::vector::erase(iterator position) and says specifically
Invalidates all the iterators and references after the point of the erase.
Is the iterator at the point of the erase not invalidated? Specifically if I have a vector with a single element and I copy begin() iterator into a local variable and then call
vec.erase(vec.begin())
Will that iterator I have in a local variable get invalidated or not?
Will the iterators be invalidated after the point of erasure or after and including the point of erasure?
I'd say that your example with erasing the only element in the vector shows that the iterator at the insertion point must be invalidated.
Anyway, in C++11, the wording has been changed (23.3.6.5/3):
Effects: Invalidates iterators and references at or after the point of the erase.
The complexity of vector::erase() says: Linear on the number of elements erased (destructions) plus the number of elements after the last element deleted (moving).
It seems to me that it is implemented as a lazy grow/shrink array. The iterator, a pointer to the data, when erased, the following data will copied into its place. And thus the iterator you keep, will point to some data else, provided that the data you erase is not the last one.
In fact, it may depend on implementation. Yet I think a lazy grow/shrink array is the best-fit implementation for vector::erase(). [Since it may depends on implementation, don't count on anything like invalidate...]
std::unordered_map<K,T> offers methods like find which return a std::unordered_map<K,T>::iterator. As far as I know, an iterator remains valid unless rehashing occurrs.
But my suspicion is that there is no guaranteed way for roundtripping ::iterator -> T * -> ::iterator, similar to the "hack" that is done in the Linux linked list. I think that because the iterator is only required to be dereferenceable via ->second, but I don't see where it is required to have a durable storage type inside the container.
So, am I right? Am I required to keep the iterator or is T *ptr = &myiterator->second later back-castable to an iterator pointer whose ->second member's address is ptr?
This question is naturally also applicable to other container's iterators.
The first question is simple: in general, there is no direct way to get an iterator from a pointer to an element or from a value. That is, to get an iterator from a value you generally need to search the container. Since std::vector<T> is required to be contiguous you can get an iterator from a pointer using a simple computation (which requires that v is non-empty):
std::vector<T> v(...);
T* ptr = ...;
std::vector<T>::iterator it(v.begin() + (ptr - &v[0]));
To track down whether iterators and values stay stable, isn't entirely trivial as the relevant guarantees are distributed over multiple clauses, e.g.:
23.2.1 [container.requirements.general] paragraph 11:
Unless otherwise specified (either explicitly or by defining a function in terms of other functions), invoking a container member function or passing a container as an argument to a library function shall not invalidate iterators to, or change the values of, objects within that container.
23.2.4 [associative.reqmts] paragraph 9:
The insert and emplace members shall not affect the validity of iterators and references to the container, and the erase members shall invalidate only iterators and references to the erased elements.
23.2.5 [unord.req] paragraph 9:
... Rehashing invalidates iterators, changes ordering between elements, and changes which buckets elements appear in, but does not invalidate pointers or references to elements. ...
23.2.5 [unord.req] paragraph 14:
The insert and emplace members shall not affect the validity of references to container elements, but may invalidate all iterators to the container. ...
23.2.5 [unord.req] paragraph 15:
The insert and emplace members shall not affect the validity of iterators if (N+n) < z * B, where N is the number of elements in the container prior to the insert operation, n is the number of elements inserted, B is the container’s bucket count, and z is the container’s maximum load factor.
The above clauses should to be the important clauses with respect to iterator validity of the associative containers. The iterator validity in the unordered associative containers entirely depends on whether the container is rehashed. It seems the rehashing can be controlled to avoid it happening by surprise.
However, the entire idea of the unordered containers is that it is very efficient to locate an object using find(). It should never be necessary to store an iterator to an element because you can find them back. Of course, if you have a std::unordered_multimap<K, V> or a std::unordered_multiset<V> you may need to know which of the equivalent elements you are looking at.
Associative containers (set, multi_set, map, multi_map, and their unordered_ brethren) do not invalidate pointers or references when you add elements to the container; they can invalidate iterators. In addition, when you remove elements, only iterators, pointers, and references to those elements are invalidated. One consequence of this is that if you take the address of an element, that address remains valid as long as that element remains in the container.
There is no portable way to directly convert from the address of an element to an iterator that points to that element. If you need to do this you have to search for the element.
In an algorithm I'm currently implementing, I need to manipulate a std::list of struct T.
T holds a reference to another instance of T, but this reference can also be "unassigned".
At first, I wanted to use a pointer to hold this reference, but using an iterator instead makes it easier to remove from the list.
My question is : how to represent the equivalent to null pointer with my iterator?
I read general solution is to use myList.end(), but in my case, I need to test whether the iterator is "null" or not, and I may add or remove elements to the list between the moment when I store the iterator and the moment I remove it from list... Should I make the iterator point to a known list containing the "null" element? Or is there a more elegant solution?
According to this (emphasis by me):
Compared to the other base sequence
containers (vector and deque), lists
are the most efficient container doing
insertions at some position other than
the beginning or the end of the
sequence, and, unlike in these, all of
the previously obtained iterators and
references remain valid after the
insertion and refer to the same
elements they were referring before.
The same applies to erasure (with the obvious exception of iterators referring to a deleted element becoming invalidated). So yes, obtaining end() will always point to the same "invalid" element and should be safe to use.
§23.1.2.8 in the standard states that insertion/deletion operations on a set/map will not invalidate any iterators to those objects (except iterators pointing to a deleted element).
Now, consider the following situation: you want to implement a graph with uniquely numbered nodes, where every node has a fixed number (let's say 4) of neighbors. Taking advantage of the above rule, you do it like this:
class Node {
private:
// iterators to neighboring nodes
std::map<int, Node>::iterator neighbors[4];
friend class Graph;
};
class Graph {
private:
std::map<int, Node> nodes;
};
(EDIT: Not literally like this due to the incompleteness of Node in line 4 (see responses/comments), but along these lines anyway)
This is good, because this way you can insert and delete nodes without invalidating the consistency of the structure (assuming you keep track of deletions and remove the deleted iterator from every node's array).
But let's say you also want to be able to store an "invalid" or "nonexistent" neighbor value. Not to worry, we can just use nodes.end()... or can we? Is there some sort of guarantee that nodes.end() at 8 AM will be the same as nodes.end() at 10 PM after a zillion insertions/deletions? That is, can I safely == compare an iterator received as a parameter to nodes.end() in some method of Graph?
And if not, would this work?
class Graph {
private:
std::map<int, Node> nodes;
std::map<int, Node>::iterator _INVALID;
public:
Graph() { _INVALID = nodes.end(); }
};
That is, can I store nodes.end() in a variable upon construction, and then use this variable whenever I want to set a neighbor to invalid state, or to compare it against a parameter in a method? Or is it possible that somewhere down the line a valid iterator pointing to an existing object will compare equal to _INVALID?
And if this doesn't work either, what can I do to leave room for an invalid neighbor value?
You write (emphasis by me):
§23.1.2.8 in the standard states that insertion/deletion operations on a set/map will not invalidate any iterators to those objects (except iterators pointing to a deleted element).
Actually, the text of 23.1.2/8 is a bit different (again, emphasis by me):
The insert members shall not affect the validity of iterators and references to the container, and the erase members shall invalidate only iterators and references to the erased elements.
I read this as: If you have a map, and somehow obtain an iterator into this map (again: it doesn't say to an object in the map), this iterator will stay valid despite insertion and removal of elements. Assuming std::map<K,V>::end() obtains an "iterator into the map", it should not be invalidated by insertion/removal.
This, of course, leaves the question whether "not invalidated" means it will always have the same value. My personal assumption is that this is not specified. However, in order for the "not invalidated" phrase to make sense, all results of std::map<K,V>::end() for the same map must always compare equal even in the face of insertions/removal:
my_map_t::iterator old_end = my_map.end();
// wildly change my_map
assert( old_end == my_map.end() );
My interpretation is that, if old_end remains "valid" throughout changes to the map (as the standard promisses), then that assertion should pass.
Disclaimer: I am not a native speaker and have a very hard time digesting that dreaded legaleze of the Holy PDF. In fact, in general I avoid it like the plague.
Oh, and my first thought also was: The question is interesting from an academic POV, but why doesn't he simply store keys instead of iterators?
23.1/7 says that end() returns an iterator that
is the past-the-end value for the container.
First, it confirms that what end() returns is the iterator. Second, it says that the iterator doesn't point to a particular element. Since deletion can only invalidate iterators that point somewhere (to the element being deleted), deletions can't invalidate end().
Well, there's nothing preventing particular collection implementation from having end() depend on the instance of collection and time of day, as long as comparisons and such work. Which means, that, perhaps, end() value may change, but old_end == end() comparison should still yield true. (edit: although after reading the comment from j_random_hacker, I doubt this paragraph itself evaluates to true ;-), not universally — see the discussion below )
I also doubt you can use std::map<int,Node>::iterator in the Node class due to the type being incomplete, yet (not sure, though).
Also, since your nodes are uniquely numbered, you can use int for keying them and reserve some value for invalid.
Iterators in (multi)sets and (multi)maps won't be invalidated in insertions and deletions and thus comparing .end() against previous stored values of .end() will always yield true.
Take as an example GNU libstdc++ implementation where .end() in maps returns the default intialized value of Rb_tree_node
From stl_tree.h:
_M_initialize()
{
this->_M_header._M_color = _S_red;
this->_M_header._M_parent = 0;
this->_M_header._M_left = &this->_M_header;
this->_M_header._M_right = &this->_M_header;
}
Assuming that (1) map implemented with red-black tree (2) you use same instance "after a zillion insertions/deletions"- answer "Yes".
Relative implmentation I can tell that all incarnation of stl I ever know use the tree algorithm.
A couple points:
1) end() references an element that is past the end of the container. It doesn't change when inserts or deletes change the container because it's not pointing to an element.
2) I think perhaps your idea of storing an array of 4 iterators in the Node could be changed to make the entire problem make more sense. What you want is to add a new iterator type to the Graph object that is capable of iterating over a single node's neighbours. The implementation of this iterator will need to access the members of the map, which possibly leads you down the path of making the Graph class extend the map collection. With the Graph class being an extended std::map, then the language changes, and you no longer need to store an invalid iterator, but instead simply need to write the algorithm to determine who is the 'next neighbour' in the map.
I think it is clear:
end() returns an iterator to the element one past the end.
Insertion/Deletion do not affect existing iterators so the returned values are always valid (unless you try to delete the element one past the end (but that would result in undefined behavior anyway)).
Thus any new iterator generated by end() (would be different but) when compared with the original using operator== would return true.
Also any intermediate values generated using the assignment operator= have a post condition that they compare equal with operator== and operator== is transitive for iterators.
So yes, it is valid to store the iterator returned by end() (but only because of the guarantees with associative containers, therefor it would not be valid for vector etc).
Remember the iterator is not necessarily a pointer. It can potentially be an object where the designer of the container has defined all the operations on the class.
I believe that this depends entirely on what type of iterator is being used.
In a vector, end() is the one past the end pointer and it will obviously change as elements are inserted and removed.
In another kind of container, the end() iterator might be a special value like NULL or a default constructed element. In this case it doesn't change because it doesn't point at anything. Instead of being a pointer-like thing, end() is just a value to compare against.
I believe that set and map iterators are the second kind, but I don't know of anything that requires them to be implemented in that way.
C++ Standard states that iterators should stay valid. And it is. Standard clearly states that in 23.1.2/8:
The insert members shall not affect the validity of iterators and references to the container, and the erase members shall invalidate only iterators and references to the erased elements.
And in 21.1/7:
end() returns an iterator which is the past-the-end value for the container.
So iterators old_end and new_end will be valid. That means that we could get --old_end (call it it1) and --new_end (call it it2) and it will be the-end value iterators (from definition of what end() returns), since iterator of an associative container is of the bidirectional iterator category (according to 23.1.2/6) and according to definition of --r operation (Table 75).
Now it1 should be equal it2 since it gives the-end value, which is only one (23.1.2/9). Then from 24.1.3 follows that: The condition that a == b implies ++a == ++b. And ++it1 and ++it2 will give old_end and new_end iterators (from definition of ++r operation Table 74). Now we get that old_end and new_end should be equal.
I had a similar question recently, but I was wondering if calling end() to retrieve an iterator for comparison purposes could possibly have race conditions.
According to the standard, two iterators are considered equivalent if both can be dereferenced and &*a == &*b or if neither can be dereferenced. Finding the bolded statement took a while and is very relevant here.
Because an std::map::iterator cannot be invalidated unless the element it points to has been removed, you're guaranteed that two iterators returned by end, regardless of what the state of the map was when they were obtained, will always compare to each other as true.