Does moving a vector invalidate iterators? - c++

If I have an iterator into vector a, then I move-construct or move-assign vector b from a, does that iterator still point to the same element (now in vector b)? Here's what I mean in code:
#include <vector>
#include <iostream>
int main(int argc, char *argv[])
{
std::vector<int>::iterator a_iter;
std::vector<int> b;
{
std::vector<int> a{1, 2, 3, 4, 5};
a_iter = a.begin() + 2;
b = std::move(a);
}
std::cout << *a_iter << std::endl; // Is a_iter valid here?
return 0;
}
Is a_iter still valid since a has been moved into b, or is the iterator invalidated by the move? For reference, std::vector::swap does not invalidate iterators.

While it might be reasonable to assume that iterators are still valid after a move, I don't think the Standard actually guarantees this. Therefore, the iterators are in an undefined state after the move.
There is no reference I can find in the Standard which specifically states that iterators that existed before a move are still valid after the move.
On the surface, it would seem to be perfectly reasonable to assume that an iterator is typically implemented as pointers in to the controlled sequence. If that's the case, then the iterators would still be valid after the move.
But the implementation of an iterator is implementation-defined. Meaning, so long as the iterator on a particular platform meets the requirements set forth by the Standard, it can be implemented in any way whatsoever. It could, in theory, be implemented as a combination of a pointer back to the vector class along with an index. If that's the case, then the iterators would become invalid after the move.
Whether or not an iterator is actually implemented this way is irrelevant. It could be implemented this way, so without a specific guarantee from the Standard that post-move iterators are still valid, you cannot assume that they are. Bear in mind also that there is such a guarantee for iterators after a swap. This was specifically clarified from the previous Standard. Perhaps it was simply an oversight of the Std comittee to not make a similar clarification for iterators after a move, but in any case there is no such guarantee.
Therefore, the long and the short of it is you can't assume your iterators are still good after a move.
EDIT:
23.2.1/11 in Draft n3242 states that:
Unless otherwise specified (either explicitly or by defining a
function in terms of other functions), invoking a container member
function or passing a container as an argument to a library function
shall not invalidate iterators to, or change the values of, objects
within that container.
This might lead one to conclude that the iterators are valid after a move, but I disagree. In your example code, a_iter was an iterator in to the vector a. After the move, that container, a has certainly been changed. My conclusion is the above clause does not apply in this case.

I think the edit that changed move construction to move assignment changes the answer.
At least if I'm reading table 96 correctly, the complexity for move construction is given as "note B", which is constant complexity for anything except std::array. The complexity for move assignment, however, is given as linear.
As such, the move construction has essentially no choice but to copy the pointer from the source, in which case it's hard to see how the iterators could become invalid.
For move assignment, however, the linear complexity means it could choose to move individual elements from the source to the destination, in which case the iterators will almost certainly become invalid.
The possibility of move assignment of elements is reinforced by the description: "All existing elements of a are either move assigned to or destroyed". The "destroyed" part would correspond to destroying the existing contents, and "stealing" the pointer from the source -- but the "move assigned to" would indicate moving individual elements from source to destination instead.

tl;dr : Yes, moving a std::vector<T, A> possibly invalidates the iterators
The common case (with std::allocator in place) is that invalidation does not happen but there is no guarantee and switching compilers or even the next compiler update might make your code behave incorrect if you rely on the fact the your implementation currently does not invalidate the iterators.
On move assignment:
The question whether std::vector iterators can actually remain valid after move-assignment is connected with the allocator awareness of the vector template and depends on the allocator type (and possibly the respective instances thereof).
In every implementation I have seen, move-assignment of a std::vector<T, std::allocator<T>>1 will not actually invalidate iterators or pointers. There is a problem however, when it comes down to making use of this, as the standard just cannot guarantee that iterators remain valid for any move-assignment of a std::vector instance in general, because the container is allocator aware.
Custom allocators may have state and if they do not propagate on move assignment and do not compare equal, the vector must allocate storage for the moved elements using its own allocator.
Let:
std::vector<T, A> a{/*...*/};
std::vector<T, A> b;
b = std::move(a);
Now if
std::allocator_traits<A>::propagate_on_container_move_assignment::value == false &&
std::allocator_traits<A>::is_always_equal::value == false && (possibly as of c++17)
a.get_allocator() != b.get_allocator()
then b will allocate new storage and move elements of a one by one into that storage, thus invalidating all iterators, pointers and references.
The reason is that fulfillment of above condition 1. forbidds move assignment of the allocator on container move. Therefore, we have to deal with two different instances of the allocator. If those two allocator objects now neither always compare equal (2.) nor actually compare equal, then both allocators have a different state. An allocator x may not be able to deallocate memory of another allocator y having a different state and therefore a container with allocator x cannot just steal memory from a container which allocated its memory via y.
If the allocator propagates on move assignment or if both allocators compare equal, then an implementation will very likely choose to just make b own as data because it can be sure to be able to deallocate the storage properly.
1: std::allocator_traits<std::allocator<T>>::propagate_on_container_move_assignment and std::allocator_traits<std::allocator<T>>::is_always_equal both are typdefs for std::true_type (for any non-specialized std::allocator).
On move construction:
std::vector<T, A> a{/*...*/};
std::vector<T, A> b(std::move(a));
The move constructor of an allocator aware container will move-construct its allocator instance from the allocator instance of the container which the current expression is moving from. Thus, the proper deallocation capability is ensured and the memory can (and in fact will) be stolen because move construction is (except for std::array) bound to have constant complexity.
Note: There is still no guarantee for iterators to remain valid even for move construction.
On swap:
Demanding the iterators of two vectors to remain valid after a swap (now just pointing into the respective swapped container) is easy because swapping only has defined behaviour if
std::allocator_traits<A>::propagate_on_container_swap::value == true ||
a.get_allocator() == b.get_allocator()
Thus, if the allocators do not propagate on swap and if they do not compare equal, swapping the containers is undefined behaviour in the first place.

Since there's nothing to keep an iterator from keeping a reference or pointer to the original container, I'd say you can't rely on the iterators remaining valid unless you find an explicit guarantee in the standard.

Related

Are references / pointers guaranteed to be valid after moving std::deque?

Is it safe to assume that any pointers I have to elements inside of an std::deque are still valid after moving the deque to another one with the move constructor?
For std::vector I cannot see any reason why they wouldn't be, but I'm not familiar enough with std::deque to be sure I can make the same assumption.
Pointers to elements would remain valid. After move construction:
After container move construction (overload (8)), references, pointers, and iterators (other than the end iterator) to other remain valid, but refer to elements that are now in *this. The current standard makes this guarantee via the blanket statement in [container.requirements.general]/12, and a more direct guarantee is under consideration via LWG 2321.

Does the spec specify what happens to pointers that point into std::move()'d containers? [duplicate]

If I have an iterator into vector a, then I move-construct or move-assign vector b from a, does that iterator still point to the same element (now in vector b)? Here's what I mean in code:
#include <vector>
#include <iostream>
int main(int argc, char *argv[])
{
std::vector<int>::iterator a_iter;
std::vector<int> b;
{
std::vector<int> a{1, 2, 3, 4, 5};
a_iter = a.begin() + 2;
b = std::move(a);
}
std::cout << *a_iter << std::endl; // Is a_iter valid here?
return 0;
}
Is a_iter still valid since a has been moved into b, or is the iterator invalidated by the move? For reference, std::vector::swap does not invalidate iterators.
While it might be reasonable to assume that iterators are still valid after a move, I don't think the Standard actually guarantees this. Therefore, the iterators are in an undefined state after the move.
There is no reference I can find in the Standard which specifically states that iterators that existed before a move are still valid after the move.
On the surface, it would seem to be perfectly reasonable to assume that an iterator is typically implemented as pointers in to the controlled sequence. If that's the case, then the iterators would still be valid after the move.
But the implementation of an iterator is implementation-defined. Meaning, so long as the iterator on a particular platform meets the requirements set forth by the Standard, it can be implemented in any way whatsoever. It could, in theory, be implemented as a combination of a pointer back to the vector class along with an index. If that's the case, then the iterators would become invalid after the move.
Whether or not an iterator is actually implemented this way is irrelevant. It could be implemented this way, so without a specific guarantee from the Standard that post-move iterators are still valid, you cannot assume that they are. Bear in mind also that there is such a guarantee for iterators after a swap. This was specifically clarified from the previous Standard. Perhaps it was simply an oversight of the Std comittee to not make a similar clarification for iterators after a move, but in any case there is no such guarantee.
Therefore, the long and the short of it is you can't assume your iterators are still good after a move.
EDIT:
23.2.1/11 in Draft n3242 states that:
Unless otherwise specified (either explicitly or by defining a
function in terms of other functions), invoking a container member
function or passing a container as an argument to a library function
shall not invalidate iterators to, or change the values of, objects
within that container.
This might lead one to conclude that the iterators are valid after a move, but I disagree. In your example code, a_iter was an iterator in to the vector a. After the move, that container, a has certainly been changed. My conclusion is the above clause does not apply in this case.
I think the edit that changed move construction to move assignment changes the answer.
At least if I'm reading table 96 correctly, the complexity for move construction is given as "note B", which is constant complexity for anything except std::array. The complexity for move assignment, however, is given as linear.
As such, the move construction has essentially no choice but to copy the pointer from the source, in which case it's hard to see how the iterators could become invalid.
For move assignment, however, the linear complexity means it could choose to move individual elements from the source to the destination, in which case the iterators will almost certainly become invalid.
The possibility of move assignment of elements is reinforced by the description: "All existing elements of a are either move assigned to or destroyed". The "destroyed" part would correspond to destroying the existing contents, and "stealing" the pointer from the source -- but the "move assigned to" would indicate moving individual elements from source to destination instead.
tl;dr : Yes, moving a std::vector<T, A> possibly invalidates the iterators
The common case (with std::allocator in place) is that invalidation does not happen but there is no guarantee and switching compilers or even the next compiler update might make your code behave incorrect if you rely on the fact the your implementation currently does not invalidate the iterators.
On move assignment:
The question whether std::vector iterators can actually remain valid after move-assignment is connected with the allocator awareness of the vector template and depends on the allocator type (and possibly the respective instances thereof).
In every implementation I have seen, move-assignment of a std::vector<T, std::allocator<T>>1 will not actually invalidate iterators or pointers. There is a problem however, when it comes down to making use of this, as the standard just cannot guarantee that iterators remain valid for any move-assignment of a std::vector instance in general, because the container is allocator aware.
Custom allocators may have state and if they do not propagate on move assignment and do not compare equal, the vector must allocate storage for the moved elements using its own allocator.
Let:
std::vector<T, A> a{/*...*/};
std::vector<T, A> b;
b = std::move(a);
Now if
std::allocator_traits<A>::propagate_on_container_move_assignment::value == false &&
std::allocator_traits<A>::is_always_equal::value == false && (possibly as of c++17)
a.get_allocator() != b.get_allocator()
then b will allocate new storage and move elements of a one by one into that storage, thus invalidating all iterators, pointers and references.
The reason is that fulfillment of above condition 1. forbidds move assignment of the allocator on container move. Therefore, we have to deal with two different instances of the allocator. If those two allocator objects now neither always compare equal (2.) nor actually compare equal, then both allocators have a different state. An allocator x may not be able to deallocate memory of another allocator y having a different state and therefore a container with allocator x cannot just steal memory from a container which allocated its memory via y.
If the allocator propagates on move assignment or if both allocators compare equal, then an implementation will very likely choose to just make b own as data because it can be sure to be able to deallocate the storage properly.
1: std::allocator_traits<std::allocator<T>>::propagate_on_container_move_assignment and std::allocator_traits<std::allocator<T>>::is_always_equal both are typdefs for std::true_type (for any non-specialized std::allocator).
On move construction:
std::vector<T, A> a{/*...*/};
std::vector<T, A> b(std::move(a));
The move constructor of an allocator aware container will move-construct its allocator instance from the allocator instance of the container which the current expression is moving from. Thus, the proper deallocation capability is ensured and the memory can (and in fact will) be stolen because move construction is (except for std::array) bound to have constant complexity.
Note: There is still no guarantee for iterators to remain valid even for move construction.
On swap:
Demanding the iterators of two vectors to remain valid after a swap (now just pointing into the respective swapped container) is easy because swapping only has defined behaviour if
std::allocator_traits<A>::propagate_on_container_swap::value == true ||
a.get_allocator() == b.get_allocator()
Thus, if the allocators do not propagate on swap and if they do not compare equal, swapping the containers is undefined behaviour in the first place.
Since there's nothing to keep an iterator from keeping a reference or pointer to the original container, I'd say you can't rely on the iterators remaining valid unless you find an explicit guarantee in the standard.

Using the address of an item in stable vector

items in std::vector are dynamically allocated and their addresses may change when a reallocation happens. So, it is not possible to depend on their addresses because it is not stable.
On the other hand, if I have a std::vector which contains some items and I do not have any intention to change anything about it during its life cycle, is it valid (well-defined) to use the addresses of its items?
Example:
std::vector<foo> foos;
foos.reserve(100);
for(size_t i=0;i<100;++i){
foos.emplace_back(make_random_foo());
}
//From now no one can touch foos
auto ptr_to_the_fifth_foo=&foos[4];
In other words, does the standard guarantee that noting will affect the vector items addresses since I did not do that by my self?
If no member function of the std::vector is called, the vector may not be changed at all and as such the contents remain the same and all pointers stay valid.
In your example you call operator[](size_type n) which is defined in the standard as being equivalent to *(a.begin() + n).
A std::vector is a container and therefore, the container requirements hold which state:
Unless otherwise specified (either explicitly or by defining a function in terms of other functions), invoking a container member function or passing a container as an argument to a library function shall not invalidate iterators to, or change the values of, objects within that container.
Since begin() is not specified to invalidate any iterators to the container, operator[] won't either.
Yes.
Pointers and references to elements are only invalidated when their iterator is invalidated.
Iterators are invalidated when capacity has to grow (when size passes capacity), or when you insert/remove elements before that element in the vector. They can also be invalidated when a container is moved-to or moved-from, but it might not occur.
I believe swap is specified to not invalidate iterators, but rather make them refer to the "new home" as it where (and hence, the pointers/references to the "new home" in the "new vector") (ie, the buffer ownership changes). Move-assign does not make this promise. I do not remember off the top of my head if move-construct does.

Move-assignment slower than copy-assignment -- bug, feature, or unspecified?

I recently realized that the addition of move semantics in C++11 (or at least my implementation of it, Visual C++) has actively (and quite dramatically) broken one of my optimizations.
Consider the following code:
#include <vector>
int main()
{
typedef std::vector<std::vector<int> > LookupTable;
LookupTable values(100); // make a new table
values[0].push_back(1); // populate some entries
// Now clear the table but keep its buffers allocated for later use
values = LookupTable(values.size());
return values[0].capacity();
}
I followed this kind of pattern to perform container recycling: I would re-use the same container instead of destroying and recreating it, to avoid unnecessary heap deallocation and (immediate) reallocation.
On C++03, this worked fine -- that means this code used to return 1, because the vectors were copied elementwise, while their underlying buffers were kept as-is. Consequently I could modify each inner vector knowing that it could use the same buffer as it had before.
On C++11, however, I noticed that this results in a move of the right-hand side onto the left-hand side, which performs an element-wise move-assignment to each vector on the left-hand side. This in turn causes the vector to discard its old buffer, suddenly reducing its capacity to zero. Consequently, my application now slows down considerably due to excess heap allocations/deallocations.
My question is: is this behavior a bug, or is it intentional? Is it even specified by the standard at all?
Update:
I just realized that correctness of this particular behavior may depend on whether or not a = A() can invalidate iterators that point to the elements of a. However, I don't know what the iterator invalidation rules for move-assignment are, so if you're aware of them it may be worth mentioning those in your answer.
C++11
The difference in the behaviours in the OP between C++03 and C++11 are due to the way move assignment is implemented.
There are two main options:
Destroy all elements of the LHS. Deallocate the LHS's underlying storage. Move the underlying buffer (the pointers) from the RHS to the LHS.
Move-assign from the elements of the RHS to the elements of the LHS. Destroy any excess elements of the LHS or move-construct new elements in the LHS if the RHS has more.
I think it is possible to use option 2 with copies, if moving is not noexcept.
Option 1 invalidates all references/pointers/iterators to the LHS, and preserves all iterators etc. of the RHS. It needs O(LHS.size()) destructions, but the buffer movement itself is O(1).
Option 2 invalidates only iterators to excess elements of the LHS which are destroyed, or all iterators if a reallocation of the LHS occurs. It is O(LHS.size() + RHS.size()) since all elements of both sides need to be taken care of (copied or destroyed).
As far as I can tell, there is no guarantee which one happens in C++11 (see next section).
In theory, you can use option 1 whenever you can deallocate the underlying buffer with the allocator that is stored in the LHS after the operation. This can be achieved in two ways:
If two allocators compare equal, one can be used to deallocate the storage allocated via the other one. Therefore, if the allocators of LHS and RHS compare equal before the move, you can use option 1. This is a run-time decision.
If the allocator can be propagated (moved or copied) from the RHS to the LHS, this new allocator in the LHS can be used to deallocate the storage of the RHS. Whether or not an allocator is propagated is determined by allocator_traits<your_allocator :: propagate_on_container_move_assignment. This is decided by type properties, i.e. a compile-time decision.
C++11 minus defects / C++1y
After LWG 2321 (which is still open), we have the guarantee that:
no move constructor (or move assignment operator when
allocator_traits<allocator_type> :: propagate_on_container_move_assignment :: value is true) of a container (except for array) invalidates any references,
pointers, or iterators referring to the elements of the source
container. [ Note: The end() iterator does not refer to any element, so
it may be invalidated. — end note ]
This requires that move-assignment for those allocators which are propagated on move assignment has to move the pointers of the vector object, but must not move the elements of the vector. (option 1)
The default allocator, after LWG defect 2103, is propagated during move-assignment of the container, hence the trick in the OP is forbidden to move the individual elements.
My question is: is this behavior a bug, or is it intentional? Is it even specified by the standard at all?
No, yes, no (arguably).
See this answer for a detailed description of how vector move assignment must work. As you are using the std::allocator, C++11 puts you into case 2, which many on the committee considered a defect, and has been corrected to case 1 for C++14.
Both case 1 and case 2 have identical run time behavior, but case 2 has additional compile-time requirements on the vector::value_type. Both case 1 and case 2 result in memory ownership being transferred from the rhs to the lhs during the move assignment, giving you the results you observe.
This is not a bug. It is intentional. It is specified by C++11 and forward. Yes, there are some minor defects as dyp pointed out in his answer. But none of these defects will change the behavior you are seeing.
As was pointed out in the comments, the simplest fix for you is to create an as_lvalue helper and use that:
template <class T>
constexpr
inline
T const&
as_lvalue(T&& t)
{
return t;
}
// ...
// Now clear the table but keep its buffers allocated for later use
values = as_lvalue(LookupTable(values.size()));
This is zero-cost, and returns you to precisely the C++03 behavior. It might not pass code review though. It would be clearer for you to iterate through and clear each element in the outer vector:
// Now clear the table but keep its buffers allocated for later use
for (auto& v : values)
v.clear();
The latter is what I recommend. The former is (imho) obfuscated.

Should std::vector::swap() with stateful allocators invalidate all iterators?

Given allocators a1 and a2, where a1 != a2,
and std::vectors v1(a1) and v2(a2)
then v1.swap(v2) invalidates all iterators.
Is this expected behavior?
In general, swap never invalidates iterators. However, another rule comes into play when the allocators are different. In that case the behavior depends on allocator_traits<a1>::propagate_on_container_swap::value and allocator_traits<a2>::propagate_on_container_swap::value. If both are true, the allocators are exchanged along with the data, all iterators remain valid. If either is false, behavior is undefined, so the particular behavior exhibited by VC++ 2010 is allowed.
From [container.requirements.general] (wording from n3290):
Allocator replacement is performed by copy assignment, move assignment, or swapping of the allocator only if allocator_traits<allocatortype>::propagate_on_container_copy_assignment::value,
allocator_traits<allocatortype>::propagate_on_container_move_assignment::value, or allocator_traits<allocatortype>::propagate_on_container_swap::value is true within the implementation of the corresponding container operation. The behavior of a call to a container’s swap function is undefined unless the objects being swapped have allocators that compare equal or allocator_traits<allocatortype>::propagate_on_container_swap::value is true.
and
Every iterator referring to an element in one container before the swap shall refer to the same element in the other container after the swap
and
Unless otherwise specified ... no swap() function invalidates any references, pointers, or iterators referring to the elements of the containers being swapped.
23.3.6.5 does not specify alternate rules for vector::swap().