Suppose I have a vector of ints,
std::vector<int> numbers;
that is populated with a bunch of values, then I say do this (where an entry exists at 43)
int *oneNumber = &numbers[43];
Is oneNumber guaranteed to always be pointing at the int at index 43, even if say I resize numbers to something like numbers.resize(46)?
I'm not 100% sure what expected behaviour is here, I know vectors are guaranteed to be contiguous but not sure if that continuity would also mean all the indices in the vector will remain in the same place throughout its life.
Is oneNumber guaranteed to always be pointing at the int at index 43
Yes, this is guaranteed by the standard.
even if say I resize numbers to something like numbers.resize(46)?
No. Once you resize, add, or remove anything to the vector, all addresses and iterators to it are invalidated. This is because the vector may need to be reallocated with new memory locations.
Your paranoia is correct. Resizing a std::vector can cause its memory location to change. That means your oneNumber is now pointing to an old memory location that has been freed, and so accessing it is undefined behavior.
Pointers, references, and iterators to std::vector elements are guaranteed to stay put as long as you only append to the std::vector and the size of the std::vector doesn't grow beyond its capacity() at the time the pointer, reference, or iterator was obtained. Once it gets resized beyond the capacity() all pointers, references, and iterators to this std::vector become invalidated. Note that things are invalidated as well when inserting somewhere else than the end of the std::vector.
If you want to have your objects stay put and you only insert new elements at the end or the beginning, you can use std::deque. Pointers and references to elements in the std::deque get only invalidated when you insert into the middle of the std::deque or when removing from the middle or when removing the referenced object. Note that iterators to elements in the std::deque get invalidated every time you insert an element into the std::deque or remove any element from it.
As all the others have said, when you call .resize() on a vector your pointers become invalidated because the (old array) may be completely deallocated, and an entirely new one may be re-allocated and your data copied into it.
One workaround for this is don't store pointers into an STL vector. Instead, store integer indices.
So in your example,
std::vector<int> numbers;
int *oneNumber = &numbers[43]; // no. pointers invalidated after .resize or possibly .push_back.
int oneNumberIndex = 43 ; // yes. indices remain valid through .resize/.push_back
No - the vector can be reallocated when it grows. Usually once the vector doubles in size.
From the C++11 standard
1 Remarks: Causes reallocation if the new size is greater than the old capacity. If no
reallocation happens, all the iterators and references before the insertion point
remain valid. If an exception is thrown other than by the copy constructor, move
constructor, assignment operator, or move assignment operator of T or by any
InputIterator operation there are no effects. If an exception is thrown by the move
constructor of a non-CopyInsertable T, the effects are unspecified.
When you use a vector's resize() or reserve() function to increase the capacity of the vector, it may need to reallocate memory for the array-backing. If it does reallocate, the new memory will not be located at the same address, so the address stored in oneNumber will no longer point to the right place.
Again, this depends on how many elements the vector is currently being used to store and the requested size. Depending on the specifics, the vector may be able to resize without reallocating, but you should definitely not assume that this will be the case.
Once you changed the capacity of the vector, the data was copied to another memory block, and the origin data is deleted.
Related
I have a 2D vector declared as such:
vector<vector<uint16_t>> vector;
After this is used and I need to clear all memory associated with it, even the elements in the [][X] dimension, can I simply do:
vector.clear();
Or do I have to go through each of the vectors inside and call .clear() on them, before finally calling it on the main vector? If that is the case, what would be a clean solution for that?
Since a vector neatly destructs its elements and deallocates the used memory when itself gets destructed, and v.clear() also destructs the elements in the vector, calling v.clear() is a perfectly valid way to delete an entire 2D vector.
Note that if you want also the memory for v itself cleaned up you need to call v.shrink_to_fit() after v.clear().
With a 2D vector, does calling clear() on the 1st level clear all the memory associated
with that vector?
Yes, clear erases all elements from the container.
Quoting from here,
std::vector<T,Allocator>::clear
After this call, size() returns zero.
Invalidates any references, pointers, or iterators referring to contained elements. Any
past-the-end iterators are also invalidated.
But this does not affect allocation/deallocation of internal storage in any way whatsoever. So, if you meant to ask,
With a 2D vector, does calling clear() on the 1st level FREE all the memory associated
with that vector?
No. As per standard,
Leaves the capacity() of the vector unchanged (note: the standard's restriction on the
changes to capacity is in the specification of vector::reserve, see 1)
So, if you wish to free up memory, either go for shrink_to_fit() or better swap contents with an empty vector. This is what most but not all implementations do.
v.swap(std::vector<T>());
v.swap(std::vector<std::vector<T>>()); // if vector of vectors
When vector needs more memory it will reallocate memory somewhere, I don't know where yet! and then pointers become invalid, is there any good explanation on this?
I mean where they go, what happen to my containers? ( not linked list ones )
Short answer: Everything will be fine. Don't worry about this and get back to work.
Medium answer: Adding elements to or removing them from a vector invalidates all iterators and references/pointers (possibly with the exception of removing from the back). Simple as that. Don't refer to any old iterators and obtain new ones after such an operation. Example:
std::vector<int> v = get_vector();
int & a = v[6];
int * b = &v[7];
std::vector<int>::iterator c = v.begin();
std::advance(it, 8);
v.resize(100);
Now a, b and c are all invalid: You cannot use a, and you cannot dereference b or c.
Long answer: The vector keeps track of dynamic memory. When the memory is exhausted, it allocates a new, larger chunk elsewhere and copies (or moves) all the old elements over (and then frees up the old memory, destroying the old objects). Memory allocation and deallocation is done by the allocator (typically std::allocator<T>), which in turn usually invokes ::operator new() to fetch memory, which in turn usually calls malloc(). Details may vary and depend on your platform. In any event, any previously held references, pointers or iterators are no longer valid (presumably because they refer to the now-freed memory, though it's not specified in the standard why they're invalid).
When you add or remove items from a vector, all iterators (and pointers) to items within it are invalidated. If you need to store a pointer to an item in a vector, then make the vector const, or use a different container.
It shouldn't matter to you where the vector stores things. You don't need to do anything, just let it do its job.
When you use std::vector, the class takes care of all the details regarding memory allocation, pointers, resizing and so on.
The vector class exposes its contents through iterators and references. Mutations of the vector will potentially invalidate iterators and references because reallocation may be necessary.
It is valid to access the contents using pointers because the vector class guarantees to store its elements at contiguous memory locations. Clearly any mutation of the list will potentially invalidate any pointers to its contents, because of potential reallocation. Therefore, if you ever access an element using pointers, you must regard those pointers as invalid once you mutate the vector. In short the same rules apply to pointers to the contents as do to references.
If you want to maintain a reference to an item in the vector, and have this reference be valid even after mutation, then you should remember the index rather than a pointer or reference to the item. In that case it is perfectly safe to add to the end of the vector and your index value still refers to the same element.
There are some confusion for me in using deque container.
I compared a vector with a deque, I entered Integer values dynamically and observed that after a few insertions vector's objects start moving around and addresses have been changed which seemed logical. However deque's objects stayed on the same place in the memory even after I entered a few hundred integers.
This observation gives me the idea that deque reserves a much larger memory than vector but if it is true then what is the point of having dynamic memory instead of static? Even if it does, it will run out of memory at somewhere and need to change the place on the memory, So the next question is does it move every object or just start using memory somewhere else and links it with previous location?
deque container supports iterator arithmetic but is it safe to use it? i want to know how deque manage the memory not how one might prefer to use it.
A deque is a double linked list of mini vectors. That means addresses are stable.
Iterator arithmetic is valid unless operation which invalidates iterators is performed.
This is true for vectors too
From this std::deque reference:
... typical implementations use a sequence of individually allocated fixed-size arrays
You could think of it like a list of arrays.
A deque is typically implemented as a sequence of fixed-length pages of elements. If you append elements, when a page is full a new one is allocated and added at the end of the pages index. This guarantees that, if you add or remove elements only at the end or at the beginning, the ones that have already been stored aren't moved around in memory (in standardese speak, references to existing elements aren't invalidated by push_back, pop_back, push_front and pop_front).
when you push_back() items into an std::vector, and retain the pointers to the objects in the vector via the back() reference -- are you guaranteed (assuming no deletions occur) that the address of the objects in the vector will remain the same?
It seems like my vector changes the pointers of the objects I use, such that if I push 10 items into it, and retain the pointers to those 10 items by remembering the back() reference after each push_back.
if your vector is to store objects, not pointers to objects, are the addresses of those objects subject to constant change upon pushing more items?
Any method that causes the vector to resize itself will invalidate all iterators, pointers, and references to the elements contained within. This can be avoided by reserving mememory, or using boost::stable_vector.
23.3.6.5/1:
Remarks: Causes reallocation if the new size is greater than the old capacity. If no reallocation happens,
all the iterators and references before the insertion point remain valid.
No, std::vector is not a stable container, i.e. pointers and iterators may get invalidated by resizing the vector (or, better, by the corresponding reallocation). If you want to avoid this behaviour, use boost::stable_vector or std::list or std::deque instead (I would prefer the last). Or, more easily, you can simply store your locations by indices.
For more information, consider also the answer to this question here.
It's not guaranteed. If you push_back enough items to exceed the size of the memory buffer that's the backing store of the vector, a new buffer will be created, all the contents will be copied to the new location, and the old buffer will be deleted. At that point, old pointers (as well as iterators!) will be invalid.
If you know exactly how much maximum space you will ever need, you could set the size of the vector's buffer to that size when you create it, to avoid reallocation. However, I prefer to store "references" to elements of a vector as a reference to the vector and a size_t index into the vector, instead of using pointers. It's not necessarily slower than pointers (depending on the CPU type) but, even if it is, it won't be much slower and in my opinion it's worth it for the peace of mind knowing that no matter how the vector is used in the future or reallocated, it'll still refer to the proper element.
I read in The C++ Programming Language : Special Edition
Don't use iterators into a resized vector
Consider this example.
vector< int >::iterator it = foo.begin();
while ( it != foo.end() ) {
if ( // something ) {
foo.push_back( // some num );
}
++it;
}
Is there a problem with this? After the vector was resized, would the foo.end() in the loop condition be pushed forward 1?
P.S. In addition, what if vector had reserved space for x number of ints. If push_back didn't violate this space, would it still be an issue ( I would assume so if it.end() points to one past the last element in the vector that contains something ).
Yes, there is a problem with it.
Any call to push_back has the potential to invalidate all iterators into a vector.
foo.end() will always retrieve the valid end iterator (which may be different to the value last returned by foo.end()), but it may have been invalidated. This means that incrementing it or comparing it may caused undefined behaviour.
Yes, there's a problem. Regardless of foo.end(), it may be invalidated by the push_back(). Edit: (i.e. it's not just that the end could change; it's possible that the whole buffer for the vector may be reallocated, so all iterators become invalid).
Yes, there's a problem with that. push_back invalidates any iterators for the vector you called it on. So after calling push_back, it is not even legal to execute ++it. This is a case of undefined behavior, so it may sometimes work and it may sometimes fail but you should never rely on it working.
As others have said, the push_back() may invalidate all iterators to the vector. The reason for this is that the data in a vector is stored in a region of contiguous memory. If the push_back() or any other operation that resizes the vector causes the size of the vector to go over the capacity of the allocated region that region will be reallocated and end up at a different place in memory, while all the iterators will still be referencing the old memory region.