I just wrote some basic code that pushes in a few values, deletes a value by using an iterator that points to it(erase). The set does not contain that value, however the iterator still points to the deleted value.
Isn't this counter-intuitive? Why does this happen?
// erasing from set
#include <iostream>
#include <set>
int main ()
{
std::set<int> myset;
std::set<int>::iterator it;
// insert some values:
for (int i=1; i<10; i++) myset.insert(i*10); // 10 20 30 40 50 60 70 80 90
it = myset.begin();
++it; // "it" points now to 20
myset.erase (it);
std::cout << *it << std::endl; // still prints 20
std::cout << "myset contains:";
for (it=myset.begin(); it!=myset.end(); ++it)
std::cout << ' ' << *it;
std::cout << '\n';
return 0;
}
You have an invalid set iterator. The standard prescribes no rules for what happens when you dereference an invalid set iterator, so the behavior is undefined. It is allowed to return 20. It is allowed to do anything else, for that matter.
The set does not contain that value, however the iterator still points to the deleted value.
No, it doesn't. Not really.
Isn't this counter-intuitive? Why does this happen?
Because the memory underneath that iterator still happens to contain the bits that make up the value 20. That doesn't mean it's valid memory, or that those bits will always have that value.
It's just a ghost.
You erased it from the set, but the iterator is still pointing to the memory that it was pointing to before you erased it. Unless the erase did something to invalidate that memory (e.g. re-organize the entire set and write over it), the memory and its contents still exist.
HOWEVER it is "dead". You should not reference it. This is a real problem...you can't iterate over a set, calling "erase" on iterators, and expect the contents of the iterate to still be valid. As far as I know, you can't cache iterators and expect to reference them if you are erasing the contents of the set using them.
This is also true when iterating over a list<.>. It is tempting to iterate over a list and use the iterator to erase(.) certain elements. But the erase(.) call breaks the linkage, so your iterator is no longer valid.
This is also true when you iterate over a vector<.>. But it is more obvious there. If I am at element N and call erase on it, the size of the underlying contiguous elements just got smaller by 1.
In general, it is probably a good idea to avoid operations that can effect the allocation of the underlying container (insert, erase, push_xxx, etc.) while using an iterator that will be subsequently referenced (e.g. loop, dereference after operation, etc.).
Related
I am attempting to create a buffer-overflow-proof program and I was considering using an stl vector for this, but I have read that this still would not perform bounds checking and could still be hacked. I am mainly concerned about overriding a return or function call or another variable in my program. Would a vector work in this case?
You can declare a vector like this
std::vector<int> v;
This is an empty vector of ints called v. It has 0 elements. Attempting to access any of its elements is undefined behavior. So yes, it's possible to create a buffer overflow in a std::vector if you treat it poorly.
v[0] = 1; // undefined because we're out of bounds
v[1] = 2; // as is this
Normally when adding elements to a std::vector, you use push_back or emplace_back, though, not direct element access
v.push_back(0); // v is now 1 element long: { 0 }
v.emplace_back(1); // v is now 2 elements long: { 0 , 1 }
Using the [] operator does give you a reference to the underlying item, so you can use it both for retrieval and assignment, as long as you're sure the index is valid. You can use the size() function to determine what the appropriate max element index is (it's size() - 1)
for (std::size_t i = 0; i < v.size(); ++i)
{
v[i] = 10;
}
// v was size 2, we've replaced both elements at 0 and 1 with value 10
When using your std::vector, use the interface provided to you. Use push_back or emplace_back or insert to add elements. pop_back or erase to remove elements. A std::vector knows its begin and end, so you can iterate through it like this:
// prints the elements in the vector
for (auto it = v.begin(); it != v.end(); ++it)
std::cout << *it << " ";
std::cout << "\n";
or a ranged-based for loop, since it provides the begin() and end() functions
// prints the elements in the vector
for (const auto& elem : v)
std::cout << elem << " ";
std::cout << "\n";
It's possible to overrun your std::vector buffer, but very unlikely if you follow the rules. A std::vector is nicer than an array, so if you don't mind your elements existing in the free store (which is most non-high-performance applications), go for it. The ease of use and flexibility is well worth the extra overhead.
Further, because the data in your std::vector does NOT exist on the stack (while a local array is placed on the stack) you're very unlikely to overwrite your stack if you do overrun your std::vector. If you're lucky, your program will crash, but don't rely on this.
In conclusion, a std::vector can have its buffer overrun, but its interface makes it very hard to do so. If you ever do overrun its buffer, you're overwriting data in the free store, not on the stack, so you won't be modifying your return address. While you could potentially have a second item in the free store be written over by a vector buffer overrun, I believe this is non-deterministic. While anyone with access to your source code with an array could easily identify what input would produce a stack overwrite, if any.
See https://en.cppreference.com/w/cpp/container/vector for more information on the std::vector interface.
I would like to loop through elements of a vector in C++.
I am very new at this so I don't understand the details very well.
For example:
for (elements in vector) {
if () {
check something
else {
//else add another element to the vector
vectorname.push_back(n)
}
}
Its the for (vector elements) that I am having trouble with.
You'd normally use what's called a range-based for loop for this:
for (auto element : your_vector)
if (condition(element))
// whatever
else
your_vector.push_back(something);
But note: modifying a vector in the middle of iteration is generally a poor idea. And if your basic notion is to add the element if it's not already present, you may want to look up std::set, std::map, std::unordered_set or std::unordered_map instead.
In order to do this properly (and safely), you need to understand how std::vector works.
vector capatity
You may know that a vector works much like an array with "infinite" size. Meaning, it can hold as many elements as you want, as long as you have enough memory to hold them. But how does it do that?
A vector has an internal buffer (think of it like an array allocated with new) that may be the same size as the elements you're storing, but generally it's larger. It uses the extra space in the buffer to insert any new elements that you want to insert when you use push_back().
The amount of elements the vector has is known as its size, and the amount of elements it can hold is known as its capacity. You can query those via the size() and capacity() member functions.
However, this extra space must end at some point. That's when the magic happens: When the vector notices it doesn't have enough memory to hold more elements, it allocates a new buffer, larger1 than the previous one, and copies all elements to it. The important thing to notice here is that the new buffer will have a different address. As we continue with this explanation, keep this in mind.
iterators
Now, we need to talk about iterators. I don't know how much of C++ you have studied yet, but think of an old plain array:
int my_array[5] = {1,2,3,4,5};
you can take the address of the first element by doing:
int* begin = my_array;
and you can take the address of the end of the array (more specifically, one past the last element) by doing:
int* end = begin + sizeof(my_array)/sizeof(int);
if you have these addresses, one way to iterate the array and print all elements would be:
for (int* it = begin; it < end; ++it) {
std::cout << *it;
}
An iterator works much like a pointer. If you increment it (like we do with the pointer using ++it above), it will point to the next element. If you dereference it (again, like we do with the pointer using *it above), it will return the element it is pointing to.
std::vector provides us with two member functions, begin() and end(), that return iterators analogous to our begin and end pointers above. This is what you need to keep in mind from this section: Internally, these iterators have pointers that point to the elements in the vector's internal buffer.
a simpler way to iterate
Theoretically, you can use std::vector::begin() and std::vector::end to iterate a vector like this:
std::vector<int> v{1,2,3,4,5};
for (std::vector<int>::iterator it = v.begin; it != v.end(); ++it) {
std::cout << *it;
}
Note that, apart from the ugly type of it, this is exactly the same as our pointer example. C++ introduced the keyword auto, that lets us get rid of these ugly types, when we don't really need to know them:
std::vector<int> v{1,2,3,4,5};
for (auto it = v.begin; it != v.end(); ++it) {
std::cout << *it;
}
This works exactly the same (in fact, it has the exact same type), but now we don't need to type (or read) that uglyness.
But, there's an even better way. C++ has also introduced range-based for:
std::vector<int> v{1,2,3,4,5};
for (auto it : v) {
std::cout << it;
}
the range-based for construct does several things for you:
It calls v.begin() and v.end()2 to get the upper and lower bounds of the range we're going to iterate;
Keeps an internal iterator (let's call it i), and calls ++i on every step of the loop;
Dereferences the iterator (by calling *i) and stores it in the it variable for us. This means we do not need to dereference it ourselves (note how the std::cout << it line looks different from the other examples)
putting it all together
Let's do a small exercise. We're going to iterate a vector of numbers, and, for each odd number, we are going to insert a new elements equal to 2*n.
This is the naive way that we could probably think at first:
std::vector<int> v{1,2,3,4,5};
for (int i : v) {
if (i%2==1) {
v.push_back(i*2);
}
}
Of course, this is wrong! Vector v will start with a capacity of 5. This means that, when we try using push_back for the first time, it will allocate a new buffer.
If the buffer was reallocated, its address has changed. Then, what happens to the internal pointer that the range-based for is using to iterate the vector? It no longer points to the buffer!
This it what we call a reference invalidation. Look at the reference for std::vector::push_back. At the very beginning, it says:
If the new size() is greater than capacity() then all iterators and references (including the past-the-end iterator) are invalidated. Otherwise only the past-the-end iterator is invalidated.
Once the range-based for tries to increment and dereference the now invalid pointer, bad things will happen.
There are several ways to avoid this. For instance, in this particular algorithm, I know that we can never insert more than n new elements. This means that the size of the vector can never go past 2n after the loop has ended. With this knowledge in hand, I can increase the vector's capacity beforehand:
std::vector<int> v{1,2,3,4,5};
v.reserve(v.size()*2); // Increases the capacity of the vector to at least size*2.
// The code bellow now works properly!
for (int i : v) {
if (i%2==1) {
v.push_back(i*2);
}
}
If for some reason I don't know this information for a particular algorithm, I can use a separate vector to store the new elements, and then add them to our vector at the end:
std::vector<int> v{1,2,3,4,5};
std::vector<int> doubles;
for (int i : v) {
if (i%2==1) {
doubles.push_back(i*2);
}
}
// Reserving space is not necessary because the vector will allocate
// memory if it needs to anyway, but this does makes things faster
v.reserve(v.size() + doubles.size());
// There's a standard algorithm (std::copy), that, when used in conjunction with
// std::back_inserter, does this for us, but I find that the code bellow is more
// readable.
for (int i : doubles) {
v.push_back(i);
}
Finally, there's the old plain for, using an int to iterate. The iterator cannot be invalidated because it holds an index, instead of a pointer to the internal buffer:
std::vector<int> v{1,2,3,4,5};
for (int i = 0; i < v.size(); ++i) {
if (v[i]%2==1) {
doubles.push_back(v[i]*2);
}
}
Hopefully by now, you understand the advantages and drawbacks of each method. Happy studies!
1 How much larger depends on the implementation. Generally, implementations choose to allocate a new buffer of twice the size of the current buffer.
2 This is a small lie. The whole story is a bit more complicated: It actually tries to call begin(v) and end(v). Because vector is in the std namespace, it ends up calling std::begin and std::end, which, in turn, call v.begin() and v.end(). All of this machinery is there to ensure that the range-based for works not only with standard containers, but also with anything with a proper implementation for begin and end. That includes, for instance, regular plain arrays.
Here is the quick code snippet using iterators to iterate the vector-
#include<iostream>
#include<iterator> // for iterators to include
#include<vector> // for vectors to include
using namespace std;
int main()
{
vector<int> ar = { 1, 2, 3, 4, 5 };
// Declaring iterator to a vector
vector<int>::iterator ptr;
// Displaying vector elements using begin() and end()
cout << "The vector elements are : ";
for (ptr = ar.begin(); ptr < ar.end(); ptr++)
cout << *ptr << " ";
return 0;
}
Article to read more - Iterate through a C++ Vector using a 'for' loop
.
Hope it will help.
Try this,
#include<iostream>
#include<vector>
int main()
{
std::vector<int> vec(5);
for(int i=0;i<10;i++)
{
if(i<vec.size())
vec[i]=i;
else
vec.push_back(i);
}
for(int i=0;i<vec.size();i++)
std::cout<<vec[i];
return 0;
}
Output:
0123456789
Process returned 0 (0x0) execution time : 0.328 s
Press any key to continue.
Here is a code snippet that I was looking at:
vector<int> iv = {1,2,3,4,5,6,7,8,9,10};
auto iter = iv.begin(), mid = iv.begin() + iv.size()/2;
for(int count = 100; count; --count ) {
iter = iv.insert(iter, - 1);
cout << "capacity = " << iv.capacity() << "*mid = " << *mid << endl;
}
As per iterator invalidation rules:
vector: all iterators and references before the point of insertion are unaffected, unless the new container size is greater than the previous capacity (in which case all iterators and references are invalidated)[23.2.4.3/1] Iterator invalidation rules
I understand that since I am reassigning the value of "iter" at each insert operation, perhaps I am able to maintain it's validity (please correct me if I am wrong). However, the iterator "mid" remain valid in this case even when I am not tampering with it in the loop and also when the capacity of the vector is changing.
So, how is "mid" able to update itself after reallocation ?
To know whether mid is changing at all or not, I changed line 4 in the code to:
iv.insert(iter, -1); // Did not assign it back to iter.
Printing the results of dereferencing the value at mid suggests the change and perhaps also that iter is invalidated. (Again, please correct me if I am wrong).
Your understanding is correct. Once the capacity increases, any iterator becomes invalid. The mid iterator becomes invalid even when capacity didn't changes but it basically points to the previous element.
So the original code would work at least for iter, however mid would become unusable upon first insertion. With the modification the code is completely invalid.
Usually the implementation of vector iterator is just a simple pointer that points to some element to backing array. Therefore when capacity changes and array is reallocated, any such iterator is no longer valid as the pointer points to no longer valid memory. As a result, you may see either garbage, you may get segmentation fault or randomly see correct values. When capacity doesn't change, the elements in the array may be moved forward so you could see the previous element in the iterators after the insert point, but only in case there was no empty elements at the beginning of array (e.g. start was greater then zero). But all those are specific to implementation and therefore standard clearly specifies that most of the above is undefined behavior.
I am very puzzled at the result of this bit of code:
std::vector<int> v;
std::cout << (v.end() - v.begin()) << std::endl;
v.reserve(1);
std::cout << (v.end() - v.begin()) << std::endl;
v[9] = 0;
std::cout << (v.end() - v.begin()) << std::endl;
The output:
0
0
0
So... first of all... end() does not point to the end of the internal array but the last occupied cell... ok, that is why the result of the iterator subtraction is still 0 after reserve(1). But, why is it still 0 after one cell has been filled. I expected 1 as the result, because end() should now return the iterator to the second internal array cell.
Furthermore, why on earth am I not getting a seg fault for accessing the tenth cell with v[9] = 0, while the vector is only 1 cell long?
First of all, end() gives you an iterator to one beyond the last element in the vector. And if the vector is empty then begin() can't return anything else than the same as end().
Then when you call reserve() you don't actually create any elements, you only reserve some memory so the vector don't have to reallocate when you do add elements.
Finally, when you do
v[9] = 0;
you are indexing the vector out of bounds which leads to undefined behavior as you write to memory you don't own. UB often leads to crashes, but it doesn't have too, it may seem to work when in reality it doesn't.
As a note on the last part, the [] operator doesn't have bounds-checking, which is why it will accept indexing out of bounds. If you want bounds-checking you should use at().
v[9] = 0;, you're just accessing the vector out of bound, it's UB. It may crash in some cases, and may not. Nothing is guaranteed.
And v[9] = 0;, you don't add element at all. You need to use push_back or resize:
v.push_back(0); // now it has 1 element
v.resize(10); // now it has 10 elements
EDIT
why does v[index] not create an element?
Because std::vector::operator[] just doesn't do that.
Returns a reference to the element at specified location pos. No bounds checking is performed.
Unlike std::map::operator[], this operator never inserts a new element into the container.
So it's supposed that the vector has the sufficient elements for the subscript operator.
BTW: What do you suppose vector should do when you write v[9] = 0? Before set the 10th element to 0, it has to push 10 elements first. And How to set their values? All 0? So, it won't do it, the issue just depends on yourself.
This is a guess, but hopefully a helpful one.
You will only get a segfault when you attempt to access an address that has not been assigned to your process' memory space. When the OS gives memory to a program, it generally does so in 4KB increments. Because of this, you can access past the end of some arrays/vectors without triggering a segfault, but not others.
Dream output:
/* DREAM OUTPUT:
INT: 1
TESTINT: 1
TESTINT: 2
TESTINT: 23
TESTINT: 24
TESTINT: 25
TESTINT: 3
TESTINT: 4
TESTINT: 5
TESTINT: 6
INT: 23
INT: 24
INT: 25
INT: 3
INT: 4
INT: 5
INT: 6
Problem
ERROR 1: Not erasing the '2' causes a bizzare effect.
ERROR 2: Erasing the '2' causes memory corruption.
Code
#include <cstdlib>
#include <iostream>
#include <vector>
int main(int argc, char* argv[])
{
std::vector<int> someInts;
someInts.push_back(1);
someInts.push_back(2);
someInts.push_back(3);
someInts.push_back(4);
someInts.push_back(5);
someInts.push_back(6);
for(std::vector<int>::iterator currentInt = someInts.begin();
currentInt != someInts.end(); ++currentInt)
if(*currentInt == 2)
{
std::vector<int> someNewInts;
someNewInts.push_back(23);
someNewInts.push_back(24);
someNewInts.push_back(25);
someInts.insert(currentInt + 1, someNewInts.begin(), someNewInts.end());
//someInts.erase(currentInt);
for(std::vector<int>::iterator testInt = someInts.begin();
testInt != someInts.end(); ++testInt)
std::cout << "TESTINT: " << *testInt << '\n';
}
else
std::cout << "INT: " << *currentInt << '\n';
return 0;
}
The code is pretty self-explanatory, but I'd like to know what's going on here. This is a replica using ints of what's happening in a much larger project. It baffles me.
Inserting elements into a vector causes the iterators asociated with it to be invalid, since the vector can grow and thus it reallocates its internal storage space.
As someInts.erase(currentInt); invalidates currentInt you can't use it until you set it right.
It so happens that erase() returns a valid iterator in the list to continue with.
An iterator that designates the first element remaining beyond any elements removed, or a pointer to the end of the vector if no such element exists.
Try
currentInt = someInts.erase(currentInt);
which would put the outer loop at '23' the start of your test data and step to '24' for the next loop.
You need to understand the differences between the stl collections.
A Vector is a continuous (usually) block of memory. Whem you insert into the middle, it tries to be helpful by re-allocating enough memory for the existing data plus the new, then copying it all to the right places and letting you continue as if nothing had happened. However, as you're finding - something has happened. Your iterators that used to refer to the old memory block, are still pointing there - but the data has been moved. You get memory errors if you try to use them.
One answer is to determine where the iterator used to point, and update it to point to the new location. Typically, people use the [] operator for this, but you can use begin() + x (where x is the index into the vector).
Alternatively, use a collection whose iterators are not invalidated by inserting. The best one for this is the list. Lists are constructed from little blocks of memory (1 per item) with a pointer to the next block along. This makes insertion very quick and easy as no memory needs to be modified, just the pointers to the blocks either side of the new item. Your iterator will still be valid too!
Erasing is just the same, except once you delete the item your iterator refers to, its invalid (obviously) so you cannot make any operation on it. Even ++ operator will not work as the memory might have changed in a vector, or the list pointers be different. So, you can first get an iterator to the next element, store it and then use that once you've deleted an item, or use the return value from the erase() method.
If you were to use list as the collection instead of vector, you would not get random-access and it might use more memory but you would have constant-time insertion in the middle of the collection and doing so would not invalidate your iterators.
The exception would be the one you were erasing, so you would not be able to ++ it at the end of the loop. You would have to handle this situation by storing a copy of its next element.