I want to grow a ::std::vector at runtime, like that:
::std::vector<int> i;
i.push_back(100);
i.push_back(10);
At some point, the vector is finished, and i do not need the extra functionality ::std::vector provides any more, so I want to convert it to a C-Array:
int* v = i.data();
Because I will do that more than once, I want to deallocate all the heap memory ::std::vector reserved, but I want to keep the data, like that (pseudo-code):
free(/*everything from ::std::vector but the list*/);
Can anybody give me a few pointers on that?
Thanks in advance,
Jack
In all the implementations I have seen, the allocated part of a vector is... the array itself. If you know that it will no longer grow, you can try to shrink it, to release possibly unused memory:
i.shrink_to_fit();
int* v = i.data();
Of course, nothing guarantees that the implementation will do anything, but the only foolproof alternative would be to allocated a new array, move data from the vector to the array and clear the vector.
int *v = new int[i.size];
memcpy(v, i.data, i.size * sizeof(int));
i.clear();
But I really doubt that you will have a major gain that way...
You can use the contents of a std::vector as a C array without the need to copy or release anything. Just make sure that std::vector outlives the need for your pointer to be valid (and that further modifications which could trigger a reallocation are done on the vector itself).
When you obtain a pointer to internal storage through data() you're not actually allocating anything, just referring to the already allocated memory.
The only additional precaution you could use to save memory is to use shrink_to_fit() to release any excess memory used as spare capacity (though it's not guaranteed).
You have two options:
Keep data in vector, but call shrink_to_fit. All overhead you'll have - is an extra pointer (to the vector end). It is available since C++ 11
Copy data to an external array and destroy the vector object:
Here is the example:
std::vector<int> vec;
// fill vector
std::unique_ptr<int[]> arr(new int[vec.size()]);
std::copy(vec.begin(), vec.end(), arr.get());
Related
I have come against a problem with my pointer vectors...
And i got an idea what the problem might be:
When I create a pointer to a vector, the pointer reserves the size of the vector on the heap. So that basicly means, that the pointer now points to the memory of the vector without anything inside... When i now resize or pushback the vector, will the pointer now still point to the whole memory of the vector or just the memory which has been allocated at the beginning?
I also want to know, if there are some tricks you can do to fix that (if what i think is true). Is the "vector.reserve(n)" a method of accomplishing this? Or is there something i could do to overwrite the pointers memory adress, to a vector after it has been initialised?
"When I create a pointer to a vector, the pointer reserves the size of the vector on the heap.
No, it doesn't! When you create a pointer to a vector, you have a pointer. Nothing more. It doesn't point to anything yet, and it certainly hasn't reserved any "heap memory" for you.
You still need to actually create the vector that will be pointed-to.
std::vector<int>* ptr1; // just a pointer;
ptr1 = new std::vector<int>(); // there we go;
delete ptr1; // don't forget this;
auto ptr2 = std::make_shared<std::vector<int>>(); // alternatively...
Y'know, it's very rare that you need to dynamically-allocate a container. Usually you just want to construct it in the usual fashion:
std::vector<int> v;
That's it. No need for pointers.
When i now resize or pushback the vector, will the pointer now still point to the whole memory of the vector or just the memory which has been allocated at the beginning?
Regardless of how you constructed/allocated it, the vector itself never moves spontaneously (only the dynamic memory that it is managing for you internally), so you do not need to worry about this.
I also want to know, if there are some tricks you can do to fix that
No need, as there's nothing to fix.
Is the "vector.reserve(n)" a method of accomplishing this?
Theoretically, if this were a problem (which it isn't) then, yes, this could possibly form the basis of a solution.
Vector is class that has internal pointer to vector's elements using continuous block of memory in the heap. Long enough for all reserved vector's elements (capacity() method)
So, if you create vector (in local scope's stack OR in the heap - doesn't matter) it creates this layout
[ vector [ ptr-to-data ] ] --> HEAP: [ 1 ][ 2 ][ 3 ]
vector<int> v1(3); // whole vector instance in the stack
vector<int> *pv2 = new vector<int>(3); // pointer to vector in the heap
each of these 2 vector instances has its pointer to its elements in the heap as well
Vector manages its pointer to the data internally.
When you push_back() more elements than its current .capacty() it will re-allocate new continuous block of memory and copy-construct or move all old elements to this new block.
I am working with STL library and my goal is to minimize the data reallocation cases.
I was wndering, does
std::vector::assign(size_type n, const value_type& val)
reallocated the data if the size is not changed or does is actually just assign the new values (for example, using operator=) ?
The STL documentation at http://www.cplusplus.com/ sais the following (C++98):
In the fill version (2), the new contents are n elements, each initialized to a copy of val.
If a reallocation happens,the storage needed is allocated using the internal allocator.
Any elements held in the container before the call are destroyed and replaced by newly constructed elements (no assignments of elements take place).
This causes an automatic reallocation of the allocated storage space if -and only if- the new vector size surpasses the current vector capacity.
The phrase "no assignments of elements take place" make it all a little confusing.
So for example, I want to have a vector of classes (for example, cv::Vec3i of OpenCV). Does this mean, that
the destructor or constructor of cv::Vec3i will be called?
a direct copy of Vec3i memory will be made and fills the vector?
what happens, if my class allocates memory at run time with new
operator? This memory cannot be accounted for by plain memory
copying. Does it mean, that assign() should not be used for such
objects?
EDIT: the whole purpose of using assign in this case is to set all values in the vector to 0 (in case I have std::vector< cv::Vec3i > v). It will be done many-many times. The size of std::vector itself will not be changed.
what i want to do (in a shorter way) is the following:
for(int i=0; i<v.size(); i++)
for(int j=0; j<3; j++)
v[i][j] = 0;
right now I am interested in C++98
std::vector.assign(...) does not reallocate the vector if it does not have to grow it. Still, it must copy the actual element.
If you want to know what the standard guarantees, look in the standard: C++11 standard plus minor editorial changes.
I assume that you have a vector filled with some data and you call an assign on it, this will:
destroy all the elements of the vector (their destructor is called) like a call to clear()
fill the now-empty vector with n copies of the given object, which must have a copy constructor.
So if your class allocates some memory you have to:
take care of this in your copy constructor
free it in the destructor
Reallocations happen when the size exceed the allocated memory (vector capacity). You can prevent this with a call to reserve(). However I think that assign() is smart enough to allocate all the memory it needs (if this is more than the already allocated) before starting to fill the vector and after having cleared it.
You may want to avoid reallocations because of their cost, but if you are trying to avoid them because your objects cannot handle properly, then I strongly discourage you to put them in a vector.
The semantics of assign are defined the standard in a quite straightforward way:
void assign(size_type n, const T& t);
Effects:
erase(begin(), end());
insert(begin(), n, t);
This means that first the destructors of the elements will be called. The copies of t are made in what is now a raw storage left after the lifetime of the elements ended.
The requirements are that value_type is MoveAssignable (for when erase doesn't erase to the end of the container and needs to move elements towards the beginning).
insert overload used here requires that value_type is CopyInsertable and CopyAssignable.
In any case, vector is oblivious to how your class is managing its own resources. That's on you to take care of. See The Rule of Three.
As in the case of vector::resize method,
std::vector::assign(size_type n, const value_type& val)
will initialize each element to a copy of "val". I prefer to use resize as it minimizes the number of object instanciations/destructions, but it does the same. Use resize if you want to minimize the data realloc, but keep in mind the following:
While doing this is safe for certain data structures, be aware that assigning /pushing elements of a class containing pointers to dynamically allocated data (e.g. with a new in the constructor) may cause havoc.
If your class dynamically allocates data then you should reimplement the YourClass::operator= to copy the data to the new object instead of copying the pointer.
Hope that it helps!
I am totally confused with regards to deleting things in C++. If I declare an array of objects and if I use the clear() member function. Can I be sure that the memory was released?
For example :
tempObject obj1;
tempObject obj2;
vector<tempObject> tempVector;
tempVector.pushback(obj1);
tempVector.pushback(obj2);
Can I safely call clear to free up all the memory? Or do I need to iterate through to delete one by one?
tempVector.clear();
If this scenario is changed to a pointer of objects, will the answer be the same as above?
vector<tempObject> *tempVector;
//push objects....
tempVector->clear();
You can call clear, and that will destroy all the objects, but that will not free the memory. Looping through the individual elements will not help either (what action would you even propose to take on the objects?) What you can do is this:
vector<tempObject>().swap(tempVector);
That will create an empty vector with no memory allocated and swap it with tempVector, effectively deallocating the memory.
C++11 also has the function shrink_to_fit, which you could call after the call to clear(), and it would theoretically shrink the capacity to fit the size (which is now 0). This is however, a non-binding request, and your implementation is free to ignore it.
There are two separate things here:
object lifetime
storage duration
For example:
{
vector<MyObject> v;
// do some stuff, push some objects onto v
v.clear(); // 1
// maybe do some more stuff
} // 2
At 1, you clear v: this destroys all the objects it was storing. Each gets its destructor called, if your wrote one, and anything owned by that MyObject is now released.
However, vector v has the right to keep the raw storage around in case you want it later.
If you decide to push some more things into it between 1 and 2, this saves time as it can reuse the old memory.
At 2, the vector v goes out of scope: any objects you pushed into it since 1 will be destroyed (as if you'd explicitly called clear again), but now the underlying storage is also released (v won't be around to reuse it any more).
If I change the example so v becomes a pointer to a dynamically-allocated vector, you need to explicitly delete it, as the pointer going out of scope at 2 doesn't do that for you. It's better to use something like std::unique_ptr in that case, but if you don't and v is leaked, the storage it allocated will be leaked as well. As above, you need to make sure v is deleted, and calling clear isn't sufficient.
vector::clear() does not free memory allocated by the vector to store objects; it calls destructors for the objects it holds.
For example, if the vector uses an array as a backing store and currently contains 10 elements, then calling clear() will call the destructor of each object in the array, but the backing array will not be deallocated, so there is still sizeof(T) * 10 bytes allocated to the vector (at least). size() will be 0, but size() returns the number of elements in the vector, not necessarily the size of the backing store.
As for your second question, anything you allocate with new you must deallocate with delete. You typically do not maintain a pointer to a vector for this reason. There is rarely (if ever) a good reason to do this and you prevent the vector from being cleaned up when it leaves scope. However, calling clear() will still act the same way regardless of how it was allocated.
if I use the clear() member function. Can I be sure that the memory was released?
No, the clear() member function destroys every object contained in the vector, but it leaves the capacity of the vector unchanged. It affects the vector's size, but not the capacity.
If you want to change the capacity of a vector, you can use the clear-and-minimize idiom, i.e., create a (temporary) empty vector and then swap both vectors.
You can easily see how each approach affects capacity. Consider the following function template that calls the clear() member function on the passed vector:
template<typename T>
auto clear(std::vector<T>& vec) {
vec.clear();
return vec.capacity();
}
Now, consider the function template empty_swap() that swaps the passed vector with an empty one:
template<typename T>
auto empty_swap(std::vector<T>& vec) {
std::vector<T>().swap(vec);
return vec.capacity();
}
Both function templates return the capacity of the vector at the moment of returning, then:
std::vector<double> v(1000), u(1000);
std::cout << clear(v) << '\n';
std::cout << empty_swap(u) << '\n';
outputs:
1000
0
Move semantics allows for a straightforward way to release memory, by simply applying the assignment (=) operator from an empty rvalue:
std::vector<int> vec(100, 0);
std::cout << vec.capacity(); // 100
vec = vector<int>(); // Same as "vector<int>().swap(vec)";
std::cout << vec.capacity(); // 0
It is as much efficient as the "swap()"-based method described in other answers (indeed, both are conceptually doing the same thing). When it comes to readability, however, the assignment version makes a better job.
You can free memory used by vector by this way:
//Removes all elements in vector
v.clear()
//Frees the memory which is not used by the vector
v.shrink_to_fit();
If you need to use the vector over and over again and your current code declares it repeatedly within your loop or on every function call, it is likely that you will run out of memory. I suggest that you declare it outside, pass them as pointers in your functions and use:
my_arr.resize()
This way, you keep using the same memory sequence for your vectors instead of requesting for new sequences every time.
Hope this helped.
Note: resizing it to different sizes may add random values. Pass an integer such as 0 to initialise them, if required.
Just on the following code do I need to allocate memory with new for floatArray or will the copy function allocate memory for me?
Also is it okay to copy a constant vector to an array this way? Why?
vector<float> floatVector;
//filled floatVector here
float *floatArray = new float[floatVector.size()];
copy(floatVector.begin(),floatVector.end(),floatArray);
delete[] floatArray;
std::copy doesn't allocate memory, you should do it yourself.
If you don't want to allocate the array first then you may use a back_inserter iterator to push elements one by one into certain containers. For some containers this is less efficient (I think) but can be very handy sometimes.
#include<iterator>
#include<vector>
#include<deque>
std::vector<float> floatVector(10,1.0);
std::deque<float> floatDeque; //memory not allocated
//insert elements of vector into deque.
std::copy(floatVector.begin(), floatVector.end(), std::back_inserter(floatDeque));
copy will not allocate for you, so yes you need to allocate it.
std::copy copies by using the arguments assignment ("=") operator or the arguments copy constructor (this is probably implementation dependent, I'm not sure right now). It does nothing but iterate through the range beginning at param.begin() to param.end() and do something like:
while (first!=last) *result++ = *first++;
Thus, you need to allocate the memory necessary yourself. Otherwise it will create undefined behaviour.
Is it OK to copy a constant vector to an array this way? Depends on what you want to do. Generally, it's fine, since the values are transferred to the target by value, not by reference, hence const-correctness is not broken.
For example, this works fine:
std::vector<int> vec;
vec.push_back(1);
vec.push_back(2);
const std::vector<int> constvec(vec);
int *arrray = new int[2];
std::copy(constvec.begin(),constvec.end(),arrray);
copy function never allocate memory but you have to allocate memory for static datastructures but for dynamic datastructures, it'll allocate memory automatically.
and this process is not good but its better to use another vector instead of that floatarray
How does std::vector implement the management of the changing number of elements: Does it use realloc() function, or does it use a linked list?
Thanks.
It uses the allocator that was given to it as the second template parameter. Like this then. Say it is in push_back, let t be the object to be pushed:
...
if(_size == _capacity) { // size is never greater than capacity
// reallocate
T * _begin1 = alloc.allocate(_capacity * 2, 0);
size_type _capacity1 = _capacity * 2;
// copy construct items (copy over from old location).
for(size_type i=0; i<_size; i++)
alloc.construct(_begin1 + i, *(_begin + i));
alloc.construct(_begin1 + _size, t);
// destruct old ones. dtors are not allowed to throw here.
// if they do, behavior is undefined (17.4.3.6/2)
for(size_type i=0;i<_size; i++)
alloc.destroy(_begin + i);
alloc.deallocate(_begin, _capacity);
// set new stuff, after everything worked out nicely
_begin = _begin1;
_capacity = _capacity1;
} else { // size less than capacity
// tell the allocator to allocate an object at the right
// memory place previously allocated
alloc.construct(_begin + _size, t);
}
_size++; // now, we have one more item in us
...
Something like that. The allocator will care about allocating memory. It keeps the steps of allocating memory and constructing object into that memory apart, so it can preallocate memory, but not yet call constructors. During reallocate, the vector has to take care about exceptions being thrown by copy constructors, which complicates the matter somewhat. The above is just some pseudo code snippet - not real code and probably contains many bugs. If the size gets above the capacity, it asks the allocator to allocate a new greater block of memory, if not then it just constructs at the previously allocated space.
The exact semantics of this depend on the allocator. If it is the standard allocator, construct will do
new ((void*)(_start + n)) T(t); // known as "placement new"
And the allocate allocate will just get memory from ::operator new. destroy would call the destructor
(_start + n)->~T();
All that is abstracted behind the allocator and the vector just uses it. A stack or pooling allocator could work completely different. Some key points about vector that are important
After a call to reserve(N), you can have up to N items inserted into your vector without risking a reallocation. Until then, that is as long as size() <= capacity(), references and iterators to elements of it remain valid.
Vector's storage is contiguous. You can treat &v[0] as a buffer containing as many elements you have currently in your vector.
One of the hard-and-fast rules of vectors is that the data will be stored in one contiguous block of memory.
That way you know you can theoretically do this:
const Widget* pWidgetArrayBegin = &(vecWidget[0]);
You can then pass pWidgetArrayBegin into functions that want an array as a parameter.
The only exception to this is the std::vector<bool> specialisation. It actually isn't bools at all, but that's another story.
So the std::vector will reallocate the memory, and will not use a linked list.
This means you can shoot yourself in the foot by doing this:
Widget* pInteresting = &(vecWidget.back());
vecWidget.push_back(anotherWidget);
For all you know, the push_back call could have caused the vector to shift its contents to an entirely new block of memory, invalidating pInteresting.
The memory managed by std::vector is guaranteed to be continuous, such that you can treat &vec[0] as a pointer to the beginning of a dynamic array.
Given this, how it actually manages it's reallocations is implementation specific.
std::vector stored data in contiguous memory blocks.
Suppose we declare a vector as
std::vector intvect;
So initially a memory of x elements will be created . Here x is implementation depended.
If user is inserting more than x elements than a new memory block will be created of 2x (twice the size)elements and initial vector is copied into this memory block.
Thats why it is always recommended to reserve memory for vector by calling reserve
function.
intvect.reserve(100);
so as to avoid deletion and copying of vector data.