I am working with STL library and my goal is to minimize the data reallocation cases.
I was wndering, does
std::vector::assign(size_type n, const value_type& val)
reallocated the data if the size is not changed or does is actually just assign the new values (for example, using operator=) ?
The STL documentation at http://www.cplusplus.com/ sais the following (C++98):
In the fill version (2), the new contents are n elements, each initialized to a copy of val.
If a reallocation happens,the storage needed is allocated using the internal allocator.
Any elements held in the container before the call are destroyed and replaced by newly constructed elements (no assignments of elements take place).
This causes an automatic reallocation of the allocated storage space if -and only if- the new vector size surpasses the current vector capacity.
The phrase "no assignments of elements take place" make it all a little confusing.
So for example, I want to have a vector of classes (for example, cv::Vec3i of OpenCV). Does this mean, that
the destructor or constructor of cv::Vec3i will be called?
a direct copy of Vec3i memory will be made and fills the vector?
what happens, if my class allocates memory at run time with new
operator? This memory cannot be accounted for by plain memory
copying. Does it mean, that assign() should not be used for such
objects?
EDIT: the whole purpose of using assign in this case is to set all values in the vector to 0 (in case I have std::vector< cv::Vec3i > v). It will be done many-many times. The size of std::vector itself will not be changed.
what i want to do (in a shorter way) is the following:
for(int i=0; i<v.size(); i++)
for(int j=0; j<3; j++)
v[i][j] = 0;
right now I am interested in C++98
std::vector.assign(...) does not reallocate the vector if it does not have to grow it. Still, it must copy the actual element.
If you want to know what the standard guarantees, look in the standard: C++11 standard plus minor editorial changes.
I assume that you have a vector filled with some data and you call an assign on it, this will:
destroy all the elements of the vector (their destructor is called) like a call to clear()
fill the now-empty vector with n copies of the given object, which must have a copy constructor.
So if your class allocates some memory you have to:
take care of this in your copy constructor
free it in the destructor
Reallocations happen when the size exceed the allocated memory (vector capacity). You can prevent this with a call to reserve(). However I think that assign() is smart enough to allocate all the memory it needs (if this is more than the already allocated) before starting to fill the vector and after having cleared it.
You may want to avoid reallocations because of their cost, but if you are trying to avoid them because your objects cannot handle properly, then I strongly discourage you to put them in a vector.
The semantics of assign are defined the standard in a quite straightforward way:
void assign(size_type n, const T& t);
Effects:
erase(begin(), end());
insert(begin(), n, t);
This means that first the destructors of the elements will be called. The copies of t are made in what is now a raw storage left after the lifetime of the elements ended.
The requirements are that value_type is MoveAssignable (for when erase doesn't erase to the end of the container and needs to move elements towards the beginning).
insert overload used here requires that value_type is CopyInsertable and CopyAssignable.
In any case, vector is oblivious to how your class is managing its own resources. That's on you to take care of. See The Rule of Three.
As in the case of vector::resize method,
std::vector::assign(size_type n, const value_type& val)
will initialize each element to a copy of "val". I prefer to use resize as it minimizes the number of object instanciations/destructions, but it does the same. Use resize if you want to minimize the data realloc, but keep in mind the following:
While doing this is safe for certain data structures, be aware that assigning /pushing elements of a class containing pointers to dynamically allocated data (e.g. with a new in the constructor) may cause havoc.
If your class dynamically allocates data then you should reimplement the YourClass::operator= to copy the data to the new object instead of copying the pointer.
Hope that it helps!
Related
I want to grow a ::std::vector at runtime, like that:
::std::vector<int> i;
i.push_back(100);
i.push_back(10);
At some point, the vector is finished, and i do not need the extra functionality ::std::vector provides any more, so I want to convert it to a C-Array:
int* v = i.data();
Because I will do that more than once, I want to deallocate all the heap memory ::std::vector reserved, but I want to keep the data, like that (pseudo-code):
free(/*everything from ::std::vector but the list*/);
Can anybody give me a few pointers on that?
Thanks in advance,
Jack
In all the implementations I have seen, the allocated part of a vector is... the array itself. If you know that it will no longer grow, you can try to shrink it, to release possibly unused memory:
i.shrink_to_fit();
int* v = i.data();
Of course, nothing guarantees that the implementation will do anything, but the only foolproof alternative would be to allocated a new array, move data from the vector to the array and clear the vector.
int *v = new int[i.size];
memcpy(v, i.data, i.size * sizeof(int));
i.clear();
But I really doubt that you will have a major gain that way...
You can use the contents of a std::vector as a C array without the need to copy or release anything. Just make sure that std::vector outlives the need for your pointer to be valid (and that further modifications which could trigger a reallocation are done on the vector itself).
When you obtain a pointer to internal storage through data() you're not actually allocating anything, just referring to the already allocated memory.
The only additional precaution you could use to save memory is to use shrink_to_fit() to release any excess memory used as spare capacity (though it's not guaranteed).
You have two options:
Keep data in vector, but call shrink_to_fit. All overhead you'll have - is an extra pointer (to the vector end). It is available since C++ 11
Copy data to an external array and destroy the vector object:
Here is the example:
std::vector<int> vec;
// fill vector
std::unique_ptr<int[]> arr(new int[vec.size()]);
std::copy(vec.begin(), vec.end(), arr.get());
I am currently optimising my code and I have a question regarding the std::vector
I have a class MyClass which I have rewritten the copy/move constructor and their corresponding operators.
MyClass(const std::string& id, int x);
MyClass(const MyClass& other);
MyClass(MyClass&& other);
~MyClass();
MyClass& operator=(const MyClass& other);
MyClass& opratror*(MyClass&& other);
I created a vector and tried the following
std::vector<MyClass> vec;
MyClass a("A", 1);
vec.push_back(a); //#1
vec.emplace_back("B", 2); //#2
vec.push_back(MyClass("C", 3)); //#3
in #1 the copy constructor is called (which I know vector store by values so it copies a)
in #2 it saves a copy constructor call only calls the constructor
in #3 it calls the constructor and move constructor
But what I found is that, at #2, #3 where the vector is not empty, every push back / emplace / emplace_back triggers the copy/destroy of existing items.
in #2, it copies "A" and destroyed the existing "A"
in #3, it does the same to "A" and "B"
it seems the vector resorts all the items whenever the array changes.
Does it mean that using a class vector could make things inefficient?
Is that the best solution to use a vector storing pointers so that there's not copies / destructors calls during resorting, only pointer copying?
Thanks
Not resorts, but reallocates. A vector is required, by contract, to store its values contiguously, like a plain array. The only way to guarantee contiguous storage is to allocate one chunk of memory. Once you got it, you're done. You can't make it bigger. All you can do is allocate a larger chunk and copy everything over and then delete the older smaller block of memory. That's what you're seeing.
A vector typically reserves some additional extra space, to accomodate new elements that might get added (so that this copying doesn't happen with every push_back), but when the vector is small, initially, only a little bit of extra space gets reserved for future growth, and this reallocation still happens quite often. But as the vector grows in size, more and more extra space gets reserved, and the reallocation happens less often.
If you know in advance how much values you are going to push_back(), you can use reserve() up front to preallocate the additional space up front, and minimize the reallocation.
If you know you're going to add ten more values to the vector:
vec.reserve(vec.size()+10);
If the vector already has at least ten more values it can accept without reallocating, then this does nothing. Otherwise the vector gets reallocated with enough extra space for at least ten additional values. The next ten push_backs are guaranteed not to cause reallocation.
You need to make your move ctor noexcept(false); otherwise in many cases it cannot safely be used.
Imagine having a vector of 10 elements in a buffer that fits 10. We resize it to fit 15, and move the first 3 of them.
Moving the 4th throws. How are we supposed to return the vector to a sane state?
Instead, the C++ standard demands that throwing move not be used, and it copies the 10 elements to the new buffer. When that succeeds, it can then destroy the old buffer.
With noexcept move, it can without fail move the 10 elements to the new buffer.
If you want to have an array as a member variable of a class there are two main options:
A: Allocate the memory on the heap
class X
{
int * arr;
public:
UnionFind(int numNodes)
{
arr = new int[numNodes];
}
}
B: Use a vector
class X
{
vector <int> arr;
public:
UnionFind(int numNodes)
{
arr.resize(numNodes);
}
}
Which of these is the preferred method? I know one drawback of heap allocated arrays is that you need to take care of deleting the memory yourself.
As a small side question, if an object of X is created on the heap is vector <int> arr also in the heap within the object? If so, how come vector <int> arr does not manually need to be deleted?
When you have the choice between a dynamically allocated C-style array and a std::vector<>, choose the vector.
It is safe, does all the alloc/realloc/resizing for you
It makes you code more flexible, readable, and easier to maintain
It is extremely efficient in most use cases
It provides explicit iterators, and plenty of member functions, including size()
Many implementations will do index checking in debug mode to catch out-of-bounds errors
Note that std::array exists for most of the cases where a C-array would be preferable (e.g., when allocation on the stack is preferred)
You should prefer vector:
the vector and vector's elements' destructors are guaranteed to run at the appropriate times
things like .push_back are massively easier and more concise to use correctly than coding your own checks on "capacity" and resizing/copy-constructing/moving in a C++ object-aware fashion
it's easier to apply algorithms to Standard containers, use the iterators etc.
it will be easier to move from vector to another Standard container (list, stack, map, unordered_map, deque etc) if evolving application/library needs suggest it
vector has some housekeeping information that's useful: size(), capacity()
before C++11 there was a single performance issue compared to using new[] and delete[] - you couldn't do an up-front "sizing" of the vector to hold however-many elements without copy-constructing their values from a prototypical element (constructor "2" here, and resize here) - that meant the constructor or resize had to iterate over every element doing copy construction even if the default constructor was a no-op (e.g. deliberately leaving all members uninitialised)
this is very rarely relevant or problematic, and indeed the C++ behaviour was generally safer
because it's a proper class, you can (whether you should is another matter) overload operator<<, operator>> for conveniently streaming arbitrary vectors
if an object of X is created on the heap is vector <int> arr also in the heap within the object? If so, how come vector <int> arr does not manually need to be deleted?
Yes, the vector object itself will be embedded within X, so will be on the heap too (similarly, it could be embedded in an automatic/stack variable, a global/namespace/static variable, a thread-specific variable etc.). That said, the vector object contains a pointer which tracks any further memory needed for elements in the vector, and that memory is by default dynamically allocated (i.e. on the heap) regardless of where the vector object itself is held.
Member variables with destructors (such as any vector) have them called automatically by the containing class's destructor.
I am totally confused with regards to deleting things in C++. If I declare an array of objects and if I use the clear() member function. Can I be sure that the memory was released?
For example :
tempObject obj1;
tempObject obj2;
vector<tempObject> tempVector;
tempVector.pushback(obj1);
tempVector.pushback(obj2);
Can I safely call clear to free up all the memory? Or do I need to iterate through to delete one by one?
tempVector.clear();
If this scenario is changed to a pointer of objects, will the answer be the same as above?
vector<tempObject> *tempVector;
//push objects....
tempVector->clear();
You can call clear, and that will destroy all the objects, but that will not free the memory. Looping through the individual elements will not help either (what action would you even propose to take on the objects?) What you can do is this:
vector<tempObject>().swap(tempVector);
That will create an empty vector with no memory allocated and swap it with tempVector, effectively deallocating the memory.
C++11 also has the function shrink_to_fit, which you could call after the call to clear(), and it would theoretically shrink the capacity to fit the size (which is now 0). This is however, a non-binding request, and your implementation is free to ignore it.
There are two separate things here:
object lifetime
storage duration
For example:
{
vector<MyObject> v;
// do some stuff, push some objects onto v
v.clear(); // 1
// maybe do some more stuff
} // 2
At 1, you clear v: this destroys all the objects it was storing. Each gets its destructor called, if your wrote one, and anything owned by that MyObject is now released.
However, vector v has the right to keep the raw storage around in case you want it later.
If you decide to push some more things into it between 1 and 2, this saves time as it can reuse the old memory.
At 2, the vector v goes out of scope: any objects you pushed into it since 1 will be destroyed (as if you'd explicitly called clear again), but now the underlying storage is also released (v won't be around to reuse it any more).
If I change the example so v becomes a pointer to a dynamically-allocated vector, you need to explicitly delete it, as the pointer going out of scope at 2 doesn't do that for you. It's better to use something like std::unique_ptr in that case, but if you don't and v is leaked, the storage it allocated will be leaked as well. As above, you need to make sure v is deleted, and calling clear isn't sufficient.
vector::clear() does not free memory allocated by the vector to store objects; it calls destructors for the objects it holds.
For example, if the vector uses an array as a backing store and currently contains 10 elements, then calling clear() will call the destructor of each object in the array, but the backing array will not be deallocated, so there is still sizeof(T) * 10 bytes allocated to the vector (at least). size() will be 0, but size() returns the number of elements in the vector, not necessarily the size of the backing store.
As for your second question, anything you allocate with new you must deallocate with delete. You typically do not maintain a pointer to a vector for this reason. There is rarely (if ever) a good reason to do this and you prevent the vector from being cleaned up when it leaves scope. However, calling clear() will still act the same way regardless of how it was allocated.
if I use the clear() member function. Can I be sure that the memory was released?
No, the clear() member function destroys every object contained in the vector, but it leaves the capacity of the vector unchanged. It affects the vector's size, but not the capacity.
If you want to change the capacity of a vector, you can use the clear-and-minimize idiom, i.e., create a (temporary) empty vector and then swap both vectors.
You can easily see how each approach affects capacity. Consider the following function template that calls the clear() member function on the passed vector:
template<typename T>
auto clear(std::vector<T>& vec) {
vec.clear();
return vec.capacity();
}
Now, consider the function template empty_swap() that swaps the passed vector with an empty one:
template<typename T>
auto empty_swap(std::vector<T>& vec) {
std::vector<T>().swap(vec);
return vec.capacity();
}
Both function templates return the capacity of the vector at the moment of returning, then:
std::vector<double> v(1000), u(1000);
std::cout << clear(v) << '\n';
std::cout << empty_swap(u) << '\n';
outputs:
1000
0
Move semantics allows for a straightforward way to release memory, by simply applying the assignment (=) operator from an empty rvalue:
std::vector<int> vec(100, 0);
std::cout << vec.capacity(); // 100
vec = vector<int>(); // Same as "vector<int>().swap(vec)";
std::cout << vec.capacity(); // 0
It is as much efficient as the "swap()"-based method described in other answers (indeed, both are conceptually doing the same thing). When it comes to readability, however, the assignment version makes a better job.
You can free memory used by vector by this way:
//Removes all elements in vector
v.clear()
//Frees the memory which is not used by the vector
v.shrink_to_fit();
If you need to use the vector over and over again and your current code declares it repeatedly within your loop or on every function call, it is likely that you will run out of memory. I suggest that you declare it outside, pass them as pointers in your functions and use:
my_arr.resize()
This way, you keep using the same memory sequence for your vectors instead of requesting for new sequences every time.
Hope this helped.
Note: resizing it to different sizes may add random values. Pass an integer such as 0 to initialise them, if required.
How does std::vector implement the management of the changing number of elements: Does it use realloc() function, or does it use a linked list?
Thanks.
It uses the allocator that was given to it as the second template parameter. Like this then. Say it is in push_back, let t be the object to be pushed:
...
if(_size == _capacity) { // size is never greater than capacity
// reallocate
T * _begin1 = alloc.allocate(_capacity * 2, 0);
size_type _capacity1 = _capacity * 2;
// copy construct items (copy over from old location).
for(size_type i=0; i<_size; i++)
alloc.construct(_begin1 + i, *(_begin + i));
alloc.construct(_begin1 + _size, t);
// destruct old ones. dtors are not allowed to throw here.
// if they do, behavior is undefined (17.4.3.6/2)
for(size_type i=0;i<_size; i++)
alloc.destroy(_begin + i);
alloc.deallocate(_begin, _capacity);
// set new stuff, after everything worked out nicely
_begin = _begin1;
_capacity = _capacity1;
} else { // size less than capacity
// tell the allocator to allocate an object at the right
// memory place previously allocated
alloc.construct(_begin + _size, t);
}
_size++; // now, we have one more item in us
...
Something like that. The allocator will care about allocating memory. It keeps the steps of allocating memory and constructing object into that memory apart, so it can preallocate memory, but not yet call constructors. During reallocate, the vector has to take care about exceptions being thrown by copy constructors, which complicates the matter somewhat. The above is just some pseudo code snippet - not real code and probably contains many bugs. If the size gets above the capacity, it asks the allocator to allocate a new greater block of memory, if not then it just constructs at the previously allocated space.
The exact semantics of this depend on the allocator. If it is the standard allocator, construct will do
new ((void*)(_start + n)) T(t); // known as "placement new"
And the allocate allocate will just get memory from ::operator new. destroy would call the destructor
(_start + n)->~T();
All that is abstracted behind the allocator and the vector just uses it. A stack or pooling allocator could work completely different. Some key points about vector that are important
After a call to reserve(N), you can have up to N items inserted into your vector without risking a reallocation. Until then, that is as long as size() <= capacity(), references and iterators to elements of it remain valid.
Vector's storage is contiguous. You can treat &v[0] as a buffer containing as many elements you have currently in your vector.
One of the hard-and-fast rules of vectors is that the data will be stored in one contiguous block of memory.
That way you know you can theoretically do this:
const Widget* pWidgetArrayBegin = &(vecWidget[0]);
You can then pass pWidgetArrayBegin into functions that want an array as a parameter.
The only exception to this is the std::vector<bool> specialisation. It actually isn't bools at all, but that's another story.
So the std::vector will reallocate the memory, and will not use a linked list.
This means you can shoot yourself in the foot by doing this:
Widget* pInteresting = &(vecWidget.back());
vecWidget.push_back(anotherWidget);
For all you know, the push_back call could have caused the vector to shift its contents to an entirely new block of memory, invalidating pInteresting.
The memory managed by std::vector is guaranteed to be continuous, such that you can treat &vec[0] as a pointer to the beginning of a dynamic array.
Given this, how it actually manages it's reallocations is implementation specific.
std::vector stored data in contiguous memory blocks.
Suppose we declare a vector as
std::vector intvect;
So initially a memory of x elements will be created . Here x is implementation depended.
If user is inserting more than x elements than a new memory block will be created of 2x (twice the size)elements and initial vector is copied into this memory block.
Thats why it is always recommended to reserve memory for vector by calling reserve
function.
intvect.reserve(100);
so as to avoid deletion and copying of vector data.