A std::vector<T> has the property of storing its elements continuously in memory. But what about a std::vector<std::vector<T>>? The elements within an individual std::vector<T> are continuous, but are the vectors themselves continuous in memory (that is, the whole data kept in the outer vector would be one memory block)? Would not this imply, that if I resize one of the inner vectors I would have to copy (or move) many objects to preserve continuity?
And what about a std::array<std::array<T,N>,N>? Here I would assume memory continuity just as for T[N][N]
A std::vector<T> internally stores a pointer to dynamically allocated memory. So while the elements of a single std::vector<T> will be continuous in memory there is no relationship to any pointers that those elements store themselves.
Therefore a std::vector<std::vector<T>> will not have the elements of the "inner vectors" stored continuously in relation to each other. It would also be impossible for a resize of one of those "inner vectors" to affect the others, since at the point where the resize happens there is no information about any relationship to other std::vector<T> objects.
On the other hand a std::array is a (thin) wrapper around a C-style array and uses neither dynamic allocation nor is resizable, so a std::array<std::array<T,N>,N> will have the same memory layout as a T[N][N] array.
Related
vector container holds object in continuous memory. it is easy to understand for cases like vector. but what if it is a vector of vectors, like vector>, each vector in this vector of vectors can have different length. how does it manage the memory? Does it allocate a fixed length vector every time we push in a new vector? if so, what will happen if the first vector grows out of size during push_back. would it trigger a full vector of vector reallocate and copy/move?
A vector is a pointer to a dynamic array. If you push_back and find you're out of space in the array you have, you allocate a new, bigger array, copy over everything from the old array, and then stick the new value in.
If you have a vector of vectors, the same holds true for each of the inner vectors.
What you need to understand here is that a vector of vectors (unlike a 2D array), is not contiguous in memory. Each of the inner vectors' arrays can be stored anywhere in memory. Or in other words, "each vector in a vector of vectors is a completely different vector. Each with their own, completely separate and separately managed buffer.1"
1. Thanks to user4581301 for this!
A vector contains a pointer to a contiguous memory block. When it runs out of memory, it allocates a new memory block. A vector of vectors is just a vector of pointers to memory blocks. Although each memory block is a contiguous block, they are not necessarily contiguous to each other, that is, not necessarily when one vector ends, the next one starts, there is almost always a gap.
Why the not necessarily and almost always semantics? Because it depends on the memory allocator you're using and on the operating system internals. Ultimately, it's (one of) the job(s) of the OS to allocate and serve memory blocks to user-space programs.
I apologize if this is too basic a question to ask here; I've been reading a bit about std::vector and I understand that it has to resize when the size reaches capacity. This can be an expensive operation if the objects it contains are large since it has to copy every one of them.
My question is: how does std::vector handle the actual size of the objects it contains growing? Suppose I initialize some
std::vector<Obj> vec(100);
And each Obj is initialized to something very small. But what if I then do something like
vec[14].ENGORGIO()
So that it now takes much more memory. Since the elements of std::vector are stored in contiguous memory, does this mean that it has to resize (expensively!)
To avoid this kind of thing, should I instead be storing pointers in a vector rather than the objects themselves? Like so
std::vector< std::unique_ptr<Obj> > vec;
Thank you
The size of a data type in C++ is set at compile time and cannot change. The data type may contain references to other objects, and these other objects can vary in size, but the size of the reference, and thus the data type, is unchanging.
Consider a vector of vectors. The inner vector may contain 0 elements or billions and billions and it will always be the same size. The outer vector only knows that it contains 0 or more vectors and knows nothing about what is contained inside the inner vectors.
You do not have to worry about this.
Each type has a size, and all objects of that type have the same size. The size of an object never changes throughout the program. Therefore the scenario that you describe does not exist.
The way vector "resizes" without changing the size of the vector object is achieved through indirection. The elements are not part of the memory of the object. Instead, the vector points to a buffer in dynamic memory. Resizing is done by creating a bigger dynamic buffer, and copying the elements from the old one before destroying the old buffer.
How is a 2D area layout in memory? Especially if its a staggered area. Given, to my understanding, that memory is contiguous going from Max down to 0, does the computer allocate each area in the area one after the other? If so, should one of the areas in the area need to be resized, does it shift all the other areas down as to make space for the newly sized area?
If specifics are needed:
C++17/14/11
Clang
linux x86
Revision: (thanks user4581301)
I'm referring to having a vector<vector<T>> where T is some defined type. I'm not talking template programming here unless that doesn't change anything.
The precise details of how std::vector is implemented will vary from compiler to compiler, but more than likely, a std::vector contains a size_t member that stores the length and a pointer to the storage. It allocates this storage using whatever allocator you specify in the template, but the default is to use new, which allocates them off the heap. You probably know this, but typically the heap is the area of RAM below the stack in memory, which grows from the bottom up as the stack grows from the top down, and which the runtime manages by tracking which blocks of it are free.
The storage managed by a std::vectoris a contiguous array of objects, so a vector of twenty vectors of T would contain at least a size_t storing the value 20, and a pointer to an array of twenty structures each containing a length and a pointer. Each of those pointers would point to an array of T, stored contiguously in memory.
If you instead create a rectangular two-dimensional array, such as T table[ROWS][COLUMNS], or a std::array< std::array<T, COLUMNS>, ROWS >, you will instead get a single continuous block of T elements stored in row-major order, that is: all the elements of row 0, followed by all the elements of row 1, and so on.
If you know the dimensions of the matrix in advance, the rectangular array will be more efficient because you’ll only need to allocate one block of memory. This is faster because you’ll only need to call the allocator and the destructor one time, instead of once per row, and also because it will be in one place, not split up over many different locations, and therefore the single block is more likely to be in the processor’s cache.
vectors are thin wrappers around a dynamically allocated array of their elements. For a vector<vector<T>>, this means that the outer vector's internal array contains the inner vector structures, but the inner vectors allocate and manage their own internal arrays separately (the structure contains a pointer to the managed array).
Essentially, the 2D aspect is purely in the program logic; the elements of any given "row" are contiguous, but there is no specified spacial relationship between the rows.
True 2D arrays (where the underlying memory is allocated as a single block) only really happen with C-style arrays declared with 2D syntax (int foo[10][20];) and nested std::array types, or POD types following the same basic design.
I have multiple vectors of structure objects for different structures. Now I want to reuse the same memory for all the vector objects.i.e,Once my work is done with one vector i want to erase its elements from the memory and assign that memory to the other vector.
i.e.The first vector is of one structure type object and the second vector is a structure type object of a completely different structure.
I am using windows 8.1 64-bit.
When you erase vector elements, the memory allocated for the vector elements is not freed until you call std::vector::shrink_to_fit. Thus you do not have to do special actions to reuse the allocated memory.
It is not clean what you mean under
Once my work is done with one vector i want to erase its elements from
the memory and assign that memory to the other vector.
You could continue using the same vector with the same memory, or you can call v1.swap(v2) to exchange allocated memories of two vectors, or you can move allocated memory of one vector to another v2 = std::move(v1).
It is applicable to vectors containing elements of a same type or pointers, that is not suitable to your case.
Being able to move allocated memory from a vector of one type to a vector containing another is not a feature supported by std::vector. I would suggest writing/finding another container that fits your needs.
I know vectors are guaranteed to be contiguous memory, and so are arrays. So what happens when I do something like this:
std::vector<uint8_t> my_array[10];
my_array[2].push_back(11);
my_array[2].push_back(7);
What would the memory look like? If both need to be contiguous, would every element of the array after my_array[2] be pushed forward a byte every time I do a push_back() on my_array[2]?
Would this be the same situation as when I have an array of structs, where the structs have a member that has a variable size, such as a string or another vector?
Memory footprint of std::vector consists of two parts:
The memory for the std::vector object itself (very small, and independent of the size), and
The memory for the data of the vector (depends on the number of elements in the vector).
The first kind of data will be contiguous in an array; the second kind of data is allocated dynamically, so it would not be contiguous in an array.
This would not be the same as with a C struct that has a flexible data member, because the data portion of std::vector is not always allocated in the same kind of memory, let alone being adjacent to it. The vector itself may be allocated in static, dynamic, or automatic memory areas, while its data is always in the dynamic area. Moreover, when vector is resized, the memory for its data may be moved to a different region.
Each time you call push_back, std::vector checks if it has enough dynamic memory to accommodate the next data element. If there is not enough memory, then the vector allocates a bigger chunk of memory, and moves its current content there before pushing the new item.
The vector memory structure is contiguous in memory; however std::vector's all contain a pointer pointing to dynamically allocated memory for the actual storage (which is very very likely not contiguous).
Knowing this, std::vector::push_back will only check to see if the (external) dynamically allocated array has enough capacity to hold the new item, if not it will reallocate space. A push_back on the first vector that overflows will not cause the second vector in the array to reallocate memory, that isn't how it works.
Also, there is no such thing as a struct having a variable size, the size of structures and classes have to be known at compile time.
std::string also has a fixed size, although you may think it is variable, because it also (like vector) has a pointer to the char* it contains.