If I have an array of vectors, will the vector be limited in its resizing ability due to the contiguous storage nature of arrays?
Yes, but not in the way you're thinking.
Vectors have to find contiguous address space for their content. Memory fragmentation may cause the largest contiguous block to be smaller than total free memory. And having many vectors makes fragmentation more likely.
No; internally, vectors hold pointers to the memory blocks, not the block themselves.
resize won't affect the array's memory at all. The vectors have a pointer to the actual storage, so resizing affects some other memory that has nothing to do with the array's. All that will be in the array is basically just pointers pointing to possibly different lengths of memory blocks.
Furthermore, if you have something like this:
std::vector<int> arr [5];
The array's memory will be on the stack, and the vectors' memory will be on the heap. Totally different!
Related
I'm having some troubles with a large array of vectors in C++.
Basicaly I wan't an array of 2 millions elements, and each elements is a vector<int> (It's for building an adjacency list).
So when I do vector<int> myList[10] it works great, but when i do vector<int> myList[2000000] it does not work and I don't know why.
I tried to do unsigned long int var = 2000000; vector<int> myList[var]; but still the same error. (I don't know what is the error, my program just crash)
If you have any idea,
Thanks
There's a big difference between heap and stack memory. The heap is the nice big space where you can dynamically allocate gigabytes of memory - the stack is much more constrained in terms of allocation size (and is determined at compile time).
If defining a local variable, that means it lives on the stack (like your array). With 2 million elements, that's at least 2MB being allocated (or assuming ~24B of stack usage per vector, more like 48MB), which is quite a lot for the stack, and likely causes the crash. Dynamically allocating an array of vectors (or preferably just allocating a vector of vectors) ensures that the bulk of the memory is being allocated from the heap, which might prevent this crash.
You can also increase the size of the stack using compiler flags, but that's generally not preferable to just dynamic allocation.
This is helpfull
Dynamically allocate memory for my_List. or
Declare your array of vector of int's(my_List) as a global variable and size a `const. Thier storage locations are by design big enough to allocate such large mermory size.
For local variable, the stack segment might be to small to allocate 2e6*24B.
vector container holds object in continuous memory. it is easy to understand for cases like vector. but what if it is a vector of vectors, like vector>, each vector in this vector of vectors can have different length. how does it manage the memory? Does it allocate a fixed length vector every time we push in a new vector? if so, what will happen if the first vector grows out of size during push_back. would it trigger a full vector of vector reallocate and copy/move?
A vector is a pointer to a dynamic array. If you push_back and find you're out of space in the array you have, you allocate a new, bigger array, copy over everything from the old array, and then stick the new value in.
If you have a vector of vectors, the same holds true for each of the inner vectors.
What you need to understand here is that a vector of vectors (unlike a 2D array), is not contiguous in memory. Each of the inner vectors' arrays can be stored anywhere in memory. Or in other words, "each vector in a vector of vectors is a completely different vector. Each with their own, completely separate and separately managed buffer.1"
1. Thanks to user4581301 for this!
A vector contains a pointer to a contiguous memory block. When it runs out of memory, it allocates a new memory block. A vector of vectors is just a vector of pointers to memory blocks. Although each memory block is a contiguous block, they are not necessarily contiguous to each other, that is, not necessarily when one vector ends, the next one starts, there is almost always a gap.
Why the not necessarily and almost always semantics? Because it depends on the memory allocator you're using and on the operating system internals. Ultimately, it's (one of) the job(s) of the OS to allocate and serve memory blocks to user-space programs.
I know vectors are guaranteed to be contiguous memory, and so are arrays. So what happens when I do something like this:
std::vector<uint8_t> my_array[10];
my_array[2].push_back(11);
my_array[2].push_back(7);
What would the memory look like? If both need to be contiguous, would every element of the array after my_array[2] be pushed forward a byte every time I do a push_back() on my_array[2]?
Would this be the same situation as when I have an array of structs, where the structs have a member that has a variable size, such as a string or another vector?
Memory footprint of std::vector consists of two parts:
The memory for the std::vector object itself (very small, and independent of the size), and
The memory for the data of the vector (depends on the number of elements in the vector).
The first kind of data will be contiguous in an array; the second kind of data is allocated dynamically, so it would not be contiguous in an array.
This would not be the same as with a C struct that has a flexible data member, because the data portion of std::vector is not always allocated in the same kind of memory, let alone being adjacent to it. The vector itself may be allocated in static, dynamic, or automatic memory areas, while its data is always in the dynamic area. Moreover, when vector is resized, the memory for its data may be moved to a different region.
Each time you call push_back, std::vector checks if it has enough dynamic memory to accommodate the next data element. If there is not enough memory, then the vector allocates a bigger chunk of memory, and moves its current content there before pushing the new item.
The vector memory structure is contiguous in memory; however std::vector's all contain a pointer pointing to dynamically allocated memory for the actual storage (which is very very likely not contiguous).
Knowing this, std::vector::push_back will only check to see if the (external) dynamically allocated array has enough capacity to hold the new item, if not it will reallocate space. A push_back on the first vector that overflows will not cause the second vector in the array to reallocate memory, that isn't how it works.
Also, there is no such thing as a struct having a variable size, the size of structures and classes have to be known at compile time.
std::string also has a fixed size, although you may think it is variable, because it also (like vector) has a pointer to the char* it contains.
Probably, this is a very basic question, but here goes anyways. I have an array of size say 10. But, while assigning integers to that array I give only 8 elements. Can I free the memory of 2 elements that are not used ?
No, you can't. For dynamic allocation, you can only free or delete memory that was allocated with malloc or new. The exact same amount with the exact same pointer. For automatic variables, the memory will be freed automatically.
But since this is C++, use a std::vector instead. Please.
It depends on how you got your array in the first place.
If it is an array that is allocated in the automatic or static storage (i.e. a local or a global) there is nothing you can free, because you did not allocate anything (the compiler did it for you).
If this is a dynamically allocated array, you can achieve the same effect by creating a smaller array with only eight elements, copying the original values into it, and then freeing the original array. This does not guarantee that the amount of memory allocated to your program would necessarily go down, because the allocator of the eight-element array is allowed to allocate space for more elements. If the numbers are 10000 and 8000, on the other hand, you will almost certainly get some savings (although the standard does not guarantee it either).
What are the advantages of using dynamic memory allocation to create an array compared to the ordinary allocation of elements in an array?
You don't have to know the size of the array in advance, or over-allocate memory to account for large arrays. This allows your program to be more efficient with its memory use.
Arrays created dynamically can typically be larger than those created automatically, and can have longer lifetimes. But both have numerous disadvantages compared to using a std::vector.
1) Dynamic memory allocation allows for lots of liberty when it comes to managing the lifecycle of an object.
2) The size of the array can also be controlled with more liberty.
int x[100]; is fixed size and you can not expand it. Its lifetime is tied to the context where it was created and can not be passed around different functions/methods.
int *x = new int[n]; ... delete[] x; can be reallocated so it can resize and n does not have to be known in the compile time (so you can ask user how many numbers do she need and create an array of that size). As pointed by #Neil Butterworth, this is creating array on the heap and can be larger in size, while the static variant is creating array on the stack.
std::vector wraps a lot of this magic reallocation code and probably this is something you should use in your code.
The basic advantages of a dynamically allocated array are that you can determine the size of a dynamically allocated array at run-time, and, because the array is on the heap rather than the stack, it can be larger than a static array. It is important to remember though, that std::vector does this work for you. The occasions on which you should roll your own array class are few and far between. Use std::vector,
When you are allocating arrays then size of the array has to be given at the compile time. In that case most of the times, either you are allocating more memory or less memory.
This is where dynamic memory allocation helps you out so that you can actually allocate only those chunks of memory that are really needed and as and when you are done with the memory management you can free the memory.
In nutshell, it helps programmer to create Array of size which user says.