I have a question regarding vectors used in c++. I know unlike array there is no limit on vectors. I have a graph with 6 million vertices and I am using vector of class. When I am trying to insert nodes into vector it is failing by saying bad memory allocation. where as it is working perfectly over 2 million nodes. I know bad allocation means it s failing due to pointers I am using in my code but to me this does not seems the case. My question is it possible that it is failing due to the large size of graph as limit on vector is increased. If it is is there any way we can increase that Limit.
First of all you should verify how much memory a single element requires. What is the size of one vertex/node? (You can verify that by using the sizeof operator). Consider that if the answer is, say, 50 bytes, you need 50 bytes times 6 million vertices = 300 MBytes.
Then, consider the next problem: in a vector the memory must be contiguous. This means your program will ask the OS to give it a contiguous chunk of 300 MBytes, and there's no guarantee this chunk is available even if the available memory is more than 300 MB. You might have to split your data, or to choose another, non-contiguous container. RAM fragmentation is impossible to control, which means if you run your program and it works, maybe you run it again and it doesn't work (or vice versa).
Another possible approach is to resize the vector manually, instead of letting it choose its new size automatically. The vector tries to anticipate some future growth, so if it has to grow it will try to allocate more capacity than is needed. This extra capacity might be the difference between having enough memory and not having it. You can use std::vector::reserve for this, though I think the exact behaviour is implementation dependent - it might still decide to reserve more than the amount you have requested.
One more option you have is to optimize the data types you are using. For example, if inside your vertex class you are using 32-bit integers while you only need 16 bits, you might use int16_t which would take half the space. See the full list of fixed size variables at CPP Reference.
There is std::vector::max_size that you can use to see the maximum number of elements the the vector you declared can potentially hold.
Return maximum size
Returns the maximum number of elements that the
vector can hold.
This is the maximum potential size the container can reach due to
known system or library implementation limitations, but the container
is by no means guaranteed to be able to reach that size: it can still
fail to allocate storage at any point before that size is reached.
Related
This is in continuation of my last question. I am failed to understand the memory taken up by vector. Problem skeleton:
Consider an vector which is an collection of lists and lists is an collection of pointers. Exactly like:
std::vector<std::list<ABC*> > vec;
where ABC is my class. We work on 64bit machines, so size of pointer is 8 bytes.
At the start of my flow in the project, I resize this vector to an number so that I can store lists at respective indexes.
vec.resize(613284686);
At this point, capacity and size of the vector would be 613284686. Right. After resizing, I am inserting the lists at corresponding indexes as:
// Some where down in the program, make these lists. Simple push for now.
std::list<ABC*> l1;
l1.push_back(<pointer_to_class_ABC>);
l1.push_back(<pointer_to_class_ABC>);
// Copy the list at location
setInfo(613284686, l1);
void setInfo(uint64_t index, std::list<ABC*> list>) {
std::copy(list.begin(), list.end(), std::back_inserter(vec.at(index));
}
Alright. So inserting is done. Notable things are:
Size of vector is : 613284686
Entries in the vector is : 3638243731 // Calculated this by going over vector indexes and add the size of std::lists at each index.
Now, since there are 3638243731 entries of pointers, I would expect memory taken by this vector is ~30Gb. 3638243731 * 8(bytes) = ~30Gb.
BUT BUT When I have this data in memory, memory peaks to, 400G.
And then I clear up this vector with:
std::vector<std::list<nl_net> >& ccInfo = getVec(); // getVec defined somewhere and return me original vec.
std::vector<std::list<nl_net> >::iterator it = ccInfo.begin();
for(; it != ccInfo.end(); ++it) {
(*it).clear();
}
ccInfo.clear(); // Since it is an reference
std::vector<std::list<nl_net> >().swap(ccInfo); // This makes the capacity of the vector 0.
Well, after clearing up this vector, memory drops down to 100G. That is too much holding from an vector.
Would you all like to correct me what I am failing to understand here?
P.S. I can not reproduce it on smaller cases and it is coming in my project.
vec.resize(613284686);
At this point, capacity and size of the vector would be 613284686
It would be at least 613284686. It could be more.
std::vector<std::list<nl_net> >().swap(ccInfo); // This makes the capacity of the vector 0.
Technically, there is no guarantee by the standard that a default constructed vector wouldn't have capacity other than 0... But in practice, this is probably true.
Now, since there are 3638243731 entries of pointers, I would expect memory taken by this vector is ~30Gb. 3638243731 * 8(bytes)
But the vector doesn't contain pointers. It contains std::list<ABC*> objects. So, you should expect vec.capacity() * sizeof(std::list<ABC*>) bytes used by the buffer of the vector itself. Each list has at least a pointer to beginning and the end.
Furthermore, you should expect each element in each of the lists to use memory as well. Since the list is doubly linked, you should expect about two pointers plus the data (a third pointer) worth of memory for each element.
Also, each pointer in the lists apparently points to an ABC object, and each of those use sizeof(ABC) memory as well.
Furthermore, since each element of the linked lists are allocated separately, and each dynamic allocation requires book-keeping so that they can be individually de-allocated, and each allocation must be aligned to the maximum native alignment, and the free store may have fragmented during the execution, there will be much overhead associated with each dynamic allocation.
Well, after clearing up this vector, memory drops down to 100G.
It is quite typical for the language implementation to retain (some) memory it has allocated from the OS. If your target system documents an implementation specific function for explicitly requesting release of such memory, then you could attempt using that.
However, if the vector buffer wasn't the latest dynamic allocation, then its deallocation may have left a massive reusable area in the free store, but if there exists later allocations, then all that memory might not be releasable back to the OS.
Even if the langauge implementation has released the memory to the OS, it is quite typical for the OS to keep the memory mapped for the process until another process actually needs the memory for something else. So, depending on how you're measuring memory use, the results might not necessarily be meaningful.
General rules of thumb that may be useful:
Don't use a vector unless you use all (or most) of the indices. In case where you don't, consider a sparse array instead (there is no standard container for such data structure though).
When using vector, reserve before resize if you know the upper bound of allocation.
Don't use linked lists without a good reason.
Don't rely on getting all memory back from peak usage (back to the OS that is; The memory is still usable for further dynamic allocations).
Don't stress about virtual memory usage.
std::list is a fragmented memory container. Typically each node MUST have the data it is storing, plus the 2 prev/next pointers, and then you have to add in the space required within the OS allocation table (typically 16 or 32 bytes per allocation - depending on OS). You then have to account for the fact all allocations must be returned on a 16byte boundary (on Intel/AMD based 64bit machines anyway).
So using the example of std::list<ABC*> the size of a pointer is 8, however you will need at least 48bytes to store each element (at least).
So memory usage for ONLY the list entries is going to be around: 3638243731 * 48(bytes) = ~162Gb.
This is of course assuming that there is no memory fragmentation (where there may be a block of 62bytes free, and the OS returns the entire block of 62 rather than the 48 requested). We are also assuming here that the OS has a minimum allocation size of 48 bytes (and not say, 64bytes, which would not be overly silly, but would push the usage up far higher).
The size of the std::lists themselves within the vector comes to around 18GB. So in total we are looking at 180Gb at least to store that vector. It would not be beyond the realm of possibility that the extra allocations are additional OS book keeping info, for all of those individual memory allocations (e.g. lists of loaded memory pages, lists of swapped out memory pages, the read/write/mmap permissions, etc, etc).
As a final note, instead of using swap on a newly constructed vector, you can just use shrink to fit.
ccInfo.clear();
ccInfo.shrinkToFit();
The main vector needs some more consideration. I get the impression it will always be a fixed size. So why not use a std::array instead? A std::vector always allocates more memory than it needs to allow for growth. The bigger your vector the bigger the reservation of memory to allow for more even growth. The reasononing behind is to keep relocations in memory to a minimum. Relocations on really big vectors take up huge amounts of time so a lot of extra memory is reserved to prevent this.
No vector function that can delete elements (such as vector::clear and ::erase) also deallocates memory (e.g. lower the capacity). The size will decrease but the capacity doesn't. Again, this is meant to prevent relocations; if you delete you are also very likely to add again. ::shrink_to_fit also doesn't guarantuee you that all of the used memory is released.*
Next is the choice of a list to store elements. Is a list really applicable? Lists are strong in random access/insertion/removal operations. Are you really constantly adding and removing ABC objects to the list in random locations? Or is another container type with different properties but with contiguous memory more suitable? Another std::vector or std::array perhaps. If the answer is yes than you're pretty much stuck with a list and its scattered memory allocations. If no, than you could win back a lot of memory by using a different container type.
So, what is it you really want to do? Do you really need dynamic growth on both the main container and its elements? Do you really need random manipulation? Or can you use fixed-size arrays for both container and ABC objects and use iteration instead? When contemplating this you might want to read up on the available containers and their properties on en.cppreference.com. It will help you decide what is most appropriate.
*For the fun of it I dug around in VS2017's implementation and it creates an entirely new vector without the growth segment, copies the old elements and then reassigns the internal pointers of the old vector to the new one while deleting the old memory. So at least with that compiler you can count on memory being released.
C++.
I want to create an array and store elements in it. And I can estimate upper bound for amount of elements that would be true for most cases, let's say about 98% of cases. Is it better in terms of speed and beauty to create static array with size equal to upper bound instead of dynamic allocation?
To be more specific, let's say that amount of elements varying between 10000 and 60000 and 2 bytes per element. And in very rare cases that amount can be higher than 60000(in such case I'll have to make reallocation).
Is it okay to statically allocate array of size 60000 and use part of it, and in some cases reallocate to bigger size, or this practice is too ugly?
Just use std::vector, which lets you resize as needed and reserve an estimated maximum size to improve performance. You can even make this static/global if you want to, but then you have the usual issues with global variables.
Unless you are creating new instances of the vector very frequently, which I doubt with such a large size, this should have good performance.
The template std::vector is made to exactly tackle such problems. It will automatically allocate new memory for new elements, ensuring efficient use of memory, and preventing time expensive operations, like reallocation, as well.
So I was running a memory usage test in my program where I was adding 20 elements each to two separate vectors each frame (~60fps). I expected that at some point I would start seeing a memory leak, but instead memory usage remained constant until a certain critical point. Around 700,000 elements total it spiked and then leveled off again at the new plateau.
I have a feeling this has something to do with the allocation of the vectors being automatically increased at that point, but I'm not sure and can't find anything online. It also wouldn't really explain why so much extra memory was allocated at that point (Private Bytes on the CPU jumped from ~800 to ~900, and System GPU Memory jumped from ~20 to ~140). Here are the Process Explorer graphs for both CPU and GPU:
Note: the dips in both CPU and GPU usage are from me pausing the program after seeing the spike.
Can anyone explain this to me?
Edit: Here's a simpler, more general test:
The total usage is obviously a lot lower, but same idea.
When you add an element to an empty vector, it will allocate enough space via new for several elements. Like 16 maybe. It does this because resizing the array to a larger buffer is slow, so it allocates more than it needs. If it allocates room for 16 elements, that means you can push back 15 more before it needs to bother with another call to new. Each time it grows significantly more. If you have 500 elements (and it's out of room) and you push back one more, it may allocate room for 750. Or maybe even 1000. Or 2000. Plenty of room.
It turns out that when you (or the vector) call new, you get this from the program's memory manager. If the program's memory manager doesn't have enough memory available, it will ask the operating system for a HUGE amount of memory, because operating system calls are themselves slow, and messing with page mappings is slow. So when vector asks for room for 200 bytes, the program's memory manager may actually grab like 65536 bytes, and then give the vector only 200 of that, and save the remaining 65336 bytes for the next call(s) to new. Because of this, you (or vector) can call new many many times before you have to bother the operating system again, and things go quickly.
But this has a side effect: The operating system can't actually tell how much memory your program is really using. All it knows is that you allocated 65536 from it, so it reports that. As you push back elements in the vector, eventually the vector runs out of capacity, and asks more from the program's memory manager. And as it does that more and more, and the operating system reports the same memory usage, because it can't see. Eventually the memory manager runs out of capacity, and asks for more from the operating system. The operating system allocates another huge block (65536? 131072?) and you see a sudden large spike in memory usage.
The vector size at which this happens isn't set, it depends on what else has also been allocated, and what order they were allocated and deallocated. Even the things that you deleted still affect things, it's pretty darn complicated. Also, the rate at which vector grows depending on your library implementation, and the amount of memory the program's memory manager grabs from the OS also varies depending on factors even I don't know about.
I have no idea why the GPU's memory would spike, it depends on what you were doing with your program. But note that there is less GPU memory total, it's entirely possible it grew by a smaller amount than "private bytes".
Vectors use a dynamically allocated array to store their elements.This array may need to be reallocated in order to grow in size when new elements are inserted, which implies allocating a new array and moving all elements to it. This is a relatively expensive task in terms of processing time, and thus, vectors do not reallocate each time an element is added to the container. Instead, vector containers may allocate some extra storage to accommodate for possible growth, and thus the container may have an actual capacity greater than the storage strictly needed to contain its elements (i.e., its size). This explains your plateaus. The capacity enlargement is done by doubling the current one. It may be the case that after a finite number of doubling, its quadrupling the capacity. Which will explain your spike.
Vector Size and Capacity
For good performance by allocating more memory than ever needed.
size()
Returns the current number of elements
empty()
Returns whether the container is empty (equivalent to 0==size() but faster)
capacity()
Returns the maximum possible number of elements without reallocation
reserve(num)
Enlarges capacity, if not enough yet
The capacity of a vector is important, because
Reallocation invalidates all references, pointers, and iterators for elements.
Reallocation takes time by moving all elements to new heap location.
Reallocation size increment depends on the actual vector implementation.
Code example for using reserve(num):
std::vector<int> v1; // Create an empty vector
v1.reserve(80); // Reserve memory for 80 elements
I have an issue with a C++ program. I think it's a problem of memory.
In my program i'm used to create some enormous std::vector (i use reserve to allocate some memory). With vector size of 1 000 000, it's ok but if i increase this number (about ten millions), my program will freeze my PC and i can do nothing except waiting for a crash (or end of the program if i'm lucky). My vector contains a structure called Point which contain a vector of double.
I used valgrind to check if there is a memory lack. But no. According to it, there is no problem. Maybe using a vector of objects is not advised ? Or maybe is there some system parameters to check or something ? Or simply, the vector is too big for the computer ?
What do you think about this ?
Disclaimer
Note that this answer assumes a few things about your machine; the exact memory usage and error potential depends on your environment. And of course it is even easier to crash when you don't compute on 2d-Points, but e.g. 4d-points, which are common in computer graphics for example, or even larger Points for other numeric purposes.
About your problem
That's quite a lot of memory to allocate:
#include <iostream>
#include <vector>
struct Point {
std::vector<double> coords;
};
int main () {
std::cout << sizeof(Point) << std::endl;
}
This prints 12, which is the size in bytes of an empty Point. If you have 2-dimensional points, add another 2*sizeof(double)=8 to that per element, i.e. you now have a total of 20 bytes per Point.
With 10s of millions of elements, you request 200s of millions of bytes of data, e.g. for 20 million elements, you request 400 million bytes. While this does not exceed the maximum index into an std::vector, it is possible that the OS does not have that much contiguous memory free for you.
Also, your vectors memory needs to be copied quite often in order to be able to grow. This happens for example when you push_back, so when you already have a 400MiB vector, upon the next push_back you might have your old version of the vector, plus the newly allocated 400MiB*X memory, so you may easily exceed the 1000MiB temporarilly, et cetera.
Optimizations (high level; preferred)
Do you need to actually store the data all time? Can you use a similar algorithm which does not require so much storage? Can you refactor your code so that storage is reduced? Can you core some data out when you know it will take some time until you need it again?
Optimizations (low level)
If you know the number of elements before creating your outer vector, use the std::vector constructor which you can tell an initial size:
vector<Foo> foo(12) // initialize have 12 elements
Of course you can optimize a lot for memory; e.g. if you know you always only have 2d-Points, just have two doubles as members: 20 bytes -> 16 bytes. When you do not really need the precision of double, use float: 16 bytes -> 8 bytes. That's an optimization to $2/5$:
// struct Point { std::vector<double> coords; }; <-- old
struct Point { float x, y; }; // <-- new
If this is still not enough, an ad-hoc solution could be std::deque, or another, non-contiguous container: No temporal memory "doubling" because no resizing needed; also no need for the OS to find you such contiguous block of memory.
You can also use compression mechanisms, or indexed data, or fixed point numbers. But it depends on your exact circumstances.
struct Point { signed char x, y; }; // <-- or even this? examine a proper type
struct Point { short x_index, y_index; };
Without seeing your code, this is just speculation, but I suspect it is in large part due to your attempt to allocate a massive amount of memory that is contiguous. std::vector is guaranteed to be in contiguous memory, so if you try to allocate a large amount of space, the operating system has to try to find a block of memory that large that it can use. This may not be a problem for 2MB, but if you are suddenly trying to allocate 200MB or 2GB of contiguous memory ...
Additionally, anytime you add a new element to the vector and it is forced to resize, all of the existing elements must be copied into the new space allocated. If you have 9 million elements and adding the 9,000,001 element requires a resize, that is 9 million elements that have to be moved. As your vector gets larger, this copy time takes longer.
Try using std::deque instead. It is will basically allocate pages (that will be contiguous), but each page can be allocated wherever it can fit.
When using vector, "Out of Memory" show.
To fix it, I use max_size() to check, then reserve or push_back.
If the max_size() is bigger the reserved value, it should be ok, but it is no! Then what's the meaning of max_size()?
I compile below demo in windows 7 and Visual studio 2010. My PC has 4GB RAM. When the reseverd is 1/2 of max_size(), it fails.
max_size()=2^32/sizeof(CPoint3D)-1=268435455
It's ok when 1/4 of max_size() in the demo. In my real project, It's ok until 1/10.
What's the right of max size of vector, how to enlarge it?
I got the "out of memory" error when I push a lot of elements into std::vector. To avoid the error, I checked with vector::max_size() first, and use vector::reserve() to pre-allocate the memory. However, it doesn't work. In a demo project, the error occurs when I reserve 1/4 of the max_size. In the real project, the error occurs when I reserve 1/10 of it. I'm running Windows 7 and compiling with Visual Studio 2010. My computer has 4GB RAM.
If max_size doesn't work, how do I find out the maximum number of elements I can allocate for a vector?
max_size() returns the maximum number of elements a vector can possibly hold. That is, the absolute limit when taking into account things like the addressing limits using the integral types it might store and the address space limits of the operating system.
This doesn't mean you can actually make a vector hold that many elements. It just means you can never store more. Also just because you have 4 gigs of RAM doesn't mean you can actually create a single contiguous buffer that occupies 4 gigs of RAM or anywhere close. There are other factors to consider like memory fragmentation (you might only be able to page a one gig memory block into physical memory due to it).
If you really need this many elements in a container, a contiguous sequence is probably not a good choice. For data sets that large, you may need something that can be paged in bits and pieces like std::deque.
max_size() tells you the design limit of the class, but memory shortage can limit the real size to something smaller. There's not generally any way to find what the lower limit might be though (e.g., it might change from one moment to another, depending on how much memory is used by other programs).
The problem is that vector tries to allocate a contiguous block of memory, which might not be available at that time, even though the total available memory may be much larger.
I would suggest to use std::deque as it does not require to allocate a contiguous block of memory.
vector::capacity() gives the maximum number of elements that can be stored in the vector without having a re-allocation, one which can potentially fail from std::bad_alloc.
vector::max_size() has a different meaning, roughly similar to (INT_MAX / sizeof(element)).
For more information on Windows memory management, see the MSDN article