I came across having the need to create 2 or 3-level nested vectors but I had this issue of memory not clearing correctly. So I made this simple test:
void test(){
vector<vector<double>> myContainer;
vector<double> vec = {1};
for (int i=0; i<20000000; ++i)
{
myContainer.push_back(vec);
}
FreeAll(myContainer);
}
where FreeAll() is a template function defined as follows:
template <typename T>
void FreeAll( T & t ) {
T tmp;
t.swap(tmp);
}
Now, invoking function test() in main(), we will find out that a lot of left over memory is still there even after leaving the test function's scope and that memory is not cleared up until the main terminates.
Could be the reason for this that I create too much vectors? also there is no any kind of memory leak here as all the storage is automatic.
In this very context, there is no any sort of memory leaks and memory observations occurred from Ubuntu's default system monitor.
So calling test() 4 times in main() and check if the last 3 calls would allocate more memory or reuse it from the previous allocation and it turns out that no more memory would be allocated and the memory is indeed reusable and it will be only freed once the program terminates. On the contrary, creating a vector of double and pushing the same number of doubles and call FreeAll(), it will actually free the memory instantly and it's returned to the OS.
Related
In C++, the importance of deallocating memory when the program is either exiting or no longer serves a purpose is important. So if this is allocation of a dynamic array
char** dynamicArr = nullptr;
for (int i = 0; i < x; i++) {
mapPtr[i] = new char[y];
}
and this is deallocation of a dynamic array
for (int i = 0; i < x; i++) {
delete[] mapPtr[i];
}
delete[] mapPtr;
However, when it comes to vectors, I noticed that my global vector with 0 elements inside seems to be causing some memory leaks.
I've read up on this link, a user commented that
No. The std::vector will automatically de-allocate the memory it uses
Screenshot of my debugging.
I have also tried these steps to clear the vector as well as make sure the vector inside the struct citySummInfo has shrunk to fit and clear hopefully not getting any memory leak but to no avail. Is there any way that I'm doing it wrong?
As what #JohnFilleau have mentioned
_CrtDumpMemoryLeaks() should be called at the point in the program where you want to see what is remaining on the heap. Since your
vectors are statically allocated, they will not have been destroyed at
the time you call this function.
_CrtDumpMemoryLeaks() is meant to place right before the program terminates, and since my vectors are statically allocated, it has not been deallocated at the time when _CrtDumpMemoryLeaks() has been called hence the "leaks".
Suppose I have a forever loop to create hashmap:
void createMap() {
map<int, int> mymap;
for (int i = 0; i < INT_MAX; i++) {
mymap[i] = i;
}
mymap.clear(); // <-- this line doesn't seem to make a difference in memory growth
}
int main (void) {
while (1) {
createMap();
}
return 0;
}
I watched the code run and on MacOS, watching the Activity Monitor, the application keeps growing the memory usage with or without the mymap.clear() at end of the createMap() function.
Shouldn't memory usage be constant for the case where mymap.clear() is used?
What's the general recommendation for using STL data containers? Need to .clear() before end of function?
I asked in another forum, the folks there helped me understand the answer. It turns out, I didn't wait long enough to exit createMap function nor do I have enough memory to sustain this program.
It takes INT_MAX=2147483647 elements to be created, and for each map = 24 bytes element of pair<int, int> = 8 bytes.
Total minimum memory = 2.147483647^9 * 8 + 24 = 17179869200 bytes ~= 17.2 GB.
I reduced the size of the elements and tested both with and without .clear() the program grew and reduce in size accordingly.
The container you create is bound to the scope of your function. If the function returns, its lifetime ends. And as std::map owns its data, the memory it allocates is freed upon destruction.
Your code hence constantly allocates and frees the same amount of memory. Memory consumption is hence constant, although the exact memory locations will probably differ. This also means that you should not manually call clear at the end of this function. Use clear when you want to empty a container that you intend to continue using afterwards.
As a side note, std::map is not a hash map (std::unordered_map is one).
I'm running into a problem where I use too much memory on the stack. I'm using several large arrays that I only need between steps in my code. Basically I need to know how to release the memory used by an array variable that's created as:
float arrayName[length][width];
To intentionally release some auto storage (items on the 'stack'), you can do the following - basically you simply limit the scope of your variables
change code from:
//...
float arrayName[length][width];
// ...
change code to:
//...
{
float arrayName[length][width];
// use arrayName here
//... still in-scope
} // scope limit
// all of arrayName released from stack
{
// stack is available for other use, so try
uint32_t u32[3][length][width];
// use u32 here
//... still in-scope
} // scope ended
// all of u32 released from stack
// better yet, use std::vector or another container
std::vector<uint32_t> bigArry;
NOTE: a vector uses a finite amount of stack (24 bytes on my system),
regardless of how many elements you put into it!
You should use vectors for things like this. It is a part of the C++ standard library and is very optimized in most implementations. The memory taken up by the vector will automatically get released when the vector goes out of scope. So you will never have to free up the memory yourself.
Another benefit with using a vector is that you do not have to worry about running out of stack space since all the "array" memory taken up by the vector is located on the heap of the program.
For examples http://en.cppreference.com/w/cpp/container/vector/vector
Other than that if you think your program memory is never going to be enough then you should consider using the disk as another storage mechanism. Databases work this way. They store most of their data on disk.
You won't need any special statements.
The array will be released on function return or exiting the scope if it is local variable having automatic storage duration, or on exiting the program if it is static variable (declared outside functions).
You may want to allocate the memory on the heap if you are running into a situation where you are running out of memory on the stack. In this case you'll want to new up the array.
float** my_array = new float* [rowCount];
for(int i = 0; i < rowCount; ++i)
{
my_array[i] = new float[columnCount];
}
// and delete it later
for(int i = 0; i < rowCount; ++i)
{
delete [] my_array[i];
}
delete [] my_array;
I am gritting a big numerical analysis program where I NEED to optimize memory hardly.
In my program there is some points where I need to compute a matrix and then use it in some functions. Let's call the matrix f. f is winSize x winSize, a variable that is calculated previous of f creations.
I have several functions, this are the important ones:
double** getF(some params to get f,int winSize){
double** f=new double*[winSize];
for(int i=0;i<winSize;i++)
f[i]=new double[winSize];
/*
Some stuff to fill f
*/
}
void freeF(double** f,int winSize){
for(int i=0;i<winSize;i++)
delete [] f[i];
delete [] f;
}
And the program goes like:
int winSize=computeWindowSize();
double** f=getF(someparams,winSize);
hiIamaFunctionThatWantsToReadF(params...,f,winSize);
soIwant(params2...,f,winSize);
iamJustCurious(f,winSize);
freeF(f,winSize)
Note that nobody modifies f, just reads it.
My problem is that this does NOT free the memory. I think the problem is in the way I handle f, but not sure....
My questions are:
Why is this not working?
What should I do to make it work?
EDIT:
How do I check that the memory is not freed?
I have to run this piece of code 1000x1400 times, and in few iterations the program crashes because there is no more free RAM memory in my computer... That may be a sign of not freed memory.
Then I tried to create memory OUTSIDE the function and then deleting OUTSIDE the function.
In that way I can see memory being allocated and deleted. But once inside the functions I can see the allocation but never the dealocation.
it appears that there is too little information to say what your code doesn't work.
maybe it's in the code not shown.
but you can just use a construct guaranteed to work.
instead of
double** f=getF(someparams,winSize);
do
std::vector<double> f( winSize*winSize );
then instead of
f[y][x]
do
f[y*winSize + x]
of course you can define a function to do that indexing
and of course you can wrap it in a matrix class
but essentially, that's one way to make things work :)
note that the std::vector takes care both of allocation and deallocation, and it can also be resized and copied if you want
Let's have a function func which allocates memory for vec. Also let's func is called many times in a loop.
void func ()
{
std::vector<int> vec;
vec.resize(someNumber);
}
for (int i=0; i<someNumber; ++i)
{
func ();
}
How I can allocate memory for vec only once in a loop?
There are two possible solutions I can see. One is to have the function take a reference to an std::vector<int> that it can use.
void func (std::vector<int> & vec)
{
vec.resize(someNumber);
}
Another possibility is to create a functor that uses a private vector, and can therefore reuse the allocation, assuming it is called multiple times.
class SomeFunctor
{
public:
void operator()();
private:
std::vector<int> vec;
}
void SomeFunctor::operator()()
{
vec.resize(someNumber);
}
This can be used like this:
SomeFunctor func;
for (int i=0; i<someNumber; ++i)
{
func ();
}
Note that calling resize() on the vector means that the vector will allocate new storage for the existing elements, copy the existing elements over, and deallocate the old storage. This defeats the entire purpose of holding onto a vector object, as the vector's internal allocation will never be reused! Consider using reserve() instead, which will never shrink (but may enlarge) the allocation.
The optimal solution might be none of these approaches, but something far simpler. Your example code has not given enough detail to know if these approaches will be beneficial.
As a side note, remember that premature optimization is a bad thing. Code first, then run a profiler and see where the bottlenecks are -- usually they will be around collecting user input or other things like network and disk I/O. I'm not saying that you shouldn't try to predict possible bottlenecks and engineer around them, but don't excessively worry about every possible function call and allocation/deallocation.
If this is a working application and not a prototype, run a profiler. You are very likely to find that any bottlenecks lie elsewhere.
As Beta said, every time the function func() is called a new vector is created and is destroyed when the function exits. If you want to have a single vector which gets resized which each call, you may want to do something like
std::vector<int> vec;
for (int i=0; i<someNumber; ++i)
{
vec.resize(i);
}
or you can direcly initialise your vector with the max memory you will need as
std::vector<int> vec(someNumber);