I have a vector<vector<double>> elem and I want to deallocate its memory many times in my program.
I tried using
vector<vector<double>>().swap(elem);
Or even a for cicle
for(int i=0; i<elem.size();i++)
vector<double>().swap(elem[i]);
vector<vector<double>>().swap(elem);
elem.resize(dim, vector<double>(0));
(I want the first dimension to be a certain number dim)
But when I call
cout<<elem[0].size();
numerous times in my program, the output keeps growing, even if I've just used the aforementioned method. This issue isn't present with the "main" size of the vector.
i.e.
cout<<elem.size();
always outputs dim
EDIT: I know about clear() but I want to deallocate the vector, shrink_to_fit() doesn't work either. Also this is implemented in a function out of the main one, as follows:
void arrayReset(vector<vector<double>> elem) {
for(int i=0; i<elem.size();i++)
vector<double>().swap(elem[i]);
vector<vector<double>>().swap(elem);
elem.resize(dim, vector<double>(0));
}
Your new function void arrayReset(vector<vector<double>> elem) { gets a COPY of your vector and [possibly] cleans it; you never see it in the calling function.
If you pass your vector by reference, you would manipulate the original vector.
How to free memory for vector
The way is the same for all vectors regardless of the element type.
Step 1: Remove the elements of the vector. Simplest way is the clear member function. After this step, the size member function will return 0.
Step 2: Call shrink_to_fit member function which requests the memory to be deallocated. After this step, capacity may return 0.
Technically, shrink_to_fit is a request that is not required to be honoured by the language implementation. The only guaranteed way to deallocate the memory is to destroy the vector. Example:
{
std::vector<std::vector<double> vector;
// use vector here
}
// memory has been deallocated
I want to deallocate its memory many times in my program.
Note that this is typically slower than not deallocating many times. I recommend making sure that you want something that is actually useful.
I came across having the need to create 2 or 3-level nested vectors but I had this issue of memory not clearing correctly. So I made this simple test:
void test(){
vector<vector<double>> myContainer;
vector<double> vec = {1};
for (int i=0; i<20000000; ++i)
{
myContainer.push_back(vec);
}
FreeAll(myContainer);
}
where FreeAll() is a template function defined as follows:
template <typename T>
void FreeAll( T & t ) {
T tmp;
t.swap(tmp);
}
Now, invoking function test() in main(), we will find out that a lot of left over memory is still there even after leaving the test function's scope and that memory is not cleared up until the main terminates.
Could be the reason for this that I create too much vectors? also there is no any kind of memory leak here as all the storage is automatic.
In this very context, there is no any sort of memory leaks and memory observations occurred from Ubuntu's default system monitor.
So calling test() 4 times in main() and check if the last 3 calls would allocate more memory or reuse it from the previous allocation and it turns out that no more memory would be allocated and the memory is indeed reusable and it will be only freed once the program terminates. On the contrary, creating a vector of double and pushing the same number of doubles and call FreeAll(), it will actually free the memory instantly and it's returned to the OS.
I have a class Set:
class Set
{
public:
//Default constructor
Set ();
//Some more functions...
private:
int *p;
const int K = 10;
int numval = 0; //Number of ints in the array
//Other variables...
};
The default constructor:
Set::Set()
{
p = new int[K]; //Allocate memory for array with 10 ints
}
If I in some other function would fill the array with 10 ints and then add an other one, what would happen? The compiler doesn't crash and I'm able to print the 11:th int. But since I havn't allocated memory for it, where is it stored?
Example:
Set1 += 5;
Would add 5 to the array with the following operator overloader.
const Set& Set::operator+=(const int x)
{
p[numval] = x; //Add next int after the last int in the array
numval++; //Increment number of ints
return *this;
}
If I in some other function would fill the array with 10 ints and then add an other one, what would happen?
You'd write into whatever memory came after the end of the array, causing undefined behaviour: perhaps causing no obvious problems, perhaps corrupting some unrelated data (or the metadata used to manage the heap), or perhaps crashing if there was no writable memory there.
But since I havn't allocated memory for it, where is it stored?
It isn't stored anywhere, in the sense of having storage allocated for it. There's just nothing to stop you writing to arbitrary memory locations beyond the end of an array. Be careful not to do that.
Computer memory is linear. It's one huge row of cells (bytes). Every cell has 2 neighbours (except the first and the last ones, obviously). Allocating memory is just an act of telling "this part is mine". It's really nothing more than a promise: you promise to not write outside your plot and in return you get promise noone else would write inside it. So what happens when you write outside of your allocated area? You break your promise. There may be someone's else's plot right next to yours, there might be unused space. Nothing really happens when you write outside your area. Real problem arises when rightful owner comes back and tries to pick up what he left - and it turns out to be something else, something you put there. (Of course it's possible that your plot lies next to something system considers important. In that case, OS stations guards on the border, and they shot to kill any trespassers on sight.)
It is your job as a programmer to make your program keep it's promises. When processes break their promises, bad things may or may not happen - to them or to other processes.
I recently found this won't work in my global CUDA C++ code that I plan to compile and later to be called in Matlab:
int M = 10; float V[M];
or if I were to import M value from the matlab host code.
But this works:
float V[10];
I was told there exists a function called new that I can use to avoid this problem, but I read online and am still quite confused how to use this new function, and it seems only to apply to host code, is that right? If so, it won't apply to my case then, since my host code is in matlab. Is this a way to get around this, so that I don't have to change vector lengths one by one? Thank you!
I don't know anything about MATLAB or CUDA, but your problem is in C++. Arrays declared like that must have sizes fixed at compile-time.
Solution 1: Fix the size
Declare your variable M const. These are equivalent:
int const M = 10;
const int M = 10;
The compiler would then know that it can assume these variables will always have the same value no matter how you run the program.
Solution 2: C-style dynamic allocation
Dynamic allocation with new and delete. Arrays allocated on the abstract section of memory called the "free-store" (rather than on the "stack", like those arrays you have) can determine their sizes on the fly. You use it like this:
float * V = new V[M]; //V is a pointer to some freestore memory
//You use it and pass it like you would a normal array:
V[2] = 5.5;
int x = some_func(V);
//But after you're done, you should manually free the memory
delete [] V; //don't forget the [], since you used [] in the allocation
I don't recommend this, because of the possiblity of forgetting to delete the memory.
Solution 3: Automatic memory management with C++'s vector
In C++, the work of memory management can be hidden behind structures called classes.
#include<vector>
using std::vector;
vector<float> V(M); //V is a vector of floats, with initial size M
//You use it like a normal array
V[2] = 5.5;
//But to pass it as an array, you need to pass a pointer to the first element
int x = some_func(&V[0]); //read as &(V[0]): pass in the address of V[0]
Solution 3b: CUDA-compatible vector
Thrust is a C++ template library for CUDA based on the Standard Template Library (STL). Thrust allows you to implement high performance parallel applications with minimal programming effort through a high-level interface that is fully interoperable with CUDA C.
http://docs.nvidia.com/cuda/thrust/#vectors
Conclusion
If you're using fixed sizes, I recommend solution 1. If you're using sizes determined during runtime, I recommend vector.
(By the way, when you pass an ordinary array to a function, you are actually passing a pointer to the first element, NOT the array. The name of the array is automatically converted to a pointer type.)
Hy all,
I believe that the following piece of code is generating memory leak?
/* External function to dynamically allocate a vector */
template <class T>
T *dvector(int n){
T *v;
v = (T *)malloc(n*sizeof(T));
return v;
}
/* Function that calls DVECTOR and, after computation, frees it */
void DiscontinuousGalerkin_Domain::computeFaceInviscidFluxes(){
int e,f,n,p;
double *Left_Conserved;
Left_Conserved = dvector<double>(NumberOfProperties);
//do stuff with Left_Conserved
//
free(Left_Conserved);
return;
}
I thought that, by passing the pointer to DVECTOR, it would allocate it and return the correct address, so that free(Left_Conserved) would successfully deallocate. However, it does not seem to be the case.
NOTE: I have also tested with new/delete replacing malloc/free without success either.
I have a similar piece of code for allocating a 2-D array. I decided to manage vectors/arrays like that because I am using them a lot, and I also would like to understand a bit deeper memory management with C++.
So, I would pretty much like to keep an external function to allocate vectors and arrays for me. What's the catch here to avoid the memory leak?
EDIT
I have been using the DVECTOR function to allocate user-defined types as well, so that is potentially a problem, I guess, since I don't have constructors being called.
Even though in the piece of code before I free the Left_Conserved vector, I also would like to otherwise allocate a vector and left it "open" to be assessed through its pointer by other functions. If using BOOST, it will automatically clean the allocation upon the end of the function, so, I won't get a "public" array with BOOST, right? I suppose that's easily fixed with NEW, but what would be the better way for a matrix?
It has just occurred me that I pass the pointer as an argument to other functions. Now, BOOST seems not to be enjoying it that much and compilation exits with errors.
So, I stand still with the need for a pointer to a vector or a matrix, that accepts user-defined types, that will be passed as an argument to other functions. The vector (or matrix) would most likely be allocated in an external function, and freed in another suitable function. (I just wouldn't like to be copying the loop and new stuff for allocating the matrix everywhere in the code!)
Here is what I'd like to do:
template <class T>
T **dmatrix(int m, int n){
T **A;
A = (T **)malloc(m*sizeof(T *));
A[0] = (T *)malloc(m*n*sizeof(T));
for(int i=1;i<m;i++){
A[i] = A[i-1]+n;
}
return A;
}
void Element::setElement(int Ptot, int Qtot){
double **MassMatrix;
MassMatrix = dmatrix<myT>(Ptot,Qtot);
FillInTheMatrix(MassMatrix);
return;
}
There is no memory leak there, but you should use new/delete[] instead of malloc/free. Especially since your function is templated.
If you ever want to to use a type which has a non-trivial constructor, your malloc based function is broken since it doesn't call any constructors.
I'd replace "dvector" with simply doing this:
void DiscontinuousGalerkin_Domain::computeFaceInviscidFluxes(){
double *Left_Conserved = new double[NumberOfProperties];
//do stuff with Left_Conserved
//
delete[] Left_Conserved;
}
It is functionally equivalent (except it can call constructors for other types). It is simpler and requires less code. Plus every c++ programmer will instantly know what is going on since it doesn't involve an extra function.
Better yet, use smart pointers to completely avoid memory leaks:
void DiscontinuousGalerkin_Domain::computeFaceInviscidFluxes(){
boost::scoped_array<double> Left_Conserved(new double[NumberOfProperties]);
//do stuff with Left_Conserved
//
}
As many smart programmers like to say "the best code is the code you don't have to write"
EDIT: Why do you believe that the code you posted leaks memory?
EDIT: I saw your comment to another post saying
At code execution command top shows
allocated memory growing
indefinitely!
This may be completely normal (or may not be) depending on your allocation pattern. Usually the way heaps work is that they often grow, but don't often shrink (this is to favor subsequent allocations). Completely symmetric allocations and frees should allow the application to stabilize at a certain amount of usage.
For example:
while(1) {
free(malloc(100));
}
shouldn't result in continuous growth because the heap is highly likely to give the same block for each malloc.
So my question to you is. Does it grow "indefinitely" or does it simply not shrink?
EDIT:
You have asked what to do about a 2D array. Personally, I would use a class to wrap the details. I'd either use a library (I believe boost has a n-dimmentional array class), or rolling your own shouldn't be too hard. Something like this may be sufficient:
http://www.codef00.com/code/matrix.h
Usage goes like this:
Matrix<int> m(2, 3);
m[1][2] = 10;
It is technically more efficient to use something like operator() for indexing a matrix wrapper class, but in this case I chose to simulate native array syntax. If efficiency is really important, it can be made as efficient as native arrays.
EDIT: another question. What platform are you developing on? If it is *nix, then I would recommend valgrind to help pinpoint your memory leak. Since the code you've provided is clearly not the problem.
I don't know of any, but I am sure that windows also has memory profiling tools.
EDIT: for a matrix if you insist on using plain old arrays, why not just allocate it as a single contiguous block and do simple math on indexing like this:
T *const p = new T[width * height];
then to access an element, just do this:
p[y * width + x] = whatever;
this way you do a delete[] on the pointer whether it is a 1D or 2D array.
There is no visible memory leak, however there is a high risk for a memory leak with code like this. Try to always wrap up resources in an object (RAII).
std::vector does exactly what you want :
void DiscontinuousGalerkin_Domain::computeFaceInviscidFluxes(){
int e,f,n,p;
std::vector<double> Left_Conserved(NumOfProperties);//create vector with "NumOfProperties" initial entries
//do stuff with Left_Conserved
//exactly same usage !
for (int i = 0; i < NumOfProperties; i++){//loop should be "for (int i = 0; i < Left_Conserved.size();i++)" .size() == NumOfProperties ! (if you didn't add or remove any elements since construction
Left_Conserved[i] = e*f + n*p*i;//made up operation
}
Left_Conserved.push_back(1.0);//vector automatically grows..no need to manually realloc
assert(Left_Conserved.size() == NumOfProperties + 1);//yay - vector knows it's size
//you don't have to care about the memory, the Left_Conserved OBJECT will clean it up (in the destructor which is automatically called when scope is left)
return;
}
EDIT: added a few example operations. You really should read about STL-containers, they are worth it !
EDIT 2 : for 2d you can use :
std::vector<std::vector<double> >
like someone suggested in the comments. but usage with 2d is a little more tricky. You should first look into the 1d-case to know what's happening (enlarging vectors etc.)
No, as long as you aren't doing anything drastic between the call to your dvector template and the free, you aren't leaking any memory. What tells you there is a memory leak?
May I ask, why you chose to create your own arrays instead of using STL containers like vector or list? That'd certainly save you a lot of trobule.
I don't see memory leak in this code.
If you write programs on c++ - use new/delete instead malloc/free.
For avoid possible memory leaks use smart pointers or stl containers.
What happens if you pass a negative value for n to dvector?
Perhaps you should consider changing your function signature to take an unsigned type as the argument:
template< typename T >
T * dvector( std::size_t n );
Also, as a matter of style, I suggest always providing your own memory release function any time you are providing a memory allocation function. As it is now, callers rely on knowledge that dvector is implemented using malloc (and that free is the appropriate release call). Something like this:
template< typename T >
void dvector_free( T * p ) { free( p ); }
As others have suggested, doing this as an RAII class would be more robust. And finally, as others have also suggested, there are plenty of existing, time-tested libraries to do this so you may not need to roll your own at all.
So, some important concepts discussed here helped me to solve the memory leaking out in my code. There were two main bugs:
The allocation with malloc of my user-defined types was buggy. However, when I changed it to new, leaking got even worse, and that's because one my user-defined types had a constructor calling an external function with no parameters and no correct memory management. Since I called that function after the constructor, there was no bug in the processing itself, but only on memory allocation. So new and a correct constructor solved one of the main memory leaks.
The other leaking was related to a buggy memory-deallocation command, which I was able to isolate with Valgrind (and a bit a patience to get its output correctly). So, here's the bug (and, please, don't call me a moron!):
if (something){
//do stuff
return; //and here it is!!! =P
}
free();
return;
And that's where an RAII, as I understood, would avoid misprogramming just like that. I haven't actually changed it to a std::vector or a boost::scoped_array coding yet because it is still not clear to me if a can pass them as parameter to other functions. So, I still must be careful with delete[].
Anyway, memory leaking is gone (by now...) =D