I have some code that contains a self-made hashtable using calloc and malloc for memory allocation. I would like to modify these parts using a shared_ptr with a custom deleter that frees automatically the allocated memory.
The code is part of the mmseg chinese segmenter algorithm, it is working great but is such a mess as it leaves memory leaks. I am considering to rewrite that code using a unordered_map or such, but for now I would like to make these changes.
I read the answers on similar questions, such as shared_ptr with malloc and free or Accessing calloc'd data through a shared_ptr, but I have problems to use that on the code below.
I have these lines where I am not able to wrap the calls with the smart pointer. So maybe somebody can help me out with this:
struct Word {
unsigned char nbytes; /* number of bytes */
char length; /* number of characters */
unsigned short freq;
char text[word_embed_len];
};
struct Entry {
Word *word;
Entry *next;
};
static Entry **new_bins = static_cast<Entry **>(std::calloc(init_size,
sizeof(Entry *)));
Entry *entry, ...;
...
new_bins[hash_val] = entry;
....
free(new_bins);
The above calloc call I would feed shared pointer with the result of calloc, such as
std::shared_ptr<Entry *> binsPtr(new_bins, freePtr());
I am not shure if this is correct.
mmseg uses a pool allocation routine with malloc(), which looks like this:
inline void *pool_alloc(int len) {
void *mem = _pool_base;
if (len <= _pool_size) {
_pool_size -= len;
_pool_base += len;
return mem;
}
_pool_base = static_cast<char *>(std::malloc(REALLOC_SIZE));
mem = _pool_base;
_pool_base += len;
_pool_size = REALLOC_SIZE - len;
return mem;
}
The allocator then is called like this:
Entry *entry = bins[h];
...
entry = static_cast<Entry *>(pool_alloc(sizeof(Entry)));
entry->word = word;
entry->next = NULL;
bins[h] = entry;
Is it possible to modify the pool_alloc routine such as I could wrap the malloc() with a shared pointer an define a custom deleter (maybe even skip the complete pool_alloc fct and just use a shared_ptr), something like
std::shared_ptr<Entry> entry((Entry *)malloc(sizeof(Entry)), freePtr());
struct freePtr {
void operator()(void* x) {
free(x);
}
};
Would be great if somebody could help me out on this. Thanks in advance!
Update:
I coded a simple memory pool class for my problem, so all pointers get destroyed automatically. The wrapped calloc() in the shared_ptr seems to work fine and works as expected. Valgrind reports no more memory leaks and errors.
OP writes:
I coded a simple memory pool class for my problem, so all pointers get destroyed automatically. The wrapped calloc() in the shared_ptr seems to work fine and works as expected. Valgrind reports no more memory leaks and errors.
In other words, changing the code fixed the bugs. :) This question could safely be deleted at this point.
Related
After a few years, I discovered a memory leak bug in my code. Unfortunately the bug was not causing any issues to be noticed until I found out about it indirectly.
Below is a function addElement() from class c_string which adds a new element to a chain. My old way of doing it is like this:
class c_string
{
private:
char *chain;
size_t chain_total;
public:
void addElement(char ch);
};
void c_string::addElement(char ch)
{
char *tmpChain = new char[chain_total+1];
for(size_t i=0;i<chain_total;i++)
tmpChain[i] = chain[i];
tmpChain[chain_total++] = ch;
delete chain;
chain = new char[chain_total];
chain = tmpChain;
}
I found out that by doing chain = tmpChain; I am causing a memory leak.
My question is, what is the best way to handle it?
Please note, in this class I am not using any STL function, because I am simply writing my own string.
The best way to do it is simply drop the second allocation, it serves no purpose.
void c_string::addElement(char ch)
{
char *tmpChain = new char[chain_total+1];
for(size_t i=0;i<chain_total;i++)
tmpChain[i] = chain[i];
tmpChain[chain_total++] = ch;
delete[] chain;
chain = tmpChain;
}
The above is correct and even has the strong exception guarantee.
Of course even if you do not want to use std::string, std::unique_ptr would is still safer than raw new+delete and you would get the rule of five for free instead of having to implement it on your own.
From performance standpoint, you might be interested in Why is it common practice to double array capacity when full? and of course std::memcpy or std::copy instead of the manual loop.
I would like experts review on the following dynamic memory allocation process and suggest whether there are any memory leaks. Following code is not real code in use, but trying understand concept of memory allocations and de-allocation in different ways.
class ObjMapData
{
private:
int* itsMapData;
........
public:
ObjMapData();
~ObjMapData(){if(itsMapData!= NULL) delete[] itsMapData;}
ClearMemory() {if(itsMapData!= NULL) {delete[] itsMapData; itsMapData= NULL}}
.......
void SetMapData(int* ptrData) { itsMapData = ptrData;} // or should I use int*&ptrData??
int* GetMapData() const { return itsMapData;}
}
Now can I do the following without any memory leaks?
bool Function1(ObjMapData& objMyMap)
{
//populate the ptrData with some data values using many different ways
int* ptrData = new int[1000]; // usually a variable from binary file header
......................
objMyMap.SetMapData(ptrData);
//don't want to delete the memory yet
return true;
}
bool Function2(ObjMapData& objMyMap)
{
int* ptrData = objMyMap.GetMapData();
//do some work such as writing updated data into a binary file
}
bool Function3(ObjMapData& objMyMap)
{
//populate the data
bool bStatus = Function1(objMyMap);
int* ptrData = objMyMap.GetMapData();
//update the map data using ptrData variable
..........
bStatus = Function2(objMyMap); // write data to a binary file
objMyMap.ClearMemory(); // not real code in use, but for understanding the concept
bStatus = Function1(objMyMap); // re-allocate memory
ptrData = objMyMap.GetMapData();
//update the map data using ptrData variable
objMyMap.SetMapData(ptrData); // Do I need to set again or member pointer get updated automatically?
return true
}
int main()
{
ObjMapData objMyMap;
bool bStatus = Function3(objMyMap);
//default destructor of objMyMap can take care of the allocated memory cleanup
return 0;
}
Thank you for your time to confirm the dynamic memory allocation..
Although this may seem to be more to do with style than your question about memory leaks, I would handle the data privately within the class:
class ObjMapData
{
private:
int* itsMapData;
// consider adding 'datasize' member variable
........
public:
ObjMapData(){ itsMapData=NULL; }; // initialise member variable(s)!
~ObjMapData(){ delete[] itsMapData;}
ClearMemory() { delete[] itsMapData; itsMapData=NULL; }
.......
void CreateMapData(int size) { ClearMemory(); itsMapData= new int[size]; }
void FillDataFrom( ???? ) { ???? };
int* GetMapData() const { return itsMapData;}
}
You are then in a better position to improve the class by adding copy constructor and assignment methods which will prevent memory leaks when you use the class.
EDIT
You ask:
My concern here is which of the following is right: void
SetMapData(int* ptrData) Vs void SetMapData(int*&ptrData)
Both are 'right' in the sense that both allow the external (to the class) pointer to be copied and used within your class - with respect to 'memory leaks' it depends on which part of your code you want to manage the memory you allocated. You could:
Have a class handle allocation/deallocation internally
Allocate memory, use some class to manipulate it, deallocate memory outside class
Have a class allocate memory and later deallocate it outside the class
Allocate memory and have some class manipulate and deallocate it.
Usually I find 1 and 2 make more sense than 3 or 4. i.e. it is easier to follow what is going on, less likely to hide errors and so on.
However, as far as 'leaking memory' is concerned: it does not matter where the pointer to an allocated memory block is, how it has been copied, assigned or referenced - it is it's value as a memory address which is important. So, as long as you new and delete that memory address correctly you will not leak memory (whether those actions are inside a class or not).
If, in your application, you need to allocate/deallocate the int array external to your class, it does make some sense for the member functions reference the pointer as a hint to the reader that the class is not responsible for its deallocation - but some decent comments should make that clear anyway :)
Over the years I've come across umpteen bugs due to the mishandling of the "passing of ownership" of allocated memory (more so with good ol 'C') where some piece of code has been written assuming either that it has to free a block or someone else will do it.
Does that answer your question or have I missed the point?
i saw some code like below, but i didn't see any delete statement, is there any memory leak problem?
struct RStatus
{
int fid;
int status;
};
void Foo()
{
vector<RStatus*> rsVec;
RStatus* rs = null;
rs = new RStatus(); // memory allocated here!
rs->fid = 0
rs->status = 0;
rsVec.push_back(rs);
}
If you use vector<RStatus*>, then you have to use delete, otherwise you will have a memory leak.
However, if you use vector<RStatus>, then you don't have to use delete — this is recommended1.
1. If you want to use pointers, then the recommendation is that you should be using smart pointers such as std::unique_ptr, or std::shared_ptr.
Yes, you should free your memory allocated :
struct RStatus
{
int fid;
int status;
};
void Foo()
{
vector<RStatus*> rsVec;
RStatus* rs = null;
rs = new RStatus(); // memory allocated here!
rs->fid = 0
rs->status = 0;
rsVec.push_back(rs);
// free :
unsigned int size = rsVec.size();
for (unsigned int i = 0; i < size; i++ )
delete rsVec[i]; // delete because you used new
}
If you don't do that, all the memory will never be released at the vector destruction.
I would suggest you to use std::vector<RStatus> instead of std::vector<RStatus*>.
You may also used smart ptr. You can found some documentation about it here : http://www.cplusplus.com/reference/memory/shared_ptr/
EDIT: As suggested in comments, if an exception is thrown at rsVec.push_back(rs), the memory allocated will be lost, that's why smart pointers would be a better solution. Or again, use std::vector<RStatus> instead.
Yes, there is a memory leak: the pointer to the created structure is lost after the vector is destroyed, and the memory is never released.
Unless someone performs a delete for each element of rsVec before clearing or destroying the vector.
Yes, that code leaks the RStatus.
It also does nothing else: possibly the real code's vector gets passed to some function that takes posession of the vector's contents.
Tracking down memory leaks is generally not a local problem: every use of that pointer has to, in theory, be examine to figure out if it leaks. Techniques like 'if I allocate it, delete it' and RAII (including smart pointers) attempt to make it more local, so you can tell from an incomplete program if there is a leak.
use boost::shared_ptr if you don't want to bother yourself with a deletion of allocated objects.
http://www.boost.org/doc/libs/1_54_0/libs/smart_ptr/shared_ptr.htm
struct RStatus
{
int fid;
int status;
};
void Foo()
{
vector<shared_ptr<RStatus> > rsVec;
shared_ptr<RStatus> rs = shared_ptr<RStatus>(); // empty shared_ptr
rs.reset(new RStatus()); // memory allocated here!
rs->fid = 0
rs->status = 0;
rsVec.push_back(rs); // shared_ptr is copied
}// vector is destroyed and shared_ptrs also
be careful however not to mixed up things using both shared_ptr and ordinary, raw pointers to avoid situation when shared_ptr tries to delete object which is already deleted
I heard a lot of memory leak vulnerability, but I could not find a real work example of memory leak, could you provide a real work example of memory leak, maybe of some big open source project and explain the solution to me
thx.
It's really simple actually. In your main put:
char* c = new char[4];
Then exit. That's a memory leak. Any new that doesn't get followed by delete is a leak.
This answer has some good examples, but like my comment has said, it will be fairly hard to find a released application with a leak that an outside observer can look at and easily identify.
I am screaming, cursing and yelling everyday about code like this in our (huge) legacy code base:
// returns raw pointer with changing conventions who's the owner...
HelpFoo* Foo::GetFoo(Bar* pBar, OtherFoo* pFoo)
{
// all 'local' variables even those allocated on freestore declared
// and initialized in a large block at the beginning of the function/method
HelpFoo *A = new HelpFoo;
OtherHelpFoo *B, *C;
EvenMore *D = new EvenMore;
// and so on, these blocks can be huge...
// a complicated spaghetti code in here, with dozens of nested 'ifs'
if (/* some expression */) {
} else if (/* some other expression */) {
// and so on... then suddenly:
if (/* some other nested expression */) {
// I forgot that I've allocated other memory at the beginning...
return A;
}
}
// some miserably written logic here and suddenly
if (D) delete D; return A;
// call to some other function with cryptical name without any
// kind of knowledge what happens with the resource:
FooTakesReferenceToPointer(&A);
// suddenly returning something completely different
// what should I free, A, D...?
return C;
}
I tried to write in comments what the problems are. Clearly, forget about exceptions. The spaghetti code is so bad that nobody can tell what the logic actually is. Therefore it is really, really easy to forget to free memory and that happens very, very frequently. Solution 1: Throw away and rewrite everything. Solution 2: Keep spaghetti as it is, replace all newed resources by smart pointers and make_shared or make_unique, let compiler yell. Of course, first write a test suite (which didn't exist before) to guarantee the same (often screwed) behaviour for all possible sets of inputs (which are not documented).
EDIT
As james said this is undefined behaviourso no promises
You could do something like this:
#include <vector>
class Base
{
public:
Base()
{
baseData = new char [1024];
}
~Base()
{
delete [] baseData;
}
private:
char* baseData;
};
class Derived : public Base
{
public:
Derived()
{
derivedData = new char[1024];
}
~Derived()
{
delete [] derivedData;
}
private:
char* derivedData;
};
int main()
{
std::vector<Base*> datablocks;
datablocks.push_back(new Base());
datablocks.push_back(new Derived());
for(unsigned int i = 0; i < datablocks.size(); ++i)
{
delete datablocks[i];
}
datablocks.clear();
return 0;
}
The data in the Derived class wont be removed here since we are calling delete on a Base* and the Base class does not declare a virtual destructor.
A lot examples could be given here. Just allocate some memory and do not free it.
A good example for this would be the following:
char* pBuffer = new char[ 1024 ]; // or something else, dynamically allocated
// do something here
// now suppose, calling f() throws
f();
// do some other things
delete[] pBuffer;
When f() throws, if the exception is not caught, delete[] will never be executed. Thus, memory leak.
This is one of the best examples why smart pointers should be used.
Another example would be - a function, returning pointer to dynamically allocated memory. The user, often, may forget to free this memory. Something like:
char
char* f()
{
return new char[ 1024 ];
}
//...
// in some other function
char* pSomething = f();
// do some stuff here and return
Imagine you're processing network data and create polymorphic "message objects" based on the data:
while (true)
{
char buf[1024];
size_t len = read_from_network(buf, 1024); // fictitious, for demonstration only
Message * p = Message::Parse(buf, len); // allocates new, dynamic, concrete object
engine.process(p);
}
The engine object may chose to store the object somewhere and use it again later, and if nobody takes care of deleting it, you have a perfect leak.
While the other answers give enough hints, some 'real world' memory leaks which I have seen in our applications.
I don't remember if this was found before or after the release, but, I guess that doesn't matter.
void f()
{
BYTE* b = NULL;
f = open a file;
while (!f.end())
{
int size = getNextRecordSize(f);
b = new BYTE;
readNextRecord(f,b);
process record;
}
delete b;
}
Bit hard to detect this. The reviewers might take it for granted that the memory is deleted properly by seeing the delete call. However, it deletes only the memory allocated for the last record. Rest is leaked.
class A
{
public:
BYTE* get()
{
allocate a new buffer, copy the someData buffer and return that.
The client is expected to delete it
};
private:
BYTE* someData;
};
void f()
{
A a;
B.initialize(a.get()); // It is so convenient to use the pointer. It is not obvious from the function name
// that the result of get has to be deleted.
}
One example I often run across in our code is in image understanding functions, where a temporary 8bit memory is allocated, and never released (yeah, I know, when you do a new, do a delete right afterwards...)
unsigned char* dataToBeUsed = new unsigned char[imgsize];
memcpy(dataToBeUsed, original, imgsize);
// use and process the data here
return value;
The allocated memory is never released -> memory leak. Windows will kill the memory when the application is exited completely, but before that within the application that memory is just lost -> leaked.
A memory leak occurs when the programmer has the memory leak of forgetting to free allocated memory :-)
linebuffer = new char[4096];
/* do things */
/* forget to free memory */
Normally, if you cause a memory leak and then exit the program, it is not harmful, since the operating system normally frees the resources allocated by the program. The problem arises when your application runs for a long period of time (for example, a service). If your program causes memory leaks, then you will run out system's memory, unless the operating system has mechanisms to avoid that; in such case, it will terminate your program.
So, be careful and eat fish: it's very good for memory :-)
To give you a real-world example, a bit of googling turned up this memory leak in 389 Directory Server (a RedHat Open Source product).
Just lose the pointer to dynamically allocated memory:
void foo()
{
int *arr = new int[100];
}
pointer segfault problems...
I've been doing c++ for some weeks meanwhile but i ran again into that issue.
basically i have these classes given. I cant change them. I start with an instance of _ns3__importAuftragResponse kout;
class SOAP_CMAC _ns3__importAuftragResponse
{
public:
ns2__SOAPImportResult *return_;
...
class SOAP_CMAC ns2__SOAPImportResult
{
public:
bool *error;
int *numberOfIgnoreds;
....
My code needs to check for the numberOfIgnoreds
first approach
ns2__SOAPImportResult* imp_result;
imp_result = kout.return_;
int num;
num = *imp_result->numberOfIgnoreds;
or i use
ns2__SOAPImportResult imp_result;
imp_result = *(kout.return_);
int* num;
*num = *imp_result.numberOfIgnoreds;
I mostly get segmentation fault
I know generally what happens at runtime but cant come up with the correct ode. PLease help.
EDIT
made progress thx to your answer, Nawaz , but still need some understanding
ns2__SOAPImportResult * imp_ptr = new ns2__SOAPImportResult;
imp_ptr = kout.return_;
int * num = new (int);
// next line segfaults
*num = *imp_ptr->numberOfIgnoreds;
what's hard for me to understand is, how or why allocate memory for something that is already "there" as there is the member return_ of the object kout
So is it correct to say I need to allocate memory for the variable I assign it to (which is of same type of course)?
Most likely you've not allocated memory for the following members which you're using in the code you've quoted.
ns2__SOAPImportResult *return_; //in the class _ns3__importAuftragResponse
int *numberOfIgnoreds; //in the class ns2__SOAPImportResult
Other than this I don't see anything where things might go wrong!
Make sure you allocate memory for these members (and all other pointers in your program) before using them. You can use new to allocate memory. Or alternatively, you can use malloc() as well. Whatever you use, use it consistently, and deallocate the memory once you done, using delete or free() respectively!
This looks like gsoap. In that case you must use soap_malloc to allocate memory which you return.
For example on the FAQ page, you will find this example:
int ns__itoa(struct soap *soap, int i, char **a)
{ *a = (char*)soap_malloc(soap, 11);
sprintf(*a, "%d", i);
return SOAP_OK;
}