I am trying to write a small memory leak detection tool.
My idea is to track the dynamic memory allocation life with in my application
to determine any invalid access of memory or un deleted memory which might cause my application to core over a time of use.
I want to write a simple interface to override new and delete.
And in my overridden new i wanted to print function line address etc. And then call the standard new.
Have anyone already tried this? I am not sure whether i can call standard new from my class specific new operator.
There's a couple of articles here:
http://www.codeproject.com/Articles/8448/Memory-Leak-Detection
http://www.codeproject.com/Articles/19361/Memory-Leak-Detection-in-C
http://www.codeproject.com/Articles/9815/Visual-Leak-Detector-Enhanced-Memory-Leak-Detectio
http://www.codeproject.com/Articles/393957/Cplusplus-Memory-Leak-Finder
In this question i have proposed an ad-hoc solution to your problem: Immediate detection of heap corruption errors on Windows. How?
In general, you can replace your new and delete with this code:
DWORD PageSize = 0;
inline void SetPageSize()
{
if ( !PageSize )
{
SYSTEM_INFO sysInfo;
GetSystemInfo(&sysInfo);
PageSize = sysInfo.dwPageSize;
}
}
void* operator new (size_t nSize)
{
SetPageSize();
size_t Extra = nSize % PageSize;
nSize = nSize + ( PageSize - Extra );
return Ptr = VirtualAlloc( 0, nSize, MEM_COMMIT, PAGE_READWRITE);
}
void operator delete (void* pPtr)
{
MEMORY_BASIC_INFORMATION mbi;
VirtualQuery(pPtr, &mbi, sizeof(mbi));
// leave pages in reserved state, but free the physical memory
VirtualFree(pPtr, 0, MEM_DECOMMIT);
DWORD OldProtect;
// protect the address space, so noone can access those pages
VirtualProtect(pPtr, mbi.RegionSize, PAGE_NOACCESS, &OldProtect);
}
to determine invalid access of memory. In the further discussion there you will find ideas for leaks detection and detection of other memory errors.
If you want to call global new and delete, you can use :: global namespace prefix:
return ::new(nSize);
Calling ::operator new from a class-specific operator new is fine, and quite common. You can't call global operator new from a replacement version of ::operator new because if you replace the global operator new the old one doesn't exist.
The described approach sounds reasonable. You can enhance it and store some bits of identifying data on each allocation into some data structure. This way you can track all the non-freed memory upon program termination without the need to inspect all allocation/deallocation logs.
In order to detect corruptions you can wrap each allocation with paddings before the start and after the end and fill it with some predefined pattern. Upon memory freeing you can verify that the patterns are still in place. This will give you a good chance of detecting corruption.
You can call global new with scope prefix: ::new.
Related
I have build a set C++ containing classes on top of the BluetoothAPIs apis.
I can enumerate open handles to services, characteristics and descriptors. I can read characteristic values. The issue that I have is that I cannot write to a characteristic value.
Below is the code use to write the characteristic value
void BleGattCharacteristic::setValue(UCHAR * data, ULONG size){
if (pGattCharacteristic->IsSignedWritable || pGattCharacteristic->IsWritable || pGattCharacteristic->IsWritableWithoutResponse)
{
size_t required_size = sizeof(BTH_LE_GATT_CHARACTERISTIC_VALUE) + size;
PBTH_LE_GATT_CHARACTERISTIC_VALUE gatt_value = (PBTH_LE_GATT_CHARACTERISTIC_VALUE)malloc(required_size);
ZeroMemory(gatt_value, required_size);
gatt_value->DataSize = (ULONG)size;
memcpy(gatt_value->Data, data, size);
HRESULT hr = BluetoothGATTSetCharacteristicValue(bleDeviceContext.getBleServiceHandle(), pGattCharacteristic, gatt_value, NULL, BLUETOOTH_GATT_FLAG_NONE);
free(gatt_value);
if (HRESULT_FROM_WIN32(S_OK) != hr)
{
stringstream msg;
msg << "Unable to write the characeristic value. Reason: ["
<< Util.getLastError(hr) << "]";
throw BleException(msg.str());
}
}
else
{
throw BleException("characteristic is not writable");
}}
The call to bleDeviceContext.getBleServiceHandle() returns the open handle to the device info service.
pGattCharacteristics is the pointer to the characteristic to write too. It was opened with a call to BluetoothGATTGetCharacteristics.
I have tried different combinations of the flags with no difference in the return code.
I have also tried using the handle to the device not to the service. In that case I get an ERROR_INVALID_FUNCTION return error code.
I would appreciate any pointers as to what I am doing wrong or what other possible options I could try.
1- You have to use the Service Handle, right.
2- I don't know how you designed your class, and then how you allocate some memory for the Characteristic's Value itself.
What I do (to be sure to have enough and proper memory for Value's data):
a) at init of the Value object, call ::BluetoothGATTGetCharacteristicValue twice, to get the needed size and then actually allocate some internal memory for it.
b) when using it, set the inner memory to what it may , then call ::BluetoothGATTSetCharacteristicValue
hr=::BluetoothGATTSetCharacteristicValue(
handle,
(PBTH_LE_GATT_CHARACTERISTIC)Characteristic,
value,//actually a (PBTH_LE_GATT_CHARACTERISTIC_VALUE) to allocated memory
0,//BTH_LE_GATT_RELIABLE_WRITE_CONTEXT ReliableWriteContext,
BLUETOOTH_GATT_FLAG_NONE)
So a few things:
typedef struct _BTH_LE_GATT_CHARACTERISTIC_VALUE {
ULONG DataSize;
UCHAR Data[];
} BTH_LE_GATT_CHARACTERISTIC_VALUE, *PBTH_LE_GATT_CHARACTERISTIC_VALUE;
is how the data structure used in the parameter CharacteristicValue is defined. Please note that Data is NOT an allocated array, but rather a pointer. So accessing Data[0] is undefined behavior and could be accessing anywhere in memory. Rather you need to do gatt_value.Data = &data; setting the pointer to the address of the input parameter.
Secondly the documentation is quite clear as to why you might get ERROR_INVALID_FUNCTION; if another reliable write is already pending then this write will fail. You should consider retry logic in that case.
As for E_INVALIDARG I'd assume it's related to the undefined behavior but I'd check after fixing the other issues previously mentioned.
I'm currently developing application using gSoap library and has some misunderstanding of proper usage library. I has generated proxy object (-j flag) which wrapped my own classes, as you can see below. Application must work 24/7 and connect simultaneously to many cameras (~50 cameras), so after every request i need to clear all temporary data. Is it normal usage to call soap_destroy() and soap_end() after every request? Because it seem's overkill to do it after each request. May be exists another option of proper usage?
DeviceBindingProxy::destroy()
{
soap_destroy(this->soap);
soap_end(this->soap);
}
class OnvifDeviceService : public Domain::IDeviceService
{
public:
OnvifDeviceService()
: m_deviceProxy(new DeviceBindingProxy)
{
soap_register_plugin(m_deviceProxy->soap, soap_wsse);
}
int OnvifDeviceService::getDeviceInformation(const Access::Domain::Endpoint &endpoint, Domain::DeviceInformation *information)
{
_tds__GetDeviceInformation tds__GetDeviceInformation;
_tds__GetDeviceInformationResponse tds__GetDeviceInformationResponse;
setupUserPasswordToProxy(endpoint);
m_deviceProxy->soap_endpoint = endpoint.endpoint().c_str();
int result = m_deviceProxy->GetDeviceInformation(&tds__GetDeviceInformation, tds__GetDeviceInformationResponse);
m_deviceProxy->soap_endpoint = NULL;
if (result != SOAP_OK) {
Common::Infrastructure::printSoapError("Fail to get device information.", m_deviceProxy->soap);
m_deviceProxy->destroy();
return -1;
}
*information = Domain::DeviceInformation(tds__GetDeviceInformationResponse.Manufacturer,
tds__GetDeviceInformationResponse.Model,
tds__GetDeviceInformationResponse.FirmwareVersion);
m_deviceProxy->destroy();
return 0;
}
}
To ensure proper allocation and deallocation of managed data:
soap_destroy(soap);
soap_end(soap);
You want to do this often to avoid memory to fill up with old data. These calls remove all deserialized data and data you allocated with the soap_new_X() and soap_malloc() functions.
All managed allocations are deleted with soap_destroy() followed by soap_end(). After that, you can start allocating again and delete again, etc.
To allocate managed data:
SomeClass *obj = soap_new_SomeClass(soap);
You can use soap_malloc for raw managed allocation, or to allocate an array of pointers, or a C string:
const char *s = soap_malloc(soap, 100);
Remember that malloc is not safe in C++. Better is to allocate std::string with:
std::string *s = soap_new_std__string(soap);
Arrays can be allocated with the second parameter, e.g. an array of 10 strings:
std::string *s = soap_new_std__string(soap, 10);
If you want to preserve data that otherwise gets deleted with these calls, use:
soap_unlink(soap, obj);
Now obj can be removed later with delete obj. But be aware that all pointer members in obj that point to managed data have become invalid after soap_destroy() and soap_end(). So you may have to invoke soap_unlink() on these members or risk dangling pointers.
A new cool feature of gSOAP is to generate deep copy and delete function for any data structures automatically, which saves a HUGE amount of coding time:
SomeClass *otherobj = soap_dup_SomeClass(NULL, obj);
This duplicates obj to unmanaged heap space. This is a deep copy that checks for cycles in the object graph and removes such cycles to avoid deletion issues. You can also duplicate the whole (cyclic) managed object to another context by using soap instead of NULL for the first argument of soap_dup_SomeClass.
To deep delete:
soap_del_SomeClass(obj);
This deletes obj but also the data pointed to by its members, and so on.
To use the soap_dup_X and soap_del_X functions use soapcpp2 with options -Ec and -Ed, respectively.
In principle, static and stack-allocated data can be serialized just as well. But consider using the managed heap instead.
See https://www.genivia.com/doc/databinding/html/index.html#memory2 for more details and examples.
Hope this helps.
The way memory has to be handled is described in Section 9.3 of the GSoap documentation.
I'm writing a function that looks like this:
Nan::MaybeLocal<v8::Object> getIconForFile(const char * str) {
NSImage * icon = [[NSWorkspace sharedWorkspace] iconForFile:[NSString stringWithUTF8String:str]];
NSData * tiffData = [icon TIFFRepresentation];
unsigned int length = [tiffData length];
//TODO this is causing `malloc: *** error for object 0x10a202000: pointer being freed was not allocated`
char * iconBuff = (char *)[tiffData bytes];
Nan::MaybeLocal<v8::Object> ret = Nan::NewBuffer(iconBuff, length);
return ret;
}
It works as expected except when it gets run by node.js, it throws malloc: *** error for object 0x10a202000: pointer being freed was not allocated. I've tried different things using malloc, etc but nothing is working. I understand that Nan::NewBuffer is trying to free the buffer data somehow and that's where the problem is coming from. Maybe the iconBuff variable is allocated to the stack and when it goes out of scope and Nan::NewBuffer tries to free it, it's freeing a null pointer? I'm not sure and I'm sort of lost :(
Here's the code that "fixed" it but #uliwitness pointed out in his answer that it still has memory management problems:
Nan::MaybeLocal<v8::Object> getIconForFile(const char * str) {
NSImage * icon = [[NSWorkspace sharedWorkspace] iconForFile:[NSString stringWithUTF8String:str]];
NSData * tiffData = [icon TIFFRepresentation];
unsigned int length = [tiffData length];
char * iconBuff = new char[length];
[tiffData getBytes:iconBuff length:length];
Nan::MaybeLocal<v8::Object> ret = Nan::NewBuffer(iconBuff, length);
return ret;
}
Here's the specific code I ended up going with based on #uliwitness' answer:
Nan::MaybeLocal<v8::Object> getIconForFile(const char * str) {
#autoreleasepool {
NSImage * icon = [[NSWorkspace sharedWorkspace] iconForFile:[NSString stringWithUTF8String:str]];
NSData * tiffData = [icon TIFFRepresentation];
unsigned int length = [tiffData length];
return Nan::CopyBuffer((char *) [tiffData bytes], length);
}
}
This seems to be the most elegant solution and from some quick and dirty testing, seems to result in a smaller resident set size in node.js over many invocations, for whatever that's worth.
I googled for documentation about Nan::NewBuffer, and found this page: https://git.silpion.de/users/baum/repos/knoesel-mqttdocker-environment/browse/devicemock/node_modules/nan/doc/buffers.md
This says:
Note that when creating a Buffer using Nan::NewBuffer() and an
existing char*, it is assumed that the ownership of the pointer is
being transferred to the new Buffer for management. When a
node::Buffer instance is garbage collected and a FreeCallback has not
been specified, data will be disposed of via a call to free(). You
must not free the memory space manually once you have created a Buffer
in this way.
So the code in your own answer is incorrect, because a buffer created using new char[] needs to be disposed of using delete [] (this is different from delete, FWIW), however you're giving it to a function that promises to call free on it. You're just accidentally turning off the error message, not fixing the error.
So at the least, you should use malloc instead of new there.
This page also mentions another function, Nan::CopyBuffer(), which it describes as:
Similar to Nan::NewBuffer() except that an implicit memcpy will occur
within Node. Calls node::Buffer::Copy(). Management of the char* is
left to the user, you should manually free the memory space if
necessary as the new Buffer will have its own copy.
That sounds like a better choice for your original code. You can pass it the NSData's bytes and it will even do the copying for you. You don't have to worry about "manually freeing the memory space", as the memory is owned by the NSData, which will take care of disposing of it when it is released (If you're using ARC, ARC will release the NSData, if you're not ARC, you should add an #autoreleasepool so it gets released).
PS - At the bottom of that page, it also mentions that you could use a Nan::FreeCallback and pass it to Nan::NewBuffer. If you provide one there that calls delete [] on the buffer it is given, the code from your answer would work as well. But really, why write additional code for something Nan::CopyBuffer will apparently already do for you?
I am analyzing a memory dump created by debugdiag. It shows CreateErrorinfo method call which leads to memory leak like below,
I am using proper map files for mydll and myanotherdll both. What is the meaning of CreateErrorInfo ? how it's leading to memory leak?
Function Source Destination
mfc90u!operator new+33
mfc90u!CPlex::Create+1f mfc90u!operator new
kernel32!TlsSetValueStub
kernel32!TlsSetValueStub
MYANOTHERDLL!CreateErrorInfo+188e2
MYDLL!MyClas::OnTimer+a3 ......\myfile.cpp # 4639
MYDLL!CMainFrame::OnTimer+71 ......\mainfrm.cpp # 1246
mfc90u!CWnd::OnWndMsg+407
mfc90u!AfxCallWndProc+a3
user32!MDIClientWndProcW
mfc90u!__sse2_available_init+657b
mfc90u!CWnd::WindowProc+24
mfc90u!AfxCallWndProc+a3
mfc90u!AfxWndProc+37 mfc90u!AfxCallWndProc
mfc90u!AfxWndProcBase+56 mfc90u!AfxWndProc
mfc90u!AfxWndProcBase
Is this related to not releasing an interface? Interface from CreatorErroInfo must be released by client:
ICreateErrorInfo* pErrorInfo = nullptr;
HRESULT hr = ::CreateErrorInfo(&pErrorInfo);
if (pErrorInfo)
{
pErrorInfo->Release();
}
Even better to use ATL's smart pointers:
CComPtr<ICreateErrorInfo> ptrErrorInfo;
HRESULT hr = ::CreateErrorInfo(&ptrErrorInfo);
if (ptrErrorInfo)
{
//no release necessary
}
CreateErrorInfo creates an instance of a generic error object.
This function returns a pointer to a generic error object, which you can use with QueryInterface on ICreateErrorInfo to set its contents. I believe you should check the state of ICreateErrorInfo pointer for more details in your code.
I have a code that has a large number of mallocs and device-specific API mallocs (I'm programming on a GPU, so cudaMalloc).
Basically my end of my beginning of my code is a big smorgasbord of allocation calls, while my closing section is deallocation calls.
As I've encapsulated my global data in structures, the deallocations are quite long, but at least I can break them into a separate function. On the other hand, I would like a shorter solution. Additionally an automatic deallocator would reduce the risk of memory leaks created if I forget to explicitly write the deallocation in the global allocator function.
I was wondering whether it'd be possible to write some sort of templated class wrapper that can allow me to "register" variables during the malloc/cudaMalloc process, and then at the end of simulation do a mass loop-based deallocation (deregistration). To be clear I don't want to type out individual deallocations (free/cudaFrees), because again this is long and undesirable, and the assumption would be that anything I register won't be deallocated until the device simulation is complete and main is terminating.
A benefit here is that if I register a new simulation duration variable, it will automatically deallocate, so there's no danger of me forgetting do deallocate it and creating a memory leak.
Is such a wrapper possible?
Would you suggest doing it?
If so, how?
Thanks in advance!
An idea:
Create both functions, one that allocates memory and provides valid pointers after register them in a "list" of allocated pointers. In the second method, loop this list and deallocate all pointers:
// ask for new allocated pointer that will be registered automatically in list of pointers.
pointer1 = allocatePointer(size, listOfPointers);
pointer2 = allocatePointer(size, listOfPointers);
...
// deallocate all pointers
deallocatePointers(listOfPointers);
Even, you may use different listOfPointers depending of your simulation scope:
listOfPointer1 = getNewListOfPointers();
listOfPointer2 = getNewListOfPointers();
....
p1 = allocatePointer(size, listOfPointer1);
p2 = allocatePointer(size, listOfPointer2);
...
deallocatePointers(listOfPointers1);
...
deallocatePointers(listOfPointers2);
There are many ways to skin a cat, as they say.
I would recommend thrust's device_vector as a memory management tool. It abstracts allocation, deallocation, and memcpy in CUDA. It also gives you access to all the algorithms that Thrust provides.
I wouldn't recommend keeping random lists of unrelated pointers as Tio Pepe recommends. Instead you should encapsulate related data into a class. Even if you use thrust::device_vector you may want to encapsulate multiple related vectors and operations on them into a class.
The best choice is probably to use the smart pointers from C++ boost library, if that is an option.
If not, the best you can hope for in C is a program design that allows you to write allocation and deallocation in one place. Perhaps something like the following pseudo code:
while(!terminate_program)
{
switch(state_machine)
{
case STATE_PREOPERATIONAL:
myclass_init(); // only necessary for non-global/static objects
myclass_mem_manager();
state_machine = STATE_RUNNING;
break;
case STATE_RUNNING:
myclass_do_stuff();
...
break;
...
case STATE_EXIT:
myclass_mem_manager();
terminate_program = true;
break;
}
void myclass_init()
{
ptr_x = NULL;
ptr_y = NULL;
/* Where ptr_x, ptr_y are some of the many objects to allocate/deallocate.
If ptr is a global/static, (static storage duration) it is
already set to NULL automatically and this function isn't
necessary */
}
void myclass_mem_manager()
{
ptr_x = mem_manage (ptr_x, items_x*sizeof(Type_x));
ptr_y = mem_manage (ptr_y, items_y*sizeof(Type_y));
}
static void* mem_manage (const void* ptr, size_t bytes_n)
{
if(ptr == NULL)
{
ptr = malloc(bytes_n);
if (ptr == NULL)
{} // error handling
}
else
{
free(ptr);
ptr = NULL;
}
return ptr;
}