Native node.js addon error pointer being freed was not allocated - c++

I'm writing a function that looks like this:
Nan::MaybeLocal<v8::Object> getIconForFile(const char * str) {
NSImage * icon = [[NSWorkspace sharedWorkspace] iconForFile:[NSString stringWithUTF8String:str]];
NSData * tiffData = [icon TIFFRepresentation];
unsigned int length = [tiffData length];
//TODO this is causing `malloc: *** error for object 0x10a202000: pointer being freed was not allocated`
char * iconBuff = (char *)[tiffData bytes];
Nan::MaybeLocal<v8::Object> ret = Nan::NewBuffer(iconBuff, length);
return ret;
}
It works as expected except when it gets run by node.js, it throws malloc: *** error for object 0x10a202000: pointer being freed was not allocated. I've tried different things using malloc, etc but nothing is working. I understand that Nan::NewBuffer is trying to free the buffer data somehow and that's where the problem is coming from. Maybe the iconBuff variable is allocated to the stack and when it goes out of scope and Nan::NewBuffer tries to free it, it's freeing a null pointer? I'm not sure and I'm sort of lost :(
Here's the code that "fixed" it but #uliwitness pointed out in his answer that it still has memory management problems:
Nan::MaybeLocal<v8::Object> getIconForFile(const char * str) {
NSImage * icon = [[NSWorkspace sharedWorkspace] iconForFile:[NSString stringWithUTF8String:str]];
NSData * tiffData = [icon TIFFRepresentation];
unsigned int length = [tiffData length];
char * iconBuff = new char[length];
[tiffData getBytes:iconBuff length:length];
Nan::MaybeLocal<v8::Object> ret = Nan::NewBuffer(iconBuff, length);
return ret;
}
Here's the specific code I ended up going with based on #uliwitness' answer:
Nan::MaybeLocal<v8::Object> getIconForFile(const char * str) {
#autoreleasepool {
NSImage * icon = [[NSWorkspace sharedWorkspace] iconForFile:[NSString stringWithUTF8String:str]];
NSData * tiffData = [icon TIFFRepresentation];
unsigned int length = [tiffData length];
return Nan::CopyBuffer((char *) [tiffData bytes], length);
}
}
This seems to be the most elegant solution and from some quick and dirty testing, seems to result in a smaller resident set size in node.js over many invocations, for whatever that's worth.

I googled for documentation about Nan::NewBuffer, and found this page: https://git.silpion.de/users/baum/repos/knoesel-mqttdocker-environment/browse/devicemock/node_modules/nan/doc/buffers.md
This says:
Note that when creating a Buffer using Nan::NewBuffer() and an
existing char*, it is assumed that the ownership of the pointer is
being transferred to the new Buffer for management. When a
node::Buffer instance is garbage collected and a FreeCallback has not
been specified, data will be disposed of via a call to free(). You
must not free the memory space manually once you have created a Buffer
in this way.
So the code in your own answer is incorrect, because a buffer created using new char[] needs to be disposed of using delete [] (this is different from delete, FWIW), however you're giving it to a function that promises to call free on it. You're just accidentally turning off the error message, not fixing the error.
So at the least, you should use malloc instead of new there.
This page also mentions another function, Nan::CopyBuffer(), which it describes as:
Similar to Nan::NewBuffer() except that an implicit memcpy will occur
within Node. Calls node::Buffer::Copy(). Management of the char* is
left to the user, you should manually free the memory space if
necessary as the new Buffer will have its own copy.
That sounds like a better choice for your original code. You can pass it the NSData's bytes and it will even do the copying for you. You don't have to worry about "manually freeing the memory space", as the memory is owned by the NSData, which will take care of disposing of it when it is released (If you're using ARC, ARC will release the NSData, if you're not ARC, you should add an #autoreleasepool so it gets released).
PS - At the bottom of that page, it also mentions that you could use a Nan::FreeCallback and pass it to Nan::NewBuffer. If you provide one there that calls delete [] on the buffer it is given, the code from your answer would work as well. But really, why write additional code for something Nan::CopyBuffer will apparently already do for you?

Related

How to return base64 decoded image bytes through a methods formal parameter of type char**

Being new to C++, I am still struggling with pointers-to-pointers and I am not sure if my method below is returning decoded image bytes properly.
This method gets a base64 encoded image string from API. The method has to follow this signature as it is part of legacy code that is not allowed to abbreviate from the way it was written originally. So the signature has to stay the same. Also, I have omitted here async calls and continuations, exceptions etc for code simplicity.
int __declspec(dllexport) GetInfoAndPicture(CString uid, char **image, long *imageSize)
{
CString request = "";
request.Format(url);
http_client httpClient(url);
http_request msg(methods::POST);
...
http_response httpResponse;
httpResponse = httpClient.request(msg).get(); //blocking
web::json::value jsonValue = httpResponse.extract_json().get();
if (jsonValue.has_string_field(L"img"))
{
web::json::value base64EncodedImageValue = jsonValue.at(L"img");
utility::string_t imageString = base64EncodedImageValue.as_string();
std::vector<unsigned char> imageBytes = utility::conversions::from_base64(imageString);
image = (char**)&imageBytes; //Is this the way to pass image bytes back?
*imageSize = imageBytes.size();
}
...
}
The caller calls this method like so:
char mUid[64];
char* mImage;
long mImageSize;
...
resultCode = GetInfoAndPicture(mUid, &mImage, &mImageSize);
//process image given its data and its size
I know what pointer to pointer is, my question is specific to this line
image = (char**)&imageBytes;
Is this the correct way to return the image decoded from base64 into the calling code via the char** image formal parameter given the above method signature and method call?
I do get error "Program .... File: minkernel\crts\ucrt\src\appcrt\convert\isctype.cpp ... "Expression c >= -1 && c <= 255"" which I believe is related to the fact that this line is not correctly passing data back.
Give the requirements there isn't any way to avoid allocating more memory and copying the bytes. You cannot use the vector directly because that is local to the GetInfoAndPicture function and will be destroyed when that function exits.
If I understand the API correctly then this is what you need to do
//*image = new char[imageBytes.size()]; //use this if caller calls delete[] to deallocate memory
*image = (char*)malloc(imageBytes.size()); //use this if caller calls free(image) to deallocate memory
std::copy(imageBytes.begin(), imageBytes.end(), *image);
*imageSize = imageBytes.size();
Maybe there is some way in your utility::conversions functions of decoding directly to a character array instead of to a vector, but only you would know about that.
The problem is with allocating (and freeing) memory for that image; who is responsible for that?
You can't (shouldn't) allocate memory in one module and free it in another.
Your two options are:
Allocate large enough buffer on the caller side, and have DLL use it utility::conversions::from_base64(). The issue here is: what is large enough? Some Win APIs provide an additional method to query the required size. Doesn't fit this scenario as the DLL would either have to get that image for the second time, or hold it (indefinitely) until you ask for it.
Allocate required buffer in the DLL and return a pointer to it. You need to ensure that it won't be freed until the caller request to free it (in a separate API).

Adobe String Memory Leak - Where to Call External Library Entry Point to Free Memory?

I've created an After Effects script that extracts data from JSON files downloaded from an HTTPS URL. The problem is with the C++ DLL I've coded to download it and pass it back to the script. Even though it has been working fine, there was one instance of memory leak - After Effects issued a popup saying, "STRING MEMORY LEAK".
I'm new to C++ but I've managed to compose a DLL that downloads the files based on the examples provided with the After Effects installation (samplelib and basicexternalobject) as well as by Microsoft's C++ documentation. The Adobe JavaScript Tools Guide says that the method "ESFreeMem()" must be "called to free memory allocated for a null-terminated string passed to or from library functions". The problem is I don't know how or where to use it. I'm using After Effects CC 15.0.0 (build 180) on Windows 7.
This is the C++ function that gets some parameters from the javascript caller and returns a string with the JSON contents. If it fails it returns a bool (FALSE) so that the script can do what is necessary in this case.
extern "C" TvgAfx_Com_API long DownloadJson(TaggedData* argv, long argc, TaggedData * result)
{
//... first I check the arguments passed
// The returned value type
result->type = kTypeString;
//Converts from string into LPCWSTR ---------------------------------------------------
std::wstring stemp = s2ws(argv[0].data.string);
LPCWSTR jsonLink = stemp.c_str();
std::wstring stemp02 = s2ws(argv[1].data.string);
LPCWSTR jsonHeader = stemp02.c_str();
//--------------------------------------------------------------------------------------
//Class that does the HTTP request
WinHttpClient client(jsonLink, jsonHeader);
//Synchronous request
if (client.SendHttpsRequest())
{
string httpResponse = client.GetHttpResponse();
if (httpResponse.length() > 0)
{
//Sends response string back to javascript
result->data.string = getNewBuffer(httpResponse);
}
else
{
//Sends FALSE back to javascript
result->type = kTypeBool;
result->data.intval = 0;
}
}
else
{
//Sends FALSE back to javascript
result->type = kTypeBool;
result->data.intval = 0;
}
return kESErrOK;
}
The class WinHttpClient that does the actual request frees the memory allocated to the buffer that holds the response. Here's a piece of code:
// Read the data.
ZeroMemory(pszOutBuffer, dwSize + 1);
if (!WinHttpReadData(hRequest, (LPVOID)pszOutBuffer, dwSize, &dwDownloaded))
{
//Log error
}
else
{
resource.append(pszOutBuffer).c_str();
}
// Free the memory allocated to the buffer.
delete[] pszOutBuffer;
This is the function that the Adobe example uses to hold the string that will be returned to javascript:
//brief Utility function to handle strings and memory clean up
static char* getNewBuffer(string& s)
{
// Dynamically allocate memory buffer to hold the string
// to pass back to JavaScript
char* buff = new char[1 + s.length()];
memset(buff, 0, s.length() + 1);
strcpy(buff, s.c_str());
return buff;
}
Now, the manual says this method must be implemented:
/**
* \brief Free any string memory which has been returned as function result.
* JavaScipt calls this function to release the memory associated with the string.
* Used for the direct interface.
*
* \param *p Pointer to the string
*/
extern "C" SAMPLIB void ESFreeMem (void* p)
{
if (p)
free (p);
}
What I understand from this is that the memory associated with the json string returned must be released. But didn't the request class already do it? I just don't know where to call this method and what to pass on to it. I would appreciate any help. Thanks a lot!
When you create a DLL ExternalObject for After Effects, it's really just an interface that you call via some ExtendScript script. You have to load the ExternalObject in the AE Script and once loaded, the methods / functions you create in the C++ class can be called from the script.
You have to know how to load the DLL in an ExtendScript script file. Then, you can call the methods of the DLL.

Objective-C ARC: is it correct to use a block as C++ callback

I have a App written in C++ and Objective-C. The C++ part receive data from a remote camera and call Objective-C callback(Block) to display. Objective-C is ARC enabled.
When the display view loaded, I set a block to C++ part, when the data is coming, C++ will call this block to update the display.
The code as follows:
- (void)viewDidLoad
{
__weak CameraVC *weakSelf = self;
self.callback_func = ^int(int iWidth, int iHeight, int iDataLen, void *pData) {
NSData *data = [NSData dataWithBytes:pData length:iDataLen];
[weakSelf performSelectorOnMainThread:#selector(updateDisplayWithData:)
withObject:data waitUntilDone:YES];
return 0;
};
setDisplayCallback(self.callback_func);
}
- (void)updateDisplayWithData:(NSData *)data
{
self.imageView.image = [UIImage imageWithData:data];
}
setDisplayCallback() is a C++ function to set a callback.
When the app runs, in the xcode panel shows the app memory usage, and it is always increasing, hours later, the app crashed, I think it is memory leak?
I have try to comment the code:
// self.imageView.image = [UIImage imageWithData:data];
The memory leak stopped.
Question 1 Is there retain cycle to cause this memory leak?
I have try to replace the block code from:
self.callback_func = ^int(int iWidth, int iHeight, int iDataLen, void *pData) {
NSData *data = [NSData dataWithBytes:pData length:iDataLen];
[weakSelf performSelectorOnMainThread:#selector(updateDisplayWithData:)
withObject:data waitUntilDone:YES];
return 0;
};
to:
self.callback_func = ^int(int iWidth, int iHeight, int iDataLen, void *pData) {
NSData *data = [NSData dataWithBytesNoCopy:pData length:iDataLen freeWhenDone:YES];
[weakSelf performSelectorOnMainThread:#selector(updateDisplayWithData:)
withObject:data waitUntilDone:YES];
return 0;
};
The memory leak problem reduced, but still have.
Question 2 What's the difference between dataWithBytes and dataWithBytesNoCopy?
UPDATE
I try to set this single file with no-ARC, and modify the code:
- (void)viewDidLoad
{
self.callback_func = ^int(int iWidth, int iHeight, int iDataLen, void *pData) {
#autoreleasepool {
NSData *data = [NSData dataWithBytes:pData length:iDataLen];
[self performSelectorOnMainThread:#selector(updateDisplayWithData:)
withObject:data waitUntilDone:YES];
}
return 0;
};
setDisplayCallback(self.callback_func);
}
The memory usage is stable. I'm curious what's the problem in my ARC version code.
In backwards order:
The difference between -dataWithBytes... and -dataWithBytesNoCopy... is there in the name. Normally, if you create an NSData using a raw byte array, the NSData will copy all the bytes internally so that it knows its underlying data cannot be modified independently (and it can also manage the memory however it wants to). If you use the ...NoCopy variant, you're telling the NSData to use the buffer you hand it as its storage, and not to allocate any more. This has the upside of not using any more memory of course (important for larger buffers maybe) but the downside that you need to be a lot more careful about what pointer you hand it and what you do with that pointer afterwards. If you pass in YES for freeWhenDone as you are doing here, you are telling the NSData to call free() on the pointer when it is itself deallocated. This assumes it was allocated directly from malloc and effectively transfers ownership of a malloc buffer to the NSData.
Is there a retain cycle in your code? No, there is not an obvious one.
So what you're seeing depends on what you mean by "leak", what else is happening in the surrounding code, and what you expect to see. What you appear to be doing is taking the bytes from pData, either copying them into an NSData or transferring their ownership to an NSData, and then using that data to create a UIImage, and putting that into a UIImageView. How you should handle the memory management of the NSData depends on the ownership of pData, which you don't show. (Who mallocs it? Who is expected to free it? When?). Creating the UIImage from the image data will use memory of course, but will not "leak" it. If you comment that out, then you'll for sure use less memory, but obviously won't get the image. :) (The NSData itself will be deallocated always just after the -performSelector... is complete, because it goes out of scope afterwards, taking the buffer with it in both your cases.)

JNI how to release it

I'm a JNI navice, a issue about release memory has blocked me, the scenario like below:
If you have convert a ArrayList from Java to C, and put the ArrayList elements from String array like:
jclass arrlist_cls = (*env)->FindClass(env,"java/util/ArrayList");
if(NULL==arrlist_cls)return;
jmethodID m_get = (*env)->GetMethodID(env,arrlist_cls,"get","(I)Ljava/lang/Object");
jmethodID m_size = (*env)->GetMethodID(env,arrlist_cls,"size","()I");
//get the ArrayList field from object
jfieldID fidArrStr = (*env)->GetFieldID(env,helloObj,"arrStr",
"Ljava/util/ArrayList;");
jobject ArrObj = (*env)->GetObjectField(env,paramsObj,fidArrStr);
if(NULL==ArrObj)return;
int len = (*env)->CallIntMethod(env,ArrObj,size_method);
int i=0; const char **ArrStr;
for(i=0;i<len;i++){
jstring jstr = (jstring)(*env)->CallObjectMethod(env,ArrObj,get_method,i);
ArrStr[i]=jstr;
}
How can i release it like single String?you know, the single String could use the method of ReleaseStringUTFChars to release,
how can i release a array object memory? Because String is not primitive type, so i can't find ReleaseArrayElements method to use.
I mean how to release the ArrStr.
This code won't even work: it will get a SIGSEGV when you execute ArrStr[i] = ... So I don't know why you're worrying about termination yet.
You need to allocate ArrStr before you can use it. How you allocate it determines how you release it. If you allocate it with new you must release it with delete. If you allocate it as an array, i.e. char *ArrStr[len], it will disappear when the method exits: no release required.
If you're asking about how to deallocate the jstrings returned by CallObjectMethod(), again they are released automatically by JNI when you exit this JNI method. You can release them explicitly prior to that, if necessary, with DeleteLocalRef().

How to write a small memory leak detection in C++?

I am trying to write a small memory leak detection tool.
My idea is to track the dynamic memory allocation life with in my application
to determine any invalid access of memory or un deleted memory which might cause my application to core over a time of use.
I want to write a simple interface to override new and delete.
And in my overridden new i wanted to print function line address etc. And then call the standard new.
Have anyone already tried this? I am not sure whether i can call standard new from my class specific new operator.
There's a couple of articles here:
http://www.codeproject.com/Articles/8448/Memory-Leak-Detection
http://www.codeproject.com/Articles/19361/Memory-Leak-Detection-in-C
http://www.codeproject.com/Articles/9815/Visual-Leak-Detector-Enhanced-Memory-Leak-Detectio
http://www.codeproject.com/Articles/393957/Cplusplus-Memory-Leak-Finder
In this question i have proposed an ad-hoc solution to your problem: Immediate detection of heap corruption errors on Windows. How?
In general, you can replace your new and delete with this code:
DWORD PageSize = 0;
inline void SetPageSize()
{
if ( !PageSize )
{
SYSTEM_INFO sysInfo;
GetSystemInfo(&sysInfo);
PageSize = sysInfo.dwPageSize;
}
}
void* operator new (size_t nSize)
{
SetPageSize();
size_t Extra = nSize % PageSize;
nSize = nSize + ( PageSize - Extra );
return Ptr = VirtualAlloc( 0, nSize, MEM_COMMIT, PAGE_READWRITE);
}
void operator delete (void* pPtr)
{
MEMORY_BASIC_INFORMATION mbi;
VirtualQuery(pPtr, &mbi, sizeof(mbi));
// leave pages in reserved state, but free the physical memory
VirtualFree(pPtr, 0, MEM_DECOMMIT);
DWORD OldProtect;
// protect the address space, so noone can access those pages
VirtualProtect(pPtr, mbi.RegionSize, PAGE_NOACCESS, &OldProtect);
}
to determine invalid access of memory. In the further discussion there you will find ideas for leaks detection and detection of other memory errors.
If you want to call global new and delete, you can use :: global namespace prefix:
return ::new(nSize);
Calling ::operator new from a class-specific operator new is fine, and quite common. You can't call global operator new from a replacement version of ::operator new because if you replace the global operator new the old one doesn't exist.
The described approach sounds reasonable. You can enhance it and store some bits of identifying data on each allocation into some data structure. This way you can track all the non-freed memory upon program termination without the need to inspect all allocation/deallocation logs.
In order to detect corruptions you can wrap each allocation with paddings before the start and after the end and fill it with some predefined pattern. Upon memory freeing you can verify that the patterns are still in place. This will give you a good chance of detecting corruption.
You can call global new with scope prefix: ::new.