nodejs native c++ npm module memory error, cairo image processing - c++

Question

I've been bugging TJ on node-canvas about a code speed up I'm working on in a fork of a node module he authored and maintains.
I found Canvas.toBuffer() to be killing our pipeline resources and created an alternative that would simply convert from Canvas into an Image without going through a png buffer/media url. The problem is that cairo is a mysterious beast, and there's an additional level of concern about memory allocated within node modules as not to get GC'd by mother v8. I've added the proper HandleScopes to all required functions which access V8 data.
I was able to test the Canvas.loadImage(image) method thousands of times on my mac setup (6.18), as well as stand alone tests on our ubuntu/production servers running the same version of node. But when the code is run as a background process/server and coordinated by Gearman I'm getting some "interesting" memory/segfaults.
In addition I'm having trouble calling any of the methods of classes defined in node-canvas that aren't inline within header files. As a side question What's the best way to create common native source code packages that other node modules can rely on?
I've tried recreating the problem and running it with gdb, node_g, and all the node modules built with symbols and debug flags. But the error crops up in a lib outside of the source I can get a stack trace for.
for reference here's where I call loadImageData and while it runs locally under a variety of conditions, in our production environment when carefully tucked away within a frame server it appears to be causing segfaults (spent the day yesterday trying to gdb node_g our server code but the frame servers are kicked off by gearman... TL;DR didn't get a root cause stack trace)
https://github.com/victusfate/node-canvas/blob/master/src/Canvas.cc#L497
Handle<Value>
Canvas::LoadImage(const Arguments &args) {
HandleScope scope;
LogStream mout(LOG_DEBUG,"node-canvas.paint.ccode.Canvas.LoadImage");
mout << "Canvas::LoadImage top " << LogStream::endl;
Canvas *canvas = ObjectWrap::Unwrap<Canvas>(args.This());
if (args.Length() < 1) {
mout << "Canvas::LoadImage Error requires one argument of Image type " << LogStream::endl;
return ThrowException(Exception::TypeError(String::New("Canvas::LoadImage requires one argument of Image type")));
}
Local<Object> obj = args[0]->ToObject();
Image *img = ObjectWrap::Unwrap<Image>(obj);
canvas->loadImageData(img);
return Undefined();
}
void Canvas::loadImageData(Image *img) {
LogStream mout(LOG_DEBUG,"node-canvas.paint.ccode.Canvas.loadImageData");
if (this->isPDF()) {
mout << "Canvas::loadImageData pdf canvas type " << LogStream::endl;
cairo_surface_finish(this->surface());
closure_t *closure = (closure_t *) this->closure();
int w = cairo_image_surface_get_width(this->surface());
int h = cairo_image_surface_get_height(this->surface());
img->loadFromDataBuffer(closure->data,w,h);
mout << "Canvas::loadImageData pdf type, finished loading image" << LogStream::endl;
}
else {
mout << "Canvas::loadImageData data canvas type " << LogStream::endl;
cairo_surface_flush(this->surface());
int w = cairo_image_surface_get_width(this->surface());
int h = cairo_image_surface_get_height(this->surface());
img->loadFromDataBuffer(cairo_image_surface_get_data(this->surface()),w,h);
mout << "Canvas::loadImageData image type, finished loading image" << LogStream::endl;
}
}
and here's what the current method in Image looks like (I removed some commented out logging info)
https://github.com/victusfate/node-canvas/blob/master/src/Image.cc#L240
/*
* load from data buffer width*height*4 bytes
*/
cairo_status_t
Image::loadFromDataBuffer(uint8_t *buf, int width, int height) {
this->clearData();
int stride = cairo_format_stride_for_width (CAIRO_FORMAT_ARGB32, width); // 4*width + ?
this->_surface = cairo_image_surface_create_for_data(buf,CAIRO_FORMAT_ARGB32,width,height,stride);
this->data_mode = DATA_IMAGE;
this->loaded();
cairo_status_t status = cairo_surface_status(_surface);
if (status) return status;
return CAIRO_STATUS_SUCCESS;
}
Any help, pro tips, assistance, or words of encouragement would be appreciated.
Originally from google groups

Answer 1

Got it!
I was working on another library today that uses cairomm and discovered surfaces created from data buffers require those buffers to live on as long as the surface does.
http://www.cairographics.org/manual/cairo-Image-Surfaces.html#cairo-image-surface-create-for-data
"Creates an image surface for the provided pixel data. The output buffer must be kept around until the cairo_surface_t is destroyed or cairo_surface_finish() is called on the surface. The initial contents of data will be used as the initial image contents; you must explicitly clear the buffer, using, for example, cairo_rectangle() and cairo_fill() if you want it cleared."
I introduced a surface that was created from a temporary buffer.
Easy solution in node-canvas fork:
There's a member variable called _data which I can assign a locally malloced data buffer to, which will live on as long as the cairo surface does.
Solution:
A general way to copy a buffer into a surface is to create a temporary surface from the buffer, then draw from the temporary surface onto an allocated surface and let cairo manage it's own memory.
It would look something like this with the c api to cairo to implement.
cairo_surface_t *pTmp = cairo_image_surface_create_for_data (
data
, CAIRO_FORMAT_ARGB32
, width
, height
, cairo_format_stride_for_width(CAIRO_FORMAT_ARGB32, width));
_surface = cairo_image_surface_create ( CAIRO_FORMAT_ARGB32
, width
, height);
cairo_t *cr = cairo_create (_surface);
cairo_set_source_surface (cr, pTmp, x, y);
cairo_paint (cr);

Answer 2

In addition I'm having trouble calling any of the methods of classes
defined in node-canvas that aren't inline within header files. As a
side question What's the best way to create common native source code
packages that other node modules can rely on?
While I don't have an answer for the memory issue/seg fault I was running into in our staging environment. I do have an answer for constructing reusable libraries with native node modules.
I'm using git submodules for all the independent native node modules, and added a conditional preprocessor definition for each of their wscript or binding.gyp files to specify whether or not to generate a shared object .node module.
update Alternatively unique init function names or namespaces can surround the module initialization call (moved to this setup).
In addition I'll be using this new package to aid in debugging or rewriting code sections (I can't spend too much time debugging utilization of several remote libraries).
in wscript or binding.gyp
flags = ['-D_NAME_NODE_MODULE', '-O3', '-Wall', '-D_FILE_OFFSET_BITS=64', '-D_LARGEFILE_SOURCE', '-msse2']
then in an initialization file
#ifdef _NAME_NODE_MODULE
extern "C" {
static void init(Handle<Object> target) {
HandleScope scope;
NODE_SET_METHOD(target, "someFunction", someFunction);
}
NODE_MODULE(moduleName, init);
}
#endif
This way a node native module is only added to when the flag is set. Otherwise it can be linked to normally (like in another node module).