Processing flutter images with C++ openCV - c++

Question

I am working on a project, where I want to process my images using C++ OpenCV.
For simplicity's sake, I just want to convert Uint8List to cv::Mat and back.
Following this tutorial, I managed to make a pipeline that doesn't crash the app. Specifically:
I created a function in a .cpp that takes the pointer to my Uint8List, rawBytes, and encodes it as a .jpg:
int encodeIm(int h, int w, uchar *rawBytes, uchar **encodedOutput) {
cv::Mat img = cv::Mat(h, w, CV_8UC3, rawBytes); //CV_8UC3
vector<uchar> buf;
cv:imencode(".jpg", img, buf); // save output into buf. Note that Dart Image.memory can process either .png or .jpg, which is why we're doing this encoding
*encodedOutput = (unsigned char *) malloc(buf.size());
for (int i=0; i < buf.size(); i++)
(*encodedOutput)[i] = buf[i];
return (int) buf.size();
}
Then I wrote a function in a .dart that calls my c++ encodeIm(int h, int w, uchar *rawBytes, uchar **encodedOutput):
//allocate memory heap for the image
Pointer<Uint8> imgPtr = malloc.allocate(imgBytes.lengthInBytes);
//allocate just 8 bytes to store a pointer that will be malloced in C++ that points to our variably sized encoded image
Pointer<Pointer<Uint8>> encodedImgPtr = malloc.allocate(8);
//copy the image data into the memory heap we just allocated
imgPtr.asTypedList(imgBytes.length).setAll(0, imgBytes);
//c++ image processing
//image in memory heap -> processing... -> processed image in memory heap
int encodedImgLen = _encodeIm(height, width, imgPtr, encodedImgPtr);
//
//retrieve the image data from the memory heap
Pointer<Uint8> cppPointer = encodedImgPtr.elementAt(0).value;
Uint8List encodedImBytes = cppPointer.asTypedList(encodedImgLen);
//myImg = Image.memory(encodedImBytes);
return encodedImBytes;
//free memory heap
//malloc.free(imgPtr);
//malloc.free(cppPointer);
//malloc.free(encodedImgPtr); // always frees 8 bytes
}
Then I linked c++ with dart via:
final DynamicLibrary nativeLib = Platform.isAndroid
? DynamicLibrary.open("libnative_opencv.so")
: DynamicLibrary.process();
final int Function(int height, int width, Pointer<Uint8> bytes, Pointer<Pointer<Uint8>> encodedOutput)
_encodeIm = nativeLib
.lookup<NativeFunction<Int32 Function(Int32 height, Int32 width,
Pointer<Uint8> bytes, Pointer<Pointer<Uint8>> encodedOutput)>>('encodeIm').asFunction();
And finally I show the result in Flutter via:
Image.memory(...)
Now, the pipeline doesn't crash, which means I haven't goofed up memory handling completely, but it doesn't return the original image either, which means I did mess up somewhere.
Original image:
Pipeline output:

Answer 1

Thanks to Richard Heap's guidance in the comments, I managed to fix the pipeline by changing my matrix definition from
cv::Mat img = cv::Mat(h, w, CV_8UC3, rawBytes);
to
vector<uint8_t> buffer(rawBytes, rawBytes + inBytesCount);
Mat img = imdecode(buffer, IMREAD_COLOR);
where inBytesCount is the length of imgBytes.