I am using libjpeg to transform image buffer from OpenCV Mat and write it to a memory location
Here is the code:
bool mat2jpeg(cv::Mat frame, unsigned char **outbuffer
, long unsigned int *outlen) {
unsigned char *outdata = frame.data;
struct jpeg_compress_struct cinfo = { 0 };
struct jpeg_error_mgr jerr;
JSAMPROW row_ptr[1];
int row_stride;
*outbuffer = NULL;
*outlen = 0;
cinfo.err = jpeg_std_error(&jerr);
jpeg_create_compress(&cinfo);
jpeg_mem_dest(&cinfo, outbuffer, outlen);
jpeg_set_quality(&cinfo, JPEG_QUALITY, TRUE);
cinfo.image_width = frame.cols;
cinfo.image_height = frame.rows;
cinfo.input_components = 1;
cinfo.in_color_space = JCS_GRAYSCALE;
jpeg_set_defaults(&cinfo);
jpeg_start_compress(&cinfo, TRUE);
row_stride = frame.cols;
while (cinfo.next_scanline < cinfo.image_height) {
row_ptr[0] = &outdata[cinfo.next_scanline * row_stride];
jpeg_write_scanlines(&cinfo, row_ptr, 1);
}
jpeg_finish_compress(&cinfo);
jpeg_destroy_compress(&cinfo);
return true;
}
The thing is I cannot deallocate outbuffer anywhere.
This is how I am using the function:
long unsigned int * __size__ = nullptr;
unsigned char * _buf = nullptr;
mat2jpeg(_img, &_buf, __size__);
both free(_buf) and free(*_buf) fails
it seems i am trying to free the head of heap by doing so.
and mat2jpeg won't accept a pointer to pointer for outbuffer. any idea?
I think your problem may be with your __size__ variable. Its not allocated anywhere. According to my reading of the libjpeg source code that means the buffer is never allocated and the program calls a fatal error function.
I think you need to call it like this:
long unsigned int __size__ = 0; // not a pointer
unsigned char * _buf = nullptr;
mat2jpeg(_img, &_buf, &__size__); // send address of __size__
Then you should be able to deallocate the buffer with:
free(_buf);
I have verified that it is the dll that caused the issue. I tried to recompiled libjpeg as static library and everything now works like a charm.
In my case there was no way to free the memory image pointer, the only way to do that was reserving enough memory to the image, that way the library will not reserve memory for me and I have the control over the memory, the memory will be part of my own application and not library's dll or .lib:
//previous code...
struct jpeg_compress_struct cinfo;
//reserving the enough memory for my image (width * height)
unsigned char* _image = (unsigned char*)malloc(Width * Height);
//putting the reserved size into _imageSize
_imageSize = Width * Height;
//call the function like this:
jpeg_mem_dest(&cinfo, &_image, &_imageSize);
................
//releasing the reserved memory
free(_image);
NOTE: if you put _imageSize = 0, the library will assume that you have not reserve memory and the own library will do it.. so you need to put in _imageSize the amount of bytes reserved in _image
That way you have total control over the reserved memory and you can release it whenever you want in your software..
Related
I'd like to return a Mat object to Unity from c++ code. However i get access violation error at c++ part like that
Unity Editor [version: Unity 2017.3.0f3_a9f86dcd79df]
SaliencyCV.dll caused an Access Violation (0xc0000005)
in module SaliencyCV.dll at 0033:270027f0.
Error occurred at 2018-03-06_235212.
C:\Program Files\Unity\Editor\Unity.exe, run by Dilara.
43% memory in use.
16266 MB physical memory [9199 MB free].
18698 MB paging file [9861 MB free].
134217728 MB user address space [134185466 MB free].
Read from location 990d0000 caused an access violation.
Here is c++ code:
uchar* cppMethod(uchar* frameData, int WIDTH, int HEIGHT, int* rows, int* cols)
{
Mat img(HEIGHT, WIDTH, CV_8UC3);
img.data = frameData;
flip(img, img, 0);
Mat result = calculateSaliency(img);
*rows = result.rows;
*cols = result.cols;
int length = result.rows * result.cols * 3;
uchar* tmpArr = result.data;
uchar* resultArray = new uchar[length];
for (int i = 0; i < length; i++)
{
resultArray[i] = tmpArr[i];
}
return resultArray;
}
Can someone help me?
You should call the correct Mat constructor, which accepts external data pointer, to make the object not release/destruct the corresponding memory location data points to. You can read about this behaviour in Mat::release().
The problem with your code is that
Mat img(HEIGHT, WIDTH, CV_8UC3) allocates a memory block of type CV_8UC3 of size HEIGHT*WIDTH, which is not used (because you are changing the data member variable to point to a different memory location, anyways),
At function exit, img is destructed, which results in a call to release(), which in turn destructs frameData, which is not the intended behaviour.
Change your first two lines to read
Mat img(HEIGHT, WIDTH, CV_8UC3, frameData);
And if you are passing resultArray to C#, where you are most likely not managing the pointed-to-memory's lifetime, you would be most likely having memory leaks. #Programmer has already suggested in his answer to your previous question that you should allocate the memory in C#, pass it to C++, and write in-place in the C++ side.
In short, you should have something like:
#include <algorithm>
void cppMethod(uchar *frameData, uchar *out, const int WIDTH, const int HEIGHT,
int *rows, int *cols) {
/* this constructor will not manage frameData's lifetime */
Mat img(HEIGHT, WIDTH, CV_8UC3, frameData);
/* in-place operation */
flip(img, img, 0);
/* local variable --- it will be destructed properly */
Mat result = calculateSaliency(img);
/* well-defined if rows and cols are scalars passed by reference */
*rows = result.rows;
*cols = result.cols;
/* make sure length will not overflow */
int length = result.rows * result.cols * 3;
/* you don't need this */
// uchar *tmpArr = result.data;
/* you sholuld NOT do this */
// uchar *resultArray = new uchar[length];
// use std::copy from <algorithm>
// for (int i = 0; i < length; i++) {
// resultArray[i] = tmpArr[i];
// }
std::copy(result.data, result.data + length, out);
// return resultArray;
}
I'm struggling at the moment and hope that someone can help me.
I have to get an Image out of a SQL database (like with SQLGetData) and than convert that data to a CImage so I can view it in my program.
Thanks for any help!
SQLGetData(m_Hstmt, col, SQL_C_BINARY, BinaryPtr, 0, &cbData)
The problem can be reduced to loading a CImage from a byte array, since that is what you get from SQLGetData.
You did not indicate whether you mean to use ATL or MFC, but in both cases it is a little bit awkward as there is no such thing as a public ::LoadFromBuffer function.
This answer should do:
https://stackoverflow.com/a/6759701/1132334
It explains how to create a bitmap structure from a byte buffer and construct a CImage from there.
It is going to be tricky if you need to handle different picture formats. In this case, write the raw bytes to a memory mapped file and then use the CImage::Load(IStream*) overload.
EDIT: its all been done before... https://stackoverflow.com/a/14035492/1132334 and https://stackoverflow.com/a/38710933/1132334
Thanks for all the replies so far. Acoording to #dlatikay answer, this is my code so far. But I'm not sure about the types and somehow my Image stayes black (when I save it to file system)
heres my code so far.
SQLLEN cbData;
CImage image;
BYTE* imgBits;
m_Rc = SQLGetData(m_Hstmt, 1, SQL_C_BINARY, imgBits, 0, &cbData);
if (SQL_SUCCEEDED(m_Rc))
{
width = 317;
height = 159;
BITMAPINFOHEADER bmInfohdr;
// Create the header info
bmInfohdr.biSize = sizeof(BITMAPINFOHEADER);
bmInfohdr.biWidth = width;
bmInfohdr.biHeight = -height;
bmInfohdr.biPlanes = 1;
bmInfohdr.biBitCount = 8 * 8;
bmInfohdr.biCompression = BI_RGB;
bmInfohdr.biSizeImage = width*height * 8;
bmInfohdr.biXPelsPerMeter = 0;
bmInfohdr.biYPelsPerMeter = 0;
bmInfohdr.biClrUsed = 0;
bmInfohdr.biClrImportant = 0;
BITMAPINFO bmInfo;
bmInfo.bmiHeader = bmInfohdr;
bmInfo.bmiColors[0].rgbBlue = 255;
// Allocate some memory and some pointers
unsigned char * p24Img = new unsigned char[width*height * 3];
BYTE *pTemp, *ptr;
pTemp = (BYTE*)imgBits;
ptr = p24Img;
// Convert image from RGB to BGR
for (DWORD index = 0; index < width*height; index++)
{
unsigned char r = *(pTemp++);
unsigned char g = *(pTemp++);
unsigned char b = *(pTemp++);
*(ptr++) = b;
*(ptr++) = g;
*(ptr++) = r;
}
// Create the CImage
image.Create(width, height, 8, NULL);
image.Save(_T("c:\\temp\\image1.bmp")); // for testing
}
I wrote an npm module for capturing webcam input on linux. The captured frame in yuyv format is converted to rgb24 and after compressed to a jpeg image. In the jpeg compression there appears to be a memory leak. So the usage of memory increases continuously.
Image* rgb24_to_jpeg(Image *img, Image *jpeg) { // img = RGB24
jpeg_compress_struct cinfo;
jpeg_error_mgr jerr;
cinfo.err = jpeg_std_error(&jerr);
jerr.trace_level = 10;
jpeg_create_compress(&cinfo);
unsigned char *imgd = new unsigned char[img->size];
long unsigned int size = 0;
jpeg_mem_dest(&cinfo, &imgd, &size);
cinfo.image_width = img->width;
cinfo.image_height = img->height;
cinfo.input_components = 3;
cinfo.in_color_space = JCS_RGB;
jpeg_set_defaults(&cinfo);
jpeg_set_quality(&cinfo, 100, true);
jpeg_start_compress(&cinfo, true);
int row_stride = cinfo.image_width * 3;
JSAMPROW row_pointer[1];
while (cinfo.next_scanline < cinfo.image_height) {
row_pointer[0] = &img->data[cinfo.next_scanline * row_stride];
jpeg_write_scanlines(&cinfo, row_pointer, 1);
}
jpeg_finish_compress(&cinfo);
jpeg_destroy_compress(&cinfo);
// size += 512; // TODO: actual value to expand jpeg buffer... JPEG header?
if (jpeg->data == NULL) {
jpeg->data = (unsigned char *) malloc(size);
} else {
jpeg->data = (unsigned char *) realloc(jpeg->data, size);
}
memcpy(jpeg->data, imgd, size);
delete[] imgd;
jpeg->size = size;
return jpeg;
}
The rgb24 and jpeg buffers are reallocated on every cycle. So it looks like the leak is inside libjpeg layer. Is this true or I simply made a mistake somewhere in the code?
Note: the compressed image shall not be saved as a file, since the data might be used for live streaming.
You are using the jpeg_mem_dest in a wrong way - the second parameter is pointer to pointer to char because it is actually set by the library and then you must free it after you are done. Now you are initializing it with a pointer, it gets overwritten and you free the memory region allocated by the library but the original memory region is leaked.
This is how you should change your function:
Image* rgb24_to_jpeg(Image *img, Image *jpeg) { // img = RGB24
jpeg_compress_struct cinfo;
jpeg_error_mgr jerr;
cinfo.err = jpeg_std_error(&jerr);
jerr.trace_level = 10;
jpeg_create_compress(&cinfo);
unsigned char *imgd = 0;
long unsigned int size = 0;
cinfo.image_width = img->width;
cinfo.image_height = img->height;
cinfo.input_components = 3;
cinfo.in_color_space = JCS_RGB;
jpeg_set_defaults(&cinfo);
jpeg_set_quality(&cinfo, 100, true);
jpeg_mem_dest(&cinfo, &imgd, &size); // imgd will be set by the library
jpeg_start_compress(&cinfo, true);
int row_stride = cinfo.image_width * 3;
JSAMPROW row_pointer[1];
while (cinfo.next_scanline < cinfo.image_height) {
row_pointer[0] = &img->data[cinfo.next_scanline * row_stride];
jpeg_write_scanlines(&cinfo, row_pointer, 1);
}
jpeg_finish_compress(&cinfo);
jpeg_destroy_compress(&cinfo);
// size += 512; // TODO: actual value to expand jpeg buffer... JPEG header?
if (jpeg->data == NULL) {
jpeg->data = (unsigned char *) malloc(size);
} else if (jpeg->size != size) {
jpeg->data = (unsigned char *) realloc(jpeg->data, size);
}
memcpy(jpeg->data, imgd, size);
free(imgd); // dispose of imgd when you are done
jpeg->size = size;
return jpeg;
}
This snippet form jpeg_mem_dest explains the memory management:
if (*outbuffer == NULL || *outsize == 0) {
/* Allocate initial buffer */
dest->newbuffer = *outbuffer = (unsigned char *) malloc(OUTPUT_BUF_SIZE);
if (dest->newbuffer == NULL)
ERREXIT1(cinfo, JERR_OUT_OF_MEMORY, 10);
*outsize = OUTPUT_BUF_SIZE;
}
So, if you pass a an empty pointer or a zero sized buffer the library will perform an allocation for you. Thus - another approach is also to set the size correctly and then you can use the originally supplied pointer
In my case I did not solve the issue with previous answer, there was no way to free the memory image pointer, the only way to do that was reserving enough memory to the image and that way the library will not reserve memory and I have the control over the memory and is on the same heap of my application and not on the library's heap, here is my example:
//previous code...
struct jpeg_compress_struct cinfo;
//reserving the enough memory for my image (width * height)
unsigned char* _image = (unsigned char*)malloc(Width * Height);
//putting the reserved size into _imageSize
_imageSize = Width * Height;
//call the function like this:
jpeg_mem_dest(&cinfo, &_image, &_imageSize);
................
//releasing the reserved memory
free(_image);
NOTE: if you put _imageSize = 0, the library will assume that you have not reserve memory and the own library will do it.. so you need to put in _imageSize the amount of bytes reserved in _image
That way you have total control over the reserved memory and you can release it whenever you want in your software..
This is my function and it says
stack around variable cinfo was corrupted.
and it says that problem is in line 551 which is last line in this posted code.
struct jpeg_compress_struct cinfo;
struct jpeg_error_mgr jerr;
FILE* pOutFile; //Target file
int nSampsPerRow; //Physical row width in image buffer
JSAMPARRAY jsmpArray; //Pixel RGB buffer for JPEG file
cinfo.err = jpeg_std_error(&jerr); //Use default error handling (ugly!)
jpeg_create_compress(&cinfo);
if ((pOutFile = fopen(csJpeg, "wb")) == NULL)
{
*pcsMsg = "Cannot open ";
*pcsMsg += csJpeg;
jpeg_destroy_compress(&cinfo);
return FALSE;
}
jpeg_stdio_dest(&cinfo, pOutFile);
LPBITMAPINFOHEADER lpbi = (LPBITMAPINFOHEADER)hDib;
cinfo.image_width = lpbi->biWidth; //Image width and height, in pixels
cinfo.image_height = lpbi->biHeight;
cinfo.input_components = 3; //Color components per pixel
//(RGB_PIXELSIZE - see jmorecfg.h)
cinfo.in_color_space = JCS_RGB; //Colorspace of input image
jpeg_set_defaults(&cinfo);
jpeg_set_quality(&cinfo,
nQuality, //Quality: 0-100 scale
TRUE); //Limit to baseline-JPEG values
jpeg_start_compress(&cinfo, TRUE);
//JSAMPLEs per row in output buffer
nSampsPerRow = cinfo.image_width * cinfo.input_components;
//Allocate array of pixel RGB values
jsmpArray = (*cinfo.mem->alloc_sarray)
((j_common_ptr) &cinfo,
JPOOL_IMAGE,
nSampsPerRow,
cinfo.image_height);
if (DibToSamps(hDib,
nSampsPerRow,
cinfo,
jsmpArray,
pcsMsg))
{
//Write the array of scan lines to the JPEG file
(void)jpeg_write_scanlines(&cinfo,
jsmpArray,
cinfo.image_height);
}
jpeg_finish_compress(&cinfo); //Always finish
fclose(pOutFile);
jpeg_destroy_compress(&cinfo); //Free resources
if (*pcsMsg != "")
return FALSE;
else
return TRUE;
}
stack trace:> WindowBitmap.exe!CBitmapFile::JpegFromDib(void * hDib, int nQuality, ATL::CStringT > > csJpeg, ATL::CStringT > > * pcsMsg) Line 551 + 0x28 bytes C++
error message: Run-Time Check Failure #2 - Stack around the variable 'cinfo' was corrupted.
pcsMsg definition:
CString* pcsMsg
jpeg_destroy_compress(&cinfo);
frees the memory of an automatic variable, which is freed a second time when cinfo goes out of scope.
The function jpeg_destroy_compress() shall deallocate and release all memory associated with the compression object.
I need to create a CImage from a byte array (actually, its an array of unsigned char, but I can cast to whatever form is necessary). The byte array is in the form "RGBRGBRGB...". The new image needs to contain a copy of the image bytes, rather than using the memory of the byte array itself.
I have tried many different ways of achieving this -- including going through various HBITMAP creation functions, trying to use BitBlt -- and nothing so far has worked.
To test whether the function works, it should pass this test:
BYTE* imgBits;
int width;
int height;
int Bpp; // BYTES per pixel (e.g. 3)
getImage(&imgBits, &width, &height, &Bpp); // get the image bits
// This is the magic function I need!!!
CImage img = createCImage(imgBits, width, height, Bpp);
// Test the image
BYTE* data = img.GetBits(); // data should now have the same data as imgBits
All implementations of createCImage() so far have ended up with data pointing to an empty (zero filled) array.
CImage supports DIBs quite neatly and has a SetPixel() method so you could presumably do something like this (uncompiled, untested code ahead!):
CImage img;
img.Create(width, height, 24 /* bpp */, 0 /* No alpha channel */);
int nPixel = 0;
for(int row = 0; row < height; row++)
{
for(int col = 0; col < width; col++)
{
BYTE r = imgBits[nPixel++];
BYTE g = imgBits[nPixel++];
BYTE b = imgBits[nPixel++];
img.SetPixel(row, col, RGB(r, g, b));
}
}
Maybe not the most efficient method but I should think it is the simplest approach.
Use memcpy to copy the data, then SetDIBits or SetDIBitsToDevice depending on what you need to do. Take care though, the scanlines of the raw image data are aligned on 4-byte boundaries (IIRC, it's been a few years since I did this) so the data you get back from GetDIBits will never be exactly the same as the original data (well it might, depending on the image size).
So most likely you will need to memcpy scanline by scanline.
Thanks everyone, I managed to solve it in the end with your help. It mainly involved #tinman and #Roel's suggestion to use SetDIBitsToDevice(), but it involved a bit of extra bit-twiddling and memory management, so I thought I'd share my end-point here.
In the code below, I assume that width, height and Bpp (Bytes per pixel) are set, and that data is a pointer to the array of RGB pixel values.
// Create the header info
bmInfohdr.biSize = sizeof(BITMAPINFOHEADER);
bmInfohdr.biWidth = width;
bmInfohdr.biHeight = -height;
bmInfohdr.biPlanes = 1;
bmInfohdr.biBitCount = Bpp*8;
bmInfohdr.biCompression = BI_RGB;
bmInfohdr.biSizeImage = width*height*Bpp;
bmInfohdr.biXPelsPerMeter = 0;
bmInfohdr.biYPelsPerMeter = 0;
bmInfohdr.biClrUsed = 0;
bmInfohdr.biClrImportant = 0;
BITMAPINFO bmInfo;
bmInfo.bmiHeader = bmInfohdr;
bmInfo.bmiColors[0].rgbBlue=255;
// Allocate some memory and some pointers
unsigned char * p24Img = new unsigned char[width*height*3];
BYTE *pTemp,*ptr;
pTemp=(BYTE*)data;
ptr=p24Img;
// Convert image from RGB to BGR
for (DWORD index = 0; index < width*height ; index++)
{
unsigned char r = *(pTemp++);
unsigned char g = *(pTemp++);
unsigned char b = *(pTemp++);
*(ptr++) = b;
*(ptr++) = g;
*(ptr++) = r;
}
// Create the CImage
CImage im;
im.Create(width, height, 24, NULL);
HDC dc = im.GetDC();
SetDIBitsToDevice(dc, 0,0,width,height,0,0, 0, height, p24Img, &bmInfo, DIB_RGB_COLORS);
im.ReleaseDC();
delete[] p24Img;
Here is a simpler solution. You can use GetPixelAddress(...) instead of all this BITMAPHEADERINFO and SedDIBitsToDevice. Another problem I have solved was with 8-bit images, which need to have the color table defined.
CImage outImage;
outImage.Create(width, height, channelCount * 8);
int lineSize = width * channelCount;
if (channelCount == 1)
{
// Define the color table
RGBQUAD* tab = new RGBQUAD[256];
for (int i = 0; i < 256; ++i)
{
tab[i].rgbRed = i;
tab[i].rgbGreen = i;
tab[i].rgbBlue = i;
tab[i].rgbReserved = 0;
}
outImage.SetColorTable(0, 256, tab);
delete[] tab;
}
// Copy pixel values
// Warining: does not convert from RGB to BGR
for ( int i = 0; i < height; i++ )
{
void* dst = outImage.GetPixelAddress(0, i);
const void* src = /* put the pointer to the i'th source row here */;
memcpy(dst, src, lineSize);
}