opencv Mat deallocation memory corruption - c++

I am struggling with release version of my opencv wrapper function.
The function code runs fine, but upon function block completition, memory access violation happens.
This problem does not appear in debug mode. The segfault happens upon freeing the heap.
int Myfunc(Arr1D_floatHdl FeatArrHdl, IMAQ_Image *img, someparams
*Params)
{
ImageInfo *Info = NULL;
//IplImage *CVImage = NULL;
Info = (ImageInfo*)img->address;
CheckImage(Info, Info);
//CVImage = cvCreateImageHeader( cvSize(Info->xRes, Info->yRes), IPL_DEPTH_8U, 4);
//CVImage->imageData = (char*)Info->imageStart;
//CVImage->widthStep = Info->xRes*sizeof(IPL_DEPTH_8U);
cv::Mat BGRAimg = cv::Mat(Info->yRes, Info->xRes, CV_8UC4, (char*)Info->imageStart, sizeof(CV_8UC4)*Info->xRes);
//cv::Mat BGRAimg(CVImage);
//cv::Mat BGRAimg = imread( "MyImg.png", cv::IMREAD_COLOR );
cv::Mat GREYimg;
cv::cvtColor(BGRAimg, GREYimg, CV_BGR2GRAY);
Here is the code where I create Mat object from user supplied data.
I tried to create IplImage first (commented version in code) and use Mat constructor with IplImage argument, but eneded up with the same problem.
I know I am doing something very wrong during the Mat construction, since manualy loading the image from disk does not cause the issue.
After creating the Mat object, all its parameters are correct and the image is fine. When comparing with the grey matrix created of it, it has refcount NULL, which I have read is perfectly fine since it is supposed to keep user data intact.
Please help.
UPDATE to give more information
Thank you for suggestions. I am obviously prone to create such errors, I am new to C/C++.
Unfortunately, the access violation still persists.
Here is the complete wrapper function as it is. I tried to narrow down the problem, and skipping the HOG.compute function I do no longer get memory corruption. Skipping the memcpy acrobatics in the end, I still get the memory corrupted.
int GetHOGFeatures(Arr1D_floatHdl FeatArrHdl, IMAQ_Image *img, HogParams *Params) //returns -1 on HOG window parameters missmatch
{
ImageInfo *Info = NULL;
Info = (ImageInfo*)img->address;
CheckImage(Info, Info);
cv::Mat BGRAimg = cv::Mat(Info->yRes, Info->xRes, CV_8UC4, (char*)Info->imageStart, sizeof(cv::Vec4b)*Info->xRes);
cv::Mat GREYimg;
cv::cvtColor(BGRAimg, GREYimg, CV_BGRA2GRAY);
//set params into hog object
cv::HOGDescriptor hog;
hog.winSize = cv::Size(Params->winsize_width, Params->winsize_height);
hog.blockSize = cv::Size(Params->blocksize_width, Params->blocksize_height);
hog.blockStride = cv::Size(Params->blockstride_x, Params->blockstride_y);
hog.cellSize = cv::Size(Params->cellsize_width, Params->cellsize_height);
hog.nbins = Params->nBins;
hog.derivAperture = Params->derivAperture;
hog.winSigma = Params->win_sigma;
hog.L2HysThreshold = Params->threshold_L2hys;
hog.gammaCorrection = (Params->gammaCorrection != 0);
MgErr error = mgNoErr;
cv::vector<float> ders;
cv::vector<cv::Point> locations;
try
{
//winstride - step of window
//padding - borderpadding
//raises exception with incorrect params ... todo replace trycatch with paramchecking
hog.compute(GREYimg, ders, cv::Size(Params->winstride_x, Params->winstride_y), cv::Size(0,0), locations);
}
catch(...)
{
return -1;
}
//copy out the data into LabView
error = DSSetHandleSize(FeatArrHdl, sizeof(int32_t) + ders.size()*sizeof(float));
memcpy((*FeatArrHdl)->Arr, ders.data(), sizeof(float)*ders.size());
(*FeatArrHdl)->dimSize = ders.size();
return error;
}
I am running this function with following parameters:
Window size 32
Block size 16
Cell size 8
Block stride 8
Window stride 32
the rest of parameters is default.
I decided to include the look of the Mat object once constructed, I hope it can help.
This is the BGRA constructed from user data. It is supposed to be 640*640 BGRA
BGRAimg {flags=1124024344 dims=2 rows=640 ...} cv::Mat
flags 1124024344 int
dims 2 int
rows 640 int
cols 640 int
data 0x12250040 "e9%" unsigned char *
101 'e' unsigned char
refcount 0x00000000 int *
CXX0030: Error: expression cannot be evaluated
datastart 0x12250040 "e9%" unsigned char *
101 'e' unsigned char
dataend 0x123e0040 "" unsigned char *
0 unsigned char
datalimit 0x123e0040 "" unsigned char *
0 unsigned char
allocator 0x00000000 cv::MatAllocator *
__vfptr CXX0030: Error: expression cannot be evaluated
size {p=0x0012f44c } cv::Mat::MSize
p 0x0012f44c int *
640 int
step {p=0x0012f474 buf=0x0012f474 } cv::Mat::MStep
p 0x0012f474 unsigned int *
2560 unsigned int
buf 0x0012f474 unsigned int [2]
[0] 2560 unsigned int
[1] 4 unsigned int
And the Grey image that enters the HOG descriptors calculator
GREYimg {flags=1124024320 dims=2 rows=640 ...} cv::Mat
flags 1124024320 int
dims 2 int
rows 640 int
cols 640 int
refcount 0x0c867ff0 int *
1 int
dataend 0x0c867ff0 "" unsigned char *
1 '' unsigned char
datalimit 0x0c867ff0 "" unsigned char *
1 '' unsigned char
allocator 0x00000000 cv::MatAllocator *
__vfptr CXX0030: Error: expression cannot be evaluated
size {p=0x0012f40c } cv::Mat::MSize
p 0x0012f40c int *
640 int
step {p=0x0012f434 buf=0x0012f434 } cv::Mat::MStep
p 0x0012f434 unsigned int *
640 unsigned int
buf 0x0012f434 unsigned int [2]
[0] 640 unsigned int
[1] 1 unsigned int
I had to ommit the data and datastart fields, because unlike for the BGRA image MSVS actually shows some data in it.
UPDATE2
changed multi-threaded for multi-threaded DLL in project properities, and the issue is gone.
The problem persisted even if I was using code like this :
int dim = 32;
BYTE *mydata = NULL;
mydata = (BYTE*)malloc(sizeof(BYTE)*dim*dim);
Mat img;
img = Mat(Size(dim,dim), CV_8U, mydata, dim*sizeof(BYTE));
Might this indicate my code was not the cause, and this is somewhat opencv x windows runtime issue, or did I just hide the problem ?
UPDATE3
After reading something about microsoft runtime, I decided to check how was my opencv built, and it is using /MD, and I was building with /MT. I hope this was the cause.

this might not work like you expect:
sizeof(CV_8UC4)*Info->xRes
CV_8UC4 is an enum, not a type, you can't use sizeof() here.
if your data is continuous, you probably might just skip the stride param completely, or:
sizeof(Vec4b)*Info->xRes
another thing:
your BGRAimg has 4 channels, right ? so, use
cv::cvtColor(BGRAimg, GREYimg, CV_BGRA2GRAY);
instead

Related

What causes this segmentation fault (core dumped) error at cudaMemcpy when copying to GPU?

I have been trying to fix segmentation fault (core dumped) error messages with a toy program when calling cudaMemcpy. It works for small images, but for bigger images it normally fails; I say normally, because it has sometimes succeeded when debugging with valgrind (more about that below). I have looked at similar questions, but have been unable to find the answer; sorry if this is a duplicate! I am just learning out (and following programming massively parallel processors).
Here is my code, cleaned up:
#include <opencv2/core.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/highgui.hpp>
#include "opencv2/imgproc/imgproc.hpp"
#include <cuda.h>
#include <iostream>
#include <cuda_runtime.h>
using namespace cv;
using namespace std;
__global__ void
colorToGreyKernel(unsigned char* outPic, unsigned char* inPic, unsigned int width, unsigned int height){
// printf("trying \n" );
int Col = blockDim.x * blockIdx.x + threadIdx.x;
int Row = blockDim.y * blockIdx.y + threadIdx.y;
if( Col < width && Row < height){
int greyOffset = Row * width + Col;
int rgbOffset = greyOffset * 3;
unsigned char b = inPic[rgbOffset];
unsigned char g = inPic[rgbOffset +1];
unsigned char r = inPic[rgbOffset +2];
outPic[greyOffset] = 0.21f*r + 0.71f*g + 0.07f*b;
}
}
#define gpuErrchk(ans) { gpuAssert((ans), __FILE__, __LINE__); }
inline void gpuAssert(cudaError_t code, const char *file, int line, bool abort=true)
{
bool test = code == cudaSuccess;
// cout << "code " << std::boolalpha<< test;
if (code != cudaSuccess)
{
// const char *errorStr = NULL;
fprintf(stderr,"GPUassert: %s %s %d\n", cudaGetErrorString(code), file, line);
if (abort) exit(code);
}
}
int main(int argc, char** argv )
{
if ( argc != 2 )
{
printf("usage: DisplayImage.out <Image_Path>\n");
return -1;
}
Mat image;
unsigned int imSize[2] = {400,400};
unsigned char* inPic = NULL;
unsigned char* outPic = NULL;
gpuErrchk(cudaMalloc(&inPic, imSize[0] * imSize[1] * 3 * sizeof(CV_8U)));
gpuErrchk(cudaMalloc(&outPic, imSize[0] * imSize[1] * sizeof(CV_8U)));
image = imread( argv[1], IMREAD_COLOR );
resize(image, image, Size(imSize[0],imSize[1]));
Mat greyImg(image.rows, image.cols, CV_8U, Scalar(125));
size_t size = image.cols * image.rows * image.channels() * sizeof(CV_8U);
// This is where it always fails for bigger images
gpuErrchk(cudaMemcpy(inPic,(void*) &image.data[0], size, cudaMemcpyHostToDevice));
gpuErrchk(cudaMemcpy(outPic, (void*)&greyImg.data[0], size/3, cudaMemcpyHostToDevice));
dim3 dimGrid(ceil(image.rows/16.0),ceil(image.cols/16.0),1);
dim3 dimBlock(16,16,1);
colorToGreyKernel<<<dimGrid, dimBlock>>>(outPic, inPic,(int) image.rows,(int) image.cols);
cudaDeviceSynchronize();
gpuErrchk(cudaGetLastError());
gpuErrchk(cudaMemcpy(greyImg.data, outPic, size / 3, cudaMemcpyDeviceToHost));
namedWindow("Display Image", WINDOW_AUTOSIZE );
imshow("Display Image", greyImg);
waitKey(0);
cudaFree(&inPic[0]);
cudaFree(&outPic[0]);
return 0;
}
I'm able to allocate on the device, but the copying fails for bigger images. I've tried it using opencv::cuda, and I can load any picture and do cvtColor on the device without resizing, so I conclude it's not memory (similar when looking at nvidia-smi).
When I run using valgrind, I get a lot of Invalid write of size 8 errors around this point, all referencing to libcuda. I know it's this particular memcopy that's the problem, by isolating it. Sometimes it also works in valgrind, but I've gathered that this is normal. I don't have experience with valgrind yet, but the memory issues don't make sense to me (I'm trying to copy to the device, so why a segmentation fault which is related to the host?).
My question is simple, where does the error come from and how to fix this?
NVCC = 11.1
gpu = GeForce GTX 960M (not a lot, but that shouldn't matter)
Again, I am new to programming in Cuda, but have tried what I can think of and can not isolate the problem! Thanks for your help.
The problem here relates to your usage of OpenCV. An item like CV_8U is not a type, it is a compiler #define. Therefore sizeof(CV_8U) is not doing what you think it is doing. Your intended usage should be to capture the size of the underlying type (e.g. unsigned char, i.e. a type size of 1). However, sizeof(CV_8U) returns evidently the size of an integer, which is 4.
As a result of that, your calculation of size is wrong (4x too large). As a result of that, when the cudaMemcpy operation attempts to access &image.data[0] for size bytes, it will attempt to copy past the end of the buffer. For small images, the overrun doesn't trigger the run time check/limit. For a large enough size calculation (large enough image) you will hit a seg fault. Although the failure is triggered within a CUDA call, the origin of the error is outside of CUDA.
One possible solution is to replace your usage of sizeof(CV_8U) with something like sizeof(unsigned char). Since that size is 1, you can also just delete the multiplication by sizeof(CV_8U) and get the same behavior.
You can also avoid this sort of allocation and let OpenCV do the allocation (and host-device data copying) work for you as demonstrated in the answer here and here

pass Mat object C++ to Unity

I'd like to return a Mat object to Unity from c++ code. However i get access violation error at c++ part like that
Unity Editor [version: Unity 2017.3.0f3_a9f86dcd79df]
SaliencyCV.dll caused an Access Violation (0xc0000005)
in module SaliencyCV.dll at 0033:270027f0.
Error occurred at 2018-03-06_235212.
C:\Program Files\Unity\Editor\Unity.exe, run by Dilara.
43% memory in use.
16266 MB physical memory [9199 MB free].
18698 MB paging file [9861 MB free].
134217728 MB user address space [134185466 MB free].
Read from location 990d0000 caused an access violation.
Here is c++ code:
uchar* cppMethod(uchar* frameData, int WIDTH, int HEIGHT, int* rows, int* cols)
{
Mat img(HEIGHT, WIDTH, CV_8UC3);
img.data = frameData;
flip(img, img, 0);
Mat result = calculateSaliency(img);
*rows = result.rows;
*cols = result.cols;
int length = result.rows * result.cols * 3;
uchar* tmpArr = result.data;
uchar* resultArray = new uchar[length];
for (int i = 0; i < length; i++)
{
resultArray[i] = tmpArr[i];
}
return resultArray;
}
Can someone help me?
You should call the correct Mat constructor, which accepts external data pointer, to make the object not release/destruct the corresponding memory location data points to. You can read about this behaviour in Mat::release().
The problem with your code is that
Mat img(HEIGHT, WIDTH, CV_8UC3) allocates a memory block of type CV_8UC3 of size HEIGHT*WIDTH, which is not used (because you are changing the data member variable to point to a different memory location, anyways),
At function exit, img is destructed, which results in a call to release(), which in turn destructs frameData, which is not the intended behaviour.
Change your first two lines to read
Mat img(HEIGHT, WIDTH, CV_8UC3, frameData);
And if you are passing resultArray to C#, where you are most likely not managing the pointed-to-memory's lifetime, you would be most likely having memory leaks. #Programmer has already suggested in his answer to your previous question that you should allocate the memory in C#, pass it to C++, and write in-place in the C++ side.
In short, you should have something like:
#include <algorithm>
void cppMethod(uchar *frameData, uchar *out, const int WIDTH, const int HEIGHT,
int *rows, int *cols) {
/* this constructor will not manage frameData's lifetime */
Mat img(HEIGHT, WIDTH, CV_8UC3, frameData);
/* in-place operation */
flip(img, img, 0);
/* local variable --- it will be destructed properly */
Mat result = calculateSaliency(img);
/* well-defined if rows and cols are scalars passed by reference */
*rows = result.rows;
*cols = result.cols;
/* make sure length will not overflow */
int length = result.rows * result.cols * 3;
/* you don't need this */
// uchar *tmpArr = result.data;
/* you sholuld NOT do this */
// uchar *resultArray = new uchar[length];
// use std::copy from <algorithm>
// for (int i = 0; i < length; i++) {
// resultArray[i] = tmpArr[i];
// }
std::copy(result.data, result.data + length, out);
// return resultArray;
}

How can I use openimageIO to store RGB values in arrays? (using C++, OpenGL)

I am using openimageIO to read and display an image from a JPG file, and I now need to store the RGB values in arrays so that I can manipulate and re-display them later.
I want to do something like this:
for (int i=0; i<picturesize;i++)
{
Rarray[i]=pixelredvalue;
Garray[i]=pixelgreenvalue;
Barray[i]=pixelbluevalue;
}
This is an openimageIO source that I found online: https://people.cs.clemson.edu/~dhouse/courses/404/papers/openimageio.pdf
"Section 3.2: Advanced Image Output" (pg 35) is the closest to what I'm doing, but I don't understand how I can use the channels to write pixel data to arrays. I also don't fully understand the difference between "writing" and "storing in an array". This is the piece of code in the reference that I am talking about:
int channels = 4;
ImageSpec spec (width, length, channels, TypeDesc::UINT8);
spec.channelnames.clear ();
spec.channelnames.push_back ("R");
spec.channelnames.push_back ("G");
spec.channelnames.push_back ("B");
spec.channelnames.push_back ("A");
I managed to read the image and display it using the code in the reference, but now I need to store all the pixel values in my array.
Here is another useful piece of code from the link, but again, I can't understand how to retrieve the individual RGB values and place them into an array:
#include <OpenImageIO/imageio.h>
OIIO_NAMESPACE_USING
...
const char *filename = "foo.jpg";
const int xres = 640, yres = 480;
const int channels = 3; // RGB
unsigned char pixels[xres*yres*channels];
ImageOutput *out = ImageOutput::create (filename);
if (! out)
return;
ImageSpec spec (xres, yres, channels, TypeDesc::UINT8);
out->open (filename, spec);
out->write_image (TypeDesc::UINT8, pixels);
out->close ();
ImageOutput::destroy (out);
But this is about writing to a file, and still does not solve my problem. This is on page 35.
Let's assume, that your code which reads an image, looks like this (snippet from OpenImageIO 1.7 Programmer Documentation, Chapter 4.1 Image Input Made Simple, page 55):
ImageInput *in = ImageInput::open (filename);
const ImageSpec &spec = in->spec();
int xres = spec.width;
int yres = spec.height;
int channels = spec.nchannels;
std::vector<unsigned char> pixels (xres*yres*channels);
in->read_image (TypeDesc::UINT8, &pixels[0]);
in->close();
ImageInput::destroy (in);
Now all the bytes of the image are contained in std::vector<unsigned char> pixels.
If you want to access the RGB valuse of the pixel at positon x, y, the you can do it like this:
int pixel_addr = (y * yres + x) * channels;
unsigned char red = pixels[pixel_addr];
unsigned char green = pixels[pixel_addr + 1];
unsigned char blue = pixels[pixel_addr + 2];
Since all the pixels are stored in pixels, there is no reason to store them in separate arrays for the 3 color channels.
But if you want to store the red, green and blue values in separated arrays, then you can do it like this:
std::vector<unsigned char> Rarray(x_res*yres);
std::vector<unsigned char> Garray(x_res*yres);
std::vector<unsigned char> Barray(x_res*yres);
for (int i=0; i<x_res*yres; i++)
{
Rarray[i] = pixels[i*channels];
Garray[i] = pixels[i*channels + 1];
Barray[i] = pixels[i*channels + 2];
}
Of course the pixels have to be tightly packed to pixels (line alignment of 1).

Saving a QImage from a memory buffer

Here is basically what I'm doing :
I have a video stream (YUV format). Each frame is extracted into a buffer (frameBytes). Later, this buffer is used to do the YUV->RGB conversion, then transfered in an IplImage. The IplImage is then transfered into a cv::Mat and displayed in an OpenGL context. Everything works fine.
What I would like to do is bypass the IplImage and cv::Mat section to directly work with the frameBytes buffer in OpenGL and do the conversion in the shaders.
This explaination is just for the context, since the problem I'm having is simpler.
To see if I can work with the buffer earlier, I try to copy it with memcpy and then save it in a QImage then in a file.
Here is my code for this part :
unsigned char *mycopy = new unsigned char[1920*1080*3];
memcpy(mycopy, frameBytes, sizeof(1920*1080*3));
QImage *img = new QImage(mycopy, 1920, 1080, QImage::Format_RGB888);
img->save("image.jpg",0,-1);
frameBytes contains the YUV data from the video stream. I know it's YUV and I'm trying to create a QImage with RGB888 format but since QImage doesn't support the format, I didn't make the conversion there, I thought it would still save an image but with the wrong colors so I don't care for the moment (Maybe this assumption is wrong ?).
Problem is, the image saved is black.
Just for more information, here is an example where I use frameBytes for the YUV->RGB conversion.
void DeckLinkCaptureDelegate::convertFrameToOpenCV(void* frameBytes, IplImage * m_RGB){
if(!m_RGB) m_RGB = cvCreateImage(cvSize(1920, 1080), IPL_DEPTH_8U, 3);
unsigned char* pData = (unsigned char *) frameBytes;
for(int i = 0, j=0; i < 1920 * 1080 * 3; i+=6, j+=4)
{
unsigned char u = pData[j];
unsigned char y = pData[j+1];
unsigned char v = pData[j+2];
//fprintf(stderr, "%d\n", v);
m_RGB->imageData[i+2] = 1.0*y + 8 + 1.402*(v-128); // r
m_RGB->imageData[i+1] = 1.0*y - 0.34413*(u-128) - 0.71414*(v-128); // g
m_RGB->imageData[i] = 1.0*y + 1.772*(u-128) + 0; // b
y = pData[j+3];
m_RGB->imageData[i+5] = 1.0*y + 8 + 1.402*(v-128); // r
m_RGB->imageData[i+4] = 1.0*y - 0.34413*(u-128) - 0.71414*(v-128); // g
m_RGB->imageData[i+3] = 1.0*y + 1.772*(u-128) + 0;
}
}
Fixing the bug
You didn't copy all the data to your new buffer:
unsigned char *mycopy = new unsigned char[1920*1080*3];
memcpy(mycopy, frameBytes, sizeof(1920*1080*3));
That sizeof in there means that you're only copying an int-sized block, rather than 6MB. It looks like an accidental holdover from using a static array? Replace it with
const size_t bufsize = 1920*1080*3;
auto *mycopy = new unsigned char[bufsize];
memcpy(mycopy, frameBytes, bufsize);
A simpler approach
Alternatively, instead of doing the memory allocation yourself (and being responsible for delete[]ing it after the QImage is destructed), you could copy the image instead:
const unsigned char *bytes = frameBytes;
QImage img = QImage(bytes, 1920, 1080, QImage::Format_RGB888).copy();
The way this works is that we create a temporary QImage using frameBytes as its source (we pass it as pointer to const to insist it's read-only). We then copy() the whole of it to a new QImage, throwing away the temporary. The copy() does something similar to your code above, but we're now saved from having to do the calculations, eliminating some of the consequent potential for error.
Note also that I prefer to pass QImage by value. Although this might seem inefficient, most (copyable) Qt types are designed as reference-counted copy-on-write structures, and can be safely used this way, eliminating another class of errors (memory management). This is true of Qt's collection types, too, and very useful.

How do I correctly load the char array of a grayscale BMP using CImage?

I have the following code:
cout<<"Please enter the name of your BMP image file: "<<endl;
cin>>fname;
nP = fname.c_str();
CImage input = CImage();
input.Load(nP);
// allocate space for host source image
unsigned char *pHI, *pCI;
width = input.GetWidth();
height = input.GetHeight();
pCI = (unsigned char *)input.GetBits();
pHI = (unsigned char *)malloc(sizeof(unsigned char) * width * height);
// fill array with CImage array content
srand (time(NULL));
for (int cnt = 0; cnt < sizeof(unsigned char) * width * height; cnt++){
pHI[cnt] = pCI[cnt];
}
But the program gives me an error when I try to get the width and height.
"Debug Assertion Failed! ... Expression: m_hBitmap !=0"
If you have any ideas as to what could be causing this / what I should change, I'd appreciate the help!
: )
First thing to check is that input.Load() is successful. It returns an HRESULT and you should check the value of this. It's going to be a clue as to what's going on.
Link to CImage::Load()
You can interpret what an HRESULT means here:
Details on HRESULT
Good luck, but more information is needed.
After edit: Further more, you can only use CImage::Load() for DIB sections. See this link for more info: CImage class reference