Memory error on displaying opencv matrix - c++

Question

I made a class consuming a serialized Opencv matrix. It is working fine and the Matrix is deserialized. If I try to display it inside the method of the class using the imshow method, it will work perfectly well, displaying without error. However, I am passing a parameter reference to a Matrix pointer from my main function in order to further process this matrix. When I try to display it in the main, I end up with a segmentation fault.
The weird part is that if I try to display the matrix in both the class method and the Main, I will end up with two windows working fine (sometimes I will get a segmentation fault but most of the time it works good).
If I remove the displaying code from the method, I won't even be able to display one frame before getting a segmentation fault.
I have tried using a shared_ptr, passing a pointer to the pointer instead of a reference, or even simply returning the value.
My code is kind of messy but the main aim of it is testing.
Here is the code of the method :
void VideoConsumer::getVideoFrame(cv::Mat* &mat) {
Message msg = _consumer->poll();
mat = NULL;
if(!msg) {
cerr << "No message received" << endl;
return;
}
if(msg.get_error()) {
if(!msg.is_eof()) {
cerr << "[+] Received error notification: " << msg.get_error() << endl;
}
return;
}
Document document;
string jsonPayload = "";
for(auto i=msg.get_payload().begin(); i != msg.get_payload().end();i++) {
jsonPayload += *i;
}
document.Parse(jsonPayload.c_str());
if(document.HasMember("rows") && document.HasMember("cols") && document.HasMember("data")) {
int rows = document["rows"].GetInt();
int cols = document["cols"].GetInt();
int type = document["type"].GetInt();
string data = document["data"].GetString();
std::vector<BYTE> decodedBytes = base64_decode(data);
stringstream ss;
for(int i=0;i< decodedBytes.size(); i++) {
ss << decodedBytes[i];
}
string decoded_data = ss.str();
cout << "Constructed string" << endl;
mat = new cv::Mat(rows,cols,type,(void *)decoded_data.data());
/*cv::imshow("test",*mat);
while(cv::waitKey(10) != 27)*/ //This is where it is displayed
return;
} else {
return;
}
}
And the code in the main :
...
if(parser.has("stream")) {
VideoConsumer consumer("localhost:9092","video-stream-topic","testId2");
consumer.setConsumer();
while(1) {
Mat *frame = NULL;
consumer.getVideoFrame(frame);
if(frame == NULL) {
cout << "Null frame" << endl;
continue;
}
if(!frame->empty() && frame->rows > 0 && frame->cols > 0) {
imshow("Test",*frame);
waitKey(10);
frame->release();
}
}
}
I am completely out of ideas and have tried every single thing I knew or found on my researches.
EDIT : Added frame->release() in order to free the allocation, still same issue.

Answer 1

There is a problem in your matrix initialization... Specifically in here:
mat = new cv::Mat(rows,cols,type,(void *)decoded_data.data());
That one is this constructor
Mat (int rows, int cols, int type, void *data, size_t step=AUTO_STEP)
which in the documentation says the following about the *data parameter
data Pointer to the user data. Matrix constructors that take data and
step parameters do not allocate matrix data. Instead, they just
initialize the matrix header that points to the specified data, which
means that no data is copied. This operation is very efficient and can
be used to process external data using OpenCV functions. The external
data is not automatically deallocated, so you should take care of it.
This means that as soon it goes out of scope (the function exits) the string which you created (decoded_data) will exit and the data will be deallocated by the string, and then your cv::Mat will have a reference to a data that is not valid anymore...
You can always initialize the matrix with something like
cv::Mat(rows,cols,type)
and then use something like std::memcpy or similar to copy the data to the mat.data member. Actually, AFAIK it is not needed to pass byte to string and then to mat which is casted to void and then to uchar....
try something like:
mat = cv::Mat(rows,cols,type);
std::memcpy(&decodedBytes[0], mat.data, decodedBytes.size());
Just a small warning for this solution, you need to do a check that decodedBytes is not empty and that mat.data has enough space to receive all the contents of decodedBytes. To do this check just make sure:
// size in bytes to copy == size of the allocated data of mat in bytes
decodedBytes.size() == (mat.elemSize() * mat.rows * mat.cols)
A couple of remarks more that may not be an issue now, but may bite you later:
Do not use cv::Mat pointers... The behaviour of cv::Mat is already like a smart pointer.
Beware of the casting/copy of data from sign to unsign and viceversa :) I think now it is done correctly, but this may become a problem later.