Qt - Transform cv::Mat to QImage in worker thread crashes - c++

Question

Introduction
What i want is really simple: I want to start the process of reading a binary file via a button in a main ui thread, let a seperate thread handle the processing including transformation into QImage and return this image back to the main ui thread where it should shown in a label.
Therefore i use Qt's signal/slot mechanism and its thread functionality.
I allready have a single thread working solution, but when using the threading attempt it crashes on totally arbitrary steps which i don't understand because the whole process is totally encapsulated and not time critical...
Working single-threading solution:
TragVisMain is a QMainWindow:
class TragVisMain : public QMainWindow
Pushing the button readBinSingle starts the process:
void TragVisMain::on_readBinSingle_clicked()
{
// Open binary file
FILE *imageBinFile = nullptr;
imageBinFile = fopen("imageFile.bin", "rb");
if(imageBinFile == NULL) {
return;
}
// Get binary file size
fseek(imageBinFile, 0, SEEK_END); // seek to end of file
size_t size = static_cast<size_t>(ftell(imageBinFile)); // get current file pointer
fseek(imageBinFile, 0, SEEK_SET);
// Read binary file
void *imageData = malloc(size);
fread(imageData, 1, size, imageBinFile);
// Create cv::Mat
cv::Mat openCvImage(1024, 1280, CV_16UC1, imageData);
openCvImage.convertTo(openCvImage, CV_8UC1, 0.04); // Convert to 8 Bit greyscale
// Transform to QImage
QImage qImage(
openCvImage.data,
1280,
1024,
QImage::Format_Grayscale8
);
// Show image in label, 'imageLabel' is class member
imageLabel.setPixmap(QPixmap::fromImage(qImage));
imageLabel.show();
}
This works like a charm, but of course the ui is blocked.
Not working multi-threading solution
As you will see the code is basically the same as above just moving to another class DoCameraStuff. So here is are the main components for this purpose in the TragVisMain header file:
namespace Ui {
class TragVisMain;
}
class TragVisMain : public QMainWindow
{
Q_OBJECT
public:
explicit TragVisMain(QWidget *parent = nullptr);
~TragVisMain();
private:
Ui::TragVisMain *ui;
DoCameraStuff dcs;
QLabel imageLabel;
QThread workerThread;
public slots:
void setImage(const QImage &img); // Called after image processing
// I have also tried normal parameter 'setImage(QImage img)', non-const referenc 'setImage(const QImage &img)' and pointer 'setImage(QImage *img)'
private slots:
void on_readBin_clicked(); // Emits 'loadBinaryImage'
// void on_readBinSingle_clicked();
signals:
void loadBinaryImage(); // Starts image processing
};
DoCameraStuff is just a QObject:
class DoCameraStuff : public QObject
{
Q_OBJECT
public:
explicit DoCameraStuff(QObject *parent = nullptr);
public slots:
void readBinaryAndShowBinPic();
signals:
void showQImage(const QImage &image);
};
Moving dcs to the workerThread and connecting signals and slots happens in the constructor of TragVisMain:
TragVisMain::TragVisMain(QWidget *parent) :
QMainWindow(parent),
ui(new Ui::TragVisMain),
dcs()
{
ui->setupUi(this);
dcs.moveToThread(&workerThread);
// Object should be deletable after the thread finished!
connect(&workerThread, &QThread::finished, &dcs, &QObject::deleteLater);
// For starting the image processing
connect(this, &TragVisMain::loadBinaryImage, &dcs, &DoCameraStuff::readBinaryAndShowBinPic);
// Showing QImage after image processing is done
connect(&dcs, &DoCameraStuff::showQImage, this, &TragVisMain::setImage);
// Start workerThread
workerThread.start();
}
Starting the image processing happens through pressing the readBinSingle button:
void TragVisMain::on_readBin_clicked()
{
emit loadBinaryImage(); // slot: readBinaryAndShowBinPic
}
The process happens in DoCameraStuff::readBinaryAndShowBinPic:
void DoCameraStuff::readBinaryAndShowBinPic() {
// Exact same code from single thread solution:
// Open binary file
FILE *imageBinFile = nullptr;
imageBinFile = fopen("imageFile.bin", "rb");
if(imageBinFile == NULL) {
return;
}
// Get binary file size
fseek(imageBinFile, 0, SEEK_END); // seek to end of file
size_t size = static_cast<size_t>(ftell(imageBinFile)); // get current file pointer
fseek(imageBinFile, 0, SEEK_SET);
// Read binary file
void *imageData = malloc(size);
fread(imageData, 1, size, imageBinFile);
// Create cv::Mat
cv::Mat openCvImage(1024, 1280, CV_16UC1, imageData);
openCvImage.convertTo(openCvImage, CV_8UC1, 0.04); // Convert to 8 Bit greyscale
// Transform to QImage
QImage qImage(
openCvImage.data,
1280,
1024,
QImage::Format_Grayscale8
);
// Send qImage to 'TragVisMain'
emit showQImage(qImage);
}
Showing the image in TragVisMain::setImage:
void TragVisMain::setImage(const QImage &img)
{
imageLabel.setPixmap(QPixmap::fromImage(img));
imageLabel.show();
}
Problem
Well the multi-threading attempt just crashed the whole Application without any message during different steps. But honestly i don't have any idea. For me the DoCameraStuff class is a standard worker class doing time-independent stuff in a member function without any critical relations.
I also checked if some of the used function inside DoCameraStuff::readBinaryAndShowBinPic are not thread-safe but I couldn't find any problems regarding <cstdio>, cv::Mat and QImage in equivalent conditions.
So:
Why does the multi-threading attempt crash?
What changes need to be applied, so that the process in the thread does not crash?
I allways appreciate your help.

Answer 1

It doesn't have a chance of working, since the QImage wraps transient data that then is deallocated by cvMat. At the minimum you should emit the copy of the image (literally qImage.copy(). Ideally, you'd wrap the cvMat lifetime management in QImage's deleter, so that the copy is not necessary, and the matrix gets destroyed along with the image that wraps it. Since format conversions are usually needed between cv::Mat and QImage, it's probably best to perform the conversion between the source cv::Mat and another cv::Mat that wraps the QImage-owned memory. That solution avoids memory reallocations, since the QImage can be retained in the class performing the conversion. It is then also comaptible with Qt 4, where QImage doesn't support deleters.
See this answer for a complete example of an OpenCV video capture Qt-based widget viewer, and this answer for a complete example of multi-format conversion from cv::Mat to QImage.