How to convert QImage to opencv Mat - c++

I have searched on the internet but I cannot find a method of converting a QImage(or QPixmap) to a OpenCV Mat. How would I do this?.
Any help is appreciated.

If the QImage will still exist, and you just need to perform a quick operation on it then you can construct a cv::Mat using the QImage memory:
cv::Mat mat(image.height(), image.width(), CV_8UC3, (cv::Scalar*)image.scanLine(0));
This assumes that the QImage is 3-channels, ie RGB888.
If the QImage is going away then you need to copy the data, see Qimage to cv::Mat convertion strange behaviour.
If QImage is Format_ARGB32_Premultiplied (the preferred format) then you will need to convert each pixel to OpenCV's BGR layout. The cv::cvtcolor() function can convert ARGB to RGB in the latest versions.
Or you can use QImage::convertToFormat() to convert to RGB before copying the data.

One year after you issued this question there've been great answers on the internet:
Convert between cv::mat and Qimage correctly
Converting Between cv::Mat and QImage or QPixmap
But the way I see it, if you're working with Qt and OpenCV at the same time then type QImage is probably just for displaying, that case you might want to use QPixmap since it's optimized for displaying. So this is what I do:
Load image as cv::Mat, if you'd like to display the image, convert to QPixmap using the non-copy method introduced in the second article.
Do your image processing in cv::Mat.
Any time during the workflow you can call upon something like Mat2QPixmap() to get realtime result.
Never convert QPixmap to cv::Mat, there's no sense doing it considering the purpose of each type.

The answer to this with Qt 5.11 (and probably some earlier versions):
cv::Mat mat(image.height(), image.width(),CV_8UC3, image.bits());
// image.scanline() does not exist,
//and height/width is interchanged for a matrix
Again the QImage is assumed to be RGB888 (ie QImage::Format_RGB888)

My attempt in OpenCV 3.1+ style code:
void qimage_to_mat(const QImage& image, cv::OutputArray out) {
switch(image.format()) {
case QImage::Format_Invalid:
{
cv::Mat empty;
empty.copyTo(out);
break;
}
case QImage::Format_RGB32:
{
cv::Mat view(image.height(),image.width(),CV_8UC4,(void *)image.constBits(),image.bytesPerLine());
view.copyTo(out);
break;
}
case QImage::Format_RGB888:
{
cv::Mat view(image.height(),image.width(),CV_8UC3,(void *)image.constBits(),image.bytesPerLine());
cvtColor(view, out, cv::COLOR_RGB2BGR);
break;
}
default:
{
QImage conv = image.convertToFormat(QImage::Format_ARGB32);
cv::Mat view(conv.height(),conv.width(),CV_8UC4,(void *)conv.constBits(),conv.bytesPerLine());
view.copyTo(out);
break;
}
}
}
void mat_to_qimage(cv::InputArray image, QImage& out)
{
switch(image.type())
{
case CV_8UC4:
{
cv::Mat view(image.getMat());
QImage view2(view.data, view.cols, view.rows, view.step[0], QImage::Format_ARGB32);
out = view2.copy();
break;
}
case CV_8UC3:
{
cv::Mat mat;
cvtColor(image, mat, cv::COLOR_BGR2BGRA); //COLOR_BGR2RGB doesn't behave so use RGBA
QImage view(mat.data, mat.cols, mat.rows, mat.step[0], QImage::Format_ARGB32);
out = view.copy();
break;
}
case CV_8UC1:
{
cv::Mat mat;
cvtColor(image, mat, cv::COLOR_GRAY2BGRA);
QImage view(mat.data, mat.cols, mat.rows, mat.step[0], QImage::Format_ARGB32);
out = view.copy();
break;
}
default:
{
throw invalid_argument("Image format not supported");
break;
}
}
}

cv::Mat to_cvmat(QImage img)
{
img = img.convertToFormat(QImage::Format_RGB888, Qt::ColorOnly).rgbSwapped();
return cv::Mat(img.height(), img.width(), CV_8UC3, img.bits(), img.bytesPerLine()).clone();
}

Related

Converting cv::mat to QImage

like the title says I am trying to convert a cv::mat to a QImage. What I am doing is using the equalizeHist() function on the mat and then converting it to a QImage to display in widget window in Qt. I know the mat works and loads the image correctly because the equalized image will show in the new window with imshow(), however when converting this mat to a QImage, I can not get it to display in the window. I believe the problem is with the conversion from the mat to QImage but cant find the issue. Below is a part of my code snippet.
Mat image2= imread(directoryImage1.toStdString(),0);
//cv::cvtColor(image2,image2,COLOR_BGR2GRAY);
Mat histEquImg;
equalizeHist(image2,histEquImg);
imshow("Histogram Equalized Image 2", histEquImg);
//QImage img=QImage((uchar*) histEquImg.data, histEquImg.cols, histEquImg.rows, histEquImg.step, QImage::Format_ARGB32);
imageObject= new QImage((uchar*) histEquImg.data, histEquImg.cols, histEquImg.rows, histEquImg.step, QImage::Format_RGB888);
image = QPixmap::fromImage(*imageObject);
scene=new QGraphicsScene(this); //create a frame for image 2
scene->addPixmap(image); //put image 1 inside of the frame
ui->graphicsView_4->setScene(scene); //put the frame, which contains image 3, to the GUI
ui->graphicsView_4->fitInView(scene->sceneRect(),Qt::KeepAspectRatio); //keep the dimension ratio of image 3
No errors occur and the program doesnt crash.
Thanks in advance.
Your problem is the conversion of the QImage to cv::Mat, when using the flag 0 in cv::imread implies the reading is grayscale, and you are using the conversion with the format QImage::Format_RGB888. I use the following function to make the conversion of cv::Mat to QImage:
static QImage MatToQImage(const cv::Mat& mat)
{
// 8-bits unsigned, NO. OF CHANNELS=1
if(mat.type()==CV_8UC1)
{
// Set the color table (used to translate colour indexes to qRgb values)
QVector<QRgb> colorTable;
for (int i=0; i<256; i++)
colorTable.push_back(qRgb(i,i,i));
// Copy input Mat
const uchar *qImageBuffer = (const uchar*)mat.data;
// Create QImage with same dimensions as input Mat
QImage img(qImageBuffer, mat.cols, mat.rows, mat.step, QImage::Format_Indexed8);
img.setColorTable(colorTable);
return img;
}
// 8-bits unsigned, NO. OF CHANNELS=3
if(mat.type()==CV_8UC3)
{
// Copy input Mat
const uchar *qImageBuffer = (const uchar*)mat.data;
// Create QImage with same dimensions as input Mat
QImage img(qImageBuffer, mat.cols, mat.rows, mat.step, QImage::Format_RGB888);
return img.rgbSwapped();
}
return QImage();
}
After that I see that you have misconceptions of how QGraphicsView and QGraphicsScene work when commenting: put the frame, which contains image 3, to the GUI, with ui->graphicsView_4->setScene(scene); you are not setting a frame but a scene, and the scene should only be set once and preferably in the constructor.
// constructor
scene = new QGraphicsScene(this);
ui->graphicsView->setScene(scene);
So when you want to load the image just use the scene:
cv::Mat image= cv::imread(filename.toStdString(), CV_LOAD_IMAGE_GRAYSCALE);
cv::Mat histEquImg;
equalizeHist(image, histEquImg);
QImage qimage = MatToQImage(histEquImg);
QPixmap pixmap = QPixmap::fromImage(qimage);
scene->addPixmap(pixmap);
ui->graphicsView->fitInView(scene->sceneRect(), Qt::KeepAspectRatio);
The complete example can be found in the following link.

Displaying opencv Mat image into QGraphicsView [duplicate]

//Sorry for my english.
Tell me please, what I am doing wrong?
I have read a lot about this. And write some code, but I have a terrible result.
As I understand
in Opencv CV_8UC3 is the same as QImage::Format_RGB888 , except BRG and RGB accordingly.
to read cv::Mat in this format I can do:
cv::Mat mat1 = cv::imread("bugero.jpg",3);
So, to convert cv::Mat to QImage I can do:
QImage Mat2QImage(cv::Mat const& src)
{
cv::Mat temp(src.cols,src.rows,src.type());
cvtColor(src, temp,CV_BGR2RGB);
QImage dest= QImage((uchar*) temp.data, temp.cols, temp.rows, temp.step, QImage::Format_RGB888);
return dest;
}
I made temp mat becouse I want to have copy of data in QImage.
Then.
To convert it back I Have to do:
cv::Mat QImage2Mat(QImage const& src)
{
QImage temp = src.copy();
cv::Mat res(temp.height(),temp.width(),CV_8UC3,(uchar*)temp.bits(),temp.bytesPerLine());
cvtColor(res, res,CV_BGR2RGB);
return res;
}
I have inserted cvtColor(res, res,CV_BGR2RGB); to make cv Mat with BGR colors. I do not exactly know what in inside this function cvtColor(res, res,CV_BGR2RGB);, But I decided
that if cvtColor(res, res,CV_BGR2RGB); change places R and B, that will chage places of this colors back, because I did not found CV_BGR2RGB.
So, I wrote short sample program
#include <QApplication>
#include <QtGui>
#include <cv.h>
#include "opencv2/highgui/highgui.hpp"
QImage Mat2QImage(cv::Mat const& src)
{
cv::Mat temp(src.cols,src.rows,src.type()); // make the same cv::Mat
cvtColor(src, temp,CV_BGR2RGB); // cvtColor Makes a copt, that what i need
QImage dest= QImage((uchar*) temp.data, temp.cols, temp.rows, temp.step, QImage::Format_RGB888);
return dest;
}
cv::Mat QImage2Mat(QImage const& src)
{
QImage temp = src.copy();
cv::Mat res(temp.height(),temp.width(),CV_8UC3,(uchar*)temp.bits(),temp.bytesPerLine());
cvtColor(res, res,CV_BGR2RGB); // make convert colort to BGR !
return res;
}
int main(int argc, char *argv[])
{
QApplication a(argc, argv);
QWidget W1;
QWidget W2;
QLabel imlab1(&W1);
QLabel imlab2(&W2);
W1.setWindowTitle("Convert cv::Mat to QImage First time");
W2.setWindowTitle("Convert cv::Mat to QImage Second time");
cv::Mat mat1 = cv::imread("bugero.jpg",3);
QImage qim1 = Mat2QImage(mat1);
cv::Mat mat2 = QImage2Mat(qim1);
QImage qim2 = Mat2QImage(mat2);
cv::Mat mat3 = QImage2Mat(qim2);
cv::imshow("First Mat",mat1);
imlab1.setPixmap(QPixmap::fromImage(qim1));
W1.setFixedSize(qim1.size());
cv::imshow("Convert QImage to cv::Mat firstly",mat2);
imlab2.setPixmap(QPixmap::fromImage(qim2));
W2.setFixedSize(qim2.size());
cv::imshow("Convert QImage to cv::Mat secondly",mat2);
W1.show();
W2.show();
return a.exec();
}
and .pro file
INCLUDEPATH += /usr/local/include/opencv /usr/local/include/opencv2
LIBS += -lopencv_core -lopencv_imgproc\
-lopencv_highgui
QT += gui
QT += core
SOURCES += \
QcvMat.cpp \
And I have got a BAD result!!!
Is there some? People,I need help!
I added some debug info to get cv::Mat.step and QImage.bytesPerLine() and it is different.
alex#lenovo /media/Files/Programming/Cpp/tests/QImagecvMat $ ./QcvMat
cv step 942
QImage bytesPerLine 944
cv step 942
QImage bytesPerLine 944
What does it means and may be problem in it?
Code looks fine with one exception.
Memory management. cv::Mat doesn't work like QImage in this mater. Remember that QImage is using copy on write mechanism and shares memory for each copy.
cv::Mat also shares memory but it doesn't do copy on write (I'm also new with open cv (2 weeks) so I can't explain yet exactly how it works but I've stumbled on some crushes because of that)!
Another thing is that when you are creating QImage from memory image is using this memory and doesn't take ownership of it.
Final outcome is that on Linux and Qt5 your code is crashes because of problems with memory management. On your screen shot you can see at the top of second window that something strange is going on and you see some memory trash.
So I've corrected your conversion functions it works perfectly:
QImage Mat2QImage(cv::Mat const& src)
{
cv::Mat temp; // make the same cv::Mat
cvtColor(src, temp,CV_BGR2RGB); // cvtColor Makes a copt, that what i need
QImage dest((const uchar *) temp.data, temp.cols, temp.rows, temp.step, QImage::Format_RGB888);
dest.bits(); // enforce deep copy, see documentation
// of QImage::QImage ( const uchar * data, int width, int height, Format format )
return dest;
}
cv::Mat QImage2Mat(QImage const& src)
{
cv::Mat tmp(src.height(),src.width(),CV_8UC3,(uchar*)src.bits(),src.bytesPerLine());
cv::Mat result; // deep copy just in case (my lack of knowledge with open cv)
cvtColor(tmp, result,CV_BGR2RGB);
return result;
}
So we both have to do a reading about memory management in open-CV :).
OFFTOPIC:
Best way to include openCV in qt projects on Linux is to add to pro file something like:
# add open CV
unix {
CONFIG += link_pkgconfig
PKGCONFIG += opencv
}
You will be free of path problems when moving code to another machine.
Thanks a Lot! It is realy works!
But. Why does memory whas spoiled?
First. I have some memory load incv::Mat
cv::Mat mat1 = cv::imread("bugero.jpg",3);
mat1 - [=====================================]
then I put a copy of this cvMat to other cv:Mat
cv::Mat temp(src.cols,src.rows,src.type());
cvtColor(src, temp,CV_BGR2RGB);
mat1 - [=========================================]
temp - [=========================================]
then make QImage from this data
QImage dest= QImage((uchar*) temp.data, temp.cols, temp.rows, temp.step, QImage::Format_RGB888);
mat1 - [============================================]
temp - > [============================================]
/
dest --/
And then temp goes out of scope and delete it self? QImage does not take ownership of it so memory in temp1 and dest marked as free, and there compiler can put other data? Is I am right?

convert Qimage to cvMat 64FC3 format

i have searched a lot on the internet but i have only found how to convert Qimage to RGB format, i want to convert an Qimage to cv mat format CV_64FC3.
i have really bad results when i work with CV_8UC3
here is my code :
QImage myImage;
myImage.load("C://images//PolarImage300915163358.bmp");
QLabel myLabel;
myLabel.setPixmap(QPixmap::fromImage(myImage));
//myLabel.show();
cv::Mat image1 = QImage2Mat(myImage);
Mat img;
image1.convertTo(img, CV_64FC3, 1.0 / 255.0);
and here is the function that i used :
cv::Mat QImage2Mat(QImage const& src)
{
cv::Mat tmp(src.height(),src.width(),CV_8UC3,(uchar*)src.bits(),src.bytesPerLine());
cv::Mat result; // deep copy just in case (my lack of knowledge with open cv)
cvtColor(tmp, result,CV_BGR2RGB);
return result;
}
please help me i m new to both opencv and Qt
Not sure what you mean with bad results, but you are assuming that QImage also loads the image as OpenCV (BGR). In the documentation it tells you that they use ARGB.
So, knowing this you have 2 options:
Convert to QImage::Format_RGB888 the Qimage using the function convertToFormat and then this line cvtColor(tmp, result,CV_BGR2RGB); is not needed, since it will be already in RGB.
Use CV_8UC4 when creating the cv::Mat and then drop the first channel (channel alpha) using either split and join or mixchannels.
i have found what was going wrong, in fact, Qimage has a fourth channel for alpha so when you read the Qimage data you need to put it in CV_8UC4
here is the code :
Mat QImage2Mat(const QImage& src) {
cv::Mat mat = cv::Mat(src.height(), src.width(), CV_8UC4, (uchar*)src.bits(), src.bytesPerLine());
cv::Mat result = cv::Mat(mat.rows, mat.cols, CV_8UC3 );
int from_to[] = { 0,0, 1,1, 2,2 };
cv::mixChannels( &mat, 1, &result, 1, from_to, 3 );
return result;
}

Qt OpenCV - SIGSEGV when displaying transformed frames

I have an app that has to pull frames from video, transform one a little, transform one a lot, and simultaneously display them in GUI. In a worker thread, there's an OpenCV loop:
while(1) {
cv::VideoCapture kalibrowanyPlik;
kalibrowanyPlik.open(kalibracja.toStdString()); //open file from url
int maxFrames = kalibrowanyPlik.get(CV_CAP_PROP_FRAME_COUNT);
for(int i=0; i<maxFrames; i++) //I thought it crashed when finished reading the first time around
{
cv::Mat frame;
cv::Mat gray;
cv::Mat color;
kalibrowanyPlik.read(frame);
cv::cvtColor(frame, gray, CV_BGR2GRAY);
cv::cvtColor(frame, color, CV_BGR2RGB);
QImage image((uchar*)color.data, color.cols, color.rows,QImage::Format_RGB888);
QImage processedImage((uchar*)gray.data, gray.cols, gray.rows,QImage::Format_Indexed8);
emit progressChanged(image, processedImage);
QThread::msleep(50);
}
}
And this is how frames are placed in GUI
void MainWindow::onProgressChagned(QImage image, QImage processedImage) {
QPixmap processed = QPixmap::fromImage(processedImage);
processed = processed.scaledToHeight(379);
ui->labelHsv->clear();
ui->labelHsv->setPixmap(processed);
QPixmap original = QPixmap::fromImage(image); //debug points SIGSEGV here
original = original.scaledToHeight(379);
ui->labelKalibracja->clear();
ui->labelKalibracja->setPixmap(original);
}
The RGB image always crashes, grayscale image never crashes (tested). Why is the RGB image crashing?
edit: I've just discovered that if I change msleep(50) to msleep(100) it executes perfectly. But I don't want that. I need at least 25 frames per second, 10 is not acceptable... why would that cause a SIGSEGV
Standard issue. Problem is memory management!
See my other answer. In comments there is a good link.
So in your code QImage doesn't copy and doesn't take ownership of memory of matrix. And later on when matrix is destroyed and QImage tries access this memory (QImage is copied by creating shallow copy) you have a segfault.
Here is a code form this link (I've tweak it a bit), for some reason this site has some administration issues (some quota exceeded), that is why I'm pasting it here.
inline QImage cvMatToQImage( const cv::Mat &inMat )
{
switch ( inMat.type() )
{
// 8-bit, 4 channel
case CV_8UC4:
{
QImage image( inMat.data, inMat.cols, inMat.rows, inMat.step, QImage::Format_RGB32 );
QImage copy(image);
copy.bits(); //enforce deep copy
return copy;
}
// 8-bit, 3 channel
case CV_8UC3:
{
QImage image( inMat.data, inMat.cols, inMat.rows, inMat.step, QImage::Format_RGB888 );
return image.rgbSwapped();
}
// 8-bit, 1 channel
case CV_8UC1:
{
static QVector<QRgb> sColorTable;
// only create our color table once
if ( sColorTable.isEmpty() )
{
for ( int i = 0; i < 256; ++i )
sColorTable.push_back( qRgb( i, i, i ) );
}
QImage image( inMat.data, inMat.cols, inMat.rows, inMat.step, QImage::Format_Indexed8 );
image.setColorTable( sColorTable );
QImage copy(image);
copy.bits(); //enforce deep copy
return copy;
}
default:
qWarning() << "ASM::cvMatToQImage() - cv::Mat image type not handled in switch:" << inMat.type();
break;
}
return QImage();
}
Your code should utilize this functions like that:
while(1) {
cv::VideoCapture kalibrowanyPlik;
kalibrowanyPlik.open(kalibracja.toStdString()); //open file from url
int maxFrames = kalibrowanyPlik.get(CV_CAP_PROP_FRAME_COUNT);
for(int i=0; i<maxFrames; i++) //I thought it crashed when finished reading the first time around
{
cv::Mat frame;
cv::Mat gray;
kalibrowanyPlik.read(frame);
cv::cvtColor(frame, gray, CV_BGR2GRAY);
QImage image(cvMatToQImage(frame));
QImage processedImage(cvMatToQImage(gray));
emit progressChanged(image, processedImage);
QThread::msleep(10); // this is bad see comments below
}
}
Use of msleep is in 95% cases bad! Remove this loop and create slot which will be invoked by signal from QTimer.
Another solution will be to use a timer :
void ??::timerEvent(QTimerEvent*){
if(kalibrowanssky.isOpened())
cv::Mat frame;
cv::Mat gray;
cv::Mat color;
kalibrowanyPlik.read(frame);
cv::cvtColor(frame, gray, CV_BGR2GRAY);
cv::cvtColor(frame, color, CV_BGR2RGB);
ui->labelHsv->setPixmap(QPixmap::fromImage(Mat2QImage(color)));
ui->labelKalibracja->setPixmap(QPixmap::fromImage(Mat2QImage(gray)));
}
In your main :
cv::VideoCapture kalibrowanyPlik;
startTimer(1000/25); // 25 frames by second
And the function Mat2QImage (I found it here : how to convert an opencv cv::Mat to qimage) :
QImage ??::Mat2QImage(cv::Mat const& src) {
cv::Mat temp;
cvtColor(src, temp,CV_BGR2RGB);
QImage dest((const uchar *) temp.data, temp.cols, temp.rows, temp.step, QImage::Format_RGB888);
dest.bits();
return dest;
}

Canon LiveView: image convertion to OpenCV Mat

I'm trying to show LiveView image in real time. I use EDSDK 2.14 + Qt5 + opencv+mingw32 under Windows. I'm not very sophisticated in image processing so now I have the following problem. I use example from Canon EDSDK and all was ok until this part of code:
//
// Display image
//
I googled a lot of examples but all of them was written on C# or MFC or VB. Also I found advise to use libjpegTurbo for decompressing image and then showing it using opencv. I tried to use libjpegTurbo but failed to undestand what to do :(. Maybe somebody here have code example of the conversion LiveView stream to opencv Mat or QImage (because I use Qt)?
Here is what worked for me after following the SAMPLE 10 from the Canon EDSDK Reference. It's a starting point for a more robust solution.
In the downloadEvfData function, I replaced the "Display Image" part by the code bellow:
unsigned char *data = NULL;
EdsUInt32 size = 0;
EdsSize coords ;
// get image coordinates
EdsGetPropertyData(evfImage, kEdsPropsID_Evf_CoordinateSystem, 0, sizeof(coords), &coords);
// get buffer pointer and size
EdsGetPointer(stream, (EdsVoid**)&data);
EdsGetLenth(stream, &size);
//
// release stream and evfImage
//
// create mat object
Mat img(coords.height, coords.width, CV_8U, data);
image = imdecode(img, CV_LOAD_IMAGE_COLOR);
I've also changed the function signature:
EdsError downloadEvfData(EdsCameraRef camera, Mat& image)
And in the main function:
Mat image;
namedWindow("main", WINDOW_NORMAL);
startLiveView(camera);
for(;;) {
dowloadEvfData(camera, image);
imshow("main", image);
if (waitkey(10) >= 0);
break;
}
Based on the Canon EDSDKs example, you may append your EdsStreamRef 'stream' data with its correct length into a QByteArray. Then, use for example the following to parse the raw data from the QByteArray as a JPG into a new QImage:
QImage my_image = QImage::fromData(limagedata,"JPG"); Once it's in a QImage you can convert it into a OpenCV cv::Mat (see How to convert QImage to opencv Mat)
Well it depends on the format of the liveview-stream.
There must be some kind of delimiter in it and you need then to convert each image and update your QImage with it.
Check out this tutorial for more information: Canon EDSDK Tutorial in C#
QImage img = QImage::fromData(data, length, "JPG");
m_image = QImageToMat(img);
// -----------------------------------------
cv::Mat MainWindow::QImageToMat(QImage& src)
{
cv::Mat tmp(src.height(),src.width(),CV_8UC4,(uchar*)src.bits(),src.bytesPerLine());
cv::Mat result = tmp.clone();
return result;
}
// -------------------------
void MainWindow::ShowVideo()
{
namedWindow("yunhu",WINDOW_NORMAL);
while(1)
{
requestLiveViewImage();
if(m_image.data != NULL)
{
imshow("yunhu", m_image);
cvWaitKey(50);
}
}
}