Canon LiveView: image convertion to OpenCV Mat - c++

I'm trying to show LiveView image in real time. I use EDSDK 2.14 + Qt5 + opencv+mingw32 under Windows. I'm not very sophisticated in image processing so now I have the following problem. I use example from Canon EDSDK and all was ok until this part of code:
//
// Display image
//
I googled a lot of examples but all of them was written on C# or MFC or VB. Also I found advise to use libjpegTurbo for decompressing image and then showing it using opencv. I tried to use libjpegTurbo but failed to undestand what to do :(. Maybe somebody here have code example of the conversion LiveView stream to opencv Mat or QImage (because I use Qt)?

Here is what worked for me after following the SAMPLE 10 from the Canon EDSDK Reference. It's a starting point for a more robust solution.
In the downloadEvfData function, I replaced the "Display Image" part by the code bellow:
unsigned char *data = NULL;
EdsUInt32 size = 0;
EdsSize coords ;
// get image coordinates
EdsGetPropertyData(evfImage, kEdsPropsID_Evf_CoordinateSystem, 0, sizeof(coords), &coords);
// get buffer pointer and size
EdsGetPointer(stream, (EdsVoid**)&data);
EdsGetLenth(stream, &size);
//
// release stream and evfImage
//
// create mat object
Mat img(coords.height, coords.width, CV_8U, data);
image = imdecode(img, CV_LOAD_IMAGE_COLOR);
I've also changed the function signature:
EdsError downloadEvfData(EdsCameraRef camera, Mat& image)
And in the main function:
Mat image;
namedWindow("main", WINDOW_NORMAL);
startLiveView(camera);
for(;;) {
dowloadEvfData(camera, image);
imshow("main", image);
if (waitkey(10) >= 0);
break;
}

Based on the Canon EDSDKs example, you may append your EdsStreamRef 'stream' data with its correct length into a QByteArray. Then, use for example the following to parse the raw data from the QByteArray as a JPG into a new QImage:
QImage my_image = QImage::fromData(limagedata,"JPG"); Once it's in a QImage you can convert it into a OpenCV cv::Mat (see How to convert QImage to opencv Mat)

Well it depends on the format of the liveview-stream.
There must be some kind of delimiter in it and you need then to convert each image and update your QImage with it.
Check out this tutorial for more information: Canon EDSDK Tutorial in C#

QImage img = QImage::fromData(data, length, "JPG");
m_image = QImageToMat(img);
// -----------------------------------------
cv::Mat MainWindow::QImageToMat(QImage& src)
{
cv::Mat tmp(src.height(),src.width(),CV_8UC4,(uchar*)src.bits(),src.bytesPerLine());
cv::Mat result = tmp.clone();
return result;
}
// -------------------------
void MainWindow::ShowVideo()
{
namedWindow("yunhu",WINDOW_NORMAL);
while(1)
{
requestLiveViewImage();
if(m_image.data != NULL)
{
imshow("yunhu", m_image);
cvWaitKey(50);
}
}
}

Related

How can I encode and decode the depth image with opencv? I use the following code to get a depth image in webots, but fail. Thanks a lot

// `s` is the webots's rangefinder
auto buffer = (float*)s->getRangeImage();
cv::Mat img(cv::Size(width, height), CV_32FC1, buffer); // image data is float*
// check and success.
// imshow("pic", img);
// cvWaitKey(0);
// encode:
std::vector<uchar> newBuffer;
cv::imencode(".jpg", img, newBuffer);
// decode
auto img_decode = cv::imdecode(newBuffer, cv::IMREAD_UNCHANGED);
// fail and get a black image
imshow("pic", img_decode);
cvWaitKey(0);
I have try to use other params in function cv::imdecode like cv::IMREAD_ANYDEPTH but fail. I am searching for a long time on net. But no use. Please help or try to give some ideas how to achieve this.
If your data is real, i.e. floating point, and single channel, you could try writing to a PFM or TIFF format image which will support that data type.
As #wohlstad mentions in the comments, JPEG is uint8 so only supports integers in the range 0..255.

Absdiff in openCV can compile but show black image

I have been trying to use absdiff to find the motion in an image,but unfortunately it fail,i am new to OpenCV. The coding supposed to use absdiff to determine whether any motion is happening around or not, but the output is a pitch black for diff1,diff2 and motion. Meanwhile,next_mframe,current_mframe, prev_mframe shows grayscale images. While, result shows a clear and normal image. I use this as my reference http://manmade2.com/simple-home-surveillance-with-opencv-c-and-raspberry-pi/. I think the all the image memory is loaded with the same frame and compare, that explain why its a pitch black. Is there any others method i miss there? I am using RTSP to pass camera RAW image to ROS.
void imageCallback(const sensor_msgs::ImageConstPtr&msg_ptr){
CvPoint center;
int radius, posX, posY;
cv_bridge::CvImagePtr cv_image; //To parse image_raw from rstp
try
{
cv_image = cv_bridge::toCvCopy(msg_ptr, enc::BGR8);
}
catch (cv_bridge::Exception& e)
{
ROS_ERROR("cv_bridge exception: %s", e.what());
return;
}
frame = new IplImage(cv_image->image); //frame now holding raw_image
frame1 = new IplImage(cv_image->image);
frame2 = new IplImage(cv_image->image);
frame3 = new IplImage(cv_image->image);
matriximage = cvarrToMat(frame);
cvtColor(matriximage,matriximage,CV_RGB2GRAY); //grayscale
prev_mframe = cvarrToMat(frame1);
cvtColor(prev_mframe,prev_mframe,CV_RGB2GRAY); //grayscale
current_mframe = cvarrToMat(frame2);
cvtColor(current_mframe,current_mframe,CV_RGB2GRAY); //grayscale
next_mframe = cvarrToMat(frame3);
cvtColor(next_mframe,next_mframe,CV_RGB2GRAY); //grayscale
// Maximum deviation of the image, the higher the value, the more motion is allowed
int max_deviation = 20;
result=matriximage;
//rellocate image in right order
prev_mframe = current_mframe;
current_mframe = next_mframe;
next_mframe = matriximage;
//motion=difflmg(prev_mframe,current_mframe,next_mframe);
absdiff(prev_mframe,next_mframe,diff1); //Here should show black and white image
absdiff(next_mframe,current_mframe,diff2);
bitwise_and(diff1,diff2,motion);
threshold(motion,motion,35,255,CV_THRESH_BINARY);
erode(motion,motion,kernel_ero);
imshow("Motion Detection",result);
imshow("diff1",diff1); //I tried to output the image but its all black
imshow("diff2",diff2); //same here, I tried to output the image but its all black
imshow("diff1",motion);
imshow("nextframe",next_mframe);
imshow("motion",motion);
char c =cvWaitKey(3); }
I change the cv_bridge method to VideoCap, its seem to functions well, cv_bridge just cannot save the image even through i change the IplImage to Mat format. Maybe there is other ways, but as for now, i will go with this method fist.
VideoCapture cap(0);
Tracker(void)
{
//check if camera worked
if(!cap.isOpened())
{
cout<<"cannot open the Video cam"<<endl;
}
cout<<"camera is opening"<<endl;
cap>>prev_mframe;
cvtColor(prev_mframe,prev_mframe,CV_RGB2GRAY); // capture 3 frame and convert to grayscale
cap>>current_mframe;
cvtColor(current_mframe,current_mframe,CV_RGB2GRAY);
cap>>next_mframe;
cvtColor(next_mframe,next_mframe,CV_RGB2GRAY);
//rellocate image in right order
current_mframe.copyTo(prev_mframe);
next_mframe.copyTo(current_mframe);
matriximage.copyTo(next_mframe);
motion = diffImg(prev_mframe, current_mframe, next_mframe);
}

Open CV image saving in smaller size without compressions

The question i have is the following:
I have a camera( with resolution Resolution : 640 x 480 px) and I get an image from that camera (I get an 8 bit/ pixel grayscale image) after the image acquisition I save the image in a bmp format. My code is the followig :
Mat img2(640,480,CV_8UC1,0);
cap.read(img2);
bool succes = imwrite("D:\\TestImage3.bmp",img2);
if(!succes){
cout << "Failed to save the image";
return -1;
}
namedWindow("myWindow",CV_WINDOW_AUTOSIZE);
imshow("myWindow",img2);
The saved image is very large almost 1 MB and i want a smaller image without losing any information (without compresing the image)???
The second question on this topic is:
even if the image is gray some times I still get some rgb noise, its like I would have set a 3 channel setting instead of 1 channel setting for my image
If anyone knows the answer please let me know, I would be very grateful
Thanks for your time!
You can save your image as PNG which is an lossless image compression format.
bool succes = imwrite("D:\\TestImage3.png",img2);
With the cv::imwrite function you can pass additional parameters depending on the image format.
PNG is a lossless image format but you can still chose the level of compression for example :
Mat img2;
cap.read(img2);
cvtColor(img2, img2, CV_BGR2GRAY); // Convert to single channel
vector<int> compression_params;
compression_params.push_back(CV_IMWRITE_PNG_COMPRESSION);
compression_params.push_back(9);
bool succes = imwrite("D:\\TestImage3.bmp", img2, compression_params);
if(!succes)
{
cout << "Failed to save the image"; return -1;
}
imshow("myWindow",img2);
waitKey(0);
Just use the default constructor for Mat with no params.
Mat img2;
cap.read(img2);
cvtColor(img2, img2, CV_BGR2GRAY); // Convert to single channel
bool succes = imwrite("D:\\TestImage3.bmp", img2);
if(!succes)
{
cout << "Failed to save the image"; return -1;
}
imshow("myWindow",img2);
waitKey(0);
Also, bmp is known for its large uncompressed size. Use .png instead.

How to convert QImage to opencv Mat

I have searched on the internet but I cannot find a method of converting a QImage(or QPixmap) to a OpenCV Mat. How would I do this?.
Any help is appreciated.
If the QImage will still exist, and you just need to perform a quick operation on it then you can construct a cv::Mat using the QImage memory:
cv::Mat mat(image.height(), image.width(), CV_8UC3, (cv::Scalar*)image.scanLine(0));
This assumes that the QImage is 3-channels, ie RGB888.
If the QImage is going away then you need to copy the data, see Qimage to cv::Mat convertion strange behaviour.
If QImage is Format_ARGB32_Premultiplied (the preferred format) then you will need to convert each pixel to OpenCV's BGR layout. The cv::cvtcolor() function can convert ARGB to RGB in the latest versions.
Or you can use QImage::convertToFormat() to convert to RGB before copying the data.
One year after you issued this question there've been great answers on the internet:
Convert between cv::mat and Qimage correctly
Converting Between cv::Mat and QImage or QPixmap
But the way I see it, if you're working with Qt and OpenCV at the same time then type QImage is probably just for displaying, that case you might want to use QPixmap since it's optimized for displaying. So this is what I do:
Load image as cv::Mat, if you'd like to display the image, convert to QPixmap using the non-copy method introduced in the second article.
Do your image processing in cv::Mat.
Any time during the workflow you can call upon something like Mat2QPixmap() to get realtime result.
Never convert QPixmap to cv::Mat, there's no sense doing it considering the purpose of each type.
The answer to this with Qt 5.11 (and probably some earlier versions):
cv::Mat mat(image.height(), image.width(),CV_8UC3, image.bits());
// image.scanline() does not exist,
//and height/width is interchanged for a matrix
Again the QImage is assumed to be RGB888 (ie QImage::Format_RGB888)
My attempt in OpenCV 3.1+ style code:
void qimage_to_mat(const QImage& image, cv::OutputArray out) {
switch(image.format()) {
case QImage::Format_Invalid:
{
cv::Mat empty;
empty.copyTo(out);
break;
}
case QImage::Format_RGB32:
{
cv::Mat view(image.height(),image.width(),CV_8UC4,(void *)image.constBits(),image.bytesPerLine());
view.copyTo(out);
break;
}
case QImage::Format_RGB888:
{
cv::Mat view(image.height(),image.width(),CV_8UC3,(void *)image.constBits(),image.bytesPerLine());
cvtColor(view, out, cv::COLOR_RGB2BGR);
break;
}
default:
{
QImage conv = image.convertToFormat(QImage::Format_ARGB32);
cv::Mat view(conv.height(),conv.width(),CV_8UC4,(void *)conv.constBits(),conv.bytesPerLine());
view.copyTo(out);
break;
}
}
}
void mat_to_qimage(cv::InputArray image, QImage& out)
{
switch(image.type())
{
case CV_8UC4:
{
cv::Mat view(image.getMat());
QImage view2(view.data, view.cols, view.rows, view.step[0], QImage::Format_ARGB32);
out = view2.copy();
break;
}
case CV_8UC3:
{
cv::Mat mat;
cvtColor(image, mat, cv::COLOR_BGR2BGRA); //COLOR_BGR2RGB doesn't behave so use RGBA
QImage view(mat.data, mat.cols, mat.rows, mat.step[0], QImage::Format_ARGB32);
out = view.copy();
break;
}
case CV_8UC1:
{
cv::Mat mat;
cvtColor(image, mat, cv::COLOR_GRAY2BGRA);
QImage view(mat.data, mat.cols, mat.rows, mat.step[0], QImage::Format_ARGB32);
out = view.copy();
break;
}
default:
{
throw invalid_argument("Image format not supported");
break;
}
}
}
cv::Mat to_cvmat(QImage img)
{
img = img.convertToFormat(QImage::Format_RGB888, Qt::ColorOnly).rgbSwapped();
return cv::Mat(img.height(), img.width(), CV_8UC3, img.bits(), img.bytesPerLine()).clone();
}

OpenCV : convert the pointer in memory to image

I have a grabber which can get the images and show them on the screen with the following code
while((lastPicNr = Fg_getLastPicNumberBlockingEx(fg,lastPicNr+1,0,10,_memoryAllc))<200) {
iPtr=(unsigned char*)Fg_getImagePtrEx(fg,lastPicNr,0,_memoryAllc);
::DrawBuffer(nId,iPtr,lastPicNr,"testing"); }
but I want to use the pointer to the image data and display them with OpenCV, cause I need to do the processing on the pixels. my camera is a CCD mono camera and the depth of the pixels is 8bits. I am new to OpenCV, is there any option in opencv that can get the return of the (unsigned char*)Fg_getImagePtrEx(fg,lastPicNr,0,_memoryAllc); and disply it on the screen? or get the data from the iPtr pointer an allow me to use the image data?
Creating an IplImage from unsigned char* raw_data takes 2 important instructions: cvCreateImageHeader() and cvSetData():
// 1 channel for mono camera, and for RGB would be 3
int channels = 1;
IplImage* cv_image = cvCreateImageHeader(cvSize(width,height), IPL_DEPTH_8U, channels);
if (!cv_image)
{
// print error, failed to allocate image!
}
cvSetData(cv_image, raw_data, cv_image->widthStep);
cvNamedWindow("win1", CV_WINDOW_AUTOSIZE);
cvShowImage("win1", cv_image);
cvWaitKey(10);
// release resources
cvReleaseImageHeader(&cv_image);
cvDestroyWindow("win1");
I haven't tested the code, but the roadmap for the code you are looking for is there.
If you are using C++, I don't understand why your are not doing it the simple way like this:
If your camera is supported, I would do it this way:
cv::VideoCapture capture(0);
if(!capture.isOpened()) {
// print error
return -1;
}
cv::namedWindow("viewer");
cv::Mat frame;
while( true )
{
capture >> frame;
// ... processing here
cv::imshow("viewer", frame);
int c = cv::waitKey(10);
if( (char)c == 'c' ) { break; } // press c to quit
}
I would recommend starting to read the docs and tutorials which you can find here.