image window not responding opencv during program running - c++

I am running a code something like that;
Mat img1 = imread("C:\\input0.png");
namedWindow("original");
imshow("original", img1 );
int lowThreshold = 20;
int ratio = 2;
int kernel_size = 5;
Mat edge_map;
Mat gray_image;
cvtColor(img1, gray_image, CV_BGR2GRAY);
Canny(gray_image, edge_map, lowThreshold, lowThreshold*ratio, kernel_size);
namedWindow("Edge Image");
imshow("Edge Image", edge_map);
Mat result_image = produce_the_result_image(img1, edge_map);
namedWindow("Final Image");
imshow("Final Image", result_image );
int key = 1;
while (key != 'q') {
key = waitKey(5);
}
Till shows the last image (Final Image window), other windows show 'not responding', however, after the last function finishes (produce_the_result_image), which lasts 2-3 minutos and shows the all image windows, the error disappears. Is that normal?
Thanks!

Images in OpenCV will not display or respond until you have called waitKey();. So if after your first two calls to imshow you call waitKey(1);, the image will display (it will wait one millisecond for a key press and then it will become nonresponsive). If you call waitKey(); with no arguments or use a while loop similar to the one at the end of the code, the images will display and respond. Once you press a key, the images will become nonresponsive again.
So yes, this is normal behavior for OpenCV. Note that it applies to all windows at once: either they are all responsive, if waitKey is currently being called, or none of them are responsive.

Related

how to resize the image through imshow (opencv , imshow )?

I am trying to show an image. but imshow shows the image with big scales so that I can see the image's pixels(as shown in the image below)
as you see pixels are too big. and it is not nice. I expected something like this:
is there any way to resize the image's window size?
I am using Linux vscode
here is my code:
int main()
{
Mat O_image = imread("lena.jpg");
namedWindow("hamid", CV_WINDOW_KEEPRATIO);
imshow("hamid", O_image);
waitKey(0);
return 0;
}
the pointer changes when I move the mouse to the edges but I can't resize it.
You can use a cv::namedWindow with WindowFlags to control the size of the output window.
cv::Mat img;
img = cv::imread("Lena.jpg", CV_LOAD_IMAGE_GRAYSCALE);
cv::namedWindow("A image of Lena", CV_WINDOW_NORMAL); //CV_WINDOW_NORMAL to enable the user to resize the window
cv::imshow("A image of Lena", img);

cv::imshow in opencv is only displaying parts of a composite image, but displaying the parts separately works. Why?

Objective and problem
I'm trying to process a video file on the fly using OpenCV 3.4.1 by grabbing each frame, converting to grayscale, then doing Canny edge detection on it. In order to display the images (on the fly as well), I created a Mat class with 3 additional headers that is three times as wide as the original frame. The 3 extra headers represent the images I would like to display in the composite, and are positioned to the 1st, 2nd and 3rd horizontal segment of the composite.
After image processing however, the display of the composite image is not as expected: the first segment (where the original frame should be) is completely black, while the other segments (of processed images) are displayed fine. If, on the other hand, I display the ROIs one by one in separate windows, all the images look fine.
These are the things I tried to overcome this issue:
use .copyTo to actually copy the data into the appropriate image segments. The result was the same.
I put the Canny image to the compOrigPart ROI, and it did display in the first segment, so it is not a problem with the definition of the ROIs.
Define the composite as three channel image
In the loop convert it to grayscale
put processed images into it
convert back to BGR
put the original in.
This time around the whole composite was black, nothing showed.
As per gameon67's suggestion, I tried to create a namedWindow as well, but that doesn't help either.
Code:
int main() {
cv::VideoCapture vid("./Vid.avi");
if (!vid.isOpened()) return -1;
int frameWidth = vid.get(cv::CAP_PROP_FRAME_WIDTH);
int frameHeight = vid.get(cv::CAP_PROP_FRAME_HEIGHT);
int frameFormat = vid.get(cv::CAP_PROP_FORMAT);
cv::Scalar fontColor(250, 250, 250);
cv::Point textPos(20, 20);
cv::Mat frame;
cv::Mat compositeFrame(frameHeight, frameWidth*3, frameFormat);
cv::Mat compOrigPart(compositeFrame, cv::Range(0, frameHeight), cv::Range(0, frameWidth));
cv::Mat compBwPart(compositeFrame, cv::Range(0, frameHeight), cv::Range(frameWidth, frameWidth*2));
cv::Mat compEdgePart(compositeFrame, cv::Range(0, frameHeight), cv::Range(frameWidth*2, frameWidth*3));
while (vid.read(frame)) {
if (frame.empty()) break;
cv::cvtColor(frame, compBwPart, cv::COLOR_BGR2GRAY);
cv::Canny(compBwPart, compEdgePart, 100, 150);
compOrigPart = frame;
cv::putText(compOrigPart, "Original", textPos, cv::FONT_HERSHEY_PLAIN, 1, fontColor);
cv::putText(compBwPart, "GrayScale", textPos, cv::FONT_HERSHEY_PLAIN, 1, fontColor);
cv::putText(compEdgePart, "Canny edge detection", textPos, cv::FONT_HERSHEY_PLAIN, 1, fontColor);
cv::imshow("Composite of Original, BW and Canny frames", compositeFrame);
cv::imshow("Original", compOrigPart);
cv::imshow("BW", compBwPart);
cv::imshow("Canny", compEdgePart);
cv::waitKey(33);
}
}
Questions
Why can't I display the entirety of the composite image in a single window, while displaying them separately is OK?
What is the difference between these displays? The data is obviously there, as evidenced by the separate windows.
Why only the original frame is misbehaving?
Your compBwPart and compEdgePart are grayscale images so the Mat type is CV8UC1 - single channel and therefore your compositeFrame is in grayscale too. If you want to combine these two images with a color image you have to convert it to BGR first and then fill the compOrigPart.
while (vid.read(frame)) {
if (frame.empty()) break;
cv::cvtColor(frame, compBwPart, cv::COLOR_BGR2GRAY);
cv::Canny(compBwPart, compEdgePart, 100, 150);
cv::cvtColor(compositeFrame, compositeFrame, cv::COLOR_GRAY2BGR);
frame.copyTo(compositeFrame(cv::Rect(0, 0, frameWidth, frameHeight)));
cv::putText(compOrigPart, "Original", textPos, cv::FONT_HERSHEY_PLAIN, 1, fontColor); //the rest of your code
This is a combination of several issues.
The first problem is that you set the type of compositeFrame to the value returned by vid.get(cv::CAP_PROP_FORMAT). Unfortunately that property doesn't seem entirely reliable -- I've just had it return 0 (meaning CV_8UC1) after opening a color video, and then getting 3 channel (CV_8UC3) frames. Since you want to have the compositeFrame the same type as the input frame, this won't work.
To work around it, instead of using those properties, I'd lazy initialize compositeFrame and the 3 ROIs after receiving the first frame (based on it's dimensions and type).
The next set of problems lies in those two statements:
cv::cvtColor(frame, compBwPart, cv::COLOR_BGR2GRAY);
cv::Canny(compBwPart, compEdgePart, 100, 150);
In this case assumption is made that frame is BGR (since you're trying to convert), meaning compositeFrame and its ROIs are also BGR. Unfortunately, in both cases you're writing a grayscale image into the ROI. This will cause a reallocation, and the target Mat will cease to be a ROI.
To correct this, use temporary Mats for the grayscale data, and use cvtColor to turn it back to BGR to write into the ROIs.
Similar problem lies in the following statement:
compOrigPart = frame;
That's a shallow copy, meaning it will just make compOrigPart another reference to frame (and therefore it will cease to be a ROI of compositeFrame).
What you need is a deep copy, using copyTo (note that the data types still need to match, but that was fixed earlier).
Finally, even though you try to be flexible regarding the type of the input video (judging by the vid.get(cv::CAP_PROP_FORMAT)), the rest of the code really assumes that the input is 3 channel, and will break if it isn't.
At the least, there should be some assertion to cover this expectation.
Putting this all together:
#include <opencv2/opencv.hpp>
int main()
{
cv::VideoCapture vid("./Vid.avi");
if (!vid.isOpened()) return -1;
cv::Scalar fontColor(250, 250, 250);
cv::Point textPos(20, 20);
cv::Mat frame, frame_gray, edges_gray;
cv::Mat compositeFrame;
cv::Mat compOrigPart, compBwPart, compEdgePart; // ROIs
while (vid.read(frame)) {
if (frame.empty()) break;
if (compositeFrame.empty()) {
// The rest of code assumes video to be BGR (i.e. 3 channel)
CV_Assert(frame.type() == CV_8UC3);
// Lazy initialize once we have the first frame
compositeFrame = cv::Mat(frame.rows, frame.cols * 3, frame.type());
compOrigPart = compositeFrame(cv::Range::all(), cv::Range(0, frame.cols));
compBwPart = compositeFrame(cv::Range::all(), cv::Range(frame.cols, frame.cols * 2));
compEdgePart = compositeFrame(cv::Range::all(), cv::Range(frame.cols * 2, frame.cols * 3));
}
cv::cvtColor(frame, frame_gray, cv::COLOR_BGR2GRAY);
cv::Canny(frame_gray, edges_gray, 100, 150);
// Deep copy data to the ROI
frame.copyTo(compOrigPart);
// The ROI is BGR, so we need to convert back
cv::cvtColor(frame_gray, compBwPart, cv::COLOR_GRAY2BGR);
cv::cvtColor(edges_gray, compEdgePart, cv::COLOR_GRAY2BGR);
cv::putText(compOrigPart, "Original", textPos, cv::FONT_HERSHEY_PLAIN, 1, fontColor);
cv::putText(compBwPart, "GrayScale", textPos, cv::FONT_HERSHEY_PLAIN, 1, fontColor);
cv::putText(compEdgePart, "Canny edge detection", textPos, cv::FONT_HERSHEY_PLAIN, 1, fontColor);
cv::imshow("Composite of Original, BW and Canny frames", compositeFrame);
cv::imshow("Original", compOrigPart);
cv::imshow("BW", compBwPart);
cv::imshow("Canny", compEdgePart);
cv::waitKey(33);
}
}
Screenshot of the composite window (using some random test video off the web):

Absdiff in openCV can compile but show black image

I have been trying to use absdiff to find the motion in an image,but unfortunately it fail,i am new to OpenCV. The coding supposed to use absdiff to determine whether any motion is happening around or not, but the output is a pitch black for diff1,diff2 and motion. Meanwhile,next_mframe,current_mframe, prev_mframe shows grayscale images. While, result shows a clear and normal image. I use this as my reference http://manmade2.com/simple-home-surveillance-with-opencv-c-and-raspberry-pi/. I think the all the image memory is loaded with the same frame and compare, that explain why its a pitch black. Is there any others method i miss there? I am using RTSP to pass camera RAW image to ROS.
void imageCallback(const sensor_msgs::ImageConstPtr&msg_ptr){
CvPoint center;
int radius, posX, posY;
cv_bridge::CvImagePtr cv_image; //To parse image_raw from rstp
try
{
cv_image = cv_bridge::toCvCopy(msg_ptr, enc::BGR8);
}
catch (cv_bridge::Exception& e)
{
ROS_ERROR("cv_bridge exception: %s", e.what());
return;
}
frame = new IplImage(cv_image->image); //frame now holding raw_image
frame1 = new IplImage(cv_image->image);
frame2 = new IplImage(cv_image->image);
frame3 = new IplImage(cv_image->image);
matriximage = cvarrToMat(frame);
cvtColor(matriximage,matriximage,CV_RGB2GRAY); //grayscale
prev_mframe = cvarrToMat(frame1);
cvtColor(prev_mframe,prev_mframe,CV_RGB2GRAY); //grayscale
current_mframe = cvarrToMat(frame2);
cvtColor(current_mframe,current_mframe,CV_RGB2GRAY); //grayscale
next_mframe = cvarrToMat(frame3);
cvtColor(next_mframe,next_mframe,CV_RGB2GRAY); //grayscale
// Maximum deviation of the image, the higher the value, the more motion is allowed
int max_deviation = 20;
result=matriximage;
//rellocate image in right order
prev_mframe = current_mframe;
current_mframe = next_mframe;
next_mframe = matriximage;
//motion=difflmg(prev_mframe,current_mframe,next_mframe);
absdiff(prev_mframe,next_mframe,diff1); //Here should show black and white image
absdiff(next_mframe,current_mframe,diff2);
bitwise_and(diff1,diff2,motion);
threshold(motion,motion,35,255,CV_THRESH_BINARY);
erode(motion,motion,kernel_ero);
imshow("Motion Detection",result);
imshow("diff1",diff1); //I tried to output the image but its all black
imshow("diff2",diff2); //same here, I tried to output the image but its all black
imshow("diff1",motion);
imshow("nextframe",next_mframe);
imshow("motion",motion);
char c =cvWaitKey(3); }
I change the cv_bridge method to VideoCap, its seem to functions well, cv_bridge just cannot save the image even through i change the IplImage to Mat format. Maybe there is other ways, but as for now, i will go with this method fist.
VideoCapture cap(0);
Tracker(void)
{
//check if camera worked
if(!cap.isOpened())
{
cout<<"cannot open the Video cam"<<endl;
}
cout<<"camera is opening"<<endl;
cap>>prev_mframe;
cvtColor(prev_mframe,prev_mframe,CV_RGB2GRAY); // capture 3 frame and convert to grayscale
cap>>current_mframe;
cvtColor(current_mframe,current_mframe,CV_RGB2GRAY);
cap>>next_mframe;
cvtColor(next_mframe,next_mframe,CV_RGB2GRAY);
//rellocate image in right order
current_mframe.copyTo(prev_mframe);
next_mframe.copyTo(current_mframe);
matriximage.copyTo(next_mframe);
motion = diffImg(prev_mframe, current_mframe, next_mframe);
}

OpenCV Black and White video empty

I have some code to record a video using openCV. It works fine for recording colour video, but I'd like to record black and white.
When I call cvtColor to black and white I get an empty video. I'd really like to know what I'm doing wrong.
VideoCapture cap(1); // open the default camera
cap.set(CV_CAP_PROP_FPS, fps);
cap.set(CV_CAP_PROP_FRAME_WIDTH, 1280);
cap.set(CV_CAP_PROP_FRAME_HEIGHT, 720);
if(!cap.isOpened()) // check if we succeeded
return -1;
VideoWriter writer(filename, CV_FOURCC('M','P','4','2'), fps, Size(1280, 720), true);
int count = 0;
for(;;)
{
count++;
Mat frame;
cap >> frame;
//cvtColor(frame, frame, CV_BGR2GRAY);
writer.write(frame);
}
The above code produces a perfectly fine video, but when cvtColor is uncommented the file is empty.
I was trying to make b/w video with XVID codec and got empty film too (5kB length), until I made FFMPEG libraries available to program (put them together with program or in PATH-ed directory, Windows OS).
OpenCV checks for FFMPEG presence and uses it if available.
And aside remark - use the second Mat for b/w frame - it will save computer time (with single object every capture causes two reallocations/reinitializations)
cvtColor(frame, bwframe, CV_BGR2GRAY);

Cant get output result using cvCornerHarris()

I just want to try the openCV function -- cvCornerHarris. Here is my c++ code:
//image file
char imagePath[256] = "./images/lena512color.tiff";
printf("%s\n", imagePath);
IplImage* srcImg = cvLoadImage(imagePath, 1);
if(NULL == srcImg){
printf("Can not open image file(s).\n");
return -1;
}
IplImage* srcImgGry = cvCreateImage(cvGetSize(srcImg), IPL_DEPTH_8U, 1);
cvCvtColor(srcImg, srcImgGry, CV_RGB2GRAY);
// Canny and Harris expect grayscale (8-bit) input.
// And output of harris image must be 32-bit float .
IplImage* harrisImg = cvCreateImage(cvGetSize(srcImg), IPL_DEPTH_32F, 1);
IplImage* cannyImg = cvCreateImage(cvGetSize(srcImg), IPL_DEPTH_8U, 1);
//// Corner detection using Harris-corner
cvCornerHarris(srcImgGry, harrisImg, 5, 5, 0.04);
cvCanny(srcImgGry, cannyImg, 50, 100, 3);
// (5)Display the result
cvNamedWindow ("Img", CV_WINDOW_AUTOSIZE);
cvShowImage ("Img", srcImgGry);
cvNamedWindow ("Harris", CV_WINDOW_AUTOSIZE);
cvShowImage ("Harris", harrisImg);
cvNamedWindow ("Canny", CV_WINDOW_AUTOSIZE);
cvShowImage ("Canny", cannyImg);
cvWaitKey (0);
cvDestroyWindow ("Harris");
cvDestroyWindow ("Img");
cvReleaseImage (&srcImg);
cvReleaseImage (&srcImgGry);
cvReleaseImage (&harrisImg);
cvReleaseImage (&cannyImg);
I can get a expected output image of cvCanny (cannyImg) but the output image of cvCornerHarris (harrisImg)is an black image with nothing on it.
Please help to explain how to use this function cvCornerHarris. Thanks!
It's all about parameters! People tend to believe that there are magical parameters that will work for all types of images and scenarios. Unfortunately, this doesn't happen in the real world.
The parameters used to process one image may not produce the same level of results when applied to other type of image. Now, consider the following code:
IplImage* colored = cvLoadImage("house.jpg", CV_LOAD_IMAGE_UNCHANGED);
if (!colored)
{
printf("Can not open image file(s).\n");
return -1;
}
IplImage* gray = cvCreateImage(cvGetSize(colored), IPL_DEPTH_8U, 1);
cvCvtColor(colored, gray, CV_RGB2GRAY);
IplImage* harris = cvCreateImage(cvGetSize(colored), IPL_DEPTH_32F, 1);
cvCornerHarris(gray, harris, 3, 11, 0.07);
cvNamedWindow("Harris", CV_WINDOW_AUTOSIZE);
cvShowImage ("Harris", harris);
As you can see below, these parameters produced a decent result (to my point of view). However, keep in mind that they won't probably work for you. Bad parameters will produce a black image (i.e. will detect nothing) as you have observed on your tests.
The answer is: take a look at the docs to see what those parameters mean and how they influence the result. Most importantly, play with them until they produce images that satisfy your needs.
Input image:
(source: 123desenhosparacolorir.com)
Output: