Sorry if the title gave you the wrong idea, I tried to make it as brief as possible. In short, what I am trying to do is detect a face with Viola-Jones algorithm (already implemented), save it in a separate image, convert that image to grayscale, then slap the grayscale image back into its' original position, resulting in a webcam display with all faces (and any false positives, I suppose) colored gray and surrounded by a green rectangle. However, I get the following error message:
Unhandled exception at 0x771115de in proba.exe: Microsoft C++ exception: cv::Exception at >memory location 0x003ef2c8..
This is my code (the relevant part), any suggestion/advice would be appreciated:
face_cascade.detectMultiScale( frame_gray, faces, 1.1, 2, 0|CV_HAAR_SCALE_IMAGE, Size(30, 30) );
for( int i = 0; i < faces.size(); i++ )
{
Point pt1(faces[i].x + faces[i].width, faces[i].y + faces[i].height);
Point pt2(faces[i].x, faces[i].y);
Rect myROI(pt1, pt2);
Mat croppedImage;
Mat(frame, myROI).copyTo(croppedImage);
cvtColor(croppedImage, croppedImage, CV_BGR2GRAY ); //the last four lines process the image
croppedImage.copyTo(frame(Rect(pt1, croppedImage.size()))); //this should copy the image back into its' original location
rectangle(frame, pt1, pt2, cvScalar(0, 255, 0, 0), 1, 8, 0);
}
//-- Show what you got
imshow( window_name, frame );
And sorry if I'm missing the obvious answer.
Your cropped grayscale image croppedImage is a 1 channel image but you are trying to overlay it onto 3 channel RGB image frame. In other words, the function copyTo in
croppedImage.copyTo(frame(Rect(pt1, croppedImage.size())));
expects croppedImage to have the same number of channels as frame. This is why you are getting the error.
EDIT To solve your issue you may try convert your grayscale cropped image back to RGB format (it will still look like the grayscale image). Something like
cvtColor(croppedImage, croppedImage, CV_BGR2GRAY ); // to grayscale
cvtColor(croppedImage, croppedImage, CV_GRAY2BGR ); // to RGB
croppedImage.copyTo(frame(Rect(pt1, croppedImage.size())));
Related
]2
I would like to get a square shape from the right image above. But when I try to get it, it also includes other protruding parts because they have similar color. Are there any solutions to get the result like below? (The square lines are not 100 % straight. They are little distorted.)
This is the code I wrote.
cv::Mat img_gray, img, clahe_img, threshold_img, bitwise_img, morph_img;
cv::Mat rectified_CCD_img = cv::imread('img.png')
cv::Mat kernel = cv::Mat::ones(99, 99, CV_8U);
clahe = cv::createCLAHE(10, cv::Size(100, 100));
cv::cvtColor(rectified_CCD_img, img_gray, cv::COLOR_BGR2GRAY);
cv::medianBlur(img_gray, img, 33);
clahe->apply(img, clahe_img);
cv::threshold(clahe_img, threshold_img, 0, 255, cv::THRESH_OTSU);
cv::bitwise_not(threshold_img, bitwise_img);
cv::morphologyEx(bitwise_img, morph_img, cv::MORPH_OPEN, kernel);
That's the original image:
Google Drive link
For this specific image my pipeline would be very simple:
Binary threshold the image with a fixed threshold. The rectangle is quite dark compared to the rest of the image.
Morphological opening with a large rectangular kernel to get rid of the "noise".
To get a perfect rectangle, determine the bounding rectangle of the remaining part, and draw a white rectangle.
That'd be the whole code:
// Read image
cv::Mat img = cv::imread("OTH61.png", cv::IMREAD_GRAYSCALE);
// Binary threshold image at fixed threshold
cv::Mat img_thr;
cv::threshold(img, img_thr, 32, 255, cv::THRESH_BINARY_INV);
// Morphological opening with large rectangular kernel
cv::Mat img_mop;
cv::morphologyEx(img_thr, img_mop, cv::MORPH_OPEN, cv::Mat::ones(51, 51, CV_8UC1));
// Draw rectangle w.r.t. to the bounding rectangle of the remaining part
cv::rectangle(img_mop, cv::boundingRect(img_mop), 255, cv::FILLED);
The thresholded image:
The morphological opened image:
The cleaned image:
I have a black and white image with lines. some of these lines, however, are not perfectly connected where they should be (though they are close) I have attached an example.
I want to make it so that the lines are close to 1px thick. I have been playing with a few ideas, but not having much sucess. I have tried dilate erote, and dilate like such:
int dsize = 5;
cv::Mat element = getStructuringElement(cv::MORPH_CROSS,
cv::Size(2*dsize + 1, 2*dsize + 1),
cv::Point( dsize, dsize ) );
cv::dilate( src, src, element );
Is there a better way, as op[p[osed to just dilating and eroding to do specifically what I am after?
There is at least a couple of solutions we can try out, but I'm gonna need more info about your problem. For example, are you trying to close the (in)complete contour of a detected object? How much "contour degradation" are you willing to take to approximate a fully closed contour?
Here's a first and very basic solution, assuming you need a 1 pixel width contour. It involves dilating the image N times and then applying a thinning/skeletonize transformation. (The function is part of the Extended Image Processing module of OpenCV ).
Let's see the code:
#include <opencv2/ximgproc.hpp>
//Read input image:
std::string imagePath = "C://opencvImages//lineImg.png";
cv::Mat imageInput= cv::imread( imagePath );
//Convert it to grayscale:
cv::Mat grayImg;
cv::cvtColor( imageInput, grayImg, cv::COLOR_BGR2GRAY );
//Get binary image via Otsu:
cv::threshold( grayImg, grayImg, 0, 255 , cv::THRESH_OTSU );
//Dilate the binary image with 5 iterations:
cv::Mat morphKernel = cv::getStructuringElement( cv::MORPH_RECT, cv::Size(3, 3) );
int morphIterations = 5;
cv::morphologyEx( grayImg, grayImg, cv::MORPH_DILATE, morphKernel, cv::Point(-1,-1), morphIterations );
This is the Dilated image:
//Get the skeleton:
cv::Mat skel;
int algorithmType = 1;
cv::ximgproc::thinning( grayImg, skel, algorithmType );
This is the Skeleton Image. The line has been "thinned" back to a width of 1 pixel:
I don't know if this is good enough for your application, but, as I said, depending on what you are doing we can try a couple of alternative solutions.
Is it you who draw the lines to the mat, It seems like the problem should be take in hands before.
You should draw line in a bigger cv::mat then resize to make your line thicker.
if you want to have complete line, don't draw each points on the map but line between points to get line from bresenham.
Objective and problem
I'm trying to process a video file on the fly using OpenCV 3.4.1 by grabbing each frame, converting to grayscale, then doing Canny edge detection on it. In order to display the images (on the fly as well), I created a Mat class with 3 additional headers that is three times as wide as the original frame. The 3 extra headers represent the images I would like to display in the composite, and are positioned to the 1st, 2nd and 3rd horizontal segment of the composite.
After image processing however, the display of the composite image is not as expected: the first segment (where the original frame should be) is completely black, while the other segments (of processed images) are displayed fine. If, on the other hand, I display the ROIs one by one in separate windows, all the images look fine.
These are the things I tried to overcome this issue:
use .copyTo to actually copy the data into the appropriate image segments. The result was the same.
I put the Canny image to the compOrigPart ROI, and it did display in the first segment, so it is not a problem with the definition of the ROIs.
Define the composite as three channel image
In the loop convert it to grayscale
put processed images into it
convert back to BGR
put the original in.
This time around the whole composite was black, nothing showed.
As per gameon67's suggestion, I tried to create a namedWindow as well, but that doesn't help either.
Code:
int main() {
cv::VideoCapture vid("./Vid.avi");
if (!vid.isOpened()) return -1;
int frameWidth = vid.get(cv::CAP_PROP_FRAME_WIDTH);
int frameHeight = vid.get(cv::CAP_PROP_FRAME_HEIGHT);
int frameFormat = vid.get(cv::CAP_PROP_FORMAT);
cv::Scalar fontColor(250, 250, 250);
cv::Point textPos(20, 20);
cv::Mat frame;
cv::Mat compositeFrame(frameHeight, frameWidth*3, frameFormat);
cv::Mat compOrigPart(compositeFrame, cv::Range(0, frameHeight), cv::Range(0, frameWidth));
cv::Mat compBwPart(compositeFrame, cv::Range(0, frameHeight), cv::Range(frameWidth, frameWidth*2));
cv::Mat compEdgePart(compositeFrame, cv::Range(0, frameHeight), cv::Range(frameWidth*2, frameWidth*3));
while (vid.read(frame)) {
if (frame.empty()) break;
cv::cvtColor(frame, compBwPart, cv::COLOR_BGR2GRAY);
cv::Canny(compBwPart, compEdgePart, 100, 150);
compOrigPart = frame;
cv::putText(compOrigPart, "Original", textPos, cv::FONT_HERSHEY_PLAIN, 1, fontColor);
cv::putText(compBwPart, "GrayScale", textPos, cv::FONT_HERSHEY_PLAIN, 1, fontColor);
cv::putText(compEdgePart, "Canny edge detection", textPos, cv::FONT_HERSHEY_PLAIN, 1, fontColor);
cv::imshow("Composite of Original, BW and Canny frames", compositeFrame);
cv::imshow("Original", compOrigPart);
cv::imshow("BW", compBwPart);
cv::imshow("Canny", compEdgePart);
cv::waitKey(33);
}
}
Questions
Why can't I display the entirety of the composite image in a single window, while displaying them separately is OK?
What is the difference between these displays? The data is obviously there, as evidenced by the separate windows.
Why only the original frame is misbehaving?
Your compBwPart and compEdgePart are grayscale images so the Mat type is CV8UC1 - single channel and therefore your compositeFrame is in grayscale too. If you want to combine these two images with a color image you have to convert it to BGR first and then fill the compOrigPart.
while (vid.read(frame)) {
if (frame.empty()) break;
cv::cvtColor(frame, compBwPart, cv::COLOR_BGR2GRAY);
cv::Canny(compBwPart, compEdgePart, 100, 150);
cv::cvtColor(compositeFrame, compositeFrame, cv::COLOR_GRAY2BGR);
frame.copyTo(compositeFrame(cv::Rect(0, 0, frameWidth, frameHeight)));
cv::putText(compOrigPart, "Original", textPos, cv::FONT_HERSHEY_PLAIN, 1, fontColor); //the rest of your code
This is a combination of several issues.
The first problem is that you set the type of compositeFrame to the value returned by vid.get(cv::CAP_PROP_FORMAT). Unfortunately that property doesn't seem entirely reliable -- I've just had it return 0 (meaning CV_8UC1) after opening a color video, and then getting 3 channel (CV_8UC3) frames. Since you want to have the compositeFrame the same type as the input frame, this won't work.
To work around it, instead of using those properties, I'd lazy initialize compositeFrame and the 3 ROIs after receiving the first frame (based on it's dimensions and type).
The next set of problems lies in those two statements:
cv::cvtColor(frame, compBwPart, cv::COLOR_BGR2GRAY);
cv::Canny(compBwPart, compEdgePart, 100, 150);
In this case assumption is made that frame is BGR (since you're trying to convert), meaning compositeFrame and its ROIs are also BGR. Unfortunately, in both cases you're writing a grayscale image into the ROI. This will cause a reallocation, and the target Mat will cease to be a ROI.
To correct this, use temporary Mats for the grayscale data, and use cvtColor to turn it back to BGR to write into the ROIs.
Similar problem lies in the following statement:
compOrigPart = frame;
That's a shallow copy, meaning it will just make compOrigPart another reference to frame (and therefore it will cease to be a ROI of compositeFrame).
What you need is a deep copy, using copyTo (note that the data types still need to match, but that was fixed earlier).
Finally, even though you try to be flexible regarding the type of the input video (judging by the vid.get(cv::CAP_PROP_FORMAT)), the rest of the code really assumes that the input is 3 channel, and will break if it isn't.
At the least, there should be some assertion to cover this expectation.
Putting this all together:
#include <opencv2/opencv.hpp>
int main()
{
cv::VideoCapture vid("./Vid.avi");
if (!vid.isOpened()) return -1;
cv::Scalar fontColor(250, 250, 250);
cv::Point textPos(20, 20);
cv::Mat frame, frame_gray, edges_gray;
cv::Mat compositeFrame;
cv::Mat compOrigPart, compBwPart, compEdgePart; // ROIs
while (vid.read(frame)) {
if (frame.empty()) break;
if (compositeFrame.empty()) {
// The rest of code assumes video to be BGR (i.e. 3 channel)
CV_Assert(frame.type() == CV_8UC3);
// Lazy initialize once we have the first frame
compositeFrame = cv::Mat(frame.rows, frame.cols * 3, frame.type());
compOrigPart = compositeFrame(cv::Range::all(), cv::Range(0, frame.cols));
compBwPart = compositeFrame(cv::Range::all(), cv::Range(frame.cols, frame.cols * 2));
compEdgePart = compositeFrame(cv::Range::all(), cv::Range(frame.cols * 2, frame.cols * 3));
}
cv::cvtColor(frame, frame_gray, cv::COLOR_BGR2GRAY);
cv::Canny(frame_gray, edges_gray, 100, 150);
// Deep copy data to the ROI
frame.copyTo(compOrigPart);
// The ROI is BGR, so we need to convert back
cv::cvtColor(frame_gray, compBwPart, cv::COLOR_GRAY2BGR);
cv::cvtColor(edges_gray, compEdgePart, cv::COLOR_GRAY2BGR);
cv::putText(compOrigPart, "Original", textPos, cv::FONT_HERSHEY_PLAIN, 1, fontColor);
cv::putText(compBwPart, "GrayScale", textPos, cv::FONT_HERSHEY_PLAIN, 1, fontColor);
cv::putText(compEdgePart, "Canny edge detection", textPos, cv::FONT_HERSHEY_PLAIN, 1, fontColor);
cv::imshow("Composite of Original, BW and Canny frames", compositeFrame);
cv::imshow("Original", compOrigPart);
cv::imshow("BW", compBwPart);
cv::imshow("Canny", compEdgePart);
cv::waitKey(33);
}
}
Screenshot of the composite window (using some random test video off the web):
I just start to learn opencv, I have defined a vector like:
vector<Point2f> cornersB;
and after that i have done some calculations like:goodFeaturesToTrack,cornerSubPix and calcOpticalFlowPyrLK using cornersB.
And now I want to show cornerB to see the points that has been drawn, my code is:
pointmat = Mat(cornersB);
imshow("Window", pointmat);
But I got error said that bad number of channels (Source image must have 1, 3 or 4 channels) in cvConvertImage.
Anyone can teach me how to show the points of cornerB in an image?
I just want to see the points (points in white and the background in black).
The simpler is to use cv::drawKeypoints
drawKeypoints( InputArray image, const std::vector<KeyPoint>& keypoints, InputOutputArray outImage,const Scalar& color=Scalar::all(-1), int flags=DrawMatchesFlags::DEFAULT );
In your case, let define a black image as image:
cv::Mat image(512,512,CV_8U)
image.setTo(0);
Then convert cornersB to cv::KeyPoint kp_cornerB and define the color as white with CV_RGB(255, 255, 255)
std::vector<cv::KeyPoint> kp_cornerB ;
// TODO convert cornersB to kp_cornerB
cv::Mat pointmat;
cv::drawKeypoints(image, kp_cornerB, pointmat, CV_RGB(255, 255, 255));
imshow("Window", pointmat);
The conversion can be done with a for loop on the vector:
for(vector<Point2f>::const_iterator it = cornersB.begin();
it != cornersB.end(); it++) {
cv::KeyPoint kp(*it, 8);
kp_cornerB.push_back(kp);
}
Here, the value '8' is the 'size' of the keypoint.
Making the usual blob tracker with OpenCV and cvBlobsLib, I've come across this problem and it seems no one else had it, which makes me sad. I get the RGB/BGR frame, choose the color to isolate, treshold it into b/w, find the blobs and add the bounding rectangle on each blob, but when I display the final image, the box is stretched on the x-axis: when the object is on the left the box is close to it (although around 2.5 times larger), and as it moves to the right the box moves faster (= more and more far from the object) until it reaches the right end of the window when the object isn't even halfway. This doesn't happen on the y-axis, where everything is fine. It's not a problem with rectangles, it happens when I use fillBlob aswell, the blob shape comes out stretched and misaligned. Also, it's not a problem related to image capturing, since I've tried with kinect (OpenNI), webcam and even using a single image (imread()), and I verified that every ImageGenerator, Mat, IplImage used were 640x480, 8bit depth, for which I used AUTOSIZE for the namedWindow (enlarging to fullscreen window doesn't help either). Showing the BGR frame and the tresholded image gives no problems, they both fit into the window, but the detected blobs seem to belong to a different resolution space when I merge them with the original image. Here's the code, not much has changed from the usual examples found online everywhere:
//[...]
namedWindow("Color Image", CV_WINDOW_AUTOSIZE);
namedWindow("Color Tracking", CV_WINDOW_AUTOSIZE);
//[...] I already got the two cv::Mat I need, imgBGR and imgTresh
CBlobResult blobs;
CBlob *currentBlob;
Point pt1, pt2;
Rect rect;
//had to do Mat to IplImage conversion, since cvBlobsLib doesn't like mats
IplImage iplTresh = imgTresh;
IplImage iplBGR = imgBGR;
blobs = CBlobResult(&iplTresh, NULL, 0);
blobs.Filter(blobs, B_EXCLUDE, CBlobGetArea(), B_LESS, 100);
int nBlobs = blobs.GetNumBlobs();
for (int i = 0; i < nBlobs; i++)
{
currentBlob = blobs.GetBlob(i);
rect = currentBlob->GetBoundingBox();
pt1.x = rect.x;
pt1.y = rect.y;
pt2.x = rect.x + rect.width;
pt2.y = rect.y + rect.height;
cvRectangle(&iplBGR, pt1, pt2, cvScalar(255, 255, 255, 0), 3, 8, 0);
}
//[...]
imshow("Color Image", imgBGR);
imshow("Color Tracking", imgTresh);
The "[...]" is code that shouldn't have nothing to do with this issue, but if you need further info on how I handled the images, let me know and I'll post it.
Based on the fact that the way I capture the image doesn't change anything, that BGR frame and B/W image are well shown, and that after getting blobs any way of displaying them gives the same (wrong) result, the problem must be something between CBlobResult() and matrix2ipl conversion, but I don't really know how to find it out.
Oh god, I spent ages to write the whole problem and the next day I found the answer almost casually. As I created the B/W matrix for tresholding, I didn't make it single-channel; I copied the BGR matrix type, thus having a treshold image with 3 channels which resulted in a widthStep 3 times the frame width. Resolved creating cv::Mat imgTresh with CV_8UC1 as type.