I have a contours finder program based on opencv, now I'm trying to get the number of corners in each founded contour using Harris corners detector, my problem is that I have to get one elements of the contours
............................
std::vector<std::vector<cv::Point>> contours;
...........................
for ( int i =0;i <contours.size(); i++){
if(!contours[i].empty()){
Harris.detect(cv::Mat(contours[i])); // here crashes the program because the dimensions don't fit ????
Harris.getCorners(approx,0.4);
std::cout <<"size \n"<< approx.size()<<std::endl;
}
}
.........................
UPDATE
I checked the code again and the program crashed in this part of the Harris class :
void HarrisDetector::detect(const cv::Mat& image) {
// Harris computation
cv::cornerHarris(image,cornerStrength, // here crashs the program
neighbourhood,// neighborhood size
aperture, // aperture size
k); // Harris parameter
// internal threshold computation
double minStrength; // not used
cv::minMaxLoc(cornerStrength,&minStrength,&maxStrength);
//local maxima detection
cv::Mat dilated; // temporary image
cv::dilate(cornerStrength,dilated,cv::Mat());
cv::compare(cornerStrength,dilated,localMax,cv::CMP_EQ);
}
any ideaa
You can use method argument in cv::findContours function to some approximation and then use the contours[i].size() to get a number of corners.
Related
I want to detect the bounding rectangle of an German ID card within an image by using OpenCV.
This is what my code looks like:
capture >> frame;
cv::resize(frame, frame, cv::Size(512,256));
cv::Mat grayScaledFrame, blurredFrame, cannyFrame;
cv::cvtColor(frame, grayScaledFrame, cv::COLOR_BGR2GRAY);
cv::GaussianBlur(grayScaledFrame, blurredFrame, cv::Size(9,9), 1);
cv::Canny(blurredFrame, cannyFrame, 40, 70);
// CONTOURS
std::vector<std::vector<cv::Point>> contours;
cv::findContours(cannyFrame, contours, cv::RETR_EXTERNAL, cv::CHAIN_APPROX_SIMPLE);
// SORT
int maxArea = 0;
std::vector<cv::Point> contour;
for(int i = 0; i < contours.size(); i++) {
int thisArea = cv::contourArea(contours.at(i));
if(thisArea > maxArea) {
maxArea = thisArea;
contour = contours.at(i);
}
}
cv::Rect borderBox = cv::boundingRect(contour);
cv::rectangle(cannyFrame, borderBox, cv::Scalar{255, 32, 32}, 8);
cv::imshow("Webcam", cannyFrame);
The result looks like this:
RESULT
There are some rectangles detected but not the big one I'm interested in.
I've already tried different thresholds for Canny and also different kernel sizes for Gaussian Blur.
Best regards
First of all, as the environmental conditions change, the parameters of the code change, so it is necessary to standardize the environment (light, distance to the object, etc.).
To get this detection right, put the card at a fixed distance from the camera and calculate the area of the rectangles.
When the card is at a certain distance from the camera, you get approximate reference values of the card's area. Then, when drawing a rectangle, you use values within a specified tolerance range.
I tried to calculate contour perimeter using arcLength. The contour is read from the file into Mat which is a black and white picture of contour only.
However, when I pass this Mat into function it throws an error:
Assertion failed (curve.checkVector(2) >= 0 && (curve.depth() == CV_32F || curve.depth() == CV_32S)) in arcLength
I have figured out that the actual cause is that curve.checkVector(2) returns -1. Although I have read the documentation about this method I still do not understand how to fix this error.
Here is the test image with corner points (1,1), (1,21), (21,21), (21,1)
The contour should be (from OpenCV doc):
Input vector of 2D points, stored in std::vector or Mat.
not a b/w image.
You can compute the perimeter is different ways. The most robust is to use findContours to find external contours only (RETR_EXTERNAL), and call arcLength on that contour.
A few examples:
#include <opencv2\opencv.hpp>
#include <vector>
using namespace std;
using namespace cv;
int main()
{
Mat1b img = imread("path_to_image", IMREAD_GRAYSCALE);
// Method 1: length of unsorted points
// NOTE: doesn't work!
vector<Point> points;
findNonZero(img, points);
double len1 = arcLength(points, true);
// 848.78
// Method 2: length of the external contour
vector<vector<Point>> contours;
findContours(img.clone(), contours, RETR_EXTERNAL, CHAIN_APPROX_NONE); // Retrieve only external contour
double len2 = arcLength(contours[0], true);
// 80
// Method 3: length of convex hull of contour
// NOTE: convex hull based methods work reliably only for convex shapes.
vector<Point> hull1;
convexHull(contours[0], hull1);
double len3 = arcLength(hull1, true);
// 80
// Method 4: length of convex hull of unsorted points
// NOTE: convex hull based methods work reliably only for convex shapes.
vector<Point> hull2;
convexHull(points, hull2);
double len4 = arcLength(hull2, true);
// 80
// Method 5: number of points in the contour
// NOTE: this will simply count the number of points in the contour.
// It works only if:
// 1) findContours was used with option CHAIN_APPROX_NONE.
// 2) the contours is thin (has thickness of 1 pixel).
double len5 = contours[0].size();
// 80
return 0;
}
What I'm trying to do is measure the thickness of the eyeglasses frames. I had the idea to measure the thickness of the frame's contours (may be a better way?). I have so far outlined the frame of the glasses, but there are gaps where the lines don't meet. I thought about using HoughLinesP, but I'm not sure if this is what I need.
So far I have conducted the following steps:
Convert image to grayscale
Create ROI around the eye/glasses area
Blur the image
Dilate the image (have done this to remove any thin framed glasses)
Conduct Canny edge detection
Found contours
These are the results:
This is my code so far:
//convert to grayscale
cv::Mat grayscaleImg;
cv::cvtColor( img, grayscaleImg, CV_BGR2GRAY );
//create ROI
cv::Mat eyeAreaROI(grayscaleImg, centreEyesRect);
cv::imshow("roi", eyeAreaROI);
//blur
cv::Mat blurredROI;
cv::blur(eyeAreaROI, blurredROI, Size(3,3));
cv::imshow("blurred", blurredROI);
//dilate thin lines
cv::Mat dilated_dst;
int dilate_elem = 0;
int dilate_size = 1;
int dilate_type = MORPH_RECT;
cv::Mat element = getStructuringElement(dilate_type,
cv::Size(2*dilate_size + 1, 2*dilate_size+1),
cv::Point(dilate_size, dilate_size));
cv::dilate(blurredROI, dilated_dst, element);
cv::imshow("dilate", dilated_dst);
//edge detection
int lowThreshold = 100;
int ratio = 3;
int kernel_size = 3;
cv::Canny(dilated_dst, dilated_dst, lowThreshold, lowThreshold*ratio, kernel_size);
//create matrix of the same type and size as ROI
Mat dst;
dst.create(eyeAreaROI.size(), dilated_dst.type());
dst = Scalar::all(0);
dilated_dst.copyTo(dst, dilated_dst);
cv::imshow("edges", dst);
//join the lines and fill in
vector<Vec4i> hierarchy;
vector<vector<Point>> contours;
cv::findContours(dilated_dst, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE);
cv::imshow("contours", dilated_dst);
I'm not entirely sure what the next steps would be, or as I said above, if I should use HoughLinesP and how to implement it. Any help is very much appreciated!
I think there are 2 main problems.
segment the glasses frame
find the thickness of the segmented frame
I'll now post a way to segment the glasses of your sample image. Maybe this method will work for different images too, but you'll probably have to adjust parameters, or you might be able to use the main ideas.
Main idea is:
First, find the biggest contour in the image, which should be the glasses. Second, find the two biggest contours within the previous found biggest contour, which should be the glasses within the frame!
I use this image as input (which should be your blurred but not dilated image):
// this functions finds the biggest X contours. Probably there are faster ways, but it should work...
std::vector<std::vector<cv::Point>> findBiggestContours(std::vector<std::vector<cv::Point>> contours, int amount)
{
std::vector<std::vector<cv::Point>> sortedContours;
if(amount <= 0) amount = contours.size();
if(amount > contours.size()) amount = contours.size();
for(int chosen = 0; chosen < amount; )
{
double biggestContourArea = 0;
int biggestContourID = -1;
for(unsigned int i=0; i<contours.size() && contours.size(); ++i)
{
double tmpArea = cv::contourArea(contours[i]);
if(tmpArea > biggestContourArea)
{
biggestContourArea = tmpArea;
biggestContourID = i;
}
}
if(biggestContourID >= 0)
{
//std::cout << "found area: " << biggestContourArea << std::endl;
// found biggest contour
// add contour to sorted contours vector:
sortedContours.push_back(contours[biggestContourID]);
chosen++;
// remove biggest contour from original vector:
contours[biggestContourID] = contours.back();
contours.pop_back();
}
else
{
// should never happen except for broken contours with size 0?!?
return sortedContours;
}
}
return sortedContours;
}
int main()
{
cv::Mat input = cv::imread("../Data/glass2.png", CV_LOAD_IMAGE_GRAYSCALE);
cv::Mat inputColors = cv::imread("../Data/glass2.png"); // used for displaying later
cv::imshow("input", input);
//edge detection
int lowThreshold = 100;
int ratio = 3;
int kernel_size = 3;
cv::Mat canny;
cv::Canny(input, canny, lowThreshold, lowThreshold*ratio, kernel_size);
cv::imshow("canny", canny);
// close gaps with "close operator"
cv::Mat mask = canny.clone();
cv::dilate(mask,mask,cv::Mat());
cv::dilate(mask,mask,cv::Mat());
cv::dilate(mask,mask,cv::Mat());
cv::erode(mask,mask,cv::Mat());
cv::erode(mask,mask,cv::Mat());
cv::erode(mask,mask,cv::Mat());
cv::imshow("closed mask",mask);
// extract outermost contour
std::vector<cv::Vec4i> hierarchy;
std::vector<std::vector<cv::Point>> contours;
//cv::findContours(mask, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE);
cv::findContours(mask, contours, hierarchy, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
// find biggest contour which should be the outer contour of the frame
std::vector<std::vector<cv::Point>> biggestContour;
biggestContour = findBiggestContours(contours,1); // find the one biggest contour
if(biggestContour.size() < 1)
{
std::cout << "Error: no outer frame of glasses found" << std::endl;
return 1;
}
// draw contour on an empty image
cv::Mat outerFrame = cv::Mat::zeros(mask.rows, mask.cols, CV_8UC1);
cv::drawContours(outerFrame,biggestContour,0,cv::Scalar(255),-1);
cv::imshow("outer frame border", outerFrame);
// now find the glasses which should be the outer contours within the frame. therefore erode the outer border ;)
cv::Mat glassesMask = outerFrame.clone();
cv::erode(glassesMask,glassesMask, cv::Mat());
cv::imshow("eroded outer",glassesMask);
// after erosion if we dilate, it's an Open-Operator which can be used to clean the image.
cv::Mat cleanedOuter;
cv::dilate(glassesMask,cleanedOuter, cv::Mat());
cv::imshow("cleaned outer",cleanedOuter);
// use the outer frame mask as a mask for copying canny edges. The result should be the inner edges inside the frame only
cv::Mat glassesInner;
canny.copyTo(glassesInner, glassesMask);
// there is small gap in the contour which unfortunately cant be closed with a closing operator...
cv::dilate(glassesInner, glassesInner, cv::Mat());
//cv::erode(glassesInner, glassesInner, cv::Mat());
// this part was cheated... in fact we would like to erode directly after dilation to not modify the thickness but just close small gaps.
cv::imshow("innerCanny", glassesInner);
// extract contours from within the frame
std::vector<cv::Vec4i> hierarchyInner;
std::vector<std::vector<cv::Point>> contoursInner;
//cv::findContours(glassesInner, contoursInner, hierarchyInner, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE);
cv::findContours(glassesInner, contoursInner, hierarchyInner, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
// find the two biggest contours which should be the glasses within the frame
std::vector<std::vector<cv::Point>> biggestInnerContours;
biggestInnerContours = findBiggestContours(contoursInner,2); // find the one biggest contour
if(biggestInnerContours.size() < 1)
{
std::cout << "Error: no inner frames of glasses found" << std::endl;
return 1;
}
// draw the 2 biggest contours which should be the inner glasses
cv::Mat innerGlasses = cv::Mat::zeros(mask.rows, mask.cols, CV_8UC1);
for(unsigned int i=0; i<biggestInnerContours.size(); ++i)
cv::drawContours(innerGlasses,biggestInnerContours,i,cv::Scalar(255),-1);
cv::imshow("inner frame border", innerGlasses);
// since we dilated earlier and didnt erode quite afterwards, we have to erode here... this is a bit of cheating :-(
cv::erode(innerGlasses,innerGlasses,cv::Mat() );
// remove the inner glasses from the frame mask
cv::Mat fullGlassesMask = cleanedOuter - innerGlasses;
cv::imshow("complete glasses mask", fullGlassesMask);
// color code the result to get an impression of segmentation quality
cv::Mat outputColors1 = inputColors.clone();
cv::Mat outputColors2 = inputColors.clone();
for(int y=0; y<fullGlassesMask.rows; ++y)
for(int x=0; x<fullGlassesMask.cols; ++x)
{
if(!fullGlassesMask.at<unsigned char>(y,x))
outputColors1.at<cv::Vec3b>(y,x)[1] = 255;
else
outputColors2.at<cv::Vec3b>(y,x)[1] = 255;
}
cv::imshow("output", outputColors1);
/*
cv::imwrite("../Data/Output/face_colored.png", outputColors1);
cv::imwrite("../Data/Output/glasses_colored.png", outputColors2);
cv::imwrite("../Data/Output/glasses_fullMask.png", fullGlassesMask);
*/
cv::waitKey(-1);
return 0;
}
I get this result for segmentation:
the overlay in original image will give you an impression of quality:
and inverse:
There are some tricky parts in the code and it's not tidied up yet. I hope it's understandable.
The next step would be to compute the thickness of the the segmented frame. My suggestion is to compute the distance transform of the inversed mask. From this you will want to compute a ridge detection or skeletonize the mask to find the ridge. After that use the median value of ridge distances.
Anyways I hope this posting can help you a little, although it's not a solution yet.
Depending on lighting, frame color etc this may or may not work but how about simple color detection to separate the frame ? Frame color will usually be a lot darker than human skin. You'll end up with a binary image (just black and white) and by calculating the number (area) of black pixels you get the area of the frame.
Another possible way is to get better edge detection, by adjusting/dilating/eroding/both until you get better contours. You will also need to differentiate the contour from the lenses and then apply cvContourArea.
The program I'm working right now is almost done but I'm not very satisfy with the result. By using Canny algorithm, I managed to get a very clear of the object's contour but the program has some problem to recognize the contour and draw the contour with a red line. The program:
void setwindowSettings(){
namedWindow("Contours", CV_WINDOW_AUTOSIZE);
createTrackbar("LowerC", "Contours", &lowerC, 255, NULL);
createTrackbar("UpperC", "Contours", &upperC, 255, NULL);
}
void wait(void)
{
long t=30000000;
while(t--);
}
int main(void)
{
VideoCapture cap(0); // open the default camera
if(!cap.isOpened()) // check if we succeeded
return -1;
Mat frame,foreground,image;
double pt1, pt2, area;
Rect rect;
int i;
vector<vector<Point> > contours;
vector<vector<Point> > largest_contours;
namedWindow("Capture", CV_WINDOW_AUTOSIZE);
setwindowSettings();
while(1){
cap >> frame; // get a new frame from camera
if( frame.empty() )
break;
image=frame.clone();
cvtColor(image,foreground,CV_BGR2GRAY);
GaussianBlur(foreground,foreground,Size(9,11),0,0);
Canny(foreground,foreground,lowerC,upperC,3);
findContours(foreground,contours,CV_RETR_EXTERNAL,CV_CHAIN_APPROX_SIMPLE);
if(contours.empty())
continue;
double largest_area = 0;
for( i= 0; i < contours.size(); i++){ // get the largest contour
area = fabs(contourArea(contours[i]));
if(area >= largest_area){
largest_area = area;
largest_contours.clear();
largest_contours.push_back(contours[i]);
}
}
if(largest_area>=3000){ // draw the largest contour if exceeded minimum largest area
drawContours(image,largest_contours,-1,Scalar(0,0,255),2);
printf("area = %.f\n",largest_area);
}
wait();
imshow( "Capture",image );
imshow("Contours",foreground);
if(waitKey(30) >= 0) break;
}
// the camera will be deinitialized automatically in VideoCapture destructor
return 0;
}
Program summary:
Get images from camera
Noise filtration (Convert to gray → blur → Canny)
Find contours
Find the largest contour and its area in the image aka the object
Draw a red line around the object and print out the largest area
Rinse and repeat
And the results:
Rarely I got what I want; Contour detected, red line drawn (GOOD ONE):
...and usually I got this; No contour detected, not red line (BAD ONE):
The chances to get the GOOD ONE are about 1/20 which is not very good. Also, the line of the object's contour in Contours screen will blink when the red line appears around the object (see the GOOD ONE picture).
I'm using one of my object (A small black square box) for this question but please note that the main objective of this object detection program is to detect the object regardless of its shape or its color.
So my questions are:
Why I still get the BAD ONES despite the contour is as clear as day?
Can anyone share a better idea on how to improve the contour detection? (i.e better blur algorithm)
How to avoid the contour's line from blinking when the red line is drawn around the object?
EDIT: I just discovered that contour's line blinking is not because of the red line drawn around it (either with drawContoursor line function) but it happens after the largest contour is detected by findContours function and calculated as the largest contour.
For question about no. 3 click HERE. VIDEO HERE, CLICK IT!!!
Thanks in advance.
PS: I'm using OpenCV 2.4.3 on Ms Visual C++ 2010 Exp.
Since you are using the fact of largest contour so I presume you are trying to detect the largest object appearing in the field of view of the camera.I wonder why the window light/bright light source at top right doesn't create any contour(may be due to blurring). You can store the background image and subtract it from the image where the object appears. This way you can derive the object.You can apply a contour finding in the difference image.absdiff(frame_now,frame_backgrnd,diff) where diff is the difference image.
If the object is in motion and you want to detect you can use optical flow combined with largest contour to detect the object.
Try doing you process without the blurring function and then detect the largest contourArea.
For plotting the points try this
for(int i = 1;i<(int)largest_contours[0].size();i++)
line(image,largest_contours[0][i-1],largest_contours[0][i],cv::Scalar(0,0,255),2,8,0);
I'm trying to locate some regions of a frame, the frame is in Ycbcr color space. and I have to select those regions based on their Y values.
so I wrote this code:
Mat frame. ychannel;
VideoCapture cap(1);
int key =0;
int maxV , minV;
Point max, min;
while(key != 27){
cap >> frame;
cvtColor(frame,yframe,CV_RGB_YCrCb); // converting to YCbCr color space
extractChannel(yframe, yframe, 0); // extracting the Y channel
cv::minMaxLoc(yframe,&minV,&maxV,&min,&max);
cv::threshold(outf,outf,(maxV-10),(maxV),CV_THRESH_TOZERO);
/**
Now I want to use :
cv::rectangle()
but I want to draw a rect around any pixel (see the picture bellow)that's higher than (maxV-10)
and that during the streaming
**/
key = waitKey(1);
}
I draw this picture hopping that it helps to understand what I what to do .
thanks for your help.
Once you have applied your threshold you will end up with a binary image containing a number of connected components, if you want to draw a rectangle around each component then you first need to detect those components.
The OpenCV function findContours does just that, pass it your binary image, and it will provide you with a vector of vectors of points which trace the boundary of each component in your image.
cv::Mat binaryImage;
std::vector<std::vector<cv::Point>> contours;
cv::findContours(binaryImage, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE)
Then all you need to do is find the bounding rectangle of each of those sets of points and draw them to your output image.
for (int i=0; i<contours.size(); ++i)
{
cv::Rect r = cv::boundingRect(contours.at(i));
cv::rectangle(outputImage, r, CV_RGB(255,0,0));
}
You have to find the each of the connected components, and draw their bounding box.