I am able to find the contours and labelled all of them. Now I want to remove some contours from the image and need some specific contours only using opencv.
I have used following code to get the contours. This code is working fine for me to get the contours and its labels: as you see the binary image and its contours in the picture. Here, I want to remove contours which are above contour 45 and below contour 22. Basically, I need the center part between the two long horizontal lines.
int main(int argc, char *argv[])
{
if (argc != 2) {
cerr << "usage: "<<argv[0]<< "<input_file with path>"<<endl;
return -1;
}
cv::Mat im_bw = cv::imread(argv[1],cv::IMREAD_GRAYSCALE); //loading an image//
// Binarize the output image
cv::Mat binarized_image;
cv::threshold(im_bw, binarized_image, 128, 255, cv::THRESH_BINARY);
cv::imshow("binary_image.png", binarized_image);
cv::waitKey(0);
vector<vector<cv::Point>> contours;
vector<cv::Vec4i> hierarchy;
cv::findContours(binarized_image,contours,hierarchy,cv::RETR_TREE,cv::CHAIN_APPROX_SIMPLE);
cv::Mat mask = cv::Mat::zeros(im_bw.size(),CV_8UC3);
cv::drawContours(mask,contours,-1,cv::Scalar(0,255,255),1);
for( int i=0; i<contours.size(); i++)
{
cv::putText(mask,to_string(i),contours[i][0],1,1,cv::Scalar(255,0,0),1);
}
cout<<"Contours : "<<contours.size()<<endl;
for(cv::Vec4i k:hierarchy)
{
cout<<k<<endl;
}
cv::imshow("Contours_binary_image.png", mask);
cv::waitKey(0);
return 0;
}
There are probably a few ways you could do it depending on what you care about. Do you want it to be fast, or accurate?
Here is a way you could do it geometrically that should be fairly accurate, and with some optimizations, could be pretty fast.
You'll need a way to identify contours 45 and 22. In the question you identified them as the two large horizontal lines. That is a good place to start.
A simple way to do it would be to iterate through all the contours and keep track of the min and max point values. The horizontal lines will be the largest distance in the min/max X direction and have a relatively small distance between min and max Y. It will probably requires some tweaking and defining some more rules for what is allowed to be considered a "horizontal line".
Once you have the horizontal lines identified, the next step is removing all the ones above and below them. Doing this for the top and bottom will be the same, just in opposite directions. I'll walk through how you could do it with the top line.
Each contour is made up of smaller individual line segments. For each linesegment in the horizontal line's contour, check that every other contour (each segment in said contour) is either above it, or intersects with it. If either is true, then you can remove it. For a large number of contours, or very complex contours, this will be quite slow. There are a number of optimizations you could make. You could simplify the contour shapes. Or compute bounding boxes around them and check the bounding box is above the horizontal line, if it intersects, you can look at it closer and see if the line itself intersects.
Here are the steps I can suggest you to handle this issue:
First you need to get the mass center of each contour by using
opencv moments. This will give you the weighted centeral point
of contours as x,y coordinates.
Then make a filter according to the mass center's y-axis. Between 45th and 22th contour's y-axis values will be the valid contours.
Only draw the contours which are valid according to your filter.
This may help to find the mass centers.
Related
I am currently working on a project based on detecting contours in a picture. I'm analyzing images with lines only, such as this rectangle, among other pictures (also colorful). The problem is that using my code I get an image of "two rectangles" as a result, instead of one. I understand the reason behind it: Canny detects edges on both sides of the line, because of the gradient. But, is there any way to remove one of the rectangles (or lines in other pictures) or merge them into one?
Fragment of my code:
cvtColor(source, imgGray, CV_RGB2GRAY);
GaussianBlur(imgGray, imgGauss, Size(5, 5), 1);
Canny(imgGauss, imgCanny, 100, 200);
findContours(imgCanny, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE);
Mat imgDraw = Mat::zeros(imgCanny.size(), CV_8UC3);
for (int i = 0; i< contours.size(); i++)
{
Scalar color(255, 255, 255);
drawContours(imgDraw, contours, i, color, 1.5, 8, hierarchy, 0);
cout << "Hierarchy: " << hierarchy[i] << endl;
for (int j = 0; j < contours[i].size(); j++)
{
cout << contours[i][j] << endl;
}
}
I've tried to use CV_RETR_CCOMP mode in findContours as in the example code for drawContours function but I get the same result.
I could probably try geometrical figures comparison, but as far as I know, with more complicated shapes (e.g. human face) that wouldn't work.
Another question, why is findContours detecting four contours instead of two (for those two rectangles)?
I'm sorry if the question appeared somewhere else, I'll be happy if you send me a link to it.
Any edge detector detects the transition from white to black and also the transition from black to white. That is why in your line it detects two edges (the left side of the line and the right side).
If you test with a filled black square, it will detect only one edge. The same happens with a filled white square in a black background.
For line detection, Hough transform may be better.
For the second question: Find contours detects a white square, that inside has a black square, that inside has another white square, that inside has another black square. It doesn't see the lines, it only sees filled objects one inside the other. In the contour object you can get the colour of the squares and also it tells you if a contour is inside another. Maybe you can ignore the contours that are inside another.
I am trying to implement Automatic perspective correction in my iOS program and when I use the test image I found on the tutorial everything works as expected. But when I take a picture I get back a weird result.
I am using code found in this tutorial
When I give it an image that looks like this:
I get this as the result:
Here is what dst gives me that might help.
I am using this to call the method which contains the code.
quadSegmentation(Img, bw, dst, quad);
Can anyone tell me when I am getting so many green lines compared to the tutorial? And how I might be able to fix this and properly crop the image to only contain the card?
For perspective transform you need,
source points->Coordinates of quadrangle vertices in the source image.
destination points-> Coordinates of the corresponding quadrangle vertices in the destination image.
Here we will calculate these point by contour process.
Calculate Coordinates of quadrangle vertices in the source image
You will get the your card as contour by just by blurring, thresholding, then find contour, find largest contour etc..
After finding largest contour just calculate approximates a polygonal curve, here you should get 4 Point which represent corners of your card. You can adjust the parameter epsilon to make 4 co-ordinates.
Calculate Coordinates of the corresponding quadrangle vertices in the destination image
This can be easily find out by calculating bounding rectangle for largest contour.
In below image the red rectangle represent source points and green for destination points.
Adjust the co-ordinates order and Apply Perspective transform
Here I manually adjust the co-ordinates order and you can use some sorting algorithm.
Then calculate transformation matrix and apply wrapPrespective
See the final result
Code
Mat src=imread("card.jpg");
Mat thr;
cvtColor(src,thr,CV_BGR2GRAY);
threshold( thr, thr, 70, 255,CV_THRESH_BINARY );
vector< vector <Point> > contours; // Vector for storing contour
vector< Vec4i > hierarchy;
int largest_contour_index=0;
int largest_area=0;
Mat dst(src.rows,src.cols,CV_8UC1,Scalar::all(0)); //create destination image
findContours( thr.clone(), contours, hierarchy,CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE ); // Find the contours in the image
for( int i = 0; i< contours.size(); i++ ){
double a=contourArea( contours[i],false); // Find the area of contour
if(a>largest_area){
largest_area=a;
largest_contour_index=i; //Store the index of largest contour
}
}
drawContours( dst,contours, largest_contour_index, Scalar(255,255,255),CV_FILLED, 8, hierarchy );
vector<vector<Point> > contours_poly(1);
approxPolyDP( Mat(contours[largest_contour_index]), contours_poly[0],5, true );
Rect boundRect=boundingRect(contours[largest_contour_index]);
if(contours_poly[0].size()==4){
std::vector<Point2f> quad_pts;
std::vector<Point2f> squre_pts;
quad_pts.push_back(Point2f(contours_poly[0][0].x,contours_poly[0][0].y));
quad_pts.push_back(Point2f(contours_poly[0][1].x,contours_poly[0][1].y));
quad_pts.push_back(Point2f(contours_poly[0][3].x,contours_poly[0][3].y));
quad_pts.push_back(Point2f(contours_poly[0][2].x,contours_poly[0][2].y));
squre_pts.push_back(Point2f(boundRect.x,boundRect.y));
squre_pts.push_back(Point2f(boundRect.x,boundRect.y+boundRect.height));
squre_pts.push_back(Point2f(boundRect.x+boundRect.width,boundRect.y));
squre_pts.push_back(Point2f(boundRect.x+boundRect.width,boundRect.y+boundRect.height));
Mat transmtx = getPerspectiveTransform(quad_pts,squre_pts);
Mat transformed = Mat::zeros(src.rows, src.cols, CV_8UC3);
warpPerspective(src, transformed, transmtx, src.size());
Point P1=contours_poly[0][0];
Point P2=contours_poly[0][1];
Point P3=contours_poly[0][2];
Point P4=contours_poly[0][3];
line(src,P1,P2, Scalar(0,0,255),1,CV_AA,0);
line(src,P2,P3, Scalar(0,0,255),1,CV_AA,0);
line(src,P3,P4, Scalar(0,0,255),1,CV_AA,0);
line(src,P4,P1, Scalar(0,0,255),1,CV_AA,0);
rectangle(src,boundRect,Scalar(0,255,0),1,8,0);
rectangle(transformed,boundRect,Scalar(0,255,0),1,8,0);
imshow("quadrilateral", transformed);
imshow("thr",thr);
imshow("dst",dst);
imshow("src",src);
imwrite("result1.jpg",dst);
imwrite("result2.jpg",src);
imwrite("result3.jpg",transformed);
waitKey();
}
else
cout<<"Make sure that your are getting 4 corner using approxPolyDP..."<<endl;
teethe This typically happens when you rely on somebody else code to solve your particular problem instead of adopting the code. Look at the processing stages and also the difference between their and your image (it is a good idea by the way to start with their image and make sure the code works):
Get the edge map. - will probably work since your edges are fine
Detect lines with Hough transform. - fail since you have lines not only on the contour but also inside of your card. So expect a lot of false alarm lines
Get the corners by finding intersections between lines. - fail for the above mentioned reason
Check if the approximate polygonal curve has 4 vertices. - fail
Determine top-left, bottom-left, top-right, and bottom-right corner. - fail
Apply the perspective transformation. - fail completely
To fix your problem you have to ensure that only lines on the periphery are extracted. If you always have a dark background you can use this fact to discard the lines with other contrasts/polarities. Alternatively you can extract all the lines and then select the ones that are closest to the image boundary (if your background doesn't have lines).
I'm using a canny edge detection and a finding contours function (both OpenCV) to create markers for the watershed transform. Everything works fine but I'm not 100% satisfied with the results. The reason is that some edges are missing and therefore important information is lost. In more detail, I got a bunch of windows (front views), which are rectangles, after the watershed transform I end up with something like this:
but I would rather have nice rectangles, that are complete and not open to one side. While maintaining irregular shapes (bushes in front of the house, cars..) Any ideas how I could solve this problem?I thought about overlaying the whole image with a grid, but I can't make it work.
Thank you very much.
Here is my code:
Mat gray;
cvtColor(im, gray, CV_BGR2GRAY);
// Use Canny instead of threshold to catch squares with gradient shading
Mat bw;
Canny(gray, bw, 0, 100, 5, true);
// Find contours
vector<vector<Point> > contours;
vector<Vec4i> hierarchy;
findContours( bw, contours, hierarchy,
CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE );
// watershed
Mat markers(bw.size(), CV_32S);
markers = Scalar::all(0);
int idx = 0;
int compCount = 0;
for( ; idx >= 0; idx = hierarchy[idx][0], compCount++ ) {
if (fabs(contourArea(contours[compCount])) < min_size )
continue;
drawContours(markers, contours, idx, Scalar::all(compCount+1), 1, 8, hierarchy, INT_MAX);
}
watershed( im, markers );
As requested, here is the original image, the image I would like to get and my output:
And I would like to have a segmentation like this (although over segmentation does not hurt, I just need to make sure, I get all the details):
While I get something like this:
(please ignore the colours, they are not important for this question and are just a result of my overall program). This is only one example, if you want, I can show you more, also please have a look at the etrims dataset, all my pictures are from there.
Two things -
1) As already mentioned, edge detection results in spurious edges being picked up.
2) Using these edges as markers for watershed segmentation results in over-segmentation because every marker produces a segmented region in the output.
Strategy -
(i) Preprocessing: Smooth the image heavily (morphological opening by reconstruction can be used for homogenizing the intensities without significantly affecting edges you are interested in).
(ii) Markers: Instead of using edges as seeds, I'd use the local extrema. Ideally, we want one marker for every region we want segmented.
(iii) Segmentation: Find the gradient magnitude (range filtering is also a good option) of the image from step (i) and use that as the segmentation function.
Using this strategy, I get the following segmentation.
Alternatively, after step (i), you can use Canny edge detection and do some morphological cleanup (to fill contours and remove edges that remain). This is what I get.
These are not exactly the expected segmentation (some objects like the car are not detected), but are a good start.
Edit: The MATLAB code used to generate the images -
% convert to grayscale
img = rgb2gray(origImg);
% create an appropriate structuring element
w_size = 20;
seSquare = strel('square', w_size);
% opening by reconstruction - to smooth dark regions
imgEroded = imerode(img, seSquare);
imgRecon = imreconstruct(imgEroded, img);
% invert and repeat - to smooth bright regions
imgReconComp = imcomplement(imgRecon);
imgEroded2 = imerode(imgReconComp, seSquare);
imgRecon2 = imreconstruct(imgEroded2, imgReconComp);
% get foreground markers
fgm = imregionalmax(imgRecon2);
% get background markers - this step can be skipped
% in which case only fgm would be the marker image
% and the segmentation would be different
distTrans = bwdist(fgm);
wLines= watershed(distTrans);
bgm = wLines == 0;
% get the segmentation function and impose markers
% perform watershed segmentation
seSquare3 = strel('square', 3);
rangeImg = rangefilt(imgRecon2, getnhood(seSquare3));
segFunc = imimposemin(rangeImg, fgm | bgm);
grayLabel = watershed(segFunc);
rgbLabel= label2rgb(grayLabel);
figure, imshow(rgbLabel); title('Output using Watershed')
% alternatively, extract edges from the preprocessed image
% perform morph cleanup
bwEdges = edge(imgRecon2, 'canny');
bwFilled = imfill(bwEdges, 'holes');
bwRegions = imopen(bwFilled, seSquare3);
grayLabel = bwlabel(bwRegions);
rgbLabel = label2rgb(grayLabel, 'jet', 'k');
figure, imshow(rgbLabel); title('Output using Canny')
from the looks of the desired output and the program's output, it seems that the edge detector is finding spurious edges. Canny edge detector contains a low-pass filter, but it might help for you to do a separate Gaussian low-pass filtering step before you actually run the Canny edge detector.
Other than that, it is difficult to achieve the desired result. For e.g., look at the top-most windows in the picture. They have distinct colors --- the frame, the shadow of the frame, and the window. The boundaries of these colors will be detected as edges by the Edge detector.
I'm trying to make a program to detect an object in any shape using a video camera/webcam based on Canny filter and contour finding function. Here is my program:
int main( int argc, char** argv )
{
CvCapture *cam;
CvMoments moments;
CvMemStorage* storage = cvCreateMemStorage(0);
CvSeq* contours = NULL;
CvSeq* contours2 = NULL;
CvPoint2D32f center;
int i;
cam=cvCaptureFromCAM(0);
if(cam==NULL){
fprintf(stderr,"Cannot find any camera. \n");
return -1;
}
while(1){
IplImage *img=cvQueryFrame(cam);
if(img==NULL){return -1;}
IplImage *src_gray= cvCreateImage( cvSize(img->width,img->height), 8, 1);
cvCvtColor( img, src_gray, CV_BGR2GRAY );
cvSmooth( src_gray, src_gray, CV_GAUSSIAN, 5, 11);
cvCanny(src_gray, src_gray, 70, 200, 3);
cvFindContours( src_gray, storage, &contours, sizeof(CvContour), CV_RETR_EXTERNAL, CV_CHAIN_APPROX_NONE, cvPoint(0,0));
if(contours==NULL){ contours=contours2;}
contours2=contours;
cvMoments(contours, &moments, 1);
double m_00 = cvGetSpatialMoment( &moments, 0, 0 );
double m_10 = cvGetSpatialMoment( &moments, 1, 0 );
double m_01 = cvGetSpatialMoment( &moments, 0, 1 );
float gravityX = (m_10 / m_00)-150;
float gravityY = (m_01 / m_00)-150;
if(gravityY>=0&&gravityX>=0){
printf("center point=(%.f, %.f) \n",gravityX,gravityY); }
for (; contours != 0; contours = contours->h_next){
CvScalar color = CV_RGB(250,0,0);
cvDrawContours(img,contours,color,color,-1,-1, 8, cvPoint(0,0));
}
cvShowImage( "Input", img );
cvShowImage( "Contours", src_gray );
cvClearMemStorage(storage);
if(cvWaitKey(33)>=0) break;
}
cvDestroyWindow("Contours");
cvDestroyWindow("Source");
cvReleaseCapture(&cam);
}
This program will detect all contours captured by the camera and the average coordinate of the contours will be printed. My question is how to filter out only one object/contour so I can get more precise (x,y) position of the object? If possible, can anyone show me how to mark the center of the object by using (x,y) coordinates?
Thanks in advance. Cheers
p/s:Sorry I couldn't upload a screenshot yet but if anything helps, here's the link.
Edit: To make my question more clear:
For example, if I only want to filter out only the square from my screenshot above, what should I do?
The object I want to filter out has the biggest contour area and most importantly has a shape(any shape), not a straight or a curve line
I'm still experimenting with the smooth and canny values so if anybody have the problem to detect the contours using my program please alter the values.
I think it can be solved fairly easy. I would suggest some morphological operations before contour detection. Also, I would suggest filtering "out" smaller elements, and getting the biggest element as the only one still in the image.
I suggest:
for filtering out lines (straight or curved): you have to decide what do you yourself consider a border between a "line" and a "shape". Let's say you consider all the objects of a thickness 5 pixel or more to be objects, while the ones that are less than 5 pixels across to be lines. An morphological opening that uses a 5x5 square or a 3-pixel sized diamond shape as a structuring element would take care of this.
for filtering out small objects in general: if objects are of arbitrary shapes, purely morphological opening won't do: you have to do an algebraic opening. A special type of algebraic openings is an area opening: an operation that removes all the connected components in the image that have (pixel) area smaller than a given threshold. If you have an upper bound on the size of uninteresting objects, or a lower bound on the size of interesting ones, that value should be used as a threshold. You can probably get a similar effect with a larger morphological opening, but it will not be so flexible.
for filtering out all the objects except the largest: it sounds like removing connected components from the smallest one to the largest one should work. Try labeling the connected components. On a binary (black & white image), this image transformation works by creating a greyscale image, labeling the background as 0 (black), and each component with a different, increasing grey value. In the end, pixels of each object are marked by a different value. You can now simply look at the gray level histogram, and find the grey value with the most pixels. Set all the other grey levels to 0 (black), and the only object left in the image is the biggest one.
The suggestions are written from the simplest to the most complex ones. Still, I think OpenCV can be of help with any of these. Morphological erosion, dilation, opening and closing are implemented in OpenCV. I think you might need to construct an algebraic opening operator on your own (or play with combining OpenCV basic morphology), but I'm sure OpenCV can help you with both labeling the connected components and examining the histogram of the resulting greyscale image.
In the end, when only pixels from one object are left, you do the Canny contour detection.
This is a blob processing problem that can not be solved (easily) by OpenCV itself. Have a look at cvBlobsLib. This library is extends OpenCV with functions/classes for connected component labeling.
http://opencv.willowgarage.com/wiki/cvBlobsLib
i want the hand image to be a black and white shape of the hand. here's a sample of the input and the desired output:
using a threshold doesn't give the desired output because some of the colors inside the hand are the same with the background color. how can i get the desired output?
Adaptive threshold, find contours, floodfill?
Basically, adaptive threshold turns your image into black and white, but takes the threshold level based on local conditions around each pixel - that way, you should avoid the problem you're experiencing with an ordinary threshold. In fact, I'm not sure why anyone would ever want to use a normal threshold.
If that doesn't work, an alternative approach is to find the largest contour in the image, draw it onto a separate matrix and then floodfill everything inside it with black. (Floodfill is like the bucket tool in MSPaint - it starts at a particular pixel, and fills in everything connected to that pixel which is the same colour with another colour of your choice.)
Possibly the most robust approach against various lighting conditions is to do them all in the sequence at the top. But you may be able to get away with only the threshold or the countours/floodfill.
By the way, perhaps the trickiest part is actually finding the contours, because findContours returns an arraylist/vector/whatever (depends on the platform I think) of MatOfPoints. MatOfPoint is a subclass of Mat but you can't draw it directly - you need to use drawContours. Here's some code for OpenCV4Android that I know works:
private Mat drawLargestContour(Mat input) {
/** Allocates and returns a black matrix with the
* largest contour of the input matrix drawn in white. */
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Imgproc.findContours(input, contours, new Mat() /* hierarchy */,
Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);
double maxArea = 0;
int index = -1;
for (MatOfPoint contour : contours) { // iterate over every contour in the list
double area = Imgproc.contourArea(contour);
if (area > maxArea) {
maxArea = area;
index = contours.indexOf(contour);
}
}
if (index == -1) {
Log.e(TAG, "Fatal error: no contours in the image!");
}
Mat border = new Mat(input.rows(), input.cols(), CvType.CV_8UC1); // initialized to 0 (black) by default because it's Java :)
Imgproc.drawContours(border, contours, index, new Scalar(255)); // 255 = draw contours in white
return border;
}
Two quick things you can try:
After thresholding you can:
Do a morphological closing,
or, the most straightforward: cv::findContours, keep the largest if it's more than one, then draw it using cv::fillConvexPoly and you will get this mask. (fillConvexPoly will fill the holes for you)