What preprocessing step to take before HoughLines - c++

I have written a program to
Combined Edge information and colour information to form the image I would need to detect straight line from
Then I use findContour and drawContour to redraw the evidence map
after which I use some thinning algorithm to collapse the line into one single line
Use HoughLinesP to calculate for line Segment. Only have traces of the line. And most importantly have missed out the horizontal line
From these set of line Segment, calculate the intersection point (not included in code)
From the intersection, draw a quadrilateral from the intersection point, vertices(called it v1 and v2, furthest from the intersection point) of the horizontal/vertical line and a vertex calculate based on the reflection of the intersection point on the line between v1 and v2
However, it is not working as i think it should. I think the problem lies with that the interior border of the rectangle is not filled. Should I use morphology eg dilation then erode. I have run out of ideas to preprocess the two "cues" image before trying to detect for intersection with Hough Transform
Need everyone help!
THanks in advance
Below is my code snippet to generate the following
int FindBoxes(cv::Mat& colorMap,cv::Mat& edgeMap)
{
cv::Mat frame;
// colorMap is a coloured filtered map while edgeMap is an edge filter Map. frame will be the colour i want
cv::bitwise_and(colorMap, edgeMap, frame);
// A trial method by using findContour to get the interior line filled up
std::vector<std::vector<cv::Point> > contours;
std::vector<cv::Vec4i> hierarchy;
cv::findContours(frame, contours, hierarchy, CV_RETR_EXTERNAL,
cv::CHAIN_APPROX_SIMPLE);
cv::Mat Map = cv::Mat::zeros(cueMap1.size(), CV_8U);
cv::drawContours(Map, contours, -1, cv::Scalar(255, 255, 255), CV_FILLED);
// thin the line to collapse it into one single line for Hough Detection
cv::Mat thin;
thinning(Map, Map);
cv::GaussianBlur(Map, Map, cv::Size(5,5),1.0,1.0);
std::vector<cv::Vec4i> lines;
cv::HoughLinesP(Map, lines,1, CV_PI/90, 2, 30, 0);
return 0;
}

Related

How to remove unwanted contours using opencv?

I am able to find the contours and labelled all of them. Now I want to remove some contours from the image and need some specific contours only using opencv.
I have used following code to get the contours. This code is working fine for me to get the contours and its labels: as you see the binary image and its contours in the picture. Here, I want to remove contours which are above contour 45 and below contour 22. Basically, I need the center part between the two long horizontal lines.
int main(int argc, char *argv[])
{
if (argc != 2) {
cerr << "usage: "<<argv[0]<< "<input_file with path>"<<endl;
return -1;
}
cv::Mat im_bw = cv::imread(argv[1],cv::IMREAD_GRAYSCALE); //loading an image//
// Binarize the output image
cv::Mat binarized_image;
cv::threshold(im_bw, binarized_image, 128, 255, cv::THRESH_BINARY);
cv::imshow("binary_image.png", binarized_image);
cv::waitKey(0);
vector<vector<cv::Point>> contours;
vector<cv::Vec4i> hierarchy;
cv::findContours(binarized_image,contours,hierarchy,cv::RETR_TREE,cv::CHAIN_APPROX_SIMPLE);
cv::Mat mask = cv::Mat::zeros(im_bw.size(),CV_8UC3);
cv::drawContours(mask,contours,-1,cv::Scalar(0,255,255),1);
for( int i=0; i<contours.size(); i++)
{
cv::putText(mask,to_string(i),contours[i][0],1,1,cv::Scalar(255,0,0),1);
}
cout<<"Contours : "<<contours.size()<<endl;
for(cv::Vec4i k:hierarchy)
{
cout<<k<<endl;
}
cv::imshow("Contours_binary_image.png", mask);
cv::waitKey(0);
return 0;
}
There are probably a few ways you could do it depending on what you care about. Do you want it to be fast, or accurate?
Here is a way you could do it geometrically that should be fairly accurate, and with some optimizations, could be pretty fast.
You'll need a way to identify contours 45 and 22. In the question you identified them as the two large horizontal lines. That is a good place to start.
A simple way to do it would be to iterate through all the contours and keep track of the min and max point values. The horizontal lines will be the largest distance in the min/max X direction and have a relatively small distance between min and max Y. It will probably requires some tweaking and defining some more rules for what is allowed to be considered a "horizontal line".
Once you have the horizontal lines identified, the next step is removing all the ones above and below them. Doing this for the top and bottom will be the same, just in opposite directions. I'll walk through how you could do it with the top line.
Each contour is made up of smaller individual line segments. For each linesegment in the horizontal line's contour, check that every other contour (each segment in said contour) is either above it, or intersects with it. If either is true, then you can remove it. For a large number of contours, or very complex contours, this will be quite slow. There are a number of optimizations you could make. You could simplify the contour shapes. Or compute bounding boxes around them and check the bounding box is above the horizontal line, if it intersects, you can look at it closer and see if the line itself intersects.
Here are the steps I can suggest you to handle this issue:
First you need to get the mass center of each contour by using
opencv moments. This will give you the weighted centeral point
of contours as x,y coordinates.
Then make a filter according to the mass center's y-axis. Between 45th and 22th contour's y-axis values will be the valid contours.
Only draw the contours which are valid according to your filter.
This may help to find the mass centers.

How to straighten curved line using OpenCV?

I have image with curved line like this :
I couldn't find a technique to straighten curved line using OpenCV. It is similar to this post Straightening a curved contour, but my question is specific to coding using opencv (in C++ is better).
So far, I'm only able to find the contour of the curved line.
int main()
{
Mat src; Mat src_gray;
src = imread("D:/2.jpg");
cvtColor(src, src_gray, COLOR_BGR2GRAY);
cv::blur(src_gray, src_gray, Size(1, 15));
Canny(src_gray, src_gray, 100, 200, 3);
/// Find contours
vector<vector<Point> > contours;
vector<Vec4i> hierarchy;
RNG rng(12345);
findContours(src_gray, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE, Point(0, 0));
/// Draw contours
Mat drawing = Mat::zeros(src_gray.size(), CV_8UC3);
for (int i = 0; i < contours.size(); i++)
{
drawContours(drawing, contours, i, (255), 1, 8, hierarchy, 0, Point());
}
imshow("Result window", drawing);
imwrite("D:/C_Backup_Folder/Ivan_codes/VideoStitcher/result/2_res.jpg", drawing);
cv::waitKey();
return 0;
}
But I have no idea how to determine which line is curved and not, and how to straighten it. Is it possible? Any help would be appreciated.
Here is my suggestion:
Before everything, resize your image into a much bigger image (for example 5 times bigger). Then do what you did before, and get the contours. Find the right-most pixel of each contour, and then survey all pixel of that contour and count the horizontal distance of each pixels to the right-most pixel and make a shift for that row (entire row). This method makes a right shift to some rows and left shift to the others.
If you have multiple contours, calculate this shift value for every one of them in every single row and compute their "mean" value, and do the shift according to that mean value for each row.
At the end resize back your image. This is the simplest and fastest thing I could think of.

Detecting Circles without using Hough Circles

I have an image of a circle, I want to find the circle but not using hough circles.
I found a way, linked here.
But I can't find the transition coordinates from white to black as I don't know the x and y coordinates in the circle. What other methods are there, or how can I make that approach work?
This is my test image:
Another approach (that is useful for more than just circles) would be to find the image contours and do image moment analysis on the circle to find it's centre of mass:
I recommend learning them if you'e going to move forward with image processing. They're pretty helpful approaches that transform images into more useful structures.
One possible approach is to first threshold the image to get rid of some of the noise around the circle. Then you can extract the edge of the circle using Canny edge detection. Finally, findNonZero to get a list of pixel coordinates.
I first did a quick prototype with Python:
import cv2
import numpy as np
img = cv2.imread('circle.png', 0)
mask = cv2.threshold(img, 127, 255, cv2.THRESH_BINARY)[1]
edges = cv2.Canny(mask, 20, 100)
points = np.array([p[0] for p in cv2.findNonZero(edges)])
And then ported it to C++, adding some extra code to save all the intermediate images and plot the found pixels.
#include <opencv2/opencv.hpp>
int main()
{
cv::Mat img(cv::imread("circle.png", 0));
cv::Mat mask;
cv::threshold(img, mask, 127, 255, cv::THRESH_BINARY);
cv::imwrite("circle_1.png", mask);
cv::Mat edges;
cv::Canny(mask, edges, 20, 100);
cv::imwrite("circle_2.png", edges);
std::vector<cv::Point2i> points;
cv::findNonZero(edges, points);
cv::Mat output(cv::Mat::zeros(edges.size(), CV_8UC3));
for (auto const& p : points) {
output.at<cv::Vec3b>(p) = cv::Vec3b(127, 255, 127);
}
cv::imwrite("circle_3.png", output);
}
Output of threshold:
Output of Canny:
Re-plotted pixels:

Proper way to determine thresholding parameters

I am trying to find triangles (blue contours) and trapezoids (yellow contours) in real time. In general it's okay.
But there is some problems. First it's a false positives. Triangles become trapezoids and vice versa. And I don't know how to how to solve this problem.
Second it's "noise". . I tried to check area of the figure, but the noise can be equal to the area. So it did not help so much. The noise depends on the thresholding parameters. cv::adaptiveThresholddoes not help at all. It's adds even more noise (and it so SLOW) erode and dilate cant fix it in a proper way
And here is my code.
cv::Mat detect(cv::Mat imageRGB)
{
//RGB -> GRAY
cv::Mat imageGray;
cv::cvtColor(imageRGB, imageGray, CV_BGR2GRAY);
//Bluring it
cv::Mat image;
cv::GaussianBlur(imageGray, image, cv::Size(5,5), 2);
//Thresholding
cv::threshold(image, image, 100, 255, CV_THRESH_BINARY_INV);
//SLOW and NOISE
//cv::adaptiveThreshold(image, image, 255.0, CV_ADAPTIVE_THRESH_GAUSSIAN_C, CV_THRESH_BINARY, 21, 0);
//Calculating canny params.
cv::Scalar mu;
cv::Scalar sigma;
cv::meanStdDev(image, mu, sigma);
cv::Mat imageCanny;
cv::Canny(image,
imageCanny,
mu.val[0] + sigma.val[0],
mu.val[0] - sigma.val[0]);
//Detecting conturs.
std::vector<std::vector<cv::Point> > contours;
std::vector<cv::Vec4i> hierarchy;
cv::findContours(imageCanny, contours, hierarchy,CV_RETR_TREE, CV_CHAIN_APPROX_NONE);
//Hierarchy is not needed here so clear it.
hierarchy.clear();
for (std::size_t i = 0; i < contours.size(); i++)
{
//fitEllipse need at last 5 points.
if (contours.at(i).size() < 5)
{
continue;
}
//Skip small contours.
if (std::fabs(cv::contourArea(contours.at(i))) < 800.0)
{
continue;
}
//Calculating RotatedRect from contours NOT from hull
//because fitEllipse need at last 5 points.
cv::RotatedRect bEllipse = cv::fitEllipse(contours.at(i));
//Finds the convex hull of a point set.
std::vector<cv::Point> hull;
cv::convexHull(contours.at(i), hull, true);
//Approx it, so we'll get 3 point for triangles
//and 4 points for trapez.
cv::approxPolyDP(hull, hull, 15, true);
//Is our contour convex. It's mast be.
if (!cv::isContourConvex(hull))
{
continue;
}
//Triangle
if (hull.size() == 3)
{
cv::drawContours(imageRGB, contours, i, cv::Scalar(255, 0, 0), 2);
cv::circle(imageRGB, bEllipse.center, 3, cv::Scalar(0, 255, 0), 2);
}
//trapez
if (hull.size() == 4)
{
cv::drawContours(imageRGB, contours, i, cv::Scalar(0, 255, 255), 2);
cv::circle(imageRGB, bEllipse.center, 3, cv::Scalar(0, 0, 255), 2);
}
}
return imageRGB;
}
So... In general all problems coused by wrong thresholding parameters, how can I calculete it in a proper way (automatically, of course)? And how can I can (lol, sorry for my english) prevent false positives?
Thesholding - i think that you should try Otsu binarization - here is some theory and a nice picture and here is documentation. This kind of thresholding generally is trying to find 2 most common values in image and use average value of them as a threshold value.
Alternatively consider using HSV color space, it might be easier to distinguish black and white regions from other regions. Another idea is to use inRange function (in RGB or in HSV color space - should work in woth situations) - you need to find 2 ranges (one from black regions and one for white) and search only for those regions (using inRange function) - look at this post.
Another way to accomplish this task might be using some library for blob extraction like this one or blob extractor which is part of OpenCV.
Distinguish triangle from trapezoid - i see 2 basic ways to improve you solution here:
in this line cv::approxPolyDP(hull, hull, 15, true); make third parameter (15 in this situation) not a constant value, but some part of contour area or length. Definitely it should adapt to contour size, it can't be just a canstant value. It's hard to say how to calculate it without some testing - try to start with 1-5% of contour area or length (i would start with length, but this is just my guess) and see whether this value is fine/to big/to small an check other values if needed. Unfortunetely there is no other way, but finding this equation manually shouldn't take very long time.
when you have 4 or 5 points calculate the equations of lines which join consecutive points (point 1 with point 2, point 2 with point 3, etc don't forget to calculate line between first point and last point), than check whether any 2 of those lines are parallel (or at least are close to being parallel - angle between them is close to 0 degress) - if you find any parallel lines than this contour is trapezoid, otherwise it's a triangle.

Automatic perspective correction OpenCV

I am trying to implement Automatic perspective correction in my iOS program and when I use the test image I found on the tutorial everything works as expected. But when I take a picture I get back a weird result.
I am using code found in this tutorial
When I give it an image that looks like this:
I get this as the result:
Here is what dst gives me that might help.
I am using this to call the method which contains the code.
quadSegmentation(Img, bw, dst, quad);
Can anyone tell me when I am getting so many green lines compared to the tutorial? And how I might be able to fix this and properly crop the image to only contain the card?
For perspective transform you need,
source points->Coordinates of quadrangle vertices in the source image.
destination points-> Coordinates of the corresponding quadrangle vertices in the destination image.
Here we will calculate these point by contour process.
Calculate Coordinates of quadrangle vertices in the source image
You will get the your card as contour by just by blurring, thresholding, then find contour, find largest contour etc..
After finding largest contour just calculate approximates a polygonal curve, here you should get 4 Point which represent corners of your card. You can adjust the parameter epsilon to make 4 co-ordinates.
Calculate Coordinates of the corresponding quadrangle vertices in the destination image
This can be easily find out by calculating bounding rectangle for largest contour.
In below image the red rectangle represent source points and green for destination points.
Adjust the co-ordinates order and Apply Perspective transform
Here I manually adjust the co-ordinates order and you can use some sorting algorithm.
Then calculate transformation matrix and apply wrapPrespective
See the final result
Code
Mat src=imread("card.jpg");
Mat thr;
cvtColor(src,thr,CV_BGR2GRAY);
threshold( thr, thr, 70, 255,CV_THRESH_BINARY );
vector< vector <Point> > contours; // Vector for storing contour
vector< Vec4i > hierarchy;
int largest_contour_index=0;
int largest_area=0;
Mat dst(src.rows,src.cols,CV_8UC1,Scalar::all(0)); //create destination image
findContours( thr.clone(), contours, hierarchy,CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE ); // Find the contours in the image
for( int i = 0; i< contours.size(); i++ ){
double a=contourArea( contours[i],false); // Find the area of contour
if(a>largest_area){
largest_area=a;
largest_contour_index=i; //Store the index of largest contour
}
}
drawContours( dst,contours, largest_contour_index, Scalar(255,255,255),CV_FILLED, 8, hierarchy );
vector<vector<Point> > contours_poly(1);
approxPolyDP( Mat(contours[largest_contour_index]), contours_poly[0],5, true );
Rect boundRect=boundingRect(contours[largest_contour_index]);
if(contours_poly[0].size()==4){
std::vector<Point2f> quad_pts;
std::vector<Point2f> squre_pts;
quad_pts.push_back(Point2f(contours_poly[0][0].x,contours_poly[0][0].y));
quad_pts.push_back(Point2f(contours_poly[0][1].x,contours_poly[0][1].y));
quad_pts.push_back(Point2f(contours_poly[0][3].x,contours_poly[0][3].y));
quad_pts.push_back(Point2f(contours_poly[0][2].x,contours_poly[0][2].y));
squre_pts.push_back(Point2f(boundRect.x,boundRect.y));
squre_pts.push_back(Point2f(boundRect.x,boundRect.y+boundRect.height));
squre_pts.push_back(Point2f(boundRect.x+boundRect.width,boundRect.y));
squre_pts.push_back(Point2f(boundRect.x+boundRect.width,boundRect.y+boundRect.height));
Mat transmtx = getPerspectiveTransform(quad_pts,squre_pts);
Mat transformed = Mat::zeros(src.rows, src.cols, CV_8UC3);
warpPerspective(src, transformed, transmtx, src.size());
Point P1=contours_poly[0][0];
Point P2=contours_poly[0][1];
Point P3=contours_poly[0][2];
Point P4=contours_poly[0][3];
line(src,P1,P2, Scalar(0,0,255),1,CV_AA,0);
line(src,P2,P3, Scalar(0,0,255),1,CV_AA,0);
line(src,P3,P4, Scalar(0,0,255),1,CV_AA,0);
line(src,P4,P1, Scalar(0,0,255),1,CV_AA,0);
rectangle(src,boundRect,Scalar(0,255,0),1,8,0);
rectangle(transformed,boundRect,Scalar(0,255,0),1,8,0);
imshow("quadrilateral", transformed);
imshow("thr",thr);
imshow("dst",dst);
imshow("src",src);
imwrite("result1.jpg",dst);
imwrite("result2.jpg",src);
imwrite("result3.jpg",transformed);
waitKey();
}
else
cout<<"Make sure that your are getting 4 corner using approxPolyDP..."<<endl;
teethe This typically happens when you rely on somebody else code to solve your particular problem instead of adopting the code. Look at the processing stages and also the difference between their and your image (it is a good idea by the way to start with their image and make sure the code works):
Get the edge map. - will probably work since your edges are fine
Detect lines with Hough transform. - fail since you have lines not only on the contour but also inside of your card. So expect a lot of false alarm lines
Get the corners by finding intersections between lines. - fail for the above mentioned reason
Check if the approximate polygonal curve has 4 vertices. - fail
Determine top-left, bottom-left, top-right, and bottom-right corner. - fail
Apply the perspective transformation. - fail completely
To fix your problem you have to ensure that only lines on the periphery are extracted. If you always have a dark background you can use this fact to discard the lines with other contrasts/polarities. Alternatively you can extract all the lines and then select the ones that are closest to the image boundary (if your background doesn't have lines).