I implemented the Hough Lines Transform in OpenCV (c++) and I get strange artifacts in the Hough Space. The following picture shows the Hough Space. The distance rho is depicted in the rows while the 180 columns represent the angle from 0 to 179 degree. If you zoom in on column 45 and 135 you see a vertical line with alternating dark and bright pixels.
http://imgur.com/NDtMn6S
For higher thresholds the lines of the fence are detected fine but when I lower the threshold the artifacts can be seen as 45° or 135° rotated lines in the final picture:
Detected lines for medium threshold
At first I thought it was a mistake in my implementation of the Hough Lines method but get similar lines for medium thresholds using OpenCV's Hough Line method. I also encounter the same problem when using Canny instead of Sobel.
So the question is: why do I get these artifacts and how can I get rid of them? I wasn't able to find anything about this and any help would be appreciated.
This is the code I used with the OpenCV Hough Lines method:
// read in input image and convert to grayscale
Mat frame = imread("fence.png", CV_LOAD_IMAGE_COLOR);
Mat final_out;
frame.copyTo(final_out);
Mat img, gx, gy, mag, angle;
cvtColor(frame, img, CV_BGR2GRAY);
// get the thresholded maggnitude image
Sobel(img, gx, CV_64F, 1, 0);
Sobel(img, gy, CV_64F, 0, 1);
cartToPolar(gx, gy, mag, angle);
normalize(mag, mag, 0, 255, NORM_MINMAX);
mag.convertTo(mag, CV_8U);
threshold(mag, mag, 55, 255.0, THRESH_BINARY);
// apply the hough lines transform and draw the lines
vector<Vec2f> lines;
HoughLines(mag, lines, 1, CV_PI / 180, 240);
for( size_t i = 0; i < lines.size(); i++ )
{
float rho = lines[i][0], theta = lines[i][1];
Point pt1, pt2;
pt1.x = 0;
pt1.y = (rho - pt1.x * cos(theta))/sin(theta);
pt2.x = mag.cols;
pt2.y = (rho - pt2.x * cos(theta))/sin(theta);
line(final_out, pt1, pt2, Scalar(0,0,255), 1, CV_AA);
}
// show the image
imshow("final_image", final_out);
cvWaitKey();
Answering the question - you can't get rid of such artifact - it's mathematical by nature due to discrete nature of image and pixels' grid orthogonality. The only way is to exclude exact 45 degree from the analysis.
I found the source - the bright pixels of anomaly are produced by the next issue:
Red dots - exactly 45 degree bright anomaly - you can see they are doubled making stairs pattern - which doubles number of pixels involved in accumulation.
Blue dots - exactly 45 degree dim anomaly - making chess-board pattern
Green dots - 44 degree line - you can see it's alternate doubling and chess patterns - which mediates anomaly.
If you look on whole picture of Hough transform matrix you will see how brightness slowly shifting across whole picture - representing slowly changing this alternation ratio depending on angle. However, due to nature of the pixel grid, at exactly 45 degree it makes this anomaly very acute. I don't know how to deal with it yet...
Stumbled across this and maybe useful to future people.
The image is inverted; the algorithm is accumulating the white pixels which obviously there are more of along the diagonals of the image. The lines you are looking for are black, which means they are zero valued and not considered.
Related
I have an Mat object derived using the canny edge detectors, I extracted contours from such image using the findContours function. Now what I'd like to do for each of such contours would be somehow check the colour on both sides.
For the "colour" bit I've discretized HSI color space, however I'm very confused on how I could "pick the colours" in both sides given a contour.
Is there a way to easily do this?
You can use the image that you apply the Canny edge detector to do this. Take the gradient of that image. Gradient is a vector. As shown in the wiki page image (shown below), the gradient points in the direction of the greatest rate of increase. If you take the negative gradient, then it points in the direction of the greatest rate of decrease. Therefore, if you sample the gradient of the image at contour points, positive and negative gradients at those points should point to the regions either side of contour points. So, you can sample points along these directions to get an idea about the colors you want.
Image gradient:
Sample python code shows how this is done for the simple image shown below. It uses Sobel to calculate the gradient.
Input image:
Canny edges and sampled points:
Green: point on contour
Red: point in the positive gradient direction
Blue: point in the negative gradient direction
import cv2
import numpy as np
from matplotlib import pyplot as plt
im = cv2.imread('grad.png', 0)
dx = cv2.Sobel(im, cv2.CV_32F, 1, 0)
dy = cv2.Sobel(im, cv2.CV_32F, 0, 1)
edge = cv2.Canny(im, 64, 192)
dx = dx / np.sqrt(dx*dx + dy*dy + 0.01)
dy = dy / np.sqrt(dx*dx + dy*dy + 0.01)
r = 20
y, x = np.nonzero(edge)
pos1 = (np.int32(x[128]+r*dx[y[128], x[128]]), np.int32(y[128]+r*dy[y[128], x[128]]))
pos2 = (np.int32(x[128]-r*dx[y[128], x[128]]), np.int32(y[128]-r*dy[y[128], x[128]]))
im2 = cv2.cvtColor(edge, cv2.COLOR_GRAY2BGR)
cv2.circle(im2, pos1, 10, (255, 0, 0), 1)
cv2.circle(im2, pos2, 10, (0, 0, 255), 1)
cv2.circle(im2, (x[128], y[128]), 10, (0, 255, 0), 1)
plt.imshow(im2)
I'm using OpenCV 3.2.
I'd like to extract and draw all lines in this image.
For this, I first obtain the contours of the image. For example, I'm using the Canny algorithm, with a double threshold 100 (low) and 200 (high).
Mat image = cv::imread(<image_path>, cv::IMREAD_GRAYSCALE);
cv::Mat contours;
cv::Canny(image, contours, 100, 200);
Then, I call the HoughLines function with a resolution of 1 pixel and π / 45 radians. I just want those lines which have a length of at least 60 pixels.
std::vector<cv::Vec2f> lines;
cv::HoughLines(canny, lines, 1, CV_PI/45, 60);
This returns me a vector lines with the rho p and theta θ parameters in the Hough space of the desired lines. As we know, the line going through a contour pixel (x_i, y_i) is:
p = x_i cos(θ) + y_i sin(θ)
We know p and θ, so we know all the pixels in this line. Two easy points to calculate are A with x_i = 0 and B with y_i = 0.
A = (0, p / sin(θ))
B = (p / cos(θ), 0)
Let's draw them with the line function in blue color.
cv::cvtColor(image, image, CV_GRAY2BGR);
for (unsigned int i = 0; i < lines.size(); ++i) {
float p = lines[i][0];
float theta = lines[i][1];
cv::Point a(0, static_cast<int>(p / std::sin(theta)));
cv::Point b(static_cast<int>(p / std::cos(theta)), 0);
cv::line(image, a, b, cv::Scalar(255, 0, 0));
}
The result is that it only draws me 6 lines, of a total of 14 obtained. As you can see, only those lines that intersect the row 0 and column 0 of the image are drawn. What is the same, those lines which have A and B points in the image boundary. The rest of the lines have these points outside the image.
How can I achieve to draw all the lines in an easy way? I can calculate all the pixels of the obtained lines and draw them (we know them), but I'd like to draw them by minimizing lines of code and using OpenCV api.
Having a Mat that is square area of grayscale pixels. How to create a straight line whose direction is created as a perpendicular to most pixel values change direction (average gradient, aerage over the whole Mat, the result would be just one direction (which can be then drawn as a line))?
For example having
it would look like
How can one do such thing in OpenCV (in python or C++)?
An OpenCV implementation would look something like the following. It solves the problem in a similar fashion as explained in the answer by Mark Setchell, except that normalising the image does not have any effect on the resulting direction.
Mat img = imread("img.png", IMREAD_GRAYSCALE);
// compute the image derivatives for both the x and y direction
Mat dx, dy;
Sobel(img, dx, CV_32F, 1, 0);
Sobel(img, dy, CV_32F, 0, 1);
Scalar average_dx = mean(dx);
Scalar average_dy = mean(dy);
double average_gradient = atan2(-average_dy[0], average_dx[0]);
cout << "average_gradient = " << average_gradient << endl;
And to display the resulting direction
Point center = Point(img.cols/2, img.rows/2);
Point direction = Point(cos(average_gradient) * 100, -sin(average_gradient) * 100);
Mat img_rgb = imread("img.png"); // read the image in colour
line(img_rgb, center, center + direction, Scalar(0,0,255));
imshow("image", img_rgb);
waitKey();
I can't easily tell you how to do it with OpenCV, but I can tell you the method and demonstrate using ImageMagick just at the command-line.
First, I think you need to convert the image to grayscale and normalise it to the full range of black to white - like this:
convert gradient.png -colorspace gray -normalize stage1.png
Then you need to calculate the X-gradient and the Y-gradient of the image using a Sobel filter and then take the inverse tan of the Y-gradient over the X-gradient:
convert stage1.png -define convolve:scale='50%!' -bias 50% \
\( -clone 0 -morphology Convolve Sobel:0 \) \
\( -clone 0 -morphology Convolve Sobel:90 \) \
-fx '0.5+atan2(v-0.5,0.5-u)/pi/2' result.jpg
Then the mean value of the pixels in result.jpg is the direction of your line.
You can see the coefficients used in the convolution for X- and Y-gradient like this:
convert xc: -define morphology:showkernel=1 -morphology Convolve Sobel:0 null:
Kernel "Sobel" of size 3x3+1+1 with values from -2 to 2
Forming a output range from -4 to 4 (Zero-Summing)
0: 1 0 -1
1: 2 0 -2
2: 1 0 -1
convert xc: -define morphology:showkernel=1 -morphology Convolve Sobel:90 null:
Kernel "Sobel#90" of size 3x3+1+1 with values from -2 to 2
Forming a output range from -4 to 4 (Zero-Summing)
0: 1 2 1
1: 0 0 0
2: -1 -2 -1
See Wikipedia here - specifically this line:
Convert the image to grayscale and classify its pixels based on the gray level. For classification, you can use something like Otsu method or kmeans with 2 clusters. Then take the morphological gradient to detect the doundary.
Here are the classified pixels and the boundary using Otsu method.
Now find the non-zero pixels of the boundary image and fit a 2D line to those pixels using the fitLine function that finds a weighted least squares line or use this RANSAC implementation. fitLine gives a normalized vector collinear to the line. Using this vector, you can find an orthogonal vector to it.
I get [0.983035, -0.183421] for the collinear vector using the code below. So, [0.183421 0.983035] is orthogonal to this vector.
Here, in the left image, the red line is the least squares line and the blue line is a perpendicular line to the red one. In the right image, red line is the least squares line and the green one is the line fitted using the RANSAC library mentioned above.
Mat im = imread("LP24W.png", 0);
Mat bw, gr;
threshold(im, bw, 0, 255, CV_THRESH_BINARY|CV_THRESH_OTSU);
Mat kernel = getStructuringElement(MORPH_ELLIPSE, Size(3, 3));
morphologyEx(bw, gr, CV_MOP_GRADIENT, kernel);
vector<vector<Point>> contours;
findContours(gr, contours, CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE);
vector<Point> points;
for (vector<Point>& cont: contours)
{
points.insert(points.end(), cont.begin(), cont.end());
}
Vec4f line;
fitLine(points, line, CV_DIST_L2, 0, 0.01, 0.01);
cout << line << endl;
With OpenCV and C++ I'm trying to detect the lines of a street from an input video. I'm using HoughLinesP and I'd like to detect ONLY the lines that delimit the street, so not horizontal or vertical for example.
Using
HoughLinesP(dst, lines, 1, CV_PI/180, 8, 80, 3),
I detect all the lines so I changed double theta (CV_PI/180) to this
HoughLinesP(dst, lines, 10*CV_PI/180<=theta<=80*CV_PI/180 & 110*CV_PI/180<=theta<=170*CV_PI/180, 8, 80, 3);
But it doesn't work because the console display only the video without any type of lines.
The fourth argument to HoughLinesP is not an angle value that tell opencv to detect only lines which are oriented from the OX axis (ie in polar coordinates). Rather than, the angle value passed in tells opencv algorithm to iterate from 0 to PI (or 2*PI, depends how algorithm is implemented) having this angle as iteration step, e.g. iterating from 0 to PI by PI/180 will take 180 iterations in HoughLinesP trying to find a line for a given (r,alpha).
The solution to find lines having polar coords with given angle range (not the most robust one) could be to detect all lines with HoughLinesP and then iterate over them, calculate angle coorinate and filter out those which are having polar angle coord in a given range.
EDIT (a draft of the algorithm using C++11):
vector<Vec4i> detectedLines;
HoughLinesP(dst, lines, 1, CV_PI/180, 50, 50, 10 );
vector<Vec4i> filteredLines(detectedLines.size());
const float downAngleRange = 30*CV_PI/180;
const float upAngleRange = 60*CV_PI/180;
auto it = copy_if(detectedLines.begin(), detectedLines.end(),
filteredLines.begin(),
[](const Vec4i &v) {
float angle = calculateAnglePolarCord(v);
return angle <= upAngleRange && angle >= downAngleRange;
});
filteredLInes.resize(std::distance(filteredLines.begin(),it));
Where calculateAnglePolarCord is a method which on given line calculats its second (angle) polar coordinate.
Remember to implement good float comparison technique.
I wanted to detect ellipse in an image. Since I was learning Mathematica at that time, I asked a question here and got a satisfactory result from the answer below, which used the RANSAC algorithm to detect ellipse.
However, recently I need to port it to OpenCV, but there are some functions that only exist in Mathematica. One of the key function is the "GradientOrientationFilter" function.
Since there are five parameters for a general ellipse, I need to sample five points to determine one. Howevere, the more sampling points indicates the lower chance to have a good guess, which leads to the lower success rate in ellipse detection. Therefore, the answer from Mathematica add another condition, that is the gradient of the image must be parallel to the gradient of the ellipse equation. Anyway, we'll only need three points to determine one ellipse using least square from the Mathematica approach. The result is quite good.
However, when I try to find the image gradient using Sobel or Scharr operator in OpenCV, it is not good enough, which always leads to the bad result.
How to calculate the gradient or the tangent of an image accurately? Thanks!
Result with gradient, three points
Result without gradient, five points
----------updated----------
I did some edge detect and median blur beforehand and draw the result on the edge image. My original test image is like this:
In general, my final goal is to detect the ellipse in a scene or on an object. Something like this:
That's why I choose to use RANSAC to fit the ellipse from edge points.
As for your final goal, you may try
findContours and [fitEllipse] in OpenCV
The pseudo code will be
1) some image process
2) find all contours
3) fit each contours by fitEllipse
here is part of code I use before
[... image process ....you get a bwimage ]
vector<vector<Point> > contours;
findContours(bwimage, contours, CV_RETR_LIST, CV_CHAIN_APPROX_NONE);
for(size_t i = 0; i < contours.size(); i++)
{
size_t count = contours[i].size();
Mat pointsf;
Mat(contours[i]).convertTo(pointsf, CV_32F);
RotatedRect box = fitEllipse(pointsf);
/* You can put some limitation about size and aspect ratio here */
if( box.size.width > 20 &&
box.size.height > 20 &&
box.size.width < 80 &&
box.size.height < 80 )
{
if( MAX(box.size.width, box.size.height) > MIN(box.size.width, box.size.height)*30 )
continue;
//drawContours(SrcImage, contours, (int)i, Scalar::all(255), 1, 8);
ellipse(SrcImage, box, Scalar(0,0,255), 1, CV_AA);
ellipse(SrcImage, box.center, box.size*0.5f, box.angle, 0, 360, Scalar(200,255,255), 1, CV_AA);
}
}
imshow("result", SrcImage);
If you focus on ellipse(no other shape), you can treat the value of the pixels of the ellipse as mass of the points.
Then you can calculate the moment of inertial Ixx, Iyy, Ixy to find out the angle, theta, which can rotate a general ellipse back to a canonical form (X-Xc)^2/a + (Y-Yc)^2/b = 1.
Then you can find out Xc and Yc by the center of mass.
Then you can find out a and b by min X and min Y.
--------------- update -----------
This method can apply to filled ellipse too.
More than one ellipse on a single image will fail unless you segment them first.
Let me explain more,
I will use C to represent cos(theta) and S to represent sin(theta)
After rotation to canonical form, the new X is [eq0] X=xC-yS and Y is Y=xS+yC where x and y are original positions.
The rotation will give you min IYY.
[eq1]
IYY= Sum(m*Y*Y) = Sum{m*(xS+yC)(xS+yC)} = Sum{ m(xxSS+yyCC+xySC) = Ixx*S^2 + Iyy*C^2 + Ixy*S*C
For min IYY, d(IYY)/d(theta) = 0 that is
2IxxSC - 2IyySC + Ixy(CC-SS) = 0
2(Ixx-Iyy)/Ixy = (SS-CC)/SC = S/C+C/S = Z+1/Z
While programming, the LHS is just a number, let's said N
Z^2 - NZ +1 =0
So there are two roots of Z hence theta, let's said Z1 and Z2, one will min the IYY and the other will max the IYY.
----------- pseudo code --------
Compute Ixx, Iyy, Ixy for a hollow or filled ellipse.
Compute theta1=atan(Z1) and theta2=atan(Z2)
Put These two theta into eq1 find which is smaller. Then you get theta.
Go back to those non-zero pixels, transfer them to new X and Y by the theta you found.
Find center of mass Xc Yc and min X and min Y by sort().
-------------- by hand -----------
If you need the original equation of the ellipse
Just put [eq0] into the canonical form
You're using terms in an unusual way.
Normally for images, the term "gradient" is interpreted as if the image is a mathematical function f(x,y). This gives us a (df/dx, df/dy) vector in each point.
Yet you're looking at the image as if it's a function y = f(x) and the gradient would be f(x)/dx.
Now, if you look at your image, you'll see that the two interpretations are definitely related. Your ellipse is drawn as a set of contrasting pixels, and as a result there are two sharp gradients in the image - the inner and outer. These of course correspond to the two normal vectors, and therefore are in opposite directions.
Also note that your image has pixels. The gradient is also pixelated. The way your ellipse is drawn, with a single pixel width means that your local gradient takes on only values that are a multiple of 45 degrees:
▄▄ ▄▀ ▌ ▀▄