The method below to remove lens distortion from a camera was written more than ten years ago and i am trying to understand how the approximation works.
void Distortion::removeLensDistortion(const V2 &dist, V2 &ideal) const
{
const int ITERATIONS=10;
V2 xn;
xn.x=(dist.x - lensParam.centerX)/lensParam.fX;
xn.y=(dist.y - lensParam.centerY)/lensParam.fY;
V2 x=xn;
for (int i=0;i<ITERATIONS;i++) {
double r2 = Utilities::square(x.x)+Utilities::square(x.y);
double r4 = Utilities::square(r2);
double rad=1+lensParam.kc1 * r2 + lensParam.kc2 * r4;
x.x/=rad;
x.y/=rad;
}
ideal.x=x.x*lensParam.fX+lensParam.centerX;
ideal.y=x.y*lensParam.fY+lensParam.centerY;
}
As a reminder:
lensParam.centerX and lensParam.centerY is the principal point
lensParam.fX and lensParam.fY is the focal length in pixel
lensParam.kc1 and lensParam.kc2 are the first two radial distortion coefficients. This is k_1 and k_2 in the formula below.
The formula to add lens distortion given the first two radial distortion parameters is as follows:
x_distorted = x_undistorted * (1+k_1 * r² + k_2 * r^4)
y_distorted = y_undistorted * (1+k_1 * r² + k_2 * r^4)
where r²=(x_undistorted)²+(y_undistorted)² and r^4=(r²)²
In the code above, the term (1+k_1 * r² + k_2 * r^4) is calculated and saved in the variable rad and the distorted x is divided by rad in each of the ten iterations.
All of the cameras we use have a pincushion distortion (so k_1<0)
The question is how is this algorithm approximating the undistorted image points?
Do you know if there is any paper in which this algorithm is proposed?
The opencv undistortion may be a bit similar, so that link may be useful but it is not quite the same though.
Related
Could someone help me understand the following function that draws a polygon of N sides (i.e. 3 being a triangle and 4 being a square):
float theta = atan(pos.x, pos.y);
float rotate_angle = 2 * PI / N;
float d = cos(floor(0.5 + theta / rotate_angle) * rotate_angle - theta) * length(pos);
What I understand from this illustration is that:
we're interested in finding the angle indicated by the red curve (call it alpha)
cos(alpha) * length will project the green line onto the blue line
by comparing the size of said projection with that of the blue line (radius of circle), we know whether a test point is inside or outside of the polygon we're trying to draw
Question
Why does alpha equal floor(0.5 + theta / rotate_angle) * rotate_angle - theta? Where does 0.5 come from? What's the significance of theta / rotate_angle?
What I have read:
[1] https://codepen.io/nik-lever/full/ZPKmmx
[2] https://thndl.com/square-shaped-shaders.html
[3] https://thebookofshaders.com/07
Simply, floor(0.5 + x) = round(x). However, because round(x) may not be available in some environments (e.g. in GLES2), floor(0.5 + x) is to be used instead.
Then, since n = round(theta / rotate_angle) gives edge section index which contains pos (e.g. n =-1, 0 or 1 for a triangle) , n * rotate_angle is the angle of edge center point(=blue line) which is nearest to the theta.
Therefore, alpha = n * rotate_angle - theta is certainly relative angle from pos to the nearest center, where -rotate_angle/2 < alpha <= rotate_angle/2.
Checking pos's projection length to the center point direction, it's possible to tell inside or outside. To detect discrete direction of polygon edges('s orthogonal vectors) seamlessly, round() function is used.
I am searching for an algorithm (using OpenCV C or C++) which does this:
Given the boundary image, I want to find the local curvature at all points and color map it, which is what is done in the image displayed above. I got this image from Wikipedia but haven't been able to find out a way to color the boundary in this way. Kindly let me know how it can be done.
If you observe the boundary, red denotes boundary has high slope, yellow shows that the boundary is almost linear.
How can this be done?
Edit
Just to give you an idea of how I was trying to do this since two days:
I used the openCV functions convexHull and convexityDefects but realized that I am going in the wrong direction. I have to work only on the contours/boundaries of the binary image.
You can solve the problem by fitting a path of cubic Bezier curves to the boundary, then taking the curvature analytically.
[elaborated]
The boundary consists of a list of points in x, y at pixel centres, each point 1px or root 2 px form the next in the list. You need to fit a smooth cubic Bezier path to this, using a technique by Schnider in Graphics Gems (Gems 1, pp 612, An algorithm for Fitting digitized curves).
The step along the curve taking tiny steps which are always sub-pixel, and
take the curvature using
double BezierCurve::Curvature(double t) const
{
// Nice mathematically perfect formula
//Vector2 d1 = Tangent(t);
//Vector2 d2 = Deriv2(t);
//return (d1.x * d2.y - d1.y * d2.x) / pow(d1.x * d1.x + d1.y * d1.y, 1.5);
// Get the cubic coefficients like this, I store them in the Bezier
// class
/*
a = p3 + 3.0 * p1 - 3.0 * p2 - p0;
b = 3.0 * p0 - 6.0 * p1 + 3.0 * p2;
c = 3.0 * p1 - 3.0 * p0;
d = p0;
*/
double dx, dy, ddx, ddy;
dx = 3 * this->ax * t*t + 2 * this->bx * t + this->cx;
ddx = 6 * this->ax * t + 2 * this->bx;
dy = 3 * this->ay * t*t + 2 * this->by * t + this->cy;
ddy = 6 * this->ay * t + 2 * this->by;
if (dx == 0 && dy == 0)
return 0;
return (dx*ddy - ddx*dy) / ((dx*dx + dy*dy)*sqrt(dx*dx + dy*dy));
}
OpenCV findContours used with mode= CV_RETR_EXTERNAL and method= CV_CHAIN_APPROX_NONE will give you all boundary pixels ordered such as two subsequent points are neighbors.
To get the radius of a circumference by three points, there are a lot of info in the Web. Because you only need the radius, not the center, this stackexchange answer is fast.
In pseudo code:
vector_of_points = OpenCV::findContours(...)
p1 = vector start
p2, p3 are next points in vector
//boundary is circular, so in the first loop pass we must adjust
p2 = next point
p3 = last point
//Use p1 as our iterator
while ( p1 <= vector.end )
{
//curvature
radius = calculateRadius(p1, p2, p3)
//set color for pixel p2
setColor(p, radius)
increment p1, p2, p3
adjust for start point = end point
}
The following link suggests that we can convert dual fisheye coordinates to equirectangular coordinates using the following equations:
// 2D fisheye to 3D vector
phi = r * aperture / 2
theta = atan2(y, x)
// 3D vector to longitude/latitude
longitude = atan2(Py, Px)
latitude = atan2(Pz, (Px^2 + Py^2)^(0.5))
// 3D vector to 2D equirectangular
x = longitude / PI
y = 2 * latitude / PI
I applied to above equations to write my source code like this:
const float FOV = 220.0f * PI / 180.0f;
float r = sqrt(u*u + v*v);
float theta = atan2(v, u);
float phi = r * FOV * 0.5f;
float px = u;
float py = r * sin(phi);
float pz = v;
float longitude = atan2(py, px); // theta
float latitude = atan2(pz, sqrt(px*px + py*py)); // phi
x = longitude / PI;
y = 2.0f * latitude / PI;
Unfortunately my math is not good enough to understand this and not sure if I write the above code correctly, where I tried to guess the values for px, py and pz.
Assume my camera FOV is 220 degrees, and the camera resolution is 2880x1440, I would expect the point (358, 224) for rear camera in the overlapped area and the point (2563, 197) for front camera in the overlapped area would both map to a coordinate close to (2205, 1009). However the actual mapping points are (515.966370,1834.647949) and (1644.442017,1853.060669) respectively, which are both very far away from (2205,1009). Please kindly suggest how to fix the above code. Many thanks!
You are building the equirectangular image, so I would suggest you to use the inverse mapping.
Start with pixel locations in the target image you are painting. Convert the 2D location to longitude/latitude. Then convert that to a 3D point on the surface of the unit sphere. Then convert from the 3D point to a location in the 2D fisheye source image. In Paul Bourke page, you would start with the bottom equation, then the rightmost one, then the topmost one.
Use landmark points like 90° long 0° lat, to verify the results make sense at each step.
The final result should be a location in the source fisheye image in the [-1..+1] range. Remap to pixel or to UV as needed. Since the source is split in two eye images you will also need a mapping from target (equirect) longitudes to the correct source sub-image.
I can't figure out how to merge circles in C++. I accomplished to union two polygons using Boost Geometry, however, the problem is that I don't know how to transform polygons to circles (if that is possible at all in Boost Geometry).
No visual representation of the geometry is necessary, in the end I would like to transform it to WKT format.
Is Boost Geometry the right approach or are there better libraries for that?
Thank you,
Andy
You can approximate circle with center point C and radius R using regular polygon with N vertices (choose N depending on needed precision). Vertex coordinates:
V[i].X = C.X + R * Cos(i * 2 * Pi / N)
V[i].Y = C.Y + R * Sin(i * 2 * Pi / N)
I'm trying to use LogPolar transform to obtain the scale and the rotation angle from two images. Below are two 300x300 sample images. The first rectangle is 100x100, and the second rectangle is 150x150, rotated by 45 degree.
The algorithm:
Convert both images to LogPolar.
Find the translational shift using Phase Correlation.
Convert the translational shift to scale and rotation angle (how to do this?).
My code:
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/imgproc/imgproc_c.h>
#include <opencv2/highgui/highgui.hpp>
#include <iostream>
int main()
{
cv::Mat a = cv::imread("rect1.png", 0);
cv::Mat b = cv::imread("rect2.png", 0);
if (a.empty() || b.empty())
return -1;
cv::imshow("a", a);
cv::imshow("b", b);
cv::Mat pa = cv::Mat::zeros(a.size(), CV_8UC1);
cv::Mat pb = cv::Mat::zeros(b.size(), CV_8UC1);
IplImage ipl_a = a, ipl_pa = pa;
IplImage ipl_b = b, ipl_pb = pb;
cvLogPolar(&ipl_a, &ipl_pa, cvPoint2D32f(a.cols >> 1, a.rows >> 1), 40);
cvLogPolar(&ipl_b, &ipl_pb, cvPoint2D32f(b.cols >> 1, b.rows >> 1), 40);
cv::imshow("logpolar a", pa);
cv::imshow("logpolar b", pb);
cv::Mat pa_64f, pb_64f;
pa.convertTo(pa_64f, CV_64F);
pb.convertTo(pb_64f, CV_64F);
cv::Point2d pt = cv::phaseCorrelate(pa_64f, pb_64f);
std::cout << "Shift = " << pt
<< "Rotation = " << cv::format("%.2f", pt.y*180/(a.cols >> 1))
<< std::endl;
cv::waitKey(0);
return 0;
}
The log polar images:
For the sample image images above, the translational shift is (16.2986, 36.9105). I have successfully obtain the rotation angle, which is 44.29. But I have difficulty in calculating the scale. How to convert the given translational shift to obtain the scale?
You have two Images f1, f2 with f1(m, n) = f2(m/a , n/a) That is f1 is scaled by factor a
In logarithmic notation that is equivalent to f1(log m, log n) = f2(logm − log a, log n − log a) where log a is the shift in your phasecorrelated image.
Compare B. S. Reddy, B. N. Chatterji: An FFT-Based Technique for Translation, Rotation and
Scale-Invariant Image Registration, IEEE Transactions On Image Processing Vol. 5
No. 8, IEEE, 1996
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.185.4387&rep=rep1&type=pdf
here is python version
which tells
ir = abs(ifft2((f0 * f1.conjugate()) / r0))
i0, i1 = numpy.unravel_index(numpy.argmax(ir), ir.shape)
angle = 180.0 * i0 / ir.shape[0]
scale = log_base ** i1
The value for the scale factor is indeed exp(pt.y). However, since you used a "magnitude scale parameter" of 40 for the cvLogPolar function, you now need to divide pt.x by 40 to get the correct value for the displacement:
Scale = exp( pt.x / 40) = exp(16.2986 / 40) = 1.503
The value of the "magnitude scale parameter" for the cvLogPolar function does not affect the displacement produced by the rotation angle pt.x, because according to the math, it cancels out. For that reason, your formula for the rotation gives the correct value.
On another note, I believe the formula for the rotation should actually be:
Rotation = pt.y*360/(a.cols)
But, for some strange reason, the ">> 1" that you added is causing the result to be multiplied by 2 (which I believe you compensated for by multiplying by 180 instead of 360?) Remove it, and you'll see what I mean.
Also, ">>1" is causing a division by 2 in:
cvPoint2D32f(a.cols >> 1, a.rows >> 1)
If to set the center parameter of the cvLogPolar function to the center of the image (which is what you want):
cvPoint2D32f(a.cols/2, a.rows/2)
and
cvPoint2D32f(b.cols/2, b.rows/2)
then, you'll also get the correct value for the rotation (i.e. the same value that you got), and for the scale.
This thread was helpful in getting me started on rotation-invariant phase correlation, so I hope my input will help resolve any lingering issues.
We aim to calculate the scale and rotation (which is incorrectly calculated in the code). Let's start by gathering the equations from the logPolar docs. There they state the following:
(1) I = (dx,dy) = (x-center.x, y-center.y)
(2) rho = M * ln(magnitude(I))
(3) phi = Ky * angle(I)_0..360
Note: rho is pt.x and phi is pt.y in the code above
We also know that
(4) M = src.cols/ln(maxRadius)
(5) Ky = src.rows/360
First, let's solve for scale. Solving for magnitude(I) (i.e. scale) in equation 2, we get
(6) magnitude(I) = scale = exp(rho/M)
Then we substitute for M and simplify to get
(7) magnitude(I) = scale = exp(rho*ln(maxRadius)/src.cols) = pow(maxRadius, rho/src.cols)
Now let's solve for rotation. Solving for angle(I) (i.e. rotation) in equation 3, we get
(8) angle(I) = rotation = phi/Ky
Then we substitute for Ky and simplify to get
(9) angle(I) = rotation = phi*360/src.rows
So, scale and rotation can be calculated using equations 7 and 9, respectively. It might be worth noting that you should use equation 4 for calculation M and Point2f center( (float)a.cols/2, (float)a.rows/2 ) for calculating center as opposed to what is in the code above. There are good bits of info in this logpolar example opencv code.
From the values by phase correlation, the coordinates are rectangular coordinates hence (16.2986, 36.9105) are (x,y). The scale is calculated as
scale = log((x^2 + y^ 2)^0.5) which is approximately 1.6(near to 1.5).
When we calculate angle by using formulae theta = arctan(y/x) = 66(approx).
The theta value is way of the real value(45 in this case).