I need to know if a point lies on a segment in 2d.
I already solved that issue using Boost and intersects between a point and a segment.
Question
is it possible to use the Eigen library only?
Why:
I am currently in the middle of an algorithm that uses Eigen, and I don't want to copy my points again and again in a loop to a Boost geometry construct to check for this.
This is what I currently do and it works. But it adds a dependency.
This is a contrived code:
using Line2 = Eigen::Hyperplane <double, 2>;
using Vec2 = Eigen::Vector2d;
Vec2 a( 0, 0 );
Vec2 b( 1, 1 );
Vec2 d(70, 100);
Line2 ab = Line2::Through( a, b );
auto res = ab.normal();
auto projection_point = ab.projection(d); <- that projection point is not on the segment ab
//.. here I test with Boost and loop again if necessary
if there is no alternative I will keep doing it the way I do, but I am sure someone knows better out there.
I don't want to write the function myself (otherwise, I prefer to keep using Boost). I just want to rely on Eigen.
Thanks
I suggest checking the distance of the point and its projection onto the line against an epsilon. Plus a check against the distance from origin, plus a check (via a dot product) that the point lies in the correct direction from the origin.
Eigen has a parameterized line type for this. It isn't a line segment but an origin plus direction vector. So the maximum distance from the origin has to be computed and stored separately.
using PLine2d = Eigen::ParameterizedLine<double, 2>;
Eigen::Vector2d a = ..., b = ..., d = ...;
PLine2d line = Pline2d::Through(a, b);
double sqrlength = (b - a).squaredNorm();
Eigen::Vector2d projection = line.projection(d);
Eigen::Vector2d fromOrigin = projection - line.origin();
bool isOnLineSegment = projection.isApprox(d) &&
fromOrigin.squaredNorm() <= sqrlength &&
fromOrigin.dot(line.direction()) >= 0.f;
I haven't counted operations but there is a good chance that the code you linked is shorter and faster when implemented in Eigen. Cross and dot products are readily available, after all.
Replace squaredNorm() with stableNorm() if desired. In 2D hypotNorm() would also work well.
A potential issue is isApprox() which, as the documentations states, fails close to zero. Therefore depending on your data, an extended check may be necessary such as
(projection.isApprox(d) || (d.isZero() && projection.isZero()))
All have tuneable epsilon parameters.
Related
I recently became aware of Cell noise based procedural generation techniques. I came across this website which explained the concept of cellular (or Worley) noise quite well. Upon further reading, the author links to Voro-Noise in which Inigo Quilez (the author of Voro-Noise) proceeds to give a rough overview of his shader. I don't understand a few parts of it however. Why is the hash function created as such:
vec3 hash3( vec2 p ) {
vec3 q = vec3( dot(p,vec2(127.1,311.7)),
dot(p,vec2(269.5,183.3)),
dot(p,vec2(419.2,371.9)) );
return fract(sin(q)*43758.5453);
}
I'm not sure the significance of the numbers nor why sin is used here, it doesn't appear to be explained on his page.
I also don't understand why there is a need to iterate over 25 points instead of the 9 used in previous examples.
for (int j=-2; j<=2; j++) {
for (int i=-2; i<=2; i++) {
...
}
}
changing the '2' and '-2' in each loop to '1' and '-1' seems to create artifacts and change the nature of the blur.
Why are we taking the dot product of the distance difference and itself? (which I guess finds the length of the vector squared?)
vec2 r = g - f + o.xy;
float d = dot(r,r);
And finally, while the author of voro-noise goes into what the power function is, he didn't really go into detail into why this works. why do we create K in this way (the author talks about raising smooth step to the power of 1, but I don't see that being a possibility here) Why do we create ww, multiply it by the offsets z component and why do we divide va by wt? I understand we add some sort of interpolation value between all points based on their contribution, but I just don't understand why the interpolator is constructed as is
float k = 1.0+63.0*pow(1.0-v,4.0);
float va = 0.0;
float wt = 0.0;
...
float ww = pow( 1.0-smoothstep(0.0,1.414,sqrt(d)), k );
va += o.z*ww;
wt += ww;
...
return va/wt;
How does the hermitian interpolator (smoothstep here) help here? From what I understand we are passing 0, sqrt(2) and the sqrt(length of difference vector) here.
After reading several posts about getting the 2D transformation of 2D points from one image to another, estimateRigidTransform() seems to be the recommendation. I'm trying to use it. I modified the source code (to change the RANSAC parameters, because its hardcoded, and the hardcoded parameters are not very good)(the source code for this function is in lkpyramid.cpp). I have read up on how RANSAC works, and am trying to understand the steps in estimateRigidTransform().
// choose random 3 non-complanar points from A & B
...
// additional check for non-complanar vectors
a[0] = pA[idx[0]];
a[1] = pA[idx[1]];
a[2] = pA[idx[2]];
b[0] = pB[idx[0]];
b[1] = pB[idx[1]];
b[2] = pB[idx[2]];
double dax1 = a[1].x - a[0].x, day1 = a[1].y - a[0].y;
double dax2 = a[2].x - a[0].x, day2 = a[2].y - a[0].y;
double dbx1 = b[1].x - b[0].x, dby1 = b[1].y - b[0].y;
double dbx2 = b[2].x - b[0].x, dby2 = b[2].y - b[0].y;
const double eps = 0.01;
if( fabs(dax1*day2 - day1*dax2) < eps*std::sqrt(dax1*dax1+day1*day1)*std::sqrt(dax2*dax2+day2*day2) ||
fabs(dbx1*dby2 - dby1*dbx2) < eps*std::sqrt(dbx1*dbx1+dby1*dby1)*std::sqrt(dbx2*dbx2+dby2*dby2) )
continue;
Is it a typo that it uses non-coplanar vectors? I mean the 2D points are all on the same plane right?
My second question is what is that if condition doing? I know that the left hand side (gives the area of triangle times 2) would be zero or near zero if the points are collinear, and the right hand side is the multiplication of the lengths of 2 sides of the triangle.
Collinearity is preserved in affine transformations (such as the one you are probably estimating), but this transformations also calculate also changes in rotations in point of view (as if you rotated the object in a 3d world). However, these points will be collinear as well, so for the algorithm it may have not a unique solution. Look at the pictures:
imagine selecting 3 center points of each black square in the first row in the first image. Then map it to the same centers in the next image. It may generate a mapping to that solution, but also a mapping to a zoom version of the first one. The same may happen with the third one, just that this time may map to a zoom out version of the first one (without any other change). However if the points are not collinear, for example, 3 corner squares centers, it will find a unique mapping.
I hope this helps you to clarify your doubts. If not, leave a comment
I'm following this tutorial, which uses Features2D + Homography. If I have known camera matrix for each image, how can I optimize the result? I tried some images, but it didn't work well.
//Edit
After reading some materials, I think I should rectify two image first. But the rectification is not perfect, so a vertical line on image 1 correspond a vertical band on image 2 generally. Are there any good algorithms?
I'm not sure if I understand your problem. You want to find corresponding points between the images or you want to improve the correctness of your matches by use of the camera intrinsics?
In principle, in order to use camera geometry for finding matches, you would need the fundamental or essential matrix, depending on wether you know the camera intrinsics (i.e. calibrated camera). That means, you would need an estimate for the relative rotation and translation of the camera. Then, by computing the epipolar lines corresponding to the features found in one image, you would need to search along those lines in the second image to find the best match. However, I think it would be better to simply rely on automatic feature matching. Given the fundamental/essential matrix, you could try your luck with correctMatches, which will move the correspondences such that the reprojection error is minimised.
Tips for better matches
To increase the stability and saliency of automatic matches, it usually pays to
Adjust the parameters of the feature detector
Try different detection algorithms
Perform a ratio test to filter out those keypoints which have a very similar second-best match and are therefore unstable. This is done like this:
Mat descriptors_1, descriptors_2; // obtained from feature detector
BFMatcher matcher;
vector<DMatch> matches;
matcher = BFMatcher(NORM_L2, false); // norm depends on feature detector
vector<vector<DMatch>> match_candidates;
const float ratio = 0.8; // or something
matcher.knnMatch(descriptors_1, descriptors_2, match_candidates, 2);
for (int i = 0; i < match_candidates.size(); i++)
{
if (match_candidates[i][0].distance < ratio * match_candidates[i][1].distance)
matches.push_back(match_candidates[i][0]);
}
A more involved way of filtering would be to compute the reprojection error for each keypoint in the first frame. This means to compute the corresponding epipolar line in the second image and then checking how far its supposed matching point is away from that line. Throwing away those points whose distance exceeds some threshold would remove the matches which are incompatible with the epiploar geometry (which I assume would be known). Computing the error can be done like this (I honestly do not remember where I took this code from and I may have modified it a bit, also the SO editor is buggy when code is inside lists, sorry for the bad formatting):
double computeReprojectionError(vector& imgpts1, vector& imgpts2, Mat& inlier_mask, const Mat& F)
{
double err = 0;
vector lines[2];
int npt = sum(inlier_mask)[0];
// strip outliers so validation is constrained to the correspondences
// which were used to estimate F
vector imgpts1_copy(npt),
imgpts2_copy(npt);
int c = 0;
for (int k = 0; k < inlier_mask.size().height; k++)
{
if (inlier_mask.at(0,k) == 1)
{
imgpts1_copy[c] = imgpts1[k];
imgpts2_copy[c] = imgpts2[k];
c++;
}
}
Mat imgpt[2] = { Mat(imgpts1_copy), Mat(imgpts2_copy) };
computeCorrespondEpilines(imgpt[0], 1, F, lines[0]);
computeCorrespondEpilines(imgpt1, 2, F, lines1);
for(int j = 0; j < npt; j++ )
{
// error is computed as the distance between a point u_l = (x,y) and the epipolar line of its corresponding point u_r in the second image plus the reverse, so errij = d(u_l, F^T * u_r) + d(u_r, F*u_l)
Point2f u_l = imgpts1_copy[j], // for the purpose of this function, we imagine imgpts1 to be the "left" image and imgpts2 the "right" one. Doesn't make a difference
u_r = imgpts2_copy[j];
float a2 = lines1[j][0], // epipolar line
b2 = lines1[j]1,
c2 = lines1[j][2];
float norm_factor2 = sqrt(pow(a2, 2) + pow(b2, 2));
float a1 = lines[0][j][0],
b1 = lines[0][j]1,
c1 = lines[0][j][2];
float norm_factor1 = sqrt(pow(a1, 2) + pow(b1, 2));
double errij =
fabs(u_l.x * a2 + u_l.y * b2 + c2) / norm_factor2 +
fabs(u_r.x * a1 + u_r.y * b1 + c1) / norm_factor1; // distance of (x,y) to line (a,b,c) = ax + by + c / (a^2 + b^2)
err += errij; // at this point, apply threshold and mark bad matches
}
return err / npt;
}
The point is, grab the fundamental matrix, use it to compute epilines for all the points and then compute the distance (the lines are given in a parametric form so you need to do some algebra to get the distance). This is somewhat similar in outcome to what findFundamentalMat with the RANSAC method does. It returns a mask wherein for each match there is either a 1, meaning that it was used to estimate the matrix, or a 0 if it was thrown out. But estimating the fundamental Matrix like this will probably be less accurate than using chessboards.
EDIT: Looks like oarfish beat me to it, but I'll leave this here.
The fundamental matrix (F) defines a mapping from a point in the left image to a line in the right image on which the corresponding point must lie, assuming perfect calibration. This is the epipolar line, i.e. the line though the point in the left image and the two epipoles of the stereo camera pair. For references, see these lecture notes and this chapter of the HZ book.
Given a set of point correspondences in the left and right images: (p_L, p_R), from SURF (or any other feature matcher), and given F, the constraint from epipolar geometry of the stereo pair says that p_R should lie on the epipolar line projected by p_L onto the right image, i.e.
In practice, calibration errors from noise as well as erroneous feature matches lead to a non-zero value.
However, using this idea, you can then perform outlier removal by rejecting those feature matches for which this equation is greater than a certain threshold value, i.e. reject (p_L, p_R) if and only if:
When selecting this threshold, keep in mind that it is the distance in image space of a point from an epipolar line that you are willing to tolerate, which in some sense is your epipolar error tolerance.
Degenerate case: To visually imagine what this means, let us assume that the stereo pair differ only in a pure X-translation. Then the epipolar lines are horizontal. This means that you can connect the feature matched point pairs by a line and reject those pairs whose line slope is not close to zero. The equation above is a generalization of this idea to arbitrary stereo rotation and translation, which is accounted for by the matrix F.
Your specific images: It looks like your feature matches are sparse. I suggest instead to use a dense feature matching approach so that after outlier removal, you are still left with a sufficient number of good-quality matches. I'm not sure which dense feature matcher is already implemented in OpenCV, but I suggest starting here.
Giving your pictures, your are trying to do a stereo matching.
This page will be helpfull. The rectification you want can be done using stereoCalibrate then stereoRectify.
The result (from the doc):
In order to find the Fundamental Matrix, you need correct correspondances, but in order to get good correspondances, you need a good estimate of the fundamental matrix. This might sound like an impossible chicken-and-the-egg-problem, but there is well established methods to do this; RANSAC.
It randomly selects a small set of correspondances, uses those to calculate a fundamental matrix (using the 7 or 8 point algorithm) and then tests how many of the other correspondences that comply with this matrix (using the method described by scribbleink for measuring the distance between point and epipolar line). It keeps testing new combinations of correspondances for a certain number of iterations and selects the one with the most inliers.
This is already implemented in OpenCV as cv::findFundamentalMat (http://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#findfundamentalmat). Select the method CV_FM_RANSAC to use ransac to remove bad correspondances. It will output a list of all the inlier correspondances.
The requirement for this is that all the points does not lie on the same plane.
This question is on the OpenCV functions findHomography, getPerspectiveTransform & getAffineTransform
What is the difference between findHomography and getPerspectiveTransform?. My understanding from the documentation is that getPerspectiveTransform computes the transform using 4 correspondences (which is the minimum required to compute a homography/perspective transform) where as findHomography computes the transform even if you provide more than 4 correspondencies (presumably using something like a least squares method?).
Is this correct?
(In which case the only reason OpenCV still continues to support getPerspectiveTransform should be legacy? )
My next concern is that I want to know if there is an equivalent to findHomography for computing an Affine transformation? i.e. a function which uses a least squares or an equivalent robust method to compute and affine transformation.
According to the documentation getAffineTransform takes in only 3 correspondences (which is the min required to compute an affine transform).
Best,
Q #1: Right, the findHomography tries to find the best transform between two sets of points. It uses something smarter than least squares, called RANSAC, which has the ability to reject outliers - if at least 50% + 1 of your data points are OK, RANSAC will do its best to find them, and build a reliable transform.
The getPerspectiveTransform has a lot of useful reasons to stay - it is the base for findHomography, and it is useful in many situations where you only have 4 points, and you know they are the correct ones. The findHomography is usually used with sets of points detected automatically - you can find many of them, but with low confidence. getPerspectiveTransform is good when you kn ow for sure 4 corners - like manual marking, or automatic detection of a rectangle.
Q #2 There is no equivalent for affine transforms. You can use findHomography, because affine transforms are a subset of homographies.
I concur with everything #vasile has written. I just want to add some observations:
getPerspectiveTransform() and getAffineTransform() are meant to work on 4 or 3 points (respectively), that are known to be correct correspondences. On real-life images taken with a real camera, you can never get correspondences that accurate, not with automatic nor manual marking of the corresponding points.
There are always outliers. Just look at the simple case of wanting to fit a curve through points (e.g. take a generative equation with noise y1 = f(x) = 3.12x + gauss_noise or y2 = g(x) = 0.1x^2 + 3.1x + gauss_noise): it will be much more easier to find a good quadratic function to estimate the points in both cases, than a good linear one. Quadratic might be an overkill, but in most cases will not be (after removing outliers), and if you want to fit a straight line there you better be mightily sure that is the right model, otherwise you are going to get unusable results.
That said, if you are mightily sure that affine transform is the right one, here's a suggestion:
use findHomography, that has RANSAC incorporated in to the functionality, to get rid of the outliers and get an initial estimate of the image transformation
select 3 correct matches-correspondances (that fit with the homography found), or reproject 3 points from the 1st image to the 2nd (using the homography)
use those 3 matches (that are as close to correct as you can get) in getAffineTransform()
wrap all of that in your own findAffine() if you want - and voila!
Re Q#2, estimateRigidTransform is the oversampled equivalent of getAffineTransform. I don't know if it was in OCV when this was first posted, but it's available in 2.4.
There is an easy solution for the finding the Affine transform for the system of over-determined equations.
Note that in general an Affine transform finds a solution to the over-determined system of linear equations Ax=B by using a pseudo-inverse or a similar technique, so
x = (A At )-1 At B
Moreover, this is handled in the core openCV functionality by a simple call to solve(A, B, X).
Familiarize yourself with the code of Affine transform in opencv/modules/imgproc/src/imgwarp.cpp: it really does just two things:
a. rearranges inputs to create a system Ax=B;
b. then calls solve(A, B, X);
NOTE: ignore the function comments in the openCV code - they are confusing and don’t reflect the actual ordering of the elements in the matrices. If you are solving [u, v]’= Affine * [x, y, 1] the rearrangement is:
x1 y1 1 0 0 1
0 0 0 x1 y1 1
x2 y2 1 0 0 1
A = 0 0 0 x2 y2 1
x3 y3 1 0 0 1
0 0 0 x3 y3 1
X = [Affine11, Affine12, Affine13, Affine21, Affine22, Affine23]’
u1 v1
B = u2 v2
u3 v3
All you need to do is to add more points. To make Solve(A, B, X) work on over-determined system add DECOMP_SVD parameter. To see the powerpoint slides on the topic, use this link. If you’d like to learn more about the pseudo-inverse in the context of computer vision, the best source is: ComputerVision, see chapter 15 and appendix C.
If you are still unsure how to add more points see my code below:
// extension for n points;
cv::Mat getAffineTransformOverdetermined( const Point2f src[], const Point2f dst[], int n )
{
Mat M(2, 3, CV_64F), X(6, 1, CV_64F, M.data); // output
double* a = (double*)malloc(12*n*sizeof(double));
double* b = (double*)malloc(2*n*sizeof(double));
Mat A(2*n, 6, CV_64F, a), B(2*n, 1, CV_64F, b); // input
for( int i = 0; i < n; i++ )
{
int j = i*12; // 2 equations (in x, y) with 6 members: skip 12 elements
int k = i*12+6; // second equation: skip extra 6 elements
a[j] = a[k+3] = src[i].x;
a[j+1] = a[k+4] = src[i].y;
a[j+2] = a[k+5] = 1;
a[j+3] = a[j+4] = a[j+5] = 0;
a[k] = a[k+1] = a[k+2] = 0;
b[i*2] = dst[i].x;
b[i*2+1] = dst[i].y;
}
solve( A, B, X, DECOMP_SVD );
delete a;
delete b;
return M;
}
// call original transform
vector<Point2f> src(3);
vector<Point2f> dst(3);
src[0] = Point2f(0.0, 0.0);src[1] = Point2f(1.0, 0.0);src[2] = Point2f(0.0, 1.0);
dst[0] = Point2f(0.0, 0.0);dst[1] = Point2f(1.0, 0.0);dst[2] = Point2f(0.0, 1.0);
Mat M = getAffineTransform(Mat(src), Mat(dst));
cout<<M<<endl;
// call new transform
src.resize(4); src[3] = Point2f(22, 2);
dst.resize(4); dst[3] = Point2f(22, 2);
Mat M2 = getAffineTransformOverdetermined(src.data(), dst.data(), src.size());
cout<<M2<<endl;
getAffineTransform:affine transform is combination of translation, scale, shear, and rotation
https://www.mathworks.com/discovery/affine-transformation.html
https://www.tutorialspoint.com/computer_graphics/2d_transformation.htm
getPerspectiveTransform:perspective transform is project mapping
enter image description here
I have two isosurfaces (skull and skin). given point A on skull isosurface, i calculated the normal at point A using "double *pos = pickerCell->GetPickNormal()".
when i print pos, this is what i got: -6.2367, 1.98263, -0.9823
could someone explain to me what these 3 values mean?
I would like to find the intersection point of this normal of point A with the skin isosurface.
Could I use IntersectWithLine() function to do so? If yes, the line in my case would then be the normal? what is the start and end point of the normal?
Or is there a better way of doing ?
As you've found, you need to define the line to intersect with as two points. What is commonly done is to start at the point, P, that you picked (the same point where the normal, v, was computed) and compute two points, A = P + v delta and B = P - v delta where you have to set delta using context (if your model is in a unit cube, delta might be something like .01, where if your model has units of size 1000, delta may be 1, etc.).
Also, I'm not sure why the normal the cell picker returns is not normalized, but I'm assuming that if you normalize it it is the surface normal. I'd call it something other than 'pos' to avoid confusion (as it is a direction, not a position).