I am trying to compute the coordinates correspondence of several points between two images.
I have a group of points whose correspondences are known, I use them with OpenCV's findFundamentalMatrix() in order to find the fundamental matrix.
I verified that x^T * F * x' = (0) for each point, and the result is always right or very close.
The thing is, now I'd like to use the coordinates of a point on the first image (y) and the fundamental matrix (F) in order to find the coordinates of the point on the second image (y'). I first thought about simply using the equation above, but given only the z of the y' point, there can be an infinity of solutions.
How else can I use the fundamental matrix to compute the translations ?
To be more clear: knowing the fundamental matrix "linking" two projections, how can I use it to translate the coordinates of any known point (a, b, 1) from the first projection to the second projection?
Considering that we know a, b and F in this equation: (a', b", 1)T * F * (a, b, 1) = (0)
I had made a simple drawing as an example: http://i.imgur.com/drNr2.jpg . The idea is to find the coordinates of the red dot (xq, yq) in projection 2, considering that we know its coordinates in projection 1 and the ones of all other points in both projections (and some other ones as the algorithm to find the fundamental matrix actually requires at least 8 points)
Another precision: in my example, known points are coplanar, but the researched point will not necessarily be.
I hope that made my problem more clear :)
The fundamental matrix transforms points from one image to lines in the other. Could you elaborate more on
How else can I use the fundamental matrix to compute the translations?
please. Telling us what you want to achieve perhaps with an example would help too.
Edit: If you have calibrated the camera you can compute the essential matrix, E, from the fundamental matrix, F. E transforms a point in one image to a point in the other. But of course, the requirement is to have the internal matrix. If K is the internal matrix E=transpose(K)FK.
The other method is to find the corresponding line for a point in the other image and then search along this line for the patch most similar in appearance to the patch surrounding the point in the first image. There are some other ways too but really need more information about the problem to tell which suits your case.
Edit 2: in the drawing you have got the points are coplanar. Hence, a homography maps the point positions between the two images, and there is no need to find the fundamental matrix. OpenCV has a function for estimating homographies, which needs only four points.
Given:
Point correspondences a in image 1.
Goal:
Finding corresponding points b laying on the so called epipolar line L in image 2.
How?
| x0 | | x1 |
a = | y0 | , b = | y1 |
| 1 | | 1 |
L = F * a
|F00 F01 F02|
F = |F10 F11 F12|
|F20 F21 F22|
The following equation must be fulfilled to obtain b in image 2:
a' * F * b = 0.
Note: a' = transpose(a).
For some reason I could not add a comment due to a lack of reputation.
I have been studying this field for about a month now and hopefully I can answer the many questions left unanswered that have also puzzled me when I was studying the topic.
#M2X
A fundamental matrix is a mapping from a point in image plane 1 to a line in image plane 2. The lines are a special type of lines called epipolar lines and are formed by the intersection of the image plane and the plane constructed from the origin of the 2 cameras and the 3D point. So it is not possible to determine a point-point mapping using the Fundamental matrix unless you have some additional information or constraints.
#Jukurrpa
A homography is a point to point mapping such that parallel lines map to parallel lines. One can prove that this mapping is linear, then since linear maps a equivalent to matrices, the homography can be defined by a matrix.
A set of 3D points lying on a plane projected to the image plane maps parallel lines to parallel lines, so a homography will work in your case. Methods of estimating homograph from a given set of points is outlined in the book (multiple view geometry in computer vision). Given corresponding points in both images you can find homography by using iterative approaches (Gradient Descent) or closed form solutions (Singular Value Decomposition).
Related
As a relative beginner in this topic, I have read the literature but I am not sure about how to manipulate the equations to my purposes and would like advice on tackling this topic.
Preamble:
I have 2 cameras in a stereo rig which have been calibrated, thus extracting data structures such as each camera's Camera Matrix K1 and K2, as well as the Fundamental, Essential, Rotation and Translation matrices, F, E, R and T respectively. Also after rectifying one has the projection matrices P1 and P2 as well as the disparity matrix Q.
My aim is however to test the triangulation method of OpenCV, and to this end I would like to use synthetic Images where the correspondence between the points in image1 and image2 is exact.
My idea was to take an image of a chessboard with one camera, and use findCorners() and cornerSubPix() to get image points in the left camera, let's call them imagePoints1.
To get synthetically generated Image Points with exactly corresponding points on the left camera's image plane, I intend to use the property
x2'Fx1 = 0, given the F matrix and x1 (which represents one homogenous 2D point from imagePoints1)
to generate said set of Image Points.
This is where I am stuck, since he obvious solution would be to have a zero-vector to make this equation work. Otherwise I get a parametric solution. How do I then get non-zero points that fulfill this property x2'Fx1 = 0 given x1 and F ?
Thank you.
The closest thing I've found to help explain what I need is here in this question: Draw equidistant points on a spiral
However, that's not exactly what I want.
The spiral to draw is an archimedean spiral and the points obtained must be equidistant from each other. (Quote: From the question linked above.)
This is precisely what I want given the Archimedean Spiral equation, .
There is a specific set of data a user can input, they are NOT based on spirals but circular figures in general. They are as follows: center point [X,Y,Z], radius, horizontal separation [can be called X separation, depends on figure], and vertical separation [can be called Y separation, depends on figure], and most importantly degrees of rotation. I'd like the horizontal separation to be the distance between consecutive points since they are the ones that need to be the same distance between each other. I'd also like vertical separation to be the distance between the 'parallel' curves.
So given that specific input selection (and yes, some can be ignored), how can I iterate through all of the consecutive, equidistant points it would take to reach the input degrees (which can be very large but is finite) and return the X and Y point of each point of those points?
Basically what I'm try to achieve is a loop from zero to the number of degrees in the input, given all of the rest of the input and my preferences noted above, and drawing a point for all of the equidistant, consecutive points (if you decide to represent using code, just represent the drawing using a 'print').
I'm having a hard time explaining, but I think I got it pretty much covered. The points on this graph are exactly what I need:
Assuming a 2D case and an archimedean spiral centered around zero (a=0), so with equation . Successive lines are then apart, so to obtain a 'vertical spacing' of , set .
The length of the arc from the centre to a point at given angle is given by Wolfram, but his solution is difficult to working with. Instead, we can approximate the length of the arc (using a very rough for-large-theta approximation) to . Rearranging, , allowing us to determine what angles correspond to the desired 'horizontal spacing'. If this approximation is not good enough, I would look at using something like Newton-Raphson. The question you link to uses also uses an approximation, although not the same one.
Finally, recognising that polar coordinates translate to cartesian as follows: ; .
I get the following:
This is generated by the following MATLAB code, but it should be straight-forward enough to translate to C++ if this is what you actually need.
% Entered by user
vertspacing = 1;
horzspacing = 1;
thetamax = 10*pi;
% Calculation of (x,y) - underlying archimedean spiral.
b = vertspacing/2/pi;
theta = 0:0.01:thetamax;
x = b*theta.*cos(theta);
y = b*theta.*sin(theta);
% Calculation of equidistant (xi,yi) points on spiral.
smax = 0.5*b*thetamax.*thetamax;
s = 0:horzspacing:smax;
thetai = sqrt(2*s/b);
xi = b*thetai.*cos(thetai);
yi = b*thetai.*sin(thetai);
I have already done the comparison of 2 images of same scene which are taken by one camera with different view angles(say left and right) using SURF in emgucv (C#). And it gave me a 3x3 homography matrix for 2D transformation. But now I want to make those 2 images in 3D environment (using DirectX). To do that I need to calculate relative location and orientation of 2nd image(right) to the 1st image(left) in 3D form. How can I calculate Rotation and Translate matrices for 2nd image?
I need also z value for 2nd image.
I read something called 'Homograhy decomposition'. Is it the way?
Is there anybody who familiar with homography decomposition and is there any algorithm which it implement?
Thanks in advance for any help.
Homography only works for planar scenes (ie: all of your points are coplanar). If that is the case then the homography is a projective transformation and it can be decomposed into its components.
But if your scene isn't coplanar (which I think is the case from your description) then it's going to take a bit more work. Instead of a homography you need to calculate the fundamental matrix (which emgucv will do for you). The fundamental matrix is a combination of the camera intrinsic matrix (K), the relative rotation (R) and translation (t) between the two views. Recovering the rotation and translation is pretty straight forward if you know K. It looks like emgucv has methods for camera calibration. I am not familiar with their particular method but these generally involve taking several images of a scene with know geometry.
To figure out camera motion (exact rotation and translation up to a scaling factor) you need
Calculate fundamental matrix F, for example, using eight-point
algorithm
Calculate Essential matrix E = A’FA, where A is intrinsic camera matrix
Decompose E which is by definition Tx * R via SVD into E=ULV’
Create a special 3x3 matrix
0 -1 0
W = 1 0 0
0 0 1
that helps to run decomposition:
R = UW-1VT, Tx = ULWUT, where
0 -tx ty
Tx = tz 0 -tx
-ty tx 0
Since E can have an arbitrary sign and W can be replace by Winv we have 4 distinct solution and have to select the one which produces most points in front of the camera.
It's been a while since you asked this question. By now, there are some good references on this problem.
One of them is "invitation to 3D image" by Ma, chapter 5 of it is free here http://vision.ucla.edu//MASKS/chapters.html
Also, Vision Toolbox of Peter Corke includes the tools to perform this. However, he does not explain much math of the decomposition
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Finding whether a point lies inside a rectangle or not
There is an interview question that is, "How to determine whether a point lies inside a rectangle"
Note that the rectangle could be rotated as well. So the simple solution of checking point inside the rectangle doesn't stands valid here...
Please share your thoughts on this question..
I found a link on internet, and was trying to understand it, but failed.... Please if any body out here can give complete solution with bit of computer graphics logic, because i have forgotten all the basics....
How to determine if a point is inside rectangle.
Pick a point that's definitely outside the rectangle. Then create a segment from that point to the point in question. Solve the linear equations for intersections between that segment and the segments that make up the rectangle. If you get exactly one intersection, the point is inside the rectangle. Otherwise (0 or 2 intersections), it's outside.
This is trivial to extend to essentially any polygon -- an odd number of intersections means the point is inside the polygon, and an even number means it's outside.
Edit: It may not be immediately obvious, so I'll emphasize that the point we pick outside the rectangle (polygon) is entirely arbitrary. We can pick whatever point we want as long as we're sure it's outside the polygon. To keep our computations easy, what we'll typically do is pick (Px, infinity) (where Px is the x coordinate of the point P that we're testing) -- that is, what we're creating is essentially a vertical ray. That simplifies testing a bit, because we only have to test against one end-point to find an intersection. It also simplifies solving the linear equations to the point that it's barely recognizable as solving linear equations anymore. We really just need to compute the Y coordinate for the line at the Px, and see if it's greater than Py. As such, solving the linear equation breaks down to:
checking whether that X value is within the range of X values for the segment
if it is, plugging the X value into the equation of the line
testing whether the resulting Y value is greater than Py
If those pass, we have an intersection. Also note that the tests can be carried out in parallel (handy if we're doing this on parallel hardware like a GPU).
Simple solution that works in N dimensions for convex polyhedra, of which a 2-dimensional rectangle is a special case:
Represent the polyhedron as the intersection of half-spaces, each defined by a unit normal vector and the distance of the surface hyperplane from the origin along the normal.
For each of these half-spaces, take the dot product of point in question with the defining normal vector. The point is in the half-space if and only if the dot product is less than [or equal to] the defining distance.
The point is inside the polyhedron if and only if it's in every one of the half-spaces.
For a rectangle defined as a counter-clockwise sequence of edges, step 1 amounts to rotating the edges each by 90 degrees clockwise to get the normals, then intersecting the normal line with the line containing the edge to find the distance to the origin.
Assuming step 1 is complete, testing a point takes at most 8 multiplications, 4 additions, and 4 comparisons.
If you want to, you can optimize the process a bit since you have rectangles (and thus opposite sides have opposite normals). Now you're only looking at 2 normals rather than 4, and a range of dot product values which indicate points that lie between the opposite sides. So now you're down to 4 multiplications, 2 additions, and 4 comparisons.
You can also get lucky if the first test you make shows that the point is outside the rectangle, in which case it's just 2 multiplications, 1 addition, and 1-2 comparisons.
This is far from the best solution... But if you have the points in consecutive order, call them a, b, c, and d with an x and a y field, you can use the cross product of the vectors between your point p and each of the consecutive pairs.
If you always get the same sign for the result (i.e., all are positive or all are negative) then you're inside the rectangle; otherwise, you're outside.
Define a new coordinate system with two rectangle sides as unit vectors and transform the coordinate of the point into the new coordinate system. If both coordinates are between 0 and 1, it's inside.
In equations (assuming A,B,C,D are corners of the rectangle, P is the point, _x and _y are the x and y components):
P_x = A_x + x * (B_x - A_x) + y * (D_x - A_x)
P_y = A_y + x * (B_y - A_y) + y * (D_y - A_y)
Solve for x and y and check if they are between 0 and 1
Written as linear equation system (A,B,C,D,P are vectors of length 2):
[ | ] [x] [ ]
[B-A | D-A] * [ ] = [P-A]
[ | ] [y] [ ]
Solving is easy as it has only two dimensions and you can be sure that you are not singular.
You can rotate and move your reference system so it matches position and rotation of the rectangle. Now it is just a matter of simple comparisons between coordinates. This is more a mathematical way, so not the fastest (bellieve #Platinum Azure's one is)
Since the rectangle could be rotated, you might want to consider an algorithm that is used to determine whether a point is interior to a convex polygon.
You could also compute the rotation angle of the rectangle, then transform both the rectangle and the point to axially align the rectangle. Then check to see if the transformed point is inside the axially aligned rectangle.
Finding whether a point lies within a bounded region like rectangle is part of the classic clipping algorithms. Refer to the wikipedia articles on Clipping and Line Clipping to know more about it.
Following the spirit of #Jerry Coffin: create segments from rectangle corners to the point in question. Solve the linear equations. Slope is tan(a). Sum up all seq arctangents diff, if it is 2*PI and each diff < PI - point is inside the rectangle.
Edit Probably enough just check for each sequential difference < Pi...
I am trying to solve the current problem using GPU capabilities: "given a point cloud P and an oriented plane described by a point and a normal (Pp, Np) return the points in the cloud which lye at a distance equal or less than EPSILON from the plane".
Talking with a colleague of mine I converged toward the following solution:
1) prepare a vertex buffer of the points with an attached texture coordinate such that every point has a different vertex coordinate
2) set projection status to orthogonal
3) rotate the mesh such that the normal of the plane is aligned with the -z axis and offset it such that x,y,z=0 corresponds to Pp
4) set the z-clipping plane such that z:[-EPSILON;+EPSILON]
5) render to a texture
6) retrieve the texture from the graphic card
7) read the texture from the graphic card and see what points were rendered (in terms of their indexes), which are the points within the desired distance range.
Now the problems are the following:
q1) Do I need to open a window-frame to be able to do such operation? I am working within MATLAB and calling MEX-C++. By experience I know that as soon as you open a new frame the whole suit crashes miserably!
q2) what's the primitive to give a GLPoint a texture coordinate?
q3) I am not too clear how the render to a texture would be implemented? any reference, tutorial would be awesome...
q4) How would you retrieve this texture from the card? again, any reference, tutorial would be awesome...
I am on a tight schedule, thus, it would be nice if you could point me out the names of the techniques I should learn about, rather to the GLSL specification document and the OpenGL API as somebody has done. Those are a tiny bit too vague answers to my question.
Thanks a lot for any comment.
p.s.
Also notice that I would rather not use any resource like CUDA if possible, thus, getting something which uses
as much OpenGL elements as possible without requiring me to write a new shader.
Note: cross posted at
http://www.opengl.org/discussion_boards/ubbthreads.php?ubb=showflat&Number=245911#Post245911
It's simple:
Let n be the normal of the plane and x be the point.
n_u = n/norm(n) //this is a normal vector of unit length
d = scalarprod(n,x) //this is the distance of the plane to the origin
for each point p_i
d_i = abs(scalarprod(p_i,n) - d) //this is the distance of the point to the plane
Obviously "scalarprod" means "scalar product" and "abs" means "absolute value".
If you wonder why just read the article on scalar products at wikipedia.
Ok first as a little disclaimer: I know nothing about 3D programming.
Now my purely mathematical idea:
Given a plane by a normal N (of unit length) and a distance L of the plane to the center (the point [0/0/0]). The distance of a point X to the plane is given by the scalar product of N and X minus L the distance to the center. Hence you only have to check wether
|n . x - L| <= epsilon
. being the scalar product and | | the absolute value
Of course you have to intersect the plane with the normal first to get the distance L.
Maybe this helps.
I have one question for Andrea Tagliasacchi, Why?
Only if you are looking at 1000s of points and possible 100s of planes, would there would be any benefit from using the method outlined. As apposed to dot producting the point and plane, as outlined my Corporal Touchy.
Also due to the finite nature of pixels you'll often find two or more points will project to the same pixel in the texture.
If you still want to do this, I could work up a sample glut program in C++, but how this would help with MATLAB I don't know, as I'm unfamiliar with it.
IT seems to me you should be able to implement something similar to Corporal Touchy's method a a vertex program rather than in a for loop, right? Maybe use a C API to GPU programming, such as CUDA?