I am trying to image coordinates to 3D coordinates. Using the solvePnP function (in C++)has given me 3X1 rotation matrix and 3X1 translation matrix. But isn't the [R|t] matrix supposed to be 3X4?
Any help will be greatly appreciated!
From the OpenCV documentation for solvePnP:
"rvec – Output rotation vector (see Rodrigues() ) that, together with tvec , brings points from the model coordinate system to the camera coordinate system."
Following the link to Rodrigues():
src – Input rotation vector (3x1 or 1x3) or rotation matrix (3x3).
dst – Output rotation matrix (3x3) or rotation vector (3x1 or 1x3), respectively.
Related
I'm working on an AR program. I have done this steps:
Detect corners of a chess board with OpenCV
Use SolvePNP to find rvec and tvec
Apply Rodrigues on rvec to get R_mat
Use hconcat(R_mat, tvec, P); to concatenate R_mat and tvec to get Projection matrix
Apply decomposeProjectionMatrix on P to get new translation,T2, vector and eulerAngles
now my problem is in translation vectors just tvec[2] or translation over Z-axis is correct
and in rotations again in Euler Angles (from decomposeProjectionMatrix) just eulerAngles[2] or rotation around z-axis is correct.
I don't know hoa can I get Translation and Rotation related to X-axis and Y-axis correctly.
I'm gonna use these in OpenGL to Augment a cube on the pattern.
In my code, result of translation over X and Y axis is very larger than windows size; and rotaion around X and Y axis are very small and always near the zero (less than 0.0001)
Any Idea how to get correct meaningful T and R?
P.S : I'm using an Identity matrix as camera matrix and zero matrix for dist coeff matrix.
I am reading the following documentation: http://docs.opencv.org/doc/tutorials/calib3d/camera_calibration/camera_calibration.html
I have managed to successfully calibrate the camera obtaining the camera matrix and the distortion matrix.
I had two sub-questions:
1) How do I use the distortion matrix as I don't know 'r'?
2) For all the views I have the rotation and translation vectors which transform the object points (given in the model coordinate space) to the image points (given in the world coordinate space). So a total of 6 coordinates per image(3 rotational, 3 translational). How do I make use of this information to obtain the rotational-translational matrix?
Any help would be appreciated. Thanks!
Answers in order:
1) "r" is the pixel's radius with respect to the distortion center. That is:
r = sqrt((x - x_c)^2 + (y - y_c)^2)
where (x_c, y_c) is the center of the nonlinear distortion (i.e. the point in the image that has zero nonlinear distortion. This is usually (and approximately) identified with the principal point, i.e. the intersection of the camera focal axis with the image plane. The coordinates of the principal point are in the 3rd column of the matrix of the camera intrinsic paramers.
2) Use Rodrigues's formula to convert between rotation vectors and rotation matrices.
I have some 2D points set (X,Y) corresponding to a 3D points set (X,Y,Z).
2D points were captured from camera and 3D points were the real coordinate according to world base. I want to find the transformation matrix between them, that is to say, how to convert other 2D points to 3D points.
I have try getPespectiveTransform function, but it didnt work in this problem.
How can I write a regression to find this transform matrix ?
You can use solvePnP of OpenCV, it gives you rotation and translation matrix. In this answer you can see in more detail:
Camera position in world coordinate from cv::solvePnP
I am trying to develop an augmented reality program that overlays a 3d object on top of a marker. The model does not move along(proportionately) with the marker. Here are the list of things that I did
1) Using opencv: a) I used the solvepnp method to find rvecs and tvecs. b) I also used the rodrigues method to find the rotation matrix and appended the tvecs vector to get the projection matrix. c) Just for testing I made some points and lines and projected them to make a cube. This works perfectly fine and I am getting a good output.
2) Using irrlicht: a) I tried to place a 3d model(at position(0,0,0) and rotation(0,0,0)) with the camera feed running in the background. b) Using the rotation matrix found using rodrigues in opencv I calculated the pitch, yaw and roll values from this post("http://planning.cs.uiuc.edu/node103.html") and passed the value onto the rotation field. In the position field I passed the tvecs values. The tvecs values are tvecs[0], -tvecs[1], tvecs[2].
The model is moving in the correct directions but it is not moving proportionately. Meaning, if I move the marker 100 pixels in the x direction, the model only moves 20 pixels(the values 100 and 20 are not measured, I just took arbitrary values illustrate the example). Similarly for y axis and z axis. I do know I have to introduce another transformation matrix that maps the opencv camera coordinates to irrlicht camera coordinates and its a 4x4 matrix. But I do not know how to find it. Also the opencv's projections matrix [R|t] is a 3x4 matrix and it yields a 2d point that is to be projected. The 4x4 matrix mapping between opencv and irrlicht requires a 3d point(made homogeneous) to be fed into a 4x4 matrix. How do I achieve that?
The 4x4 matrix You are writing about seems to be M=[ R|t; 0 1]. t is 3x1 translation vector. To get the transformed coordinates v' of 4x1 ([x y z 1]^T) point v just do v'=Mt.
Your problem with scaling may be also caused by difference in units used for camera calibration in OpenCV and those used by the other library.
I have calibrated my camera with OpenCV (findChessboard etc) so I have:
- Camera Distortion Coefficients & Intrinsics matrix
- Camera Pose information (Translation & Rotation, computed separatedly via other means) as Euler Angles & a 4x4
- 2D points within the camera frame
How can I convert these 2D points into 3D unit vectors pointing out into the world? I tried using cv::undistortPoints but that doesn't seem to do it (only returns 2D remapped points), and I'm not exactly sure what method of matrix math to use to model the camera via the Camera intrinsics I have.
Convert your 2d point into a homogenous point (give it a third coordinate equal to 1) and then multiply by the inverse of your camera intrinsics matrix. For example
cv::Matx31f hom_pt(point_in_image.x, point_in_image.y, 1);
hom_pt = camera_intrinsics_mat.inv()*hom_pt; //put in world coordinates
cv::Point3f origin(0,0,0);
cv::Point3f direction(hom_pt(0),hom_pt(1),hom_pt(2));
//To get a unit vector, direction just needs to be normalized
direction *= 1/cv::norm(direction);
origin and direction now define the ray in world space corresponding to that image point. Note that here the origin is centered on the camera, you can use your camera pose to transform to a different origin. Distortion coefficients map from your actual camera to the pinhole camera model and should be used at the very beginning to find your actual 2d coordinate. The steps then are
Undistort 2d coordinate with distortion coefficients
Convert to ray (as shown above)
Move that ray to whatever coordinate system you like.