I have taken a photo A using an RGB camera. And I know the position of a point g in photo A. The camera needs to do a camera calibration. Now I want to know the position of point g after calibration. I am using the code as following, but I want to get the point position, not image. How can I do that? Can you give me some advice?
initUndistortRectifyMap(
cameraMatrix,
distCoeffs,
Mat(),
Mat(),
Size(640, 480),
CV_32FC1,
map1, map2);
remap(A, B, map1, map2, cv::INTER_LINEAR);
Point2f g = Point2f(...,...);//i want to get the new position of the point not image B
Just get coordinates using maps:
x,y - coordinates after (not before),as pasbi correctly noticed in comments, mapping.
(map1(y,x),map2(y,x)) - coordinates before mapping
In other words:
map1.at<float>(y,x) contains source x coordinates for each destination point
p(x,y).
map2.at<float>(y,x) contains source y coordinates for each destination point
p(x,y).
See documentation on remap function.
best method i found was to recreate a camera matrix, with inverted parameters. work to a certain extent, with like basic images modifications
undistortPoints() is your need。
// src_pts are points in raw(distort) img, rectify_pt_vec are in rectifyImageL
// RL, PL are from stereoRectify()
cv::undistortPoints(src_pts, rectify_pt_vec, cameraMatrixL, distCoeffL, RL, PL);
how to get point in srcimg from dstimg just like pasbi commented below.
Related
I am trying to perform 3D Reconstruction(Structure From Motion) from Multiple Images of Planar Markers. I very new to MVG and openCV.
As far I have understood I have to do the following steps:
Identify corresponding 2D corner points in the one images.
Calculate the Camera Pose of the first image us cv::solvePNP(assuming the
origin to be center of the marker).
Repeat 1 and 2 for the second image.
Estimate the relative motion of the camera by Rot_relative = R2 - R1,
Trans_relative = T2-T1.
Now assume the first camera to be the origin construct the 3x4 Projection
Matrix for both views, P1 =[I|0]*CameraMatrix(known by Calibration) and P2 =
[Rot_relative |Trans_relative ].
Use the created projection matrices and 2D corner points to triangulate the
3D coordinate using cv::triangulatePoints(P1,P2,point1,point2,OutMat)
The 3D coordinate can be found by dividing the each rows of OutMat by the 4th
row.
I was hoping to keep my "First View" as my origin and iterate
through n views repeating steps from 1-7(I suppose its called Global SFM).
I was hoping to get (n-1)3D points of the corners with "The first View as origin" which we could optimize using Bundle Adjustment.
But the result I get is very disappointing the 3D points calculated are displaced by a huge factor.
These are questions:
1.Is there something wrong with the steps I followed?
2.Should I use cv::findHomography() and cv::decomposeHomographyMat() to find the
relative motion of the camera?
3.Should point1 and point2 in cv::triangulatePoints(P1,P2,point1,point2,OutMat)
be normalized and undistorted? If yes, how should the "Outmat" be interpreted?
Please anyone who has insights towards the topic, can you point out my mistake?
P.S. I have come to above understanding after reading "MultiView Geometry in Computer Vision"
Please find the code snippet below:
cv::Mat Reconstruction::Triangulate(std::vector<cv::Point2f>
ImagePointsFirstView, std::vector<cv::Point2f>ImagePointsSecondView)
{
cv::Mat rVectFirstView, tVecFristView;
cv::Mat rVectSecondView, tVecSecondView;
cv::Mat RotMatFirstView = cv::Mat(3, 3, CV_64F);
cv::Mat RotMatSecondView = cv::Mat(3, 3, CV_64F);
cv::solvePnP(RealWorldPoints, ImagePointsFirstView, cameraMatrix, distortionMatrix, rVectFirstView, tVecFristView);
cv::solvePnP(RealWorldPoints, ImagePointsSecondView, cameraMatrix, distortionMatrix, rVectSecondView, tVecSecondView);
cv::Rodrigues(rVectFirstView, RotMatFirstView);
cv::Rodrigues(rVectSecondView, RotMatSecondView);
cv::Mat RelativeRot = RotMatFirstView-RotMatSecondView ;
cv::Mat RelativeTrans = tVecFristView-tVecSecondView ;
cv::Mat RelativePose;
cv::hconcat(RelativeRot, RelativeTrans, RelativePose);
cv::Mat ProjectionMatrix_0 = cameraMatrix*cv::Mat::eye(3, 4, CV_64F);
cv::Mat ProjectionMatrix_1 = cameraMatrix* RelativePose;
cv::Mat X;
cv::undistortPoints(ImagePointsFirstView, ImagePointsFirstView, cameraMatrix, distortionMatrix, cameraMatrix);
cv::undistortPoints(ImagePointsSecondView, ImagePointsSecondView, cameraMatrix, distortionMatrix, cameraMatrix);
cv::triangulatePoints(ProjectionMatrix_0, ProjectionMatrix_1, ImagePointsFirstView, ImagePointsSecondView, X);
X.row(0) = X.row(0) / X.row(3);
X.row(1) = X.row(1) / X.row(3);
X.row(2) = X.row(2) / X.row(3);
return X;
}
Can anyone give me more explanation about the opencv function solvepnp()?
The opencv documentation says
bool cv::solvePnP (
InputArray objectPoints,
InputArray imagePoints,
InputArray cameraMatrix,
InputArray distCoeffs,
OutputArray rvec,
OutputArray tvec,
bool useExtrinsicGuess = false,
int flags = SOLVEPNP_ITERATIVE)
I'm wondering what the objectPoints, imagePoints and cameraMatrix are. I have once calibrated my camera and have a parameter xml file from it, can i use this?
It is used when you have for example a 3D model of an object and you have a view of it in the real world, it will give you an approximate position and orientation of the camera towards the object.
For example:
objectPoints – Array of object points in the object coordinate space, 3xN/Nx3 1-channel or 1xN/Nx1 3-channel, where N is the number of points. vector can be also passed here.
imagePoints – Array of corresponding image points, 2xN/Nx2 1-channel or 1xN/Nx1 2-channel, where N is the number of points. vector can be also passed here.
You can find the rest at this link
I have a fish-eye camera in the ceiling and I want to locate some points on the floor. I have put the origin of my reference system (real world) just below the camera and I want to know the position of every object in centimeters. This picture shows this:
Reference system - Real world
Firstly, I have done the camera calibration and I have obtained the next result with an RMS of 1.11:
Undistorted image after calibration
As a result of the calibration I obtained intrinsic parameters (camera matrix), so I used cv::solvePnP to get rotation and translation vectors. For apply this I marked some points in the undistorted image (in pixels) and I measured them in real world according to my reference system.
For example, the origin is in the center of a 1024x768 image, so:
Point 0: ImagePoint(512, 384) [pixels] --> ObjectPoint(0,0) [centimeters]
The next code shows this:
std::vector<cv::Point2f> imagePointsPix;
std::vector<cv::Point3f> objectPointsCm;
imagePointsPix.push_back(cv::Point2f(512.,384.));
imagePointsPix.push_back(cv::Point2f(404.,512.));
imagePointsPix.push_back(cv::Point2f(666.,211.));
imagePointsPix.push_back(cv::Point2f(519.,66.));
objectPointsCm.push_back(cv::Point3f(0., 0., 0.));
objectPointsCm.push_back(cv::Point3f(-80.,-132.,0.));
objectPointsCm.push_back(cv::Point3f(120.,188.,0.));
objectPointsCm.push_back(cv::Point3f(-40.,268.,0.));
cv::Mat rvec(1,3,cv::DataType<double>::type);
cv::Mat tvec(1,3,cv::DataType<double>::type);
cv::Mat rotationMatrix(3,3,cv::DataType<double>::type);
cv::solvePnP(objectPointsCm, imagePointsPix, cameraMatrix, distCoeffs, rvec, tvec, 0, SOLVEPNP_ITERATIVE);
cv::Rodrigues(rvec,rotationMatrix);
Now I have the camera matrix, the rotation matrix and the traslation vector, so by using this as reference I am able to compute any point if I have its position in pixels. This is the code:
cv::Mat uvPoint = cv::Mat::ones(3,1,cv::DataType<double>::type); //u,v,1
uvPoint.at<double>(0,0) = 512.; //img point for which we want its real coordinates
uvPoint.at<double>(1,0) = 384.;
cv::Mat tempMat, tempMat2;
double s;
tempMat = rotationMatrix.inv() * cameraMatrix.inv() * uvPoint;
tempMat2 = rotationMatrix.inv() * tvec;
s = 0 + tempMat2.at<double>(2,0); //before 0 it was 285, which represents the height Zconst
s /= tempMat.at<double>(2,0);
std::cout << "P = " << rotationMatrix.inv() * (s * cameraMatrix.inv() * uvPoint - tvec) << std::endl;
I get this results for the same points I used for obtaining my parameters:
Point 0 --> (0.213, 3.391) (it should be (0,0)) ERROR: 3.69 cm
Point 1 --> (-68.28, -112.82) (it should be (-80, -132)) ERROR: 17.49 cm
Point 2 --> (84.48, 137.61) (it should be (120, 188)) ERROR: 49.62 cm
The rest of points also show an error too big... I have used more points but the results do not improve. I don't know where I went wrong, could anyone help me?
Thanks in advance.
It looks like you may be effectively undistorting your image twice from solvePNP's perspective. This is due to passing in the distortion coefficients along with point correspondences that are already derived from an undistorted image.
Try passing the actual camera matrix from your calibration to solvePNP instead of an identity matrix, but still pass NULL for the distortion coefficients to avoid the double-undistortion.
Finally I have found out that the error was caused by the distortion coefficients, i.e. my calibration. I set the cameraMatrix to the Identity matrix (eye(3)) and the distCoefficients to NULL so that solvePNP assumed I have a perfect camera. Using this approach I obtained an error much lower. I will have to make a better calibration.
I have a program that detects objects in a live video stream. I am looking to compensate for the distortion in the camera, I have used the OpenCV calibration tool and produced an XML file with the relevant parameters.
However I am unsure how to then apply this using the undistort function, it is my understanding that this will need to be applied to each frame as it is captured?
void undistort(InputArray src, OutputArray dst, InputArray cameraMatrix, InputArray distCoeffs, InputArray newCameraMatrix=noArray() )
I am having trouble identifying each of these parameters, below is my current understanding.
undistorted(currentFrame, resultsWindow, calibrationFile, notSure, notSure);
Is this function called as below:
if(captureOpen == false){
img_scene = cvCaptureFromFile(videoFeed);
}
while(1) {
image = cvQueryFrame(img_scene);
undistort();
undistorted(currentFrame, resultsWindow, calibrationFile, notSure,
notSure);
No, that will not work. You need to manually read your XML file beforehand and fill the corresponding parameters with the data found in the file. The file should contain the camera matrix (look for cx, cy, fx, fy values) and the distortion parameters (k1, k2, k3, p1, p2, etc.).
The documentation for undistort for 2.4.x is here : http://docs.opencv.org/2.4/modules/imgproc/doc/geometric_transformations.html#undistort
Typically src is a Mat containing the current frame. dst an output Mat of the same size that will be filled with the undistorted image. You will have to convert that back to your preferred format or display it in the window. cameraMatrix is a 3x3 Mat that you have filled with your camera intrinsics. distCoeffs is usually a 1x4 or 1x5 Mat containing the distorsion coeffs. (Note that p1 and p2 must be written right after k2).
I'm new to openCV using version 2.4.9
I am trying to generate a 3D projection of points from a sequence of images without any knowledge of the camera parameters nor have camera used with me to calibrate. The camera used had a fish eye lens.
I used goodFeaturesToTrack() for detecting feature points followed by LK implementation in openCV to track the feature points in the sequence of images. Using these points I was successfully able to estimate the Fundamental Matrix from findFundamentalMat() and implemented stereoRectifyUncalibrated() to generate rectification homography matrices H1 and H2.Then I have computed Rotation matrix R from H as
R = cameraMatrix^{-1}*H*cameraMatrix
Now I need to undistort my images after rectification. Either by initUndistortRectifyMap() and remap() or directly by undistort(), but both the functions also require "distortion co-efficients" to compute corrected image.
I tried to find various methods to estimate those parameters, neither the documentation of the camera model is made available by the company, nor I could find any other method apart from calibrating camera using chessboards or circles grid.
How do I do it??
Am I doing it right?
Is there any other better method?
Can someone kindly help?
Thanks in Advance.
//Code
//Fundamental Matrix
Mat fundamental_matrix = findFundamentalMat(points[0], points[1], FM_RANSAC, 3, 0.99);
cout<<"F:\n" <<fundamental_matrix<<endl;
//Rectification Homographies
Mat H1, H2,F;
F = fundamental_matrix;
stereoRectifyUncalibrated(points[0],points[1], F, image.size(), H1, H2, 3);
cout<<"H1:\n" <<H1<<endl;
cout<<"H2:\n" <<H2<<endl;
//calculating Rotation matrix from homographic maps
Mat fInv= fundamental_matrix.inv();
R = (fInv)*H1*fundamental_matrix;
// Mat distCoeffs = Mat::zeros(8, 1, CV_64F);
initUndistortRectifyMap(fundamental_matrix, distCoeffs, R, fundamental_matrix, image.size() ,CV_32FC1, map1, map2);
//How to compute distCoeffs without a camera nor prior knowledge.Thank You