I'm trying to perform fisheye camera calibration via OpenCV 3.4.0 (C++, MS Windows).
I used cv::fisheye::calibrate to make K and D (camera matrix and radial distortion coeffitients matrix). Then I used cv::fisheye::initUndistortRectifyMap to produce maps for X and Y coordinates.
And finally I used cv::remap to undistort image from fisheye camera via maps from initUndistortRectifyMap.
Everything looks right, but OpenCV dewarps only a central part of fisheye image.
Edges are moved outside.
I'd like to dewarp the whole image.
I tried to change focal length in K matrix manually, and got undistorted edges, but they became very very blurry.
I found some results in this task. For example
https://www.youtube.com/watch?v=Ll8KCnCw4iU
and
https://www.youtube.com/watch?v=p1kCR1i2nF0
As far as you can see these results are very similar with my results.
Does anybody have a solution of this problem?
I analyzed a lot of papers in the last 2 weeks. I think I found the source of the problem. OpenCV 3.4.0 fisheye undistortion method is based on a pin-hole camera model. We have an angle between the optical axis of the camera and the ray of light from some object. We also have an angle between the direction to an undistorted point of this object and the camera optical axis. If the fisheye image was undistorted correctly, these two angles would be equal. FOV of my fisheye camera is 180 degrees. It means that distance fromthe undistorted image center and the point corresponding to the edge of the undistorted image is equal to infinity.
In other words if we have a fisheye camera with FOV around 180 degrees, undistortion (via OpenCV) of 100% of fisheye image surface is impossible.
It can be achieved, only that using a projection instead of trying to undistort it.
More info here OpenCV fisheye calibration cuts too much of the resulting image
Example result:
Related
I am interested in finding the Rotation Matrix of an Aruco Marker from a Stereo Camera.
I know that estimateposesinglemarkers gives a Rotation Vector (which can be converted to matrix via Rodrigues)and Translation Vector but the values are not that stable and is supposedly written for MonoCam.
I can get Stable 3D points of the Marker from a Stereo Camera, however i am struggling in creating a Rotation Matrix. My Main goal is to achieve what Ali has achieved in this following blog Relative Position of Aruco Markers.
I have tried working with Euler Angles from here by creating a plane of the Aruco Marker from the 3D points that i get from the Stereo Camera but in vain.
I know my algorithm is failing because the values of the Relative Co-ordinates keeps on changing on moving the camera which should not happen as the Relative Co-ordinates b/w the Markers Should remain Constant.
I have a properly Calibrated camera with all the required matrices.
I tried using SolvePnP, but i believe it gives Rvecs and Tvecs which when combined together brings points from the model coordinate system to the camera coordinate system.
Any idea on how i can create the Rotation Matrix of the Marker with my fairly Stable 3D points so that on moving the camera, the relative Co-ordinates doesn't Change ?
Thanks in Advance.
Assume I have a camera that has been calibrated using the full camera frame to obtain the camera matrix and distortion coefficients. Also, assume that I have a 3D world point expressed in that camera's frame.
I know that I can use cv::projectPoints() with rvec=tvec=(0,0,0), the camera matrix, and distortion coefficients to project the point to the (full) distorted frame. I also know that if I am receiving an ROI from the camera (which is a cropped portion of the full distorted frame), I can adjust for the ROI simply by subtracting the (x,y) coordinate of the top left corner of the ROI from the result of cv::projectPoints(). Lastly, I know that if I use cv::projectPoints() with rvec=tvec=(0,0,0), the camera matrix, and zero distortion coefficients I can project the point to the full undistorted frame (correct me if I'm wrong, but I think this requires that you use the same camera matrix in cv::undistort() and don't use a newCameraMatrix).
How do I handle the case where I want to project to the undistorted version of an ROI that I am receiving (i.e. I get a (distorted) ROI and then use cv::undistort() on it using the method described here to account for the fact that it's an ROI, and then I want to project the 3D point to that resulting image)?
If there is a better way to go about all this I am open to suggestions as well. My goal is that I want to be able to project 3D points to distorted and undistorted frames with or without the presence of an ROI where the ROI is always originally defined by the feed from the camera and therefore always defined in the distorted frame (i.e. 4 different cases: distorted full frame, distorted ROI, undistorted full frame, undistorted version of distorted ROI).
I have calibrated my stereo cameras using OpenCV's stereoCalibrate() and rectified them using stereoRectify(). This is successful (cameras' epipolar lines correctly aligned and can obtain accurate 3D point locations from corresponding 2D point left/right pairs).
I'm now trying to use WebGL-based three.js to display the same 3D points in the same projection as I have in OpenCV. In other words, if I overlaid the calibrated and rectified 2D image from my left camera onto the three.js output, the three.js 3D points should visually align with where they are on the OpenCV 2D image. This could be used for augmented reality.
Just dealing with the left camera (camera 1):
stereoRectify() provides projection matrix P1 and rectification transform R1. Using only P1 I can convert a 3D point to the correct 2D screen position in OpenCV. However, I am having difficulty using P1 to get an equivalent camera.projectionMatrix in three.js.
I'm using the OpenCV camera matrix to OpenGL projection conversion suggested by Guillaume Noctua here. I'm taking the camera matrix to be the top-left 3x3 of P1. That produces a three.js camera view that looks similar, but not quite aligned (camera appears rotated along all axis by a degree or so, with perhaps some other small but clearly erroneous distortions/translations). So I'm missing something. Do I need to use the rectification transform R1 somehow too? I've tried using it as a rotation matrix and rotating the three.js camera by this amount but it doesn't improve.
Any thoughts on using OpenCV's P1 and R1 matrix to make an equivalent camera view in three.js would be much appreciated.
I have a reference image A with a known position and I want to calculate the relative position of the camera at image B (i.e. tx, ty, tz in meters). The images are taken with the same camera so the camera matrix stays the same. I'm using SIFT to detect and compute the keypoints and descriptors in both images and match them with FLANN. From there I can get the homography matrix which I decompose with cv::decomposeHomography(..). This function is based on this paper: PDF.
In this paper it is stated, that the translation matrix is normalized by d*, which is the plane depth.
In order to get the correct translation I need to know the plane depth. Is there a way to get this without knowing the size of an object found in the image?
The 3D translation computed using homography decomposition is only computable up to an unknown scale factor. This is a classical problem with computing 3D geometry from monocular images using only apparent motion in the images. Typically 3D reconstructions from monocular images are called metric reconstructions for this reason (rather than Euclidean reconstructions where scale is resolved). To resolve the scale factor some more information is needed, such as knowing the depth of a point on the plane or the distance moved by the camera between images.
I have an image with a circle in it, and I use the openCV methods to detect it and display its edges and center on the image before the image is rectified and undistorted.
I rectify the image and undistort it using InitUndistortRectifyMap in OpenCV. After remaping, the image is warped and the circle has an oval shape due to the change in perspective. The position coordinates of the center do obviuosly change as well.
I cannot do the circle detection step after rectifying because this will produce inaccurate results, due to the perspective change.
My question is, how can I find the position of the center after the image has been undistorted and rectified?
There is an undistortPoints function which is able to transform vector of Point2f or Point2d.