I have a rotatable camera mounted in a fixed position. The camera can be rotated with API. I want to do 3D reconstruction about the environment around the camera.
Is it possible to do 3D reconstruction in my setup ? I have read some theories about 3D reconstruction with two cameras. What are the major differences between my setup and a two-camera setup ?
Any tutorials/blogs/samples are welcome ;)
Triangulating points isn't possible during pure rotational motion. Geometrically you can think about how the direction vectors to a feature match from the two camera poses will be parallel and will never meet or will meet at an arbitrary point so the results from triangulation won't mean anything. The best approach for this would have to be some depth estimation from monocular images using deep learning.
Related
I am interested in finding the Rotation Matrix of an Aruco Marker from a Stereo Camera.
I know that estimateposesinglemarkers gives a Rotation Vector (which can be converted to matrix via Rodrigues)and Translation Vector but the values are not that stable and is supposedly written for MonoCam.
I can get Stable 3D points of the Marker from a Stereo Camera, however i am struggling in creating a Rotation Matrix. My Main goal is to achieve what Ali has achieved in this following blog Relative Position of Aruco Markers.
I have tried working with Euler Angles from here by creating a plane of the Aruco Marker from the 3D points that i get from the Stereo Camera but in vain.
I know my algorithm is failing because the values of the Relative Co-ordinates keeps on changing on moving the camera which should not happen as the Relative Co-ordinates b/w the Markers Should remain Constant.
I have a properly Calibrated camera with all the required matrices.
I tried using SolvePnP, but i believe it gives Rvecs and Tvecs which when combined together brings points from the model coordinate system to the camera coordinate system.
Any idea on how i can create the Rotation Matrix of the Marker with my fairly Stable 3D points so that on moving the camera, the relative Co-ordinates doesn't Change ?
Thanks in Advance.
Currently I am using the triangulatePoints function from OpenCV to triangulate two 2D points from two calibrated cameras into a 3D Point.
cv::triangulatePoints(calibration[camera1Index].getProjectionMatrix(), calibration[camera2Index].getProjectionMatrix(), points1, points2, points4D);
My code uses the projection matrix of the cameras and the undistorted point for each camera to find the 3D point.
The problem is that sometimes the cameras send points that can't be correlated.
My question is if there is any method to find the zone where a point from the first camera should be projected in the second camera and discard all the points from the second camera outside of that zone.
I've been searching and I guess homography is the solution here but I wasn't able to find an approach for this algorithm or if there's an existing method in OpenCV for this.
Thanks in advance.
I am using 3 ArUco Markers stuck on a 3D head phantom model to do pose estimation using OpenCV in C++. My algorithm for pose estimation is giving me the translation with respect to the camera, but I want to now know the coordinates of the marker with respect to the model coordinate system. Therefore I have scanned the head model using a 3D scanner and have an object file and the texture file with me. My question is what is the easiest or best way to get the coordinates of the markers with respect to the head model. Should I use OpenGL, blender or some other software for it? Looking for some pointers or advice.
Sounds like you have the coordinates for the markers with respect to the camera as the coordinate system, so coordinates in "eye space" or camera space. Which is when you have coordinates where the camera is at the origin.
This article has a brilliant diagram which explain the different spaces and how to transform in to different spaces:
http://antongerdelan.net/opengl/raycasting.html
If you want these same coordinates but in model space you need the matrices that will get you in to that space.
In this case you are going from eye/camera space -> model space so you need to multiply those coordinates by the inverse view matrix then by the inverse model matrix. Then your coordinate would be in model space.
But this is a lot more difficult when you are using a physical camera, as opposed to a software camera, in OpenGL for example.
To do that you will need to use OpenCV to obtain your camera's intrinsic and extrinsic parameters.
See this tutorial for more details:
https://docs.opencv.org/3.1.0/dc/dbb/tutorial_py_calibration.html
By rotating the pinhole camera at it's optical center, Just asking generally, can we get 3D informations? if yes, Can you explain or provide sources or terms so I can read it up?
Given only this rotation you can not get any 3D information. Even if there might be some disparity for the pixels close to the border, in real world situations this will most likely not work.
As a good reference I can point you to the book from Multiple View Geometry in Computer Vision from Hartley and Zisserman.
I'm attempting to animate a skeleton in C++. I currently have the coordinates of all the joints and which joints connect to which.
Does anyone know how I would go about, say, raising the arm up by 90 degrees and then calculate the new coordinates for all the joints further down the arm?
I'm guessing I'd have to get the vectors for each bone and rotate those and take it from there, but I'm not sure how to go about doing that.
(I'm using openGL to display them)
Calculate the point and axis you wish to rotate around. The axis is perpendicular to the rotation plane. The point is the shoulder in your case. You would rotate all points in the arm hierarchy the same amount. There are many examples on the web for rotation about an arbitrary axis. Here's one: http://paulbourke.net/geometry/rotate/
Another way to go about it is to reinterpret your skeleton using local transformations: ie each new bone has a transformation from its parent bone's world space. This is useful for forward kinematics (FK) where you simply pose a skeleton based on local rotations. Most motion capture data is stored in this way. To calculate the world co-ordinates of each joint, you must multiply all local matrices up the hierarchy.
If you currently only have skeleton joint positions in world space it is a pain to generate the local transformation matrices, because you don't necessarily know the local matrix of each joint. This is a bigger topic. I did it years ago when I worked on the motion capture retargetting module in a 3dsmax plugin called CAT.
I worked with two popular formats: BVH and HTR. From memory, BVH uses global positions (and is a pain), whereas HTR uses local joint matrices and is much easier to import.