I am working on a fiducial marker system (like Aruco) to obtain a 3d pose of markers (3d coordinates (x, y, z) and the roll, pitch, yaw of the marker) with respect to the camera. The overall setup is as shown in the figure.
Marker-Camera
Right now, for some reason, I am getting the pose representation of camera with respect to the marker (Thus, considering marker as an origin). But for my purpose, I want the pose representation of the marker, with respect to the camera. I cannot make changes in the way I am getting the pose, and I must use an external transformation. Currently, I using C++ Eigen library.
From what I have read so far, I have to do a rotation around the yaw (z) axis and then translate the obtained pose by the translation vector (x,y,z). But I am not sure how to represent this in Eigen. I tried to define my pose as Affine3f but I am not getting correct results.
Can anyone help me? Thanks!
If you are using ArUco, this might answer your questions: https://stackoverflow.com/a/59754199/8371691
However, if you are using some other marker system, the most robust way is to construct the attitude matrix and take inverse.
It is not clear how you represent your pose, but whether you use Euler angles or quaternion, it can be easily converted into an attitude matrix, R.
Then, the inverse transformation is simply taking inverse of R.
But given the nature of the configuration space that R belongs to, the inverse of R is also the transpose of R, which is computationally less expensive.
In Eigen, it's simply R.transpose().
If you are using ArUco with OpenCV, you can simply use built-in Rodrigues function.
But, if you are using ArUco, rvec is actually the rotation of the marker with respect to the camera frame.
Related
I am trying to estimate initial camera pose with respect to ECEF system using EXIF metadata. I have the a DJI Drone's gimbal Roll, Pitch and Yaw readings along with the Latitude, Longitude and Altitude. Assuming these values are fairly accurate, I am hoping to obtain the transformation matrix from Image to ECEF to find all the objects (out of a collection of objects whose location and altitude is known) that lie in a particular image along with their pixel coordinates, and then imporve the pose estimation at a later stage. I have tried the Pix4d way by converting roll, yaw and pitch to omega, phi and kappa, and then find a rotation matrix from PATB coordinate system to ECEF coordinates. I am not even sure if the values of Omega,Phi and Kappa obtained are correct, but assuming they are, the rotation matrix that I obtain from them is orthonormal, so I cannot directly use it for constructing the 4x4 transformation matrix, I probably need a scale factor for that.
Here is the link for Pix4D article - https://support.pix4d.com/hc/en-us/articles/205678146-How-to-convert-Yaw-Pitch-Roll-to-Omega-Phi-Kappa-
Please help me, I have been scouring through websites and papers for a week now to find a solution, but to no avail.
If you work in Python or Matlab, I recommend using the excellent nvector library for these problems - I use it every day exactly for these things.
I've got a question related to multiple view geometry.
I'm currently dealing with a problem where I have a number of images collected by a drone flying around an object of interest. This object is planar, and I am hoping to eventually stitch the images together.
Letting aside the classical way of identifying corresponding feature pairs, computing a homography and warping/blending, I want to see what information related to this task I can infer from prior known data.
Specifically, for each acquired image I know the following two things: I know the correspondence between the central point of my image and a point on the object of interest (on whose plane I would eventually want to warp my image). I also have a normal vector to the plane of each image.
So, knowing the centre point (in object-centric world coordinates) and the normal, I can derive the plane equation of each image.
My question is, knowing the plane equation of 2 images is it possible to compute a homography (or part of the transformation matrix, such as the rotation) between the 2?
I get the feeling that this may seem like a very straightforward/obvious answer to someone with deep knowledge of visual geometry but since it's not my strongest point I'd like to double check...
Thanks in advance!
Your "normal" is the direction of the focal axis of the camera.
So, IIUC, you have a 3D point that projects on the image center in both images, which is another way of saying that (absent other information) the motion of the camera consists of the focal axis orbiting about a point on the ground plane, plus an arbitrary rotation about the focal axis, plus an arbitrary translation along the focal axis.
The motion has a non-zero baseline, therefore the transformation between images is generally not a homography. However, the portion of the image occupied by the ground plane does, of course, transform as a homography.
Such a motion is defined by 5 parameters, e.g. the 3 components of the rotation vector for the orbit, plus the the angle of rotation about the focal axis, plus the displacement along the focal axis. However the one point correspondence you have gives you only two equations.
It follows that you don't have enough information to constrain the homography between the images of the ground plane.
TL;DR
I have a quaternion representing the orientation of a sphere (an Earth globe). From the quaternion I wish to derive a latitude/longitude. I can visualize in my mind the process, but am weak with the math (matrices/quaternions) and not much better with the code (still learning OpenGL/GLM). How can I achieve this? This is for use in OpenGL using c++ and the GLM library.
Long Version
I am making a mapping program based on a globe of the Earth - not unlike Google Earth, but for a customized purpose that GE cannot be adapted to.
I'm doing this in C++ using OpenGL with the GLM library.
I have successfully coded the sphere and am using a quaternion directly to represent it's orientation. No Euler angles involved. I can rotate the globe using mouse motions thus rotating the globe on arbitrary axes depending on the current viewpoint and orientation.
However, I would like to get a latitude and longitude of a point on the sphere, not only for the user, but for some internal program use as well.
I can visualize that this MUST be possible. Imagine a sphere in world space with no rotations applied. Assuming OpenGL's right hand rule, the north pole points up the Y axis with the equator parallel on the X/Z plane. The latitude/longitude up the Y axis is thus 90N and something else E/W (degenerate). The prime meridian would be on the +Z axis.
If the globe/sphere is rotated arbitrarily the globe's north pole is now somewhere else. This point can be mapped to a latitude/longitude of the original sphere before rotation. Imagine two overlaying spheres, one the globe which is rotated, and the other a fixed reference.
(Actually, it would be in reverse. The latitude/longitude I seek is the point on the rotated sphere that correlates to the north pole of the unrotated reference sphere)
In my mind it seems that somehow I should be able to get the vector of the Earth globe's orientation axis from it's quaternion and compare it to that of the unrotated sphere. But I just can't seem to grok how to do that. (I guess I still don't fully understand mats and quats and have only blundered into my success so far)
I'm hoping to achieve this without needing a crash course in the deep math. I'm looking for a solution/understanding/guidance from the point of view of being able to use the GLM library to achieve my goal. Ideally a code sample with some general explanation. I learn best from example.
FYI, in my code the rotation of the globe/sphere is totally independent of the camera (which does use Euler angles) so it can be moved independently. So I can't use anything from the camera to determine this.
Maybe you could try to follow that link (ie. use boost ;) ) from that thread Longitude / Latitude to quaternion and then deduct the inverse of that conversion.
Or you could also go add a step by converting your quaternion into Euler angle?
Now what I have is the 3D point sets as well as the projection parameters of the camera. Given two 2D point sets projected from the 3D point by using the camera and transformed camera(by rotation and translation), there should be an intuitive way to estimate the camera motion...I read some parts of Zisserman's book "Muliple view Geometry in Computer Vision", but I still did not get the solution..
Are there any hints, how can the rigid motion be estimated in this case?
THANKS!!
What you are looking for is a solution to the PnP problem. OpenCV has a function which should work called solvePnP. Just to be clear, for this to work you need point locations in world space, a camera matrix, and the points projections onto the image plane. It will then tell you the rotation and translation of the camera or points depending on how you choose to think of it.
Adding to the previous answer, Eigen has an implementation of Umeyama's method for estimation of the rigid transformation between two sets of 3d points. You can use it to get an initial estimation, and then refine it using an optimization algorithm and considering the projections of the 3d points onto the images too. For example, you could try to minimize the reprojection error between 2d points on the first image and projections of the 3d points after you bring them from the reference frame of one camera to the the reference frame of the other using the previously estimated transformation. You can do this in both ways, using the transformation and its inverse, and try to minimize the bidirectional reprojection error. I'd recommend the paper "Stereo visual odometry for autonomous ground robots", by Andrew Howard, as well as some of its references for a better explanation, especially if you are considering an outlier removal/inlier detection step before the actual motion estimation.
i have an object in 3d space that i want to align according to a vector.
i already got the Y-rotation out by doing an atan2 on the x and z component of the vector. but i would also like to have an X-rotation to make the object look downwards or upwards.
imagine a plane that does it's pitch yaw roll, just without the roll.
i am using openGL to set the rotations so i will need an Y-angle and an X-angle.
I would not use Euler angles, but rather a Euler axis/angle. For that matter, this is what Opengl glRotate uses as input.
If all you want is to map a vector to another vector, there are an infinite number of rotations to do that. For the shortest one, (the one with the smallest angle of rotation), you can use the vector found by the cross product of your from and to unit vectors.
axis = from X to
from there, the angle of rotation can be found from from.to = cos(theta) (assuming unit vectors)
theta = arccos(from.to)
glRotate(axis, theta) will then transform from to to.
But as I said, this is only one of many rotations that can do the job. You need a full referencial to define better how you want the transform done.
You should use some form of quaternion interpolation (Spherical Linear Interpolation) to animate your object going from its current orientation to this new orientation.
If you store the orientations using Quaternions (vector space math), then you can get the shortest path between two orientations very easily. For a great article, please read Understanding Slerp, Then Not Using It.
If you use Euler angles, you will be subject to gimbal lock and some really weird edge cases.
Actually...take a look at this article. It describes Euler Angles which I believe is what you want here.