The following link suggests that we can convert dual fisheye coordinates to equirectangular coordinates using the following equations:
// 2D fisheye to 3D vector
phi = r * aperture / 2
theta = atan2(y, x)
// 3D vector to longitude/latitude
longitude = atan2(Py, Px)
latitude = atan2(Pz, (Px^2 + Py^2)^(0.5))
// 3D vector to 2D equirectangular
x = longitude / PI
y = 2 * latitude / PI
I applied to above equations to write my source code like this:
const float FOV = 220.0f * PI / 180.0f;
float r = sqrt(u*u + v*v);
float theta = atan2(v, u);
float phi = r * FOV * 0.5f;
float px = u;
float py = r * sin(phi);
float pz = v;
float longitude = atan2(py, px); // theta
float latitude = atan2(pz, sqrt(px*px + py*py)); // phi
x = longitude / PI;
y = 2.0f * latitude / PI;
Unfortunately my math is not good enough to understand this and not sure if I write the above code correctly, where I tried to guess the values for px, py and pz.
Assume my camera FOV is 220 degrees, and the camera resolution is 2880x1440, I would expect the point (358, 224) for rear camera in the overlapped area and the point (2563, 197) for front camera in the overlapped area would both map to a coordinate close to (2205, 1009). However the actual mapping points are (515.966370,1834.647949) and (1644.442017,1853.060669) respectively, which are both very far away from (2205,1009). Please kindly suggest how to fix the above code. Many thanks!
You are building the equirectangular image, so I would suggest you to use the inverse mapping.
Start with pixel locations in the target image you are painting. Convert the 2D location to longitude/latitude. Then convert that to a 3D point on the surface of the unit sphere. Then convert from the 3D point to a location in the 2D fisheye source image. In Paul Bourke page, you would start with the bottom equation, then the rightmost one, then the topmost one.
Use landmark points like 90° long 0° lat, to verify the results make sense at each step.
The final result should be a location in the source fisheye image in the [-1..+1] range. Remap to pixel or to UV as needed. Since the source is split in two eye images you will also need a mapping from target (equirect) longitudes to the correct source sub-image.
Related
I'm generating an image with text in C++ using library called ImageMagick, and the DrawableRotation function does a rotation around the point of coordinates (0;0). (because it's an image it's top left corner )
The issue here is that I need to rotate my text label by a certain degree to put it on top of rectagle that has the same angle. ( text in box )
But the boxes are drown using the 4 points coordinates, I have their relative angle, which meens it's always positive, but some boxes are vertical, others are horizontal, and others are angled in between.
here's an exemple:
exemple of 3 boxes with different angles and their text
I would like to know if Image magick has an other way to rotate the text around itself or if there is a mathematical way to rotate the text so it has the good angle and then calculate the values I need to had to the coordinates to put it back at it's original coordinates.
I tried manually adding values to compensate the change in x;y but as all boxes have different angles, it's not dynamic enough, some labels get lost randomly on the image.
Well, I couldn't find a way to overcome this using the ImageMagick library directly, so what I do is make the rotation transformation around the origin before using the rotation formula from ImageMagick but in the opposite direction. But I also needed to map the angle (in degrees) from 90 to -90 so the text is always in the best direction to read. I'll post sample code for the rotation down below :
std::pair<float, float> MyImage::coorRotation(float x, float y, float angle)
{
float x_ = x*cos(angle) - y*sin(angle);
float y_ = x*sin(angle) + y*cos(angle);
return (std::make_pair(x_, y_));
}
for the rotation compensation and :
void MyImage::drawTextOnShapes()
float degrees = std::abs((atan2(this.shape.delta y, this.shape.delta x) * 180.0) / PI); // delta y = y2 - y1 and delta x = x2 - x1
if (degrees > 90)
degrees -= 180; // to map between 90 and -90
float radian = (degrees * PI) / 180;
float x = this.shape.center.x;
float y = this.shape.center.y;
std::pair coords = std::make_pair(x, y);
this->img.strokeColor("white");
coords = this->coorRotation(x, y, radian);
drawlist.push_back(DrawableRotation(-degrees));
stream << std::fixed << std::setprecision(2) << wall.length; // to get a .10 precision on the float output
drawlist.push_back(DrawableText(coords.first - textLenInPixels/2, coords.second, stream.str().append("m"))); // m for meters
this->img.draw(drawlist);
drawlist.clear();
this->saveImage();
}
And here's the result:
I'm currently stuck on achieving an equirectangular rotation on a 360° image with OpenCV because of my mathematical understanding (nearly zero) of projections and rotations matrixes.
The result of a such rotation would be exactly what you can see here: https://www.youtube.com/watch?v=l1N0lEKIeLA
I found some code here: https://github.com/FoxelSA/libgnomonic/wiki/Equirectangular-rotation_v0.1 but I didn't succeed to apply it to opencv
If someone has any idea how to apply it for an OpenCV Mat and Pitch, Yaw, Roll angles it would be highly appreciated!
Thanks!
Instead of talking about yaw, pitch and roll, I'll talk here about Euler angles x, y and z.
To perform a rotation of your equirectangular mapping, you can follow this procedure:
Consider coordinates (i2, j2) in your result image. We'll try to find which color to put here. These coordinates correspond to a point on the sphere with latitude lat2 = 180 * i2 / image.height and longitude lon2 = 360 * j2 / image.width. Compute the corresponding 3D vector v2.
Compute the rotation matrix R with angles x, y and z (look at the formulas here). Take the transpose of this matrix to get the inverse rotation from the new image to the old one. We'll name this inverse rotation matrix Rt.
Compute v1 = Rt * v2. Then compute the latitude lat1 and longitude lon1 of v1.
Find the color in the original image at coordinates i1 = image.height * lat1 / 180 and j1 = image.width * lon1 / 360. This might not be integer coordinates. You might have to interpolate between several pixels to get your value. This is the color of the pixel at position (i2, j2) in your new image.
You'll need to look at how to convert between 3D vectors on a sphere and their latitude and longitude angles but this shouldn't be too hard to find. The algorithm described here should be rather straightforward to implement.
Let me know if I made any mistake as I haven't tested it myself.
I have a 3D scene with a movable camera. I have the 3D coordinates for that camera (x, y, z -> Y being the height) and the X and Y rotation (Up/Down, Left/Right).
I want get the coordinates (x1, z1) in the floor where I'm looking at.
Basically if the camera is at (0, 4096, 0) (4096 is the height) and my xRotation is 45º and my yRotation is 0, I will be looking at the point on the floor (4096, 0, 0)
I was trying to program it but i got stuck with the trigonometry. Help me with it.
The following code is what I have right now and not fully working:
float x1, z1, anguloX, anguloY;
anguloX = (90 - Xrotation) / 180 * Pi;
anguloY = (90 - Yrotation) / 180 * Pi;
x1 = Yposition * tan(anguloX) * cos(anguloY);
z1 = Yposition * tan(anguloY) * cos(anguloY);
x1 += Xposition;
z1 += Zposition;
Do not bother yourself with this kind of trigonometry, It is often more practical and understandable to use matrices and transformation to do this kind of stuff. I suggest to try this approach instead which is called Ray Casting:
Calculate the direction of camera sight, you must get the looking
direction of camera
Use ray casting to determine the intersection point of sight ray and
scene meshes(or just a plane that represents the floor).
Now about the first, here is the pseudo code :
XRotMat = CreateRotationMatrixAroundXAxis(verticalCameraAngel);
YRotMat = CreateRotationMatrixAroundYAxis(horizentalCameraAngel);
CameraSightDir = YRotMat*XRotMa*initialCameraDir;
and about the second step:
SightRay.Source = Camera.Position;
SightRay.Direction = CameraSightDir;
Intersection = IntersectRayWithPlane(SightRay , FloorPlane);
IntersectRayWithPlane is quite a simple procedure, you can read about it here.
The problem is I have two points in 3D space where y+ is up, x+ is to the right, and z+ is towards you. I want to orientate a cylinder between them that is the length of of the distance between both points, so that both its center ends touch the two points. I got the cylinder to translate to the location at the center of the two points, and I need help coming up with a rotation matrix to apply to the cylinder, so that it is orientated the correct way. My transformation matrix for the entire thing looks like this:
translate(center point) * rotateX(some X degrees) * rotateZ(some Z degrees)
The translation is applied last, that way I can get it to the correct orientation before I translate it.
Here is what I have so far for this:
mat4 getTransformation(vec3 point, vec3 parent)
{
float deltaX = point.x - parent.x;
float deltaY = point.y - parent.y;
float deltaZ = point.z - parent.z;
float yRotation = atan2f(deltaZ, deltaX) * (180.0 / M_PI);
float xRotation = atan2f(deltaZ, deltaY) * (180.0 / M_PI);
float zRotation = atan2f(deltaX, deltaY) * (-180.0 / M_PI);
if(point.y < parent.y)
{
zRotation = atan2f(deltaX, deltaY) * (180.0 / M_PI);
}
vec3 center = vec3((point.x + parent.x)/2.0, (point.y + parent.y)/2.0, (point.z + parent.z)/2.0);
mat4 translation = Translate(center);
return translation * RotateX(xRotation) * RotateZ(zRotation) * Scale(radius, 1, radius) * Scale(0.1, 0.1, 0.1);
}
I tried a solution given down below, but it did not seem to work at all
mat4 getTransformation(vec3 parent, vec3 point)
{
// moves base of cylinder to origin and gives it unit scaling
mat4 scaleFactor = Translate(0, 0.5, 0) * Scale(radius/2.0, 1/2.0, radius/2.0) * cylinderModel;
float length = sqrtf(pow((point.x - parent.x), 2) + pow((point.y - parent.y), 2) + pow((point.z - parent.z), 2));
vec3 direction = normalize(point - parent);
float pitch = acos(direction.y);
float yaw = atan2(direction.z, direction.x);
return Translate(parent) * Scale(length, length, length) * RotateX(pitch) * RotateY(yaw) * scaleFactor;
}
After running the above code I get this:
Every black point is a point with its parent being the point that spawned it (the one before it) I want the branches to fit into the points. Basically I am trying to implement the space colonization algorithm for random tree generation. I got most of it, but I want to map the branches to it so it looks good. I can use GL_LINES just to make a generic connection, but if I get this working it will look so much prettier. The algorithm is explained here.
Here is an image of what I am trying to do (pardon my paint skills)
Well, there's an arbitrary number of rotation matrices satisfying your constraints. But any will do. Instead of trying to figure out a specific rotation, we're just going to write down the matrix directly. Say your cylinder, when no transformation is applied, has its axis along the Z axis. So you have to transform the local space Z axis toward the direction between those two points. I.e. z_t = normalize(p_1 - p_2), where normalize(a) = a / length(a).
Now we just need to make this a full 3 dimensional coordinate base. We start with an arbitrary vector that's not parallel to z_t. Say, one of (1,0,0) or (0,1,0) or (0,0,1); use the scalar product ·(also called inner, or dot product) with z_t and use the vector for which the absolute value is the smallest, let's call this vector u.
In pseudocode:
# Start with (1,0,0)
mindotabs = abs( z_t · (1,0,0) )
minvec = (1,0,0)
for u_ in (0,1,0), (0,0,1):
dotabs = z_t · u_
if dotabs < mindotabs:
mindotabs = dotabs
minvec = u_
u = minvec_
Then you orthogonalize that vector yielding a local y transformation y_t = normalize(u - z_t · u).
Finally create the x transformation by taking the cross product x_t = z_t × y_t
To move the cylinder into place you combine that with a matching translation matrix.
Transformation matrices are effectively just the axes of the space you're "coming from" written down as if seen from the other space. So the resulting matrix, which is the rotation matrix you're looking for is simply the vectors x_t, y_t and z_t side by side as a matrix. OpenGL uses so called homogenuous matrices, so you have to pad it to a 4×4 form using a 0,0,0,1 bottommost row and rightmost column.
That you can load then into OpenGL; if using fixed functio using glMultMatrix to apply the rotation, or if using shader to multiply onto the matrix you're eventually pass to glUniform.
Begin with a unit length cylinder which has one of its ends, which I call C1, at the origin (note that your image indicates that your cylinder has its center at the origin, but you can easily transform that to what I begin with). The other end, which I call C2, is then at (0,1,0).
I'd like to call your two points in world coordinates P1 and P2 and we want to locate C1 on P1 and C2 to P2.
Start with translating the cylinder by P1, which successfully locates C1 to P1.
Then scale the cylinder by distance(P1, P2), since it originally had length 1.
The remaining rotation can be computed using spherical coordinates. If you're not familiar with this type of coordinate system: it's like GPS coordinates: two angles; one around the pole axis (in your case the world's Y-axis) which we typically call yaw, the other one is a pitch angle (in your case the X axis in model space). These two angles can be computed by converting P2-P1 (i.e. the local offset of P2 with respect to P1) into spherical coordinates. First rotate the object with the pitch angle around X, then with yaw around Y.
Something like this will do it (pseudo-code):
Matrix getTransformation(Point P1, Point P2) {
float length = distance(P1, P2);
Point direction = normalize(P2 - P1);
float pitch = acos(direction.y);
float yaw = atan2(direction.z, direction.x);
return translate(P1) * scaleY(length) * rotateX(pitch) * rotateY(yaw);
}
Call the axis of the cylinder A. The second rotation (about X) can't change the angle between A and X, so we have to get that angle right with the first rotation (about Z).
Call the destination vector (the one between the two points) B. Take -acos(BX/BY), and that's the angle of the first rotation.
Take B again, ignore the X component, and look at its projection in the (Y, Z) plane. Take acos(BZ/BY), and that's the angle of the second rotation.
for a project I need to compute the real world position and orientation of a camera
with respect to a known object.
I have a set of photos, each displays a chessboard from different points of view.
Using CalibrateCamera and solvePnP I am able to reproject Points in 2d, to get a AR-thing.
So my situation is as such:
Intrinsic parameters are known
Distortioncoefficients are known
translation Vector and rotation Vector are known per photo.
I simply cannot figure out how to compute the position of the camera. My guess was:
invert translation vector. (=t')
transform rotation vector to degree (seems to be radian) and invert
use rodriguez on rotation vector
compute RotationMatrix * t'
But the results are somehow totally off...
Basically I want to to compute a ray for each pixel in world coordinates.
If more informations on my problem are needed, I'd be glad to answer quickly.
I dont' get it... somehow the rays are still off. This is my Code btw:
Mat image1CamPos = tvecs[0].clone(); //From calibrateCamera
Mat rot = rvecs[0].clone(); //From calibrateCamera
Rodrigues(rot, rot);
rot = rot.t();
//Position of Camera
Mat pos = rot * image1CamPos;
//Ray-Normal (( (double)mk[i][k].x) are known image-points)
float x = (( (double)mk[i][0].x) / fx) - (cx / fx);
float y = (( (double)mk[i][0].y) / fy) - (cy / fy);
float z = 1;
float mag = sqrt(x*x + y*y + z*z);
x /= mag;
y /= mag;
z /= mag;
Mat unit(3, 1, CV_64F);
unit.at<double>(0, 0) = x;
unit.at<double>(1, 0) = y;
unit.at<double>(2, 0) = z;
//Rotation of Ray
Mat rot = stof1 * unit;
But when plotting this, the rays are off :/
The translation t (3x1 vector) and rotation R (3x3 matrix) of an object with respect to the camera equals the coordinate transformation from object into camera space, which is given by:
v' = R * v + t
The inversion of the rotation matrix is simply the transposed:
R^-1 = R^T
Knowing this, you can easily resolve the transformation (first eq.) to v:
v = R^T * v' - R^T * t
This is the transformation from camera into object space, i.e., the position of the camera with respect to the object (rotation = R^T and translation = -R^T * t).
You can simply get a 4x4 homogeneous transformation matrix from this:
T = ( R^T -R^T * t )
( 0 1 )
If you now have any point in camera coordinates, you can transform it into object coordiantes:
p' = T * (x, y, z, 1)^T
So, if you'd like to project a ray from a pixel with coordinates (a,b) (probably you will need to define the center of the image, i.e. the principal point as reported by CalibrateCamera, as (0,0)) -- let that pixel be P = (a,b)^T. Its 3D coordinates in camera space are then P_3D = (a,b,0)^T. Let's project a ray 100 pixel in positive z-direction, i.e. to the point Q_3D = (a,b,100)^T. All you need to do is transform both 3D coordinates into the object coordinate system using the transformation matrix T and you should be able to draw a line between both points in object space. However, make sure that you don't confuse units: CalibrateCamera will report pixel values while your object coordinate system might be defined in, e.g., cm or mm.