I'm having trouble converting my OpenGL scene coordiates to my screen coordinates.
I thought I needed to multiply my coordinates with the modelview matrix then with the projection matrix to get the ndc. But i'm getting weird coordinates.
Here is my piece of code
GLKVector3 coor = GLKVector3Make(point.x, point.y, 0);
GLKMatrix4 modelview = GLKMatrix4MakeWithArray(glProjectionMatrix);
GLKMatrix4 projetion = GLKMatrix4MakeWithArray(modelViewMatrix.data);
GLKVector3 eyeCoor = GLKMatrix4MultiplyVector3(modelview, coor);
GLKVector3 ndcCoor = GLKMatrix4MultiplyVector3(projetion,eyeCoor);
CGPoint p = CGPointMake(ndcCoor.x, ndcCoor.y);
Any idea ?
The code seems perfectly valid, but you should use 4D vectors for these homogeneous transforms.
So,
GLKVector4 coor = GLKVector4Make(point.x, point.y, 0, 1);
/// I hope those matrices are fine
GLKMatrix4 modelview = GLKMatrix4MakeWithArray(glProjectionMatrix);
GLKMatrix4 projetion = GLKMatrix4MakeWithArray(modelViewMatrix.data);
GLKVector4 eyeCoor = GLKMatrix4MultiplyVector4(modelview, coor);
GLKVector4 ndcCoor = GLKMatrix4MultiplyVector4(projetion,eyeCoor);
float XScr = ndcCoor.x / ndcCoor.w;
float YScr = ndcCoor.y / ndcCoor.w;
CGPoint p = CGPointMake(XScr, YScr);
If you want XScr and YScr to be in [0..1] range, then add
XScr = (XScr + 1.0f) * 0.5f;
YScr = (YScr + 1.0f) * 0.5f;
conversion.
Even easier: use the GLKit Math function GLKMathProject.
GLKVector3 GLKMathProject (
GLKVector3 object,
GLKMatrix4 model,
GLKMatrix4 projection,
int *viewport
);
So, in your case, e.g.
int viewport[] = {0, 0, 320, 480};
GLKVector3 windowVector = GLKMathProject(coor, modelview, projetion, viewport);
CGPoint p = CGPointMake(windowVector.x, windowVector.y);
Note that the origin is at lower left, so if you're using UIKit coordinates where the origin is at the upper left, then switch the y coordinate, e.g.
CGPoint p = CGPointMake(windowVector.x, window.bounds.size.height - windowVector.y);
Related
In my SharpGL project (C#) I have used the Unproject function in order to get the world coordinates from mouse coordinates.
This procedure, quite trivial, fails when the drawing is scaled. I found many articles about this issue, but no one suited me.
When I say scaled means that in draw main proc i apply this code:
_gl.Scale(_params.ScaleFactor, _params.ScaleFactor, _params.ScaleFactor);
Then, when I intercept the mouse move I want to visualize the world coords. These coordinates are precise when the scale factor is 1, but when I change it these are wrong.
for example:
a world point (10, 10)
scaled 1 is detected (10, 10)
scaled 1,25 is detected (8, 8)
scaled 1,25 is detected (6.65, 6.65)
This is my simple code, consider that scale_factor is just passed for debugging.
public static XglVertex GetWorldCoords(this OpenGL gl, int x, int y, float scale_factor)
{
double worldX = 0;
double worldY = 0;
double worldZ = 0;
int[] viewport = new int[4];
double[] modelview = new double[16];
double[] projection = new double[16];
gl.GetDouble(OpenGL.GL_MODELVIEW_MATRIX, modelview); //get the modelview info
gl.GetDouble(OpenGL.GL_PROJECTION_MATRIX, projection); //get the projection matrix info
gl.GetInteger(OpenGL.GL_VIEWPORT, viewport); //get the viewport info
float winX = (float)x;
float winY = (float)viewport[3] - (float)y;
float winZ = 0;
//get the world coordinates from the screen coordinates
gl.UnProject(winX, winY, winZ, modelview, projection, viewport, ref worldX, ref worldY, ref worldZ);
XglVertex vres = new XglVertex((float)worldX, (float)worldY, (float)worldZ);
Debug.Print(string.Format("World Coordinate: x = {0}, y = {1}, z = {2}, sf = {3}", vres.X, vres.Y, vres.Z, scale_factor));
return vres;
}
I found a solution!
there were in the main draw procedure a portion of code that alterate results of UnProject function.
gl.PushMatrix();
.Translate(_mouse.CursorPosition.X * _params.ScaleFactor, _mouse.CursorPosition.Y * _params.ScaleFactor, _mouse.CursorPosition.Z * _params.ScaleFactor);
foreach (var el in _cursor.Elements) //objects to be drawn
{
gl.LineWidth(1f);
gl.Begin(el.Mode);
foreach (var v in el.Vertex)
{
gl.Color(v.Color.R, v.Color.G, v.Color.B, v.Color.A);
gl.Vertex(v.X, v.Y, v.Z);
}
gl.End();
}
gl.PopMatrix();
I have a pipeline that uses model, view and projection matrices to render a triangle mesh.
I am trying to implement a ray tracer that will pick out the object I'm clicking on by projecting the ray origin and direction by the inverse of the transformations.
When I just had a model (no view or projection) in the vertex shader I had
Vector4f ray_origin = model.inverse() * Vector4f(xworld, yworld, 0, 1);
Vector4f ray_direction = model.inverse() * Vector4f(0, 0, -1, 0);
and everything worked perfectly. However, I added a view and projection matrix and then changed the code to be
Vector4f ray_origin = model.inverse() * view.inverse() * projection.inverse() * Vector4f(xworld, yworld, 0, 1);
Vector4f ray_direction = model.inverse() * view.inverse() * projection.inverse() * Vector4f(0, 0, -1, 0);
and nothing is working anymore. What am I doing wrong?
If you use perspective projection, then I recommend to define the ray by a point on the near plane and another one on the far plane, in normalized device space. The z coordinate of the near plane is -1 and the z coordinate of the far plane 1. The x and y coordinate have to be the "click" position on the screen in the range [-1, 1] The coordinate of the bottom left is (-1, -1) and the coordinate of the top right is (1, 1). The window or mouse coordinates can be mapped linear to the NDCs x and y coordinates:
float x_ndc = 2.0 * mouse_x/window_width - 1.0;
flaot y_ndc = 1.0 - 2.0 * mouse_y/window_height; // flipped
Vector4f p_near_ndc = Vector4f(x_ndc, y_ndc, -1, 1); // z near = -1
Vector4f p_far_ndc = Vector4f(x_ndc, y_ndc, 1, 1); // z far = 1
A point in normalized device space can be transformed to model space by the inverse projection matrix, then the inverse view matrix and finally the inverse model matrix:
Vector4f p_near_h = model.inverse() * view.inverse() * projection.inverse() * p_near_ndc;
Vector4f p_far_h = model.inverse() * view.inverse() * projection.inverse() * p_far_ndc;
After this the point is a Homogeneous coordinates, which can be transformed by a Perspective divide to a Cartesian coordinate:
Vector3f p0 = p_near_h.head<3>() / p_near_h.w();
Vector3f p1 = p_far_h.head<3>() / p_far_h.w();
The "ray" in model space, defined by point r and a normalized direction d finally is:
Vector3f r = p0;
Vector3f d = (p1 - p0).normalized()
I am building a modeling program and I'd like to do transformations on objects in their own space and then assign that single object to a group to rotate around another axis which the group rotates around. However, I'd also like to be able to do transformations in the object's own space when it's combined.
Manipulating the individual object, I pick the object's center.
glm::mat4 transform;
transform = glm::translate(transform, - obj.meshCenter);
glm::mat4 transform1;
transform1 = glm::translate(transform1, obj.meshCenter);
obj.rotation = transform1*obj.thisRot*transform;
I then send this off to the shader,
glUniformMatrix4fv(modelLoc, 1, GL_FALSE, glm::value_ptr(obj.translation*obj.rotation*objscale);
I would now like to rotate this object around another axis, say an axis of (5,0,0) of 45 degrees.
I now have:
glm::mat4 groupR;
groupR = glm::rotate(groupR,glm::degrees(45.0f),glm::vec3(5,0,0));
obj.groupRotation = groupR;
glUniformMatrix4fv(modelLoc, 1, GL_FALSE,
glm::value_ptr(obj.groupRotation*obj.translation*obj.rotation*objscale)
I've now moved the object from it's local space to the Group space.
I'm having a bit of difficulty now operating tranformations in the object's own space when combined with the Group's rotation.
I've had limited success when I set the groupR axis to (0,1,0) like so:
///Translating object in its own space///
glm::mat4 R = obj.groupRotation;
obj.translation = glm::inverse(R) * obj.translate * R;
the problem here is that this will only translate the object correctly in it's own space if the axis of rotation of R (Group's rotation) is equal to (0,1,0):
///Rotating object in its own space///
glm::mat4 R = obj.groupRotation;
obj.rotation = glm::inverse(R) * obj.rot * R;
Again, the rotations are incorrect. I'm thinking that maybe I have to undo the groupR's axis translation? and then re-apply it somewhere?
Let's assume we have an object that is moved, rotated and scaled, and we define a transformation matrix as follows:
glm::mat4 objTrans ...; // translation
glm::mat4 objRot ...; // roation
glm::mat4 objScale ...; // scaling
glm::mat4 objMat = objTrans * objRot * objScale;
And we have rotation matrix that we want to run on the object. In this case we have rotation around the Z-axis:
foat angle ...; // rotation angle
glm::mat4 rotMat = glm::rotate( angle, glm::vec3( 0.0, 0.0, 1.0 ) );
We have several rotations we can do with this information.
First we want to rotate the object on its local axis:
glm::mat4 modelMat = objMat * rotMat;
A Rotation around the worlds origin can be performed like this:
glm::mat4 modelMat = rotMat * objMat;
In order to rotate around the origin of the object in the world coordinate system, we must eliminate the rotation of the object:
glm::mat4 modelMat = objMat * (glm::inverse(objRot) * rotMat * objRot);
A Rotation around the worlds origin in relation to the object you have to do the opposite:
glm::mat4 modelMat = (objRot * rotMat * glm::inverse(objRot)) * objMat;
If you have a complete transformations matrix for an object and you do not know the rotation part, then it can be easily determined.
Note that a transformation matrix usually looks like this:
( X-axis.x, X-axis.y, X-axis.z, 0 )
( Y-axis.x, Y-axis.y, Y-axis.z, 0 )
( Z-axis.x, Z-axis.y, Z-axis.z, 0 )
( trans.x, trans.y, trans.z, 1 )
To generate a rotation only matrix you have to extract the normalized axis vectors:
glm::mat4 a ...; // any matrix
glm::vec3 x = glm::normalize( a[0][0], a[0][1], a[0][2] );
glm::vec3 y = glm::normalize( a[1][0], a[1][1], a[1][2] );
glm::vec3 z = glm::normalize( a[2][0], a[2][1], a[2][2] );
glm::mat4 r;
r[0][0] = x[0]; r[0][1] = x[1]; r[0][2] = x[2]; r[0][3] = 0.0f;
r[1][0] = y[0]; r[1][1] = y[1]; r[1][2] = y[2]; r[0][3] = 0.0f;
r[2][0] = z[0]; r[2][1] = z[1]; r[2][2] = z[2]; r[0][3] = 0.0f;
r[3][0] = 0.0f; r[3][1] = 0.0f; r[3][2] = 0.0f; r[0][3] = 1.0f;
Here is a partial answer to the behavior I want and the setup I used. This seems to be what I need to do to get the correct transforms in object space while apart of a group rotation.
Here I have a model composed of 7 different individual meshes that is rotated around the origin of (0,5,0) on the y Axis, this is just an arbitrary rotation I choose for testing.
for (int i = 0; i < models.at(currentSelectedPointer.x)->meshes.size()i++)
{
glm::mat4 rotMat;
rotMat = glm::translate(rotMat, glm::vec3(5, 0, 0));
rotMat = glm::rotate(rotMat, f, glm::vec3(0, 1.0, 0.0));
rotMat = glm::translate(rotMat, glm::vec3(-5, 0, 0));
models.at(currentSelectedPointer.x)->meshes.at(i).groupRotation = rotMat;
}
all the meshes are now rotating around (0,5,0) as a group, not at (0,5,0), on the Y axis.
to do the correct rotation transform on a single object in it's own object space, I have to undo the location of the groupRotation's origin (Sorry for the messy code, but I did it in steps like this to keep everything seperated and easily disectable). Also the individual object has an identity matrix for both it's translation and it's scale.
//These will be used to shift the groupRotation origin back to the
// origin in order to rotate around the object's origin.
glm::mat4 gotoGroupAxis;
gotoGroupAxis= glm::translate(gotoGroupAxis, glm::vec3(5, 0, 0));
glm::mat4 goBack ;
goBack = glm::translate(goBack , glm::vec3(-5, 0, 0));
////////Group rotation and it's inverse
glm::mat4 tempGroupRot = goBack *obj.groupRotation*gotoGroupAxis;
glm::mat4 tempGroupRotInverse= glm::inverse(tempGroupRot);
//thisRot and lastRot are matrix variables I use to accumulate and
//save rotations
obj.thisRot = tempGroupRotInverse*
glm::toMat4(currentRotation)*tempGroupRot *
obj.lastRot;
//now I translate the object's rotation origin to it's center.
glm::mat4 transform = glm::translate(transform, -obj.meshCenter);
glm::mat4 transform1 = glm::translate(transform1, obj.meshCenter);
//Finally I rotate the object in it's own space.
obj.rotation = transform1*obj.thisRot*transform;
Update:
//Translation works as well with
obj.finalTranslation= tempGroupRotInverse*
obj.translation * tempGroupRot ;
This is only a partial answer because I'm going to be doing transforms on an object level and group level and I'm almost certain that something will go wrong down the line that hasn't been taken into account by the answer I've posted.
I'm attempting to implement an arcball style camera. I use glm::lookAt to keep the camera pointed at a target, and then move it around the surface of a sphere using azimuth/inclination angles to rotate the view.
I'm running into an issue where the view gets flipped upside down when the azimuth approaches 90 degrees.
Here's the relevant code:
Get projection and view martrices. Runs in the main loop
void Visual::updateModelViewProjection()
{
model = glm::mat4();
projection = glm::mat4();
view = glm::mat4();
projection = glm::perspective
(
(float)glm::radians(camera.Zoom),
(float)width / height, // aspect ratio
0.1f, // near clipping plane
10000.0f // far clipping plane
);
view = glm::lookAt(camera.Position, camera.Target, camera.Up);
}
Mouse move event, for camera rotation
void Visual::cursor_position_callback(GLFWwindow* window, double xpos, double ypos)
{
if (leftMousePressed)
{
...
}
if (rightMousePressed)
{
GLfloat xoffset = (xpos - cursorPrevX) / 4.0;
GLfloat yoffset = (cursorPrevY - ypos) / 4.0;
camera.inclination += yoffset;
camera.azimuth -= xoffset;
if (camera.inclination > 89.0f)
camera.inclination = 89.0f;
if (camera.inclination < 1.0f)
camera.inclination = 1.0f;
if (camera.azimuth > 359.0f)
camera.azimuth = 359.0f;
if (camera.azimuth < 1.0f)
camera.azimuth = 1.0f;
float radius = glm::distance(camera.Position, camera.Target);
camera.Position[0] = camera.Target[0] + radius * cos(glm::radians(camera.azimuth)) * sin(glm::radians(camera.inclination));
camera.Position[1] = camera.Target[1] + radius * sin(glm::radians(camera.azimuth)) * sin(glm::radians(camera.inclination));
camera.Position[2] = camera.Target[2] + radius * cos(glm::radians(camera.inclination));
camera.updateCameraVectors();
}
cursorPrevX = xpos;
cursorPrevY = ypos;
}
Calculate camera orientation vectors
void updateCameraVectors()
{
Front = glm::normalize(Target-Position);
Right = glm::rotate(glm::normalize(glm::cross(Front, {0.0, 1.0, 0.0})), glm::radians(90.0f), Front);
Up = glm::normalize(glm::cross(Front, Right));
}
I'm pretty sure it's related to the way I calculate my camera's right vector, but I cannot figure out how to compensate.
Has anyone run into this before? Any suggestions?
It's a common mistake to use lookAt for rotating the camera. You should not. The backward/right/up directions are the columns of your view matrix. If you already have them then you don't even need lookAt, which tries to redo some of your calculations. On the other hand, lookAt doesn't help you in finding those vectors in the first place.
Instead build the view matrix first as a composition of translations and rotations, and then extract those vectors from its columns:
void Visual::cursor_position_callback(GLFWwindow* window, double xpos, double ypos)
{
...
if (rightMousePressed)
{
GLfloat xoffset = (xpos - cursorPrevX) / 4.0;
GLfloat yoffset = (cursorPrevY - ypos) / 4.0;
camera.inclination = std::clamp(camera.inclination + yoffset, -90.f, 90.f);
camera.azimuth = fmodf(camera.azimuth + xoffset, 360.f);
view = glm::mat4();
view = glm::translate(view, glm::vec3(0.f, 0.f, camera.radius)); // add camera.radius to control the distance-from-target
view = glm::rotate(view, glm::radians(camera.inclination + 90.f), glm::vec3(1.f,0.f,0.f));
view = glm::rotate(view, glm::radians(camera.azimuth), glm::vec3(0.f,0.f,1.f));
view = glm::translate(view, camera.Target);
camera.Right = glm::column(view, 0);
camera.Up = glm::column(view, 1);
camera.Front = -glm::column(view, 2); // minus because OpenGL camera looks towards negative Z.
camera.Position = glm::column(view, 3);
view = glm::inverse(view);
}
...
}
Then remove the code that calculates view and the direction vectors from updateModelViewProjection and updateCameraVectors.
Disclaimer: this code is untested. You might need to fix a minus sign somewhere, order of operations, or the conventions might mismatch (Z is up or Y is up, etc...).
I try to use what many people seem to find a good way, I call gluUnproject 2 times with different z-values and then try to calculate the direction vector for the ray from these 2 vectors.
I read this question and tried to use the structure there for my own code:
glGetFloat(GL_MODELVIEW_MATRIX, modelBuffer);
glGetFloat(GL_PROJECTION_MATRIX, projBuffer);
glGetInteger(GL_VIEWPORT, viewBuffer);
gluUnProject(mouseX, mouseY, 0.0f, modelBuffer, projBuffer, viewBuffer, startBuffer);
gluUnProject(mouseX, mouseY, 1.0f, modelBuffer, projBuffer, viewBuffer, endBuffer);
start = vecmath.vector(startBuffer.get(0), startBuffer.get(1), startBuffer.get(2));
end = vecmath.vector(endBuffer.get(0), endBuffer.get(1), endBuffer.get(2));
direction = vecmath.vector(end.x()-start.x(), end.y()-start.y(), end.z()-start.z());
But this only returns the Homogeneous Clip Coordinates (I believe), since they only range from -1 to 1 on every axis.
How to actually get coordinates from which I can create a ray?
EDIT: This is how I construct the matrices:
Matrix projectionMatrix = vecmath.perspectiveMatrix(60f, aspect, 0.1f,
100f);
//The matrix of the camera = viewMatrix
setTransformation(vecmath.lookatMatrix(eye, center, up));
//And every object sets a ModelMatrix in it's display method
Matrix modelMatrix = parentMatrix.mult(vecmath
.translationMatrix(translation));
modelMatrix = modelMatrix.mult(vecmath.rotationMatrix(1, 0, 1, angle));
EDIT 2:
This is how the function looks right now:
private void calcMouseInWorldPosition(float mouseX, float mouseY, Matrix proj, Matrix view) {
Vector start = vecmath.vector(0, 0, 0);
Vector end = vecmath.vector(0, 0, 0);
FloatBuffer modelBuffer = BufferUtils.createFloatBuffer(16);
modelBuffer.put(view.asArray());
modelBuffer.rewind();
FloatBuffer projBuffer = BufferUtils.createFloatBuffer(16);
projBuffer.put(proj.asArray());
projBuffer.rewind();
FloatBuffer startBuffer = BufferUtils.createFloatBuffer(16);
FloatBuffer endBuffer = BufferUtils.createFloatBuffer(16);
IntBuffer viewBuffer = BufferUtils.createIntBuffer(16);
//The two calls for projection and modelView matrix are disabled here,
as I use my own matrices in this case
// glGetFloat(GL_MODELVIEW_MATRIX, modelBuffer);
// glGetFloat(GL_PROJECTION_MATRIX, projBuffer);
glGetInteger(GL_VIEWPORT, viewBuffer);
//I know this is really ugly and bad, but I know that the height and width is always 600
// and this is just for testing purposes
mouseY = 600 - mouseY;
gluUnProject(mouseX, mouseY, 0.0f, modelBuffer, projBuffer, viewBuffer, startBuffer);
gluUnProject(mouseX, mouseY, 1.0f, modelBuffer, projBuffer, viewBuffer, endBuffer);
start = vecmath.vector(startBuffer.get(0), startBuffer.get(1), startBuffer.get(2));
end = vecmath.vector(endBuffer.get(0), endBuffer.get(1), endBuffer.get(2));
direction = vecmath.vector(end.x()-start.x(), end.y()-start.y(), end.z()-start.z());
}
I'm trying to use my own projection and view matrix, but this only seems to give weirder results.
With the GlGet... stuff I get this for a click in the upper right corner:
start: (0.97333336, -0.98, -1.0)
end: (0.97333336, -0.98, 1.0)
When I use my own stuff I get this for the same position:
start: (-2.4399707, -0.55425626, -14.202201)
end: (-2.4399707, -0.55425626, -16.198204)
Now I actually need a modelView matrix instead of just the view matrix, but I don't know how I am supposed to get it, since it is altered and created anew in every display call of every object.
But is this really the problem? In this tutorial he says "Normally, to get into clip space from eye space we multiply the vector by a projection matrix. We can go backwards by multiplying by the inverse of this matrix." and in the next step he multiplies again by the inverse of the view matrix, so I thought this is what I should actually do?
EDIT 3:
Here I tried what user42813 suggested:
Matrix view = cam.getTransformation();
view = view.invertRigid();
mouseY = height - mouseY - 1;
//Here I only these values, because the Z and W values would be 0
//following your suggestion, so no use adding them here
float tempX = view.get(0, 0) * mouseX + view.get(1, 0) * mouseY;
float tempY = view.get(0, 1) * mouseX + view.get(1, 1) * mouseY;
float tempZ = view.get(0, 2) * mouseX + view.get(1, 2) * mouseY;
origin = vecmath.vector(tempX, tempY, tempZ);
direction = cam.getDirection();
But now the direction and origin values are always the same:
origin: (-0.04557252, -0.0020000197, -0.9989586)
direction: (-0.04557252, -0.0020000197, -0.9989586)
Ok I finally managed to work this out, maybe this will help someone.
I found some formula for this and did this with the coordinates that I was getting, which ranged from -1 to 1:
float tempX = (float) (start.x() * 0.1f * Math.tan(Math.PI * 60f / 360));
float tempY = (float) (start.y() * 0.1f * Math.tan(Math.PI * 60f / 360) * height / width);
float tempZ = -0.1f;
direction = vecmath.vector(tempX, tempY, tempZ); //create new vector with these x,y,z
direction = view.transformDirection(direction);
//multiply this new vector with the INVERSED viewMatrix
origin = view.getPosition(); //set the origin to the position values of the matrix (the right column)
I dont really use deprecated opengl but i would share my thought,
First it would be helpfull if you show us how you build your View matrix,
Second the View matrix you have is in the local space of the camera,
now typically you would multiply your mouseX and (ScreenHeight - mouseY - 1) by the View matrix (i think the inverse of that matrix sorry, not sure!) then you will have the mouse coordinates in camera space, then you will add the Forward vector to that vector created by the mouse, then you will have it, it would look something like that:
float mouseCoord[] = { mouseX, screen_heihgt - mouseY - 1, 0, 0 }; /* 0, 0 because we multipling by a matrix 4.*/
mouseCoord = multiply( ViewMatrix /*Or: inverse(ViewMatrix)*/, mouseCoord );
float ray[] = add( mouseCoord, forwardVector );