UnProject fails to get the world coordinate when scaling - opengl

In my SharpGL project (C#) I have used the Unproject function in order to get the world coordinates from mouse coordinates.
This procedure, quite trivial, fails when the drawing is scaled. I found many articles about this issue, but no one suited me.
When I say scaled means that in draw main proc i apply this code:
_gl.Scale(_params.ScaleFactor, _params.ScaleFactor, _params.ScaleFactor);
Then, when I intercept the mouse move I want to visualize the world coords. These coordinates are precise when the scale factor is 1, but when I change it these are wrong.
for example:
a world point (10, 10)
scaled 1 is detected (10, 10)
scaled 1,25 is detected (8, 8)
scaled 1,25 is detected (6.65, 6.65)
This is my simple code, consider that scale_factor is just passed for debugging.
public static XglVertex GetWorldCoords(this OpenGL gl, int x, int y, float scale_factor)
{
double worldX = 0;
double worldY = 0;
double worldZ = 0;
int[] viewport = new int[4];
double[] modelview = new double[16];
double[] projection = new double[16];
gl.GetDouble(OpenGL.GL_MODELVIEW_MATRIX, modelview); //get the modelview info
gl.GetDouble(OpenGL.GL_PROJECTION_MATRIX, projection); //get the projection matrix info
gl.GetInteger(OpenGL.GL_VIEWPORT, viewport); //get the viewport info
float winX = (float)x;
float winY = (float)viewport[3] - (float)y;
float winZ = 0;
//get the world coordinates from the screen coordinates
gl.UnProject(winX, winY, winZ, modelview, projection, viewport, ref worldX, ref worldY, ref worldZ);
XglVertex vres = new XglVertex((float)worldX, (float)worldY, (float)worldZ);
Debug.Print(string.Format("World Coordinate: x = {0}, y = {1}, z = {2}, sf = {3}", vres.X, vres.Y, vres.Z, scale_factor));
return vres;
}

I found a solution!
there were in the main draw procedure a portion of code that alterate results of UnProject function.
gl.PushMatrix();
.Translate(_mouse.CursorPosition.X * _params.ScaleFactor, _mouse.CursorPosition.Y * _params.ScaleFactor, _mouse.CursorPosition.Z * _params.ScaleFactor);
foreach (var el in _cursor.Elements) //objects to be drawn
{
gl.LineWidth(1f);
gl.Begin(el.Mode);
foreach (var v in el.Vertex)
{
gl.Color(v.Color.R, v.Color.G, v.Color.B, v.Color.A);
gl.Vertex(v.X, v.Y, v.Z);
}
gl.End();
}
gl.PopMatrix();

Related

camera matrix to OpenSceneGraph camera

I have a problem.
I Have a OpenGl project and i want to set the camera matrix of the OpenGl into a OpenSceneGraph camera to have the same view.
I've this code to get OpenGl camera :
GLdouble camera_pos[3];
double rationZoom = // RatioZoom
int* iViewport = // the ViewPort
double* projetMatrix = // Projection Matrix
double* modelMatrix = // ModelView matrix
// screenCenter is the position of my model.
double screenCenterX = // The center in x Axis
double screenCenterY = // The center in y Axis
gluUnProject((iViewport[2] - iViewport[0]) / 2, (iViewport[3] - iViewport[1]) / 2,
0.0, modelMatrix, projetMatrix, iViewport, &camera_pos[0], &camera_pos[1], &camera_pos[2]);`
//Camera_pos is the position X,Y,Z of my camera.
And in OpenSceneGraph i make this code to set the camera with eye/center/up to LookAt of the camera (to have the same view as OpenGl) :
// i use a zoom ration to have the same distance.
double X = ((camera_pos[0]/2) - ((screenCenterX)))/rationZoom;
double Y = ((camera_pos[1]/2) - ((screenCenterY)))/rationZoom;
double Z = ((camera_pos[2]/2)) / rationZoom;
osg::Vec3 updateCameraPosition(X, Y, Z);
osg::Matrix matrixCameraPosition;
matrixCameraPosition.setTrans(updateCameraPosition);
// Yes, for the center i invert the position matrix of the camera
osg::Matrix matrixCameraCenter;
matrixCameraCenter.invert(matrixCameraPosition);
osg::Vec3f eye(matrixCameraPosition.getTrans());
osg::Vec3f up(0, 0, 1);
osg::Vec3f centre = osg::Vec3f(matrixCameraCenter.getTrans().x(),
matrixCameraCenter.getTrans().y(),
matrixCameraCenter.getTrans().z());
// And a set the view into the camera
nodeCamera->setViewMatrixAsLookAt(eye, centre, up);
For the initialisation of the position i've no problem but if i panning the model of the OpenGl project i don't have the same view.
If I'm not mistaken for OpenGl the coordinate system is : X-left, Y-up and Z-backward, and for OpenSceneGraph this is : X-left, Y-backward, Z-up.
Maybe this is the problem and i have to set the Y up instead of Z in OpenSceneGraph ?
I have solved my problem.
I don't need to calculate the camera or set the setViewMatrixAsLookAt but just to get the modelview matrix and the projection matrix of OpenGl and set to the OpenScenGraph camera, like this:
double* projetMatrix = // the Projection matrix
double* modelMatrix = // The modelView matrix
osg::Matrixd modelViewMatrix(modelMatrix[0], modelMatrix[1], modelMatrix[2], modelMatrix[3],
modelMatrix[4], modelMatrix[5], modelMatrix[6], modelMatrix[7],
modelMatrix[8], modelMatrix[9], modelMatrix[10], modelMatrix[11],
modelMatrix[12], modelMatrix[13], modelMatrix[14], modelMatrix[15]
);
osg::Matrixd projectionMatrix(projetMatrix[0], projetMatrix[1], projetMatrix[2], projetMatrix[3],
projetMatrix[4], projetMatrix[5], projetMatrix[6], projetMatrix[7],
projetMatrix[8], projetMatrix[9], projetMatrix[10], projetMatrix[11],
projetMatrix[12], projetMatrix[13], projetMatrix[14], projetMatrix[15]
);
And :
camera->setViewMatrix(modelViewMatrix);
camera->setProjectionMatrix(projectionMatrix);
camera->setViewport(new osg::Viewport(iViewport[0], iViewport[1], iViewport[2], iViewport[3]));

Implement camera with off-axis projection

I'm trying to create a 3D viewer for a parallax barrier display, but I'm stuck with camera movements. You can see a parallax barrier display at: displayblocks.org
Multiple views are needed for this effect, this tutorial provide code for calculating the interViewpointDistance depending of the display properties and so selecting the head Position.
Here are the parts of the code involved in the matrix creation:
for (y = 0; y < viewsCountY; y++) {
for (x = 0; x <= viewsCountX; x++) {
viewMatrix = glm::mat4(1.0f);
// selection of the head Position
float cameraX = (float(x - int(viewsCountX / 2))) * interViewpointDistance;
float cameraY = (float(y - int(mviewsCountY / 2))) * interViewpointDistance;
camera.Position = glm::vec3(camera.Position.x + cameraX, camera.Position.y + cameraY, camera.Position.z);
// Move the apex of the frustum to the origin.
viewMatrix = glm::translate(viewMatrix -camera.Position);
projectionMatrix = get_off_Axis_Projection_Matrix();
// render's stuff
// (...)
// glfwSwapBuffers();
}
}
The following code is the projection matrix function. I use the Robert Kooima's paper generalized perspective projection.
glm::mat4 get_off_Axis_Projection_Matrix() {
glm::vec3 Pe = camera.Position;
// space corners coordinates (space points)
glm::vec3 Pa = glm::vec3(screenSizeX, -screenSizeY, 0.0);
glm::vec3 Pb = glm::vec3(screenSizeX, -screenSizeY, 0.0);
glm::vec3 Pc = glm::vec3(screenSizeX, screenSizeY, 0.0);
// Compute an orthonormal basis for the screen.
glm::vec3 Vr = Pb - Pa;
Vr = glm::normalize(Vr);
glm::vec3 Vu = Pc - Pa;
Vu = glm::normalize(Vu);
glm::vec3 Vn = glm::cross(Vr, Vu);
Vn = glm::normalize(Vn);
// Compute the screen corner vectors.
glm::vec3 Va = Pa - Pe;
glm::vec3 Vb = Pb - Pe;
glm::vec3 Vc = Pc - Pe;
//-- Find the distance from the eye to screen plane.
float d = -glm::dot(Va, Vn);
// Find the extent of the perpendicular projection.
float left = glm::dot(Va, Vr) * const_near / d;
float right = glm::dot(Vr, Vb) * const_near / d;
float bottom = glm::dot(Vu, Va) * const_near / d;
float top = glm::dot(Vu, Vc) * const_near / d;
// Load the perpendicular projection.
return glm::frustum(left, right, bottom, top, const_near, const_far + d);
}
These two methods works, and I can see that my multiple views are well projected.
But I cant manage to make a camera that works normally, like in a FPS, with Tilt and Pan.
This code for example give me the "head tracking" effect (but with the mouse), it was handy to test projections, but this is not what I'm looking for.
float cameraX = (mouseX - windowWidth / 2) / (windowWidth * headDisplacementFactor);
float cameraY = (mouseY - windowHeight / 2) / (windowHeight * headDisplacementFactor);
camera.Position = glm::vec3(cameraX, cameraY, 60.0f);
viewMatrix = glm::translate(viewMatrix, -camera.Position);
My camera class works if viewmatrix is created with lookAt. But with the off-axis projection, using lookAt will rotate the scene, by which the correspondence between near plane and screen plane will be lost.
I may need to translate/rotate the space corners coordinates Pa, Pb, Pc, used to create the frustum, but I don't know how.

Draw 2d box outline around 3d object

I want to draw a 2d box (smallest possible containing the object) around a 3d object with OpenGL.
Image: http://imgur.com/h1Vyy4b
What I have is:
Camera X/Y/Z/yaw/pitch, Object X/Y/Z/width/height/depth
I can draw on a 2d surface and a 3d surface.
How would I go about this?
I went here and found a function for getting screen coordinates out of your 3D points:
function point2D get2dPoint(Point3D point3D, Matrix viewMatrix,
Matrix projectionMatrix, int width, int height) {
Matrix4 viewProjectionMatrix = projectionMatrix * viewMatrix;
//transform world to clipping coordinates
point3D = viewProjectionMatrix.multiply(point3D);
int winX = (int) Math.round((( point3D.getX() + 1 ) / 2.0) *
width );
//we calculate -point3D.getY() because the screen Y axis is
//oriented top->down
int winY = (int) Math.round((( 1 - point3D.getY() ) / 2.0) *
height );
return new Point2D(winX, winY);
}
If your not sure how to get the matrices:
glGetDoublev (GL_MODELVIEW_MATRIX, mvmatrix);
glGetDoublev (GL_PROJECTION_MATRIX,pjmatrix);
After getting your 2D coordinates you go like this: (pseudo code)
int minX, maxX, minY, maxY;
for each 2dpoint p:
if (p.x<minX) minX=p.x;
if (p.x>maxX) maxX=p.x;
if (p.y<minY) minY=p.y;
if (p.y>maxY) maxY=p.y;
Then you draw a box with
P1=(minX,minY)
P2=(maxX,minY)
P3=(maxX,maxY)
P4=(minX,maxY)

get mouse in world coordinates with 2 gluUnproject calls to create ray

I try to use what many people seem to find a good way, I call gluUnproject 2 times with different z-values and then try to calculate the direction vector for the ray from these 2 vectors.
I read this question and tried to use the structure there for my own code:
glGetFloat(GL_MODELVIEW_MATRIX, modelBuffer);
glGetFloat(GL_PROJECTION_MATRIX, projBuffer);
glGetInteger(GL_VIEWPORT, viewBuffer);
gluUnProject(mouseX, mouseY, 0.0f, modelBuffer, projBuffer, viewBuffer, startBuffer);
gluUnProject(mouseX, mouseY, 1.0f, modelBuffer, projBuffer, viewBuffer, endBuffer);
start = vecmath.vector(startBuffer.get(0), startBuffer.get(1), startBuffer.get(2));
end = vecmath.vector(endBuffer.get(0), endBuffer.get(1), endBuffer.get(2));
direction = vecmath.vector(end.x()-start.x(), end.y()-start.y(), end.z()-start.z());
But this only returns the Homogeneous Clip Coordinates (I believe), since they only range from -1 to 1 on every axis.
How to actually get coordinates from which I can create a ray?
EDIT: This is how I construct the matrices:
Matrix projectionMatrix = vecmath.perspectiveMatrix(60f, aspect, 0.1f,
100f);
//The matrix of the camera = viewMatrix
setTransformation(vecmath.lookatMatrix(eye, center, up));
//And every object sets a ModelMatrix in it's display method
Matrix modelMatrix = parentMatrix.mult(vecmath
.translationMatrix(translation));
modelMatrix = modelMatrix.mult(vecmath.rotationMatrix(1, 0, 1, angle));
EDIT 2:
This is how the function looks right now:
private void calcMouseInWorldPosition(float mouseX, float mouseY, Matrix proj, Matrix view) {
Vector start = vecmath.vector(0, 0, 0);
Vector end = vecmath.vector(0, 0, 0);
FloatBuffer modelBuffer = BufferUtils.createFloatBuffer(16);
modelBuffer.put(view.asArray());
modelBuffer.rewind();
FloatBuffer projBuffer = BufferUtils.createFloatBuffer(16);
projBuffer.put(proj.asArray());
projBuffer.rewind();
FloatBuffer startBuffer = BufferUtils.createFloatBuffer(16);
FloatBuffer endBuffer = BufferUtils.createFloatBuffer(16);
IntBuffer viewBuffer = BufferUtils.createIntBuffer(16);
//The two calls for projection and modelView matrix are disabled here,
as I use my own matrices in this case
// glGetFloat(GL_MODELVIEW_MATRIX, modelBuffer);
// glGetFloat(GL_PROJECTION_MATRIX, projBuffer);
glGetInteger(GL_VIEWPORT, viewBuffer);
//I know this is really ugly and bad, but I know that the height and width is always 600
// and this is just for testing purposes
mouseY = 600 - mouseY;
gluUnProject(mouseX, mouseY, 0.0f, modelBuffer, projBuffer, viewBuffer, startBuffer);
gluUnProject(mouseX, mouseY, 1.0f, modelBuffer, projBuffer, viewBuffer, endBuffer);
start = vecmath.vector(startBuffer.get(0), startBuffer.get(1), startBuffer.get(2));
end = vecmath.vector(endBuffer.get(0), endBuffer.get(1), endBuffer.get(2));
direction = vecmath.vector(end.x()-start.x(), end.y()-start.y(), end.z()-start.z());
}
I'm trying to use my own projection and view matrix, but this only seems to give weirder results.
With the GlGet... stuff I get this for a click in the upper right corner:
start: (0.97333336, -0.98, -1.0)
end: (0.97333336, -0.98, 1.0)
When I use my own stuff I get this for the same position:
start: (-2.4399707, -0.55425626, -14.202201)
end: (-2.4399707, -0.55425626, -16.198204)
Now I actually need a modelView matrix instead of just the view matrix, but I don't know how I am supposed to get it, since it is altered and created anew in every display call of every object.
But is this really the problem? In this tutorial he says "Normally, to get into clip space from eye space we multiply the vector by a projection matrix. We can go backwards by multiplying by the inverse of this matrix." and in the next step he multiplies again by the inverse of the view matrix, so I thought this is what I should actually do?
EDIT 3:
Here I tried what user42813 suggested:
Matrix view = cam.getTransformation();
view = view.invertRigid();
mouseY = height - mouseY - 1;
//Here I only these values, because the Z and W values would be 0
//following your suggestion, so no use adding them here
float tempX = view.get(0, 0) * mouseX + view.get(1, 0) * mouseY;
float tempY = view.get(0, 1) * mouseX + view.get(1, 1) * mouseY;
float tempZ = view.get(0, 2) * mouseX + view.get(1, 2) * mouseY;
origin = vecmath.vector(tempX, tempY, tempZ);
direction = cam.getDirection();
But now the direction and origin values are always the same:
origin: (-0.04557252, -0.0020000197, -0.9989586)
direction: (-0.04557252, -0.0020000197, -0.9989586)
Ok I finally managed to work this out, maybe this will help someone.
I found some formula for this and did this with the coordinates that I was getting, which ranged from -1 to 1:
float tempX = (float) (start.x() * 0.1f * Math.tan(Math.PI * 60f / 360));
float tempY = (float) (start.y() * 0.1f * Math.tan(Math.PI * 60f / 360) * height / width);
float tempZ = -0.1f;
direction = vecmath.vector(tempX, tempY, tempZ); //create new vector with these x,y,z
direction = view.transformDirection(direction);
//multiply this new vector with the INVERSED viewMatrix
origin = view.getPosition(); //set the origin to the position values of the matrix (the right column)
I dont really use deprecated opengl but i would share my thought,
First it would be helpfull if you show us how you build your View matrix,
Second the View matrix you have is in the local space of the camera,
now typically you would multiply your mouseX and (ScreenHeight - mouseY - 1) by the View matrix (i think the inverse of that matrix sorry, not sure!) then you will have the mouse coordinates in camera space, then you will add the Forward vector to that vector created by the mouse, then you will have it, it would look something like that:
float mouseCoord[] = { mouseX, screen_heihgt - mouseY - 1, 0, 0 }; /* 0, 0 because we multipling by a matrix 4.*/
mouseCoord = multiply( ViewMatrix /*Or: inverse(ViewMatrix)*/, mouseCoord );
float ray[] = add( mouseCoord, forwardVector );

OpenGL scene coordinates to screen coordinates

I'm having trouble converting my OpenGL scene coordiates to my screen coordinates.
I thought I needed to multiply my coordinates with the modelview matrix then with the projection matrix to get the ndc. But i'm getting weird coordinates.
Here is my piece of code
GLKVector3 coor = GLKVector3Make(point.x, point.y, 0);
GLKMatrix4 modelview = GLKMatrix4MakeWithArray(glProjectionMatrix);
GLKMatrix4 projetion = GLKMatrix4MakeWithArray(modelViewMatrix.data);
GLKVector3 eyeCoor = GLKMatrix4MultiplyVector3(modelview, coor);
GLKVector3 ndcCoor = GLKMatrix4MultiplyVector3(projetion,eyeCoor);
CGPoint p = CGPointMake(ndcCoor.x, ndcCoor.y);
Any idea ?
The code seems perfectly valid, but you should use 4D vectors for these homogeneous transforms.
So,
GLKVector4 coor = GLKVector4Make(point.x, point.y, 0, 1);
/// I hope those matrices are fine
GLKMatrix4 modelview = GLKMatrix4MakeWithArray(glProjectionMatrix);
GLKMatrix4 projetion = GLKMatrix4MakeWithArray(modelViewMatrix.data);
GLKVector4 eyeCoor = GLKMatrix4MultiplyVector4(modelview, coor);
GLKVector4 ndcCoor = GLKMatrix4MultiplyVector4(projetion,eyeCoor);
float XScr = ndcCoor.x / ndcCoor.w;
float YScr = ndcCoor.y / ndcCoor.w;
CGPoint p = CGPointMake(XScr, YScr);
If you want XScr and YScr to be in [0..1] range, then add
XScr = (XScr + 1.0f) * 0.5f;
YScr = (YScr + 1.0f) * 0.5f;
conversion.
Even easier: use the GLKit Math function GLKMathProject.
GLKVector3 GLKMathProject (
GLKVector3 object,
GLKMatrix4 model,
GLKMatrix4 projection,
int *viewport
);
So, in your case, e.g.
int viewport[] = {0, 0, 320, 480};
GLKVector3 windowVector = GLKMathProject(coor, modelview, projetion, viewport);
CGPoint p = CGPointMake(windowVector.x, windowVector.y);
Note that the origin is at lower left, so if you're using UIKit coordinates where the origin is at the upper left, then switch the y coordinate, e.g.
CGPoint p = CGPointMake(windowVector.x, window.bounds.size.height - windowVector.y);