Using gluUnProject() to draw a quad at the cursor position - opengl

I am using SDL with OpenGL and up until now I have done everything in 2d (Using glOrtho())
Now I want to try drawing things in 3D but of course since I have now thrown a 3rd dimension into the mixture, things are getting more complicated.
What I want to do is, take the screen co-ordinates of the cursor, translate them to (?)world co-ordinates (Co-ordinates I can use in OpenGL since I now have perspective etc.), and draw a quad at that position (The mouse cursor being at the center of the quad)
I have read a few tutorials on gluUnProject() and I sort of understand what I am supposed to do, but since I am learning by myself it is very easy to get confused and not properly understand what I am doing.
As such, I have mainly copied from examples I have found.
Below is my code (I took out error checking etc in an attempt to cut it down a bit)
As you can see, I have mouseX, mouseY and mouseZ variables, of which mouseX and mouseY get their values from SDL.
I tried to use glReadPixel() to get mouseZ but I am not sure if I am doing it right.
The problem is that when I click, the quad is not drawn in the correct position (I guess partly due to the fact that I don't know how to get mouseZ properly, so I just replaced it in gluUnProject() with 1.0?)
I am using the new co-ordinates (wx, wy, wz) in the glTranslatef() function but once again because I don't know how to get mouseZ properly I am guessing this is the reason it doesn't work properly.
If I replace wz with -499 it seems to work ok, but I need to know how to get accurate results without manually finding the correct (Or almost correct) numbers
If anybody can tell me what I am doing wrong, or give me any advice it would be greatly appreciated.
#include <SDL/SDL.h>
#include <SDL/SDL_opengl.h>
SDL_Surface* screen = 0;
SDL_Event event;
bool isRunning = true;
int mouseX, mouseY, mouseZ = 0;
GLint viewport[4];
GLdouble mvmatrix[16], projmatrix[16];
GLint realY; /* OpenGL y coordinate position */
GLdouble wx, wy, wz; /* returned world x, y, z coords */
int main(int argc, char **argv) {
SDL_Init(SDL_INIT_EVERYTHING);
screen = SDL_SetVideoMode(800, 600, 32, SDL_OPENGL);
glEnable(GL_DEPTH_TEST);
glDepthMask(GL_TRUE);
glViewport(0, 0, 800, 600);
glClearDepth(1.f);
glClearColor(0, 0, 0, 0);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
gluPerspective(90.f, 1.f, 1.f, 500.f);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
while(isRunning) {
while(SDL_PollEvent(&event)) {
if(event.type == SDL_QUIT) {
isRunning = false;
}
if(event.type == SDL_MOUSEBUTTONDOWN) {
if(event.button.button == SDL_BUTTON_LEFT) {
SDL_GetMouseState(&mouseX, &mouseY);
glGetIntegerv(GL_VIEWPORT, viewport);
glGetDoublev(GL_MODELVIEW_MATRIX, mvmatrix);
glGetDoublev(GL_PROJECTION_MATRIX, projmatrix);
realY = viewport[3] - (GLint) mouseY - 1;
glReadPixels(mouseX, realY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &mouseZ);
gluUnProject((GLdouble)mouseX, (GLdouble)realY, 1.0, mvmatrix, projmatrix, viewport, &wx, &wy, &wz);
}
}
}
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glPushMatrix();
glTranslatef(wx, wy, wz);
glBegin(GL_QUADS);
glColor4f(1.f, 0.f, 0.f, 1.f);
glVertex3f(-20.f, -20.f, 0.f);
glColor4f(0.f, 1.f, 0.f, 1.f);
glVertex3f(-20.f, 20.f, 0.f);
glColor4f(0.f, 0.f, 1.f, 1.f);
glVertex3f(20.f, 20.f, 0.f);
glColor4f(1.f, 0.f, 1.f, 1.f);
glVertex3f(20.f, -20.f, 0.f);
glEnd();
glPopMatrix();
SDL_GL_SwapBuffers();
}
SDL_FreeSurface(screen);
return 0;
}

I found that if I added the following:
GLfloat depth[2];
then changed the glReadPixels() from:
glReadPixels(mouseX, realY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &mouseZ);
to:
glReadPixels(mouseX, realY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, depth);
then I just changed gluUnProject() from:
gluUnProject((GLdouble)mouseX, (GLdouble)realY, mouseZ, mvmatrix, projmatrix, viewport, &wx, &wy, &wz);
to:
gluUnProject((GLdouble) mouseX, (GLdouble) realY, depth[0], mvmatrix, projmatrix, viewport, &wx, &wy, &wz);
then I could get the Z value I wanted.
Then I just added a small value to wz, for example: wz += 1; to make sure the quad appeared above the place I clicked and it seems to work as I intended now.

To guess a Z you need to have a meaning for this z :
Usually the meaning is that you want to intersect the line corresponding to (camera origin,pixel) with a 3d object with is displayed. You can use the Z buffer for this.
If not ( you want to stick a polygon to the cursor with no 3D relation ), just use the 2D functions, you can just alternate between 3D and 2D projections while drawing in the same frame (with either the projection matrix or with 2D specific functions), thus you don't need a z.
Setting a default value for Z is a bad idea if it has no rationale in your design.
There is only one value of Z which will give you correct pixel coordinates.
The link between 2D and 3D coordinates in openGL is that OpenGl uses a projection plane at Z = 1.
And then scales the pixel coordinates from [-1,1] to pixels coordinates.

Related

Swap Y and Z Axis in OpenGL

So basically i have created an application that renders a curve in a 2D-Dimension (X and Y). This application also uses a haptic device, which is represented as a cursor inside of the frame where the curve is also rendered.
The only thing i want to achieve is to map the y-axis of my projection to my z-axis of the haptic workspace of my haptic device. So basically i want to mantain the logic of the curve and cursor rendering and only change the used axis for my haptic workspace. I used this question: Swap axis y and z
as a guidance. Practically i just have to use glRotatef(90, 1, 0, 0) to achieve what i want. I tried rotating only the projection matrix of my haptic workspace, but this sadly doesnt work for me.
This is how my normal curve looks:
Those are the results, while trying different options:
This is my code
reshapeCallback:
void reshapeCallback(int width, int height) {
glViewport(0, 0, width, height);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glOrtho(minX, maxX, minY, maxY, -1, 11);
// glRotatef(90, 1, 0, 0) [4th Try]
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
// glRotatef(90, 1, 0, 0) [5th Try]
updateWorkspace();
}
updateWorkspace():
void updateWorkspace() {
GLdouble modelview[16];
GLdouble projection[16];
GLint viewport[4];
glGetDoublev(GL_MODELVIEW_MATRIX, modelview);
//glRotatef(90, 1, 0, 0) [2nd try]
glGetDoublev(GL_PROJECTION_MATRIX, projection);
//glRotatef(90, 1, 0, 0) [1st try]
glGetIntegerv(GL_VIEWPORT, viewport);
hlMatrixMode(HL_TOUCHWORKSPACE);
hlLoadIdentity();
//glRotatef(90, 1, 0, 0) [3rd try]
// Fit haptic workspace to view volume.
hluFitWorkspace(projection);
// Compute cursor scale.
gCursorScale = hluScreenToModelScale(modelview, projection, viewport);
gCursorScale *= CURSOR_SIZE_PIXELS;
}
drawLine():
void drawLine() {
static GLuint displayList = 0;
if (displayList)
{
glCallList(displayList);
}
else
{
displayList = glGenLists(1);
glNewList(displayList, GL_COMPILE_AND_EXECUTE);
glPushAttrib(GL_ENABLE_BIT | GL_LIGHTING_BIT);
glDisable(GL_LIGHTING);
glLineWidth(2.0);
glBegin(GL_LINE_STRIP);
for (int i = 0; i < vertices.size(); i += 2) {
glVertex2f(vertices[i], vertices[i + 1]);
}
glEnd();
glPopAttrib();
glEndList();
}
}
I numerated the tries of glRotatef() for different positions. I hope someone can give me a hint where my thought process went wrong.
Point the first: For the love of God USE SHADERS AND NOT OLD OPENGL.
With shaders, it is a trivial task of swapping 2 arguments, changing
gl_Position = vec4(x, y, z, 1);
to
gl_Position = vec4(x, z, y, 1);
I would recommend looking at this tutorial for shaders.
Other that that, in my opinion the code looks like it should work, but if it doesn't I know how finicky glRotate can be, so I really have no advice other that to try things until it works.
Sorry I couldn't give a full, direct answer.

UnProjected mouse coordinates are between 0-1

I'm trying to create a ray from my mouse location out into 3D space, and apparently in order to do that I need to "UnProject()" it.
Doing so will give me a value between 0 & 1 for each axis.
This can't be right for drawing a "Ray" or a line from the viewport, can it? All this is, is a percentage essentially of my mouse to viewport size.
If this is actually right, then I don't understand the following:
I draw triangles that have vertices that are not constrained from 0-1, rather they are coordinates like (0,100,0), (100,100,0), (100,0,0), And these draw perfectly fine
But also, drawing the vertices that are unprojected from my mouse coordinates as lines/points also draw perfectly fine.
How the heck would I then compare my mouse coordinates to the coordinates of my objects?
If this is actually wrong, then what can cause such an error?
I tried unprojecting my own object's vertices, and those aren't constrained from 0-1.
I don't know whether or not the way I handle my "projections" when rendering is even compatible with gluUnproject. I've been just doing it the way these tutorials here show it (near bottom): http://qt-project.org/wiki/Developer-Guides#28810c65dd0f273a567b83a48839d275
This is the way I try to get my mouse coordinates:
GLdouble modelViewMatrix[16];
GLdouble projectionMatrix[16];
GLint viewport[4];
GLfloat winX, winY, winZ;
glGetDoublev(GL_MODELVIEW_MATRIX, modelViewMatrix);
glGetDoublev(GL_PROJECTION_MATRIX, projectionMatrix);
glGetIntegerv(GL_VIEWPORT, viewport);
winX = (float)x;
winY = (float)viewport[3] - (float)y;
glReadPixels( winX, winY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &winZ );
GLdouble nearPlaneLocation[3];
gluUnProject(winX, winY, 0, modelViewMatrix, projectionMatrix,
viewport, &nearPlaneLocation[0], &nearPlaneLocation[1],
&nearPlaneLocation[2]);
GLdouble farPlaneLocation[3];
gluUnProject(winX, winY, 1, modelViewMatrix, projectionMatrix,
viewport, &farPlaneLocation[0], &farPlaneLocation[1],
&farPlaneLocation[2]);
QVector3D nearP = QVector3D(nearPlaneLocation[0], nearPlaneLocation[1],
nearPlaneLocation[2]);
QVector3D farP = QVector3D(farPlaneLocation[0], farPlaneLocation[1],
farPlaneLocation[2]);
Perhaps my actual projections are off?
void oglWidget::paintGL()
{
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
QMatrix4x4 mMatrix;
QMatrix4x4 vMatrix;
QMatrix4x4 cameraTransformation;
cameraTransformation.rotate(alpha, 0, 1, 0);
cameraTransformation.rotate(beta, 1, 0, 0);
QVector3D cameraPosition = cameraTransformation * QVector3D(camX, camY, distance);
QVector3D cameraUpDirection = cameraTransformation * QVector3D(0, 1, 0);
vMatrix.lookAt(cameraPosition, QVector3D(camX, camY, 0), cameraUpDirection);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
gluLookAt(cameraPosition.x(), cameraPosition.y(), cameraPosition.z(), camX, camY, 0, cameraUpDirection.x(), cameraUpDirection.y(), cameraUpDirection.z());
shaderProgram.bind();
shaderProgram.setUniformValue("mvpMatrix", pMatrix * vMatrix * mMatrix);
shaderProgram.setUniformValue("texture", 0);
for (int x = 0; x < tileCount; x++)
{
shaderProgram.setAttributeArray("vertex", tiles[x]->vertices.constData());
shaderProgram.enableAttributeArray("vertex");
shaderProgram.setAttributeArray("textureCoordinate", textureCoordinates.constData());
shaderProgram.enableAttributeArray("textureCoordinate");
//Triangle Drawing
glTexImage2D( GL_TEXTURE_2D, 0, GL_RGBA, tiles[x]->image.width(), tiles[x]->image.height(), 0, GL_RGBA, GL_UNSIGNED_BYTE, tiles[x]->image.bits());
glDrawArrays(GL_TRIANGLES, 0, tiles[x]->vertices.size());
}
shaderProgram.release();
}
Where as, pMatrix is a 4x4 matrix, controlled during resize events like:
pMatrix.setToIdentity();
pMatrix.perspective(fov, (float) width / (float) height, 0.001, 10000);
glViewport(0, 0, width, height);
and my vertex shader is set up like this:
uniform mat4 mvpMatrix;
in vec4 vertex;
in vec2 textureCoordinate;
out vec2 varyingTextureCoordinate;
void main(void)
{
varyingTextureCoordinate = textureCoordinate;
gl_Position = mvpMatrix * vertex;
}
glReadPixels takes integers (x and y) and you don't seem to be using winZ for some reason in gluUnProject.
Try it like this:
gluUnProject(winX, winY, winZ, glView, glProjection, viewport, &posX, &posY, &posZ);
Also, if you want the ray to stop when it meets something in the depth buffer then don't clear the depth buffer after rendering. If you do a glClear(GL_DEPTH_BUFFER_BIT) then the ray should go as far as the far clip you set in your projection matrix.
I also have no idea why you need to call it more than once. The last three floats will be the target vector and you can just use your camera position as the source of the ray (depending on what you are doing).
Part of my problem here was poorly describing it. I accidentally left residual code from frantically testing, resulting in bits of "read Pixel" functions and related nonsense which wasn't useful for solving the problem.
The rest of my problem was due to inconsistent data types for the matrices, and trying to pull matrices from OpenGL when it never had them stored in the first place.
The problem was solved by:
Using GLM to hold all my matrices
performing the calculations myself (inverse view matrix * inverse model matrix * inverse projection matrix) * vector holding NDC converted screen space coordinates (range of -1 to 1: x or y divided by width or height, * 2 - 1), which also has a z of -1 or 1 for the far or near planes, and a w of 1.
Divide result by the fourth spot of the vector.
I still do not know why unprojecting doesn't work for me, as I got the wrong results with GLU as well as GLM's unproject function, but doing it manually worked for me.
Since my problem extended over quite a great length of time, and took up several questions, I owe credit to a few individuals who helped me along the way:
srobins of facepunch, in this thread
derhass from here, in this question, and this discussion

gluUnproject returning 0, seems to be related to modelview matrix

I'm working on a 2D image viewer, I want to retrieve openGL mouse position on texture but I can't get it to work if glTranslatef() or glScalef() calls are made on the modelview matrix.
I'm using a QGLWidget , of the famous Qt library.
Here are the important calls :
Resize function :
void ViewerGL::resizeGL(int width, int height){
glViewport (0, 0, width, height);
Display function :
void ViewerGL::paintGL()
{ int w = width();
int h = height();
glMatrixMode (GL_PROJECTION);
glLoadIdentity();
//transX,transY are for panning around the image in the viewer
float left = (0.f+transX) ;
float right = (w+transX) ;
float bottom = (h-transY);
float top = (0.f-transY) ;
glOrtho(left, right, top, bottom, -1, 1);
... later in paintGL:
glMatrixMode (GL_MODELVIEW);
glLoadIdentity ();
//padx,pady are used to translate the image from the bottom left corner
// to the center of the viewer
float padx,pady;
padx= ((float)width() - _dw.w()*zoomFactor)/2.f; // _dw.w is the width of the texture
pady =((float)height() - _dw.h()*zoomFactor)/2.f ;// _dw.h is the height of the texture
glTranslatef( padx , pady, 0);
//zoomX,zoomY are the position at which the user required a zoom
glTranslatef(-zoomX,-zoomY, 0.f);
glScalef(zoomFactor, zoomFactor,0.f);
glTranslatef(zoomX ,zoomY, 0.f);
Now here is my function to retrieve the openGL coordinates :
QPoint ViewerGL::openGLpos(int x,int y){
GLint viewport[4];
GLdouble modelview[16];
GLdouble projection[16];
GLfloat winX=0, winY=0, winZ=0;
GLdouble posX=0, posY=0, posZ=0;
glGetDoublev( GL_MODELVIEW_MATRIX, modelview );
glGetDoublev( GL_PROJECTION_MATRIX, projection );
glGetIntegerv( GL_VIEWPORT, viewport );
winX = (float)x;
winY = height()- y;
if(winY == 0) winY =1.f;
glReadPixels( x, winY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &winZ );
gluUnProject( winX, winY, winZ, modelview, projection, viewport, &posX, &posY, &posZ);
return QPoint(posX,posY);
}
So far , here is what I noticed:
The code like this returns always (0,0) and GLU_FALSE is returned from gluUnproject. I read somewhere on a forum that it could be because of the modelview matrix, so I put the identity matrix instead, but,if I do it, I get exactly the coordinates of the mouse in the window...
Before , I dealt with the zoom using the orthographic projection, but I couldn't make it work perfectly, so to make it simpler I decided to retrieve openGL position, and use glTranslatef/glScalef instead .
If I remove all the translating / scaling stuff in the paintGL function, everything is working...but the zoom doesn't work :x)
I'm requesting your help to make this damned zoom to point working, using the gluUnProject solution;)
Aigth , nevermind, I found the solution : I was zeroing out the z in glScalef(x,y,z)
so it made the matrix non-invertible...

glTranslatef Equivalent without it

Ok so I'm hooking into a game to retrieve data from it and use it. I got as far as hooking text (via CallLists).
The game uses:
glNewlist()
glBegin(GL_QUADS)
glVertex2i(....); //Stored the location of each char in the bitmap above..
glTexCoords2f(....); //Not sure what this is..
glEnd()
glEndList()
glCallList(876); //Represents a single character in the above bitmap.
glLoadIdentity(); //Resets the matrix.
glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE, GL_REPLACE);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_RGB, GL_TEXTURE);
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_RGB, GL_PREVIOUS);
glTranslatef(336, 196, 0); //Places it on screen somehow! :S? This is what I need to know.
glColor4ub(0, 0, 0, 255); //Colours the text.
LoadIdentity(); //Resets the matrix and does the next character.
glCallList(877); //Next char.
To render text to the screen. Is there a way I can figure out the coords of the text on the screen? I have access to all functions via Detours.
I'm not sure what the glTranslate did. How can I get the X and Y of the text?
I've used this to project the coords from glTranslate but it still projects it wrong. What do I pass to my WorldVector? It is just a struct with X, Y, Z. I've passed it the glTranslate coords but that doesn't work.
bool WorldToScreen(GLfloat &X, GLfloat &Y, Vector3D World, GLdouble* ModelViewMatrix, GLdouble* ProjectionMatrix)
{
GLint ViewPort[4];
GLdouble Screen[3];
glGetIntegerv(GL_VIEWPORT, ViewPort);
if(gluProject(World.X, World.Y, World.Z, ModelViewMatrix, ProjectionMatrix, ViewPort, &Screen[0], &Screen[1], &Screen[2]) == GL_TRUE)
{
X = Screen[0];
Y = ViewPort[3] - Screen[1];
return true;
}
return false;
}
That really depends, if you are drawing your text in orthographic mode, whatever you pass into glTranslatef is the actual screen coordinate, where if you are in perspective mode, you will have to pass them through the transformation pipeline to get the screen coordinates, I believe the function for doing this would be in the GLU library called gluProject, where gluUnProject would bring screen coordinates to world space
translate to world position
translate to view position
divide by W (Copy of Z) to get projection coordinates
ScreenX = Px * ScreenWidth/2 + ScreenWidth/2
ScreenY = -Py * ScreenWidth/2 + ScreenWidth/2
Here is an example of translating and calling your list in orthographic
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glOrtho(0.0, SCREEN_WIDTH, SCREEN_HEIGHT, 0.0, -1.0, 1.0);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
glTranslatef(336.0f, 196.0f, 0.0f);
glColor4f(1.0f, 0.0f, 0.0f, 1.0f); //Red
glCallList(877); //Or whatever list you wish to call
At this point, you may want to have the width of the next character to write, and simply translate value to put your text directly to the right of it,
By the way, there is a great free to use library called FreeType 2 , Blizzard uses it for there games, as well as myself, the former of which gives it good credibility.
If I am still not answering your question be sure to let me know

handling click on 3d or 2d figure in opengl

how to handle clik on specific 3d or 2d object in opengl, for example i have the following code
void Widget::drawCircle(float radius) {
glBegin(GL_TRIANGLE_FAN);
for (int i = 0; i < 360; i++) {
float degInRad = i*DEG2RAD;
glVertex2f(cos(degInRad) * radius, sin(degInRad) * radius);
}
glEnd();
}
So i need to handle click on this circle, is there any solutions for this problem?
When I need to detect clicks, I usually do my ordinary draw loop, but instead of drawing the objects with texturing, lighting and other effects enabled, I draw each of them with flat/no shading, each in a different color. I then check the color on the pixel the mouse is on, and map backwards from the color returned from the framebuffer to the object that I drew with that color.
Perhaps this technique is useful for you, too.
Take a look into this nehe tutorial item. It is very complex, but it shows how opengl picking works. In my opinion, if you need it, you are better with some game engine then with opengl.
Here is another (similar) way of selecting items in opengl.
opengl mouse raytracing will provide you with all details how to select items in opengl. This thread even provides the code how it is done :
Vector3 World::projectedMouse(float mx, float my){
GLdouble model_view[16];
GLint viewport[4];
GLdouble projection[16];
GLfloat winX, winY, winZ;
GLdouble dx, dy, dz;
glGetDoublev(GL_MODELVIEW_MATRIX, model_view);
glGetDoublev(GL_PROJECTION_MATRIX, projection);
glGetIntegerv(GL_VIEWPORT, viewport);
winX = (float)mx;
winY = (float)viewport[3] - (float)my;
glReadPixels ((int)mx, (int)winY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &winZ);
gluUnProject(winX, winY, 0, model_view, projection, viewport, &bx, &by, &bz);
Vector3 pr2 = Vector3(bx, by, bz);
glColor3f(1,0,0);
glBegin (GL_LINE_LOOP);
glVertex3f(player->getPosition().x, player->getPosition().y + 100, player->getPosition().z); // 0
glVertex3f(pr.x,pr.y,pr.z); // 1
glVertex3f(player->getPosition().x, player->getPosition().y, player->getPosition().z); // 0
glEnd();
return pr;
}