calling glm::unproject() correctly, confused - c++

Question

I'm trying to use glm::unproject() to convert my SDL mouse coordinates into a world position vector, on the x/z-plane. Basically I want to figure out which "x/z" coordinate the user clicked on with a mouse.
From other stack overflow answers I came up needing to call glm::unproject(). I think I'm passing it the wrong arguments, because the values I'm getting back for the world position (printed std std::cerr) aren't world position values as I would expect.
Am I constructing the arguments to glm::unproject() correctly below? Specifically should I be combing the camera's world position and the view matrix (computed using glm::lookAt) to compute the modelview matrix passed into glm::unproject?
struct Dimensions {
int x, y, w, h;
};
glm::mat4
Camera::view_matrix() const
{
// VIEW matrix is created by looking at some target member
auto const& target = target_->translation;
auto const position_xyz = world_position();
glm::vec3 const UP{0, 1, 0};
return glm::lookAt(position_xyz, target, UP);
}
glm::mat4
Camera::projection_matrix() const
{
auto const fov = glm::radians(90.0f);
return glm::perspective(fov, 4.0f/3.0f, 0.1f, 200.0f);
}
glm::vec3
calculate_worldpos(Camera const& camera, int const mouse_x, int const mouse_y)
{
float const width = 1024.0f, height = 768.0f;
glm::vec4 const viewport = glm::vec4(0.0f, 0.0f, width, height);
glm::mat4 const modelview = camera.view_matrix();
glm::mat4 const projection = camera.projection_matrix();
float z = 0.0;
glm::vec3 screenPos = glm::vec3(mouse_x, height - mouse_y - 1, z);
std::cerr << "screenpos: xyz: '" << glm::to_string(screenPos) << "'\n";
glm::vec3 worldPos = glm::unProject(screenPos, modelview, projection, viewport);
std::cerr << "worldpos: xyz: '" << glm::to_string(worldPos) << "'\n";
return worldPos;
}
In the image below, I have the follow setup.
camera lookAt target = (0, 0, 0)
camera world position = (-0.009, 5.107, -0.368)
(mouse_x, mouse_y, mouse_z) = (286, 393, 0)
If you look at the image below, you can see that my mouse is hovering over the world position (3, 0, 0) as shown by the grid. I would expect calculating the world position of my mouse (as shown in the picture) would return me the vector (3, 0, 0). It does not, instead I get the vector: (0.049, 5.007, -0.360).
Does anyone see where I might be going wrong? I'm assuming I'm making some kind of incorrect assumption somewhere.

Answer 1

Your assumption is wrong: glm::unproject returns the worldspace coordinates of the input given by a xy-position in pixel coordinates and a z-coordinate storing the depth value. On every pixel on the screen, there is an infinite number of points in worldspace that project to this pixel (All that lie on the ray going from the projecting center through this pixel). Which one you want is identified by choosing the depth coordinate which than results in one specific point on this ray. Choosing z = 0 means that the result will always be a point on the near-plane of the camera.
What you are actually looking for is the intersection of this ray (going through the camera position and the calculated point) and the xz-plane (where y=0).
The ray is given by the two points on it (camera position C, near plane point P) as follows:
-0.009 0.058
C + l * (P-C) = ( 5.107 ) + l * ( -0.100 )
-0.368 0.008
, where l is a free variable.
As already said, we are looking for the intersection point (a,b) with the y=0 plane, thus we can formulate the following equation:
-0.009 0.058 a
( 5.107 ) + l * ( -0.100 ) = ( 0 )
-0.368 0.008 b
Solving the y-equation (5.107 + l * -0.1 = 0) for l results in l = 51.07. Pasting back in the equations for x and z yields:
a = -0.009 + 51.07 * 0.058 = 2.95306
b = -0.368 + 51.07 * 0.008 = 0.04056
Which is close to the expected worldspace position. The difference is most probably given by the fact that you just showed rounded numbers in the question. For accuracy reasons, I would also not calculate a point on the near-plane but one on the far plane (z=1) since the near-plane distance is usually quite small and could lead to numerical issues.
Conclusion: All values supplied are correct, but you were just not calculating what you expected.