How to convert mouse coordinate on screen to 3D coordinate - c++

Question

I'm creating a 3D application using GLUT in C++.
Now, I want to implement a method similar to this:
Vector3* MyClass::get3DObjectfromMouse(int mouseX, int mouseY);
How can I implement this method?

Answer 1

As it was commented by Andon M. Coleman, one way you can achieve this is by doing a ray/object intersection test, with unprojected screen coordinates. This technique is commonly known as picking.
A pseudo-C++ code for picking:
Assume we have a 3D object type/class:
class Object3D { ... };
A 3D picking function would return a list of all objects that are intersected by a line going from the given 2D point in the near plane to the same point in the far plane.
struct LineSegment
{
Vector3 start;
Vector3 end;
};
Object3D[] Pick(float x, float y)
{
LineSegment lineSeg;
Object3D[] intersectedObjs;
// Do both un-projections for z-near (0) and z-far (1).
// This produces a line segment going from z-near to far.
UnProject(x, y, /* z = */ 0.0, modelViewMatrix, projectionMatrix, viewport, lineSeg.start);
UnProject(x, y, /* z = */ 1.0, modelViewMatrix, projectionMatrix, viewport, lineSeg.end);
// Iterate all object in the scene or in the current view:
for (Object3D obj : scene)
{
if (TestLineIntersection(obj, lineSeg))
{
// This object is crossed by the picking line.
intersectedObjs.Add(obj);
}
}
// Optionally you might want sort them from distance
// to the camera/viewer before returning the intersections.
return intersectedObjs;
}
And the UnProject() function would look like this:
bool UnProject(float winX, float winY, float winZ,
const Matrix4 & modelView, const Matrix4 & projection,
const ScreenRect viewport, Vector3 & worldCoordinates)
{
// Compute (projection x modelView) ^ -1:
const Matrix4 m = inverse(projection * modelView);
// Need to invert Y since screen Y-origin point down,
// while 3D Y-origin points up (this is an OpenGL only requirement):
winY = viewport.Height() - winY;
// Transformation of normalized coordinates between -1 and 1:
Vector4 in;
in[0] = (winX - viewport.X()) / viewport.Width() * 2.0 - 1.0;
in[1] = (winY - viewport.Y()) / viewport.Height() * 2.0 - 1.0;
in[2] = 2.0 * winZ - 1.0;
in[3] = 1.0;
// To world coordinates:
Vector4 out(m * in);
if (out[3] == 0.0) // Avoid a division by zero
{
worldCoordinates = Vector3Zero;
return false;
}
out[3] = 1.0 / out[3];
worldCoordinates[0] = out[0] * out[3];
worldCoordinates[1] = out[1] * out[3];
worldCoordinates[2] = out[2] * out[3];
return true;
}
To clarify, TestLineIntersection() does a line vs AABB intersection test. The bounding box should be transformed to world-space, since it is usually expressed as a set of points in local model-space.
bool TestLineIntersection(const Object3D & obj, const LineSegment & lineSeg)
{
AABB aabb = obj.GetAABB();
aabb.TransformBy(obj.modelMatrix);
return aabb.LineIntersection(lineSeg.start, lineSeg.end);
}
// AABB.cpp:
bool AABB::LineIntersection(const Vector3 & start, const Vector3 & end) const
{
const Vector3 center = (mins + maxs) * 0.5;
const Vector3 extents = maxs - center;
const Vector3 lineDir = 0.5 * (end - start);
const Vector3 lineCenter = start + lineDir;
const Vector3 dir = lineCenter - center;
const float ld0 = Mathf::Abs(lineDir[0]);
if (Mathf::Abs(dir[0]) > (extents[0] + ld0))
{
return false;
}
const float ld1 = Mathf::Abs(lineDir[1]);
if (Mathf::Abs(dir[1]) > (extents[1] + ld1))
{
return false;
}
const float ld2 = Mathf::Abs(lineDir[2]);
if (Mathf::Abs(dir[2]) > (extents[2] + ld2))
{
return false;
}
const Vector3 vCross = cross(lineDir, dir);
if (Mathf::Abs(vCross[0]) > (extents[1] * ld2 + extents[2] * ld1))
{
return false;
}
if (Mathf::Abs(vCross[1]) > (extents[0] * ld2 + extents[2] * ld0))
{
return false;
}
if (Mathf::Abs(vCross[2]) > (extents[0] * ld1 + extents[1] * ld0))
{
return false;
}
return true;
}