ok when i used to draw stuff using gdi+ i used to know the coordinates for objects but now when im using meshes in directx 9 i have no idea how to get the coordinates for the object so i can find if the mouse over an object. any idea how to find the coordinates ?
You need to cast the mouse position into the world and convert it to world-space coordinates, which then are tested against the various objects. You may be able to find a library to do this for you, I know OpenGL supports picking and most wrappers offer enhanced functions for that, but the principle is:
Find the mouse coordinates in the window. Using those coordinates, cast a ray (whether you actually use a ray in the system or simply do similar math isn't a big deal here) into the world. You'll use the current view matrix ("camera" angle and position) to calculate the direction and origin of the ray. Using that ray, test against your objects, their bounding boxes or geometry (whichever you choose) to find the object. Using the intersection coordinates, find the object that is at that location.
You can also use the depth buffer for this very easily, if your scene is relatively static. Simply render with a depth texture set as the Z buffer, then use the depth, mouse position and view matrix to find the point of intersection.
It may be possible to do this in reverse, that is, map each object to the appropriate screen coordinates, but you will likely run into issues with depth sorting and overlapping areas. Also, it can be unnecessarily slow mapping every object to window areas each frame.
Related
In my program I have a gizmo wich moves any objects in the scene. As I already know, is the usual way of storing any transformations is store that transformation in model matrices of this objects and execute any transformation directly in the shader. BUT also in my program I implement a classic ray-picking algorythm wich works only with a real transformed data. A ray detect any intersection with real(transformed) vertex position. How is the common way to solve this conflict:
Multiply any transformation immediatly on CPU and store transformed data. I think it's a clear way but it's expensive: for example I drag my object on screen in during 100 frames, and each frame I convert the delta of moving to matrices and multiply whole data by it.
Store any transformation in matrices until the mouse picking will starts and then quick multiply verticies by matrix to prepare data for picking. This is very fancy but there is ways to optimize it.
Which is the more performant way. Maybe there is some other method?
Update for Robinson:
I think you misunderstood me. Or I did not fully understand you. I have a box and a sphere and I move it by gizmo (I edit their model matrix) on 1,0,0 and 0,1,0 respectively. His model matrix is now different. HERE I get data that I need for ray-picking - ever objects has own individual place.
Then I transform the entire scene to eye space (view matrix) and then to clip space (projetion matrix) and render it. My ray makes return journey from viewport to world space (unproject a view and a projection matrix) and should interacts with the actual scene. My ray transformed rather than scene!
My question was how to interact with the objects wich the real place is unknown until it's will render (or transformed)? Or may be I'm not on the right track and I should have done it differently - multiply entire data each step (it's expensive, look at my first question).
You use ray-picking which technically is "get x,y screen coordinates, transform them to NDC and set the z as anyone in the range [-1,1]; and finally transform them all back to world coordinates".
This is useful when you want to intersect a ray from the point of view (the camera) to "mouse coordinates" AND you want to do all of this intersection calculations on CPU side.
Notice you can do it even when nothing is drawn in the screen, just mouse coordinates are needed; well, plus the viewport and the current transformations, but you know them before any glDrawxxx command.
Next, the question is: what are you going to do with that ray or intersections?
You may wish to modify some property (like color) or position. Right?
How many objects are to be modified? If it's just a bunch then it's OK to do it on CPU modifying the data to send to GPU. But if you have thousands of objects then think of the hardware-accelerated way: keep their coordinates but send the new tranformation matrices and properties to GPU and let it do the hard work.
If you are worried about some objects stay as before but others get modified, remember that you can draw groups of objects that share the matrices and other uniforms with a single glDrawxxx call. If you have different groups, use several glDrawxxx calls with different uniforms, even different shaders.
What is the best way to drag 3D point with mouse picking. Issue is not the picking but to dragging in 3D space.
There are two ways I am thinking, one is to get the View to World coordinates using gluUnProject and translate the 3D point. The problem in this case is, it only world on surfaces with Depth value (with glReadPixels), if mouse leaves the surface it gives the maximum or minimum depth values based on winZ component of gluUnProject. And it doesn't work in some cases.
The second way is to drag along XY, XZ, YZ plane using GL_MODELVIEW_MATRIX.
But the problem in this case, is how would we know that we are on XY or XZ or YZ plane? How can we know that the front view of the trackball is in XY plane, and what if we want to drag on the side plane not the front plane?
So, is there any way that gives me the accurate 2D to 3D coordinate so that I can drag 3D point easily in every direction without considering the plane cases? There must be some ways, I have seen 3D softwares, they have perfect dragging feature.
I'm used to solving these user-interaction problems somewhat naively (perhaps not in a mathematically optimal way) but "well enough" considering that they're not very performance-critical (the user interaction parts, not necessarily the resulting modifications to the scene).
For unconstrained free dragging of an object, the method you described using unproject tends to work quite well to often give near pixel-perfect dragging with a slight tweak:
... instead of using glReadPixels to try to extract screen depth, you want a notion of a geometric object/mesh when the user is picking and/or has selected. Now just project the center point of that object to get your screen depth. Then you can drag around in screen X/Y, keeping the same Z you got from this projection, and unproject to get the resulting translation delta from previous center to new center to transform the object. This also makes it "feel" like you're dragging from the center of the object which tends to be quite intuitive.
For auto-constrained dragging, a quick way to detect that is to first grab a 'viewplane normal'. A quick way (might make mathematicians frown) using those projection/unprojection functions you're used to is to unproject two points at the center of the viewport in screenspace (one with a near z value and one with a far z value) and get a unit vector in between those two points. Now you can find the world axis closest to that normal using dot product. The other two world axises define the world plane we want to drag along.
Then it becomes a simple matter of using those handy unprojection functions again to get a ray along the mouse cursor. After that, you can do repeated ray/plane intersections as you're dragging the cursor around to compute a translation vector from the delta.
For more flexible constraints, a gizmo (aka manipulator, basically a 3D widget) can come in handy so that the user can indicate what kind of drag constraint he wants (planar, axis, unconstrained, etc) based on which parts of the gizmo he picks/drags. For axis constraints, ray/line or line/line intersection is handy.
As requested in the comments, to retrieve a ray from a viewport (C++-ish pseudocode):
// Get a ray from the current cursor position (screen_x and screen_y).
const float near = 0.0f;
const float far = 1.0f;
Vec3 ray_org = unproject(Vec3(screen_x, screen_y, near));
Vec3 ray_dir = unproject(Vec3(screen_x, screen_y, far));
ray_dir -= ray_org;
// Normalize ray_dir (rsqrt should handle zero cases to avoid divide by zero).
const float rlen = rsqrt(ray_dir[0]*ray_dir[0] +
ray_dir[1]*ray_dir[1] +
ray_dir[2]*ray_dir[2]);
ray_dir[0] *= rlen;
ray_dir[1] *= rlen;
ray_dir[2] *= rlen;
Then we do a ray/plane intersection with the ray obtained from the mouse cursor to figure out where the ray intersects the plane when the user begins dragging (the intersection will give us a 3D point). After that, it's just translating the object by the deltas between the points gathered from repeatedly doing this as the user drags the mouse around. The object should intuitively follow the mouse while being moved along a planar constraint.
Axis dragging is basically the same idea, but we turn the ray into a line and do a line/line intersection (mouse line against the line for the axis constraint, giving us a nearest point since the lines generally won't perfectly intersect), giving us back a 3D point from which we can use the deltas to translate the object along the constrained axis.
Note that there are tricky edge cases involved with axis/planar dragging constraints. For example, if a plane is perpendicular to the viewing plane (or close), it can shoot off the object into infinity. The same kind of case exists with axis dragging along a line that is perpendicular, like trying to drag along the Z axis from a front viewport (X/Y viewing plane). So it's worth detecting those cases where the line/plane is perpendicular (or close) and prevent dragging in such cases, but that can be done after you get the basic concept working.
Another trick worth noting to improve the way things "feel" for some cases is to hide the mouse cursor. For example, with axis constraints, the mouse cursor could end up becoming very distant from the axis itself, and it might look/feel weird. So I've seen a number of commercial packages simply hide the mouse cursor in this case to avoid revealing that discrepancy between the mouse and the gizmo/handle, and it tends to feel a bit more natural as a result. When the user releases the mouse button, the mouse cursor is moved to the visual center of the handle. Note that you shouldn't do this hidden-cursor dragging for tablets (they're a bit of an exception).
This picking/dragging/intersection stuff can be very difficult to debug, so it's worth tackling it in babysteps. Set small goals for yourself, like just clicking a mouse button in a viewport somewhere to create a ray. Then you can orbit around and make sure the ray was created in the right position. Next you can try a simple test to see if that ray intersects a plane in the world (say X/Y) plane, and create/visualize the intersection point between the ray and plane, and make sure that's correct. Take it in small, patient babysteps, pacing yourself, and you'll have smooth, confident progress. Try to do too much at once and you can have very discouraging jarring progress trying to figure out where you went wrong.
It's a very interesting toic. As well as the methods you described color tagging coloring the z-buffer is also a method. There have been similar discussions about the same topic here:
Generic picking solution for 3D scenes with vertex-shader-based geometry deformation applied
OpenGL - Picking (fastest way)
How to get object coordinates from screen coordinates?
This is also similar to object picking that has been discussed fully including their pros and cons:
http://www.opengl-tutorial.org/miscellaneous/clicking-on-objects/picking-with-an-opengl-hack/
Actually my answer is there is no unique best way to do this. As you also mentioned they have pros and cons. I personally like gluunproject as it is easy and its results are okay. In addition, you cannot move a vertex at any direction using the screen space as screen space is 2D and there is not a unique way to map it back to th 3D geometry space. Hope it helps :).
I have a couple of 3D objects in OpenGL in a processing sketch and I need to find out if the mouse is hovering over those objects. Since there is constant transformation I can't compare the original coordinates and size to the mouse position. I already found the screenX() and screenY() methods which return the translated screen coordinates after transformation and translation but I would still need to get the displayed size after rotation.
Determining which object the mouse is over is called picking and there are 2 main approaches:
Color picking. Draw each object using a different color into the back buffer (this is only done when picking, the colored objects are never displayed on screen). Then use glReadPixels to read the pixel under the cursor and check its color to determine which object it is. If the mouse isn't over an object you'll get the background color. More details here: Lighthouse 3D Picking Tutorial, Color Coding
Ray casting. You cast a ray through the cursor location into the scene and check if it intersects any objects. More details here: Mouse picking with ray casting
From reading your description option 1 would probably be simpler and do what you need.
I am quite new to CGAL as well as OpenGL. I know that CGAL provides a Qt interface to display objects but I want to use only OpenGL and I am able to render polyhedrons and nef polyhedrons in openGL(I referred to polyhedron demo). Question is, how to display polyhedrons of different size efficiently in openGL. I apply translation in my program using glTranslatef to view the objects properly. Problem is, it may not work for each and every object because of the difference in the size. Therefore I need to apply translations based on the size of the object. If I can find the longest diagonal of the object this may be possible by adjusting the value of the parameters that I pass to glTranslatef(). Is there any way to do this in CGAL?
Treat your objects as a collection of points, and create a bounding volume from it. The size of the bounding volume should give you the scaling required. For example, you might wish to center the view around the center of the bounding sphere, and scale the view based on its radius.
See the chapter on bounding volumes.
Also, you probably want to use glScale to scale the view in addition to glTranslate to center it.
I'm working with Qt and QWt3D Plotting tools, and extending them to provide some 3-D and 2-D plotting functionality that I need, so I'm learning some OpenGL in the process.
I am currently able to plot points using OpenGL, but only as circles (or "squares" by turning anti-aliasing off). These points act the way I like - i.e. they don't change size as I zoom in, although their x/y/z locations move appropriately as I zoom, pan, etc.
What I'd like to be able to do is plot points using a myriad of shapes (^,<,>,*,., etc.). From what I understand of OpenGL (which isn't very much) this is not trivial to accomplish because OpenGL treats everything as a "real" 3-D object, so zooming in on any openGL shape but a "point" changes the object's projected size.
After doing some reading, I think there are (at least) 2 possible solutions to this problem:
Use OpenGL textures. This doesn't seem to difficult, but I believe that the texture images will get larger and smaller as I zoom in - is that correct?
Use OpenGL polygons, lines, etc. and draw *'s, triangles, or whatever. But here again I run into the same problem - how do I prevent OpenGL from re-sizing the "points" as I zoom?
Is the solution to simply bite the bullet and re-draw the whole data set each time the user zooms or pans to make sure that the points stay the same size? Is there some way to just tell openGL to not re-calculate an object's size?
Sorry if this is in the OpenGL doc somewhere - I could not find it.
What you want is called a "point sprite." OpenGL1.4 supports these through the ARB_point_sprite extension.
Try this tutorial
http://www.ploksoftware.org/ExNihilo/pages/Tutorialpointsprite.htm
and see if it's what you're looking for.
The scene is re-drawn every time the user zooms or pans, anyway, so you might as well re-calculate the size.
You suggested using a textured poly, or using polygons directly, which sound like good ideas to me. It sounds like you want the plot points to remain in the correct position in the graph as the camera moves, but you want to prevent them from changing size when the user zooms. To do this, just resize the plot point polygons so the ratio between the polygon's size and the distance to the camera remains constant. If you've got a lot of plot points, computing the distance to the camera might get expensive because of the square-root involved, but a lookup table would probably solve that.
In addition to resizing, you'll want to keep the plot points facing the camera, so billboarding is your solution, there.
An alternative is to project each of the 3D plot point locations to find out their 2D screen coordinates. Then simply render the polygons at those screen coordinates. No re-scaling necessary. However, gluProject is quite slow, so I'd be very surprised if this wasn't orders of magnitude slower than simply rescaling the plot point polygons like I first suggested.
Good luck!
There's no easy way to do what you want to do. You'll have to dynamically resize the primitives you're drawing depending on the camera's current zoom. You can use a technique known as billboarding to make sure that your objects always face the camera.