How can one mix ortographic and perspective projection in openGL?
Some 2d elements have to be drawn in screen space (no scaling, rotation, etc..)
These 2d elements have a z position, they have to appear in front/behind of other 3d elements.
So i set up orographic projection, draw all 2d elements, then setup perspective projection and draw all 3d elements.
The result is that all 2d elements are drawn on top. It seems that the z values from the orto projection and the z values from the perspective projection are not compatible (GL_DEPTH_TEST).
Separately all 2d and all 3d elements work fine, the problem is when i try to mix them.
Does the prespective projection changes the z values? In what way?
Is it possible to use z values from orto projection mixed with z values from perspective projection for depth test, or this whole concept is flawed?
Bare opengl1.5
It seems that the z values from the orto projection and the z values from the perspective projection are not compatible (GL_DEPTH_TEST).
That is indeed the case. Perspective transformation maps the Z values nonlinear to the depth buffer values. The usual way to address this problem is to copy the depth buffer after the perspective pass into a depth texture and use that as an additional input in the fragment shader of the orthographic drawn stuff, reverse the nonlinearity in the depth input and compare the incoming Z coordinate with that; then discard appropriately.
It's also possible to emit linear depth values in the perspective drawn geometry fragment shaders, however the depth nonlinearity of perspective projection has its purpose; without it you loose depth precision where it matters most, close to the point of view.
Related
OpenGL spec:
It says: However, depth values for polygons must be interpolated by (14.10).
Why? Are the z coordinates depth values in camera space? If so, we should use perspective correctly barycentric coordinates to interpolate them, isn't it?(like equation 14.9)
Update:
So the z coordinates are NDC coordinates(which already divided by w). I have a small demo which implement a rasterizer. When I use linear interpolation of the NDC z coordinates, the result is a bit unusual(image below). While I use perspective correctly interpolation of camera z coordinates, the result is ok.
This is the perspective projection matrix I use:
Why? Are the z coordinates depth values in camera space? If so, we should use perspective correctly barycentric coordinates to interpolate them, isn't it?
No, they are not. They are in window space, meaning they already have been divided by w. It is correct that if you wanted to interpolate camrea space z, you would have to apply perspective correction. But for NDC and window space Z this would be wrong - after all, the perspective transformation (as achieved by perspective projection matrix and perspective divide) still maps straight lines to straight lines, and flat trinagles to flat triangles. That's why we use the hyperbolically distorted Z values as depth in the first place. This is also a property that is exploited for the hierarchical depth test optimization. Have a look at my answer here for some more details, including a few diagrams.
I am drawing a stack of decals on a quad. Same geometry, different textures. Z-fighting is the obvious result. I cannot control the rendering order or use glPolygonoffset due to batched rendering. So I adjust depth values inside the vertex shader.
gl_Position = uMVPMatrix * pos;
gl_Position.z += aDepthLayer * uMinStep * gl_Position.w;
gl_Position holds clip coordinates. That means a change in z will move a vertex along its view ray and bring it to the front or push it to the back. For normalized device coordinates the clip coords get divided by gl_Position.w (=-Zclip). As a result the depth buffer does not have linear distribution and has higher resolution towards the near plane. By premultiplying gl_Position.w that should be fixed and I should be able to apply a flat amount (uMinStep) to the NDC.
That minimum step should be something like 1/(2^GL_DEPTH_BITS -1). Or, since NDC space goes from -1.0 to 1.0, it might have to be twice that amount. However it does not work with these values. The minStep is roughly 0.00000006 but it does not bring a texture to the front. Neither when I double that value. If I drop a zero (scale by 10), it works. (Yay, thats something!)
But it does not work evenly along the frustum. A value that brings a texture in front of another while the quad is close to the near plane does not necessarily do the same when the quad is close to the far plane. The same effect happens when I make the frustum deeper. I would expect that behaviour if I was changing eye coordinates, because of the nonlinear z-Buffer distribution. But it seems that premultiplying gl_Position.w is not enough to counter that.
Am I missing some part of the transformations that happen to clip coords? Do I need to use a different formula in general? Do I have to include the depth range [0,1] somehow?
Could the different behaviour along the frustum be a result of nonlinear floating point precision instead of nonlinear z-Buffer distribution? So maybe the calculation is correct, but the minStep just cannot be handled correctly by floats at some point in the pipeline?
The general question: How do I calculate a z-Shift for gl_Position (clip coordinates) that will create a fixed change in the depth buffer later? How can I make sure that the z-Shift will bring one texture in front of another no matter where in the frustum the quad is placed?
Some material:
OpenGL depth buffer faq
https://www.opengl.org/archives/resources/faq/technical/depthbuffer.htm
Same with better readable formulas (but some typos, be careful)
https://www.opengl.org/wiki/Depth_Buffer_Precision
Calculation from eye coords to z-buffer. Most of that happens already when I multiply the projection matrix.
http://www.sjbaker.org/steve/omniv/love_your_z_buffer.html
Explanation about the elements in the projection matrix that turn into the A and B parts in most depth buffer calculation formulas.
http://www.songho.ca/opengl/gl_projectionmatrix.html
I've rendered an 3d object and its 2d projection in the image is correct. However now I want to shift the 2d projected object by some pixels. How do I achieve that?
Note that simply translating the 3d object doesn't work because under perspective projection the 2d projected object could change. My goal is to just shift the 2d object in the image without changing its shape and size.
If you're using the programmable pipeline, you can apply the translation after you applied the projection transformation.
The only thing you have to be careful about is that the transformed coordinates after applying the projection matrix have a w coordinate that will be used for the perspective division. To make the additional translation amount constant in screen space, you'll have to multiply it by w. The key fragments of the vertex shader would look like this:
in vec4 Position;
uniform mat4 ModelViewProjMat;
uniform vec2 TranslationOffset;
void main() {
gl_Position = ModelViewProjMat * Position;
gl_Position.xy += TranslationOffset * gl_Position.w;
}
After the perspective division by w, this will result in a fixed offset.
Another possibility that works with both the programmable and fixed pipeline is that you shift the viewport. Say if the window size is vpWidth times vpHeight, and the offset you want to apply is (xOffset, yOffset), you can set the viewport to:
glViewport(xOffset, yOffset, vpWidth + xOffset, vpHeight + yOffset);
One caveat here is that the geometry will still be clipped by the same view volume, but only be shifted by the viewport transform after clipping was applied. If the geometry would fit completely inside the original viewport, this will work fine. But if the geometry would have been clipped originally, it will still be clipped with the same planes, even though it might actually be inside the window after the shift is applied.
As an addition to Reto Koradi's answer: You don't need shaders and you don't need to modify the viewport you use (which has the clipping issues mentioned in the answer). You can simply modifiy the projection matrix by pre-multiplying some translation (which in effect will be applied last, after the projective transformation):
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glTranstlate(x,y,z); // <- this one was added
glFrustum(...) or gluPerspective(...) or whatever you use
glFrustum and gluPerspective will multiply the current matrix with the projective transfrom matrix they build, that is why one typically loads identity first. However, it doesn't necessarily have to be identity, and this case is one of the rare cases where one should load something else.
Since you want to shift in pixels, but that transformation is applied in clip space, you need some unit conversions. Since the clip space is just the homogenous representation of the normalized device space, where the frustum is [-1,1] in all 3 dimensions (so the viewport is 2x2 units big in that space), you can use the following:
glTranslate(x * 2.0f/viewport_width, y * 2.0f/viewport_height, 0.0f);
to shift the output by (x,y) pixels.
Note that while I wrote this for fixed-function GL, the math will of course work with shaders as well, and you can simply modify the projection matrix used by the shader in the same way.
I usually find matrix libraries building both modelview and cameras matrices from the RUB (right-up-back) vectors, as depicted in these pages:
http://3dengine.org/Right-up-back_from_modelview
http://3dengine.org/Modelview_matrix
Is the RUB tuple just a common standard?
Otherwise, is there a reason the RUB vectors are preferred over any other orientation (such as forward-up-right)?
Particularly if you're using the programmable pipeline, you have almost complete freedom about the coordinate system you work in, and how you transform your geometry. But once all your transformations are applied in the vertex shader (resulting in the vector assigned to gl_Position), there is still a fixed function block in the pipeline between the vertex shader and fragment shader. That fixed function block relies on the transformed vertices being in a well defined coordinate system.
gl_Position is in a coordinate system called "clip coordinates", which then turns into "normalized device coordinates" (NDC) after dividing by the w coordinate of the vector.
Based on the vector in NDC, the fixed function rasterization block generates pixels. It will use the first coordinate to map to the horizontal window direction, and the second coordinate to map to the vertical window direction. The third coordinate will be used to calculate the depth, which can be used for depth testing.
This means that after all transformations are applied, the first coordinate has to be left-right, the second coordinate has to be bottom-up, and the third coordinate has to be front-back (well, it could be back-front if you change the depth test).
If you use a classic setup with modelview and projection matrix, it makes sense to use the modelview matrix to transform the original geometry into this orientation, and then use the projection matrix to apply e.g. a perspective.
I don't think there's anything stopping you from using a different orientation as the result of the modelview transformation, and then include a rotation in the projection matrix to transform the whole thing into the correct clip coordinate space. But I don't see a benefit, and it looks like it would just add unnecessary confusion.
Is there a way using OpenGL or GLUT to project a point from the model-view matrix into an associated texture matrix? If not, is there a commonly used library that achieves this? I want to modify the texture of an object according to a ray cast in 3D space.
The simplest case would be:
A ray is cast which intersects a quad, mapped with a single texture.
The point of intersection is converted to a value in texture space clamped between [0.0,1.0] in the x and y axis.
A 3x3 patch of pixels centered around the rounded value of the resulting texture point is set to an alpha value of 0.( or another RGBA value which is convenient, for the desired effect).
To illustrate here is a more complex version of the question using a sphere, the pink box shows the replaced pixels.
I just specify texture points for mapping in OpenGL, I don't actually know how the pixels are projected onto the sphere. Basically I need to to the inverse of that projection, but I don't quite know how to do that math, especially on more complex shapes like a sphere or an arbitrary convex hull. I assume that you can somehow find a planar polygon that makes up the shape, which the ray is intersecting, and from there the inverse projection of a quad or triangle would be trivial.
Some equations, articles and/or example code would be nice.
There are a few ways you could accomplish what you're trying to do:
Project a world coordinate point into normalized device coordinates (NDCs) by doing the model-view and projection transformation matrix multiplications by yourself (or if you're using old-style OpenGL, call gluProject), and perform the perspective division step. If you use a depth coordinate of zero, this would correspond to intersecting your ray at the imaging plane. The only other correction you'd need to do map from NDCs (which are in the range [-1,1] in x and y) into texture space by dividing the resulting coordinate by two, and then shifting by .5.
Skip the ray tracing all together, and bind your texture as a framebuffer attachment to a framebuffer object, and then render a big point (or sprite) that modifies the colors in the neighborhood of the intersection as you want. You could use the same model-view and projection matrices, and will (probably) only need to update the viewport to match the texture resolution.
So I found a solution that is a little complicated, but does the trick.
For complex geometry you must determine which quad or triangle was intersected, and use this as the plane. The quad must be planar(obviously).
Draw a plane in the identity matrix with dimensions 1x1x0, map the texture on points identical to the model geometry.
Transform the plane, and store the inverse of each transform matrix in a stack
Find the point at which the the plane is intersected
Transform this point using the inverse matrix stack until it returns to identity matrix(it should have no depth(
Convert this point from 1x1 space into pixel space by multiplying the point by the number of pixels and rounding. Or start your 2D combining logic here.