I have an image of 640x320 and a rectangle (2 triangles) with vertices (0,0) (0,319), (639,0), (639,319).
I want to render using (Opengl 4) the rectangle with the texture without normalize the vertices because I need to work with the same units.
You can treat your coordinates of rectangle in window space you convert them to ndc space
xndc = (xw-x)/(width/2)-1
yndc = (yw-y)/(height/2)-1
x y width and height should be 0 0 640 320
Xw and Yw will be your rectangle vertices.
One by one calculate vertices in NDC space [-1 1] range.
You can now pass them directly to shader to draw no need of any transformation just set proper viewport.
For texture coordinates do same as above which will bring them in [-1 1] range but you will further need to convert them in [0 1] you can use following formula
Uv = Uvndc *0.5 + 0.5
Use this yo bring x and y or uv to range [0 1].
Now you can use this uv to do texture mapping in shader.
Related
I'm using pyopengl to get a depth map.
I am able to get a normalized depth map using glReadPixels(). How can I revert the normalized values to the actual depth in world coordinates?
I've tried playing with glDepthRange(), but it always performs some normalization. Can I disable the normalization at all?
When you draw your geometry, your vertex shader is supposed to transform everything into normalized device coordinates (where each component is between -1 and 1) via the view/projection matrix. There is no way to avoid it, everything outside of this range will get clipped (or clamped, if you enable depth clamping). Then, these device coordinates are transformed into window coordinates - X and Y coordinates are mapped into range specified with glViewport and Z into range set with glDepthRange.
You can't disable normalization, because the final values are required to be in 0..1 range. But you can apply the reverse transformation: first, map your depth values back to -1..1 range (if you didn't use glDepthRange, all you have to do is multiply them by 2 and subtract 1). Then, you need to apply the inverse of your projection matrix - you can either do that explicitly by calculating its inverse, or avoid matrix operations by looking into how your perspective matrix is calculated. For a typical matrix, the inverse transform will be
zNorm = 2 * zBuffer - 1
zView = 2 * near * far / ((far - near) * zNorm - near - far)
(Note that zView will be negative, between -near and -far, because in OpenGL your Z axis normally points towards the camera).
Although normally you don't want only depth - you want the full 3D points, so you might as well reconstruct the vector in normalized coordinates and then apply the inverse projection/view transform.
After the projection to the viewport, the coordinates of the scene are normalized device coordinates (NDC). The normalized device space is a cube, with the left, bottom, front coordinate of (-1, -1, -1) and the right, top, back coordinate of (1, 1, 1). The geometry in this cube is "visible" on the viewport (unless it is covered).
The Z coordinate of the normalized device space, is mapped to the depth range (glDepthRange), which is general in [0, 1].
How the z-coordinate of the view space is transformed to a normalized device Z-coordinate and further a depth, depends on the projection matrix.
While at Orthographic Projection, the Z component is calculated by the linear function, at Perspective Projection, the Z component is calculated by the rational function.
See How to render depth linearly in modern OpenGL with gl_FragCoord.z in fragment shader?.
This means , to convert form the depth of the depth buffer to the original Z-coordinate, the projection (Orthographic or Perspective), and the near plane and far plane has to be known.
In the following is assumed that the depth range is in [0, 1] and depth is a value in this range:
Orthographic Projection
n = near, f = far
z_eye = depth * (f-n) + n;
z_linear = z_eye
Perspective Projection
n = near, f = far
z_ndc = 2 * depth - 1.0;
z_eye = 2 * n * f / (f + n - z_ndc * (f - n));
If the perspective projection matrix is known this can be done as follows:
A = prj_mat[2][2]
B = prj_mat[3][2]
z_eye = B / (A + z_ndc)
Note, in any case transformation by the inverse projection matrix, would transform a normalized device coordinate to a coordinate in view space.
I have to render 2 scenes separately and embed one of them into another scene as a plane. The sub scene that is rendered as a plane will use a view matrix calculated from relative camera position and perspective matrix considering distance and calculated skew to render sub scene as if that scene is placed actually on the point.
For describing more detail, this is a figure to describe the simpler case.
(In this case, we have the sub scene on the center line of the main frustum)
It is easy to calculate perspective matrix visualized as red frustum by using these parameters.
However, it is very difficult for me to solve the other case. If there were the sub scene outside of the center line, I should skew the projection matrix to correspond with scene outside.
I think this is kind of oblique perspective projection. And also this is very similar to render mirror. How do I calculate this perspective matrix?
As #Rabbid76 already pointed out this is just a standard asymmetric frustum. For that, you just need to know the coordinates of the rectangle on the near plane you are going to use, in eye-space.
However, there is also another option: You can also modify the existing projection matrix. That approach will be easier if you know the position of your rectangle in window coordinates or normalized devices coordinates. You can simply pre-multiply scale and translation matrices to select any sub-region of your original frustum.
Let's assume that your viewport is w * h pixels wide, and starts at (0,0) in the window. And you want to create a frustum which just renders a sub-rectangle which starts at the lower left corner of pixel (x,y), and which is a pixels wide and b pixels tall.
Convert to NDC:
x_ndc = (x / w) * 2 - 1 and y_ndc = (y / h) * 2 - 1
a_ndc = (a / w) * 2 and b_ndc = (b / h) * 2
Create a scale and translation transform which maps the range [x_ndc, x_ndc+a_ndc] to [-1,1], and similiar for y:
( 2/a_ndc 0 0 -2*x_ndc/a_ndc-1 )
M = ( 0 2/b_ndc 0 -2*y_ndc/b_ndc-1 )
( 0 0 1 0 )
( 0 0 0 1 )
(note that the factor 2 is going to be cancled out. Instead of going to [-1,1] NDC space in step 1, we could also just have used the normalized [0,1], I just wanted to use the standard spaces.)
Pre-Multiply M to the original projection matrix P:
P' = M * P
Note that even though we defined the transformation in NDC space, and P works in clip space before the division, the math still will work out. By using the homogenous coordinates, the translation part of M will be scaled by w accordingly. The resulting matrix will just be a general asymmetric projection matrix.
Now this does not adjust the near and far clipping planes of the original projection. But you can adjust them in the very same way by adding appropriate scale and translation to the z coordinate.
Also note that using this approach, you are not even restricted to selecting an axis-parallel rectangle, you can also rotate or skew it arbitrarily, so basically, you can select an arbitrary parallelogram in window space.
How do I calculate this perspective matrix?
An asymmetric perspective (column major order) projection matrix is set up like this:
m[16] = [
2*n/(r-l), 0, 0, 0,
0, 2*n/(t-b), 0, 0,
(r+l)/(r-l), (t+b)/(t-b), -(f+n)/(f-n), -1,
0, 0, -2*f*n/(f-n), 0];
Where r, l, b, and t are the left, right, bottom and top distances to the frustum planes on the near plane. n and f are the distances to the near and far plane.
In common, in a framework or a library a projection matrix like this is set up by a function called frustum.
e.g.
OpenGL Mathematics: glm::frustum
OpenGL fixed function pipeline: glFrustum
Given 4 points in eye coordinates (call them ABCD), all of which are in front of the camera (Z<0), how can I calculate a 4x4 projection matrix that will project points ABCD exactly onto the 4 corners of the viewport?
The goal is to make an orthogonal rendering of a rectangular quad which exists somewhere in the scene (with vertices ABCD) as if it was viewed square-on, but where the rendering also includes any objects that block line of sight to the quad from the actual camera location.
I think the answer may involve a homography which maps eye coords to NDC, and I've used OpenCV's cv2.getPerspectiveTransform() to calculate one. But converting the 3x3 homography to a 4x4 projection matrix is proving a challenge.
I can correctly map X and Y from eye coords to NDC using this matrix, but only when A,B,C,D have the same Z coordinate:
H[0,0] H[0,1] 0 H[0,2]
H[1,0] H[1,1] 0 H[1,2]
0 0 0 0
H[2,0] H[2,1] 0 1
I can't work out how to compute Z while keeping X and Y. This handy guide to projection matrices shows that to correctly map Z from [-near,-far] to [-1,1], the bottom rows of the projection matrix should be:
0 0 -(f+n)/(f-n) -2fn/(f-n)
0 0 -1 0
I feel like I'm holding all the pieces of the puzzle but can't work out how to put them together.
The target is to draw a shape, lets say a triangle, pixel-perfect (vertices shall be specified in pixels) and be able to transform it in the 3rd dimension.
I've tried it with a orthogonal projection matrix and everything works fine, but the shape doesn't have any depth - if I rotate it around the Y axis it looks like I would just scale it around the X axis. (because a orthogonal projection obviously behaves like this). Now I want to try it with a perspective projection. But with this projection, the coordinate system changes completely, and due to this I can't specify my triangles verticies with pixels. Also if the size of my window changes, the size of shape changes too (because of the changed coordinate system).
Is there any way to change the coordinate system of the perspective projection so that I can specify my vertices like if I would use the orthogonal projection? Or do anyone have a Idea how to achieve the target described in the first sentence?
The projection matrix describes the mapping from 3D points of a scene, to 2D points of the viewport. It transforms from eye space to the clip space, and the coordinates in the clip space are transformed to the normalized device coordinates (NDC) by dividing with the w component of the clip coordinates. The NDC are in range (-1,-1,-1) to (1,1,1).
At Perspective Projection the projection matrix describes the mapping from 3D points in the world as they are seen from of a pinhole camera, to 2D points of the viewport. The eye space coordinates in the camera frustum (a truncated pyramid) are mapped to a cube (the normalized device coordinates).
Perspective Projection Matrix:
r = right, l = left, b = bottom, t = top, n = near, f = far
2*n/(r-l) 0 0 0
0 2*n/(t-b) 0 0
(r+l)/(r-l) (t+b)/(t-b) -(f+n)/(f-n) -1
0 0 -2*f*n/(f-n) 0
where:
aspect = w / h
tanFov = tan( fov_y * 0.5 );
prjMat[0][0] = 2*n/(r-l) = 1.0 / (tanFov * aspect)
prjMat[1][1] = 2*n/(t-b) = 1.0 / tanFov
I assume that the view matrix is the identity matrix, and thus the view space coordinates are equal to the world coordinates.
If you want to draw a polygon, where the vertex coordinates are translated 1:1 into pixels, then you have to draw the polygon in parallel plane to the viewport. This means all points have to be draw with the same depth. The depth has to choose that way, that the transformation of a point in normalized device coordinates, by the inverse projection matrix gives the vertex coordinates in pixel. Note, the homogeneous coordinates given by the transformation with the inverse projection matrix, have to be divided by the w component of the homogeneous coordinates, to get cartesian coordinates.
This means, that the depth of the plane depends on the field of view angle of the projection:
Assuming you set up a perspective projection like this:
float vp_w = .... // width of the viewport in pixel
float vp_h = .... // height of the viewport in pixel
float fov_y = ..... // field of view angle (y axis) of the view port in degrees < 180°
gluPerspective( fov_y, vp_w / vp_h, 1.0, vp_h*2.0f );
Then the depthZ of the plane with a 1:1 relation of vertex coordinates and pixels, will be calculated like this:
float angRad = fov_y * PI / 180.0;
float depthZ = -vp_h / (2.0 * tan( angRad / 2.0 ));
Note, the center point of the projection to the view port is (0,0), so the bottom left corner point of the plane is (-vp_w/2, -vp_h/2, depthZ) and the top right corner point is (vp_w/2, vp_h/2, depthZ). Ensure, that the near plane of the perspective projetion is less than -depthZ and the far plane is greater than -depthZ.
See further:
Both depth buffer and triangle face orientation are reversed in OpenGL
Transform the modelMatrix
Setting the scene
I'm working on a feature in scenekit where i have a camera at the center of a sphere. The sphere has a texture wrapped around it. Let's say it was a 360 degree image captured inside of a room.
So far
I have identified the positions on the sphere that correspond to the corners of the floor. I can extract and create a new flat 2d plane that matches the dimensions of the floor from the camera's perspective. E.g. If the room had a long rectangular floor, I'd create a trapezoid shaped plane.
Problem
But I would like for the new 2d plane to have the texture of the floor, not just the shape. How do I do this given that what I want to extract is not the original texture image, but the result of its projection onto the sphere?
FYI I'm pretty new to scenekit and 3d graphics stuff and I'm even newer to opengl
I assume that your image is structured in a way that lets you directly pick a pixel given an arbitrary direction. E.g. if the azimuth of the direction is mapped to the image's x-coordinate and the height of the direction to the image's y-coordinate, you would convert the direction to these parameters and pick the color at those coordinates. If that is not the case, you have to find the intersection of the according ray (starting at the camera) with the sphere and find the texture coordinate at that intersection. You can then pick the color using this texture coordinate.
Now, you have basically two options. The first option is generating a new texture for the plane. The second option is sampling the spherical image from a shader.
Option 1 - Generate a new texture
You know the extent of your plane, so you can generate a new texture whose dimensions are proportional to the plane's extents. You can use an arbitrary resolution. All you then need to do is fill the pixels of this texture. For this, you just generate the ray for a given pixel and find the according color in the spherical image like so:
input: d1, d2, d3, d3 (the four direction vectors of the plane corners)
// d3 +------+ d4
// d1 +------+ d2
for x from 0 to texture width
for y from 0 to texture height
//Find the direction vector for this pixel through bilinear interpolation
a = x / (width - 1) //horizontal interpolation parameter
b = y / (height - 1) //vertical interpolation parameter
d = (1 - a) * ((1 - b) * d1 + b * d3) + a * ((1 - b) * d2 + b * d4)
normalize d
//Sample the spherical image at d
color = sample(d)
//write the color to the new planar texture
texture(x, y) = color
next
next
Then, you have a new texture that you can apply to the plane. Barycentric interpolation might be more appropriate if you express the plane as two triangles. But as long as the plane is rectangular, the results will be the same.
Note that the sample() method depends on your image structure and needs to be implemented appropriately.
Option 2 - Sample in a shader
In option 2, you do the same thing as in option 1. But you do it in a fragment shader. You employ the vertices of the plane with their respective directions (this might be just the vertex position) and let the GPU interpolate them. This gives you directly the direction d, which you can use. Here is some pseudo shader code:
in vec3 direction;
out vec4 color;
void main()
{
color = sample(normalize(direction));
}
If your image is a cube map, you can even let the GPU do the sampling.