So there is a difference between "column major" and "row major" matrix notation conventions, which everyone seems to say "pick the one you like". OpenGL seems to want it's input matrices represented in memory as "column major" and there is the difference between pre-multiply and post-multiply.
So in a typical 3d graphics system you have transforms for:
modelSourceVertex -> ModelToWorld -> WorldToEye(Camera) -> CameraToClip(projection to 2x2x2 cube) -> ClipToPixels ( to 2d vertices in pixel device space internal to GL set by "viewport" )
One tends to precalculate the sequences that do not change much ie: WorldToClip to optimize.
I see in shaders a number of things.
mat4 MVP = Model * Projection;
gl_Position = modelPosition * MVP;
# **********
gl_position = Projection * Model * modelPosition;
# **********
mat4 MVP = ModelMatrix * ViewMatrix * ProjectionMatrix;
gl_Position = modelPosition * MVP;
# **********
etc
Many of those "crowd sourced" samples seem to be naive and often ModelToWorld and WorldToEye are identity or simply rotated so they will work even if the matrix math is ordered improperly.
My question is, is there a "proper" or "most widely accepted" convention ?
In a world where shader libraries are becoming more common I want to adhere to the "best" way in our system.
Related
The background:
I am writing some terrain visualiser and I am trying to decouple the rendering from the terrain generation.
At the moment, the generator returns some array of triangles and colours, and these are bound in OpenGL by the rendering code (using OpenTK).
So far I have a very simple shader which handles the rotation of the sphere.
The problem:
I would like the application to be able to display the results either as a 3D object, or as a 2D projection of the sphere (let's assume Mercator for simplicity).
I had thought, this would be simple — I should compile an alternative shader for such cases. So, I have a vertex shader which almost works:
precision highp float;
uniform mat4 projection_matrix;
uniform mat4 modelview_matrix;
in vec3 in_position;
in vec3 in_normal;
in vec3 base_colour;
out vec3 normal;
out vec3 colour2;
vec3 fromSphere(in vec3 cart)
{
vec3 spherical;
spherical.x = atan(cart.x, cart.y) / 6;
float xy = sqrt(cart.x * cart.x + cart.y * cart.y);
spherical.y = atan(xy, cart.z) / 4;
spherical.z = -1.0 + (spherical.x * spherical.x) * 0.1;
return spherical;
}
void main(void)
{
normal = vec3(0,0,1);
normal = (modelview_matrix * vec4(in_normal, 0)).xyz;
colour2 = base_colour;
//gl_Position = projection_matrix * modelview_matrix * vec4(fromSphere(in_position), 1);
gl_Position = vec4(fromSphere(in_position), 1);
}
However, it has a couple of obvious issues (see images below)
Saw-tooth pattern where triangle crosses the cut meridian
Polar region is not well defined
3D case (Typical shader):
2D case (above shader)
Both of these seem to reduce to the statement "A triangle in 3-dimensional space is not always even a single polygon on the projection". (... and this is before any discussion about whether great circle segments from the sphere are expected to be lines after projection ...).
(the 1+x^2 term in z is already a hack to make it a little better - this ensures the projection not flat so that any stray edges (ie. ones that straddle the cut meridian) are safely behind the image).
The question: Is what I want to achieve possible with a VertexShader / FragmentShader approach? If not, what's the alternative? I think I can re-write the application side to pre-transform the points (and cull / add extra polygons where needed) but it will need to know where the cut line for the projection is — and I feel that this information is analogous to the modelViewMatrix in the 3D case... which means taking this logic out of the shader seems a step backwards.
Thanks!
shader_type spatial;
void fragment ()
{
vec2 i_resolution = 1.0 / SCREEN_PIXEL_SIZE ;
...
//fragColor = ...;
COLOR = ...; 'Constants cannot be modified' And this is other problem on spatial shader
}
SCREEN_PIXEL_SIZE don't work on spatial shader? how to get the resolution?
As you have noticed SCREEN_PIXEL_SIZE does not exist in Spatial Shader (it exists in Canvas Shaders).
The equivalent of SCREEN_PIXEL_SIZE for Spatial Shaders is 1.0/VIEWPORT_SIZE. That is, VIEWPORT_SIZE = 1.0/SCREEN_PIXEL_SIZE.
Since 1.0/SCREEN_PIXEL_SIZE is what you want, you can use VIEWPORT_SIZE directly. It gives you size in pixels of the viewport (where the shader is being drawn, be it the screen, a window, or a texture).
Furthermore, if you are going to do FRAGCOORD.xy/VIEWPORT_SIZE, you can use SCREEN_UV instead. That is SCREEN_UV = FRAGCOORD.xy/VIEWPORT_SIZE. As you would expect, FRAGCOORD is the fragment coordinates in pixels, while SCREEN_UV gives you normalized coordinates (0 to 1).
By the way, if you don't want the material to depend on the position on screen, you may swap SCREEN_UV for UV.
By the way yes, I know SCREEN_UV says "screen" and not "viewport". Some naming can be confusing.
In your spatial shader you don't write to COLOR, you write ALBEDO and ALPHA instead. Godot will notice if you write to ALPHA or not, and take that into account for deciding render order (so potentially transparent materials are rendered after opaque ones).
Speaking of confusing naming…
mat4 ModelToWorld = WORLD_MATRIX; // Common name: Model Matrix
mat4 WorldToModel = inverse(WORLD_MATRIX); // Common name: Inverse Model Matrix
mat4 WorldToCamera = INV_CAMERA_MATRIX; // Common name: View Matrix
mat4 CameraToWorld = CAMERA_MATRIX; // Common name: Inverse View Matrix
mat4 ModelToCamera = MODELVIEW_MATRIX; // Common name: View Model Matrix
mat4 CameraToModel = inverse(MODELVIEW_MATRIX); // Common name: Inverse View Model Matrix
mat4 CameraToClip = PROJECTION_MATRIX; // Common name: Projection Matrix
mat4 ClipToCamera = INV_PROJECTION_MATRIX; // Common name: Inverse Projection Matrix
Please refer to Spatial Shaders, for the list of built-ins (including those mentioned above) and render modes.
I'm developping a little 3D Engine using OpenGL and GLSL. I currently use Texture Buffer Objects (TBOs) to store all my matrices (Proj, View, Model and Shadow Matrices). But I did some researches on what is the best way to handle matrices (I mean the most efficient way) within a graphic engine, without any success. The goal is to store a maximum of matrices into a minimum number of TBO and occur a minimum of state changes and a minimum of exchanges between the GPU and client code (glBufferSubData).
I propose 2 different methods (with their advantages and disadvantages):
Here's a scene example:
1 Camera (1 ProjMatrix, 1 ViewMatrix)
5 boxes (5 ModelMatrix)
Here's an example of a simple vertex shader I use:
#version 400
/*
** Vertex attributes.
*/
layout (location = 0) in vec4 VertexPosition;
layout (location = 1) in vec2 VertexTexture;
/*
** Uniform matrix buffer.
*/
uniform samplerBuffer matrixBuffer;
/*
** Matrix buffer offset.
*/
uniform int MatrixBufferOffset;
/*
** Output variables.
*/
out vec2 TexCoords;
/*
** Returns matrix4x4 from texture cache.
*/
mat4 Get_Matrix(int offset)
{
return (mat4(texelFetch(
matrixBuffer, offset), texelFetch(
matrixBuffer, offset + 1), texelFetch(matrixBuffer, offset + 2),
texelFetch(matrixBuffer, offset + 3)));
}
/*
** Vertex shader entry point.
*/
void main(void)
{
TexCoords = VertexTexture;
{
mat4 ModelViewProjMatrix = Get_Matrix(
MatrixBufferOffset);
gl_Position = ModelViewProjMatrix * VertexPosition;
}
}
1) The method I currently use: in my vertex shader I use to use ModelViewProjMatrix (needed for rasterization(gl_Position)), ModelViewMatrix (for lighting calculations) and ModelMatrix. So to avoid useless calculation within the vertex shader I've decided to store the ModelViewProjMatrix, the ModelViewMatrix and the ModelMatrix for each mesh node inlined in the TBO as follow:
TBO = {[ModelViewProj_Box1][ModelView_Box1][Model_Box1]|[ModelViewProj_Box2]...}
Advantages: I don't need to compute the product Proj * View * Model (ModelViewProj for example) for each vertex shader (the matrices are pre-calculated).
Disadvantages: if I move the camera I need to update all the ModelViewProj and ModelView matrices. So, a lot of informations to update.
2) I thought about an other way, I think more efficient: store once the projection matrix, once the view matrix and finally each box scene node model matrix once again this way:
TBO = {[ProjMatrix][ViewMatrix][ModelMatrix_Box1][ModelMatrix_Box2]...}
So my vertex shader will look like this:
#version 400
/*
** Vertex attributes.
*/
layout (location = 0) in vec4 VertexPosition;
layout (location = 1) in vec2 VertexTexture;
/*
** Uniform matrix buffer.
*/
uniform samplerBuffer matrixBuffer;
/*
** Matrix buffer offset.
*/
uniform int MatrixBufferOffset;
/*
** Output variables.
*/
out vec2 TexCoords;
/*
** Returns matrix4x4 from texture cache.
*/
mat4 Get_Matrix(int offset)
{
return (mat4(texelFetch(
matrixBuffer, offset), texelFetch(
matrixBuffer, offset + 1), texelFetch(matrixBuffer, offset + 2),
texelFetch(matrixBuffer, offset + 3)));
}
/*
** Vertex shader entry point.
*/
void main(void)
{
TexCoords = VertexTexture;
{
mat4 ProjMatrix = Get_Matrix(MatrixBufferOffset);
mat4 ViewMatrix = Get_Matrix(MatrixBufferOffset + 4);
mat4 ModelMatrix = Get_Matrix(MatrixBufferOffset + 8);
gl_Position = ProjMatrix * ViewMatrix * ModelMatrix * VertexPosition;
}
}
Advantages: The TBO contains the exact number of matrices used. The update is highly targeted (if I move the camera I only updates the view matrix, if I resize the window I only updates the projection matrix and finally if a object is moving only its model matrix will be updated).
Disadvantages: I need to compute fo each vertex within the vertex shader the ModelViewProjMatrix. Plus, if the scene is composed of a huge number of object with each of them owning a different model matrix, I probably need to create a new TBO. Consequently, I will loose the proj/view matrix information because I won't be connect to the right TBO, which bring us to my third method.
3) Store the Projection and View matrix in a TBO and all the other model matrices within another or others TBO(s) as follow:
TBO_0 = {[ProjMatrix][ViewMatrix]}
TBO_1 = {[ModelMatrix_Box1][ModelMatrix_Box2]...}
What do you think of my 3 methods ? Which one is the best for you?
Thanks a lot in advance for your help!
The solution 3 is what most engines do, except they use uniform buffers (constant buffers) instead of texture buffers. Also they don't generally group all the model matrices together in the same buffer, they usually are grouped by object type (because same objects are drawn at once with instancing) and sometimes by frequency of update (objects that never move are in the same buffer so that it never needs to be updated).
Also glBufferSubData can be pretty slow; updating a buffer is often slower than just binding a different one, because of all the synchronization happening inside the driver. There is a very good book chapter about that, freely available on the Internet, called "OpenGL Insights: Asynchronous Buffer Transfers" (Google it to find it).
EDIT: The nvidia article you linked in the comments is very interesting. They recommend using glMultiDrawElements to make several draw calls at once (that's the main trick, everything else is there because of that decision). That can reduce the CPU work in the driver a lot, but that also mean that it's a lot more complicated to provide all the data required to draw the objects: you have to build/update bigger buffers for the matrices / material values and, you also need to use something like bindless textures to be able to have different textures for each object. So, interesting, but more complicated.
And glMultiDrawElements is only important if you want to draw a lot of different objects. Their examples have 68000-98000 different meshes, that's really a lot. In a game, for example, you usually have lots of instances of the same objects, but only a few hundred of different objects (maximum). In the end, it depends on what your 3D engine needs to render.
So I have a .3ds mesh that I imported into my project and it's quite large so I wanted to scale it down in size using glscalef. However the rendering algorithm I use in my shader makes use of normal vector values and so after scaling my rendering algorithm no longer works exactly as it should. So how do remedy this? Is there a glscalef for normals as well?
Normal vectors are transformed by the transposed inverse of the modelview matrix. However a second constraint is, that normals be of unit length and scaling the modelview changes that. So in your shader, you should apply a normalization step
#version 330
uniform mat4x4 MV;
uniform mat4x4 P;
uniform mat4x4 N; // = transpose(inv(MV));
in vec3 vertex_pos; // vertes position attribute
in vec3 vertex_normal; // vertex normal attribute
out vec3 trf_normal;
void main()
{
trf_normal = normalize( (N * vec4(vertex_normal, 1)).xyz );
gl_Position = P * MV * vertex_pos;
}
Note that "normalization" is the process of turning a vector into colinear vector of its own with unit length, and has nothing to do with the concept of surface normals.
To transform normals, you need to multiply them by the inverse transpose of the transformation matrix. There are several explanations of why this is the case, the best one is probably here
Normal is transformed by MV^IT, the inverse transpose of the modelview matrix, read the redbook, it explains everything.
VC++ 2010, OpenGL, GLSL, SDL
I am moving over to shaders, and have run into a problem that originally occured while working with the ogl pipeline. That is, the position of the light seems to point in whatever direction my camera faces. In the ogl pipeline it was just the specular highlight, which was fixable with:
glLightModelf(GL_LIGHT_MODEL_LOCAL_VIEWER, 1.0f);
Here are the two shaders:
Vertex
varying vec3 lightDir,normal;
void main()
{
normal = normalize(gl_NormalMatrix * gl_Normal);
lightDir = normalize(vec3(gl_LightSource[0].position));
gl_TexCoord[0] = gl_MultiTexCoord0;
gl_Position = ftransform();
}
Fragment
varying vec3 lightDir,normal;
uniform sampler2D tex;
void main()
{
vec3 ct,cf;
vec4 texel;
float intensity,at,af;
intensity = max(dot(lightDir,normalize(normal)),0.0);
cf = intensity * (gl_FrontMaterial.diffuse).rgb +
gl_FrontMaterial.ambient.rgb;
af = gl_FrontMaterial.diffuse.a;
texel = texture2D(tex,gl_TexCoord[0].st);
ct = texel.rgb;
at = texel.a;
gl_FragColor = vec4(ct * cf, at * af);
}
Any help would be much appreciated!
The question is: What coordinate system (reference frame) do you want the lights to be in? Probably "the world".
OpenGL's fixed-function pipeline, however, has no notion of world coordinates, because it uses a modelview matrix, which transforms directly from eye (camera) coordinates to model coordinates. In order to have “fixed” lights, you could do one of these:
The classic OpenGL approach is to, every frame, set up the modelview matrix to be the view transform only (that is, be the coordinate system you want to specify your light positions in) and then use glLight to set the position (which is specified to apply the modelview matrix to the input).
Since you are using shaders, you could also have separate model and view matrices and have your shader apply both (rather than using ftransform) to vertices, but only the view matrix to lights. However, this means more per-vertex matrix operations and is probably not an especially good idea unless you are looking for clarity rather than performance.