I'm implementing tangent space normal mapping in my OpenGL app, and I have a few questions.
1) I know that, naturally, the TBN matrix is not always orthogonal because the texture co-ordinates might have skewing. I know that you can re-orthogonalize it using the Gramm-Schmidt process. My question is this - Does the Gramm-Schmidt process introduce visible artifacts? Will I get the best visual quality by using pure unmodified normal/tangent/bitangent?
2) I notice that in a lot of tutorials, they do the lighting calculations in tangent space instead of view space. Why is this? I plan to use a deferred renderer, so my normals have to be saved into a buffer, am I correct in that they should be in view space when saved? What would I be missing out on by using view space instead of tangent space in the calculations? If I'm using view space, do I need the inverse of the TBN matrix?
3) In the fragment shader, do the tangent and bitangent have to be re-normalized? After multiplying the incoming (normalized) bump map normal, does that then have to be renormalized? Would the orthogonality (see question 1) affect this? What vectors do or do not need to be renormalized within the fragment shader?
Does the Gramm-Schmidt process introduce visible artifacts?
It depends on the desired outcome. But the usual answer would be yes, since normal map vectors are in tangent space. Orthogonalizing tangent space will skew normal mapping calculations if the texture coordinates define a nonorthogonal base.
I notice that in a lot of tutorials, they do the lighting calculations in tangent space instead of view space. Why is this?
Doing normal mapping in view space would require each normal map texel to be transformed into view space. This is more expensive, than transforming lighting vectors into tangent space in the vertex shader and let the barycentric interpolation stage (which is far more efficient) to its work.
In the fragment shader, do the tangent and bitangent have to be re-normalized?
No. You should normalize the individual vectors after transforming them (normal map, direction to light source and viewpoint) but before doing the illumination calculations.
Related
I'm implementing the PHONG shading model in OpenGL. I need the normal, viewer direction, and the light direction for each fragment. A lot of demos pass in these vectors in world coordinates from vertex shader. Maybe it's because there isn't much difference between the normalized world coordinate vectors and the normalized perspective coordinate vectors?
I'm thinking for the "true" PHONG solution, these vectors should be transformed to be perspective coordinate system in vertex shader and then perform the .w divide in fragment shader because they are not the gl_position. Is this thinking correct?
Edit:
From this link seems to suggest OpenGl's varying qualifier requires the original 'Z' coordinate of the fragment to perform correct perspective interpolation. See https://www.opengl.org/wiki/Type_Qualifier_(GLSL)#Interpolation_qualifiers
So the question I'm wondering can OpenGL derive the Z-value from the depth value?
Edit: Yes it can. Getting the true z value from the depth buffer
First, you cannot forgo the division-by-W step. Why? Because it's hard-wired. It happens as part of OpenGL's fixed-functionality. The gl_Position your last vertex processing step generates will have its W component divided into the other three.
Now, you could try to trick your way around that, by sticking 1.0 in the gl_Position's W, and passing it as some unrelated output. But the W component is a crucial part of perspective-correct interpolation. By faking your transforms this way, you lose that.
And that's kinda important. So unless you intend to re-interpolate all of your per-vertex outputs in the FS and perform perspective-correct interpolation, this just isn't going to work.
Second, post-projective space, when using a perspective projection, is a non-linear transformation, relative to world space. This means that parallel lines are no longer parallel. This also means that vector directions don't point at what they used to point at. So your light direction doesn't necessarily point at where your light is.
Oh, and distances are not linear either. So light attenuation no longer makes sense, since the attenuation factors were designed in a space linearly equivalent to world space. And post-projection space is not.
Here's an image to give you an idea of what I'm talking about:
What you see on the left is a rendering in world space. What you see on the right is the same scene as on the left, only viewed in post-projection space.
That is not a reasonable space to do lighting in.
I'm currently trying to implement bump mapping which requires to have a "tangent space". I read through some tutorials, specifically the following two:
http://www.terathon.com/code/tangent.html
http://www.opengl-tutorial.org/intermediate-tutorials/tutorial-13-normal-mapping/
Both tutorials avoid expensive matrix computation in the fragment shader which would be required if the shading computation would happen in the camera space as usual (as I'm used to, at least).
They introduce the tangent space which might be different per vertex (or even per fragment if the surface is smoothed). If I understand it correctly, for efficient bump mapping (i.e. to minimize computations in the fragment shader), they convert everything needed for light computation into this tangent space using the vertex shader. But I wonder if model space is a good alternative to compute light shading in.
My questions regarding this topic are:
For shading computation in tangent space, what exactly do I pass between vertex and fragment shaders? Do I really need to convert light positions in tangent space, requiring O(number of lights) of varying variables? This will for example not work for deferred shading or if the light positions aren't known for some other reason in the vertex shader. There has to be a (still efficient) alternative, which I guess is shading computation in model space.
If I pass model space varyings, is it a good idea to still perform shading computations in tangent space, i.e. convert light positions in fragment shader? Or is it better to perform shading computations in model space? Which will be faster? (In both cases I need a TBN matrix, but one case requires a model-to-tangent transform, the other a tangent-to-model transform.)
I currently pass per-vertex normal, tangent and bitangent (orhtonormal) to the vertex shader. If I understand it correctly, the orthonormalization is only required if I want to quickly build a model-to-tangent space matrix which requires inversion of a matrix containing the TBN vectors. If they are orthogonal, this is simply a transposition. But if I don't need vectors in the tangent space, I don't need an inversion but simply the original TBN vectors in a matrix which is then the tangent-to-model matrix. Wouldn't this simplify everything?
Normal mapping is usually done in tangent space because the normal maps are given in this space. So if you pre-transform the (relatively little) input data to tangent space in the vertex shader, you don't need extra computation in the fragment shader. That requires that all input data is available, of course. I haven't done bump mapping with deferred shading, but using the model space seems to be a good idea. World space would probably be even better, because you'll need world space vectors in the end to render to the G-buffers.
If you pass model space vectors, I would recommend to perform the calculations in this space. Then the fragment shader would have to transform one normal from tangent space to model space. In the other case it would have to transform n light attributes from model space to tangent space, which should take n times longer.
If you don't need the inverse TBN matrix, a non-orthonormal coordinate system should be fine. At least I don't see any reason, why it should not.
I am learning OpenGL graphics, and am getting into shadows. The tutorials that I am reading are telling me to transform my normals and light vector to camera space. Why is this? Why can't you just keep the coords in model space?
A follow up question to this is how to handle model transformations. I am unable to find a definitive answer. I currently have this code:
vec3 normCamSpace = normalize(mat3(V) * normal);"
vec3 dirToLight = (V*vec4(lightPos, 0.0)).xyz;"
float cosTheta = clamp(dot(normCamSpace, dirToLight),0,1);"
V is the view matrix, or the camera matrix. I am unsure how to move or edit the light when the model changes in position, rotation, and scale.
The main reason is, that usually your light positions will not be given in model space, but world space. However for illumination to work efficiently all calculations must happen in a common space. In your usual transformation chain, model local coordinates are transformed by the modelview matrix directly into view space
p_view = MV · p_local
Since you normally have only one modelview matrix it would be cumbersome to separate this steap into something like
p_world = M · p_local
p_view = V · p_world
For that you required MV to be separated.
Since the projection transformation traditionally happens as a separate step, view space is the natural "common lower ground" on which illumination calculation to base on. It just involves transforming transforming your light positions from world to view space, and since light positions are not very complex, this is done on the CPU and the pretransformed light positions given as shader.
Note that nothing is stopping you from performing illumination calculations in world space, or model local space. It just takes transforming the light positions correctly.
I am learning OpenGL graphics, and am getting into shadows. The tutorials that I am reading are telling me to transform my normals and light vector to camera space. Why is this? Why can't you just keep the coords in model space?
Actually if you're the one writing the shader, you can use whatever coordinate space you want. IMO calculating lighting in world space feels more "natural", but that's matter of taste.
However, there are two small details:
You cannot "naturally" calculate lighting in object space, if your object is a skinned mesh (character model animated by bones). Such model will require world space or view space. If your object can be only translated and rotated (affine transforms only), then lighting can be easily calculated in model/object space. I think some game engines actualy worked this way.
If you use camera space, you can drop one subtraction when calculating specular highlights. Blinn/phong specular models require vector to(or from) eye to calculate specular factor. In camera space vector from eye to point is equal to point position. This is a very small optimization and it probably isn't worth the effort.
I can't seem to understand the OpenGL pipeline process from a vertex to a pixel.
Can anyone tell me how important are vertex normals on these two shading techinques? As far as i know, in gouraud, lighting is calculated at each vertex, then the result color is interpolated across the polygon between vertices (is this done in fragment operations, before rasterizing?), and phong shading consists of interpolating first the vertices normals and then calculating the illumination on each of these normals.
Another thing is when bump mapping is applied to lets say a plane (2 triangles) and a brick texture as diffuse with its respect bump map, all of this with gouraud shading.
Bump mapping consist on altering the normals by a gradient depending on a bump map. But what normals does it alter and when (at the fragment shader?) if there are only 4 normals (4 vertices = plane), and all 4 are the same. In Gouraud you interpolate the color of each vertex after the illumination calculation, but this calculation is done after altering the normals.
How does the lighting work?
Vertex normals are absoloutely essential for both Gouraud and Phong shading.
In Gouraud shading the lighting is calculated per vertex and then interpolated across the triangle.
In Phong shading the normal is interpolated across the triangle and then the calculation is done per-pixel/fragment.
Bump-mapping refers to a range of different technologies. When doing normal mapping (probably the most common variety these days) the normals, bi-tangent (often erroneously called bi-normal) and tangent are calculated per-vertex to build a basis matrix. This basis matrix is then interpolated across the triangle. The normal retrieved from the normal map is then transformed by this basis matrix and then the lighting is performed per pixel.
There are extensions to the normal mapping technique above that allow bumps to hide other bumps behind them. This is, usually, performed by storing a height map along with the normal map and then ray marching through the height map to find parts that are being obscured. This technique is called Relief Mapping.
There are other older forms such as DUDV bump mapping (Which was implemented in DirectX 6 as Environment Mapped, bump mapping or EMBM).
You also have emboss bump mapping which was a really early way of doing bump mapping
Edit: In answer to your comment, emboss bump mapping CAN be performed on gouraud shaded triangles. Other forms of bump-mapping are, necessarily, per-pixel (due to the fact they work by modifying the surface normals on a per-pixel (or, at least, per-texel) basis). I wouldn't be surprised if there were other methods that can be performed with per-vertex lighting but I can't think of any off the top of my head. The results will look pretty rubbish compared to doing it on a per-pixel basis, though.
Re: Tangents and Bi-Tangents are actually quite simple once you get your head round them (took me years though, tbh ;)). Any 3D coordinate frame can be defined by a set of vectors that form an orthogonal basis matrix. By setting up the normal, tangent and bi-tangent per vertex you are merely setting up the coordinate frame at each vertex. From this you have the ability to transform a world or object space vector into the triangle's own coordinate frame. From here you can simply translate a light vector (or position) into the coordinate frame of a given pixel on the surface of the triangle. This then means that the normals in the normal map don't need to be stored in the object's space and hence as those triangles move around (when being animated, for example) the normals are already being handled in their own local space.
Normal mapping, one of the techniques to simulate bumped surfaces basically perturbs the per-pixel normals before you compute the light equation on that pixel.
For example, one way to implement requires you to interpolate surface normals and binormal (2 of the tangent space basis) and compute the third per-pixel (2+1 vectors which are the tangent basis). This technique also requires to interpolate the light vector. With those 3 (2+1 computed) vectors (named tagent space basis) you have a way to change the light vector from object space into tagent space. This is because these 3 vectors can be arranged as a 3x3 matrix which can be used to change the basis of your light direction vector.
Then it is simply a matter of using that tagent-space light vector and compute the light equation per pixel, where it most basic form would be a dot product between the tagent-space light vector and the normal map (your bump texture).
This is how a normal maps looks like (the normal component is stored in each channel of the texture and is already in tangent space):
This is one way, you can compute things in view space but the above is more easy to understand.
Old bump mapping was way simpler and was also kind of a fake effect.
All bump mapping techniques operate at pixel level, as they perturb in one way or other, how the surface is rendered. Even the old emboss bump mapping did some computation per pixel.
EDIT: I added a few more clarifications, when I have some spare minutes I will try to add some math and examples. Although there are great resources out there that explain this in great detail.
First of all, you don't need to understand the whole graphics pipeline to write a simple shader :). But, of course, you should know whats going on. You could read the graphics pipeline chapter in real-time rendering, 3rd edition (möller, hofmann, akenine-moller). What you describe is per-vertex and per-fragment lighting. For both calculations the vertex normals are part of the equation. For the bump mapping shader you alter the interpolated normals. So after rasterization you have fragments where missing data has to be caculated to determine the final pixel color.
Bump mapping in OpenGL shaders is usually done in tangent space, which has the normal, tangent and binormal as base vectors.
According to my book, OpenGL Shading Language, it is required that the base vectors are consistently oriented across the surface of the object for the lighting equations to interpolate correctly. It also defines that by consistent, it means consistent with respect to the normal map texture coordinates.
So given the vertex positions, normals and normal map texture coordinates for an arbitrary mesh, how can I calculate consistent tangent vectors?
Calculating tangent and bitangent vectors so they orient correctly with texture coordinates, and correctly match the normals is actually fairly complicated.
A good code sample I have used in the past is this one:
http://www.terathon.com/code/tangent.html
Crytek also has a presentation on this topic. Their implementation also solves many common problems with tangent space calculation:
http://crytek.com/cryengine/presentations/triangle-mesh-tangent-space-calculation