I am writing a Phong lighting shader and I have a hard time deciding whether the value I pass to gl_FragColor should be normalized or not.
If I use normalized values, the lighting is a bit weird. For example an object far away from a light source (un-lighted) would have its color determined by the sum of the emissive component, ambient component and global ambiental light. Let us say that adds up to (0.3, 0.3, 0.3). The normal for this is roughly (0.57, 0.57, 0.57), which is quite more luminous than what I'm expecting.
However, if I use non-normalized values, for close objects the specular areas get really, really bright and I have to make sure I generally use low values for my material constants.
As a note, I am normalizing only the RGB component and the alpha component is always 1.
I am a bit swamped and I could not find anything related to this. Either that or my searches were completely wrong.
No. Normalizing the color creates an interesting effect, but I think you don't really want it most, if not all of the time.
Normalization of the color output causes loss of information, even though it may seem to provide greater detail to a scene in some cases. If all your fragments have their color normalized, it means that all RGB vectors have their norm equal to 1. This means that there are colors that simply cannot exist in your output: white (norm = sqrt(3)), bright colors such as yellow (norm = sqrt(2)), dark colors such as dark red (norm(0.5, 0.0, 0.0) = 0.5), etc. Another issue you may face is normalizing zero vectors (i.e. black).
Another way to understand why color normalization is wrong, think about the less general case of rendering a grayscale image. As there is only one color component, normalization does not really make sense at all as it would make all your colors 1.0.
The problem with using the values without normalization arises from the fact that your output image has to have its color values clamped to a fixed interval: [0, 255] or [0.0, 1.0]. As the specular parts of your object reflect more light than those that only reflect diffuse light, quite possibly the computed color value may excede even (1.0, 1.0, 1.0) and get clamped to white for most of the specular area, therefore these areas become, perhaps, too bright.
A simple solution would be to lower the material constant values, or the light intensity. You could go one step further and make sure that the values for the material constants and light intensity are chosen such that the computed color value cannot excede (1.0, 1.0, 1.0). The same result could be achieved with a simple division of the computed color value if consistent values are used for all the materials and all the lights in the scene, but it is kind of overkill, as the scene would probably be too dark.
The more complex, but better looking solution involves HDR rendering and exposure filters such as bloom to obtain more photo-realistic images. This basically means rendering the scene into a float buffer which can handle a greater range than the [0, 255] RGB buffer, then simulating the camera or human eye behavior of adapting to a certain light intensity and the image artefacts caused by this mechanism (i.e. bloom).
Related
Demo https://codepen.io/Andreslav/pen/wvmjzwe
Scheme: Top left - the original.
Top right - the result.
Bottom right - rounding coordinates when extracting color.
The problem can be solved this way, but then the result is less smoothed:
coord = floor(coord) + .5;
How to make it better? Make it so that when calculating the average color, the program ignores the color of transparent pixels?
Maybe there are some settings that I haven't figured out..
Updated the demo
The result is even better after such an adjustment:
vec4 color = texture2D(texture, coord / texSize);
vec4 color_ = texture2D(texture, coordNoRound / texSize);
if(color_.a != color.a) {
color.a *= color_.a;
}
On the preview: bottom left. But this is not an ideal option, the correction is partial. The problem is relevant.
This appears to be a premultiplied alpha problem. And it's not as much of a glsl problem as it is a glfx problem.
Here's what happens:
Consider the RGBA values of two adjacent pixels at the edge of your source image. It would be something like this:
[R G B A ] [R, G, B, A]
[1.0, 1.0, 1.0, 1.0] [?, ?, ?, 0]
Meaning that there is a fully opaque, fully-white pixel to the left, and then comes a fully-transparent (A=0) pixel to the right.
But what are the RGB values of a completely transparent pixel?
They are technically ill-defined (this fact is the core problem which needs to be solved). In practice, pretty much every image processing software will put [0, 0, 0] there.
So the pixels are actually like this:
[R G B A ] [R, G, B, A]
[1.0, 1.0, 1.0, 1.0] [0, 0, 0, 0]
What happens if your swirl shader samples the texture halfway between those 2 pixels? You get [0.5, 0.5, 0.5, 0.5]. That's color [0.5 0.5 0.5], with 0.5 Alpha. Which is gray, not white.
The generally chosen solution to this problem is premultiplied alpha. Which means that, for any given RGBA color, the RGB components are defined so that they don't range from 0 .. 1.0, but instead from 0 .. A. With that definition, color [0.5 0.5 0.5 0.5] is now "0.5 A, with maximum RGB, which is white". One side effect of this definition is that the RGB values of a fully transparent pixel are no longer ill-defined; they must now be exactly [0, 0, 0].
As you can see, we didn't really change any values, instead, we just defined that our result is now correct. Of course, we still need to tell the other parts of the graphics pipeline of our definition.
Premultiplied alpha is not the only solution to the problem. Now that you know what's happening, you might be able to come up with your own solution. But pretty much all modern graphics pipelines expect that you are working with premultiplied alpha all the time. So the correct solution would be to make that true. That means:
(1) You need to make sure that your input texture also has premultiplied alpha, i.e. all its RGB values must be multiplied with their alpha value. This is generally what game engines do, all their textures have premultiplied alpha. Either every pixel must already be edited in the source file, or you do the multiplication once for each pixel in the texture after loading the image.
AND
(2) You need to convince every alpha blending component in your rendering pipeline to use premultiplied alpha blending, instead of "normal" alpha blending. It seems you use the "glfx" framework, I don't know glfx, so I don't know how you can make it blend correctly. Maybe check the docs. In case you are using raw OpenGL/WebGL, then this is the way to tell the pipeline that it should assume premultiplied alpha values when blending:
gl.blendEquation(gl.FUNC_ADD); // Normally not needed because it's the default
gl.blendFunc(gl.ONE, gl.ONE_MINUS_SRC_ALPHA);
(This can be derived from the analyzing the formula for source-over alpha blending, but without the last division step.)
The above code tells OpenGL/WebGL that every time it's blending two pixels on top of another, it should calculate the final RGBA values in a way that's correct assuming that both the "top" and the "bottom" pixel has premultiplied alpha applied to it.
For higher level APIs (for example, GDI+), you can typically specify the pixel format of images, where there is a separation between RGBA and RGBPA, in which case the API will automatically choose correct blending. That may not be true for glfx though. In essence, you always need to be aware whether the pixel format of your textures and drawing surfaces have premultiplied alpha or not, there is no magic code that always works correctly.
(Also note that using premultiplied alpha has other advantages too.)
For a quick fix, it appears that the framework you're using performs alpha blending so that it expects non-premultiplied alpha values. So you could just undo the premultiplication by adding this at the end:
color.rgb /= color.a;
gl_FragColor = color;
But for correctness, you still need to premultiply the alpha values of your input texture.
Because at the rounded corners, your input texture contains pixels which are fully white, but semi-transparent; their RGBA values would look like this:
[1.0, 1.0, 1.0, 0.8]
For the blending code to work correctly, the values should be
[0.8, 0.8, 0.8, 0.8]
,
because otherwise the line color.rgb /= color.a; would give you RGB values greater than 1.
I'm writing a PBR shader and I'm trying to determine the correct way to generate the alpha value that is output for the image. The goal is to have a final image that is 'premultiplied alpha', so it can be used with another Over composite operation later on (say, compositing it over another background image). This all works great, except for the case of specular highlights on transparent surfaces.
Reading this article:
https://google.github.io/filament/Filament.md.html#lighting/transparencyandtranslucencylighting
In particular:
Observe a window and you will see that the diffuse reflectance is
transparent. On the other hand, the brighter the specular reflectance,
the less opaque the window appears. This effect can be seen in figure
63: the scene is properly reflected onto the glass surfaces but the
specular highlight of the sun is bright enough to appear opaque.
They also include the code snippet:
// baseColor has already been premultiplied
vec4 shadeSurface(vec4 baseColor) {
float alpha = baseColor.a;
vec3 diffuseColor = evaluateDiffuseLighting();
vec3 specularColor = evaluateSpecularLighting();
return vec4(diffuseColor + specularColor, alpha);
}
When rendering glass the metallic level would be 0, so it's not pulling from the baseColor at all, and just using the specular level. This all makes sense and means specular highlights can still occur even on the transparent glass, as per the above quote. I don't want to just multiply by alpha at the end, since my baseColor texture would already be a premultiplied alpha image, such as something containing a decal. For example let's say I'm drawing a pane of glass with a sticker on it that's the letter 'A'.
My question is: for a pixel that has a specular highlight on a transparent portion of the glass, what should the alpha value be for compositing to work downstream? The alpha would be 0 according to the above code, which would make an Over operation later on behave poorly. I say this since a property of pre-multiplied alpha is that max(R,G,B) <= A, but this specular highlight would have max(R,G,B) > A. This would result in the over operation being additive between the specular highlight and the background it's being composited over, blowing out the final image.
Should I set the alpha to max(specular, alpha), to give a more useful alpha value for these pixels? Is there a more 'correct' way to handle this?
Thanks
The idea behind the pre-multiplied approach is we assume that the alpha/opacity only affects the diffuse (i.e. the light that goes inside the material/surface), whereas the specular is assumed to be unaffected (at least for the case of a dielectric, the light is simply reflected off the surface un-tinted).
In essence, you should have :
return vec4(diffuseColor * alpha + specularColor, alpha);
Where the alpha is 'pre-multiplied' with the diffuse colour, while the specular colour is left at full intensity (as it should).
Your blend mode will also need to follow suit : off the top of my head one/one for the source/destination colours since the colour will need to be rendered in an additive fashion.
I want to implement gamma correction on my OpenGL 3D renderer, I understand that it's absolutely relevant on texture loaded in sRGB, so I do this:
vec4 texColor;
texColor = texture(src_tex_unit0, texCoordVarying);
texColor = vec4(pow(texColor.rgb,vec3(2.2)),1.0);
vec4 colorPreGamma = texColor * (vec4(ambient,1.0) + vec4(diffuse,1.0));
fragColor = vec4(pow(colorPreGamma.rgb, vec3(1.0/gamma)),1.0);
But my question is about solid color, when the surface of the 3D object I want lit is not textured but just colored by a per vertex RGB value. In this case, do I have to transform my color in Linear space, and after the lighting operation, transform back to gamma space like I do for a texture?
Does this apply when my light are colored?
In this case, do I have to transform my color in Linear space, and after the lighting operation, transform back to gamma space like I do for a texture?
That depends: what colorspace are your colors in?
You're not doing this correction because of where they come from; you're doing it because of what the colors actually are. If the value is not linear, then you must linearize it before using it, regardless of where it comes from.
You are ultimately responsible for putting that color there. So you must have to know whether that color is in linear RGB or sRGB colorspace. And if the color is not linear, then you have to linearize it before you can get meaningful numbers from it.
In OpenGL there isn't a huge distinction between color data and other kinds of data: if you have a vec3 you can access components as .xyz or .rgb. It's all just data.
So ask yourself this: "Do I have to gamma correct my vertex positions in the vertex shader?"
Of course not, because your vertex positions are already in linear space. So if you are similarly setting your vertex colors in linear space, again no gamma correction is needed.
In other words, do you imagine vec3(0.5, 0.5, 0.5) as being a gray that is visually halfway between black and white? Then you need gamma correction.
Do you imagine it as being mathematically halfway between black and white (in terms of measurable light intensity)? Then it's already linear.
I have a scene that is rendered to texture via FBO and I am sampling it from a fragment shader, drawing regions of it using primitives rather than drawing a full-screen quad: I'm conserving resources by only generating the fragments I'll need.
To test this, I am issuing the exact same geometry as my texture-render, which means that the rasterization pattern produced should be exactly the same: When my fragment shader looks up its texture with the varying coordinate it was given it should match up perfectly with the other values it was given.
Here's how I'm giving my fragment shader the coordinates to auto-texture the geometry with my fullscreen texture:
// Vertex shader
uniform mat4 proj_modelview_mat;
out vec2 f_sceneCoord;
void main(void) {
gl_Position = proj_modelview_mat * vec4(in_pos,0.0,1.0);
f_sceneCoord = (gl_Position.xy + vec2(1,1)) * 0.5;
}
I'm working in 2D so I didn't concern myself with the perspective divide here. I just set the sceneCoord value using the clip-space position scaled back from [-1,1] to [0,1].
uniform sampler2D scene;
in vec2 f_sceneCoord;
//in vec4 gl_FragCoord;
in float f_alpha;
out vec4 out_fragColor;
void main (void) {
//vec4 color = texelFetch(scene,ivec2(gl_FragCoord.xy - vec2(0.5,0.5)),0);
vec4 color = texture(scene,f_sceneCoord);
if (color.a == f_alpha) {
out_fragColor = vec4(color.rgb,1);
} else
out_fragColor = vec4(1,0,0,1);
}
Notice I spit out a red fragment if my alpha's don't match up. The texture render sets the alpha for each rendered object to a specific index so I know what matches up with what.
Sorry I don't have a picture to show but it's very clear that my pixels are off by (0.5,0.5): I get a thin, one pixel red border around my objects, on their bottom and left sides, that pops in and out. It's quite "transient" looking. The giveaway is that it only shows up on the bottom and left sides of objects.
Notice I have a line commented out which uses texelFetch: This method works, and I no longer get my red fragments showing up. However I'd like to get this working right with texture and normalized texture coordinates because I think more hardware will support that. Perhaps the real question is, is it possible to get this right without sending in my viewport resolution via a uniform? There's gotta be a way to avoid that!
Update: I tried shifting the texture access by half a pixel, quarter of a pixel, one hundredth of a pixel, it all made it worse and produced a solid border of wrong values all around the edges: It seems like my gl_Position.xy+vec2(1,1))*0.5 trick sets the right values, but sampling is just off by just a little somehow. This is quite strange... See the red fragments? When objects are in motion they shimmer in and out ever so slightly. It means the alpha values I set aren't matching up perfectly on those pixels.
It's not critical for me to get pixel perfect accuracy for that alpha-index-check for my actual application but this behavior is just not what I expected.
Well, first consider dropping that f_sceneCoord varying and just using gl_FragCoord / screenSize as texture coordinate (you already have this in your example, but the -0.5 is rubbish), with screenSize being a uniform (maybe pre-divided). This should work almost exact, because by default gl_FragCoord is at the pixel center (meaning i+0.5) and OpenGL returns exact texel values when sampling the texture at the texel center ((i+0.5)/textureSize).
This may still introduce very very very slight deviations form exact texel values (if any) due to finite precision and such. But then again, you will likely want to use a filtering mode of GL_NEAREST for such one-to-one texture-to-screen mappings, anyway. Actually your exsiting f_sceneCoord approach may already work well and it's just those small rounding issues prevented by GL_NEAREST that create your artefacts. But then again, you still don't need that f_sceneCoord thing.
EDIT: Regarding the portability of texelFetch. That function was introduced with GLSL 1.30 (~SM4/GL3/DX10-hardware, ~GeForce 8), I think. But this version is already required by the new in/out syntax you're using (in contrast to the old varying/attribute syntax). So if you're not gonna change these, assuming texelFetch as given is absolutely no problem and might also be slightly faster than texture (which also requires GLSL 1.30, in contrast to the old texture2D), by circumventing filtering completely.
If you are working in perfect X,Y [0,1] with no rounding errors that's great... But sometimes - especially if working with polar coords, you might consider aligning your calculated coords to the texture 'grid'...
I use:
// align it to the nearest centered texel
curPt -= mod(curPt, (0.5 / vec2(imgW, imgH)));
works like a charm and I no longer get random rounding errors at the screen edges...
From graphics view, is material just the the images applying to geometric object?
We can define material as a set of data which describes how a surface reacts to light.
In the Phong-Blinn shading model, the material is defined by several sets of data:
(rgb) ambient - (see below)
(rgb) diffuse - how strongly it diffuses the incoming light of a given color
(rgb) specular - how well it reflects the incoming light of a given color
(number) shininess - how perfect (how small and focused) is the reflection. Bigger value = smaller "shining spot".
The ambient value is just added to the final color - it is there to emulate "secondary light reflections". It's usually set to the same hue as diffuse, but usually of smaller intensity.
By balancing ambient/diffuse/specular/shininess parameters, you may make the surface resemble different real-world materials.
Also, those parameters may be defined either per-vertex or per-pixel (as a texture). It is common to take the values of ambient and diffuse from a colourful texture and specify constant specular and shininess, but you also could have 3 different textures for ambient, diffuse and specular colours - in order to simulate a sophisticated material which reflects the light in different way depending on the position.
There might be more parameters involved depending on what effects you want to use, for example an additional value for glowing surfaces, etc.
Material usually refers to the colour of the geometric object, while the image is the texture. The material will specify how the object reacts to ambient and direct lighting, it's reflectance, transparency etc.
They can be combined in various ways to produce different effects.
For example the texture might completely override the material so that the underlying colour has no effect on the final scene.
In other cases the texture might be blended with the material so that the same texture effect can be applied to different objects (red car, blue car etc.).