Convert SRGB texture to linear in OpenGL - c++

I am currently trying to properly implement gamma correction in my renderer. I have set my framebuffer to be SRGB with glEnable(GL_FRAMEBUFFER_SRGB) and now I am left with importing the SRGB textures properly. I know three approaches to do this:
Convert value in shader: vec3 realColor = pow(sampledColor, 2.2)
Make OpenGL do it for me: glTexImage2D(..., ...,GL_SRGB, ..., ..., ..., GL_RGB, ..., ...);
Convert the values directly:
for (GLubyte* pixel = image; pixel < image + size; ++pixel)
*pixel = GLubyte(pow(*pixel, 2.2f) + 0.5f);
Now I'm trying to use the third approach, but it doesn't work.
It is super slow (I know it has to loop through all the pixels but still).
It makes everything look completely wrong (see image below).
Here are some images.
No gamma correction:
Method 2 (correction in when sampling in fragment shader)
Something weird when trying method 3
So now my question is what's wrong with method 3 cause it looks completely different from the correct result (assuming that method 2 is correct, which if I think it is).

I have set my framebuffer to be SRGB with glEnable(GL_FRAMEBUFFER_SRGB);
That doesn't set your framebuffer to a sRGB format - it only enables sRGB conversion if the framebuffer is using an sRGB format already - they only use of the GL_FRAMEBUFFER_SRGB enable state is to actually disable sRGB conversion on frambeuffers which have an sRGB format. You still have to specifically request your windows' default framebuffer to be sRGB capabable (or might be lucky to get one without asking for it, but that will differ greatly on implementations and platforms), or you have to create an sRGB texture or render-target if you render to an FBO.
Convert the values directly:
for (GLubyte* pixel = image; pixel < image + size; ++pixel)
*pixel = GLubyte(pow(*pixel, 2.2f) + 0.5f);
First of all pow(x,2.2) is not the correct formula for sRGB - the real one uses a small linear segment near 0 and the power of 2.4 for the rest - using a power of 2.2 is just some further approximation.
However, the bigger problem with this approach is that GLubyte is an 8 Bit unsigned integer type with the range [0,255] and doing a pow(...,2.2) on that yields a value in [0,196964.7], which when converted back to GLubyte will ignore the higher bits and basically calculate the modulo 256, so you will get really useless results. Conceptually, you need 255.0 * pow(x/255.0,2.2) which could of course be further simplified.
The big problem here is that by doing this conversion, you basically loose a lot of precision due to the non-linear distortion of your value range.
If you do such a conversion before-hand, you would have to use higher precision textures to store the linearized color values (like 16 bit half float per channel), just keeping the stuff as 8bit UNORM is a complete disaster - and that is also why GPUs do the conversion directly when accessing the texture, so that you don't have to blow up the memory footprint of your textures by a factor of 2.
So I really doubt that your approach 3 would be "importing the SRGB textures properly". It will just destroy any fidelity even if done right. Approaches1 and 2 do not have that problem, but approach 1 is just silly considering that the hardware will do that for you for free. so I really wonder why you even consider 1 and 3 at all.

Related

Forward rendering multiple rendering passes

I'm trying to implement PBR into my simple OpenGL renderer and trying to use multiple lighting passes, I'm using one pass per light for rendering as follow:
1- First pass = depth
2- Second pass = ambient
3- [3 .. n] for all the lights in the scene.
I'm using the blending function glBlendFunc(GL_ONE, GL_ONE) for passes [3..n], and i'm doing a Gamma Correction at the end of each fragment shader.
But i still have a problem with the output image it just looks noisy specially when i'm using texture maps.
Is there anything wrong with those steps or is there any improvement to this process?
So basically, what you're calculating is
f(x) = a^gamma + b^gamma + ...
However, what you actually want (and as noted by #NicolBolas in the comments already) is
g(x) = (a + b + ...)^gamma
Now f(x) and g(x) will only equal each other in the rather useless cases like gamma=1. You simply cannot additively decompose a nonlinear function like power that way.
The correct solution is to blend everything together in linear space, and doing the gamma correction afterwards, on the total sum of the linear contributions of each light source.
However, implementing this will lead to a couple of technical issues. First and foremost, the standard 8 bit per channel are just not precise enough to store linear color values. Using such a format for the accumulation step will result in strongly visible color banding artifacts. There are two approaches to solve this:
Use a higher bit-per-channel format for the accumulation framebuffer. You will need a separate gamma correction pass, so you need to set up render-to-texture via a FBO. GL_RGBA16F appears as a particularily good format for this. Since you use a PBR lighting model, you can then also work with color values outside [0,1], and instead of a simple gamma correction, apply a proper tone mapping in the final pass. Note that while you may not need an alpha chanell, still use an RGBA format here, the RGB formats are simply not required color buffer formats by the GL spec, so they may not be supported universally.
Store the data still in 8 bit-per-component format, gamma corrected. The key here is that the blending must still be done in linear space, so the destination framebuffer color values must be re-linearized prior to blending. This can be achieved by using a framebuffer with GL_SRGB8_ALPHA8 format and enabling GL_FRAMEBUFFER_SRGB. In that case, the GPU will automatically apply the standard sRGB gamma correction when writing the fragment color to the framebuffer (which currently your fragment shader does), but it will also lead to the sRGB linearization when accessing to those values, including for blending. The OpenGL 4.6 core profile spec states in section "17.3.6.1 Blend equation":
If FRAMEBUFFER_SRGB is enabled and the value of FRAMEBUFFER_ATTACHMENT_COLOR_ENCODING for the framebuffer attachment corresponding
to the destination buffer is SRGB (see section 9.2.3), the R, G, and B destination
color values (after conversion from fixed-point to floating-point) are considered to
be encoded for the sRGB color space and hence must be linearized prior to their
use in blending. Each R, G, and B component is converted in the same fashion
described for sRGB texture components in section 8.24.
Approach 1 will be the much more general approach, while approach 2 has a couple of drawbacks:
the linearization/delinerization is done multiple times, potentially wasting some GPU processing power
due to still using only 8 bit integer, the overall quality will be lower. After each blending step, the results are rounded to the next representable number, so you will get much more quantization noise.
you are still limited to color values in [0,1] and cannot (easily) do more interesting tone mapping and HDR rendering effects
However, approach 2 also has advantages:
you do not need a separate final gamma correction pass
if your platform / window system does support sRGB framebuffers, you can directly create an sRGB pixeformat/visual for your window, and do not need any rende-to-texture step at all. Basically, requesting an sRGB framebuffer and enabling GL_FRAMEBUFFER_SRGB will be enough to make this work.

8bit sRGB and OpenGL confusion

I would like to ask a clarification about the sRGB color space and its 8bit per channel representation in OpenGL. After digging in the subject, I found out that storing images in sRGB is just made to compensate the opposite gamma operation done by the monitor which "imitates" what the old CTR did by nature for compatibility reasons. The fact that the human eye has a similiar non linear response is just a coincidence and has nothing to do with gamma correction (as many confusing articles claim) as the output will be linear again anyway.
Assuming that this understanding is right, in OpenGL we have the GL_SRGB8_ALPHA8 format for textures, which has 8bit per channel. However since the range of 0~255 is the same as a linear RGB texture, does this mean that to convert a linear 8bit per channel texture to sRGB the color values remain unchanged and a simple "flag" tells OpenGL: Look this 0~255 range is not linear so interpret them as a curve?
What about sRGB 16bit per channel images (ex. 16bit PNGs) ?
The range is the same. The values are not.
Your linear values before storing are floating-point. So they have greater precision than an 8-bit-per-channel format.
If you store them in a linearRGB image, you're taking the input range [0, 1] and evenly mapping them to [0, 255].
But if you do sRGB conversion, then you're taking the [0, 1] range and mapping them to [0, 255] via a non-linear gamma mapping of approximately 2.2. While this non-linear mapping does not magically create more values, what it does do is effectively give you more precision in the higher parts of the range than the lower parts.
In sRGB conversion, values in the input range [0.5, 1] are mapped to [56, 255]. That's over 75% of the output range that's covered by 50% of the input range. This gives you a better representation of the larger values in your input.
A linear mapping loses precision evenly. The sRGB mapping loses more precision in the darker areas than the lighter. Or to put it another way, it preserves more precision in the lighter areas than the linear mapping.
For a memory vs. visual quality tradeoff, sRGB comes out better overall than linearRGB 8-bit-per-channel.
However since the range of 0~255 is the same as a linear RGB texture, does this mean that to convert a linear 8bit per channel texture to sRGB the color values remain unchanged and a simple "flag" tells OpenGL: Look this 0~255 range is not linear so interpret them as a curve?
It depends on what operation you're talking about.
An sRGB texture is a texture that stores its RGB information in the sRGB colorspace. However, shader operations are assumed to want data in the linearRGB colorspace, not sRGB. So using an sRGB format means that texture fetches will convert the pixels they read from sRGB to linearRGB.
Writes from a fragment shader to an FBO-attached image using an sRGB format may or may not perform conversion. Here, conversion has to be explicitly enabled with GL_FRAMEBUFFER_SRGB. The idea being that some operations will generate values in the sRGB colorspace (GUIs, for example. Most images were created in the sRGB colorspace), while others will generate values in linearRGB (normal rendering). So you have an option to turn on or off conversion.
The conversion also allows blending to read sRGB destination pixels, convert them to linear, blend with the incoming linearRGB values, and then convert them back to sRGB for writing.
Uploads to and downloads from an sRGB image will write and read the pixel values in the sRGB colorspace directly.
What about sRGB 16bit per channel images (ex. 16bit PNGs) ?
What about them? OpenGL has no 16-bit-per-channel sRGB formats.
sRGB conversion is typically done via a 256-entry table lookup. For every sRGB value, there is a pre-computed linear one.
So, just like any other case where an image format offers something that OpenGL doesn't match, you'll have to manually convert them.
Gamma is Good
I found out that storing images in sRGB is just made to compensate the opposite gamma operation done by the monitor...the fact that the human eye has a similiar non linear response is just a coincidence and has nothing to do with gamma correction ... Assuming that this understanding is right,
That understanding is not correct. Storing images with a gamma curve is necessary to increase the data density in dark areas for perceptual reasons. Without gamma, you would need 12 bits for linear for the same image fidelity you get with 8 bits and a gamma curve.
Old school TVs could have been designed to use a linearized signal, but in fact the design techniques of the day increased gamma from the theoretical 1.5 to the commonly used 2.4. As a function of system design the use of gamma reduced the perception of noise in the broadcast signal.
A gamma-type of transfer curve makes the most of a limited bandwidth by weighting the available bandwidth per human non-linear visual perception.
OPENGL
As for your question, If the internal format parameter is GL_SRGB, GL_SRGB8, GL_SRGB_ALPHA, or GL_SRGB8_ALPHA8, the texture is treated as if the red, green, or blue components are encoded in the sRGB color space.
This means that the values in the image are assumed to be gamma-encoded values as opposed to GL_RGB8 which are assumed to be linear.
However since the range of 0~255 is the same as a linear RGB texture, does this mean that to convert a linear 8bit per channel texture to sRGB the color values remain unchanged and a simple "flag" tells OpenGL: Look this 0~255 range is not linear so interpret them as a curve? What about sRGB 16bit per channel images (ex. 16bit PNGs) ?
The range or bit depth has nothing at all to do with a gamma curve being used or not.
An image needs gamma for perceptual reasons. A linear texture map or bump map does not. If you use the GL_SRGB8 tag on a linear bump map, then GL will use sRGB gamma on that linear data which you do NOT want to do - that is it will apply a power curve of approximately 2.2 to linear 1.0 values, and this is not want you want, unless your bump map IS an image with a gamma curve.
The sRGB tag is there so that when you have an sRGB image which has the color values encoded with a ~1/2.2 curve, those values become linearized.

How to get a floating-point color from GLSL

I am currently faced with a problem closely related to the OpenGL pipeline, and the use of shaders.
Indeed, I work on a project whose one of the steps consists of reading pixels from an image that we generate using OpenGL, with as much accuracy as possible : I mean that instead of reading integers, I would like to read float numbers. (So, instead of reading the value (134, 208, 108) for a pixel, I would like to obtain something like (134.180, 207.686, 108.413), for example.)
For this project, I used both vertex and fragment shaders to render my scene. I assume that the color computed and returned by the fragment shader, is a vector of 4 floats (one per RGBA component) belonging to the "continuous" [0, 1] internal. But, how can I get it in my C++ file ? Is there a way of doing it ?
I thought of calling the glReadPixels() function just after having rendered my scene in a buffer, by setting the format argument to GL_RGBA, and the data type of the pixel data to GL_FLOAT. But I have the feeling that the values associated to the pixels that we read, have already been casted to a integer in the meanwhile, because the float numbers that I finally get, correspond to the interval [0, 255] clamped to [0, 1], without any gain in precision. A closer look on the OpenGL spectifications strengthens this idea : I think there is indeed a cast somewhere between rendering my scene, and callingglReadPixels().
Do you have any idea about the way I can reach my objective ?
The GL_RGBA format returned by the fragment shader stores pixels components in 8-bit integers. You should use a floating point format, such as GL_RGBA16F or GL_RGBA32F, where 16 and 32 are the depths for each component.

GL_FRAMEBUFFER_SRGB_EXT banding problems (gamma correction)

Consider the following code. imageDataf is a float*. In fact, as the code shows it consist of float4 values created by a ray tracer. Of course, the color values are in linear space and I need them gamma corrected for output on screen.
So what I can do is a simple for loop with a gamma correction of 2.2 (see for loop). Also, i can use GL_FRAMEBUFFER_SRGB_EXT, which works almost correclty but has "banding" problems.
Left is using GL_FRAMEBUFFER_SRGB_EXT, right is manual gamma correction. Right picture looks perfect. There may be some difficulties to spot it on some monitors. Does anyone have a clue how to fix this problem? I would like to do gamma correction for "free" as the CPU version makes the GUI a bit laggy. Note that the actual ray tracing is done in another thread using GPU(optix) so in fact its about as fast in rendering performance.
GLboolean sRGB = GL_FALSE;
glGetBooleanv( GL_FRAMEBUFFER_SRGB_CAPABLE_EXT, &sRGB );
if (sRGB) {
//glEnable(GL_FRAMEBUFFER_SRGB_EXT);
}
for(int i = 0; i < 768*768*4; i++)
{
imageDataf[i] = (float)powf(imageDataf[i], 1.0f/2.2f);
}
glPixelStorei(GL_UNPACK_ALIGNMENT, 8);
glDrawPixels( static_cast<GLsizei>( buffer_width ), static_cast<GLsizei>( buffer_height ),
GL_RGBA, GL_FLOAT, (GLvoid*)imageDataf);
//glDisable(GL_FRAMEBUFFER_SRGB_EXT);
When GL_FRAMEBUFFER_SRGB is enabled, this means that OpenGL will assume that the colors for a fragment are in a linear colorspace. Therefore, when it writes them to an sRGB-format image, it will convert them internally from linear to sRGB. Except... your pixels are not linear. You already converted them to a non-linear colorspace.
However, I'll assume that you simply forgot an if statement in there. I'll assume that if the framebuffer is sRGB capable, you skip the loop and upload the data directly. So instead, I'll explain why you're getting banding.
You're getting banding because the OpenGL operation you asked for does the following. For each color you specify:
Clamp the floats to the [0, 1] range.
Convert the floats to unsigned, normalized, 8-bit integers.
Generate a fragment with that unsigned, normalized, 8-bit color.
Convert the unsigned, normalized, 8-bit fragment color from linear RGB space to sRGB space and store it.
Steps 1-3 all come from the use of glDrawPixels. Your problem is step 2. You want to keep your floating-point values as floats. Yet you insist on using the fixed-function pipeline (ie: glDrawPixels), which forces a conversion from float to unsigned normalized integers.
If you uploaded your data to a float texture and used a proper fragment shader to render this texture (even just a simple gl_FragColor = texture(tex, texCoord); shader), you'd be fine. The shader pipeline uses floating-point math, not integer math. So no such conversion would occur.
In short: stop using glDrawPixels.

What exactly is a floating point texture?

I tried reading the OpenGL ARB_texture_float spec, but I still cannot get it in my head..
And how is floating point data related to just normal 8-bit per channel RGBA or RGB data from an image that I am loading into a texture?
Here is a read a little bit here about it.
Basically floating point texture is a texture in which data is of floating point type :)
That is it is not clamped. So if you have 3.14f in your texture you will read the same value in the shader.
You may create them with different numbers of channels. Also you may crate 16 or 32 bit textures depending on the format. e.g.
// create 32bit 4 component texture, each component has type float
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA32F, 16, 16, 0, GL_RGBA, GL_FLOAT, data);
where data could be like this:
float data[16][16];
for(int i=0;i<16*16;++i) data[i] = sin(i*M_PI/180.0f); // whatever
then in shader you can get exactly same (if you use FLOAT32 texture) value.
e.g.
uniform sampler2D myFloatTex;
float value = texture2D(myFloatTex, texcoord.xy);
If you were using 16bit format, say GL_RGBA16F, then whenever you read in shader you will have a convertion. So, to avoid this you may use half4 type:
half4 value = texture2D(my16BitTex, texcoord.xy);
So, basically, difference between the normalized 8bit and floating point texture is that in the first case your values will be brought to [0..1] range and clamped, whereas in latter you will receive your values as is ( except for 16<->32 conversion, see my example above).
Not that you'd probably want to use them with FBO as a render target, in this case you need to know that not all of the formats may be attached as a render target. E.g. you cannot attach Luminance and intensity formats.
Also not all hardware supports filtering of floating point textures, so you need to check it first for your case if you need it.
Hope this helps.
FP textures have a special designated range of internal formats (RGBA_16F,RGBA_32F,etc).
Regular textures store fixed-point data, so reading from them gives you [0,1] range values. Contrary, FP textures give you [-inf,+inf] range as a result (not necessarily with a higher precision).
In many cases (like HDR rendering) you can easily proceed without FP textures, just by transforming the values to fit in [0,1] range. But there are cases like deferred rendering when you may want to store, for example, world-space coordinate without caring about their range.