OpenGL ReadPixels (Screenshot) Alpha - opengl

Question

I'm using a Tile Rendering (using glReadPixels) setup to take screenshots of a scene. My output looks correct:
But looking at the alpha channel the pixels are still transparent even if the transparent texture is rendered on top of an opaque texture.
In particular the "inside" of the car should be completely opaque where we see the ceiling, but partially transparent when going through the back windows.
Is there a method for testing the alpha component of each texture at a pixel, not just the closest one?

Answer 1

I think you got some useful direction from the comments and the other answer, but not the solution in full detail. Instead of just giving the result, let me walk through it, so that you know how to figure it out yourself next time.
I'm assuming that you draw your translucent objects back to front, using a GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA blend function. You don't directly mention that, but it's fairly standard, and consistent with what you're seeing. The correct solution will also need an alpha component in the framebuffer, but you have that already, otherwise you would get nothing when reading back the alpha.
To illustrate the whole process, I'm going to use two examples on the way. I'll only list the (R, A) components for the colors, G and B would behave just like R.
Draw a layer with color (R1, 1.0), then a layer with (R2, 0.4) on top of it.
Draw a layer with color (R1, 0.5), then a layer with (R2, 0.4) on top of it.
The background color is (Rb, 0.0), you always want to clear with an alpha value of 0.0 for this kind of blending.
First, let's calculate the result we want to achieve for the colors:
For case 1, drawing the first layer completely covers the background, since it has alpha = 1.0. Then we blend the second layer on top of it. Since it has alpha = 0.4, we keep 60% of the first layer, and add 40% of the second layer. So the color we want is
0.6 * R1 + 0.4 * R2
For case 1, drawing the first layer keeps 50% of the background background, since it has alpha = 0.5. So the color so far is
0.5 * Rb + 0.5 * R1
Then we blend the second layer on top of it. Again we keep 60% of the previous color, and add 40% of the second layer. So the color we want is
0.6 * (0.5 * Rb + 0.5 * R1) + 0.4 * R2
= 0.3 * Rb + 0.3 * R1 + 0.4 * R2
Now, let's figure out what we want the result for alpha to be:
For case 1, our first layer was completely opaque. One way of looking at opacity is as a measure of what fraction of light is absorbed by the object. Once we have a layer that absorbs all the light, anything else we render will not change that. Our total alpha should be
1.0
For case 2, we have one layer that absorbs 50% of the light, and one that absorbs 40% of the remaining light. Since 40% of 50% is 20%, a total of 70% is absorbed. Our total alpha should be
0.7
Using GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA for blending gives you the desired result for the color. But as you noticed, not for alpha. Doing the calculation on the example:
Case 1: Drawing layer 1, SRC_ALPHA is 1.0, the source value is S = (R1, 1.0) and the destination value is D = (Rb, 0.0). So the blend function evaluates as
SRC_ALPHA * S + ONE_MINUS_SRC_ALPHA * D
= 1.0 * (R1, 1.0) + 0.0 * (Rb, 0.0)
= (R1, 1.0)
This is written to the framebuffer, and becomes the destination value for drawing layer 2. The source for layer 2 is (R2, 0.4). Evaluating with 0.4 for SRC_ALPHA gives
SRC_ALPHA * S + ONE_MINUS_SRC_ALPHA * D
= 0.4 * (R2, 0.4) + 0.6 * (R1, 1.0)
= (0.4 * R2 + 0.6 * R1, 0.76)
Case 2: Drawing layer 1, SRC_ALPHA is 0.5, the source value is S = (R1, 0.5) and the destination value is D = (Rb, 0.0). So the blend function evaluates as
SRC_ALPHA * S + ONE_MINUS_SRC_ALPHA * D
= 0.5 * (R1, 0.5) + 0.5 * (Rb, 0.0)
= (0.5 * R1 + 0.5 * Rb, 0.25).
This is written to the framebuffer, and becomes the destination value for drawing layer 2. The source for layer 2 is (R2, 0.4). Evaluating with 0.4 for SRC_ALPHA gives
SRC_ALPHA * S + ONE_MINUS_SRC_ALPHA * D
= 0.4 * (R2, 0.4) + 0.6 * (0.5 * R1 + 0.5 * Rb, 0.25)
= (0.4 * R2 + 0.3 * R1 + 0.3 * Rb, 0.31).
So we confirmed what you already knew: We get the desired colors, but the wrong alphas. How do we fix this? We need a different blend function for alpha. Fortunately OpenGL has glBlendFuncSeparate(), which allows us to do exactly that. All we need to figure out is what blend function to use for alpha. Here is the thought process:
Let's say we already rendered some translucent objects, with a total alpha of A1, which is stored in the framebuffer. What we rendered so far absorbs a fraction A1 of the total light, and lets a fraction 1.0 - A1 pass through. We render another layer with alpha A2 on top of it. This layer absorbs a fraction A2 of the light that passed through before, so it absorbs an additional (1.0 - A1) * A2 of all the light. We need to add this to the amount of light that was already absorbed, so that a total of (1.0 - A1) * A2 + A1 is now absorbed.
All that's left to do is translate that into an OpenGL blend equation. A2 is the source value S, and A1 the destination value D. So our desired alpha result becomes
(1.0 - A1) * A2 + A1
= (1.0 - A1) * S + 1.0 * D
What I called A1 is the alpha value in the framebuffer, which is referred to as DST_ALPHA in the blend function specification. So we use ONE_MINUS_DST_ALPHA to match our source multiplier of 1.0 - A1. We use GL_ONE to match the destination multiplier 1.0.
So the blend function parameters for alpha are (GL_ONE_MINUS_DST_ALPHA, GL_ONE), and the complete blend function call is:
glBlendFuncSeparate(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA,
GL_ONE_MINUS_DST_ALPHA, GL_ONE);
We can double check the math for alpha with the examples one more time:
Case 1: Drawing layer 1, DST_ALPHA is 0.0, the source value is S = (.., 1.0) and the destination value is D = (.., 0.0). So the blend function evaluates as
ONE_MINUS_DST_ALPHA * S + ONE * D
= 1.0 * (.., 1.0) + 1.0 * (.., 0.0)
= (.., 1.0)
This is written to the framebuffer, and becomes the destination value for drawing layer 2. The source for layer 2 is (.., 0.4), and DST_ALPHA is now 1.0. Evaluating the blend equation for layer 2 gives
ONE_MINUS_DST_ALPHA * S + ONE * D
= 0.0 * (.., 0.4) + 1.0 * (.., 1.0)
= (.., 1.0)
We got the desired alpha value of 1.0!
Case 2: Drawing layer 1, DST_ALPHA is 0.0, the source value is S = (.., 0.5) and the destination value is D = (.., 0.0). So the blend function evaluates as
ONE_MINUS_DST_ALPHA * S + ONE * D
= 1.0 * (.., 0.5) + 1.0 * (.., 0.0)
= (.., 0.5)
This is written to the framebuffer, and becomes the destination value for drawing layer 2. The source for layer 2 is (.., 0.4), and DST_ALPHA is now 0.5. Evaluating the blend equation for layer 2 gives
ONE_MINUS_DST_ALPHA * S + ONE * D
= 0.5 * (.., 0.4) + 1.0 * (.., 0.5)
= (.., 0.7)
We got the desired alpha value of 0.7!

Answer 2

Is there a method for testing the alpha component of each texture at a pixel, not just the closest one?
No. OpenGL stores one RGBA value for each pixel; there's no way to get the previous values (as that would need a ton of RAM).
What alpha values get written to the framebuffer depend on your alpha blending equation, which you can set with glBlendFunc or glBlendFuncSeparate. See the blending page on the OpenGL wiki for more info, and this JavaScript app lets you see the effects of various blending modes.