Directional shadow mapping in deferred shading - opengl

Question

I'm implementing directional shadow mapping in deferred shading.
First, I render a depth map from light view (orthogonal projection).
Result:
I intend to do VSM so above buffer is R32G32 storing depth and depth * depth.
Then for a full-screen shading pass for shadow (after a lighting pass), I write the following pixel shader:
#version 330
in vec2 texCoord; // screen coordinate
out vec3 fragColor; // output color on the screen
uniform mat4 lightViewProjMat; // lightView * lightProjection (ortho)
uniform sampler2D sceneTexture; // lit scene with one directional light
uniform sampler2D shadowMapTexture;
uniform sampler2D scenePosTexture; // store fragment's 3D position
void main() {
vec3 fragPos = texture(scenePosTexture, texCoord).xyz; // get 3D position of pixel
vec4 fragPosLightSpace = lightViewProjMat * vec4(fragPos, 1.0); // project it to light-space view: lightView * lightProjection
// projective texture mapping
vec3 coord = fragPosLightSpace.xyz / fragPosLightSpace.w;
coord = coord * 0.5 + 0.5;
float lightViewDepth; // depth value in the depth buffer - the maximum depth that light can see
float currentDepth; // depth of screen pixel, maybe not visible to the light, that's how shadow mapping works
vec2 moments; // depth and depth * depth for later variance shadow mapping
moments = texture(shadowMapTexture, coord.xy).xy;
lightViewDepth = moments.x;
currentDepth = fragPosLightSpace.z;
float lit_factor = 0;
if (currentDepth <= lightViewDepth)
lit_factor = 1; // pixel is visible to the light
else
lit_factor = 0; // the light doesn't see this pixel
// I don't do VSM yet, just want to see black or full-color pixels
fragColor = texture(sceneTexture, texCoord).rgb * lit_factor;
}
The rendered result is a black screen, but if I hard coded the lit_factor to be 1, result is:
Basically that's how the sceneTexture looks like.
So I think either my depth value is wrong, which is unlikely, or my projection (light space projection in above shader / projective texture mapping) is wrong. Could you validate it for me?
My shadow map generation code is:
// vertex shader
#version 330 compatibility
uniform mat4 lightViewMat; // lightView
uniform mat4 lightViewProjMat; // lightView * lightProj
in vec3 in_vertex;
out float depth;
void main() {
vec4 vert = vec4(in_vertex, 1.0);
depth = (lightViewMat * vert).z / (500 * 0.2); // 500 is far value, this line tunes the depth precision
gl_Position = lightViewProjMat * vert;
}
// pixel shader
#version 330
in float depth;
out vec2 out_depth;
void main() {
out_depth = vec2(depth, depth * depth);
}

Answer 1

The z component of the fragment shader built in variable gl_FragCoord contains the depth value in range [0.0, 1.0]. This is the value which you shoud store to the depth map:
out_depth = vec2(gl_FragCoord.z, depth * depth);
After the calculation
vec3 fragPos = texture(scenePosTexture, texCoord).xyz; // get 3D position of pixel
vec4 fragPosLightSpace = lightViewProjMat * vec4(fragPos, 1.0); // project it to light-space view: lightView * lightProjection
vec3 ndc_coord = fragPosLightSpace.xyz / fragPosLightSpace.w;
the variable ndc_coord contains a normalized device coordinate, where all components are in range [-1.0, 1.0].
The z component of the normalized device coordiante can be conveted to the depth value (if the depth range is [0.0, 1.0]), by
float currentDepth = ndc_coord.z * 0.5 + 0.5;
This value can be compared to the value from the depth map, because currentDepth and lightViewDepth are calcualted by the same view matrix and projection matrix:
moments = texture(shadowMapTexture, coord.xy).xy;
lightViewDepth = moments.x;
if (currentDepth <= lightViewDepth)
lit_factor = 1; // pixel is visible to the light
else
lit_factor = 0; // the light doesn't see this pixel

Answer 2

This is the depth you store in the shadow map:
depth = (lightViewMat * vert).z / (500 * 0.2);
This is the depth you compare the read back value to:
vec4 fragPosLightSpace = lightViewProjMat * vec4(fragPos, 1.0);
currentDepth = fragPosLightSpace.z;
If fragPos is in world space then I assume lightViewMat * vert == fragPos. You are compressing depth by dividing by 500 * 0.2, but that does not equal to fragPosLightSpace.z.
Hint: Write out the value of currentDepth in one channel and the value from the shadow map in another channel, you can then compare them visually or in RenderDoc or similar.