Understanding image dithering and how they help blending CSM - opengl

Question

So I wish to implement dithering as a blend mode between my cascade shadow map splits.
I had no idea what they were so I've watched this video to try and understand it.As far as I understand it it's a way to map an image colors to a limited pallet while trying to maintain a convincing gradient between different colored pixels.
Now from this video I understand how to calculate what color my eye will see based on the weights of the dithering pattern. What I do not understand is how we take an image with 4 bytes pixels data and for example trying to map it to 1 byte pixel data. How can we map each pixel color in the original image to a dither pattern that it's weighted average will look as if it's the original color if we're basically limited? Say we were limited to only 5 colors, I'm guessing not every possible weighted average combination of dither pattern using these 5 pallet color could result in the original pixel color so how can this be achieved? Also is a dither pattern is calculated for each pixel to achieve a dithered image?
Besides these general question about image dithering I'm still having difficulties understanding how this technique is helping us blend between cascade splits, where as far as actually implementing it in code, I've seen an example where it uses the space coordinates of a fragment and calculate a dither (Not sure what it's calculating actually because it doesn't return a matrix it returns a float):
float GetDither2(ivec2 p)
{
float d = 0.0;
if((p.x & 1) != (p.y & 1))
d += 2.0;
if((p.y & 1) == 1)
d += 1.0;
d *= 0.25;
return d;
}
float GetDither4(ivec2 p)
{
float d = GetDither2(p);
d = d * 0.25 + GetDither2(p >> 1);
return d;
}
float threshold = GetDither4(ivec2(gl_FragCoord.xy));
if(factor <= threshold)
{
// sample current cascade
}
else
{
// sample next cascade
}
And then it samples either cascade map based on this returned float.
So my brain can't translate what I learned that you can have a dither pattern to simulate large color pattern, into this example that uses a returned float as a threshold factor and compares it to some blend factor just to sample from either shadow map. So it made me more confused.
Would appreciate a good explanation of this 🙏
EDIT:
Ok I see correlation between the algorithm I was provided with to the wikipedia article about ordered dithering, which as far as I understand is the preferred dithering algorithm because according to the article:
Additionally, because the location of the dithering patterns always
stays the same relative to the display frame, it is less prone to
jitter than error-diffusion methods, making it suitable for
animations.
Now I see the code tries to get this threshold value for a given space coordinate although it seems to me it got it a bit wrong because the following calculation of threshold is a follows:
Mpre(i,j) = (Mint(i,j)+1) / n^2
And it needs to set: float d = 1.0 instead of float d = 0.0 if Im not mistaken.
Secondly, I’m not sure how left shifting the ivec2 space coordinate (I’m not even sure what’s the behavior of bitwise shift on vector in glsl…) but I assumes it just component bitwise operation, and I tried plug-in (head calculating) for a given space coordinate (2,1) (according to my assumptions about the bitwise operation) and got different threshold result for what should be the threshold value of this position in a 4x4 Bayer matrix.
So I'm skeptic about how well this code implements the ordered dithering algorithm.
Secondly I’m still not sure how this threshold value has anything to do with choosing between shadow map 1 or 2, and not just reducing color pallet of a given pixel, this logic hasn’t settled in my mind yet as I do not understand the use of dithering threshold value for a given space coordinate to choose the right map to sample from.
Lastly won’t choosing space coordinate will cause jitters? Given fragment in world position (x,y,z) who’s shadowed. Given this fragment space coordinate for a given frame are (i,j). If the camera moves won’t this fragment space coordinate bound to change making the dither threshold calculated for this fragment change with each movement causing jitters of the dither pattern?
EDIT2:
Tried to blend the maps as follow although result not look so good any ideas?
const int indexMatrix8x8[64] = int[](
0, 32, 8, 40, 2, 34, 10, 42,
48, 16, 56, 24, 50, 18, 58, 26,
12, 44, 4, 36, 14, 46, 6, 38,
60, 28, 52, 20, 62, 30, 54, 22,
3, 35, 11, 43, 1, 33, 9, 41,
51, 19, 59, 27, 49, 17, 57, 25,
15, 47, 7, 39, 13, 45, 5, 37,
63, 31, 55, 23, 61, 29, 53, 21
);
for (int i = 0; i < NR_LIGHT_SPACE; i++) {
if (fs_in.v_FragPosClipSpaceZ <= u_CascadeEndClipSpace[i]) {
shadow = isInShadow(fs_in.v_FragPosLightSpace[i], normal, lightDirection, i) * u_ShadowStrength;
int x = int(mod(gl_FragCoord.x, 8));
int y = int(mod(gl_FragCoord.y, 8));
float threshold = (indexMatrix8x8[(x + y * 8)] + 1) / 64.0;
if (u_CascadeBlend >= threshold)
{
shadow = isInShadow(fs_in.v_FragPosLightSpace[i + 1], normal, lightDirection, i + 1) * u_ShadowStrength;
}
}
break;
}
}
Basically if I understand what I'm doing is getting the threshold value from the matrix for each space coordinate of a shadowed pixel and if it's (using probability) higher than a blend factor than I sample the second map instead.
Here're the results:
The larger red box is where the split between map occurs.
The smaller red box goes to show that there's some dither pattern but the image isn't so blended as I think it should.

Answer 1

First of all I have no knowledge about CSM so I focus on dithering and blending. Firstly see these:
my very simple dithering in C++ I come up with
image dithering routine that accepts an amount of dithering? and its variation
They basically answers you question about how to compute the dithering pattern/pixels.
Also its important to have good palette for dithering that reduce your 24/32 bpp into 8 bpp (or less). There are 2 basic approaches
reduce colors (color quantization)
so compute histogram of original image and pick significant colors from it that more or less cover whole image information. For more info see:
Effective gif/image color quantization?
dithering palette
dithering use averaging of pixels to generate desired color so we need to have such colors that can generate all possible colors we want. So its good to have few (2..4) shades of each base color (R,G,B,C,M,Y) and some (>=4) shades of gray. From these you can combine any color and intensity you want (if you have enough pixels)
#1 is the best but it is per image related so you need to compute palette for each image. That can be problem as that computation is nasty CPU hungry stuff. Also on old 256 color modes you could not show 2 different palettes at the same time (which with true color is no more a problem anymore) so dithering is usually better choice.
You can even combine the two for impressive results.
The better the used palette is the less grainy the result is ...
The standard VGA 16 and 256 color palettes where specially designed for dithering so its a good idea to use them...
Standard VGA 16 color palette:
Standard VGA 256 color palette:
Here also C++ code for the 256 colors:
//---------------------------------------------------------------------------
//--- EGA VGA pallete -------------------------------------------------------
//---------------------------------------------------------------------------
#ifndef _vgapal_h
#define _vgapal_h
//---------------------------------------------------------------------------
unsigned int vgapal[256]=
{
0x00000000,0x00220000,0x00002200,0x00222200,
0x00000022,0x00220022,0x00001522,0x00222222,
0x00151515,0x00371515,0x00153715,0x00373715,
0x00151537,0x00371537,0x00153737,0x00373737,
0x00000000,0x00050505,0x00000000,0x00030303,
0x00060606,0x00111111,0x00141414,0x00101010,
0x00141414,0x00202020,0x00242424,0x00202020,
0x00252525,0x00323232,0x00303030,0x00373737,
0x00370000,0x00370010,0x00370017,0x00370027,
0x00370037,0x00270037,0x00170037,0x00100037,
0x00000037,0x00001037,0x00001737,0x00002737,
0x00003737,0x00003727,0x00003717,0x00003710,
0x00003700,0x00103700,0x00173700,0x00273700,
0x00373700,0x00372700,0x00371700,0x00371000,
0x00371717,0x00371727,0x00371727,0x00371737,
0x00371737,0x00371737,0x00271737,0x00271737,
0x00171737,0x00172737,0x00172737,0x00173737,
0x00173737,0x00173737,0x00173727,0x00173727,
0x00173717,0x00273717,0x00273717,0x00373717,
0x00373717,0x00373717,0x00372717,0x00372717,
0x00372525,0x00372531,0x00372536,0x00372532,
0x00372537,0x00322537,0x00362537,0x00312537,
0x00252537,0x00253137,0x00253637,0x00253237,
0x00253737,0x00253732,0x00253736,0x00253731,
0x00253725,0x00313725,0x00363725,0x00323725,
0x00373725,0x00373225,0x00373625,0x00373125,
0x00140000,0x00140007,0x00140006,0x00140015,
0x00140014,0x00150014,0x00060014,0x00070014,
0x00000014,0x00000714,0x00000614,0x00001514,
0x00001414,0x00001415,0x00001406,0x00001407,
0x00001400,0x00071400,0x00061400,0x00151400,
0x00141400,0x00141500,0x00140600,0x00140700,
0x00140606,0x00140611,0x00140615,0x00140610,
0x00140614,0x00100614,0x00150614,0x00110614,
0x00060614,0x00061114,0x00061514,0x00061014,
0x00061414,0x00061410,0x00061415,0x00061411,
0x00061406,0x00111406,0x00151406,0x00101406,
0x00141406,0x00141006,0x00141506,0x00141106,
0x00141414,0x00141416,0x00141410,0x00141412,
0x00141414,0x00121414,0x00101414,0x00161414,
0x00141414,0x00141614,0x00141014,0x00141214,
0x00141414,0x00141412,0x00141410,0x00141416,
0x00141414,0x00161414,0x00101414,0x00121414,
0x00141414,0x00141214,0x00141014,0x00141614,
0x00100000,0x00100004,0x00100000,0x00100004,
0x00100010,0x00040010,0x00000010,0x00040010,
0x00000010,0x00000410,0x00000010,0x00000410,
0x00001010,0x00001004,0x00001000,0x00001004,
0x00001000,0x00041000,0x00001000,0x00041000,
0x00101000,0x00100400,0x00100000,0x00100400,
0x00100000,0x00100002,0x00100004,0x00100006,
0x00100010,0x00060010,0x00040010,0x00020010,
0x00000010,0x00000210,0x00000410,0x00000610,
0x00001010,0x00001006,0x00001004,0x00001002,
0x00001000,0x00021000,0x00041000,0x00061000,
0x00101000,0x00100600,0x00100400,0x00100200,
0x00100303,0x00100304,0x00100305,0x00100307,
0x00100310,0x00070310,0x00050310,0x00040310,
0x00030310,0x00030410,0x00030510,0x00030710,
0x00031010,0x00031007,0x00031005,0x00031004,
0x00031003,0x00041003,0x00051003,0x00071003,
0x00101003,0x00100703,0x00100503,0x00100403,
0x00000000,0x00000000,0x00000000,0x00000000,
0x00000000,0x00000000,0x00000000,0x00000000,
};
//---------------------------------------------------------------------------
class _vgapal_init_class
{
public: _vgapal_init_class();
} vgapal_init_class;
//---------------------------------------------------------------------------
_vgapal_init_class::_vgapal_init_class()
{
int i;
BYTE a;
union { unsigned int dd; BYTE db[4]; } c;
for (i=0;i<256;i++)
{
c.dd=vgapal[i];
c.dd=c.dd<<2;
a =c.db[0];
c.db[0]=c.db[2];
c.db[2]= a;
vgapal[i]=c.dd;
}
}
//---------------------------------------------------------------------------
#endif
//---------------------------------------------------------------------------
//--- end. ------------------------------------------------------------------
//---------------------------------------------------------------------------
Now back to your question about blending by dithering
Blending is merging of 2 images of the same resolution together by some amount (weights). So each pixel color is computed like this:
color = w0*color0 + w1*color1;
where color? are pixels in source images and w? are weights where all weights together sum up to 1:
w0 + w1 = 1;
here example:
Draw tbitmap with scale and alpha channel faster
and preview (the dots are dithering from my GIF encoder):
But Blending by dithering is done differently. Instead of Blending colors we use some percentage of pixels from one image and others from the second one. So:
if (Random()<w0) color = color0;
else color = color1;
Where Random() returns pseudo random number in range <0,1>. As you can see no combinig of colors is done simply you just chose from which image you copy the pixel... Here preview:
Now the dots are caused by the blending by dithering as the intensities of the images are very far away of each other so it does not look good but if you dither relatively similar images (like your shadow maps layers) the result should be good enough (with almost no performance penalty).
To speed up this its usual to precompute the Random() outputs for some box (8x8, 16x16 , ...) and use that for whole image (its a bit blocky but that is sort of used as a fun effect ...). This way it can be done also branchlessly (if you store pointers to source images instead of random value). Also it can be done fully on integers (withou fixed precision) if the weights are integers for example <0..255> ...
Now to make cascade/transition from image0 to image1 or what ever just simply do something like this:
for (w0=1.0;w0>=0.0;w0-=0.05)
{
w1=1.0-w0;
render blended images;
Sleep(100);
}
render image1;

Answer 2

I got the dither blend to work in my code as follows:
for (int i = 0; i < NR_LIGHT_SPACE; i++) {
if (fs_in.v_FragPosClipSpaceZ <= u_CascadeEndClipSpace[i])
{
float fade = fadedShadowStrength(fs_in.v_FragPosClipSpaceZ, 1.0 / u_CascadeEndClipSpace[i], 1.0 / u_CascadeBlend);
if (fade < 1.0) {
int x = int(mod(gl_FragCoord.x, 8));
int y = int(mod(gl_FragCoord.y, 8));
float threshold = (indexMatrix8x8[(x + y * 8)] + 1) / 64.0;
if (fade < threshold)
{
shadow = isInShadow(fs_in.v_FragPosLightSpace[i + 1], normal, lightDirection, i + 1) * u_ShadowStrength;
}
else
{
shadow = isInShadow(fs_in.v_FragPosLightSpace[i], normal, lightDirection, i) * u_ShadowStrength;
}
}
else
{
shadow = isInShadow(fs_in.v_FragPosLightSpace[i], normal, lightDirection, i) * u_ShadowStrength;
}
break;
}
}
First check if we're close to the cascade split by a fading factor taking into account frag position clip space and the end of the cascade clip-space with fadedShadowStrength (I use this function for normal blending between cascade to know when to start blending, basically if blending factor u_CascadeBlend is set to 0.1 for example then we blend when we're atleast 90% into the current cascade (z clip space wise).
Then if we need to fade (if (fade <1.0)) I just compare the fade factor to the threshold from the matrix and choose shadow map accordingly.
Results: