Texture splatting more than 5 textures - opengl

Question

I'm looking for a solution to store more than 5 diffrent texture in one chunk of a terrain and I've found part of this solution only. I have to pack all my textures (512x512) into one big texture (8192x8192) then in shader I have to calculate texture id using r,g,b values from splat texture. But how to calculate this id from r,g,b values. What is a formula?

Answer 1

So you got map (left texture/image) where ID's of surface type is stored per terrain segment. Than you have texture atlas (right texture) with 16x16=256 textures and want to produce combined texel for terrain coloring (bottom image).
map texture
You did not specify its format and encoding but I suspect that is what you ask for. So you got 256 textures meaning you need 8bit per each ID. If you use 32bit RGBA integer format you can store up to 4 ID's.
encoding/decoding
If you need more then that use either more map textures or lover the number of textures in atlas. For example for 5 IDs you have floor(32/5)=6 bits/ID so you would need to have max 64 textures in your atlas (but still got 2 spare bits left so one ID could still use 256 or two ID's could use 128 textures.)
Of coarse for all this you need integer sampling for map texture so just in case see:
Precise control over texture bits in GLSL
instead of floating r,g,b values. The math equation on integers for 5 IDs GPU decoding from r8g8b8a8 format could look like this (untested just illustrative):
uniform usampler2D tex;
uvec4 a=texture(tex, ...); // fetch 4x8 bits
uint b=0; // construct 32 bit int
b|=uint(a.r);
b|=uint(a.g)<<8;
b|=uint(a.b)<<16;
b|=uint(a.a)<<32;
uint ID1=(b )&0x3F; // decode ID's
uint ID2=(b>> 6)&0x3F;
uint ID3=(b>>12)&0x3F;
uint ID4=(b>>18)&0x3F;
uint ID5=(b>>24)&0x3F;
And similarly CPU encoding:
uint a=0; // construct 32 bit int
a|=(ID1&0x3F);
a|=(ID2&0x3F)<< 6;
a|=(ID3&0x3F)<<12;
a|=(ID4&0x3F)<<18;
a|=(ID5&0x3F)<<24;
map_texture_data[...]=a;
The codes are not tested so there may by reverse RGBA order or wrongly assumed uint bit-widths (they should match the rem-ed values.