WebGL: Loop index cannot be compared with non-constant expression - glsl

I have the a webgl blur shader:
precision mediump float;
precision mediump int;
uniform sampler2D u_image;
uniform float blur;
uniform int u_horizontalpass; // 0 or 1 to indicate vertical or horizontal pass
uniform float sigma; // The sigma value for the gaussian function: higher value means more blur
// A good value for 9x9 is around 3 to 5
// A good value for 7x7 is around 2.5 to 4
// A good value for 5x5 is around 2 to 3.5
// ... play around with this based on what you need :)
varying vec4 v_texCoord;
const vec2 texOffset = vec2(1.0, 1.0);
// uniform vec2 texOffset;
const float PI = 3.14159265;
void main() {
vec2 p = v_texCoord.st;
float numBlurPixelsPerSide = blur / 2.0;
// Incremental Gaussian Coefficent Calculation (See GPU Gems 3 pp. 877 - 889)
vec3 incrementalGaussian;
incrementalGaussian.x = 1.0 / (sqrt(2.0 * PI) * sigma);
incrementalGaussian.y = exp(-0.5 / (sigma * sigma));
incrementalGaussian.z = incrementalGaussian.y * incrementalGaussian.y;
vec4 avgValue = vec4(0.0, 0.0, 0.0, 0.0);
float coefficientSum = 0.0;
// Take the central sample first...
avgValue += texture2D(u_image, p) * incrementalGaussian.x;
coefficientSum += incrementalGaussian.x;
incrementalGaussian.xy *= incrementalGaussian.yz;
// Go through the remaining 8 vertical samples (4 on each side of the center)
for (float i = 1.0; i <= numBlurPixelsPerSide; i += 1.0) {
avgValue += texture2D(u_image, p - i * texOffset) * incrementalGaussian.x;
avgValue += texture2D(u_image, p + i * texOffset) * incrementalGaussian.x;
coefficientSum += 2.0 * incrementalGaussian.x;
incrementalGaussian.xy *= incrementalGaussian.yz;
}
gl_FragColor = avgValue / coefficientSum;
}
When I build, I get the following error message:
webgl-renderer.js?2eb3:137 Uncaught could not compile shader:ERROR:
0:38: 'i' : Loop index cannot be compared with non-constant expression
I have also tried to use just the uniform float blur to compare i to. Is there any way to fix this?
The problem is further detailed here: https://www.khronos.org/webgl/public-mailing-list/archives/1012/msg00063.php
The solution that I've found looking around is to only use a constant expression when comparing a loop var. This doesn't fit with what I need to do which is vary how many times I'm looping based on the blur radius.
Any thoughts on this?

This happens because on some hardware, GLSL loops are un-rolled into native GPU instructions. This means there needs to be a hard upper limit to the number of passes through the for loop, that governs how many copies of the loop's inner code will be generated. If you replace numBlurPixelsPerSide with a const float or even a #define directive, and the shader compiler can then determine the number of passes at compile time, and generate the code accordingly. But with a uniform there, the upper limit is not known at compile time.
There's an interesting wrinkle in this rule: You're allowed to break or call an early return out of a for loop, even though the max iterations must be discernible at compile time. For example, consider this tiny Mandelbrot shader. This is hardly the prettiest fractal on GLSL Sandbox, but I chose it for its small size:
precision mediump float;
uniform float time;
uniform vec2 mouse;
uniform vec2 resolution;
varying vec2 surfacePosition;
const float max_its = 100.;
float mandelbrot(vec2 z){
vec2 c = z;
for(float i=0.;i<max_its;i++){ // for loop is here.
if(dot(z,z)>4.) return i; // conditional early return here.
z = vec2(z.x*z.x-z.y*z.y,2.*z.x*z.y)+c;
}
return max_its;
}
void main( void ) {
vec2 p = surfacePosition;
gl_FragColor = vec4(mandelbrot(p)/max_its);
}
In this example, max_its is a const so the compiler knows the upper limit and can un-roll this loop if it needs to. Inside the loop, a return statement offers a way to leave the loop early for pixels that are outside of the Mandelbrot set.
You still don't want to set the max iterations too high, as this can produce a lot of GPU instructions and possibly hurt performance.

Try something like this:
const float MAX_ITERATIONS = 100.0;
// Go through the remaining 8 vertical samples (4 on each side of the center)
for (float i = 1.0; i <= MAX_ITERATIONS; i += 1.0) {
if (i >= numBlurPixelsPerSide){break;}
avgValue += texture2D(u_image, p - i * texOffset) * incrementalGaussian.x;
avgValue += texture2D(u_image, p + i * texOffset) * incrementalGaussian.x;
coefficientSum += 2.0 * incrementalGaussian.x;
incrementalGaussian.xy *= incrementalGaussian.yz;
}

Sometimes you can use my very simple solving of issue.
My fragment of the shader source code:
const int cloudPointsWidth = %s;
for ( int i = 0; i < cloudPointsWidth; i++ ) {
//TO DO something
}
You can see '%' : syntax error above. But I am replace %s to a number in my javascript code before use my shader. For example:
vertexCode = vertexCode.replace( '%s', 10 );
vertexCode is my shader source code.
Everytime if I want to change cloudPointsWidth, I am destroying my old shader and creating new shader with new cloudPointsWidth .
Hope sometimes my solving can to help you.

You can just do a for loop with large constant number and use a break.
for(int i = 0; i < 1000000; ++i)
{
// your code here
if(i >= n){
break;
}
}

I've had similar problem with image downsampling shader. The code is basically the same:
for (int dx = -2 * SCALE_FACTOR; dx < 2 * SCALE_FACTOR; dx += 2) {
for (int dy = -2 * SCALE_FACTOR; dy < 2 * SCALE_FACTOR; dy += 2) {
/* accumulate fragment's color */
}
}
What I've ended up doing is using preprocessor and creating separate shader programs for every SCALE_FACTOR used (luckily, only 4 was needed). To achieve that, a small helper function was implemented to add #define ... statements to shader code:
function insertDefines (shaderCode, defines) {
var defineString = '';
for (var define in defines) {
if (defines.hasOwnProperty(define)) {
defineString +=
'#define ' + define + ' ' + defines[define] + '\n';
}
}
var versionIdx = shaderCode.indexOf('#version');
if (versionIdx == -1) {
return defineString + shaderCode;
}
var nextLineIdx = shaderCode.indexOf('\n', versionIdx) + 1;
return shaderCode.slice(0, nextLineIdx) +
defineString +
shaderCode.slice(nextLineIdx);
}
The implementation is a bit tricky because if the code already has #version preprocessor statement in it, all other statements have to follow it.
Then I've added a check for SCALE_FACROR being defined:
#ifndef SCALE_FACTOR
# error SCALE_FACTOR is undefined
#endif
And in my javascript code I've done something like this:
var SCALE_FACTORS = [4, 8, 16, 32],
shaderCode, // the code of my shader
shaderPrograms = SCALE_FACTORS.map(function (factor) {
var codeWithDefines = insertDefines(shaderCode, { SCALE_FACTOR: factor });
/* compile shaders, link program, return */
});

I use opengl es3 on android and solve this problem by using extension above the beginning of program like this:
#extension GL_EXT_gpu_shader5 : require
I don't know whether it work on webGL, but you can try it.
Hope it can help.

You can also use template litterals to set the length of the loop
onBeforeCompile(shader) {
const array = [1,2,3,4,5];
shader.uniforms.myArray = { value: array };
let token = "#include <begin_vertex>";
const insert = `
uniform float myArray[${array.length}];
for ( int i = 0; i < ${array.length}; i++ ) {
float test = myArray[ i ];
}
`;
shader.vertexShader = shader.vertexShader.replace(token, token + insert);
}

Related

How many calculations can a GPU do with a given amount of FLOPS?

I have the following shader rendering voxels using raycasting:
#version 460
#extension GL_ARB_separate_shader_objects : enable
#pragma optionNV(unroll all)
layout(binding = 3, std140) uniform compVarsOb {
float time;
float phiA;
float thetaA;
vec3 camPos;
float fov;
int voxWidth;
int voxHeight;
int voxDepth;
} cvo;
layout(binding = 2, rgba8) uniform writeonly image2D img;
float hash3(vec2 xy){
xy = mod(xy, .19);
float h = dot(xy.yyx, vec3(.013, 27.15, 2027.3));
h *= h;
h *= fract(h);
return fract(h);
}
//layout(binding = 4) uniform sampler3D voxels;
layout(binding = 4, std140) buffer vData{
vec4 voxels[];
};
float greaterThan(float a, float b){
float d = a - b;
return (1. + (d / abs(d)))/2.;
}
float lesserThan(float a, float b){
float d = a - b;
return (1. - (d / abs(d)))/2.;
}
float withinBounds(ivec3 li){
vec3 l = vec3(li);
return greaterThan(l.x, 0.) * lesserThan(l.x, cvo.voxWidth) * greaterThan(l.y, 0.) * lesserThan(l.y, cvo.voxHeight) * greaterThan(l.z, 0.) * lesserThan(l.z, cvo.voxDepth);
}
vec4 quaternionMult(vec4 a, vec4 b){
return vec4(a.x * b.x - dot(a.yzw, b.yzw), a.x*b.yzw + b.x*a.yzw + cross(a.yzw, b.yzw));
}
void main()
{
vec2 iResolution = vec2(2560., 1440.);
vec2 fragCoord = gl_GlobalInvocationID.xy;
ivec2 fragI = ivec2(gl_GlobalInvocationID.xy);
vec2 iMouse = vec2(.5);
vec2 uv = fragCoord/iResolution.xy;
ivec2 uvI = ivec2(uv);
vec2 muv = iMouse.xy / iResolution.xy;
float iTime = cvo.time;
vec3 col = vec3(0.);
float screenRatio = iResolution.y / iResolution.x;
//Setting up the ray directions and other information about the point and camera
//##############################################################################
//camera direction angles phi (xy plane) and theta (xz plane)
float phi = cvo.phiA;//radians(360. * (1. - muv.x));
float theta = cvo.thetaA;//radians(180. * (1. - muv.y));
//get the camera direction as the basis for the rotation (each ray direction is a rotation of the camera direciton vector)
//it is in quarternion form here so its a vec4 instead of a vec3
vec4 camD = vec4(0., cos(phi) * sin(theta), sin(phi) * sin(theta), cos(theta));
float rad90 = radians(90.);
float fov = cvo.fov;
float xAng = radians(fov * (.5 - uv.x));
//replace "fov" with "(fov + (110. * pow(.5 - uv.x, 2.)))" below to add a counteractment to the fisheye lens effect
//it basically counteracts the artifact with quaternions that happens when you rotate by a large angle on one axis then try to rotate on another axis perpendicular, it just rotates around it thus making the new direction lesser
float yAng = radians(fov * screenRatio * (uv.y - .5));
//get the axes that the quarternions should be based around (perpendicular to the camera plane or dv)
vec3 xRotAxis = vec3(cos(phi) * sin(theta - rad90), sin(phi) * sin(theta - rad90), cos(theta - rad90));
vec3 yRotAxis = cross(xRotAxis, camD.yzw);//vec3(cos(phi - rad90) * sin(theta), sin(phi - rad90) * sin(theta), cos(theta));
//get the quarternions of the ray direction rotations
vec4 xQuat = vec4(cos(xAng / 2.), xRotAxis * sin(xAng / 2.));
vec4 yQuat = vec4(cos(yAng / 2.), yRotAxis * sin(yAng / 2.));
//combine the rotations
vec4 compQuat = quaternionMult(yQuat, xQuat);
//get the conjugate of the compQuart
vec4 conjComp = vec4(compQuat.x, -compQuat.yzw);
//ray direction
vec3 rayD = normalize(quaternionMult(quaternionMult(compQuat, camD), conjComp).yzw);
//camera location
vec3 cam = cvo.camPos;//vec3(cos(iTime), 0., 0.);
//point location and radius
//vec3 p = vec3(0., (5. * iTime) + 1., 0.);
float pr = .00001;
//############################################
//hit = 1. means that nothing has been hit or everything has been completely transparent
float hit = 1.;
vec3 locf = vec3(0.);
ivec3 loc = ivec3(0);
int locI = 0;
vec4 v = vec4(0.);
for(int i = 0; i < 20; i++){
locf = vec3((i * rayD * .4) + cam);
loc = ivec3(locf);
//adjust loc for the buffer indexing
locI = loc.x + loc.y * cvo.voxWidth + loc.z * cvo.voxWidth * cvo.voxHeight;
//vec4 v = texelFetch(voxels, loc, 0);//;imageLoad(voxels, loc);//texelFetch(voxels, ivec3((i * rayD) + cam), 0);
v = voxels[locI];// * withinBounds(loc);
if(locf.x < 0. || locf.x > cvo.voxWidth || locf.y < 0. || locf.y > cvo.voxHeight || locf.z < 0. || locf.z > cvo.voxDepth){
v = vec4(0.);
}
col += v.xyz * hit * v.w;
hit -= v.w;
if(hit <= 0.){
//col = vec3(v.w / 5.);
break;
}
}
//col = imageLoad(voxels, ivec3(uvI, 1)).xyz;//texelFetch(voxels, ivec3(fragI / 10, 1), 0).xyz;
//col = vec3(phi / radians(180.));
//col = texture(iChannel0, uv).xyz;
//col = voxels[(fragI.x / 10) + (fragI.y / 10) * cvo.voxWidth].xyz;
//col = vec3(rayD.z);
imageStore(img, fragI, vec4(col,1.0));
}
It produces this:
The problem is when I change the loop (end of main()) for the amount of voxels I want to iterate over to more than 2 (its at 20 right now), the fps absolutely tanks. Yet I feel that my GPU is capable of way more than 2 iterations of a not so demanding loop, so I'm not sure what is going on.
I am running on an RTX 2060 Super which here is said to be capable of 7.81 * 10^12 FLOPS. If I'm understanding it correctly, this means that if I want to run a compute shader at 144 fps at 1440p, I would be allowed a total of (7.81 * 10^12)/(144 * 2560 * 1440) FLOPS in my shader. That comes out to about 14712 FLOPS in the compute shader which is way more than I have in my compute shader right now, yet my code only runs at an average of 30 fps when the loop is at 20 iterations. I can only get 144+ fps when I cut the loop down to 1 or 2 iterations (which at that point is basically like not having the loop at all). Are loops just horribly unoptimized for compute shaders? Where am I going wrong?
The flops GPU is capable of is usually calculated given perfect conditions with perfect code. That is, every computation is a Fused Multiply Add, is able to start right after the previous one with data already in cache/registers, and every single core is working. Achieving such conditions is the problem of writing code on GPUs.
GPUs normally create multiple threads per core in order to reduce penalty from having to wait for memory accesses. Caches are used to work with large bandwith usage, but they are relatively small and they require memory locality(data being stored physically close) for effective usage. Some GPUs have a separate texture cache, abusing which might be wise. Drivers also try to store textures and images in an efficient way, at least by making pixels close by - also close in memory, but potentially using some special hardware.
In compute shaders, threads are created in large blocks and then assigned to Compute Units (terminology differs between vendors, basically sort of a meta-core, with own cache and a bunch of cores). The size of block a is defined in shader with layout(local_size_x = X​, local_size_y = Y​, local_size_z = Z​) in;, where total number of threads equals to the multiple of the dimensions. Threads within a block can communicate with shared variables (they are usually stored within same space as l1 cache) and synchronise with barrier() and alike. GPUs have multiple compute units, and to make them work multiple blocks should be launched(the values in vkCmdDispatch() are the number of groups launched).
Threads are also implicitly grouped in SIMD-like groups (nVidia calls it SIMT - single instruction multiple threads). Every thread in one such group performs the same instruction(but there are some differences with newer nVidia cards). In case where an if() or for(), while(), etc. makes only some threads execute a portion of code, part of cores are disabled, potentially wasting performance. Usually they have size of 32 or 64(AMD), so thread blocks should be created as a multiple of that number. Some functionalities of such groups are exposed with subgroup extensions.
Since you are working with voxels, i'd imagine that using 3D textures or images instead of a buffer may be good for caches. Figuring out how to make threads cooperate and make use of the fast shared memory should also be a good idea.

GLSL-ES3(webGL2): how to test extensions from fragment shader?

In webGL1 it was possible to test the availability of a GLSL extension from a fragment shader using (for instance) #ifdef GL_EXT_shader_texture_lod .
It seems to no longer be working in webGL2 (=GLSL-ES3.0): Extensions are not the same, but for instance #ifdef GL_EXT_color_buffer_float seems false despite https://webglreport.com/?v=2 tells that the extension is there.
Or what am I doing wrong ?
It's up to the extension whether or not it adds a flag to GLSL
EXT_shader_texture_lod is specifically an extension that effects GLSL. It's spec says it adds that macro
The GLSL macro GL_EXT_shader_texture_lod is defined as 1.
EXT_color_buffer_float is not an extension that affects GLSL. It's spec does not mention any GLSL macros. No change from WebGL1
Those flags though are mostly nonsense in WebGL anyway. You can trivially do your own string manipulation
const shaderTextureLodExt = gl.getExtension('EXT_shader_texture_lod');
const shader = `
#if ${shaderTextureLodExt ? 1 : 0}
... code if shader texture lod exists
#else
... code if shader texture lod does not exist
#endif
...
`;
Or a thousand other ways to manipulate shader strings.
Here's another
const colorBufferFloatExt = gl.getExtension('EXT_color_buffer_float');
function replaceIfDefs(s) {
return `
${colorBufferExtension ? '#define EXTENSION_color_buffer_float' : ''}
${s.replace(/GL_EXT_color_buffer_float/g 'EXTENSION_color_buffer_float')}
`;
}
const shader = replaceIfDefs(`
#ifdef GL_EXT_color_buffer_float
...
#endif
...
`);
etc...
Also since there never was a GL_EXT_color_buffer_float even in OpenGL there isn't much point in calling the macro GL_EXT_color_buffer_float. In fact it would arguably be a bad idea because it would end up looking like an official specified macro even though it's not. Best to chose your own name that doesn't start with GL_ .
Also consider that using #ifdef might not even be a good idea since you can just use string manipulation. For example
const colorBufferFloatExt = gl.getExtension('EXT_color_buffer_float');
const snippet = colorBufferFloatExt
? `
float decode_float(vec4 v) {
return v;
}
`
: `
float decode_float(vec4 v) {
vec4 bits = v * 255.0;
float sign = mix(-1.0, 1.0, step(bits[3], 128.0));
float expo = floor(mod(bits[3] + 0.1, 128.0)) * 2.0 +
floor((bits[2] + 0.1) / 128.0) - 127.0;
float sig = bits[0] +
bits[1] * 256.0 +
floor(mod(bits[2] + 0.1, 128.0)) * 256.0 * 256.0;
return sign * (1.0 + sig / 8388607.0) * pow(2.0, expo);
}
`;
const shader = `
precision highp float;
${snippet}
uniform sampler2D data;
uniform vec2 dataSize;
void main(
vec4 d = texture2D(data, gl_FragCoord.xy / dataSize);
vec4 v = decode_float(d) * 2.0;
gl_FragColor = v;
}
`;
...etc...

How to do dynamic loop in WebGL GLSL [duplicate]

I have the a webgl blur shader:
precision mediump float;
precision mediump int;
uniform sampler2D u_image;
uniform float blur;
uniform int u_horizontalpass; // 0 or 1 to indicate vertical or horizontal pass
uniform float sigma; // The sigma value for the gaussian function: higher value means more blur
// A good value for 9x9 is around 3 to 5
// A good value for 7x7 is around 2.5 to 4
// A good value for 5x5 is around 2 to 3.5
// ... play around with this based on what you need :)
varying vec4 v_texCoord;
const vec2 texOffset = vec2(1.0, 1.0);
// uniform vec2 texOffset;
const float PI = 3.14159265;
void main() {
vec2 p = v_texCoord.st;
float numBlurPixelsPerSide = blur / 2.0;
// Incremental Gaussian Coefficent Calculation (See GPU Gems 3 pp. 877 - 889)
vec3 incrementalGaussian;
incrementalGaussian.x = 1.0 / (sqrt(2.0 * PI) * sigma);
incrementalGaussian.y = exp(-0.5 / (sigma * sigma));
incrementalGaussian.z = incrementalGaussian.y * incrementalGaussian.y;
vec4 avgValue = vec4(0.0, 0.0, 0.0, 0.0);
float coefficientSum = 0.0;
// Take the central sample first...
avgValue += texture2D(u_image, p) * incrementalGaussian.x;
coefficientSum += incrementalGaussian.x;
incrementalGaussian.xy *= incrementalGaussian.yz;
// Go through the remaining 8 vertical samples (4 on each side of the center)
for (float i = 1.0; i <= numBlurPixelsPerSide; i += 1.0) {
avgValue += texture2D(u_image, p - i * texOffset) * incrementalGaussian.x;
avgValue += texture2D(u_image, p + i * texOffset) * incrementalGaussian.x;
coefficientSum += 2.0 * incrementalGaussian.x;
incrementalGaussian.xy *= incrementalGaussian.yz;
}
gl_FragColor = avgValue / coefficientSum;
}
When I build, I get the following error message:
webgl-renderer.js?2eb3:137 Uncaught could not compile shader:ERROR:
0:38: 'i' : Loop index cannot be compared with non-constant expression
I have also tried to use just the uniform float blur to compare i to. Is there any way to fix this?
The problem is further detailed here: https://www.khronos.org/webgl/public-mailing-list/archives/1012/msg00063.php
The solution that I've found looking around is to only use a constant expression when comparing a loop var. This doesn't fit with what I need to do which is vary how many times I'm looping based on the blur radius.
Any thoughts on this?
This happens because on some hardware, GLSL loops are un-rolled into native GPU instructions. This means there needs to be a hard upper limit to the number of passes through the for loop, that governs how many copies of the loop's inner code will be generated. If you replace numBlurPixelsPerSide with a const float or even a #define directive, and the shader compiler can then determine the number of passes at compile time, and generate the code accordingly. But with a uniform there, the upper limit is not known at compile time.
There's an interesting wrinkle in this rule: You're allowed to break or call an early return out of a for loop, even though the max iterations must be discernible at compile time. For example, consider this tiny Mandelbrot shader. This is hardly the prettiest fractal on GLSL Sandbox, but I chose it for its small size:
precision mediump float;
uniform float time;
uniform vec2 mouse;
uniform vec2 resolution;
varying vec2 surfacePosition;
const float max_its = 100.;
float mandelbrot(vec2 z){
vec2 c = z;
for(float i=0.;i<max_its;i++){ // for loop is here.
if(dot(z,z)>4.) return i; // conditional early return here.
z = vec2(z.x*z.x-z.y*z.y,2.*z.x*z.y)+c;
}
return max_its;
}
void main( void ) {
vec2 p = surfacePosition;
gl_FragColor = vec4(mandelbrot(p)/max_its);
}
In this example, max_its is a const so the compiler knows the upper limit and can un-roll this loop if it needs to. Inside the loop, a return statement offers a way to leave the loop early for pixels that are outside of the Mandelbrot set.
You still don't want to set the max iterations too high, as this can produce a lot of GPU instructions and possibly hurt performance.
Try something like this:
const float MAX_ITERATIONS = 100.0;
// Go through the remaining 8 vertical samples (4 on each side of the center)
for (float i = 1.0; i <= MAX_ITERATIONS; i += 1.0) {
if (i >= numBlurPixelsPerSide){break;}
avgValue += texture2D(u_image, p - i * texOffset) * incrementalGaussian.x;
avgValue += texture2D(u_image, p + i * texOffset) * incrementalGaussian.x;
coefficientSum += 2.0 * incrementalGaussian.x;
incrementalGaussian.xy *= incrementalGaussian.yz;
}
Sometimes you can use my very simple solving of issue.
My fragment of the shader source code:
const int cloudPointsWidth = %s;
for ( int i = 0; i < cloudPointsWidth; i++ ) {
//TO DO something
}
You can see '%' : syntax error above. But I am replace %s to a number in my javascript code before use my shader. For example:
vertexCode = vertexCode.replace( '%s', 10 );
vertexCode is my shader source code.
Everytime if I want to change cloudPointsWidth, I am destroying my old shader and creating new shader with new cloudPointsWidth .
Hope sometimes my solving can to help you.
You can just do a for loop with large constant number and use a break.
for(int i = 0; i < 1000000; ++i)
{
// your code here
if(i >= n){
break;
}
}
I've had similar problem with image downsampling shader. The code is basically the same:
for (int dx = -2 * SCALE_FACTOR; dx < 2 * SCALE_FACTOR; dx += 2) {
for (int dy = -2 * SCALE_FACTOR; dy < 2 * SCALE_FACTOR; dy += 2) {
/* accumulate fragment's color */
}
}
What I've ended up doing is using preprocessor and creating separate shader programs for every SCALE_FACTOR used (luckily, only 4 was needed). To achieve that, a small helper function was implemented to add #define ... statements to shader code:
function insertDefines (shaderCode, defines) {
var defineString = '';
for (var define in defines) {
if (defines.hasOwnProperty(define)) {
defineString +=
'#define ' + define + ' ' + defines[define] + '\n';
}
}
var versionIdx = shaderCode.indexOf('#version');
if (versionIdx == -1) {
return defineString + shaderCode;
}
var nextLineIdx = shaderCode.indexOf('\n', versionIdx) + 1;
return shaderCode.slice(0, nextLineIdx) +
defineString +
shaderCode.slice(nextLineIdx);
}
The implementation is a bit tricky because if the code already has #version preprocessor statement in it, all other statements have to follow it.
Then I've added a check for SCALE_FACROR being defined:
#ifndef SCALE_FACTOR
# error SCALE_FACTOR is undefined
#endif
And in my javascript code I've done something like this:
var SCALE_FACTORS = [4, 8, 16, 32],
shaderCode, // the code of my shader
shaderPrograms = SCALE_FACTORS.map(function (factor) {
var codeWithDefines = insertDefines(shaderCode, { SCALE_FACTOR: factor });
/* compile shaders, link program, return */
});
I use opengl es3 on android and solve this problem by using extension above the beginning of program like this:
#extension GL_EXT_gpu_shader5 : require
I don't know whether it work on webGL, but you can try it.
Hope it can help.
You can also use template litterals to set the length of the loop
onBeforeCompile(shader) {
const array = [1,2,3,4,5];
shader.uniforms.myArray = { value: array };
let token = "#include <begin_vertex>";
const insert = `
uniform float myArray[${array.length}];
for ( int i = 0; i < ${array.length}; i++ ) {
float test = myArray[ i ];
}
`;
shader.vertexShader = shader.vertexShader.replace(token, token + insert);
}

Deriving uncertainty values from a noise texture?

I'm trying to implement Sketchy Drawings. I'm at the part of the process which calls for the use of the noise texture to derive uncertainty values that will provide an offset into the edge map.
Here is a picture of my edge map for a torus:
And here is the noise texture I've gotten using the Perlin function as suggested:
I have these saved as textures in edgeTexture and noiseTexture respectively.
Now I'm stuck on the section where you have to offset the texture coordinates of the edge map by uncertainty values derived from the noise texture. This image is from the book:
offs = turbulence(s, t);
offt = turbulence(1 - s, 1 - t);
I'm ignoring the 2x2 matrix for the time being. Here is my current fragment shader attempt and the result it produces:
#version 330
out vec4 vFragColor;
uniform sampler2D edgeTexture;
uniform sampler2D noiseTexture;
smooth in vec2 vTexCoords;
float turbulence(float s, float t)
{
float sum = 0;
float scale = 1;
float s1 = 1;
vec2 coords = vec2(s,t);
for (int i=0; i < 10; i++)
{
vec4 noise = texture(noiseTexture, 0.25 * s1 * coords);
sum += scale * noise.x;
scale = scale / 2;
s1 = s1 * 2;
}
return sum;
}
void main( void )
{
float off_s = turbulence(vTexCoords.s, vTexCoords.t);
float off_t = turbulence(1 - vTexCoords.s, 1 - vTexCoords.t);
vFragColor = texture(edgeTexture, vTexCoords + vec2(off_s, off_t));
}
Clearly my addition to the vTexCoords is way off, but I can't see why. I have tried several other turbulence function definitions but none were close to the desired output so I'm thinking my overall approach is flawed somewhere. Any help here is greatly appreciated, and please comment if I haven't been clear. The desired output for a torus would just look like a roughly drawn circle I would imagine.
Your turbulence function will return values in the range (0,1). Firstly you need to change this to get values centered on 0. This should be done inside the loop in the function or you'll end up with a strange distribution. So firstly, I think you should change the line:
vec4 noise = texture(noiseTexture, 0.25 * s1 * coords);
to
vec4 noise = texture(noiseTexture, 0.25 * s1 * coords) * 2.0 - 1.0;
You then need to scale the offset so that you're not sampling the edge texture too far away from the fragment being drawn. Change:
vFragColor = texture(edgeTexture, vTexCoords + vec2(off_s, off_t));
to
vFragColor = texture(edgeTexture, vTexCoords + vec2(off_s, off_t) * off_scale);
where off_scale is some small value (perhaps around 0.05) chosen by experimentation.

Gaussian-distributed pseudo-random number generator in GLSL [duplicate]

As the GPU driver vendors don't usually bother to implement noiseX in GLSL, I'm looking for a "graphics randomization swiss army knife" utility function set, preferably optimised to use within GPU shaders. I prefer GLSL, but code any language will do for me, I'm ok with translating it on my own to GLSL.
Specifically, I'd expect:
a) Pseudo-random functions - N-dimensional, uniform distribution over [-1,1] or over [0,1], calculated from M-dimensional seed (ideally being any value, but I'm OK with having the seed restrained to, say, 0..1 for uniform result distribution). Something like:
float random (T seed);
vec2 random2 (T seed);
vec3 random3 (T seed);
vec4 random4 (T seed);
// T being either float, vec2, vec3, vec4 - ideally.
b) Continous noise like Perlin Noise - again, N-dimensional, +- uniform distribution, with constrained set of values and, well, looking good (some options to configure the appearance like Perlin levels could be useful too). I'd expect signatures like:
float noise (T coord, TT seed);
vec2 noise2 (T coord, TT seed);
// ...
I'm not very much into random number generation theory, so I'd most eagerly go for a pre-made solution, but I'd also appreciate answers like "here's a very good, efficient 1D rand(), and let me explain you how to make a good N-dimensional rand() on top of it..." .
For very simple pseudorandom-looking stuff, I use this oneliner that I found on the internet somewhere:
float rand(vec2 co){
return fract(sin(dot(co, vec2(12.9898, 78.233))) * 43758.5453);
}
You can also generate a noise texture using whatever PRNG you like, then upload this in the normal fashion and sample the values in your shader; I can dig up a code sample later if you'd like.
Also, check out this file for GLSL implementations of Perlin and Simplex noise, by Stefan Gustavson.
It occurs to me that you could use a simple integer hash function and insert the result into a float's mantissa. IIRC the GLSL spec guarantees 32-bit unsigned integers and IEEE binary32 float representation so it should be perfectly portable.
I gave this a try just now. The results are very good: it looks exactly like static with every input I tried, no visible patterns at all. In contrast the popular sin/fract snippet has fairly pronounced diagonal lines on my GPU given the same inputs.
One disadvantage is that it requires GLSL v3.30. And although it seems fast enough, I haven't empirically quantified its performance. AMD's Shader Analyzer claims 13.33 pixels per clock for the vec2 version on a HD5870. Contrast with 16 pixels per clock for the sin/fract snippet. So it is certainly a little slower.
Here's my implementation. I left it in various permutations of the idea to make it easier to derive your own functions from.
/*
static.frag
by Spatial
05 July 2013
*/
#version 330 core
uniform float time;
out vec4 fragment;
// A single iteration of Bob Jenkins' One-At-A-Time hashing algorithm.
uint hash( uint x ) {
x += ( x << 10u );
x ^= ( x >> 6u );
x += ( x << 3u );
x ^= ( x >> 11u );
x += ( x << 15u );
return x;
}
// Compound versions of the hashing algorithm I whipped together.
uint hash( uvec2 v ) { return hash( v.x ^ hash(v.y) ); }
uint hash( uvec3 v ) { return hash( v.x ^ hash(v.y) ^ hash(v.z) ); }
uint hash( uvec4 v ) { return hash( v.x ^ hash(v.y) ^ hash(v.z) ^ hash(v.w) ); }
// Construct a float with half-open range [0:1] using low 23 bits.
// All zeroes yields 0.0, all ones yields the next smallest representable value below 1.0.
float floatConstruct( uint m ) {
const uint ieeeMantissa = 0x007FFFFFu; // binary32 mantissa bitmask
const uint ieeeOne = 0x3F800000u; // 1.0 in IEEE binary32
m &= ieeeMantissa; // Keep only mantissa bits (fractional part)
m |= ieeeOne; // Add fractional part to 1.0
float f = uintBitsToFloat( m ); // Range [1:2]
return f - 1.0; // Range [0:1]
}
// Pseudo-random value in half-open range [0:1].
float random( float x ) { return floatConstruct(hash(floatBitsToUint(x))); }
float random( vec2 v ) { return floatConstruct(hash(floatBitsToUint(v))); }
float random( vec3 v ) { return floatConstruct(hash(floatBitsToUint(v))); }
float random( vec4 v ) { return floatConstruct(hash(floatBitsToUint(v))); }
void main()
{
vec3 inputs = vec3( gl_FragCoord.xy, time ); // Spatial and temporal inputs
float rand = random( inputs ); // Random per-pixel value
vec3 luma = vec3( rand ); // Expand to RGB
fragment = vec4( luma, 1.0 );
}
Screenshot:
I inspected the screenshot in an image editing program. There are 256 colours and the average value is 127, meaning the distribution is uniform and covers the expected range.
Gustavson's implementation uses a 1D texture
No it doesn't, not since 2005. It's just that people insist on downloading the old version. The version that is on the link you supplied uses only 8-bit 2D textures.
The new version by Ian McEwan of Ashima and myself does not use a texture, but runs at around half the speed on typical desktop platforms with lots of texture bandwidth. On mobile platforms, the textureless version might be faster because texturing is often a significant bottleneck.
Our actively maintained source repository is:
https://github.com/ashima/webgl-noise
A collection of both the textureless and texture-using versions of noise is here (using only 2D textures):
http://www.itn.liu.se/~stegu/simplexnoise/GLSL-noise-vs-noise.zip
If you have any specific questions, feel free to e-mail me directly (my email address can be found in the classicnoise*.glsl sources.)
Gold Noise
// Gold Noise ©2015 dcerisano#standard3d.com
// - based on the Golden Ratio
// - uniform normalized distribution
// - fastest static noise generator function (also runs at low precision)
// - use with indicated fractional seeding method.
float PHI = 1.61803398874989484820459; // Φ = Golden Ratio
float gold_noise(in vec2 xy, in float seed){
return fract(tan(distance(xy*PHI, xy)*seed)*xy.x);
}
See Gold Noise in your browser right now!
This function has improved random distribution over the current function in #appas' answer as of Sept 9, 2017:
The #appas function is also incomplete, given there is no seed supplied (uv is not a seed - same for every frame), and does not work with low precision chipsets. Gold Noise runs at low precision by default (much faster).
There is also a nice implementation described here by McEwan and #StefanGustavson that looks like Perlin noise, but "does not require any setup, i.e. not textures nor uniform arrays. Just add it to your shader source code and call it wherever you want".
That's very handy, especially given that Gustavson's earlier implementation, which #dep linked to, uses a 1D texture, which is not supported in GLSL ES (the shader language of WebGL).
After the initial posting of this question in 2010, a lot has changed in the realm of good random functions and hardware support for them.
Looking at the accepted answer from today's perspective, this algorithm is very bad in uniformity of the random numbers drawn from it. And the uniformity suffers a lot depending on the magnitude of the input values and visible artifacts/patterns will become apparent when sampling from it for e.g. ray/path tracing applications.
There have been many different functions (most of them integer hashing) being devised for this task, for different input and output dimensionality, most of which are being evaluated in the 2020 JCGT paper Hash Functions for GPU Rendering. Depending on your needs you could select a function from the list of proposed functions in that paper and simply from the accompanying Shadertoy.
One that isn't covered in this paper but that has served me very well without any noticeably patterns on any input magnitude values is also one that I want to highlight.
Other classes of algorithms use low-discrepancy sequences to draw pseudo-random numbers from, such as the Sobol squence with Owen-Nayar scrambling. Eric Heitz has done some amazing research in this area, as well with his A Low-Discrepancy Sampler that Distributes Monte Carlo Errors as a Blue Noise in Screen Space paper.
Another example of this is the (so far latest) JCGT paper Practical Hash-based Owen Scrambling, which applies Owen scrambling to a different hash function (namely Laine-Karras).
Yet other classes use algorithms that produce noise patterns with desirable frequency spectrums, such as blue noise, that is particularly "pleasing" to the eyes.
(I realize that good StackOverflow answers should provide the algorithms as source code and not as links because those can break, but there are way too many different algorithms nowadays and I intend for this answer to be a summary of known-good algorithms today)
Do use this:
highp float rand(vec2 co)
{
highp float a = 12.9898;
highp float b = 78.233;
highp float c = 43758.5453;
highp float dt= dot(co.xy ,vec2(a,b));
highp float sn= mod(dt,3.14);
return fract(sin(sn) * c);
}
Don't use this:
float rand(vec2 co){
return fract(sin(dot(co.xy ,vec2(12.9898,78.233))) * 43758.5453);
}
You can find the explanation in Improvements to the canonical one-liner GLSL rand() for OpenGL ES 2.0
hash:
Nowadays webGL2.0 is there so integers are available in (w)GLSL.
-> for quality portable hash (at similar cost than ugly float hashes) we can now use "serious" hashing techniques.
IQ implemented some in https://www.shadertoy.com/view/XlXcW4 (and more)
E.g.:
const uint k = 1103515245U; // GLIB C
//const uint k = 134775813U; // Delphi and Turbo Pascal
//const uint k = 20170906U; // Today's date (use three days ago's dateif you want a prime)
//const uint k = 1664525U; // Numerical Recipes
vec3 hash( uvec3 x )
{
x = ((x>>8U)^x.yzx)*k;
x = ((x>>8U)^x.yzx)*k;
x = ((x>>8U)^x.yzx)*k;
return vec3(x)*(1.0/float(0xffffffffU));
}
Just found this version of 3d noise for GPU, alledgedly it is the fastest one available:
#ifndef __noise_hlsl_
#define __noise_hlsl_
// hash based 3d value noise
// function taken from https://www.shadertoy.com/view/XslGRr
// Created by inigo quilez - iq/2013
// License Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
// ported from GLSL to HLSL
float hash( float n )
{
return frac(sin(n)*43758.5453);
}
float noise( float3 x )
{
// The noise function returns a value in the range -1.0f -> 1.0f
float3 p = floor(x);
float3 f = frac(x);
f = f*f*(3.0-2.0*f);
float n = p.x + p.y*57.0 + 113.0*p.z;
return lerp(lerp(lerp( hash(n+0.0), hash(n+1.0),f.x),
lerp( hash(n+57.0), hash(n+58.0),f.x),f.y),
lerp(lerp( hash(n+113.0), hash(n+114.0),f.x),
lerp( hash(n+170.0), hash(n+171.0),f.x),f.y),f.z);
}
#endif
A straight, jagged version of 1d Perlin, essentially a random lfo zigzag.
half rn(float xx){
half x0=floor(xx);
half x1=x0+1;
half v0 = frac(sin (x0*.014686)*31718.927+x0);
half v1 = frac(sin (x1*.014686)*31718.927+x1);
return (v0*(1-frac(xx))+v1*(frac(xx)))*2-1*sin(xx);
}
I also have found 1-2-3-4d perlin noise on shadertoy owner inigo quilez perlin tutorial website, and voronoi and so forth, he has full fast implementations and codes for them.
I have translated one of Ken Perlin's Java implementations into GLSL and used it in a couple projects on ShaderToy.
Below is the GLSL interpretation I did:
int b(int N, int B) { return N>>B & 1; }
int T[] = int[](0x15,0x38,0x32,0x2c,0x0d,0x13,0x07,0x2a);
int A[] = int[](0,0,0);
int b(int i, int j, int k, int B) { return T[b(i,B)<<2 | b(j,B)<<1 | b(k,B)]; }
int shuffle(int i, int j, int k) {
return b(i,j,k,0) + b(j,k,i,1) + b(k,i,j,2) + b(i,j,k,3) +
b(j,k,i,4) + b(k,i,j,5) + b(i,j,k,6) + b(j,k,i,7) ;
}
float K(int a, vec3 uvw, vec3 ijk)
{
float s = float(A[0]+A[1]+A[2])/6.0;
float x = uvw.x - float(A[0]) + s,
y = uvw.y - float(A[1]) + s,
z = uvw.z - float(A[2]) + s,
t = 0.6 - x * x - y * y - z * z;
int h = shuffle(int(ijk.x) + A[0], int(ijk.y) + A[1], int(ijk.z) + A[2]);
A[a]++;
if (t < 0.0)
return 0.0;
int b5 = h>>5 & 1, b4 = h>>4 & 1, b3 = h>>3 & 1, b2= h>>2 & 1, b = h & 3;
float p = b==1?x:b==2?y:z, q = b==1?y:b==2?z:x, r = b==1?z:b==2?x:y;
p = (b5==b3 ? -p : p); q = (b5==b4 ? -q : q); r = (b5!=(b4^b3) ? -r : r);
t *= t;
return 8.0 * t * t * (p + (b==0 ? q+r : b2==0 ? q : r));
}
float noise(float x, float y, float z)
{
float s = (x + y + z) / 3.0;
vec3 ijk = vec3(int(floor(x+s)), int(floor(y+s)), int(floor(z+s)));
s = float(ijk.x + ijk.y + ijk.z) / 6.0;
vec3 uvw = vec3(x - float(ijk.x) + s, y - float(ijk.y) + s, z - float(ijk.z) + s);
A[0] = A[1] = A[2] = 0;
int hi = uvw.x >= uvw.z ? uvw.x >= uvw.y ? 0 : 1 : uvw.y >= uvw.z ? 1 : 2;
int lo = uvw.x < uvw.z ? uvw.x < uvw.y ? 0 : 1 : uvw.y < uvw.z ? 1 : 2;
return K(hi, uvw, ijk) + K(3 - hi - lo, uvw, ijk) + K(lo, uvw, ijk) + K(0, uvw, ijk);
}
I translated it from Appendix B from Chapter 2 of Ken Perlin's Noise Hardware at this source:
https://www.csee.umbc.edu/~olano/s2002c36/ch02.pdf
Here is a public shade I did on Shader Toy that uses the posted noise function:
https://www.shadertoy.com/view/3slXzM
Some other good sources I found on the subject of noise during my research include:
https://thebookofshaders.com/11/
https://mzucker.github.io/html/perlin-noise-math-faq.html
https://rmarcus.info/blog/2018/03/04/perlin-noise.html
http://flafla2.github.io/2014/08/09/perlinnoise.html
https://mrl.nyu.edu/~perlin/noise/
https://rmarcus.info/blog/assets/perlin/perlin_paper.pdf
https://developer.nvidia.com/gpugems/GPUGems/gpugems_ch05.html
I highly recommend the book of shaders as it not only provides a great interactive explanation of noise, but other shader concepts as well.
EDIT:
Might be able to optimize the translated code by using some of the hardware-accelerated functions available in GLSL. Will update this post if I end up doing this.
lygia, a multi-language shader library
If you don't want to copy / paste the functions into your shader, you can also use lygia, a multi-language shader library. It contains a few generative functions like cnoise, fbm, noised, pnoise, random, snoise in both GLSL and HLSL. And many other awesome functions as well. For this to work it:
Relays on #include "file" which is defined by Khronos GLSL standard and suported by most engines and enviroments (like glslViewer, glsl-canvas VS Code pluging, Unity, etc. ).
Example: cnoise
Using cnoise.glsl with #include:
#ifdef GL_ES
precision mediump float;
#endif
uniform vec2 u_resolution;
uniform float u_time;
#include "lygia/generative/cnoise.glsl"
void main (void) {
vec2 st = gl_FragCoord.xy / u_resolution.xy;
vec3 color = vec3(cnoise(vec3(st * 5.0, u_time)));
gl_FragColor = vec4(color, 1.0);
}
To run this example I used glslViewer.
Please see below an example how to add white noise to the rendered texture.
The solution is to use two textures: original and pure white noise, like this one: wiki white noise
private static final String VERTEX_SHADER =
"uniform mat4 uMVPMatrix;\n" +
"uniform mat4 uMVMatrix;\n" +
"uniform mat4 uSTMatrix;\n" +
"attribute vec4 aPosition;\n" +
"attribute vec4 aTextureCoord;\n" +
"varying vec2 vTextureCoord;\n" +
"varying vec4 vInCamPosition;\n" +
"void main() {\n" +
" vTextureCoord = (uSTMatrix * aTextureCoord).xy;\n" +
" gl_Position = uMVPMatrix * aPosition;\n" +
"}\n";
private static final String FRAGMENT_SHADER =
"precision mediump float;\n" +
"uniform sampler2D sTextureUnit;\n" +
"uniform sampler2D sNoiseTextureUnit;\n" +
"uniform float uNoseFactor;\n" +
"varying vec2 vTextureCoord;\n" +
"varying vec4 vInCamPosition;\n" +
"void main() {\n" +
" gl_FragColor = texture2D(sTextureUnit, vTextureCoord);\n" +
" vec4 vRandChosenColor = texture2D(sNoiseTextureUnit, fract(vTextureCoord + uNoseFactor));\n" +
" gl_FragColor.r += (0.05 * vRandChosenColor.r);\n" +
" gl_FragColor.g += (0.05 * vRandChosenColor.g);\n" +
" gl_FragColor.b += (0.05 * vRandChosenColor.b);\n" +
"}\n";
The fragment shared contains parameter uNoiseFactor which is updated on every rendering by main application:
float noiseValue = (float)(mRand.nextInt() % 1000)/1000;
int noiseFactorUniformHandle = GLES20.glGetUniformLocation( mProgram, "sNoiseTextureUnit");
GLES20.glUniform1f(noiseFactorUniformHandle, noiseFactor);
FWIW I had the same questions and I needed it to be implemented in WebGL 1.0, so I couldn't use a few of the examples given in previous answers. I tried the Gold Noise mentioned before, but the use of PHI doesn't really click for me. (distance(xy * PHI, xy) * seed just equals length(xy) * (1.0 - PHI) * seed so I don't see how the magic of PHI should be put to work when it gets directly multiplied by seed?
Anyway, I did something similar just without PHI and instead added some variation at another place, basically I take the tan of the distance between xy and some random point lying outside of the frame to the top right and then multiply with the distance between xy and another such random point lying in the bottom left (so there is no accidental match between these points). Looks pretty decent as far as I can see. Click to generate new frames.
(function main() {
const dim = [512, 512];
twgl.setDefaults({ attribPrefix: "a_" });
const gl = twgl.getContext(document.querySelector("canvas"));
gl.canvas.width = dim[0];
gl.canvas.height = dim[1];
const bfi = twgl.primitives.createXYQuadBufferInfo(gl);
const pgi = twgl.createProgramInfo(gl, ["vs", "fs"]);
gl.canvas.onclick = (() => {
twgl.bindFramebufferInfo(gl, null);
gl.useProgram(pgi.program);
twgl.setUniforms(pgi, {
u_resolution: dim,
u_seed: Array(4).fill().map(Math.random)
});
twgl.setBuffersAndAttributes(gl, pgi, bfi);
twgl.drawBufferInfo(gl, bfi);
});
})();
<script src="https://twgljs.org/dist/4.x/twgl-full.min.js"></script>
<script id="vs" type="x-shader/x-vertex">
attribute vec4 a_position;
attribute vec2 a_texcoord;
void main() {
gl_Position = a_position;
}
</script>
<script id="fs" type="x-shader/x-fragment">
precision highp float;
uniform vec2 u_resolution;
uniform vec2 u_seed[2];
void main() {
float uni = fract(
tan(distance(
gl_FragCoord.xy,
u_resolution * (u_seed[0] + 1.0)
)) * distance(
gl_FragCoord.xy,
u_resolution * (u_seed[1] - 2.0)
)
);
gl_FragColor = vec4(uni, uni, uni, 1.0);
}
</script>
<canvas></canvas>