WebGL - is there an alternative to embedding shaders in HTML? - glsl

Question

The popular way of using GLSL shaders in WebGL seems to be to embed them in the main html file.
The vertex and fragments shaders are embedded in tags like:
<script id="shader-fs" type="x-shader/x-fragment">
This is the same convention I see in the WebGL samples in the Mozilla Developer Network page.
This works fine for simple apps, but when you have a complex app with a number of shaders, the html file gets cluttered. (I keep editing the wrong shader!) Also if you want to reuse your shaders, this scheme is inconvenient.
So I was thinking about putting these shaders in a separate XML files and loading them using XMLHttpRequest(). Then I saw that someone else had the same idea:
http://webreflection.blogspot.com/2010/09/fragment-and-vertex-shaders-my-way-to.html
I like the suggestion to use .c files, since that gives you syntax highlighting and other editor conveniences for GLSL.
But the issue with the above approach is that (as far as I understand) XMLHttpRequest() cannot load a local .c file - ie, on the client side - while you are developing and testing the WebGL app. But it is cumbersome to keep uploading it to the server during this process.
So if I want to keep the shaders out of the html file, is the only option to embed them as strings in the code? But that would make it hard to write as well as debug...
I'd appreciate any suggestions on managing multiple GLSL shaders in WebGL apps.
Regards
Edit (May 05 2011)
Since I use a Mac for development, I decided to enable Apache server, and put my webgl code under http://localhost/~username/. This sidesteps the issue of file: protocol being disabled during development. Now the javascript file loading code works locally since http: is used, rather than file:. Just thought I'd put this up here in case anyone finds it useful.

Answer 1

Yup, a local server's really the only way to go if you want to use XHR. I've written a bunch of WebGL lessons, and have often considered moving away from embedding the shaders in the HTML, but have been scared off by amount of explanation about web security I'd need to write...
Fortunately it's super easy to run a server. Just open a shell then
cd path-to-files
python -m SimpleHTTPServer
Then point your browser to
http://localhost:8000
That works for simple cases like textures and GLSL. For video and audio streaming see
What is a faster alternative to Python's http.server (or SimpleHTTPServer)?
On the other hand every browser that supports WebGL supports ES6 mutli-line template literals so if you don't care about old browsers you can just put you shaders in JavaScript using backticks like this
var vertexShaderSource = `
attribute vec4 position;
uniform mat4 u_matrix;
void main() {
gl_Position = u_matrix * position;
}
`;

Answer 2

EDIT 2021 this answer is antiquated. You should probably look for something else.
I've been using require.js's text plugin.
Here's a snippet:
define(
/* Dependencies (I also loaded the gl-matrix library) */
["glmatrix", "text!shaders/fragment.shader", "text!shaders/vertex.shader"],
/* Callback when all has been loaded */
function(glmatrix, fragmentShaderCode, vertexShaderCode) {
....
var vertexShader = gl.createShader(gl.VERTEX_SHADER);
gl.shaderSource(vertexShader, vertexShaderCode);
gl.compileShader(vertexShader);
....
}
);
The directory structure is as follows:
~require-gl-shaders/
|~js/
| |+lib/
| |~shaders/
| | |-fragment.shader
| | `-vertex.shader
| |-glmatrix.js - gl-matrix library
| |-shader.js
| |-text.js - require.js's text plugin
|-index.html
|-main.js
`-require.js - the require.js library
Personally, I had a little bit of learning curve with require, but it really helped me keep cleaner code.

Answer 3

Following #droidballoon's hint I ended up using stack.gl which "is an open software ecosystem for WebGL, built on top of browserify and npm".
Its glslify provides a browserify transform which can be used in conjunction with gl-shader in order to load shaders. The Javascript would look something like this:
var glslify = require('glslify');
var loadShader = require('gl-shader');
var createContext = require('gl-context');
var canvas = document.createElement('canvas');
var gl = createContext(canvas);
var shader = loadShader(
gl,
glslify('./shader.vert'),
glslify('./shader.frag')
);

Answer 4

My buddy created a nice utils object with some handy functions for this type of scenario. You would store your shaders in plain text files in a folder called "shaders":
filename : vertex.shader
attribute vec3 blah;
uniform mat4 uMVMatrix;
uniform mat4 uPMatrix;
uniform mat3 uNMatrix;
void main(void) {
magic goes here
}
filename : fragment.shader
#ifdef GL_ES
precision highp float;
#endif
varying vec4 vYadaYada;
uniform sampler2D uSampler;
void main(void) {
fragic magic goes here
}
And you simply call this to create a new program with these shader files:
var shaderProgram = utils.addShaderProg(gl, 'vertex.shader', 'fragment.shader');
And here is the sweet util object to handle biz:
utils = {};
utils.allShaders = {};
utils.SHADER_TYPE_FRAGMENT = "x-shader/x-fragment";
utils.SHADER_TYPE_VERTEX = "x-shader/x-vertex";
utils.addShaderProg = function (gl, vertex, fragment) {
utils.loadShader(vertex, utils.SHADER_TYPE_VERTEX);
utils.loadShader(fragment, utils.SHADER_TYPE_FRAGMENT);
var vertexShader = utils.getShader(gl, vertex);
var fragmentShader = utils.getShader(gl, fragment);
var prog = gl.createProgram();
gl.attachShader(prog, vertexShader);
gl.attachShader(prog, fragmentShader);
gl.linkProgram(prog);
if (!gl.getProgramParameter(prog, gl.LINK_STATUS)) {alert("Could not initialise main shaders");}
return prog;
};
utils.loadShader = function(file, type) {
var cache, shader;
$.ajax({
async: false, // need to wait... todo: deferred?
url: "shaders/" + file, //todo: use global config for shaders folder?
success: function(result) {
cache = {script: result, type: type};
}
});
// store in global cache
uilts.allShaders[file] = cache;
};
utils.getShader = function (gl, id) {
//get the shader object from our main.shaders repository
var shaderObj = utils.allShaders[id];
var shaderScript = shaderObj.script;
var shaderType = shaderObj.type;
//create the right shader
var shader;
if (shaderType == "x-shader/x-fragment") {
shader = gl.createShader(gl.FRAGMENT_SHADER);
} else if (shaderType == "x-shader/x-vertex") {
shader = gl.createShader(gl.VERTEX_SHADER);
} else {
return null;
}
//wire up the shader and compile
gl.shaderSource(shader, shaderScript);
gl.compileShader(shader);
//if things didn't go so well alert
if (!gl.getShaderParameter(shader, gl.COMPILE_STATUS)) {
alert(gl.getShaderInfoLog(shader));
return null;
}
//return the shader reference
return shader;
};//end:getShader
Thanks buddy for the sweet codeezy.. enjoy his contribution to the webgl community.. makes it way easier to simplify program / shader management.

Answer 5

I am using this:
https://www.npmjs.com/package/webpack-glsl-loader
It fits priority to keep syntax highlighting from having proper glsl files instead of text fragments.
I'll report later how it works.
[edit Aug-17, 2015] This approach is working fine for me. It assumes webpack is in your build flow, but that's not such a bad thing.
[edit 11-June-2016] https://github.com/kulicuu/Spacewar_WebGL_React has a working example for importing glsl files through a Webpack build. The game itself should be developed over the coming week.

Answer 6

A nice way of doing it is through the browserify-shader extension to Browserify.

Answer 7

If you can use server-side scripting, you could write a small script that reads in the shader files and returns a JavaScript file with the scripts in a global object. That way you can include it using plain-old <script src="shader?prefix=foo"> and edit the scripts as .c files.
Something like this Ruby CGI script
require 'cgi'
require 'json'
cgi = CGI.new
prefix = File.expand_path(cgi["prefix"])
cwd = Dir.getwd + "/"
exit!(1) unless prefix.start_with?(cwd)
shader = prefix + ".c"
source = File.read(shader)
cgi.out("text/javascript") {
<<-EOF
if (typeof Shaders == 'undefined') Shaders = {};
Shaders[#{cgi["prefix"]}] = #{source.to_json};
EOF
}

Answer 8

You can place your shaders in different files just like you put your javascript code in different files. This library https://github.com/codecruzer/webgl-shader-loader-js accomplishes that with a familiar syntax:
Example Usage (taken verbatim from above page):
[index.html]:
<script data-src="shaders/particles/vertex.js" data-name="particles"
type="x-shader/x-vertex"></script>
<script data-src="shaders/particles/fragment.js" data-name="particles"
type="x-shader/x-fragment"></script>
[example.js]:
SHADER_LOADER.load (
function (data)
{
var particlesVertexShader = data.particles.vertex;
var particlesFragmentShader = data.particles.fragment;
}
);

Answer 9

Is not the exact solution, but is good for me.
I use Pug (old Jade) for compile HTML, and I use includes inside shaders script tags
script#vertexShader(type="x-shader/x-vertex")
include shader.vert
script#fragmentShader(type="x-shader/x-fragment")
include shader.frag
The result is the same, a HTML with the code inline, but you can work the shader separately.

Answer 10

I've also been using Require.js to organize my files, but rather than use the text plugin, like #Vlr suggests, I have a script which takes the shaders and converts it into a Require.js module which I can then use elsewhere. So a shader file, simple.frag like this:
uniform vec3 uColor;
void main() {
gl_FragColor = vec4(uColor, 1.0);
}
Will be converted into a file shader.js:
define( [], function() {
return {
fragment: {
simple: [
"uniform vec3 uColor;",
"void main() {",
" gl_FragColor = vec4(uColor, 1.0);",
"}",
].join("\n"),
},
}
} );
Which looks messy, but the idea isn't that it is human readable. Then if I want to use this shader someplace, I just pull in the shader module and access it using shader.fragment.simple, like so:
var simple = new THREE.ShaderMaterial( {
vertexShader: shader.vertex.simple,
fragmentShader: shader.fragment.simple
} );
I've written up a blog post with more details and links to demo code here: http://www.pheelicks.com/2013/12/webgl-working-with-glsl-source-files/

Answer 11

Might not be the best way but I'm using php. I put the shaders in a separate file and then you just use:
<?php include('shaders.html'); ?>
works great for me.

Answer 12

Use C macro #include and gcc -E (-E key runs preprocessor without the compiler)
Add this to your js file:
const shader = `
#include "shader.fg"
`
and use shell after:
mov main.js main.c
gcc -E --no-warnings main.c | sed '/^#.*/d' > main.js
sed here just deletes extra comments generated by preprocessor
It works! ;)

Answer 13

You can use JSONP as an alternative to XMLHttpRequest, which is fine for local file system based loading.
Your shader files can be named anything you like (for automatic syntax highlighting)
vertex-shader.glsl:
JSONP('vertex-shader',`
attribute vec4 a_position;
void main() {
gl_Position = a_position;
}
`)
Then we have our index.html file which includes the 2 script files
index.html
<canvas id="c"></canvas>
<script src="vertex-shader.glsl">
<script src="main.js"></script>
Inside main.js you include the JSONP function, which handles the shader files
var shaders = {}
function JSONP(name,contents) {
shaders[name] = contents
}
// ... main webGL handler code
You could dynamically insert new script tags if you wanted to load new shaders in.