I was trying to configure my stencil buffer so that, when enabled, it would set when the pixel drawn is not transparent (thus creating a map of pixels that light can collide with). What I've done is:
glClearStencil(0); //clear stencil
glStencilFunc(GL_EQUAL, 0xFF, 0x000000FF); //only where alpha (mask : 0x000000FF) is 0xFF (opaque)
glStencilOp(GL_INCR, GL_KEEP, GL_KEEP); //increment if passes (if it is opaque)
render(); //withing this method I sometimes disable the whole thing to draw the floor, for example
Then, i use the following code to test:
/* TURN OFF STENCIL */
glEnable(GL_STENCIL_TEST); //re-enable
glStencilFunc(GL_EQUAL, 0, 1); //if the test is equal to 1
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP); //do not change stencil buffer
ImageInfo.drawColorSquare(0, 0, Configurations.SCREEN_WIDTH, Configurations.GAME_HEIGHT, Color.BLUE); //drwa blue square
glDisable(GL_STENCIL_TEST); //disable
However, there are two problems:
It doesn't seem to be ignoring transparent pixels, as it should;
If a region overlaps with another, then it reverses - for example, it sets to one, then another region is drawn in the same area, and it reset it to 0 again.
I don't know why that is happening. Probably something wrong with my mask, I guess - I wasn't absolutely sure how many pixels OpneGL used in the Color Buffer.
Also, GL_INCR should add up to the max, and not go back, according to the documentation. Since my stencil buffer size is one bit, it should set to one, try to increase again, fail, and keep on one (instead of reseting).
The Stencil Test is independent of what happens in the color buffer. Setting glStencilFunc
( http://www.opengl.org/sdk/docs/man/xhtml/glStencilFunc.xml )
you can specify how the Stencil Test interacts with what is already stored in the Stencil Buffer.
Setting glStencilOp ( http://www.opengl.org/sdk/docs/man/xhtml/glStencilOp.xml ) gives you the possibility of using the result of the depth test to perform the Stencil Test.
A good tutorial that explain the Stencil Test and a very instructive algorithm based upon it can be found here http://ogldev.atspace.co.uk/www/tutorial37/tutorial37.html
The stencil buffer is usually an 8-bit buffer that reserves a small portion of the memory normally used for the depth buffer and is used for advanced rejection of fragments; things like masking to arbitrary shapes rather than using rectangular scissor boxes. It has nothing to do with your color buffer, and to make sure that fragments that have a specific alpha value do not affect the pixels on screen you would use something called an alpha test.
In core OpenGL 3, the fixed-function alpha test is no longer supported, so you would have to implement it in a fragment shader and then discard if it failed to meet your condition.
Related
What I'm trying to achieve is illustrated on the image below:
Let's say we have stencil buffer in a state so only red section is filled at the moment. What actions do I need to perform when I update stencil buffer with a section which is marked yellow, so in the end only green section would be the final state of the stencil buffer?
I need this to achieve nested element content clipping, to prevent content of the element to be rendered beyond the boundaries of them both combined.
So far I have tried various boolean operations involving stencil test to no avail, which brought more confusion than any progress.
Please note that scissor test is not actual for this task, because elements may have arbitrary shape and rotation.
If you've a hierarchical structure,then you've layers. Each child is only allowed to draw in the region which is defined by its parent. So each layer has to count up the stencil buffer (GL_INCR) and is only allowed to draw, if the current value of the stencil buffer is equal to the layer "depth":
int layer = 0;
glStencilFunc(GL_ALWAYS, 1, 0xff);
glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE);
// draw initial layer
// ...
layer ++;
glStencilFunc(GL_EQUAL, layer, 0xff);
glStencilOp(GL_KEEP, GL_KEEP, GL_INCR);
// draw layer 1
// ...
layer ++;
glStencilFunc(GL_EQUAL, layer, 0xff);
glStencilOp(GL_KEEP, GL_KEEP, GL_INCR);
// draw layer 2
// ...
Note, in general the stencil buffer has 8 bits, this limits the number of layers to 256.
I'm rendering reflective surfaces in my 3D engine. I need to perform two stencil operations. First, I draw out the reflective surface with depth testing to find visible areas of the surface. I then need to draw out models into a G-Buffer, which is also stencilled to find areas to draw my skybox.
Draw Surface: always draw, write bit #1
Draw Models: draw only if bit #1 is set, write bit #2
How would I do this using OpenGL? I'm unsure of the relationship between the glStencilFunc ref and mask values, and the glDepthMask value.
glStencilOp
glStencilFunc
glStencilMask
The docs are quite specific, but it's not always intuitive or obvious what to do if you simply want to create and then activate a mask. I use these as a simple starting point...
Create the mask
Clear the stencil buffer to zeroes and write 1s to all pixels you draw to.
void createStencilMask()
{
// Clear the stencil buffer with zeroes.
// This assumes glClearStencil() is unchanged.
glClear(GL_STENCIL_BUFFER_BIT);
// Enable stencil raster ops
// 1. Enables writing to the stencil buffer (glStencilOp)
// 2. If stencil test (glStencilFunc) fails, no frags are written
glEnable(GL_STENCIL_TEST);
// sfail GL_KEEP - keep original value if stencil test fails
// dpfail GL_KEEP - also keep if the stencil passes but depth test fails
// dppass GL_REPLACE - write only if both pass and you'd normally see it
// this writes the value of 'ref' to the buffer
glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);
// func GL_ALWAYS - the stencil test always passes
// ref 1 - replace with ones instead of zeroes
// mask 1 - only operate on the first bit (don't need any more)
// Assumes glStencilMask() is unchanged
glStencilFunc(GL_ALWAYS, 1, 1);
}
Call the above function, and draw your stuff. You can set glDepthMask and glColorMask so this doesn't actually affect the current colour/depth and your scene, only the stencil buffer.
Draw, using the mask
Only draw to pixels with 1s from the previous step.
void useStencilMask()
{
// Enable stencil raster ops
glEnable(GL_STENCIL_TEST);
// Just using the test, don't need to replace anything.
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
// Only render if the current pixel stencil value is one.
glStencilFunc(GL_EQUAL, 1, 1);
}
Draw and then leave the test disabled glDisable(GL_STENCIL_TEST) when you're done.
Reading and writing different bits
Now to focus on your question...
This bit's a little tricky because the same ref value of glStencilFunc() is used in both the stencil test and as the value to replace. However, you can get around this with masks:
Use mask in glStencilFunc() to ignore bits when testing/reading.
Use glStencilMask() to stop certain bits from getting written.
In your case you don't need to mask out the first bit from being written because it's already set. Simply update useStencilMask() to glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE) and glStencilFunc(GL_EQUAL, 3, 1);. Because mask is 1, only the first bit is used in the test for equality. However the whole ref of 3 (which is 0b11) is written. You could use glStencilMask(2) (which is 0b10) to stop the first bit from being written but it's already one so it doesn't matter.
You could also make use of GL_INCR which would set the second bit and remove the first. Or perhaps clear with ones and use GL_ZERO to mark both your bits.
I would like it so that when mesh A (the character), is behind mesh B (a wall), it is still rendered but with a solid gray color.
I'm beginning opengles 2.0 and I'm still unsure as to go about this. From what I understand the depth buffer allows meshes to fight out who will be seen in the fragments they encompass, also there are various blend functions that could possibly involved in this, finally the stencil buffer looks like it would also have this desirable functionality.
So is there a way to output different colors through the shader based on a failed depth test? Is there a way to do this through blending? Or must I use the stencil buffer some how?
And what is this technique called for future reference? I've seen it used in a lot of video games.
This can be done using the stencil buffer. The stencil buffer gives each pixel some additional bits which can be used as a bitmask or a counter. In your case you'd configure the stencil test unit to set a specific bitmask when the depth test for the character fails (because it's obstructed by the well). Then you switch the stencil test mode operation to pass the stencil test for this specific bitmask, and render a full viewport, solid quad in the desired color, with depth testing and depth writes disabled.
Code
I strongly recommend you dive deep into the documentation for the stencil test unit. It's a very powerful mechanism, often overlooked. Your particular problem would be solved by the following. I stuggest you take this example code, read it in parallel to the stencil test functions references glStencilFunc, glStencilOp.
You must add a stencil buffer to your frame buffer's pixel format – how you do that is platform dependent. For example, if you're using GLUT, then you'd add |GLUT_STENCIL to the format bitmask of glutInitDisplayMode; on iOS you'd set a property on your GLKView; etc. Once you've added a stencil buffer, you should clear it along with your other render buffers by adding |GL_STENCIL_BUFFER_BIT to the initial glClear call of each drawing.
GLint const silouhette_stencil_mask = 0x1;
void display()
{
/* ... */
glEnable(GL_DEPTH_TEST);
glDepthMask(GL_TRUE);
glDepthFunc(GL_LESS);
glDisable(GL_STENCIL_TEST);
/* The following two are not necessary according to specification.
* But drivers can be buggy and this makes sure we don't run into
* issues caused by not wanting to change the stencil buffer, but
* it happening anyway due to a buggy driver.
*/
glStencilFunc(GL_NEVER, 0, 0);
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
draw_the_wall();
glEnable(GL_STENCIL_TEST);
glStencilFunc(GL_ALWAYS, silouhette_stencil_mask, 0xffffffff);
glStencilOp(GL_KEEP, GL_REPLACE, GL_KEEP);
draw_the_character();
glStencilFunc(GL_EQUAL, silouhette_stencil_mask, 0xffffffff);
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
glDisable(GL_DEPTH_TEST);
glDepthMask(GL_FALSE);
draw_full_viewport_solid_color();
/* ... */
}
I have drawn multiple different colored polygons on the screen, now I have to draw another polygon of different color, but this polygon should be drawn only on those pixels which have a specific color.
I render each of the different colored polygons at same time in their own "layers", (= one color at a time). They can cover each other; newest layer covers all previous layers. The black color in the image is the "no polygons" area: empty space, and it should ignore that too.
So, basically I just render polygons, and then the N'th (not first) layer of polygons must be masked with the next polygon layer, and nothing else under it should be affected.
Image of the method needed:
What method can I use to achieve this with OpenGL ? I would prefer non-shader solution for this, if possible(?).
The only method I can do currently is to render each of the layers separately into the memory, then go through the pixels myself and combine the layers "manually", but that seems like a very slow method, doable though, but the speed is important here.
To use the stencil buffer for this, what you can do is:
Make sure you request a context that has a stencil buffer, this is windowing system specific so I won't cover it here. Call glGet(GL_STENCIL_BITS) to make sure that you get a sufficient number of bits.
The stencil buffer maintains an integer alongside each pixel, and allows you to modify it as things are drawn. As you draw each layer, set up the stencil buffer to set to a specific value when you draw each layer. You do this with
//draw layer N
glEnable(GL_STENCIL_TEST);
glStencilFunc(GL_ALWAYS, N, -1);
glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);
Now as you draw each layer, the last polygon that was drawn to the screen also stores it's layer number into the stencil buffer.
At this point when you want to go back and draw your green star, you just tell it to only draw on pixels where the stencil buffer is equal to N.
//draw only where stencil == N
glEnable(GL_STENCIL_TEST);
glStencilFunc(GL_EQUAL, N, -1);
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
drawStar();
I want to set up multiple stencils in OpenGL and then draw into multiple combinations of them.
For example:
I have two rectangular stencils like this:
Then I want to draw into:
anywhere
left rectangle (blue+purple)
right rectangle (purple+red)
middle rectangle (purple)
whole colored area (blue+purple+red)
I have found that it is possible to declare multiple stencils in different bits of stencil buffer but I don't know how.
How do I setup glStencilOp and glStencilFunc for this to work?
Or can I (should I) use glScissor for that?
I don't currently know, if it's possible to configure the stencil buffer for being able to do all the above 5 steps without making any changes to the stencil buffer between them. It would be easy if glStencilOp provided bitwise OR, but it doesn't and with just using increment or decrement you would have to draw the rectangles multiple times.
But if the regions are always rectangles, why not just use the scissor test? So the first 3 steps (or actually 2 and 3) can be done by just setting the rectangle's region with glScissor and enabling the scissor test (glEnable(GL_SCISSOR_TEST)).
For the middle one (step 4) you either compute the purple intersection rectangle yourself and use the scissor test again, or you use the stencil test:
glEnable(GL_STENCIL_TEST);
glStencilFunc(/*whatever*/);
glStencilOp(GL_INCR, GL_INCR, GL_INCR); //increase the stencil value
//draw both rectangles
glStencilFunc(GL_EQUAL, 2, 0xFFFFFFFF); //only draw where both rectangles are
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP); //don't change the stencil buffer
//draw things
So we first draw both rectangles and increase the stencil value everywhere they are drawn. Then we draw our things everywhere the stencil value is 2, meaning both rectangles were drawn.
For the 5th step you use the same stencil buffer, but with
glStencilFunc(GL_LEQUAL, 1, 0xFFFFFFFF);
for the second pass. This way you draw something everywhere the stencil buffer is at least 1, meaning at least one rectangle was drawn.
For more than two rectangles it can get more complicated and you need to play around a bit to find the most optimal way.