I try to use the stencil buffer of a FBO in OpenGL, but I can't get it to work. I bound a depth24_stencil8 texture to the FBO both for the depth and stencil targets. As a simple test, I tried:
/* Enable FBO */
glEnable(GL_STENCIL_TEST);
glStencilFunc(GL_NEVER, 1, 0xff);
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
glBegin(GL_TRIANGLE);
/* Draw some triangles */
glEnd();
glDisable(GL_STENCIL_TEST);
/* Disable FBO and render it on screen as a texture. */
As I use GL_NEVER, nothing should be rendered at all, but I can see the triangles. This is like if there were no stencil at all, but I cannot understand why. Trying this code without FBOs works, so I think I use the stencil functions correctly. I don't have any idea how to solve this problem. Did anyone already use a stencil with FBOs?
My bad, I was not attaching correctly the stencil buffer to my FBO. Strange thing is that my fbo status was not indicating any error or bad attachement, so I was persuaded it was OK... this problem has driven me crazy but now it seems to work.
Related
I have a serious problem understanding how stencil buffering in OpenGL works. I followed the examples of a tutorial and the code works of course, but I really don't get at what points I'm writing to the stencil buffer. To me it seems as I'm setting up the stencil tests, masks and everything, but then I just call glDrawArrays(...)
and write the triangles to the output without applying the stencil buffer at all.
I also don't understand why I clear the buffer using glClear(GL_STENCIL_BUFFER_BIT)
before I even draw the triangles. I would be very thankful if somebody could enlighten me.
// Draws the initial cube facing upwards
glDrawArrays(GL_TRIANGLES, 0, 36);
//enable stencil testing
glEnable(GL_STENCIL_TEST);
// Draws the floor between the upward cube and downward cube
glStencilFunc(GL_ALWAYS, 1, 0xFF); /*sets the test function, mask and ref value*/
glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE); /*defines what happens on restults*/
glStencilMask(0xFF); /*Mask that is applied on the bits before they are written to the stencil buffer*/
glDepthMask(GL_FALSE);
glClear(GL_STENCIL_BUFFER_BIT); /*Why do I clear the buffer here?!?*/
glDrawArrays(GL_TRIANGLES, 36, 6);
// Draws the downward cube (reflection)
glStencilFunc(GL_EQUAL, 1, 0xFF);
glDepthMask(GL_TRUE);
glStencilMask(0x00);
model = glm::scale(
glm::translate(model, glm::vec3(0, 0, -1)),
glm::vec3(1, 1, -1)
);
glUniformMatrix4fv(uniModel, 1, GL_FALSE, glm::value_ptr(model));
glUniform3f(uniColor, 0.3f, 0.3f, 0.3f);
glDrawArrays(GL_TRIANGLES, 0, 36);
glUniform3f(uniColor, 1.0f, 1.0f, 1.0f);
glDisable(GL_STENCIL_TEST);
The third value passed to glStencilOp (GL_REPLACE) tells OpenGL to set the stencil buffer to the ref value previously specified in glStencilFunc whenever both the stencil test and the depth test are passed. In this case: 1.
So then the call to glDrawArrays updates the stencil buffer to 1 everyplace where it draws successfully (i.e., passes both tests).
(And the reason you clear it first, is to get rid of any 1's that happened to be there when you started: you don't want them acting as false positives where you didn't draw triangles).
See https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glStencilOp.xml for details.
Okay I think I finally get it. And just to provide the answer to anybody else who might have the same thinking barrier I had here is what was my problem.
In my head I thought I have to write the values somehow to the stencil buffer first and afterwards I will write them to the display. Which is wrong as #NicolBolas kindly stated.
The command glDrawArrays (while stencilTesting is active) writes to the display as well as to the stencil buffer. By that for everything following we have the stencil of the in my example floor in the buffer and can use it to evaluate whether or not a pixel will be drawn. By that we can effectively make the second cube be drawn only on the Floor and not outside of it. Due to setting glStencilMask to all zero before we draw the cube we make sure it doesn't affect the stencil buffer even though the stencil test is applied on the action. If anybody else stumbles upon this I hope this might help. And if this is complete nonsense, please anybody leave a comment with the mistake in my thinking.
Edit: Rendering the skybox before all other objects in the scene fixed this problem.
I've seen the question here but adding
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
doesn't appear to help.
I'm trying to render a spherical Skybox for my scene and for some reason when I Disable depth testing before doing so, the Skybox is the only thing rendered.
[Render other objects..]
// Disable depth test & mask, faceculling
glDisable(GL_DEPTH_TEST); // Adding this makes everything else invisible
glDepthMask(GL_FALSE);
glCullFace(GL_FRONT);
[Render texture onto inside of sphere..]
// Re-enable faceculling, & depth
glDepthMask(GL_TRUE);
glEnable(GL_DEPTH_TEST);
glCullFace(GL_BACK);
Any idea why this might be happening?
I wasn't sure of what code to include to keep this clear, so don't hesitate to ask for more.
Just to let you know, this:
glDisable(GL_DEPTH_TEST);
Disables both depth test and writing, so you don't need both that and set the depth mask to GL_FALSE.
When you clear the depth buffer each frame by default it should clear it the maximum value, probably 1.0. By default the depth function is GL_LESS meaning any depth value coming out of the fragment shader less than the one in the depth buffer passes and is written to the framebuffer.
It seems to me that what you're doing is clearing the depth buffer to 1.0, disabling depth testing and writing, drawing your objects, then enabling depth testing and writing and drawing your skybox. The problem with this is that the drawing of your objects doesn't write anything to the depth buffer, and so when it comes time to draw your skybox (with depth testing enabled) all the pixel depth values in the buffer are 1.0 (because you never wrote anything to it), and because the depth function is GL_LESS every pixel you draw of your skybox passes the depth test and is written to the framebuffer.
If there is a special need to have your objects always drawn in front of the skybox, for example the skybox follows the camera position around, then:
1) Disable the depth writing.
2) Draw the skybox.
3) Enable the depth writing.
4) Draw your objects.
Well, yes. That's what the depth test is for. Without it, there's nothing to indicate to OpenGL that the skybox shouldn't be rendered on top of everything else.
If you don't want this to happen, don't disable the depth test… or draw the skybox before everything else, instead of afterwards.
I know that it is a very bad idea to read/write from/to the same texture/location, because this would result in undefined behaviour.
But in my case, if depth testing is disabled and I read the depth values in a shader, is it ok to do the stencil testing at the same time as reading the depth values within the same texture?
In my opinion there should not be any problems, because i'm not reading the stencil buffer values in the shader. Or, could there by any hardware related problems when the texture is bound for reading in a shader and OpenGL uses it to do the stencil testing?
This texture is filled with depth/stencil values. I wan't to avoid some heavy BRDF lighting(Directional light) calculations on specific pixels(the sky).
Example code:
//Contains the depth/stencil texture(deferredDepthStencilTextureID).
//Two FBO's are sharing the same depth/stencil texture.
lightAccumulationFBO->bind();
glEnable(GL_STENCIL_TEST);
glStencilFunc(GL_EQUAL, 1, 0xFF);
//Disable writing to the stencil buffer, i.e. all the bits is write-protected.
glStencilMask(0x00);
glDisable(GL_DEPTH_TEST);
glEnable(GL_BLEND);
glBlendFunc(GL_ONE, GL_ONE); //Additive blending. Light accumulation.
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, deferredDepthStencilTextureID); //GL_DEPTH24_STENCIL8
//Bind other textures here...
shader.bind();
//Uniforms here...
postProcessQuad->renderQuad();
shader.unbind();
//Unbind all the textures here.
...
glDisable(GL_BLEND);
glEnable(GL_DEPTH_TEST);
glDisable(GL_STENCIL_TEST);
lightAccumulationFBO->unbind();
But in my case, if depth testing is disabled and I read the depth values in a shader, is it ok to do the stencil testing at the same time as reading the depth values within the same texture?
No. Whether an operation is defined or not is based on the images attached to the FBO and read from. Not the components of said images. And no, write masking will not save you from undefined behavior.
So unless your stencil texture is separate from your depth texture, that's not going to work. And good luck finding hardware that will allow you to separate depth/stencil images.
Even with GL 4.5/NV/ARB_texture_barrier, the answer is still no. That functionality makes certain exceptions to the above rule, but only for operations that are due to fragment shader outputs. Stencil test operations are not fragment shader outputs, so they don't apply.
I am writing an SDL2/modern OpenGL application that uses stencil buffers. I have written the following code in my renderer:
glEnable(GL_STENCIL_FUNC);
glClearStencil(0);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);
glStencilFunc(GL_NEVER, 1, 0xFF); //Always fail the stencil test
glStencilOp(GL_REPLACE, GL_KEEP, GL_KEEP); //Set the pixels which failed to 1
glStencilMask(0xFF);
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
//Drawing small rectangle here
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
glStencilFunc(GL_EQUAL, 1, 0xFF); //Only pass the stencil test where the pixel is 1 in the stencil buffer
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP); //Dont change the stencil buffer any further
//Drawing big rectangle here
glDisable(GL_STENCIL_FUNC);
The goal of the code above is to only draw the part of the big rectangle that fits in the small rectangle. Unfortunately, when I run the code the opposite happens, it renders the big rectangle with a hole in it the size of the small rectangle.
I have tried many more stencil functions, but they all result in the same, and this seems like it should work. So, does anybody have any ideas or can tell me where I am going wrong?
I apperantly don't have the reputation to embed pictures in my post but:
Intended result:
http://i.imgur.com/RpbHzCV.jpg
Actual result:
http://i.imgur.com/Z5qDqHk.jpg
The call glEnable(GL_STENCIL_FUNC); is just wrong, the correct enum is GL_STENCIL_TEST. So your code doesn't use the stencil buffer at all.
I can only guess why you get the result you got, then: Your code might draw the first rectangle into the depth buffer, so when you draw the second one, the fragments in that area might fail the depth test. So even when you correctly enable the stencil test, you still have to take care about the depht buffer here.
I would like it so that when mesh A (the character), is behind mesh B (a wall), it is still rendered but with a solid gray color.
I'm beginning opengles 2.0 and I'm still unsure as to go about this. From what I understand the depth buffer allows meshes to fight out who will be seen in the fragments they encompass, also there are various blend functions that could possibly involved in this, finally the stencil buffer looks like it would also have this desirable functionality.
So is there a way to output different colors through the shader based on a failed depth test? Is there a way to do this through blending? Or must I use the stencil buffer some how?
And what is this technique called for future reference? I've seen it used in a lot of video games.
This can be done using the stencil buffer. The stencil buffer gives each pixel some additional bits which can be used as a bitmask or a counter. In your case you'd configure the stencil test unit to set a specific bitmask when the depth test for the character fails (because it's obstructed by the well). Then you switch the stencil test mode operation to pass the stencil test for this specific bitmask, and render a full viewport, solid quad in the desired color, with depth testing and depth writes disabled.
Code
I strongly recommend you dive deep into the documentation for the stencil test unit. It's a very powerful mechanism, often overlooked. Your particular problem would be solved by the following. I stuggest you take this example code, read it in parallel to the stencil test functions references glStencilFunc, glStencilOp.
You must add a stencil buffer to your frame buffer's pixel format – how you do that is platform dependent. For example, if you're using GLUT, then you'd add |GLUT_STENCIL to the format bitmask of glutInitDisplayMode; on iOS you'd set a property on your GLKView; etc. Once you've added a stencil buffer, you should clear it along with your other render buffers by adding |GL_STENCIL_BUFFER_BIT to the initial glClear call of each drawing.
GLint const silouhette_stencil_mask = 0x1;
void display()
{
/* ... */
glEnable(GL_DEPTH_TEST);
glDepthMask(GL_TRUE);
glDepthFunc(GL_LESS);
glDisable(GL_STENCIL_TEST);
/* The following two are not necessary according to specification.
* But drivers can be buggy and this makes sure we don't run into
* issues caused by not wanting to change the stencil buffer, but
* it happening anyway due to a buggy driver.
*/
glStencilFunc(GL_NEVER, 0, 0);
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
draw_the_wall();
glEnable(GL_STENCIL_TEST);
glStencilFunc(GL_ALWAYS, silouhette_stencil_mask, 0xffffffff);
glStencilOp(GL_KEEP, GL_REPLACE, GL_KEEP);
draw_the_character();
glStencilFunc(GL_EQUAL, silouhette_stencil_mask, 0xffffffff);
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
glDisable(GL_DEPTH_TEST);
glDepthMask(GL_FALSE);
draw_full_viewport_solid_color();
/* ... */
}