Write gl_FragDepth while still executing depth pre-test - c++

Given a depth-prepass renderer, I have the minimum depth values a given fragment can possibly contain- thus, it makes no sense to consider any fragments farther than denoted.
Now, I have a shader which writes to gl_FragDepth, however is guaranteed to write a value greater than or equal to the depth value of its polygonal face. How can I still execute a depth-pretest (ie, if the fragment depth is farther than the buffer value, discard without shader execution), but allows me to write a value different (greater) than interpolated face depth if it passes the pre-test?

Starting with OpenGL 4.2 (GLSL 4.20), the functionality you're looking for is available as a layout qualifier on gl_FragDepth. It allows you to specify your intent on how you are going to modify the depth output in the fragment shader. For example, the following specifies that you are only going to change the depth value to make in greater:
layout (depth_greater) out float gl_FragDepth;
This will allow the early depth test to still be used when the depth is modified in the fragment shader. If you do not follow the "contract" you establish with this qualifier, you will get undefined behavior. For example, with the qualifier above, if you make the depth smaller, fragments that would otherwise be visible may get eliminated.
The functionality is based on the GL_AMD_conservative_depth and GL_ARB_conservative_depth extensions. If you want to use it with OpenGL versions lower than 4.2, you could check for the presence of these extension, and use it in one of its extension forms if it's available.

Related

Optimize for expensive fragment shader

I'm rendering multiple layers of flat triangles with a raytracer in the fragment shader. The upper layers have holes, and I'm looking for a way how I can avoid running the shader for pixels that are filled already by one of the upper layers, i.e. I want only the parts of the lower layers rendered that lie in the holes in the upper layers. Of course, if there's a hole or not is not known unless the fragment shader did his thing for a layer.
As far as I understand, I cannot use early depth testing because there, the depth values are interpolated between the vertices and do not come from the fragment shader. Is there a way to "emulate" that behavior?
The best way to solve this issue is to not use layers. You are only using layers because of the limitations of using a 3D texture to store your scene data. So... don't do that.
SSBOs and buffer textures (if your hardware is too old for SSBOs) can access more memory than a 3D texture. And you could even employ manual swizzling of the data to improve cache locality if that is viable.
As far as I understand, I cannot use early depth testing because there, the depth values are interpolated between the vertices and do not come from the fragment shader.
This is correct insofar as you cannot use early depth tests, but it is incorrect as to why.
The "depth" provided by the VS doesn't need to be the depth of the actual fragment. You are rendering your scene in layers, presumably with each layer being a full-screen quad. By definition, everything in one layer of rendering is beneath everything in a lower layer. So the absolute depth value doesn't matter; what matters is whether there is something from a higher layer over this fragment.
So each layer could get its own depth value, with lower layers getting a lower depth value. The exact value is arbitrary and irrelevant; what matters is that higher layers have higher values.
The reason this doesn't work is this: if your raytracing algorithm detects a miss within a layer (a "hole"), you must discard that fragment. And the use of discard at all turns off early depth testing in most hardware, since the depth testing logic is usually tied to the depth writing logic (it is an atomic read/conditional-modify/conditional-write).

Any way to obtain percentage coverage of fragment (pixel) by primitive in hlsl/glsl fragment shader?

When the rasterizer invokes on primitive it split it into the collection of fragments (pixels). Next, the fragment shader called for every obtained pixel. Is there any way for me to have additional float parameter in my fragment shader, that will store information about how much the exact pixel is covered by the source primitive? This should have non-trivial value from 0-1 on triangle border pixels. Obviously it will be 1 on every "inside" triangle pixel.
I want rasterizer calculate and pass this value for me.
I thoight the "coservative rasterization" could help with that, but as I understand it uses for slightly different tasks (mostly for collision detection).
Also, as I understand there is no build-in method to do that. May be I can change the rasterized nature to do this? Is it possible?
When rendering to a multisampled framebuffer, you can look at the gl_SampleMaskIn[] bitmask array in the fragment shader to detect how many samples will be covered by the current fragment. This is about as close as you're going to get, and it's not great for what you want.
Obviously, it has the limitation of having the same granularity as the sample locations within a pixel. But the full mask also may be fewer than the number of samples in the framebuffer. If the renderer decides to generate multiple fragments per-pixel during multisample rasterization, the sample mask that any such fragments will only be for the samples that this particular fragment will write.
So if you have a 16-sample multisample framebuffer, the implementation may generate 4 fragments per-pixel, each covering a distinct set of 4 samples. So the sample bitmask for a fragment will never have more than 4 bits, even though you asked for 16x multisample rendering. And there's basically nothing you can do to detect if this is happening (outside of doing tests on specific hardware). All of this is implementation-defined.
Basically, what you want isn't really available; gl_SampleMask is the closest you can get, and how useful it is will be very implementation-dependent.
Maybe one could use GL_POLYGON_SMOOTH somehow for this, since as far as I understand it does exactly this, calculate the coverage of the current fragment and then modulates the fragment's alpha based on this

Is it possible to depth test against a depth texture I am also sampling, in the same draw call?

Context:
I am using a deferred rendering setup, where in the first stage I have two FBO's: one is the GBuffer, for storing the normals, albedo, and material information for all visible fragments. This FBO has a 32-bit depth texture. This gets drawn into in a geometry pass, before any lighting is calculated.
The second FBO is color-only, and starts off black, but accumulates lighting over several passes, from lighting shaders that sample from the GBuffer and write to the color-only buffer using additive blending.
The problem is, I would really like to utilize early depth testing in order to have my lighting ONLY calculate for fragments that contain actual geometry (Not just sky). The best way I can think of to do this is to use depth testing to fail any pixels that have a depth of one in the case of sunlight, or to fail any pixels that lie behind the sphere of influence for point lights. However, I don't think I can bind this depth texture to my color FBO, since I also sample from it inside the lighting shader to calculate the fragments position in world-space.
So my question is: Is there a way to use the same depth texture for both the early depth test, and for sampling inside the shader? Or if not, is there some other (reasonably performant) way of rejecting pixels that don't have geometry in them? I will not be writing to this depth texture at all in my lighting pass.
I only have to target modern graphics hardware on PC's (So I can use any common extensions, or openGL 4.6 features).
There are rules in OpenGL about reading from data in a shader that's also being updated due to a framebuffer operation. Those rules used to be quite strict. Indeed, pre-GL 4.4, the rules were so strict that what you're trying to do was actually undefined behavior. That is, if an image from a texture was attached to the rendering FBO, and you took a sample from that texture in a way such that it was at all possible to be reading from the attached image, you got undefined behavior. Never mind if your write mask meant that no writing happened; it was UB.
Fortunately, it's well-defined now. You only get UB if you're doing an actual write, not merely because you have an image attached to the FBO. And by "now," I mean basically any hardware made in the last 10 years. While ARB_texture_barrier and GL 4.5 are fairly recent, their predecessor NV_texture_barrier is actually quite old. And despite being an NVIDIA extension by name, it was so widely implemented that it is even available on MacOS implementations.

How can I write a different value to the depth buffer than the value used for depth comparisson?

In Open GL, is it possible to render a polygon with a regular depth test enabled, but when the depth buffer value is actually written to the depth buffer, I want to write a custom value?
(The reason is I'm rendering a particle system, which should be depth-tested against the geometry in the scene, but I want to write a very large depth value where the particle system is located, thus utilizing the depth-blur post-processing step to further blur the particle system)
Update
To further refine the question, is it possible without rendering in multiple passes?
You don't.
OpenGL does not permit you to lie. The depth test tests the value in the depth buffer against the incoming value to be written in the depth buffer. Once that test passes, the tested depth value will be written to the depth buffer. Period.
You cannot (ab)use the depth test to do something other than testing depth values.
ARB_conservative_depth/GL 4.2 does allow you a very limited form of this. Basically, you promise the implementation that you will only change the depth in a way that makes it smaller or larger than the original value. But even then, it would only work in specific cases, and then only so long as you stay within the limits you specified.
Enforcing early fragment tests will similarly not allow you to do this. The specification explicitly states that the depth will be written before your shader executes. So anything you write to gl_FragDepth will be ignored.
One way to do it in a single pass is by doing the depth-test "manually".
Set glDepthFunc to GL_ALWAYS
Then in the fragment shader, sample the current value of the depth buffer and depending on it discard the fragment using discard;
To sample the current value of the depth buffer, you either need ARM_shader_framebuffer_fetch_depth_stencil (usually on mobile platforms) or NV_texture_barrier. The later however will yield undefined results if multiple particles of the same drawcall render on top of each other, while the former in this case will use the written depth value of the last particle rendered at the same location.
You can also do it without any extension by copying the current depth buffer into a depth texture before you render the particles and then use that depth texture for your manual depth test. That also avoids that particles which render on top of each other would interfere, as they'll all use the old depth value for the manual test.
You can use gl_FragDepth in fragment shader to write your custom values.
gl_FragDepth = 0.3;
https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/gl_FragDepth.xhtml

What happens to the depth buffer if I discard a fragment in a shader using early_fragment_tests?

I'm using a fragment shader which discards some fragments using the discard keyword. My shader also uses the early_fragment_tests ( image store load obliges ).
EDIT :
I do not write the gl_FragDepth, I let the standard OpenGL handle the depth value.
Will my depth buffer be updated with the fragment's depth before the discard keyword is executed?
EDIT :
It does not seems like it on my NVidia Quadro 600 and K5000.
Any clue where I could find that information? FYI, I searched http://www.opengl.org/registry/specs/ARB/shader_image_load_store.txt. I found close enough topics but not that particular one.
Will my depth buffer be updated with the fragment's depth before the discard keyword is executed?
No, this sort of behavior is explicitly forbidden in a shader that contains discard or that writes an arbitrary value to gl_FragDepth. This is because in such a shader, the depth of your fragment after it is shaded may be unrelated the position generated during initial rasterization (pre-shading).
Without writing to gl_FragDepth or discarding, the depth of a fragment is actually known long before the actual fragment shader executes and this forms the foundation for early depth tests. Rasterization/shading can be skipped for some (individual tiled regions) or all of a primitive if it can be determined that it would have failed a depth test before the fragment shader is evaluated, but if the fragment shader itself is what determines a fragment's depth, then all bets are off.
There is an exception to this rule in DX11 / OpenGL 4.x. If you write your shaders in such a way that you can guarantee the output depth will always preserve the result of a depth test (same result as the depth generated during rasterization), early fragment tests can be enabled in a shader that uses discard or writes to gl_FragDepth. This feature is known as conservative depth, and unless you use this it is generally understood that discard is going to break early depth optimizations across the board.
Now, since you should never write to the depth buffer before you know whether the value you are writing passes or fails a depth test (gl_FragDepth may be different) or if the fragment even survives (discard may be used), you can see why a primitive shaded by a fragment shader that contains discard cannot write to the depth buffer before the shader is evaluated.
I think the information you are looking for is on that page:
If early fragment tests are enabled, any depth value computed by the
fragment shader has no effect. Additionally, the depth buffer, stencil
buffer, and occlusion query sample counts may be updated even for
fragments or samples that would be discarded after fragment shader
execution due to per-fragment operations such as alpha-to-coverage or
alpha tests.
The word "may" in "the depth buffer, [etc.] may be updated", implies it is implementation dependent (or completely random).