Hello I'm trying to implement Bloom effect in my program. In fact, I've already implemented the effect using a highlight pass and a separate gaussian blur pass.
Here's an example:
Bright pass texture:
Gaussian blur render pass (2 internal passes for this effect):
And finally the final pass (brightPass + BlurPass):
(I want to precise I don't have implemented HDR tone mapping yet).
But I found a very interesting article from intel:
https://software.intel.com/en-us/articles/compute-shader-hdr-and-bloom
It said :
"First the bright pass is performed where values below a specified threshold are filtered out. The bright pass output is then downscaled by half 4 times. Each of the downscaled bright pass outputs are blurred with a separable Gaussian filter and then added to the next higher resolution bright pass output."
I understood how it works but I did not understand how to perform texture downscaling using OpenGL. However I know if I use glGenerateMipmap() function (at the initialization of my FBO) with 4 mipmap levels I think I will have my 4 downscaled texture directly with the desired format (1/16, 1/8, 1/4 and 1/2) like it's written in the article.
But my problem is I can't find the way to do this!
Does it exist a way to bind the other textures generated using mipmaping and use them in the fragment shader? Should I have to render the bright pass 4 times with 4 separates FBOs applying different formats (1/16, ...). I think it's a solution but it's probably not correct about the performance. I think the bright pass should be rendered once using the window max size and then using the downscaled textures (mipmap) already loaded in memory but I don't know how to bind and use them into my shaders! I'm really lost.
Thanks a lot in advance for your help!
You may use textureLod, as BDL suggested, but I think that there is no real need for this.
Once the required mipmap levels are generated, the right level will be selected automatically depending on the size of the active render buffer.
The main idea of the article you have mentioned, is that they perform several blurring steps on differently sized sources.
After mipmap levels were generated, you will have 4 mipmap levels that represent 1/16, 1/8, 1/4 and 1/2 scaled textures from bright pass.
Each of this buffers you should blur, using the render buffer of the same size (1/16, 1/8, 1/4 and 1/2 correspondingly).
If the render buffer, you are rendering to is 1/16 scale of the source texture from the bright pass, than the 4-th mipmap level will be used. So you do not need to specify mipmap level manually.
You can access a specific mipmap-level in a shader using the
gvec4 textureLod(gsampler2D sampler, vec2 P, float lod);
method, where lod specifies the mipmap level. (link to documentation).
Related
I'm trying to implement PBR into my simple OpenGL renderer and trying to use multiple lighting passes, I'm using one pass per light for rendering as follow:
1- First pass = depth
2- Second pass = ambient
3- [3 .. n] for all the lights in the scene.
I'm using the blending function glBlendFunc(GL_ONE, GL_ONE) for passes [3..n], and i'm doing a Gamma Correction at the end of each fragment shader.
But i still have a problem with the output image it just looks noisy specially when i'm using texture maps.
Is there anything wrong with those steps or is there any improvement to this process?
So basically, what you're calculating is
f(x) = a^gamma + b^gamma + ...
However, what you actually want (and as noted by #NicolBolas in the comments already) is
g(x) = (a + b + ...)^gamma
Now f(x) and g(x) will only equal each other in the rather useless cases like gamma=1. You simply cannot additively decompose a nonlinear function like power that way.
The correct solution is to blend everything together in linear space, and doing the gamma correction afterwards, on the total sum of the linear contributions of each light source.
However, implementing this will lead to a couple of technical issues. First and foremost, the standard 8 bit per channel are just not precise enough to store linear color values. Using such a format for the accumulation step will result in strongly visible color banding artifacts. There are two approaches to solve this:
Use a higher bit-per-channel format for the accumulation framebuffer. You will need a separate gamma correction pass, so you need to set up render-to-texture via a FBO. GL_RGBA16F appears as a particularily good format for this. Since you use a PBR lighting model, you can then also work with color values outside [0,1], and instead of a simple gamma correction, apply a proper tone mapping in the final pass. Note that while you may not need an alpha chanell, still use an RGBA format here, the RGB formats are simply not required color buffer formats by the GL spec, so they may not be supported universally.
Store the data still in 8 bit-per-component format, gamma corrected. The key here is that the blending must still be done in linear space, so the destination framebuffer color values must be re-linearized prior to blending. This can be achieved by using a framebuffer with GL_SRGB8_ALPHA8 format and enabling GL_FRAMEBUFFER_SRGB. In that case, the GPU will automatically apply the standard sRGB gamma correction when writing the fragment color to the framebuffer (which currently your fragment shader does), but it will also lead to the sRGB linearization when accessing to those values, including for blending. The OpenGL 4.6 core profile spec states in section "17.3.6.1 Blend equation":
If FRAMEBUFFER_SRGB is enabled and the value of FRAMEBUFFER_ATTACHMENT_COLOR_ENCODING for the framebuffer attachment corresponding
to the destination buffer is SRGB (see section 9.2.3), the R, G, and B destination
color values (after conversion from fixed-point to floating-point) are considered to
be encoded for the sRGB color space and hence must be linearized prior to their
use in blending. Each R, G, and B component is converted in the same fashion
described for sRGB texture components in section 8.24.
Approach 1 will be the much more general approach, while approach 2 has a couple of drawbacks:
the linearization/delinerization is done multiple times, potentially wasting some GPU processing power
due to still using only 8 bit integer, the overall quality will be lower. After each blending step, the results are rounded to the next representable number, so you will get much more quantization noise.
you are still limited to color values in [0,1] and cannot (easily) do more interesting tone mapping and HDR rendering effects
However, approach 2 also has advantages:
you do not need a separate final gamma correction pass
if your platform / window system does support sRGB framebuffers, you can directly create an sRGB pixeformat/visual for your window, and do not need any rende-to-texture step at all. Basically, requesting an sRGB framebuffer and enabling GL_FRAMEBUFFER_SRGB will be enough to make this work.
I'm currently implementing automatically adapting exposure for use with HDR in OpenGL. For this I need to retrieve the average brightness of all pixels in the previous frame.
I've not managed to find any solid explanations of how to do this. As far as I can see there are two ways to go about it.
Use glReadPixels to copy the framebuffer to memory and average them on the CPU. This is likely to be painfully slow and doesn't make good use of the GPU.
Take the frame and render it to successively smaller FBOs using linear filtering. This lets the GPU do most of the work but it's going to require a lot of FBOs (roughly 10 for a 1080p screen).
There has got to be a better way of getting average scene brightness. Does anyone have any suggestions?
There are two options that come into my mind:
Using glGenerateMipmap, which calculates the average of a 2x2 window, leaving you with the average scene brightness at the smallest level. This can be retrieved using textureLod function in a shader. Since each mipmap level has half the size of the previous one, the correct level will be log2(max), where max is the returned value of GL_MAX_TEXTURE_SIZE.
Using compute shaders to do basically the same thing glGenerateMipmap does, but with a bigger window size, which could potentially be faster (although I never tested this).
Your Option 2 is not much different from using glGenerateMipmap on the texture, just that you don't need to hassle with any client side objects like FBOs. So basically, rendering to mipmap level 0 of the texture, letting the GL generate the mipmap pyramid, and reading back just the highest level 1x1 image is probably the easiest way to get some approximation of the average color value.
For implementing a physically accurate motion blur by actually rendering at intermediate locations, it seems that to do this correctly I need a special blending function. Additive blending would only work on a black background, and the standard "transparency" function (GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA) may look okay for small numbers of samples, but it is physically inaccurate because samples rendered at the end will contribute more to the resulting color.
The function I need has to produce a color which is the weighted average of the original and destination colors, depending on the number of samples covering a fragment. However I can generalize this to better account for rendering differences between samples: Suppose I am to render a blurred object n times. Treating color as a 3-vector, Let D be the color DEST - SRC. I want each render to add D/n to the source color.
Can this be done using the fixed-function pipeline? The glBlendFunc reference is rather cryptic, at least to me. It seems like this can be done either trivially or is impossible. It seems like I would want to set alpha to 1/n. For the behavior I just described, am I in need of a GL_DEST_MINUS_SRC_COLOR option?
I also have a related question: At which stage does this blending operation occur? Before or after the fragment shader program? Would i be able to access the source and destination colors in a fragment shader?
I know that one way to accomplish what I want is by using an accumulation buffer. I do not want to do this because it is a waste of memory and fillrate.
The solution I ended up using to implement my effect is a combination of additive blending and a render target that I access as a texture from the fragment shader.
I'd like to be able to produce this effect, to be specific, the color-crawl / color-shift.
Is this possible with OpenGL shaders, or do I need to use another technique?
I'm new to OpenGL and I'd like try this as a getting started exercise, however if there's a better way of doing this, ultimately I want to produce this effect.
FYI I'm using Cinder as my OpenGL framework.
I know this isn't much information, but I'm having trouble even finding out what this effect is really called, so I can't google it.
I can't help you with the name of the effect, but I have an idea to produce this effect. My understanding is that each color component is shifted by some amount. A simple translation to the right of left of individual color components produced the black and white original image:
Steps to get the image you want
Get the source black and white image in a texture. If it's the result of other rendering, copy it to a texture.
Render a full screen quad (or the size you want) with texture coordinates from (0,0) to (1,1) and with the texture attached.
Apply a fragment shader that samples 3 times the input texture with a different shift in texture coordinates. e.g. -2 texels, 0 texel and +2 texel offsets. You can expirement and try more samples if you want and at different offsets.
Combine those 3 samples by keeping only 1 color component of each.
Alternate if performance doesn't matter or shaders are not available
Don't use a pixel shader but instead on OpenGL blending with the ADD function. Render 3 times that same full screen quad with the texture attached and use the texture matrix to offset the lookups each time. Mask the output colormask differently for each pass and you get the same result: pass 1 => red, pass 2 => green, pass 3 => blue.
The effect you're looking for is called chromatic abberation, you can it look up at Wikipedia. You were given a solution already, but I think it's my duty being a physicist, to give you a deeper understanding of what is going on, and how the effect can be generalized.
Remember that every camera has some aperture and light usually is described as waves. The interaction of waves with an aperture is called diffraction, but when it comes down mathematically it's just a convolution of the wave function with the fourier transform of the aperture function. Diffraction depends on the wavelength, so this creates a spatial shift depending on the color. The other effect contributing is dispersion, i.e. the dependence on refraction of the wavelength. Again diffraction can be described by a convolution.
Now convolutions can be chained up, yielding a total convolution kernel. In the case of Gauss blurring filter the convolution kernel is a Gauss distribution identical in all channels. But you can have different convolution kernels for each target channel. What #bernie suggestet are actually box convolution kernels, shifted by a few pixels in each channel.
This is a nice tutorial about convolution filtering with GLSL. You may use for loops as well instead of unrolling the loops.
http://www.ozone3d.net/tutorials/image_filtering_p2.php
I suggest you use some Gauss shaped kernels, with the blurring for red and blue being stronger than green, and of course slightly shifted center points.
GeexLab have a demo of Chromatic Abberation, with source in their Shader Library here.
I have an application where I need take the average intensity of an image for around 1 million images. It "feels" like a job for a GPU fragment shader, but fragment shaders are for per-pixel local computations, while image averaging is a global operation.
One approach I considered is loading the image into a texture, applying a 2x2 box-blur, load the result back into a N/2 x N/2 texture and repeating until the output is 1x1. However, this would take log n applications of the shader.
Is there a way to do it in one pass? Or should I just break down and use CUDA/OpenCL?
The summation operation is a specific case of the "reduction," a standard operation in CUDA and OpenCL libraries. A nice writeup on it is available on the cuda demos page. In CUDA, Thrust and CUDPP are just two examples of libraries that provide reduction. I'm less familiar with OpenCL, but CLPP seems to be a good library that provides reduction. Just copy your color buffer to an OpenGL pixel buffer object and use the appropriate OpenGL interoperability call to make that pixel buffer's memory accessible in CUDA/OpenCL.
If it must be done using the opengl API (as the original question required), the solution is to render to a texture, create a mipmap of the texture, and read in the 1x1 texture. You have to set the filtering right (bilinear is appropriate, I think), but it should get close to the right answer, modulo precision error.
My gut tells me to attempt your implementation in OpenCL. You can optimize for your image size and graphics hardware by breaking up the images into bespoke chunks of data that are then summed in parallel. Could be very fast indeed.
Fragment shaders are great for convolutions but that result is usually written to the gl_FragColor so it makes sense. Ultimately you will have to loop over every pixel in the texture and sum the result which is then read back in the main program. Generating image statistics perhaps not what the fragment shader was designed for and its not clear that a major performance gain is to be had since its not guaranteed a particular buffer is located in GPU memory.
It sounds like you may be applying this algorithm to a real-time motion detection scenario, or some other automated feature detection application. It may be faster to compute some statistics from a sample of pixels rather than the entire image and then build a machine learning classifier.
Best of luck to you in any case!
It doesn't need CUDA if you like to stick to GLSL. Like in the CUDA solution mentioned here, it can be done in a fragment shader staight forward. However, you need about log(resolution) draw calls.
Just set up a shader that takes 2x2 pixel samples from the original image, and output the average sum of those. The result is an image with half resolution in both axes. Repeat that until the image is 1x1 px.
Some considerations: Use GL_FLOAT luminance textures if avaliable, to get an more precise sum. Use glViewport to quarter the rendering area in each stage. The result then ends up in the top left pixel of your framebuffer.