Say I have a 512x512 pixels texture that I am displaying on 256x256 pixels on the screen.
In that case "the level-of-detail function used when sampling from the texture determines that the texture should be minified" according to my GL_TEXTURE_MIN_FILTER which is GL_LINEAR.
As a result 2x2 pixels will be minified to 1 pixel (distance weighted linear average).
Is there some way that I can control the minification?
Say I instead want 4x4 or 8x8 pixels to be minified to 1 pixel since I prefer a coarse or rasterized image ;-).
Alternatively is there some way I can achieve the same effect in the shader code?
If you want to precisely control the filtering, write an appropriate fragment shader, use the texelFetch function to access the unfiltered texture data, then implement the filter in the shader.
If you're going for a Taylor approximation of the filtering kernel, keep in mind, that you can make use of bilinear mipmap filtering (i.e. GL_TEXTURE_MIN_FILTER := GL_LINEAR_MIPMAP_LINEAR) to implement the 0th and 1st order terms of the Taylor expansion.
Related
I read the Khronos wiki on this, but I don't really understand what it is saying. What exactly does textureGrad do?
I think it samples multiple mipmap levels and computes some color mixing using the explicit derivative vectors given to it, but I am not sure.
When you sample a texture, you need the specific texture coordinates to sample the texture data at. For sake of simplicity, I'm going to assume a 2D texture, so the texture coordinates are a 2D vector (s,t). (The explanation is analogous for other dimensionalities).
If you want to texture-map a triangle, one typically uses one of two strategies to get to the texture coordinates:
The texture coordinates are part of the model. Every vertex contains the 2D texture coordinates as a vertex attribute. During rasterization, those texture coordinates are interpolated across the primitive.
You specify a mathematic mapping. For example, you could define some function mapping the 3D object coordinates to some 2D texture coordinates. You can for example define some projection, and project the texture onto a surface, just like a real projector would project an image onto some real-world objects.
In either case, each fragment generated when rasterizing the typically gets different texture coordinates, so each drawn pixel on the screen will get a different part of the texture.
The key point is this: each fragment has 2D pixel coordinates (x,y) as well as 2D texture coordinates (s,t), so we can basically interpret this relationship as a mathematical function:
(s,t) = T(x,y)
Since this is a vector function in the 2D pixel position vector (x,y), we can also build the partial derivatives along x direction (to the right), and y direction (upwards), which are telling use the rate of change of the texture coordinates along those directions.
And the dTdx and dTdy in textureGrad are just that.
So what does the GPU need this for?
When you want to actually filter the texture (in contrast to simple point sampling), you need to know the pixel footprint in texture space. Each single fragment represents the area of one pixel on the screen, and you are going to use a single color value from the texture to represent the whole pixel (multisampling aside). The pixel footprint now represent the actual area the pixel would have in texture space. We could calculate it by interpolating the texcoords not for the pixel center, but for the 4 pixel corners. The resulting texcoords would form a trapezoid in texture space.
When you minify the texture, several texels are mapped to the same pixel (so the pixel footprint is large in texture space). When you maginify it, each pixel will represent only a fraction of the corresponding texel (so the footprint is quiete small).
The texture footprint tells you:
if the texture is minified or magnified (GL has different filter settings for each case)
how many texels would be mapped to each pixel, so which mipmap level would be appropriate
how much anisotropy there is in the pixel footprint. Each pixel on the screen and each texel in texture space is basically a square, but the pixel footprint might significantly deviate from than, and can be much taller than wide or the over way around (especially in situations with high perspective distortion). Classic bilinear or trilinear texture filters always use a square filter footprint, but the anisotropic texture filter will uses this information to
actually generate a filter footprint which more closely matches that of the actual pixel footprint (to avoid to mix in texel data which shouldn't really belong to the pixel).
Instead of calculating the texture coordinates at all pixel corners, we are going to use the partial derivatives at the fragment center as an approximation for the pixel footprint.
The following diagram shows the geometric relationship:
This represents the footprint of four neighboring pixels (2x2) in texture space, so the uniform grid are the texels, and the 4 trapezoids represent the 4 pixel footprints.
Now calculating the actual derivatives would imply that we have some more or less explicit formula T(x,y) as described above. GPUs usually use another approximation:
the just look at the actual texcoords the the neighboring fragments (which are going to be calculated anyway) in each 2x2 pixel block, and just approximate the footprint by finite differencing - the just subtracting the actual texcoords for neighboring fragments from each other.
The result is shown as the dotted parallelogram in the diagram.
In hardware, this is implemented so that always 2x2 pixel quads are shaded in parallel in the same warp/wavefront/SIMD-Group. The GLSL derivative functions like dFdx and dFdy simply work by subtracting the actual values of the neighboring fragments. And the standard texture function just internally uses this mechanism on the texture coordinate argument. The textureGrad functions bypass that and allow you to specify your own values, which means you control the what pixel footprint the GPU assumes when doing the actual filtering / mipmap level selection.
I have a 3D texture. Each texel contains a transparency value on the alpha channel.
I need to generate my mipmaps in such a way that it always takes the values of the texel with he maximum alpha value.
In other words if there are 4 texels 3 with a transparency value of 0 and one with a transparency value of 1 the resulting mipmap texel should be 1.
How can I achieve this?
If I need to write my own shaders, what is the optimal way to do it?
EDIT:
My question, to put it more clearly is:
Do I need to manually create a shader that does this or is there a way to use built in functions of opengl to save me the trouble?
In order to do that, you'll need to render to each layer of each mipmap with a custom shader that computes max of 8 samples from the upper level.
This can be done by attaching each layer of the rendered mipmap to a framebuffer (using glFramebufferTexture3D), and, in the shader, sampling from the same texture by using texelFetch (lod parameter specifies the mipmap to sample from).
I'm trying to code a texture reprojection using a UV gBuffer (this is a texture that contains the UV desired value for mapping at that pixel)
I think that this should be easy to understand just by seeing this picture (I cannot attach due low reputation):
http://www.andvfx.com/wp-content/uploads/2012/12/3-objectes.jpg
The first image (the black/yellow/red/green one) is the UV gBuffer, it represents the uv values, the second one is the diffuse channel and the third the desired result.
Making this on OpenGL is pretty trivial.
Draw a simple rectangle and use as fragmented shader this pseudo-code:
float2 newUV=texture(UVgbufferTex,gl_TexCoord[0]).xy;
float3 finalcolor=texture(DIFFgbufferTex,newUV);
return float4(finalcolor,0);
OpenGL takes care about selecting the mipmap level, the anisotropic filtering etc, meanwhile if I make this on regular CPU process I get a single pixel for finalcolor so my result is crispy.
Any advice here? I was wondering about computing manually a kind of mipmaps and select the level by checking the contiguous pixel but not sure if this is the right way, also I doubt how to deal with that since it could be changing fast on horizontal but slower on vertical or viceversa.
In fact I don't know how this is computed internally on OpenGL/DirectX since I used this kind of code for a long time but never thought about the internals.
You are on the right track.
To select mipmap level or apply anisotropic filtering you need a gradient. That gradient comes naturally in GL (in fragment shaders) because it is computed for all interpolated variables after rasterization. This all becomes quite obvious if you ever try to sample a texture using mipmap filtering in a vertex shader.
You can compute the LOD (lambda) as such:
ρ = max (((du/dx)2 + (dv/dx)2)1/2
, ((du/dy)2 + (dv/dy)2)1/2)
λ = log2 ρ
The texture is picked basing on the size on the screen after reprojection. After you emit a triangle, check the rasterization size and pick the appropriate mipmap.
As for filtering, it's not that hard to implement i.e. bilinear filtering manually.
I have an open GL quad that is rendered with a grayscale gradient. I would like to colorize it by applying a filter, something like:
If color = 0,0,0 then set color to 255,255,255
If color = 0,0,1 then set color to 255,255,254
etc, or some scheme I decide on.
Note the reason I do this in grayscale because the algorithm I'm using was designed to be drawn in grayscale and then colorized since the colors may not be known immediately.
This would be similar to the java LookupOp http://download.oracle.com/javase/6/docs/api/java/awt/image/LookupOp.html.
Is there a way to do this in openGL?
thanks,
Jeff
You could interpret those colours from the grayscale gradient as 1-D texture coordinates and then specify your look-up table as a 1-D texture. This seems to fit your situation.
Alternatively, you can use a fragment program (shader) to perform arbitrary colour transformations on individual pixels.
Some more explanation: What is a texture? A texture, conceptually, is some kind of lookup function, with some additional logic on top.
A 2-D texture is something which for any pair of coordinates (s,t) or (x,y) in the range of [0,0] - [1,1] yields a specific colour (RGB, RGBA, L, whatever). Additionally it has some settings like warping or filtering.
Underneath, a texture is described by discrete data of a given "density" - perhaps 16x16, perhaps 256x512. The filtering process makes it possible to specify a colour for any real number between [0,0] and [1,1] (by mixing/interpolating neighbouring texels or just taking the nearest one).
A 1-D texture is identical, except that it maps just a single real value to a colour. Therefore, it can be thought of as a specific type of a "lookup table". You can consider it equivalent to a 2-D texture based on a 1xN image.
If you have a grayscale gradient, you may render it directly by treating the gradient value as a colour - or you can treat it as texture coordinates (= indices in the lookup table) and using the 1-D texture for an arbitrary colour space transform.
You'd just need to translate the gradient values (from 0..255 range) to the [0..1] range of texture indices. I'd recommend something like out = (in+0.5)/256.0. The 0.5 makes for the half-texel offset as we want to point to the middle of a texel (a value inside a texture), not to a corner between 2 values.
To only have the exact RGB values from lookup table (= 1-D texture), also set the texture filters to GL_NEAREST.
BTW: Note that if you already need another texture to draw the gradient, then it gets a bit more complicated, because you'd want to treat the values received from one texture as coordinates for another texture - and I believe you'd need pixel shaders for that. Not that shaders are complicated or anything... they are extremely handy when you learn the basics.
I'm rendering a certain scene into a texture and then I need to process that image in some simple way. How I'm doing this now is to read the texture using glReadPixels() and then process it on the CPU. This is however too slow so I was thinking about moving the processing to the GPU.
The simplest setup to do this I could think of is to display a simple white quad that takes up the entire viewport in an orthogonal projection and then write the image processing bit as a fragment shader. This will allow many instances of the processing to run in parallel as well as to access any pixel of the texture it requires for the processing.
Is this a viable course of action? is it common to do things this way?
Is there maybe a better way to do it?
Yes, this is the usual way of doing things.
Render something into a texture.
Draw a fullscreen quad with a shader that reads that texture and does some operations.
Simple effects (e.g. grayscale, color correction, etc.) can be done by reading one pixel and outputting one pixel in the fragment shader. More complex operations (e.g. swirling patterns) can be done by reading one pixel from offset location and outputting one pixel. Even more complex operations can be done by reading multiple pixels.
In some cases multiple temporary textures would be needed. E.g. blur with high radius is often done this way:
Render into a texture.
Render into another (smaller) texture, with a shader that computes each output pixel as average of multiple source pixels.
Use this smaller texture to render into another small texture, with a shader that does proper Gaussian blur or something.
... repeat
In all of the above cases though, each output pixel should be independent of other output pixels. It can use one more more input pixels just fine.
An example of processing operation that does not map well is Summed Area Table, where each output pixel is dependent on input pixel and the value of adjacent output pixel. Still, it is possible to do those kinds on the GPU (example pdf).
Yes, it's the normal way to do image processing. The color of the quad doesn't really matter if you'll be setting the color for every pixel. Depending on your application, you might need to careful about pixel sampling issues (i.e. ensuring that you sample from exactly the correct pixel on the source texture, rather than halfway between two pixels).