I'm building a LIDAR simulator in OpenGL. This means that the fragment shader returns the length of the light vector (the distance) in place of one of the color channels, normalized by the distance to the far plane (so it'll be between 0 and 1). In other words, I use red to indicate light intensity and blue to indicate distance; and I set green to 0. Alpha is unused, but I keep it at 1.
Here's my test object, which happens to be a rock:
I then write the pixel data to a file and load it into a point cloud visualizer (one point per pixel) — basically the default. When I do that, it becomes clear that all of my points are in discrete planes each located at a different depth:
I tried plotting the same data in R. It doesn't show up initially with the default histogram because the density of the planes is pretty high. But when I set the breaks to about 60, I get this:
.
I've tried shrinking the distance between the near and far planes, in case it was a precision issue. First I was doing 1–1000, and now I'm at 1–500. It may have decreased the distance between planes, but I can't tell, because it means the camera has to be closer to the object.
Is there something I'm missing? Does this have to do with the fact that I disabled anti-aliasing? (Anti-aliasing was causing even worse periodic artifacts, but between the camera and the object instead. I disabled line smoothing, polygon smoothing, and multisampling, and that took care of that particular problem.)
Edit
These are the two places the distance calculation is performed:
The vertex shader calculates ec_pos, the position of the vertex relative to the camera.
The fragment shader calculates light_dir0 from ec_pos and the camera position and uses this to compute a distance.
Is it because I'm calculating ec_pos in the vertex shader? How can I calculate ec_pos in the fragment shader instead?
There are several possible issues I can think of.
(1) Your depth precision. The far plane has very little effect on resolution; the near plane is what's important. See Learning to Love your Z-Buffer.
(2) The more probable explanation, based on what you've provided, is the conversion/saving of the pixel data. The shader outputs floating point values, but these are stored in the framebuffer, which will typically have only 8bits per channel. For color, what that means is that your floats will be mapped to the underlying 8-bit (fixed width, integer) representation, therefore only possessing 256 values.
If you want to output pixel data as the true floats they are, you should make a 32-bit floating point RGBA FBO (with e.g. GL_RGBA32F or something similar). This will store actual floats. Then, when your data from the GPU, it will return the original shader values.
I suppose you could alternately encode a single float in a vec4 with some multiplication, if you don't have a FBO implementation handy.
Related
I read the Khronos wiki on this, but I don't really understand what it is saying. What exactly does textureGrad do?
I think it samples multiple mipmap levels and computes some color mixing using the explicit derivative vectors given to it, but I am not sure.
When you sample a texture, you need the specific texture coordinates to sample the texture data at. For sake of simplicity, I'm going to assume a 2D texture, so the texture coordinates are a 2D vector (s,t). (The explanation is analogous for other dimensionalities).
If you want to texture-map a triangle, one typically uses one of two strategies to get to the texture coordinates:
The texture coordinates are part of the model. Every vertex contains the 2D texture coordinates as a vertex attribute. During rasterization, those texture coordinates are interpolated across the primitive.
You specify a mathematic mapping. For example, you could define some function mapping the 3D object coordinates to some 2D texture coordinates. You can for example define some projection, and project the texture onto a surface, just like a real projector would project an image onto some real-world objects.
In either case, each fragment generated when rasterizing the typically gets different texture coordinates, so each drawn pixel on the screen will get a different part of the texture.
The key point is this: each fragment has 2D pixel coordinates (x,y) as well as 2D texture coordinates (s,t), so we can basically interpret this relationship as a mathematical function:
(s,t) = T(x,y)
Since this is a vector function in the 2D pixel position vector (x,y), we can also build the partial derivatives along x direction (to the right), and y direction (upwards), which are telling use the rate of change of the texture coordinates along those directions.
And the dTdx and dTdy in textureGrad are just that.
So what does the GPU need this for?
When you want to actually filter the texture (in contrast to simple point sampling), you need to know the pixel footprint in texture space. Each single fragment represents the area of one pixel on the screen, and you are going to use a single color value from the texture to represent the whole pixel (multisampling aside). The pixel footprint now represent the actual area the pixel would have in texture space. We could calculate it by interpolating the texcoords not for the pixel center, but for the 4 pixel corners. The resulting texcoords would form a trapezoid in texture space.
When you minify the texture, several texels are mapped to the same pixel (so the pixel footprint is large in texture space). When you maginify it, each pixel will represent only a fraction of the corresponding texel (so the footprint is quiete small).
The texture footprint tells you:
if the texture is minified or magnified (GL has different filter settings for each case)
how many texels would be mapped to each pixel, so which mipmap level would be appropriate
how much anisotropy there is in the pixel footprint. Each pixel on the screen and each texel in texture space is basically a square, but the pixel footprint might significantly deviate from than, and can be much taller than wide or the over way around (especially in situations with high perspective distortion). Classic bilinear or trilinear texture filters always use a square filter footprint, but the anisotropic texture filter will uses this information to
actually generate a filter footprint which more closely matches that of the actual pixel footprint (to avoid to mix in texel data which shouldn't really belong to the pixel).
Instead of calculating the texture coordinates at all pixel corners, we are going to use the partial derivatives at the fragment center as an approximation for the pixel footprint.
The following diagram shows the geometric relationship:
This represents the footprint of four neighboring pixels (2x2) in texture space, so the uniform grid are the texels, and the 4 trapezoids represent the 4 pixel footprints.
Now calculating the actual derivatives would imply that we have some more or less explicit formula T(x,y) as described above. GPUs usually use another approximation:
the just look at the actual texcoords the the neighboring fragments (which are going to be calculated anyway) in each 2x2 pixel block, and just approximate the footprint by finite differencing - the just subtracting the actual texcoords for neighboring fragments from each other.
The result is shown as the dotted parallelogram in the diagram.
In hardware, this is implemented so that always 2x2 pixel quads are shaded in parallel in the same warp/wavefront/SIMD-Group. The GLSL derivative functions like dFdx and dFdy simply work by subtracting the actual values of the neighboring fragments. And the standard texture function just internally uses this mechanism on the texture coordinate argument. The textureGrad functions bypass that and allow you to specify your own values, which means you control the what pixel footprint the GPU assumes when doing the actual filtering / mipmap level selection.
I have a application that will encode data for bearing and intensity using 32 bits. My fragment shader already decodes the values and then sets the color depending on bearing and intensity.
I'm wondering if it's also possible, via shader, to change the size (and possibly shape) of the drawn pixel.
As an example, let's say we have 4 possible values for intensity, then 0 would cause a single pixel to be drawn, 1 would draw a 2x2 square, 2 a 4x4 square and 3 a circle with a radius of 6 pixels.
In the past, we had to do all this on the CPU side and I was hoping to offload this job to the GPU.
No, fragment shaders cannot affect the "size" of the data they write. Once something has been rasterized into fragments, it doesn't really have a "size" anymore.
If you're rendering GL_POINTS primitives, you can change their size from the vertex shader. As for point sizes, it's rather difficult to ensure that a particular point covers an exact number of fragments.
The first thing that came into my mind is doing something similiar to blur technique, but instead of bluring the texture, we use it to look for neighbouring texels with a range to check if it has the intensity above 1.0f. if yes, then set the current texel color to for example red.
If you're using a fbo that is 1:1 in comparison to window size, use 1/width and 1/height in texture coordinates to get approximately 1 pixel (well not exactly because it is not a pixel but texel, just nearly)
Although this work just fine, the downside of this is it is very expensive as it will have n^2 complexity and probably some branching.
Edit: after thinking awhile this might not work for size with even number
TL;DR I'm computing a depth map in a fragment shader and then trying to use that map in a vertex shader to see if vertices are 'in view' or not and the vertices don't line up with the fragment texel coordinates. The imprecision causes rendering artifacts, and I'm seeking alternatives for filtering vertices based on depth.
Background. I am very loosely attempting to implement a scheme outlined in this paper (http://dash.harvard.edu/handle/1/4138746). The idea is to represent arbitrary virtual objects as lots of tangent discs. While they wanted to replace triangles in some graphics card of the future, I'm implementing this on conventional cards; my discs are just fans of triangles ("Discs") around center points ("Points").
This is targeting WebGL.
The strategy I intend to use, similar to what's done in the paper, is:
Render the Discs in a Depth-Only pass.
In a second (or more) pass, compute what's visible based solely on which Points are "visible" - ie their depth is <= the depth from the Depth-Only pass at that x and y.
I believe the authors of the paper used a gaussian blur on top of the equivalent of a GL_POINTS render applied to the Points (ie re-using the depth buffer from the DepthOnly pass, not clearing it) to actually render their object. It's hard to say: the process is unfortunately a one line comment, and I'm unsure of how to duplicate it in WebGL anyway (a naive gaussian blur will just blur in the background pixels that weren't touched by the GL_POINTS call).
Instead, I'm hoping to do something slightly different, by rerendering the discs in a second pass instead as cones (center of disc becomes apex of cone, think "close the umbrella") and effectively computing a voronoi diagram on the surface of the object (ala redbook http://www.glprogramming.com/red/chapter14.html#name19). The idea is that an output pixel is the color value of the first disc to reach it when growing radiuses from 0 -> their natural size.
The crux of the problem is that only discs whose centers pass the depth test in the first pass should be allowed to carry on (as cones) to the 2nd pass. Because what's true at the disc center applies to the whole disc/cone, I believe this requires evaluating a depth test at a vertex or object level, and not at a fragment level.
Since WebGL support for accessing depth buffers is still poor, in my first pass I am packing depth info into an RGBA Framebuffer in a fragment shader. I then intended to use this in the vertex shader of the second pass via a sampler2D; any disc center that was closer than the relative texture2D() lookup would be allowed on to the second pass; otherwise I would hack "discarding" the vertex (its alpha would be set to 0 or some flag set that would cause discard of fragments associated with the disc/cone or etc).
This actually kind of worked but it caused horrendous z-fighting between discs that were close together (very small perturbations wildly changed which discs were visible). I believe there is some floating point error between depth->rgba->depth. More importantly, though, the depth texture is being set by fragment texel coords, but I'm looking up vertices, which almost certainly don't line up exactly on top of relevant texel coordinates; so I get depth +/- noise, essentially, and the noise is the issue. Adding or subtracting .000001 or something isn't sufficient: you trade Type I errors for Type II. My render became more accurate when I switched from NEAREST to LINEAR for the depth texture interpolation, but it still wasn't good enough.
How else can I determine which disc's centers would be visible in a given render, so that I can do a second vertex/fragment (or more) pass focused on objects associated with those points? Or: is there a better way to go about this in general?
i have a 2k x 1k image with randomly placed "target" pixels. these pixels are pure red.
in a frag/pixel shader, for each pixel that is not red (target color), i need to find the distance to the closest red pixel. i'll use this distance value to create a gradient.
i found this answer, which seems the closest to my problem ---
Finding closest non-black pixel in an image fast ---
but it's not glsl specific.
i have the option to send my red target pixels into the frag shader as a texture buffer array. but i think it would be cleaner if i didn't need to.
A shader cannot read and write to the same texture because that would introduce too many constraints and complexities about the sequence of execution and would make caching much more difficult. So you're talking about sending some data about the red pixels in and getting the distance information out.
Fragment shaders run in parallel and it's much more expensive to perform random-access texture reads than to read from a location that is known outside of the shader, primarily due to pipelining considerations. The pre-programmable situation where sampling coordinates are known at vertices and then interpolated across the face of the geometry is still the most optimal way to access a texture.
So, writing a shader that, for each pixel, did a search outwards for a red pixel would be extremely inefficient. It's definitely possible, doing much the algorithm you link to, but probably not the smartest way around.
Ideally you'd phrase things the other way around and use some sort of accumulation. So:
clear your output buffer to its maximal values;
for each red location:
for every output fragment, work out the distance from the location;
check what distance is already stored for that fragment;
if the new distance is less than that stored, replace the stored version.
The easiest way to do that in OpenGL is likely going to be to use a depth buffer, because that has the per-fragment steps (2) and (3) implemented directly in hardware.
So for each each fragment you're going to calculate the distance from the current red fragment. You're going to output that as depth. When you're finished with all red dots you can use the depth buffer as input to a shader that outputs appropriate colours.
To avoid 2000 red spots turning into a 2000-pass drawing algorithm which would quickly run up against memory bandwidth, you'll probably want to write a single shader that does a large number of red dots at once and outputs a single depth value.
You should check GL_MAX_UNIFORM_LOCATIONS to find out how many uniforms you can push at once. It's guaranteed to be at least 1024 on recent versions of desktop OpenGL. You'll probably want to generate your shader dynamically.
Why don't you gaussian blur the whole image with a large radius, but at each iteration keep adding the red pixels back into the equation at full intensity so they bleed out. The red channel of the final blur would be your distance values - the higher values are closer to the red pixels. It's an approximation, but then you can make use of heavily optimised blur shaders.
I'm currently programming shadow mapping (cascaded shadow mapping, to be precise) into my c++ opengl engine. I therefore want to have a texture containing the distance between my light source and every pixel in my shadow map. Which texture type should I use?
I saw that there is a GL_DEPTH_COMPONENT texture internal format, but it scales the data I want to give the texture to [0,1]. Should I invert my length once when I create the shadow map then a second time during my final rendering to get back the real length? It seems quite useless!
Is there a way to use textures to store lengths without inverting them 2 times? (once at the texture creation, once during its usage).
I'm not sure what you mean with invert (I'm sure you cannot mean to invert the distance as this won't work). What you do is transform the distance to the light source into the [0,1] range.
This can be done by constructing a usual projection matrix for the light source's view and applying this to the vertices in the shadow map construction pass. This way their distance to the light source is written into the depth buffer (to which you can connect a texture with GL_DEPTH_COMPONENT format either by glCopyTexSubImage or FBOs). In the final pass you of course use the same projection matrix to compute the texture coordinates for the shadow map using projective texturing (using a sampler2DShadow sampler when using GLSL).
But this transformation is not linear, as the depth buffer has a higher precision near the viewer (or light source in this case). Another disadvantage is that you have to know the valid range of the distance values (the farthest point your light source affects). Using shaders (which I assume you do), you can make this transformation linear by just dividing the distance to the light source by this maximum distance and manually assign this to the fragment's depth value (gl_FragDepth in GLSL), which is what you probably meant by "invert".
The division (and knowledge of the maximum distance) can be prevented by using a floating point texture for the light distance and just writing the distance out as a color channel and then performing the depth comparison in the final pass yourself (using a normal sampler2D). But linearly filtering floating point textures is only supported on newer hardware and I'm not sure if this will be faster than a single division per fragment. But the advantage of this way is, that this opens the path for things like "variance shadow maps", which won't work that good with normal ubyte textures (because of the low precision) and neither with depth textures.
So to sum up, GL_DEPTH_COMPONENT is just a good compromise between ubyte textures (which lack the neccessary precision, as GL_DEPTH_COMPONENT should have at least 16bit precision) and float textures (which are not that fast or completely supported on older hardware). But due to its fixed point format you won't get around a transformation into the [0,1]-range (be it linear of projective). I'm not sure if floating point textures would be faster, as you only spare one division, but if you are on the newest hardware supporting linear (or even trilinear) filtering of float textures and 1 or 2 component float textures and render targets, it might be worth a try.
Of course, if you are using the fixed function pipeline you have only GL_DEPTH_COMPONENT as an option, but regarding your question I assume you are using shaders.