I've written a GLSL shader to emulate a vintage arcade game's indexed color tile-based graphics. I made a couple of shaders, one that does this with point sprites, and another using polygons. The point sprite shader converts gl_PointCoord to a pixel coordinate within each tile like so:
vec2 pixelFloat = gl_PointCoord * tileSizeInPixels;
ivec2 pixel = ivec2(int(pixelFloat.x), int(pixelFloat.y));
// pixel is now used in conjunction with a tile 'ID' uniform
// to locate indexed colors with a texture lookup from a
// large texture representing the game's ROM, with GL_NEAREST filtering.
// very clever 😋
The polygon shader instead uses an attribute buffer to pass pixel coordinates (which range {0.0 … 32.0} for a 32-pixel square tile, for example). After conversion to int, each fragment within the tile sees pixel coordinate values ranging x {0 … 31} y {0 … 31}, except:
This worked fine apart from artefacts sometimes showing at the edge of the tile with the higher numbered pixel coordinate at certain resolutions. I guessed that would be due to the fragment being at just the right location to be right on the maximum value of either gl_PointCoord or the vertex attribute value of 32.0, causing that fragment to sample the wrong tile.
These artefacts went away when I clamped the pixel ivec like this:
vec2 pixelFloat = gl_PointCoord * tileSizeInPixels;
ivec2 pixel = ivec2(
min(int(pixelFloat.x), tileSizeInPixels - 1),
min(int(pixelFloat.y), tileSizeInPixels - 1));
which solved the problem and didn't introduce any new artefacts.
My question is: Is there some way of controlling the interpolation of gl_PointCoord or my pixel coordinate attribute such that we can guarantee the interpolated value will range
minimum value <= interpolated value < maximum value
as opposed to
minimum value <= interpolated value <= maximum value
Is there some way I can avoid using min() here?
NB: GL_CLAMP_* is not an option here, as the pixel coordinate is used to look up the pixel's index color from a much larger texture, which is essentially the game's sprite ROM loaded into a single large texture buffer.
Related
Setting the scene
I'm working on a feature in scenekit where i have a camera at the center of a sphere. The sphere has a texture wrapped around it. Let's say it was a 360 degree image captured inside of a room.
So far
I have identified the positions on the sphere that correspond to the corners of the floor. I can extract and create a new flat 2d plane that matches the dimensions of the floor from the camera's perspective. E.g. If the room had a long rectangular floor, I'd create a trapezoid shaped plane.
Problem
But I would like for the new 2d plane to have the texture of the floor, not just the shape. How do I do this given that what I want to extract is not the original texture image, but the result of its projection onto the sphere?
FYI I'm pretty new to scenekit and 3d graphics stuff and I'm even newer to opengl
I assume that your image is structured in a way that lets you directly pick a pixel given an arbitrary direction. E.g. if the azimuth of the direction is mapped to the image's x-coordinate and the height of the direction to the image's y-coordinate, you would convert the direction to these parameters and pick the color at those coordinates. If that is not the case, you have to find the intersection of the according ray (starting at the camera) with the sphere and find the texture coordinate at that intersection. You can then pick the color using this texture coordinate.
Now, you have basically two options. The first option is generating a new texture for the plane. The second option is sampling the spherical image from a shader.
Option 1 - Generate a new texture
You know the extent of your plane, so you can generate a new texture whose dimensions are proportional to the plane's extents. You can use an arbitrary resolution. All you then need to do is fill the pixels of this texture. For this, you just generate the ray for a given pixel and find the according color in the spherical image like so:
input: d1, d2, d3, d3 (the four direction vectors of the plane corners)
// d3 +------+ d4
// d1 +------+ d2
for x from 0 to texture width
for y from 0 to texture height
//Find the direction vector for this pixel through bilinear interpolation
a = x / (width - 1) //horizontal interpolation parameter
b = y / (height - 1) //vertical interpolation parameter
d = (1 - a) * ((1 - b) * d1 + b * d3) + a * ((1 - b) * d2 + b * d4)
normalize d
//Sample the spherical image at d
color = sample(d)
//write the color to the new planar texture
texture(x, y) = color
next
next
Then, you have a new texture that you can apply to the plane. Barycentric interpolation might be more appropriate if you express the plane as two triangles. But as long as the plane is rectangular, the results will be the same.
Note that the sample() method depends on your image structure and needs to be implemented appropriately.
Option 2 - Sample in a shader
In option 2, you do the same thing as in option 1. But you do it in a fragment shader. You employ the vertices of the plane with their respective directions (this might be just the vertex position) and let the GPU interpolate them. This gives you directly the direction d, which you can use. Here is some pseudo shader code:
in vec3 direction;
out vec4 color;
void main()
{
color = sample(normalize(direction));
}
If your image is a cube map, you can even let the GPU do the sampling.
Here's my situation: I need to draw a rectangle on the screen for my game's Gui. I don't really care how big this rectangle is or might be, I want to be able to handle any situation. How I'm doing it right now is I store a single VAO that contains only a very basic quad, then I re-draw this quad using uniforms to modify the size and position of it on the screen each time.
The VAO contains 4 vec4 vertices:
0, 0, 0, 0;
1, 0, 1, 0;
0, 1, 0, 1;
1, 1, 1, 1;
And then I draw it as a GL_TRIANGLE_STRIP. The XY of each vertex is it's position, and the ZW is it's texture co-ordinates*. I pass in the rect for the gui element I'm currently drawing as a uniform vec4, which offsets the vertex positions in the vertex shader like so:
vertex.xy *= guiRect.zw;
vertex.xy += guiRect.xy;
And then I convert the vertex from screen pixel co-ordinates into OpenGL NDC co-ordinates:
gl_Position = vec4(((vertex.xy / screenSize) * 2) -1, 0, 1);
This changes the range from [0, screenWidth | screenHeight] to [-1, 1].
My problem comes in when I want to do texture wrapping. Simply passing vTexCoord = vertex.zw; is fine when I want to stretch a texture, but not for wrapping. Ideally, I want to modify the texture co-ordinates such that 1 pixel on the screen is equal to 1 texel in the gui texture. Texture co-ordinates going beyond [0, 1] is fine at this stage, and is in fact exactly what I'm looking for.
I plan to implement texture atlasses for my gui textures, but managing the offsets and bounds of the appropriate sub-texture will be handled in the fragment shader - as far as the vertex shader is concerned, our quad is using one solid texture with [0, 1] co-ordinates, and wrapping accordingly.
*Note: I'm aware that this particular vertex format isn't neccesarily useful for this particular case, I could be using vec2 vertices instead. For the sake of convenience I'm using the same vertex format for all of my 2D rendering, and other objects ie text actually do need those ZW components. I might change this in the future.
TL/DR: Given the size of the screen, the size of a texture, and the location/size of a quad, how do you calculate texture co-ordinates in a vertex shader such that pixels and texels have a 1:1 correspondence, with wrapping?
That is really very easy math: You just need to relate the two spaces in some way. And you already formulated a rule which allows you to do so: a window space pixel is to map to a texel.
Let's assume we have both vec2 screenSize and vec2 texSize which are the unnormalized dimensions in pixels/texels.
I'm not 100% sure what exactly you wan't to achieve. There is something missing: you actaully did not specify where the origin of the texture shall lie. Should it always be but to the bottom left corner of the quad? Or should it be just gloablly at the bottom left corner of the viewport? I'll assume the lattter here, but it should be easy to adjust this for the first case.
What we now need is a mapping between the [-1,1]^2 NDC in x and y to s and t. Let's first map it to [0,1]^2. If we have that, we can simply multiply the coords by screenSize/texSize to get the desired effect. So in the end, you get
vec2 texcoords = ((gl_Position.xy * 0.5) + 0.5) * screenSize/texSize;
You of course already have caclulated (gl_Position.xy * 0.5) + 0.5) * screenSize implicitely, so this could be changed to:
vec2 texcoords = vertex.xy / texSize;
I am rendering a point based terrain from loaded heightmap data - but the points change their texturing depending on where the camera position is. To demonstrate the bug (and the fact that this isnt occuring from a z-buffering problem) I have taken screenshots with the points rendered at a fixed 5 pixel size from very slightly different camera positions (same angle), shown bellow:
PS: The images are large enough if you drag them into a new tab, didn't realise stack would scale them down this much.
State 1:
State 2:
The code to generate points is relatively simple so I'm posting this merely to rule out the option - mapArray is a single dimensional float array and copied to a VBO:
for(j = 0; j < mHeight; j++)
{
for(i = 0; i < mWidth; i++)
{
height = bitmapImage[k];
mapArray[k++] = 5 * i;
mapArray[k++] = height;
mapArray[k++] = 5 * j;
}
}
I find it more likely that I need to adjust my fragment shader because I'm not great with shaders- although I'm unsure where I could have gone wrong with such simple code and guess it's probably just not fit for purpose (with point based rendering). Bellow is my frag shader:
in varying vec2 TexCoordA;
uniform sampler2D myTextureSampler;
void main(){
gl_FragColor = texture2D(myTextureSampler, TexCoordA.st) * gl_Color;
}
Edit (requested info):
OpenGL version 4.4 no texture flags used.
TexCoordA is passed into the shader directly from my Vertex shader with no alterations at all. Self calculated UV's using this:
float* UVs = new float[mNumberPoints * 2];
k = 0;
for(j = 0; j < mHeight; j++)
{
for(i = 0; i < mWidth; i++)
{
UVs[k++] = (1.0f/(float)mWidth) * i;
UVs[k++] = (1.0f/(float)mHeight) * j;
}
}
This looks just like a subpixel accurate texture mapping side-effect. The problem with texture mapping implementation is that it needs to interpolate the texture coordinates on the actual rasterized pixels (fragments). When your camera is moving, the roundoff error from real position to the integer pixel position affects texture mapping, and is normally required for jitter-free animation (otherwise all the textures would jump by seemingly random subpixel amounts as the camera moves. There was a great tutorial on this topic by Paul Nettle.
You can try to fix this by not sampling texel corners but trying to sample texel centers (add half size of the texel to your point texture coordinates).
Another thing you can try is to compensate for the subpixel accurate rendering by calculating the difference between the rasterized integer coordinate (which you need to calculate yourself in a shader) and the real position. That could be enough to make the sampled texels more stable.
Finally, size matters. If your texture is large, the errors in the interpolation of the finite-precision texture coordinates can introduce these kinds of artifacts. Why not use GL_TEXTURE_2D_ARRAY with a separate layer for each color tile? You could also clamp the S and T texcoords to edge of the texture to avoid this more elegantly.
Just a guess: How are your point rendering parameters set? Perhaps the distance attenuation (GL_POINT_DISTANCE_ATTENUATION ) along with GL_POINT_SIZE_MIN and GL_POINT_SIZE_MAX are causing different fragment sizes depending on camera position. On the other hand I think I remember that when using a vertex shader these functionality is disabled and the vertex shader must decide about the size. I did it once by using
//point size calculation based on z-value as done by distance attenuation
float psFactor = sqrt( 1.0 / (pointParam[0] + pointParam[1] * camDist + pointParam[2] * camDist * camDist) );
gl_PointSize = pointParam[3] * psFactor;
where pointParam holds the three coefficients and the min point size:
uniform vec4 pointParam; // parameters for point size calculation [a b c min]
You may play around by setting your point size in the vertex shader directly with gl_PointSize = [value].
In my deferred renderer, the depth buffer holds values in the range from about 0.9700 to 1.0000. I found this out by drawing pixels in a given depth range black. This is the shader code I used.
bool inrange(in float position)
{
float z = texture2D(depth, coord).r;
return abs(position - z) < 0.0001;
}
void main()
{
if(inrange(0.970)) image = vec3(0);
else result = texture2D(image, coord);
}
And this is how it looks like. I tried several depth values. This image is for the depth value 0.998.
I use a as texture of the type GL_DEPTH24_STENCIL8 and format GL_FLOAT_32_UNSIGNED_INT_24_8_REV as depth and stencil attachment.
Is there an explanation for my depth values to be in this range? Is it default behavior? I would like them to be in the range of 0.0000 and 1.0000.
Sounds like a perspective near clipping plane very close to the origin. When building the perspective matrix try to set the near clipping as far as possible from the origin.
i need to get the color at a particular coordinate from a texture. There are 2 ways i can do this, by getting and looking at the raw png data, or by sampling my generated opengl texture. Is it possible to sample an opengl texture to get the color (RGBA) at a given UV or XY coord? If so, how?
Off the top of my head, your options are
Fetch the entire texture using glGetTexImage() and check the texel you're interested in.
Draw the texel you're interested in (eg. by rendering a GL_POINTS primitive), then grab the pixel where you rendered it from the framebuffer by using glReadPixels.
Keep a copy of the texture image handy and leave OpenGL out of it.
Options 1 and 2 are horribly inefficient (although you could speed 2 up somewhat by using pixel-buffer-objects and doing the copy asynchronously). So my favourite by FAR is option 3.
Edit: If you have the GL_APPLE_client_storage extension (ie. you're on a Mac or iPhone) then that's option 4 which is the winner by a long way.
The most efficient way I've found to do it is to access the texture data (you should have our PNG decoded to make into a texture anyway) and interpolate between the texels yourself. Assuming your texcoords are [0,1], multiply texwidthu and texheightv and then use that to find the position on the texture. If they're whole numbers, just use the pixel directly, otherwise use the int parts to find the bordering pixels and interpolate between them based on the fractional parts.
Here's some HLSL-like psuedocode for it. Should be fairly clear:
float3 sample(float2 coord, texture tex) {
float x = tex.w * coord.x; // Get X coord in texture
int ix = (int) x; // Get X coord as whole number
float y = tex.h * coord.y;
int iy = (int) y;
float3 x1 = getTexel(ix, iy); // Get top-left pixel
float3 x2 = getTexel(ix+1, iy); // Get top-right pixel
float3 y1 = getTexel(ix, iy+1); // Get bottom-left pixel
float3 y2 = getTexel(ix+1, iy+1); // Get bottom-right pixel
float3 top = interpolate(x1, x2, frac(x)); // Interpolate between top two pixels based on the fractional part of the X coord
float3 bottom = interpolate(y1, y2, frac(x)); // Interpolate between bottom two pixels
return interpolate(top, bottom, frac(y)); // Interpolate between top and bottom based on fractional Y coord
}
As others have suggested, reading back a texture from VRAM is horribly inefficient and should be avoided like the plague if you're even remotely interested in performance.
Two workable solutions as far as I know:
Keep a copy of the pixeldata handy (wastes memory though)
Do it using a shader