Precisely map World Position to UV Texture-Coordinates (OpenGL Compute Shader) - opengl

Question

I need help to precisely sample from my 3D Texture in the OpenGL (4.5) Compute Shader given a world position (within the domain of the texture dimensions). More precisely, I need help with my uv() function which maps world coordinates to the exact corresponding texture coordinates.
I want linear interpolation of the data, so my current approach uses texture(). But this results in errors around 0.001 compared to the expected values.
However, if I use texelFetch() and mix() to manually mimick the linear interpolation of texture() as stated in the specification (p. 248), I can reduce the error to 0.0000001 (which is desired). You can see an example of how I implemented it below in the Code section.
This is the function which I currently use inside the Compute Shader to calculate my uv-coordinates:
vec3 uv(const vec3 position) {
return (position + 0.5) / textureSize(tex[0], 0);
}
Though this one is often suggested across the internet, my results are not perfectly aligned.
Example
To elaborate, I have floating point data stored in a Texture as GL_RGB32F. For simplicity my example here uses scalar GL_R32F. The data has dimensions of, e.g., 20x20x20 (but can be arbitrary). I operate in the data domain [0, 19]^3 and want to exactly map my current position to the texture domain [0, 1]^3 to index the data at this position.
I have a test texture which alternates between 0 and 1 on the x-axis and therefore should interpolate for vec3(2.2, 0, 0) to 0.2.
As stated above, I tested texture() and texelFetch() + mix(). My manual interpolation evaluates to 0.200000003 which is fine. But calling texture() evaluates to 0.199218750, a quite high error compared. Strangely, manual interpolation and automatic interpolation evaluate to the same (correct) value for integer positions and the mid between integer positions (e.g., for vec3(2.0, 0, 0), vec3(2.5, 0, 0) and vec3(3.0, 0, 0)).
A visual example with actual calculated values:
uv(x, y, z) = ((x, y, z) + 0.5) / (20, 20, 20)
19| 1 |
| |
..| uv ..|
| (2.2, 3.0) ===> | (0.135, 0.175)
1 | x | x
|___________ |___________
0 1 .. 19 0 1
Code
I use C++, OpenGL 4.5 and globjects as a wrapper for OpenGL. The texture buffers are created and configured as depicted below.
// Texture buffer creation
t = globjects::Texture::createDefault(gl::GLenum::GL_TEXTURE_3D);
t->setParameter(gl::GL_TEXTURE_WRAP_S, gl::GL_CLAMP_TO_EDGE);
t->setParameter(gl::GL_TEXTURE_WRAP_T, gl::GL_CLAMP_TO_EDGE);
t->setParameter(gl::GL_TEXTURE_WRAP_R, gl::GL_CLAMP_TO_EDGE);
t->setParameter(gl::GL_TEXTURE_MIN_FILTER, gl::GL_LINEAR);
t->setParameter(gl::GL_TEXTURE_MAG_FILTER, gl::GL_LINEAR);
The Compute Shader is invocated.
// datatex holds image information
t->image3D(0, gl::GL_RGB32F, datatex->dimensions, 0, gl::GL_RGB, gl::GL_FLOAT, (const uint8_t*) datatex->data());
// ... (Make texture resident)
gl::glDispatchCompute(1, 1, 1);
// ... (Make texture not resident)
The Compute Shader, summarized to the important parts, is as follows:
#version 450
#extension GL_ARB_bindless_texture : enable
layout(local_size_x = 1, local_size_y = 1, local_size_z = 1) in;
layout(binding=0) uniform samplers
{
sampler3D tex[1];
};
vec3 uv(const vec3 position) {
return (position + 0.5) / textureSize(tex[0], 0);
}
void main() {
// Automatic interpolation
vec4 correct1 = texture(tex[0], uv(vec3(2.0,0,0), 0);
vec4 correct2 = texture(tex[0], uv(vec3(2.5,0,0), 0);
vec4 correct3 = texture(tex[0], uv(vec3(3.0,0,0), 0);
vec4 wrong = texture(tex[0], uv(vec3(2.1,0,0), 0);
// Manual interpolation on x-axis
vec3 pos = vec3(2.1,0,0);
vec4 v0 = texelFetch(tex[0], ivec3(floor(pos.x), pos.yz), 0);
vec4 v1 = texelFetch(tex[0], ivec3(ceil(pos.x), pos.yz), 0);
vec4 correct4 = mix(v0, v1, fract(pos.x));
}
I'd love your input, I'm at my end.. Thanks!
System
Also, I'm trying to achieve this on an NVIDIA GPU.

Answer 1

The texture units of GPUs are only needed to sample with 8bit precision in the fraction as of the D3D11 specs. This explains the small error which does not happen on (normalized) integer or mid-integer coordinates.
The fractional precision can also be queried in Vulkan via subTexelPrecisionBits and the online Vulkan database shows that there is no GPU as of today which offers more than 8 bits of precision in the fraction during sampling.
Performing linear interpolation in the shader itself offers the full float32 precision.