i've got a little problem with my water effect
as you can see here, it doesn't show up the right way.
another screen with a diffrent texture applied shows the error in the transform something more clearly
my HLSL code:
V2P vs(float4 inPos : POSITION, float2 inTex: TEXCOORD)
{
V2P Output = (V2P)0;
float4x4 viewproj = mul (matView, matProjection);
float4x4 worldviewproj = mul (matWorld,viewproj);
float4x4 reflviewproj = mul (matRLView, matProjection);
float4x4 reflworldviewproj = mul (matWorld, reflviewproj);
Output.Position = mul(inPos, worldviewproj);
Output.RLMapTex = mul(inPos, reflworldviewproj);
return Output;
}
P2F ps(V2P PSIn)
{
P2F Output = (P2F)0;
float2 ProjectedTexCoords;
ProjectedTexCoords.x = PSIn.RLMapTex.x / PSIn.RLMapTex.w /2.0f + 0.5f;
ProjectedTexCoords.y = -PSIn.RLMapTex.y / PSIn.RLMapTex.w /2.0f + 0.5f;
float2 ProjectedRefCoords;
ProjectedRefCoords.x = ( PSIn.Position.x / PSIn.Position.w) /2.0f + 0.5f;
ProjectedRefCoords.y = (-PSIn.Position.y / PSIn.Position.w) /2.0f + 0.5f;
Output.Color = tex2D(samRLMap, ProjectedTexCoords);
return Output;
}
the reflection map is rendered on a render target while flipping the y value of the eye along the waterheight. (and with up vector 0,-1,0)
so, my question: what could be the cuase of this?
I guess i found it, the matrix i used for the reflected view, is wrong.
When i use the standard view, it works fine
I'm not clear on why you are changing x. Doesn't it stay the same as y is flipped? As in
float2 ProjectedTexCoords;
ProjectedTexCoords.x = PSIn.RLMapTex.x / PSIn.RLMapTex.w;
ProjectedTexCoords.y = -PSIn.RLMapTex.y / PSIn.RLMapTex.w /2.0f + 0.5f;
Looks like a texture that is repeating its edge pixels. In other words, you might be doing a texture lookup beyond the texture boundaries. Are you sure your reflection map is big enough?
Maybe try setting the output colour to red if the texture coordinates are out of range? (I don't speak HLSL so I don't know the syntax for this, but I'm sure it is possible.)
Or enlarge the reflection map?
These kinds of issues can be hard to debug even if you can see the full source code, so this is more of a suggestion where to look, not an actual answer. My attempt at psychic debugging.
Related
So, basically, I'm trying to make a OBS Filter that displaces the pixels based on a lightmap/luminance map. I decided to learn how to make a filter by following this tutorial. But, in this tutorial, they don't explain much in terms of pixel displacement. So, I made a function that basically gets the brightness value of a texture I input and tested it by changing the pixel's alpha value with the red value of the texture:
float4 get_displacement(float2 position)
{
float2 pattern_uv = position / pattern_size;
float4 pattern_sample = pattern_texture.Sample(linear_wrap, pattern_uv / scale);
return pattern_sample;
}
float4 pixel_shader(pixel_data pixel) : TARGET
{
float4 source_sample = image.Sample(linear_wrap, pixel.uv);
if (pattern_size.x <= 0){
return source_sample;
}
float2 position = pixel.uv * float2(width, height);
float4 lightmap = get_displacement(position);
return float4(source_sample.rgb, lightmap.r);
return source_sample;
}
Which results to this (Note: The green is from a colour source that's behind the image to show the alpha value)
But, for some reason, when I try it with the vertex_shader, the function that decides where the pixel is rendered at, it seems to not work:
pixel_data vertex_shader(vertex_data vertex)
{
pixel_data pixel;
pixel.uv = vertex.uv;
if (pattern_size.x <= 0){
pixel.pos = mul(float4(vertex.pos.xyz, 1.0), ViewProj);
return pixel;
}
float2 position = vertex.uv * float2(width, height);
float4 lightmap = get_displacement(position);
pixel.pos = mul(float4(vertex.pos.x + (lightmap.r * testRamp1), vertex.pos.yz, 1.0), ViewProj);
return pixel;
}
(Note: testRamp1 is used as a value that I can change from a slider inside of OBS via some filter Properties)
The result that I'm expecting is something similar to this
To see if the issue was from me changing the XY position, I tested it using this function:
pixel_data vertex_shader(vertex_data vertex)
{
pixel_data pixel;
pixel.uv = vertex.uv;
pixel.pos = mul(float4(vertex.pos.x + 100, vertex.pos.yz, 1.0), ViewProj);
return pixel;
}
And it gave me an expected result.
I also changed the 100 with the testRamp1 value, and it works just the same based on the value of the slider.
So, I then tested if it was from the pixels needing to all move the same distance as each other. So, I change the function to this:
pixel_data vertex_shader(vertex_data vertex)
{
pixel_data pixel;
pixel.uv = vertex.uv;
pixel.pos = mul(float4(vertex.pos.x + (vertex.uv.x * testRamp1), vertex.pos.yz, 1.0), ViewProj);
return pixel;
}
Which then gives me either a squashed image when testRamp1 is set to a negative value, and it gives me a stretched image when it's set it to a positive value.
But as soon as I try to get the value of an image, may it be the pattern or from the source image, it no longer works(not even the filter parameters appear). For example, I used, this function to use the values of the source image:
pixel_data vertex_shader(vertex_data vertex)
{
pixel_data pixel;
float4 source_sample = image.Sample(linear_wrap, vertex.uv);
pixel.uv = vertex.uv;
pixel.pos = mul(float4(vertex.pos.x + (source_sample.r * testRamp1), vertex.pos.yz, 1.0), ViewProj);
return pixel;
}
At this point, I'm at a loss of words as to what could be causing this issue
First of all, the vertex shader is not what you want to use for this kind of effect. What you actually want to do, is to sample the image in the pixel shader, but offset the UV values slightly by your displacement before you pass them to the Sample function.
The primary reason, why you don't want to do this in the vertex shader, is, that the number of vertices is usually much smaller than the number of pixels - in the worst case, you only have 4 vertices in total (one for each corner of your screen), so the granuality of things you can do in the vertex shader is rather coarse. (Note: I'm not too familiar with OBS filters, and don't know how many vertices OBS dispatches, but certainly much less than the number of pixels you have on your screen).
Now, the reason why your vertex shader didn't work at all, is a bit more technical. In short, you can't use Sample in a vertex shader, you'd have to use SampleLevel or SampleGrad instead (note that these functions require more parameters). This is because Sample automatically calculates a UV gradient between adjacent pixels, to figure out the level of detail that is needed for your texture (whether or not it actually has multiple levels of detail). But the vertex shader operates on vertices, not on pixels, so the concept of an "adjacent pixel" doesn't make sense in a vertex shader - thus, the Sample method doesn't work.
I'm working on a WebGL project to create isolines on a 3D surface on macOS/amd GPU. My idea is to colour the pixels based on elevation in fragment shader. With some optimizations, I can achieve a relatively consistent line width and I am happy about that. However when I tested it on windows it behaves differently.
Then I figured out it's because of fwidth(). I use fwidth() to prevent fragment shader from coloring the whole horizontal plane when it happens to locate at a isolevel. Please see the screenshot:
I solved this issue by adding the follow glsl line:
if (fwidth(vPositionZ) < 0.001) { /**then do not colour isoline on these pixels**/ };
It works very well on macOS since I got this:
.
However, on windows/nvidia GPU all isolines are gone because fwidth(vPositionZ) always evaluates to 0.0. Which doesn't make sense to me.
What am I doing wrong? Is there any better way to solve the issue presented in the first screenshot? Thank you all!
EDIT:
Here I attach my fragment shader. It's simplified but I think that's all relevant. I know looping is slow but for now I'm not worried about it.
uniform float zmin; // min elevation
uniform vec3 lineColor;
varying float vPositionZ; // elevation value for each vertex
float interval;
vec3 originColor = finalColor.rgb; // original surface color
for ( int i = 0; i < COUNT; i ++ ) {
float elevation = zmin + float( i + 1 ) * interval;
lineColor = mix( originColor, lineColor, step( 0.001, fwidth(vPositionZ)));
if ( vPositionZ <= elevation + lineWidth && vPositionZ >= elevation - lineWidth ) {
finalColor.rgb = lineColor;
}
// same thing but without condition:
// finalColor.rgb = mix( mix( originColor, lineColor, step(elevation - lineWidth, vPositionZ) ),
// originColor,
// step(elevation + lineWidth, vPositionZ) );
}
gl_FragColor = finalColor;
Environment: WebGL2.0, es version 300, chrome browser.
Put fwidth(vPosistionZ) before the loop will work. Otherwise, fwidth() evaluates anything to 0 if it's inside a loop.
I suspect this is a bug with Nvidia GPU.
I am currently working on my first OpenGL based game engine. I need normal mapping as a feature, but it isn't working correctly.
Here is an animation of what is Happening
The artifacts are affected by the angle between the light and the normals on the surface. Camera movement does not affect it in any way. I am also (at least for now) going the route of the less efficient method where the normal extracted from the normal map is converted into view space rather than converting everything to tangent space.
Here are the relevant pieces of my code:
Generating Tangents and Bitangents
for(int k=0;k<(int)mb->getIndexCount();k+=3)
{
unsigned int i1 = mb->getIndex(k);
unsigned int i2 = mb->getIndex(k+1);
unsigned int i3 = mb->getIndex(k+2);
JGE_v3f v0 = mb->getVertexPosition(i1);
JGE_v3f v1 = mb->getVertexPosition(i2);
JGE_v3f v2 = mb->getVertexPosition(i3);
JGE_v2f uv0 = mb->getVertexUV(i1);
JGE_v2f uv1 = mb->getVertexUV(i2);
JGE_v2f uv2 = mb->getVertexUV(i3);
JGE_v3f deltaPos1 = v1-v0;
JGE_v3f deltaPos2 = v2-v0;
JGE_v2f deltaUV1 = uv1-uv0;
JGE_v2f deltaUV2 = uv2-uv0;
float ur = deltaUV1.x * deltaUV2.y - deltaUV1.y * deltaUV2.x;
if(ur != 0)
{
float r = 1.0 / ur;
JGE_v3f tangent;
JGE_v3f bitangent;
tangent = ((deltaPos1 * deltaUV2.y) - (deltaPos2 * deltaUV1.y)) * r;
tangent.normalize();
bitangent = ((deltaPos1 * -deltaUV2.x) + (deltaPos2 * deltaUV1.x)) * r;
bitangent.normalize();
tans[i1] += tangent;
tans[i2] += tangent;
tans[i3] += tangent;
btans[i1] += bitangent;
btans[i2] += bitangent;
btans[i3] += bitangent;
}
}
Calculating the TBN matrix in the Vertex Shader
(mNormal corrects the normal for non-uniform scales)
vec3 T = normalize((mVW * vec4(tangent, 0.0)).xyz);
tnormal = normalize((mNormal * n).xyz);
vec3 B = normalize((mVW * vec4(bitangent, 0.0)).xyz);
tmTBN = transpose(mat3(
T.x, B.x, tnormal.x,
T.y, B.y, tnormal.y,
T.z, B.z, tnormal.z));
Finally here is where I use the sampled normal from the normal map and attempt to convert it to view space in the Fragment Shader
fnormal = normalize(nmapcolor.xyz * 2.0 - 1.0);
fnormal = normalize(tmTBN * fnormal);
"nmapcolor" is the sampled color from the normal map.
"fnormal" is then used like normal in the lighting calculations.
I have been trying to solve this for so long and have absolutely no idea how to get this working. Any help would be greatly appreciated.
EDIT - I slightly modified the code to work in world space and outputted the results. The big platform does not have normal mapping (and it works correctly) while the smaller platform does.
I added in what direction the normals are facing. They should both be generally the same color, but they're clearly different. Seems the mTBN matrix isn't transforming the tangent space normal into world (and normally view) space properly.
Well... I solved the problem. Turns out my normal mapping implementation was perfect. The problem actually was in my texture class. This is, of course, my first time writing an OpenGL rendering engine, and I did not realize that the unlock() function in my texture class saved ALL my textures as GL_SRGB_ALPHA including normal maps. Only diffuse map textures should be GL_SRGB_ALPHA. Temporarily forcing all textures to load as GL_RGBA fixed the problem.
Can't believe I had this problem for 11 months, only to find it was something so small.
I have been searching online for a while now on why my geometric attenuation term for my physically based shader (Which I posted a question about not too long ago) and I cannot seem to come up with a result. The function I'm trying to implement can be found here: http://blog.selfshadow.com/publications/s2013-shading-course/karis/s2013_pbs_epic_notes_v2.pdf
This is my current iteration of the function.
vec3 Gsub(vec3 v) // Sub Function of G
{
float k = ((roughness + 1) * (roughness + 1)) / 8;
float fdotv = dot(fNormal, v);
return vec3((fdotv) / ((fdotv) * (1.0 - k) + k));
}
vec3 G(vec3 l, vec3 v, vec3 h) // Geometric Attenuation Term - Schlick Modified (k = a/2)
{
return Gsub(l) * Gsub(v);
}
This is the current result of the above in my application:
You can clearly see the strange artifacts on the left side, which should not be present.
One of the things I thought was an issue was my normals. I believe this is the issue, because whenever I put the same function into the Disney BRDF editor (http://www.disneyanimation.com/technology/brdf.html) I get correct results. I believe it is the normals because whenever I view the normals in Disney's application, I get this.
These normals differ from my normals, which -should- be correct:
I use the same model in both applications, and the normals are stored inside the model file. Can anyone give any insight into this?
Additionally I'd like to mention that these are the operations done on my normals:
Vertex Shader
mat3 normalMatrix = mat3(transpose(inverse(ModelView)));
inputNormal = normalize(normalMatrix * vNormal);
Fragment Shader
fNormal = normalize(inputNormal);
P.S. Please excuse my rushy-code, I've been trying to get this to work for a while.
I am writing a viewer for a proprietary mesh & animation format in OpenGL.
During rendering a transformation matrix is created for each bone (node) and is applied to the vertices that bone is attached to.
It is possible for a bone to be marked as "Billboarded" which as most everyone knows, means it should always face the camera.
So the idea is to generate a matrix for that bone which when used to transform the vertices it's attached to, causes the vertices to be billboarded.
On my test model it should look like this:
However currently it looks like this:
Note, that despite its incorrect orientation, it is billboarded. As in no matter which direction the camera looks, those vertices are always facing that direction at that orientation.
My code for generating the matrix for bones marked as billboarded is:
mat4 view;
glGetFloatv(GL_MODELVIEW_MATRIX, (float*)&view);
vec4 camPos = vec4(-view[3].x, -view[3].y, -view[3].z,1);
vec3 camUp = vec3(view[0].y, view[1].y, view[2].y);
// zero the translation in the matrix, so we can use the matrix to transform
// camera postion to world coordinates using the view matrix
view[3].x = view[3].y = view[3].z = 0;
// the view matrix is how to get to the gluLookAt pos from what we gave as
// input for the camera position, so to go the other way we need to reverse
// the rotation. Transposing the matrix will do this.
{
float * matrix = (float*)&view;
float temp[16];
// copy this into temp
memcpy(temp, matrix, sizeof(float) * 16);
matrix[1] = temp[4]; matrix[4] = temp[1];
matrix[2] = temp[8]; matrix[8] = temp[2];
matrix[6] = temp[9]; matrix[9] = temp[6];
}
// get the correct position of the camera in world space
camPos = view * camPos;
//vec3 pos = pivot;
vec3 look = glm::normalize(vec3(camPos.x-pos.x,camPos.y-pos.y,camPos.z-pos.z));
vec3 right = glm::cross(camUp,look);
vec3 up = glm::cross(look,right);
mat4 bmatrix;
bmatrix[0].x = right.x;
bmatrix[0].y = right.y;
bmatrix[0].z = right.z;
bmatrix[0].w = 0;
bmatrix[1].x = up.x;
bmatrix[1].y = up.y;
bmatrix[1].z = up.z;
bmatrix[1].w = 0;
bmatrix[2].x = look.x;
bmatrix[2].y = look.y;
bmatrix[2].z = look.z;
bmatrix[2].w = 0;
bmatrix[3].x = pos.x;
bmatrix[3].y = pos.y;
bmatrix[3].z = pos.z;
bmatrix[3].w = 1;
I am using GLM to do the math involved.
Though this part of the code is based off of the tutorial here, other parts of the code are based off of an open source program similar to the one I'm building. However that program was written for DirectX and I haven't had much luck directly converting. The (working) directX code for billboarding looks like this:
D3DXMatrixRotationY(&CameraRotationMatrixY, -Camera.GetPitch());
D3DXMatrixRotationZ(&CameraRotationMatrixZ, Camera.GetYaw());
D3DXMatrixMultiply(&CameraRotationMatrix, &CameraRotationMatrixY, &CameraRotationMatrixZ);
D3DXQuaternionRotationMatrix(&CameraRotation, &CameraRotationMatrix);
D3DXMatrixTransformation(&CameraRotationMatrix, NULL, NULL, NULL, &ModelBaseData->PivotPoint, &CameraRotation, NULL);
D3DXMatrixDecompose(&Scaling, &Rotation, &Translation, &BaseMatrix);
D3DXMatrixTransformation(&RotationMatrix, NULL, NULL, NULL, &ModelBaseData->PivotPoint, &Rotation, NULL);
D3DXMatrixMultiply(&TempMatrix, &CameraRotationMatrix, &RotationMatrix);
D3DXMatrixMultiply(&BaseMatrix, &TempMatrix, &BaseMatrix);
Note the results are stored in baseMatrix in the directX version.
EDIT2: Here's the code I came up with when I tried to modify my code according to datenwolf's suggestions. I'm pretty sure I made some mistakes still. This attempt creates heavily distorted results with one end of the object directly in the camera.
mat4 view;
glGetFloatv(GL_MODELVIEW_MATRIX, (float*)&view);
vec3 pos = vec3(calculatedMatrix[3].x,calculatedMatrix[3].y,calculatedMatrix[3].z);
mat4 inverted = glm::inverse(view);
vec4 plook = inverted * vec4(0,0,0,1);
vec3 look = vec3(plook.x,plook.y,plook.z);
vec3 right = orthogonalize(vec3(view[0].x,view[1].x,view[2].x),look);
vec3 up = orthogonalize(vec3(view[0].y,view[1].y,view[2].y),look);
mat4 bmatrix;
bmatrix[0].x = right.x;
bmatrix[0].y = right.y;
bmatrix[0].z = right.z;
bmatrix[0].w = 0;
bmatrix[1].x = up.x;
bmatrix[1].y = up.y;
bmatrix[1].z = up.z;
bmatrix[1].w = 0;
bmatrix[2].x = look.x;
bmatrix[2].y = look.y;
bmatrix[2].z = look.z;
bmatrix[2].w = 0;
bmatrix[3].x = pos.x;
bmatrix[3].y = pos.y;
bmatrix[3].z = pos.z;
bmatrix[3].w = 1;
calculatedMatrix = bmatrix;
vec3 orthogonalize(vec3 toOrtho, vec3 orthoAgainst) {
float bottom = (orthoAgainst.x*orthoAgainst.x)+(orthoAgainst.y*orthoAgainst.y)+(orthoAgainst.z*orthoAgainst.z);
float top = (toOrtho.x*orthoAgainst.x)+(toOrtho.y*orthoAgainst.y)+(toOrtho.z*orthoAgainst.z);
return toOrtho - top/bottom*orthoAgainst;
}
Creating a parallel to view billboard matrix is as simple as setting the upper left 3×3 submatrix of the total modelview matrix to identity. There are only some cases where you actually require the actual look vector.
Anyway, you're thinking far too complicated. All your tinkering with the matrix completely misses the point. Namely that the modelview transformation assumes that the camera is always at (0,0,0) and moves world and models in opposite. What you try to do is finding the vector in model space that points towards the camera. Which is simply the vector that will point toward (0,0,0) after transformation.
So all we have to do is invert the modelview matrix and transform (0,0,0,1) with it. That's your look vector. For your calculations of right and up vectors orthogonalize the 1st (X) and 2nd (Y) column of the modelview matrix against that look vector.
Figured it out myself. It turns out the model format I'm using uses different axes for billboarding. Most billboarding implementations (including the one I used) use the X,Y coordinates to position the billboarded object. The format I was reading uses Y and Z.
The thing to look for is that there was a billboarding effect, but facing the wrong direction. To fix this I played with the different camera vectors until I arrived at the correct matrix calculation:
bmatrix[1].x = right.x;
bmatrix[1].y = right.y;
bmatrix[1].z = right.z;
bmatrix[1].w = 0;
bmatrix[2].x = up.x;
bmatrix[2].y = up.y;
bmatrix[2].z = up.z;
bmatrix[2].w = 0;
bmatrix[0].x = look.x;
bmatrix[0].y = look.y;
bmatrix[0].z = look.z;
bmatrix[0].w = 0;
bmatrix[3].x = pos.x;
bmatrix[3].y = pos.y;
bmatrix[3].z = pos.z;
bmatrix[3].w = 1;
My attempts to follow datenwolf's advice did not succeed and at this time he hasn't offered any additional explanation so I'm unsure why. Thanks anyways!