openGL cubemap reflections in view space are wrong - opengl

Question

I am following this tutorial and I managed to add a cube map to my scene. Then I tried to add reflections to my object, differently from the tutorial, I made my GLSL code in view space. However, the reflections seem a bit off. They are always reflecting the same side whatever angle you are facing, in my case, you always see a rock on the reflected object, but the rock is only on one side of my cube map.
Here is a video showing the effect:
.
I tried with other shaped objects, like a cube, and the effect is the same. I also found this book, that shows an example of a view space reflections, and it seems I am doing something similar to it, but it still won't result in the desired effect.
My vertex shader code:
#version 330 core
layout (location = 0) in vec3 aPos;
layout (location = 1) in vec3 normal;
layout (location = 2) in vec2 aTexCoord;
uniform mat4 Model;
uniform mat4 View;
uniform mat4 Projection;
out vec2 TexCoord;
out vec3 aNormal;
out vec3 FragPos;
void main()
{
aNormal = mat3(transpose(inverse(View * Model))) * normal;
FragPos = vec3(View * Model * vec4(aPos,1.0));
gl_Position = Projection * vec4(FragPos,1.0);
TexCoord = aTexCoord;
}
My vertex code:
#version 330 core
out vec4 FragColor;
in vec3 FragPos;
in vec3 aNormal;
uniform samplerCube skybox;
void main(){
vec3 I = normalize(FragPos);
vec3 R = reflect(I,normalize(aNormal));
FragColor = vec4(texture(skybox,R).rgb,1.0);
}

Answer 1

Since you do the computations, in the fragment shader, in view space, the reflected vector (R) is a vector in view space, too. The cubemap (skybox) represents a map of the environment, in world space.
You have to transform R form view space to world space. That can be done by the inverse view matrix The inverse matrix can be computed by the glsl built-in function inverse:
#version 330 core
out vec4 FragColor;
in vec3 FragPos;
in vec3 aNormal;
uniform samplerCube skybox;
uniform mat4 View;
void main() {
vec3 I = normalize(FragPos);
vec3 viewR = reflect(I, normalize(aNormal));
vec3 worldR = inverse(mat3(View)) * viewR;
FragColor = vec4(texture(skybox, worldR).rgb, 1.0);
}
Note, the view matrix transforms form world space to view space, this the inverse view matrix transforms form view space to world space. See also Invertible matrix.

Answer 2

This is a late answer but I just wanted to give additional information why it is behaving like this.
Imagine your reflective object is simply a 6 sided cube. Each face can be thought of as a mirror. Now because you are in view space every coordinate of that mirror plane that is visible from your viewpoint does have a negative z value. Lets us look at the point directly at the center. This vector looks like (0, 0, -z) and because the side of the cube is like a mirror it will get reflected directly back to you (0, 0, +z). So you end up sampling from GL_TEXTURE_CUBE_MAP_POSITIVE_Z of your cube map.
In shader code it looks like:
vec3 V = normalize(-frag_pos_view_space); // vector from fragment to view point (0,0,0) in view space
vec3 R = reflect(-V, N); // invert V because reflect expects incident vector
vec3 color = texture(skybox, R).xyz;
Now, let us move to the other side of the cube and look at that mirror plane. In view space, the coordinate you are looking at is still (0,0,-z) at the center, will be reflected around the normal and gets back to you, so the reflected vector again looks like (0,0, +z). This means even if you are at the other side of your cube you will sample the same face in your cube map.
So what you have to do is go back into world space using the inverse of your view matrix. If in addition then you rendered the skybox itself by applying a rotation you will even have to transform your reflected vector with the inverse of the model matrix that you used to transform the skybox otherwise the reflections will still be wrong.