Flying camera using glm is acting weird - c++

Question

I´m working on a Vulkan application and want to implement a "flying camera" which can move around anywhere freely.
I placed a cube in the middle of the scene that I can fly around.
The camera position works perfectly but the rotation acts weirdly as soon as I move around the cube.
When I´m on the opposite side of the cube the UP and DOWN directions that I give via my mouse are inverted and when I´m on either side of the cube they just don´t work at all. Anywhere between that it just does weird circles. Note that this only affects up and down movements not left or right.
Here is a demonstration of how it looks for me where I only move my mouse up and down in this order on every side of the cube (except when moving around it) I apologize for the bad frame rate, i had to convert it to gif and lower the quality:
https://imgur.com/HxknkQV
Quick explanation of the video:
Firstly, while it might have looked like I moved my mouse left and right while not moving looking at the sides, I didn´t. I only went up and down every time except for when I moved into the positions. The other thing is that while for the opposite site of the cube it might have looked like the rotation worked, it was actually inverted.
This is the code for my Camera:
#pragma once
#define GLM_FORCE_DEPTH_ZERO_TO_ONE
#include <glm/glm.hpp>
#include <glm/gtc/matrix_transform.hpp>
#include <glm/gtx/rotate_vector.hpp>
class Camera {
private:
glm::mat4 _model;
glm::mat4 _view;
glm::mat4 _projection;
glm::vec3 _position;
glm::vec3 _up;
glm::vec3 _moveSpeed = glm::vec3(0.08f);
float _mouseSens = 0.005f;
public:
glm::vec3 _direction;
enum MovementType { FORWARD, BACKWARD, LEFT, RIGHT, UP, DOWN };
Camera(uint32_t width, uint32_t height){
_model = glm::mat4(1.0f);
_projection = glm::perspective(glm::radians(90.0f), width / (float)height, 0.01f, 100.0f);
_direction = glm::vec3(-2, 0, 0);
_up = glm::vec3(0.0f, 0.0f, 1.0f);
_position = glm::vec3(2.0f, 0.0f, 0.0f);
_projection[1][1] *= -1; //Because Vulkan uses different axis, hence the up Vector being different to an OpenGL application for example
}
void rotate(float amount, glm::vec3 axis){
_direction = glm::rotate(_direction, amount * _mouseSens, axis);
}
void move(MovementType movement) {
switch (movement)
{
case FORWARD:
_position += _direction * _moveSpeed;
break;
case BACKWARD:
_position -= _direction * _moveSpeed;
break;
case LEFT:
_position -= glm::normalize(glm::cross(_direction, _up)) * _moveSpeed;
break;
case RIGHT:
_position += glm::normalize(glm::cross(_direction, _up)) * _moveSpeed;
break;
case UP:
_position += _up * _moveSpeed;
break;
case DOWN:
_position -= _up * _moveSpeed;
break;
}
}
glm::mat4 getMVP() {
_view = glm::lookAt(_position, _position + _direction, _up);
return _projection * _view * _model;
}
};
Any help would be gladly appreciated as I am not really that good in Vector and Matrix calculations, and really don´t know how to fix this. Thanks.

Answer 1

It looks to me as if you were rotating the camera in world space (but I can't tell for sure because the code that invokes Camera::rotate is not included in your question).
If my assumption is correct, rotating in camera space should solve the problem. I.e. assuming that Camera::rotate performs a rotation relative to the axes of the current camera's space, you'll have to transform that back into world space, which can be done with the inverse of _view:
void rotate(float amount, glm::vec3 axis){
auto directionCamSpace = glm::rotate(_direction, amount * _mouseSens, axis);
_directionWorldSpace = glm::mat3(glm::inverse(_view)) * _direction;
}
And then use _directionWorldSpace with glm::lookAt:
glm::mat4 getMVP() {
_view = glm::lookAt(_position, _position + _directionWorldSpace, _up);
return _projection * _view * _model;
}
I am afraid that it might be that this does not lead you to the final solution of your problem yet and that further/other artefacts occur, but it should at least get you one step further.
The best way to implement such a camera would probably be to use quaternions to track the rotation of the camera, rotate the camera's coordinate system with the accumulated quaternion rotations and then use glm::inverse to compute the view matrix from the rotated camera's coordinate system. (You wouldn't need glm::lookAt at all with that approach.)