How to correctly represent 3D rotation in games - opengl

In most 3D platform games, only rotation around the Y axis is needed since the player is always positioned upright.
However, for a 3D space game where the player needs to be rotated on all axises, what is the best way to represent the rotation?
I first tried using Euler angles:
glRotatef(anglex, 1.0f, 0.0f, 0.0f);
glRotatef(angley, 0.0f, 1.0f, 0.0f);
glRotatef(anglez, 0.0f, 0.0f, 1.0f);
The problem I had with this approach is that after each rotation, the axises change. For example, when anglex and angley are 0, anglez rotates the ship around its wings, however if anglex or angley are non zero, this is no longer true. I want anglez to always rotate around the wings, irrelevant of anglex and angley.
I read that quaternions can be used to exhibit this desired behavior however was unable to achieve it in practice.
I assume my issue is due to the fact that I am basically still using Euler angles, but am converting the rotation to its quaternion representation before usage.
struct quaternion q = eulerToQuaternion(anglex, angley, anglez);
struct matrix m = quaternionToMatrix(q);
glMultMatrix(&m);
However, if storing each X, Y, and Z angle directly is incorrect, how do I say "Rotate the ship around the wings (or any consistent axis) by 1 degree" when my rotation is stored as a quaternion?
Additionally, I want to be able to translate the model at the angle that it is rotated by. Say I have just a quaternion with q.x, q.y, q.z, and q.w, how can I move it?

Quaternions are very good way to represent rotations, because they are efficient, but I prefer to represent the full state "position and orientation" by 4x4 matrices.
So, imagine you have a 4x4 matrix for every object in the scene. Initially, when the object is unrotated and untraslated, this matrix is the identity matrix, this is what I will call "original state". Suppose, for instance, the nose of your ship points towards -z in its original state, so a rotation matrix that spin the ship along the z axis is:
Matrix4 around_z(radian angle) {
c = cos(angle);
s = sin(angle);
return Matrix4(c, -s, 0, 0,
s, c, 0, 0,
0, 0, 1, 0,
0, 0, 0, 1);
}
now, if your ship is anywhere in space and rotated to any direction, and lets call this state t, if you want to spin the ship around z axis for an angle amount as if it was on its "original state", it would be:
t = t * around_z(angle);
And when drawing with OpenGL, t is what you multiply for every vertex of that ship. This assumes you are using column vectors (as OpenGL does), and be aware that matrices in OpenGL are stored columns first.
Basically, your problem seems to be with the order you are applying your rotations. See, quaternions and matrices multiplication are non-commutative. So, if instead, you write:
t = around_z(angle) * t;
You will have the around_z rotation applied not to the "original state" z, but to global coordinate z, with the ship already affected by the initial transformation (roatated and translated). This is the same thing when you call the glRotate and glTranslate functions. The order they are called matters.
Being a little more specific for your problem: you have the absolute translation trans, and the rotation around its center rot. You would update each object in your scene with something like:
void update(quaternion delta_rot, vector delta_trans) {
rot = rot * delta_rot;
trans = trans + rot.apply(delta_trans);
}
Where delta_rot and delta_trans are both expressed in coordinates relative to the original state, so, if you want to propel your ship forward 0.5 units, your delta_trans would be (0, 0, -0.5). To draw, it would be something like:
void draw() {
// Apply the absolute translation first
glLoadIdentity();
glTranslatevf(&trans);
// Apply the absolute rotation last
struct matrix m = quaternionToMatrix(q);
glMultMatrix(&m);
// This sequence is equivalent to:
// final_vertex_position = translation_matrix * rotation_matrix * vertex;
// ... draw stuff
}
The order of the calls I choose by reading the manual for glTranslate and glMultMatrix, to guarantee the order the transformations are applied.
About rot.apply()
As explained at Wikipedia article Quaternions and spatial rotation, to apply a rotation described by quaternion q on a vector p, it would be rp = q * p * q^(-1), where rp is the newly rotated vector. If you have a working quaternion library implemented on your game, you should either already have this operation implemented, or should implement it now, because this is the core of using quaternions as rotations.
For instance, if you have a quaternion that describes a rotation of 90° around (0,0,1), if you apply it to (1,0,0), you will have the vector (0,1,0), i.e. you have the original vector rotated by the quaternion. This is equivalent to converting your quaternion to matrix, and doing a matrix to colum-vector multiplication (by matrix multiplication rules, it yields another column-vector, the rotated vector).

Related

How to Pitch Camera Around Origin

I am trying to implement a camera which orbits around the origin, where I have successfully implemented the ability to yaw using the gluLookat function. I am trying to implement pitch, but have a few issues with the outcome (pitch only works if I yaw to a certain point and then pitch).
Here is my attempt so far:
float distance, // radius (from origin) updated by -, + keys
pitch, // angle in degrees updated from W, S keys (increments of +- 10)
yaw; // angle in degrees updated from A, D keys (increments of +- 10)
view = lookAt(
Eigen::Vector3f(distance * sin(toRadians(pitch)) * cos(toRadians(yaw)), distance * sin(toRadians(pitch)) * sin(toRadians(yaw)), distance * cos(toRadians(pitch))),
Eigen::Vector3f(0.0f, 0.0f, 0.0f),
Eigen::Vector3f(0.0f, 0.0f, 1.0f));
proj = perspective(toRadians(90.0f), static_cast<float>(width) / height, 1.0f, 10.0f);
I feel like my issue is the Up vector, but I'm not sure how to update it properly(and at the same time I think its fine, as I always want the orientation of the camera to stay the same, I really just want to move the position of the camera)
Edit: I wanted to add that I'm calculating the position based info found here: http://tutorial.math.lamar.edu/Classes/CalcIII/SphericalCoords.aspx I'm not sure if the math discussed here directly translates over so please correct me if wrong.
It might be a matter of interpretation. Your code looks correct but pitch might not have the meaning that you think.
When pitch is 0, the camera is located at the north pole of the sphere (0, 0, 1). This is a bit problematic since your up-vector and view direction become parallel and you will not get a valid transform. Then, when pitch increases, the camera moves south until it reaches the south pole when pitch=PI. Your code should work for any point that is not at the poles. You might want to swap sin(pitch) and cos(pitch) to start at the equator when pitch=0 (and support positive and negative pitch).
Actually, I prefer to model this kind of camera more directly as a combination of matrices:
view = Tr(0, 0, -distance) * RotX(-pitch) * RotY(-yaw)
Tr is a translation matrix, RotX is a rotation about the x-axis, and RotY is a rotation about the y-axis. This assumes that the y-axis is up. If you want another axis to be up, you can just add an according rotation matrix. E.g., if you want the z-axis to be up, then
view = Tr(0, 0, -distance) * RotX(-pitch) * RotY(-yaw) * RotX(-Pi/2)

Converting glm::lookat matrix to quaternion and back

I am using glm to create a camera class, and I am running into some problems with a lookat function. I am using a quaternion to represent rotation, but I want to use glm's prewritten lookat function to avoid duplicating code. This is my lookat function right now:
void Camera::LookAt(float x, float y, float z) {
glm::mat4 lookMat = glm::lookAt(position, glm::vec3(x, y, z), glm::vec3(0, 1, 0));
rotation = glm::toQuat(lookMat);
}
However when I call LookAt(0.0f,0.0f,0.0f), my camera is not rotated to that point. When I call glm::eulerangles(rotation) after the lookat call, I get a vec3 with the following values: (180.0f, 0.0f, 180.0f). position is (0.0f,0.0f,-10.0f), so I should not have any rotation at all to look at 0,0,0. This is the function which builds the view matrix:
glm::mat4 Camera::GetView() {
view = glm::toMat4(rotation) * glm::translate(glm::mat4(), position);
return view;
}
Why am I not getting the correct quaternion, and how can I fix my code?
Solution:
You have to invert the rotation of the quaternion by conjugating it:
using namespace glm;
quat orientation = conjugate(toQuat(lookAt(vecA, vecB, up)));
Explanation:
The lookAt function is a replacement for gluLookAt, which is used to construct a view matrix.
The view matrix is used to rotate the world around the viewer, and is therefore the inverse of the cameras transform.
By taking the inverse of the inverse, you can get the actual transform.
I ran into something similar, the short answer is your lookMat might need to be inverted/transposed, because it is a camera rotation (at least in my case), as opposed to a world rotation. Rotating the world would be a inverse of a camera rotation.
I have a m_current_quat which is a quaternion that stores the current camera rotation. I debugged the issue by printing out the matrix produced by glm::lookAt, and comparing with the resulting matrix that I get by applying m_current_quat and a translation by m_camera_position. Here is the relevant code for my test.
void PrintMatrix(const GLfloat m[16], const string &str)
{
printf("%s:\n", str.c_str());
for (int i=0; i<4; i++)
{
printf("[");
//for (int j=i*4+0; j<i*4+4; j++) // row major, 0, 1, 2, 3
for (int j=i+0; j<16; j+=4) // OpenGL is column major by default, 0, 4, 8, 12
{
//printf("%d, ", j); // print matrix index
printf("%.2f, ", m[j]);
}
printf("]\n");
}
printf("\n");
}
void CameraQuaternion::SetLookAt(glm::vec3 look_at)
{
m_camera_look_at = look_at;
// update the initial camera direction and up
//m_initial_camera_direction = glm::normalize(m_camera_look_at - m_camera_position);
//glm::vec3 initial_right_vector = glm::cross(m_initial_camera_direction, glm::vec3(0, 1, 0));
//m_initial_camera_up = glm::cross(initial_right_vector, m_initial_camera_direction);
m_camera_direction = glm::normalize(m_camera_look_at - m_camera_position);
glm::vec3 right_vector = glm::cross(m_camera_direction, glm::vec3(0, 1, 0));
m_camera_up = glm::cross(right_vector, m_camera_direction);
glm::mat4 lookat_matrix = glm::lookAt(m_camera_position, m_camera_look_at, m_camera_up);
// Note: m_current_quat quat stores the camera rotation with respect to the camera space
// The lookat_matrix produces a transformation for world space, where we rotate the world
// with the camera at the origin
// Our m_current_quat need to be an inverse, which is accompolished by transposing the lookat_matrix
// since the rotation matrix is orthonormal.
m_current_quat = glm::toQuat(glm::transpose(lookat_matrix));
// Testing: Make sure our model view matrix after gluLookAt, glmLookAt, and m_current_quat agrees
GLfloat current_model_view_matrix[16];
//Test 1: gluLookAt
gluLookAt(m_camera_position.x, m_camera_position.y, m_camera_position.z,
m_camera_look_at.x, m_camera_look_at.y, m_camera_look_at.z,
m_camera_up.x, m_camera_up.y, m_camera_up.z);
glGetFloatv(GL_MODELVIEW_MATRIX, current_model_view_matrix);
PrintMatrix(current_model_view_matrix, "Model view after gluLookAt");
//Test 2: glm::lookAt
lookat_matrix = glm::lookAt(m_camera_position, m_camera_look_at, m_camera_up);
PrintMatrix(glm::value_ptr(lookat_matrix), "Model view after glm::lookAt");
//Test 3: m_current_quat
glLoadIdentity();
glMultMatrixf( glm::value_ptr( glm::transpose(glm::mat4_cast(m_current_quat))) );
glTranslatef(-m_camera_position.x, -m_camera_position.y, -m_camera_position.z);
glGetFloatv(GL_MODELVIEW_MATRIX, current_model_view_matrix);
PrintMatrix(current_model_view_matrix, "Model view after quaternion transform");
return;
}
Hope this helps.
I want to use glm's prewritten lookat function to avoid duplicating code.
But it's not duplicating code. The matrix that comes out of glm::lookat is just a mat4. Going through the conversion from a quaternion to 3 vectors, only so that glm::lookat can convert it back into an orientation is just a waste of time. You've already done 85% of lookat's job; just do the rest.
You are getting the (or better: a) correct rotation.
When I call glm::eulerangles(rotation) after the lookat call, I get a
vec3 with the following values: (180.0f, 0.0f, 180.0f). position is
(0.0f,0.0f,-10.0f), so I should not have any rotation at all to look
at 0,0,0.
glm is following the conventions of the old fixed-function GL. And there, eye space was defined as the camera placed at origin, with x pointng to the right, y up and looking in -z direction. Since you want to look in positive z direction, the camera has to turn. Now, as a human, I would have described that as a rotation of 180 degrees around y, but a rotation of 180 degrees around x in combination with another 180 degrees rotation aroundz will have the same effect.
When multiplied by the LookAt view matrix, the world-space vectors are rotated (brought) into the camera's view while the camera's orientation is kept in place.
So an actual rotation of the camera by 45 degress to the right is achieved with a matrix which applies a 45 degree rotation to the left to all the world-space vertices.
For a Camera object you would need to get its local forward and up direction vectors in order to calculate a lookAt view matrix.
viewMatrix = glm::lookAtLH (position, position + camera_forward, camera_up);
When using quaternions to store the orientation of an object (be it a camera or anything else), usually this rotation quat is used to calculate the vectors which define its local-space (left-handed one in the below example):
glm::vec3 camera_forward = rotation * glm::vec3(0,0,1); // +Z is forward direction
glm::vec3 camera_right = rotation * glm::vec3(1,0,0); // +X is right direction
glm::vec3 camera_up = rotation * glm::vec3(0,1,0); // +Y is up direction
Thus, the world-space directions should be rotated 45 degress to the right in order to reflect the correct orientation of the camera.
This is why the lookMat or the quat obtained from it cannot be directly used for this purpose, since the orientation they describe is a reversed one.
Correct rotation can be done in two ways:
Calculate the inverse of the lookAt matrix and multiply the world-space direction vectors by this rotation matrix
(more efficient) Convert the LookAt matrix into a quaternion and conjugate it instead of applying glm::inverse, since the result is a unit quat and for such quats the inverse is equal to the conjugate.
Your LookAt should look like this:
void Camera::LookAt(float x, float y, float z) {
glm::mat4 lookMat = glm::lookAt(position, glm::vec3(x, y, z), glm::vec3(0, 1, 0));
rotation = glm::conjugate( glm::quat_cast(lookMat));
}

How to properly rotate a quaternion along all axis?

I want to code a first person camera with its rotation stored in a quaternion. Unfortunately there is something wrong with the rotation.
The following function is responsible to rotate the camera. The parameters Mouse and Speed pass the mouse movement and rotation speed. Then the function fetches the rotation quaternion, rotates it and stores the result. By the way, I'm using Bullet Physics that is where the types and functions come from.
void Rotate(vec2 Mouse, float Speed)
{
btTransform transform = camera->getWorldTransform();
btQuaternion rotation = transform.getRotation();
Mouse = Mouse * Speed; // apply mouse sensitivity
btQuaternion change(Mouse.y, Mouse.x, 0); // create quaternion from angles
rotation = change * rotation; // rotate camera by that
transform.setRotation(rotation);
camera->setWorldTransform(transform);
}
To illustrate the resulting camera rotation when the mouse moves, I show you a hand drawing. On the left side the wrong rotation the camera actually performs is shown. On the right side the desired correct case is shown. The arrows indicate how the camera is rotate when moving the mouse up (in orange) and down (in blue).
As you can see, as long as the yaw is zero, the rotation is correct. But the more yaw it has, the smaller the circles in which the camera rotates become. In contrast, the circles should always run along the whole sphere like a longitude.
I am not very familiar with quaternions, so here I ask how to correctly rotate them.
I found out how to properly rotate a quaternion on my own. The key was to find vectors for the axis I want to rotate around. Those are used to create quaternions from axis and angle, when angle is the amount to rotate around the actual axis.
The following code shows what I ended up with. It also allows to roll the camera, which might be useful some time.
void Rotate(btVector3 Amount, float Sensitivity)
{
// fetch current rotation
btTransform transform = camera->getWorldTransform();
btQuaternion rotation = transform.getRotation();
// apply mouse sensitivity
Amount *= Sensitivity;
// create orientation vectors
btVector3 up(0, 1, 0);
btVector3 lookat = quatRotate(rotation, btVector3(0, 0, 1));
btVector3 forward = btVector3(lookat.getX(), 0, lookat.getZ()).normalize();
btVector3 side = btCross(up, forward);
// rotate camera with quaternions created from axis and angle
rotation = btQuaternion(up, Amount.getY()) * rotation;
rotation = btQuaternion(side, Amount.getX()) * rotation;
rotation = btQuaternion(forward, Amount.getZ()) * rotation;
// set new rotation
transform.setRotation(rotation);
camera->setWorldTransform(transform);
}
Since I rarely found information about quaternion rotation, I'll spend some time further explaining the code above.
Fetching and setting the rotation is specific to the physics engine and isn't related to this question so I won't elaborate on this. The next part, multiplying the amount by a mouse sensitivity should be really clear. Let's continue with the direction vectors.
The up vector depends on your own implementation. Most conveniently, the positive Y axis points up, therefore we end up with 0, 1, 0.
The lookat vector represents the direction the camera looks at. We simply rotate a unit vector pointing forward by the camera rotation quaternion. Again, the forward pointing vector depends on your conventions. If the Y axis is up, the positive Z axis might point forward, which is 0, 0, 1.
Do not mix that up with the next vector. It's named forward which references to the camera rotation. Therefore we just need to project the lookat vector to the ground. In this case, we simply take the lookat vector and ignore the up pointing component. For neatness we normalize that vector.
The side vector points leftwards from the camera orientation. Therefore it lies perpendicular to both the up and the forward vector and we can use the cross product to compute it.
Given those vectors, we can correctly rotate the camera quaternion around them. Which you start with, Z, Y or Z, depends on the Euler angle sequence which is, again, a convention varying from application to application. Since I want to rotations to be applied in Y X Z order, I do the following.
First, rotate the camera around the up axis by the amount for the Y rotation. This is yaw.
Then rotate around the side axis, which points leftwards, by the X amount. It's pitch.
And lastly, rotate around the forward vector by the Z amount to apply roll.
To apply those rotations, we need to multiply the quaternions create by axis and angle with the current camera rotation. Lastly we apply the resulted quaternion to the body in the physics simulation.
Matrices and pitch/yaw/roll both having their limitations, I do not use them anymore but use instead quaternions. I rotate the view vector and recalculate first the camera vectors, then the view matrix in regard to the rotated view vector.
void Camera::rotateViewVector(glm::quat quat) {
glm::quat rotatedViewQuat;
quat = glm::normalize(quat);
m_viewVector = glm::normalize(m_viewVector);
glm::quat viewQuat(0.0f,
m_viewVector.x,
m_viewVector.y,
m_viewVector.z);
viewQuat = glm::normalize(viewQuat);
rotatedViewQuat = (quat * viewQuat) * glm::conjugate(quat);
rotatedViewQuat = glm::normalize(rotatedViewQuat);
m_viewVector = glm::normalize(glm::vec3(rotatedViewQuat.x, rotatedViewQuat.y, rotatedViewQuat.z));
m_rightVector = glm::normalize(glm::cross(glm::vec3(0.0f, 1.0f, 0.0f), m_viewVector));
m_upVector = glm::normalize(glm::cross(m_viewVector, m_rightVector));
}

Camera Rotation using lookAt

Hey guys I am trying to do a Camera class that uses lookAt from glm library. I have 4 points, the first one is eye, that is the camera position in space, the second one is the look, that is the point where the camera is looking at, the third one is the upp, that set the orientation of camera, and the forth is side, that is a cross product of look - eye and upp -eye.
So in the end I got a base of 3 vectors all of them with origin in the eye point. I got a coordinate system of the camera.
In my camera class, I want to be able to rotate about the coordinate system of the camera, not the coordinate system of the world. So what I am doing is rotate about one of the axis of the coordinate system of the camera.
I construct the class with initial values like this:
void Observer::initialize(glm::vec3 eye, glm::vec3 look, glm::vec3 upp, glm::vec3 side)
{
this->eye = eye; // (0.0, 0.0, 0.0)
this->look = look; // (0.0, 0.0, -1.0)
this->upp = upp; // (0.0, 1.0, 0.0)
this->side = side; // (1.0, 0.0, 0.0)
}
When I want to rotate the coordinate system about the x axis for example I call the function from glm like this:
void Observer::pitch(GLfloat pitch)
{
glm::mat4 rotate(1.0f);
rotate = glm::rotate(rotate, pitch, side - eye);
look = glm::vec3(rotate * glm::vec4(look, 1.0f));
upp = glm::vec3(rotate * glm::vec4(upp, 1.0f));
}
So far, I am understanding that all my points still form a coordinate system for the camera and all vector are perpendicular between each other.
But then I use these points I got with the lookAt function, to position the camera in the world.
glm::mat4 view = glm::lookAt(eye, look, upp);
And multiply this matrix with the modelview matrix from OpenGL
If I start to rotate a lot, the camera after a few rotations "reflect" the rotation, like I was rotating in the other way (I don't know how to describe what is really happening in a better way =s).
I really don't understand what is happening. I should normalize the vectors after I apply the rotation? Am I having a problem with gimbal lock (I don't know a lot about gimbal lock)?
When you do incremental rotations on vectors as you have done, numerical errors mount. When error causes the up and look-at vectors to point nearly in the same direction or opposite each other, the camera transformation calculation is unstable and whacky things can happen. Gimbal lock. Length changes cause different problems.
A solution that has worked for me is to re-orthogonalize the up and look-at vectors after each rotation. To do this, compute their cross-product L, then adjust (really replace) the up vector by crossing L with the lookAt. After all this re-normalize both up and look-at to unit length.
Though the orthogonalization-normalization is a fast operation, you don't really have to do it with every camera motion.
Note that when you correct the vectors like this you are actually doing part of the lookAt matrix calculation, so consider implementing your own to avoid an unnecessary cross product. See e.g. this prior SO article on this topic.

OpenGL simultaneous translate and rotate around local axis

I am working on an application that has similar functionality to MotionBuilder in its viewport interactions. It has three buttons:
Button 1 rotates the viewport around X and Y depending on X/Y mouse drags.
Button 2 translates the viewport around X and Y depending on X/Y mouse drags.
Button 3 "zooms" the viewport by translating along Z.
The code is simple:
glTranslatef(posX,posY,posZ);
glRotatef(rotX, 1, 0, 0);
glRotatef(rotY, 0, 1, 0);
Now, the problem is that if I translate first, the translation will be correct but the rotation then follows the world axis. I've also tried rotating first:
glRotatef(rotX, 1, 0, 0);
glRotatef(rotY, 0, 1, 0);
glTranslatef(posX,posY,posZ);
^ the rotation works, but the translation works according to world axis.
My question is, how can I do both so I achieve the translation from code snippet one and the rotation from code snippet 2.
EDIT
I drew this rather crude image to illustrate what I mean by world and local rotations/translations. I need the camera to rotate and translate around its local axis.
http://i45.tinypic.com/2lnu3rs.jpg
Ok, the image makes things a bit clearer.
If you were just talking about an object, then your first code snippet would be fine, but for the camera it's quite different.
Since there's technically no object as a 'camera' in opengl, what you're doing when building a camera is just moving everything by the inverse of how you're moving the camera. I.e. you don't move the camera up by +1 on the Y axis, you just move the world by -1 on the y axis, which achieves the same visual effect of having a camera.
Imagine you have a camera at position (Cx, Cy, Cz), and it has x/y rotation angles (CRx, CRy). If this were just a regular object, and not the camera, you would transform this by:
glTranslate(Cx, Cy, Cz);
glRotate(CRx, 1, 0, 0);
glRotate(CRy, 0, 1, 0);
But because this is the camera, we need to do the inverse of this operation instead (we just want to move the world by (-Cx, -Cy, and -Cz) to emulate the moving of a 'camera'. To invert the matrix, you just have to do the opposite of each individual transform, and do them in reverse order.
glRotate(-CRy, 0, 1, 0);
glRotate(-CRx, 1, 0, 0);
glTranslate(-Cx, -Cy, -Cz);
I think this will give you the kind of camera you're mentioning in your image.
I suggest that you bite the apple and implement a camera class that stores the current state of the camera (position, view direction, up vector, right vector) and manipulate that state according to your control scheme. Then you can set up the projection matrix using gluLookAt(). Then, the order of operations becomes unimportant. Here is an example:
Let camPos be the current position of the camera, camView its view direction, camUp the up vector and camRight the right vector.
To translate the camera by moveDelta, simply add moveDelta to camPos. Rotation is a bit more difficult, but if you understand quaternions you'll be able to understand it quickly.
First you need to create a quaternion for each of your two rotations. I assume that your horizontal rotation is always about the positive Z axis (which points at the "ceiling" if you will). Let hQuat be the quaternion representing the horizontal rotation. I further assume that you want to rotate the camera about its right axis for your vertical rotation (creating a pitch effect). For this, you must apply the horizontal rotation to the camera's current angle. The result is the rotation axis for your vertical rotation hQuat. The total rotation quaternion is then rQuat = hQuat * vQuat. Then you apply rQuat to the camera's view direction, up, and right vectors.
Quat hRot(rotX, 0, 0, 1); // creates a quaternion that rotates by angle rotX about the positive Z axis
Vec3f vAxis = hRot * camRight; // applies hRot to the camera's right vector
Quat vRot(rotY, vAxis); // creates a quaternion that rotates by angle rotY about the rotated camera's right vector
Quat rQuat = hRot * vRot; // creates the total rotation
camUp = rQuat * camUp;
camRight = rQuat * camRight;
camView = rQuat * camView;
Hope this helps you solve your problem.
glRotate always works around the origin. If you do:
glPushMatrix();
glTranslated(x,y,z);
glRotated(theta,1,0,0);
glTranslated(-x,-y,-z);
drawObject();
glPopMatrix();
Then the 'object' is rotate around (x,y,z) instead of the origin, because you moved (x,y,z) to the origin, did the rotation, and then pushed (x,y,z) back where it started.
However, I don't think that's going to be enough to get the effect you're describing. If you always want transformations to be done with respect to the current frame of reference, then you need to keep track of the transformation matrix yourself. This why people use Quaternion based cameras.