Move in direction of rotation in opengl - opengl

I'm really new to OpenGL but have quite a good grasp of basic trigonometry (forgotten quite a bit since school!) but I'm having trouble with this.
I have a character that moves forwards and backwards on the Z axis. To move left and right, rather than strafing I want them to rotate (when pressing left and right arrow keys respectively) and then when they press forwards/backwards again they move in the direction they're facing.
So what I did was have the left/right functions add/subtract small amounts onto an angle variable that is used to draw the character's rotation. Then the forward/backward functions add/subtract small amounts to the x and z axes variables. They are as follows (for the backwards):
z -= 0.005f * Math.cos(heading);
x -= 0.005f * Math.sin(heading)
The heading variable is the angle which is manipulated by the left and right arrow keys.
I thought this would work because when the player is going straight forward the heading is 0 and so cos(0) = 1 and sin(0) = 0 which means they move nowhere on the X but forwards the full 0.005 amount on the Z. I guess my rudimentary trigonometry knowledge wasn't completely sound because if I turn a bit then move forwards they go in that direction but if I move a bit more then move forwards they go in that same line if it was rotated 90 degrees, and continues then again as if it was 180 degrees then 270 degrees etc.
EDIT: I'll try and explain that better, basically if I press forwards after turning to the left it will go in the direction I want, but if I let go of forward, turn a bit more and then press forward again, the angle has increased as it should have but the direction is like 90 degrees around from the direction it should be going. Sorry I can't really explain this well.
EDIT: Okay, I'm getting some weird problems that I think might be causing the strange "90 degrees" problem, When I get the character to look 90 degrees (through hard-coding heading=90) left/right, cos(heading) should be 0 right? But for some reason it is coming out as -0.44, and If I cos-1(-0.44) I get 116.1, is it something to do with Math.cos() wanting the angle in radians or something? I'm completely lost here.
Is this the right way of going about this problem? I'm completely stuck just trial and erroring around with minus signs...
Any reply is appreciated,
Thanks
InfinitiFizz
(Also, I know I should use a deltaTime for the speed/rotation values of the character not a hard-coded 0.005f but I want to get this problem out of the way first before I sort that).

It should be * not +
based on:
x = r cos (theta)
y = r sin (theta)
you will want:
z -= 0.005f * Math.cos(heading);
x -= 0.005f * Math.sin(heading);
If that's what you're after? I must admit, I didn't fully understand your description of what actually happens.
EDIT: you'll probably want to use a larger "radius" than 0.005, depending on how your coordinate system is working.

Related

Make sprite walk along an angle

I am trying to make a sprite walk towards another sprite,
movement_angle = (atan2((y - target->y),(x - target->x)));
if(isWalkable(game,delta))
{
y -= ((int)(movementspeed)*delta)*sin(((movement_angle)));
x -= ((int)(movementspeed)*delta)*cos(((movement_angle)));
}
I am calculating the angle and adding the values to my x and y cords.
But the movement is not perfect, it does not really follow as I wish, takes weird turns and so on.
It's hard to say exactly what's going wrong from your code, but here are some things you could look at:
To get a vector from a point to target you'll need to subtract target->x by the point's x and target->y by the point's y
At least movement_angle needs to be a floating point number, though you'll likely want to use doubles for all your 3D geometry values
Why bother with trigonometric functions? You want to walk directly from point x, y to target->x, target->y, why not just use a vector, normalize it, and multiply by your movementspeed and delta?
Well you are casting an algebraic result to an int. I think that may be causing your problem.

How do you rotate a sprite based on mouse position?

Basically, I have a sprite that I render using SDL 2.0 that I can rotate a variable amount around a center orgin point of the texture clockwise using SDL_RenderCopyEx(). I want to rotate it based on the mouse position by using the angle x between my physical slope line and my two straight lines based off of my base line. The base line I'm talking about can be represented mathematically as x = orgin_x, where orgin_x is the rotation orgin. The other line is a segment along the baseline that connects the horizontal line end point to the orgin_x point vertically. With the angle to the mouse cursor being the one I want to find to rotate my character.
Please no complicated math symbols. I would rather the formula be posted in C-style format, and please explain the logic behind the math so I can maybe understand what's happening and fix similar future problems if needed.
Some basic trigonometry. You can use atan2(delta_y, delta_x). With this you will get your angle in RAD. To get your angle in degree, because RenderCopyEx use Degree for angle, you need to convert your angle. You got 360 Degree and 2*PI Rad for a full circle. So
angle_deg = (atan2(delta_y, delta_x)*180.0000)/3.1416
Now you got your angle to do a RenderCopyEx
BTW :
delta_y = origin_y - mouse_y
AND
delta_x = origin_x - mouse_x

Inbetweening a rotation

I have an image (let's say it's a simple rectangle) positioned on the left of my screen, which I can move up and down. When moving it upwards, I use some simple trigonometry to rotate it so that the rectangle "points" towards the upper right corner of the screen. When moving downwards, it points towards the lower left corner of the screen.
Given that my application uses the following coordinate system:
I use the following code to achieve the rotation:
// moving upwards
rotation = -atan2(position.y , res.x - position.x));
// moving downwards
rotation = atan2(res.y - position.y , res.x - position.x));
where res is the reference point and position is the position (upper left corner) of our rectangle image. (For information on atan2(): atan2() on cplusplus.com).
This works just fine: it rotates more when farther away from the reference point (res). However, let's say the image is all the way at the bottom of the screen. If we move it upwards, it will very suddenly rotate. I would like to 'inbetween' this rotation, so that it is smoothened out.
What I mean by suddenly rotating is this:
Let's say the rectangle is not moving in frame n: therefore its rotation is 0 degrees. I then press the up arrow, which makes it calculate the angle. In frame n+1, the angle is 30 degrees (for example). This is ofcourse not very smooth.
Is my question clear? How do I go about this?
You can incrementally change the angle on each frame. For a very "smooth" rotation effect, you can use
target_angle = ...
current_angle += (target_angle - current_angle) * smoothing_factor
where smoothing_factor gives the rate at which current_angle should converge to target_angle. For example, a value of 1 would be instantaneous, a value of 0.1 would probably give a smooth effect.
By doing this you may encounter the wraparound issue whereby something like going from 10 degrees to 350 degrees would go the wrong way. In such a case, use
target_angle = ...
current_angle += diff(target_angle, current_angle) * smoothing_factor
where
diff(a, b) {
return atan2(sin(a - b), cos(a - b))
}
This nice angle difference formula is taken from another question.

Implementing a complex rotation-based camera

I am implementing a 3D engine for spatial visualisation, and am writing a camera with the following navigation features:
Rotate the camera (ie, analogous to rotating your head)
Rotate around an arbitrary 3D point (a point in space, which is probably not in the center of the screen; the camera needs to rotate around this keeping the same relative look direction, ie the look direction changes too. This does not look directly at the chosen rotation point)
Pan in the camera's plane (so move up/down or left/right in the plane orthogonal to the camera's look vector)
The camera is not supposed to roll - that is, 'up' remains up. Because of this I represent the camera with a location and two angles, rotations around the X and Y axes (Z would be roll.) The view matrix is then recalculated using the camera location and these two angles. This works great for pan and rotating the eye, but not for rotating around an arbitrary point. Instead I get the following behaviour:
The eye itself apparently moving further up or down than it should
The eye not moving up or down at all when m_dRotationX is 0 or pi. (Gimbal lock? How can I avoid this?)
The eye's rotation being inverted (changing the rotation makes it look further up when it should look further down, down when it should look further up) when m_dRotationX is between pi and 2pi.
(a) What is causing this 'drift' in rotation?
This may be gimbal lock. If so, the standard answer to this is 'use quaternions to represent rotation', said many times here on SO (1, 2, 3 for example), but unfortunately without concrete details (example. This is the best answer I've found so far; it's rare.) I've struggled to implemented a camera using quaternions combining the above two types of rotations. I am, in fact, building a quaternion using the two rotations, but a commenter below said there was no reason - it's fine to immediately build the matrix.
This occurs when changing the X and Y rotations (which represent the camera look direction) when rotating around a point, but does not occur simply when directly changing the rotations, i.e. rotating the camera around itself. To me, this doesn't make sense. It's the same values.
(b) Would a different approach (quaternions, for example) be better for this camera? If so, how do I implement all three camera navigation features above?
If a different approach would be better, then please consider providing a concrete implemented example of that approach. (I am using DirectX9 and C++, and the D3DX* library the SDK provides.) In this second case, I will add and award a bounty in a couple of days when I can add one to the question. This might sound like I'm jumping the gun, but I'm low on time and need to implement or solve this quickly (this is a commercial project with a tight deadline.) A detailed answer will also improve the SO archives, because most camera answers I've read so far are light on code.
Thanks for your help :)
Some clarifications
Thanks for the comments and answer so far! I'll try to clarify a few things about the problem:
The view matrix is recalculated from the camera position and the two angles whenever one of those things changes. The matrix itself is never accumulated (i.e. updated) - it is recalculated afresh. However, the camera position and the two angle variables are accumulated (whenever the mouse moves, for example, one or both of the angles will have a small amount added or subtracted, based on the number of pixels the mouse moved up-down and/or left-right onscreen.)
Commenter JCooper states I'm suffering from gimbal lock, and I need to:
add another rotation onto your transform that rotates the eyePos to be
completely in the y-z plane before you apply the transformation, and
then another rotation that moves it back afterward. Rotate around the
y axis by the following angle immediately before and after applying
the yaw-pitch-roll matrix (one of the angles will need to be negated;
trying it out is the fastest way to decide which).
double fixAngle = atan2(oEyeTranslated.z,oEyeTranslated.x);
Unfortunately, when implementing this as described, my eye shoots off above the scene at a very fast rate due to one of the rotations. I'm sure my code is simply a bad implementation of this description, but I still need something more concrete. In general, I find unspecific text descriptions of algorithms are less useful than commented, explained implementations. I am adding a bounty for a concrete, working example that integrates with the code below (i.e. with the other navigation methods, too.) This is because I would like to understand the solution, as well as have something that works, and because I need to implement something that works quickly since I am on a tight deadline.
Please, if you answer with a text description of the algorithm, make sure it is detailed enough to implement ('Rotate around Y, then transform, then rotate back' may make sense to you but lacks the details to know what you mean. Good answers are clear, signposted, will allow others to understand even with a different basis, are 'solid weatherproof information boards.')
In turn, I have tried to be clear describing the problem, and if I can make it clearer please let me know.
My current code
To implement the above three navigation features, in a mouse move event moving based on the pixels the cursor has moved:
// Adjust this to change rotation speed when dragging (units are radians per pixel mouse moves)
// This is both rotating the eye, and rotating around a point
static const double dRotatePixelScale = 0.001;
// Adjust this to change pan speed (units are meters per pixel mouse moves)
static const double dPanPixelScale = 0.15;
switch (m_eCurrentNavigation) {
case ENavigation::eRotatePoint: {
// Rotating around m_oRotateAroundPos
const double dX = (double)(m_oLastMousePos.x - roMousePos.x) * dRotatePixelScale * D3DX_PI;
const double dY = (double)(m_oLastMousePos.y - roMousePos.y) * dRotatePixelScale * D3DX_PI;
// To rotate around the point, translate so the point is at (0,0,0) (this makes the point
// the origin so the eye rotates around the origin), rotate, translate back
// However, the camera is represented as an eye plus two (X and Y) rotation angles
// This needs to keep the same relative rotation.
// Rotate the eye around the point
const D3DXVECTOR3 oEyeTranslated = m_oEyePos - m_oRotateAroundPos;
D3DXMATRIX oRotationMatrix;
D3DXMatrixRotationYawPitchRoll(&oRotationMatrix, dX, dY, 0.0);
D3DXVECTOR4 oEyeRotated;
D3DXVec3Transform(&oEyeRotated, &oEyeTranslated, &oRotationMatrix);
m_oEyePos = D3DXVECTOR3(oEyeRotated.x, oEyeRotated.y, oEyeRotated.z) + m_oRotateAroundPos;
// Increment rotation to keep the same relative look angles
RotateXAxis(dX);
RotateYAxis(dY);
break;
}
case ENavigation::ePanPlane: {
const double dX = (double)(m_oLastMousePos.x - roMousePos.x) * dPanPixelScale;
const double dY = (double)(m_oLastMousePos.y - roMousePos.y) * dPanPixelScale;
m_oEyePos += GetXAxis() * dX; // GetX/YAxis reads from the view matrix, so increments correctly
m_oEyePos += GetYAxis() * -dY; // Inverted compared to screen coords
break;
}
case ENavigation::eRotateEye: {
// Rotate in radians around local (camera not scene space) X and Y axes
const double dX = (double)(m_oLastMousePos.x - roMousePos.x) * dRotatePixelScale * D3DX_PI;
const double dY = (double)(m_oLastMousePos.y - roMousePos.y) * dRotatePixelScale * D3DX_PI;
RotateXAxis(dX);
RotateYAxis(dY);
break;
}
The RotateXAxis and RotateYAxis methods are very simple:
void Camera::RotateXAxis(const double dRadians) {
m_dRotationX += dRadians;
m_dRotationX = fmod(m_dRotationX, 2 * D3DX_PI); // Keep in valid circular range
}
void Camera::RotateYAxis(const double dRadians) {
m_dRotationY += dRadians;
// Limit it so you don't rotate around when looking up and down
m_dRotationY = std::min(m_dRotationY, D3DX_PI * 0.49); // Almost fully up
m_dRotationY = std::max(m_dRotationY, D3DX_PI * -0.49); // Almost fully down
}
And to generate the view matrix from this:
void Camera::UpdateView() const {
const D3DXVECTOR3 oEyePos(GetEyePos());
const D3DXVECTOR3 oUpVector(0.0f, 1.0f, 0.0f); // Keep up "up", always.
// Generate a rotation matrix via a quaternion
D3DXQUATERNION oRotationQuat;
D3DXQuaternionRotationYawPitchRoll(&oRotationQuat, m_dRotationX, m_dRotationY, 0.0);
D3DXMATRIX oRotationMatrix;
D3DXMatrixRotationQuaternion(&oRotationMatrix, &oRotationQuat);
// Generate view matrix by looking at a point 1 unit ahead of the eye (transformed by the above
// rotation)
D3DXVECTOR3 oForward(0.0, 0.0, 1.0);
D3DXVECTOR4 oForward4;
D3DXVec3Transform(&oForward4, &oForward, &oRotationMatrix);
D3DXVECTOR3 oTarget = oEyePos + D3DXVECTOR3(oForward4.x, oForward4.y, oForward4.z); // eye pos + look vector = look target position
D3DXMatrixLookAtLH(&m_oViewMatrix, &oEyePos, &oTarget, &oUpVector);
}
It seems to me that "Roll" shouldn't be possible given the way you form your view matrix. Regardless of all the other code (some of which does look a little funny), the call D3DXMatrixLookAtLH(&m_oViewMatrix, &oEyePos, &oTarget, &oUpVector); should create a matrix without roll when given [0,1,0] as an 'Up' vector unless oTarget-oEyePos happens to be parallel to the up vector. This doesn't seem to be the case since you're restricting m_dRotationY to be within (-.49pi,+.49pi).
Perhaps you can clarify how you know that 'roll' is happening. Do you have a ground plane and the horizon line of that ground plane is departing from horizontal?
As an aside, in UpdateView, the D3DXQuaternionRotationYawPitchRoll seems completely unnecessary since you immediately turn around and change it into a matrix. Just use D3DXMatrixRotationYawPitchRoll as you did in the mouse event. Quaternions are used in cameras because they're a convenient way to accumulate rotations happening in eye coordinates. Since you're only using two axes of rotation in a strict order, your way of accumulating angles should be fine. The vector transformation of (0,0,1) isn't really necessary either. The oRotationMatrix should already have those values in the (_31,_32,_33) entries.
Update
Given that it's not roll, here's the problem: you create a rotation matrix to move the eye in world coordinates, but you want the pitch to happen in camera coordinates. Since roll isn't allowed and yaw is performed last, yaw is always the same in both the world and camera frames of reference. Consider the images below:
Your code works fine for local pitch and yaw because those are accomplished in camera coordinates.
But when you rotate around a reference point, you are creating a rotation matrix that is in world coordinates and using that to rotate the camera center. This works okay if the camera's coordinate system happens to line up with the world's. However, if you don't check to see if you're up against the pitch limit before you rotate the camera position, you will get crazy behavior when you hit that limit. The camera will suddenly start to skate around the world--still 'rotating' around the reference point, but no longer changing orientation.
If the camera's axes don't line up with the world's, strange things will happen. In the extreme case, the camera won't move at all because you're trying to make it roll.
The above is what would normally happen, but since you handle the camera orientation separately, the camera doesn't actually roll.
Instead, it stays upright, but you get strange translation going on.
One way to handle this would be to (1)always put the camera into a canonical position and orientation relative to the reference point, (2)make your rotation, and then (3)put it back when you're done (e.g., similar to the way that you translate the reference point to the origin, apply the Yaw-Pitch rotation, and then translate back). Thinking more about it, however, this probably isn't the best way to go.
Update 2
I think that Generic Human's answer is probably the best. The question remains as to how much pitch should be applied if the rotation is off-axis, but for now, we'll ignore that. Maybe it'll give you acceptable results.
The essence of the answer is this: Before mouse movement, your camera is at c1 = m_oEyePos and being oriented by M1 = D3DXMatrixRotationYawPitchRoll(&M_1,m_dRotationX,m_dRotationY,0). Consider the reference point a = m_oRotateAroundPos. From the point of view of the camera, this point is a'=M1(a-c1).
You want to change the orientation of the camera to M2 = D3DXMatrixRotationYawPitchRoll(&M_2,m_dRotationX+dX,m_dRotationY+dY,0). [Important: Since you won't allow m_dRotationY to fall outside of a specific range, you should make sure that dY doesn't violate that constraint.] As the camera changes orientation, you also want its position to rotate around a to a new point c2. This means that a won't change from the perspective of the camera. I.e., M1(a-c1)==M2(a-c2).
So we solve for c2 (remember that the transpose of a rotation matrix is the same as the inverse):
M2TM1(a-c1)==(a-c2) =>
-M2TM1(a-c1)+a==c2
Now if we look at this as a transformation being applied to c1, then we can see that it is first negated, then translated by a, then rotated by M1, then rotated by M2T, negated again, and then translated by a again. These are transformations that graphics libraries are good at and they can all be squished into a single transformation matrix.
#Generic Human deserves credit for the answer, but here's code for it. Of course, you need to implement the function to validate a change in pitch before it's applied, but that's simple. This code probably has a couple typos since I haven't tried to compile:
case ENavigation::eRotatePoint: {
const double dX = (double)(m_oLastMousePos.x - roMousePos.x) * dRotatePixelScale * D3DX_PI;
double dY = (double)(m_oLastMousePos.y - roMousePos.y) * dRotatePixelScale * D3DX_PI;
dY = validatePitch(dY); // dY needs to be kept within bounds so that m_dRotationY is within bounds
D3DXMATRIX oRotationMatrix1; // The camera orientation before mouse-change
D3DXMatrixRotationYawPitchRoll(&oRotationMatrix1, m_dRotationX, m_dRotationY, 0.0);
D3DXMATRIX oRotationMatrix2; // The camera orientation after mouse-change
D3DXMatrixRotationYawPitchRoll(&oRotationMatrix2, m_dRotationX + dX, m_dRotationY + dY, 0.0);
D3DXMATRIX oRotationMatrix2Inv; // The inverse of the orientation
D3DXMatrixTranspose(&oRotationMatrix2Inv,&oRotationMatrix2); // Transpose is the same in this case
D3DXMATRIX oScaleMatrix; // Negative scaling matrix for negating the translation
D3DXMatrixScaling(&oScaleMatrix,-1,-1,-1);
D3DXMATRIX oTranslationMatrix; // Translation by the reference point
D3DXMatrixTranslation(&oTranslationMatrix,
m_oRotateAroundPos.x,m_oRotateAroundPos.y,m_oRotateAroundPos.z);
D3DXMATRIX oTransformMatrix; // The full transform for the eyePos.
// We assume the matrix multiply protects against variable aliasing
D3DXMatrixMultiply(&oTransformMatrix,&oScaleMatrix,&oTranslationMatrix);
D3DXMatrixMultiply(&oTransformMatrix,&oTransformMatrix,&oRotationMatrix1);
D3DXMatrixMultiply(&oTransformMatrix,&oTransformMatrix,&oRotationMatrix2Inv);
D3DXMatrixMultiply(&oTransformMatrix,&oTransformMatrix,&oScaleMatrix);
D3DXMatrixMultiply(&oTransformMatrix,&oTransformMatrix,&oTranslationMatrix);
D3DXVECTOR4 oEyeFinal;
D3DXVec3Transform(&oEyeFinal, &m_oEyePos, &oTransformMatrix);
m_oEyePos = D3DXVECTOR3(oEyeFinal.x, oEyeFinal.y, oEyeFinal.z)
// Increment rotation to keep the same relative look angles
RotateXAxis(dX);
RotateYAxis(dY);
break;
}
I think there is a much simpler solution that lets you sidestep all rotation issues.
Notation: A is the point we want to rotate around, C is the original camera location, M is the original camera rotation matrix that maps global coordinates to the camera's local viewport.
Make a note of the local coordinates of A, which are equal to A' = M × (A - C).
Rotate the camera like you would in normal "eye rotation" mode. Update the view matrix M so that it is modified to M2 and C remains unchanged.
Now we would like to find C2 such that A' = M2 × (A - C2).
This is easily done by the equation C2 = A - M2-1 × A'.
Voilà, the camera has been rotated and because the local coordinates of A are unchanged, A remains at the same location and the same scale and distance.
As an added bonus, the rotation behavior is now consistent between "eye rotation" and "point rotation" mode.
You rotate around the point by repeatedly applying small rotation matrices, this probably cause the drift (small precision errors add up) and I bet you will not really do a perfect circle after some time. Since the angles for the view use simple 1-dimension double, they have much less drift.
A possible fix would be to store a dedicated yaw/pitch and relative position from the point when you enter that view mode, and using those to do the math. This requires a bit more bookkeeping, since you need to update those when moving the camera. Note that it will also make the camera move if the point move, which I think is an improvement.
If I understand correctly, you are satisfied with the rotation component in the final matrix (save for inverted rotation controls in the problem #3), but not with the translation part, is that so?
The problem seems to come from the fact that you treating them differently: you are recalculating the rotation part from scratch every time, but accumulate the translation part (m_oEyePos). Other comments mention precision problems, but it's actually more significant than just FP precision: accumulating rotations from small yaw/pitch values is simply not the same---mathematically---as making one big rotation from the accumulated yaw/pitch. Hence the rotation/translation discrepancy. To fix this, try recalculating eye position from scratch simultaneously with the rotation part, similarly to how you find "oTarget = oEyePos + ...":
oEyePos = m_oRotateAroundPos - dist * D3DXVECTOR3(oForward4.x, oForward4.y, oForward4.z)
dist can be fixed or calculated from the old eye position. That will keep the rotation point in the screen center; in the more general case (which you are interested in), -dist * oForward here should be replaced by the old/initial m_oEyePos - m_oRotateAroundPos multiplied by the old/initial camera rotation to bring it to the camera space (finding a constant offset vector in camera's coordinate system), then multiplied by the inverted new camera rotation to get the new direction in the world.
This will, of course, be subject to gimbal lock when the pitch is straight up or down. You'll need to define precisely what behavior you expect in these cases to solve this part. On the other hand, locking at m_dRotationX=0 or =pi is rather strange (this is yaw, not pitch, right?) and might be related to the above.

Confused about degrees and OpenGL/GLUT camera movement/rotation

NOTICE: I have edited the question below which is more relevant to my real issue than the text right below, you can skip this if you but I'll leave it here for historic reasons.
To see if I get this right, a float in C is the same as a value in radians right? I mean, 360º = 6.28318531 radians and I just noticed on my OpenGL app that a full rotation goes from 0.0 to 6.28, which seems to add up correctly. I just want to make sure I got that right.
I'm using a float (let's call it anglePitch) from 0.0 to 360.0 (it's easier to read in degrees and avoids casting int to float all the time) and all the code I see on the web uses some kind of DEG2RAD() macro which is defined as DEG2RAD 3.141593f / 180. In the end it would be something like this:
anglePitch += direction * 1; // direction will be 1 or -1
refY = tan(anglePitch * DEG2RAD);
This really does a full rotation but that full rotation will be when anglePitch = 180 and anglePitch * DEG2RAD = 3.14, but a full rotation should be 360|6.28. If I change the macro to any of the following:
#define DEG2RAD 3.141593f / 360
#define DEG2RAD 3.141593f / 2 / 180
It works as expected, a full rotation will happen when anglePitch = 360.
What am I missing here and what should I use to properly convert angles to radians/floats?
IMPORTANT EDIT (REAL QUESTION):
I understand now the code I see everywhere on the web about DEG2RAD, I'm just too dumb at math (yeah, I know, it's important when working with this kind of stuff). So I'm going to rephrase my question:
I have now added this to my code:
#define PI 3.141592654f
#define DEG2RAD(d) (d * PI / 180)
Now, when working the pitch/yawn angles in degrees, which are floats, once again, to avoid casting all the time, I just use the DEG2RAD macro and the degree value will be correctly converted to radians. These values will be passed to sin/cos/tan functions and will return the proper values to be used in GLUT camera.
Now the real question, where I was really confused before but couldn't explain myself better:
angleYaw += direction * ROTATE_SPEED;
refX = sin(DEG2RAD(angleYaw));
refZ = -cos(DEG2RAD(angleYaw));
This code will be executed when I press the LEFT/RIGHT keys and the camera will rotate in the Y axis accordingly. A full rotation goes from 0º to 360º.
anglePitch += direction * ROTATE_SPEED;
refY = tan(DEG2RAD(anglePitch));
This is similar code and will be executed when I press the UP/DOWN keys and the camera will rotate in the X axis. But in this situation, a full rotation goes from 0º to 180º degrees and that's what's really confusing me. I'm sure it has something to do with the tangent function but I can't get my head around it.
Is there way I could use sin/cos (as I do in the yawn code) to achieve the same rotation? What is the right way, the most simple code I can add/fix and what makes more sense to create a full pitch rotation from 0º to 360º?
360° = 2 * Pi, Pi = 3.141593…
Radians are defined by the arc length of an angle along a circle of radius 1. The circumfence of a circle is 2*r*Pi, so one full turn on a unit circle has an arc length of 2*Pi = 6.28…
The measure of angles in degrees stem from the fact, that by aligning 6 equilateral triangles you span a full turn. So we have 6 triangles, each making up a 6th of the turn, so the old babylonians divided a circle into pieces of 1/(6*6) = 1/36, and to further refine it this was subdivded by 10. That's why we ended up with 360° in a full circle. This number is arbitrarily choosen, though.
So if there are 2*Pi/360° this makes Pi/180° = 3.141593…/180° which is the conversion factor from degrees to radians. The reciprocal, 180°/Pi = 180/3.141593…
Why on earth the old OpenGL function glRotate and GLU's gluPerspective used degrees instead of radians I cannot fathom. From a mathematical point of view only radians make sense. Which I think is most beautifully demonstrated by Euler's equation
e^(i*Pi) - 1 = 0
There you have it, all the important numbers of mathematics in one single equation. What's this got to do with angles? Well:
e^(i*alpha) = cos(alpha) + i * sin(alpha), alpha is in radians!
EDIT, with respect to modified question:
Your angles being floats is all fine. Why would you even think degress being integers I cannot understand. Normally you don't have to define PI yourself, it comes predefined in math.h, usually called M_PI, M_2PI, and M_PI2 for Pi, 2*Pi and Pi/2. You also should change your macro, the way it's written now can create strange effects.
#define DEG2RAD(d) ( (d) * M_PI/180. )
GLUT has no camera at all. GLUT is a rather dumb OpenGL framework I recommend not using. You probably refer to gluLookAt.
Those obstacles out of the way let's see what you're doing there. Remember that trigonometric functions operate on the unit circle. Let the angle 0 point towards the right and angles increment counterclockwise. Then sin(a) is defined as the amount of rightwards and cos(a) and the amount of forwards to reach the point at angle a on the unit circle. This is what the refX and refZ are getting assigned to.
refY however makes no sense written that way. tan = sin/cos so as we approach n*pi/2 (i.e. 90°) it diverges to +/- infinity. At least it explains your pi/180° cyclic range, because that's the period of tan.
I was first thinking that tan may have been used to normalize the direction vector, but didn't make sense either. The factor would have been 1./sqrt(sin²(Pitch) + 1)
I double checked: using tan there does the right thing.
EDIT2: I don't see where your problem is: The pitch angle is -90° to +90°, which makes perfect sense. Go get yourself a globe (of the earth): The east-west coordinates (longitude) go from -180° to +180°, the south-north coordinate (latitude) goes -90° to +90°. Think about it: Any larger coordinate range would create ambiguities.
The only good suggestion I offer you is: Grab some math text book and bend your mind around spherical coordinates! Sorry to tell you that way. Whatever you have works perfectly fine, you just need to understand sperical geometry.
You're using the terms Yaw and Pitch. Those are normally used in Euler angles. Now unfortunately Euler angles, which compelling at first, cause serious trouble later on (like gimbal lock). You should not use them at all. It may also be a good idea if you used some pencil/sticks/whatever to decompose the rotations you're intending with your hands to understand their mechanics.
And by the way: There are also non-integer degrees. Just hop over to http://maps.google.com to see them in action (just select some place and let http://maps.google.com give you the link to it).
'float' is a type, like int or double. radians and degrees are units of measure, both of which can be represented with any precision you want. i.e., there's no reason you can't have 22.5 degrees, and keep that value in a float.
a full rotation in radians is 2*pi, about 6.283, whereas a full rotation in degrees is 360. You can convert between them by dividing out the starting unit's full circle, then multiplying by the desired unit's full circle.
for example, to get from 90 degrees to radians, first divide out the degrees. 90 over 360 is 0.25 (note this value is in 'revolutions'). Now multiply that 0.25 by 6.283 to arrive at 1.571 radians.
follow up
the reason you're seeing your pitch cycle twice as fast as it should is precisely because you're using tan(pitch) to compute the Y component. What you should have is that the Y component depends on sin(pitch). i.e., try changing
refY = tan(DEG2RAD(anglePitch));
to
refY = sin(DEG2RAD(anglePitch));
a technical detail: the numbers that go into the look matrix should all be in the range of -1 to +1, and if you were to inspect the values you're feeding to refY, and run your pitch outside of -45 to +45 degrees, you'd see the problem; tan() runs off to infinity at +/-90 degrees.
also, note that casting a value from int to float in no sense converts between degrees and radians. casting just gives you the nearest equivalent value in the new storage type. for example, if you cast the integer 22 to floating point, you get 22.0f, whereas if you cast 33.3333f to type int, you'd be left with 33. when working with angles, you really should just stick with floating point, unless you're constrained by working with an embedded processor or something. this is especially important with radians, where whole number increments represent leaps of (about) 57.3 degrees.
Assuming that your ref components are intended to be used as your look-at vector, I think what you need is
refY = sin(DEG2RAD(anglePitch));
XZfactor = cos(DEG2RAD(anglePitch));
refX = XZfactor*sin(DEG2RAD(angleYaw));
refZ = -XZfactor*cos(DEG2RAD(angleYaw));