Skeletal animation COLLADA matrix multiplication - opengl

I'm trying to implement skeletal animation in a small program I'm writing. The idea is to calculate the transformation matrix on the CPU every frame by interpolating keyframe data, then feeding this data to my vertex shader which multiplies my vertices by this matrix like this:
vec4 v = animationMatrices[int(boneIndices.x)] * gl_Vertex * boneWeights.x;
Where boneWeights and boneIndices are attributes and animationMatrices is a uniform array of transformation matrices updated every frame before drawing. (The idea is to have multiple bones affecting one vertex eventually, but right now I'm testing with one bone per vertex so just taking the weight.x and indices.x is enough).
Now the problem is calculating the transformation matrix for each bone. My transformation matrix for the single joint is good, the problem is that it always takes (0,0,0) as pivot instead of the pivot. I took the joint matrices from the COLLADA which correctly shows my skeleton when I draw them like this:
public void Draw()
{
GL.PushMatrix();
drawBone(Root);
GL.PopMatrix();
}
private void drawBone(Bone b)
{
GL.PointSize(50);
GL.MultMatrix(ref b.restMatrix);
GL.Begin(BeginMode.Points);
GL.Color3((byte)0, (byte)50, (byte)0);
if (b.Name == "Blades")
{
GL.Vertex3(0, 0, 0);
}
GL.End();
foreach (Bone bc in b.Children)
{
GL.PushMatrix();
drawBone(bc);
GL.PopMatrix();
}
}
So now to calculate the actual matrix I've tried:
Matrix4 jointMatrix = b.restMatrixInv * boneTransform * b.restMatrix;
or according to the collada documentation (this doesn't really make sense to me):
Matrix4 jointMatrix = b.restMatrix * b.restMatrixInv * boneTransform;
And I know I also have to put the parent matrix in here somewhere, I'm guessing something like this:
Matrix4 jointMatrix = b.restMatrixInv * boneTransform * b.restMatrix * b.Parent.jointMatrix;
But at the moment I'm mostly just confused and any push in the right direction would help. I really need to get this order right...

Related

Skinning Animation - Weights Destroy Mesh

I am in the process of writing an animation system with my own Collada parser and am running into an issue that I can't wrap my head around.
I have collected my mesh/skin information (vertices, normals, jointIds, weights, etc), my skeleton information (joints, their local transforms, inverse bind position, hierarchy structure), and my animation (keyframe transform position for each joint, timestamp).
My issue is that with everything calculated and then implemented in the shader (the summation of weights multiplied by the joint transform and vertex position) - I get the following:
When I remove the weight multiplication, the mesh remains fully intact - however the skin doesn't actually follow the animation. I am at a lost as I feel as though the math is correct, but very obviously I am going wrong somewhere. Would someone be able to shine light on the aspect I have misinterpreted?
Here is my current understanding and implementation:
After collecting all of the joint's localTransforms and hierarchy, I calculate their inverse bind transfromation matrix. To do this I multiple each joints localTransform with their parentLocalTransform to get a bindTransform. Inverting that bindTransform results in their inverseBindTransform. Below is my code for that:
// Recursively collect each Joints InverseBindTransform -
// root joint's local position is an identity matrix.
// Function is only called once after data collection.
void Joint::CalcInverseBindTransform(glm::mat4 parentLocalPosition)
{
glm::mat4 bindTransform = parentLocalPosition * m_LocalTransform;
m_InverseBindPoseMatrix = glm::inverse(bindTransform);
for (Joint child : Children) {
child.CalcInverseBindTransform(bindTransform);
}
}
Within my animator during an animation, for each joint I take the two JointTransforms for the two frame's my currentTime is in between and I calculate the interpolated JointTransform. (JointTransform simply has a vec3 for position and quaternion for rotation). I do this for every joint and then apply those interpolated values to each Joint by again recursively muliplying the new frameLocalTransform by their parentLocalTransform. I take that bindTransform and multiply it by the invBindTransform and then transpose the matrix. Below is the code for that:
std::unordered_map<int, glm::mat4> Animator::InterpolatePoses(float time) {
std::unordered_map<int, glm::mat4> poses;
if (IsPlaying()) {
for (std::pair<int, JointTransform> keyframe : m_PreviousFrame.GetJointKeyFrames()) {
JointTransform previousFrame = m_PreviousFrame.GetJointKeyFrames()[keyframe.first];
JointTransform nextFrame = m_NextFrame.GetJointKeyFrames()[keyframe.first];
JointTransform interpolated = JointTransform::Interpolate(previousFrame, nextFrame, time);
poses[keyframe.first] = interpolated.getLocalTransform();
}
}
return poses;
}
void Animator::ApplyPosesToJoints(std::unordered_map<int, glm::mat4> newPose, Joint* j, glm::mat4 parentTransform)
{
if (IsPlaying()) {
glm::mat4 currentPose = newPose[j->GetJointId()];
glm::mat4 modelSpaceJoint = parentTransform * currentPose;
for (Joint child : j->GetChildren()) {
ApplyPosesToJoints(newPose, &child, modelSpaceJoint);
}
modelSpaceJoint = glm::transpose(j->GetInvBindPosition() * modelSpaceJoint);
j->SetAnimationTransform(modelSpaceJoint);
}
}
I then collect all the newly AnimatedTransforms for each joint and send them to the shader:
void AnimationModel::Render(bool& pass)
{
[...]
std::vector<glm::mat4> transforms = GetJointTransforms();
for (int i = 0; i < transforms.size(); ++i) {
m_Shader->SetMat4f(transforms[i], ("JointTransforms[" + std::to_string(i) + "]").c_str());
}
[...]
}
void AnimationModel::AddJointsToArray(Joint current, std::vector<glm::mat4>& matrix)
{
glm::mat4 finalMatrix = current.GetAnimatedTransform();
matrix.push_back(finalMatrix);
for (Joint child : current.GetChildren()) {
AddJointsToArray(child, matrix);
}
}
In the shader, I simply follow the summation formula that can be found all over the web when researchiing this topic:
for (int i = 0; i < total_weight_amnt; ++i) {
mat4 jointTransform = JointTransforms[jointIds[i]];
vec4 newVertexPos = jointTransform * vec4(pos, 1.0);
total_pos += newVertexPos * weights[i];
[...]
---------- Reply to Normalizing Weights ------------
There were a few weights summing above 1, but after solving the error in my code the model looked like this:
For calculating the weights - I loop through all preadded weights in the vector, and if I find a weight that is less than the weight I'm looking to add - I replace that weight in that position. Otherwise, I append the weight onto the end of the vector. If there are less weights in my vector than my specified max_weights (which is 4) - I fill in the remaining weights/jointIds with 0.
I understand when something is going wrong in skinning animations, there can be alot of different areas the problem is occuring. As such, for future googlers experiencing the same issue I am - take this as more of a list of suggestions of what you could be doing wrong rather than absolutely doing wrong.
For my problem - I had the right idea but wrong approach in a lot of minor areas. Which brought me fairly close but, as they say, no cigar.
I had no need to calculate the Inverse Bind Pose myself, Collada's Inverse Bind Pose (sometimes/often declared as an "offsetMatrix") is more than perfect. This wasn't a problem more as I was just doing unnecessary calculations.
In a Collada file, they often provide you more "joints" or "nodes" in the hierarchy than what is needed for the animation. Prior to the start of your actual animated "joints", there is the scene and an initial armature "node" type. The scene is typically an identity matrix that was manipulated based on your "up axis" upon reading in the Collada file. The Node type will determine the overall size of each joint in the skeleton - so if it wasn't resized, its probably the identity matrix. Make sure your hierarchy still contains ALL nodes/joints listed in the hierarchy. I very much was not doing so - which greatly distorted my globalPosition (BindPose).
If you are representing your Joint's transforms rotation through quaternions (which is highly recommended), make sure the resulted quaternion is normalized after interpolating between two rotated positions.
On the same note - when combining the Rotation and Transform into your final matrix - make sure your order of multiplication and the final output is correct.
Finally - your last skinning matrix is comprised of your joints InvBindMatrix * GlobalPosition * GlobalInverseRootTransform (<- this is the inverse of the local transfrom from your "scene" node mentioned in (1), remember?).
Based on your prior matrix multiplications up to this point, you may or may not need to transpose this final matrix.
And with that - I was able to successfully animate my model!
One final note - my mesh and animation files are added in separately. If your animations are in separate files from your mesh, make sure you collect the skinning/joint information from the files with an animation rather than the file with the mesh. I list my steps for loading in a model and then giving it multiple animations through different files:
Load in the Mesh (This contains Vertices,Normals,TexCoords,JointIds,Weights)
Load in the animation file (This gives Skeleton, InverseBindPositions, and other needed info to bind skeleton to mesh) - Once skeleton and binding info is collected, gather first animation info from that file as well.
For another animation, the above Skeleton should work fine for any other animation on the same mesh/model - just read in the animation information and store in your chosen data structure. Repeat step 3 til happy.

OpenGL Camera Strafing doesnt work

I'm currently working on a OpenGL FrameWork/Engine and as far as the OpenGL part goes, I'm quite satisfied with my results.
On the other hand I have a serious problem getting a Camera to work.
Moving along the Z-Axis works well, but as soon as I start to strafe (moving along the X-Axis), the whole Scene get screwed.
You can see the result of strafing in the image below.
The left part shows the actual scene, the right part shows the scene resulting from a strafe movement.
My code is the following.
In Constructor:
//Info is a Struct with Default values
m_projectionMatrix = glm::perspective(
info.fieldOfView, width / height, //info.fov = 90
info.nearPlane, info.farPlane // info.near = 0.1f, info.far = 1000
);
//m_pos = glm::vec3(0.0f,0.0f,0.0f), info.target = glm::vec3(0.0f, 0.0f, -1.0f)
m_viewMatrix = glm::lookAt(m_pos, m_pos + info.target, Camera::UP);
//combine projection and view
m_vpMatrix = m_projectionMatrix * m_viewMatrix;
In the "Update"-Method I'm currently doing the following:
glm::mat4x4 Camera::GetVPMatrix()
{
m_vpMatrix = glm::translate(m_vpMatrix, m_deltaPos);
return m_vpMatrix;
}
As far as i know:
The projection matrix achieves the actual perspective view. The view matrix, initially, translates and rotates the whole scene, that it is centered?
So why translating the VP-Matrix by any Z-Value works just fine, but by an X-Value doesn't?
I would like to achive a camera behaviour like this:
Initial Cam Pos is (0,0,0) and "Center" is e.g. (0,0,-1).
Then after Translation by X = 5: Cam Pos is (5,0,0) and Center is (5,0,-1).
Edit: Additional Question.
Why is the Z-Coordinate affekted by VP-Transformation?
Thanks for any help!
Best regards, Christoph.
Okay, I finally got the solution... As you can see, I am using GLM for my matrix math. GLM stores its matrices values in column major order. Open GL wants column major ordered matrices, too. C/C++ native 2d Array layout is row major, so most of the available OpenGL/C++ tutorials state, that one should use
glUniformMatrix4fv(location, 1, GL_TRUE, &mat[0][0]);
With GL_TRUE meaning, that the matrix should be converted (transposed?) from row major to column major order. Because of my matrices already beeing in column major format, that makes absolutely no sense...
Changing the above to
glUniformMatrix4fv(location, 1, GL_FALSE, &mat[0][0]);
fixed my problem...
Matrix math is not my strong point so I can't explain why your current approach doesn't work, though I have my suspicions (translating the projection matrix doesn't seem right). The following should allow you to move the camera though:
// update your camera position
m_pos = new_pos;
// update the view matrix
m_viewMatrix = glm::lookAt(m_pos, m_pos + info.target, Camera::UP);
// update the view-projection matrix, projection matrix never changes
m_vpMatrix = m_projectionMatrix * m_viewMatrix;

Assimp calculating bones

Ive been trying to load my skinned collada mesh using assimp for the last few days but am having an extremely hard time. Currently I am just trying to render it in its normal "pose" without any transformations or animations but the mesh comes very deformed. I think the problem lies in my bones, not my weights so I am only posting the bone section.
Here is my bone loading code:
// this is done for every bone in a for loop:
Bone new_bone;
new_bone.name = std::string(bone->mName.data);
aiMatrix4x4 b_matrix = bone->mOffsetMatrix;
aiMatrix4x4 g_inv = scene->mRootNode->mTransformation;
g_inv.Inverse(); // according to a tutorial, you have to multiply by inverse root
b_matrix = b_matrix * g_inv;
memcpy(new_bone.matrix, &b_matrix, sizeof(float) * 16);
Bones.push_back(new_bone);
Then I simply send this to my shader with
glUniformMatrix4fv(MatrixArray, bone_count, GL_FALSE, &MATRIX_BUFFER[0][0][0]);
And apply it in the vertex shader with:
mat4 Bone = V_MatrixArray[int(Indicies[0])] * Weights[0];
Bone += V_MatrixArray[int(Indicies[1])] * Weights[1];
Bone += V_MatrixArray[int(Indicies[2])] * Weights[2];
Bone += V_MatrixArray[int(Indicies[3])] * Weights[3];
vec4 v = Bone * vec4(Vertex, 1);
gl_Position = MVP * vec4(v.xyz, 1);
This code mostly works, however my mesh is very deformed.... It looks like this:
According to the reserach ive done so far:
I do not need to transpose my matrices since assimp uses OpenGL column major
I do not need to read the nodes of the scene yet since they are for
animation
Please correct me if I am mistaken about these last 2 things.
I managed to solve it. Turns out you DO need to read the nodes, even for a simple bind pose without animation. For any future readers with this issue, each bones matrix = root_node_inverse * matrix concatenated from node heirechy * bone offset matrix.

Applying World Transform

New to OpenGl I am trying to make sure i did this part right, im told to build the world matrix from the position, scaling and rotation information.
From the material i found online my understanding is
p^world = p^world∗p^Model
P^Model = Scaling * Rotation * Translation
Therefore i coded the following:
glm::mat4 Model::GetWorldMatrix() const
{
// #TODO 2, you must build the world matrix from the position, scaling and rotation informations
glm::mat4 pModel = GetScaling() * GetRotationAngle() * GetPosition();
glm::mat4 worldMatrix(1.0f);
worldMatrix = worldMatrix* pModel;
// #TODO 4 - Maybe you should use the parent world transform when you do hierarchical modeling
return worldMatrix;
}
void Model::SetPosition(glm::vec3 position)
{
mPosition = position;
}
void Model::SetScaling(glm::vec3 scaling)
{
mScaling = scaling;
}
void Model::SetRotation(glm::vec3 axis, float angleDegrees)
{
mRotationAxis = axis;
mRotationAngleInDegrees = angleDegrees;
}
Is this correct?? Thank you for your time and help.
The way to do it is to save one 4x4 Matrix for every Model. This matrix is although called the ModelMatrix.
All important informations (position,rotation,scale) of the object are saved in this matrix. If you want to translate,rotate or scale your object you generate a transformation matrix and multiply it from the left side to your model matrix. model = trafo * model; You can generate these transformation matrices with GLM http://glm.g-truc.net/0.9.2/api/a00245.html .

Reverse vertex winding order using matrices

I'm currently implementing reflections (Using the render-to-texture method), and so far it works, the only problem is that all objects in the reflected version are rendered inside out.
I'd like to avoid changing the internal OpenGL vertex winding order to make sure I don't interfere with other rendering operations too much.
Instead, I'd like to transform my reflection matrix to do the reversing. (Which should be just another reflection?)
This is my reflection matrix (which transforms the view matrix of the camera):
glm::mat4 matReflection(
1.f -2.f *n.x *n.x,-2.f *n.x *n.y,-2.f *n.x *n.z,-2.f *n.x *d,
-2.f *n.x *n.y,1.f -2.f *n.y *n.y,-2.f *n.y *n.z,-2.f *n.y *d,
-2.f *n.x *n.z,-2.f *n.y *n.z,1.f -2.f *n.z *n.z,-2.f *n.z *d,
0.f,0.f,0.f,1.f
);
n = normal of the plane of the reflection; d = distance of the plane
Is it even possible to reverse the vertex order through a matrix transformation? If so, how exactly?
If you apply a matrix to scale the y-axis by -1 after all other transformations (including projection), you will end up with an upside-down image (which you can use by including something like uv.y = 1-uv.y somewhere in your pipeline).