I am in the process of rolling my own openGL framework, and know how to draw 3d objects ... etc...
But how do you define relationships between 3d objects that may have a joint?
Or how do you define the 3d object as being a "bone"?
Are there any good resources?
As OpenGL is only a graphics library and not a 3D modeling framework the task of defining and using "bones" falls onto you.
There are different ways of actually implementing it, but the general idea is:
You treat each part of your model as a bone (e.g. head, torso, lower legs, upper legs, etc).
Each bone has a parent which it is connected to (e.g. the parent of the lower left leg is the upper left leg).
Thus each bone has a number of children.
Now you define each bone's position as a relative position to the parent bone. When displaying a bone you now multiply it's relative position with the parent bone's relative position to get the absolute position.
To visualize:
Think of it as a doll. When you grab the doll's arm and move it around, the relative position (and rotation) of the hand won't change. Its absolute position WILL change because you've moved one of its parents around.
When tried skeletal animations I learnt most of it from this link:
http://content.gpwiki.org/index.php/OpenGL:Tutorials:Basic_Bones_System
But how do you define relationships between 3d objects that may have a joint?
OpenGL does not care about these things. I't a pure drawing API. So it's upon you to unleash your creativity and define such structures yourself. The usual approach to skeletal animatio is having a bone/rig system, where each bone has an orientation (represented by a quaternion or a 3×3 matrix) a length and a list of bones attached to it further, i.e. some kind of tree.
I'd define this structure as
typedef float quaternion[4];
struct Bone {
quaternion orientation;
float length;
int n_subbones;
Bone *subbones;
};
In addition to that you need a pivot from where the rig starts. I'd do it like this
typedef float vec3[3];
struct GeomObjectBase {
vec3 position;
quaternion orientation;
};
struct BoneRig {
struct GeomObjectBase gob;
struct Bone pivot_bone;
}
Next you need some functions that iterate through this structure, generate the matrix palette out of it, so that it can be applied to the model mesh.
note: I'm using freeglut
Totally irrelevant
Related
I have already loaded vertices, materials, normals, weights, joint IDs, joints themselves and the parent children info (hierarchy), I have also managed to render it all, and when I rotate or translate one of the joints, the children rotate with the parent.
My problem is, the parent rotates on a wrong point or offset (hopefully you understand what I mean), this means, that I've gotten the initial offsets wrong, right? To get the start t-pose, I'm guessing I don't need rotation or translation, only the offset of the position of the joint, but I have no clue of how to get it, been stuck for ages. In the Collada file, there is a transform for each joint, I've loaded that one also, but I don't know how to implement it correctly, my 3d model gets deformed and looks wrong.
If you answer this question please make it as if you where explaining it to a monkey (me), and step by step if possible, I'm unfamiliar with these bind and inverse bind terms, and very confused. I think that if i manage to get this, I'll eventually figure out the rest of skeletal animation myself, so it's just this little thing.
I've recently gotten bones, joints and nodes working, so I'll try to explain exactly how I achieved it. Do note, I am using Assimp to import my DAE files, but as far as I know, Assimp doesn't do any processing on the data, so this explanation should directly relate to the data in the Collada file.
I'm just learning all this myself, so I may get things wrong. If I do, anyone, please tell me and I will update this answer accordingly.
Semantics
A mesh is a set of vertices, normals, texture coordinates and faces. The points stored in a mesh are in a bind pose, or a rest pose. This is often, but not always, a T-pose.
A skin is a controller. It refers to a single mesh, and contains the list of bones that will modify that mesh (this is where the bones are stored). You can think of the skin element as the actual model (or part of the model) that will be rendered.
A bone is a flat list of names and associated matrices. There is no hierarchical data here, it is simply a flat list. The hierarchy is provided by the nodes that refer to the bones.
A node, or joint, is a hierarchical data element. They are stored in a hierarchy, with a parent node having zero or more child nodes. A node may be linked to zero or more bones, and may be linked to zero or more skins. There should only be one root node. A joint is the same as a node, so I will refer to joins as nodes.
Do note that nodes and bones are separate. You do not modify a bone to animate your model. Instead, you modify a node, which gets applied to the bone when the model is rendered.
Skin
A skin is the thing you will render. A skin always refers to one single mesh. You can have multiple skins in a DAE file, as part of the same model (or, scene). Sometimes, a model will reuse meshes by transforming them. For instance, you may have a mesh for a single arm, and reuse that arm, mirrored, for the other side of the body. I believe that is what the bind_shape_matrix value of a skin is used for. So far, I haven't used this, and my matrices are always identity, so I cannot speak as to it's usage.
Bone
A bone is what applies transformations to your model. You do not modify bones. Instead, you modify the nodes that control the bones. More on this later.
A bone consists of the following:
A name, used to find the node that controls this bone (Name_array)
A bind pose matrix, sometimes called an "inverse bind matrix" or an "offset matrix" (bind_poses array)
A list of vertex indices that the bone will affect (vertex_weights element)
A list of weights of the same length above, that tell how much the bone will affect that vertex. (weights array)
Node
A node is a hierarchical data element, describing how the model gets transformed when rendered. You will always start with one root node, and travel up the node tree, applying transforms in sequence. I use a depth-first algorithm for this.
The node tells how the model, skins, and bones should be transformed when rendering or animating.
A node may refer to a skin. This means that skin will be used as part of the render for this model. If you see a node refer to a skin, it gets included when rendering.
A node consists of the following:
A name (sid attribute)
A transform matrix (transform element)
Child nodes (node elements)
GlobalInverseTransform Matrix
The GlobalInverseTransform matrix is calculated by taking the Transform matrix of the first node, and inverting it. Simple as that.
The Algorithm
Now we can get to the good bits - the actual skinning and rendering.
Calculating a node's LocalTransform
Each node should have a matrix, called the LocalTransform matrix. This matrix isn't in the DAE file, but is calculated by your software. It is basically the accumulation of the Transform matrices of the node, and all its parents.
First step is to traverse the node hierarchy.
Start at the first node, and calculate the LocalTransform for the node, using the Transform matrix of the node, and the LocalTransform of the parent. If the node has no parent, use an identity matrix as the parent's LocalTransform matrix.
Node.LocalTransform = ParentNode.LocalTransform * Node.Transform
Repeat this process recursively for every child node in this node.
Calculating a bone's FinalTransform matrix
Just like a node, a bone should have a FinalTransform matrix. Again, this is not stored in the DAE file, it is calculated by your software as part of the render process.
For each mesh used, for each bone in that mesh, apply the following algorithm:
For each mesh used:
For each bone in mesh:
If a node with the same name exists:
Bone.FinalTransform = Bone.InverseBind * Node.LocalTransform * GlobalInverseTransform
Otherwise:
Bone.FinalTransform = Bone.InverseBind * GlobalInverseTransform
We now have the FinalTransform matrix for each bone in the model.
Calculating a vertex's position
Once we have all the bones calculated, we can then transform the mesh's points into their final render locations. This is the algorithm I use. This is not the "correct" way to do this, as it should be calculated by a vertex shader on-the-fly, but it works to demonstrate what's happening.
From the root node:
For each mesh referred to by node:
Create an array to hold the transformed vertices, the same size as your source vertices array.
Create an array to hold the transformed normals, the same size as your source vertices array (normals and vertices arrays should be the same length at the beginning.
If the mesh has no bones:
Copy source vertices and source normals to output arrays - mesh is not skinned
Otherwise:
For every bone in the mesh:
For every weight in the bone:
OutputVertexArray(Weight.VertexIndex) = Mesh.InputVertexArray(Weight.VertexIndex) * Bone.FinalTransform * Weight.TransformWeight
OutputNormalArray(Weight.VertexIndex) = Normalize(Mesh.InputNormalArray(Weight.VertexIndex) * Bone.FinalTransform * Weight.TransformWeight)
Render the mesh, using OutputVertexArray, OutputNormalArray, Mesh.InputTexCoordsArray and the mesh's face indices.
Recursively call this process for each child node.
This should get you a correctly rendered output.
Note that with this system, it is possible to re-use a mesh more than once.
Animating
Just a quick note on animating. I haven't done much with this, and Assimp hides much of the gory details of Collada (and introduces its own form of gore), but to use predefined animations from your file, you do some interpolation of translations, rotations and scales to come up with a matrix that represents a node's animated state at as single point in time.
Remember, matrix construction follows the TRS (translate, rotate, scale) convention, where translations happen first, then rotations, then scale.
AnimatedNodeTransform = TranslationMatrix * RotationMatrix * ScaleMatrix
The generated matrix completely replaces the node's Transform matrix - it is not combined with the matrix.
I am still trying to work out how to perform on-the-fly animation (think Inverse Kinematics) correctly. For some models I try, it works great. I can apply a quaternion to the node’s Transform matrix and it will work. However, some other models will do strange things, like rotate the node around the origin, so I think I’m still missing something there. If I finally solve this, I will update this section to reflect what I discover.
Hope this helps. If I've missed anything, or gotten anything wrong, anyone please feel free to correct me. I am only learning this stuff myself. If I notice any mistakes, I will edit the answer.
Also, be aware that I use Direct3D, so my matrix multiplication order is probably reversed from yours. You will likely need to flip the multiplication order of some of the operations in my answer.
I have been trying to implement skeletal animation in my own 3D openGL/c++ game engine using ASSIMP to import the models.
I think that the problem is caused by the calculation of the bone matrices, loaded as uniform variables in the shader because if I assign them to identities, the mesh is loaded in its bind pose.
These are the functions I use to initialize and calculate the ready bone matrices:
void Joint::AddToSceneGraph(GameObject* root)
{
GameObject* parent = root->FindChild(m_name);
//m_globalInverseMatrix = inverse(root->GetTransform().GetMatrix());
parent->AddComponent(this);
while (parent->GetParent() != root)
parent = parent->GetParent();
parent->GetTransform().SetParent(NULL); (1)
}
mat4 Joint::GetMatrix()
{
mat4 ans = /*m_globalInverseMatrix*/GetTransform().GetMatrix()*m_offsetMatrix;
return ans;
}
Since I am trying to render the model in its bind pose I wont supply you the code of calculating animation matrices.
Some clarifications - I have a Transform class which has Transform* parent and whose GetMatrix() method calculates the matrix from the parent's one, scaling and translating vec3s and rotation quaternion so the relation between parent and child is taken in consideration. I assume that the parent's transform of the root joint has to be NULL thus its matrix - identity, which is the purpose of (1). Although I am not sure for this assumption, I am sure that the node with the root of the skeleton and the node, containing the meshes, are siblings.
Also I am not sure weather I should use m_globalInverseMatrix and furthermore what exactly is its purpose.
In general I think the main issue I have is misunderstanding of how ASSIMP calculates the offset matrix or so called inverse bind pose matrix and how to invert its effect. This issue results in the model looking "packed" in itself :
From the docs in Assimp, the offset matrix of the bones transforms from the mesh space to the bone space.
I managed to render the bind pos with a hierarchy similar to yours (if I have not misunderstood, you have a root node with two children, one spawning the skeleton hierarchy and the other spawning the mesh). What you have to do is set the m_globalInverseMatrix to the inverse transform of the node containing the mesh. Then, each bone's transform is:
boneTransform = m_globalInverseMatrix * boneHierarchyTransforms * boneOffsetMatrix
where the boneHierarchyTransform comes from traversing the tree up to each bone node and the boneOffsetMatrix is the one you mention and I referred to in the first paragraph.
I'm trying to do skeletal animation in OpenGL using Assimp as my model import library.
What exactly do I need to the with the bones' offsetMatrix variable? What do I need to multiply it by?
Let's take for instance this code, which I used to animate characters in a game I worked. I used Assimp too, to load bone information and I read myself the OGL tutorial already pointed out by Nico.
glm::mat4 getParentTransform()
{
if (this->parent)
return parent->nodeTransform;
else
return glm::mat4(1.0f);
}
void updateSkeleton(Bone* bone = NULL)
{
bone->nodeTransform = bone->getParentTransform() // This retrieve the transformation one level above in the tree
* bone->transform //bone->transform is the assimp matrix assimp_node->mTransformation
* bone->localTransform; //this is your T * R matrix
bone->finalTransform = inverseGlobal // which is scene->mRootNode->mTransformation from assimp
* bone->nodeTransform //defined above
* bone->boneOffset; //which is ai_mesh->mBones[i]->mOffsetMatrix
for (int i = 0; i < bone->children.size(); i++) {
updateSkeleton (&bone->children[i]);
}
}
Essentially the GlobalTransform as it is referred in the tutorial Skeletal Animation with Assimp or properly the transform of the root node scene->mRootNode->mTransformation is the transformation from local space to global space. To give you an example, when in a 3D modeler (let's pick Blender for instance) you create your mesh or you load your character, it is usually positioned (by default) at the origin of the Cartesian plane and its rotation is set to the identity quaternion.
However you can translate/rotate your mesh/character from the origin (0,0,0) to somewhere else and have in a single scene even multiple meshes with different positions. When you load them, especially if you do skeletal animation, it is mandatory to translate them back in local space (i.e. back at the origin 0,0,0 ) and this is the reason why you have to multiply everything by the InverseGlobal (which brings back your mesh to local space).
After that you need to multiply it by the node transform which is the multiplication of the parentTransform (the transformation one level up in the tree, this is the overall transform) the transform (formerly the assimp_node->mTransformation which is just the transformation of the bone relative to the node's parent) and your local transformation (any T * R) you want to apply to do: forward kinematic, inverse kinematic or key-frame interpolation.
Eventually there is the boneOffset (ai_mesh->mBones[i]->mOffsetMatrix) that transforms from mesh space to bone space in bind pose as stated in the documentation.
Here there is a link to GitHub if you want to look at the whole code for my Skeleton class.
Hope it helps.
The offset matrix defines the transform (translation, scale, rotation) that transforms the vertex in mesh space, and converts it to "bone" space. As an example consider the following vertex and a bone with the following properties;
Vertex Position<0, 1, 2>
Bone Position<10, 2, 4>
Bone Rotation<0,0,0,1> // Note - no rotation
Bone Scale<1, 1, 1>
If we multiply a vertex by the offset Matrix in this case we would get a vertex position of <-10, -1, 2>.
How do we use this? You have two options on how to use this matrix which is down to how we store the vertex data in the vertex buffers. The options are;
1) Store the mesh vertices in mesh space
2) Store the mesh vertices in bone space
In the case of #1, we would take the offsetMatrix and apply it to the vertices that are influenced by the bone as we build the vertex buffer. And then when we animate the mesh, we later apply the animated matrix for that bone.
In the case of #2, we would use the offsetMatrix in combination with the animation matrix for that bone when transforming the vertices stored in the vertex buffer. So it would be something like (Note: you may have to switch the matrix concatenations around here);
anim_vertex = (offset_matrix * anim_matrix) * mesh_vertex
Does this help?
As I already assumed, the mOffsetMatrix is the inverse bind pose matrix. This tutorial states the correct transformations that you need for linear blend skinning:
You first need to evaluate your animation state. This will give you a system transform from animated bone space to world space for every bone (GlobalTransformation in the tutorial). The mOffsetMatrix is the system transform from world space to bind pose bone space. Therefore, what you do for skinning is the following (assuming that a specific vertex is influenced by a single bone): Transform the vertex to bone space with mOffsetMatrix. Now assume an animated bone and transform the intermediate result back from animated bone space to world space. So:
boneMatrix[i] = animationMatrix[i] * mOffsetMatrix[i]
If the vertex is influenced by multiple bones, LBS simply averages the results. That's where the weights come into play. Skinning is usually implemented in a vertex shader:
vec4 result = vec4(0);
for each influencing bone i
result += weight[i] * boneMatrix[i] * vertexPos;
Usually, the maximum number of influencing bones is fixed and you can unroll the for loop.
The tutorial uses an additional m_GlobalInverseTransform for the boneMatrix. However, I have no clue why they do that. Basically, this undoes the overall transformation of the entire scene. Probably it is used to center the model in the view.
Im currently implementing an Octree for my bachelor thesis project.
My Octree takes a std::vector as argument:
octree::Octree::Octree(std::vector<const D3DXVECTOR3*> vec) :
m_vertices(std::vector<D3DXVECTOR3*>()),
m_size(m_vertices.size())
{
int i=0;
for(;i<m_size;++i) {
m_vertices.push_back(new D3DXVECTOR3(*vec.at(i)));
}
}
Im asking for what is typically used to store the vertices in before rendering them and making any culling test etc to them.
I kept this very simple for now, all i have is a function that renders a grid. Some snippets:
#define GRIDFVF (D3DFVF_XYZ | D3DFVF_DIFFUSE)
struct GridVertex {
D3DXVECTOR3 position;
DWORD color;
};
g_dev->SetTransform(D3DTS_WORLD, &matIdentity);
g_dev->SetStreamSource(0, g_buffer, 0, sizeof(GridVertex));
g_dev->SetTexture(0, NULL);
g_dev->DrawPrimitive(D3DPT_LINELIST, 0, GridSize * 4 + 2);
Now when rendering this i use my custom struct GridVertex, thats saves a D3DXVECTOR9 for pos and a DWORD for the color value and the tell the GPU by setting the flexible vertex format to GRIDFVF.
But in my Octree i only want to store the positions to perform the test if certain vertices are inside nodes within my Octree and so on. Therefore I thought of creating another class called SceneManager and storing all values within an std::vector and finally pass it to my Octree class, that does the test and afterwards pass the checked vertices to the GPU.
Would this be a solid solution or whats common to implement something like this?
Thanks in advance
Generally, one does not put the actual render geometry vertices themselves in the octree or whatever spatial partitioning structure one uses. That level of granularity is not useful, because if a set of vertices that make up a model spans partition nodes such that some subset of those vertices would be culled, you couldn't properly draw the model.
What you'd typically want to do is have an object representing an entity and its bounds within the world (axis-oriented bounding boxes, or bounding spheres, are simple and efficient bounding volumes, for example). Each entity is also associated with (or can be associated with by some other subsystem) rendering geometry. The entities themselves are sorted within the octree.
Then, you use your octree to determine which entities are visible, and submit all of their associated render geometry to the card.
I have implemented a small scene graph to be rendered by OpenGL, all the objects derive from a common Node class and during OpenGL frame rendering I just call the visit method of the root node and it traverses the graph recursively. The first matrix I pass when beginning traversal is the camera matrix.
The visit method looks like this:
void Node::visit(const QMatrix4x4 &mv) {
QMatrix4x4 m = mv;
m.rotate(m_rot);
m.translate(m_pos);
m.scale(m_scale);
m_effectiveMV = m;
for (int i = 0; i < m_children.size(); i++) {
m_children[i]->visit(m_effectiveMV);
}
draw(); // draws if this node has anything to draw,
// otherwise just transformation.
}
The problem I experience is, when I set rotation for a child node, the rotation happens relative to the parent node, not around the node itself. Can anyone spot what I'm doing wrong here?
Assuming your matrix methods are doing the right thing, translation should be the first one in the list:
m.translate(m_pos);
m.rotate(m_rot);
m.scale(m_scale);
This will first scale and rotate the vertex, then translate it into the parents system and so on.
Matrix operations are not commutative, i.e. the order in which matrix multiplication happens does matter. Rotating something first, then translate it, is different to first translating and then rotate/orbit around the original center.
Instead of creating the transformation by successive application of various transformations I recommend building it directly. The upper left 3×3 is the rotation part, which you can copy directly from the rotation matrix. Scaling multiplies x,y,z factors to the 1st, 2nd and 3rd column of the matrix. Translation is the 4th column. The rotation is either stored as a 3×3 matrix or a quaternion. Don't use Euler angles, they are numerically instable.