I am in the process of writing an animation system with my own Collada parser and am running into an issue that I can't wrap my head around.
I have collected my mesh/skin information (vertices, normals, jointIds, weights, etc), my skeleton information (joints, their local transforms, inverse bind position, hierarchy structure), and my animation (keyframe transform position for each joint, timestamp).
My issue is that with everything calculated and then implemented in the shader (the summation of weights multiplied by the joint transform and vertex position) - I get the following:
When I remove the weight multiplication, the mesh remains fully intact - however the skin doesn't actually follow the animation. I am at a lost as I feel as though the math is correct, but very obviously I am going wrong somewhere. Would someone be able to shine light on the aspect I have misinterpreted?
Here is my current understanding and implementation:
After collecting all of the joint's localTransforms and hierarchy, I calculate their inverse bind transfromation matrix. To do this I multiple each joints localTransform with their parentLocalTransform to get a bindTransform. Inverting that bindTransform results in their inverseBindTransform. Below is my code for that:
// Recursively collect each Joints InverseBindTransform -
// root joint's local position is an identity matrix.
// Function is only called once after data collection.
void Joint::CalcInverseBindTransform(glm::mat4 parentLocalPosition)
{
glm::mat4 bindTransform = parentLocalPosition * m_LocalTransform;
m_InverseBindPoseMatrix = glm::inverse(bindTransform);
for (Joint child : Children) {
child.CalcInverseBindTransform(bindTransform);
}
}
Within my animator during an animation, for each joint I take the two JointTransforms for the two frame's my currentTime is in between and I calculate the interpolated JointTransform. (JointTransform simply has a vec3 for position and quaternion for rotation). I do this for every joint and then apply those interpolated values to each Joint by again recursively muliplying the new frameLocalTransform by their parentLocalTransform. I take that bindTransform and multiply it by the invBindTransform and then transpose the matrix. Below is the code for that:
std::unordered_map<int, glm::mat4> Animator::InterpolatePoses(float time) {
std::unordered_map<int, glm::mat4> poses;
if (IsPlaying()) {
for (std::pair<int, JointTransform> keyframe : m_PreviousFrame.GetJointKeyFrames()) {
JointTransform previousFrame = m_PreviousFrame.GetJointKeyFrames()[keyframe.first];
JointTransform nextFrame = m_NextFrame.GetJointKeyFrames()[keyframe.first];
JointTransform interpolated = JointTransform::Interpolate(previousFrame, nextFrame, time);
poses[keyframe.first] = interpolated.getLocalTransform();
}
}
return poses;
}
void Animator::ApplyPosesToJoints(std::unordered_map<int, glm::mat4> newPose, Joint* j, glm::mat4 parentTransform)
{
if (IsPlaying()) {
glm::mat4 currentPose = newPose[j->GetJointId()];
glm::mat4 modelSpaceJoint = parentTransform * currentPose;
for (Joint child : j->GetChildren()) {
ApplyPosesToJoints(newPose, &child, modelSpaceJoint);
}
modelSpaceJoint = glm::transpose(j->GetInvBindPosition() * modelSpaceJoint);
j->SetAnimationTransform(modelSpaceJoint);
}
}
I then collect all the newly AnimatedTransforms for each joint and send them to the shader:
void AnimationModel::Render(bool& pass)
{
[...]
std::vector<glm::mat4> transforms = GetJointTransforms();
for (int i = 0; i < transforms.size(); ++i) {
m_Shader->SetMat4f(transforms[i], ("JointTransforms[" + std::to_string(i) + "]").c_str());
}
[...]
}
void AnimationModel::AddJointsToArray(Joint current, std::vector<glm::mat4>& matrix)
{
glm::mat4 finalMatrix = current.GetAnimatedTransform();
matrix.push_back(finalMatrix);
for (Joint child : current.GetChildren()) {
AddJointsToArray(child, matrix);
}
}
In the shader, I simply follow the summation formula that can be found all over the web when researchiing this topic:
for (int i = 0; i < total_weight_amnt; ++i) {
mat4 jointTransform = JointTransforms[jointIds[i]];
vec4 newVertexPos = jointTransform * vec4(pos, 1.0);
total_pos += newVertexPos * weights[i];
[...]
---------- Reply to Normalizing Weights ------------
There were a few weights summing above 1, but after solving the error in my code the model looked like this:
For calculating the weights - I loop through all preadded weights in the vector, and if I find a weight that is less than the weight I'm looking to add - I replace that weight in that position. Otherwise, I append the weight onto the end of the vector. If there are less weights in my vector than my specified max_weights (which is 4) - I fill in the remaining weights/jointIds with 0.
I understand when something is going wrong in skinning animations, there can be alot of different areas the problem is occuring. As such, for future googlers experiencing the same issue I am - take this as more of a list of suggestions of what you could be doing wrong rather than absolutely doing wrong.
For my problem - I had the right idea but wrong approach in a lot of minor areas. Which brought me fairly close but, as they say, no cigar.
I had no need to calculate the Inverse Bind Pose myself, Collada's Inverse Bind Pose (sometimes/often declared as an "offsetMatrix") is more than perfect. This wasn't a problem more as I was just doing unnecessary calculations.
In a Collada file, they often provide you more "joints" or "nodes" in the hierarchy than what is needed for the animation. Prior to the start of your actual animated "joints", there is the scene and an initial armature "node" type. The scene is typically an identity matrix that was manipulated based on your "up axis" upon reading in the Collada file. The Node type will determine the overall size of each joint in the skeleton - so if it wasn't resized, its probably the identity matrix. Make sure your hierarchy still contains ALL nodes/joints listed in the hierarchy. I very much was not doing so - which greatly distorted my globalPosition (BindPose).
If you are representing your Joint's transforms rotation through quaternions (which is highly recommended), make sure the resulted quaternion is normalized after interpolating between two rotated positions.
On the same note - when combining the Rotation and Transform into your final matrix - make sure your order of multiplication and the final output is correct.
Finally - your last skinning matrix is comprised of your joints InvBindMatrix * GlobalPosition * GlobalInverseRootTransform (<- this is the inverse of the local transfrom from your "scene" node mentioned in (1), remember?).
Based on your prior matrix multiplications up to this point, you may or may not need to transpose this final matrix.
And with that - I was able to successfully animate my model!
One final note - my mesh and animation files are added in separately. If your animations are in separate files from your mesh, make sure you collect the skinning/joint information from the files with an animation rather than the file with the mesh. I list my steps for loading in a model and then giving it multiple animations through different files:
Load in the Mesh (This contains Vertices,Normals,TexCoords,JointIds,Weights)
Load in the animation file (This gives Skeleton, InverseBindPositions, and other needed info to bind skeleton to mesh) - Once skeleton and binding info is collected, gather first animation info from that file as well.
For another animation, the above Skeleton should work fine for any other animation on the same mesh/model - just read in the animation information and store in your chosen data structure. Repeat step 3 til happy.
Related
I have a simple model with a simple skeletal structure that I made in blender. Here's how it looks:
And here's the hierarchy in blender:
As you can see it has two bones: One that goes halfway up the rectangular box ("Bone"), that is completely stationary. And another bone ("Bone.001") that goes from the halfway point and up to the top, that rotates.
I've imported the mesh using AssImpNet, and extracted the rotation, scaling and position keys from the animation node channels. When I apply those transformations to the mesh, I get this result (colored by bone weight):
The motion/animation seems to play correctly, so I believe this part works correctly. Now this is where my understanding starts to break down, but I believe the crucial part I'm missing is calculating the "inverse bind pose" (I've seen a few names for it), and applying that to the bone transformations that I feed into my shader as well. But so far I haven't been able to find exactly what it is that I need to extract from AssImp's format, and multiply together to get the correct final transformation. I've only found vague explanations about traversing the node tree and "undoing" transformations from each parent node, or something.
Here's what I tried, which doesn't seem to be working:
i thought that, for the base bone ("Bone"), i would need:
- the global inverse transform
- the node transform of the node with name "Bone"
- the rotation/position/scale keys from the animation channel with name "Bone"
- the bone offset from Meshes.Bones.OffsetMatrix from the bone named "Bone"
and then multiply them together in that order.
similarly for "Bone.001":
- the global inverse transform
- the node transform of the node with name "Bone"
- the rotation/position/scale keys from the animation channel with name "Bone"
- the node transform of the node with name "Bone.001"
- the rotation/position/scale keys from the animation channel with name "Bone.001"
- the bone offset from Meshes.Bones.OffsetMatrix from the bone named "Bone.001"
My attempt at implementing this (hard coding the indexes for now just to try to get things working), note that it's using C#/AssImpNet, so the naming conventions are a bit different from C++/AssImp:
// "Bone"
public Matrix4 Bone0Transform(double time)
{
var bone0 = MathUtils.ConvertMatrix(Scene.RootNode.Children[1].Children[0].Transform);
var frame0 = GetTransformedFrame(1, TimeToFrame(1, time));
var global = MathUtils.ConvertMatrix(Scene.RootNode.Transform).Inverted();
var offset0 = MathUtils.ConvertMatrix(Scene.Meshes[0].Bones[0].OffsetMatrix);
var total = global * bone0 * frame0 * offset0;
return total;
}
// "Bone.001"
public Matrix4 Bone1Transform(double time)
{
var bone0 = MathUtils.ConvertMatrix(Scene.RootNode.Children[1].Children[0].Transform);
var bone1 = MathUtils.ConvertMatrix(Scene.RootNode.Children[1].Children[0].Children[0].Transform);
var frame0 = GetTransformedFrame(1, TimeToFrame(1, time));
var frame1 = GetTransformedFrame(2, TimeToFrame(2, time));
var global = MathUtils.ConvertMatrix(Scene.RootNode.Transform).Inverted();
var offset1 = MathUtils.ConvertMatrix(Scene.Meshes[0].Bones[1].OffsetMatrix);
var total = global * bone0 * frame0 * bone1 * frame1 * offset1;
return total;
}
GetTransformedFrame returns a Matrix4 combining the scale, rotation and position keys for the frame that corresponds to the current time, and on its own gives the result you can see in the gif where the box is colored red/green.
All this gives me is an obviously incorrect result:
So my question is this: Is my understanding of how to calculate the final bone transformations wrong? If so, what is the correct way of doing it?
I'm just wondering if there was any way which one can perform mouse picking detection onto any object. Whether it would be generated object or imported object.
[Idea] -
The idea I have in mind is that, there would be iterations with every object in the scene. Checking if the mouse ray has intersected with an object. For checking the intersection, it would check the mouse picking ray with the triangles that make up the object.
[Pros] -
I believe the benefit of this approach is that, every object can be detected with mouse picking since they all inherit from the detection method.
[Cons] -
I believe this drawbacks are mainly the speed and the method being very expensive. So would need fine tuning of optimization.
[Situation] -
In the past I have read about mouse picking and I too have implemented some basic form of mouse picking. But all those were crappy work which I am not proud of. So again today, I have re-read some of the stuff from online. Nowadays I see alot of mouse picking using color ids and shaders. I'm not too keen for this method. I'm more into a mathematical side.
So here is my mouse picking ray thingamajig.
maths::Vector3 Camera::Raycast(s32 mouse_x, s32 mouse_y)
{
// Normalized Device Coordinates
maths::Vector2 window_size = Application::GetApplication().GetWindowSize();
float x = (2.0f * mouse_x) / window_size.x - 1.0f;
float y = 1.0f;
float z = 1.0f;
maths::Vector3 normalized_device_coordinates_ray = maths::Vector3(x, y, z);
// Homogeneous Clip Coordinates
maths::Vector4 homogeneous_clip_coordinates_ray = maths::Vector4(normalized_device_coordinates_ray.x, normalized_device_coordinates_ray.y, -1.0f, 1.0f);
// 4D Eye (Camera) Coordinates
maths::Vector4 camera_ray = maths::Matrix4x4::Invert(projection_matrix_) * homogeneous_clip_coordinates_ray;
camera_ray = maths::Vector4(camera_ray.x, camera_ray.y, -1.0f, 0.0f);
// 4D World Coordinates
maths::Vector3 world_coordinates_ray = maths::Matrix4x4::Invert(view_matrix_) * camera_ray;
world_coordinates_ray = world_coordinates_ray.Normalize();
return world_coordinates_ray;
}
I have this ray plane intersection function which calculates if a certain ray as intersected with a certain plane. DUH!
Here is the code for that.
bool Camera::RayPlaneIntersection(const maths::Vector3& ray_origin, const maths::Vector3& ray_direction, const maths::Vector3& plane_origin, const maths::Vector3& plane_normal, float& distance)
{
float denominator = plane_normal.Dot(ray_direction);
if (denominator >= 1e-6) // 1e-6 = 0.000001
{
maths::Vector3 vector_subtraction = plane_origin - ray_origin;
distance = vector_subtraction.Dot(plane_normal);
return (distance >= 0);
}
return false;
}
There are many more out there. E.g. Plane Sphere Intersection, Plane Disk Intersection. These things are like very specific. So it feel that is very hard to do mouse picking intersections on a global scale. I feel this way because, for this very RayPlaneIntersection function. What I expect to do with it is, retrieve the objects in the scene and retrieve all the normals for that object (which is a pain in the ass). So now to re-emphasize my question.
Is there already a method out there which I don't know, that does mouse picking in one way for all objects? Or am I just being stupid and not knowing what to do when I have everything?
Thank you. Thank you.
Yes, it is possible to do mouse-picking with OpenGL: you render all the geometry into a special buffer that stores a unique id of the object instead of its shaded color, then you just look at what value you got at the pixel below the mouse and know the object by its id that is written there. However, although it might be simpler, it is not a particularly efficient solution if your camera or geometry constantly moves.
Instead, doing an analytical ray-object intersection is the way to go. However, you don't need to check the intersection of every triangle of every object against the ray. That would be inefficient indeed. You should cull entire objects by their bounding boxes, or even portions of the whole scene. Game engines have their own spacial index data structure to speed-up ray-object intersections. They need it not only for mouse picking, but also for collision-detection, physics simulations, AI, and what-not.
Also note that the geometry used for the picking might be different from the one used for rendering. One example that comes to mind is that of semi-transparent objects.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I am looking for a way to do a wormhole effect like this:
- https://www.youtube.com/watch?v=WeOBXmLeJWo&feature=youtu.be&t=43s
I have already found nice tunnels in the examples, but here it is a little bit more involved. Actually the space appears to be warped somehow and the movement is high velocity, so it is not just entering into a simple tunnel. Any idea how to do the space warping part of it?
I decided to add more info because this was too broad:
I have a galaxy and each star has a 3d coord, size, etc. in this galaxy. I can visit these stars with a space ship. There are very distant stars and it would take a lot of time to get to them, that's why I need warp (faster than light) speed. This does not necessarily requires a wormhole according to physics, but this app does not have to be overly realistic. I don't want to solve this with pure OpenGL so ofc. we can use shaders. I'd like to warp the space in the middle of the screen when accelerating to warp speeds. After that a tunnel effect can come, because I think it would consume a lot of resources to update every star by very high speeds, so I'd like to update only the close stars. This can't be a prerendered animation, because the destination is not always certain, so this has sometimes exploration purposes and sometimes traveling purposes. I don't think warping only the sky box is enough, but I am not sure about this.
There are 2 things going on there:
space curvature around the hole
You need to construct equation that describe the curvature of space around hole parametrized by hole parameters (mass,position,orientation) and time so you can animate it. Then from this curvature you can compute relative displacement of pixel/voxel around it. I would start with cylindrical cones with radius modulated by sin of the distance from hole +/- some animation parameters (need experimentation).
Something like this:
and for example start with (in wormhole local coordinates LCS):
r = R * sin(z*0.5*M_PI/wormhole_max_depth)
Then modulate it by additional therms. The wormhole_max_depth,R should be functions of time even linear or with some periodic therm so it is pulsating a bit.
The displacement can be done by simply computing distance of concerned point to cone surface and push it towards it the more the closer to the cone it is (inside cone voxels are considered below surface so apply max displacement strength)
particle/light/matter bursting out of the hole
I would go for this only when #1 is alredy done. It should be simple particle effect with some nice circular blended alpha texture animated on the surface of the cone from #1. I see it as few for loops with pseudo random displacement in position and speed ...
Techniques
This topic depends on how you want to do this. I see these possibilities:
Distort geometry during rendering (3D vector)
So you can apply the cone displacement directly on rendered stuff. This would be best applicable in GLSL but the geometry rendered must have small enough primitives to make this work on vertex level ...
Distort skybox/stars only (3D vector or 2D raster but objects stay unaffected)
So you apply the displacement on texture coordinates of skybox or directly on the star positions.
Distort whole rendered scene in second pass (2D raster)
This need to use 2 pass rendering and in the second pass just wrap the texture coordinates near hole.
As you got different local stars in each sector I would use star background generated from star catalogue (list of all your stars) And apply the distortion on them directly in 3D vector space (so no skybox.. option #2). And also because my engines already use such representation and rendering for the same reasons.
[Edit1] cone geometry
I haven't much time for this recently until today so I did not make much of a progress. I decided to start with cone geometry so here it is:
class wormhole
{
public:
reper rep; // coordinate system transform matrix
double R0,R1,H,t; // radiuses,depth
wormhole(){ R0=10.0; R1=100.0; H=50.0; t=0.0; };
wormhole(wormhole& a){ *this=a; };
~wormhole(){};
wormhole* operator = (const wormhole *a) { *this=*a; return this; };
/*wormhole* operator = (const wormhole &a) { ...copy... return this; };*/
void ah2xyz(double *xyz,double a,double h) // compute cone position from parameters a=<0,2pi>, h=<0,1>
{
double r,tt;
tt=t; if (t>0.5) tt=0.5; r=2.0*R0*tt; // inner radius R0
tt=t; if (t>1.0) tt=1.0; r+=(R1-r)*h*h*tt; // outer radius R1
xyz[0]=r*cos(a);
xyz[1]=r*sin(a);
xyz[2]=H*h*tt;
rep.l2g(xyz,xyz);
}
void draw_cone()
{
int e;
double a,h,da=pi2*0.04,p[3];
glColor3f(0.2,0.2,0.2);
for (h=0.0;h<=1.0;h+=0.1){ glBegin(GL_LINE_STRIP); for (e=1,a=0.0;e;a+=da) { if (a>=pi2) { e=0; a=0.0; } ah2xyz(p,a,h); glVertex3dv(p); } glEnd(); }
for (e=1,a=0.0;e;a+=da){ glBegin(GL_LINE_STRIP); for (h=0.0;h<=1.0;h+=0.1) { if (a>=pi2) { e=0; a=0.0; } ah2xyz(p,a,h); glVertex3dv(p); } glEnd(); }
}
} hole;
Where rep is my class for homogenous 4x4 transform matrix (remembering both direct and inverse matrices at the same time) function l2g just transforms from local coordinates to global. The cone parameters are:
R0 - inner cone radius when fully grown
R1 - outer cone radius when fully grown
H - the height/depth of the cone when fully grown
t - is animation parameter values <0.0,1.0> are the growth and values above 1.0 are reserved for wormhole fully grown animation
Here how it looks like:
What I would do is simply calculate a vector from the texture coordinate of the screen center to the texture coordinate of the pixel you're shading.
Then modify that vector in any way you want (time based for example) and apply it to the texture coordinate of the pixel you're shading and then use the resulting coordinate to sample your texture.
In pseudocode this would be something like this:
vec2 vector_to_screen_center = vec2(0.5) - texture_coordinate;
texture_coordinate += vector_to_screen_center * sin(time) * 0.1; // Time based modulation of the vector.
gl_FragColor = texture2D(screen_texture, texture_coordinate);
Your question does not have a GLSL tag. If you plan to do this without shaders, this is going to be hard and/or inefficient.
I have the following issue when trying to map UV-coordinates to a sphere
Here is the code I'm using to get my UV-coordinates
glm::vec2 calcUV( glm::vec3 p)
{
p = glm::normalize(p);
const float PI = 3.1415926f;
float u = ((glm::atan(p.x, p.z) / PI) + 1.0f) * 0.5f;
float v = (asin(p.y) / PI) + 0.5f;
return glm::vec2(u, v);
}
The issue was very well explained at this stackoverflow question, although, I still don't get how can I fix it. From what I've been reading, I have to create a duplicate pair of vertices. Does anyone know some good and effcient way of doing it ?
The problem you have is, that at the seam your texture coordinates "roll" back to 0, so you get the whole texture mapped, mirrored onto the seam. To avoid this you should use GL_WRAP repeat mode and at the seam finish with vertices with texture coordinates >= 1 (don't roll back to 0). Remember that a vertex consists of the whole tuple of all its attributes and vertices with different attribute values are different in the whole, so there's no point in trying to "share" the vertices.
Another way to do it is simply to pass the object coordinates of the sphere into the pixel shader, and calculate the UV in "perfect" spherical space.
Be aware that you will need to pass the local derivatives so that you don't merely reduce your seam from several pixels to one.
Otherwise, yes. You need to duplicate vertices along the same edge as u=0, and likewise repeat the vertices at the poles. In this way, your object topology will become a rectangle: just like your texture.
I'm trying to implement skeletal animation in a small program I'm writing. The idea is to calculate the transformation matrix on the CPU every frame by interpolating keyframe data, then feeding this data to my vertex shader which multiplies my vertices by this matrix like this:
vec4 v = animationMatrices[int(boneIndices.x)] * gl_Vertex * boneWeights.x;
Where boneWeights and boneIndices are attributes and animationMatrices is a uniform array of transformation matrices updated every frame before drawing. (The idea is to have multiple bones affecting one vertex eventually, but right now I'm testing with one bone per vertex so just taking the weight.x and indices.x is enough).
Now the problem is calculating the transformation matrix for each bone. My transformation matrix for the single joint is good, the problem is that it always takes (0,0,0) as pivot instead of the pivot. I took the joint matrices from the COLLADA which correctly shows my skeleton when I draw them like this:
public void Draw()
{
GL.PushMatrix();
drawBone(Root);
GL.PopMatrix();
}
private void drawBone(Bone b)
{
GL.PointSize(50);
GL.MultMatrix(ref b.restMatrix);
GL.Begin(BeginMode.Points);
GL.Color3((byte)0, (byte)50, (byte)0);
if (b.Name == "Blades")
{
GL.Vertex3(0, 0, 0);
}
GL.End();
foreach (Bone bc in b.Children)
{
GL.PushMatrix();
drawBone(bc);
GL.PopMatrix();
}
}
So now to calculate the actual matrix I've tried:
Matrix4 jointMatrix = b.restMatrixInv * boneTransform * b.restMatrix;
or according to the collada documentation (this doesn't really make sense to me):
Matrix4 jointMatrix = b.restMatrix * b.restMatrixInv * boneTransform;
And I know I also have to put the parent matrix in here somewhere, I'm guessing something like this:
Matrix4 jointMatrix = b.restMatrixInv * boneTransform * b.restMatrix * b.Parent.jointMatrix;
But at the moment I'm mostly just confused and any push in the right direction would help. I really need to get this order right...