Skeletal Animation with Parenting Issues - opengl

tl;dr: When animating a model, each joint moves correctly, but not relative to its parent joint.
I am working on a skeletal animation system using a custom built IQE loader and renderer in Lua. Nearly everything is working at this point, except the skeleton seems to be disjointed when animating. Each joint translates, rotates, and scales correctly but is not taking the position of its parent into account, thus creating some awful problems.
In referencing the IQM spec and demo, I cannot for the life of me find out what is going wrong. My Lua code is (as far as I can tell) identical to the reference C++.
Calculating Base Joint Matrices:
local base = self.active_animation.base
local inverse_base = self.active_animation.inverse_base
for i, joint in ipairs(self.data.joint) do
local pose = joint.pq
local pos = { pose[1], pose[2], pose[3] }
local rot = matrix.quaternion(pose[4], pose[5], pose[6], pose[7])
local scale = { pose[8], pose[9], pose[10] }
local m = matrix.matrix4x4()
m = m:translate(pos)
m = m:rotate(rot)
m = m:scale(scale)
local inv = m:invert()
if joint.parent > 0 then
base[i] = base[joint.parent] * m
inverse_base[i] = inv * inverse_base[joint.parent]
else
base[i] = m
inverse_base[i] = inv
end
end
Calculating Animation Frame Matrices
local buffer = {}
local base = self.active_animation.base
local inverse_base = self.active_animation.inverse_base
for k, pq in ipairs(self.active_animation.frame[self.active_animation.current_frame].pq) do
local joint = self.data.joint[k]
local pose = pq
local pos = { pose[1], pose[2], pose[3] }
local rot = matrix.quaternion(pose[4], pose[5], pose[6], pose[7])
local scale = { pose[8], pose[9], pose[10] }
local m = matrix.matrix4x4()
m = m:translate(pos)
m = m:rotate(rot)
m = m:scale(scale)
local f = matrix.matrix4x4()
if joint.parent > 0 then
f = base[joint.parent] * m * inverse_base[k]
else
f = m * inverse_base[k]
end
table.insert(buffer, f:to_vec4s())
end
The full code is here for further examination. The relevant code is in /libs/iqe.lua and is near the bottom in the functions IQE:buffer() and IQE:send_frame(). This code runs on a custom version of the LOVE game framework, and a Windows binary (and batch file) is included.
Final note: Our matrix code has been verified against other implementations and several tests.

Transformations of parent bones should effect transformations of their children. Indeed, this is achieved, by specifying transformation of particular bone in a frame of it's parent. So, usually transformation of bones are specified in their local coordinate system, that depends on it's parent. If any of the parents transformed, this transformation would effect all children, even if their local transformations didn't changed.
In your case, you once cache all absolute (relative to the root, to be precise) transformation of each node. Then you update local transforms of each node using the cache, and do not update your cache. So, how the change of local transform of a node would effect it's child, if when you update the child you use cache instead of the actual parent transform?
There is one more issue. Why you do the following thing?
f = base[joint.parent] * m * inverse_base[k]
I mean, usually it would be just:
f = base[joint.parent] * m
I guess, that transformations recorded in animation is absolute (relative to the root, to be precise). It is very strange. Usually every transformation is local. Check this issue, because this will add you lots of problems.
More over, I don't see any need to cache something in your case (except of inverse_base, that is usually not needed).
Change your IQE:send_frame() function as follows:
local buffer = {}
local transforms = {}
local inverse_base = self.active_animation.inverse_base
for k, pq in ipairs(self.active_animation.frame[self.active_animation.current_frame].pq) do
local joint = self.data.joint[k]
local pose = pq
local pos = { pose[1], pose[2], pose[3] }
local rot = matrix.quaternion(pose[4], pose[5], pose[6], pose[7])
local scale = { pose[8], pose[9], pose[10] }
local m = matrix.matrix4x4()
m = m:translate(pos)
m = m:rotate(rot)
m = m:scale(scale)
local f = matrix.matrix4x4()
if joint.parent > 0 then
transforms[k] = transforms[joint.parent] * m
f = transforms[k] * inverse_base[k]
else
f = m * inverse_base[k]
transforms[k] = m
end
table.insert(buffer, f:to_vec4s())
end
This works good for me. Try to get rid of the inverse_base and you would be able to remove all animation related code from your IQE:buffer() function
P.S. Usually, all nodes are updated by traversing down the tree. However, you update nodes by going though the list. You should be aware, that you must guarantee somehow that for any node it's children would go after it.

Related

Calculating the final bone transformation for a skeletal animation

I have a simple model with a simple skeletal structure that I made in blender. Here's how it looks:
And here's the hierarchy in blender:
As you can see it has two bones: One that goes halfway up the rectangular box ("Bone"), that is completely stationary. And another bone ("Bone.001") that goes from the halfway point and up to the top, that rotates.
I've imported the mesh using AssImpNet, and extracted the rotation, scaling and position keys from the animation node channels. When I apply those transformations to the mesh, I get this result (colored by bone weight):
The motion/animation seems to play correctly, so I believe this part works correctly. Now this is where my understanding starts to break down, but I believe the crucial part I'm missing is calculating the "inverse bind pose" (I've seen a few names for it), and applying that to the bone transformations that I feed into my shader as well. But so far I haven't been able to find exactly what it is that I need to extract from AssImp's format, and multiply together to get the correct final transformation. I've only found vague explanations about traversing the node tree and "undoing" transformations from each parent node, or something.
Here's what I tried, which doesn't seem to be working:
i thought that, for the base bone ("Bone"), i would need:
- the global inverse transform
- the node transform of the node with name "Bone"
- the rotation/position/scale keys from the animation channel with name "Bone"
- the bone offset from Meshes.Bones.OffsetMatrix from the bone named "Bone"
and then multiply them together in that order.
similarly for "Bone.001":
- the global inverse transform
- the node transform of the node with name "Bone"
- the rotation/position/scale keys from the animation channel with name "Bone"
- the node transform of the node with name "Bone.001"
- the rotation/position/scale keys from the animation channel with name "Bone.001"
- the bone offset from Meshes.Bones.OffsetMatrix from the bone named "Bone.001"
My attempt at implementing this (hard coding the indexes for now just to try to get things working), note that it's using C#/AssImpNet, so the naming conventions are a bit different from C++/AssImp:
// "Bone"
public Matrix4 Bone0Transform(double time)
{
var bone0 = MathUtils.ConvertMatrix(Scene.RootNode.Children[1].Children[0].Transform);
var frame0 = GetTransformedFrame(1, TimeToFrame(1, time));
var global = MathUtils.ConvertMatrix(Scene.RootNode.Transform).Inverted();
var offset0 = MathUtils.ConvertMatrix(Scene.Meshes[0].Bones[0].OffsetMatrix);
var total = global * bone0 * frame0 * offset0;
return total;
}
// "Bone.001"
public Matrix4 Bone1Transform(double time)
{
var bone0 = MathUtils.ConvertMatrix(Scene.RootNode.Children[1].Children[0].Transform);
var bone1 = MathUtils.ConvertMatrix(Scene.RootNode.Children[1].Children[0].Children[0].Transform);
var frame0 = GetTransformedFrame(1, TimeToFrame(1, time));
var frame1 = GetTransformedFrame(2, TimeToFrame(2, time));
var global = MathUtils.ConvertMatrix(Scene.RootNode.Transform).Inverted();
var offset1 = MathUtils.ConvertMatrix(Scene.Meshes[0].Bones[1].OffsetMatrix);
var total = global * bone0 * frame0 * bone1 * frame1 * offset1;
return total;
}
GetTransformedFrame returns a Matrix4 combining the scale, rotation and position keys for the frame that corresponds to the current time, and on its own gives the result you can see in the gif where the box is colored red/green.
All this gives me is an obviously incorrect result:
So my question is this: Is my understanding of how to calculate the final bone transformations wrong? If so, what is the correct way of doing it?

FBXSDK, using Quaternions to set rotation keys?

I am trying to write a file save application using the Autodesk FBXSDK. I have this working fine using Euler rotations, but I need to update it to use quaternions.
The relevant function is:
bool CreateScene(FbxScene* pScene, double lFocalLength, int startFrame)
{
//Create Camera
FbxNode* lMyCameraNode = FbxNode::Create(pScene, "p_camera");
//connect camera node to root node
FbxNode* lRootNode = pScene->GetRootNode();
lRootNode->ConnectSrcObject(lMyCameraNode);
FbxCamera* lMyCamera = FbxCamera::Create(pScene, "Root_camera");
lMyCameraNode->SetNodeAttribute(lMyCamera);
// Create an animation stack
FbxAnimStack* myAnimStack = FbxAnimStack::Create(pScene, "My stack");
// Create the base layer (this is mandatory)
FbxAnimLayer* pAnimLayer = FbxAnimLayer::Create(pScene, "Layer0");
myAnimStack->AddMember(pAnimLayer);
// Get the camera’s curve node for local translation.
FbxAnimCurveNode* myAnimCurveNodeRot = lMyCameraNode->LclRotation.GetCurveNode(pAnimLayer, true);
//create curve nodes
FbxAnimCurve* myRotXCurve = NULL;
FbxAnimCurve* myRotYCurve = NULL;
FbxAnimCurve* myRotZCurve = NULL;
FbxTime lTime; // For the start and stop keys. int lKeyIndex = 0; // Index for the keys that define the curve
// Get the animation curve for local rotation of the camera.
myRotXCurve = lMyCameraNode->LclRotation.GetCurve(pAnimLayer, FBXSDK_CURVENODE_COMPONENT_X, true);
myRotYCurve = lMyCameraNode->LclRotation.GetCurve(pAnimLayer, FBXSDK_CURVENODE_COMPONENT_Y, true);
myRotZCurve = lMyCameraNode->LclRotation.GetCurve(pAnimLayer, FBXSDK_CURVENODE_COMPONENT_Z, true);
//This to add keys, per frame.
float frameNumber = startFrame;
for (int i = 0; i < rec.size(); i++)
{
lTime.SetFrame(frameNumber); //frame number
//rx
lKeyIndex = myRotXCurve->KeyAdd(lTime);
myRotXCurve->KeySet(lKeyIndex, lTime, recRotX[i], FbxAnimCurveDef::eInterpolationLinear);
//ry
lKeyIndex = myRotYCurve->KeyAdd(lTime);
myRotYCurve->KeySet(lKeyIndex, lTime, recRotY[i], FbxAnimCurveDef::eInterpolationLinear);
//rz
lKeyIndex = myRotZCurve->KeyAdd(lTime);
myRotZCurve->KeySet(lKeyIndex, lTime, recRotZ[i], FbxAnimCurveDef::eInterpolationLinear);
frameNumber += 1;
}
return true;
}
I would ideally like to pass in quaternion data here, instead of the euler x,y,z values. Is this possible with the fbxsdk? or do I need to convert my quaternion data first, and continue to pass in eulers?
Thank you.
You always need to go back to Euler angles, as you can only get animation curves for the XYZ rotation. The only thing you have control over is the rotation order.
However, you can use FbxQuaternion for your calculations, then use .DecomposeSphericalXYZ() to get XYZ Euler angles.
The accepted answer does not work. Although the documentation definitely implies it should,
Create an Euler XYZ equivalent to the current quaternion.
An Autodesk employee claims that it does not
DecomposeSphericalXYZ does not convert to Euler angles
and this is borne out by my testing. In the current FBX SDK, there are at least two relatively easy ways you can convert a quat to what they call an euler, or to something suitable for LclRotation. First is via FbxAMatrix
FbxQuaternion fq = ...;
FbxAMatrix fa;
fa.SetQ(fq);
FbxVector4 fe = fa.GetR();
Second is via FbxVector::SetXYZ
FbxVector4 fe2;
fe2.SetXYZ(fq);
I've successfully gone from an XYZ rotation sequence → quaternion → euler from both methods, and retrieved the same rotation sequence. When I use DecomposeSphericalXYZ I get a slightly different FbxVector4. I haven't tried to figure out what they mean by "euler in spherical coordinates".
Years later, I hit this issue again, and found a simple answer.
After you have set all the keys that you need, just use this filter:
FbxAnimCurveFilterUnroll filter;
filter.Apply(*myAnimCurveNodeRot);
This seems to function the same as the 'Euler Filter' in Maya, or the 'Gimbal Killer' filter in Motionbuilder.

skimage - Contrast stretch using python

I am trying to preform a simple contrast stretch with python skimage, on the image opened with gdal as array of type float32. I first calculate the percentile with:
p2, p98 = np.percentile(arrayF, (P1, P2))
and then try to perform the stretch with:
img_rescale = exposure.rescale_intensity(arrayF, in_range=(p2, p98))
The returned image written to .tiff with GDAL contains only 'ones' and no data.
The cause of the problem might be in data range. For this arrayF it is between 0,0352989 and 1,03559. The script works fine when stretching the array with values 0 - 255.
Here is the function:
def contrastStrecher(Raster1, p1, p2, OutDir, OutName1):
fileNameR1 = Raster1
P1 = p1
P2 =p2
outputPath = OutDir
outputName = OutName1
extension = os.path.splitext(fileNameR1)[1]
raster1 = gdal.Open(fileNameR1, GA_ReadOnly)
colsR1 = raster1.RasterXSize
rowsR1 = raster1.RasterYSize
bandsR1 = raster1.RasterCount
driverR1 = raster1.GetDriver().ShortName
geotransformR1 = raster1.GetGeoTransform()
proj1 = raster1.GetProjection()
bandF = raster1.GetRasterBand(1)
nodataF = bandF.GetNoDataValue()
newnodata = -1.
arrayF = bandF.ReadAsArray().astype("float32")
nodatamaskF = arrayF == nodataF
arrayF[nodatamaskF] = newnodata
p2, p98 = np.percentile(arrayF, (P1, P2))
img_rescale = exposure.rescale_intensity(arrayF, in_range=(p2, p98))
del arrayF
img_rescale[nodatamaskF] = newnodata
driver = gdal.GetDriverByName(driverR1)
outraster = driver.Create(outputPath + outputName + extension, colsR1, rowsR1, 1, gdal.GDT_Float32)
outraster.SetGeoTransform(geotransformR1)
outraster.SetProjection(proj1)
outband = outraster.GetRasterBand(1)
outband.WriteArray(img_rescale)
del img_rescale
outband.FlushCache()
outband.SetNoDataValue(newnodata)
del outraster, outband
I figured out that value of newnodata interferes with the calculation. Previously I assigned a velue of -9999.9 to it and the results were as described above. Now with -1. it seems that the function outputs correct results however I'm not entirely sure of that as the nodata or newnodata value should not be included in calculation.
There is a concept called percentile stretching where everything passed the bottom and top percentile designated will be mapped to 0 and 255 respectively and all pixel values in between will be stretched to improve contrast. I am not sure that is what you want but I believe that is what's done here in this example with downloadable code: https://scikit-image.org/docs/0.9.x/auto_examples/plot_equalize.html
But you many not want for some images any mapping to 0,255 so maybe use argparse() to be able to enter these as parameters or use np.amax and np.amin to designate cut-off points, try writing the images to file and build an algorithm that suits your needs.

Creating reusable objects with Raphael

I'm stuck on a simple problem (I just started using the lib):
What is the most effective way to create sets of objects, replicate them and transform them?
Creating a bunch of circles around a center point:
var height = 150,
width = 150,
branchRadius = 20,
centerX = width / 2,
centerY = height / 2,
treeCrownCenterOffset = 10,
treeCrownRotationCenterX = centerX,
treeCrownRotationCenterY = centerY - treeCrownCenterOffset,
rotationCenter = [treeCrownRotationCenterX , treeCrownRotationCenterY],
paper = Raphael('logo', height , width ),
outerCircle = paper.circle(treeCrownRotationCenterX, treeCrownRotationCenterY, branchRadius).attr({fill:'#F00'}),
innerCircle = paper.circle(treeCrownRotationCenterX, treeCrownRotationCenterY, branchRadius - 4).attr({fill:'#000'}),
branch = paper.set([outerCircle, innerCircle]),
crown = paper.set(),
numCircles = 8, rotationStep = 360 / numCircles, i = 0;
for(; i < numCircles - 1 ; i++) {
//Cloning a branch and pushing it to the crown after transforming it
crown.push(branch.clone().transform('r' + ((i + 1) * rotationStep) + ',' + rotationCenter));
}
//Putting a second crown 200px right of the first crown
//Yes, I'm building a tree :-) No trunk at this point
crown.clone().transform('t200,0');​​​​​
If you like violin, here is the fiddle.
This is my naive code, thinking a set of cloned sets (the crown of cloned branches) will indeed be moved to position (200, 0), next to the first crown.
It doesn't work: looks like a cloned set of cloned elements cannot be moved:
crown.clone().transform('t200,0');​
Not much happens when this line is executed.
Seems like "cloning" is not doing what I expect, and that the transformations are not carried to the second (cloned) collection of objects.
The basic question is:
How does one go about creating reusable objects with Raphael?
Thanks.
You are cloning the set, but since your canvas is only 150px wide, translating it by 200 pixels is sending it off the reservation :)
When you do expand the size of the canvas, however, you will see that only one circle appears to have been cloned. This is not the case. The problem is not with the cloning but the transformation.
I find transformations to be a huge headache. The line "crown.clone().transform('t200,0');​" is applying that transformation to each object in the set, but I believe it is overriding the rotation. Even if it weren't, it would be applying the translation after the rotating, sending the circles scattering as if by centrifugal force.
I know you wanted to avoid looping through the cloned set, but this works:
var crown2 = crown.clone();
for (i = 0; i < crown2.length; i ++) {
crown2[i].transform('t200,0r' + (i * rotationStep) + ',' + rotationCenter);
​}​
Also, note that you didn't add the original branch to the set. You need this:
branch = paper.set([outerCircle, innerCircle]),
crown = paper.set(),
numCircles = 8, rotationStep = 360 / numCircles, i = 0;
//ADD ORIGINAL BRANCH TO SET
crown.push(branch);
Updated fiddle.

How to create a rubber thread in Box2D?

Using Box2d, how to create a rubber thread (rubber band / elastic rope) like Parachute Ninja (ZeptoLab)?
-(void) CreateElasticRope {
//=======Params
// Position and size
b2Vec2 lastPos = b2Vec2(4,4); //set position first body
float widthBody = 0.35;
float heightBody = 0.1;
// Body params
float density = 0.05;
float restitution = 0.5;
float friction = 0.5;
// Distance joint
float dampingRatio = 0.85;
float frequencyHz = 10;
// Rope joint
float kMaxWidth = 1.1;
// Bodies
int countBodyInChain = 10;
b2Body* prevBody;
//========Create bodies and joints
for (int k = 0; k < countBodyInChain; k++) {
b2BodyDef bodyDef;
if(k==0 || k==countBodyInChain-1) bodyDef.type = b2_staticBody; //first and last bodies are static
else bodyDef.type = b2_dynamicBody;
bodyDef.position = lastPos;
lastPos += b2Vec2(2*widthBody, 0); //modify b2Vect for next body
bodyDef.fixedRotation = YES;
b2Body* body = world->CreateBody(&bodyDef);
b2PolygonShape distBodyBox;
distBodyBox.SetAsBox(widthBody, heightBody);
b2FixtureDef fixDef;
fixDef.density = density;
fixDef.restitution = restitution;
fixDef.friction = friction;
fixDef.shape = &distBodyBox;
body->CreateFixture(&fixDef);
if(k>0) {
//Create distance joint
b2DistanceJointDef distJDef;
b2Vec2 anchor1 = prevBody->GetWorldCenter();
b2Vec2 anchor2 = body->GetWorldCenter();
distJDef.Initialize(prevBody, body, anchor1, anchor2);
distJDef.collideConnected = false;
distJDef.dampingRatio = dampingRatio;
distJDef.frequencyHz = frequencyHz;
world->CreateJoint(&distJDef);
//Create rope joint
b2RopeJointDef rDef;
rDef.maxLength = (body->GetPosition() - prevBody->GetPosition()).Length() * kMaxWidth;
rDef.localAnchorA = rDef.localAnchorB = b2Vec2_zero;
rDef.bodyA = prevBody;
rDef.bodyB = body;
world->CreateJoint(&rDef);
} //if k>0
prevBody = body;
} //for -loop
}
I use distance and rope Joints, set different values ​​of parameters dampingRatio and frequencyHz, but the effect is far from being an example (my thread for a long time coming to original state, and not so elastic.).
You can simulate springs by applying forces. At each timestep update the forces on the connected bodies (wake the bodies up if necessary too). If one of the bodies is the ground (or a static body) then you don't need to apply any force to the ground just the dynamic body.
A regular spring would apply both tension and compression forces (pull and push) depending on the deflection. In your case you have a bungee so there would be no compression force just tension (pull).
This is the formula you need:
F = K * x
Where F is the force, K is the spring stiffness (force/deflection), and x is the deflection. Deflection is computed as the difference between the initial length and the current length (the distance between connection points). The sign of the F determines if it is pulling or pushing. Once you compute F then you need to apply it along the line connecting two spring connection points. To satisfy force balance you need to apply this force in opposing directions (one of the bodies gets positive the other one gets negative force). This is because Sir Newton says so.
Here is an example (works with pyBox2D but you can easily convert this to C++)
You need spring objects with some properties. Your spring objects need to know their initial lengths, stiffness, body1, body2, connection coordinates (b1x, b1y, b2x, b2y (in local coordinates))
In your case you need to check if length < spr.initialLength, if this is True then you don't apply any force.
body1 = spr.box2DBody1
body2 = spr.box2DBody2
pA = body1.GetWorldPoint(b2Vec2(spr.box2Db1x, spr.box2Db1y))
pB = body2.GetWorldPoint(b2Vec2(spr.box2Db2x, spr.box2Db2y))
lenVector = pB - pA
length = lenVector.Length()
deltaL = length - spr.initialLength
force = spr.K * deltaL
#normalize the lenVector
if length == 0:
lenVector = b2Vec2(0.70710678118654757, 0.70710678118654757)
else:
lenVector = b2Vec2(lenVector.x / length, lenVector.y / length)
sprForce = b2Vec2(lenVector.x * force, lenVector.y * force)
body1.ApplyForce(sprForce, pA)
body2.ApplyForce(-sprForce, pB)
I very much doubt they are using any joints there. They are probably just taking the distance between the current position of the ninja guy, and the middle of the two posts, to calculate a direction and starting impulse... and just drawing two lines between the posts and the ninja guy.
The best physics implementation added to games I have seen was done by a guy with an engineering degree. He used the calculations you would do in physics / engineering translated into C++. Everything from simple gravity, recoil, thrust, to rotational velocities caused by incidental explosions. All the math was separated into a module that was distinct from the animation.
I would suggest looking up formulas for properties of elastics, and also consider that you have three situations for the elastic band:
1) A shaped force is being applied to stretch it back
2) The shape is now driven by the elastic properties of the band
3) The shape is no longer touching the band, and the band is loosely oscillating by its own weight and inertia
The closer you get to using the true physics calculations, the more realistic it will appear. I'm sure you can fudge it to make it easier on yourself, but humans are inherently good at seeing fakeness.