I am trying to get animations to work for complex glTF models I am trying the RiggedFigure.
In particular I am trying to calculate the globalTransformOfJointNode as described here: https://github.com/KhronosGroup/glTF-Tutorials/blob/master/gltfTutorial/gltfTutorial_020_Skins.md
For that I have this function:
std::vector<Eigen::Matrix4f> GetGlobalJointMatrices(
const std::vector<Eigen::Matrix4f>& local_matrices,
const std::vector<Eigen::Matrix4f>& animation_matrices,
const std::map<int, int>& skeleton,
const int start_joint)
{
std::vector<Eigen::Matrix4f> skeleton_matrices(skeleton.size());
skeleton_matrices[0] = Eigen::Matrix4f::Identity();
for(auto [child, parent]: skeleton)
{
skeleton_matrices[child - start_joint] = skeleton_matrices[parent - start_joint] *
local_matrices[child - start_joint] * animation_matrices[child - start_joint];
}
return skeleton_matrices;
}
In here local_matrices refers to the local transform of each node in the skeleton as described in the nodes array.
animation_matrices are only the animations that target nodes in the skin.
For both arrays, since the nodes are contiguous, the index i contains the value corresponding to node i + start_joint. I have verified that the animations are loaded correctly and in the correct order.
This is the result of doing:
skeleton_matrices[child - start_joint] = skeleton_matrices[parent - start_joint] * local_matrices[child - start_joint];
If I try to apply the animation matrices Instead I get:
Putting the animation matrices at other points results in similar behaviour.
I am sure I am not loading the martices incorrectly, for example these are the matrices that target nodes 6 and 10:
animation targetting node:10
rotation: 0.0245355 -0.319997 0.9446 0.0687827
translation: -0.00234646 -0.0661734 0.0278567
scales: 1 1 1
node 6:
animation targetting node:6
rotation: -0.0341418 -0.319178 0.946171 -0.0414678
translation: -0.00145852 -0.0661988 0.0278567
I went and verified that those values are correct manually, they are.
I don't understand what is wrong.
I feel the spec should be more clear about this. So as it turns out you should NOT apply the transform of thr nodes and only calculate the animation matrix, the animation matrices already contain the node matrix premultiplied.
TL;DR:
Do this:
skeleton_matrices[child - start_joint] = skeleton_matrices[parent - start_joint] *
animation_matrices[child - start_joint];
Related
I'm trying to use mrpt slam algorithm. I would like to adapt the original "icp slam app" to use lidar scans from my simulation. If I understand correctly I should use the CObservation2DRangeScan class to contain the lidar observations.
My problem is that I cannot find how to set the scan angle. I presume that the scan has to be in polar coordinates, then if setScanRange sets the range in meters, how do I set the angle?
I cannot find a proper member function within the class, I am probably missing something.
A code sample so far:
mrpt::obs::CObservation2DRangeScan::Ptr observation(new mrpt::obs::CObservation2DRangeScan);
observation->resizeScan(i32NUM_POINTS);
for(int32_t i = 0; i < i32NUM_POINTS; ++i)
{
observation->setScanRange(i, arrPoints[i].range);
//here I must set the scan angle
observation->setScanRangeValidity(i, true);
}
mrpt version: 2.2.1
Thank you in advance
Massimo
The angle is implicitly defined by the index of each range within the vector.
I just edited the class docs to better explain this.
Note that this code describes the exact relationship between indices and angles:
float Ang = -0.5f * aperture;
float dA = aperture / (m_scan.size() - 1);
if (!rightToLeft)
{
Ang = -Ang;
dA = -dA;
}
return Ang + dA * idx;
Also: Note that the program GridmapNavSimul already allows you to draw a gridmap world, drive a robot and generate simulated datasets without coding a single line... ;-)
I am in the process of writing an animation system with my own Collada parser and am running into an issue that I can't wrap my head around.
I have collected my mesh/skin information (vertices, normals, jointIds, weights, etc), my skeleton information (joints, their local transforms, inverse bind position, hierarchy structure), and my animation (keyframe transform position for each joint, timestamp).
My issue is that with everything calculated and then implemented in the shader (the summation of weights multiplied by the joint transform and vertex position) - I get the following:
When I remove the weight multiplication, the mesh remains fully intact - however the skin doesn't actually follow the animation. I am at a lost as I feel as though the math is correct, but very obviously I am going wrong somewhere. Would someone be able to shine light on the aspect I have misinterpreted?
Here is my current understanding and implementation:
After collecting all of the joint's localTransforms and hierarchy, I calculate their inverse bind transfromation matrix. To do this I multiple each joints localTransform with their parentLocalTransform to get a bindTransform. Inverting that bindTransform results in their inverseBindTransform. Below is my code for that:
// Recursively collect each Joints InverseBindTransform -
// root joint's local position is an identity matrix.
// Function is only called once after data collection.
void Joint::CalcInverseBindTransform(glm::mat4 parentLocalPosition)
{
glm::mat4 bindTransform = parentLocalPosition * m_LocalTransform;
m_InverseBindPoseMatrix = glm::inverse(bindTransform);
for (Joint child : Children) {
child.CalcInverseBindTransform(bindTransform);
}
}
Within my animator during an animation, for each joint I take the two JointTransforms for the two frame's my currentTime is in between and I calculate the interpolated JointTransform. (JointTransform simply has a vec3 for position and quaternion for rotation). I do this for every joint and then apply those interpolated values to each Joint by again recursively muliplying the new frameLocalTransform by their parentLocalTransform. I take that bindTransform and multiply it by the invBindTransform and then transpose the matrix. Below is the code for that:
std::unordered_map<int, glm::mat4> Animator::InterpolatePoses(float time) {
std::unordered_map<int, glm::mat4> poses;
if (IsPlaying()) {
for (std::pair<int, JointTransform> keyframe : m_PreviousFrame.GetJointKeyFrames()) {
JointTransform previousFrame = m_PreviousFrame.GetJointKeyFrames()[keyframe.first];
JointTransform nextFrame = m_NextFrame.GetJointKeyFrames()[keyframe.first];
JointTransform interpolated = JointTransform::Interpolate(previousFrame, nextFrame, time);
poses[keyframe.first] = interpolated.getLocalTransform();
}
}
return poses;
}
void Animator::ApplyPosesToJoints(std::unordered_map<int, glm::mat4> newPose, Joint* j, glm::mat4 parentTransform)
{
if (IsPlaying()) {
glm::mat4 currentPose = newPose[j->GetJointId()];
glm::mat4 modelSpaceJoint = parentTransform * currentPose;
for (Joint child : j->GetChildren()) {
ApplyPosesToJoints(newPose, &child, modelSpaceJoint);
}
modelSpaceJoint = glm::transpose(j->GetInvBindPosition() * modelSpaceJoint);
j->SetAnimationTransform(modelSpaceJoint);
}
}
I then collect all the newly AnimatedTransforms for each joint and send them to the shader:
void AnimationModel::Render(bool& pass)
{
[...]
std::vector<glm::mat4> transforms = GetJointTransforms();
for (int i = 0; i < transforms.size(); ++i) {
m_Shader->SetMat4f(transforms[i], ("JointTransforms[" + std::to_string(i) + "]").c_str());
}
[...]
}
void AnimationModel::AddJointsToArray(Joint current, std::vector<glm::mat4>& matrix)
{
glm::mat4 finalMatrix = current.GetAnimatedTransform();
matrix.push_back(finalMatrix);
for (Joint child : current.GetChildren()) {
AddJointsToArray(child, matrix);
}
}
In the shader, I simply follow the summation formula that can be found all over the web when researchiing this topic:
for (int i = 0; i < total_weight_amnt; ++i) {
mat4 jointTransform = JointTransforms[jointIds[i]];
vec4 newVertexPos = jointTransform * vec4(pos, 1.0);
total_pos += newVertexPos * weights[i];
[...]
---------- Reply to Normalizing Weights ------------
There were a few weights summing above 1, but after solving the error in my code the model looked like this:
For calculating the weights - I loop through all preadded weights in the vector, and if I find a weight that is less than the weight I'm looking to add - I replace that weight in that position. Otherwise, I append the weight onto the end of the vector. If there are less weights in my vector than my specified max_weights (which is 4) - I fill in the remaining weights/jointIds with 0.
I understand when something is going wrong in skinning animations, there can be alot of different areas the problem is occuring. As such, for future googlers experiencing the same issue I am - take this as more of a list of suggestions of what you could be doing wrong rather than absolutely doing wrong.
For my problem - I had the right idea but wrong approach in a lot of minor areas. Which brought me fairly close but, as they say, no cigar.
I had no need to calculate the Inverse Bind Pose myself, Collada's Inverse Bind Pose (sometimes/often declared as an "offsetMatrix") is more than perfect. This wasn't a problem more as I was just doing unnecessary calculations.
In a Collada file, they often provide you more "joints" or "nodes" in the hierarchy than what is needed for the animation. Prior to the start of your actual animated "joints", there is the scene and an initial armature "node" type. The scene is typically an identity matrix that was manipulated based on your "up axis" upon reading in the Collada file. The Node type will determine the overall size of each joint in the skeleton - so if it wasn't resized, its probably the identity matrix. Make sure your hierarchy still contains ALL nodes/joints listed in the hierarchy. I very much was not doing so - which greatly distorted my globalPosition (BindPose).
If you are representing your Joint's transforms rotation through quaternions (which is highly recommended), make sure the resulted quaternion is normalized after interpolating between two rotated positions.
On the same note - when combining the Rotation and Transform into your final matrix - make sure your order of multiplication and the final output is correct.
Finally - your last skinning matrix is comprised of your joints InvBindMatrix * GlobalPosition * GlobalInverseRootTransform (<- this is the inverse of the local transfrom from your "scene" node mentioned in (1), remember?).
Based on your prior matrix multiplications up to this point, you may or may not need to transpose this final matrix.
And with that - I was able to successfully animate my model!
One final note - my mesh and animation files are added in separately. If your animations are in separate files from your mesh, make sure you collect the skinning/joint information from the files with an animation rather than the file with the mesh. I list my steps for loading in a model and then giving it multiple animations through different files:
Load in the Mesh (This contains Vertices,Normals,TexCoords,JointIds,Weights)
Load in the animation file (This gives Skeleton, InverseBindPositions, and other needed info to bind skeleton to mesh) - Once skeleton and binding info is collected, gather first animation info from that file as well.
For another animation, the above Skeleton should work fine for any other animation on the same mesh/model - just read in the animation information and store in your chosen data structure. Repeat step 3 til happy.
I have a simple model with a simple skeletal structure that I made in blender. Here's how it looks:
And here's the hierarchy in blender:
As you can see it has two bones: One that goes halfway up the rectangular box ("Bone"), that is completely stationary. And another bone ("Bone.001") that goes from the halfway point and up to the top, that rotates.
I've imported the mesh using AssImpNet, and extracted the rotation, scaling and position keys from the animation node channels. When I apply those transformations to the mesh, I get this result (colored by bone weight):
The motion/animation seems to play correctly, so I believe this part works correctly. Now this is where my understanding starts to break down, but I believe the crucial part I'm missing is calculating the "inverse bind pose" (I've seen a few names for it), and applying that to the bone transformations that I feed into my shader as well. But so far I haven't been able to find exactly what it is that I need to extract from AssImp's format, and multiply together to get the correct final transformation. I've only found vague explanations about traversing the node tree and "undoing" transformations from each parent node, or something.
Here's what I tried, which doesn't seem to be working:
i thought that, for the base bone ("Bone"), i would need:
- the global inverse transform
- the node transform of the node with name "Bone"
- the rotation/position/scale keys from the animation channel with name "Bone"
- the bone offset from Meshes.Bones.OffsetMatrix from the bone named "Bone"
and then multiply them together in that order.
similarly for "Bone.001":
- the global inverse transform
- the node transform of the node with name "Bone"
- the rotation/position/scale keys from the animation channel with name "Bone"
- the node transform of the node with name "Bone.001"
- the rotation/position/scale keys from the animation channel with name "Bone.001"
- the bone offset from Meshes.Bones.OffsetMatrix from the bone named "Bone.001"
My attempt at implementing this (hard coding the indexes for now just to try to get things working), note that it's using C#/AssImpNet, so the naming conventions are a bit different from C++/AssImp:
// "Bone"
public Matrix4 Bone0Transform(double time)
{
var bone0 = MathUtils.ConvertMatrix(Scene.RootNode.Children[1].Children[0].Transform);
var frame0 = GetTransformedFrame(1, TimeToFrame(1, time));
var global = MathUtils.ConvertMatrix(Scene.RootNode.Transform).Inverted();
var offset0 = MathUtils.ConvertMatrix(Scene.Meshes[0].Bones[0].OffsetMatrix);
var total = global * bone0 * frame0 * offset0;
return total;
}
// "Bone.001"
public Matrix4 Bone1Transform(double time)
{
var bone0 = MathUtils.ConvertMatrix(Scene.RootNode.Children[1].Children[0].Transform);
var bone1 = MathUtils.ConvertMatrix(Scene.RootNode.Children[1].Children[0].Children[0].Transform);
var frame0 = GetTransformedFrame(1, TimeToFrame(1, time));
var frame1 = GetTransformedFrame(2, TimeToFrame(2, time));
var global = MathUtils.ConvertMatrix(Scene.RootNode.Transform).Inverted();
var offset1 = MathUtils.ConvertMatrix(Scene.Meshes[0].Bones[1].OffsetMatrix);
var total = global * bone0 * frame0 * bone1 * frame1 * offset1;
return total;
}
GetTransformedFrame returns a Matrix4 combining the scale, rotation and position keys for the frame that corresponds to the current time, and on its own gives the result you can see in the gif where the box is colored red/green.
All this gives me is an obviously incorrect result:
So my question is this: Is my understanding of how to calculate the final bone transformations wrong? If so, what is the correct way of doing it?
I'm currently working on a hobby project in which I have several thousand stars in a 2D fictional universe. I need to render these stars to the screen, but clearly I don't want to have to operate on all of them -- only the ones that are visible at any given time.
For proof of concept, I wrote a brute force algorithm that would look at every star and test its coordinates against the bounds of the player's screen:
for (const std::shared_ptr<Star>& star : stars_) {
if (moved_)
star->MoveStar(starfield_offset_, level_);
position = star->position();
if (position.x >= bounds_[0] &&
position.x <= bounds_[1] &&
position.y >= bounds_[2] &&
position.y <= bounds_[3])
target.draw(*star);
}
While this clunky method does, indeed, draw only the visible stars to the screen, it clearly operates in linear time. Since stars are only part of the background and, frankly, aren't the most important thing for the processor to be spending time filtering through, I'd like to devise a faster algorithm to reduce some of the load.
So, my current train of thought is along the lines of using binary search to find the relevant stars. For this, I would clearly need to sort my data. However, I wasn't really sure how I could go about sorting my coordinate data -- I couldn't think of any absolute ordering that would allow me to properly sort my data in ascending order (with regards to both x and y coordinates).
So, I implemented two new containers -- one for the data sorted by x coordinate, and the other by y coordinate. My original thought was to take the intersection of these two sorted sets and draw the resulting stars to screen (stars whose x and y coordinates lie within the screen bounds):
struct SortedStars {
std::vector<std::shared_ptr<Star>>::iterator begin, end;
std::vector<std::shared_ptr<Star>> stars;
} stars_x_, stars_y_;
I then sorted these containers:
// comparison objects
static struct SortX {
bool operator() (const std::shared_ptr<Star>& first, const std::shared_ptr<Star>& second)
{ return (first->position().x < second->position().x); }
bool operator() (const std::shared_ptr<Star>& first, const float val)
{ return (first->position().x < val); }
bool operator() (const float val, const std::shared_ptr<Star>& second)
{ return (val < second->position().x); }
} sort_x;
static struct SortY {
bool operator() (const std::shared_ptr<Star>& first, const std::shared_ptr<Star>& second)
{ return (first->position().y < second->position().y); }
bool operator() (const std::shared_ptr<Star>& first, const float val)
{ return (first->position().y < val); }
bool operator() (const float val, const std::shared_ptr<Star>& second)
{ return (val < second->position().y); }
} sort_y;
void Starfield::Sort() {
// clone original data (shared pointers)
stars_x_.stars = stars_;
stars_y_.stars = stars_;
// sort as needed
std::sort(stars_x_.stars.begin(), stars_x_.stars.end(), sort_x);
std::sort(stars_y_.stars.begin(), stars_y_.stars.end(), sort_y);
// set iterators to the outermost visible stars (defined by screen bounds)
// these are updated every time the screen is moved
stars_x_.begin = std::lower_bound(stars_x_.stars.begin(), stars_x_.stars.end(), bounds_[0], sort_x);
stars_x_.end = std::upper_bound(stars_x_.stars.begin(), stars_x_.stars.end(), bounds_[1], sort_x);
stars_y_.begin = std::lower_bound(stars_y_.stars.begin(), stars_y_.stars.end(), bounds_[2], sort_y);
stars_y_.end = std::upper_bound(stars_y_.stars.begin(), stars_y_.stars.end(), bounds_[3], sort_y);
return;
}
Unfortunately, I cannot seem to either come up with an appropriate comparison function for std::set_intersection or a method through which I could manually compare coordinates using my iterators.
Could you guys point me in the right direction? Feedback on my methodology or implementation is very welcome.
Thanks for your time!
There are a variety of spatial acceleration data structures that help to answer questions of 'what points are in this region'. Quadtrees are a popular solution for 2D but may be overkill for your problem. Probably the simplest approach is to have a 2D grid with points (stars) bucketed by the grid square they fall into. You then check to see which grid squares your view window overlaps and only need to look at the stars in the buckets for those squares. If you make your grid squares a bit larger than your view window size you'll only ever have to check a maximum of four buckets.
If you can zoom in and out a more complicated structure like a Quadtree might be appropriate.
I use real star data for rendering (psychosomatic style) for years and have no speed problems without any visibility ordering/selecting under OpenGL (VBO)
I usually used BSC star catalog in the past
stars up to +6.5mag
9110 stars
few years back I convert my engines to hipparcos catalog
118322 stars
3D coordinates
So unless you use too much stars it should be faster to just render them all
- How many stars are you rendering?
- How are you stars rendered? (I use blended Quad per star)
What platform/setup ...
- this worked well even on my old setup GeForce 4000 Ti, 1.3GHz single core AMD
- also in stereo 3D
what is your desired FPS ? ... I am fine with 30fps for my simulations
If you have similar values and low speed may be there is something wrong with your rendering code (not with the amount of data)...
PS.
if you have a big space to cover you can select bright stars to viewer only
after each hyperspace jump or what ever
based on relative magnitude and distance
also you use too much ifs for star selection
they are sometimes slower then the rendering
try just dot product of viewing direction and star direction vectors instead
and test the sign only (do not see what is behind)
of course if you use quads then CULL_FACE make it for you
Also i see you are calling draw for each star
that is heap trashing
try to avoid calling functions when you can
it will boost the speed a lot !!!
for example you can add a flag to each star if it should be rendered or not
and then render them with single for and no sub-calls to render function
You can try spatial R-tree which is now part of Boost Geometry library.
The application could work as follows:
You add your star's coordinate to the tree in some "absolute" coordinate system. If your stars have different sizes you probably want to add not a point but a bounding box of each star.
#include <boost/geometry/index/rtree.hpp>
#include <boost/geometry/geometries/box.hpp>
namespace bg = boost::geometry;
namespace bgi = boost::geometry::index;
typedef bg::model::point<float, 2, bg::cs::cartesian> point;
typedef bg::model::box<point> box;
typedef std::pair<box, Star*> value; //here "second" can optionally give the star index in star's storage
bgi::rtree<value> rtree;
As you build your universe, you populate the rtree:
for (auto star: stars)
{
box b(star->position(), star->position()));
bg::expand(b, point(star->radius(), star->radius());
// insert new value
rtree.insert(std::make_pair(b, star));
}
When you need to render them, you compute your screen window into "absolute" coord system and query the tree about stars which overlap your window:
box query_box(point(0, 0), point(5, 5));
std::vector<value> result_s;
rtree.query(bgi::intersects(query_box), std::back_inserter(result_s));
Here result_s will list the relevant stars and their bounding boxes.
Good luck!
I have a strange issue where my normals just do not work when I render terrain. My terrain renders just fine, so I left out all the code for calculating the terrain points from a height map and how I calculated the indices. I know I should be using shaders, but I want to get this fixed first before I move on. I am assuming the issue comes from something obvious I have overlooked in my normals generation code, which is as follows:
for (currentind = 0; currentind < indices.size() - 3; currentind+=3)
{
indtopt = indices[currentind] * 3;
point1.vects[0]=terrainpoints[indtopt];//x
point1.vects[1]=terrainpoints[indtopt+1];//y
point1.vects[2]=terrainpoints[indtopt+2];//z
indtopt = indices[currentind+1] * 3;
//second indice
//points of that indice
point2.vects[0]=terrainpoints[indtopt];//x
point2.vects[1]=terrainpoints[indtopt+1];//y
point2.vects[2]=terrainpoints[indtopt+2];//z
indtopt = indices[currentind+2] *3;
//third indice
//points of that indice
point3.vects[0]=terrainpoints[indtopt];//x
point3.vects[1]=terrainpoints[indtopt+1];//y
point3.vects[2]=terrainpoints[indtopt+2];//z
//--------------------------------------------------
point4.vects[0]=(point2.vects[0]-point1.vects[0]);
point4.vects[1]=(point2.vects[1]-point1.vects[1]);
point4.vects[2]=(point2.vects[2]-point1.vects[2]);
point5.vects[0]=(point3.vects[0]-point2.vects[0]);
point5.vects[1]=(point3.vects[1]-point2.vects[1]);
point5.vects[2]=(point3.vects[2]-point2.vects[2]);
//-------------------------------------------------
//cross product
point6.vects[0]=point4.vects[1]*point5.vects[2] - point4.vects[2]*point5.vects[1];
point6.vects[1]=point4.vects[2]*point5.vects[0] - point4.vects[0]*point5.vects[2];
point6.vects[2]=point4.vects[0]*point5.vects[1] - point4.vects[1]*point5.vects[0];
point6 = point6.normalize();
ternormals[currentind]=point6.vects[0];
ternormals[currentind+1]=point6.vects[1];
ternormals[currentind+2]=point6.vects[2];
}
Below is a picture of what the issue is in both wireframe and triangle renders:
I can post more code if need be, but I just wanted to keep this post short, so I tried to find where I thought the issue might be.
Well, for every "dark" band you're accidently flipping the normal, probably because the surface tangent vectors are passed into the cross product in the wrong order
a × b = - (b × a)
If your terrain is made of triangle strips, then you've got a bidirectional ordering, which means, that you have to either flip the operands or negate the result of the cross product for every odd row.