Kinect Scaling in C++ - c++

I am experimenting with the kinect, however I am having some problems with scaling. The below is code from the kinect-kcb and although the face tracking works for the 'mesh' I am having problems returning the scaling value for my own classes. The below code returns a correct rotation and translation which function perfectly, but the scale only ever returns 1 for a long period (despite the mesh clearly changing size) and then slowly gets smaller 0.98... etc but clearly not correct scaling values.
float scale;
float rotation[ 3 ];
float translation[ 3 ];
hr = mResult->Get3DPose( &scale, rotation, translation );
if ( SUCCEEDED( hr ) ) {
Vec3f r( rotation[ 0 ], rotation[ 1 ], rotation[ 2 ] );
Vec3f t( translation[ 0 ], translation[ 1 ], translation[ 2 ] );
face.mPoseMatrix.translate( t );
face.mPoseMatrix.rotate( r );
face.mPoseMatrix.translate( -t );
face.mPoseMatrix.translate( t );
face.mPoseMatrix.scale( Vec3f::one() * scale );
}
This scale value is used repeatedly thoughout the code, but does not seem to change often enough (example functions - not in order):
hr = mModel->Get3DShape( shapeUnits, numShapeUnits, animationUnits, numAnimationUnits, scale, rotation, translation, pts, numVertices );
hr = mModel->GetProjectedShape( &mConfigColor, mSensorData.ZoomFactor, viewOffset, shapeUnits, numShapeUnits, animationUnits,
numAnimationUnits, scale, rotation, translation, pts, numVertices );
The kinect has a function FaceModel.Scale(), however this only returns a constant value which I assume is the initial scaling value for the 3D model, and then I assumed the above scaling value would change as the user moved closer and further away from the camera.

The method IFTResult::Get3DPose among other things, gives you the face scale value. If it is equal to 1.0 then the face scale is equal to the loaded 3D model (so nothing to do?).
If when reloading the 3d model, the face value is not equal to 1.0 then you need to do work on the model.
have you tried outputting some debug info of what IFTResult::Get3DPose assigns to pScale?
its also possible that the system is failing to track, you can check this with IFTResult::GetStatus.
It may be that what you are after is the magnitude of the face rectangle. This would scale with the proximity of the image subject.
Heres a relevant code project link.

Related

Physically based camera values too small

I am currently working on a physically based camera model and came across this blog: https://placeholderart.wordpress.com/2014/11/21/implementing-a-physically-based-camera-manual-exposure/
So I tried to implement it myself in OpenGL. I thought of calculating the exposure using the function getSaturationBasedExposure and pass that value to a shader where I will multiply the final color with that value:
float getSaturationBasedExposure(float aperture,
float shutterSpeed,
float iso)
{
float l_max = (7800.0f / 65.0f) * Sqr(aperture) / (iso * shutterSpeed);
return 1.0f / l_max;
}
colorOut = color * exposure;
But the values I get from that function are way too small (like around 0.00025 etc) so I guess I am missunderstanding the returned value of that function.
In the blog a test scene is mentioned in which the scene luminance is around 4000, but I haven't seen a shader implementation working with color range from 0 to 4000+ (not even HDR goes that high, right?).
So could anyone explain me how to apply the calculations correctly to a OpenGL scene or help me understand the meaning behind the calculations?

Magick++ trimming pixels that are not of backgroundColor (pixels same color on top and bottom of image)

The following image of size 1x9 is being trimmed to 1x6 because presumably the pixel at the top is the same color as the pixel at the bottom and in the trim function, these pixels are being identified as the background color, even though the backgroundColor being reported before the execution of the trim function is #FFFFFF.
http://s1.postimage.org/a7r69yxsr/m_medium_bc.png
The only thing I am doing is executing trim on the Image. Explicitly setting backgroundColor and/or transparent() makes no difference.
Why is this occurring and is this the expected behavior?
Can this be fixed by configuration/property setting/without changing Graphicsk library code?
If not, when can this bug be fixed? Do you expect a bug of this nature to be fixed in the next few days?
Here is the code:
Magick::Image tempImage;
tempImage.read(name);
std::cout<<"size:"<<tempImage.columns()<<","<<tempImage.rows()<<std::endl;
temp=tempImage.backgroundColor();
std::cout<<"bg:"<<(std::string)temp<<std::endl;
tempImage.trim();
std::cout<<"size:"<<tempImage.columns()<<","<<tempImage.rows()<<std::endl;
I agree that this behaviour is strange, I am not a developer/maintainer of ImageMagick/Magick++ so can't comment further as to whether this is a bug or 'feature'. However I had the same issue and created this function as a workaround (note this is much faster than manually iterating the pixels, even with a pixel cache in place):
Magick::Geometry CalculateImageMagickBoundingBox( const Magick::Image & image, const Magick::Color & borderColor )
{
// Clone input image.
Magick::Image clone( image );
// Remember original image size.
const Magick::Geometry originalSize( image.columns( ), image.rows( ) );
// Extend geometry by two in width and height (one pixel border).
Magick::Geometry extendedSize( originalSize.width( ) + 2, originalSize.height( ) + 2 );
// Extend cloned canvas (center gravity so 1 pixel border of user specified colour).
clone.extent( extendedSize, borderColor, Magick::CenterGravity );
// Calculate bounding box (will use border colour, which we have set above).
Magick::Geometry boundingBox = clone.boundingBox( );
// We added 1 pixel border, so subtract this now.
boundingBox.xOff( boundingBox.xOff( ) - 1 );
boundingBox.yOff( boundingBox.yOff( ) - 1 );
// Clamp (required for cases where entire image is border colour, and therefore the right/top borders
// that we added are taken into account).
boundingBox.width( std::min( boundingBox.width( ), originalSize.width( ) ) );
boundingBox.height( std::min( boundingBox.height( ), originalSize.height( ) ) );
// Return bounding box.
return boundingBox;
}
In your particular case, you could use this function and then set the canvas size based on the geometry returned.

MVP matrix not working outside of shader?

Odd problem here, I've been converting my current project from Qt's native matrix/vector classes to Eigen's, but I've come across an issue that I can't work out.
I calculate the MVP for the shader thus:
DiagonalMatrix< double, 4 > diag( Vector4d( 1.0, 1.0, -1.0, 1.0 ) );
scrMatrix_.noalias() = projMatrix_ * diag * camMatrix_.inverse();
The diag matrix inverts the Z-axis because all my maths sees the camera's aim vector pointing into the screen, but OpenGL does the opposite. Anyway this works because the OpenGL side of the viewports appear and operate fine.
The other side of my viewport output is 2D overlay painting via Qt's paintEvent() system, grid labelling for example. So I use the same matrix to find the 3D location in the camera's clip space:
Vector4d outVec( scrMatrix_ * ( Vector4d() << inVec, 1.0 ).finished() );
Except I get totally wrong results:
inVec: 0 0 10
outVec: 11.9406 -7.20796
In this example I expected something more like outVec: 0.55 -0.15. My GLSL vertex shader performs the calculation like this:
gl_Position = scrMatrix_ * transform * vec4( inVec, 1.0 );
In the examples above transform is the identity, so I can't see any difference between the two projections, and yet the outcomes are totally different! I know this is a long shot, but can anyone see where I'm going wrong?
Update:
I reimplemented the old (working) Qt code for comparison purposes:
QVector3D qvec( vector( 0 ), vector( 1 ), vector( 2 ) );
QMatrix4x4 qmat( Affine3d( scrMatrix_ ).data() );
QPointF pnt = ( qvec * qmat ).toPointF() / 2.0;
Vs:
Vector4d vec( scrMatrix_ * ( Vector4d() << vector, 1.0 ).finished() );
QPointF pnt = QPointF( vec( 0 ), vec( 1 ) ) / 2.0;
To me they are identical, but only the Qt version works!
Well I sussed it out, you need to scale the XYZ axes of the resulting vector by the W axis scale factor (clues in the name...).
It's amazing how much Qt and OpenGL do in the background for you.

OpenCV Kalman filter

I have three gyroscope values, pitch, roll and yaw. I would like to add Kalman filter to get more accurate values. I found the opencv library, which implements a Kalman filter, but I can't understand it how is it really work.
Could you give me any help which can help me? I didn't find any related topics on the internet.
I tried to make it work for one axis.
const float A[] = { 1, 1, 0, 1 };
CvKalman* kalman;
CvMat* state = NULL;
CvMat* measurement;
void kalman_filter(float FoE_x, float prev_x)
{
const CvMat* prediction = cvKalmanPredict( kalman, 0 );
printf("KALMAN: %f %f %f\n" , prev_x, prediction->data.fl[0] , prediction->data.fl[1] );
measurement->data.fl[0] = FoE_x;
cvKalmanCorrect( kalman, measurement);
}
in main
kalman = cvCreateKalman( 2, 1, 0 );
state = cvCreateMat( 2, 1, CV_32FC1 );
measurement = cvCreateMat( 1, 1, CV_32FC1 );
cvSetIdentity( kalman->measurement_matrix,cvRealScalar(1) );
memcpy( kalman->transition_matrix->data.fl, A, sizeof(A));
cvSetIdentity( kalman->process_noise_cov, cvRealScalar(2.0) );
cvSetIdentity(kalman->measurement_noise_cov, cvRealScalar(3.0));
cvSetIdentity( kalman->error_cov_post, cvRealScalar(1222));
kalman->state_post->data.fl[0] = 0;
And I call this everytime, when I receive data from gyro:
kalman_filter(prevr, mpe->getGyrosDegrees().roll);
I thought in kalman_filter the first parameter is the previous value and the second is the currect value. I'm not and this code doesn't work... I know I have a lot of work with it, but I don't know how to continue, what to change...
It seems like you are giving too high values to the covariance matrices.
kalman->process_noise_cov is the 'process noise covariance matrix' and it is often referred in the Kalman literature as Q. The result will be smoother with lower values.
kalman->measurement_noise_cov is the 'measurement noise covariance matrix' and it is often referred in the Kalman literature as R. The result will be smoother with higher values.
The relation between those two matrices defines the amount and shape of filtering you are performing.
If the value of Q is high, it will mean that the signal you are measuring varies quickly and you need the filter to be adaptable. If it is small, then big variations will be attributed to noise in the measure.
If the value of R is high (compared to Q), it will indicate that the measuring is noisy so it will be filtered more.
Try lower values like q = 1e-5 and r = 1e-1 instead of q = 2.0 and r = 3.0.

3d model to fit in viewport

How does a 3D model handled unit wise ?
When i have a random model that i want to fit in my view port i dunno if it is too big or not, if i need to translate it to be in the middle...
I think a 3d object might have it's own origine.
You need to find a bounding volume, a shape that encloses all the object's vertices, for your object that is easier to work with than the object itself. Spheres are often used for this. Either the artist can define the sphere as part of the model information or you can work it out at run time. Calculating the optimal sphere is very hard, but you can get a good approximation using the following:
determine the min and max value of each point's x, y and z
for each vertex
min_x = min (min_x, vertex.x)
max_x = max (max_x, vertex.x)
min_y = min (min_y, vertex.y)
max_y = max (max_y, vertex.y)
min_z = min (min_z, vertex.z)
max_z = max (max_z, vertex.z)
sphere centre = (max_x + min_x) / 2, (max_y + min_y) / 2, (max_z + min_z) / 2
sphere radius = distance from centre to (max_x, max_y, max_z)
Using this sphere, determine the a world position that allows the sphere to be viewed in full - simple geometry will determine this.
Sorry, your question is very unclear. I suppose you want to center a 3D model to a viewport. You can achieve this by calculating the model's bounding box. To do this, traverse all polygons and get the minimum/maximum X/Y/Z coordinates. The bounding box given by the points (min_x,min_y,min_z) and (max_x,max_y,max_z) will contain the whole model. Now you can center the model by looking at the center of this box. With some further calculations (depending on your FOV) you can also get the left/right/upper/lower borders inside your viewport.
"so i tried to scale it down"
The best thing to do in this situation is not to transform your model at all! Leave it be. What you want to change is your camera.
First calculate the bounding box of your model somewhere in 3D space.
Next calculate the radius of it by taking the max( aabb.max.x-aabb.min.x, aabb.max.y-aabb.min.y, aabb.max.z-aabb.min.z ). It's crude but it gets the job done.
To center the object in the viewport place the camera at the object position. If Y is your forward axis subtract the radius from Y. If Z is the forward axis then subtract radius from it instead. Subtract a fudge factor to get you past the pesky near plane so your model doesn't clip out. I use quaternions in my engine with a nice lookat() method. So call lookat() and pass in the center of the bounding box. Voila! You're object is centered in the viewport regardless of where it is in the world.
This always places the camera axis aligned so you might want to get fancy and transform the camera into model space instead, subtract off the radius, then lookat() the center again. Then you're always looking at the back of the model. The key is always the lookat().
Here's some example code from my engine. It checks to see if we're trying to frame a chunk of static terrain, if so look down from a height, or a light or a static mesh. A visual is anything that draws in the scene and there are dozens of different types. A Visual::Instance is a copy of the visual, or where to draw it.
void EnvironmentView::frameSelected(){
if( m_tSelection.toInstance() ){
Visual::Instance& I = m_tSelection.toInstance().cast();
Visual* pVisual = I.toVisual();
if( pVisual->isa( StaticTerrain::classid )){
toEditorCamera().toL2W().setPosition( pt3( 0, 0, 50000 ));
toEditorCamera().lookat( pt3( 0 ));
}else if( I.toFlags()->bIsLight ){
Visual::LightInstance& L = static_cast<Visual::LightInstance&>( I );
qst3& L2W = L.toL2W();
const sphere s( L2W.toPosition(), L2W.toScale() );
const f32 y =-(s.toCenter()+s.toRadius()).y();
const f32 z = (s.toCenter()+s.toRadius()).y();
qst3& camL2W = toEditorCamera().toL2W();
camL2W.setPosition(s.toCenter()+pt3( 0, y, z ));//45 deg above
toEditorCamera().lookat( s.toCenter() );
}else{
Mesh::handle hMesh = pVisual->getMesh();
if( hMesh ){
qst3& L2W = m_tSelection.toInstance()->toL2W();
vec4x4 M;
L2W.getMatrix( M );
aabb3 b0 = hMesh->toBounds();
b0.min = M * b0.min;
b0.max = M * b0.max;
aabb3 b1;
b1 += b0.min;
b1 += b0.max;
const sphere s( b1.toSphere() );
const f32 y =-(s.toCenter()+s.toRadius()*2.5f).y();
const f32 z = (s.toCenter()+s.toRadius()*2.5f).y();
qst3& camL2W = toEditorCamera().toL2W();
camL2W.setPosition( L2W.toPosition()+pt3( 0, y, z ));//45 deg above
toEditorCamera().lookat( b1.toOrigin() );
}
}
}
}