Kalman filter - Null predicted point(s) - c++

I am trying to apply a Kalman Filter in C++ with OpenCV in order to filter some tracks. The first step to make it work for me was to predict the points with the filter from a vector of Points2f.
My code is the following one :
cv::KalmanFilter kalmanFilter(4,2,0, CV_32F);
kalmanFilter.transitionMatrix = transitionMat;
for(int i = 0 ; i < oldTrackeables.size() ; i++)
for(int j = 0 ; j < oldTrackeables[i].getTrack().size() ; j++)
{
cv::Size msmtSize(2,1);
cv::Mat measurementMat(msmtSize, CV_32F);
measurementMat.setTo(cv::Scalar(0));
measurementMat.at<float>(0) = oldTrackeables[i].getTrack()[j].x;
measurementMat.at<float>(1) = oldTrackeables[i].getTrack()[j].y;
//Initialisation of the Kalman filter
kalmanFilter.statePre.at<float>(0) = (float) oldTrackeables[i].getTrack()[j].x;
kalmanFilter.statePre.at<float>(1) = (float) oldTrackeables[i].getTrack()[j].y;
kalmanFilter.statePre.at<float>(2) = (float) 2;
kalmanFilter.statePre.at<float>(3) = (float) 3;
cv::setIdentity(kalmanFilter.measurementMatrix);
cv::setIdentity(kalmanFilter.processNoiseCov, cv::Scalar::all(1e-4));
cv::setIdentity(kalmanFilter.measurementNoiseCov, cv::Scalar::all(.1));
cv::setIdentity(kalmanFilter.errorCovPost, cv::Scalar::all(.1));
//Prediction
cv::Mat prediction = kalmanFilter.predict();
kalmanFilter.statePre.copyTo(kalmanFilter.statePost);
kalmanFilter.errorCovPre.copyTo(kalmanFilter.errorCovPost);
cv::Point predictPt(prediction.at<float>(0), prediction.at<float>(1));
cv::Point Mc = oldTrackeables[i].getMassCenter();
cv::circle(kalmat, predictPt, 16, cv::Scalar(0,255,0), 3, 2, 1);
std::cout<<"prediction : x = " << predictPt.x << " - y = " << predictPt.y <<std::endl;
std::cout<<"position captée : x = " << oldTrackeables[i].getTrack()[j].x << " - y = " << oldTrackeables[i].getTrack()[j].y << std::endl;
std::cout<<"size of frame : rows = " << frame.rows << " - width = " << frame.cols <<std::endl;
std::cout<<"size of kalmat : rows = " << kalmat.rows << " - width = " << kalmat.cols <<std::endl;
cv::imshow("kalmat", kalmat);
Where oldTrackeables[i].getTrack()[j] are just some Points2f from a vector.
The tracking is correct, but the Kalman filter does not give "correct" values for the prediction - For example, the program displays :
prediction : x = 0 - y = 0 -
position captée : x = 138.29 - y = 161.078 (position of the original point).
I've really been looking a lot for answers and trying many different ways to do it but I can't find anything that really helps me... The closer one I found was this one : http://answers.opencv.org/question/24865/why-kalman-filter-keeps-returning-the-same-prediction/ But it did not help me solve my problem...
If any of you has an element of answer of could help me understand the problem, I'd be very grateful.
Thank you.

First of all I would have moved all the init stuff outside the loop otherwise you will override the internal states in the filter. Also change the statePre to statPost
//Initialisation of the Kalman filter
kalmanFilter.statePost.at<float>(0) = (float) 0;
kalmanFilter.statePost.at<float>(1) = (float) 0;
kalmanFilter.statePost.at<float>(2) = (float) 2;
kalmanFilter.statePost.at<float>(3) = (float) 3;
cv::setIdentity(kalmanFilter.measurementMatrix);
cv::setIdentity(kalmanFilter.processNoiseCov, cv::Scalar::all(1e-4));
cv::setIdentity(kalmanFilter.measurementNoiseCov,cv::Scalar::all(.1));
cv::setIdentity(kalmanFilter.errorCovPost, cv::Scalar::all(.1));
The part:
kalmanFilter.statePre.copyTo(kalmanFilter.statePost);
kalmanFilter.errorCovPre.copyTo(kalmanFilter.errorCovPost);
should be removed since this is done internally in the predict phase.
Finally as #Mozfox says, the correct phase is not present in the loop code you provided. Add:
kalmanFilter.predict(measurementMat);

I think you are missing correction phase for measurement computation.

Related

find overlap and downsample scatter plot

I have huge data that consist of points(x,y) and need to present in a scatter plot.
To find overlap and remove invisible differences between the same point, I wrote bellow code :
void DownSampler::makeDownSample(QVector<double> keys,QVector<double> values, QVector<int> pixmapIdx, QSize resolution, sViewSize view)
{
m_calculating = true;
QTime now = QTime::currentTime();
QMap<QPair<double, double>,int> downsampledMap;
QSizeF stepValue( ( view.maxHorizontal - view.minHorizontal ) * m_downSampleRatio / (resolution.width() ),
( view.maxVertical - view.minVertical ) * m_downSampleRatio/ (resolution.height() ) ) ;
for(int index = 0 ; index < keys.size() ; index++)
{
keys[index] = round((keys[index]) / stepValue.width());
keys[index] *= stepValue.width();
values[index] = round(values[index] / stepValue.height());
values[index] *= stepValue.height();
//remove same items
if((keys[index] >= view.minHorizontal) && (keys[index] <= view.maxHorizontal)) {
if((values[index] >= view.minVertical) && (values[index] <= view.maxVertical)) {
QPair<double, double> pairValue = qMakePair(keys[index], values[index]);
if(!downsampledMap.contains(pairValue)){
downsampledMap.insert(pairValue, pixmapIdx[index]);
}
}
}
}
QVector<int> retPixmapIdx;
QVector<double> retKey, retValue;
for(QMap<QPair<double, double>,int>::iterator iter = downsampledMap.begin() ; iter != downsampledMap.end() ; ++iter)
{
retKey.append(iter.key().first);
retValue.append(iter.key().second);
retPixmapIdx.append(iter.value());
}
emit downSampledPlotReady(retKey, retValue, retPixmapIdx);
m_calculating = false;
// qDebug() << __FUNCTION__ << "firstPointSize ==> "<< keys.size() << "downsampledSize ==> " << retKey.size() << "time ==> " << now.msecsTo(QTime::currentTime());
}
as you can see, at first, I rounded all positions to a pixel size step that expects to present data and after all iterate to all created boxes and return the created box list.
according to this code, every box can contain many points, we return just a single position and therefore overlap point was removed.
it's my handwriting code but I think it hasn't the best-expected performance.
my question is :
1- is there any library or algorithm that collects all scatter data like our dependency?
2- is an improvement in our source code?
the point cloud library and Grid Voxel in 2D aspects can process and downsample point clouds perfectly.

Does Opencv 3 SVM trainAuto scale labels too?

I'm using OpenCV 3.0.0. When running OpenCV's SVM example 1 I noticed that when auto trained instead of trained, the predicted values are between 0 and 1. I don't see the same behavior when I run the same example directly with libsvm.
Is this a bug or an intentional scaling of the labels along with the other features? The behavior also seems to be undocumented.
Here is the code I'm running:
// Set up training data
size_t numberOfSamples = 4;
cv::Mat1i labelsMat(numberOfSamples, 1);
labelsMat(0, 0) = 1;
labelsMat(1, 0) = -1;
labelsMat(2, 0) = -1;
labelsMat(3, 0) = -1;
cv::Mat1f trainingDataMat(numberOfSamples, 2);
// Sample 0
trainingDataMat(0, 0) = 501;
trainingDataMat(0, 1) = 10;
// Sample 1
trainingDataMat(1, 0) = 255;
trainingDataMat(1, 1) = 10;
// Sample 2
trainingDataMat(2, 0) = 501;
trainingDataMat(2, 1) = 255;
// Sample 3
trainingDataMat(3, 0) = 10;
trainingDataMat(3, 1) = 501;
// Set up SVM's parameters
cv::Ptr<cv::ml::SVM> svm = cv::ml::SVM::create();
svm->setType(cv::ml::SVM::C_SVC);
svm->setKernel(cv::ml::SVM::LINEAR);
svm->setTermCriteria(cv::TermCriteria(cv::TermCriteria::MAX_ITER, 100, 1e-6));
// Train the SVM with given parameters
cv::Ptr<cv::ml::TrainData> td =
cv::ml::TrainData::create(trainingDataMat, cv::ml::ROW_SAMPLE, labelsMat);
// train the SVM
// svm->train(td);
// or auto train
svm->trainAuto(td);
// predict
// first point used for training
cv::Mat point1 = (cv::Mat_<float>(1, 2) << 501, 10);
float response1 = svm->predict(point1);
// second point used for training
cv::Mat point2 = (cv::Mat_<float>(1, 2) << 255, 10);
float response2 = svm->predict(point2);
std::cout << "first point: " << response1 << "\n" <<
<< "second point: " << response2 << std::endl;
If ran with trainAuto it will output 0 and 1 instead of -1 and 1.
Apparently it was a bug that is fixed now:
http://code.opencv.org/issues/4464

How do I set the value of an input tensor in c++?

I'm trying to run a sample through a pre trained model on ios. session->Run() takes as input a tensor to my understanding. I have initialized a tensor, but how do i set it's value? I don't have much experience using C++.
I have successfully created a test model that accepts 3 dimensional tensor of shape {1, 1, 10}.
I pulled the following line of code from Tensorflow's simple example to create the input tensor.
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/ios_examples/simple/RunModelViewController.mm#L189
tensorflow::Tensor input_tensor(tensorflow::DT_FLOAT, tensorflow::TensorShape({1,1,10}));
From here, I cannot figure out how I would set the data of input_tensor. I would like to set the tensor to something like {{{.0, .1, .2, .3, .4, .5, .6, .7, .8, .9}}}
I had a similar problem and was trying to set the tensor input values in C++ for a model trained in Python. The model is a simple NN with one hidden layer to learn to calculate the XOR operation.
I first created an output graph file with both the graph structure and the model parameters by following steps 1-4 of this nice post: https://medium.com/#hamedmp/exporting-trained-tensorflow-models-to-c-the-right-way-cf24b609d183#.j4l51ptvb.
Then in C++ (the TensorFlow iOS simple example), I used the following code:
tensorflow::Tensor input_tensor(tensorflow::DT_FLOAT, tensorflow::TensorShape({4,2}));
// input_tensor_mapped is an interface to the data of a tensor and used to copy data into the tensor
auto input_tensor_mapped = input_tensor.tensor<float, 2>();
// set the (4,2) possible input values for XOR
input_tensor_mapped(0, 0) = 0.0;
input_tensor_mapped(0, 1) = 0.0;
input_tensor_mapped(1, 0) = 0.0;
input_tensor_mapped(1, 1) = 1.0;
input_tensor_mapped(2, 0) = 1.0;
input_tensor_mapped(2, 1) = 0.0;
input_tensor_mapped(3, 0) = 1.0;
input_tensor_mapped(3, 1) = 1.0;
tensorflow::Status run_status = session->Run({{input_layer, input_tensor}},
{output_layer}, {}, &outputs);
After this, GetTopN(output->flat<float>(), kNumResults, kThreshold, &top_results); returns the same 4 values (0.94433498, 0.94425952, 0.06565627, 0.05823805), as in my Python test code for XOR after the model is trained, in top_results.
So if your tensor's shape is {1,1,10}, you can set the values as follows:
auto input_tensor_mapped = input_tensor.tensor<float, 3>();
input_tensor_mapped(0, 0, 0) = 0.0;
input_tensor_mapped(0, 0, 1) = 0.1;
....
input_tensor_mapped(0, 0, 9) = 0.9;
Credit: the answer at How do I pass an OpenCV Mat into a C++ Tensorflow graph? is very helpful.
If you want to directly set the value of a tensor you can use few utilities functions provided by the Tensor interface. For the most common linear access you can use flat<T>.
From tensor_test
void ExpectClose(const Tensor& x, const Tensor& y, double atol, double rtol) {
auto Tx = x.flat<T>();
auto Ty = y.flat<T>();
for (int i = 0; i < Tx.size(); ++i) {
if (!IsClose(Tx(i), Ty(i), atol, rtol)) {
LOG(ERROR) << "x = " << x.DebugString();
LOG(ERROR) << "y = " << y.DebugString();
LOG(ERROR) << "atol = " << atol << " rtol = " << rtol
<< " tol = " << atol + rtol * std::fabs(Tx(i));
EXPECT_TRUE(false) << i << "-th element is not close " << Tx(i) << " vs. "
<< Ty(i);
}
}
}
to create a tensor you can use one of the constructors
Tensor(DT_FLOAT, new TensorShape(..))
If you want to set the value of a tensor or a placeholder at run time you need to pass it through the Run() interface:
Status run_status = session->Run({{input_layer, resized_tensor}},
{output_layer}, {}, &outputs);
if (!run_status.ok()) {
LOG(ERROR) << "Running model failed: " << run_status;
return -1;
}
If you want to have a predefine value of a tensor you can use the Const constructor
tensorflow::ops::Const({input_height, input_width})

adding float openCV3.0

I actually have a problem on openCV3.0.
I used 12 gabor filters(12 differents orientation) on 1 image and stocked them.
Now I want to add all those images and then divide by 12 each value to obtain the mean of the 12 filters.
Because those image are RGB, I have to work on each channel separatly.
The problem is : when I add all the values, I obtain values > 12 while all the values are between 0 and 1.
The part of the code bugged :
for (i = 0; i < gaborV.size(); ++i) { //gaborV contain the 12 gabor filters
std::vector<cv::Mat> vec_split; //I split because of the 3 channels
cv::split(gaborV[i], vec_split);
for (int k = 0; k < imgCol.rows; ++k) {
for (int j = 0; j < imgCol.cols; ++j) {
if (k == 1 && j == 1)
std::cout << mat_X.at<float>(k, j) << " " << vec_split[0].at<float>(k, j) << std::endl;
mat_X.at<float>(k, j) += vec_split[0].at<float>(k, j);
mat_Y.at<float>(k, j) += vec_split[1].at<float>(k, j);
mat_Z.at<float>(k, j) += vec_split[2].at<float>(k, j);
}
}
}
and mat_X, mat_Y and mat_Z are created as follow :
mat_X = mat_Y = mat_Z = cv::Mat(cvSize(imgColNormalize.cols, imgColNormalize.rows), CV_32FC1, cvScalar(0.));
As I said, all values in vec_split are between 0 and 1, but when I'm out of the loop, mat_X, mat_Y and mat_Z contain values > 12..
The output of the cout I used :
0 0.507358
1.54751 0.496143
3.00963 0.528832
4.53887 0.465426
... and at the end I have 15.9459
And i don't understand since 0 + 0.507358 != 1.54751; 1.54751 + 0.496143 != 3.00963 ...
Do someone understand the problem?
Thanks for all!
I think the problem is here:
mat_X = mat_Y = mat_Z = cv::Mat(cvSize(imgColNormalize.cols,
imgColNormalize.rows), CV_32FC1, cvScalar(0.));
The way you initialise these arrays results in all three cv::Mat objects referencing the same data. Only one Mat is created and so your code increments the values in this array three times.
For info, OpenCV uses a reference counting mechanism with cv::Mat and the assignment operator simply creates a new reference to existing data. If you wanted to create a genuine deep-copy of a cv::Mat, you would need to use cv::Mat::clone().
So, instead, initialise like so:
mat_X = cv::Mat(cvSize(imgColNormalize.cols, imgColNormalize.rows), CV_32FC1, cvScalar(0.));
mat_Y = cv::Mat(cvSize(imgColNormalize.cols, imgColNormalize.rows), CV_32FC1, cvScalar(0.));
mat_Z = cv::Mat(cvSize(imgColNormalize.cols, imgColNormalize.rows), CV_32FC1, cvScalar(0.));
An excerpt from the documentation copied below for posterity:

Interfering Vector in glBegin()

I am trying to implement code for an assignment to render skeleton and mesh animations. In my glBegin(GL_TRIANGLES) section, I have some vectors that appear to be interfering with my information when it shouldn't.
glBegin(GL_TRIANGLES);
for (int i = 0; i < mesh->nfaces.size(); i += 1)
for (int k = 0; k < 3; k += 1) {
int j = k;//2 - k;
glm::vec4 myPointPrime;
myPointPrime.w = 1;
myPoint.x = ecks = mesh->vertex[mesh->faces[i][j]][0];
myPoint.y = why = mesh->vertex[mesh->faces[i][j]][1];
myPoint.z = zed = mesh->vertex[mesh->faces[i][j]][2];
// Stuff vvvv THIS CAUSES PROBLEMS
for (int t = 0; t < mySkeleton->vertex.at(i).size(); t++) {
myPointPrime += mySkeleton->vertex[i][j] * MyXformations * myPoint;
}
glNormal3f(mesh->normal[mesh->nfaces[i][j]][0],
mesh->normal[mesh->nfaces[i][j]][1],
mesh->normal[mesh->nfaces[i][j]][2]);
glVertex3f(mesh->vertex[mesh->faces[i][j]][0],
mesh->vertex[mesh->faces[i][j]][1],
mesh->vertex[mesh->faces[i][j]][2]);
// glVertex3f(myPointPrime.x, myPointPrime.y, myPointPrime.z);
// glVertex3f(myPoint.x, myPoint.y, myPoint.z);
}
glEnd();
The myPointPrime += ... code is doing something weird to my Vertex calls, the scene won't render unless I comment out that for loop.
If I comment out the loop, then the scene renders, but I think I kinda need the loop if animating something like 16,000 vertexes is going to have any performance at all.
Is having that there kind of like having it automatically multiply with the glVertex calls?
Edit:
Below is another version of the code I hope should be more clear, instead of calculating the points in the actual drawing code I change the whole mesh to supposedly follow the skeleton each frame, but nothing is rendered.
for (int vertex_i = 0; vertex_i < mesh->nfaces.size(); vertex_i++) {
for (int k = 0; k < 3; k += 1) {
int j = k;//2 - k;
pointp.x = 0;
pointp.y = 0;
pointp.z = 0;
for (int t = 0; t < mySkeleton->vertex.at(vertex_i).size(); t++) {
point.x = mesh->vertex[mesh->faces[vertex_i][j]][0];
point.y = mesh->vertex[mesh->faces[vertex_i][j]][1];
point.z = mesh->vertex[mesh->faces[vertex_i][j]][2];
//glPushMatrix();
pointp += mySkeleton->vertex[vertex_i][t] * myTranslationMatrix * myRotationMatrix * point;
cout << "PointP X: " << pointp.x << " PointP Y: " << pointp.y << " PointP Z: " << pointp.z << endl;
mesh->vertex[mesh->faces[vertex_i][j]][0] = pointp.x;
mesh->vertex[mesh->faces[vertex_i][j]][1] = pointp.y;
mesh->vertex[mesh->faces[vertex_i][j]][2] = pointp.z;
//myPointPrime += MyXformations * myPoint;
}
}
}
My assumption is that maybe the calculations for pointp isn't doing what I think its doing?
mySkeleton->vertex[vertex_i][t] is a vector from my 'skeleton' class, it holds all of the weights for every vertex, there are 17 weights per vertex.
"MyXformations" is a 4x4 matrix passed from my skeleton animation function that holds the last known key frame and this is applied to the vertexes.
point is the current point in the vertex.
Your loop variable is t. However, you refer to j in the loop. Looks to me like your loop might simply be crashing for larger values of j.
You're not using t inside the for loop. Is this expected?
mySkeleton->vertex[i][j] looks like it's out of bounds since j should be for mesh->faces/mesh->nfaces.
Also you can use glNormal3fv and glVertex3fv with arrays.
With out of bounds memory operations you can get all sorts of weird stuff happening, although I can't see any out of bound writes. Your * operators don't modify the objects do they?
If you're worried about performance. You shouldn't be using immediate mode. Instead, put all your data on the GPU with buffer objects (including join/bone transformations) and animate on the fly in the vertex shader.
This is from a few years ago, but worth a read: Animated Crowd Rendering.