I noticed that opencv stereoCalibrate() changes the focal lengths in camera matrices even though I've set appropriate flag (ie CV_CALIB_FIX_FOCAL_LENGTH). I'm using two identical cameras with the same focal length set mechanically on lens and furthermore I know the sensor size so I can compute intrinsic camera matrix manually what actually I do.
Here you have some output form the stereo calibration program - camera matrices before and after stereoCalibrate().
std::cout << "Before calibration: " << std::endl;
std::cout << "C1: " << _cameraMatrixA << std::endl;
std::cout << "C2: " << _cameraMatrixB << std::endl;
double error = cv::stereoCalibrate(objectPoints, imagePointsA, imagePointsB, _cameraMatrixA, _distCoeffsA, _cameraMatrixB, _distCoeffsB, _imageSize,
R, T, E, F,
cv::TermCriteria((cv::TermCriteria::COUNT + cv::TermCriteria::EPS), 30, 9.999999999999e-7), CV_CALIB_FIX_FOCAL_LENGTH | CV_CALIB_FIX_PRINCIPAL_POINT);
std::cout << "After calibration: " << std::endl;
std::cout << "C1: " << _cameraMatrixA << std::endl;
std::cout << "C2: " << _cameraMatrixB << std::endl;
Before calibration:
C1: [6203.076923076923, 0, 1280; 0,
6203.076923076923, 960; 0, 0, 1]
C2: [6203.076923076923, 0, 1280; 0, 6203.076923076923, 960; 0, 0,
1]
After calibration:
C1: [6311.77650416514, 0, 1279.5; 0, 6331.34531760757, 959.5; 0,
0, 1]
C2: [6152.655897294907, 0, 1279.5; 0, 6206.591406832492, 959.5; 0,
0, 1]
I think this is weird opencv behavior. Anyone faced similar problem? I know it is easy to solve, I can just set focal lengths to camera matrices after stereo calibration.
In order to do what you want, you have to call stereoCalibrate with flags:
CV_CALIB_USE_INTRINSIC_GUESS | CV_CALIB_FIX_FOCAL_LENGTH | CV_CALIB_FIX_PRINCIPAL_POINT
If you do not use the CV_CALIB_USE_INTRINSIC_GUESS flag, stereoCalibrate will first initialize the camera matrices and distortion coefficients itself and then fix part of them in the subsequent optimization. This is stated in the documentation, although rather unclearly and without mentionning that critical flag:
Besides the stereo-related information, the function can also perform a full calibration of each of two cameras. However, due to the high dimensionality of the parameter space and noise in the input data, the function can diverge from the correct solution. If the intrinsic parameters can be estimated with high accuracy for each of the cameras individually (for example, using calibrateCamera() ), you are recommended to do so [...].
Using CV_CALIB_USE_INTRINSIC_GUESS in addition to any of the CV_CALIB_FIX_* flags tells the function to use what you are passing as input, otherwise, this input is simply ignored and overwritten.
the CV_CALIB_FIX_FOCAL_LENGTH flag causes the optimization routine to just use the Fx and Fy that were passed in the intrinsic matrix.
Related
I have a publish-subscribe type of a node that receives pose information (position and orientation) from the subscribed data stream and it should compute the inverse and publish out.
In order to do so I'm creating a 4-by-4 homogeneous transformation matrix from the original pose data.
Inverse it using the Eigen C++ template library, convert the transformation matrix back to position and orientation form and publish it.
When I plotted the published data stream I noticed some noise so I ended up publishing the original data too for comparison, here is what I did:
convert original_pose to TF matrix, named as original_TF
convert original_TF back to pose, named as original_pose_
publish original_pose_
inverse original_TF assign to inverted_TF
convert inverted_TF to pose, named as inverted_pose_
publish inverted_pose_
When I plot the X, Y, Z position fields, I'm seeing a significant amount of noise (spikes and notches in the visual below) in the inverted pose data. Since I'm using the same functions to convert the original pose to TF and back, I know that those equations aren't the source of the noise.
Blue is the original, whereas red is the inverted.
Here is the code. Really nothing extraordinary.
bool inverse_matrix(std::vector<std::vector<double> > & input, std::vector<std::vector<double> > & output)
{
// TODO: Currently only supports 4-by-4 matrices, I can make this configurable.
// see https://eigen.tuxfamily.org/dox/group__TutorialMatrixClass.html
Eigen::Matrix4d input_matrix;
Eigen::Matrix4d output_matrix;
Eigen::VectorXcd input_eivals;
Eigen::VectorXcd output_eivals;
input_matrix << input[0][0], input[0][1], input[0][2], input[0][3],
input[1][0], input[1][1], input[1][2], input[1][3],
input[2][0], input[2][1], input[2][2], input[2][3],
input[3][0], input[3][1], input[3][2], input[3][3];
cout << "Here is the matrix input:\n" << input_matrix << endl;
input_eivals = input_matrix.eigenvalues();
cout << "The eigenvalues of the input_eivals are:" << endl << input_eivals << endl;
if(input_matrix.determinant() == 0) { return false; }
output_matrix = input_matrix.inverse();
cout << "Here is the matrix output:\n" << output_matrix << endl;
output_eivals = output_matrix.eigenvalues();
cout << "The eigenvalues of the output_eivals are:" << endl << output_eivals << endl;
// Copy output_matrix to output
for (int i = 0; i < 16; ++i)
{
int in = i/4;
int im = i%4;
output[in][im] = output_matrix(in, im);
}
return true;
}
-- Edit 1 --
I printed out the eigenvalues of the input and output matrices of the inverse_matrix function.
Here is the matrix input:
0.99916 -0.00155684 -0.0409514 0.505506
0.00342358 -0.992614 0.121267 0.19625
-0.0408377 -0.121305 -0.991775 1.64257
0 0 0 1
The eigenvalues of the input_eivals are:
(1,0)
(-0.992614,0.121312)
(-0.992614,-0.121312)
(1,0)
Here is the matrix output:
0.99916 0.00342358 -0.0408377 -0.438674
-0.00155684 -0.992614 -0.121305 0.39484
-0.0409514 0.121267 -0.991775 1.62597
-0 -0 0 1
The eigenvalues of the output_eivals are:
(1,0)
(-0.992614,0.121312)
(-0.992614,-0.121312)
(1,0)
-- Edit 2 --
I don't quite understand what you are plotting. Is it original_pose.{X,Y,Z} and inverted_pose.{X,Y,Z}? Then the "spikes" will really depend on the orientation-part of the matrix.
I am plotting original_pose_{position.x, position.y, position.z} and inverted_pose_{position.x, position.y, position.z} where the complete data that's published is <variable_name>{position.x, position.y, position.z, orientation.w, orientation.x, orientation.y, orientation.z}.
Can you elaborate on "the "spikes" will really depend on the orientation-part of the matrix."?
Also, how is your description related to the code-snippet? (I don't see any matching variable names).
I've identified that the source of the noise is the inversion, which is the item number 4 in my description: inverse original_TF assign to inverted_TF. To relate one another, I'm calling the function as follows:
isSuccess = inverse_matrix(original_TF, inverted_TF);
How do you store "poses" (is that the vector<vector> in your snippet)?
Yes, I'm storing them in 2-dimensional vectors of type double.
At any point, do you use Eigen::Transform to store transformations, or just plain Eigen::Matrix4d?
No, I'm only using Eigen::Matrix4d locally in the inverse_matrix function to be able to make use of the Eigen library for computation.
I used a DLIB parallel_for loop to do some processing and add coordinates to a vector> that has been declared outside the loop. But I cannot use the vector.push_back() function from within the loop.
Verified whether there are any declaration issues.
Passed the vector pointer to the parallel_for loop lambda function.
//Store cordinates of respective face_image
std::vector<pair<int,int>> xy_coords;
//Create a dlib image window
window.clear_overlay();
window.set_image(dlib_frame);
auto detections = f_detector(dlib_frame);
dlib::parallel_for(0, detections.size(), [&,detections,xy_coords](long i)
{
auto det_face = detections[i];
//Display Face data to the user
cout << "Face Found! " << "Area: " << det_face.area() << "X: " <<det_face.left() << "Y: " << det_face.bottom() << endl;
//Get the Shape details from the face
auto shape = sp(dlib_frame, det_face);
//Extract Face Image from frame
matrix<rgb_pixel> face_img;
extract_image_chip(dlib_frame, get_face_chip_details(shape, 150, 0.25), face_img);
faces.push_back(face_img);
//Add the coordinates to the coordinates vector
xy_coords.push_back(std::pair<int,int>((int)det_face.left(),(int)det_face.bottom()));
//Add face to dlib image window
window.add_overlay(det_face);
});
Your lambda is capturing xy_coords by copy, the one you're pushing into inside the lamdba is not the same outside. Try capturing it by reference like so [&,&xy_coords,detections] or just [&,detections].
See this for more info:
https://en.cppreference.com/w/cpp/language/lambda#Lambda_capture
I am trying to develop an Artificial Neural Network in C++ using OpenCV 3.4.1 with the aim of being able to recognise 33 different characters, including both numbers and letters, but the results I am obtaining are always wrong.
I have tested my code with different parameters' values like the alpha and beta of the sigmoid function that I am using for training, the backpropagation parameters, or the number of hidden nodes but, although the result varies sometimes, it normally tends to be a vector of the following shape:
Classification result:
[20.855789, -0.033862107, -0.0053131776, 0.026316155, -0.0032050854,
0.036046479, -0.025410429, -0.017537225, 0.015429396, -0.023276867, 0.013653283, -0.025660357, -0.051959664, -0.0032470606, 0.032143779, -0.011631044, 0.022339549, 0.041757714, 0.04414707, -0.044756029, 0.042280547, 0.012204648, 0.026924053, 0.016814215, -0.028257577, 0.05190875, -0.0070033628, -0.0084492415, -0.040644459, 0.00022287761, -0.0376678, -0.0021550131, -0.015310903]
That is, independently of which character I test, it is always predicting that the analysed character is the one in the first position of the characters vector, which corresponds to number '1'.
The training data is obtained from an .XML I have created, which contains 474 samples (rows) with 265 attributes each (cols). As for the training classes, following some advice I found in a previous question in this forum, it is obtained from another .XML file that contains 474 rows, one for each training sample, and 33 columns, one for each character/class.
I attach the code below so that you can perhaps kindly guess what I am doing wrong and I am so thankful in advance for any help you can offer! :)
//Create the Neural Network
Mat_<int> layerSizes(1, 3);
layerSizes(0, 0) = numFeaturesPerSample;
layerSizes(0, 1) = nlayers;
layerSizes(0, 2) = numClasses;
//Set ANN params
Ptr<ANN_MLP> network = ANN_MLP::create();
network->setLayerSizes(layerSizes);
network->setActivationFunction(ANN_MLP::SIGMOID_SYM, 0.6, 1);
network->setTrainMethod(ANN_MLP::BACKPROP, 0.1, 0.1);
Ptr<TrainData> trainData = TrainData::create(TrainingData, ROW_SAMPLE, classes);
network->train(trainData);
//Predict
if (network->isTrained())
{
trained = true;
Mat results;
cout << "Predict:" << endl;
network->predict(features, results);
cout << "Prediction done!" << endl;
cout << endl << "Classification result: " << endl << results << endl;
//We need to know where in output is the max val, the x (cols) is the class.
Point maxLoc;
double maxVal;
minMaxLoc(results, 0, &maxVal, 0, &maxLoc);
return maxLoc.x;
}
This is a part of my code and it's result in opengl/c++(using visual studio 2013):
GLint *raspos = new GLint[];
glRasterPos2i(56, 56);
glGetIntegerv(GL_CURRENT_RASTER_POSITION, raspos);
cout << " , X : " << raspos[0] << " and " << " Y : " << raspos[1];
result
X : 125 and Y : 125
i can't understand what's going on! why glRasterPos2i changes the arguments ?
The coordinates passed to glRasterPos are subject to the transformation pipeline. The values you retrieve is the raster position in window coordinates after undergoing those transformations.
Because the raster position is transform by the current projection and modelview matrices just like an ordinary vertex is, but querying GL_CURRENT_RASTER_POSITION is retrieving the window space coordinates.
I'm using SFML to draw in C++. It was going well until I tried accessing the position of a circle I drew on the screen. Code:
sf::Shape RootCircle = sf::Shape::Circle(300, 30, 30, sf::Color::Blue);
App.Draw(RootCircle);
cout << "X: " << RootCircle.GetPosition().x << endl;
cout << "Y: " << RootCircle.GetPosition().y << endl;
It's consisting telling me that the x and y positions are set to 0. What am I missing?
By calling the sf::Shape::Circle() constructor, only the offset relative to the position is set to 300,30. To actually set the position of the circle, you need to call:
rootCircle.SetPosition(300.0f, 30.0f);
Note that by setting the position to 300,30, whatever offset is specified in the Circle() constructor will be relative to the actual position specified.