Camera calibration data retrieval - c++

Question

So I need help with OpenCV in C++
Basically I have a camera that has some radial distortion and I am able to undistort it using the provided examples/samples in OpenCV.
But currently I have to recalibrate the camera each time the program is run. But the example generates an XML file for a reason right... To make use of those values...
My problem is I'm not sure which values and how to use those values from the XML file to undistort the camera without having to go through the entire calibration again.
I tried finding examples of this use online but for some reason nothing related to my problem came up...
Supposedly we are supposed to be able to take the values from the output XML file and use them directly in the program so that we don't have to recalibrate the camera each time.
But currently that's exactly what my program is doing :/
I really hope someone can help me with this
Thanks a lot :)

Answer 1

First, You have to create Camera Matrix with Camera Matrix values from XML file.
Mat cameraMatrix = new Mat(new Size(3,3), CvType.CV_64FC1);
cameraMatrix.put(0,0,3275.907);
cameraMatrix.put(0,1,0);
cameraMatrix.put(0,2,2069.153);
cameraMatrix.put(1,0,0);
cameraMatrix.put(1,1,3270.752);
cameraMatrix.put(1,2,1139.271);
cameraMatrix.put(2,0,0);
cameraMatrix.put(2,1,0);
cameraMatrix.put(2,2,1);
Second, create Distortion Matrix with Distortion_Coefficients from XML file.
Mat distortionMatrix = new Mat(new Size(4,1), CvType.CV_64FC1);
distortionMatrix.put(0,0,-0.006934);
distortionMatrix.put(0,1,-0.047680);
distortionMatrix.put(0,2,0.002173);
distortionMatrix.put(0,3,0.002580);
Finally, use OpenCV method.
Mat map1 = new Mat();
Mat map2 = new Mat();
Mat temp = new Mat();
Imgproc.initUndistortRectifyMap(cameraMatrix, distortionMatrix, temp, cameraMatrix, src.size(), CvType.CV_32FC1, map1, map2);
And you can get two matrix map1, map2 which use for undistortion.
If you get this two matrix, you don't have to re-calibrate every time.
Just use remap and undistortion will be done.
Imgproc.remap(mat, undistortPicture, map1, map2, Imgproc.INTER_LINEAR);
refer this link.

Answer 2

Alright so I was able to extract the 4 things that I think was necessary from the output xml file. Essentially I made a new class that I named CalibSet and just extracted the data from the xml file via the "tfs[""] >> xxx;" at the bottom of the code.
class CalibSet
{
public:
Size Boardsize; // The size of the board -> Number of items by width and height
Size image; // image size
String calibtime;
Mat CamMat; // camera matrix
Mat DistCoeff; // distortion coefficient
Mat PViewReprojErr; // per view reprojection error
float SqSize; // The size of a square in your defined unit (point, millimeter,etc).
float avg_reproj_error;
int NrFrames; // The number of frames to use from the input for calibration
int Flags;
bool imagePoints; // Write detected feature points
bool ExtrinsicParams; // Write extrinsic parameters
bool GridPoints; // Write refined 3D target grid points
bool fisheyemodel; // use fisheye camera model for calibration
void write(FileStorage& fs) const //Write serialization for this class
{
fs << "{"
<<"nr_of_frames" << NrFrames
<<"image_width" << image.width
<<"image_height" << image.height
<<"board_width" << Boardsize.width
<<"board_height" << Boardsize.height
<<"square_size" << SqSize
<<"flags" << Flags
<<"fisheye_model" << fisheyemodel
<<"camera_matrix" << CamMat
<<"distortion_coefficients" << DistCoeff
<<"avg_reprojection_error" << avg_reproj_error
<<"per_view_reprojection_errors" << PViewReprojErr
<<"extrinsic_parameters" << ExtrinsicParams
<< "}";
}
void read(const FileNode& node) //Read serialization for this class
{
node["calibration_time"] >> calibtime;
node["nr_of_frames"] >> NrFrames;
node["image_width"] >> image.width;
node["image_height"] >> image.height;
node["board_width"] >> Boardsize.width;
node["board_height"] >> Boardsize.height;
node["square_size"] >> SqSize;
node["flags"] >> Flags;
node["fisheye_model"] >> fisheyemodel;
node["camera_matrix"] >> CamMat;
node["distortion_coefficients"] >> DistCoeff;
node["avg_reprojection_error"] >> avg_reproj_error;
node["per_view_reprojection_errors"] >> PViewReprojErr;
node["extrinsic_parameters"] >> ExtrinsicParams;
}
};
CalibSet CS;
FileStorage tfs(inputCalibFile, FileStorage::READ); // Read the settings
if (!tfs.isOpened())
{
cout << "Could not open the calibration file: \"" << inputCalibFile << "\"" << endl;
return -1;
}
tfs["camera_matrix"] >> CS.CamMat;
tfs["distortion_coefficients"] >> CS.DistCoeff;
tfs["image_width"] >> CS.image.width;
tfs["image_height"] >> CS.image.height;
tfs.release(); // close Settings file
And after this I use the function "undistort" to correct the live camera frames that I stored in frame and put the corrected image in rframe
flip(frame, frame, -1); // flip image vertically so that it's not upside down
cv::undistort(frame, rframe, CS.CamMat, CS.DistCoeff);
flip(rframe, rframe, +1); // flip image horizontally
It's important to make sure that the orientation of photos taken for sampling is exactly the same as the one used later on (including mirroring vertically or horizontally) else the image will still be distorted after using "undistort"
After this I can get an undistorted image as intended BUT the frame rate is extremely low (around 10-20FPS) and I'd appreciate any help in optimising the process if possible; to allow for higher frame rate from the live camera feed