I want to undistort a camera image. The undistort function of OpenCV is too slow, so I want to split it like mentioned in the documentation into the 2 calls of initUndistortRectifyMap (as init step) and remap (in the render loop).
At first, I tried a test program with the principal approach:
//create source matrix
cv::Mat srcImg(res.first, res.second, cvFormat, const_cast<char*>(pImg));
//fill matrices
cv::Mat cam(3, 3, cv::DataType<float>::type);
cam.at<float>(0, 0) = 528.53618582196384f;
cam.at<float>(0, 1) = 0.0f;
cam.at<float>(0, 2) = 314.01736116032430f;
cam.at<float>(1, 0) = 0.0f;
cam.at<float>(1, 1) = 532.01912214324500f;
cam.at<float>(1, 2) = 231.43930864205211f;
cam.at<float>(2, 0) = 0.0f;
cam.at<float>(2, 1) = 0.0f;
cam.at<float>(2, 2) = 1.0f;
cv::Mat dist(5, 1, cv::DataType<float>::type);
dist.at<float>(0, 0) = -0.11839989180635836f;
dist.at<float>(1, 0) = 0.25425420873955445f;
dist.at<float>(2, 0) = 0.0013269901775205413f;
dist.at<float>(3, 0) = 0.0015787467748277866f;
dist.at<float>(4, 0) = -0.11567938093172066f;
cv::Mat map1, map2;
cv::initUndistortRectifyMap(cam, dist, cv::Mat(), cam, cv::Size(res.second, res.first), CV_32FC1, map1, map2);
cv::remap(srcImg, *m_undistImg, map1, map2, cv::INTER_CUBIC);
The format of my camera image is BGRA. The code compiles and starts, but the resulting image is wrong:
Any ideas, what's wrong with my code?
It works, yes. To be honest, I don't remember exactly what the problem was. I interchanged width and height or somethink like that.
This is my running code:
//create source matrix
cv::Mat srcImg(resolution.second, resolution.first, cvFormat, const_cast<unsigned char*>(pSrcImg));
//look if an update of the maps is necessary
if ((resolution.first != m_width) || (m_height != resolution.second))
{
m_width = resolution.first;
m_height = resolution.second;
cv::initUndistortRectifyMap(*m_camData, *m_distData, cv::Mat(), *m_camData, cv::Size(resolution.first, resolution.second), CV_32FC1, *m_undistMap1, *m_undistMap2);
}
//create undistorted image
cv::remap(srcImg, *m_undistortedImg, *m_undistMap1, *m_undistMap2, cv::INTER_LINEAR);
return reinterpret_cast<unsigned char*>(m_undistortedImg->data);
Related
Currently I'm multiplying a 4x4 mat and a Point3f in a function this way,
Is there a more efficient way to do this multiplication?
Note: I also need to convert the result back to a cv::Point3f
Thanks
cv::Mat cam_to_world(4, 4, CV_32FC1);
///....
cv::Mat pointA_cam(4, 1, CV_32FC1);
pointA_cam.at<float>(0, 0) = OAvec_cam.x;
pointA_cam.at<float>(1, 0) = OAvec_cam.y;
pointA_cam.at<float>(2, 0) = OAvec_cam.z;
pointA_cam.at<float>(3, 0) = 1.0;
cv::Point3f point_A_wld;
cv::Mat point_A_world = cam_to_world*pointA_cam;
point_A_wld.x = point_A_world.at<float>(0, 0);
point_A_wld.y = point_A_world.at<float>(1, 0);
point_A_wld.z = point_A_world.at<float>(2, 0);
This is what I have, if anyone has a more compact solution it would help
cv::Mat cam_to_world(4, 4, CV_32FC1);
cv::Mat pointA_cam = cam_to_world*Mat(cv::Vec4f(OAvec_cam.x,OAvec_cam.y,OAvec_cam.z,1.0));
cv::Point3f point_A_wld(cam_to_world.at<float>(0,0),cam_to_world.at<float>(1,0),cam_to_world.at<float>(2,0));
I am trying to set predefined values to homography and then use function warpPerspective that will warp my image. First i used findHomography function and displayed result:
H = findHomography(obj, scene, CV_RANSAC);
for( int i=0; i<H.rows; i++){
for( int j=0; j<H.cols; j++){
printf("H: %d %d: %lf\n",i,j,H.at<double>(i,j));
}
}
warpPerspective(image1, result, H, cv::Size(image1.cols + image2.cols, image1.rows));
This works as it is supposed to and i get these values
After that i tried to set values for H and call warpPerspective like this:
H.at<double>(0, 0) = 0.766912;
H.at<double>(0, 1) = 0.053191;
H.at<double>(0, 2) = 637.961151;
H.at<double>(1, 0) = -0.118426;
H.at<double>(1, 1) = 0.965682;
H.at<double>(1, 2) = 3.405685;
H.at<double>(2, 0) = -0.000232;
H.at<double>(2, 1) = 0.000019;
H.at<double>(2, 2) = 1.000000;
warpPerspective(image1, result, H, cv::Size(image1.cols + image2.cols, image1.rows));
And now i get System NullReferenceException, do you have any idea why is this failing?
Okay i got help on OpenCV forum, my declaration of H was like this
cv::Mat H;
this was okay for function fingHomography, but when i wanted to add values manually, i had to declare H like this:
cv::Mat H(3, 3, CV_64FC1);
Nvidia's cuDNN for deep learning has a rather interesting format for images called CHW. I have a cv::Mat img; that I want to convert to a one-dimensional vector of floats. The problem that I'm having is that the format of the 1D vector for CHW is (RR...R, GG..G,BB..B).
So I'm curious as to how I can extract the channel values for each pixel and order them for this format.
I faced with same problem and and solve it in that way:
#include <opencv2/opencv.hpp>
cv::Mat hwc2chw(const cv::Mat &image){
std::vector<cv::Mat> rgb_images;
cv::split(image, rgb_images);
// Stretch one-channel images to vector
cv::Mat m_flat_r = rgb_images[0].reshape(1,1);
cv::Mat m_flat_g = rgb_images[1].reshape(1,1);
cv::Mat m_flat_b = rgb_images[2].reshape(1,1);
// Now we can rearrange channels if need
cv::Mat matArray[] = { m_flat_r, m_flat_g, m_flat_b};
cv::Mat flat_image;
// Concatenate three vectors to one
cv::hconcat( matArray, 3, flat_image );
return flat_image;
}
P.S. If input image isn't in RGB format, you can change channel order in matArray creation line.
Use cv::dnn::blobFromImage:
cv::Mat bgr_image = cv::imread(imageFileName);
cv::Mat chw_image = cv::dnn::blobFromImage
(
bgr_image,
1.0, // scale factor
cv::Size(), // spatial size for output image
cv::Scalar(), // mean
true, // swapRB: BGR to RGB
false, // crop
CV_32F // Depth of output blob. Choose CV_32F or CV_8U.
);
const float* data = reinterpret_cast<const float*>(chw_image.data);
int data_length = 1 * 3 * bgr_image.rows * bgr_image.cols;
You can either iterate over the image manually and copy the values into the right place, or you can use something like cv::extractChannel to copy the channels one by one like so:
#include <opencv2/opencv.hpp>
int main()
{
//create dummy 3 channel float image
cv::Mat sourceRGB(cv::Size(100,100),CV_32FC3);
auto size = sourceRGB.size();
for (int y = 0; y < size.height; ++y)
{
for (int x = 0; x < size.width; ++x)
{
float* pxl = sourceRGB.ptr<float>(x, y);
*pxl = x / 100.0f;
*(pxl+1) = y / 100.0f;
*(pxl + 2) = (y / 100.0f) * (x / 100.0f);
}
}
cv::imshow("test", sourceRGB);
cv::waitKey(0);
//create single image with all 3 channels one after the other
cv::Size newsize(size.width,size.height*3);
cv::Mat destination(newsize,CV_32FC1);
//copy the channels from the source image to the destination
for (int i = 0; i < sourceRGB.channels(); ++i)
{
cv::extractChannel(
sourceRGB,
cv::Mat(
size.height,
size.width,
CV_32FC1,
&(destination.at<float>(size.height*size.width*i))),
i);
}
cv::imshow("test", destination);
cv::waitKey(0);
return 0;
}
I was trying to write Point2f imagePoints to a Mat image in openCV. I was following the link below.
Create Mat from vector<point2f>
But I am getting 'Assertion failed' error. Please help.
Code:
std::vector<cv::Point3d> objectPoints;
std::vector<cv::Point2d> imagePoints;
cv::Mat intrisicMat(3, 3, cv::DataType<double>::type);
intrisicMat.at<double>(0, 0) = param.focalLength.first;
intrisicMat.at<double>(0, 1) = 0;
intrisicMat.at<double>(0, 2) = param.principalPoint.first;
intrisicMat.at<double>(1, 0) = 0;
intrisicMat.at<double>(1, 1) = param.focalLength.second;
intrisicMat.at<double>(1, 2) = param.principalPoint.second;
intrisicMat.at<double>(2, 0) = 0;
intrisicMat.at<double>(2, 1) = 0;
intrisicMat.at<double>(2, 2) = 1;
cv::Mat rVec(3, 1, cv::DataType<double>::type); // Rotation vector
rVec.at<double>(0) = 0;
rVec.at<double>(1) = 0;
rVec.at<double>(2) = 0;
cv::Mat tVec(3, 1, cv::DataType<double>::type); // Translation vector
tVec.at<double>(0) = 0;
tVec.at<double>(1) = 0;
tVec.at<double>(2) = 0;
cv::Mat distCoeffs(5, 1, cv::DataType<double>::type); // Distortion vector
distCoeffs.at<double>(0) = param.distortionRadial.at(0);
distCoeffs.at<double>(1) = param.distortionRadial.at(1);
distCoeffs.at<double>(2) = param.distortionTangential.first;
distCoeffs.at<double>(3) = param.distortionTangential.second;
distCoeffs.at<double>(4) = param.distortionRadial.at(2);
projectPoints(objectPoints, rVec, tVec, intrisicMat, distCoeffs, imagePoints);
Mat depthImage = Mat(imagePoints);
imwrite("E:/softwares/1.8.0.71/bin/depthImage.jpg", depthImage);
cout << "depthImage.channels()=" << depthImage.channels() << endl;
Error:
OpenCV Error: Assertion failed (image.channels() == 1 || image.channels() == 3 || image.channels() == 4) in cv::imwrite_, file E:\softwares\opencv-3.1.0\opencv-3.1.0\modules\imgcodecs\src\loadsave.cpp, line 455
My image has 2 channels. So ImWrite() is throwing assertion failed error. How can I create a Mat image using the Image points if not like this?
With what you have written in the comments, it seems that you're trying to imwrite your Mat to a file. The problem is, a Mat from Vector<Point2f> will give a 2 channels matrix, which is not compatible with any image format (grayscale, RGB or RGBA).
Moreover, please edit your main post to show the code (using markdown) so it is easier to read and then help you.
I've been trying to compute real world coordinates of points from a disparity map using the reprojectImageTo3D() function provided by OpenCV, but the output seems to be incorrect.
I have the calibration parameters, and compute the Q matrix using
stereoRectify(left_cam_matrix, left_dist_coeffs, right_cam_matrix, right_dist_coeffs, frame_size, stereo_params.R, stereo_params.T, R1, R2, P1, P2, Q, CALIB_ZERO_DISPARITY, 0, frame_size, 0, 0);
I believe this first step is correct, since the stereo frames are being rectified properly, and the distortion removal I'm performing also seems all right. The disparity map is being computed with OpenCV's block matching algorithm, and it looks good too.
The 3D points are being calculated as follows:
cv::Mat XYZ(disparity8U.size(),CV_32FC3);
reprojectImageTo3D(disparity8U, XYZ, Q, false, CV_32F);
But for some reason they form some sort of cone, and are not even close to what I'd expect, considering the disparity map. I found out that other people had a similar problem with this function, and I was wondering if someone has the solution.
Thanks in advance!
[EDIT]
stereoRectify(left_cam_matrix, left_dist_coeffs, right_cam_matrix, right_dist_coeffs,frame_size, stereo_params.R, stereo_params.T, R1, R2, P1, P2, Q, CALIB_ZERO_DISPARITY, 0, frame_size, 0, 0);
initUndistortRectifyMap(left_cam_matrix, left_dist_coeffs, R1, P1, frame_size,CV_32FC1, left_undist_rect_map_x, left_undist_rect_map_y);
initUndistortRectifyMap(right_cam_matrix, right_dist_coeffs, R2, P2, frame_size, CV_32FC1, right_undist_rect_map_x, right_undist_rect_map_y);
cv::remap(left_frame, left_undist_rect, left_undist_rect_map_x, left_undist_rect_map_y, CV_INTER_CUBIC, BORDER_CONSTANT, 0);
cv::remap(right_frame, right_undist_rect, right_undist_rect_map_x, right_undist_rect_map_y, CV_INTER_CUBIC, BORDER_CONSTANT, 0);
cv::Mat imgDisparity32F = Mat( left_undist_rect.rows, left_undist_rect.cols, CV_32F );
StereoBM sbm(StereoBM::BASIC_PRESET,80,5);
sbm.state->preFilterSize = 15;
sbm.state->preFilterCap = 20;
sbm.state->SADWindowSize = 11;
sbm.state->minDisparity = 0;
sbm.state->numberOfDisparities = 80;
sbm.state->textureThreshold = 0;
sbm.state->uniquenessRatio = 8;
sbm.state->speckleWindowSize = 0;
sbm.state->speckleRange = 0;
// Compute disparity
sbm(left_undist_rect, right_undist_rect, imgDisparity32F, CV_32F );
// Compute world coordinates from the disparity image
cv::Mat XYZ(disparity32F.size(),CV_32FC3);
reprojectImageTo3D(disparity32F, XYZ, Q, false, CV_32F);
print_3D_points(disparity32F, XYZ);
[EDIT]
Adding the code used to compute 3D coords from disparity:
cv::Vec3f *StereoFrame::compute_3D_world_coordinates(int row, int col,
shared_ptr<StereoParameters> stereo_params_sptr){
cv::Mat Q_32F;
stereo_params_sptr->Q_sptr->convertTo(Q_32F,CV_32F);
cv::Mat_<float> vec(4,1);
vec(0) = col;
vec(1) = row;
vec(2) = this->disparity_sptr->at<float>(row,col);
// Discard points with 0 disparity
if(vec(2)==0) return NULL;
vec(3)=1;
vec = Q_32F*vec;
vec /= vec(3);
// Discard points that are too far from the camera, and thus are highly
// unreliable
if(abs(vec(0))>10 || abs(vec(1))>10 || abs(vec(2))>10) return NULL;
cv::Vec3f *point3f = new cv::Vec3f();
(*point3f)[0] = vec(0);
(*point3f)[1] = vec(1);
(*point3f)[2] = vec(2);
return point3f;
}
Your code seems fine to me. It could be a bug with the reprojectImageTo3D. Try to replace it with the following code (which has the same role):
cv::Mat_<cv::Vec3f> XYZ(disparity32F.rows,disparity32F.cols); // Output point cloud
cv::Mat_<float> vec_tmp(4,1);
for(int y=0; y<disparity32F.rows; ++y) {
for(int x=0; x<disparity32F.cols; ++x) {
vec_tmp(0)=x; vec_tmp(1)=y; vec_tmp(2)=disparity32F.at<float>(y,x); vec_tmp(3)=1;
vec_tmp = Q*vec_tmp;
vec_tmp /= vec_tmp(3);
cv::Vec3f &point = XYZ.at<cv::Vec3f>(y,x);
point[0] = vec_tmp(0);
point[1] = vec_tmp(1);
point[2] = vec_tmp(2);
}
}
I never used reprojectImageTo3D, however I am using successfully code similar to the snippet above.
[Initial answer]
As it is explained in the documentation for StereoBM, if you request a CV_16S disparity map, you have to divide each disparity value by 16 before using them.
Hence, you should convert the disparity map as follows before using it:
imgDisparity16S.convertTo( imgDisparity32F, CV_32F, 1./16);
You can also directly request a CV_32F disparity map from the StereoBM structure, in which case you directy get the true disparities.