I'm trying to convert an OpenCV 3-channel Mat to a 3D Eigen Tensor.
So far, I can convert 1-channel grayscale Mat by:
cv::Mat mat = cv::imread("/image/path.png", cv::IMREAD_GRAYSCALE);
Eigen::MatrixXd myMatrix;
cv::cv2eigen(mat, myMatrix);
My attempt to convert a BGR mat to a Tensor have been:
cv::Mat mat = cv::imread("/image/path.png", cv::IMREAD_COLOR);
Eigen::MatrixXd temp;
cv::cv2eigen(mat, temp);
Eigen::Tensor<double, 3> myTensor = Eigen::TensorMap<Eigen::Tensor<double, 3>>(temp.data(), 3, mat.rows, mat.cols);
However, I'm getting the following error :
libc++abi.dylib: terminating with uncaught exception of type cv::Exception: OpenCV(4.1.0) /tmp/opencv-20190505-12101-14vk1fh/opencv-4.1.0/modules/core/src/matrix_wrap.cpp:1195:
error: (-215:Assertion failed) !fixedType() || ((Mat*)obj)->type() == mtype in function 'create'
in the line: cv::cv2eigen(mat, temp);
Any help is appreciated!
The answer might be disappointing for you.
After going through 12 pages, My conclusion is you have to separate the RGB to individual channel MAT and then convert to eigenmatrix. Or create your own Eigen type and opencv convert function
In OpenCV it is tested like this. It only allows a single channel greyscale image
https://github.com/daviddoria/Examples/blob/master/c%2B%2B/OpenCV/ConvertToEigen/ConvertToEigen.cxx
And in OpenCV it is implemented like this. Which dont give you much option for custom type aka cv::scalar to eigen std::vector
https://github.com/stonier/opencv2/blob/master/modules/core/include/opencv2/core/eigen.hpp
And according to this post,
https://stackoverflow.com/questions/32277887/using-eigen-array-of-arrays-for-rgb-images
I think Eigen was not meant to be used in this way (with vectors as
"scalar" types).
they also have the difficulting in dealing with RGB image in eigen.
Take note that Opencv Scalar and eigen Scalar has a different meaning
It is possible to do so if and only if you use your own datatype aka matrix
So you either choose to store the 3 channel info in 3 eigen matrix and you can use default eigen and opencv routing.
Mat src = imread("img.png",CV_LOAD_IMAGE_COLOR); //load image
Mat bgr[3]; //destination array
split(src,bgr);//split source
//Note: OpenCV uses BGR color order
imshow("blue.png",bgr[0]); //blue channel
imshow("green.png",bgr[1]); //green channel
imshow("red.png",bgr[2]); //red channel
Eigen::MatrixXd bm,gm,rm;
cv::cv2eigen(bgr[0], bm);
cv::cv2eigen(bgr[1], gm);
cv::cv2eigen(bgr[2], rm);
Or you can define your own type and write you own version of the opencv cv2eigen function
custom eigen type follow this. and it wont be pretty
https://eigen.tuxfamily.org/dox/TopicCustomizing_CustomScalar.html
https://eigen.tuxfamily.org/dox/TopicNewExpressionType.html
Rewrite your own cv2eigen_custom function similar to this
https://github.com/stonier/opencv2/blob/master/modules/core/include/opencv2/core/eigen.hpp
So good luck.
Edit
Since you need tensor. forget about cv function
Mat image;
image = imread(argv[1], CV_LOAD_IMAGE_COLOR);
Tensor<float, 3> t_3d(image.rows, image.cols, 3);
// t_3d(i, j, k) where i is row j is column and k is channel.
for (int i = 0; i < image.rows; i++)
for (int j = 0; j < image.cols; j++)
{
t_3d(i, j, 0) = (float)image.at<cv::Vec3b>(i,j)[0];
t_3d(i, j, 1) = (float)image.at<cv::Vec3b>(i,j)[1];
t_3d(i, j, 2) = (float)image.at<cv::Vec3b>(i,j)[2];
//cv ref Mat.at<data_Type>(row_num, col_num)
}
watch out for i,j as em not sure about the order. I only write the code based on reference. didnt compile for it.
Also watch out for image type to tensor type cast problem. Some times you might not get what you wanted.
this code should in principle solve your problem
Edit number 2
following the example of this
int storage[128]; // 2 x 4 x 2 x 8 = 128
TensorMap<Tensor<int, 4>> t_4d(storage, 2, 4, 2, 8);
Applied to your case is
cv::Mat frame=imread('myimg.ppm');
TensorMap<Tensor<float, 3>> t_3d(frame.data, image.rows, image.cols, 3);
problem is I'm not sure this will work or not. Even it works, you still have to figure out how the inside data is being organized so that you can get the shape correctly. Good luck
Updated answer - OpenCV now has conversion functions for Eigen::Tensor which will solve your problem. I needed this same functionality too so I made a contribution back to the project for everyone to use. See the documentation here:
https://docs.opencv.org/3.4/d0/daf/group__core__eigen.html
Note: if you want RGB order, you will still need to reorder the channels in OpenCV before converting to Eigen::Tensor
Related
Not quite understanding why this code works:
cv::Mat img = cv::imread('pic.jpg', -1);
cv::Mat padded;
std::uint16_t m = cv::getOptimalDFTSize(img.rows); // This will be 256
std::uint16_t n = cv::getOptimalDFTSize(img.cols); // This will be 256
cv::copyMakeBorder(img, padded, 0, m - img.rows, 0, n - img.cols,
cv::BORDER_CONSTANT, cv::Scalar::all(0)); // With my inputs, this effectively just copies img into padded
cv::Mat planes[] = { cv::Mat_<float>(padded),cv:: Mat::zeros(padded.size(), CV_32F) };
cv::Mat dft_img;
cv::merge(planes, 2, dft_img);
cv::dft(dft_img, dft_img);
cv::split(dft_img, planes);
But this breaks with an exception in memory:
cv::Mat img = cv::imread('pic.jpg', -1); // I know this image is 256x256
cv::Mat dft_img = cv::Mat::zeros(256,256,CV_32F); // Hard coding for simplicity atm
cv::dft(img,dft_img);
I'm having trouble understanding the documentation for dft() https://docs.opencv.org/2.4/modules/core/doc/operations_on_arrays.html#dft,
and other functions and classes for that matter.
I think it has something to do with dft_img not being a multichannel array in the second segment, but I'm lost on how to initialize such an array short of copying the first segment of code.
Secondly, when trying to access either planes[0] or planes[1] and modify their values with:
planes[0].at<double>(indexi,indexj) = 0;
I get another exception in memory, though I also see a new page that says mat.inl.hpp not found. Using Visual Studio, OpenCV 3.4.3, a novice with C++ but intermediate with signal processing, any help is appreciated.
You did not specify what exception you got, but an important point is that input of the dft function must be a floating point number, either 32 bits or 64 bits floating point number. Another point is that try not to use raw arrays if you are not comfortable with c++. I would even suggest that if using c++ is not mandotary, prefer python for OpenCV. Here is a working example dft code:
// read your image
cv::Mat img = cv::imread("a2.jpg", CV_LOAD_IMAGE_GRAYSCALE); // I know this image is 256x256
// convert it to floating point
//normalization is optional(depends on library and you I guess?)
cv::Mat floatImage;
img.convertTo(floatImage, CV_32FC1, 1.0/255.0);
// create a placeholder Mat variable to hold output of dft
std::vector<cv::Mat> dftOutputs;
dftOutputs.push_back(floatImage);
dftOutputs.push_back(cv::Mat::zeros(floatImage.size(), CV_32F));
cv::Mat dftOutput;
cv::merge(dftOutputs, dftOutput);
// perform dft
cv::dft(dftOutput, dftOutput);
// separate real and complex outputs back
cv::split(dftOutput, dftOutputs);
I changed code from the tutorial a little to make it easier to understand. If you would like to obtain magnitude image and such, you can follow the tutorial after split function.
How can I convert vector<Point2d> to Mat.
Mat newImg = Mat(ImagePoints);
imwrite("E:/softwares/1.8.0.71/bin/newImg.png", newImg);
This is not working since imWrite() will only accept channel 1 or 3 or 4 and the Image Points are 2 channel.
I am using OpenCV version 3.
Here is the answer:
Dont worry about the type casting. Using integers in double. But this is just to give the gist of the solution.
std::vector< cv::Point2d> points;
for(int i =0; i < 10; i++)
{
points.push_back(cv::Point2d(i,i));
}
cv::Mat_<cv::Point2d> matrix(points);
std::cout<<matrix.at<cv::Point2d>(1);
But if you want to save this Mat then use XML. Imwrite wont write the Mat.
I have 2 vectors (p1 and p2) of point3f variables which represent 2 3D pointclouds. In order to match these two point clouds I want to use SVD to find a transformation for this. The problem is that SVD requires a matrix (p1*p2 transpose). My question is how do I convert a vector of size Y to a Yx3 matrix?
I tried cv::Mat p1Matrix(p1) but this gives me a row vector with two dimensions.I also found fitLine but I think this only works for 2D.
Thank you in advance.
How about something like:
cv::Mat p1copy(3, p1.size(), CV_32FC1);
for (size_t i = 0, end = p1.size(); i < end; ++i) {
p1copy.at<float>(0, i) = p1[i].x;
p1copy.at<float>(1, i) = p1[i].y;
p1copy.at<float>(2, i) = p1[i].z;
}
If this gives you the desired result, you can make the code faster by using a pointer instead of the rather slow at<>() function.
I use reshape function for convert vector of points to Mat.
vector<Point3f> P1,P2;
Point3f c1,c2;//center of two set
... //data association for two set
Mat A=Mat(P1).reshape(1).t();
Mat B=Mat(P2).reshape(1).t();
Mat AA,BB,CA,CB;
repeat(Mat(c1),1,P1.size(),CA);
repeat(Mat(c2),1,P2.size(),CB);
AA=A-CA;
BB=B-CB;
Mat H=AA*BB.t();
SVD svd(H);
Mat R_;
transpose(svd.u*svd.vt,R_);
if(determinant(R_)<0)
R_.at<float>(0,2)*=-1,R_.at<float>(1,2)*=-1,R_.at<float>(2,2)*=-1;
Mat t=Mat(c2)-R_*Mat(c1);
I want to ask if it is possible to access single channel matrix using img.at<T>(y, x) instead using img.ptr<T>(y, x)[0]
In the example below, I create a simple program to copy an image to another
cv::Mat inpImg = cv::imread("test.png");
cv::Mat img;
inpImg.convertTo(img, CV_8UC1); // single channel image
cv::Mat outImg(img.rows, img.cols, CV_8UC1);
for(int a = 0; a < img.cols; a++)
for(int b = 0; b < img.rows; b++)
outImg.at<uchar>(b, a) = img.at<uchar>(b, a); // This is wrong
cv::imshow("Test", outImg);
The shown result was wrong, but if I change it to
outImg.ptr<uchar>(b, a)[0] = img.ptr<uchar>(b, a)[0];
The result was correct.
I'm quite puzzled since using img.at<T>(y, x) should also be okay. I also tried with 32FC1 and float, the result is similar.
Although I know you already found it, the real reason - buried nicely in the documentation - is that cv::convertTo ignores the number of channels implied by the output type, so when you do this:
inpImg.convertTo(img, CV_8UC1);
And, assuming your input image has three channels, you actually end up with a CV_8UC3 format, which explains why your initial workaround was successful - effectively, you only took a single channel by doing this:
outImg.ptr<uchar>(b, a)[0] // takes the first channel of a CV_8UC3
This only worked by accident as the pixel should have been accessed like this:
outImg.ptr<Vec3b>(b, a)[0] // takes the blue channel of a CV_8UC3
As the data is still packed uchar in both cases, the effective reinterpretation happened to work.
As you noted, you can either convert to greyscale on loading:
cv::imread("test.png", CV_LOAD_IMAGE_GRAYSCALE)
Or, you can convert explicitly:
cv::cvtColor(inpImg, inpImg, CV_BGR2GRAY);
I am using Ubuntu 12.04 and OpenCV 2
I have written the following code :
IplImage* img =0;
img = cvLoadImage("nature.jpg");
if(img != 0)
{
Mat Img_mat(img);
std::vector<Mat> RGB;
split(Img_mat, RGB);
int data = (RGB[0]).at<int>(i,j)); /*Where i, j are inside the bounds of the matrix size .. i have checked this*/
}
The problem is I am getting negative values and very large values in the data variable. I think I have made some mistake somewhere. Can you please point it out.
I have been reading the documentation (I have not finished it fully.. it is quite large. ) But from what I have read, this should work. But it isnt. What is going wrong here?
Img_mat is a 3 channeled image. Each channel consists of pixel values uchar in data type.
So with split(Img_mat, BGR) the Img_mat is split into 3 planes of blue, green and red which are collectively stored in a vector BGR. So BGR[0] is the first (blue) plane with uchar data type pixels...hence it will be
int dataB = (int)BGR[0].at<uchar>(i,j);
int dataG = (int)BGR[1].at<uchar>(i,j);
so on...
You have to specify the correct type for cv::Mat::at(i,j). You are accessing the pixel as int, while it should be a vector of uchar. Your code should look something like this:
IplImage* img = 0;
img = cvLoadImage("nature.jpg");
if(img != 0)
{
Mat Img_mat(img);
std::vector<Mat> BGR;
split(Img_mat, BGR);
Vec3b data = BGR[0].at<Vec3b>(i,j);
// data[0] -> blue
// data[1] -> green
// data[2] -> red
}
Why are you loading an IplImage first? You are mixing the C and C++ interfaces.
Loading a cv::Mat with imread directly would be more straight-forward.
This way you can also specify the type and use the according type in your at call.