I am trying to store a IPL_DEPTH_8U, 3 channel image into an array so that I can store 100 images in memory.
To initialise my 4D array I used the following code (rows,cols,channel,stored):
int size[] = { 324, 576, 3, 100 };
CvMatND* cvImageBucket; = cvCreateMatND(3, size, CV_8U);
I then created a matrix and converted the image into the matrix
CvMat *matImage = cvCreateMat(Image->height,Image->width,CV_8UC3 );
cvConvert(Image, matImage );
How would I / access the CvMatND to copy the CvMat into it at the position of stored?
e.g. cvImageBucket(:,:,:,0) = matImage; // copied first image into array
You've tagged this as both C and C++. If you want to work in C++, you could use the (in my opinion) simpler cv::Mat structure to store each of the images, and then use these to populate a vector with all the images.
For example:
std::vector<cv::Mat> imageVector;
cv::Mat newImage;
newImage = getImage(); // where getImage() returns the next image,
// or an empty cv::Mat() if there are no more images
while (!newImage.empty())
{
// Add image to vector
imageVector.push_back(image);
// get next image
newImage = getImage();
}
I'm guessing something similar to:
for ith matImage
memcpy((char*)cvImageBucket->data+i*size[0]*size[1]*size[2],(char*)matImage->data,size[0]*size[1]*size[2]);
Although I agree with #Chris that it is best to use vector<Mat> rather than a 4D matrix, this answer is just to be a reference for those who really need to use 4D matrices in OpenCV (even though it is a very unsupported, undocumented and unexplored thing with so little available online and claimed to be working just fine!).
So, suppose you filled a vector<Mat> vec with 2D or 3D data which can be CV_8U, CV_32F etc.
One way to create a 4D matrix is
vector<int> dims = {(int)vec.size(), vec[0].rows, vec[0].cols};
Mat m(dims, vec[0].type(), &vec[0]);
However, this method fails when the vector is not continuous which is typically the case for big matrices. If you do this for a discontinuous matrix, you will get a segmentation fault or bad access error when you would like to use the matrix (i.e. copying, cloning, etc). To overcome this issue, you can copy matrices of the vector one by one into the 4D matrix as follows:
Mat m2(dims, vec[0].type());
for (auto i = 0; i < vec.size(); i++){
vec[i].copyTo(temp.at<Mat>(i));
}
Notice that both methods require the matrices to be the same resolution. Otherwise, you may get undesired results or errors.
Also, notice that you can always use for loops but it is generally not a good idea to use them when you can vectorize.
Related
Please let me know if this question is too broad, but I am trying to learn some c++ so I thought it would be a good idea to try and recreate some opencv functions.
I am still grabbing the frames or reading the image with opencv's API, but I then want to feed the cv::Mat into my custom function(s), where I modify its data and return it to display it. (For example a function to blur the image, where I pass the original Mat to a padding function, then the output of that to a fn that convolves the padded image with the blurring kernel, and returns the Mat to cv for displaying)
I am a little confused as to what the best (or right) way to do this is. OpenCV functions use a function argument as the return matrix ( cv_foo(cv::Mat src_frame, cv::Mat dst_frame) ) but I am not entirely clear how this works, so I have tried a more familiar approach, something like
cv::Mat my_foo(cv::Mat src_frame) {
// do processing on src_frame data
return dst_frame;
}
where to access the data from src_frame I use uchar* framePtr = frame.data; and to create the dst_frame I followed this suggestion
cv::Mat dst_frame = cv::Mat(n_rows, n_cols, CV_8UC3);
memcpy(dst_frame.data, &new_data_array, sizeof(new_data_array));
I have however encountered various segmentation faults that I find hard to debug, as it seems they occur almost at random (could this be due to the way I am handling the memory management with frame.data or something like that?).
So to come back to my original question, what is the best way to access, modify and pass the data from a cv::Mat in the most consistent way?
I think what would make the most intuitive sense to me (coming from numpy) would be to extract the data array from the original Mat, use that throughout my processing and then repackage it into a Mat before displaying, which would also allow me to feed any custom array into the processing without having to turn it into a Mat, but I am not sure how to best do that (or if it is the right approach).
Thank you!
EDIT:
I will try to highlight the main bug in my code.
One of the functions I am trying to replicate is a conversion from bgr to greyscale, my code looks like this
cv::Mat bgr_to_greyscale(cv::Mat& frame){
int n_rows = frame.rows;
int n_cols = frame.cols;
uchar* framePtr = frame.data;
int channels = frame.channels();
uchar grey_array[n_rows*n_cols];
for(int i=0; i<n_rows; i++){
for(int j=0; j<n_cols; j++){
uchar pixel_b = framePtr[i*n_cols*channels + j*channels];
uchar pixel_g = framePtr[i*n_cols*channels + j*channels + 1];
uchar pixel_r = framePtr[i*n_cols*channels + j*channels + 2];
uchar pixel_grey = 0.299*pixel_r + 0.587*pixel_g + 0.144*pixel_b;
grey_array[i*n_cols + j] = pixel_grey;
}
}
cv::Mat dst_frame = cv::Mat(n_rows, n_cols, CV_8UC1, &grey_array);
return dst_frame;
}
however when I display the result of this function on a sample image I get this result: the bottom part of the image looks like random noise, how can I fix this? what exactly is going wrong in my code?
Thank you!
This question is too broad to answer in any detail, but generally a cv::Mat is a wrapper around the image data akin to the way an std::vector<int> is a wrapper around a dynamically allocated array of int values or an std::string is a wrapper around a dynamically allocated array of characters with one exception: a cv::Mat will not perform a deep copy of the image data on assignment or usage of the copy constructor.
std::vector<int> b = { 1, 2, 3, 4};
std::vector<int> a = b;
// a now contains a copy of b and a[0] = 42 will not effect b.
cv::Mat b = cv::imread( ... );
cv::Mat a = b;
// a and b now wrap the same data.
But that said, you should not be using memcpy et. al. to copy a cv::Mat ... You can make copies with clone or copyTo. From the cv documentation:
Mat F = A.clone();
Mat G;
A.copyTo(G);
I have a const std::vector<cv::Mat> containing 3 matrices (3 images), and in order to use each image further in my program I need to save them in separate matrices cv::Mat.
I know that I need to iterate over vector elements, since this vector is a list of matrices but somehow I can't manage it. At the end, I also need to push 3 matrices back to a vector.
I would appreciate if someone could help me out with this. I am still learning it.
std::vector<cv::Mat> imagesRGB;
cv::Mat imgR, imgG, imgB;
for(size_t i=0; i<imagesRGB.size(); i++)
{
imagesRGB[i].copyTo(imgR);
}
In your code, note that imagesRGB is uninitialized, and its size is 0. The for loop is not evaluated. Additionally, the copyTo method copies matrix data into another matrix (like a paste function), it is not used to store a cv::Mat into a std::vector.
Your description is not clear enough, however here's an example of what I think you might be needing. If you want to split an RGB (3 channels) image into three individual mats, you can do it using the cv::split function. If you want to merge 3 individual channels into an RGB mat, you can do that via the cv::merge function. Let's see the example using this test image:
//Read input image:
cv::Mat inputImage = cv::imread( "D://opencvImages//lena512.png" );
//Split the BGR image into its channels:
cv::Mat splitImage[3];
cv::split(inputImage, splitImage);
//show images:
cv::imshow("B", splitImage[0]);
cv::imshow("G", splitImage[1]);
cv::imshow("R", splitImage[2]);
Note that I'm already storing the channels in an array. However, if you want to store the individual mats into a std::vector, you can use the push_back method:
//store the three channels into a vector:
std::vector<cv::Mat> matVector;
for( int i = 0; i < 3; i++ ){
//get current channel:
cv::Mat currentChannel = splitImage[i];
//store into vector
matVector.push_back(currentChannel);
}
//merge the three channels back into an image:
cv::Mat mergedImage;
cv::merge(matVector, mergedImage);
//show final image:
cv::imshow("Merged Image", mergedImage);
cv::waitKey(0);
This is the result:
Suppose that I collect same size, depth and channel images/matrices into a vector. So, these images are r*c*d each and I have m of them in my vector as follows.
vector<string> imgs; --> there are m image paths in these, all are r*c*d resolution
vector<Mat> vec;
for (auto img: imgs ){
vec.push_back(cv::imread(img, COLOR_BGR)); //or gray. doesn't really matter
}
Now, I want to create a 4D Matrix. For example, in python np.array(vec) would have given me that (assuming vec is a list). I would like to the same in OpenCV c++ but I couldn't find a solution for this.
I don't want to create a 4D matrix with Mat m(dims, size, type);, then iterate through all pixels and copy the value as it is very inefficient. I would like to have a technique that will just consider vec<Mat> as 4D Mat so that it is going to be super fast. Note that I can have 100 full-resolution images.
I am using Opencv4.2 and c++ on Mac.
Thanks in advance
After many hours today, I coincidentally found an answer to my question. I will leave the answer here to have a reference for those who battle with OpenCV's documentation to find the correct answer.
vector<int> dims = {m, r, c}; //dimensions
cv::Mat m (3, &dims[0], imgs[0].type(), &imgs[0]);
This creates the 4D matrix from imgs vector where the type is one of CV_8UC1, CV_8UC3 or CV_8UC4 depending on the number of channels. The good thing is it doesn't copy the vector.
Although this is not part of the question, to access a pixel in the 4D matrix, you can do the following:
int x = v1, i = v2, j = v3, c = v4; //v1-4 are some random values within their ranges
cout << (int)m.at<Mat>(x).at<Vec3b>(i,j)[c] << " "
<< (int)imgs[x].at<Vec3b> (i,j)[c] << endl;
Both will print c-th channel of i,j-th index of x-th image.
Why does the following code segment work
int sizearray[3] = {3,4,2};
Mat OneM = Mat::ones(3,4,CV_8UC1);
Mat TwoM = Mat::ones(3,4,CV_8UC1)+1;
Mat OneTwo3D = Mat::zeros(3,sizearray,CV_8UC1);
Mat OneTwo3DPlaneM;
Range OneTwo3DRanges[] = {Range(1,3),Range(1,4),Range(1,2)};
OneTwo3DPlaneM = OneTwo3D(OneTwo3DRanges);
but if I change the ranges to
Range OneTwo3DRanges[] = {Range(1,4),Range(1,4),Range(1,2)};
it blows up.
In the working one, OneTwo3DPlaneM is apparently a 2x3x1 matrix as expected given that the "end" of a Range is exclusive (confirmed by looking at the working code/range result OneTwo3DPlaneM.size.p which is [2,3,1]). However, just increasing the first range from Range(1,3) to Range(1,4) causes an error on
OneTwo3DPlaneM = OneTwo3D(OneTwo3DRanges);
Note, what I'm trying to do, eventually, is assign an image into a slice of a 3d arrray (via OneM.copyTo(OneTwo3DPlaneM) I think). The code above is just a test for that. So, I just want to create a matrix that references a plane OneTwo3D so that, for this test, I can assign the matrices OneM to the first plane and, eventually, TwoM to the second plane (after I change the third range to point to OneTwo3D's second plane)
Given that I don't have much experience with opencv I assume I'm doing something retarded. Apologies up front.
The code below seems to work Apparently, copyTo doesn't like assigning from a 2d into a slice of a 3d array. So, as shown below, I changed the OneM and TwoM from 3x4 matrices to 3x4x1 and then it worked. I guess then the question is given an image (2d) can I just reshape it to a 3d entity with a singleton 3rd dimension to copyTo happy. I'm not sure how to do that considering that reshape only works on 2d arrays.
int sizearray1[3] = {3,4,1};
int sizearray[3] = {3,4,2};
// Mat OneM = Mat::ones(3,4,CV_8UC1);
Mat OneM = Mat::ones(3,sizearray1,CV_8UC1);
// Mat TwoM = Mat::ones(3,4,CV_8UC1)+1;
Mat TwoM = Mat::ones(3,sizearray1,CV_8UC1)+1;
Mat OneTwo3D = Mat::zeros(3,sizearray,CV_8UC1);
Mat OneTwo3DPlaneM;
Range OneTwo3DRanges[] = {Range::all(),Range::all(),Range(0,1)};
OneTwo3DPlaneM = OneTwo3D(OneTwo3DRanges);
OneM.copyTo(OneTwo3DPlaneM);
OneTwo3DRanges[2] = Range(1,2);
OneTwo3DPlaneM = OneTwo3D(OneTwo3DRanges);
TwoM.copyTo(OneTwo3DPlaneM);
I have some problems with opencv flann::Index -
I'm creating index
Mat samples = Mat::zeros(vfv_net_quie.size(),24,CV_32F);
for (int i =0; i < vfv_net_quie.size();i++)
{
for (int j = 0;j<24;j++)
{
samples.at<float>(i,j)=(float)vfv_net_quie[i].vfv[j];
}
}
cv::flann::Index flann_index(
samples,
cv::flann::KDTreeIndexParams(4),
cvflann::FLANN_DIST_EUCLIDEAN
);
flann_index.save("c:\\index.fln");
A fter that I'm tryin to load it and find nearest neiborhoods
cv::flann::Index flann_index(Mat(),
cv::flann::SavedIndexParams("c:\\index.fln"),
cvflann::FLANN_DIST_EUCLIDEAN
);
cv::Mat resps(vfv_reg_quie.size(), K, CV_32F);
cv::Mat nresps(vfv_reg_quie.size(), K, CV_32S);
cv::Mat dists(vfv_reg_quie.size(), K, CV_32F);
flann_index.knnSearch(sample,nresps,dists,K,cv::flann::SearchParams(64));
And have access violation in miniflann.cpp in line
((IndexType*)index)->knnSearch(_query, _indices, _dists, knn,
(const ::cvflann::SearchParams&)get_params(params));
Please help
You should not load the flann-file into a Mat(), as it is the place where the index is stored. It is a temporary object destroyed after the constructor was called. That's why the index isn't pointing anywhere useful when you call knnSearch().
I tried following:
cv::Mat indexMat;
cv::flann::Index flann_index(
indexMat,
cv::flann::SavedIndexParams("c:\\index.fln"),
cvflann::FLANN_DIST_EUCLIDEAN
);
resulting in:
Reading FLANN index error: the saved data size (100, 64) or type (5) is different from the passed one (0, 0), 0
which means, that the matrix has to be initialized with the correct dimensions (seems very stupid to me, as I don't necessarily know, how many elements are stored in my index).
cv::Mat indexMat(samples.size(), CV_32FC1);
cv::flann::Index flann_index(
indexMat,
cv::flann::SavedIndexParams("c:\\index.fln"),
cvflann::FLANN_DIST_EUCLIDEAN
);
does the trick.
In the accepted answer is somehow not clear and misleading why the input matrix in the cv::flann::Index constructor must have the same dimension as the matrix used for generating the saved Index. I'll elaborate on #Sau's comment with an example.
KDTreeIndex was generated using as input a cv::Mat sample, and then saved. When you load it, you must provide the same sample matrix to generate it, something like (using the templated GenericIndex interface):
cv::Mat sample(sample_num, sample_size, ... /* other params */);
cv::flann::SavedIndexParams index_params("c:\\index.fln");
cv::flann::GenericIndex<cvflann::L2<float>> flann_index(sample, index_params);
L2 is the usual Euclidean distance (other types can be found in opencv2/flann/dist.h).
Now the index can be used as shown the find the K nearest neighbours of a query point:
std::vector<float> query(sample_size);
std::vector<int> indices(K);
std::vector<float> distances(K);
flann_index.knnSearch(query, indices, distances, K, cv::flann::SearchParams(64));
The matrix indices will contain the locations of the nearest neighbours in the matrix sample, which was used at first to generate the index. That's why you need to load the saved index with the very matrix used to generate the index, otherwise the returned vector will contain indices pointing to meaningless "nearest neighbours".
In addition you get a distances matrix containing how far are the found neighbours from your query point, which you can later use to perform some inverse distance weighting, for example.
Please also note that sample_size has to match across sample matrix and query point.