Using OpenCV 3.4.1 in C++ I have a 2 dimensional Mat array object. These first few lines just explain essentially how the data are obtained to fill this Mat object ("dat")
int rows=NUMBER_OF_ROWS;
int cols=NUMBER_OF_COLS;
float* rawdata;
rawdata = new float[rows*cols];
fits_read_img(FP, TFLOAT, firstpixel, npixels, &nullval, rawdata, &anynull, &status);
Then creation of the Mat object from the raw data looks like this:
cv::Mat dat = cv::Mat(rows, cols, CV_32FC1, rawdata);
So far, so good, and I've used this Mat object "dat" for many things. But now I want to create a row average of the data. That should give me a rows x 1 vector, with every column of each row reduced into a single value, the average of all the values in that row. Something like this:
So OpenCV has a reduce() function. Here is how I've tried calling it:
std::vector<float> vec(rows);
cv::reduce(dat, vec, 1, cv::REDUCE_AVG, CV_32FC1);
which segfaults. If I go the other direction, I.E.
cv::reduce(dat, vec, 0, cv::REDUCE_AVG, CV_32FC1);
then I get a 1 x cols vector containing a column average. In fact, it gives me the exact equivalent of this:
int r, c;
float sum;
for (c=0; c<cols; c++) {
sum=0;
for (r=0; r<rows; r++) {
sum += dat.at<float>(r,c);
}
std::cout << "sum=" << ss << " avg=" << ss/r << std::endl;
}
So I could certainly use a brute-force method like this to get the row average that I want, but it seems I ought to be able to get it with a single cv::reduce() call.
What am I missing?
Related
How can we get the mean of an input RGB image(3 dimensional Mat object) so that we get a gray image? The cvtColor() function of OpenCV converts the image to gray based on a pre-existing formula. I want to get the mean of all three channels and store the resultant image in another matrix. The cv::mean() function in OpenCV returns the scalar mean of all input channels.
Were this Python, with img being a RGB image, img.mean(2) would get me what I want. Successive calls of the addWeighted() function and using gray= blue/3.0 + red/3.0 +green/3.0 [ After splitting channels] yielded different results when compared with Python.
Is there anything analogous to img.mean(2) in C++ or the OpenCV library of C++?
Is there anything analogous to img.mean(2) in C++ or the OpenCV library of C++?
No, but you can easily compute that. There are a few ways of doing it:
Loop over all the image, and set each value as the mean of the input pixel values. Take care of computing the intermediate values for the mean on a type with more capacity and accuracy than uchar (here I used double) or you may end up with wrong results. You can also optimize the code further, e.g. see this question and its answers. You just need to change the function computed in the inner loop to compute the mean.
Use reduce. You can reshape you 3 channel matrix of size rows x cols to be a matrix of shape ((rows*cols) x 3), and then you can use the reduce operation with parameter REDUCE_AVG to compute the average row-wise. Then reshape the matrix to correct size. reshape operation is very fast, since you just modify the header without affecting the stored data.
Use matrix operations to sum channels. You can use split to get the matrix for each channel, and sum them. Take care to not saturate your values while summing up! (Thanks to beaker for this one.)
You can see that the first approach is faster with small matrices, but as soon as the size increase, the second approach performs much better since you take advantage of OpenCV optimizations.
The third approach works surprisingly well (thanks to matrix expressions).
Some numbers, time in ms. Time may vary on you computer depending on OpenCV optimizations enabled. Run in release!
Size : 10x10 100x100 1000x1000 10000x10000
Loop : 0.0077 0.3625 34.82 3456.71
Reduce: 1.44 1.42 8.88 716.75
Split : 0.1158 0.0656 2.26304 246.476
Code:
#include <opencv2\opencv.hpp>
#include <iostream>
using namespace std;
using namespace cv;
int main()
{
Mat3b img(1000, 1000);
randu(img, Scalar(0, 0, 0), Scalar(10, 10, 10));
{
double tic = double(getTickCount());
Mat1b mean_img(img.rows, img.cols, uchar(0));
for (int r = 0; r < img.rows; ++r) {
for (int c = 0; c < img.cols; ++c) {
const Vec3b& v = img(r, c);
mean_img(r, c) = static_cast<uchar>(round((double(v[0]) + double(v[1]) + double(v[2])) / 3.0));
}
}
double toc = (double(getTickCount()) - tic) * 1000.0 / getTickFrequency();
cout << "Loop: " << toc << endl;
}
{
double tic = double(getTickCount());
Mat1b mean_img2 = img.reshape(1, img.rows*img.cols);
reduce(mean_img2, mean_img2, 1, REDUCE_AVG);
mean_img2 = mean_img2.reshape(1, img.rows);
double toc = (double(getTickCount()) - tic) * 1000.0 / getTickFrequency();
cout << "Reduce: " << toc << endl;
}
{
double tic = double(getTickCount());
vector<Mat1b> planes;
split(img, planes);
Mat1b mean_img3;
if (img.channels() == 3) {
mean_img3 = (planes[0] + planes[1] + planes[2]) / 3.0;
}
double toc = (double(getTickCount()) - tic) * 1000.0 / getTickFrequency();
cout << "Split: " << toc << endl;
}
getchar();
return 0;
}
mean()
Calculates an average (mean) of array elements.
C++: Scalar mean(InputArray src, InputArray mask=noArray())
Python: cv2.mean(src[, mask]) → retval
C: CvScalar cvAvg(const CvArr* arr, const CvArr* mask=NULL )
Python: cv.Avg(arr, mask=None) → scalar
Parameters:
src – input array that should have from 1 to 4 channels so that the result can be stored in Scalar_ .
mask – optional operation mask.
The function mean calculates the mean value M of array elements, independently for each channel, and return it:
When all the mask elements are 0’s, the functions return Scalar::all(0) .
Also check this answer how to calculate and use cvMat mean value
I am novice in OpenCV. Recently, I have troubles finding OpenCV functions to convert from Mat to Array. I researched with .ptr and .at methods available in OpenCV APIs, but I could not get proper data. I would like to have direct conversion from Mat to Array(if available, if not to Vector). I need OpenCV functions because the code has to be undergo high level synthesis in Vivado HLS. Please help.
If the memory of the Mat mat is continuous (all its data is continuous), you can directly get its data to a 1D array:
std::vector<uchar> array(mat.rows*mat.cols*mat.channels());
if (mat.isContinuous())
array = mat.data;
Otherwise, you have to get its data row by row, e.g. to a 2D array:
uchar **array = new uchar*[mat.rows];
for (int i=0; i<mat.rows; ++i)
array[i] = new uchar[mat.cols*mat.channels()];
for (int i=0; i<mat.rows; ++i)
array[i] = mat.ptr<uchar>(i);
UPDATE: It will be easier if you're using std::vector, where you can do like this:
std::vector<uchar> array;
if (mat.isContinuous()) {
// array.assign(mat.datastart, mat.dataend); // <- has problems for sub-matrix like mat = big_mat.row(i)
array.assign(mat.data, mat.data + mat.total()*mat.channels());
} else {
for (int i = 0; i < mat.rows; ++i) {
array.insert(array.end(), mat.ptr<uchar>(i), mat.ptr<uchar>(i)+mat.cols*mat.channels());
}
}
p.s.: For cv::Mats of other types, like CV_32F, you should do like this:
std::vector<float> array;
if (mat.isContinuous()) {
// array.assign((float*)mat.datastart, (float*)mat.dataend); // <- has problems for sub-matrix like mat = big_mat.row(i)
array.assign((float*)mat.data, (float*)mat.data + mat.total()*mat.channels());
} else {
for (int i = 0; i < mat.rows; ++i) {
array.insert(array.end(), mat.ptr<float>(i), mat.ptr<float>(i)+mat.cols*mat.channels());
}
}
UPDATE2: For OpenCV Mat data continuity, it can be summarized as follows:
Matrices created by imread(), clone(), or a constructor will always be continuous.
The only time a matrix will not be continuous is when it borrows data (except the data borrowed is continuous in the big matrix, e.g. 1. single row; 2. multiple rows with full original width) from an existing matrix (i.e. created out of an ROI of a big mat).
Please check out this code snippet for demonstration.
Can be done in two lines :)
Mat to array
uchar * arr = image.isContinuous()? image.data: image.clone().data;
uint length = image.total()*image.channels();
Mat to vector
cv::Mat flat = image.reshape(1, image.total()*image.channels());
std::vector<uchar> vec = image.isContinuous()? flat : flat.clone();
Both work for any general cv::Mat.
Explanation with a working example
cv::Mat image;
image = cv::imread(argv[1], cv::IMREAD_UNCHANGED); // Read the file
cv::namedWindow("cvmat", cv::WINDOW_AUTOSIZE );// Create a window for display.
cv::imshow("cvmat", image ); // Show our image inside it.
// flatten the mat.
uint totalElements = image.total()*image.channels(); // Note: image.total() == rows*cols.
cv::Mat flat = image.reshape(1, totalElements); // 1xN mat of 1 channel, O(1) operation
if(!image.isContinuous()) {
flat = flat.clone(); // O(N),
}
// flat.data is your array pointer
auto * ptr = flat.data; // usually, its uchar*
// You have your array, its length is flat.total() [rows=1, cols=totalElements]
// Converting to vector
std::vector<uchar> vec(flat.data, flat.data + flat.total());
// Testing by reconstruction of cvMat
cv::Mat restored = cv::Mat(image.rows, image.cols, image.type(), ptr); // OR vec.data() instead of ptr
cv::namedWindow("reconstructed", cv::WINDOW_AUTOSIZE);
cv::imshow("reconstructed", restored);
cv::waitKey(0);
Extended explanation:
Mat is stored as a contiguous block of memory, if created using one of its constructors or when copied to another Mat using clone() or similar methods. To convert to an array or vector we need the address of its first block and array/vector length.
Pointer to internal memory block
Mat::data is a public uchar pointer to its memory.
But this memory may not be contiguous. As explained in other answers, we can check if mat.data is pointing to contiguous memory or not using mat.isContinous(). Unless you need extreme efficiency, you can obtain a continuous version of the mat using mat.clone() in O(N) time. (N = number of elements from all channels). However, when dealing images read by cv::imread() we will rarely ever encounter a non-continous mat.
Length of array/vector
Q: Should be row*cols*channels right?
A: Not always. It can be rows*cols*x*y*channels.
Q: Should be equal to mat.total()?
A: True for single channel mat. But not for multi-channel mat
Length of the array/vector is slightly tricky because of poor documentation of OpenCV. We have Mat::size public member which stores only the dimensions of single Mat without channels. For RGB image, Mat.size = [rows, cols] and not [rows, cols, channels]. Mat.total() returns total elements in a single channel of the mat which is equal to product of values in mat.size. For RGB image, total() = rows*cols. Thus, for any general Mat, length of continuous memory block would be mat.total()*mat.channels().
Reconstructing Mat from array/vector
Apart from array/vector we also need the original Mat's mat.size [array like] and mat.type() [int]. Then using one of the constructors that take data's pointer, we can obtain original Mat. The optional step argument is not required because our data pointer points to continuous memory. I used this method to pass Mat as Uint8Array between nodejs and C++. This avoided writing C++ bindings for cv::Mat with node-addon-api.
References:
Create memory continuous Mat
OpenCV Mat data layout
Mat from array
Here is another possible solution assuming matrix have one column( you can reshape original Mat to one column Mat via reshape):
Mat matrix= Mat::zeros(20, 1, CV_32FC1);
vector<float> vec;
matrix.col(0).copyTo(vec);
None of the provided examples here work for the generic case, which are N dimensional matrices. Anything using "rows" assumes theres columns and rows only, a 4 dimensional matrix might have more.
Here is some example code copying a non-continuous N-dimensional matrix into a continuous memory stream - then converts it back into a Cv::Mat
#include <iostream>
#include <cstdint>
#include <cstring>
#include <opencv2/opencv.hpp>
int main(int argc, char**argv)
{
if ( argc != 2 )
{
std::cerr << "Usage: " << argv[0] << " <Image_Path>\n";
return -1;
}
cv::Mat origSource = cv::imread(argv[1],1);
if (!origSource.data) {
std::cerr << "Can't read image";
return -1;
}
// this will select a subsection of the original source image - WITHOUT copying the data
// (the header will point to a region of interest, adjusting data pointers and row step sizes)
cv::Mat sourceMat = origSource(cv::Range(origSource.size[0]/4,(3*origSource.size[0])/4),cv::Range(origSource.size[1]/4,(3*origSource.size[1])/4));
// correctly copy the contents of an N dimensional cv::Mat
// works just as fast as copying a 2D mat, but has much more difficult to read code :)
// see http://stackoverflow.com/questions/18882242/how-do-i-get-the-size-of-a-multi-dimensional-cvmat-mat-or-matnd
// copy this code in your own cvMat_To_Char_Array() function which really OpenCV should provide somehow...
// keep in mind that even Mat::clone() aligns each row at a 4 byte boundary, so uneven sized images always have stepgaps
size_t totalsize = sourceMat.step[sourceMat.dims-1];
const size_t rowsize = sourceMat.step[sourceMat.dims-1] * sourceMat.size[sourceMat.dims-1];
size_t coordinates[sourceMat.dims-1] = {0};
std::cout << "Image dimensions: ";
for (int t=0;t<sourceMat.dims;t++)
{
// calculate total size of multi dimensional matrix by multiplying dimensions
totalsize*=sourceMat.size[t];
std::cout << (t>0?" X ":"") << sourceMat.size[t];
}
// Allocate destination image buffer
uint8_t * imagebuffer = new uint8_t[totalsize];
size_t srcptr=0,dptr=0;
std::cout << std::endl;
std::cout << "One pixel in image has " << sourceMat.step[sourceMat.dims-1] << " bytes" <<std::endl;
std::cout << "Copying data in blocks of " << rowsize << " bytes" << std::endl ;
std::cout << "Total size is " << totalsize << " bytes" << std::endl;
while (dptr<totalsize) {
// we copy entire rows at once, so lowest iterator is always [dims-2]
// this is legal since OpenCV does not use 1 dimensional matrices internally (a 1D matrix is a 2d matrix with only 1 row)
std::memcpy(&imagebuffer[dptr],&(((uint8_t*)sourceMat.data)[srcptr]),rowsize);
// destination matrix has no gaps so rows follow each other directly
dptr += rowsize;
// src matrix can have gaps so we need to calculate the address of the start of the next row the hard way
// see *brief* text in opencv2/core/mat.hpp for address calculation
coordinates[sourceMat.dims-2]++;
srcptr = 0;
for (int t=sourceMat.dims-2;t>=0;t--) {
if (coordinates[t]>=sourceMat.size[t]) {
if (t==0) break;
coordinates[t]=0;
coordinates[t-1]++;
}
srcptr += sourceMat.step[t]*coordinates[t];
}
}
// this constructor assumes that imagebuffer is gap-less (if not, a complete array of step sizes must be given, too)
cv::Mat destination=cv::Mat(sourceMat.dims, sourceMat.size, sourceMat.type(), (void*)imagebuffer);
// and just to proof that sourceImage points to the same memory as origSource, we strike it through
cv::line(sourceMat,cv::Point(0,0),cv::Point(sourceMat.size[1],sourceMat.size[0]),CV_RGB(255,0,0),3);
cv::imshow("original image",origSource);
cv::imshow("partial image",sourceMat);
cv::imshow("copied image",destination);
while (cv::waitKey(60)!='q');
}
Instead of getting image row by row, you can put it directly to an array. For CV_8U type image, you can use byte array, for other types check here.
Mat img; // Should be CV_8U for using byte[]
int size = (int)img.total() * img.channels();
byte[] data = new byte[size];
img.get(0, 0, data); // Gets all pixels
byte * matToBytes(Mat image)
{
int size = image.total() * image.elemSize();
byte * bytes = new byte[size]; //delete[] later
std::memcpy(bytes,image.data,size * sizeof(byte));
}
You can use iterators:
Mat matrix = ...;
std::vector<float> vec(matrix.begin<float>(), matrix.end<float>());
cv::Mat m;
m.create(10, 10, CV_32FC3);
float *array = (float *)malloc( 3*sizeof(float)*10*10 );
cv::MatConstIterator_<cv::Vec3f> it = m.begin<cv::Vec3f>();
for (unsigned i = 0; it != m.end<cv::Vec3f>(); it++ ) {
for ( unsigned j = 0; j < 3; j++ ) {
*(array + i ) = (*it)[j];
i++;
}
}
Now you have a float array. In case of 8 bit, simply change float to uchar, Vec3f to Vec3b and CV_32FC3 to CV_8UC3.
If you know that your img is 3 channel, than you can try this code
Vec3b* dados = new Vec3b[img.rows*img.cols];
for (int i = 0; i < img.rows; i++)
for(int j=0;j<img.cols; j++)
dados[3*i*img.cols+j] =img.at<Vec3b>(i,j);
If you wanna check the (i,j) vec3b you can write
std::cout << (Vec3b)img.at<Vec3b>(i,j) << std::endl;
std::cout << (Vec3b)dados[3*i*img.cols+j] << std::endl;
Since answer above is not very accurate as mentioned in its comments but its "edit queue is full", I have to add correct one-liners.
Mat(uchar, 1 channel) to vector(uchar):
std::vector<uchar> vec = (image.isContinuous() ? image : image.clone()).reshape(1, 1); // data copy here
vector(any type) to Mat(the same type):
Mat m(vec, false); // false(by default) -- do not copy data
Say I need to find a specific element in a cv::Mat which can be a row vector in my case (though for more general case Mat can be more than one dimension).
The data type of the target can be as simple as char, int, double etc.
There is an existing post: How to find if an item is present in a std::vector? which explained how to find the element in a std::vector. Therefore, one way to do this can be: 1) converting the cv::Mat to std::vector; 2) used the method in the post to find the element.
However, I need to do this searching operation hundreds of times per row. When I have hundreds of rows need to be processed, the performance can be an issue.
I am wondering how's the performance of above method (convert + search) and is there any more efficient way to do this (maybe search element directly using cv::Mat without conversion)?
p.s: Here is a post for Converting a row of cv::Mat to std::vector
Combining those two answers and depending on the mat type (here CV_64F) you get:
bool findValue(const cv::Mat &mat, double value) {
for(int i = 0;i < mat.rows;i++) {
const double* row = mat.ptr<double>(i);
if(std::find(row, row + mat.cols, value) != row + mat.cols)
return true;
}
return false;
}
(see find docs for more information). Of course first converting mat row to a vector and then using std::find on that vector is slower than using find directly on pointer to a row array.
EDIT: After some more research, it is not quite hard to develop a generic version:
template <class T>
bool findValue(const cv::Mat &mat, T value) {
for(int i = 0;i < mat.rows;i++) {
const T* row = mat.ptr<T>(i);
if(std::find(row, row + mat.cols, value) != row + mat.cols)
return true;
}
return false;
}
I tested it on more complex data types:
cv::Mat matDouble = cv::Mat::zeros(10, 10, CV_64F);
cv::Mat matRGB = cv::Mat(10, 10, CV_8UC3, cv::Scalar(255, 255, 255));
std::cout << findValue(matDouble, 0.0) << std::endl;
std::cout << findValue(matDouble,1.0) << std::endl;
std::cout << findValue(matRGB, cv::Scalar(255, 255, 255)) << std::endl;
std::cout << findValue(matRGB, cv::Scalar(255, 255, 254)) << std::endl;
And what surprised me the output is:
1
0
0 // should be 1, right?
0
The problem is with the cv::Scalar size structure. No matter of the version of the constructor we're using (ie oone, two, three or four arguments) the size is... constant. This is no so surprising cause this is still the same structure, on my machine the size is 32 bytes (by default cv::Scalar is type of double so on my machine double is 8 bytes and 4 * 8 = 32). So the find goes strictly wrong, cause it assumes size of the element in the array as 32 bytes and it should be 3 bytes.
So don't use std::find with cv::Scalar! However it works with the primitive data types remarkable well and efficient.
EDIT2 (after berak's comment):
Yes, you can use cv::Vec3b with find and it seems working well although it have not done more testing than simply correct test:
cv::Mat matRGB = cv::Mat(10, 10, CV_8UC3, cv::Scalar(255, 255, 255));
std::cout << findValue(matRGB, cv::Vec3b(255, 255, 255)) << std::endl;
std::cout << findValue(matRGB, cv::Vec3b(255, 255, 254)) << std::endl;
(still you have to use Scalar in the Mat constructor, but it does not matter and the Mat is properly initialized). Now the output is as expected:
1
0
I have a matrix of descriptors from FREAK description extraction where each row is a descriptor of 64 elements.
I need to create a vector <int*> from this matrix due to the system requirements. I tried this so far:
Mat _descriptors;
std::vector<int*> descriptors;
int row;
for (int i=0; i<_descriptors.rows;i++)
{
row =(int) _descriptors.row(i).data;
descriptors.push_back( & row );
}
Is this correct or is there a better way for this?
All the values in descriptors will point to the variable row on the stack with this code.
Looking at the definition of a opencv Mat, row returns by value:
// returns a new matrix header for the specified row
Mat row(int y) const;
Accessing the data in _descriptors directly and stepping with provided stride member variable step should work however:
Mat _descriptors;
std::vector<int*> descriptors;
for (int i=0; i<_descriptors.rows;i++)
{
descriptors.push_back((int*)(_descriptors.data + i * _descriptors.step));
}
When you are creating a new object for the vector descriptors, you are creating an object with the address of the row variable and not it's actual value. To store the value, if it is this what you want, the code should look like this.
Mat _descriptors;
std::vector<int> descriptors;
int row;
for (int i=0; i<_descriptors.rows;i++)
{
row =(int) _descriptors.row(i).data;
descriptors.push_back( row );
}
I also change the vector<int*> descriptors to vector<int> descriptors.
When iterating through a cv::Mat to output all of its values, you could do the following:
for (int r = 0; r < t.rows; r++)
{
for (int c = 0; c < t.cols; c++)
{
std::cout << t.at<float>(r,c) << ", ";
}
std::cout << std::endl << std::endl;
}
However, if the information stored in that array is not of type float, this will crash. How do you enforce the type of this matrix?
for 2 Mats nonFloatMat and floatMat when the first is the source matrix and the last is the destination matrix do
nonFloatMat.convertTo(floatMat, CV_32F); //if its a color image use CV_32FC3, or
// CV_32FC4 if its RGBA
convertTo's reference can be found here.
Edit:
and by the way, the code will work if you'll change the float to the correct concrete type of the matrix ie if the mat is a CV_8U you can do t.at<char>(r,c) etc.
While Boaz correctly answers your questions, converting your matrix adds a new problem Your function will allocate some new memory, will convert data, and then will perform calculations.
This adds a significant overhead for your function. The solution is to make a branch for every data type you suppose to support. This is the way OpenCV handles multiple data types (of course, it's a bit smarter than if-else)
if(mat.type()==CV_32FC1)
{
}
else if(mat.type()==CV_32FC3)
{
}
else if(mat.type()==CV_8UC3)
{
}
...
another solution, if your only aim is to print the matrix is
cout << myMat << endl;
Isn't it great?
You can also do this with the template wrapper class called Mat_.
So, to "force" the type of the matrix to float you could do the following:
Mat_<float> A(100, 100);
Also, another way to ensure a matrix is of a certain type (this is done quite a bit in the OpenCV source) is to use an assertion.
Like this:
void functionOnlyUsesFloatMats(const Mat& src)
{
CV_Assert( src.type() == CV_32FC1 );
// now you can be assured that only single-channel float matrices reach this code...
}
Or, you could check the type before proceeding, and convert if needed:
Mat floatMat;
if(src.type() != CV_32FC1)
{
src.convertTo(floatMat, CV_32F);
}
else
{
floatMat = src;
}
// proceed with float calculations...