How can we get the mean of an input RGB image(3 dimensional Mat object) so that we get a gray image? The cvtColor() function of OpenCV converts the image to gray based on a pre-existing formula. I want to get the mean of all three channels and store the resultant image in another matrix. The cv::mean() function in OpenCV returns the scalar mean of all input channels.
Were this Python, with img being a RGB image, img.mean(2) would get me what I want. Successive calls of the addWeighted() function and using gray= blue/3.0 + red/3.0 +green/3.0 [ After splitting channels] yielded different results when compared with Python.
Is there anything analogous to img.mean(2) in C++ or the OpenCV library of C++?
Is there anything analogous to img.mean(2) in C++ or the OpenCV library of C++?
No, but you can easily compute that. There are a few ways of doing it:
Loop over all the image, and set each value as the mean of the input pixel values. Take care of computing the intermediate values for the mean on a type with more capacity and accuracy than uchar (here I used double) or you may end up with wrong results. You can also optimize the code further, e.g. see this question and its answers. You just need to change the function computed in the inner loop to compute the mean.
Use reduce. You can reshape you 3 channel matrix of size rows x cols to be a matrix of shape ((rows*cols) x 3), and then you can use the reduce operation with parameter REDUCE_AVG to compute the average row-wise. Then reshape the matrix to correct size. reshape operation is very fast, since you just modify the header without affecting the stored data.
Use matrix operations to sum channels. You can use split to get the matrix for each channel, and sum them. Take care to not saturate your values while summing up! (Thanks to beaker for this one.)
You can see that the first approach is faster with small matrices, but as soon as the size increase, the second approach performs much better since you take advantage of OpenCV optimizations.
The third approach works surprisingly well (thanks to matrix expressions).
Some numbers, time in ms. Time may vary on you computer depending on OpenCV optimizations enabled. Run in release!
Size : 10x10 100x100 1000x1000 10000x10000
Loop : 0.0077 0.3625 34.82 3456.71
Reduce: 1.44 1.42 8.88 716.75
Split : 0.1158 0.0656 2.26304 246.476
Code:
#include <opencv2\opencv.hpp>
#include <iostream>
using namespace std;
using namespace cv;
int main()
{
Mat3b img(1000, 1000);
randu(img, Scalar(0, 0, 0), Scalar(10, 10, 10));
{
double tic = double(getTickCount());
Mat1b mean_img(img.rows, img.cols, uchar(0));
for (int r = 0; r < img.rows; ++r) {
for (int c = 0; c < img.cols; ++c) {
const Vec3b& v = img(r, c);
mean_img(r, c) = static_cast<uchar>(round((double(v[0]) + double(v[1]) + double(v[2])) / 3.0));
}
}
double toc = (double(getTickCount()) - tic) * 1000.0 / getTickFrequency();
cout << "Loop: " << toc << endl;
}
{
double tic = double(getTickCount());
Mat1b mean_img2 = img.reshape(1, img.rows*img.cols);
reduce(mean_img2, mean_img2, 1, REDUCE_AVG);
mean_img2 = mean_img2.reshape(1, img.rows);
double toc = (double(getTickCount()) - tic) * 1000.0 / getTickFrequency();
cout << "Reduce: " << toc << endl;
}
{
double tic = double(getTickCount());
vector<Mat1b> planes;
split(img, planes);
Mat1b mean_img3;
if (img.channels() == 3) {
mean_img3 = (planes[0] + planes[1] + planes[2]) / 3.0;
}
double toc = (double(getTickCount()) - tic) * 1000.0 / getTickFrequency();
cout << "Split: " << toc << endl;
}
getchar();
return 0;
}
mean()
Calculates an average (mean) of array elements.
C++: Scalar mean(InputArray src, InputArray mask=noArray())
Python: cv2.mean(src[, mask]) → retval
C: CvScalar cvAvg(const CvArr* arr, const CvArr* mask=NULL )
Python: cv.Avg(arr, mask=None) → scalar
Parameters:
src – input array that should have from 1 to 4 channels so that the result can be stored in Scalar_ .
mask – optional operation mask.
The function mean calculates the mean value M of array elements, independently for each channel, and return it:
When all the mask elements are 0’s, the functions return Scalar::all(0) .
Also check this answer how to calculate and use cvMat mean value
Related
I have a matrix img (480*640 pixel, float 64 bits) on which I apply a complex mask. After this, I need to multiply my matrix by a value but in order to win time I want to do this multiplication only on the non-zero elements because for now the multiplication is too long because I have to iterate the operation 2000 times on 2000 different matrix but with the same mask. So I found the index (on x/y axes) of the nonzero pixels which I keep in a vector of Point. But I don't succeed to use this vector to do the multplication only on the pixels indexed in this same vector.
Here is an example (with a simple mask) to understand my problem :
Mat img_temp(480, 640, CV_64FC1);
Mat img = img_temp.clone();
Mat mask = Mat::ones(img.size(), CV_8UC1);
double value = 3.56;
// Apply mask
img_temp.copyTo(img, mask);
// Finding non zero elements
vector<Point> nonZero;
findNonZero(img, nonZero);
// Previous multiplication (long because on all pixels)
Mat result = img.clone()*value;
// What I wish to do : multiplication only on non-zero pixels (not functional)
Mat result = Mat::zeros(img.size(), CV_64FC1);
result.at<int>(nonZero) = img.at(nonZero).clone() * value
What is tricky is that my pixels are not on a range (for example pixels 3, 4 and 50, 51 on a line).
Thank you in advance.
I would suggest using Mat.convertTo.
Basically, for the parameter alpha, which is the scaling factor, use the value of the mask (3.56 in your case). Make sure that the Mat is of type CV_32 or CV_64.
This will be faster than finding all non-zero pixels, saving their coordinates in a Vector and iterating (it was faster for me in Java).
Hope it helps!
Constructing vector of points will also increase computation time. I think you should consider iterating over all pixels and multiply if the pixel is not equal to zero.
Iterating will be faster if you have the matrix as raw data.
If you do
Mat result = img*value;
Instead of
Mat result = img.clone()*value;
The speed will be almost 10 times as fast
I have also tested your suggestion with vector but this is even slower than your first solution.
Below the code I used to test your firs suggestion
cv::Mat multMask(cv::Mat &img, std::vector<cv::Point> mask, double fact)
{
if (img.type() != CV_64FC1) throw "invalid format";
cv::Mat res = cv::Mat::zeros(img.size(), img.type());
int iLen = (int)mask.size();
for (int i = 0; i < iLen; i++)
{
cv::Point &p = mask[i];
((double*)(res.data + res.step.p[0] * p.y))[p.x] = ((double*)(img.data + img.step.p[0] * p.y))[p.x] * fact;
}
return res;
}
In my trials with images of 1409x900 and 960x696, it takes 2.5 ms on average to split channels of a RGB image using OpenCV in my 64-bit 6-core 3.2 GHz Windows machine.
vector<cv::Mat> channels;
cv::split(img, channels);
I found that this is almost similar amount of time for the other image processing (boolean operation + morphological opening).
Considering my code only uses an image of a channel from the splitting, I wonder if there is any faster way of extracting single channel from a RGB image, preferably with OpenCV.
UPDATE
As #DanMašek pointed out, there was another function mixChannels that can extract a single channel image from multi-channel. I've tested about 2000 images with the same sizes. mixChannels took about 1 ms on average. For now, I am satisfied with the result. But post your answer if you can make it faster.
cv::Mat channel(img.rows, img.cols, CV_8UC1);
int from_to[] = { sel_channel,0 };
mixChannels(&img, 1, &channel, 1, from_to, 1);
Two simple options come to mind here.
You mention that you perform this operation repeatedly on images captured from a camera. Therefore it is safe to assume that the images are always the same size.
Allocations of cv::Mat have a non-negligible overhead, so in this case it would be beneficial to reuse the channel Mats. (i.e. allocate the destination images when you receive the first frame, and then just overwrite the contents for subsequent frames)
The additional benefit of this approach is (quite likely) reducing memory fragmentation. This can become a real problem for 32bit code.
You mention that you're interested in only one specific channel (which the user may select arbitrarily). That means you could use cv::mixChannels, which gives you the flexibility in selecting what channels and how you want to extract them.
That means you can extract data for only a single channel, theoretically (depending on the implementation -- study the source code for more details) avoiding the overhead in extracting and/or copying the data for the channels you're not interested in.
Let's make a test program evaluating the 4 possible combinations of the approaches outlined above.
Variant 0: cv::split without reuse
Variant 1: cv::split with reuse
Variant 2: cv::mixChannels without reuse
Variant 3: cv::mixChannels with reuse
NB: I just use static for simplicity here, usually i'd make this member variable in a class that wraps the algorithm.
#include <opencv2/opencv.hpp>
#include <chrono>
#include <cstdint>
#include <iostream>
#include <vector>
#define SELECTED_CHANNEL 1
cv::Mat variant_0(cv::Mat const& img)
{
std::vector<cv::Mat> channels;
cv::split(img, channels);
return channels[SELECTED_CHANNEL];
}
cv::Mat variant_1(cv::Mat const& img)
{
static std::vector<cv::Mat> channels;
cv::split(img, channels);
return channels[SELECTED_CHANNEL];
}
cv::Mat variant_2(cv::Mat const& img)
{
// NB: output Mat must be preallocated
cv::Mat channel(img.rows, img.cols, CV_8UC1);
int from_to[] = { SELECTED_CHANNEL, 0 };
cv::mixChannels(&img, 1, &channel, 1, from_to, 1);
return channel;
}
cv::Mat variant_3(cv::Mat const& img)
{
// NB: output Mat must be preallocated
static cv::Mat channel(img.rows, img.cols, CV_8UC1);
int from_to[] = { SELECTED_CHANNEL, 0 };
cv::mixChannels(&img, 1, &channel, 1, from_to, 1);
return channel;
}
template<typename T>
void timeit(std::string const& title, T f)
{
using std::chrono::high_resolution_clock;
using std::chrono::duration_cast;
using std::chrono::microseconds;
cv::Mat img(1024,1024, CV_8UC3);
cv::randu(img, 0, 256);
int32_t const STEPS(1024);
high_resolution_clock::time_point t1 = high_resolution_clock::now();
for (uint32_t i(0); i < STEPS; ++i) {
cv::Mat result = f(img);
}
high_resolution_clock::time_point t2 = high_resolution_clock::now();
auto duration = duration_cast<microseconds>(t2 - t1).count();
double t_ms(static_cast<double>(duration) / 1000.0);
std::cout << title << "\n"
<< "Total = " << t_ms << " ms\n"
<< "Iteration = " << (t_ms / STEPS) << " ms\n"
<< "FPS = " << (STEPS / t_ms * 1000.0) << "\n"
<< "\n";
}
int main()
{
for (uint8_t i(0); i < 2; ++i) {
timeit("Variant 0", variant_0);
timeit("Variant 1", variant_1);
timeit("Variant 2", variant_2);
timeit("Variant 3", variant_3);
std::cout << "--------------------------\n\n";
}
return 0;
}
Output for the second pass (so we avoid any warmup costs).
Note: Running this on i7-4930K, using OpenCV 3.1.0 (64-bit, MSVC12.0), Windows 10 -- YMMV, especially with CPUs that have AVX2
Variant 0
Total = 1518.69 ms
Iteration = 1.48309 ms
FPS = 674.267
Variant 1
Total = 359.048 ms
Iteration = 0.350633 ms
FPS = 2851.99
Variant 2
Total = 820.223 ms
Iteration = 0.800999 ms
FPS = 1248.44
Variant 3
Total = 427.089 ms
Iteration = 0.417079 ms
FPS = 2397.63
Interestingly, cv::split with reuse wins here. Feel free to edit the answer and add timings from different platforms/CPU generations (especially if the proportions differ radically).
It also seems that with my setup, none of this is parallelized quite well, so that may be another possible path at speeding this up (something like cv::parallel_for_).
OpenCV docs say A.mul(B) is per-element multiplication. Yet the following code produces the following output, and then gives this error:
OpenCV Error: Sizes of input arguments do not match
.
cout << laplacian_pyramids[i][numLevels - 1 - l].rows << endl;
cout << gaussian_weight_pyramids[i][l].rows << endl;
cout << laplacian_pyramids[i][numLevels - 1 - l].cols << endl;
cout << gaussian_weight_pyramids[i][l].cols << endl;
Gives:
339
339
571
571
Then:
Mat prod = gaussian_weight_pyramids[i][l].mul(laplacian_pyramids[i][numLevels - 1 - l]);
gives the error. I tried Mat::multiply to a similar effect.
I would recommend converting single channel to three channels:
Mat A = Mat::zeros(100, 200, CV_32FC1);
Mat B = Mat::zeros(100, 200, CV_32FC3);
// Mat C = A.mul(B); // Sizes of input arguments do not match
Mat Afc3;
Mat t[] = {A, A, A};
merge(t, 3, Afc3);
Mat C = Afc3.mul(B); // now Afc3 has 3 channels ans it is type of 32_FC3
// we can multiply each elem in B by the same coef from A
But if B it is a CV_8UC3 type, it does not work because opencv would not allow to multiply Mats which have different types of pixels. In that case, convert 8UC3 to 32FC3 remebering to scale each pixel by 1/255.0 beacuse each pixel in 32FC3 has a value between 0.0 and 1.0 (and of course each pixel in 8UC3 has a value between 0 and 255).
Mat A = Mat::zeros(100, 200, CV_32FC1);
Mat B = Mat::zeros(100, 200, CV_8UC3);
// Mat C = A.mul(B);
Mat Afc3, Bfc3;
Mat t[] = {A, A, A};
merge(t, 3, Afc3);
B.convertTo(Bfc3, CV_32FC3, 1/255.0);
Mat C = Afc3.mul(Bfc3);
There can be 2 reasons for such error: different number of channels or different type of data (for example if first matrix contain unsigned char and second matrix contain unsigned short). Of course there can be both reasons. In general there 3 types of solutions for problems like the one you encountered:
1) Write your own 'for' loop that will do the operation you need. You won't benefit from optimizations that might be present in OpenCV functions but other solutions will have their own overheads. You can see this tutorial about how to access pixels in efficient way.
2) Use functions like 'merge' or 'convertTo' in order to create input of same type and number of channels. See answer posted by #marol for code example. In this solution the main overhead is copy of data. That means extra time and space. This is reasonable solution if you are going to perform multiple operations with both images. But if all you need is simple multiplication it won't be very effective.
3) Use workarounds. For example if your matrices have same type but differ in number of channels you can use reshape function:
// two matrices of same size but different number of channels
Mat laplac(100, 200, CV_32FC3);
Mat gauss(100, 200, CV_32FC1);
// turn them into single channel matrices. they have NxM rows and 1 or 3 columns.
// note that there no copy of data. any change in them will affect original matrices
Mat laplac2 = laplac.reshape( 1, laplac.rows*laplac.cols );
Mat gauss2 = gauss.reshape( 1, gauss.rows*gauss.cols ;
// perform multiplication
laplac2.col(0) = laplac2.col(0).mul(gauss2);
laplac2.col(1) = laplac2.col(1).mul(gauss2);
laplac2.col(2) = laplac2.col(2).mul(gauss2);
This way you are using only OpenCV build-in functions without copy overhead. But I doubt that this will be any faster than solution-1, because solution-1 is more efficient in terms of memory access.
In any case you won't have nice and clean operation that takes exactly one line :(
I am looking to normalize the pixel values of an image to the range [0..1] using C++/OpenCV. However, when I do the normalization using either image *= 1./255 or the normalize function the pixel values are rounded down to zero. I have tried setting the image to type CV_32FC3.
Below is the code I have:
Mat image;
image = imread(imageLoc, CV_LOAD_IMAGE_COLOR | CV_LOAD_IMAGE_ANYDEPTH);
Mat tempImage;
// (didn't work) tempImage *= 1./255;
image.convertTo(tempImage, CV_32F, 3);
normalize(image, tempImage, 0, 1, CV_MINMAX);
int r = 100;
int c = 150;
uchar* ptr = (uchar*)(tempImage.data + r * tempImage.step);
Vec3f tempVals;
tempVals.val[0] = ptr[3*c+1];
tempVals.val[1] = ptr[3*c+2];
tempVals.val[2] = ptr[3*c+3];
cout<<" temp image - "<< tempVals << endl;
uchar* ptr2 = (uchar*)(image.data + r * image.step);
Vec3f imVals;
imVals.val[0] = ptr2[3*c+1];
imVals.val[1] = ptr2[3*c+2];
imVals.val[2] = ptr2[3*c+3];
cout<<" image - "<< imVals << endl;
This produces the following output in the console:
temp image - [0, 0, 0]
image - [90, 78, 60]
You can make convertTo() do the normalization for you:
image.convertTo(tempImage, CV_32FC3, 1.f/255);
You are passing 3 to convertTo(), presumably as channel-count, but that's not the correct signature.
I used the normalize function and it worked (Java):
Core.normalize(src,dst,0.0,1.0,Core.NORM_MINMAX,CvType.CV_32FC1);
You should use a 32F depth for your destination image. I believe the reason for this, is that since you need to get decimal values, you should use an a non-integer OpenCV data type. According to this table, the float types correspond to the 32F depth. I chose the number of channels to be 1 and it worked; CV_32FC1
Remember also that it's unlikely to spot any visual difference in the image.
Finally, since you probably have thousands of pixels in your image, your console might seem that it's printing only zeros. However due to the large amount of data, try to use CTRL+F to see what's going on. Hope this helps.
I am relatively new to C++ and coding in general and have run into a problem when attempting to convert an image to a floating point image. I am attempting to do this to eliminate round off errors with calculating the mean and standard deviation of pixel intensity for images as it starts to effect data quite substantially. My code is below.
Mat img = imread("Cells2.tif");
cv::namedWindow("stuff", CV_WINDOW_NORMAL);
cv::imshow("stuff",img);
CvMat cvmat = img;
Mat dst = cvCreateImage(cvGetSize(&cvmat),IPL_DEPTH_32F,1);
cvConvertScale(&cvmat,&dst);
cvScale(&dst,&dst,1.0/255);
cvNamedWindow("Test",CV_WINDOW_NORMAL);
cvShowImage("Test",&dst);
And I am running into this error
OpenCV Error: Bad argument (Array should be CvMat or IplImage) in an unknown function, file ......\modules\core\src\array.cpp, line 1238
I've looked everywhere and everyone was saying to convert img to CvMat which I attempted above.
When I did that as above code shows I get
OpenCV Error: Bad argument (Unknown array type) in unknown function, file ......\modules\core\src\matrix.cpp line 697
Thanks for your help in advance.
Just use C++ OpenCV interface instead of C interface and use convertTo function to convert between data types.
Mat img = imread("Cells2.tif");
cv::imshow("source",img);
Mat dst; // destination image
// check if we have RGB or grayscale image
if (img.channels() == 3) {
// convert 3-channel (RGB) 8-bit uchar image to 32 bit float
src.convertTo(dst, CV_32FC3);
}
else if (img.channels() == 1) {
// convert 1-chanel (grayscale) 8-bit uchar image to 32 bit float
img1.convertTo(dst, CV_32FC1);
}
// display output, note that to display dst image correctly
// we have to divide each element of dst by 255 to keep
// the pixel values in the range [0,1].
cv::imshow("output",dst/255);
waitKey();
Second part of the question To calculate the mean of all elements in dst
cv::Salar avg_pixel;
double avg;
// note that Scalar is a vector.
// If your image is RGB, Scalar will contain 3 values,
// representing color values for each channel.
avg_pixel = cv::mean(dst);
if (dst.channels() == 3) {
//if 3 channels
avg = (avg_pixel[0] + avg_pixel[1] + avg_pixel[2]) / 3;
}
if(dst.channels() == 1) {
avg = avg_pixel[0];
}
cout << "average element of m: " << avg << endl;
Here is my code for calculating the average in C++ OpenCV.
int NumPixels = img.total();
double avg;
double c;
for(int y = 0; y <= img.cols; y++)
for(int x = 0; x <= dst.rows; x++)
c+=img.at<uchar>(x,y);
avg = c/NumPixels;
cout << "Avg Value\n" << 255*avg;
For MATLAB I just load the image and take Q = mean(img(:)); which returns 1776.23
And for the return of 1612.36 I used cv:Scalar z = mean(dst);