Cycle through pixels with opencv - c++

How would I be able to cycle through an image using opencv as if it were a 2d array to get the rgb values of each pixel? Also, would a mat be preferable over an iplimage for this operation?

cv::Mat is preferred over IplImage because it simplifies your code
cv::Mat img = cv::imread("lenna.png");
for(int i=0; i<img.rows; i++)
for(int j=0; j<img.cols; j++)
// You can now access the pixel value with cv::Vec3b
std::cout << img.at<cv::Vec3b>(i,j)[0] << " " << img.at<cv::Vec3b>(i,j)[1] << " " << img.at<cv::Vec3b>(i,j)[2] << std::endl;
This assumes that you need to use the RGB values together. If you don't, you can uses cv::split to get each channel separately. See etarion's answer for the link with example.
Also, in my cases, you simply need the image in gray-scale. Then, you can load the image in grayscale and access it as an array of uchar.
cv::Mat img = cv::imread("lenna.png",0);
for(int i=0; i<img.rows; i++)
for(int j=0; j<img.cols; j++)
std::cout << img.at<uchar>(i,j) << std::endl;
UPDATE: Using split to get the 3 channels
cv::Mat img = cv::imread("lenna.png");
std::vector<cv::Mat> three_channels = cv::split(img);
// Now I can access each channel separately
for(int i=0; i<img.rows; i++)
for(int j=0; j<img.cols; j++)
std::cout << three_channels[0].at<uchar>(i,j) << " " << three_channels[1].at<uchar>(i,j) << " " << three_channels[2].at<uchar>(i,j) << std::endl;
// Similarly for the other two channels
UPDATE: Thanks to entarion for spotting the error I introduced when copying and pasting from the cv::Vec3b example.

Since OpenCV 3.0, there are official and fastest way to run function all over the pixel in cv::Mat.
void cv::Mat::forEach (const Functor& operation)
If you use this function, operation is runs on multi core automatically.
Disclosure : I'm contributor of this feature.

If you use C++, use the C++ interface of opencv and then you can access the members via http://docs.opencv.org/2.4/doc/tutorials/core/how_to_scan_images/how_to_scan_images.html#the-efficient-way or using cv::Mat::at(), for example.

This is an old question but needs to get updated since opencv is being actively developed. Recently, OpenCV has introduced parallel_for_ which complies with c++11 lambda functions. Here is the example
parallel_for_(Range(0 , img.rows * img.cols), [&](const Range& range){
for(int r = range.start; r<range.end; r++ )
{
int i = r / img.cols;
int j = r % img.cols;
img.ptr<uchar>(i)[j] = doSomethingWithPixel(img.at<uchar>(i,j));
}
});
This is mention-worthy that this method uses the CPU cores in modern computer architectures.

Since OpenCV 3.3 (see changelog) it is also possible to use C++11 style for loops:
// Example 1
Mat_<Vec3b> img = imread("lena.jpg");
for( auto& pixel: img ) {
pixel[0] = gamma_lut[pixel[0]];
pixel[1] = gamma_lut[pixel[1]];
pixel[2] = gamma_lut[pixel[2]];
}
// Example 2
Mat_<float> img2 = imread("float_image.exr", cv::IMREAD_UNCHANGED);
for(auto& p : img2) p *= 2;

The docs show a well written comparison of different ways to iterate over a Mat image here.
The fastest way is to use C style pointers. Here is the code copied from the docs:
Mat& ScanImageAndReduceC(Mat& I, const uchar* const table)
{
// accept only char type matrices
CV_Assert(I.depth() != sizeof(uchar));
int channels = I.channels();
int nRows = I.rows;
int nCols = I.cols * channels;
if (I.isContinuous())
{
nCols *= nRows;
nRows = 1;
}
int i,j;
uchar* p;
for( i = 0; i < nRows; ++i)
{
p = I.ptr<uchar>(i);
for ( j = 0; j < nCols; ++j)
{
p[j] = table[p[j]];
}
}
return I;
}
Accessing the elements with the at is quite slow.
Note that if your operation can be performed using a lookup table, the built in function LUT is by far the fastest (also described in the docs).

If you want to modify RGB pixels one by one, the example below will help!
void LoopPixels(cv::Mat &img) {
// Accept only char type matrices
CV_Assert(img.depth() == CV_8U);
// Get the channel count (3 = rgb, 4 = rgba, etc.)
const int channels = img.channels();
switch (channels) {
case 1:
{
// Single colour
cv::MatIterator_<uchar> it, end;
for (it = img.begin<uchar>(), end = img.end<uchar>(); it != end; ++it)
*it = 255;
break;
}
case 3:
{
// RGB Color
cv::MatIterator_<cv::Vec3b> it, end;
for (it = img.begin<cv::Vec3b>(), end = img.end<cv::Vec3b>(); it != end; ++it) {
uchar &r = (*it)[2];
uchar &g = (*it)[1];
uchar &b = (*it)[0];
// Modify r, g, b values
// E.g. r = 255; g = 0; b = 0;
}
break;
}
}
}

Related

Run time error assigning cv::Mat element value using cv::Mat::at

In the following code I would like to assign a values to elements of a Mat variable in a loop. I get the runtime error below.
pair<Mat, Mat> meshgrid(vector<int> x, vector<int> y) {
int sx = (int)x.size();
int sy = (int)y.size();
Mat xmat = Mat::ones(sy, sx, CV_16U);
Mat ymat = Mat::ones(sy, sy, CV_16U);
for (int i = 0; i < sx; i++) {
for (int j = 0; j < sy; j++) {
xmat.at<int>(i, j) = j; // <------- here is place of error.
cout << j << "\t";
}
cout << endl;
}
for (int i = 0; i < sx; i++) {
for (int j = 0; j < sy; j++) {
ymat.at<int>(i, j) = i; // <------- here is place of error.
cout << j << "\t";
}
cout << endl;
}
return make_pair(xmat, ymat);
}
This picture when debuging;
This is the run time error I get:
OpenCV(...) Error: Assertion failed
(((((sizeof(size_t)<<28)|0x8442211) >> ((traits::Depth<_Tp>::value) &
((1 << 3) - 1))*4) & 15) == elemSize1()) in cv::Mat::at, file
...\include\opencv2\core\mat.inl.hpp, line 1108
Thank you for your answers.
I assume you meant to generate output similar to numpy.meshgrid, and Matlab meshgrid.
There are several errors in your code:
The cv::Mat is initialized with type CV_16U (i.e. 16 bit unsigned value), but when you access the elements with at you use int (which is 32bit signed).
You should change it to at<unsigned short> (or change the type of the cv::Mat to 32 bit signed - CV_32S).
You initialized the cv::Mat with wrong sizes: xmat has size of (sy, sx), but ymat has size of (sy, sy).
The indices (row, col) you used to access the mat elements were incorrect. To make it easier to use correctly, I changed the names of the dimentions to rows, cols,
and the loop indices to iRow, iCol.
The values in the matrices should come from the values in x and y vectors (not the indices).
See updated code below (and the notes following it regarding the changes):
#include <opencv2/core/core.hpp>
#include <vector>
#include <iostream>
std::pair<cv::Mat, cv::Mat> meshgrid(std::vector<unsigned short> const & x, std::vector<unsigned short> const & y)
{
int cols = static_cast<int>(x.size());
int rows = static_cast<int>(y.size());
cv::Mat xmat(rows, cols, CV_16U);
cv::Mat ymat(rows, cols, CV_16U);
for (int iRow = 0; iRow < rows; ++iRow) {
auto * pRowData = xmat.ptr<unsigned short>(iRow);
for (int iCol = 0; iCol < cols; ++iCol) {
pRowData[iCol] = x[iCol];
std::cout << pRowData[iCol] << "\t";
}
std::cout << std::endl;
}
std::cout << std::endl;
for (int iRow = 0; iRow < rows; ++iRow) {
auto * pRowData = ymat.ptr<unsigned short>(iRow);
for (int iCol = 0; iCol < cols; ++iCol) {
pRowData[iCol] = y[iRow];
std::cout << pRowData[iCol] << "\t";
}
std::cout << std::endl;
}
return std::make_pair(std::move(xmat), std::move(ymat));
}
int main()
{
std::vector<unsigned short> xxx{ 1,2 };
std::vector<unsigned short> yyy{ 10,11,12 };
auto p = meshgrid(xxx, yyy);
return 0;
}
Output:
1 2
1 2
1 2
10 10
11 11
12 12
Some notes:
I might have misunderstood which values you wanted to set in the cv::Mat's. But at least now you have code that does not crash. You can change the assigned values as you wish.
Using at to access cv::Mat elements one after the other is very inefficient, because at contains some validations for every access.
It's a lot more efficient to use the cv::Mat method ptr, that gives you a pointer to the data of a row. Then you can use this pointer to traverse the row more efficiently - see above
In any method, it is more efficient to traverse a cv::Mat one row after another (and not column by column). This causes you to access continous memory, and decrease the number of cache misses.
In your real code, it's better to separate calculations from I/O. Therefore it's better if your meshgrid function will only create the matrices. Print them outside if you need.
No need to initialize the cv::Mats to ones, because immediatly afterwards we set the values for all elements.
In my code x and y are passed to the function by const refernce. It is more efficient (avoid copy) and also forces the compiler to verify the vectors are not modified.
Better to avoid using namespace std - see here Why is "using namespace std;" considered bad practice?.
From similar reasons I recomend to avoid using namespace cv as well.

Understanding nested array c++ in image scanning context

I came across this sample code on openCV library. What does the line p[j] = table[p[j]] do? I have come across multi dimensional arrays but not something like this before.
Mat& ScanImageAndReduceC(Mat& I, const uchar* const table)
{
// accept only char type matrices
CV_Assert(I.depth() == CV_8U);
int channels = I.channels();
int nRows = I.rows;
int nCols = I.cols * channels;
if (I.isContinuous())
{
nCols *= nRows;
nRows = 1;
}
int i,j;
uchar* p;
for( i = 0; i < nRows; ++i)
{
p = I.ptr<uchar>(i);
for ( j = 0; j < nCols; ++j)
{
p[j] = table[p[j]];
}
}
return I;
}
It is doing color replacement by using a table where each pixel intensity maps to some other value. Commonly used for techniques like color grading, histogram adjustment, or even thresholding.
Here, the table contains unsigned char values and is being indexed by the value of the pixel. The pixel's intensity p[i] is used as an index into the table, and the value at that index is then written to that pixel, replacing its original value.
It is a lookup table conversion.
The pixels of image(I) would be converted by means of table.
For example, the pixel with value 100 would be changed to 10 if table[100]=10.
Your sample code is introduced in OpenCV tutorial which is well explained of what the code does.
https://docs.opencv.org/master/db/da5/tutorial_how_to_scan_images.html

Calculating the vertical gradient of 2D image causes strange output

I want to apply a simple derive/gradient filter, [-1, 0, 1], to an image from a .ppm file.
The raw binary data from the .ppm file is read into a one-dimensional array:
uint8_t* raw_image_data;
size_t n_rows, n_cols, depth;
// Open the file as an input binary file
std::ifstream file;
file.open("test_image.ppm", std::ios::in | std::ios::binary);
if (!file.is_open()) { /* error */ }
std::string temp_line;
// Check that it's a valid P6 file
if (!(std::getline(file, temp_line) && temp_line == "P6")) {}
// Then skip all the comments (lines that begin with a #)
while (std::getline(file, temp_line) && temp_line.at(0) == '#');
// Try read in the info about the number of rows and columns
try {
n_rows = std::stoi(temp_line.substr(0, temp_line.find(' ')));
n_cols = std::stoi(temp_line.substr(temp_line.find(' ')+1,temp_line.size()));
std::getline(file, temp_line);
depth = std::stoi(temp_line);
} catch (const std::invalid_argument & e) { /* stoi has failed */}
// Allocate memory and read in all image data from ppm
raw_image_data = new uint8_t[n_rows*n_cols*3];
file.read((char*)raw_image_data, n_rows*n_cols*3);
file.close();
I then read a grayscale image from the data into a two-dimensional array, called image_grayscale:
uint8_t** image_grayscale;
image_grayscale = new uint8_t*[n_rows];
for (size_t i = 0; i < n_rows; ++i) {
image_grayscale[i] = new uint8_t[n_cols];
}
// Convert linear array of raw image data to 2d grayscale image
size_t counter = 0;
for (size_t r = 0; r < n_rows; ++r) {
for (size_t c = 0; c < n_cols; ++c) {
image_grayscale[r][c] = 0.21*raw_image_data[counter]
+ 0.72*raw_image_data[counter+1]
+ 0.07*raw_image_data[counter+2];
counter += 3;
}
}
I want to write the resulting filtered image to another two-dimensional array, gradient_magnitude:
uint32_t** gradient_magnitude;
// Allocate memory
gradient_magnitude = new uint32_t*[n_rows];
for (size_t i = 0; i < n_rows; ++i) {
gradient_magnitude[i] = new uint32_t[n_cols];
}
// Filtering operation
int32_t grad_h, grad_v;
for (int r = 1; r < n_rows-1; ++r) {
for (int c = 1; c < n_cols-1; ++c) {
grad_h = image_grayscale[r][c+1] - image_grayscale[r][c-1];
grad_v = image_grayscale[r+1][c] - image_grayscale[r-1][c];
gradient_magnitude[r][c] = std::sqrt(pow(grad_h, 2) + pow(grad_v, 2));
}
}
Finally, I write the filtered image to a .ppm output.
std::ofstream out;
out.open("output.ppm", std::ios::out | std::ios::binary);
// ppm header
out << "P6\n" << n_rows << " " << n_cols << "\n" << "255\n";
// Write data to file
for (int r = 0; r < n_rows; ++r) {
for (int c = 0; c < n_cols; ++c) {
for (int i = 0; i < 3; ++i) {
out.write((char*) &gradient_magnitude[r][c],1);
}
}
}
out.close();
The output image, however, is a mess.
When I simply set grad_v = 0; in the loop (i.e. solely calculate the horizontal gradient), the output is seemingly correct:
When I instead set grad_h = 0; (i.e. solely calculate the vertical gradient), the output is strange:
It seems like part of the image has been circularly shifted, but I cannot understand why. Moreover, I have tried with many images and the same issue occurs.
Can anyone see any issues? Thanks so much!
Ok, first clue is that the image looks circularly shifted. This hints that strides are wrong. The core of your problem is simple:
n_rows = std::stoi(temp_line.substr(0, temp_line.find(' ')));
n_cols = std::stoi(temp_line.substr(temp_line.find(' ')+1,temp_line.size()));
but in the documentation you can read:
Each PPM image consists of the following:
A "magic number" for identifying the file type. A ppm image's magic number is the two
characters "P6".
Whitespace (blanks, TABs, CRs, LFs).
A width, formatted as ASCII characters in decimal.
Whitespace.
A height, again in ASCII decimal.
[...]
Width is columns, height is rows. So that's the classical error that you get when implementing image processing stuff: swapping rows and columns.
From a didactic point of view, why are you doing this mistake? My guess: poor debugging tools. After making a working example from your question (effort that I would have saved if you had provided a MCVE), I run to the end of image loading and used Image Watch to see the content of your image with #mem(raw_image_data, UINT8, 3, n_cols, n_rows, n_cols*3). Result:
Ok, let's try to swap them: #mem(raw_image_data, UINT8, 3, n_rows, n_cols, n_rows*3). Result:
Much better. Unfortunately I don't know how to specify RGB instead of BGR in Image Watch #mem pseudo command, so the wrong colors.
Then we come back to your code: please compile with all warnings on. Then I'd use more of the std::stream features for parsing your input and less std::stoi() or find(). Avoid memory allocation by using std::vector and make a (possibly template) class for images. Even if you stick to your pointer to pointer, don't make multiple new for each row: make a single new for the pointer at row 0, and have the other pointers point to it:
uint8_t** image_grayscale = new uint8_t*[n_rows];
image_grayscale[0] = new uint8_t[n_rows*n_cols];
for (size_t i = 1; i < n_rows; ++i) {
image_grayscale[i] = image_grayscale[i - 1] + n_cols;
}
Same effect, but easier to deallocate and to manage as a single piece of memory. For example, saving as a PGM becomes:
{
std::ofstream out("output.pgm", std::ios::binary);
out << "P5\n" << n_rows << " " << n_cols << "\n" << "255\n";
out.write(reinterpret_cast<char*>(image_grayscale[0]), n_rows*n_cols);
}
Fill your borders! Using the single allocation style I showed you you can do it as:
uint32_t** gradient_magnitude = new uint32_t*[n_rows];
gradient_magnitude[0] = new uint32_t[n_rows*n_cols];
for (size_t i = 1; i < n_rows; ++i) {
gradient_magnitude[i] = gradient_magnitude[i - 1] + n_cols;
}
std::fill_n(gradient_magnitude[0], n_rows*n_cols, 0);
Finally the gradient magnitude is an integer value between 0 and 360 (you used a uint32_t). Then you save only the least significant byte of it! Of course it's wrong. You need to map from [0,360] to [0,255]. How? You can saturate (if greater than 255 set to 255) or apply a linear scaling (*255/360). Of course you can do also other things, but it's not important.
Here you can see the result on a zoomed version of the three cases: saturate, scale, only LSB (wrong):
With the wrong version you see dark pixels where the value should be higer than 255.

Efficient way to loop through pixels of 16-bit Mat in OpenCV

I'm trying to make very simple (LUT-like) operations on a 16-bit gray-scale OpenCV Mat, which is efficient and doesn't slow down the debugger.
While there is a very detailed page in the documentation addressing exactly this issue, it fails to point out that most of those methods are only available on 8-bit images (including the perfect, optimized LUT function).
I tried the following methods:
uchar* p = mat_depth.data;
for (unsigned int i = 0; i < depth_width * depth_height * sizeof(unsigned short); ++i)
{
*p = ...;
*p++;
}
Really fast, unfortunately only supporting uchart (just like LUT).
int i = 0;
for (int row = 0; row < depth_height; row++)
{
for (int col = 0; col < depth_width; col++)
{
i = mat_depth.at<short>(row, col);
i = ..
mat_depth.at<short>(row, col) = i;
}
}
Adapted from this answer: https://stackoverflow.com/a/27225293/518169. Didn't work for me, and it was very slow.
cv::MatIterator_<ushort> it, end;
for (it = mat_depth.begin<ushort>(), end = mat_depth.end<ushort>(); it != end; ++it)
{
*it = ...;
}
Works well, however it uses a lot of CPU and makes the debugger super slow.
This answer https://stackoverflow.com/a/27099697/518169 points out to the source code of the built-in LUT function, however it only mentions advanced optimization techniques, like IPP and OpenCL.
What I'm looking for is a very simple loop like the first code, but for ushorts.
What method do you recommend for solving this problem? I'm not looking for extreme optimization, just something on par with the performance of the single-for-loop on .data.
I implemented Michael's and Kornel's suggestion and benchmarked them both in release and debug modes.
code:
cv::Mat LUT_16(cv::Mat &mat, ushort table[])
{
int limit = mat.rows * mat.cols;
ushort* p = mat.ptr<ushort>(0);
for (int i = 0; i < limit; ++i)
{
p[i] = table[p[i]];
}
return mat;
}
cv::Mat LUT_16_reinterpret_cast(cv::Mat &mat, ushort table[])
{
int limit = mat.rows * mat.cols;
ushort* ptr = reinterpret_cast<ushort*>(mat.data);
for (int i = 0; i < limit; i++, ptr++)
{
*ptr = table[*ptr];
}
return mat;
}
cv::Mat LUT_16_if(cv::Mat &mat)
{
int limit = mat.rows * mat.cols;
ushort* ptr = reinterpret_cast<ushort*>(mat.data);
for (int i = 0; i < limit; i++, ptr++)
{
if (*ptr == 0){
*ptr = 65535;
}
else{
*ptr *= 100;
}
}
return mat;
}
ushort* tablegen_zero()
{
static ushort table[65536];
for (int i = 0; i < 65536; ++i)
{
if (i == 0)
{
table[i] = 65535;
}
else
{
table[i] = i;
}
}
return table;
}
The results are the following (release/debug):
LUT_16: 0.202 ms / 0.773 ms
LUT_16_reinterpret_cast: 0.184 ms / 0.801 ms
LUT_16_if: 0.249 ms / 0.860 ms
So the conclusion is that reinterpret_cast is the faster by 9% in release mode, while the ptr one is faster by 4% in debug mode.
It's also interesting to see that directly calling the if function instead of applying a LUT only makes it slower by 0.065 ms.
Specs: streaming 640x480x16-bit grayscale image, Visual Studio 2013, i7 4750HQ.
OpenCV implementation is based on polymorphism and runtime dispatching over templates. In OpenCV version the use of templates is limited to a fixed set of primitive data types. That is, array elements should have one of the following types:
8-bit unsigned integer (uchar)
8-bit signed integer (schar)
16-bit unsigned integer (ushort)
16-bit signed integer (short)
32-bit signed integer (int)
32-bit floating-point number (float)
64-bit floating-point number (double)
a tuple of several elements where all elements have the same type (one of the above).
In case your cv::Mat is continues you can use pointer arithmetics to go through the whole data pointer and you should only use the appropriate pointer type to your cv::Mat.
Furthermore, keep it mind that cv::Mats are not always continuous (it can be a ROI, padded, or created from pixel pointer) and iterating over them with pointers will crash.
An example loop:
cv::Mat cvmat16sc1 = cv::Mat::eye(10, 10, CV_16SC1);
if (cvmat16sc1.data)
{
if (!cvmat16sc1.isContinuous())
{
cvmat16sc1 = cvmat16sc1.clone();
}
short* ptr = reinterpret_cast<short*>(cvmat16sc1.data);
for (int i = 0; i < cvmat16sc1.cols * cvmat16sc1.rows; i++, ptr++)
{
if (*ptr == 1)
std::cout << i << ": " << *ptr << std::endl;
}
}
Best solution for your problem is already written in the tutorial that you mentioned, in the chapter named "The efficient way". All you need is to replace every instance of uchar with ushort. No other changes are needed.

Multi-Dimensional data in a Matrix in OpenCV with C++

I want to declare, populate, access a Multi-Dimensional Matrix in OpenCV (C++) which is compatible with namespace cv. I found no quick and easy to learn examples on them. Can you please help me out?
Here is a short example from the NAryMatIterator documentation; it shows how to create, populate, and process a multi-dimensional matrix in OpenCV:
void computeNormalizedColorHist(const Mat& image, Mat& hist, int N, double minProb)
{
const int histSize[] = {N, N, N};
// make sure that the histogram has a proper size and type
hist.create(3, histSize, CV_32F);
// and clear it
hist = Scalar(0);
// the loop below assumes that the image
// is a 8-bit 3-channel. check it.
CV_Assert(image.type() == CV_8UC3);
MatConstIterator_<Vec3b> it = image.begin<Vec3b>(),
it_end = image.end<Vec3b>();
for( ; it != it_end; ++it )
{
const Vec3b& pix = *it;
hist.at<float>(pix[0]*N/256, pix[1]*N/256, pix[2]*N/256) += 1.f;
}
minProb *= image.rows*image.cols;
Mat plane;
NAryMatIterator it(&hist, &plane, 1);
double s = 0;
// iterate through the matrix. on each iteration
// it.planes[*] (of type Mat) will be set to the current plane.
for(int p = 0; p < it.nplanes; p++, ++it)
{
threshold(it.planes[0], it.planes[0], minProb, 0, THRESH_TOZERO);
s += sum(it.planes[0])[0];
}
s = 1./s;
it = NAryMatIterator(&hist, &plane, 1);
for(int p = 0; p < it.nplanes; p++, ++it)
it.planes[0] *= s;
}
Also, check out the cv::compareHist function for another usage example of the NAryMatIterator here.
To create a multi-dimensional matrix that is of size 100x100x3, using floats, one channel, and with all elements initialized to 10 you write like this:
int size[3] = { 100, 100, 3 };
cv::Mat M(3, size, CV_32FC1, cv::Scalar(10));
To loop over and output the elements in the matrix you can do:
for (int i = 0; i < 100; i++)
for (int j = 0; j < 100; j++)
for (int k = 0; k < 3; k++)
std::cout << M.at<cv::Vec3f>(i,j)[k] << ", ";
However, beware of the troubles with using multi-dimensional matrices as documented here: How do i get the size of a multi-dimensional cv::Mat? (Mat, or MatND)