I am trying to sort pixel values of an image (example 80x20) from lowest to highest.
Below is the some code:
bool sortPixel(int first, int second)
{
return (first < second);
}
vector<int>vect_sortPixel;
for(int y=0; y<height; y++)
{
for(int x=0; x<width; x++)
{
vect_sortPixel.push_back(cvGetReal2D(srcImg, y, x));
sort(vect_sortPixel.begin(), vect_sortPixel.end(), sortPixel);
}
}
But it takes quite long time to compute. Any suggestion to reduce the processing time?
Thank you.
Don't use getReal2D. It's quite slow.
Convert image to cv::Mat or Mat. Use its data pointer to get the pixel values. Mat.data() will give you pointer to the original matrix. Use that.
And as far as sorting is concerned, I would advise you to first make an array of all the pixels, then sort it using Merge sort (time complexity O(n log n))
#include<opencv2/highgui/highgui.hpp>
#include<stdio.h>
using namespace cv;
using namespace std;
int main()
{
Mat img = imread("filename.jpg",CV_LOAD_IMAGE_COLOR);
unsigned char *input = (unsigned char*)(img.data);
int i,j,r,g,b;
for(int i = 0;i < img.cols;i++){
for(int j = 0;j < img.rows;j++){
b = input[img.cols * j + i] ;
g = input[img.cols * j+ i + 1];
r = input[img.cols *j + i +2];
}
}
return 0;
}
Using this you can access pixel values from the main matrix.
Warning: This is not how you compare it. I'm suggesting that by using something like this, you can access pixel values.
Mat.data() gives you pointer to the original matrix. This matrix is a 1 D matrix with all the given pixel values.
Image => (x,y,z),(x1,y1,z1), etc..
Mat(original matrix) => x,y,z,x1,y1,z1,...
If you still have some doubts regarding how to extract data from Mat, visit this link OpenCV get pixel channel value from Mat image
and here's a link regarding Merge Sort http://www.cplusplus.happycodings.com/Algorithms/code17.html
There are few problems in your code:
As Froyo already said you use cvGetReal2D which is actually not very fast. You have to convert your cvMat to cv::Mat. To do this there's cv::Mat constructor:
// converts old-style CvMat to the new matrix; the data is not copied by default
Mat(const CvMat* m, bool copyData=false);
And after this use direct pixels acces as mentioned in this SO question.
Another problem is that you use push_back which actually also not very fast. You know the size of array, so why don't you allocate needed memory at the beginning? Like this:
vector<int> vect_sortPixel(mat.cols*mat.rows);
And than just use vect_sortPixel[i] to get needed pixel.
Why do you call sort in the loop? You have to call it after loop, when array is already created! Default STL's sort should work fast:
Complexity
Approximately N*logN comparisons on average (where N is
last-first). In the worst case, up to N^2, depending on specific
sorting algorithm used by library implementation.
Related
The question is quite straightforward. I'll also explain what I do in case there is a faster way to do this without optimizing this specific way.
I go through an image and its rgb values. I have bins of size 256 for each color. So for every pixel I calculate the 3 bins of its rgb values. The bins essentially give me the index to access data for the specific color in a large vector. With this data, I do some calculations which are irrelevant. What I want to optimize is the accessing part.
Keep in mind that the large vector has an extra dimension. Every pixel belongs to some defined areas of the image. For every area it belongs to, it has an element in the big vector. So, if a pixel belongs in 4 areas(eg 3,9,12,13) then the data I want to access is: data[colorIndex][3],data[colorIndex][9],data[colorIndex][12],data[colorIndex][13].
I think that's enough to explain the code which is the following:
//Just filling with data for the sake of the example
int cols = 200; int rows = 200;
cv::Mat image(200, 200, CV_8UC3);
image.setTo(Scalar(100, 100, 100));
int numberOfAreas = 50;
//For every pixel (first dimension) we have a vector<int> containing ones for every area the pixel belongs to.
//For this example, every pixel belongs to every area.
vector<vector<int>> areasThePixelBelongs(200 * 200, vector<int>(numberOfAreas, 1));
int numberOfBins = 32;
int sizeOfBin = 256 / numberOfBins;
vector<vector<float>> data(pow(numberOfBins, 3), vector<float>(numberOfAreas, 1));
//Filling complete
//Part I need to optimize
uchar* matPointer;
for (int y = 0; y < rows; y++) {
matPointer = image.ptr<uchar>(y);
for (int x = 0; x < cols; x++) {
int red = matPointer[x * 3 + 2];
int green = matPointer[x * 3 + 1];
int blue = matPointer[x * 3];
int binNumberRed = red / sizeOfBin;
int binNumberGreen = green / sizeOfBin;
int binNumberBlue = blue / sizeOfBin;
//Instead of a 3d vector where I access the elements like: color[binNumberRed][binNumberGreen][binNumberBlue]
//I use a 1d vector where I just have to calculate the 1d index as follows
int index = binNumberRed * numberOfBins * numberOfBins + binNumberGreen * numberOfBins + binNumberBlue;
vector<int>& areasOfPixel = areasThePixelBelongs[y*cols+x];
int numberOfPixelAreas = areasOfPixel.size();
for (int i = 0; i < numberOfPixelAreas; i++) {
float valueOfInterest = data[index][areasOfPixel[i]];
//Some calculations here...
}
}
}
Would it be better accessing each mat element as a Vec3b? I think I'm essentially accessing an element 3 times for each pixel using uchar. Would accessing one Vec3b be faster?
First of all vector<vector<T>> is not efficiently stored in memory as it is not contiguous. This as often a big impact on performance and should be avoided as mush as possible (especially when the inner arrays are of the same size). Instead of this, you can use std::array for fixed-size arrays or a flatten std::vector (with the size dim1 * dim2 * ... dimN).
Moreover, the loop is a good candidate for parallelization. You can parallelize this code easily with OpenMP. This assumes Some calculations here can be implemented in a thread-safe way (you should be careful about shared writes if any). If this code is embarrassingly-parallel, then the resulting parallel code can be much faster. Still, using multi-threading introduces some overhead which may be too big compared to the overall computation time (which is highly dependent of the content in Some calculations here).
Finally, regarding the content in Some calculations here it may or may not be possible to adapt the code so the compiler use SIMD instructions. The data[index][areasOfPixel[i]] will likely prevent most compiler to do that, but the following computation could be. Note that software prefetching and gather instructions may help to speed up a bit the data[index][areasOfPixel[i]] operation.
Note that the way you access pixels should not have a significant impact on the runtime as the computation should be bounded by the speed of the inner loop iterating on areas containing some unknown code (unless this unknown code actually access pixels too).
In Python I normally use functions like vstack, stack, etc to easily create a 3D array by stacking 2D arrays one onto another.
Is there any way to do this in C++?
In particular, I have loaded a image into a Mat variable with OpenCV like:
cv::Mat im = cv::imread("image.png", 0);
I would like to make a 3D array/Mat of N layers by stacking copies of that Mat variable.
EDIT: This new 3D matrix has to be "travellable" by adding an integer to any of its components, such that if I am in the position (x1,y1,1) and I add +1 to the last component, I arrive to (x1,y1,2). Similarly for any of the coordinates/components of the 3D matrix.
SOLVED: Both answers from #Aram and #Nejc do exactly what expected. I set #Nejc 's answer as the correct one for his shorter code.
The Numpy function vstack returns a contiguous array. Any C++ solution that produces vectors or arrays of cv::Mat objects does not reflect the behaviour of vstack in this regard, becase separate "layers" belonging to individual cv::Mat objects will not be stored in contiguous buffer (unless a careful allocation of underlying buffers is done in advance of course).
I present the solution that copies all arrays into a three-dimensional cv::Mat object with a contiguous buffer. As far as the idea goes, this answer is similar to Aram's answer. But instead of assigning pixel values one by one, I take advantage of OpenCV functions. At the beginning I allocate the matrix which has a size N X ROWS X COLS, where N is the number of 2D images I want to "stack" and ROWS x COLS are dimensions of each of these images.
Then I make N steps. On every step, I obtain the pointer to the location of the first element along the "outer" dimension. I pass that pointer to the constructor of temporary Mat object that acts as a kind of wrapper around the memory chunk of size ROWS x COLS (but no copies are made) that begins at the address that is pointed-at by pointer. I then use copyTo method to copy i-th image into that memory chunk. Code for N = 2:
cv::Mat img0 = cv::imread("image0.png", CV_IMREAD_GRAYSCALE);
cv::Mat img1 = cv::imread("image1.png", CV_IMREAD_GRAYSCALE);
cv::Mat images[2] = {img0, img1}; // you can also use vector or some other container
int dims[3] = { 2, img0.rows, img0.cols }; // dimensions of new image
cv::Mat joined(3, dims, CV_8U); // same element type (CV_8U) as input images
for(int i = 0; i < 2; ++i)
{
uint8_t* ptr = &joined.at<uint8_t>(i, 0, 0); // pointer to first element of slice i
cv::Mat destination(img0.rows, img0.cols, CV_8U, (void*)ptr); // no data copy, see documentation
images[i].copyTo(destination);
}
This answer is in response to the question above of:
In Python I normally use functions like vstack, stack, etc to easily create a 3D array by stacking 2D arrays one onto another.
This is certainly possible, you can add matrices into a vector which would be your "stack"
For instance you could use a
std::vector<cv::Mat>>
This would give you a vector of mats, which would be one slice, and then you could "layer" those by adding more slices vector
If you then want to have multiple stacks you can add that vector into another vector:
std::vector<std::vector<cv::Mat>>
To add matrix to an array you do:
myVector.push_back(matrix);
Edit for question below
In such case, could I travel from one position (x1, y1, z1) to an immediately upper position doing (x1,y1,z1+1), such that my new position in the matrix would be (x1,y1,z2)?
You'll end up with something that looks a lot like this. If you have a matrix at element 1 in your vector, it doesn't really have any relationship to the element[2] except for the fact that you have added it into that point. If you want to build relationships then you will need to code that in yourself.
You can actually create a 3D or ND mat with opencv, you need to use the constructor that takes the dimensions as input. Then copy each matrix into (this case) the 3D array
#include <opencv2/opencv.hpp>
using namespace cv;
using namespace std;
int main() {
// Dimensions for the constructor... set dims[0..2] to what you want
int dims[] = {5, 5, 5}; // 5x5x5 3d mat
Mat m = Mat::zeros(5, 5, CV_8UC1);
for (size_t i = 0; i < 5; i++) {
for (size_t k = 0; k < 5; k++) {
m.at<uchar>(i, k) = i + k;
}
}
// Mat with constructor specifying 3 dimensions with dimensions sizes in dims.
Mat 3DMat = Mat(3, dims, CV_8UC1);
// We fill our 3d mat.
for (size_t i = 0; i < m2.size[0]; i++) {
for (size_t k = 0; k < m2.size[1]; k++) {
for (size_t j = 0; j < m2.size[2]; j++) {
3DMat.at<uchar>(i, k, j) = m.at<uchar>(k, j);
}
}
}
// We print it to show the 5x5x5 array.
for (size_t i = 0; i < m2.size[0]; i++) {
for (size_t k = 0; k < m2.size[1]; k++) {
for (size_t j = 0; j < m2.size[2]; j++) {
std::cout << (int) 3DMat.at<uchar>(i, k, j) << " ";
}
std::cout << endl;
}
std::cout << endl;
}
return 0;
}
Based on the question and comments, I think you are looking for something like this:
std::vector<cv::Mat> vec_im;
//In side for loop:
vec_im.push_back(im);
Then, you can access it by:
Scalar intensity_1 = vec_im[z1].at<uchar>(y, x);
Scalar intensity_2 = vec_im[z2].at<uchar>(y, x);
This assumes that the image is single channel.
Suppose I have a Mat of indices (locations) called B, We can say that this Mat has dimensions of 1 x 100 and We suppose to have another Mat, called A, full of data of the same dimensions of B.
Now, I would access to the data of A with B. Usually I would create a for loop and I would take for each elements of B, the right elements of A. For the most fussy of the site, this is the code that I would write:
for(int i=0; i < B.cols; i++){
int index = B.at<int>(0, i);
std::cout<<A.at<int>(0, index)<<std:endl;
}
Ok, now that I showed you what I could do, I ask you if there is a way to access the matrix A, always using the B indices, in a more intelligent and fast way. As someone could do in python thanks to the numpy.take() function.
This operation is called remapping. In OpenCV, you can use function cv::remap for this purpose.
Below I present the very basic example of how remap algorithm works; please note that I don't handle border conditions in this example, but cv::remap does - it allows you to use mirroring, clamping, etc. to specify what happens if the indices exceed the dimensions of the image. I also don't show how interpolation is done; check the cv::remap documentation that I've linked to above.
If you are going to use remapping you will probably have to convert indices to floating point; you will also have to introduce another array of indices that should be trivial (all equal to 0) if your image is one-dimensional. If this starts to represent a problem because of performance, I'd suggest you implement the 1-D remap equivalent yourself. But benchmark first before optimizing, of course.
For all the details, check the documentation, which covers everything you need to know to use te algorithm.
cv::Mat<float> remap_example(cv::Mat<float> image,
cv::Mat<float> positions_x,
cv::Mat<float> positions_y)
{
// sizes of positions arrays must be the same
int size_x = positions_x.cols;
int size_y = positions_x.rows;
auto out = cv::Mat<float>(size_y, size_x);
for(int y = 0; y < size_y; ++y)
for(int x = 0; x < size_x; ++x)
{
float ps_x = positions_x(x, y);
float ps_y = positions_y(x, y);
// use interpolation to determine intensity at image(ps_x, ps_y),
// at this point also handle border conditions
// float interpolated = bilinear_interpolation(image, ps_x, ps_y);
out(x, y) = interpolated;
}
return out;
}
One fast way is to use pointer for both A (data) and B (indexes).
const int* pA = A.ptr<int>(0);
const int* pIndexB = B.ptr<int>(0);
int sum = 0;
for(int i = 0; i < Bi.cols; ++i)
{
sum += pA[*pIndexB++];
}
Note: Be carefull with pixel type, in this case (as you write in your code) is int!
Note2: Using cout for each point access put the optimization useless!
Note3: In this article Satya compare four methods for pixel access and fastest seems "foreach": https://www.learnopencv.com/parallel-pixel-access-in-opencv-using-foreach/
I guess it's such an easy question (I'm coming from Java), but I can't figure out how it works.
I simply want to increment an vector element by one. The reason for this is, that I want to compute a histogram out of image values. But whatever I try I just can accomplish to assign a value to the vector. But not to increment it by one!
This is my histogram function:
void histogram(unsigned char** image, int height,
int width, vector<unsigned char>& histogramArray) {
for (int i = 0; i < width; i++) {
for (int j = 0; j < height; j++) {
// histogramArray[1] = (int)histogramArray[1] + (int)1;
// add histogram position by one if greylevel occured
histogramArray[(int)image[i][j]]++;
}
}
// display output
for (int i = 0; i < 256; i++) {
cout << "Position: " << i << endl;
cout << "Histogram Value: " << (int)histogramArray[i] << endl;
}
}
But whatever I try to add one to the histogramArray position, it leads to just 0 in the output. I'm only allowed to assign concrete values like:
histogramArray[1] = 2;
Is there any simple and easy way? I though iterators are hopefully not necesarry at this point, because I know the exakt index position where I want to increment something.
EDIT:
I'm so sorry, I should have been more precise with my question, thank you for your help so far! The code above is working, but it shows a different mean value out of the histogram (difference of around 90) than it should. Also the histogram values are way different than in a graphic program - even though the image values are exactly the same! Thats why I investigated the function and found out if I set the histogram to zeros and then just try to increase one element, nothing happens! This is the commented code above:
for (int i = 0; i < width; i++) {
for (int j = 0; j < height; j++) {
histogramArray[1]++;
// add histogram position by one if greylevel occured
// histogramArray[(int)image[i][j]]++;
}
}
So the position 1 remains 0, instead of having the value height*width. Because of this, I think the correct calculation histogramArray[image[i][j]]++; is also not working properly.
Do you have any explanation for this? This was my main question, I'm sorry.
Just for completeness, this is my mean function for the histogram:
unsigned char meanHistogram(vector<unsigned char>& histogram) {
int allOccurences = 0;
int allValues = 0;
for (int i = 0; i < 256; i++) {
allOccurences += histogram[i] * i;
allValues += histogram[i];
}
return (allOccurences / (float) allValues) + 0.5f;
}
And I initialize the image like this:
unsigned char** image= new unsigned char*[width];
for (int i = 0; i < width; i++) {
image[i] = new unsigned char[height];
}
But there shouldn't be any problem with the initialization code, since all other computations work perfectly and I am able to manipulate and safe the original image. But it's true, that I should change width and height - since I had only square images it didn't matter so far.
The Histogram is created like this and then the function is called like that:
vector<unsigned char> histogramArray(256);
histogram(array, adaptedHeight, adaptedWidth, histogramArray);
So do you have any clue why this part histogramArray[1]++; don't increases my histogram? histogramArray[1] remains 0 all the time! histogramArray[1] = 2; is working perfectly. Also histogramArray[(int)image[i][j]]++; seems to calculate something, but as I said, I think it's wrongly calculating.
I appreciate any help very much! The reason why I used a 2D Array is simply because it is asked for. I like the 1D version also much more, because it's way simpler!
You see, the current problem in your code is not incrementing a value versus assigning to it; it's the way you index your image. The way you've written your histogram function and the image access part puts very fine restrictions on how you need to allocate your images for this code to work.
For example, assuming your histogram function is as you've written it above, none of these image allocation strategies will work: (I've used char instead of unsigned char for brevity.)
char image [width * height]; // Obvious; "char[]" != "char **"
char * image = new char [width * height]; // "char*" != "char **"
char image [height][width]; // Most surprisingly, this won't work either.
The reason why the third case won't work is tough to explain simply. Suffice it to say that a 2D array like this will not implicitly decay into a pointer to pointer, and if it did, it would be meaningless. Contrary to what you might read in some books or hear from some people, in C/C++, arrays and pointers are not the same thing!
Anyway, for your histogram function to work correctly, you have to allocate your image like this:
char** image = new char* [height];
for (int i = 0; i < height; ++i)
image[i] = new char [width];
Now you can fill the image, for example:
for (int i = 0; i < height; ++i)
for (int j = 0; j < width; ++j)
image[i][j] = rand() % 256; // Or whatever...
On an image allocated like this, you can call your histogram function and it will work. After you're done with this image, you have to free it like this:
for (int i = 0; i < height; ++i)
delete[] image[i];
delete[] image;
For now, that's enough about allocation. I'll come back to it later.
In addition to the above, it is vital to note the order of iteration over your image. The way you've written it, you iterate over your columns on the outside, and your inner loop walks over the rows. Most (all?) image file formats and many (most?) image processing applications I've seen do it the other way around. The memory allocations I've shown above also assume that the first index is for the row, and the second is for the column. I suggest you do this too, unless you've very good reasons not to.
No matter which layout you choose for your images (the recommended row-major, or your current column-major,) it is in issue that you should always keep in your mind and take notice of.
Now, on to my recommended way of allocating and accessing images and calculating histograms.
I suggest that you allocate and free images like this:
// Allocate:
char * image = new char [height * width];
// Free:
delete[] image;
That's it; no nasty (de)allocation loops, and every image is one contiguous block of memory. When you want to access row i and column j (note which is which) you do it like this:
image[i * width + j] = 42;
char x = image[i * width + j];
And you'd calculate the histogram like this:
void histogram (
unsigned char * image, int height, int width,
// Note that the elements here are pixel-counts, not colors!
vector<unsigned> & histogram
) {
// Make sure histogram has enough room; you can do this outside as well.
if (histogram.size() < 256)
histogram.resize (256, 0);
int pixels = height * width;
for (int i = 0; i < pixels; ++i)
histogram[image[i]]++;
}
I've eliminated the printing code, which should not be there anyway. Note that I've used a single loop to go through the whole image; this is another advantage of allocating a 1D array. Also, for this particular function, it doesn't matter whether your images are row-major or column major, since it doesn't matter in what order we go through the pixels; it only matters that we go through all the pixels and nothing more.
UPDATE: After the question update, I think all of the above discussion is moot and notwithstanding! I believe the problem could be in the declaration of the histogram vector. It should be a vector of unsigned ints, not single bytes. Your problem seems to be that the value of the vector elements seem to stay at zero when your simplify the code and increment just one element, and are off from the values they need to be when you run the actual code. Well, this could be a symptom of numeric wrap-around. If the number of pixels in your image are a a multiple of 256 (e.g. 32x32 or 1024x1024 image) then it is natural that the sum of their number would be 0 mod 256.
I've already alluded to this point in my original answer. If you read my implementation of the histogram function, you see in the signature that I've declared my vector as vector<unsigned> and have put a comment above it that says this victor counts pixels, so its data type should be suitable.
I guess I should have made it bolder and clearer! I hope this solves your problem.
I'm using the following code to add some noise to an image (straight out of the OpenCV reference, page 449 -- explanation of cv::Mat::begin):
void
simulate_noise(Mat const &in, double stddev, Mat &out)
{
cv::Size s = in.size();
vector<double> noise = generate_noise(s.width*s.height, stddev);
typedef cv::Vec<unsigned char, 3> V4;
cv::MatConstIterator_<V4> in_itr = in.begin<V4>();
cv::MatConstIterator_<V4> in_end = in.end<V4>();
cv::MatIterator_<V4> out_itr = out.begin<V4>();
cv::MatIterator_<V4> out_end = out.end<V4>();
for (; in_itr != in_end && out_itr != out_end; ++in_itr, ++out_itr)
{
int noise_index = my_rand(noise.size());
for (int j = 0; j < 3; ++j)
(*out_itr)[j] = (*in_itr)[j] + noise[noise_index];
}
}
Nothing overly complicated:
in and out are allocated cv::Mat objects of the same dimensions and type
iterate over the input image in
at each position, pick a random value from noise (my_rand(int n) returns a random number in [0..n-1]
sum the pixel from in with the random noise value
put the summation result into out
I don't like this code because the following statement seems unavoidable:
typedef cv::Vec<unsigned char, 3> V4;
It has hard-coded two things:
The images have 3 channels
The channel depth is 8bpp
If I get this typedef wrong (e.g. wrong channel depth or wrong number of channels), then my program segfaults. I originally used typedef cv::Vec<unsigned char, 4> V4 to handle images with an arbitrary number of channels (the max OpenCV supports is 4), but this caused a segfault.
Is there any way I can avoid hard-coding the two things above? Ideally, I want something that's as generic as:
typedef cv::Vec<in.type(), in.size()> V4;
I know this comes late. However, the real solution to your problem is to use OpenCV functionality to do what you want to do.
create noise vector as you do already (or use the functions that OpenCV provides hint!)
shuffle noise vector so you don't need individual noise_index for each pixel; or create vector of randomised noise beforehand
build a matrix header around your shuffled/random vector: cv::Mat_<double>(noise);
use matrix operations for computation: out = in + noise; or cv::add(in, noise, out);
PROFIT!
Another advantage of this method is that OpenCV might employ multithreading, SSE or whatever to speed-up this massive-element operation, which you do not. Your code is simpler, cleaner, and OpenCV does all the nasty type handling for you.
The problem is that you need determine to determine type and number of channels at runtime, but templates need the information at compile time. You can avoid hardcoding the number of channels by either using cv::split and cv::merge, or by changing the iteration to
for(int row = 0; row < in.rows; ++row) {
unsigned char* inp = in.ptr<unsigned char>(row);
unsigned char* outp = out.ptr<unsigned char>(row);
for (int col = 0; col < in.cols; ++col) {
for (int c = 0; c < in.channels(); ++c) {
*outp++ = *inp++ + noise();
}
}
}
If you want to get rid of the dependance of the type, I'd suggest putting the above in a templated function and calling that from your function, depending on the type of the matrix.
They are hardcoded because performance is better that way.
In OpenCV1.x there is cvGet2D() , which can be used here since Mat can be casted as an IplImage.
But it's slow since each time you access a pixel the function will find out the type, size, etc. Specially inefficient in loops.