Fast method to access random image pixels and at most once - c++

I'm learning OpenCV (C++) and as a simple practice, I designed a simple effect which makes some of image pixels black or white. I want each pixel to be edited at most once; so I added address of all pixels to a vector. But it made my code very slow; specially for large images or high amounts of effect. Here is my code:
void effect1(Mat& img, float amount) // 100 ≥ amount ≥ 0
{
vector<uchar*> addresses;
int channels = img.channels();
uchar* lastAddress = img.ptr<uchar>(0) + img.total() * channels;
for (uchar* i = img.ptr<uchar>(0); i < lastAddress; i += channels) addresses.push_back(i); //Fast Enough
size_t count = img.total() * amount / 100 / 2;
for (size_t i = 0; i < count; i++)
{
size_t addressIndex = xor128() % addresses.size(); //Fast Enough, xor128() is a fast random number generator
for (size_t j = 0; j < channels; j++)
{
*(addresses[addressIndex] + j) = 255;
} //Fast Enough
addresses.erase(addresses.begin() + addressIndex); // MAKES CODE EXTREMELY SLOW
}
for (size_t i = 0; i < count; i++)
{
size_t addressIndex = xor128() % addresses.size(); //Fast Enough, xor128() is a fast random number generator
for (size_t j = 0; j < channels; j++)
{
*(addresses[addressIndex] + j) = 0;
} //Fast Enough
addresses.erase(addresses.begin() + addressIndex); // MAKES CODE EXTREMELY SLOW
}
}
I think rearranging vector items after erasing an item is what makes my code slow (if I remove addresses.erase, code will run fast).
Is there any fast method to select each random item from a collection (or a number range) at most once?
Also: I'm pretty sure such effect already exists. Does anyone know the name of it?

This answer assumes you have a random bit generator function, since std::random_shuffle requires that. I don't know how xor128 works, so I'll use the functionality of the <random> library.
If we have a population of N items, and we want to select groups of size j and k randomly from that population with no overlap, we can write down the index of each item on a card, shuffle the deck, draw j cards, and then draw k cards. Everything left over is discarded. We can achieve this with the <random> library. Answer pending on how to incorporate a custom PRNG like you implemented with xor128.
This assumes that random_device won't work on your system (many compilers implement it in a way that it will always return the same sequence) so we seed the random generator with current time like the good old fashioned srand our mother used to make.
Untested since I don't know how to use OpenCV. Anyone with a lick of experience with that please edit as appropriate.
#include <ctime> // for std::time
#include <numeric> // for std::iota
#include <random>
#include <vector>
void effect1(Mat& img, float amount, std::mt19937 g) // 0.0 ≥ amount ≥ 1.00
{
std::vector<cv::Size> ind(img.total());
std::iota(ind.begin(), ind.end(), 0); // fills with 0, 1, 2, ...
std::random_shuffle(ind.begin(), ind.end(), g);
cv::Size count = img.total() * amount;
auto white = get_white<Mat>(); // template function to return this matrix' concept of white
// could easily replace with cv::Vec3d(255,255,255)
// if all your matrices are 3 channel?
auto black = get_black<Mat>(); // same but... opposite
auto end = ind.begin() + count;
for (auto it = ind.begin(), it != end; ++it)
{
img.at(*it) = white;
}
end = (ind.begin() + 2 * count) > ind.end() ?
ind.end() :
ind.begin() + 2 * count;
for (auto it = ind.begin() + count; it != end; ++it)
{
img.at(*it) = black;
}
}
int main()
{
std::mt19937 g(std::time(nullptr)); // you normally see this seeded with random_device
// but that's broken on some implementations
// adjust as necessary for your needs
cv::Mat mat = ... // make your cv objects
effect1(mat, 0.1, g);
// display it here
}
Another approach
Instead of shuffling indices and drawing cards from a deck, assume each pixel has a random probability of switching to white, switching to black, or staying the same. If your amount is 0.4, then select a random number between 0.0 and 1.0, any result between 0.0 and 0.4 flips the pixel black, and betwen 0.4 and 0.8 flips it white, otherwise it stays the same.
General algorithm:
given probability of flipping -> f
for each pixel in image -> p:
get next random float([0.0, 1.0)) -> r
if r < f
then p <- BLACK
else if r < 2*f
then p <- WHITE
You won't get the same number of white/black pixels each time, but that's randomness! We're generating a random number for each pixel anyway for the shuffling algorithm. This has the same complexity unless I'm mistaken.

Also: I'm pretty sure such effect already exists. Does anyone know the name of it?
The effect you're describing is called salt and pepper noise. There is no direct implementation in OpenCV that I know of though.
I think rearranging vector items after erasing an item is what makes
my code slow (if I remove addresses.erase, code will run fast).
Im not sure why you add your pixels to a vector in your code, it would make much more sense and also be much more performant to directly work on the Mat object and change the pixel value directly. You could use OpenCVs inbuild Mat.at() function to directly change the pixel values to either 0 or 255.
I would create a single loop which generates random indexes in the range of your image dimension and manipulate the image pixels directly. That way you are in O(n) for your noise addition. You could also just search for "OpenCV" and "salt and pepper noise", I am sure there already are a lot of really performant implementations.

I also post a simpler code:
void saltAndPepper(Mat& img, float amount)
{
vector<size_t> pixels(img.total()); // size_t = unsigned long long
uchar channels = img.channels();
iota(pixels.begin(), pixels.end(), 0); // Fill vector with 0, 1, 2, ...
shuffle(pixels.begin(), pixels.end(), mt19937(time(nullptr))); // Shuffle the vector
size_t count = img.total() * amount / 100 / 2;
for (size_t i = 0; i < count; i++)
{
for (size_t j = 0; j < channels; j++) // Set all pixel channels (e.g. Grayscale with 1 channel or BGR with 3 channels) to 255
{
*(img.ptr<uchar>(0) + (pixels[i] * channels) + j) = 255;
}
}
for (size_t i = count; i < count*2; i++)
{
for (size_t j = 0; j < channels; j++) // Set all pixel channels (e.g. Grayscale with 1 channel or BGR with 3 channels) to 0
{
*(img.ptr<uchar>(0) + (pixels[i] * channels) + j) = 0;
}
}
}

Related

Opencv obatin certain pixel RGB value based on mask

My title may not be clear enough, but please look carefully on the following description.Thanks in advance.
I have a RGB image and a binary mask image:
Mat img = imread("test.jpg")
Mat mask = Mat::zeros(img.rows, img.cols, CV_8U);
Give some ones to the mask, assume the number of ones is N. Now the nonzero coordinates are known, based on these coordinates, we can surely obtain the corresponding pixel RGB value of the origin image.I know this can be accomplished by the following code:
Mat colors = Mat::zeros(N, 3, CV_8U);
int counter = 0;
for (int i = 0; i < mask.rows; i++)
{
for (int j = 0; j < mask.cols; j++)
{
if (mask.at<uchar>(i, j) == 1)
{
colors.at<uchar>(counter, 0) = img.at<Vec3b>(i, j)[0];
colors.at<uchar>(counter, 1) = img.at<Vec3b>(i, j)[1];
colors.at<uchar>(counter, 2) = img.at<Vec3b>(i, j)[2];
counter++;
}
}
}
And the coords will be as follows:
enter image description here
However, this two layer of for loop costs too much time. I was wondering if there is a faster method to obatin colors, hope you guys can understand what I was trying to convey.
PS:If I can use python, this can be done in only one sentence:
colors = img[mask == 1]
The .at() method is the slowest way to access Mat values in C++. Fastest is to use pointers, but best practice is an iterator. See the OpenCV tutorial on scanning images.
Just a note, even though Python's syntax is nice for something like this, it still has to loop through all of the elements at the end of the day---and since it has some overhead before this, it's de-facto slower than C++ loops with pointers. You necessarily need to loop through all the elements regardless of your library, you're doing comparisons with the mask for every element.
If you are flexible with using any other open source library using C++, try Armadillo. You can do all linear algebra operations with it and also, you can reduce above code to one line(similar to your Python code snippet).
Or
Try findNonZero()function and find all coordinates in image containing non-zero values. Check this: https://stackoverflow.com/a/19244484/7514664
Compile with optimization enabled, try profiling this version and tell us if it is faster:
vector<Vec3b> colors;
if (img.isContinuous() && mask.isContinuous()) {
auto pimg = img.ptr<Vec3b>();
for (auto pmask = mask.datastart; pmask < mask.dataend; ++pmask, ++pimg) {
if (*pmask)
colors.emplace_back(*pimg);
}
}
else {
for (int r = 0; r < img.rows; ++r) {
auto prowimg = img.ptr<Vec3b>(r);
auto prowmask = img.ptr(r);
for (int c = 0; c < img.cols; ++c) {
if (prowmask[c])
colors.emplace_back(prowimg[c]);
}
}
}
If you know the size of colors, reserve the space for it beforehand.

Intensity Histogram ++

I'm writing my own Intensity histogram for greyscale images where the number of bins is passed into the function.
This is what i have so far:
std::vector<unsigned int> Image::histogram(const int bins)
{
std::vector<unsigned int> histogram(bins ,0);
for (unsigned int i(0); i < bins; i++)
{
for (unsigned int j(0); j < m_height * m_width; ++j)
{
if (i == m_p_image[j])
{
histogram[i]++;
}
}
}
return histogram;
}
This works perfectly for 256 bins as each count is added to histogram, but for 128 bins its misses the second half of the image, I know I need to implement a way of grouping points together if the bin size is less than 256 but I'm unsure how to do this.
Your code strikes me as unnecessarily clumsy. There's no real need for the outer loop.
To answer the question you asked, however, the usual way to do this would be to use linear interpolation--that is, find the proportional position of a value in the input range, then increment the same proportional position in the output range.
for (j =0; j<height * width; j++) {
double input_pos = image[j] / 256.0;
int output_pos = int(input_pos * bin_count);
++histogram[output_pos];
}
Given that these are colors, you could (if you chose to) apply a gamma curve instead of doing linear interpolation. The reason to do that would be if you wanted to model how you see colors instead of just basing the histogram on the input numbers themselves. The difference between the two is based on the fact that vision is something like logarithmic instead of linear, so a linear histogram (especially if you're using relatively few bins compared to the number of possible input values) doesn't represent what we see very accurately.

Weird but close fft and ifft of image in c++

I wrote a program that loads, saves, and performs the fft and ifft on black and white png images. After much debugging headache, I finally got some coherent output only to find that it distorted the original image.
input:
fft:
ifft:
As far as I have tested, the pixel data in each array is stored and converted correctly. Pixels are stored in two arrays, 'data' which contains the b/w value of each pixel and 'complex_data' which is twice as long as 'data' and stores real b/w value and imaginary parts of each pixel in alternating indices. My fft algorithm operates on an array structured like 'complex_data'. After code to read commands from the user, here's the code in question:
if (cmd == "fft")
{
if (height > width) size = height;
else size = width;
N = (int)pow(2.0, ceil(log((double)size)/log(2.0)));
temp_data = (double*) malloc(sizeof(double) * width * 2); //array to hold each row of the image for processing in FFT()
for (i = 0; i < (int) height; i++)
{
for (j = 0; j < (int) width; j++)
{
temp_data[j*2] = complex_data[(i*width*2)+(j*2)];
temp_data[j*2+1] = complex_data[(i*width*2)+(j*2)+1];
}
FFT(temp_data, N, 1);
for (j = 0; j < (int) width; j++)
{
complex_data[(i*width*2)+(j*2)] = temp_data[j*2];
complex_data[(i*width*2)+(j*2)+1] = temp_data[j*2+1];
}
}
transpose(complex_data, width, height); //tested
free(temp_data);
temp_data = (double*) malloc(sizeof(double) * height * 2);
for (i = 0; i < (int) width; i++)
{
for (j = 0; j < (int) height; j++)
{
temp_data[j*2] = complex_data[(i*height*2)+(j*2)];
temp_data[j*2+1] = complex_data[(i*height*2)+(j*2)+1];
}
FFT(temp_data, N, 1);
for (j = 0; j < (int) height; j++)
{
complex_data[(i*height*2)+(j*2)] = temp_data[j*2];
complex_data[(i*height*2)+(j*2)+1] = temp_data[j*2+1];
}
}
transpose(complex_data, height, width);
free(temp_data);
free(data);
data = complex_to_real(complex_data, image.size()/4); //tested
image = bw_data_to_vector(data, image.size()/4); //tested
cout << "*** fft success ***" << endl << endl;
void FFT(double* data, unsigned long nn, int f_or_b){ // f_or_b is 1 for fft, -1 for ifft
unsigned long n, mmax, m, j, istep, i;
double wtemp, w_real, wp_real, wp_imaginary, w_imaginary, theta;
double temp_real, temp_imaginary;
// reverse-binary reindexing to separate even and odd indices
// and to allow us to compute the FFT in place
n = nn<<1;
j = 1;
for (i = 1; i < n; i += 2) {
if (j > i) {
swap(data[j-1], data[i-1]);
swap(data[j], data[i]);
}
m = nn;
while (m >= 2 && j > m) {
j -= m;
m >>= 1;
}
j += m;
};
// here begins the Danielson-Lanczos section
mmax = 2;
while (n > mmax) {
istep = mmax<<1;
theta = f_or_b * (2 * M_PI/mmax);
wtemp = sin(0.5 * theta);
wp_real = -2.0 * wtemp * wtemp;
wp_imaginary = sin(theta);
w_real = 1.0;
w_imaginary = 0.0;
for (m = 1; m < mmax; m += 2) {
for (i = m; i <= n; i += istep) {
j = i + mmax;
temp_real = w_real * data[j-1] - w_imaginary * data[j];
temp_imaginary = w_real * data[j] + w_imaginary * data[j-1];
data[j-1] = data[i-1] - temp_real;
data[j] = data[i] - temp_imaginary;
data[i-1] += temp_real;
data[i] += temp_imaginary;
}
wtemp = w_real;
w_real += w_real * wp_real - w_imaginary * wp_imaginary;
w_imaginary += w_imaginary * wp_real + wtemp * wp_imaginary;
}
mmax=istep;
}}
My ifft is the same only with the f_or_b set to -1 instead of 1. My program calls FFT() on each row, transposes the image, calls FFT() on each row again, then transposes back. Is there maybe an error with my indexing?
Not an actual answer as this question is Debug only so some hints instead:
your results are really bad
it should look like this:
first line is the actual DFFT result
Re,Im,Power is amplified by a constant otherwise you would see a black image
the last image is IDFFT of the original not amplified Re,IM result
the second line is the same but the DFFT result is wrapped by half size of image in booth x,y to match the common results in most DIP/CV texts
As you can see if you IDFFT back the wrapped results the result is not correct (checker board mask)
You have just single image as DFFT result
is it power spectrum?
or you forget to include imaginary part? to view only or perhaps also to computation somewhere as well?
is your 1D **DFFT working?**
for real data the result should be symmetric
check the links from my comment and compare the results for some sample 1D array
debug/repair your 1D FFT first and only then move to the next level
do not forget to test Real and complex data ...
your IDFFT looks BW (no gray) saturated
so did you amplify the DFFT results to see the image and used that for IDFFT instead of the original DFFT result?
also check if you do not round to integers somewhere along the computation
beware of (I)DFFT overflows/underflows
If your image pixel intensities are big and the resolution of image too then your computation could loss precision. Newer saw this in images but if your image is HDR then it is possible. This is a common problem with convolution computed by DFFT for big polynomials.
Thank you everyone for your opinions. All that stuff about memory corruption, while it makes a point, is not the root of the problem. The sizes of data I'm mallocing are not overly large, and I am freeing them in the right places. I had a lot of practice with this while learning c. The problem was not the fft algorithm either, nor even my 2D implementation of it.
All I missed was the scaling by 1/(M*N) at the very end of my ifft code. Because the image is 512x512, I needed to scale my ifft output by 1/(512*512). Also, my fft looks like white noise because the pixel data was not rescaled to fit between 0 and 255.
Suggest you look at the article http://www.yolinux.com/TUTORIALS/C++MemoryCorruptionAndMemoryLeaks.html
Christophe has a good point but he is wrong about it not being related to the problem because it seems that in modern times using malloc instead of new()/free() does not initialise memory or select best data type which would result in all problems listed below:-
Possibly causes are:
Sign of a number changing somewhere, I have seen similar issues when a platform invoke has been used on a dll and a value is passed by value instead of reference. It is caused by memory not necessarily being empty so when your image data enters it will have boolean maths performed on its values. I would suggest that you make sure memory is empty before you put your image data there.
Memory rotating right (ROR in assembly langauge) or left (ROL) . This will occur if data types are being used which do not necessarily match, eg. a signed value entering an unsigned data type or if the number of bits is different in one variable to another.
Data being lost due to an unsigned value entering a signed variable. Outcomes are 1 bit being lost because it will be used to determine negative or positive, or at extremes if twos complement takes place the number will become inverted in meaning, look for twos complement on wikipedia.
Also see how memory should be cleared/assigned before use. http://www.cprogramming.com/tutorial/memory_debugging_parallel_inspector.html

FFT of an image

I have an assignment about fftw and I was trying to write a small program to create an fft of an image. I am using CImg to read and write images. But all I get is a dark image with a single white dot :(
I'm most likely doing this the wrong way and I would appreciate if someone could explain how this should be done. I don't need the code, I just need to know what is the right way to do this.
Here is my code:
CImg<double> input("test3.bmp");
CImg<double> image_fft(input, false);
unsigned int nx = input.dimx(), ny = input.dimy();
size_t align = sizeof(Complex);
array2<Complex> in (nx, ny, align);
fft2d Forward(-1, in);
for (int i = 0; i < input.dimx(); ++i) {
for (int j = 0; j < input.dimy(); ++j) {
in(i,j) = input(i,j);
}
}
Forward.fft(in);
for (int i = 0; i < input.dimx(); ++i) {
for (int j = 0; j < input.dimy(); ++j) {
image_fft(i,j,0) = image_fft(i,j,1) = image_fft(i,j,2) = std::abs(in(i,j));
}
}
image_fft.normalize(0, 255);
image_fft.save("test.bmp");
You need to take the log of the magnitude. The single white dot is the base value (0 Hz, DC, whatever you want to call it), so it will almost ALWAYS be by far the largest component of any image you take (Since pixel values cannot be negative, the DC value will always be positive and large).
What you need to do is calculate the log (ln, whatever, some type of logarithmic calculation) of the magnitude (so after you've converted from complex to magnitude/phase form (phasor notation iirc?)) on each point before you normalize it.
Please note that the values are there, they are just REALLY small compared to the DC value, taking the log (Which makes smaller values bigger by a lot, and bigger values only slightly larger) will make the other frequencies visible.

How to smooth a histogram?

I want to smooth a histogram.
Therefore I tried to smooth the internal matrix of cvHistogram.
typedef struct CvHistogram
{
int type;
CvArr* bins;
float thresh[CV_MAX_DIM][2]; /* for uniform histograms */
float** thresh2; /* for non-uniform histograms */
CvMatND mat; /* embedded matrix header for array histograms */
}
I tried to smooth the matrix like this:
cvCalcHist( planes, hist, 0, 0 ); // Compute histogram
(...)
// smooth histogram with Gaussian Filter
cvSmooth( hist->mat, hist_img, CV_GAUSSIAN, 3, 3, 0, 0 );
Unfortunately, this is not working because cvSmooth needs a CvMat as input instead of a CvMatND. I couldn't transform CvMatND into CvMat (CvMatND is 2-dim in my case).
Is there anybody who can help me? Thanks.
You can use the same basic algorithm used for Mean filter, just calculating the average.
for(int i = 1; i < NBins - 1; ++i)
{
hist[i] = (hist[i - 1] + hist[i] + hist[i + 1]) / 3;
}
Optionally you can use a slightly more flexible algorithm allowing you to easily change the window size.
int winSize = 5;
int winMidSize = winSize / 2;
for(int i = winMidSize; i < NBins - winMidSize; ++i)
{
float mean = 0;
for(int j = i - winMidSize; j <= (i + winMidSize); ++j)
{
mean += hist[j];
}
hist[i] = mean / winSize;
}
But bear in mind that this is just one simple technique.
If you really want to do it using OpenCv tools, I recommend you access the openCv forum: http://tech.groups.yahoo.com/group/OpenCV/join
You can dramatically change the "smoothness" of a histogram by changing the number of bins you use. A good rule of thumb is to have sqrt(n) bins if you have n data points. You might try applying this heuristic to your histogram and see if you get a better result.