Robust image segmentation in OpenCV - c++

I'm trying to write an OpenCV program that counts fish eggs for someone else. It currently takes their uploaded image, normalizes, blurs, thresholds, dilates, distance transforms, thresholds again, and then finds contours (like in a typical watershed tutorial).
The problem I'm having is that the lighting conditions can vary quite a bit, so even with my adaptive threshold values, the accuracy of the algorithm also varies wildly. If there's a gradient brightness across the image it seems to do especially poorly. Sometimes the objects are very bright against the background and other times they're almost the same luminosity. Are there any particularly effective ways to find objects in varying light conditions?
Sample images:

Because anything larger than 100 pixels isn't relevant to your image, I would construct a fourier band pass filter to remove these structures.
Here is an implementation I use, based off the one in ImageJ. In this implementation the input image is mirror padded to reduce edge artifacts.
static void GenerateBandFilter(thrust::host_vector<float>& filter, const BandPassSettings& band, const FrameSize& frame)
{
//From https://imagej.nih.gov/ij/plugins/fft-filter.html
if (band.do_band_pass == false)
{
return;
}
if (frame.width != frame.height)
{
throw std::runtime_error("Frame height and width should be the same");
}
auto maxN = static_cast<int>(std::max(frame.width, frame.height));//todo make sure they are the same
auto filterLargeC = 2.0f*band.max_dx / maxN;
auto filterSmallC = 2.0f*band.min_dx / maxN;
auto scaleLargeC = filterLargeC*filterLargeC;
auto scaleSmallC = filterSmallC*filterSmallC;
auto filterLargeR = 2.0f*band.max_dy / maxN;
auto filterSmallR = 2.0f*band.min_dy / maxN;
auto scaleLargeR = filterLargeR*filterLargeR;
auto scaleSmallR = filterSmallR*filterSmallR;
// loop over rows
for (auto j = 1; j < maxN / 2; j++)
{
auto row = j * maxN;
auto backrow = (maxN - j)*maxN;
auto rowFactLarge = exp(-(j*j) * scaleLargeR);
auto rowFactSmall = exp(-(j*j) * scaleSmallR);
// loop over columns
for (auto col = 1; col < maxN / 2; col++)
{
auto backcol = maxN - col;
auto colFactLarge = exp(-(col*col) * scaleLargeC);
auto colFactSmall = exp(-(col*col) * scaleSmallC);
auto factor = (((1 - rowFactLarge*colFactLarge) * rowFactSmall*colFactSmall));
filter[col + row] *= factor;
filter[col + backrow] *= factor;
filter[backcol + row] *= factor;
filter[backcol + backrow] *= factor;
}
}
auto fixy = [&](float t){return isinf(t) ? 0 : t; };
auto rowmid = maxN * (maxN / 2);
auto rowFactLarge = fixy(exp(-(maxN / 2)*(maxN / 2) * scaleLargeR));
auto rowFactSmall = fixy(exp(-(maxN / 2)*(maxN / 2) *scaleSmallR));
filter[maxN / 2] *= ((1 - rowFactLarge) * rowFactSmall);
filter[rowmid] *= ((1 - rowFactLarge) * rowFactSmall);
filter[maxN / 2 + rowmid] *= ((1 - rowFactLarge*rowFactLarge) * rowFactSmall*rowFactSmall); //
rowFactLarge = fixy(exp(-(maxN / 2)*(maxN / 2) *scaleLargeR));
rowFactSmall = fixy(exp(-(maxN / 2)*(maxN / 2) *scaleSmallR));
for (auto col = 1; col < maxN / 2; col++){
auto backcol = maxN - col;
auto colFactLarge = exp(-(col*col) * scaleLargeC);
auto colFactSmall = exp(-(col*col) * scaleSmallC);
filter[col] *= ((1 - colFactLarge) * colFactSmall);
filter[backcol] *= ((1 - colFactLarge) * colFactSmall);
filter[col + rowmid] *= ((1 - colFactLarge*rowFactLarge) * colFactSmall*rowFactSmall);
filter[backcol + rowmid] *= ((1 - colFactLarge*rowFactLarge) * colFactSmall*rowFactSmall);
}
// loop along column 0 and expanded_width/2
auto colFactLarge = fixy(exp(-(maxN / 2)*(maxN / 2) * scaleLargeC));
auto colFactSmall = fixy(exp(-(maxN / 2)*(maxN / 2) * scaleSmallC));
for (auto j = 1; j < maxN / 2; j++) {
auto row = j * maxN;
auto backrow = (maxN - j)*maxN;
rowFactLarge = exp(-(j*j) * scaleLargeC);
rowFactSmall = exp(-(j*j) * scaleSmallC);
filter[row] *= ((1 - rowFactLarge) * rowFactSmall);
filter[backrow] *= ((1 - rowFactLarge) * rowFactSmall);
filter[row + maxN / 2] *= ((1 - rowFactLarge*colFactLarge) * rowFactSmall*colFactSmall);
filter[backrow + maxN / 2] *= ((1 - rowFactLarge*colFactLarge) * rowFactSmall*colFactSmall);
}
filter[0] = (band.remove_dc) ? 0 : filter[0];
}
You can poke around my code that uses it here: https://github.com/kandel3/DPM_PhaseRetrieval

Calculate alpha and beta values of image
image = cv::imread("F:\Dilated.jpg");
int x,y;
int a=0; // variables to be used in loop
int count=0; // variables to be used in loop
for( int y = 0; y < image.rows; y++ )
{ for( int x = 0; x < image.cols; x++ )
{ for( int c = 0; c < 3; c++ )
{
image.at<Vec3b>(y,x)[c] =
saturate_cast<uchar>( alpha*( image.at<Vec3b>(y,x)[c] ) + beta );
}
}
}

Related

Reorganizing nested loops for multithreading

I'm trying to rewrite the main loop in a physics simulation and split the workload between more threads.
It calls dostuff on every unique pair of indices and looks like this:
for (int i = 0; i < n - 1; ++i)
{
for (int j = i + 1; j < n; ++j)
{
dostuff(i, j);
}
}
I came up with two options:
//#1
//sqrt is implemented as binary search on ints, floors the result
for (int x = 0; x < n * (n - 1) / 2; ++x)
{
int i = (1 + sqrt(1 + 8 * x)) / 2;
int j = x - i * (i - 1) / 2;
dostuff(i, j);
}
//#2
for (int x = 0; x < n * n; ++x)
{
int i = x % n;
int j = x / n;
if (i < j)
dostuff(i, j);
}
And for each option, there is corresponding thread loop using shared atomic counter:
//#1
while(int x = counter.fetch_add(1) < n * (n - 1) / 2)
{
int i = (1 + sqrt(1 + 8 * x)) / 2;
int j = x - i * (i - 1) / 2;
dostuff(i, j);
}
//#2
while(int x = counter.fetch_add(1) < n * n)
{
int i = x % n;
int j = x / n;
if (i < j)
dostuff(i, j);
}
My question is, what is the best way to share the workload of the main loop between threads for n < 10^6?
EDIT:
//dostuff
Element& a = elements[i];
Element& b = elements[j];
glm::dvec3 r = b.getPosition() - a.getPosition();
double rv = glm::length(r);
double base = G / (rv * rv);
glm::dvec3 dir = glm::normalize(r);
glm::dvec3 bd = dir * base;
accelerations[i] += bd * b.getMass();
accelerations[j] -= bd * a.getMass();
Your work is a triangle. You want to.divide the triangle into k distinct pieces.
If k is a power of 2 you can do this:
a
a a
b c d
b c d d
Each of those regions are equal in size.

How do I handle edge pixels from a image without any libraries but the standart ones from C++?

I have developed a code that can read and handle the bits from a 24 bits bmp image, mostly applying filters, but now I want to make my blur filter to blur the edge pixels too. Right now I have a 1 pixel edge, I'm using a 3x3 box blur, and this is the image I get after the blur is applied:
https://i.stack.imgur.com/0Px6Z.jpg
I'm able to keep the original bits from the image if I use an if statement in my inner loop but that doesn't really help given that I want it to be blurred and not the original unblurred bits.
Here is the code:
>
for (int count = 0; count < times; ++count) {
for (int x = 1; x < H-1; ++x) {
for (int y = 1; y < W-1; ++y) {
double sum1 = 0;
double sum2 = 0;
double sum3 = 0;
for (int k = -1; k <= 1; ++k) {
for (int j = -1; j <= 1; ++j) {
sum1 += bits[((x - j) * W + (y - k)) * 3] * kernel[j + 1][k + 1];
sum2 += bits[((x - j) * W + (y - k)) * 3 + 1] * kernel[j + 1][k + 1];
sum3 += bits[((x - j) * W + (y - k)) * 3 + 2] * kernel[j + 1][k + 1];
}
}
if (sum1 <= 0) sum1 = 0;
if (sum1 >= 255) sum1 = 255;
if (sum2 <= 0) sum2 = 0;
if (sum2 >= 255) sum2 = 255;
if (sum3 <= 0) sum3 = 0;
if (sum3 >= 255) sum3 = 255;
temp[(x * W + y) * 3] = sum1;
temp[(x * W + y) * 3 + 1] = sum2;
temp[(x * W + y) * 3 + 2] = sum3;
}
}
bits = temp;
}
I know that 5 for loops nested are really slow but I would like to be able to make it work properly first, but if there are any tips on how to improve it I'm all ears.
Now as for the first loop, what it does is it applies the filter the amount of times you want.
The next two is to go through the vector as a 2d vector, and the inner 2 are for the box blur.
Important things to know: I have a vector of bits(RGB) and not just the pixels, that is why I treat them one by one(bits), also my vector is a 1d vector.

Shaping image pixel by pixel in OpenCV

So the idea is that I have and image recorded from cylindrical camera through a rectangular window in it. Image we get is a rectangular picture though it must be circular. I'm using OpenCV to move image pixel by pixel, line by line into a circle from a given rectangular picture. Problem is that pixel distribution is uneven depending on a radius. What algorithm you'd suggest to make distribution more even? Here's the code:
int main( int argc, char** argv ) {
Mat src = imread( "srcImg.jpg", 1 );
Mat dst = imread( "dstImg.jpg", 1 );
int srcH = src.rows; int srcW = src.cols;
int dstH = dst.rows; int dstW = src.cols;
//convert chamber radius to pixels
double alpha, alpha_double, alpha_triple;
int r = 1500;
double k = 210 / (500 * PI);
int l = 1;
//take pixels from source and arrange them into circles
for (int i = srcH - 1; i > 0; i--) {
for (int j = 1; j <= srcW ; j++) {
alpha = (double) (2 * PI * (r * k + i)) / j;
alpha_double = (double) (2 * PI * ((r + 15) * k + i)) / j;
alpha_triple = alpha_double = (double) (2 * PI * ((r + 30) * k + i)) / j;
int x_new = abs((int) (dstW / 2 - (r * k + i) * cos(alpha)) - 200);
int y_new = abs((int) (dstH / 2 - (3.5*(r * k + i) * sin(alpha))) + 1000);
int x_new_double = abs((int) (dstW / 2 - (r * k + i) * cos(alpha_double)) - 200);
int y_new_double = abs((int) (dstH / 2 - (3.5*(r * k + i) * sin(alpha_double))) + 1000);
int x_new_triple = abs((int) (dstW / 2 - (r * k + i) * cos(alpha_triple)) - 200);
int y_new_triple = abs((int) (dstH / 2 - (3.5*(r * k + i) * sin(alpha_triple))) + 1000);
dst.at<uchar>(x_new, y_new) = src.at<uchar>(srcH - i, srcW - j);
dst.at<uchar>(x_new_double, y_new_double) = src.at<uchar>(srcH - i, srcW - j);
dst.at<uchar>(x_new_triple, y_new_triple) = src.at<uchar>(srcH - i, srcW - j);
}
}
//make dst image grey and show all images
Mat dstGray;
cvtColor(dst, dstGray, CV_RGB2GRAY);
imshow("Source", src);
imshow("Result", dstGray);
waitKey();
return 0;
And the
This is hardly a full answer, but I would try some kind of projection mapping instead of manually accessing every pixel. There must be an openCV way to create a destination shape then say : "Take my original image and make it fit the destination shape"
This is rather trivial for rectangles as you can see here, but your hole in the middle makes it harder.

cvtColor too Slow

I'm making a project where i need to change the lightness, and contrast of an image, it's lightness not brightness.
So my code at the start was
for (int y = 0; y < dst.rows; y++) {
for (int x = 0; x < dst.cols; x++) {
int b = dst.data[dst.channels() * (dst.cols * y + x) + 0];
int g = dst.data[dst.channels() * (dst.cols * y + x) + 1];
int r = dst.data[dst.channels() * (dst.cols * y + x) + 2];
... other processing stuff i'm doing
and it's good, doing it really fast, but when i try to make the hsv to hsl conversion to set the l value that i need it gets reaaaaaaally slow;
my hsl to hsl lines of code are
cvtColor(dst, dst, CV_BGR2HSV);
Vec3b pixel = dst.at<cv::Vec3b>(y, x); // read pixel (0,0)
double H = pixel.val[0];
double S = pixel.val[1];
double V = pixel.val[2];
h = H;
l = (2 - S) * V;
s = s * V;
s /= (l <= 1) ? (l) : 2 - (l);
l /= 2;
/* i will further make here the calcs to set the l i want */
H = h;
l *= 2;
s *= (l <= 1) ? l : 2 - l;
V = (l + s) / 2;
S = (2 * s) / (l + s);
pixel.val[0] = H;
pixel.val[1] = S;
pixel.val[2] = V;
cvtColor(dst, dst, CV_HSV2BGR);
and i ran it and was slow, so i was take of the lines to see which one was making it slow and i figure out it was cvtColor(dst, dst, CV_BGR2HSV);
So there's a way to make it faster than using cvtCOlor, or its time issue is something that can be handled?
I think (I haven't opened the text editor, but it seems) that you need to generate the entire image in HSV and then call cvtColor once for the entire image. Meaning that you should call cvtColor once instead of once for every pixel. That should give you a significant boost in speed.
You would do this:
cvtColor(dst, dst, CV_BGR2HSV);
for (int y = 0; y < dst.rows; y++) {
for (int x = 0; x < dst.cols; x++) {
Vec3b pixel = dst.at<cv::Vec3b>(y, x); // read current pixel
double H = pixel.val[0];
double S = pixel.val[1];
double V = pixel.val[2];
h = H;
l = (2 - S) * V;
s = s * V;
s /= (l <= 1) ? (l) : 2 - (l);
l /= 2;
H = h;
l *= 2;
s *= (l <= 1) ? l : 2 - l;
V = (l + s) / 2;
S = (2 * s) / (l + s);
pixel.val[0] = H;
pixel.val[1] = S;
pixel.val[2] = V;
}
}
cvtColor(dst, dst, CV_HSV2BGR);

How to implement overlap add method?

I implemented my filter, where overlap add method to prevent circular convultion is used.
input - file with noise, output should be filtered file.
My result: out is slightly modified, frequencies aren`t cut
My guess is that I wrongly multiply in the frequency domain input signal on the filter kernel
(My intention is to cut off frequencies that aren't in range [300,3700]). How multiplication should be done?
I construct kernel using blackmanwindow - is my understanding correct? ( I compute amount of frequency per one sample of filter, then go through samples and see if it is in range I want to cut off I calculate frequency using formula for Blackman window.)
I just started learning DSP.
Here is my implementation (what is wrong with it???):
void DeleteFrequencies(char* fileWithNoise, char* resultFile, const int bufferSize, int lowestFrequency, int highestFrequency, int sampleRate )
{
// |1|. files
std::fstream in;
std::fstream out;
in.open (fileWithNoise, std::ios::in | std::ios::binary);
out.open(resultFile, std::ios::out | std::ios::binary);
// |2|. Filter kernel design. I shall use blackman window
// fundamental params
const int filterKernelLength = 200; // 512
const int filterMaxFrequency = sampleRate / 2; // 8000 / 2
const int frequencyPerSamle = filterMaxFrequency / filterKernelLength;
double *RealFilterResp = new double [bufferSize / 2];
double *ImmFilterResp = new double [bufferSize / 2];
// coefficients for Blackman window
const double a0 = 0.42659;
const double a1 = 0.49656;
const double a2 = 0.076849;
// construct filter kernel
for (int i = 0 ; i < bufferSize / 2; ++i)
{
if ( i >= filterKernelLength ) // padd filter kernel with zeroes
{
RealFilterResp[i] = 0;
ImmFilterResp[i] = 0;
}
else if (i * frequencyPerSamle < lowestFrequency || i * frequencyPerSamle > highestFrequency)
{
// apply blackman window (to eleminate frequencies < 300 hz and > 3700 hz)
RealFilterResp[i] = a0 - a1 * cos (2 * M_PI * i / (bufferSize / 2 - 1)) + a2 * cos (4 * M_PI / (bufferSize / 2 - 1));
ImmFilterResp[i] = a0 - a1 * cos (2 * M_PI * i / (bufferSize / 2 - 1)) + a2 * cos (4 * M_PI / (bufferSize / 2 - 1));
}
else
{
RealFilterResp[i] = 1;
ImmFilterResp[i] = 1;
}
}
// |3|. overlap add method
// calculate parameters for overlap add method (we use it to prevent circular convultion)
const int FFT_length = pow (2.0 ,(int)(log(bufferSize + filterKernelLength - 1.0)/log(2.0)) + 1.0);
double *OLAP = new double[bufferSize / 2 ]; // holds the overlapping samples from segment to segment
memset(OLAP,0, bufferSize / 2 * sizeof (double));
double *RealX = new double[bufferSize];
memset(RealX, 0, bufferSize * sizeof(double));
double *ImmX = new double[bufferSize];
memset(ImmX, 0, bufferSize * sizeof(double));
short* audioDataBuffer = new short[bufferSize];
memset(audioDataBuffer, 0 , sizeof(short) * bufferSize);
// start reading from file by chunks of bufferSize
while (in.good())
{
// get proper chunk of data
FillBufferFromFile(audioDataBuffer, bufferSize, in); // read chunk from file
ShortArrayToDoubleArray(audioDataBuffer, RealX, bufferSize); // fill RealPart
ForwardRealFFT(RealX, ImmX, bufferSize); // go to frequency domain
// perform convultion as multiplication in frequency domain
for (int j = 0; j < bufferSize / 2; ++j)
{
double tmp = RealX[j] * RealFilterResp[j] - ImmX[j] * ImmFilterResp[j];
ImmX[j] = RealX[j] * ImmFilterResp[j] + ImmX[j] * RealFilterResp[j];
RealX[j] = tmp;
}
// Inverse FFT
ReverseRealFFT(RealX, ImmX, bufferSize); // go to time domain
// add last segment overlap to this segment
for (int j = 0; j < filterKernelLength - 2; ++j )
{
RealX[j] += OLAP[j];
}
// save samples that will overlap the next segment
for (int j = bufferSize/2 + 1; j < bufferSize; ++j )
{
OLAP[j - bufferSize/2 - 1] = audioDataBuffer[j];
}
// write results
DoubleArrayToShortArray(RealX, audioDataBuffer, bufferSize);
FillFileFromBuffer(audioDataBuffer, bufferSize, out);
}
/*ReverseRealFFT(RealX, ImmX, bufferSize
);
DoubleArrayToShortArray(RealX, audioDataBuffer, bufferSize);*/
delete [] audioDataBuffer;
delete [] RealFilterResp;
delete [] ImmFilterResp;
delete [] OLAP;
delete [] RealX;
delete [] ImmX;
in.close();
out.close();
}
If your intention is to use the window method to implement the filter, the window should multiply the time-domain sequence corresponding to the infinite impulse response of the ideal bandpass filter.
Specifically, for a bandpass filter of bandwidth w0=2*pi*(3700-300)/8000 centered at wc=2*pi*(300+3700)/8000, the ideal impulse response would be (for -infinity < n < infinity):
w0*sinc(0.5*w0*n/pi) * cos(wc*n) / pi
Which you would shift to the interval [0,N-1], and then apply the window that you computed:
double sinc(double x) {
if (fabs(x)<1e-6) return 1.0;
return sin(M_PI * x)/(M_PI * x);
}
void bandpassDesign(int N, double* filterImpulseResponse) {
double w0 = 2*(3700-300)*M_PI/8000;
double wc = 2*(300+3700)*M_PI/8000;
double shift = 0.5*N;
for (int i = 0; i < bufferSize; ++i) {
double truncatedIdealResponse = w0*sinc(0.5*w0*(i-shift)/M_PI) * cos(wc*i) / M_PI;
double window = a0 - a1 * cos (2 * M_PI * i / (N- 1)) + a2 * cos (4 * M_PI * i / (N- 1));
filterImpulseResponse[i] = truncatedIdealResponse * window;
}
}
You can then take the FFT to obtain the frequency-domain coefficients. Remember that if you intend on filtering data using this filter, the time sequence has to be zero padded.
For example, if you wish to use a 1024-point FFT with the overlap-add method, and assuming a 128-point filter kernel meets your filter design specifications, you would call bandpassDesign with N=128, pad with 1024-128=896 zeros, then take the FFT.
Your window coefficients are wrong - the window function is purely real, and you are going to multiply your (complex) frequency domain data with these real coeffs. So your filter coef initialisation:
double *RealFilterResp = new double [bufferSize / 2];
double *ImmFilterResp = new double [bufferSize / 2];
if ( i >= filterKernelLength ) // padd filter kernel with zeroes
{
RealFilterResp[i] = 0;
ImmFilterResp[i] = 0;
}
else if (i * frequencyPerSamle < lowestFrequency || i * frequencyPerSamle > highestFrequency)
{
// apply blackman window (to eleminate frequencies < 300 hz and > 3700 hz)
RealFilterResp[i] = a0 - a1 * cos (2 * M_PI * i / (bufferSize / 2 - 1)) + a2 * cos (4 * M_PI / (bufferSize / 2 - 1));
ImmFilterResp[i] = a0 - a1 * cos (2 * M_PI * i / (bufferSize / 2 - 1)) + a2 * cos (4 * M_PI / (bufferSize / 2 - 1));
}
else
{
RealFilterResp[i] = 1;
ImmFilterResp[i] = 1;
}
should just be:
double *FilterResp = new double [bufferSize / 2];
if ( i >= filterKernelLength ) // padd filter kernel with zeroes
{
FilterResp[i] = 0;
}
else if (i * frequencyPerSamle < lowestFrequency || i * frequencyPerSamle > highestFrequency)
{
FilterResp[i] = a0 - a1 * cos (2 * M_PI * i / (bufferSize / 2 - 1)) + a2 * cos (4 * M_PI / (bufferSize / 2 - 1));
}
else
{
FilterResp[i] = 1;
}
and the frequency domain multiplication:
for (int j = 0; j < bufferSize / 2; ++j)
{
double tmp = RealX[j] * RealFilterResp[j] - ImmX[j] * ImmFilterResp[j];
ImmX[j] = RealX[j] * ImmFilterResp[j] + ImmX[j] * RealFilterResp[j];
RealX[j] = tmp;
}
should just be:
for (int j = 0; j < bufferSize / 2; ++j)
{
RealX[j] *= FilterResp[j];
ImmX[j] *= FilterResp[j];
}