local thresholding implementation in C++ / OpenCV - c++

Question

I would like to implement a local thresholding algorithm and I require your expertise.
my images are resized to 600x400, grayscale.
Basic Thought process on localizing:
Segment the images using a 9x9 ROI taken at each pixel and calculating the maximum intensity in the region.
create a 9x9 Kernel.
condition:
if the center pixel of the mask is above 50% of the maximum intensity, set the center pixel true.(apply mask)
my question to you:
How should I pick my kernel/mask ?
cv::Mat ROI;
cv::Mat mask(input.size(),CV_8UC1, cv::Scalar::all(0)); // create mask of 0s at first
const int kerneldepth = 1;
const int kernelsize = 9;
cv::Mat kernel = cv::Mat::ones( kernelsize, kernelsize, CV_8UC1 );
//take ROI of 9x9 and apply a threshold
for( double x = 9; x < input.cols -9; x++ ){
for( double y = 9 ; y < input.rows - 9 ; y++ ){
try{
double x_left = x - 4;
double x_right = x + 4;
double y_up = y + 4;
double y_down = y - 4;
double maxVal;
double minVal;
cv::Point anchor(kernelsize/2,kernelsize/2);
cv::Rect ROI = cv::Rect(x_left,y_down,9,9);
cv::Mat ROI_Mat = input(ROI); // a new matrix for ROI
cv::Scalar avgPixelIntensity = cv::mean( ROI_Mat ); // calculate mean
cv::minMaxLoc(ROI_Mat,&minVal,&maxVal);
if( input.at<uchar>(x,y) >= 0.5*maxVal){
cv::filter2D(input,mask,-1,kernel,anchor,0);
} else { break;}
}
catch (cv::Exception &e){
e.what();
}
}
*****************************UPDATED CODE: ******************************************
applyLocalThresh(cv::Mat &src, cv::Mat& out){
double maxVal, minVal;
cv::Mat output;
int top, bottom, left , right;
int borderType = cv::BORDER_CONSTANT;
cv::Scalar value;
top = (int) (9); bottom = (int) (9);
left = (int) (9); right = (int) (9);
output = src;
out = src;
value = 0;
cv::copyMakeBorder(src,output,top,bottom,left,right,borderType,value);
for(int y = 9; y < src.rows; y++) {
for(int x = 9; x < src.cols; x ++) {
cv::Mat ROI = src(cv::Rect(cv::Point(x-4,y-4),cv::Size(9,9)));
cv::minMaxLoc(ROI,&minVal,&maxVal);
if(src.at<uchar>(cv::Point(x-4,y-4)) >= 0.6*maxVal){
out.at<uchar>(cv::Point(x-4,y-4)) = 255;
}else{
out.at<uchar>(cv::Point(x-4,y-4));
}
}
}
}

Answer 1

You can do this with a dilation followed by a comparison in OpenCV;
im = load image here;
di = dilate im with a 9x9 kernel;
bw = im > (di * 0.5); // in OpenCV, pixels of bw are set to 255 or 0
A simple example to illustrate this with a 4x6 image and a 3x3 kernel in Matlab/Octave:
im =
1 2 3 4 5 6
2 3 4 5 6 7
3 4 5 6 7 8
4 5 6 7 8 9
di =
3 4 5 6 7 7
4 5 6 7 8 8
5 6 7 8 9 9
5 6 7 8 9 9
th = di * .5
th =
1.5000 2.0000 2.5000 3.0000 3.5000 3.5000
2.0000 2.5000 3.0000 3.5000 4.0000 4.0000
2.5000 3.0000 3.5000 4.0000 4.5000 4.5000
2.5000 3.0000 3.5000 4.0000 4.5000 4.5000
bw = im > th
bw =
0 0 1 1 1 1
0 1 1 1 1 1
1 1 1 1 1 1
1 1 1 1 1 1

Answer 2

I fear this approach is not entirely correct. Let me explain: for operations involving a kernel, one must be careful to place the center of the kernel on top of the pixel that is going to be transformed. That's because a 3x3, 5x5, 7x7, 9x9 (...) kernel just computes the value for one pixel in the image, which is the one positioned at the center [0,0] of the kernel.
If you think about how to compute the value for the first pixel of the image, the center of a 9x9 kernel is going to be placed at coordinate [0,0]. That means that 3/4 of the kernel are going to be placed at negative coordinates, i.e. coordinates that refers to pixels that don't exist:
[-4,-4][-3,-4][-2,-4][-1,-4][ 0,-4][ 1,-4][ 2,-4][ 3,-4][ 4,-4]
[-4,-3][-3,-3][-2,-3][-1,-3][ 0,-3][ 1,-3][ 2,-3][ 3,-3][ 4,-3]
[-4,-2][-3,-2][-2,-2][-1,-2][ 0,-2][ 1,-2][ 2,-2][ 3,-2][ 4,-2]
[-4,-1][-3,-1][-2,-1][-1,-1][ 0,-1][ 1,-1][ 2,-1][ 3,-1][ 4,-1]
[-4, 0][-3, 0][-2, 0][-1, 0][ 0, 0][ 1, 0][ 2, 0][ 3, 0][ 4, 0]
[-4, 1][-3, 1][-2, 1][-1, 1][ 0, 1][ 1, 1][ 2, 1][ 3, 1][ 4, 1]
[-4, 2][-3, 2][-2, 2][-1, 2][ 0, 2][ 1, 2][ 2, 2][ 3, 2][ 4, 2]
[-4, 3][-3, 3][-2, 3][-1, 3][ 0, 3][ 1, 3][ 2, 3][ 3, 3][ 4, 3]
[-4, 4][-3, 4][-2, 4][-1, 4][ 0, 4][ 1, 4][ 2, 4][ 3, 4][ 4, 4]
This is always going to happen with pixels near the border of the image. So for the computation of the first pixel, we would have to restrict the computation to 1/4 of the kernel, which refers to valid coordinates in the target image:
[ ][ ][ ][ ][ ][ ][ ][ ][ ]
[ ][ ][ ][ ][ ][ ][ ][ ][ ]
[ ][ ][ ][ ][ ][ ][ ][ ][ ]
[ ][ ][ ][ ][ ][ ][ ][ ][ ]
[ ][ ][ ][ ][ 0, 0][ 1, 0][ 2, 0][ 3, 0][ 4, 0]
[ ][ ][ ][ ][ 0, 1][ 1, 1][ 2, 1][ 3, 1][ 4, 1]
[ ][ ][ ][ ][ 0, 2][ 1, 2][ 2, 2][ 3, 2][ 4, 2]
[ ][ ][ ][ ][ 0, 3][ 1, 3][ 2, 3][ 3, 3][ 4, 3]
[ ][ ][ ][ ][ 0, 4][ 1, 4][ 2, 4][ 3, 4][ 4, 4]
So the problem with your current approach is that at some point you will setup a ROI that is going to have negative coordinates, and when these instructions are executed you will see a nice crash:
cv::Mat ROI_Mat = input(ROI); // crash
The solution is not to use a ROI and just implement that algorithm yourself. I just can't see this custom computation working with cv::filter2D(). Here's a little something to help you get started:
void local_threshold(const cv::Mat& input, cv::Mat& output)
{
if (input.channels() != 1)
{
std::cout << "local_threshold !!! input image must be single channel" << std::endl;
return;
}
output = cv::Mat(input.rows, input.cols, CV_8UC1);
double min_val = 0, max_val = 0;
for (int i = 0; i < input.rows; i++)
for (int j = 0; j < input.cols; j++)
{
cv::Mat kernel = Mat::zeros(9, 9, output.type());
// Implement logic to fill the 9x9 kernel with
// values from the input Mat, respecting boundaries.
cv::Scalar avg_intensity = cv::mean(kernel);
cv::minMaxLoc(kernel, &min_val,&max_val);
if (input.at<uchar>(i,j) > (max_val / 2))
output.at<unsigned char>(i,j) = 255;
else
output.at<unsigned char>(i,j) = 0;
}
}

Answer 3

After further thinking and finding out how to utilize my basic knowledge in programming, I came up with this code that isn't the most efficient but gets the job done.
What was the main problem with my approach? :
Boundary pixels where one of the main problems and the whole indexing operation between kernel and mask caused a slight headache.
What was my approach in solving that matter?:
My threshold says it needs a relative high intensity level to set true pixels. Therefore I filled the image with some imaginary negative pixels and made my algorithm start at the first pixel of the original. And saved the results to a mask.
Result:
SUCCESS!
Code:
double maxVal, minVal;
cv::Mat output;
int top, bottom, left , right;
int borderType = cv::BORDER_CONSTANT;
cv::Scalar value;
top = (int) (4); bottom = (int) (4);
left = (int) (4); right = (int) (4);
output = src;
out = src;
value = 0;
cv::copyMakeBorder(src,output,top,bottom,left,right,borderType,value);
for(int y = 4; y < output.rows - 4; y++) {
for(int x = 4; x < output.cols - 4; x ++) {
// apply local ROI
cv::Mat ROI = output(cv::Rect(cv::Point(x-4,y-4),cv::Size(9,9)));
cv::minMaxLoc(ROI,&minVal,&maxVal); // extract max intensity values in the ROI
if(src.at<uchar>(cv::Point(x-4,y-4)) >= 0.5*maxVal){ // apply local threshold w.r.t highest intensity level
out.at<uchar>(cv::Point(x-4,y-4)) = 255; // change pixel value in mask if true
}else{
out.at<uchar>(cv::Point(x-4,y-4)) = 0;
}
}
}
}
it needs some clean up I know but hopefully this would help others to get some idea.