Watershed boundaries closely surround one area - c++

I am trying to make an average of two blobs in OpenCV. To achieve that I was planning to use watershed algorithm on the image preprocessed in the following way:
cv::Mat common, diff, processed, result;
cv::bitwise_and(blob1, blob2, common); //calc common area of the two blobs
cv::absdiff(blob1, blob2, diff); //calc area where they differ
cv::distanceTransform(diff, processed, CV_DIST_L2, 3); //idea here is that the highest intensity
//will be in the middle of the differing area
cv::normalize(processed, processed, 0, 255, cv::NORM_MINMAX, CV_8U); //convert floats to bytes
cv::Mat watershedMarkers, watershedOutline;
common.convertTo(watershedMarkers, CV_32S, 1. / 255, 1); //change background to label 1, common area to label 2
watershedMarkers.setTo(0, processed); //set 0 (unknown) for area where blobs differ
cv::cvtColor(processed, processed, CV_GRAY2RGB); //watershed wants 3 channels
cv::watershed(processed, watershedMarkers);
cv::rectangle(watershedMarkers, cv::Rect(0, 0, watershedMarkers.cols, watershedMarkers.rows), 1); //remove the outline
//draw the boundary in red (for debugging)
watershedMarkers.convertTo(watershedOutline, CV_16S);
cv::threshold(watershedOutline, watershedOutline, 0, 255, CV_THRESH_BINARY_INV);
watershedOutline.convertTo(watershedOutline, CV_8U);
processed.setTo(cv::Scalar(CV_RGB(255, 0, 0)), watershedOutline);
//convert computed labels back to mask (blob), less relevant but shows my ultimate goal
watershedMarkers.convertTo(watershedMarkers, CV_8U);
cv::threshold(watershedMarkers, watershedMarkers, 1, 0, CV_THRESH_TOZERO_INV);
cv::bitwise_not(watershedMarkers * 255, result);
My problem with the results is that the calculated boundary is (almost) always adjacent to the area common to both blobs. Here are the pictures:
Input markers (black = 0, gray = 1, white = 2)
Watershed input image (distance transform result) with resulting outline drawn in red:
I would expect the boundary to go along the maximum intensity region of the input (that is, along the middle of the differing area). Instead (as you can see) it mostly goes around the area marked as 2, with a bit shifted to touch the background (marked as 1). Do I do something wrong here, or did I misunderstand how watershed works?

Starting from this image:
You can get the correct result simply passing an all-zero image to watershed algorithm. The "basin" is then equally filled of "water" starting from each "side" (then just remember to remove the outer border which is set by default to -1 by watershed algorithm):
Code:
#include <opencv2\opencv.hpp>
using namespace cv;
using namespace std;
int main()
{
Mat1b img = imread("path_to_image", IMREAD_GRAYSCALE);
Mat1i markers(img.rows, img.cols, int(0));
markers.setTo(1, img == 128);
markers.setTo(2, img == 255);
Mat3b image(markers.rows, markers.cols, Vec3b(0,0,0));
markers.convertTo(markers, CV_32S);
watershed(image, markers);
Mat3b result;
cvtColor(img, result, COLOR_GRAY2BGR);
result.setTo(Scalar(0, 0, 255), markers == -1);
imshow("Result", result);
waitKey();
return(0);
}

Related

Closing gaps in objects of opencv mask

I've computed the following mask of an image containing corn:
the problem is, now i have black parts inside the seeds. Is there a way to get rid of them while making sure that seeds remain disconnected from each other?
My ultimate goal is to count the seeds on the picture, using watershed algorithm. I observed when the seeds are touching each other, it introduces a imprecision to the algorithm, so I tried to introduce gaps between seeds by using canny and subtracting the borders from the mask, which is the result above.
My code so far:
auto img = GetAreaOfInterest(orig, bt); // <- just a Gauss blur with OTSU thresh
{
cv::Mat gray, edges;
cv::cvtColor(orig, gray, cv::COLOR_BGR2GRAY);
// tried bilateralFIlter here, but things only got worse
cv::Canny(gray, edges, 40, 160);
cv::dilate(edges, edges,
cv::getStructuringElement(cv::MORPH_ELLIPSE, {5, 5}),
{-1, -1}, 1);
img.setTo(0, edges != 0);
}
cv::Mat background, foreground, unknown, markers;
cv::dilate(img, background,
cv::getStructuringElement(cv::MORPH_ELLIPSE, {3,3}));
/* Get area for which we are sure it's the foreground */
cv::distanceTransform(img, img, cv::DIST_L1, 3, CV_8U);
{
double max;
cv::minMaxLoc(img, nullptr, &max);
cv::threshold(img, img, 0.46 * max, 255, cv::THRESH_BINARY);
}
img.convertTo(foreground, CV_8UC3);
show_resized("distance", foreground, 0.5);
/* Mark unknown areas */
cv::subtract(background, foreground, unknown);
/* Set up markers */
cv::connectedComponents(foreground, markers);
markers = markers + 1;
markers.setTo(0, unknown == 255);
/* Expand markers */
cv::watershed(orig, markers);
// Count objects
original image:
the most problematic image:

Braille Recognition from an image using OpenCV/C++

I am aiming to develop an iOS app that captures an image > extracts Braille dots represented in a specific color (Blue) > translates Braille letters into text using image processing techniques.
My approach was to use OpenCV/C++ to have the image be processed to extract the blue colored dots as shown in this photo into this photo
The next step is to recognize Braille in the image to translate it into text, one solution was to put a grid on the image to find the intersection points pixel color value then classify them into (1 if white , 0 if black) as represented in this photo
The problem with the proposed solution was:
how to position the grid rows/columns at the wanted positions?
how to get the coordinates and value(0 or 1)of the intersection points?
If you have any suggestion/solution about the proposed solution or any other solution please share them It would be appreciated since I don't have an experience in OpenCV/C++ field.
*note that python solutions cannot be used in iOS (as far as I know).
I attached my code for reference
+ (UIImage *)detectRedShapesInImage:(UIImage *)image{
cv::Mat mat;
UIImageToMat(image, mat);
cv::medianBlur(mat, mat, 3);
// Convert input image to HSV
cv::Mat hsv_image;
cv::cvtColor(mat, hsv_image, cv::COLOR_BGR2HSV);
// Threshold the HSV image, keep only the red (replaced it with blue) pixels
cv::Mat lower_red_hue_range;
cv::Mat upper_red_hue_range;
cv::inRange(hsv_image, cv::Scalar(0, 100, 100), cv::Scalar(10, 255, 255), lower_red_hue_range);
cv::inRange(hsv_image, cv::Scalar(160, 100, 100), cv::Scalar(179, 255, 255), upper_red_hue_range);
// Combine the above two images
cv::Mat red_hue_image;
cv::addWeighted(lower_red_hue_range, 1.0, upper_red_hue_range, 1.0, 0.0, red_hue_image);
cv::GaussianBlur(red_hue_image, red_hue_image, cv::Size(9, 9), 2, 2);
// detect circules, for now it doesnot take all circles
std::vector<cv::Vec4f> circles;
cv::HoughCircles(red_hue_image, circles, cv::HOUGH_GRADIENT, 1.0, 20, 150, 40, 0, 0);
// Loop over all detected circles and outline them on the original image
if(circles.size() == 0) std::exit(-1);
for(size_t current_circle = 0; current_circle < circles.size(); ++current_circle) {
cv::Point center(std::round(circles[current_circle][0]), std::round(circles[current_circle][1]));
int radius = std::round(circles[current_circle][2]);
cv::circle(red_hue_image, center, radius, cv::Scalar(0, 255, 0), 5);
}
UIImage *maskedShapesImg = MatToUIImage(red_hue_image);
return maskedShapesImg;}
Hints:
Assuming that the text is fairly horizontal and the spacing between lines is sufficient:
get the centers of the dots;
find the shortest horizontal and vertical distances between dots; these give you the horizontal and vertical pitch;
cluster the dots that are no more than one horizontal or two vertical pitches apart (with a safety margin); a cluster should correspond to a single character;
find the top-left corner of the characters;
find the median horizontal and vertical distance between the characters;
from this information (dot and character spacing), map the dots to the grid by predicting the grid nodes and using the nearest-neighbor rule.
This work is made a little tricky by the fact that not all character occupy the two columns and three rows of dots.

OpenCV: how can I interpret the results of inRange?

I am processing video images and I would like to detect if the video contains any pixels of a certain range of red. Is this possible?
Here is the code I am adapting from a tutorial:
#ifdef __cplusplus
- (void)processImage:(Mat&)image;
{
cv::Mat orig_image = image.clone();
cv::medianBlur(image, image, 3);
cv::Mat hsv_image;
cv::cvtColor(image, hsv_image, cv::COLOR_BGR2HSV);
cv::Mat lower_red_hue_range;
cv::Mat upper_red_hue_range;
cv::inRange(hsv_image, cv::Scalar(0, 100, 100), cv::Scalar(10, 255, 255), lower_red_hue_range);
cv::inRange(hsv_image, cv::Scalar(160, 100, 100), cv::Scalar(179, 255, 255), upper_red_hue_range);
// Interpret values here
}
Interpreting values
I would like to detect if the results from the inRange operations are nil or not. In other words I want to understand if there are any matching pixels in the original image with a colour inRange from the given lower and upper red scale. How can I interpret the results?
First you need to OR the lower and upper mask:
Mat mask = lower_red_hue_range | upper_red_hue_range;
Then you can countNonZero to see if there are non zero pixels (i.e. you found something).
int number_of_non_zero_pixels = countNonZero(mask);
It could be better to first apply morphological erosion or opening to remove small (probably noisy) blobs:
Mat kernel = getStructuringElement(MORPH_ELLIPSE, Size(3, 3));
morphologyEx(mask, mask, MORPH_OPEN, kernel); // or MORPH_ERODE
or find connected components (findContours, connectedComponentsWithStats) and prune / search for according to some criteria:
vector<vector<Point>> contours
findContours(mask.clone(), contours, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE);
double threshold_on_area = 100.0;
for(int i=0; i<contours.size(); ++i)
{
double area = countourArea(contours[i]);
if(area < threshold_on_area)
{
// don't consider this contour
continue;
}
else
{
// do something (e.g. drawing a bounding box around the contour)
Rect box = boundingRect(contours[i]);
rectangle(hsv_image, box, Scalar(0, 255, 255));
}
}

OpenCV better detection of red color?

I have the following image:
I would like to detect the red rectangle using cv::inRange method and HSV color space.
int H_MIN = 0;
int H_MAX = 10;
int S_MIN = 70;
int S_MAX = 255;
int V_MIN = 50;
int V_MAX = 255;
cv::cvtColor( input, imageHSV, cv::COLOR_BGR2HSV );
cv::inRange( imageHSV, cv::Scalar( H_MIN, S_MIN, V_MIN ), cv::Scalar( H_MAX, S_MAX, V_MAX ), imgThreshold0 );
I already created dynamic trackbars in order to change the values for HSV, but I can't get the desired result.
Any suggestion for best values (and maybe filters) to use?
In HSV space, the red color wraps around 180. So you need the H values to be both in [0,10] and [170, 180].
Try this:
#include <opencv2\opencv.hpp>
using namespace cv;
int main()
{
Mat3b bgr = imread("path_to_image");
Mat3b hsv;
cvtColor(bgr, hsv, COLOR_BGR2HSV);
Mat1b mask1, mask2;
inRange(hsv, Scalar(0, 70, 50), Scalar(10, 255, 255), mask1);
inRange(hsv, Scalar(170, 70, 50), Scalar(180, 255, 255), mask2);
Mat1b mask = mask1 | mask2;
imshow("Mask", mask);
waitKey();
return 0;
}
Your previous result:
Result adding range [170, 180]:
Another interesting approach which needs to check a single range only is:
invert the BGR image
convert to HSV
look for cyan color
This idea has been proposed by fmw42 and kindly pointed out by Mark Setchell. Thank you very much for that.
#include <opencv2\opencv.hpp>
using namespace cv;
int main()
{
Mat3b bgr = imread("path_to_image");
Mat3b bgr_inv = ~bgr;
Mat3b hsv_inv;
cvtColor(bgr_inv, hsv_inv, COLOR_BGR2HSV);
Mat1b mask;
inRange(hsv_inv, Scalar(90 - 10, 70, 50), Scalar(90 + 10, 255, 255), mask); // Cyan is 90
imshow("Mask", mask);
waitKey();
return 0;
}
While working with dominant colors such as red, blue, green and yellow; analyzing the two color channels of the LAB color space keeps things simple. All you need to do is apply a suitable threshold on either of the two color channels.
1. Detecting Red color
Background :
The LAB color space represents:
the brightness value in the image in the primary channel (L-channel)
while colors are expressed in the two remaining channels:
the color variations between red and green are expressed in the secondary channel (A-channel)
the color variations between yellow and blue are expressed in the third channel (B-channel)
Code :
import cv2
img = cv2.imread('red.png')
# convert to LAB color space
lab = cv2.cvtColor(img, cv2.COLOR_BGR2LAB)
# Perform Otsu threshold on the A-channel
th = cv2.threshold(lab[:,:,1], 127, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
Result:
I have placed the LAB converted image and the threshold image besides each other.
2. Detecting Blue color
Now lets see how to detect blue color
Sample image:
Since I am working with blue color:
Analyze the B-channel (since it expresses blue color better)
Perform inverse threshold to make the blue region appear white
(Note: the code changes below compared to the one above)
Code :
import cv2
img = cv2.imread('blue.jpg')
# convert to LAB color space
lab = cv2.cvtColor(img, cv2.COLOR_BGR2LAB)
# Perform Otsu threshold on the A-channel
th = cv2.threshold(lab[:,:,2], 127, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
Result:
Again, stacking the LAB and final image:
Conclusion :
Similar processing can be performed on green and yellow colors
Moreover segmenting a range of one of these dominant colors is also much simpler.

How to rescaling the intensity values in the original image to cover the entire dynamic range

I have a program for segmentation of lesion. In part of my program I want rescaling the intensity values in the original image to cover the entire dynamic range.
my code:
cvtColor(src, gray, CV_RGB2GRAY);
Mat kernel = Mat::ones(5, 5, CV_32F)/21;
kernel.row(0).col(0) = 0;
kernel.row(0).col(4) = 0;
kernel.row(4).col(0) = 0;
kernel.row(4).col(4) = 0;
//circular averaging low pass filter with a radius of 5, using the pill-box point spread function
filter2D(gray, gray, -1, kernel, Point(-1,-1));
threshold(gray, dst, 0, 255, THRESH_BINARY | THRESH_OTSU);
I am looking for solution for to do. It should Before threshold Done.