How to detect all candies with opencv and count them - c++

Question

I have the following picture
And I try to count all candies, but some of them intersect. And my following code doesn't work. The Code below can only identify candies if they don't intersect. But I can't have any thoughts about what to do if they intersect
int main( int argc, char** argv )
{
Mat src = imread("C:\\data\\Assignment1A.jpg",0); // reads image from file
Mat dst = Mat::zeros(src.rows, src.cols, CV_8UC3);
cv::Scalar min(245, 0, 0);
cv::Scalar max(255, 255, 255);
cv::inRange( src, min, max, dst);
namedWindow( "Source", 1 );
imshow( "Source", dst );
vector<vector<Point> > contours;
vector<Vec4i> hierarchy;
findContours( dst, contours, hierarchy,
CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE );
cout <<contours.size();
size_t count = contours.size();
int i = 0;
for(size_t i = 0; i < contours.size(); i++) {
cv::Point2f c;
float r;
cv::minEnclosingCircle( contours[i], c, r);
if (r > 10) {
i++; }
}
cout << i << " candies";
waitKey(0);
return 0;
}
Can anyone help me how to solve this problem?

Answer 1

This is probably not a very good solution (it only works under assumption that candies of the same color never overlap). (There's also this article about using distance transform and watershed, but I couldn't make it fork for this, maybe the candies overlap too much?)
Remove the white background (especially because it has a lot of noise and JPEG artifacts).
Convert to HSV and take the H channel (as suggested in the comments). Hue is useful here, because it sort of represents a color with a single value (hue) instead of three (red, green, blue), ignoring, to a degree, all those surface irregularities. You probably can't distinguish some of the ellipses below from the background, but don't worry, your computer can.
For a narrow range of H (say, [1, 20]), find contours big enough (in terms of their area) to be a single candy, but small enough not to be more than one candy. Repeat for H in range [21, 40], then [41, 60], and so on, until you reach the top hue value of 179. The contours are not too good, but good enough for counting. You can probably improve them by doing dilate/erode at every step to get rid of the junk, or something, if you need to.
The Python code to show how it works:
import cv2
import numpy as np
#get only contours of more or less the single candy size
#noise is not included because its area is too small
#overlapping candies are not included because their area is too big
#probably there are smarter ways (area/circumference ratio, ellipse fitting, etc.)
def get_contours(img):
area_threshold_max = 120*120 #nice hard-coded empirical values
area_threshold_min = 50*50
_, contours, _ = cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
areas = [cv2.contourArea(c) for c in contours]
contours = [c for n, c in enumerate(contours) if area_threshold_min < areas[n] < area_threshold_max]
return contours
#remove background
img_g = cv2.imread("candy.jpg", 0) #read as grayscale
_, thresh = cv2.threshold(img_g, 235, 1, cv2.THRESH_BINARY_INV) #mask to include everything that isn't too white
img = cv2.imread("candy.jpg") #read again as a color image
black = np.zeros(img.shape).astype(img.dtype) #black image to combine with
img = cv2.bitwise_or(img, black, mask = thresh)
#convert to HSV and take only the H value
img = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)[:, :, 0]
#find contours of a more or less appropriate size within a narrow H range,
#remove them from the image, increment H, repeat
contours_all = []
h_step = 20
for h_start in range(0, 180, h_step):
rng = np.zeros(img.shape).astype(np.uint8)
rng = cv2.inRange(img, h_start + 1, h_start + h_step)
contours = get_contours(rng)
contours_all += contours
#draw our contours with their numbers on top of the original image
img_to_draw = cv2.imread("candy.jpg")
cv2.drawContours(img_to_draw, contours_all, -1, (0, 0, 0), 2)
for i, cnt in enumerate(contours_all):
(x, y), radius = cv2.minEnclosingCircle(cnt)
cv2.putText(img_to_draw, "%d" % (i + 1), (int(x-20), int(y+10)), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 0), 3)
cv2.putText(img_to_draw, "%d candies" % len(contours_all), (5, 30), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 0), 3)
cv2.imwrite("result2.png", img_to_draw)