OpenCV3 Finger Detection - c++

Question

I am quite new to OpenCV and have just played around with it for a while doing the basic things like threshold images, etc. I'm using Visual Studio 2015 in C++ with OpenCV3. I'm trying to detect the number of fingers of my hand that are being held up using the camera. For example, if I hold up 4 fingers, I would like the program to tell me that 4 fingers are detected. So far I have been able to detect the edges of objects such as my entire hand in the camera using contours. Here is the code:
#include "opencv2\opencv.hpp"
using namespace cv;
void on_trackbar(int, void*) {
// Dummy function
}
int main(int argv, char** argc) {
Mat frame;
Mat grayFrame;
Mat hsvFrame;
Mat thesholdFrame;
VideoCapture capture;
//Trackbar variables (H,S,V)
int H_MIN = 0;
int H_MAX = 180;
int S_MIN = 0;
int S_MAX = 255;
int V_MIN = 0;
int V_MAX = 255;
namedWindow("trackbar", 0);
//create memory to store trackbar name on window
char TrackbarName[50];
sprintf(TrackbarName, "H_MIN");
sprintf(TrackbarName, "H_MAX");
sprintf(TrackbarName, "S_MIN");
sprintf(TrackbarName, "S_MAX");
sprintf(TrackbarName, "V_MIN");
sprintf(TrackbarName, "V_MAX");
createTrackbar("H_MIN", "trackbar", &H_MIN, H_MAX, on_trackbar);
createTrackbar("H_MAX", "trackbar", &H_MAX, H_MAX, on_trackbar);
createTrackbar("S_MIN", "trackbar", &S_MIN, S_MAX, on_trackbar);
createTrackbar("S_MAX", "trackbar", &S_MAX, S_MAX, on_trackbar);
createTrackbar("V_MIN", "trackbar", &V_MIN, V_MAX, on_trackbar);
createTrackbar("V_MAX", "trackbar", &V_MAX, V_MAX, on_trackbar);
capture.open(0);
std::vector<std::vector<cv::Point> > contours;
while (true){
capture >> frame;
waitKey(10);
cvtColor(frame, hsvFrame, COLOR_BGR2HSV);
//imshow("HSV", hsvFrame);
inRange(hsvFrame, Scalar(H_MIN, S_MIN, V_MIN), Scalar(H_MAX, S_MAX, V_MAX), thesholdFrame);
findContours(thesholdFrame, contours, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE);
int largest_contour_area = 0;
int largest_contour_area_index = 1;
/*for (int i = 0; i < contours.size(); i++) {
double contour_area = contourArea(contours[i], false);
if (contour_area > largest_contour_area) {
largest_contour_area = contour_area;
largest_contour_area_index = i;
}
}*/
drawContours(frame, contours, -1, (0, 255, 0), 3);
putText(frame, "NO DETECTION", Point(25, 40), 2, 1, CV_RGB(255, 255, 0), 1, 8, false);
imshow("Threshold", thesholdFrame);
imshow("Camera", frame);
}
}
After searching around for a while, I realised that it might be useful to isolate the contours of the hand only. The 'for' loop that is commented out is my attempt at implementing that. However, it doesn't seem to work. I do realise that once I uncomment that section I have to change
drawContours(frame, contours, -1, (0, 255, 0), 3);
to
drawContours(frame, contours[largest_contour_area_index], -1, (0, 255, 0), 3);
This does not seem to work. I keep getting this error(when I uncomment the for loop and replace the drawContours command with the one above):
Unhandled exception at 0x00007FF8C1537788 in FingerDetection.exe: Microsoft C++ exception: cv::Exception at memory location 0x0000000C4CAFDB50.
Furthermore, if I did somehow manage to get the program to find the edges of the largest contour in the frame(i.e the hand), how would I proceed to detect the number of fingers? I've heard things about convex hulls, etc. however I cannot find any good explanation for those exactly are. Any clarification or advice as to what to do? Keep in mind that I am very new to openCV.

Answer 1

Your second question ("how would I proceed to detect the number of fingers") might be a bit too broad to be answered here.
But the first one (i.e. why you are getting the exception) seems to have an easy answer, which I derive from the OpenCV documentation (version 2.4) for the drawContours function I found here.
According to that, the second parameter must be an array (of contours), so you cannot simply pass a single element contours[largest_contour_area_index].
If you want to draw one contour only, you still have to pass the entire array, and then the index of the contour you want to draw as third parameter (instead of -1):
drawContours(frame, contours, largest_contour_area_index, (0, 255, 0), 3);
As an additional note, it would be a good to idea to ensure that largest_contour_area_index has a value that is smaller than the number of contours in the array. Given your current code, there could be situations where the body of the for-loop is never executed, and largest_contour_area_index, which is initialized at a value of 1 could be too large.