face and eye detection not working properly - c++

Question

using this code below I want to detect face and eyes from a video,
the code runs without error but the video and detection result is not displayed when I run it , what is the problem ?
I tried it using images it works fine on some images and other images just detect faces .
#include <opencv2\opencv.hpp>
using namespace cv;
using namespace std;
int main()
{
float EYE_SX = 0.16f;
float EYE_SY = 0.26f;
float EYE_SW = 0.30f;
float EYE_SH = 0.28f;
Mat dest, gray,frame;
VideoCapture capture("m.mp4");
CascadeClassifier detector, eyes_detector;
if (!capture.isOpened()) // check if we succeeded
return -1;
if(!detector.load("haarcascade_frontalface_alt2.xml"))
cout << "No se puede abrir clasificador." << endl;
if(!eyes_detector.load("haarcascade_eye_tree_eyeglasses.xml"))
cout << "No se puede abrir clasificador para los ojos." << endl;
for (;;)
{
capture >> frame;
cvtColor(frame, gray, CV_BGR2GRAY);
equalizeHist(gray, dest);
vector<Rect> rect;
detector.detectMultiScale(dest, rect);
for (Rect rc : rect)
{
rectangle(frame,
Point(rc.x, rc.y),
Point(rc.x + rc.width, rc.y + rc.height),
CV_RGB(0, 255, 0), 2);
}
if (rect.size() > 0)
{
Mat face = dest(rect[0]).clone();
vector<Rect> leftEye, rightEye;
int leftX = cvRound(face.cols * EYE_SX);
int topY = cvRound(face.rows * EYE_SY);
int widthX = cvRound(face.cols * EYE_SW);
int heightY = cvRound(face.rows * EYE_SH);
int rightX = cvRound(face.cols * (1.0 - EYE_SX - EYE_SW));
Mat topLeftOfFace = face(Rect(leftX, topY, widthX, heightY));
Mat topRightOfFace = face(Rect(rightX, topY, widthX, heightY));
eyes_detector.detectMultiScale(topLeftOfFace, leftEye);
eyes_detector.detectMultiScale(topRightOfFace, rightEye);
if ((int)leftEye.size() > 0)
{
rectangle(frame,
Point(leftEye[0].x + leftX + rect[0].x, leftEye[0].y + topY + rect[0].y),
Point(leftEye[0].width + widthX + rect[0].x - 5, leftEye[0].height + heightY + rect[0].y),
CV_RGB(0, 255, 255), 2);
}
if ((int)rightEye.size() > 0)
{
rectangle(frame,
Point(rightEye[0].x + rightX + leftX + rect[0].x, rightEye[0].y + topY + rect[0].y),
Point(rightEye[0].width + widthX + rect[0].x + 5, rightEye[0].height + heightY + rect[0].y),
CV_RGB(0, 255, 255), 2);
}
}
}
imshow("Ojos", frame);
waitKey(0);
return 1;
}

Answer 1

So, right now, the imshow("Ojos", frame); and the waitKey(0); are only getting called right before the program ends. That's fine for images, but not for video, since you want it to happen once per frame.
If you move it up a few lines, inside that for loop (basically, just put the bracket that's one line up from it one line below it), it should start working better for videos.
However, there are a couple of other things you might want to tweak in the code - it's only going to show one right eye and one left eye. This is normally what you want to happen, but if you have false positives you might end up with somebody's hair or skin being labeled an eye, and you're none-the-wiser as to how it happens. I'd reccommend displaying all of the items in the lefteye and righteye vectors. This can be done simply by replacing those if statements (if (int)rightEye.size() > 0, etc) with
for (int i = 0; i < rightEye.size(); i++) {
rectangle(frame,
Point(rightEye[i].x + rightX + leftX + rect[i].x,
rightEye[i].y + topY + rect[i].y),
Point(rightEye[i].width + widthX + rect[i].x + 5,
rightEye[i].height + heightY + rect[i].y),
CV_RGB(0, 255, 255), 2);
}
If you're having problems with false positives or negatives, you might want to look into tweaking the parameters on detectMultiscale - right now, you're leaving everything to the defaults. Multiscale has a number of parameters that can be put in. Image and objects you already have, but there are other, such as:
scaleFactor – Parameter specifying how much the image size is
reduced at each image scale. The default is 1.1. The bigger that is, the larger a scale will happen, which will take the cascade less time but there will be more false positives.
minNeighbors – Parameter specifying how
many neighbors each candidate rectangle should have to retain it. Default is 3. The bigger that is, the more neighbors it'll search for, resulting in less false positives, but it'll take longer. Tweak it too high, and it'll start giving false negatives.
flags – Parameter with the same meaning for an old cascade as in the
function cvHaarDetectObjects. It is not used for a new cascade. Default's zero, just leave it at zero for the most part.
minSize – Minimum possible object size. Objects smaller than that are
ignored. Default is Size(0,0). I tend to bump it up just a little. Again, bigger means it's faster and less false positives, but too big will skip over whatever you're looking for.
maxSize – Maximum possible object size. Objects larger than
that are ignored. Default is I believe the max size of the image passed in. I tend to limit it to smaller than that. Smaller on this means it's faster and less false positives, but too small will skip over whatever you're looking for.
cascade_name.detectMultiScale( frame_gray, frame_rectangle, 1.1, 2, 0, Size(30, 30) ); , as an example.