Why Opencv DNN based (caffe) face detector failed to find faces? - c++

Question

By using OpenCV version 4.2.0 in c++ (VS 2019) I created project which performs face detection on the given image. I used Opencv's DNN face detector which uses res10_300x300_ssd_iter_140000_fp16.caffemodel model to detect faces. Below is the code of that function:
//variables which are used in function
const double inScaleFactor = 1.0;
const cv::Scalar meanVal = cv::Scalar(104.0, 177.0, 123.0);
const size_t inWidth = 300;
const size_t inHeight = 300;
std::vector<FaceDetectionResult> namespace_name::FaceDetection::detectFaceByOpenCVDNN(std::string filename, FaceDetectionModel model)
{
Net net;
cv::Mat frame = cv::imread(filename);
cv::Mat inputBlob;
std::vector<FaceDetectionResult> vec;
if (frame.empty())
throw std::exception("provided image file is not found or unable to open.");
int frameHeight = frame.rows;
int frameWidth = frame.cols;
if (model == FaceDetectionModel::CAFFE)
{
net = cv::dnn::readNetFromCaffe(caffeConfigFile, caffeWeightFile);
inputBlob = cv::dnn::blobFromImage(frame, inScaleFactor, cv::Size(inWidth, inHeight), meanVal, false, false);
}
else
{
net = cv::dnn::readNetFromTensorflow(tensorflowWeightFile, tensorflowConfigFile);
inputBlob = cv::dnn::blobFromImage(frame, inScaleFactor, cv::Size(inWidth, inHeight), meanVal, true, false);
}
net.setInput(inputBlob, "data");
cv::Mat detection = net.forward("detection_out");
cv::Mat detectionMat(detection.size[2], detection.size[3], CV_32F, detection.ptr<float>());
for (int i = 0; i < detectionMat.rows; i++)
{
if (detectionMat.at<float>(i, 2) >= 0.5)
{
FaceDetectionResult res;
res.faceDetected = true;
res.confidence = detectionMat.at<float>(i, 2);
res.x1 = static_cast<int>(detectionMat.at<float>(i, 3) * frameWidth);
res.y1 = static_cast<int>(detectionMat.at<float>(i, 4) * frameHeight);
res.x2 = static_cast<int>(detectionMat.at<float>(i, 5) * frameWidth);
res.y2 = static_cast<int>(detectionMat.at<float>(i, 6) * frameHeight);
vec.push_back(res);
}
#ifdef aDEBUG
else
{
cout << detectionMat.at<float>(i, 2) << endl;
}
#endif
}
return vec;
}
In the above code, after face detection I assign confidence and co-ordinates of face detected in custom class FaceDetectionResult, which a simple class having bool and int,float members as required.
Function detect faces in the given image, but while playing with this I am doing comparison with dlib's HOG+SVM face detector, So first I am doing face detection by dlib and then same image path is passed to this function.
I found some images where dlib can easily find faces in the image but opencv didn't find a single face, for example look at below image:
As you can see HOG+SVM detected 46 faces in approx 3 sec., If I pass this same image to above function then opencv did not detect a single face in it. Why? Do I need any enhancements in above code? I am not saying that function does not detect faces for any image, it does, but for some images (like above) it could not not.
For ref:
I used https://pastebin.com/9rt9reNY this python program to detect faces using dlib.

Answer 1

After a deep search, unfortunately I couldn't find a good explanation to this problem. The reason why I tried to crop image is that I assumed there can be a maximum detected face number limit. It is also not about occlusion.
I tried some image examples which includes more than 20(appx.) faces and the results were the same but when I cropped those images(decrease the number of faces), program was able to find the faces.This is also not about the resolution(sizes) of the image because the images I tried had different sizes.
I also changed and tried the all parameters(iteration number, confidentThreshold etc.) but the result still wasn't the desired one.
My assumption but not the answer:
The program doesn't let to find the faces if image includes more than a maximum number(approximately 20)
As a solution for this question, we can divide the source image into 2 parts and find the rectangles for each one then can be pasted to source image.
Note: After digging deeply on the internet, I couldnt find a topic related to this problem. I am also curious about the main reason causes this issue so any help will be appreciated. This post only includes my experiences and assumptions.

Answer 2

change this line :
inputBlob = cv::dnn::blobFromImage(frame, inScaleFactor, cv::Size(inWidth, inHeight), meanVal, false, false);
by this line :
frameHeightinputBlob = cv::dnn::blobFromImage(frame, inScaleFactor, cv::Size(inWidth, inHeight), meanVal, false, false);