Slow speed in stream webcam to do "detectMultiScale" - c++

I'm new using OpenCV. I'm doing a sample face detector application console. I'm using haarcascade to detect the face from the webcam.
I did next code:
int main(int, char**)
{
CascadeClassifier face_cascade;
face_cascade.load("haarcascade_frontalface_alt2.xml");
vector<Rect> faces;
Mat frame_gray;
const double scale_factor = 1.1;
const int min_neighbours = 2;
const int flags = 0 | CV_HAAR_SCALE_IMAGE;
VideoCapture cap(0); // open the default camera
if (!cap.isOpened()) // check if we succeeded
return -1;
Mat frame;
for (;;)
{
cvtColor(frame, frame_gray, CV_BGR2GRAY);
equalizeHist(frame_gray, frame_gray);
face_cascade.detectMultiScale(frame_gray, faces, scale_factor, min_neighbours, flags, Size(30, 30));
if (faces.size() == 0)
{
cout << "No face detected" << endl;
}
else
{
for (unsigned int i = 0; i<faces.size(); i++)
{
Point pt1(faces[i].x, faces[i].y);
Point pt2(faces[i].x + faces[i].width, faces[i].y + faces[i].height);
rectangle(frame, pt1, pt2, Scalar(0, 255, 0), 1.5, 8, 0);
}
}
if (waitKey(30) >= 0) break;
}
return 0;
}
I tested the speed from the webcam is slow. I imagine that could be by the resolution from the image (640x480). I want to know if there are any way to keep the resolution and improving the speed between every frame to do the detection.
Thanks!

You can:
Increase minimal face size from Size(30, 30) to Size(50, 50) (it improves performance in 2-3 times).
Change value of scale_factor from 1.1 to 1.2; (it improves performance in 2 times).
Use LBP detector instead of Haar detector (it is faster in 2-3 times).
Check compiler options (may be you use Debug mode).

Related

Why is haar cascade very slow opencv c++

I am using haar cascading to detect frontal faces. I have below code:
int main()
{
Mat image;
cv::VideoCapture cap;
cap.open(1);
int frame_idx = 0;
time_t fpsStartTime, fpsEndTime;
time(&fpsStartTime);
for (;;)
{
frame_idx = frame_idx + 1;
cap.read(image);
CascadeClassifier face_cascade;
face_cascade.load("<PATH");
std::vector<Rect> faces;
face_cascade.detectMultiScale(image, faces, 1.1, 2, 0 | cv::CASCADE_SCALE_IMAGE, Size(30, 30));
// Draw circles on the detected faces
for (int i = 0; i < faces.size(); i++)
{
Point center(faces[i].x + faces[i].width*0.5, faces[i].y + faces[i].height*0.5);
ellipse(image, center, Size(faces[i].width*0.5, faces[i].height*0.5), 0, 0, 360, Scalar(255, 0, 255), 4, 8, 0);
}
cv::imshow("Detected Face", image);
char k = cv::waitKey(1);
if (k == 27)
break;
time(&fpsEndTime);
double seconds = difftime(fpsEndTime, fpsStartTime);
double fps = frame_idx / seconds;
std::string fps_txt = "FPS: " + std::to_string(fps); // fps_str.str();
cout << "FPS : " << fps_txt << endl;
}
return 0;
}
This code is working fine but giving very low FPS. FPS is ~1fps which is very slow. I am running this on Windows 10 laptop with intel i5 CPU. I believe this should not be this much slow.
In debug mode, it gives ~1fps but in release mode it is 4-5fps which again is very slow. I have run some openvino demo's like pedestrian detection which uses 2 openvino model on same hardware and it gives ~17-20fps which is very good.
I am using USB 3.0 logitech brio 4k camera so this cannot be a reason of low fps. My question is why haar cascading is performing very slow. Is there anyway we can enhance its speed and make it more usable. Please help. Thanks
You should not (re)load the classifier on every frame. It should load once before processing frames.
Move the following statements out of the for loop.
CascadeClassifier face_cascade;
face_cascade.load("<PATH");
See a demo on OpenCV Docs.
Can you confirm if you are using right .lib and .dll file?
I have checked and seen that the opencv_world440.lib & opencv_world440.dll provide great speed compared to opencv_world440d.lib & opencv_world440d.dll files.
My guess is that opencv_world440d.lib & opencv_world440d.dll are for debugging so slow speed.
Note::Your lib name may vary ie.., opencv_world<"SomeNumber">d.lib & opencv_world<"SomeNumber">.lib

C++ OpenCV - Find biggest object in an webcam stream and sort it by size

My goal is to find the biggest contour of a captured webcam frame, then after it's found, find its size and determine either to be rejected or accepted.
Just to explain the objetive of this project, i am currently working for a Hygiene product's Manufacturer. There we have, in total, 6 workers that are responsible for sorting the defective soap bars out of the production line. So in order to gain this workforce for other activities, i am trying to write an algorithm to "replace" their eyes.
I've tried several methods along the way (findcontours, SimpleBlobDetection, Canny, Object tracking), but the problem that i've been facing is that i can't seem to find a way to effectively find the biggest object in a webcam image, find its size and then determine to either discard or accept it.
Below follows my newest code to find the biggest contour in an webcam stream:
#include <iostream>
#include "opencv2/highgui/highgui.hpp"
#include "opencv/cv.h"
#include "opencv2\imgproc\imgproc.hpp"
using namespace cv;
using namespace std;
int main(int argc, const char** argv)
{
Mat src;
Mat imgGrayScale;
Mat imgCanny;
Mat imgBlurred;
/// Load source image
VideoCapture capWebcam(0);
if (capWebcam.isOpened() == false)
{
cout << "Não foi possível abrir webcam!" << endl;
return(0);
}
while (capWebcam.isOpened())
{
bool blnframe = capWebcam.read(src);
if (!blnframe || src.empty())
{
cout << "Erro! Frame não lido!\n";
break;
}
int largest_area = 0;
int largest_contour_index = 0;
Rect bounding_rect;
Mat thr(src.rows, src.cols, CV_8UC1);
Mat dst(src.rows, src.cols, CV_8UC1, Scalar::all(0));
cvtColor(src, imgGrayScale, CV_BGR2GRAY); //Convert to gray
GaussianBlur(imgGrayScale, imgBlurred, Size(5, 5), 1.8);
Canny(imgBlurred, imgCanny, 45, 90); //Threshold the gray
vector<vector<Point>> contours; // Vector for storing contour
vector<Vec4i> hierarchy;
findContours(imgCanny, contours, hierarchy, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE); // Find the contours in the image
for (int i = 0; i < contours.size(); i++) // iterate through each contour.
{
double a = contourArea(contours[i], false); // Find the area of contour
if (a > largest_area)
{
largest_area = a;
largest_contour_index = i; //Store the index of largest contour
bounding_rect = boundingRect(contours[i]); // Find the bounding rectangle for biggest contour
}
}
Scalar color(255, 255, 255);
drawContours(dst, contours, largest_contour_index, color, CV_FILLED, 8, hierarchy); // Draw the largest contour using previously stored index.
rectangle(src, bounding_rect, Scalar(0, 255, 0), 1, 8, 0);
imshow("src", src);
imshow("largest Contour", dst);
waitKey(27);
}
return(0);
}
And here are the results windows that the program generates and the image of the object that i want to detect and sort.
Thank you all in advance for any clues on how to achieve my goal.

findChessboardCorners gives unexpected results

Here is my code
#include <opencv/cv.h>
#include <opencv/highgui.h>
#include<opencv2/opencv.hpp>
#include<iostream>
//#include<vector>
using namespace cv;
using namespace std;
int main()
{
VideoCapture cap = VideoCapture(0);
int successes = 0;
int numBoards = 0;
int numCornersHor = 6;
int numCornersVer = 4;
int numSquares = (numCornersHor - 1) * (numCornersVer - 1);
Size board_sz = Size(numCornersHor, numCornersVer);
vector<Point2f> corners;
for (;;)
{
Mat img;
cap >> img;
Mat gray;
cvtColor(img, gray, CV_RGB2GRAY);
if (img.empty()) break; // end of video stream
imshow("this is you, smile! :)", gray);
if (waitKey(1) == 27) break; // stop capturing by pressing ESC
bool found = findChessboardCorners(gray, board_sz, corners, CALIB_CB_ADAPTIVE_THRESH);
if (found == 1)
{
cout << corners.size()<<"\n";
cornerSubPix(gray, corners, Size(11, 11), Size(-1, -1), TermCriteria(CV_TERMCRIT_EPS | CV_TERMCRIT_ITER, 30, 0.1));
drawChessboardCorners(gray, board_sz, corners, found);
}
}
cap.release();
waitKey();
return 0;
}
The code is capturing frames from a webcam. If a chessboard is detected, the total number of found corners is printed out (I did it because I was not getting the same output as in the tutorial code and I wanted to find where the bug is).
The output:
First you should follow some ground rules:
Do not use loose papers -> print/glue the chessboard on a flat plate
Print it with a big white border to improve detection
The chessboard has to be completly inside the image (not as in your example)
Take several images with different positions of your chessboard
Second, you cant draw your contours into a 8-bit grayscale image, use an 8-bit color image instead.
And if i count correctly (count inner corners) your chessboard has the size (8,6).
I have the same problem, the number of corners is HUGE. After some search i found this solution Here.
For some reason findChessboardCorners function resizes the corners vector. I tried the solution above, it worked well with the output corners, but i still have assertion failed problem with cornerSubPix function.

face detect OpenCV + Qt + cvMat

I'm trying to migrate my old opencv face detect code from the use of an IplImage structure to the use of the Mat class from opencv.
The issue is that the code is working when there is no Qt code.
Here is the code in brute C++ in codelite:
void detectFace(Mat& img)
{
std::vector<Rect> faces;
Mat gray;
cvtColor(img, gray, CV_BGR2GRAY);
equalizeHist(gray, gray);
// Face detect
face_cascade.detectMultiScale(gray, faces, 1.1, 2,1 | CV_HAAR_SCALE_IMAGE, Size(100,100));
for (unsigned int i = 0; i < faces.size(); i++)
{
//face detect rectangle
Point upperLeftFace(faces[i].x, faces[i].y);
Point lowerRightFace(faces[i].x+faces[i].width, faces[i].y+faces[i].height);
rectangle(/*Matrice*/img, /*Rect*/upperLeftFace, /*Rect*/lowerRightFace, /*BGR Color*/Scalar(255, 255,0), /*Line height*/1, /*line type*/8);
}
//Show window
namedWindow("Alexey Eye", CV_WINDOW_AUTOSIZE);
imshow("Alexey Eye", img);
}
And here is the code when i use qt:
void Neski::m_faceDetect()
{
*att_CamCapture >> att_MatCamera;
std::vector<Rect> faces;
Mat grayMat;
cvtColor(att_MatCamera, grayMat, CV_BGR2GRAY);
equalizeHist(grayMat, grayMat);
// Face detect
face_cascade.detectMultiScale(grayMat, faces, 1.1, 2,1 | CV_HAAR_SCALE_IMAGE, Size(150,150));
for (unsigned int i = 0; i < faces.size(); i++)
{
//face detect rectangle
Point upperLeftFace(faces[i].x, faces[i].y);
Point lowerRightFace(faces[i].x+faces[i].width, faces[i].y+faces[i].height);
rectangle(/*Matrice*/att_MatCamera, /*Rect*/upperLeftFace, /*Rect*/lowerRightFace, /*BGR Color*/Scalar(255, 255,0), /*Line height*/1, /*line type*/8);
}
cvtColor(att_MatCamera, att_MatCamera, CV_BGR2RGB);
QImage att_QImageCamera((uchar*) att_MatCamera.data, att_MatCamera.cols, att_MatCamera.rows, att_MatCamera.step, QImage::Format_RGB888);
*att_PixImageCamera = QPixmap::fromImage(att_QImageCamera.scaled(640, 480),Qt::AutoColor);
att_ui->lab_image->setPixmap(*att_PixImageCamera);
}
Both codes are almost the same, but i'm lost on why is there no facedetect when i launch the program. It does show me a video from the webcam but there is no facedetect rectangle.
Does anyone have any ideas?
i got it working after hours of sleep and fresh ideas.
It was a silly mistake, i forgot to load the cascade used to detect the face.
That's why there were no error and no rectangle.
face_cascade.load(face_cascade_name);
Just put it before the line:
face_cascade.detectMultiScale(grayMat, faces, 1.1, 2,1 | CV_HAAR.......
And it works now.
But, still thank you for the great site.
Here is a screenshoot of the final result

Why does detectMultiScale detect faces only when they are close to the centre of the frame?

I am trying a very simple program to detect faces in a webcam feed. I am noticing that the faces are detected well when my face is in the centre of the frame. Whenver I move a bit to the sides, the face detector either completely misses my face or gives no detection.
Is this bias because of the way I am using the function (code appended) or is it an inherent bias in the HAAR Classifiers?
Note that in either case (my face being in the approximate centre of the frame or my face being somewhere near the boundaries), my face is completely visible, i.e so side profiles/or cutting of the face.
//A live face detector Program. Takes feed from the camera and detects face in the given frame
#include<opencv2/opencv.hpp>
#include<iostream>
#include<vector>
#include"opencv2/video/video.hpp"
using namespace cv;
using namespace std;
int main(){
cv::Mat frame;
cv::VideoCapture cap(0);
cv::namedWindow("Frame");
do{
cap >> frame;
Rect r1,r2;
vector<Rect> faces1,faces2;
CascadeClassifier cascade1;
CascadeClassifier cascade2;
//cascade1.load("C:/opencv2.4.9/sources/data/lbpcascades/lbpcascade_frontalface.xml");
cascade1.load("C:/opencv2.4.9/sources/data/haarcascades/haarcascade_frontalface_alt2.xml");
cascade2.load("C:/opencv2.4.9/sources/data/lbpcascades/lbpcascade_profileface.xml");
cascade1.detectMultiScale(frame, faces1,1.05, 6, CV_HAAR_FIND_BIGGEST_OBJECT, Size(0, 0));
cascade2.detectMultiScale(frame, faces2,1.05, 6, CV_HAAR_FIND_BIGGEST_OBJECT, Size(0, 0));
if (faces1.size()!=0){
cout << "face1 found";
r1 = faces1[0];
}
if (faces2.size()!=0){
cout << "face2 found";
r2 = faces2[0];
}
rectangle(frame, Point(r1.y,r1.x), Point(r1.y+r1.height,r1.x+r1.width), Scalar(0,255,0),2, 8);
rectangle(frame, Point(r2.y,r2.x), Point(r2.y+r2.height,r2.x+r2.width), Scalar(255,0,0),2, 8);
imshow("Frame",frame);
}while(waitKey(30) < 0);
cap.release();
return 0;
}
your haar classifier code is working well.in your code change this
rectangle(frame, Point(r1.y,r1.x), Point(r1.y+r1.height,r1.x+r1.width), Scalar(0,255,0),2, 8);
rectangle(frame, Point(r2.y,r2.x), Point(r2.y+r2.height,r2.x+r2.width), Scalar(255,0,0),2, 8);
to
rectangle(frame, Point(r1.x, r1.y), Point(r1.x + r1.width, r1.y + r1.height), Scalar(0, 255, 0), 2, 8);
rectangle(frame, Point(r2.x, r2.y), Point(r2.x + r2.width, r2.y + r2.height), Scalar(255, 0, 0), 2, 8);
it will work. you have changed the x,y values.