How to identify unknown persons in facerecognition from videos? - c++

I'm using the facial recognition from videos by Philipp Wagner ,i updated the code to work with opencv 3.2 ,i had after that a real hard time to create the appropriate face database ,but then my question is how can i give a value for the unknown persons?So far when i run my code it gives the unknown person a value from my database i'am using "0" for myself and "1" for another person. how can i set it to be "-1" for example for unknown subjects ? Here is my code so far, I tried to use threshold but didn't get any results.
#include "opencv2/core.hpp"
#include "opencv2/face.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/objdetect.hpp"
#include <iostream>
#include <fstream>
#include <sstream>
using namespace cv;
using namespace cv::face;
using namespace std;
static void read_csv(const string& filename, vector<Mat>& images, vector<int>& labels, char separator = ';') {
std::ifstream file(filename.c_str(), ifstream::in);
if (!file) {
string error_message = "No valid input file was given, please check the given filename.";
CV_Error(CV_StsBadArg, error_message);
}
string line, path, classlabel;
while (getline(file, line)) {
stringstream liness(line);
getline(liness, path, separator);
getline(liness, classlabel);
if(!path.empty() && !classlabel.empty()) {
images.push_back(imread(path, 0));
labels.push_back(atoi(classlabel.c_str()));
}
}
}
int main(int argc, const char *argv[]) {
// Check for valid command line arguments, print usage
// if no arguments were given.
if (argc != 4) {
cout << "usage: " << argv[0] << " </path/to/haar_cascade> </path/to/csv.ext> </path/to/device id>" << endl;
cout << "\t </path/to/haar_cascade> -- Path to the Haar Cascade for face detection." << endl;
cout << "\t </path/to/csv.ext> -- Path to the CSV file with the face database." << endl;
cout << "\t <device id> -- The webcam device id to grab frames from." << endl;
exit(1);
}
// Get the path to your CSV:
string fn_haar = string(argv[1]);
string fn_csv = string(argv[2]);
int deviceId = atoi(argv[3]);
// These vectors hold the images and corresponding labels:
vector<Mat> images;
vector<int> labels;
// Read in the data (fails if no valid input filename is given, but you'll get an error message):
try {
read_csv(fn_csv, images, labels);
} catch (cv::Exception& e) {
cerr << "Error opening file \"" << fn_csv << "\". Reason: " << e.msg << endl;
// nothing more we can do
exit(1);
}
// Get the height from the first image. We'll need this
// later in code to reshape the images to their original
// size AND we need to reshape incoming faces to this size:
int im_width = images[0].cols;
int im_height = images[0].rows;
// Create a FaceRecognizer and train it on the given images:
Ptr<FaceRecognizer> model = createFisherFaceRecognizer();
model->train(images, labels);
// That's it for learning the Face Recognition model. You now
// need to create the classifier for the task of Face Detection.
// We are going to use the haar cascade you have specified in the
// command line arguments:
//
CascadeClassifier haar_cascade;
haar_cascade.load(fn_haar);
// Get a handle to the Video device:
VideoCapture cap(deviceId);
// Check if we can use this device at all:
if(!cap.isOpened()) {
cerr << "Capture Device ID " << deviceId << "cannot be opened." << endl;
return -1;
}
// Holds the current frame from the Video device:
Mat frame;
for(;;) {
cap >> frame;
// Clone the current frame:
Mat original = frame.clone();
// Convert the current frame to grayscale:
Mat gray;
cvtColor(original, gray, CV_BGR2GRAY);
// Find the faces in the frame:
vector< Rect_<int> > faces;
haar_cascade.detectMultiScale(gray, faces);
// At this point you have the position of the faces in
// faces. Now we'll get the faces, make a prediction and
// annotate it in the video. Cool or what?
for(int i = 0; i < faces.size(); i++) {
// Process face by face:
Rect face_i = faces[i];
// Crop the face from the image. So simple with OpenCV C++:
Mat face = gray(face_i);
// Resizing the face is necessary for Eigenfaces and Fisherfaces. You can easily
// verify this, by reading through the face recognition tutorial coming with OpenCV.
// Resizing IS NOT NEEDED for Local Binary Patterns Histograms, so preparing the
// input data really depends on the algorithm used.
//
// I strongly encourage you to play around with the algorithms. See which work best
// in your scenario, LBPH should always be a contender for robust face recognition.
//
// Since I am showing the Fisherfaces algorithm here, I also show how to resize the
// face you have just found:
Mat face_resized;
cv::resize(face, face_resized, Size(im_width, im_height), 1.0, 1.0, INTER_CUBIC);
// Now perform the prediction, see how easy that is:
int prediction = model->predict(face_resized);
// And finally write all we've found out to the original image!
// First of all draw a green rectangle around the detected face:
rectangle(original, face_i, CV_RGB(0, 255,0), 1);
// Create the text we will annotate the box with:
string box_text = format("Prediction = %d", prediction);
// Calculate the position for annotated text (make sure we don't
// put illegal values in there):
int pos_x = std::max(face_i.tl().x - 10, 0);
int pos_y = std::max(face_i.tl().y - 10, 0);
// And now put it into the image:
putText(original, box_text, Point(pos_x, pos_y), FONT_HERSHEY_PLAIN, 1.0, CV_RGB(0,255,0), 2.0);
}
// Show the result:
imshow("face_recognizer", original);
// And display it:
char key = (char) waitKey(20);
// Exit this loop on escape:
if(key == 27)
break;
}
return 0;
}

Read up on this document: Fisher Face Recognizer. Read over every method you are using. This should give you the information you need to troubleshoot.
From the document on model->set: If the distance to the nearest neighbor is larger than the threshold, this method returns -1. In your case, you're not getting any -1's returned, meaning that your threshold may set to high which will allow faces that are not similar to return a positive match.
It looks like you have not set your threshold variable. Try setting your threshold to a lower value using: model->set("threshold", DOUBLE_VALUE_HERE);.
A threshold of 0.0 would almost always return a -1 as images would always have slight differences making their distance > 0.0. Experiment with different threshold values and see if that gives you the result you're looking for. I'd recommend starting with a value of 5.0: model->set("threshold", 5.0); and working up or down from there.

Related

use of undeclared identifier 'createFisherFaceRecognizer' in opencv C++

Here is my code below:
/*
* Copyright (c) 2011. Philipp Wagner .
* Released to public domain under terms of the BSD Simplified license.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of the organization nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* See http://www.opensource.org/licenses/bsd-license
*/
#include "opencv2/core/core.hpp"
//#include "opencv2/contrib/contrib.hpp"
#include "opencv2/face.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/objdetect/objdetect.hpp"
#include <iostream>
#include <fstream>
#include <sstream>
using namespace cv;
using namespace std;
using namespace cv::face;
static void read_csv(const string& filename, vector<Mat>& images, vector<int>& labels, char separator = ';') {
std::ifstream file(filename.c_str(), ifstream::in);
if (!file) {
string error_message = "No valid input file was given, please check the given filename.";
CV_Error(CV_StsBadArg, error_message);
}
string line, path, classlabel;
while (getline(file, line)) {
stringstream liness(line);
getline(liness, path, separator);
getline(liness, classlabel);
if(!path.empty() && !classlabel.empty()) {
images.push_back(imread(path, 0));
labels.push_back(atoi(classlabel.c_str()));
}
}
}
int main(int argc, const char *argv[]) {
// Check for valid command line arguments, print usage
// if no arguments were given.
if (argc != 4) {
cout << "usage: " << argv[0] << " </path/to/haar_cascade> </path/to/csv.ext> </path/to/device id>" << endl;
cout << "\t </path/to/haar_cascade> -- Path to the Haar Cascade for face detection." << endl;
cout << "\t </path/to/csv.ext> -- Path to the CSV file with the face database." << endl;
cout << "\t <device id> -- The webcam device id to grab frames from." << endl;
exit(1);
}
// Get the path to your CSV:
string fn_haar = string(argv[1]);
string fn_csv = string(argv[2]);
int deviceId = atoi(argv[3]);
// These vectors hold the images and corresponding labels:
vector<Mat> images;
vector<int> labels;
// Read in the data (fails if no valid input filename is given, but you'll get an error message):
try {
read_csv(fn_csv, images, labels);
} catch (cv::Exception& e) {
cerr << "Error opening file \"" << fn_csv << "\". Reason: " << e.msg << endl;
// nothing more we can do
exit(1);
}
// Get the height from the first image. We'll need this
// later in code to reshape the images to their original
// size AND we need to reshape incoming faces to this size:
int im_width = images[0].cols;
int im_height = images[0].rows;
// Create a FaceRecognizer and train it on the given images:
Ptr<FaceRecognizer> model = createFisherFaceRecognizer();
model->train(images, labels);
// That's it for learning the Face Recognition model. You now
// need to create the classifier for the task of Face Detection.
// We are going to use the haar cascade you have specified in the
// command line arguments:
//
CascadeClassifier haar_cascade;
haar_cascade.load(fn_haar);
// Get a handle to the Video device:
VideoCapture cap(deviceId);
// Check if we can use this device at all:
if(!cap.isOpened()) {
cerr << "Capture Device ID " << deviceId << "cannot be opened." << endl;
return -1;
}
// Holds the current frame from the Video device:
Mat frame;
for(;;) {
cap >> frame;
// Clone the current frame:
Mat original = frame.clone();
// Convert the current frame to grayscale:
Mat gray;
cvtColor(original, gray, CV_BGR2GRAY);
// Find the faces in the frame:
vector< Rect_<int> > faces;
haar_cascade.detectMultiScale(gray, faces);
// At this point you have the position of the faces in
// faces. Now we'll get the faces, make a prediction and
// annotate it in the video. Cool or what?
for(int i = 0; i < faces.size(); i++) {
// Process face by face:
Rect face_i = faces[i];
// Crop the face from the image. So simple with OpenCV C++:
Mat face = gray(face_i);
// Resizing the face is necessary for Eigenfaces and Fisherfaces. You can easily
// verify this, by reading through the face recognition tutorial coming with OpenCV.
// Resizing IS NOT NEEDED for Local Binary Patterns Histograms, so preparing the
// input data really depends on the algorithm used.
//
// I strongly encourage you to play around with the algorithms. See which work best
// in your scenario, LBPH should always be a contender for robust face recognition.
//
// Since I am showing the Fisherfaces algorithm here, I also show how to resize the
// face you have just found:
Mat face_resized;
cv::resize(face, face_resized, Size(im_width, im_height), 1.0, 1.0, INTER_CUBIC);
// Now perform the prediction, see how easy that is:
int prediction = model->predict(face_resized);
// And finally write all we've found out to the original image!
// First of all draw a green rectangle around the detected face:
rectangle(original, face_i, CV_RGB(0, 255,0), 1);
// Create the text we will annotate the box with:
string box_text = format("Prediction = %d", prediction);
// Calculate the position for annotated text (make sure we don't
// put illegal values in there):
int pos_x = std::max(face_i.tl().x - 10, 0);
int pos_y = std::max(face_i.tl().y - 10, 0);
// And now put it into the image:
putText(original, box_text, Point(pos_x, pos_y), FONT_HERSHEY_PLAIN, 1.0, CV_RGB(0,255,0), 2.0);
}
// Show the result:
imshow("face_recognizer", original);
// And display it:
char key = (char) waitKey(20);
// Exit this loop on escape:
if(key == 27)
break;
}
return 0;
}
You have to create the model so
Ptr<FaceRecognizer> model = FisherFaceRecognizer::create();
It is documented in the modern manual and code samples.

Error while executing Eigenfaces algorithm in OpenCV

I have a problem with the code for Eigenfaces i've found on the OpenCV tutorial page.
The code is this, exactly the same you can find on the related page (http://docs.opencv.org/modules/contrib/doc/facerec/facerec_tutorial.html#eigenfaces-in-opencv):
/*
* Copyright (c) 2011. Philipp Wagner <bytefish[at]gmx[dot]de>.
* Released to public domain under terms of the BSD Simplified license.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of the organization nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* See <http://www.opensource.org/licenses/bsd-license>
*/
#include "opencv2/core/core.hpp"
#include "opencv2/contrib/contrib.hpp"
#include "opencv2/highgui/highgui.hpp"
#include <iostream>
#include <fstream>
#include <sstream>
using namespace cv;
using namespace std;
static Mat norm_0_255(InputArray _src) {
Mat src = _src.getMat();
// Create and return normalized image:
Mat dst;
switch(src.channels()) {
case 1:
cv::normalize(_src, dst, 0, 255, NORM_MINMAX, CV_8UC1);
break;
case 3:
cv::normalize(_src, dst, 0, 255, NORM_MINMAX, CV_8UC3);
break;
default:
src.copyTo(dst);
break;
}
return dst;
}
static void read_csv(const string& filename, vector<Mat>& images, vector<int>& labels, char separator = ';') {
std::ifstream file(filename.c_str(), ifstream::in);
if (!file) {
string error_message = "No valid input file was given, please check the given filename.";
CV_Error(CV_StsBadArg, error_message);
}
string line, path, classlabel;
while (getline(file, line)) {
stringstream liness(line);
getline(liness, path, separator);
getline(liness, classlabel);
if(!path.empty() && !classlabel.empty()) {
images.push_back(imread(path, 0));
labels.push_back(atoi(classlabel.c_str()));
}
}
}
int main(int argc, const char *argv[]) {
// Check for valid command line arguments, print usage
// if no arguments were given.
if (argc < 2) {
cout << "usage: " << argv[0] << " <csv.ext> <output_folder> " << endl;
exit(1);
}
string output_folder = ".";
if (argc == 3) {
output_folder = string(argv[2]);
}
// Get the path to your CSV.
string fn_csv = string(argv[1]);
// These vectors hold the images and corresponding labels.
vector<Mat> images;
vector<int> labels;
// Read in the data. This can fail if no valid
// input filename is given.
try {
read_csv(fn_csv, images, labels);
} catch (cv::Exception& e) {
cerr << "Error opening file \"" << fn_csv << "\". Reason: " << e.msg << endl;
// nothing more we can do
exit(1);
}
// Quit if there are not enough images for this demo.
if(images.size() <= 1) {
string error_message = "This demo needs at least 2 images to work. Please add more images to your data set!";
CV_Error(CV_StsError, error_message);
}
// Get the height from the first image. We'll need this
// later in code to reshape the images to their original
// size:
int height = images[0].rows;
// The following lines simply get the last images from
// your dataset and remove it from the vector. This is
// done, so that the training data (which we learn the
// cv::FaceRecognizer on) and the test data we test
// the model with, do not overlap.
Mat testSample = images[images.size() - 1];
int testLabel = labels[labels.size() - 1];
images.pop_back();
labels.pop_back();
// The following lines create an Eigenfaces model for
// face recognition and train it with the images and
// labels read from the given CSV file.
// This here is a full PCA, if you just want to keep
// 10 principal components (read Eigenfaces), then call
// the factory method like this:
//
// cv::createEigenFaceRecognizer(10);
//
// If you want to create a FaceRecognizer with a
// confidence threshold (e.g. 123.0), call it with:
//
// cv::createEigenFaceRecognizer(10, 123.0);
//
// If you want to use _all_ Eigenfaces and have a threshold,
// then call the method like this:
//
// cv::createEigenFaceRecognizer(0, 123.0);
//
Ptr<FaceRecognizer> model = createEigenFaceRecognizer();
model->train(images, labels); //<--ERROR!!
// The following line predicts the label of a given
// test image:
int predictedLabel = model->predict(testSample);
//
// To get the confidence of a prediction call the model with:
//
// int predictedLabel = -1;
// double confidence = 0.0;
// model->predict(testSample, predictedLabel, confidence);
//
string result_message = format("Predicted class = %d / Actual class = %d.", predictedLabel, testLabel);
cout << result_message << endl;
// Here is how to get the eigenvalues of this Eigenfaces model:
Mat eigenvalues = model->getMat("eigenvalues");
// And we can do the same to display the Eigenvectors (read Eigenfaces):
Mat W = model->getMat("eigenvectors");
// Get the sample mean from the training data
Mat mean = model->getMat("mean");
// Display or save:
if(argc == 2) {
imshow("mean", norm_0_255(mean.reshape(1, images[0].rows)));
} else {
imwrite(format("%s/mean.png", output_folder.c_str()), norm_0_255(mean.reshape(1, images[0].rows)));
}
// Display or save the Eigenfaces:
for (int i = 0; i < min(10, W.cols); i++) {
string msg = format("Eigenvalue #%d = %.5f", i, eigenvalues.at<double>(i));
cout << msg << endl;
// get eigenvector #i
Mat ev = W.col(i).clone();
// Reshape to original size & normalize to [0...255] for imshow.
Mat grayscale = norm_0_255(ev.reshape(1, height));
// Show the image & apply a Jet colormap for better sensing.
Mat cgrayscale;
applyColorMap(grayscale, cgrayscale, COLORMAP_JET);
// Display or save:
if(argc == 2) {
imshow(format("eigenface_%d", i), cgrayscale);
} else {
imwrite(format("%s/eigenface_%d.png", output_folder.c_str(), i), norm_0_255(cgrayscale));
}
}
// Display or save the image reconstruction at some predefined steps:
for(int num_components = min(W.cols, 10); num_components < min(W.cols, 300); num_components+=15) {
// slice the eigenvectors from the model
Mat evs = Mat(W, Range::all(), Range(0, num_components));
Mat projection = subspaceProject(evs, mean, images[0].reshape(1,1));
Mat reconstruction = subspaceReconstruct(evs, mean, projection);
// Normalize the result:
reconstruction = norm_0_255(reconstruction.reshape(1, images[0].rows));
// Display or save:
if(argc == 2) {
imshow(format("eigenface_reconstruction_%d", num_components), reconstruction);
} else {
imwrite(format("%s/eigenface_reconstruction_%d.png", output_folder.c_str(), num_components), reconstruction);
}
}
// Display if we are not writing to an output folder:
if(argc == 2) {
waitKey(0);
}
return 0;
}
When i try to execute this code i get this error:
OpenCV Error: Image step is wrong (The matrix is not continuous, thus its number of rows can not be changed) in reshape, file /tmp/opencv-ppDuGa/opencv-2.4.9/modules/core/src
/matrix.cpp, line 802 libc++abi.dylib: terminating with uncaught exception of type cv::Exception: /tmp/opencv-ppDuGa/opencv-2.4.9/modules/core/src/matrix.cpp:802: error: (-13) The matrix is not continuous, thus its number of rows can not be changed in function reshape
Abort trap: 6
The error appears during the train phase (as i've reported on the code).
I'm using XCode on Mac with the last OS Yosemite, the dataset is AT&T Facedatabase that can be downloaded from here: http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html
Thanks in advance
for eigen and fisherfaces, the images have to get 'flattened' to 1 single row, this is only possible, if your Mat is continuous. (lbph is not constraind this way)
but i'd rather suspect, that the resp. Mat was empty, because it was not an image at all.
please check your data again very carefully. does it contain non-images (like a .txt file) ?
you probably generated a csv file via the supplied python script. again, check the outcome.
when reading the csv, try to replace
images.push_back(imread(path, 0));
with:
Mat im = imread(path, 0);
if ( im.empty() )
{
cerr << path << " was empty !" << endl;
exit(-1);
}
if ( !im.isContinuous() )
{
// .bmp files sometimes are 'padded' to multiples of 4
// (some image editors like to do that)
// if you end up here, try to:
// * convert your images to png, pbm or such
// * use im.clone(); instead (the deep copy will force continuity)
//
cerr << path << " was not continuous !" << endl;
exit(-2);
}
images.push_back(im);

OpenCV error: Image step is wrong (The matrix is not continuous)

When starting my programm through command line, have such problem:
OpenCV Error: Image step is wrong (The matrix is not continuous, thus its number of rows can not be changed) un cv::Mat::reshape, file C:\builds\2_4_PackSlave-win64-vc12-shared\opencv\modules\core\src\matrix.cpp, line 802.
Code of the program:
#include "opencv2/core/core.hpp"
#include "opencv2/contrib/contrib.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/objdetect/objdetect.hpp"
#include <iostream>
#include <fstream>
#include <sstream>
using namespace cv;
using namespace std;
static void read_csv(const string& filename, vector<Mat>& images, vector<int>& labels, char separator = ';') {
std::ifstream file(filename.c_str(), ifstream::in);
if (!file) {
string error_message = "No valid input file was given, please check the given filename.";
CV_Error(CV_StsBadArg, error_message);
}
string line, path, classlabel;
while (getline(file, line)) {
stringstream liness(line);
getline(liness, path, separator);
getline(liness, classlabel);
if (!path.empty() && !classlabel.empty()) {
images.push_back(imread(path, 0));
labels.push_back(atoi(classlabel.c_str()));
}
}
}
int main(int argc, const char *argv[]) {
// Check for valid command line arguments, print usage
// if no arguments were given.
if (argc != 4) {
cout << "usage: " << argv[0] << " </path/to/haar_cascade> </path/to/csv.ext> </path/to/device id>" << endl;
cout << "\t </path/to/haar_cascade> -- Path to the Haar Cascade for face detection." << endl;
cout << "\t </path/to/csv.ext> -- Path to the CSV file with the face database." << endl;
cout << "\t <device id> -- The webcam device id to grab frames from." << endl;
exit(1);
}
// Get the path to your CSV:
string fn_haar = string(argv[1]);
string fn_csv = string(argv[2]);
int deviceId = atoi(argv[3]);
// These vectors hold the images and corresponding labels:
vector<Mat> images;
vector<int> labels;
// Read in the data (fails if no valid input filename is given, but you'll get an error message):
try {
read_csv(fn_csv, images, labels);
}
catch (cv::Exception& e) {
cerr << "Error opening file \"" << fn_csv << "\". Reason: " << e.msg << endl;
// nothing more we can do
exit(1);
}
// Get the height from the first image. We'll need this
// later in code to reshape the images to their original
// size AND we need to reshape incoming faces to this size:
int im_width = images[0].cols;
int im_height = images[0].rows;
// Create a FaceRecognizer and train it on the given images:
Ptr<FaceRecognizer> model = createFisherFaceRecognizer();
model->train(images, labels);
// That's it for learning the Face Recognition model. You now
// need to create the classifier for the task of Face Detection.
// We are going to use the haar cascade you have specified in the
// command line arguments:
//
CascadeClassifier haar_cascade;
haar_cascade.load(fn_haar);
// Get a handle to the Video device:
VideoCapture cap(deviceId);
// Check if we can use this device at all:
if (!cap.isOpened()) {
cerr << "Capture Device ID " << deviceId << "cannot be opened." << endl;
return -1;
}
// Holds the current frame from the Video device:
Mat frame;
for (;;) {
cap >> frame;
// Clone the current frame:
Mat original = frame.clone();
// Convert the current frame to grayscale:
Mat gray;
cvtColor(original, gray, CV_BGR2GRAY);
// Find the faces in the frame:
vector< Rect_<int> > faces;
haar_cascade.detectMultiScale(gray, faces);
// At this point you have the position of the faces in
// faces. Now we'll get the faces, make a prediction and
// annotate it in the video. Cool or what?
for (int i = 0; i < faces.size(); i++) {
// Process face by face:
Rect face_i = faces[i];
// Crop the face from the image. So simple with OpenCV C++:
Mat face = gray(face_i);
// Resizing the face is necessary for Eigenfaces and Fisherfaces. You can easily
// verify this, by reading through the face recognition tutorial coming with OpenCV.
// Resizing IS NOT NEEDED for Local Binary Patterns Histograms, so preparing the
// input data really depends on the algorithm used.
//
// I strongly encourage you to play around with the algorithms. See which work best
// in your scenario, LBPH should always be a contender for robust face recognition.
//
// Since I am showing the Fisherfaces algorithm here, I also show how to resize the
// face you have just found:
Mat face_resized;
cv::resize(face, face_resized, Size(im_width, im_height), 1.0, 1.0, INTER_CUBIC);
// Now perform the prediction, see how easy that is:
int prediction = model->predict(face_resized);
// And finally write all we've found out to the original image!
// First of all draw a green rectangle around the detected face:
rectangle(original, face_i, CV_RGB(0, 255, 0), 1);
// Create the text we will annotate the box with:
string box_text = format("Prediction = %d", prediction);
// Calculate the position for annotated text (make sure we don't
// put illegal values in there):
int pos_x = std::max(face_i.tl().x - 10, 0);
int pos_y = std::max(face_i.tl().y - 10, 0);
// And now put it into the image:
putText(original, box_text, Point(pos_x, pos_y), FONT_HERSHEY_PLAIN, 1.0, CV_RGB(0, 255, 0), 2.0);
}
// Show the result:
imshow("face_recognizer", original);
// And display it:
char key = (char)waitKey(20);
// Exit this loop on escape:
if (key == 27)
break;
}
return 0;
}
What I must to do?
the FisherFaceRecognizer (Eigen, too) tries to 'flatten' the images to a single row (reshape()) for training and testing.
this does not work, if the Mat is non-continuous (because it's either padded or a submat/roi only).
( then again, 'fileNotFound' counts as 'non-continuous', too ;] )
if your images are e.g. .bmp , there's a high chance, that some image-editor padded your images, so the row-size is a factor of 4.
maybe you can get away with batch converting your imgs to .png or .pgm externally
else resizing your train images after loading them will help (anything, that makes a copy of it)
or, change this line in the loading code:
images.push_back(imread(path, 0));
to:
Mat m = imread(path, 1);
Mat m2;
cvtColor(m,m2,CV_BGR_GRAY);
images.push_back(m2);
It's a slash/anti-slash problem in your csv file.
For example mine was like this:
sujets\s1/1.pgm;0
sujets\s1/10.pgm;0
...
sujets\s1/9.pgm;0
sujets\s2/1.pgm;1
sujets\s2/10.pgm;1
...
sujets\s2/9.pgm;1
sujets\s3/1.pgm;2
sujets\s3/10.pgm;2
...
sujets\s3/9.pgm;2
sujets\s4/1.pgm;3
sujets\s4/10.pgm;3
...
sujets\s4/9.pgm;3
Changing for this:
sujets/s1/1.pgm;0
sujets/s1/10.pgm;0
...
sujets/s1/9.pgm;0
sujets/s2/1.pgm;1
sujets/s2/10.pgm;1
...
sujets/s2/9.pgm;1
sujets/s3/1.pgm;2
sujets/s3/10.pgm;2
...
sujets/s3/9.pgm;2
sujets/s4/1.pgm;3
sujets/s4/10.pgm;3
...
sujets/s4/9.pgm;3
did the tricks
I think that the problem could be in Mat face = gray(face_i).
Read the comments for clarification.
Rect face_i = faces[i];
// This operation makes a new header for the specified sub-array of
// *this, thus it is a 0(1) operation, that is, no matrix data is
// copied. So matrix elements are no longer stored continuously without
// gaps at the end of each row.
Mat face = gray(face_i);
...
Mat face_resized;
// Here new memory for face_resized should be allocated, but I'm not sure.
// If not then it is the reason of the error, because in this case
// face_resized will contain not continuous data (=face).
cv::resize(face, face_resized, Size(im_width, im_height), 1.0, 1.0, INTER_CUBIC);
...
// here reshape(face_resized) will be called and will throw the error if
// matrix is not continuous
int prediction = model->predict(face_resized);

The opencv code for face recognition does not predict correctly in visual c++

This is my code for face recognition in videos. It runs without any error but it's prediction
is wrong most of the time.I am using LBPH face recognizer to recognize the faces.
I tried using haar cascades but it does not load. so i switched to LBHP.please help me to improve the prediction.
I am using gray scale cropped images of size 500 x 500 (pixels) for training the cascade classifier.
#include <opencv2/core/core.hpp>
#include <opencv2/contrib/contrib.hpp
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/objdetect/objdetect.hpp>
#include <iostream>
#include <fstream>
#include <sstream>
using namespace cv;
using namespace std;
static void read_csv(const string& filename, vector<Mat>& images, vector<int>& labels, char separator = ';') {
std::ifstream file(filename.c_str(), ifstream::in);
if (!file) {
string error_message = "No valid input file was given, please check the given filename.";
CV_Error(CV_StsBadArg, error_message);
}
string line, path, classlabel;
while (getline(file, line)) {
stringstream liness(line);
getline(liness, path, separator);
getline(liness, classlabel);
if(!path.empty() && !classlabel.empty()) {
images.push_back(imread(path, 0));
labels.push_back(atoi(classlabel.c_str()));
}
}
}
string g_listname_t[]=
{
"ajay","Aasai","famiz"
};
int main(int argc, const char *argv[]) {
// Check for valid command line arguments, print usage
// if no arguments were given.
//if (argc != 4) {
// cout << "usage: " << argv[0] << " </path/to/haar_cascade> </path/to/csv.ext> </path/to/device id>"<<endl;
// cout << "\t </path/to/haar_cascade> -- Path to the Haar Cascade for face detection." << endl;
// cout << "\t </path/to/csv.ext> -- Path to the CSV file with the face database." << endl;
// cout << "\t <device id> -- The webcam device id to grab frames from." << endl;
// exit(1);
//}
//// Get the path to your CSV:
//string fn_haar = string(argv[1]);
//string fn_csv = string(argv[2]);
//int deviceId = atoi(argv[3]);
//// Get the path to your CSV:
// please set the correct path based on your folder
string fn_haar = "lbpcascade_frontalface.xml";
string fn_csv = "reader.ext ";
int deviceId = 0; // here is my webcam Id.
// These vectors hold the images and corresponding labels:
vector<Mat> images;
vector<int> labels;
// Read in the data (fails if no valid input filename is given, but you'll get an error message):
try {
read_csv(fn_csv, images, labels);
} catch (cv::Exception& e) {
cerr << "Error opening file \"" << fn_csv << "\". Reason: " << e.msg << endl;
// nothing more we can do
exit(1);
}
// Get the height from the first image. We'll need this
// later in code to reshape the images to their original
// size AND we need to reshape incoming faces to this size:
int im_width = images[0].cols;
int im_height = images[0].rows;
// Create a FaceRecognizer and train it on the given images:
Ptr<FaceRecognizer> model = createLBPHFaceRecognizer();
model->train(images, labels);
cout<<("Facerecognizer created");
// That's it for learning the Face Recognition model. You now
// need to create the classifier for the task of Face Detection.
// We are going to use the haar cascade you have specified in the
// command line arguments:
CascadeClassifier lbp_cascade;
if ( ! lbp_cascade.load(fn_haar) )
{
cout<<("\nlbp cascade not loaded");
}
else
{
cout<<("\nlbp cascade loaded");
}
// Get a handle to the Video device:
VideoCapture cap(deviceId);
cout<<("\nvideo device is opened");
// Check if we can use this device at all:
if(!cap.isOpened()) {
cerr << "Capture Device ID " << deviceId << "cannot be opened." << endl;
return -1;
}
// Holds the current frame from the Video device:
Mat frame;
for(;;) {
cap >> frame;
// Clone the current frame:
Mat original = frame.clone();
cout<<("\nframe is cloned");
// Convert the current frame to grayscale:
Mat gray;
//gray = imread("G:\Picture\003.jpg",0);
cvtColor(original, gray, CV_BGR2GRAY);
imshow("gray image", gray);
// And display it:
char key1 = (char) waitKey(50);
// Find the faces in the frame:
cout<<("\ncolor converted");
vector< Rect_<int> > faces;
cout<<("\ndetecting faces");
lbp_cascade.detectMultiScale(gray, faces);
// At this point you have the position of the faces in
// faces. Now we'll get the faces, make a prediction and
// annotate it in the video. Cool or what?
cout<<("\nfaces detected\n");
cout<<faces.size();
for(int i = 0; i < faces.size(); i++)
{
// Process face by face:
cout<<("\nprocessing faces");
Rect face_i = faces[i];
// Crop the face from the image. So simple with OpenCV C++:
Mat face = gray(face_i);
// Resizing the face is necessary for Eigenfaces and Fisherfaces. You can easily
// verify this, by reading through the face recognition tutorial coming with OpenCV.
// Resizing IS NOT NEEDED for Local Binary Patterns Histograms, so preparing the
// input data really depends on the algorithm used.
//
// I strongly encourage you to play around with the algorithms. See which work best
// in your scenario, LBPH should always be a contender for robust face recognition.
//
// Since I am showing the Fisherfaces algorithm here, I also show how to resize the
// face you have just found:
/*Mat face_resized;
cv::resize(face, face_resized, Size(im_width, im_height), 1.0, 1.0, INTER_CUBIC);
// Now perform the prediction, see how easy that is:
cout<<("\nface resized");
imshow("resized face image", face_resized);*/
int prediction = model->predict(face);
cout<<("\nface predicted");
// And finally write all we've found out to the original image!
// First of all draw a green rectangle around the detected face:
cout<<("\nnow writing to original");
rectangle(original, face_i, CV_RGB(0, 255,0), 1);
// Create the text we will annotate the box with:
string box_text;
box_text = format( "Prediction =",prediction);
// Get stringname
if ( prediction >= 0 && prediction <=1 )
{
box_text.append( g_listname_t[prediction] );
}
else box_text.append( "Unknown" );
// Calculate the position for annotated text (make sure we don't
// put illegal values in there):
int pos_x = std::max(face_i.tl().x - 10, 0);
int pos_y = std::max(face_i.tl().y - 10, 0);
// And now put it into the image:
putText(original, box_text, Point(pos_x, pos_y), FONT_HERSHEY_PLAIN, 1.0, CV_RGB(0,255,0), 2.0);
}
// Show the result:
imshow("face_recognizer", original);
// And display it:
char key = (char) waitKey(50);
// Exit this loop on escape:
if(key == 27)
break;
}
return 0;
}
That is an expected result if you ask me, the code which you showed is the basic one to do recognition, there are some backdrops which we need to take care of before implementing.
1) the quality of training images, how did you crop them ?
do they contain any extra information apart from face, if you used haar classifier in our opencv data to crop faces, then, the images tend to contain extra information than the face, as the rectangles are a bit large in size when compared to face.
2) there might be a chance that, even the rotated faces might be trained, so, its tough to classify with the features of rotated faces.
3) how many images, you trained the recognizer with ?, it playes a crucial role.
Answer for the first question, is most likely to be out of opencv, we cant do much about it, as there is very less probability that, we ll find a face detector which is as good and as simple as haar detector, so, we could make this as an exemption, if we can adjust with an accuracy around 70 %.
the second problem could be solved with some preprocessing techniques on training and testing dataset.
Like., aligning faces which are being rotated
follow this link, very good suggestions for face alignment are being suggested.
How to align face images c++ opencv
the third problem is solved with good number of samples which is not a hard task to achieve, take care of alignment before training, so that correct features could be extracted to classify.
there might be other factors that can improve the accuracy which I might have missed.

Open Cv fisherfaces

I have this problem,
when i run with vs2010 (debug) (open cv 2.4.0) facerec_demo.cpp gaves me the program this error
OpenCV Error: Image step is wrong (The matrix is not continuous, thus its number of rows can not be changed) in unknown function, file ......\src\opencv\modul es\core>\src\matrix.cpp, line 801
This error derives me to this line in facerec.cpp
(Fisherfaces::train(InputArray src, InputArray _lbls)
Mat data = asRowMatrix(src, CV_64FC1); <-- this gets a exeption, not handled.
and a i use at pgm img database and this is my original *facerec_demo.cpp* file
#include "stdafx.h"
#include <opencv2/opencv.hpp>
#include <iostream>
#include <fstream>
#include <vector>
#include <string>
#include <sstream>
using namespace cv;
using namespace std;
vector<string> split_at_commas(const string& row)
{
vector<string> res;
istringstream buf(row);
string s;
while (getline(buf, s, ';'))
res.push_back(s);
return res;
}
Mat toGrayscale(InputArray _src) {
Mat src = _src.getMat();
// only allow one channel
if(src.channels() != 1)
CV_Error(CV_StsBadArg, "Only Matrices with one channel are supported");
// create and return normalized image
Mat dst;
cv::normalize(_src, dst, 0, 255, NORM_MINMAX, CV_8UC1);
return dst;
}
void read_csv(const string& filename, vector<Mat>& images, vector<int>& labels, char separator = ';') {
//std::ifstream file(filename.c_str(), ifstream::in);
std::ifstream file(_T("D:\\Users\\PC ACER\\Documents\\mycsv4.csv"));
if (!file)
throw std::exception();
string line="", path="", classlabel="";
while (getline(file, line)) {
//vector<string> values = split_at_commas(line);
stringstream liness(line);
getline(liness, path, ';');
getline(liness, classlabel);
images.push_back(imread(path, 0));
labels.push_back(atoi(classlabel.c_str()));
}
}
int main(int argc, const char *argv[]) {
// check for command line arguments
if (argc != 2) {
cout << "usage: " << argv[0] << " <csv.ext>" << endl;
exit(1);
}
// path to your CSV
string fn_csv = string(argv[1]);
// images and corresponding labels
vector<Mat> images;
vector<int> labels;
// read in the data
try {
read_csv(fn_csv, images, labels);
} catch (exception&) {
cerr << "Error opening file \"" << fn_csv << "\"." << endl;
exit(1);
}
// get width and height
//int width = images[0].cols;
int height = images[0].rows;
// get test instances
Mat testSample = images[images.size() - 1];
int testLabel = labels[labels.size() - 1];
// ... and delete last element
images.pop_back();
labels.pop_back();
// build the Fisherfaces model
Ptr<FaceRecognizer> model = createFisherFaceRecognizer();
model->train(images, labels);
// test model
int predicted = model->predict(testSample);
cout << "predicted class = " << predicted << endl;
cout << "actual class = " << testLabel << endl;
// get the eigenvectors
Mat W = model->eigenvectors();
// show first 10 fisherfaces
for (int i = 0; i < min(10, W.cols); i++) {
// get eigenvector #i
Mat ev = W.col(i).clone();
// reshape to original size AND normalize between [0...255]
Mat grayscale = toGrayscale(ev.reshape(1, height));
// show image (with Jet colormap)
Mat cgrayscale;
applyColorMap(grayscale, cgrayscale, COLORMAP_JET);
imshow(format("%d", i), cgrayscale);
}
waitKey(0);
return 0;
}
I see you are using OpenCV 2.4.0. As the developer I admit the confusion is my fault: I didn't thoroughly check the input data passed to the training method back then, so people passing wrongly aligned data got error messages like yours. Most likely the error you see happens, because your training images don't have equal size. This is necessary for the Eigenfaces and Fisherfaces algorithm (not for the Local Binary Patterns Histograms). OpenCV 2.4.0 just tries to reshape the data to a matrix and blows up with the error message you see; OpenCV 2.4.2 instead checks (before training) if the input data is correctly aligned and throws a meaningful exception... with a very clear message.
This post assumes it could also be due to linking the OpenCV libraries:
Getting OpenCV Error "Image step is wrong" in Fisherfaces.train() method
If it's not linking the libraries it might be due to the image size. Resizing your training images, can easily be done OpenCV with cv::resize:
http://docs.opencv.org/modules/imgproc/doc/geometric_transformations.html#resize
But you probably should consider to switch to OpenCV 2.4.2, where all this is added:
http://opencv.org/opencv-v2-4-2-released.html
This version also comes with an extensive documentation at:
http://docs.opencv.org/trunk/modules/contrib/doc/facerec/index.html
However if you can't change to OpenCV 2.4.2 and you'll need to stay with OpenCV 2.4.0, then you could also use libfacerec:
https://github.com/bytefish/libfacerec
This is the project, that got merged into OpenCV. I made sure it works with OpenCV 2.4.0 and it'll leave you with exactely the same interface as the OpenCV 2.4.2 version. So once you feel like updating to OpenCV 2.4.2, you'll only switch the includes.
I got the same OpenCv error, I try all help that I find here, and it still gives me an exception (exception happend on .Predict() statement).
Problem was in the size of images. Size of an Images must be less then 100px (<100px) (not sure if exactly less then 100,maybe 100 would still work).
I change my pictures size of 150:150 to 80:80 and its working!
Hope I help someone, because this was annoying error.
I answered this question on another post but I want to make sure people searching for help with this error are sure to find the answer.
when you make the model
Ptr<FaceRecognizer> model = createFisherFaceRecognizer();
You need to pass two params
createFisherFaceRecognizer(int num_components=0, double threshold=DBL_MAX);
This page has more information on how createFisherFaceRecognizer works