Using Opencv to calculate the haar feature - c++

I try to calculate Haar feature using opencv (Given an image).
Input: an image
output: haar feature
For that, I am using the FeatureEvaluator from OpenCV.
But I got an exception when I try to calculate one feature.
Here is how I am doing:
Ptr<FeatureEvaluator> ptrHaar = FeatureEvaluator::create(FeatureEvaluator::HAAR);
Mat img = imread(image_path); // image of size 2048*1536 correctly loaded
ptrHaar->setImage(img, Size(100, 100));
ptrHaar->setWindow(Point(0, 0));
double res = ptrHaar->calcOrd(0); // get the exception here

I think that you need to load/create some type of Haar feature, not just create an object. Try to load some Haar cascade classifier using load method and than try to use calcOrd method.

Your code is almost right. Only is missing is to read the CascadeClassifier previously trained. You can do this as follow:
FileStorage fs( "cascade.xml", FileStorage::READ );
//2) Then, create a FileNode to access the features:
FileNode featuresNode = fs["cascade"]["features"];
//3) Create the FeatureEvaluator, as you did in your first line
//4) Read the FileNode you have created:
ptrHaar->read(featuresNode);
And continue accordingly your code.
Note that ptrHaar->calcOrd(0) will read just the first feature rectangle, if you have more to read, you will need a loop, like this:
FileNodeIterator it = featuresNode.begin(), it_end = featuresNode.end();
int idx = 0;
for( ; it != it_end; ==it, idx++ )
{
res = ptrHaar.calcOrd(idx);
}

Related

How to save DLIB object detection chips properly?

I am having trouble saving image chips generated by DLIB's face detection model. The code below details my workflow. I have attempted saving the whole image, d_image below, and that works just fine. However, when I try to save each chip I get distorted output (see example below). I'm using dlib 19.4 on Ubuntu 16.04.
// object to store raw image data
cv::Mat rawImage;
// initialize the detector
dlib::frontal_face_detector detector = dlib::get_frontal_face_detector();
// using shape predictor object to create dull_object_detections
dlib::shape_predictor sp;
dlib::deserialize(argv[1]) >> sp;
// for writing out images
int image_id = 1;
while (true){
// retrieve image size
sockt.getData(&image_size, 4, NULL);
if (image_size > 0) {
rawImage.create(1, image_size, CV_8UC1);
// load incoming data from a stream
sockt.getData(rawImage.data, image_size, MSG_WAITALL);
// reshape and correct orientation
dlib::cv_image<dlib::bgr_pixel> d_image = utils::process_frame(rawImage);
// find the daces!
std::vector<dlib::rectangle> detections = detector(d_image);
if (detections.size() > 0){
// generate additional detection data so we can use
// dlib's extract_image_chips function
std::vector<dlib::full_object_detection> shapes;
for (int idx = 0; idx < detections.size(); idx++){
dlib::full_object_detection shape = sp(d_image, detections[idx]);
shapes.push_back(shape);
}
// write each chip to disk
dlib::array<dlib::array2d<dlib::bgr_pixel>> face_chips;
dlib::extract_image_chips(d_image, dlib::get_face_chip_details(shapes), face_chips);
for (int idx = 0; idx < face_chips.size(); idx++){
std::string fname = argv[2] + std::to_string(image_id) + ".jpg";
dlib::save_jpeg(face_chips[idx], fname);
image_id++;
}
}
Example saved chip:
Edit: Added comment to utils::process_frame. This function accepts a 1xN array and decodes as a JPEG using OpenCV
Something wrong with image formats you are using:
This is OpenCV's greyscale (1-channel) image
rawImage.create(1, image_size, CV_8UC1);
This is BGR (3-channel) image
dlib::cv_image<dlib::bgr_pixel> d_image = utils::process_frame(rawImage);
Dlib should throw an exception if image has incorrect number of channels, but it does not throw it in your case. This means that somewhere in utils::process_frame(rawImage) the image format is changed into 3-channel- check image formats first
And this construction code rawImage.create(1, image_size, CV_8UC1); constructs 1-row and image_size cols image.
Something is incorrect with image size and format
Please also note, that dlib does not copy image data into dlib::cv_image<dlib::bgr_pixel> d_image and the rawImage should remain unchanged by other threads until processing is finished
Anyways, you can call dlib::toMat, get OpenCV Mat and save it with OpenCV functions
UPDATE:
one more thing here:
dlib::cv_image<dlib::bgr_pixel> d_image = utils::process_frame(rawImage);
looks like utils::process_frame returns some temporary object that is destroyed after d_image is constructed. d_image does not hold returned data and it can be lost
So I suggest you to change your code like this:
cv::Mat uncompressed;
tils::process_frame(rawImage, uncompressed);
dlib::cv_image<dlib::bgr_pixel> d_image(uncompressed);;
where process_frame should take reference to cv::Mat and save its output into it

How to form data for SVM training OpenCV3

I am trying to write utility for training svm classifier for image classification in OpenCV3. But I have Floating point exception (core dumped) error during training process.
My main problem is that I don't know, I'm not sure exactly how to form training data to feed svm.train method.
This is code which is forming training data.
TrainingDataType SVMTrainer::prepareDataForTraining() {
cv::Mat trainingData(m_numOfAllImages, 28*28, CV_32FC1);
cv::Mat trainingLabels(m_numOfAllImages, 1, CV_32FC1);
int rowNum = 0;
// Item is pair of classId (int) and vector of images.
for(auto item : m_data){
int classId = item.first;
for(auto item1 : item.second){
Mat temp = item1.reshape(1,1);
temp.copyTo(trainingData.row(rowNum));
trainingLabels.at<float>(rowNum) = item.first;
++rowNum;
}
}
return cv::ml::TrainData::create(trainingData,
cv::ml::SampleTypes::ROW_SAMPLE,
trainingLabels) ;
}
void SVMTrainer::train(std::string& configPath){
// Read and store images in memory.
formClassifierData(configPath);
m_classifier = cv::ml::SVM::create();
// Training parameters:
m_classifier->setType(cv::ml::SVM::C_SVC);
m_classifier->setKernel(cv::ml::SVM::POLY);
m_classifier->setGamma(3);
m_classifier->setDegree(3);
TrainingDataType trainData = prepareDataForTraining();
m_classifier->trainAuto(trainData);
}
All images are already prepared with dimensions 28*28, black&white.
And actual train call is in this method
Can somebody tell me what I am doing wrong.
Thanks,
Its simple. Change the label format to CV_32SC1. It will definitely resolve your issue in opencv 3.0 ml.

my application crashes at KNearest::find_nearest

I want to implement a OCR feature.
I have collected some samples and i want to use K-Nearest to implement it.
So, i use the below code to load data and initialize KNearest
KNearest knn = new KNearest;
Mat mData, mClass;
for (int i = 0; i <= 9; ++i)
{
Mat mImage = imread( FILENAME ); // the filename format is '%d.bmp', presenting a 15x15 image
Mat mFloat;
if (mImage.empty()) break; // if the file doesn't exist
mImage.convertTo(mFloat, CV_32FC1);
mData.push_back(mFloat.reshape(1, 1));
mClass.push_back( '0' + i );
}
knn->train(mData, mClass);
Then, i call the code to find best result
for (vector<Mat>::iterator it = charset.begin(); it != charset.end(); ++it)
{
Mat mFloat;
it->convertTo(mFloat, CV_32FC1); // 'it' presents a 15x15 gray image
float result = knn->find_nearest(mFloat.reshape(1, 1), knn->get_max_k());
}
But, my application crashes at find_nearest.
Anyone could help me?
I seemed to find the problem...
My sample image is a converted gray image by cvtColor, but my input image isn't.
After i add
cvtColor(mImage, mImage, COLOR_BGR2GRAY);
between
if (mImage.empty()) break;
mImage.convertTo(mFloat, CV_32FC1);
find_nearest() return a value and my application is fine.

Updating OpenCV CvNormalBayesClassifier

I'm trying to use CvNormalBayesClassifier to train my program to learn skin pixel colors. I have a set of training images and response images. The response images are in black and white, skin regions are marked white. The following is my code,
CvNormalBayesClassifier classifier;
for (int i = 0; i < numFiles; i++) {
string trainFile = "images/" + int2str(i) + ".jpg";
string responseFile = "images/" + int2str(i) + "_mask.jpg";
Mat trainData = imread(trainFile, 1);
Mat responseData = imread(responseFile, CV_LOAD_IMAGE_GRAYSCALE);
trainData = trainData.reshape(1, trainData.rows * trainData.cols);
responseData = responseData.reshape(0, responseData.rows * responseData.cols);
trainData.convertTo(trainData, CV_32FC1);
responseData.convertTo(responseData, CV_32FC1);
classifier.train(trainData, responseData, Mat(), Mat(), i != 0);
}
However, it is giving the following error,
The function/feature is not implemented (In the current implementation the new training data must have absolutely the same set of class labels as used in the original training data) in CvNormalBayesClassifier::train
Many thanks.
As the error message states, you cannot 'update' the classifier in light of new class labels. The Normal Bayes Classifier learns a Mixture of Gaussians to represent the training data. If you suddenly start adding new labels this mixture model will cease to be correct and a new model must be learned from scratch.
Ok, I found that the problem was that the black and white images have been compressed and thus contain values ranging from 0-255. Therefore, there can be a new class label in the other images.
To solve this problem, use thresholding to make the value all become 0 or 255.

how is PCA implemented on a camera captured image?

I have successfully implemented face detection part in my Face Recognition project.Now i have a rectangular region of face in an image.Now i have to implement PCA on this detected rectangular region to extract important features.I have used examples of implementing PCA on face databases.I want to know how we can pass our detected face to function implementing PCA?Is it that we pass the rectangle frame?
This is the code for my face detection.
#include "cv.h"
#include "highgui.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <assert.h>
#include <math.h>
#include <float.h>
#include <limits.h>
#include <time.h>
#include <ctype.h>
// Create a string that contains the exact cascade name
const char* cascade_name =
"haarcascade_frontalface_alt.xml";
/* "haarcascade_profileface.xml";*/
// Function prototype for detecting and drawing an object from an image
void detect_and_draw( IplImage* image );
// Main function, defines the entry point for the program.
int main( int argc, char** argv )
{
// Create a sample image
IplImage *img = cvLoadImage("Image018.jpg");
if(!img)
{
printf("could not load image");
return -1;
}
// Call the function to detect and draw the face positions
detect_and_draw(img);
// Wait for user input before quitting the program
cvWaitKey();
// Release the image
cvReleaseImage(&img);
// Destroy the window previously created with filename: "result"
cvDestroyWindow("result");
// return 0 to indicate successfull execution of the program
return 0;
}
// Function to detect and draw any faces that is present in an image
void detect_and_draw( IplImage* img )
{
// Create memory for calculations
static CvMemStorage* storage = 0;
// Create a new Haar classifier
static CvHaarClassifierCascade* cascade = 0;
int scale = 1;
// Create a new image based on the input image
IplImage* temp = cvCreateImage( cvSize(img->width/scale,img->height/scale), 8, 3 );
// Create two points to represent the face locations
CvPoint pt1, pt2;
int i;
// Load the HaarClassifierCascade
cascade = (CvHaarClassifierCascade*)cvLoad( cascade_name, 0, 0, 0 );
// Check whether the cascade has loaded successfully. Else report and error and quit
if( !cascade )
{
fprintf( stderr, "ERROR: Could not load classifier cascade\n" );
return;
}
// Allocate the memory storage
storage = cvCreateMemStorage(0);
// Create a new named window with title: result
cvNamedWindow( "result", 1 );
// Clear the memory storage which was used before
cvClearMemStorage( storage );
// Find whether the cascade is loaded, to find the faces. If yes, then:
if( cascade )
{
// There can be more than one face in an image. So create a growable sequence of faces.
// Detect the objects and store them in the sequence
CvSeq* faces = cvHaarDetectObjects( img, cascade, storage,
1.1, 2, CV_HAAR_DO_CANNY_PRUNING,
cvSize(40, 40) );
// Loop the number of faces found.
for( i = 0; i < (faces ? faces->total : 0); i++ )
{
// Create a new rectangle for drawing the face
CvRect* r = (CvRect*)cvGetSeqElem( faces, i );
// Find the dimensions of the face,and scale it if necessary
pt1.x = r->x*scale;
pt2.x = (r->x+r->width)*scale;
pt1.y = r->y*scale;
pt2.y = (r->y+r->height)*scale;
// Draw the rectangle in the input image
cvRectangle( img, pt1, pt2, CV_RGB(255,0,0), 3, 8, 0 );
}
}
// Show the image in the window named "result"
cvShowImage( "result", img );
// Release the temp image created.
cvReleaseImage( &temp );
}
Edit:
Just to notify anyone visiting this site. I have written some sample code to perform face recognition in videos using my libfacerec library:
https://github.com/bytefish/libfacerec/blob/master/samples/facerec_video.cpp
Original post:
I assume your problem is the following. You've used the Cascade Classifier cv::CascadeClassifier coming with OpenCV to detect and extract faces from images. Now you want to perform a face recognition on the images.
You want to use the Eigenfaces for face recognition. So the first thing you have to do is to learn the Eigenfaces from the images you've gathered. I rewrote the Eigenfaces class for you to make it simpler. To learn the eigenfaces simply pass a vector with your face images and the corresponding labels (the subject) either to Eigenfaces::Eigenfaces or Eigenfaces::compute. Make sure all your images have the same size, you can use cv::resize to ensure this.
Once you have computed the Eigenfaces, you can get predictions from your model. Simply call Eigenfaces::predict on a computed model. The main.cpp shows you how to use the class and its methods (for prediction, projection, reconstruction of images), here's how to get a prediction for an image.
Now I see where your problem is. You are using the old OpenCV C API. That makes it's hard to interface with the new OpenCV2 C++ API my code is written in. Not to be offending, but if you want to interface with my code you better use the OpenCV2 C++ API. I can't give a guide on learning C++ and the OpenCV2 API here, there's a lot of documentation coming with OpenCV. A good start is the OpenCV C++ Cheat Sheet (also available at http://opencv.willowgarage.com/) or the OpenCV Reference Manual.
For recognizing images from the Cascade Detector, I repeat: First learn the Eigenfaces model with the persons you want to recognize, it's shown in the example coming with my code. Then you need to get the Region Of Interest (ROI), that's the face, the Rectangle the Cascade Detector outputs. Finally you can get a prediction for the ROI from the Eigenfaces model (you have computed it above), it's shown in the example coming with my code. You probably have to convert your image to grayscale, but that's all. That's how it's done.