how to save identical faces only once - c++

I am doing a project on face detection from surveillance cameras.Now I am at the stage of face detection and I can detect faces from each frame.After detecting the face I need store that face to local folder.Now I can save each face in the specified folder.
Problem Now it is saving every faces,but I need to save identical faces only once.That means if saved one face as a jpeg image and when face detection progress again the same face is coming, so this time I don't want to save that particular face.
This is my code:
#include <cv.h>
#include <highgui.h>
#include <time.h>
#include <stdio.h>
using namespace std;
int ct=1;
int ct1=0;
IplImage *frame;
int frames;
void facedetect(IplImage* image);
void saveImage(IplImage *img,char *ex);
IplImage* resizeImage(const IplImage *origImg, int newWidth,int newHeight, bool keepAspectRatio);
const char* cascade_name="haarcascade_frontalface_default.xml";
int k=1;
int main(int argc, char** argv)
{
CvCapture *capture=cvCaptureFromFile("Arnab Goswami on Pepper spary rajagopal Complete NewsHour Debate (Mobile).3gp");
int count=1;
while(1)
{
frame = cvQueryFrame(capture);
if(count%30==0)
{
facedetect(frame);
}
count++;
}
cvReleaseCapture(&capture);
return 0;
}
void facedetect(IplImage* image)
{
ct1++;
cvNamedWindow("output");
int j=0,i,count=0,l=0,strsize;
char numstr[50];
int arr[100],arr1[100];
CvPoint ul,lr,w,h,ul1,lr1;
CvRect *r;
//int i=0;
IplImage* image1;IplImage* tmpsize;IplImage* reimg;
CvHaarClassifierCascade* cascade=(CvHaarClassifierCascade*) cvLoad(cascade_name);
CvMemStorage* storage=cvCreateMemStorage(0);
const char *extract;
if(!cascade)
{
cout<<"Coulid not load classifier cascade"<<endl;
}
if(cascade)
{
CvSeq*faces=cvHaarDetectObjects(image,cascade,storage,1.1,1,CV_HAAR_DO_CANNY_PRUNING,cvSize(10,10));
//function used for detecting faces.o/p is list of detected faces.
for(int i=0;i<(faces ? faces->total : 0);i++)
{
string s1="im",re,rename,ex=".jpeg";
sprintf(numstr, "%d", k);
re = s1 + numstr;
rename=re+ex;
char *extract1=new char[rename.size()+1];
extract1[rename.size()]=0;
memcpy(extract1,rename.c_str(),rename.size());
//Copies the values of rename.size from the location pointed by source //(rename.c_str)directly to the memory block pointed by destination(extract).
strsize=rename.size();
r=(CvRect*) cvGetSeqElem(faces,i);//draw rectangle outline around each image.
ul.x=r->x;
ul.y=r->y;
w.x=r->width;
h.y=r->height;
lr.x=(r->x + r->width);
lr.y=(r->y + r->height);
cvSetImageROI(image,cvRect(ul.x,ul.y,w.x,h.y));
image1=cvCreateImage(cvGetSize(image),image->depth,image->nChannels);
cvCopy(image, image1, NULL);
reimg=resizeImage(image1, 40, 40, true);
saveImage(reimg,extract1);
cvResetImageROI(image);
cvRectangle(image,ul,lr,CV_RGB(1,255,0),3,8,0);
j++,count++;
k++;
cout<<"frame"<<ct1<<" "<<"face"<<ct<<":"<<"x: "<<ul.x<<endl;
cout<<"frame"<<ct1<<" "<<"face"<<ct<<":"<<"y: "<<ul.y<<endl;
cout<<""<<endl;
ct++;
//cvShowImage("output",image);
}
//return image;
//cvNamedWindow("output");//creating a window.
cvShowImage("output",image);//showing resized image.
cvWaitKey(0);
}
}
void saveImage(IplImage *img,char *ex)
{
int i=0;
char path[255]="/home/athira/Image/OutputImage";
char *ext[200];
char buff[1000];
ext[i]=ex;
sprintf(buff,"%s/%s",path,ext[i]);//copy ext[i] to buff
strcat(path,buff);//concat path & buff
cvSaveImage(buff,img);
i++;
}

You are using the haar feature-based cascade classifier for object detection. As far as i know these xml files are only trained to detect the specific objects based on hundreds of evaluated pictures (see cascade classifier training).
So to compare saved images you will need another "detection" mode, because you have to compare if two faces are identical with respect to the view angle and so on (keyword: biometric data).
The keyword you're looking for is "face recognition" i think. Just build up a database based on your detected faces and use them for face recognition after that.
Edit:
Another maybe helpful link: www.shervinemami.info/faceRecognition.html

If I understood correctly, what you want is to detect faces in one frame, save a thumbnail of this face. Then, in the following frame, you want to detect faces again but only save the thumbnails for those that were not present in the first frame.
This problem is hard, because the faces captured in a video always change from one frame to the next. This is due to noise in the images, to the fact that the persons may be moving, etc. As a consequence, no two faces are ever identical in a surveillance video.
Hence, in order to achieve what you asked, you need to determine if the face you are considering has already been observed in previous frames. In its general form, this problem is not obvious one and is still the topic of a lot of research related to biometrics, pedestrian tracking and re-identification, etc. Therefore, you will have a hard time to achieve 100% effectiveness in detecting that a given face has already been observed.
However, if can accept a method that is not 100% effective, you could try the following approach:
Detect faces F0i in frame 0, with associated image position (x0i, y0i), and store the thumbnails
Compute sparse optical-flow (e.g. using KLT, see this link) on the positions (xn-1i, yn-1i) of the faces in previous frame n-1, in order to estimate their positions (xxni, yyni) in the current frame n.
Detect faces F0i in the current frame n, with associated image position (xni, yni), and save only the thumbnail of those which are not close to one of the predicted positions (xxni, yyni).
Increment n and repeat steps 2-3 using the next frame.
This is a simple algorithm using tracking to determine if a given face was already observed previously. It should be easier to implement than biometrics-based approaches, and also probably more appropriate in the context of video surveillance. However, it is not a 100% accurate, due to the limited effectivity of the optical-flow estimation and of the face detector.

Related

c++ laser detection using frame by frame analysis

I am trying to create a program that will open my camera, take a frame, and locate a laser point. I want to analyse each frame and find a red laser dot. Ideally, I would be able to insert a line at the x and y coordinate at the centroid of the laser dot that updates with each new frame. The camera I am using to take pictures of the laser is fairly high quality and takes pictures of size 2048x2048 so going through each pixel takes way too long. I need this software to run fairly quickly. I already have the code to open the camera and acquire frames.
laser image
What is the best way to write the code to locate the central point of the laser point within each frame?
Thanks,
#include <stdio.h>
#include <iostream>
#include "xiApiPlusOcv.hpp"
#include "opencv2/opencv.hpp"
using namespace cv;
using namespace std;
int main(int argc, char* argv[])
{
try
{
xiAPIplusCameraOcv cam;
// Retrieving a handle to the camera device
printf("Opening first camera...\n");
cam.OpenFirst();
cam.SetExposureTime(10000); //10000 us = 10 ms
// Note: The default parameters of each camera might be different in different API versions
cam.SetImageDataFormat(XI_RGB24);
printf("Starting acquisition...\n");
cam.StartAcquisition();
printf("First pixel value \n");
#define EXPECTED_IMAGES 200
for (int images = 0; images < EXPECTED_IMAGES; images++)
{
Mat cv_mat_image = cam.GetNextImageOcvMat();
cvWaitKey(20);
Mat outImg;
cv::resize(cv_mat_image, outImg, cv::Size(), .35, .35);;
imshow("image", outImg);
}
cam.StopAcquisition();
cam.Close();
printf("Done\n");
cvWaitKey(500);
}
catch (xiAPIplus_Exception& exp) {
printf("Error:\n");
exp.PrintError();
cvWaitKey(2000);
}
return 0;
}

OCR & OpenCV: Difference between two frames on high resolution images

According to this post OCR: Difference between two frames, I now know how to find pixel differences between two images with OpenCV.
I would like to improve this solution and use it with high resolution images (from a video) with rich content. The example above is not applicable with big images because the process is to slow (too much differences found, the "findCountours method" fills the tab with 250k elements which takes a huge time to process).
My application uses a RLE decoder to decode the compressed frames of the video. Once the frame is decoded, I would like to compare the current frame with the previous one in order to store the differences between the two frames in a "Mat" tab for example.
The goal of all of this is to be able to perform an analysis on the different pixels and to check if there is any latin character. This allows me to reduce the amount of pixels to analyze and to save precious time.
If anyone has other ideas instead of this one to perform such operations, feel free to propose it please.
Thank you for your help.
EDIT 1:
Example of two high resolution images of a computer screen. These are for the moment the perfect example of what I'm trying to analyse. As we can see there is just a window as difference between the two big images and I would like to analyze just the new "Challenge" window for any character.
EDIT 2:
I'm trying to tune the algorithm depending on the data analyzed. Typically on the two following pictures I only get the green lines as differences and no text at all (which is what is the most interesting). I'm trying to understand better how things work for this.
1st image:
2nd image:
3rd image:
As you can see I only have those green lines and never the text (at the best I can have just ONE letter when decreasing the countours[i].size())
In addition to the post you mentioned, you need to:
When you binarize the mask, use a threshold higher then 0 to remove small differences.
Remove some noise. You can find all connected components, and remove smaller ones.
Find the area of the bigger connected components. You can use convexHull and fillConvexPoly to get the mask of the different objects on screen
Copy the second image to a new image, with the given mask.
The result will look like:
Code:
#include <opencv2/opencv.hpp>
#include <vector>
using namespace std;
using namespace cv;
int main()
{
Mat3b img1 = imread("path_to_image_1");
Mat3b img2 = imread("path_to_image_2");
Mat3b diff;
absdiff(img1, img2, diff);
// Split each channel
vector<Mat1b> masks;
split(diff, masks);
// Create a black mask
Mat1b mask(diff.rows, diff.cols, uchar(0));
// OR with each channel of the N channels mask
for (int i = 0; i < masks.size(); ++i)
{
mask |= masks[i];
}
// Binarize mask
mask = mask > 100;
// Results images
vector<Mat3b> difference_images;
// Remove small blobs
//Mat kernel = getStructuringElement(MORPH_RECT, Size(5,5));
//morphologyEx(mask, mask, MORPH_OPEN, kernel);
// Find connected components
vector<vector<Point>> contours;
findContours(mask.clone(), contours, CV_RETR_EXTERNAL, CHAIN_APPROX_NONE);
for (int i = 0; i < contours.size(); ++i)
{
if (contours[i].size() > 1000)
{
Mat1b mm(mask.rows, mask.cols, uchar(0));
vector<Point> hull;
convexHull(contours[i], hull);
fillConvexPoly(mm, hull, Scalar(255));
Mat3b difference_img(img2.rows, img2.cols, Vec3b(0,0,0));
img2.copyTo(difference_img, mm);
difference_images.push_back(difference_img.clone());
}
}
return 0;
}

OpenCV C++ Color Detection and Print Mac

I'm new to OpenCV and am working on a video analysis project. Basically, I want to split my webcam into two sides (left and right), and have already figured out how to do this. However, I also want to analyze each side for red and green colors, and print out the amount of pixels that are red/green. I must have gone through every possible blog to figure this out, but alas it still doesn't work. The following code runs, however instead of detecting red as the code might suggest it seems to pick up white (all light sources and white walls). I have spent hours combing through the code but still cannot find the solution. Please help! Also note that this is being run on OSX 10.8, via Xcode. Thanks!
#include <iostream>
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/core/core.hpp"
using namespace cv;
using namespace std;
int main( int argc, char** argv )
{
VideoCapture cap(0); //capture the video from webcam
if ( !cap.isOpened() ) // if not success, exit program
{
cout << "Cannot open the web cam" << endl;
return -1;
}
namedWindow("HSVLeftRed", CV_WINDOW_AUTOSIZE);
namedWindow("HSVLeftGreen", CV_WINDOW_AUTOSIZE);
while (true) {
Mat image;
cap.read(image);
Mat HSV;
Mat threshold;
//Left Cropping
Mat leftimg = image(Rect(0, 0, 640, 720));
//Left Red Detection
cvtColor(leftimg,HSV,CV_BGR2HSV);
inRange(HSV,Scalar(0,0,150),Scalar(0,0,255),threshold);
imshow("HSVLeftRed",threshold);
//Left Green Detection
cvtColor(leftimg,HSV,CV_BGR2HSV);
inRange(HSV,Scalar(still need to find proper min values),Scalar(still need to find proper max values),threshold);
imshow("HSVLeftGreen",threshold);
}
return 0;
}
You're cropping a 640x720 area, which might not fit exactly your contents. Tip: Check your actual capture resolution with capture.get(CAP_PROP_FRAME_WIDTH) and capture.get(CAP_PROP_FRAME_HEIGHT). You might want to consider Mat threshold --> Mat thresholded. This is just some ranting :)
What I suspect is the actual issue is the threshold you use for HSV. According to cvtolor, section on RGB to HSV conversion,
On output 0 <= V <= 1.
so you should use a float representing your V threshold, i.e. 150 -> 150/255 ~= 0.58 etc.

how is PCA implemented on a camera captured image?

I have successfully implemented face detection part in my Face Recognition project.Now i have a rectangular region of face in an image.Now i have to implement PCA on this detected rectangular region to extract important features.I have used examples of implementing PCA on face databases.I want to know how we can pass our detected face to function implementing PCA?Is it that we pass the rectangle frame?
This is the code for my face detection.
#include "cv.h"
#include "highgui.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <assert.h>
#include <math.h>
#include <float.h>
#include <limits.h>
#include <time.h>
#include <ctype.h>
// Create a string that contains the exact cascade name
const char* cascade_name =
"haarcascade_frontalface_alt.xml";
/* "haarcascade_profileface.xml";*/
// Function prototype for detecting and drawing an object from an image
void detect_and_draw( IplImage* image );
// Main function, defines the entry point for the program.
int main( int argc, char** argv )
{
// Create a sample image
IplImage *img = cvLoadImage("Image018.jpg");
if(!img)
{
printf("could not load image");
return -1;
}
// Call the function to detect and draw the face positions
detect_and_draw(img);
// Wait for user input before quitting the program
cvWaitKey();
// Release the image
cvReleaseImage(&img);
// Destroy the window previously created with filename: "result"
cvDestroyWindow("result");
// return 0 to indicate successfull execution of the program
return 0;
}
// Function to detect and draw any faces that is present in an image
void detect_and_draw( IplImage* img )
{
// Create memory for calculations
static CvMemStorage* storage = 0;
// Create a new Haar classifier
static CvHaarClassifierCascade* cascade = 0;
int scale = 1;
// Create a new image based on the input image
IplImage* temp = cvCreateImage( cvSize(img->width/scale,img->height/scale), 8, 3 );
// Create two points to represent the face locations
CvPoint pt1, pt2;
int i;
// Load the HaarClassifierCascade
cascade = (CvHaarClassifierCascade*)cvLoad( cascade_name, 0, 0, 0 );
// Check whether the cascade has loaded successfully. Else report and error and quit
if( !cascade )
{
fprintf( stderr, "ERROR: Could not load classifier cascade\n" );
return;
}
// Allocate the memory storage
storage = cvCreateMemStorage(0);
// Create a new named window with title: result
cvNamedWindow( "result", 1 );
// Clear the memory storage which was used before
cvClearMemStorage( storage );
// Find whether the cascade is loaded, to find the faces. If yes, then:
if( cascade )
{
// There can be more than one face in an image. So create a growable sequence of faces.
// Detect the objects and store them in the sequence
CvSeq* faces = cvHaarDetectObjects( img, cascade, storage,
1.1, 2, CV_HAAR_DO_CANNY_PRUNING,
cvSize(40, 40) );
// Loop the number of faces found.
for( i = 0; i < (faces ? faces->total : 0); i++ )
{
// Create a new rectangle for drawing the face
CvRect* r = (CvRect*)cvGetSeqElem( faces, i );
// Find the dimensions of the face,and scale it if necessary
pt1.x = r->x*scale;
pt2.x = (r->x+r->width)*scale;
pt1.y = r->y*scale;
pt2.y = (r->y+r->height)*scale;
// Draw the rectangle in the input image
cvRectangle( img, pt1, pt2, CV_RGB(255,0,0), 3, 8, 0 );
}
}
// Show the image in the window named "result"
cvShowImage( "result", img );
// Release the temp image created.
cvReleaseImage( &temp );
}
Edit:
Just to notify anyone visiting this site. I have written some sample code to perform face recognition in videos using my libfacerec library:
https://github.com/bytefish/libfacerec/blob/master/samples/facerec_video.cpp
Original post:
I assume your problem is the following. You've used the Cascade Classifier cv::CascadeClassifier coming with OpenCV to detect and extract faces from images. Now you want to perform a face recognition on the images.
You want to use the Eigenfaces for face recognition. So the first thing you have to do is to learn the Eigenfaces from the images you've gathered. I rewrote the Eigenfaces class for you to make it simpler. To learn the eigenfaces simply pass a vector with your face images and the corresponding labels (the subject) either to Eigenfaces::Eigenfaces or Eigenfaces::compute. Make sure all your images have the same size, you can use cv::resize to ensure this.
Once you have computed the Eigenfaces, you can get predictions from your model. Simply call Eigenfaces::predict on a computed model. The main.cpp shows you how to use the class and its methods (for prediction, projection, reconstruction of images), here's how to get a prediction for an image.
Now I see where your problem is. You are using the old OpenCV C API. That makes it's hard to interface with the new OpenCV2 C++ API my code is written in. Not to be offending, but if you want to interface with my code you better use the OpenCV2 C++ API. I can't give a guide on learning C++ and the OpenCV2 API here, there's a lot of documentation coming with OpenCV. A good start is the OpenCV C++ Cheat Sheet (also available at http://opencv.willowgarage.com/) or the OpenCV Reference Manual.
For recognizing images from the Cascade Detector, I repeat: First learn the Eigenfaces model with the persons you want to recognize, it's shown in the example coming with my code. Then you need to get the Region Of Interest (ROI), that's the face, the Rectangle the Cascade Detector outputs. Finally you can get a prediction for the ROI from the Eigenfaces model (you have computed it above), it's shown in the example coming with my code. You probably have to convert your image to grayscale, but that's all. That's how it's done.

OpenCV Frame Rate Issue

I am working on a school project with OpenCV. A major part of the program will be a comparison of histograms. There will be a database of histograms and new histograms will be created from a live video feed then compared to the histograms in the database. Right now I am just trying to get the histograms created correctly from the video feed. My problem is that the program crashes or slows down dramatically at random intervals. So my question is how do I prevent the program from crashing or slowing down? OpenCV has always been kind of flaky for me, so I'm not sure if it is an issue with my code or if it is just the nature of OpenCV. If it is to do with my code I think the issue might have something to do with the frame rates (a guess/gut feeling). I am useing "cvWaitKey" to "pace" the loading of frames, but the "Learning OpenCV" book has this to say about "cvWaitKey"
c = cvWaitKey(33);
if( c == 27 ) break;
Once we have displayed the frame, we then wait for 33 ms. If the user hits a key, then c
will be set to the ASCII value of that key; if not, then it will be set to –1. If the user hits
the Esc key (ASCII 27), then we will exit the read loop. Otherwise, 33 ms will pass and
we will just execute the loop again.
It is worth noting that, in this simple example, we are not explicitly controlling
the speed of the video in any intelligent way. We are relying solely on the timer in
cvWaitKey() to pace the loading of frames. In a more sophisticated application it would
be wise to read the actual frame rate from the CvCapture structure (from the AVI) and
behave accordingly!
You will see in my code below (modified from here) that I my loop waits 10ms before starting the next execution. Often times the program will run with no issues at all, but sometimes it will crash less than a minute in, or five minutes in, there really is not pattern that I can detect. Any suggestions on how this crash( or slow down) can be prevented would be welcomed. Also I should add that I am using OpenCV 1.1 (can't ever get OpenCV 2.0 to work right), I am using Visual Studio 2008, and I create an .MSI installer package everytime I modify my code, that is, I do not debug in Visual Studio. Dependencies are cv110.dll, cxcore110.dll, and highgui110.dll. My code is below:
// SLC (Histogram).cpp : Defines the entry point for the console application.
#include "stdafx.h"
#include <cxcore.h>
#include <cv.h>
#include <cvaux.h>
#include <highgui.h>
#include <stdio.h>
#include <sstream>
#include <iostream>
using namespace std;
int main(){
CvCapture* capture = cvCaptureFromCAM(0);
if(!cvQueryFrame(capture)){
cout<<"Video capture failed, please check the camera."<<endl;
}
else{
cout<<"Video camera capture successful!"<<endl;
};
CvSize sz = cvGetSize(cvQueryFrame(capture));
IplImage* image = cvCreateImage(sz, 8, 3);
IplImage* imgHistogram = 0;
IplImage* gray = 0;
CvHistogram* hist;
cvNamedWindow("Image Source",1);
cvNamedWindow("Histogram",1);
for(;;){
image = cvQueryFrame(capture);
//Size of the histogram -1D histogram
int bins = 256;
int hsize[] = {bins};
//Max and min value of the histogram
float max_value = 0, min_value = 0;
//Value and normalized value
float value;
int normalized;
//Ranges - grayscale 0 to 256
float xranges[] = {0, 256};
float* ranges[] = {xranges};
//Create an 8 bit single channel image to hold a grayscale version of the original picture
gray = cvCreateImage(cvGetSize(image), 8, 1);
cvCvtColor(image, gray, CV_BGR2GRAY);
//Planes to obtain the histogram, in this case just one
IplImage* planes[] = {gray};
//Get the histogram and some info about it
hist = cvCreateHist(1, hsize, CV_HIST_ARRAY, ranges,1);
cvCalcHist(planes, hist, 0, NULL);
cvGetMinMaxHistValue(hist, &min_value, &max_value);
printf("Minimum Histogram Value: %f, Maximum Histogram Value: %f\n", min_value, max_value);
//Create an 8 bits single channel image to hold the histogram and paint it white
imgHistogram = cvCreateImage(cvSize(bins, 50),8,3);
cvRectangle(imgHistogram, cvPoint(0,0), cvPoint(256,50), CV_RGB(255,255,255),-1);
//Draw the histogram
for(int i=0; i < bins; i++){
value = cvQueryHistValue_1D(hist, i);
normalized = cvRound(value*50/max_value);
cvLine(imgHistogram,cvPoint(i,50), cvPoint(i,50-normalized), CV_RGB(0,0,0));
}
cvFlip(image, NULL, 1);
cvShowImage("Image Source", image);
cvShowImage("Histogram", imgHistogram);
//Page 19 paragraph 3 of "Learning OpenCV" tells us why we DO NOT use "cvReleaseImage(&image)" in this section
cvReleaseImage(&imgHistogram);
cvReleaseImage(&gray);
cvReleaseHist(&hist);
char c = cvWaitKey(10);
//if ASCII key 27 (esc) is pressed then loop breaks
if(c==27) break;
}
cvReleaseImage(&image);
cvReleaseCapture(&capture);
cvDestroyAllWindows();
}
Only a few things I can see or recommend:
Considering the build, make sure you're building in Release. Also, make sure the build of OpenCV you're using was built with OpenMP enabled, it makes an enormous difference.
Try moving your allocations outside the loop. Every loop you're re-creating gray and other images, when they should be re-used.
The other thing is your style, which makes it difficult to give good recommendations easily. It's poor style to pre-declare a bunch of variables, this is C-style. Declare your variables just prior to their use, and the code will be easier to read.
Update: I found the issue, it was actually my hardware (well the driver I think). I was using a PS3 Eye because of the amazing frame rates, but for some reason OpenCV does not like the PS3 Eye all the time. Sometimes it works great and other times not so great. I have verfied this on three computers, all of which run my code good with a standard web cam but randomly lock up when the PS3 Eye is used. Still, thank you for your suggestions GMan!