Here I say that I have tried many tutorials to implement face recognition in OpenCV 3.2 by using the FaceRecognizer class in face module. But I did not get the accepted result as I wish.
Here I want to ask and I want to know, that what is the best way or what are the conditions to be care off during training and recognizing?
What I have done to improve the accuracy:
Create (at least) 10 faces for training each person in the best quality, size, and angle.
Try to fit the face in the image.
Equalize the HIST of the images
And then I have tried all the three face recognizer (EigenFaceRecognizer, FisherFaceRecognizer, LBPHFaceRecognizer), the result all was the same, but the recognition rate was really very low, I have trained only for three persons, but also cannot recognize very well (the fist person was recognized as the second and so on problems).
Questions:
Do the training and the recognition images must be from the same
camera?
Do the training images cropped manually (photoshop -> read images then train) or this task
must be done programmatically (detect-> crop-> resize then train)?
And what are the best parameters for the each face recognizer (int num_components, double threshold)
And how to set training Algorithm to return -1 when it is an unknown
person.
Expanding on my comment, Chapter 8 in Mastering OpenCV provides really helpful tips for pre-processing faces to make aid the recognition process, such as:
taking a sample only when both eyes are detected (via haar cascade)
Geometrical transformation and cropping: This process would include scaling, rotating, and translating the images so that the eyes are aligned, followed by the removal of the forehead, chin, ears, and background from the face image.
Separate histogram equalization for left and right sides: This process standardizes the brightness and contrast on both the left- and right-hand sides of the face independently.
Smoothing: This process reduces the image noise using a bilateral filter.
Elliptical mask: The elliptical mask removes some remaining hair and background from the face image.
I've added a hacky load/save to my fork of the example code, feel free to try it/tweak it as you need it. Currently it's very limited, but it's a start.
Additionally, you should also check OpenFace and it's DNN face recognizer.
I haven't played with that yet so can't provide details, but it looks really cool.
Related
I am a new user of opencv. I am currently doing a project of performing product inspection with opencv. I was planning to extract edges of good product and the bad product then compare their edges maybe with mean square difference. However, it is already quite difficult to extract the edge clearly as the first step.
Good sample:Good product
[![enter image description here][1]][1]
When I use canny edge detection, the edge of the good product (the blue part of the picture) only includes part of the product and is as follows:
Edge of good product
[![enter image description here][2]][2]
I also tried to use adaptiveThreshold to make the greyscale picture more clear and then use the edge detection. But, the edge detected is not as good as expected because of many noise.
Therefore, I would like to ask for a solution of extracting the edge or any better way of comparing good product and bad product with opencv. Sorry for the bad english above.
This task can be made simple if some assumptions like those below are valid. For example:
Products move along the same production line, and therefore the lighting across all product images stays the same.
The products lie on a flat surface parallel to the focal plane of the camera, so the rotation of objects will be only around the axis of the lens.
The lighting can be controlled (this will help you to get better edges, and is actually used in production line image processing)
Since we cannot see the images that you added, it is a bit hard to see what exactly the situation is. But if the above assumptions are valid, image differencing combined with image signatures is one possible approach.
Another possibility is to train a Haar Cascade classifier using good and bad products. With this, the edges and all that will be taken care of. But you will have to collect a lot of data and train the classifier.
I've working on this some time now, and can't find a decent solution for this.
I use OpenCV for image processing and my workflow is something like this:
Took a picture of a tv.
Split image in to R, G, B planes - I'm starting to test using H, S, V too and seems a bit promising.
For each plane, threshold image for a range values in 0 to 255
Reduce noise, detect edges with canny, find the contours and approximate it.
Select contours that contains the center of the image (I can assume that the center of the image is inside the tv screen)
Use convexHull and HougLines to filter and refine invalid contours.
Select contours with certain area (area between 10%-90% of the image).
Keep only contours that have only 4 points.
But this is too slow (loop on each channel (RGB), then loop for the threshold, etc...) and is not good enought as it not detects many tv's.
My base code is the squares.cpp example of the OpenCV framework.
The main problems of TV Screen detection, are:
Images that are half dark and half bright or have many dark/bright items on screen.
Elements on the screen that have the same color of the tv frame.
Blurry tv edges (in some cases).
I also have searched many SO questions/answers on Rectangle detection, but all are about detecting a white page on a dark background or a fixed color object on a contrast background.
My final goal is to implement this on Android/iOS for near-real time tv screen detection. My code takes up to 4 seconds on a Galaxy Nexus.
Hope anyone could help . Thanks in advance!
Update 1: Just using canny and houghlines, does not work, because there can be many many lines, and selecting the correct ones can be very difficult. I think that some sort of "cleaning" on the image should be done first.
Update 2: This question is one of the most close to the problem, but for the tv screen, it didn't work.
Hopefully these points provide some insight:
1)
If you can properly segment the image via foreground and background, then you can easily set a bounding box around the foreground. Graph cuts are very powerful methods of segmenting images. It appears that OpenCV provides easy to use implementations for it. So, for example, you provide some brush strokes which cover "foreground" and "background" pixels, and your image is converted into a digraph which is sliced optimally to split the two. Here is a fun example:
http://docs.opencv.org/trunk/doc/py_tutorials/py_imgproc/py_grabcut/py_grabcut.html
This is a quick something I put together to illustrate its effectiveness:
2)
If you decide to continue down the edge detection route, then consider using Mathematical Morphology to "clean up" the lines you detect before trying to fit a bounding box or contour around the object.
http://en.wikipedia.org/wiki/Mathematical_morphology
3)
You could train across a dataset containing TVs and use the viola jones algorithm for object detection. Traditionally it is used for face detection but you can adapt it for TVs given enough data. For example you could script downloading images of living rooms with TVs as your positive class and living rooms without TVs as your negative class.
http://en.wikipedia.org/wiki/Viola%E2%80%93Jones_object_detection_framework
http://docs.opencv.org/trunk/doc/py_tutorials/py_objdetect/py_face_detection/py_face_detection.html
4)
You could perform image registration using cross correlation, like this nice MATLAB example demonstrates:
http://www.mathworks.com/help/images/examples/registering-an-image-using-normalized-cross-correlation.html
As for your template TV image which would be slid across the search image, you could obtain a bunch of pictures of TVs and create "Eigenscreens" similar to how Eigenfaces are used for facial recognition and generate an average TV image:
http://jeremykun.com/2011/07/27/eigenfaces/
5)
It appears OpenCV has plenty of fun tools for describing shape and structure features, which appears to be mainly what you're interested in. Worth a look if you haven't seen this already:
http://docs.opencv.org/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html
Best of luck.
I have a profile face:
and a frontal face image:
Output: aligned profile face with reference to frontal face.
Idea: I just need to know which 3 common points I can take,which will be visible on both faces and then use affineTransform and display the aligned profile face
OR any other **simple method** of doing so
development envi.:c++ and opencv 2.4.2
what I tried:
haarcascade feature detection(common detection point in both images=eye) ; it wont detect ear in frontal face
OpenCV: Shift/Align face image relative to reference Image (Image Registration) (I get error message)
As discussed here by #bytefish, finding the accurate position of the eyes in a given image is far from trivial. The Haar-cascades for finding the eyes in OpenCV produce too many false positive to be useful, moreover this approach won't be robust to image rotation.
You'll need a robust head pose estimation for aligning face images. Here are two most robust ones (with code available):
Gary B. Huang, Vidit Jain, and Erik Learned-Miller. Unsupervised joint alignment of complex images. International Conference on Computer Vision (ICCV), 2007. (Project page), (PDF Online available), (Source code)
X. Zhu, D. Ramanan. Face Detection, Pose Estimation and Landmark Localization in the Wild Computer Vision and Pattern Recognition (CVPR) Providence, Rhode Island, June 2012. (Project page), (PDF Online available), (Source code)
For example, using the method described in the second paper, you will get more robust features like that are shown in the following images. And these robust features will, in turn, ensure to generate more robust face alignment performance.
If you look for something really simple you can treat your mouth as a line on a planar object and calculate the rotation by the amount of line foreshortening. You should never smile though when you take pictures nor when you write your code.
A much cooler approach though would be to map your face as a texture to a predefined 3D model and rotate it until it correlates best with your profile view.
Of course, the right way to do this is to use a set of different head rotations to train a binary classifier that does only pairwise intensity comparisons as in Lepetit paper.
For analysis we have a sequence of images or a movie. My aim is to create a semi automatic face and eye detection for these sequences. The sequences consist of about 4000 images with a frontal capture of a person slightly moving. I want to process these images semi automatic or manual to get the two/three ROIs of the face and eyes.
I tried OpenCV's cascade classifiers but for my sequences they do not turn out to be robust (with manual controll we need to get a rate of 100%). The cascade classifiers do not give positions, eg. when the person is looking slightly to the side.
Is there any semi automatic approach out there for imagej, matlab or opencv/c++ to select/correct the rois manually if false detected or to select templates for tracking ?
If you are processing a movie, it is reasonable to assume that the motion between frames is low. The following is a possible approach.
Initialize the first frame manually (or get user input to confirm/edit the positions detected by cascade classifiers)
For the next frame, check if the features detected are too far off the original positions. You can also check if the positions of different parts are moving in an illogical manner.
Stop and get the user to correct the points, if processing in step 2 suggest errors.
Note: With OpenCV cascades, face detection is generally accurate. But eye detection is not so accurate and you might not detect both eyes in some frames. Some projects use AAMs (Active Appearance Models) to robustly track a face, and this might work for you.
For face detection, try this list of 50+ API's :
http://blog.mashape.com/post/53379410412/list-of-50-face-detection-recognition-apis
for eyes detection you can try flandmark detector:
http://cmp.felk.cvut.cz/~uricamic/flandmark/
Or STASM:
http://www.milbo.users.sonic.net/stasm/
I'm trying to make a hand detection program by using OpenCV and Haar cascade. It works quite well but it's very jerky. So I'm asking myself if this is a trouble of the haar file that would be too 'cheap' or if there's a way to refine the detection by using contours or feature detection (or may be some other techniques).
What I would like to perform would be the same as this face detection, but for hands : Face Detection (see FaceOSC)
Thanks a lot.
EDIT : here is the kind of stuff I would like to do : Hand extraction It seems that he performs it with contour detection, but how to find the hand ?
The Hand Extraction video, you gave the link, is based on skin color detection and convex hull finding.
1) Change image to YCrCb (or HSV).
2) Threshold the image so that hand becomes white and everything other to black.
3) Remove noise
4) Find center of hand (if you like).
5) Use convex hull to find sharpest points which will be finger tips.
You can get full details from this paper.
Anyway, no need of haar cascades.
obviously if the HAAR classifier-based detection results become so-called 'jerky', in my opinion which means the detection is not stable and jumps around the detecting image, then the problem is on the quality of classifier.
as far as there are enough positive/negative samples, lets say 5k/5k, the results should be quite robust already. Based on my experiences, I used 700 positive hand gesture samples and 1200 negative samples, and the results seemed satisfied to some extent. but after I used another group of 8000 positive samples and 10200 negative samples with different features included, the results were even worse than the former.
So, I would suggest you to carefully reset your training samples, such like the ratio, content features and colours.