The documentation page of expo Face Detector states that it uses Google Mobile Vision framework (Now MLKIT) to detect faces on images. I would like to obtain this model to use on a python program to obtain face landmarks on some local images that I have. These face landmark will be used to train another model later on.
However, after searching around, I was not able to obtain this exact model. The closest one I found is Face Detect by Media Pipe (https://google.github.io/mediapipe/solutions/face_detection). This model returns 6 key points (right eye, left eye, nose tip, mouth center, right ear tragion, and left ear tragion).
For the later part of my application, I require the positions of left cheek, right cheek, left mouth tip, right mouth tip (all these points are returned by expo Face Detector). Thus I am searching for the pre-trained model which returns all these features. Could anyone point to any resources where this might be available?
Related
I need to rotate images of faces. I want to understand the output of GCP face detection. For the roll angle, where is the axis supposed to be? at the mouth center? at the nose? at the bottom left of the image file?
The definition of roll angle is here: http://www.conitec.net/beta/aentity-pan.htm
GCP explanation of the output seems to be in this outdated page: https://developers.google.com/vision/face-detection-concepts
Here it seems the z-axis is located at the lower left corner, which cannot be if we want to be precise. To get the image with that rotation we will need the axis to be located in the head.
Update: I have been told that the rotation is with respect to the center of the image. Can anyone confirm this?
Cloud Vision API is the GCP service for Face Detection (let me point out that this feature doesn't support Facial Recognition). You can refer to this similar SO thread that contains additional information about Google Cloud Vision API where it may be helpful to better understand this feature.
As mentioned, in case this feature doesn't cover your current needs, you can use the Send Feedback button, located at the lower left and upper right corners of the service public documentation, as well as take a look at the Issue Tracker tool in order to raise a Vision API feature request and notify Google about this desired functionality.
Here I say that I have tried many tutorials to implement face recognition in OpenCV 3.2 by using the FaceRecognizer class in face module. But I did not get the accepted result as I wish.
Here I want to ask and I want to know, that what is the best way or what are the conditions to be care off during training and recognizing?
What I have done to improve the accuracy:
Create (at least) 10 faces for training each person in the best quality, size, and angle.
Try to fit the face in the image.
Equalize the HIST of the images
And then I have tried all the three face recognizer (EigenFaceRecognizer, FisherFaceRecognizer, LBPHFaceRecognizer), the result all was the same, but the recognition rate was really very low, I have trained only for three persons, but also cannot recognize very well (the fist person was recognized as the second and so on problems).
Questions:
Do the training and the recognition images must be from the same
camera?
Do the training images cropped manually (photoshop -> read images then train) or this task
must be done programmatically (detect-> crop-> resize then train)?
And what are the best parameters for the each face recognizer (int num_components, double threshold)
And how to set training Algorithm to return -1 when it is an unknown
person.
Expanding on my comment, Chapter 8 in Mastering OpenCV provides really helpful tips for pre-processing faces to make aid the recognition process, such as:
taking a sample only when both eyes are detected (via haar cascade)
Geometrical transformation and cropping: This process would include scaling, rotating, and translating the images so that the eyes are aligned, followed by the removal of the forehead, chin, ears, and background from the face image.
Separate histogram equalization for left and right sides: This process standardizes the brightness and contrast on both the left- and right-hand sides of the face independently.
Smoothing: This process reduces the image noise using a bilateral filter.
Elliptical mask: The elliptical mask removes some remaining hair and background from the face image.
I've added a hacky load/save to my fork of the example code, feel free to try it/tweak it as you need it. Currently it's very limited, but it's a start.
Additionally, you should also check OpenFace and it's DNN face recognizer.
I haven't played with that yet so can't provide details, but it looks really cool.
I am a new user of opencv. I am currently doing a project of performing product inspection with opencv. I was planning to extract edges of good product and the bad product then compare their edges maybe with mean square difference. However, it is already quite difficult to extract the edge clearly as the first step.
Good sample:Good product
[![enter image description here][1]][1]
When I use canny edge detection, the edge of the good product (the blue part of the picture) only includes part of the product and is as follows:
Edge of good product
[![enter image description here][2]][2]
I also tried to use adaptiveThreshold to make the greyscale picture more clear and then use the edge detection. But, the edge detected is not as good as expected because of many noise.
Therefore, I would like to ask for a solution of extracting the edge or any better way of comparing good product and bad product with opencv. Sorry for the bad english above.
This task can be made simple if some assumptions like those below are valid. For example:
Products move along the same production line, and therefore the lighting across all product images stays the same.
The products lie on a flat surface parallel to the focal plane of the camera, so the rotation of objects will be only around the axis of the lens.
The lighting can be controlled (this will help you to get better edges, and is actually used in production line image processing)
Since we cannot see the images that you added, it is a bit hard to see what exactly the situation is. But if the above assumptions are valid, image differencing combined with image signatures is one possible approach.
Another possibility is to train a Haar Cascade classifier using good and bad products. With this, the edges and all that will be taken care of. But you will have to collect a lot of data and train the classifier.
I am currently working on a robotic project: a robot must grab an cube using a Kinect camera that process cube detection and calculate coordinates.
I am new in computer vision. I first worked on static image of square in order to get a basic understanding. Using C++ and openCV, I managed to get the corners (and their x y pixel coordinates) of the square using smoothing (remove noise), edge detection (canny function), lines detection (Hough transform) and lines intersection (mathematical calculation) on an simplified picture (uniform background).
By adjusting some threshold I can achieve corners detection assuming that I have only one square and no line feature in the background.
Now is my question: do you have any direction/recommendation/advice/literature about cube recognition algorithm ?
What I have found so far involves shape detection combined with texture detection and/or learning sequence. Moreover, in their applications, they often use GPU/parallellisation computing, which I don't have...
The Kinect also provided a depth camera which gives distance of the pixel from the camera. Maybe I can use this to bypass "complicated" image processing ?
Thanks in advance.
OpenCV 3.0 with contrib includes surface_matching module.
Cameras and similar devices with the capability of sensation of 3D
structure are becoming more common. Thus, using depth and intensity
information for matching 3D objects (or parts) are of crucial
importance for computer vision. Applications range from industrial
control to guiding everyday actions for visually impaired people. The
task in recognition and pose estimation in range images aims to
identify and localize a queried 3D free-form object by matching it to
the acquired database.
http://docs.opencv.org/3.0.0/d9/d25/group__surface__matching.html
I have a profile face:
and a frontal face image:
Output: aligned profile face with reference to frontal face.
Idea: I just need to know which 3 common points I can take,which will be visible on both faces and then use affineTransform and display the aligned profile face
OR any other **simple method** of doing so
development envi.:c++ and opencv 2.4.2
what I tried:
haarcascade feature detection(common detection point in both images=eye) ; it wont detect ear in frontal face
OpenCV: Shift/Align face image relative to reference Image (Image Registration) (I get error message)
As discussed here by #bytefish, finding the accurate position of the eyes in a given image is far from trivial. The Haar-cascades for finding the eyes in OpenCV produce too many false positive to be useful, moreover this approach won't be robust to image rotation.
You'll need a robust head pose estimation for aligning face images. Here are two most robust ones (with code available):
Gary B. Huang, Vidit Jain, and Erik Learned-Miller. Unsupervised joint alignment of complex images. International Conference on Computer Vision (ICCV), 2007. (Project page), (PDF Online available), (Source code)
X. Zhu, D. Ramanan. Face Detection, Pose Estimation and Landmark Localization in the Wild Computer Vision and Pattern Recognition (CVPR) Providence, Rhode Island, June 2012. (Project page), (PDF Online available), (Source code)
For example, using the method described in the second paper, you will get more robust features like that are shown in the following images. And these robust features will, in turn, ensure to generate more robust face alignment performance.
If you look for something really simple you can treat your mouth as a line on a planar object and calculate the rotation by the amount of line foreshortening. You should never smile though when you take pictures nor when you write your code.
A much cooler approach though would be to map your face as a texture to a predefined 3D model and rotate it until it correlates best with your profile view.
Of course, the right way to do this is to use a set of different head rotations to train a binary classifier that does only pairwise intensity comparisons as in Lepetit paper.