I have a profile face:
and a frontal face image:
Output: aligned profile face with reference to frontal face.
Idea: I just need to know which 3 common points I can take,which will be visible on both faces and then use affineTransform and display the aligned profile face
OR any other **simple method** of doing so
development envi.:c++ and opencv 2.4.2
what I tried:
haarcascade feature detection(common detection point in both images=eye) ; it wont detect ear in frontal face
OpenCV: Shift/Align face image relative to reference Image (Image Registration) (I get error message)
As discussed here by #bytefish, finding the accurate position of the eyes in a given image is far from trivial. The Haar-cascades for finding the eyes in OpenCV produce too many false positive to be useful, moreover this approach won't be robust to image rotation.
You'll need a robust head pose estimation for aligning face images. Here are two most robust ones (with code available):
Gary B. Huang, Vidit Jain, and Erik Learned-Miller. Unsupervised joint alignment of complex images. International Conference on Computer Vision (ICCV), 2007. (Project page), (PDF Online available), (Source code)
X. Zhu, D. Ramanan. Face Detection, Pose Estimation and Landmark Localization in the Wild Computer Vision and Pattern Recognition (CVPR) Providence, Rhode Island, June 2012. (Project page), (PDF Online available), (Source code)
For example, using the method described in the second paper, you will get more robust features like that are shown in the following images. And these robust features will, in turn, ensure to generate more robust face alignment performance.
If you look for something really simple you can treat your mouth as a line on a planar object and calculate the rotation by the amount of line foreshortening. You should never smile though when you take pictures nor when you write your code.
A much cooler approach though would be to map your face as a texture to a predefined 3D model and rotate it until it correlates best with your profile view.
Of course, the right way to do this is to use a set of different head rotations to train a binary classifier that does only pairwise intensity comparisons as in Lepetit paper.
Related
Here I say that I have tried many tutorials to implement face recognition in OpenCV 3.2 by using the FaceRecognizer class in face module. But I did not get the accepted result as I wish.
Here I want to ask and I want to know, that what is the best way or what are the conditions to be care off during training and recognizing?
What I have done to improve the accuracy:
Create (at least) 10 faces for training each person in the best quality, size, and angle.
Try to fit the face in the image.
Equalize the HIST of the images
And then I have tried all the three face recognizer (EigenFaceRecognizer, FisherFaceRecognizer, LBPHFaceRecognizer), the result all was the same, but the recognition rate was really very low, I have trained only for three persons, but also cannot recognize very well (the fist person was recognized as the second and so on problems).
Questions:
Do the training and the recognition images must be from the same
camera?
Do the training images cropped manually (photoshop -> read images then train) or this task
must be done programmatically (detect-> crop-> resize then train)?
And what are the best parameters for the each face recognizer (int num_components, double threshold)
And how to set training Algorithm to return -1 when it is an unknown
person.
Expanding on my comment, Chapter 8 in Mastering OpenCV provides really helpful tips for pre-processing faces to make aid the recognition process, such as:
taking a sample only when both eyes are detected (via haar cascade)
Geometrical transformation and cropping: This process would include scaling, rotating, and translating the images so that the eyes are aligned, followed by the removal of the forehead, chin, ears, and background from the face image.
Separate histogram equalization for left and right sides: This process standardizes the brightness and contrast on both the left- and right-hand sides of the face independently.
Smoothing: This process reduces the image noise using a bilateral filter.
Elliptical mask: The elliptical mask removes some remaining hair and background from the face image.
I've added a hacky load/save to my fork of the example code, feel free to try it/tweak it as you need it. Currently it's very limited, but it's a start.
Additionally, you should also check OpenFace and it's DNN face recognizer.
I haven't played with that yet so can't provide details, but it looks really cool.
I'm working with SDK 1.8 and I'm getting the depth stream from the Kinect. Now, I want to hold a paper of A4 size in front of the camera and want to get the co-ordinates of the corners of that paper so I can project an image onto it.
How can I detect the corners of the paper and get the co-ordinates? Does Kinect SDK 1.8 provide that option?
Thanks
Kinect SDK 1.8 does not provide this feature itself (to my knowledge). Depending on the language which you use for coding, there most certainly are libraries which allow such an operation if you segment it into steps.
OpenCV for example is quite useful in image-processing. When I once worked with the Kinect for object recognition, I used AForge with C#.
I recommend to target the challenge as follows:
Edge Detection:
You will apply edge detection algorithms such as the Canny Filter onto the image. First you will probably - depending on the library - transform your depth picture into a greyscale picture. The resulting image will be greyscale as well and the intensity of a pixel correlates with the probability of it belonging to an edge. Using a threshold, you will binarize this picture to black/white.
Hough Transformation: is used to get the position and parameters of a line within a image, which allows further calculation. Hough Transformation is VERY sensistive to its parameters and you will spend a lot of time in tuning those to get good results.
Calculation of edge points: Assuming that your Hough Transformation was successful, you can now calculate all intersections or the given lines which will yield the points that you are looking for.
All of these steps (especially Edge Detection and Hough Transformation) have been asked/answered/discussed in this forum.
If you provide code and intermediate results or further question, you can get a more detailled answer.
p.s.
I remember that the kinect was not that accurate and that noise was a topic. Therefore you might consider using a filter before doing these operations.
I am a new user of opencv. I am currently doing a project of performing product inspection with opencv. I was planning to extract edges of good product and the bad product then compare their edges maybe with mean square difference. However, it is already quite difficult to extract the edge clearly as the first step.
Good sample:Good product
[![enter image description here][1]][1]
When I use canny edge detection, the edge of the good product (the blue part of the picture) only includes part of the product and is as follows:
Edge of good product
[![enter image description here][2]][2]
I also tried to use adaptiveThreshold to make the greyscale picture more clear and then use the edge detection. But, the edge detected is not as good as expected because of many noise.
Therefore, I would like to ask for a solution of extracting the edge or any better way of comparing good product and bad product with opencv. Sorry for the bad english above.
This task can be made simple if some assumptions like those below are valid. For example:
Products move along the same production line, and therefore the lighting across all product images stays the same.
The products lie on a flat surface parallel to the focal plane of the camera, so the rotation of objects will be only around the axis of the lens.
The lighting can be controlled (this will help you to get better edges, and is actually used in production line image processing)
Since we cannot see the images that you added, it is a bit hard to see what exactly the situation is. But if the above assumptions are valid, image differencing combined with image signatures is one possible approach.
Another possibility is to train a Haar Cascade classifier using good and bad products. With this, the edges and all that will be taken care of. But you will have to collect a lot of data and train the classifier.
I am currently working on a robotic project: a robot must grab an cube using a Kinect camera that process cube detection and calculate coordinates.
I am new in computer vision. I first worked on static image of square in order to get a basic understanding. Using C++ and openCV, I managed to get the corners (and their x y pixel coordinates) of the square using smoothing (remove noise), edge detection (canny function), lines detection (Hough transform) and lines intersection (mathematical calculation) on an simplified picture (uniform background).
By adjusting some threshold I can achieve corners detection assuming that I have only one square and no line feature in the background.
Now is my question: do you have any direction/recommendation/advice/literature about cube recognition algorithm ?
What I have found so far involves shape detection combined with texture detection and/or learning sequence. Moreover, in their applications, they often use GPU/parallellisation computing, which I don't have...
The Kinect also provided a depth camera which gives distance of the pixel from the camera. Maybe I can use this to bypass "complicated" image processing ?
Thanks in advance.
OpenCV 3.0 with contrib includes surface_matching module.
Cameras and similar devices with the capability of sensation of 3D
structure are becoming more common. Thus, using depth and intensity
information for matching 3D objects (or parts) are of crucial
importance for computer vision. Applications range from industrial
control to guiding everyday actions for visually impaired people. The
task in recognition and pose estimation in range images aims to
identify and localize a queried 3D free-form object by matching it to
the acquired database.
http://docs.opencv.org/3.0.0/d9/d25/group__surface__matching.html
I'm trying to make a hand detection program by using OpenCV and Haar cascade. It works quite well but it's very jerky. So I'm asking myself if this is a trouble of the haar file that would be too 'cheap' or if there's a way to refine the detection by using contours or feature detection (or may be some other techniques).
What I would like to perform would be the same as this face detection, but for hands : Face Detection (see FaceOSC)
Thanks a lot.
EDIT : here is the kind of stuff I would like to do : Hand extraction It seems that he performs it with contour detection, but how to find the hand ?
The Hand Extraction video, you gave the link, is based on skin color detection and convex hull finding.
1) Change image to YCrCb (or HSV).
2) Threshold the image so that hand becomes white and everything other to black.
3) Remove noise
4) Find center of hand (if you like).
5) Use convex hull to find sharpest points which will be finger tips.
You can get full details from this paper.
Anyway, no need of haar cascades.
obviously if the HAAR classifier-based detection results become so-called 'jerky', in my opinion which means the detection is not stable and jumps around the detecting image, then the problem is on the quality of classifier.
as far as there are enough positive/negative samples, lets say 5k/5k, the results should be quite robust already. Based on my experiences, I used 700 positive hand gesture samples and 1200 negative samples, and the results seemed satisfied to some extent. but after I used another group of 8000 positive samples and 10200 negative samples with different features included, the results were even worse than the former.
So, I would suggest you to carefully reset your training samples, such like the ratio, content features and colours.