I need to be able to identify an opened door in an image taken from a kinect camra mounted on a robot.
The problem is that most of the time the image won't capture the entire door but only the lower half.
So I can't just train a HOG detector for doors because it would need to be trained on the entire door frame.
I am also able to get the whole range of kinect depth images. Would it be possible to look for something like a "hole in the wall" an assume that is a door?
I recently found this algorithm on the net.
http://docs.opencv.org/master/d1/dee/tutorial_moprh_lines_detection.html#gsc.tab=0
Now that I am able to reliably identify verticl edges in the picture, i can check them against my depth senosr data, to see if they are edges of an open door.
E.g.: if the depth to the right of the edge is far greater than on the left, its probably the left edge of the door.
With this i can identify the area "door" pretty acuratly.
I hope this can help someone else as well.
Related
Here I say that I have tried many tutorials to implement face recognition in OpenCV 3.2 by using the FaceRecognizer class in face module. But I did not get the accepted result as I wish.
Here I want to ask and I want to know, that what is the best way or what are the conditions to be care off during training and recognizing?
What I have done to improve the accuracy:
Create (at least) 10 faces for training each person in the best quality, size, and angle.
Try to fit the face in the image.
Equalize the HIST of the images
And then I have tried all the three face recognizer (EigenFaceRecognizer, FisherFaceRecognizer, LBPHFaceRecognizer), the result all was the same, but the recognition rate was really very low, I have trained only for three persons, but also cannot recognize very well (the fist person was recognized as the second and so on problems).
Questions:
Do the training and the recognition images must be from the same
camera?
Do the training images cropped manually (photoshop -> read images then train) or this task
must be done programmatically (detect-> crop-> resize then train)?
And what are the best parameters for the each face recognizer (int num_components, double threshold)
And how to set training Algorithm to return -1 when it is an unknown
person.
Expanding on my comment, Chapter 8 in Mastering OpenCV provides really helpful tips for pre-processing faces to make aid the recognition process, such as:
taking a sample only when both eyes are detected (via haar cascade)
Geometrical transformation and cropping: This process would include scaling, rotating, and translating the images so that the eyes are aligned, followed by the removal of the forehead, chin, ears, and background from the face image.
Separate histogram equalization for left and right sides: This process standardizes the brightness and contrast on both the left- and right-hand sides of the face independently.
Smoothing: This process reduces the image noise using a bilateral filter.
Elliptical mask: The elliptical mask removes some remaining hair and background from the face image.
I've added a hacky load/save to my fork of the example code, feel free to try it/tweak it as you need it. Currently it's very limited, but it's a start.
Additionally, you should also check OpenFace and it's DNN face recognizer.
I haven't played with that yet so can't provide details, but it looks really cool.
Suppose I want to find out if there is a person in a bed or not using cameras and computer vision algorithms. One can assume that the camera provides RGB, infrared and depth data.
I don't really have a good idea how to solve this. So far I came up with this:
Estimate a plane using RANSAC of the bed object. This plane should be further away from the ground plane, if there is a person in the bed. This seems very unstable though, assumes that the normal height of a bed is known and can easily be broken if the bed has an adjustable head part (e.g. in a hospital)
Face detection. Try to detect a face in the bed. Probably also isn't very reliable since the face can be sideways to the camera and partly covered.
Use the infrared-image. I am not sure how much you would see through the blanket and what would happen if the person just left the bed and the bed is still warm?
Is there a good way to do this? Or, to be reliable, you would have to use pressure sensors in the bed?
Thanks!
I dont know about infrared images but for camera based video processing this kind of problem is widely studied.
If your problem is to detect a person in a bed which is "Normally empty" then I think the simplest algorithm would be to capture successive frames and calculate their difference.
The existence of human in the frame would make it different from a frame capturing only empty bed. Depending on various algorithms like this you would get different reliability.
Otherwise you can go directly for human detection in video frames. One possible algorithm is described here.
Edit:
Your problem is harder than i thought. The following approach might solve the cases.
The main idea is to use bunch of features at once to get higher accuracy and remove false positives.
Use HOG person detector at top level to detect a person's entry in the scene. If the position of the possible entry doors are known or detectable using edge lines in the scene use it to increase accuracy. (At the point of entry the diference in successive frames will be located near the doors)
Use Edge lines to track the human. And use the bed edges to track the position of the human. The edges of human should be bounded by the edges of the bed.
If the difference is located within the bed implies human is in the bed but moving.
If needed as a preprocessing step include analysis of texture, connected component to remove possible moving objects in the room for higher accuracy (for example:- movement of clothes because of air).
Also use face detectors to increase accuracy.
Infrared that camera uses has a different frequency than infrared signal from a warm object. Unless you are using military grade IR scanners you can forget about connection IR-warmth. But IR is still useful if there is limited light or you use it for depth maps.
Go with depth (Kinect style) and estimate bed as a segment at your image. It should have some features in depth (certain dimension, flatness, etc). The bed usually surrounded by walls or floor that are easy to segment out. You algorithm can also be tuned to the distance to the bed and cut it out based just on depth range.
As other people said, it will be useful to learn more about your particular goal or application. What is background or environment around the bed? how does it looks when there is no person in it? Can a person simulate his/her presence(as in prison escape scenario), etc. etc.
I am currently doing a project where I need to locate ink drops in a video, perform measurements such as volume estimation, velocity, and distance travelled before it becomes spherical.
Firstly, I would like to know whether I am along the right tracks in tackling this project. At the moment I have:
1.) Converted the original image to grayscale
2.) Applied Gaussian Blur then Canny edge detection (Click here for image)
3.) Located the white pixels using findNonZero() then calculated the sum of blocks of rows and the block with the highest concentration of white pixels and all the rows above it are cropped out). This removes the print heads in the image so the ROI is only the droplets below it.
4.) Used the findContours to find the contours. (Click here for image)
The above 4 steps are what I have done so far. Are the following steps below what I should do next?
Dilate the binary image first after cropping and before finding contours to ensure the contours will be closed and not open?
Maybe ignore the ones that are very open? (Any tips on how to actually do this?)
floodFill() every closed circles
Find the each contours' area using contourArea() (Can I then estimate the volume of the drop after this step with a few assumptions like its shape, pixel to volume ratio, etc?)
Find the centre of each contour and save it to an array so I can compare it to the centre of the same drop in the next frame. Once I know distance travelled of the centre of the droplet and the frame rate of the video I should be able to estimate velocity.
I am also unsure of how I can give a drop an ID so I can be sure I am tracking it properly and know when a new drop has entered the ROI.
Any help would be greatly appreciated, Thank You.
I think that your idea is good and can be quite easily extended to something that will satisfy you.
For clarification i will call red ROI from your image "redROI".
Find all droplets in redROI. Remember positions and IDs.
For each droplet position from previous step create a ROI similar to yellow rectangle:
For each rectangle check whether there is a droplet inside it.
If yes - probably it's the droplet from the previous frame, so the one you are looking for.
If no - you may try to search again in a little bit bigger rectangle or assume that the darkset point of this ROI is you droplet. If ROI is near bottom of redROI probably the droplet is gone - forget about it.
Note few things:
-size of the rectangle depends on how fast droplets move and whether they can move only vertical or diagonal (wind can change direction of move) too.
-before searching for droplets, check whether all rectangles are disjoint (the don't have any common area -> (Rect1 & Rect2).area() == 0 for each pair of rectangles).
-before searching for droplets in ROI make sure this ROI is inside redROI. So just use this code: roi = roi & redROI;
After finding new positions of every old droplet, search for droplets in whole redROI, so you won't miss any new droplet.
Let me know if you don't understand some part of this idea - i will try to explain it better.
Maybe ignore the ones that are very open? (Any tips on how to actually
do this?)
I'm not sure about it, so check it. Try to use CV_RETR_LIST as the third parameter of findContours and check the distance between first and the last point from returned (by findContours) contour - if the distance is big than the contour is open, if no - it is closed.
floodFill() every closed circles
You can just use drawContours and set thickness parameter to -1 - simpler and faster solution.
edit:
You can try to use optical flow as well - it's already implemented in openCV, here you can read nice tutorial about that: http://robotics.stanford.edu/~dstavens/cs223b/ (start from .pdf files)
I am at the start of developing a software using OpenCV in Microsoft Visual 2010 Express. Now what I need to know before i get into coding is the procedures i have to follow.
Overview:
I want to develop software that detects simple boxing moves such as (Left punch, right punch) and outputs the results.
Now where am struggling is what approach should i take how should i tackle this development i.e.
Capture Video Footage and be able to extract lets say every 5th frame for processing.
Do i have to extract and store this frame perhaps have a REFERENCE image to subtract the capture frame from it.
Once i capture a frame what would be the best way to process it:
* Threshold it, then
* Detect the edges, then
* Smooth the edges using some filter, then
* Draw some BOUNDING boxes....?
What is your view on this guys or am i missing something or are there better simpler ways...? Any suggestions...?
Any answer will be much appreciated
Ps...its not my homework :)
I'm not sure if analyzing only every 5th frame will be enough, because usually punches are so fast that they could be overlooked.
I assume what you actually want to find is fast forward (towards camera) movements of fists.
In case of OpenCV I would first start off with such movements of faces, since some examples are already provided on how to do that in software package.
To detect and track faces you can use CvHaarClassifierCascade, but since this won't be fast enough for runtime detection, continue tracking such found face with Lukas-Kanade. Just pick some good-to-track points inside previously found face, remember their distance from arbitrary face middle, and at each frame update it. See this guy http://www.youtube.com/watch?v=zNqCNMefyV8 - example of just some random points tracked with Lukas-Kanade. Note that unlike faces, fists may not be so easy to track since their surface is rather uniform, better check Lukas-Kanade demo in OpenCV.
Of course with each frame actual face will drift away, once in a while re-run CvHaarClassifierCascade and interpolate to it your currently held face position.
You should be able to do above for fists also, but that will require training classifier with pictures of fists (classifier trained with faces is already provided in OpenCV).
Now having fists/face tracked you may try observing what happens to the points - when someone punches they move rapidly in some direction, while on the fist that remains still they don't move to much. And so, when you calculate average movement of single points in recent frames, the higher the value, the bigger chance that there was a punch. Alternatively, if somehow you've managed to track them accurately, if distance between each of them increases, that means object is closer to camera - and so a likely punch.
Note that without at least knowing change of a size of the fist in picture, it might be hard to distinguish if a movement of hand was forward or backward, or if the user was faking it by moving fists left or right. You may have to come up with some specialized algorithm (maybe with trial and error) to detect that, like say, increase a number of screen color pixels in location that previously fist was found.
What you are looking for is the research field of action recognition e.g. www.nada.kth.se/cvap/actions/ or an possible solution is e.g the STIP ( Space-time interest points) method www.di.ens.fr/~laptev/actions/ . But finally this is a tough job if you have to deal with occlusion or different point of views.
I am familiar with openCV, a powerful open source library and using that I am dealing with farm industry project where a mouse will be injected with drug , and its been kept on so called a stage which is surrounded by cylinder with painted strips of successive white and black. So i need to find out how many times the mouse will rotate its head to words the rotation of the cylinder . (its because it has got hang of drug) . How can i achieve this any opencv experts can help me out there.
I have added an image below
Seems an interesting one, these are my preliminary suggestions...
Depends on the resolution of the camera and how far your object (mouse) is from the camera...coz mouse is a small object so the image of the mouse need to cover good number of pixels in the image to differentiate head movement...
I don't think the mouse will stick to one position..it will keep moving in the cage...so you need to track the mouse...
At every position of the mouse you need to find the position of the head with respect to the body....that you can do using template matching (create templates of the head of the mouse)
Hence more info and some sample pictures are necessary to get the clear idea of the scene
EDIT AFTER IMAGE UPLOADED
since the camera is fixed hence create a circular region of interest...so that only movement inside this circle concerns you and not the moving cylinder outside the circle
subtract the present frame from the previous frame (frame differentiation) and store the absolute of the difference in an image.
absdiff(frameNow,framePrevs,diffofFrames);
threshold the diffofFrames as required to get the current position of the rat...
Now the task is easier if the image clearly shows its nose...since the nose has a pointed shape it can be detected by some template matching....however from the image you have given its difficult to make out the nose against a black background...However I can only suggest you the following process... green circles denote the tip of the nose...all I am trying to do is to get orientation of the head w.r.t. the body....for good results you need to have good images...