I am pretty new to CV, so forgive my stupid questions...
What I want to do:
I want to recognize a RC plane in live video (for now its only a recorded video).
What I have done so far:
Differences between frames
Convert it to grey scale
GaussianBlur
Threshold
findContours
Here are some example frames:
But there are also frames with noise, so there are more objects in the frame.
I thought I could do something like this:
Use some object recognition algorithm for every contour that has been found. And compute only the feature vector for each of these bounding rectangles.
Is it possible to compute SURF/SIFT/... only for a specific patch (smaller part) of the image?
Since it will be important that the algorithm is capable of processing real time video I think it will only be possible if I don't look at the whole image all the time?! Or maybe decide for example if there are more than 10 bounding rectangles I check the whole image instead of every rectangle.
Then I will look at the next frame and try to match my feature vector with the previous one. That way I will be able to trace my objects. Once these objects cross the red line in the middle of the picture it will trigger another event. But that's not important here.
I need to make sure that not every object which is crossing or behind that red line is triggering that event. So there need to be at least 2 or 3 consecutive frames which contain that object and if it crosses then and only then the event should be triggered.
There are so many variations of object recognition algorithms, I am bit overwhelmed.
Sift/Surf/Orb/... you get what I am saying.
Can anyone give me a hint which one I should chose or if what I am doing is even making sense?
Assuming the plane location doesn't change a lot from one frame to the next, I think you should look at object tracking instead of trying to estimate the location independently in each frame.
http://docs.opencv.org/modules/video/doc/motion_analysis_and_object_tracking.html
I am currently doing a project where I need to locate ink drops in a video, perform measurements such as volume estimation, velocity, and distance travelled before it becomes spherical.
Firstly, I would like to know whether I am along the right tracks in tackling this project. At the moment I have:
1.) Converted the original image to grayscale
2.) Applied Gaussian Blur then Canny edge detection (Click here for image)
3.) Located the white pixels using findNonZero() then calculated the sum of blocks of rows and the block with the highest concentration of white pixels and all the rows above it are cropped out). This removes the print heads in the image so the ROI is only the droplets below it.
4.) Used the findContours to find the contours. (Click here for image)
The above 4 steps are what I have done so far. Are the following steps below what I should do next?
Dilate the binary image first after cropping and before finding contours to ensure the contours will be closed and not open?
Maybe ignore the ones that are very open? (Any tips on how to actually do this?)
floodFill() every closed circles
Find the each contours' area using contourArea() (Can I then estimate the volume of the drop after this step with a few assumptions like its shape, pixel to volume ratio, etc?)
Find the centre of each contour and save it to an array so I can compare it to the centre of the same drop in the next frame. Once I know distance travelled of the centre of the droplet and the frame rate of the video I should be able to estimate velocity.
I am also unsure of how I can give a drop an ID so I can be sure I am tracking it properly and know when a new drop has entered the ROI.
Any help would be greatly appreciated, Thank You.
I think that your idea is good and can be quite easily extended to something that will satisfy you.
For clarification i will call red ROI from your image "redROI".
Find all droplets in redROI. Remember positions and IDs.
For each droplet position from previous step create a ROI similar to yellow rectangle:
For each rectangle check whether there is a droplet inside it.
If yes - probably it's the droplet from the previous frame, so the one you are looking for.
If no - you may try to search again in a little bit bigger rectangle or assume that the darkset point of this ROI is you droplet. If ROI is near bottom of redROI probably the droplet is gone - forget about it.
Note few things:
-size of the rectangle depends on how fast droplets move and whether they can move only vertical or diagonal (wind can change direction of move) too.
-before searching for droplets, check whether all rectangles are disjoint (the don't have any common area -> (Rect1 & Rect2).area() == 0 for each pair of rectangles).
-before searching for droplets in ROI make sure this ROI is inside redROI. So just use this code: roi = roi & redROI;
After finding new positions of every old droplet, search for droplets in whole redROI, so you won't miss any new droplet.
Let me know if you don't understand some part of this idea - i will try to explain it better.
Maybe ignore the ones that are very open? (Any tips on how to actually
do this?)
I'm not sure about it, so check it. Try to use CV_RETR_LIST as the third parameter of findContours and check the distance between first and the last point from returned (by findContours) contour - if the distance is big than the contour is open, if no - it is closed.
floodFill() every closed circles
You can just use drawContours and set thickness parameter to -1 - simpler and faster solution.
edit:
You can try to use optical flow as well - it's already implemented in openCV, here you can read nice tutorial about that: http://robotics.stanford.edu/~dstavens/cs223b/ (start from .pdf files)
I am familiar with openCV, a powerful open source library and using that I am dealing with farm industry project where a mouse will be injected with drug , and its been kept on so called a stage which is surrounded by cylinder with painted strips of successive white and black. So i need to find out how many times the mouse will rotate its head to words the rotation of the cylinder . (its because it has got hang of drug) . How can i achieve this any opencv experts can help me out there.
I have added an image below
Seems an interesting one, these are my preliminary suggestions...
Depends on the resolution of the camera and how far your object (mouse) is from the camera...coz mouse is a small object so the image of the mouse need to cover good number of pixels in the image to differentiate head movement...
I don't think the mouse will stick to one position..it will keep moving in the cage...so you need to track the mouse...
At every position of the mouse you need to find the position of the head with respect to the body....that you can do using template matching (create templates of the head of the mouse)
Hence more info and some sample pictures are necessary to get the clear idea of the scene
EDIT AFTER IMAGE UPLOADED
since the camera is fixed hence create a circular region of interest...so that only movement inside this circle concerns you and not the moving cylinder outside the circle
subtract the present frame from the previous frame (frame differentiation) and store the absolute of the difference in an image.
absdiff(frameNow,framePrevs,diffofFrames);
threshold the diffofFrames as required to get the current position of the rat...
Now the task is easier if the image clearly shows its nose...since the nose has a pointed shape it can be detected by some template matching....however from the image you have given its difficult to make out the nose against a black background...However I can only suggest you the following process... green circles denote the tip of the nose...all I am trying to do is to get orientation of the head w.r.t. the body....for good results you need to have good images...
I am working on a simple mesh viewer implementation in C++ with basic functionality such as translation, rotation, scaling.
I'm stuck with with implementing the rotation of the object along z-axis using the mouse. What I want to implement is the following:
Click and drag the mouse vertically (almost vertical will do, as I use a simple threshold to filter slight deviations along the horizontal axis) to rotate the object along y-axis (this part is done).
Click and drag the mouse horizontally just as described above to rotate the object along x-axis (this part is done too).
For z-axis rotation, I want to detect a circular (or along an arc) mouse movement. I'm stuck with this part, and don't know how to implement this.
For the above two, i just use atan2() to determine the angle of movement. But how do I detect circular movements?
The only way to deal with this is to have a delay between the user starting to make the motion and the object rotating:
When user clicks and begins to move the mouse you need to determine if its going to become a straight line movement, or a circular one. This will require a certain amount of data to be collected before that judgement can be made.
The most extreme case would be requiring the user to make one complete circle first, then the rotation begins (in reality you could do much better than this). Just how small you are able to cut this period down to will depend on a) how precise you dictate your users actions must be, and b) how good you are with pattern recognition algorithms.
To get you started heres an outline of an extremely poor algorithm:
On user click store the x and y coordinates.
Every 1/10 of a second store the new coordinates and process_for_pattern.
in process_for_pattern you're looking for:
A period where the x coordinates and the y coordinates regularly both increase, both decrease, or one increases and one decreases. Over time if this pattern changes such that either the x or the y begins to reverse whilst the other continues as it was, then at that moment you can be fairly sure you've got a circle.
This algorithm would require the user to draw a quarter circle before it was detected, and it does not account for size, direction, or largely irregular movements.
If you really want to continue with this method you can get a much better algorithm, but you might want to reconsider your control method.
Perhaps, you should define a screen region (e.g. at window boundaries), which, when was clicked, will initiate arc movement - or use some other modifier, a button or whatever.
Then at a mouse click you capture the coordinates and center of rotation (mesh axis) in 2D screen space. This gets you a vector (mesh center, button down pos)
On every mouse move you calculate a new vector (mesh center, mouse pos) and the angle between the two vectors is the angle of rotation.
I don't think it works like that...
You could convert mouse wheel rotation to z-axis, or use quaternion camera orientation, which is able to rotate along every axis almost intuitively...
The opposite is true for quarternion camera: if one tries to rotate the mesh along a straight line, the mesh appears to rotate slightly around some other weird axis -- and to compensate that, one intuitively tries to follow some slightly curved trajectory.
It's not exactly what you want, but should come close enough.
Choose a circular region within which your movements numbered 1 and 2 work as described (in the picture this would be some region that is smaller than the red circle. However, when the user clicks outside the circular region, you save the initial click position (shown in green). This defines a point which has a certain angle relative to the x-axis of your screen (you can find this easily with some trig), and it also defines the radius of the circle on which the user is working (in red). The release of the mouse adds a second point (blue). You then find the angle this point has relative to the center of the screen and the x-axis (just like before). You then project that angle onto your circle with the radius determined by the first click. The dark red arc defines the amount of rotation of the model.
This should be enough to get you started.
That will not be a good input method, I think. Because you will always need some travel distance to discriminate between a line and a curve, which means some input delay. Here is an alternative:
Only vertical mouse having their line crossing the center of the screen are considered vertical. Same for horizontal. In other cases it's considered a rotation, and to calculate its amplitude, calculate the angle between the last mouse location and the current location relatively to the center of the screen.
Alternatively you could use the center of the selected mesh if your application works like that.
You can't detect the "circular, along an arc" mouse movement with anywhere near the precision needed for 3d model viewing. What you want is something like this: http://thetechartist.com/?p=80
You nominate an axis (x, y, or z) using either keyboard shortcuts or on-screen axis indicators that you can grab with the mouse.
This will be much more precise than trying to detect an "arc" gesture. Any "arc" recognition would necessarily involve a delay while you accumulate enough mouse samples to decide whether an arc gesture has begun or not. Gesture recognition like this is non-trivial (I've done some gesture work with the Wii-mote). Similarly, even your simple "vertical" and "horizontal" mouse movement detection will require a delay for the same reason. Any "simple threshold to filter slight deviations" will make it feel dampened and weird.
For 3d viewing you want 1:1 mouse responsiveness, and that means just explicitly nominating an axis with a shortcut key or UI etc. For x-axis rotation, just restrict it to mouse x, y-axis to mouse y if you like. For z you could similarly restrict to x or y mouse input, or just take the total 2d mouse distance travelled. It depends what feels nicest to you.
As an alternative, you could try coding up support for a 3D mouse like the 3dConnexion SpaceExplorer.
I'm trying to build a program using opencv library.
I intend to make a laser pointer mouse.
so far, the program can detect laser point and move the cursor location that location in realtime.
now i want to give the program an ability to perform click and if possible a double click.
the only idea I have, is to do this by playing with the coordinate value for certain frame and subtract current frame coordinate with last frame coordinate.
my problem is...I dont know how implement it in code
should I use array to store the coordinate?? or any other solution i could use??
thanks in advance..
A click could be represented by the laser pointer disappearing and appearing near the same spot, and only if this happens within 1 second.
You could store the coordinates of the last frames in a std::vector of CvPoint and do a simple search in this vector when the laser pointer appears again. The last 30 coordinates or so should be stored, so you will always have the coordinates of that last 1 second of recording (at 30fps).
The double click is a small enhancement of the single click. For simplicity purposes, the double click could seen as 2 single clicks being detected within 2 seconds.