I'm working on a blob matching and tracking library in C++. Currently I'm using OpenCV to detect blobs and try to match blobs in a new frame by checking the position, velocity and size of the blob. This works quite okay and I'm receiving a high blob match rate (95% or higher).
Sometimes blobs fall out of the image or new blobs appear. Now I need to give matched blobs the same ID as they had before. I'm wondering if there are typical or commonly used techniques for doing this. Or even some keywords I can use to Google on.
Thanks
http://en.wikipedia.org/wiki/Blob_extraction
I assume you have your blobs in a binary image, simply floodfill each blob with a different color/id, and register overlapping blobs between frames with the same id.
CCV is used for multiple finger tracking for multi-touch environments. Check out their tracking code. It uses a function trackKnn which uses k nearest neighbour algorithm.
You can also use Kalman Filter if blobs collide each other. Check out this SO
Related
I have an application where I have to detect the presence of some items in a scene. The items can be rotated and a little scaled (bigger or smaller). I've tried using keypoint detectors but they're not fast and accurate enough. So I've decided to first detect edges in the template and the search area, using Canny ( or a faster edge detection algo ), and then match the edges to find the position, orientation, and size of the match found.
All this needs to be done in less than a second.
I've tried using matchTemplate(), and matchShape() but the former is NOT scale and rotation invariant, and the latter doesn't work well with the actual images. Rotating the template image in order to match is also time consuming.
So far I have been able to detect the edges of the template but I don't know how to match them with the scene.
I've already gone through the following but wasn't able to get them to work (they're either using old version of OpenCV, or just not working with other images apart from those in the demo):
https://www.codeproject.com/Articles/99457/Edge-Based-Template-Matching
Angle and Scale Invariant template matching using OpenCV
https://answers.opencv.org/question/69738/object-detection-kinect-depth-images/
Can someone please suggest me an approach for this? Or a code snipped for the same if possible ?
This is my sample input image ( the parts to detect are marked in red )
These are some software that are doing this and also how I want it should be:
This topic is what I am actually dealing for a year on a project. So I will try to explain what my approach is and how I am doing that. I assume that you already did the preprocess steps(filters,brightness,exposure,calibration etc). And be sure you clean the noises on image.
Note: In my approach, I am collecting data from contours on a reference image which is my desired object. Then I am comparing these data with the other contours on the big image.
Use canny edge detection and find the contours on reference
image. You need to be sure here about that it shouldn't miss some parts of
contours. If it misses, probably preprocess part should have some
problems. The other important point is that you need to find an
appropriate mode of findContours because every modes have
different properties so you need to find an appropriate one for your
case. At the end you need to eliminate the contours which are okey
for you.
After getting contours from reference, you can find the length of
every contours using outputArray of findContours(). You can compare
these values on your big image and eliminate the contours which are
so different.
minAreaRect precisely draws a fitted, enclosing rectangle for
each contour. In my case, this function is very good to use. I am
getting 2 parameters using this function:
a) Calculate the short and long edge of fitted rectangle and compare the
values with the other contours on the big image.
b) Calculate the percentage of blackness or whiteness(if your image is
grayscale, get a percentage how many pixel close to white or black) and
compare at the end.
matchShape can be applied at the end to the rest of contours or you can also apply to all contours(I suggest first approach). Each contour is just an array so you can hold the reference contours in an array and compare them with the others at the end. After doing 3 steps and then applying matchShape is very good on my side.
I think matchTemplate is not good to use directly. I am drawing every contour to a different mat zero image(blank black surface) as a template image and then I compare with the others. Using a reference template image directly doesnt give good results.
OpenCV have some good algorithms about finding circles,convexity etc. If your situations are related with them, you can also use them as a step.
At the end, you just get the all data,values, and you can make a table in your mind. The rest is kind of statistical analysis.
Note: I think the most important part is preprocess part. So be sure about that you have a clean almost noiseless image and reference.
Note: Training can be a good solution for your case if you just want to know the objects exist or not. But if you are trying to do something for an industrial application, this is totally wrong way. I tried YOLO and haarcascade training algorithms several times and also trained some objects with them. The experiences which I get is that: they can find objects almost correctly but the center coordinates, rotation results etc. will not be totally correct even if your calibration is correct. On the other hand, training time and collecting data is painful.
You have rather bad image quality very bad light conditions, so you have only two ways:
1. To use filters -> binary threshold -> find_contours -> matchShape. But this very unstable algorithm for your object type and image quality. You will get a lot of wrong contours and its hard to filter them.
2. Haarcascades -> cut bounding box -> check the shape inside
All "special points/edge matching " algorithms will not work in such bad conditions.
I'm currently working a computer vision application with OpenCV. The application involves target identification and characteristic determination. Generally, I'm going to have a target cross into the visible region and slowly move through it in a couple of seconds. This should give me upwards of 50-60 frames from the camera in which I'll be able to find the target.
We have successfully implemented the detection algorithms using SWT and OCR (the targets all have alphanumeric identifiers, which makes them relatively easy to pick out). What I want to do is use as much of the data as possible from all 50-60 shots of each target. To do this, I need some way to identify that a particular ROI of image 2 contains the same target as another ROI from image 1.
What I'm asking for a little advice from anyone who may have come across this before. How can I easily/quickly identify, within a reasonable error margin, that ROI #2 has the same target as ROI#1? My first instinct is something like this:
Detect targets in frame 1.
Calculate certain unique features of each of the targets in frame 1. Save.
Get frame 2.
Immediately look for ROIs which have the same features as those calc'd in step 2. Grab these and send them down the line for further processing, skipping step 5.
Detect new targets in frame 2.
Pass targets to a thread to calculate shape, color, GPS coordinates, etc.
Lather, rinse, repeat.
I'm thinking that SURF or SIFT features might be a way to accomplish this, but I'm concerned that they might have trouble identifying targets as the same from frame to frame due to distortion or color fade. I don't know how to set a threshold on SIFT/SURF features.
Thank you in advance for any light you can shed on this matter.
One thing you can do is locally equalize brightness and possibly saturation levels. If you aren't using an advanced space such as YCrCb or HSV, I suggest you try them.
Can you assume that the object is not moving too fast? If you feed the previous position in the detection routine, you can decrease the size of the window you are looking at. Same thing goes with the speed, and direction of movement.
I've successfully used histogram composition and shape descriptors of a region in order to reliably detect it, you can use that or add it to a SURF/SIFT classifier.
Bascially I want to detect an object and than track it in a video (frame-by-frame).
I can detect it on the first frame with for example ORB or SIFT. But for the next frames (or say next XX frames) I would like to avoid to calulcate again all the keypoints (ORB or SIFT) to detect it again.
Considering I want to track it in a video real time, what could I do ?
A common option is using a patchtracker. That means that you just search for keypoints in an area of, for example, 8 pixels around the previous frame keypoint. You can perform cv::matchTemplate() of an area surrounding the keypoint, instead of using SIFT.
Performing a pyramidal search helps to improve frame-rate. You first search at a lower scale, if you cannot find the keypoint you double the scale.
If patchtracker fails, because the image moves too fast, you just have to reinitialize the system by applying SIFT again. I would use FAST instead of SIFT. You can use SIFT for the marker, and then FAST for detecting keypoints real-time, generating SIFT descriptors.
Detecting and tracking object in a video is a very large topic and the way to go highly depends on your application. There is no magic bullet!
If you achieve the detection part, you can try tracking by meanshift on color (maybe HSV color space) likelihood if the object you need to track is colored .. , or try template matching, or .. You need to be more specific on your needs.
you can use OpticalFlow for simple tracking, here are the steps to do it...
Find the corners of a moving object using harris corner detector or SIFT feature detector.
Give those corners and previous image(in which you found the corners of object to be tracked) and the next image to opticalflow function it will compute the corners of the same object in the next images..
Here are the links:
Link1
Link2
code
NOTE: if you are addressing problems like occlusion handling , multiple people tracking then OpticalFlow alone can't solve problems. For that kalman filter or particle filters are needed to be employed...
You can achieve almost perfect and real time tracking using TLD or CLM. Once you detect the object of interest use that bounding box to initiate predator tracking.
You can find about CMT here
https://www.gnebehay.com/cmt/
and TLD here
https://www.gnebehay.com/tld/
I am new to OpenCV. I would like to know if we can compare two images (one of the images made by photoshop i.e source image and the otherone will be taken from the camera) and find if they are same or not.
I tried to compare the images using template matching. It does not work. Can you tell me what are the other procedures which we can use for this kind of comparison?
Comparison of images can be done in different ways depending on which purpose you have in mind:
if you just want to compare whether two images are approximately equal (with a few
luminance differences), but with the same perspective and camera view, you can simply
compute a pixel-to-pixel squared difference, per color band. If the sum of squares over
the two images is smaller than a threshold the images match, otherwise not.
If one image is a black-white variant of the other, conversion of the color images is
needed (see e.g. http://www.johndcook.com/blog/2009/08/24/algorithms-convert-color-grayscale). Afterwarts simply perform the step above.
If one image is a subimage of the other, you need to perform registration of the two
images. This means determining the scale, possible rotation and XY-translation that is
necessary to lay the subimage on the larger image (for methods to register images, see:
Pluim, J.P.W., Maintz, J.B.A., Viergever, M.A. , Mutual-information-based registration of
medical images: a survey, IEEE Transactions on Medical Imaging, 2003, Volume 22, Issue 8,
pp. 986 – 1004)
If you have perspective differences, you need an algorithm for deskewing one image to
match the other as well as possible. For ways of doing deskewing look for example in
http://javaanpr.sourceforge.net/anpr.pdf from page 15 and onwards.
Good luck!
You should try SIFT. You apply SIFT to your marker (image saved in memory) and you get some descriptors (points robust to be recognized). Then you can use FAST algorithm with the camera frames in order to find the coprrespondent keypoints of the marker in the camera image.
You have many threads about this topic:
How to get a rectangle around the target object using the features extracted by SIFT in OpenCV
How to search the image for an object with SIFT and OpenCV?
OpenCV - Object matching using SURF descriptors and BruteForceMatcher
Good luck
I'm looking for a fast way to compare a frame with a running average, and determine the difference between them (in terms of giving a high value if they're very similar, and a lower value if they're not that similar). I need to compare the entire frame, not just a smaller region.
I'm already using Otsu thresholding on the images to filter out the background (not interested in the background, nor the features of the foreground - just need shapes). Is there a nice, fast way to do what I want?
The classic method for this is Normalized Cross Correlation (try cv::matchTemplate()). You will need to set a treshold to decided if images are a match. Also you can use the output (which is thresholded) to compare several images.
In OpenCV, this method in matchTemplate is explained here, and the parameter you need to pass to the function.