I try to use motion detector to detect shooting star in the video with the example of code UseMotionDetector.cpp.
If I use the motion detector with default options than nothing works.
I think that may be connected with small object size, its fast speed, or big noise.
The motions detector has a big amount of parameters (and also this) but I have no experience in using of any motion detector.
So I have some questions:
Is this task can be resolved with using of algorithms of motion detection?
Is this motion detector suit for this job?
How to tune its paraameters?
Thank in advance!
I have analysed your video and there are some troubles which don allow to work Simd::Motion::Detector properly with default settings.
And you have already listed most of them above:
The object (shooting star) has small size.
It moves too fast.
Time of its existance is short.
There is a big noise on the video.
In order to solve these troubles I changed following parameters of motion detector:
To detect objects of small size I decreased minimal object size in model:
Model model;
model.size = FSize(0.01, 0.01); // By default it is equal to FSize(0.1, 0.1).
detector.SetModel(model);
To reduce influence of fast motion:
Options options;
options.TrackingAdditionalLinking = 5; // Boosts binding of trajectory.
To resolve trouble of short object existence time:
options.ClassificationShiftMin = 0.01; // Decreases minimal shift of object to be detected.
options.ClassificationTimeMin = 0.01; // Decreases minimal life time of object to be detected.
To reduce big noise:
options.DifferenceDxFeatureWeight = 0; // Turns off gradient along X axis feature.
options.DifferenceDyFeatureWeight = 0; // Turns off gradient along Y axis feature.
detector.SetOptions(options);
And it works! I hope that I helped you.
Related
I am using the OpenCV C++ implementation of the Kalman Filter with the constant acceleration model in x and y direction. The filter works pretty well up until around 1200 frames and then it start giving very extreme predictions.
Following graph shows the the prediction vs the measurement for both x and y coordinates and you can clearly see where it goes wrong.
And this is the code that runs every frame:
TrackingResult KalmanFilter::Process(TrackingInput& input)
{
cv::Mat prediction = _kalmanFilter.predict();
Point2 predictPt(prediction.at<float>(0), prediction.at<float>(1));
cv::Mat_<float> measurement(2, 1, CV_32F);
if (input.DetectionResult.FoundBall)
{
measurement(0) = input.DetectionResult.BallPosition.x;
measurement(1) = input.DetectionResult.BallPosition.y;
cv::Mat estimated = _kalmanFilter.correct(measurement);
_ballPosition = Point2(estimated.at<float>(0), estimated.at<float>(1));
return TrackingResult(_ballPosition, predictPt, input);
}
else
{
return TrackingResult(predictPt, predictPt, input);
}
}
The code is inspired by OpenCV's official Kalman Filter example here. I initialized the transition matrix correctly by assuming the constant acceleration model. More info on this model can be found here
I tried passing constant values as measurements and it stayed very stable so I'm not entirely sure what the issue here really is. The issue can not be with the video too because I am using the camera to test it.
I can not create an MCVE because I'd have to give 1200 frame positions to reproduce it but you can visualize the values I pass in the graphs above.
Any help will be highly appreciated.
I have two sets of points, one from time t-1 and current time t. The first set was generated using goodFeaturesToTrack, and the latter from using calcOpticalFlowPyrLK(). Using these two sets of points, I then estimate a transformation matrix via estimateAffine2DPartial() in order to keep track of its scale & rotation. Code snippet is listed below:
// Precompute image pyramids
maxLvl = cv::buildOpticalFlowPyramid(_imgPrev, imPyr1, _winSize, maxLvl, true);
maxLvl = cv::buildOpticalFlowPyramid(tmpImg, imPyr2, _winSize, maxLvl, true);
// Optical flow call for tracking pixels
cv::calcOpticalFlowPyrLK(imPyr1, imPyr2, _currentPoints, nextPts, status, err, _winSize, maxLvl, _terminationCriteria, 0, 0.000001);
// Get transformation matrix between the two data sets
cv::Mat H = cv::estimateAffinePartial2D(_currentPoints, nextPts, inlier_mask, cv::RANSAC, 10.0, 2000, 0.99);
Using H, I then map my masking points using perspectiveTransform(). The result seems accurate for the first few dozen frames until I notice some drift (in terms of rotation) occurring when the object I am tracking continues to rotate (usually when rotation becomes > M_PI). I'm honestly stumped on where the culprit is, but my main suspicion is perhaps my window size for optical flow might be too small, or too big. However, tweaking the window size did not seem to help, the position of my object is still accurate, but the estimated rotation (and scale) got worse. Can anyone hope to shed a light on this?
Warm regards and thanks.
EDIT: Images attached to show drift issue
Starting Frame
First few frames -- Rotation OK
Z-Rotation Drift occurs -- see anchor line has drifted towards the red rectangle.
Lucas Kanade tracker needs more features. Guess the tracking template you provided is not good enough.
(1) Try with other feature rich real images? e.g Opencv feautre tracking template image
(2) fix scale. Since you are doing simulation, you can try to anchor the size first.
calcOpticalFlowPyrLK is widely used in visual inertial state estimation studies. such as Semi direct visual odometry or VINSMONO. You can try to find the code inside those project to see how other people is playing with the feature and parameters
I'm using OpenCV 3.3.1. I want to do a semi-dense optical flow operation using cv::calcOpticalFlowPyrLK, but I've been getting some really noticeable slowdown whenever my ROI is pretty big (Partly due to the fact that I am letting the user decide what the winSize should be, ranging from from 10 to 100). Anyways, it seems like cv::buildOpticalFlowPyramid can mitigate the slowdown by building image pyramids? I'm sorta familiar what image pyramids are, but in context of the function, I'm especially confused about what parameters I pass in, and how it impacts my function call to cv::calcOpticalFlowPyrLK. With that in mind, I now have these set of questions:
The output is, according to the documentation, is an OutputArrayOfArrays, which I take it can be a vector of cv::Mat objects. If so, what do I pass in to cv::calcOpticalFlowPyrLK for prevImg and nextImg (assuming that I need to make image pyramids for both)?
According to the docs for cv::buildOpticalFlowPyramid, you need to pass in a winSize parameter in order to calculate required padding for pyramid levels. If so, do you pass in the same winSize value when you eventually call cv::calcOpticalFlowPyrLK?
What exactly are the arguments for pyrBorder and derivBorder doing?
Lastly, and apologies if it sounds newbish, but what is the purpose of this function? I always assumed that cv::calcOpticalFlowPyrLK internally builds the image pyramids. Is it just to speed up the optical flow operation?
I hope my questions were clear, I'm still very new to OpenCV, and computer vision, but this topic is very interesting.
Thank you for your time.
EDIT:
I used the function to see if my guess was correct, so far it has worked, but I've seen no noticeable speed up. Below is how I used it:
// Building pyramids
int maxLvl = 3;
maxLvl = cv::buildOpticalFlowPyramid(imgPrev, imPyr1, cv::Size(searchSize, searchSize), maxLvl, true);
maxLvl = cv::buildOpticalFlowPyramid(tmpImg, imPyr2, cv::Size(searchSize, searchSize), maxLvl, true);
// LK optical flow call
cv::calcOpticalFlowPyrLK(imPyr1, imPyr2, currentPoints, nextPts, status, err,
cv::Size(searchSize, searchSize), maxLvl, termCrit, 0, 0.00001);
So now I'm wondering what's the purpose of preparing the image pyramids if calcOpticalFlowPyrLK does it internally?
So the point of your question is that you are trying to improve speed of optical flow tracking by tuning your input parameters.
If you want dirty and quick answer then here it is
KTL (OpenCV's calcOpticalFlowPyrLK) define a e residual function which are sum of gradient of point inside search window .
The main purpose is to find vector of point that can minimize residual function
So if you increase search window size (winSize) then it is more difficult to find that set of points.
If your really really want to do that then please read the official paper.
See the section 2.4
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.185.585&rep=rep1&type=pdf
I took it from official document
https://docs.opencv.org/2.4/modules/video/doc/motion_analysis_and_object_tracking.html#bouguet00
Hope that help
So, I'm taking over the work on an ortho-rectification algorithm that is intended to produce "accurate" results. I'm running into trouble trying to increase the accuracy and could use a little help.
Here is the basic approach.
Extract a calibration pattern from an image that was taken from a mobile phone.
Rectify the image based on a calibration pattern in the image
Scale the image to get the real world size of the scene around the pattern.
The calibration pattern is held against a flat surface, like a wall, counter, table, floor and the user takes a picture. With that picture, we want to measure artifacts on the same surface as the calibration pattern. We have tried this with calibration patterns ranging from the size of a credit card to a sheet of paper (8.5" x 11")
Here is an example input picture
With this resulting output image
Right now our measurements are usually within 1-2% of what we expect. This is sufficient for small areas (less than 25cm away from the calibration pattern. However, we'd like the algorithm to scale so that we can accurately measure a 2x2 meter area. However, at that size, the current error is too much (2-4 cm).
Here is the algorithm we are following.
// convert original image to grayscale and perform morphological dilation to reduce false matches when finding circle grid
Mat imgGray;
cvtColor(imgOriginal, imgGray, CV_BGR2GRAY);
// find calibration pattern in original image
Size patternSize(4, 11);
vector <Point2f> circleCenters_OriginalImage;
if (!findCirclesGrid(imgGray, patternSize, circleCenters_OriginalImage, CALIB_CB_ASYMMETRIC_GRID))
{
return false;
}
Point2f inputQuad[4];
inputQuad[0] = Point2f(circleCenters_OriginalImage[0].x, circleCenters_OriginalImage[0].y);
inputQuad[1] = Point2f(circleCenters_OriginalImage[3].x, circleCenters_OriginalImage[3].y);
inputQuad[2] = Point2f(circleCenters_OriginalImage[43].x, circleCenters_OriginalImage[43].y);
inputQuad[3] = Point2f(circleCenters_OriginalImage[40].x, circleCenters_OriginalImage[40].y);
// create model points for calibration pattern
vector <Point2f> circleCenters_ObjectSpace = GeneratePatternPointsInObjectSpace(circleCenters_OriginalImage[0], Distance(circleCenters_OriginalImage[0], circleCenters_OriginalImage[1]) / 2.0f, ioData.marker_up);
Point2f outputQuad[4];
outputQuad[0] = Point2f(circleCenters_ObjectSpace[0].x, circleCenters_ObjectSpace[0].y);
outputQuad[1] = Point2f(circleCenters_ObjectSpace[3].x, circleCenters_ObjectSpace[3].y);
outputQuad[2] = Point2f(circleCenters_ObjectSpace[43].x, circleCenters_ObjectSpace[43].y);
outputQuad[3] = Point2f(circleCenters_ObjectSpace[40].x, circleCenters_ObjectSpace[40].y);
Mat lambda(2,4,CV_32FC1);
lambda = Mat::zeros(imgOriginal.rows, imgOriginal.cols, imgOriginal.type());
lambda = getPerspectiveTransform(inputQuad, outputQuad);
warpPerspective(imgOriginal, imgOrthorectified, lambda, imgOrthorectified.size());
...
My Questions:
Is it reasonable to shoot for error < 0.25%? Is there a different algorithm that would yield more accurate results? What are the most valuable sources of error to identify and resolve?
As I've worked on this, I've also looked at removing pincushion / barrel distortions, and trying homographies to find the perspective transform. The best approaches I have found so far remain in the 1-2% error.
Any suggestions of where to go next would be really helpful
I am using BackgroundSubtractorMOG in OpenCV to track objects. When they appear, it works fine but the background fastly adapts so I cannot track static objects. How can I make the background adaptation slower (I dont want it fully static, just slower)?
Setting the learning rate using the constructor doesn't change that:
BackgroundSubtractorMOG pBSMOG = BackgroundSubtractorMOG(???);
How can I solve this? Thanks!
BackgroundSubtractorMOG pBSMOG = BackgroundSubtractorMOG(int history=200, int nmixtures=5, double backgroundRatio=0.7, double noiseSigma=0);
Where,
history – Length of the history.
nmixtures – Number of Gaussian mixtures.
backgroundRatio – Background ratio.
noiseSigma – Noise strength (standard deviation of the brightness or each color channel). 0 means some automatic value.
Increasing the history value will slow down the adaptation rate.
There is another function available in OpenCV:
Ptr <BackgroundSubtractorMOG2> createBackgroundSubtractorMOG2(int
history=500, double varThreshold=16, bool detectShadows=true )
This is much faster than the previous one and it can eleminate detecting shadows too.