I am using BackgroundSubtractorMOG in OpenCV to track objects. When they appear, it works fine but the background fastly adapts so I cannot track static objects. How can I make the background adaptation slower (I dont want it fully static, just slower)?
Setting the learning rate using the constructor doesn't change that:
BackgroundSubtractorMOG pBSMOG = BackgroundSubtractorMOG(???);
How can I solve this? Thanks!
BackgroundSubtractorMOG pBSMOG = BackgroundSubtractorMOG(int history=200, int nmixtures=5, double backgroundRatio=0.7, double noiseSigma=0);
Where,
history – Length of the history.
nmixtures – Number of Gaussian mixtures.
backgroundRatio – Background ratio.
noiseSigma – Noise strength (standard deviation of the brightness or each color channel). 0 means some automatic value.
Increasing the history value will slow down the adaptation rate.
There is another function available in OpenCV:
Ptr <BackgroundSubtractorMOG2> createBackgroundSubtractorMOG2(int
history=500, double varThreshold=16, bool detectShadows=true )
This is much faster than the previous one and it can eleminate detecting shadows too.
Related
I try to use motion detector to detect shooting star in the video with the example of code UseMotionDetector.cpp.
If I use the motion detector with default options than nothing works.
I think that may be connected with small object size, its fast speed, or big noise.
The motions detector has a big amount of parameters (and also this) but I have no experience in using of any motion detector.
So I have some questions:
Is this task can be resolved with using of algorithms of motion detection?
Is this motion detector suit for this job?
How to tune its paraameters?
Thank in advance!
I have analysed your video and there are some troubles which don allow to work Simd::Motion::Detector properly with default settings.
And you have already listed most of them above:
The object (shooting star) has small size.
It moves too fast.
Time of its existance is short.
There is a big noise on the video.
In order to solve these troubles I changed following parameters of motion detector:
To detect objects of small size I decreased minimal object size in model:
Model model;
model.size = FSize(0.01, 0.01); // By default it is equal to FSize(0.1, 0.1).
detector.SetModel(model);
To reduce influence of fast motion:
Options options;
options.TrackingAdditionalLinking = 5; // Boosts binding of trajectory.
To resolve trouble of short object existence time:
options.ClassificationShiftMin = 0.01; // Decreases minimal shift of object to be detected.
options.ClassificationTimeMin = 0.01; // Decreases minimal life time of object to be detected.
To reduce big noise:
options.DifferenceDxFeatureWeight = 0; // Turns off gradient along X axis feature.
options.DifferenceDyFeatureWeight = 0; // Turns off gradient along Y axis feature.
detector.SetOptions(options);
And it works! I hope that I helped you.
I have two sets of points, one from time t-1 and current time t. The first set was generated using goodFeaturesToTrack, and the latter from using calcOpticalFlowPyrLK(). Using these two sets of points, I then estimate a transformation matrix via estimateAffine2DPartial() in order to keep track of its scale & rotation. Code snippet is listed below:
// Precompute image pyramids
maxLvl = cv::buildOpticalFlowPyramid(_imgPrev, imPyr1, _winSize, maxLvl, true);
maxLvl = cv::buildOpticalFlowPyramid(tmpImg, imPyr2, _winSize, maxLvl, true);
// Optical flow call for tracking pixels
cv::calcOpticalFlowPyrLK(imPyr1, imPyr2, _currentPoints, nextPts, status, err, _winSize, maxLvl, _terminationCriteria, 0, 0.000001);
// Get transformation matrix between the two data sets
cv::Mat H = cv::estimateAffinePartial2D(_currentPoints, nextPts, inlier_mask, cv::RANSAC, 10.0, 2000, 0.99);
Using H, I then map my masking points using perspectiveTransform(). The result seems accurate for the first few dozen frames until I notice some drift (in terms of rotation) occurring when the object I am tracking continues to rotate (usually when rotation becomes > M_PI). I'm honestly stumped on where the culprit is, but my main suspicion is perhaps my window size for optical flow might be too small, or too big. However, tweaking the window size did not seem to help, the position of my object is still accurate, but the estimated rotation (and scale) got worse. Can anyone hope to shed a light on this?
Warm regards and thanks.
EDIT: Images attached to show drift issue
Starting Frame
First few frames -- Rotation OK
Z-Rotation Drift occurs -- see anchor line has drifted towards the red rectangle.
Lucas Kanade tracker needs more features. Guess the tracking template you provided is not good enough.
(1) Try with other feature rich real images? e.g Opencv feautre tracking template image
(2) fix scale. Since you are doing simulation, you can try to anchor the size first.
calcOpticalFlowPyrLK is widely used in visual inertial state estimation studies. such as Semi direct visual odometry or VINSMONO. You can try to find the code inside those project to see how other people is playing with the feature and parameters
I'm trying to make a Gtk::Image widget display a picture from a file, but prevent the widget from expanding in size, so I'm loading it from a Gdk::Pixbuf and then scaling the picture. I'm using Gdk::Pixbuf instead of GdkPixBuf because the latter one works on regular pointers, but Gtk::Image requires a Glib::RefPtr<Gdk::Pixbuf>. (Just mentioning all this in case there's a better way to achieve what I'm doing that I'm unaware of.)
auto pixbuf = Gdk::Pixbuf::create_from_file("/home/raitis/Music/WRLD/Awake EP/cover.jpg");
auto scaled = pixbuf->scale_simple(48, 48, Gdk::InterpType::NEAREST);
image->set(scaled);
Anyway, problem is that although I'm following the documentation for Gdk::Pixbuf, line 2 in my code generate the error:
error: ‘NEAREST’ is not a member of ‘Gdk::InterpType’
auto scaled = pixbuf->scale_simple(48, 48, Gdk::InterpType::NEAREST);
^~~~~~~
Trying GDK_INTERP_NEAREST instead also leads to an error. :(
no known conversion for argument 3 from ‘GdkInterpType’ to ‘Gdk::InterpType’
From the stable gtkmm gdkmm documentation, Gdk::InterpType members are:
INTERP_NEAREST
Nearest neighbor sampling; this is the fastest and lowest quality
mode. Quality is normally unacceptable when scaling down, but may be OK when
scaling up.
INTERP_TILES
This is an accurate simulation of the PostScript image operator
without any interpolation enabled.
Each pixel is rendered as a tiny parallelogram of solid color, the
edges of which are implemented with antialiasing. It resembles nearest
neighbor for enlargement, and bilinear for reduction.
INTERP_BILINEAR
Best quality/speed balance; use this mode by default.
Bilinear interpolation. For enlargement, it is equivalent to
point-sampling the ideal bilinear-interpolated image. For reduction,
it is equivalent to laying down small tiles and integrating over the
coverage area.
INTERP_HYPER
This is the slowest and highest quality reconstruction function.
It is derived from the hyperbolic filters in Wolberg's "Digital Image
Warping", and is formally defined as the hyperbolic-filter sampling
the ideal hyperbolic-filter interpolated image (the filter is designed
to be idempotent for 1:1 pixel mapping).
And from the documentation of the Gdk::Pixbuf, in the scale_simple method you'll find a reference to the interpolation type:
Leaves src unaffected. interp_type should be Gdk::INTERP_NEAREST if
you want maximum speed (but when scaling down Gdk::INTERP_NEAREST is
usually unusably ugly). The default interp_type should be
Gdk::INTERP_BILINEAR which offers reasonable quality and speed.
I'm using OpenCV 3.3.1. I want to do a semi-dense optical flow operation using cv::calcOpticalFlowPyrLK, but I've been getting some really noticeable slowdown whenever my ROI is pretty big (Partly due to the fact that I am letting the user decide what the winSize should be, ranging from from 10 to 100). Anyways, it seems like cv::buildOpticalFlowPyramid can mitigate the slowdown by building image pyramids? I'm sorta familiar what image pyramids are, but in context of the function, I'm especially confused about what parameters I pass in, and how it impacts my function call to cv::calcOpticalFlowPyrLK. With that in mind, I now have these set of questions:
The output is, according to the documentation, is an OutputArrayOfArrays, which I take it can be a vector of cv::Mat objects. If so, what do I pass in to cv::calcOpticalFlowPyrLK for prevImg and nextImg (assuming that I need to make image pyramids for both)?
According to the docs for cv::buildOpticalFlowPyramid, you need to pass in a winSize parameter in order to calculate required padding for pyramid levels. If so, do you pass in the same winSize value when you eventually call cv::calcOpticalFlowPyrLK?
What exactly are the arguments for pyrBorder and derivBorder doing?
Lastly, and apologies if it sounds newbish, but what is the purpose of this function? I always assumed that cv::calcOpticalFlowPyrLK internally builds the image pyramids. Is it just to speed up the optical flow operation?
I hope my questions were clear, I'm still very new to OpenCV, and computer vision, but this topic is very interesting.
Thank you for your time.
EDIT:
I used the function to see if my guess was correct, so far it has worked, but I've seen no noticeable speed up. Below is how I used it:
// Building pyramids
int maxLvl = 3;
maxLvl = cv::buildOpticalFlowPyramid(imgPrev, imPyr1, cv::Size(searchSize, searchSize), maxLvl, true);
maxLvl = cv::buildOpticalFlowPyramid(tmpImg, imPyr2, cv::Size(searchSize, searchSize), maxLvl, true);
// LK optical flow call
cv::calcOpticalFlowPyrLK(imPyr1, imPyr2, currentPoints, nextPts, status, err,
cv::Size(searchSize, searchSize), maxLvl, termCrit, 0, 0.00001);
So now I'm wondering what's the purpose of preparing the image pyramids if calcOpticalFlowPyrLK does it internally?
So the point of your question is that you are trying to improve speed of optical flow tracking by tuning your input parameters.
If you want dirty and quick answer then here it is
KTL (OpenCV's calcOpticalFlowPyrLK) define a e residual function which are sum of gradient of point inside search window .
The main purpose is to find vector of point that can minimize residual function
So if you increase search window size (winSize) then it is more difficult to find that set of points.
If your really really want to do that then please read the official paper.
See the section 2.4
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.185.585&rep=rep1&type=pdf
I took it from official document
https://docs.opencv.org/2.4/modules/video/doc/motion_analysis_and_object_tracking.html#bouguet00
Hope that help
I have Problem understanding all Parameter of backgroundsubtractormog2.
I looked in the code (located in bfgf_gaussmix2.cpp), but don't see the connection to the mentioned paper. For exmaple is Tb = varThreshold, but what is the name of Tb in the paper?
I am especially interested in the fat marked parameter.
Let's start with the easy parameter [my remarks]:
int nmixtures
Maximum allowed number of mixture components. Actual number is determined dynamically per pixel.
[set 0 for GMG]
uchar nShadowDetection
The value for marking shadow pixels in the output foreground mask. Default value is 127.
float fTau
Shadow threshold. The shadow is detected if the pixel is a darker version of the background. Tau is a threshold defining how much darker the shadow can be. Tau= 0.5 means that if a pixel is more than twice darker then it is not shadow.
Now to the ones i don't understand:
float backgroundRatio
Threshold defining whether the component is significant enough to be included into the background model ( corresponds to TB=1-cf from the paper??which paper??). cf=0.1 => TB=0.9 is default. For alpha=0.001, it means that the mode should exist for approximately 105 frames before it is considered foreground.
float varThresholdGen
Threshold for the squared Mahalanobis distance that helps decide when a sample is close to the existing components (corresponds to Tg). If it is not close to any component, a new component is generated. 3 sigma => Tg=3*3=9 is default. A smaller Tg value generates more components. A higher Tg value may result in a small number of components but they can grow too large. [i don't understand a word of this]
In the Constructor the variable varThreshold is used. Is it the same as varThresholdGen?
Threshold on the squared Mahalanobis distance to decide whether it is well described by the background model (see Cthr??). This parameter does not affect the background update. A typical value could be 4 sigma, that is, varThreshold=4*4=16; (see Tb??).
float fVarInit
Initial variance for the newly generated components. It affects the speed of adaptation. The parameter value is based on your estimate of the typical standard deviation from the images. OpenCV uses 15 as a reasonable value.
float fVarMin
Parameter used to further control the variance.
float fVarMax
Parameter used to further control the variance.
float fCT
Complexity reduction parameter. This parameter defines the number of samples needed to accept to prove the component exists. CT=0.05 is a default value for all the samples. By setting CT=0 you get an algorithm very similar to the standard Stauffer&Grimson algorithm.
Someone asked pretty much the same question on the OpenCV website, but without an answer.
Well, I don't think anyone could tell you which parameter is what if you don't know the details of the algorithm that you are using. Besides, you should not need anyone to tell you which parameter is what if you know the details of the algorithm. I'm telling this for detailed parameters (fCT, fVarMax, etc.) not for straightforward ones (nmixtures, nShadowDetection, etc.).
So, I think you should read the papers referenced in the documentation. Here are the links for the papers 1, 2, 3.
And also you should read this paper as well, which is the beginning of background estimation.
After reading these papers and checking out the code with, I'm sure you will understand what those parameters are.
Good luck!