I'm doing some work on tracking moving objects using a ceiling mounted downward facing camera. I've got to the point where I can detect the position of the desired object in each frame.
I'm looking into using a Kalman filter to track the object's position and speed through the scene and I've reached a stumbling block. I've set up my system and have all the required parts of the Kalman filter except the measurement variance.
I want to be able to assign a meaningful variance to each measurement to allow the correction phase to use the new information in a sensible manner. I have several measures assigned to my detected objects which could in theory be useful in determining how accurate the position should be and it seems logical to try and combine them to derive a suitable variance.
Am I approaching this in the right manner and if so, can anyone point me in the right direction to continue?
Any help greatly appreciated.
I think you are right. According to this post:
Sensor fusioning with Kalman filter
determining the variance is 100% experimental. It seems to me you have everything you need to get good estimates of the variance.
sorry for the late reply. I have personally encountered the same problem in my previous project. I found the advice given by Gustaf Hendeby in his Sensor Fusion lecture slides ( Page 10 of the slides) extremely valuable.
To summarize:
(1) The SNR of your measurement noise and your process noise determines your filter behavior. A high process noise/measurement noise ration makes your filter slower (low-pass filter), which will usually allow smoother tracking, vice versa a if you set your measurement noise low, you essentially have a high pass filter, which tends to have more jitter.
(2) There are numerous papers in the literature discuss on how to set these noise model properly. However, usually a lot of "tuning" is needed depends on your application. Usually the measurement noise is what we can measure/characterize based on the hardware specification. Therefore a recommendation is to fix "R" (measurement noise covariance) and tune Q (the process model noise covariance).
Related
I am comparing models for the detection of objects for maritime Search and Rescue (SAR) purposes. From the models that I used, I got the best results for the improved version of YOLOv3 for small object detection and for FASTER RCNN.
For YOLOv3 I got the best mAP#50, but for FASTER RCNN I got better all other metrics (precision, recall, F1 score). Now I am wondering how to read it and which model is really better in this case?
I would like to add that there are only two classes in the dataset: small and large objects. We chose this solution because the objects' distinction between classes is not as important to us as the detection of any human origin object.
However, small objects don't mean small GT bounding boxes. These are objects that actually have a small area - less than 2 square meters (e.g. people, buoys). Large objects are objects with a larger area (boats, ships, canoes, etc.).
Here are the results per category:
And two sample images from the dataset (with YOLOv3 detections):
The mAP for object detection is the average of the AP calculated for all the classes. mAP#0.5 means that it is the mAP calculated at IOU threshold 0.5.
The general definition for the Average Precision(AP) is finding the area under the precision-recall curve.
The process of plotting the model's precision and recall as a function of the model’s confidence threshold is the precision recall curve.
Precision measures how accurate is your predictions. i.e. the percentage of your predictions that are correct. Recall measures how good you find all the positives. F1 score is HM (Harmonic Mean) of precision and recall.
To answer your questions now.
How to read it and which model is really better in this case?
The mAP is a good measure of the sensitivity of the neural network. So good mAP indicates a model that's stable and consistent across difference confidence thresholds. In your case faster rcnn results indicate that precision-recall curve metric is bad compared to that of Yolov3, which means that either faster rcnn has very bad recall at higher confidence thresholds or very bad precision at lower confidence threshold compared to that of Yolov3 (especially for small objects).
Precision, Recall and F1 score are computed for given confidence threshold. I'm assuming you're running the model with default confidence threshold (could be 0.25). So higher Precision, Recall and F1 score of faster rcnn indicate that at that confidence threshold it's better in terms of all the 3 metric compared to that of Yolov3.
What metric should be more important?
In general to analyse better performing model, I would suggest you to use validation set (data set that is used to tune hyper-parameters) and test set (data set that is used to assess the performance of a fully-trained model).
Note: FP - False Positive FN - False Negative
On validation set:
Use mAP to select best performing model (model that is more stable and consistent) out of all the trained weights across iterations/epochs. Use mAP to understand whether model should be trained/tuned further or not.
Check class level AP values to ensure model is stable and good across the classes.
As per use-case/application, if you're completely tolerant to FNs and highly intolerant to FPs then to train/tune the model accordingly use Precision.
As per use-case/application, if you're completely tolerant to FPs and highly intolerant to FNs then to train/tune the model accordingly use Recall.
On test set:
If you're neutral towards FPs and FNs, then use F1 score to evaluate best performing model.
If FPs are not acceptable to you (without caring much about FNs) pick the model with higher Precision
If FNs are not acceptable to you (without caring much about FPs) pick the model with higher Recall
Once you decide metric you should be using, try out multiple confidence thresholds (say for example - 0.25, 0.35 and 0.5) for given model to understand for which confidence threshold value the metric you selected works in your favour and also to understand acceptable trade off ranges (say you want Precision of at least 80% and some decent Recall). Once confidence threshold is decided, you use it across different models to find out best performing model.
I'm currently writing a program that will simulate in very basic terms the diffusion and pressure of a gas in a 3D volume with boundaries throughout - Think for example an ant cave.
The effects I want to achieve:
1. Gas diffuses throughout the environment over time, respecting walls.
2. I'd like to measure pressure, or the compression of the gas, per grid point. The effect of this should be that if a room is opened the gas will diffuse out of the opening in a speed that reflects the pressure difference.
My problem is that I lack the knowledge to fully understand theoretical math equations, and to be honest I'm really not looking for an accurate simulation. I'd just want to achieve some of the prominent effects of the physics at play. I'm not interested in fluid dynamics (For example the simulation of smoke).
I'll be writing this program in OpenCL but happy to take any form of code examples, be it C or pseudo code.
I'm thinking I should pass in 3 3D arrays - One for the gas, one that defines the walls (eg 1 at xyz = wall), and one to store the pressure.
I'm currently assuming checking for the wall is easy enough. One simply checks each neighbour cell for it and account for the cell if its not a wall:
For each grid point,
is wallmatrix[x+1] a wall?
[diffusion math here]
is wallmatrix[x-1] a wall?
[diffusion math here]
is wallmatrix[y+1] a wall?
[diffusion math here]
etc...
But I'm just not sure what to do with the diffusion math, nor how I would include pressure in all this?
Diffusion is one of the easiest things to simulate because it's self smoothing.
For example, you could run your simulation in terms of constant time steps and keeping track of the individual particle positions, and at each time step move each particle a fixed (small) distance, but in a random direction.
There are other ways too, for example, you can do a grid based approach, where change the number of particles in each grid location.
A slight issue with your question is where you say, "diffuse out of the opening in a speed that reflects the pressure difference". Diffusion doesn't really do this, since diffusion is just the random motion of particles. I think, though, that even straight diffusion might look satisfying to you here, since the gas will diffuse out of an opening, and it will look faster. Really what will be happening though is that it will be diffusing out at the same speed as everywhere else, it's just that nothing will be diffusing back in. Still, if this isn't satisfying, then you will need to get into fluid dynamics, at least a bit, since this is how one describes how fluid behaves when there's a pressure gradient, not diffusion.
Well, this is not an easy task!
First of all: you want to simulate basic diffusion OR the complete motion of the gas? The second case isn't easy at all, but you can get an idea here.
If you just want to diffuse a gas in an static environment, things are easier but you can't measure the
total pressure, your only variable will be the local concentration of the gas.
This phenomena is governed by the Fick laws; the second one is probably what you are looking for.
Read for finite difference methods to understand how to discretize the diffusion equation.
The subject is quite big to write a complete answer here.
I'm using OpenCV to process pictures taken with a mobile phone. The pictures contain text, and they have small amounts of motion blur, which I need to remove.
What would be the most viable algorithm to use? I have tested so far Lucy-Richardson and Weiner deconvolution, but they did not yield satisfactory results.
Agree with #TheJuice, your problem lies in the PSF estimation. Usually to be able to do this from a single frame, several assumptions need to be made about the factors leading to the blur (motion of object, type of motion of the sensor, etc.).
You can find some pointers, especially on the monodimensional case, here. They use a filtering method that leaves mostly correlation from the blur, discarding spatial correlation of original image, and use this to deduce motion direction and thence the PSF. For small blurs you might be able to consider the motion as constant; otherwise you will have to use a more complex accelerated motion model.
Unfortunately, mobile phone blur is often a compound of CCD integration and non-linear motion (translation perpendicular to line of sight, yaw from wrist motion, and rotation around the wrist), so Yitzhaky and Kopeika's method will probably only yield acceptable results in a minority of cases. I know there are methods to deal with that ("depth awareness" and other) but I have never had occasion of dealing with them.
You can preview the results using photo recovery software such as Focus Magic; while they do not employ YK estimator (motion description is left to you), the remaining workflow is necessarily very similar. If your pictures are amenable to Focus Magic recovery, then probably YK method will work. If they are not (or not enough, or not enough of them to be worthwhile), then there's no point even trying to implement it.
Motion blur is a difficult problem to overcome. The best results are gained when
The speed of the camera relative to the scene is known
You have many pictures of the blurred object which you can correlate.
You do have one major advantage in that you are looking at text (which normally constitutes high contrast features). If you only apply deconvolution to high contrast (I know that the theory is often to exclude high contrast) areas of your image you should get results which may enable you to better recognise characters. Also a combination of sharpening/blurring filters pre/post processing may help.
I remember being impressed with this paper previously. Perhaps an adaption on their implementation would be worth a go.
I think the estimation of your point-spread function is likely to be more important than the algorithm used. It depends on the kind of motion blur you're trying to remove, linear motion is likely to be the easiest but is unlikely to be the kind you're trying to remove: i imagine it's non-linear caused by hand movement during the exposure.
You cannot eliminate motion blur. The information is lost forever. What you are dealing with is a CCD that is recording multiple real objects to a single pixel, smearing them together. In other words if the pixel reads 56, you cannot magically determine that the actual reading should have been 37 at time 1, and 62 at time 2, and 43 at time 3.
Another way to look at this: imagine you have 5 pictures. You then use photoshop to blend the pictures together, averaging the value of each pixel. Can you now somehow from the blended picture tell what the original 5 pictures were? No, you cannot, because you do not have the information to do that.
Given a video with a fixed background containing a lot of variation in light I am trying to detect pulses of light that occur for relatively short spans of time. When the video is played it is pretty easy for a person to distinguish the light pulses but if only shown a still frame it would be impossible to distinguish a pulse from background light.
I would like to know if there is specific terminology in machine vision that I can use to search for algorithms used to solve this problem. Also if you have any references for papers or open source software that solves this problem that would be great.
Edit: More context
The video itself is of a biological process that occurs at the sub-cellular level and while the background is fixed there is also a significant amount of random signal noise at the pixel level (there doesn't appear to be significant correlation in the noise between neighboring pixels). Note that the variation I refer to in the first paragraph is true variation and not signal noise. Since I mentioned that the process is biological it's probably also worth saying that there is no movement going on; these are just pulses of light. Also, the pulses themselves occupy enough pixels so that it is easy to discern their relative sizes.
From statistics, you could look into change point detection. The essential idea being that most of the time each (x,y) point or region, if you define some granularity of regions, has an intensity I(x,y), where I(x,y) is random, but either bounded or stochastic with some assumed distribution (e.g. normal with a given mean and standard deviation), and then it is observed with an intensity that is anomalous for that distribution. Anomaly detection would also apply, but the time series nature is more appropriate.
(If you want to go more into the statistical methodologies, it would be far more appropriate to discuss this on the statistics Stack Exchange site.)
If you look into astronomical applications, you can find papers on supernova and pulsar detection.
Update 1. Just to clarify the astronomical analogies, if the pulse is repeating, then papers on pulsars or satellites may be most appropriate. If the pulse is one-time, then papers on supernova detection would be better. If the pulse is bursty, and spatially clustered, then meteor strike detection would be better. Although spatial time series analysis, especially change point or anomaly detection, is useful, it's best to have an understanding of the stochastic phenomena of interest in order to narrow down the detection methodology.
To continue the notion of applying statistics: you might consider gridding each image frame into rectangular neighborhoods. At each time t, compute the variance (or standard deviation) of the neighborhood. Presumably, the unexcited neighborhoods will exhibit some common distribution of intensity (i.e. uniform, but most likely some form of gaussian). The presence of pulse pixels will bias that distribution in some way. When comparing a neighborhood at time t and t-1, a significant change in mean intensity (or a change in the variance, etc.) would indicate an excited neighborhood.
You might also consider looking at other measures, such as skewness and kurtosis. Assuming the initial, unexcited distribution is gaussian, the "shape" parameters could also identify differences in the pixel populations.
*Note that I'm assuming a grayscale image for simplicity, but the same principles may be applied to an RGB image.
Assuming a completely static scene with no object and camera motion, then any color deviation would be due to lighting changes.
If you detect an abrupt color/intensity change at particular pixels (i.e. brighness change above a certain allowable threshold), then this should be due to the light source turning on/off.
If you are only interested in point light sources, then any change in a region larger than the maximum apparent light source should be considered as coming from something else (e.g. the sun suddenly revealed from behind clouds).
100 periods have been collected from a 3 dimensional periodic signal. The wavelength slightly varies. The noise of the wavelength follows Gaussian distribution with zero mean. A good estimate of the wavelength is known, that is not an issue here. The noise of the amplitude may not be Gaussian and may be contaminated with outliers.
How can I compute a single period that approximates 'best' all of the collected 100 periods?
Time-series, ARMA, ARIMA, Kalman Filter, autoregression and autocorrelation seem to be keywords here.
UPDATE 1: I have no idea how time-series models work. Are they prepared for varying wavelengths? Can they handle non-smooth true signals? If a time-series model is fitted, can I compute a 'best estimate' for a single period? How?
UPDATE 2: A related question is this. Speed is not an issue in my case. Processing is done off-line, after all periods have been collected.
Origin of the problem: I am measuring acceleration during human steps at 200 Hz. After that I am trying to double integrate the data to get the vertical displacement of the center of gravity. Of course the noise introduces a HUGE error when you integrate twice. I would like to exploit periodicity to reduce this noise. Here is a crude graph of the actual data (y: acceleration in g, x: time in second) of 6 steps corresponding to 3 periods (1 left and 1 right step is a period):
My interest is now purely theoretical, as http://jap.physiology.org/content/39/1/174.abstract gives a pretty good recipe what to do.
We have used wavelets for noise suppression with similar signal measured from cows during walking.
I'm don't think the noise is so much of a problem here and the biggest peaks represent actual changes in the acceleration during walking.
I suppose that the angle of the leg and thus accelerometer changes during your experiment and you need to account for that in order to calculate the distance i.e you need to know what is the orientation of the accelerometer in each time step. See e.g this technical note for one to account for angle.
If you need get accurate measures of the position the best solution would be to get an accelerometer with a magnetometer, which also measures orientation. Something like this should work: http://www.sparkfun.com/products/10321.
EDIT: I have looked into this a bit more in the last few days because a similar project is in my to do list as well... We have not used gyros in the past, but we are doing so in the next project.
The inaccuracy in the positioning doesn't come from the white noise, but from the inaccuracy and drift of the gyro. And the error then accumulates very quickly due to the double integration. Intersense has a product called Navshoe, that addresses this problem by zeroing the error after each step (see this paper). And this is a good introduction to inertial navigation.
Periodic signal without noise has the following property:
f(a) = f(a+k), where k is the wavelength.
Next bit of information that is needed is that your signal is composed of separate samples. Every bit of information you've collected are based on samples, which are values of f() function. From 100 samples, you can get the mean value:
1/n * sum(s_i), where i is in range [0..n-1] and n = 100.
This needs to be done for every dimension of your data. If you use 3d data, it will be applied 3 times. Result would be (x,y,z) points. You can find value of s_i from the periodic signal equation simply by doing
s_i(a).x = f(a+k*i).x
s_i(a).y = f(a+k*i).y
s_i(a).z = f(a+k*i).z
If the wavelength is not accurate, this will give you additional source of error or you'll need to adjust it to match the real wavelength of each period. Since
k*i = k+k+...+k
if the wavelength varies, you'll need to use
k_1+k_2+k_3+...+k_i
instead of k*i.
Unfortunately with errors in wavelength, there will be big problems keeping this k_1..k_i chain in sync with the actual data. You'd actually need to know how to regognize the starting position of each period from your actual data. Possibly need to mark them by hand.
Now, all the mean values you calculated would be functions like this:
m(a) :: R->(x,y,z)
Now this is a curve in 3d space. More complex error models will be left as an excersize for the reader.
If you have a copy of Curve Fitting Toolbox, localized regression might be a good choice.
Curve Fitting Toolbox supports both lowess and loess localized regression models for curve and curve fitting.
There is an option for robust localized regression
The following blog post shows how to use cross validation to estimate an optimzal spaning parameter for a localized regression model, as well as techniques to estimate confidence intervals using a bootstrap.
http://blogs.mathworks.com/loren/2011/01/13/data-driven-fitting/