I am testing OpenCV pose estimation with solvePnPRansac. The method uses a parameter reprojectionError which distinguishes inliers/outliers for RANSAC. Assuming that a pose estimation returning T=T(R,t), is it possible to find the maximal estimation error $$\DeltaT_max$$ given a value of reprojectionError?
I've done some tests by formulating this question as a constraint optimization problem and see that this estimation is rather sensitive. In the sense that, with a small reprojectionEror tolerance e.g. .1, .01..., maximal error in translation can reach hundreds of millimeters, and almost all possible angle rotation. From this view point, is it reliable to use such an estimation where some small errors can be easily found e.g. round-up of pixel location of the reference point?
Edit 1:
I've found some massive tests from "Absolute Pose Estimation from Line Correspondences using Direct Linear Transformation"
It looks like to have a good estimation, better to enforce the reprojection error to be rather small e.g. <1e-6 or <1e-7 and number of matches should be near to 10 in order to have a sound estimation in position and orientation (say angle < 1 Deg and position < 1 meter which is still huge). So, regards to the reprojectionError earlier I mentioned i.e. .1, .01 is very far from accurate. With this being relate to solvePnPRansac, the default value for this reprojectionEror is ... 8 pixel! isn't this almost irrelevant? if a point is inlier, the required reprojection error should be very very small
Also, if the graph and my understanding it is right, it also can be interpreted like...to have position error <.01 meter, number of matching lines/points should reach >2000 and rotation error <.1 Deg, this number is up to > 100 to 500. These numbers of matching are very huge for cases
Related
The standard approach for validating the camera calibration is to compute the distance between the detected point and the corresponding world point reprojected in the image, this procedure validates both the intrinsic as extrinsic parameters
Now it is possible to verify the accuracy of the nonlinear distortion parameters by capturing images of lines and then undistorting the images and measuring if the lines are now straight lines.
Is there a way to verify the accuracy of the linear intrinsic parameters (optical center, focal point, skew) separate from the extrinsics?
It's tricky, tending to very tricky if you require high levels of accuracy. The problem is that all intrinsic parameters are coupled in the reprojection error.
To give you an indea of the difficulties involved, consider the case of the principal point. It can be proven that the principal point of a pinhole camera is the baricenter of the triangle formed by three independent vanishing points. This would seem to suggest a procedure for verifying it independently of the other intrinsic parameters: take one or more images collectively showing three or more pencils of parallel lines, detect and model said pencil of lines, estimate their vanishing points, etc. However, to precisely model the detected lines, so you can intersect them to find the vanishing points, you need to accurately undistort the images - and guess what, the center of the nonlinear lens distortion is often approximated by the principal point, so your "verification" procedure ends up using exactly the same estimated parameter you are trying to independently verify.
You could try to work around the above difficulty by using an alternate non-parametric model of the nonlinear distortion - for example a thin-plate spline built off a grid using a cost function that only depends on deviation from linearity - as you suggest. Then again, it's tricky to come up with such a cost function that is unbiased: simple linear-least-squares fitting a straight line won't do, since the distorted images of the line points are in general not i.i.d. with respect to the underlying undistorted line. So you need to use a "local" model for each line, typically a low-order polynomial.
In the end, you are much better off just accepting that the parameters (both intrinsic and extrinsic) are coupled, and simply base your verification on the input-output needs of your actual application: determine what is an acceptable RMS reprojection error over the image area, then use independent sets of images of a known calibration object, one that somehow models the properties of the 3D scene that are important to your application, then reproject its points and verify that the errors you get are acceptable.
I'm working on an application were I have a set of Contours(each one representing a Potential Line) and I wanna check "How straight" is that contour/shape.
The article I am using as a refrence uses the following technique:
It Matches a "segmented" line crossing the shape like so-
Then grading how "straight" is the line.
Heres an example of the Contours I am working on:
How would you go about implementing this technique?
Is there any other way of checking "How Straight" is a contour\shape?
Regards!
My first guess would be to use a coefficient of determination. That would be, fit a linear line to all your point assuming some reasonable origin where you won't receive rounding errors and calculate R^2.
A more advanced approach, if all contours are disconnected components, would be to calculate the structure model index (the link is for bone morphometry, but they explain the concept and cite the original paper.) This gives you a number that tells you how much your segment is "like a rod". This is just an idea, though. Anything that forms curves or has branches will be less and less like a rod.
I would say that it also depends on what you are using the metric for and if your contours are always generally carrying left to right.
An additional method would be to create the covariance matrix of your points, calculate the eigenvalues from that matrix, and take their ratio (where the ratio is greater than or equal to 1; otherwise, invert the ratio.) This is the basic principle behind a PCA besides the final ratio. If you have a rather linear data set (the data set varies in only one direction) then you will have a very large ratio. As the data set becomes less and less linear (or more uncorrelated) you would see the ratio approach one. A perfectly linear data set would be infinity and a perfect circle one (I believe, but I would appreciate if someone could verify this for me.) Also, working in two dimensions would mean the calculation would be computationally cheap and straight forward.
This would handle outliers very well and would be invariant to the rotation and shape of your contour. You also have a number which is always positive. The only issue would be preventing overflow when dividing the two eigenvalues. Then again you could always divide the smaller eigenvalue by the larger and your metric would be bound between zero and one, one being a circle and zero being a straight line.
Either way, you would need to test if this parameter is sensitive enough for your application.
One example for a simple algorithm is using the dot product between two segments to determine the angle between them. The formula for dot product is:
A * B = ||A|| ||B|| cos(theta)
Solving the equation for cos(theta) yields
cos(theta) = (A * B / (||A|| ||B||))
Since cos(0) = 1, cos(pi) = -1.0 and you're checking for the "straightness" of the lines, a line whose normalization of cos(theta) angles is closest to -1.0 is the straightest.
straightness = SUM(cos(theta))/(number of line segments)
where a straight line is close to -1.0, and a non-straight line approaches 1.0. Keep in mind this is a cursory evaluation of this algorithm and it obviously has edge cases and caveats that would need to be addressed in an implementation.
The trick is to use image moments. In short, you calculate the minimum inertia around an axis, the inertia around an axis perpendicular to this, and the ratio between them (which is always between 0 and 1; since inertia is non-negative)
For a straight line, the inertia along the line is zero, so the ratio is also zero. For a circle, the inertia is the same along all axis so the ratio is one. Your segmented line will be 0.01 or so as it's a fairly good match.
A simpler method is to compare the circumference of the the convex polygon containing the shape with the circumference of the shape itself. For a line, they're trivially equal, and for a not too crooked shape it's still comparable.
How does one determine the appropriate physical size of calibration board for camera calibration ? In the past, I have used ones like these printed on A4 size paper. Reasonable criteria seem to be the precision(i.e. how accurate) and completeness(i.e. are all corners detected) with a particular sized board. But, short of printing multiple sizes and trying them all, is there any other approach ?
On a related note, if we know that the "depth of field" of interest is 2 metres to 2.5 metres from the camera, would it be better to use the calibration pattern within this range or would it be better to try it ignoring such preferences ?
it is a wrong question to ask. Apart from resolution of the board that is going to be probably fine for a wide range of sizes the important question is how many boards you have to show to your camera in order to find accurately intrinsic and extrinsic parameters.
Unlike PnP or posit or extrinsic problem calibration has to find both extrinsic and intrinsic. You will have to cover camera whole field of view with images of your board plus (or rather times) different board orientations and distances. This is because converging to the right intrinsic and extrinsic requires seeing vanishing points while eliminating a bias in pixel error metrics requires uniform sampling of the field of view.
First, see this answer for general notes on manufacturing a calibration target.
Doing a complete sensitivity analysis is quite complicate. A rule of thumb I have used is to assume that my corner detector is accurate up to 1/2 a pixel worst case. Then, given an approximate field of view (which you can estimate from the lens's nominal focal length in mm and the width of the sensor chip), and a specification for the min and max distance used in the application, I worked out the sensitivity of a plane's estimated pose in various orientations and positions in the calibration volume, given a worst case error in its four corners. I then played with its size until the worst case error became acceptable - and within my budget and the other application constraints (weight, depth of field, resolution, etc).
I am trying to extract the curvature of a pulse along its profile (see the picture below). The pulse is calculated on a grid of length and height: 150 x 100 cells by using Finite Differences, implemented in C++.
I extracted all the points with the same value (contour/ level set) and marked them as the red continuous line in the picture below. The other colors are negligible.
Then I tried to find the curvature from this already noisy (due to grid discretization) contour line by the following means:
(moving average already applied)
1) Curvature via Tangents
The curvature of the line at point P is defined by:
So the curvature is the limes of angle delta over the arclength between P and N. Since my points have a certain distance between them, I could not approximate the limes enough, so that the curvature was not calculated correctly. I tested it with a circle, which naturally has a constant curvature. But I could not reproduce this (only 1 significant digit was correct).
2) Second derivative of the line parametrized by arclength
I calculated the first derivative of the line with respect to arclength, smoothed with a moving average and then took the derivative again (2nd derivative). But here I also got only 1 significant digit correct.
Unfortunately taking a derivative multiplies the already inherent noise to larger levels.
3) Approximating the line locally with a circle
Since the reciprocal of the circle radius is the curvature I used the following approach:
This worked best so far (2 correct significant digits), but I need to refine even further. So my new idea is the following:
Instead of using the values at the discrete points to determine the curvature, I want to approximate the pulse profile with a 3 dimensional spline surface. Then I extract the level set of a certain value from it to gain a smooth line of points, which I can find a nice curvature from.
So far I could not find a C++ library which can generate such a Bezier spline surface. Could you maybe point me to any?
Also do you think this approach is worth giving a shot, or will I lose too much accuracy in my curvature?
Do you know of any other approach?
With very kind regards,
Jan
edit: It seems I can not post pictures as a new user, so I removed all of them from my question, even though I find them important to explain my issue. Is there any way I can still show them?
edit2: ok, done :)
There is ALGLIB that supports various flavours of interpolation:
Polynomial interpolation
Rational interpolation
Spline interpolation
Least squares fitting (linear/nonlinear)
Bilinear and bicubic spline interpolation
Fast RBF interpolation/fitting
I don't know whether it meets all of your requirements. I personally have not worked with this library yet, but I believe cubic spline interpolation could be what you are looking for (two times differentiable).
In order to prevent an overfitting to your noisy input points you should apply some sort of smoothing mechanism, e.g. you could try if things like Moving Window Average/Gaussian/FIR filters are applicable. Also have a look at (Cubic) Smoothing Splines.
100 periods have been collected from a 3 dimensional periodic signal. The wavelength slightly varies. The noise of the wavelength follows Gaussian distribution with zero mean. A good estimate of the wavelength is known, that is not an issue here. The noise of the amplitude may not be Gaussian and may be contaminated with outliers.
How can I compute a single period that approximates 'best' all of the collected 100 periods?
Time-series, ARMA, ARIMA, Kalman Filter, autoregression and autocorrelation seem to be keywords here.
UPDATE 1: I have no idea how time-series models work. Are they prepared for varying wavelengths? Can they handle non-smooth true signals? If a time-series model is fitted, can I compute a 'best estimate' for a single period? How?
UPDATE 2: A related question is this. Speed is not an issue in my case. Processing is done off-line, after all periods have been collected.
Origin of the problem: I am measuring acceleration during human steps at 200 Hz. After that I am trying to double integrate the data to get the vertical displacement of the center of gravity. Of course the noise introduces a HUGE error when you integrate twice. I would like to exploit periodicity to reduce this noise. Here is a crude graph of the actual data (y: acceleration in g, x: time in second) of 6 steps corresponding to 3 periods (1 left and 1 right step is a period):
My interest is now purely theoretical, as http://jap.physiology.org/content/39/1/174.abstract gives a pretty good recipe what to do.
We have used wavelets for noise suppression with similar signal measured from cows during walking.
I'm don't think the noise is so much of a problem here and the biggest peaks represent actual changes in the acceleration during walking.
I suppose that the angle of the leg and thus accelerometer changes during your experiment and you need to account for that in order to calculate the distance i.e you need to know what is the orientation of the accelerometer in each time step. See e.g this technical note for one to account for angle.
If you need get accurate measures of the position the best solution would be to get an accelerometer with a magnetometer, which also measures orientation. Something like this should work: http://www.sparkfun.com/products/10321.
EDIT: I have looked into this a bit more in the last few days because a similar project is in my to do list as well... We have not used gyros in the past, but we are doing so in the next project.
The inaccuracy in the positioning doesn't come from the white noise, but from the inaccuracy and drift of the gyro. And the error then accumulates very quickly due to the double integration. Intersense has a product called Navshoe, that addresses this problem by zeroing the error after each step (see this paper). And this is a good introduction to inertial navigation.
Periodic signal without noise has the following property:
f(a) = f(a+k), where k is the wavelength.
Next bit of information that is needed is that your signal is composed of separate samples. Every bit of information you've collected are based on samples, which are values of f() function. From 100 samples, you can get the mean value:
1/n * sum(s_i), where i is in range [0..n-1] and n = 100.
This needs to be done for every dimension of your data. If you use 3d data, it will be applied 3 times. Result would be (x,y,z) points. You can find value of s_i from the periodic signal equation simply by doing
s_i(a).x = f(a+k*i).x
s_i(a).y = f(a+k*i).y
s_i(a).z = f(a+k*i).z
If the wavelength is not accurate, this will give you additional source of error or you'll need to adjust it to match the real wavelength of each period. Since
k*i = k+k+...+k
if the wavelength varies, you'll need to use
k_1+k_2+k_3+...+k_i
instead of k*i.
Unfortunately with errors in wavelength, there will be big problems keeping this k_1..k_i chain in sync with the actual data. You'd actually need to know how to regognize the starting position of each period from your actual data. Possibly need to mark them by hand.
Now, all the mean values you calculated would be functions like this:
m(a) :: R->(x,y,z)
Now this is a curve in 3d space. More complex error models will be left as an excersize for the reader.
If you have a copy of Curve Fitting Toolbox, localized regression might be a good choice.
Curve Fitting Toolbox supports both lowess and loess localized regression models for curve and curve fitting.
There is an option for robust localized regression
The following blog post shows how to use cross validation to estimate an optimzal spaning parameter for a localized regression model, as well as techniques to estimate confidence intervals using a bootstrap.
http://blogs.mathworks.com/loren/2011/01/13/data-driven-fitting/