How to retrieve outliers from ceres solver result? - c++

Question

I try to compare images using method similar to Features2D + Homography to find a known object but replace findHomography() by self-writed findAffine() function.
I use Ceres Solver to obtain optimal affine matrix considering outliers.
double translation[] = {0, 0};
double angle = 0;
double scaleFactor = 1;
ceres::Problem problem;
for (size_t i = 0; i < points1.size(); ++i) {
problem.AddResidualBlock(
new ceres::AutoDiffCostFunction<AffineResidual, 1, 2, 1, 1>(
new AffineResidual(Eigen::Vector2d(points1[i].x, points1[i].y),
Eigen::Vector2d(points2[i].x, points2[i].y))),
new ceres::HuberLoss(1.0),
translation,
&angle,
&scaleFactor);
}
ceres::Solver::Options options;
options.linear_solver_type = ceres::DENSE_QR;
options.minimizer_progress_to_stdout = true;
ceres::Solver::Summary summary;
Solve(options, &problem, &summary);
Ceres solver provide LossFunction:
Loss functions reduce the influence of residual blocks with high residuals, usually the ones corresponding to outliers.
Of course, I can transform keypoints coordinates from first image by obtained matrix, compare with second and get deviation. But ceres solver already done it inside during work.
How I can retrieve it? Did not find it in the documentation.

Answer 1

I had the similar problem. After looking into Ceres library sources (particularly into ResidualBlock::Evaluate() method) I had a conclusion that there is no explicit "outlier" status for residual block. It seems that the loss function just affects resulting cost value for a block (which is exactly described by the phrase from documentation you have quoted - "Loss functions reduce the influence of residual blocks with high residuals"). So the answer is that you cannot retrieve outliers from Ceres, there is no such feature.
Workaround might be calculating residuals for your data with the solved result, and apply loss function to them. The comment from LossFunction::Evaluate() might help:
// For a residual vector with squared 2-norm 'sq_norm', this method
// is required to fill in the value and derivatives of the loss
// function (rho in this example):
//
// out[0] = rho(sq_norm),
// out[1] = rho'(sq_norm),
// out[2] = rho''(sq_norm),
//
// Here the convention is that the contribution of a term to the
// cost function is given by 1/2 rho(s), where
//
// s = ||residuals||^2.
//
// Calling the method with a negative value of 's' is an error and
// the implementations are not required to handle that case.
//
// Most sane choices of rho() satisfy:
//
// rho(0) = 0,
// rho'(0) = 1,
// rho'(s) < 1 in outlier region,
// rho''(s) < 0 in outlier region,
//
// so that they mimic the least squares cost for small residuals.
virtual void Evaluate(double sq_norm, double out[3]) const = 0;