OpenCV - Use FLANN with ORB descriptors to match features - c++

Question

I am using OpenCV 3.2
I am trying to use FLANN to match features descriptors in a faster way than brute force.
// Ratio to the second neighbor to consider a good match.
#define RATIO 0.75
void matchFeatures(const cv::Mat &query, const cv::Mat &target,
std::vector<cv::DMatch> &goodMatches) {
std::vector<std::vector<cv::DMatch>> matches;
cv::Ptr<cv::FlannBasedMatcher> matcher = cv::FlannBasedMatcher::create();
// Find 2 best matches for each descriptor to make later the second neighbor test.
matcher->knnMatch(query, target, matches, 2);
// Second neighbor ratio test.
for (unsigned int i = 0; i < matches.size(); ++i) {
if (matches[i][0].distance < matches[i][1].distance * RATIO)
goodMatches.push_back(matches[i][0]);
}
}
This code is working me with SURF and SIFT descriptors, but not with ORB.
OpenCV Error: Unsupported format or combination of formats (type=0) in buildIndex
As it's said here, FLANN needs the descriptors to be of type CV_32F so we need to convert them.
if (query.type() != CV_32F) query.convertTo(query, CV_32F);
if (target.type() != CV_32F) target.convertTo(target, CV_32F);
However, this supposed fix is returning me another error in convertTo function.
OpenCV Error: Assertion failed (!fixedType() || ((Mat*)obj)->type() == mtype) in create
This assertion is in opencv/modules/core/src/matrix.cpp file, line 2277.
What's happening?
Code to replicate issue.
#include <opencv2/opencv.hpp>
int main(int argc, char **argv) {
// Read both images.
cv::Mat image1 = cv::imread(argv[1], cv::IMREAD_GRAYSCALE);
if (image1.empty()) {
std::cerr << "Couldn't read image in " << argv[1] << std::endl;
return 1;
}
cv::Mat image2 = cv::imread(argv[2], cv::IMREAD_GRAYSCALE);
if (image2.empty()) {
std::cerr << "Couldn't read image in " << argv[2] << std::endl;
return 1;
}
// Detect the keyPoints and compute its descriptors using ORB Detector.
std::vector<cv::KeyPoint> keyPoints1, keyPoints2;
cv::Mat descriptors1, descriptors2;
cv::Ptr<cv::ORB> detector = cv::ORB::create();
detector->detectAndCompute(image1, cv::Mat(), keyPoints1, descriptors1);
detector->detectAndCompute(image2, cv::Mat(), keyPoints2, descriptors2);
// Match features.
std::vector<cv::DMatch> matches;
matchFeatures(descriptors1, descriptors2, matches);
// Draw matches.
cv::Mat image_matches;
cv::drawMatches(image1, keyPoints1, image2, keyPoints2, matches, image_matches);
cv::imshow("Matches", image_matches);
}

Answer 1

Did you adjust the FLANN parameters?
Taken from http://docs.opencv.org/3.0-beta/doc/py_tutorials/py_feature2d/py_matcher/py_matcher.html
While using ORB, you can pass the following. The commented values are recommended as per the docs, but it didn’t provide required results in some cases. Other values worked fine.:
index_params= dict(algorithm = FLANN_INDEX_LSH,
table_number = 6, # 12
key_size = 12, # 20
multi_probe_level = 1) #2
Probably you can convert that to C++ api?
According to the comment, the C++ way is:
cv::FlannBasedMatcher matcher = cv::FlannBasedMatcher(cv::makePtr<cv::flann::LshIndexParams>(12, 20, 2));

Answer 2

Binary-string descriptors: ORB, BRIEF, BRISK, FREAK, AKAZE, etc.
Floating-point descriptors: SIFT, SURF, GLOH, etc.
Feature matching of binary descriptors can be efficiently done by comparing their Hamming distance as opposed to Euclidean distance used for floating-point descriptors.
For comparing binary descriptors in OpenCV, use FLANN + LSH index or Brute Force + Hamming distance.
http://answers.opencv.org/question/59996/flann-error-in-opencv-3/
By default, FlannBasedMatcher works as KDTreeIndex with L2 norm. This is the reason why it works well with SIFT/SURF descriptors and throws an exception for ORB descriptor.
Binary features and Locality Sensitive Hashing (LSH)
Performance comparison between binary and floating-point descriptors

Answer 3

I believe there is a bug in the OpenCV3 version: FLANN error in OpenCV 3
You need to convert your descriptors to a 'CV_32F'.