I want to match a set of binary descriptors (query data) against a larger set of binary descriptors (train data).
The matching should be done fast. I decided to try the FlannBasedMatcher of OpenCv. The matcher supports a variety of Algorithms. For binary descriptors there is Multi-Probe LSH implemented.
FlannBasedMatcher matcher (new flann::LshIndexParams(12,10,2));
std::vector<DMatch> matches;
matcher.knnMatch(query, train, matches, 2);
(I tried different LshIndexParams settings)
The Problem is that Flann matching with LSH is very slow compared to BruteForce Matching or compared to Flann matching with KDTreeIndexParams using float descriptors.
It also looks like some others experienced the same problem (see 1, 2).
In one of the answers it is suggested to try Hierarchical Clustering.
That should be faster than LSH.
I would like to use the same FlannBasedMatcher Interface as with LSH.
FlannBasedMatcher matcher (new flann::HierarchicalClusteringIndexParams());
std::vector<DMatch> matches;
matcher.knnMatch(query, train, matches, 2);
However this doesn't work with binary descriptors (see the error message):
OpenCV Error: Unsupported format or combination of formats
But the "raw" Hierarchical Clustering interface supports multiple distance types and it is possible to choose the hamming distance.
cv::flann::Index tree(train, cv::flann::HierarchicalClusteringIndexParams(), FLANN_DIST_HAMMING);
cv::Mat indices, dists;
tree.knnSearch(query, indices, dists, 2, cv::flann::SearchParams());
My questions are:
Is it possible to set a distance type for the FlannBasedMatcher to work with binary descriptors and with Hierarchical Clustering? Or can I define a custom flann::HierarchicalClusteringIndexParams() with Hamming distance? I would like to use the FlannBasedMatcher interface.
Is there an alternative faster method for matching binary descriptors? Better/faster than using FLANN with LSH or Hierarchical Clustering?
Related
Hello I try to learn a SVM with a Dataset of negative and positive examples.
Iam using the Kmeans Clustering and Bag of Words for it.
My steps are:
compute the descriptors and key points for each image with surf
put all descriptors in one mat (unclustered mat) and create the label mat(1 and -1)
using K-Means for clustering put in the unclustered mat and run the algorithm the result is the vocabulary
start the Bag of features procedure with BowImgDescriptorExtractor using the FlannBasedMatcher and the Surf detector for it
set the extracted vocabulary to BowImgDescriptorExtractor
compute bow with img, imgkeypoints
the result is a bow descriptor
train the SVM with the bowdescritpor and the labels
The syntax is correct, but if I use svm->islearned the svm returns false.
Something in my procedure is wrong. Please give me some advice what Iam doing wrong
I want to use the hierarchicalClustering function of the flann module of OpenCV 3.0. The data I want to cluster are descriptors of ORB features, so it would be reasonable to use the Hamming distance. But this does not work with OpenCV.
Is there any better solution than using the L2 distance?
This is the code I use:
cvflann::KMeansIndexParams params(32, 11, vflann::FLANN_CENTERS_KMEANSPP);
true_no_words = flann::hierarchicalClustering<cvflann::L2<unsigned char>>(in_words,out_temp,params);
I want to do something like:
cvflann::KMeansIndexParams params(32, 11, vflann::FLANN_CENTERS_KMEANSPP);
true_no_words = flann::hierarchicalClustering<cvflann::Hamming>(in_words,out_temp,params);
Currently i am working on checking duplicate images , so i am using ORB for that, the first part is almost complete, i have the descriptor vector of both the images, now as the second part i want to know how we calculate the scores using hamming distance, and what should be the threshold of saying that these are duplicates
img1 = gray_image15
img2 = gray_image25
# Initiate STAR detector
orb = cv2.ORB_create()
# find the keypoints with ORB
kp1 = orb.detect(img1,None)
kp2 = orb.detect(img2,None)
# compute the descriptors with ORB
kp1, des1 = orb.compute(img1, kp1)
kp2, des2 = orb.compute(img2, kp2)
matcher = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
matches = matcher.match(des1, des2)
# Sort them in the order of their distance.
matches = sorted(matches, key = lambda x:x.distance)
i just want to know the next step in this process so that ultimately i can print yes or no for duplicates. i am using opencv3.0.0 with python 2.7
Once you obtain the descriptors, you can use a bag-of-words model to cluster the descriptors of the reference image, that is, build a vocabulary (visual words).
Then project the descriptors of the other image on to this vocabulary.
Then you can obtain a histogram showing the distribution of each of the visual words in the two images.
Compare these two histograms using a histogram comparison technique and use a threshold to detect the duplicates. For example, if you use Bhattacharyya distance, a low value means a good match.
I don't have a python implementation of this, but you can find something similar in c++ here.
Is there a way to get the hamming distance between two matched descriptors when using the flann matcher without manually calculating it? (i.e. looping through the descriptors matched, XORing each element, then counting).
I am matching descriptors computed by ORB like so:
FlannBasedMatcher flannMatcher;
flannMatcher.match(des1, des2, matches);
If I check the distance:
cout << matches.at(0).distance;
I get the NORM_L2 distance, however my application requires the hamming distance.
Context:
The reason I want to do this is that I am generating train descriptors from a training image set using ORB, finding matches using the brute force matcher, then filtering poor matches based on hamming distance.
I then want to use the flann matcher to match descriptors on a webcam stream to these train descriptors (and show which of the training images most closely matches the current frame), but since the flann matcher doesn't seem to give the hamming distance I'm stuck when it comes to filtering out poor matches, and I get a lot of error when choosing the train image that matches best.
In OpenCV's tutorial, it is described how to create flann based matcher for SIFT and ORB (here is the ORB)
While using ORB, you can pass the following. The commented values are recommended as per the docs, but it didn't provide required results in some cases. Other values worked fine.:
FLANN_INDEX_LSH = 6
index_params= dict(algorithm = FLANN_INDEX_LSH,
table_number = 6, # 12
key_size = 12, # 20
multi_probe_level = 1) #2
https://docs.opencv.org/3.4/dc/dc3/tutorial_py_matcher.html
Try using flann::LshIndexParams as the distance type. This does Locality Sensitive Hashing (which is close to the Hamming distance)
FlannBasedMatcher matcher2(new flann::LshIndexParams(20,10,2));
See also the discussion here
I need to do BOW (bag of words) but I only have the described keypoints of the images.
For the moment, I have obtained the vocabulary using:
cv::BOWKMeansTrainer bowtrainerCN(numCenters); //num clusters
bowtrainerCN.add(allDescriptors);
cv::Mat vocabularyCN = bowtrainerCN.cluster();
So now I need to do the assignment but I can't use the compute function because it calculates the descriptors of the images and I already have that. Is there any function to do the assignment or have I to compute it manually?
Once you have built the vocabulary (codebook) using cv::BOWKMeansTrainer::cluster() method, you can then match a descriptor (with suitable size and type) to the codebook. You first have to choose the type of matcher you need with a norm to use. (see opencv doc)
For example, with cv::BFMatcher and L2 norm
// init the matcher with you pre-trained codebook
cv::Ptr<cv::DescriptorMatcher > matcher = new cv::BFMatcher(cv::NORM_L2);
matcher->add(std::vector<cv::Mat>(1, vocabulary));
// matches
std::vector<cv::DMatch> matches;
matcher->match(new_descriptors,matches);
Then the index of the closest codeword in your codebook for the new_descriptors[i] will be
matches[i].trainIdx;