I have already detected and computed SIFT keypoints and descriptors in an image (which I need for a different purpose) with OpenCV (4.3.0-dev).
Mat descriptors;
vector<KeyPoint> keypoints;
sift->detectAndCompute(image, Mat(), keypoints, descriptors);
Now I want to get keypoints and descriptors for a rectangular patch of this same image (from the previously extracted keypoints and descriptors) and do so without having to run the costly detectAndCompute() again. The only solutions I can come up with are masking and region of interest extraction, both of which require detectAndCompute() to be run again. How can this be done?
Related
I am using OpenCV's Feature2D::detectAndCompute() to compute the keypoints and descriptors of images.
The code basically look like this:
Ptr<cv::ORB> orb = cv::ORB::create();
Mat descriptors1, descriptors2...;
vector<KeyPoint> keypoints1, keypoints2...;
orb->detectAndCompute(img1, noArray(), keypoints1, descriptors1); // first instance
orb->detectAndCompute(img2, noArray(), keypoints2, descriptors2); // second instance
// more detectAndCompute...
My problem is that it takes significantly longer to do the first detectAndCompute(~0.25s) than all the detectAndCompute after it(~0.002s each). Why is this happening?
Also, regardless of how small the images are. The first detectAndCompute always takes 0.25s. Is there any way to cut down this time?
Thank you!
I'm trying to parallelize OpenCV SIFT implementation with OpenMP (hurray! for me).
First of all: seriously? OpenCV doesn't offer any parallel implementation of the best detector/descriptor available today? Only SURF has a TBB and GPU implementations.
Anyway: what I want to know is if we compute
std::vector<cv::KeyPoint> keypoints;
Mat descriptors;
detectAndCompute(image, mask, keypoints, descriptors, useProvided); //don't look mask and useProvided
It means that the descriptor relative to the i-th keypoint, i.e. keypoints[i], is the i-th descriptor, i.e. descriptors[i]?
This is important because if that's a case, in such a parallel application I have to guarantee the order which descriptors is populated (so the correspondence between keypoints and descriptors is respected).
I'm trying to compute disparity for corresponding keypoints in two epipolar stereo images.
I know how to get keypoints, descriptors and then compute DMatch, but then I'm using this simple operation to get disparity and I'm not sure if it's ok:
float disparity = keypointsLeft[matches[i].queryIdx].pt.x - keypointsRight[matches[i].trainIdx].pt.x;
(matches was defined as: vector <DMatch> matches and then computed)
Why I'm not sure?
OpenCV documentation 2.4.12 says:
1) for DescriptorExtractor::compute
void DescriptorExtractor::compute(const Mat& image, vector<KeyPoint>& keypoints, Mat& descriptors) const
keypoints – Input collection of keypoints. Keypoints for which a descriptor cannot be computed are removed and the remaining ones may be reordered. Sometimes new keypoints can be added, for example: SIFT duplicates a keypoint with several dominant orientations (for each orientation).
2) for DMatch
Class for matching keypoint descriptors: query descriptor index, train descriptor index, train image index, and distance between descriptors.
The problem is: Is the descriptor index the same as it's corresponding keypoint index, when as we can read the descriptors vector doesn't contain the data for each detected keypoint ?
I would check it for myself, but I have no idea how to do this... Maybe there is someone who also wondered about this issue.
My application is working and it looks fine, but it doesn't mean that it's ok...
Would somebody share their knowledge of OpenCV feature detection and extraction of fiducial markers?
I'm attempting to find a fiducial marker (see image below) (self-created ARTag-style using MS Paint) in a scene.
Using Harris corner detection, I can adequately locate the corners of the marker image. Similarly, using Harris corner detection, I can find most of the corners of the marker in the scene. I then use SIFT to extract descriptors for the marker image and the scene image. Then I've tried both BF and FLANN for feature matching. However, both matching algorithms tend to match the wrong corners together.
Is there something that I can do to improve the accuracy? Or are there other detection methods that would be better appropriate for this application?
Portion of code:
GoodFeaturesToTrackDetector harris_detector(6, 0.15, 10, 3, true);
vector<KeyPoint> keypoints1, keypoints2;
harris_detector.detect(im1, keypoints1);
harris_detector.detect(im2, keypoints2);
SiftDescriptorExtractor extractor;
Mat descriptors1, descriptors2;
extractor.compute( im1, keypoints1, descriptors1 );
extractor.compute( im2, keypoints2, descriptors2 );
BFMatcher matcher;
//FlannBasedMatcher matcher;
std::vector< DMatch > matches;
matcher.match( descriptors1, descriptors2, matches );
you can try to use ORB detector， which is a fusion of FAST keypoint detector and BRIEF descriptor. it is fast and better than BRIEF descriptor because the later does not compute the orientation.
you can found a example of orb's usage in samples/cpp/tutorial_code/features2D/AKAZE_tracking or enter link description here
or there is a python project which does the similar task as yours fiducial
I just wondered, if using a SurfFeatureDetector to detect keypoints and a SurfDescriptorExtractor to extract the SURF descriptors (see code below as described here) wouldn't extract the descriptors twice.
SurfFeatureDetector detector( minHessian );
std::vector<KeyPoint> keypoints;
detector.detect( img, keypoints ); //detecting keypoints, extracting descriptors without returning them
SurfDescriptorExtractor extractor;
Mat descriptors;
extractor.compute( img, keypoints, descriptors ); // extracting descriptors a second time
The openCV documentation says, those 2 classes are wrappers for the SURF() class.
The SURF::operator() is overloaded, one version taking just a keypoint vector, the other one additionally taking a vector for the descriptors.
What intrigues me... both then call the cvExtractSURF() function, which seems to extract the descriptors, no matter what... (I did not dive too deep into the C code as I find it hard to understand, so maybe I'm wrong)
But this would mean that the SurfFeatureDetector would extract descriptors without returning them. Using the SurfDescriptorExtractor in the next step just does it a second time, which seems very inefficient to me. But am I right?
You can be assured that detector does not actually compute the descriptors. The key statement to look at is line 687 of surf.cpp if( !descriptors ) continue; Features are not computed during detection, the way it should be. This kind of architecture is most likely due to the fact that surf code was "added" to OpenCV after it was designed/developed to work by itself.
As a background: note that detector and feature extractors are different things. You first "detect" points using SurfFeatureDetector where local features are extracted (using SurfDescriptorExtractor). The snippet you have is a good guide.