OpenCV drawMatches error - c++

My code consists of a section where I sort through a set of matches and define good matches based on distance. When I try to drawMatches, I receive an error:
OpenCV Error: Assertion failed (i1 >= 0 && i1 < static_cast<int>(keypoints1.size())) in drawMatches, file /home/user/OpenCV/opencv-2.4.10/modules/features2d/src/draw.cpp, line 207
terminate called after throwing an instance of 'cv::Exception'
what(): /home/user/OpenCV/opencv-2.4.10/modules/features2d/src/draw.cpp:207: error: (-215) i1 >= 0 && i1 < static_cast<int>(keypoints1.size()) in function drawMatches
draw.cpp file shows:
// draw matches
for( size_t m = 0; m < matches1to2.size(); m++ )
{
if( matchesMask.empty() || matchesMask[m] )
{
int i1 = matches1to2[m].queryIdx;
int i2 = matches1to2[m].trainIdx;
CV_Assert(i1 >= 0 && i1 < static_cast<int>(keypoints1.size()));
CV_Assert(i2 >= 0 && i2 < static_cast<int>(keypoints2.size()));
const KeyPoint &kp1 = keypoints1[i1], &kp2 = keypoints2[i2];
_drawMatch( outImg, outImg1, outImg2, kp1, kp2, matchColor, flags );
}
}
My drawMatches call follows:
Mat matchesImage;
drawMatches( im1, keypoints1, im2, keypoints2,
good_matches, matchesImage, Scalar::all(-1), Scalar::all(-1),
vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS );
Can anybody help explain this error to me?
Update:
Here is my code for the good_matches calculation
double min_dist = 10000;
double max_dist = 0;
//-- Quick calculation of max and min distances between keypoints
for( int i = 0; i < descriptors1.rows; i++ ) {
double dist = matches[i].distance;
if( dist < min_dist ) min_dist = dist;
if( dist > max_dist ) max_dist = dist;
}
printf("-- Max dist : %f \n", max_dist );
printf("-- Min dist : %f \n", min_dist );
//-- Draw only "good" matches
std::vector< DMatch > good_matches;
for( int i = 0; i < descriptors1.rows; i++ ) {
if( matches[i].distance <= max(2*min_dist, 0.02) ) {
good_matches.push_back( matches[i]);
}
}
for( int i = 0; i < (int)good_matches.size(); i++ ) {
printf( "-- Good Match [%d] Keypoint 1: %d -- Keypoint 2: %d \n",
i, good_matches[i].queryIdx, good_matches[i].trainIdx );
}
cout << "number of good matches: " << (int)good_matches.size() << endl;;
//Draw matches and save file
Mat matchesImage;
drawMatches( im1, keypoints1, im2, keypoints2,
good_matches, matchesImage, Scalar::all(-1), Scalar::all(-1),
vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS );
Update 2:
BFMatcher matcher(NORM_L2, true);
vector<DMatch> matches;
matcher.match(descriptors1, descriptors2, matches);

The problem has to do with the order you matched the points.
If you did, e.g.:
match(right_desc, left_desc)
function drawMatches will have to follow the same order. This will work (considering my matching example):
drawMatches(right_rgb, right_pts, left_rgb, left_pts, matches)
this will produce the error you have:
drawMatches(left_rgb, left_pts, right_rgb, right_pts, matches)
The order will also affect what is query and what is train (i.e. queryIdx and trainIdx), when you access the coordinates of the matches. Note that right_pts and left_pts are the keypoints described by right_desc and left_desc respectively.
Hope it helps someone.

I think you should clear the contents of the good_matches vector with the good_matches.clear() command for each iteration (if you use while(1) loop for get frame from camera, you will write good_matches.clear() command after while(1) loop:
while(1) {
good_matches.clear();
// other code ...
}

In general, good_matches is an array, which binds points from both keypoints1 and keypoints2 array. So point keypoints1[good_matches[m].queryIdx] corresponds to point keypoints2[good_matches[m].trainIdx].
As you can see, assertions in opencv code have sense.
It seems, the problem is in matches array.

I don't agree with answers above.
Parameters you pass into matcher.match(dscp1, dscp2, matches) and drawMatch(img1, kp1, img2, kp2, good_matches) are corresponding (they are both matching every descriptor in keypoints1 from keypoints2, keypoints1 is called query set while keypoints2 is called train set.)
The error might be caused by the wrong keypoints1 and keypoints2 you pass into drawMatch(). Check whether the index in good_matches correspond with kp in keypoints1. (If you reduce the number of keypoints while picking good matches, this occasion could happen.)

Related

Panoromic image construction

I am trying to construct a panoromic view from different images.
Initially I tried to stitch two images as part of panoromic construction.
The two input images I am trying to stitch are:
I used ORB feature descriptor to find features in the image,then I found out Homography matrix between these two images.
My code is:
int main(int argc, char **argv){
Mat img1 = imread(argv[1],1);
Mat img2 = imread(argv[2],1);
//-- Step 1: Detect the keypoints using orb Detector
std::vector<KeyPoint> kp2,kp1;
// Default parameters of ORB
int nfeatures=500;
float scaleFactor=1.2f;
int nlevels=8;
int edgeThreshold=15; // Changed default (31);
int firstLevel=0;
int WTA_K=2;
int scoreType=ORB::HARRIS_SCORE;
int patchSize=31;
int fastThreshold=20;
Ptr<ORB> detector = ORB::create(
nfeatures,
scaleFactor,
nlevels,
edgeThreshold,
firstLevel,
WTA_K,
scoreType,
patchSize,
fastThreshold );
Mat descriptors_img1, descriptors_img2;
//-- Step 2: Calculate descriptors (feature vectors)
detector->detect(img1, kp1,descriptors_img1);
detector->detect(img2, kp2,descriptors_img2);
Ptr<DescriptorExtractor> extractor = ORB::create();
extractor->compute(img1, kp1, descriptors_img1 );
extractor->compute(img2, kp2, descriptors_img2 );
//-- Step 3: Matching descriptor vectors using FLANN matcher
if ( descriptors_img1.empty() )
cvError(0,"MatchFinder","1st descriptor empty",__FILE__,__LINE__);
if ( descriptors_img2.empty() )
cvError(0,"MatchFinder","2nd descriptor empty",__FILE__,__LINE__);
descriptors_img1.convertTo(descriptors_img1, CV_32F);
descriptors_img2.convertTo(descriptors_img2, CV_32F);
FlannBasedMatcher matcher;
std::vector<DMatch> matches;
matcher.match(descriptors_img1,descriptors_img2,matches);
double max_dist = 0; double min_dist = 100;
//-- Quick calculation of max and min distances between keypoints
for( int i = 0; i < descriptors_img1.rows; i++ )
{
double dist = matches[i].distance;
if( dist < min_dist )
min_dist = dist;
if( dist > max_dist )
max_dist = dist;
}
printf("-- Max dist : %f \n", max_dist );
printf("-- Min dist : %f \n", min_dist );
//-- Draw only "good" matches (i.e. whose distance is less than 3*min_dist )
std::vector< DMatch > good_matches;
for( int i = 0; i < descriptors_img1.rows; i++ )
{
if( matches[i].distance < 3*min_dist )
{
good_matches.push_back( matches[i]);
}
}
Mat img_matches;
drawMatches(img1,kp1,img2,kp2,good_matches,img_matches,Scalar::all(-1),
Scalar::all(-1),vector<char>(),DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS );
std::vector<Point2f> obj;
std::vector<Point2f> scene;
for( int i = 0; i < good_matches.size(); i++ )
{
//-- Get the keypoints from the good matches
obj.push_back( kp1[ good_matches[i].queryIdx ].pt );
scene.push_back( kp2[ good_matches[i].trainIdx ].pt );
}
Mat H = findHomography( obj, scene, CV_RANSAC );
After wards some people told me to include the following code
cv::Mat result;
warpPerspective( img1, result, H, cv::Size( img1.cols+img2.cols, img1.rows) );
cv::Mat half(result, cv::Rect(0, 0, img2.cols, img2.rows) );
img2.copyTo(half);
imshow("result",result);
The result I got is
I also tried using inbuilt opencv stitch function. And I got the result
I am trying to implement stitch function so I dont want to use inbuilt opencv stitch function.
Can any one tell me where I went wrong and correct my code.Thanks in advance
Image stitching includes the following steps:
Feature finding
Find camera parameters
Warping
Exposure compensation
Seam Finding
Blending
You have to do all these steps in order to get the perfect result.
In your code you have only done the first part, that is feature finding.
You can find a detailed explanation on how image stitching works in Learn OpenCV
Also I have the code on Github
Hope this helps.

Solution for OpenCV Error: Unsupported format or combination of format - when matching ORB features with FlannBasedMatcher

I tried to find good matches using ORB.My code is as follows:
Ptr<FeatureDetector> detector = ORB::create();
Mat descriptors_img1, descriptors_img2;
//-- Step 2: Calculate descriptors (feature vectors)
detector->detect(img1, kp1,descriptors_img1);
detector->detect(img2, kp2,descriptors_img2);
Ptr<DescriptorExtractor> extractor = ORB::create();
extractor->compute(img1, kp1, descriptors_img1 );
extractor->compute(img2, kp2, descriptors_img2 );
//-- Step 3: Matching descriptor vectors using FLANN matcher
descriptors_img1.convertTo(descriptors_img1, CV_32F);
descriptors_img2.convertTo(descriptors_img2, CV_32F);
FlannBasedMatcher matcher;
std::vector<DMatch> matches;
matcher.match(descriptors_img1,descriptors_img2,matches);
double max_dist = 0; double min_dist = 100;
//-- Quick calculation of max and min distances between keypoints
for( int i = 0; i < descriptors_img1.rows; i++ )
{
double dist = matches[i].distance;
if( dist < min_dist )
min_dist = dist;
if( dist > max_dist )
max_dist = dist;
}
printf("-- Max dist : %f \n", max_dist );
printf("-- Min dist : %f \n", min_dist );
//-- Draw only "good" matches (i.e. whose distance is less than 3*min_dist )
std::vector< DMatch > good_matches;
for( int i = 0; i < descriptors_img1.rows; i++ )
{
if( matches[i].distance < 3*min_dist )
{
good_matches.push_back( matches[i]);
}
}
Mat img_matches;
drawMatches(img1,kp1,img2,kp2,good_matches,img_matches,Scalar::all(-1),
Scalar::all(-1),vector<char>(),DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS );
//-- Show detected matches
imshow( "Good Matches", img_matches );
But when I run it, I get error saying:
OpenCV Error: Unsupported format or combination of formats (type=0
) in buildIndex_, file /home/opencv-3.2.0/modules/flann/src/miniflann.cpp, line 315
terminate called after throwing an instance of 'cv::Exception'
what(): /home/opencv-3.2.0/modules/flann/src/miniflann.cpp:315: error: (-210) type=0 in function buildIndex_
I looked at similar questions ,but i didnt find my answer.After debugging I came to know that error is at
matcher.match(....);
Please help me in fixing this out.Thanks in advance
I solved this error.
You just modify the code
Ptr<ORB> detector = ORB::create()
instead of
Ptr<FeatureDetector> detector = ORB::create();
Then it worked for me.

How to match two different image in C++

I'm trying to reconstruct a 3D model of a anatomical structure. So I want to match key points in pair of X ray images. I tried it by using following code. But it didn't give correct results.
Mat tmp = cv::imread( "1.jpg", 1 );
Mat in = cv::imread( "2.jpg", 1 );
cv::SiftFeatureDetector detector( 0.0001, 1.0 );
cv::SiftDescriptorExtractor extractor;
vector<KeyPoint> keypoints1, keypoints2;
detector.detect( tmp, keypoints1 );
detector.detect( in, keypoints2 );
Mat feat1,feat2;
drawKeypoints(tmp,keypoints1,feat1,Scalar(255, 255, 255),DrawMatchesFlags::DRAW_RICH_KEYPOINTS);
drawKeypoints(in,keypoints2,feat2,Scalar(255, 255, 255),DrawMatchesFlags::DRAW_RICH_KEYPOINTS);
imwrite( "feat1.bmp", feat1 );
imwrite( "feat2.bmp", feat2 );
int key1 = keypoints1.size();
int key2 = keypoints2.size();
printf("Keypoint1=%d \nKeypoint2=%d", key1, key2);
Mat descriptor1,descriptor2;
extractor.compute( tmp, keypoints1, descriptor1 );
extractor.compute( in, keypoints2, descriptor2 );
BruteForceMatcher<L2<float> > matcher;
std::vector< DMatch > matches;
matcher.match( descriptor1, descriptor2, matches );
double max_dist = 0; double min_dist = 100;
Mat img_matches;
for( int i = 0; i < descriptor1.rows; i++ )
{ double dist = matches[i].distance;
if( dist < min_dist ) min_dist = dist;
if( dist > max_dist ) max_dist = dist;
}
printf("-- Max dist : %f \n", max_dist );
printf("-- Min dist : %f \n", min_dist );
std::vector< DMatch > good_matches;
for( int i = 0; i < descriptor1.rows; i++ )
{ if( matches[i].distance <= max(2*min_dist, 0.03) )
{ good_matches.push_back( matches[i]); }
}
drawMatches( tmp, keypoints1, in, keypoints2,
good_matches, img_matches, Scalar::all(-1), Scalar::all(-1),
vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS );
namedWindow("SIFT", CV_WINDOW_AUTOSIZE );
imshow("SIFT", img_matches);
imwrite("sift_1.jpg",img_matches);
waitKey(0);
return 0;
These are the two images
This is what i got from this code
This is very close to my expected result but it also matching wrong points. This shows few points but i need more points.
Feature detectors like SIFT or SURF are designed to work and match images that have a rich and distinctive texture. They are not designed to work with very spares binary inputs like your examples.
You might want to try them on the original X-Rays for more image context.
Alternatively, you might try a more direct global alignment model between the images.
Check out this link for some options for alignment with the findTransformECC() function.
Also see the article here.
I think you may try to use ITK, ITK is designed to complete image registration with 2D or 3D images.

C++ - OpenCV feature detection with ORB

I'm trying to extract and match features with OpenCV using ORB for detecting and FLANN for matching, and i get a really weird result. After loading my 2 images and converting them to grayscale, here's my code:
// Initiate ORB detector
Ptr<FeatureDetector> detector = ORB::create();
// find the keypoints and descriptors with ORB
detector->detect(gray_image1, keypoints_object);
detector->detect(gray_image2, keypoints_scene);
Ptr<DescriptorExtractor> extractor = ORB::create();
extractor->compute(gray_image1, keypoints_object, descriptors_object );
extractor->compute(gray_image2, keypoints_scene, descriptors_scene );
// Flann needs the descriptors to be of type CV_32F
descriptors_scene.convertTo(descriptors_scene, CV_32F);
descriptors_object.convertTo(descriptors_object, CV_32F);
FlannBasedMatcher matcher;
vector<DMatch> matches;
matcher.match( descriptors_object, descriptors_scene, matches );
double max_dist = 0; double min_dist = 100;
//-- Quick calculation of max and min distances between keypoints
for( int i = 0; i < descriptors_object.rows; i++ )
{
double dist = matches[i].distance;
if( dist < min_dist ) min_dist = dist;
if( dist > max_dist ) max_dist = dist;
}
//-- Use only "good" matches (i.e. whose distance is less than 3*min_dist )
vector< DMatch > good_matches;
for( int i = 0; i < descriptors_object.rows; i++ )
{
if( matches[i].distance < 3*min_dist )
{
good_matches.push_back( matches[i]);
}
}
vector< Point2f > obj;
vector< Point2f > scene;
for( int i = 0; i < good_matches.size(); i++ )
{
//-- Get the keypoints from the good matches
obj.push_back( keypoints_object[ good_matches[i].queryIdx ].pt );
scene.push_back( keypoints_scene[ good_matches[i].trainIdx ].pt );
}
// Find the Homography Matrix
Mat H = findHomography( obj, scene, CV_RANSAC );
// Use the Homography Matrix to warp the images
cv::Mat result;
warpPerspective(image1,result,H,Size(image1.cols+image2.cols,image1.rows));
cv::Mat half(result,cv::Rect(0,0,image2.cols,image2.rows));
image2.copyTo(half);
imshow( "Result", result );
And this is a screen shot of the weird result i'm getting:
screen shot
What might be the problem?
Thanks!
You are experiencing the results of a bad matching: The homography which fits the data is not "realistic" and thus distorts the image.
You can debug your matching with imshow( "Good Matches", img_matches ); as done in the example.
There are multiple approaches to improve your matches:
Use the crossCheck option
Use the SIFT ratio test
Use the OutputArray mask in cv::findHompgraphy to identify totally wrong homography computations
... and so on...
ORB are binary feature vectors which don't work with Flann. Use Brute Force (BFMatcher) instead.

ORB/BruteForce-drawing matches when there are none

I'm trying to write a program that uses ORB algorithm to detect and compute the keypoints of an image and a video and matches descriptor vectors using BruteForce matcher. The issue I am facing is, that every time I run the program on Visual C++, when the object that I'm trying to detect is not visible, my algorithm is drawing all the supposed matching lines between the keypoints detected(it matches all the keypoints). When the object that I'm trying to detect appears in the image I don't face this issue, in fact, I hardly get any mismatches.
This is a brief sequence of the main test:
• convert input image to grayscale
• convert input videos to grayscale
• detect keypoints and extract descriptors from input grayscale image
• detect keypoints and extract descriptors from input grayscale videos
• match descriptors(see below)
BFMatcher matcher(NORM_HAMMING);
vector<DMatch> matches;
matcher.match(descriptors_1, descriptors_2, matches);
double max_dist = 0; double min_dist = 100;
////calcularea distantelor max si min distances intre keypoints
for (int i = 0; i < descriptors_1.rows; i++)
{
double dist = matches[i].distance;
if (dist < min_dist) min_dist = dist;
if (dist > max_dist) max_dist = dist;
}
printf("-- Max dist : %f \n", max_dist);
printf("-- Min dist : %f \n", min_dist);
std::vector< DMatch > good_matches;
for (int i = 0; i < descriptors_1.rows; i++)
{
if (matches[i].distance <= max(2 * min_dist, 0.02))
{
good_matches.push_back(matches[i]);
}
}
////-- Desenarea matches-urilor "bune"
Mat img_matches;
drawMatches(img1, keypoints_1, cadruProcesat, keypoints_2,
good_matches, img_matches, Scalar::all(-1), Scalar::all(-1),
vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS);
////-- Afisare matches
imshow("good matches", img_matches);
int gm = 0;
for (int i = 0; i < (int)good_matches.size(); i++)
{
printf("-- Good Match [%d] Keypoint 1: %d -- Keypoint 2: %d \n", i, good_matches[i].queryIdx, good_matches[i].trainIdx);
gm += 1;
}
printf("%d",gm);
///////////////////////////////////////////////////////////////////
//imshow(windowName2, cadruProcesat);
switch (waitKey(10)) {
case 27:
//tasta 'esc' a fost apasata(ASCII 27)
return 0;
}
}
return 0;
Please help me find the problem.
So I've changed the value of min_dist to 15(not 100) and it seems that it works very well for patterns... but this was done by trial and error...