I'm creating the visual histograms (following the Bag of Visual Words model) of a dataset of n images using OpenCV.
This is the code:
std::vector<std::string> fileList;
std::unique_ptr<cv::BOWImgDescriptorExtractor> bowDE;
cv::Mat vocabulary;//will be filled through KMeans
...
bowDE->setVocabulary(vocabulary);
cv::Ptr<cv::Feature2D> featureDetectorDescriptor = cv::xfeatures2d::SIFT::create(0,3,0.04,10,1.6);;
for(size_t i=0;i<n;i++) {
cv::UMat img;
cv::Mat word;
imread(fileList[i], cv::IMREAD_GRAYSCALE).copyTo(img);
std::vector<cv::KeyPoint> keyPoints;
featureDetectorDescriptor->detect(img, keyPoints);
bowDE->compute(img, keyPoints, word);
histograms.push_back(word);
assert(histograms.rows==(i+1));//this is for testing
}
The problem is that after 4625 images have been inserted, the assert condition is false. I tested keypoints.size() and it results 0, no keypoints are detected!
I tried to remove the image which created the error, but another one creates the same error (so the error seems not to depend from the image).
Why this happens?
Update: actually I found out that the error was image dependent. It seems that with the default parameters cv::SIFT detects 0 keypoints with this image from Caltech101:
Related
I'm trying to use OpenCV's FAST corner detection algorithm to get an outline of an image of a ball (Not my final project, I'm using it as a simple example). For some reason, it only works on a third of the input Mat, and stretches the Keypoints across the image. I'm not sure as to what could be going wrong here to make the FAST algorithm not apply to the entire Mat.
Code:
void featureDetection(const Mat& imgIn, std::vector<KeyPoint>& pointsOut) {
int fast_threshold = 20;
bool nonmaxSuppression = true;
FAST(imgIn, pointsOut, fast_threshold, nonmaxSuppression);
}
int main(int argc, char** argv) {
Mat out = imread("ball.jpg", IMREAD_COLOR);
// Detect features
std::vector<KeyPoint> keypoints;
featureDetection(out.clone(), keypoints);
Mat out2 = out.clone();
// Draw features (Normal, missing right side)
for(KeyPoint p : keypoints) {
drawMarker(out, Point(p.pt.x / 3, p.pt.y), Scalar(0, 255, 0));
}
imwrite("out.jpg", out, std::vector<int>(0));
// Draw features (Stretched)
for(KeyPoint p : keypoints) {
drawMarker(out2, Point(p.pt.x, p.pt.y), Scalar(127, 0, 255));
}
imwrite("out2.jpg", out2, std::vector<int>(0));
}
Input image
Output 1 (keypoint.x multiplied by a factor of 1/3, but missing right side)
Output 2 (Coordinates untouched)
I'm using OpenCV 4.5.4 on MinGW.
Most keypoint detectors use grayscale images as input.
If you interpret the memory of a bgr image as grayscale, you will have 3 times the number of pixels. Y axis is still ok if the algorithm uses the width-offset per row, which most algorithms do (because this is useful when subimaging or padding is used).
I don't know whether it is a bug or a feature, that FAST doesn't check for the number of channels snd doesnt throw an exception if the wrong number of channels ist given.
You can convert the image to grayscale by cv::cvtColor with the flag cv:: COLOR_BGR2GRAY
I am trying to extract features of an image using SIFT in opencv 4.5.1, but when I try to check the result by using drawKeypoints() I keep getting this cryptic error:
OpenCV(4.5.1) Error: Assertion failed (!fixedType() || ((Mat*)obj)->type() == mtype) in cv::debug_build_guard::_OutputArray::create, file C:\build\master_winpack-build-win64-vc14\opencv\modules\core\src\matrix_wrap.cpp, line 1147
D:\School\IP2\OpenCVApplication-VS2019_OCV451_basic\x64\Debug\OpenCVApplication.exe (process 6140) exited with code -1.
To automatically close the console when debugging stops, enable Tools->Options->Debugging->Automatically close the console when debugging stops.
The problem seems to be with the drawKeypoints() function but I'm not sure what causes the problem.
The function:
vector<KeyPoint> extractFeatures(String path) {
Mat_<uchar> source = imread(path, 0);
Mat_<uchar> output(source.rows, source.cols);
vector<KeyPoint> keypoints;
Ptr<SIFT> sift = SIFT::create();
sift->detect(source, keypoints);
drawKeypoints(source, keypoints, output);
imshow("sift_result", output);
return keypoints;
}
You are getting a exception because output argument of drawKeypoints must be 3 channels colored image, and you are initializing output to 1 channel (grayscale) image.
When using: Mat output(source.rows, source.cols); or Mat output;, the drawKeypoints function creates a new colored matrix automatically.
When using the derived template matrix class Mat_<uchar>, the function drawKeypoints raises an exception!
You may replace: Mat_<uchar> output(source.rows, source.cols); with:
Mat_<Vec3b> output(source.rows, source.cols); //Create 3 color channels image (matrix).
Note:
You may also use Mat instead of Mat_:
Mat output; //The matrix is going to be dynamically allocated inside drawKeypoints function.
Note:
My current OpenCV version (4.2.0) has no SIFT support, so I used ORB instead (for testing).
Here is the code sample used for testing:
#include "opencv2/opencv.hpp"
using namespace cv;
using namespace std;
int main()
{
Mat_<uchar> source = imread("graf.png", 0);
Mat_<Vec3b> output(source.rows, source.cols); //Create 3 color channels image (matrix).
vector<KeyPoint> keypoints;
Ptr<ORB> orb = ORB::create();
orb->detect(source, keypoints);
drawKeypoints(source, keypoints, output);
imshow("orb_result", output);
waitKey(0);
destroyAllWindows();
return 0;
}
Result:
I am using ORB detector for detecting the keypoints of frame of a video but it gives me the following error:
OpenCV Error: Assertion failed (img.type() == (((0) & ((1 << 3) - 1)) + (((1)-1) << 3))) in detectAsync
The CPU functions for ORB detector works fine. But somehow for GPU it is not able to detect the keypoints of the image. I have also tried using FastFeatureDetector from CUDA but it also fails.
I am attaching my code below:
int main()
{
cv::VideoCapture input("/home/admin/Pictures/cars.mp4");
// Create image matrix
cv::Mat img,desc;
cv::cuda::GpuMat obj1;
// create vector keypoints
std::vector<cv::KeyPoint>keypoints;
// create keypoint detector
cv::Ptr<cv::cuda::ORB> detector = cv::cuda::ORB::create();
for(;;)
{
if(!input.read(img))
break;
obj1.upload(img);
detector->detect(obj1,keypoints, cv::cuda::GpuMat());
obj1.download(desc);
//Create cricle at keypoints
for(size_t i=0; i<keypoints.size(); i++)
cv::circle(desc, keypoints[i].pt, 2,cv::Scalar(0,0,255),1);
// Display Image
cv::imshow("img", desc);
char c = cv::waitKey();
// NOTE: Press any key to run the next frame
// Wait for a key to press
if(c == 27) // 27 is ESC key code
break;
}
}
The main problem is that the detector takes CV_8UC1 as the input format for Mat. But the format of my image is CV_8UC3. I have tried converting that image using img.convertTo(img1, CV_8UC1)but still it is unable to process and throws me the error OpenCV Error: Bad flag (parameter or structure field) (Unrecognized or unsupported array type) in cvGetMat.
You have to convert your image from CV_8UC3 - 3 channel image to CV_8UC1 - single channel (grayscale) image. To do so simply call
cv::cvtColor(img, img, cv::BGR2GRAY);
prior to uploading data to the GPU.
I am using Visual Studio 2010, with OpenCV 3.0. I'm trying to train a SVM and to save it into a file, but I am having problems.
My purpose is to extract the HOG features of some images and train a SVM with them. All seems to be right, but when I try to save the model in a xml file I obtain the following error:
Unhandled exception in 0x000007fefd9bb16d (KernelBase.dll) in TrainSVM.exe: Exception de MICROSOFT C++: cv::Exception at memory location 0x0026e1b0.
And then this is showed in console:
OpenCV Error: Parsing error (SVM model data is invalid, check sv_count, var_* an
d class_count tags) in cv::ml::SVMImpl::write, file C:\builds\master_PackSlave-w
in64-vc12-shared\opencv\modules\ml\src\svm.cpp, line 2027
The error seems to appear when the SVM has not been trained properly, but I don't understand where I have failed, because the line
svm->train(auxResult)
has "true" as result.
I have checked the images and they are loaded properly, anybody could help me?
Thanks in advance.
Here is the code:
String imagesPathPos = "Positivas/*.jpg"; // it has filters, too !
vector<String> fp;
glob(imagesPathPos, fp);
int tamaño = fp.size();
std::vector<cv::Point> positions;
positions.push_back(cv::Point(0,0));
std::vector<float> descriptor;
Ptr<TrainData> auxResult;
for (size_t i=0; i<fp.size(); ++i)
{
string nameFile = fp[i];
Mat img = imread(fp[i]);
cv::Mat grayImg;
cvtColor( img, grayImg, COLOR_BGR2GRAY );
hog.compute(grayImg,descriptor,winStride,trainingPadding,positions);
Mat auxDescriptor = cv::Mat(descriptor);
Mat descriptorMat(1,auxDescriptor.rows,CV_32FC1);
transpose(auxDescriptor, descriptorMat);
trainingData.push_back(descriptorMat);
trainingLabels.push_back(labelPositive);
}
String imagesPathNeg = "Negativas/*.jpg";
vector<String> fn;
glob(imagesPathNeg, fn, true);
for (size_t i=0; i<fn.size(); i++)
{
Mat img = imread(fn[i]);
cv::Mat grayImg;
cvtColor( img, grayImg, COLOR_BGR2GRAY );
hog.compute(grayImg,descriptor,Size(),Size(),positions);
Mat auxDescriptor = cv::Mat(descriptor);
Mat descriptorMat(1,auxDescriptor.rows,CV_32FC1);
transpose(auxDescriptor, descriptorMat);
trainingData.push_back(descriptorMat);
trainingLabels.push_back(labelPositive);
}
auxResult = TrainData::create(trainingData, type, trainingLabels);
svm->train(auxResult);
svm->save("output.xml");
You are defining "labelPositive" even when the images are negative. Possibly the error is there, within the loop through the vector fn:
trainingLabels.push_back(labelPositive);
You should use a parameter named "labelNegative" defined as -1.
Does someone know of the link to BRIEF implementation with OpenCV 2.4? Regards.
PS: I know such questions are generally not welcome on SO, as the primary focus is what work you have done. But there was a similar question which was quite well received.
One of the answers to that questions suggests a generic manner for SIFT, which could be extended to BRIEF. Here is my slightly modified code.
#include <opencv2/nonfree/nonfree.hpp>
#include <opencv2/highgui/highgui.hpp>
//using namespace std;
using namespace cv;
int main(int argc, char *argv[])
{
Mat image = imread("load02.jpg", CV_LOAD_IMAGE_GRAYSCALE);
cv::initModule_nonfree();
// Create smart pointer for SIFT feature detector.
Ptr<FeatureDetector> featureDetector = FeatureDetector::create("HARRIS"); // "BRIEF was initially written. Changed after answer."
vector<KeyPoint> keypoints;
// Detect the keypoints
featureDetector->detect(image, keypoints); // NOTE: featureDetector is a pointer hence the '->'.
//Similarly, we create a smart pointer to the SIFT extractor.
Ptr<DescriptorExtractor> featureExtractor = DescriptorExtractor::create("BRIEF");
// Compute the 128 dimension SIFT descriptor at each keypoint.
// Each row in "descriptors" correspond to the SIFT descriptor for each keypoint
Mat descriptors;
featureExtractor->compute(image, keypoints, descriptors);
// If you would like to draw the detected keypoint just to check
Mat outputImage;
Scalar keypointColor = Scalar(255, 0, 0); // Blue keypoints.
drawKeypoints(image, keypoints, outputImage, keypointColor, DrawMatchesFlags::DEFAULT);
namedWindow("Output");
imshow("Output", outputImage);
char c = ' ';
while ((c = waitKey(0)) != 'q'); // Keep window there until user presses 'q' to quit.
return 0;
}
The issue with this code is that it gives an error: First-chance exception at 0x00007FFB84698B9C in Project2.exe: Microsoft C++ exception: cv::Exception at memory location 0x00000071F4FBF8E0.
The error results in the function execution breaking. A tag says that execution will resume at the namedWindow("Output"); line.
Could someone please help fix this issue, or suggest a new code altogether? Thanks.
EDIT: The terminal now shows an error: Assertion failed (!outImage.empty()) in cv::drawKeypoints, file ..\..\..\..opencv\modules\features2d\src\draw.cpp, line 115. The next statement from where the code will resume remains the same, as drawKepoints is called just before it.
In OpenCV, BRIEF is a DescriptorExtractor, not a FeatureDetector. According to FeatureDetector::create, this factory method does not support "BRIEF" algorithm. In other words, FeatureDetector::create("BRIEF") returns a null pointer and your program crashes.
The general steps in feature matching are:
Find some interesting (feature) points in an image: FeatureDetector
Find a way to describe those points: DescriptorExtractor
Try to match descriptors (feature vectors) in two images: DescriptorMatcher
BRIEF is an algorithm only for step 2. You can use some other methods, HARRIS, ORB, ..., in step 1 and pass the result to step 2 using BRIEF. Besides, SIFT can be used in both step 1 and 2 because the algorithm provides methods for both steps.
Here's a simple example to use BRIEF in OpenCV. First step, find points that looks interesting (key points) in an image:
vector<KeyPoint> DetectKeyPoints(const Mat &image)
{
auto featureDetector = FeatureDetector::create("HARRIS");
vector<KeyPoint> keyPoints;
featureDetector->detect(image, keyPoints);
return keyPoints;
}
You can try any FeatureDetector algorithm instead of "HARRIS". Next step, compute the descriptors from key points:
Mat ComputeDescriptors(const Mat &image, vector<KeyPoint> &keyPoints)
{
auto featureExtractor = DescriptorExtractor::create("BRIEF");
Mat descriptors;
featureExtractor->compute(image, keyPoints, descriptors);
return descriptors;
}
You can use algorithm different than "BRIEF", too. And you can see that the algorithms in DescriptorExtractor is not the same as the algorithms in FeatureDetector. The last step, match two descriptors:
vector<DMatch> MatchTwoImage(const Mat &descriptor1, const Mat &descriptor2)
{
auto matcher = DescriptorMatcher::create("BruteForce");
vector<DMatch> matches;
matcher->match(descriptor1, descriptor2, matches);
return matches;
}
Similarly, you can try different matching algorithm other than "BruteForce". Finally back to main program, you can build the application from those functions:
auto img1 = cv::imread("image1.jpg");
auto img2 = cv::imread("image2.jpg");
auto keyPoints1 = DetectKeyPoints(img1);
auto keyPoints2 = DetectKeyPoints(img2);
auto descriptor1 = ComputeDescriptors(img1, keyPoints1);
auto descriptor2 = ComputeDescriptors(img2, keyPoints2);
auto matches = MatchTwoImage(descriptor1, descriptor2);
and use matches vector to complete your application. If you want to check the results, OpenCV also provides functions to draw results of step 1 & 3 in an image. For example, draw the matches in the final step:
Mat result;
drawMatches(img1, keyPoints1, img2, keyPoints2, matches, result);
imshow("result", result);
waitKey(0);