First some background
I have written a C++ function that detect an area of a certain color in an RGB image using OpenCV. The function is used to isolate a small colored area using the FeatureDetector: SimpleBlobDetector.
The problem I have is that this function is used in a crossplatform project. On my OSX 10.8 machine using OpenCV in Xcode this works flawlessly. However when I try to run the same piece of code on Windows using OpenCV in Visual Studio, this code crashes whenever I use:
blobDetector.detect(imgThresh, keypoints)
with an error such as this:
OpenCV Error: Assertion failed (dims <= 2 && data && (unsigned)i0 < (unsigned)size.p[0] && (unsigned)(i1*DataType<_Tp>::channels) < (unsigned)(size.p[1]*channels()) && ((((sizeof(size_t)<<28)|0x8442211) >> ((DataType<_Tp>::depth) & ((1 << 3) - 1))*4) & 15) == elemSize1()) in unknown function, file C:\slave\builds\WinInstallerMegaPack\src\opencv\modules\core\include\opencv2/core/mat.hpp, line 545
This is the only piece of OpenCV code that have given me problems so far. I tried several solutions like the ones suggested here Using FeatureDetector in OpenCV gives access violation and Access violation reading in FeatureDetector OpenCV 2.4.5 . But to no avail.
A somewhat solution to my problem was to add a threshold() call just before my call to .detect(), which appears to make it work. However I don't like this solution as it forces me to do something I don't have to (as far as I know) and because it is not necessary to do on my Mac for some reason.
Question
Can anyone explain why the following line:
threshold(imgThresh, imgThresh, 100, 255, 0);
is necessary on Windows, but not on OSX, just before the call to .detect() in the following code?
Full code snippet:
#include "ColorDetector.h"
using namespace cv;
using namespace std;
Mat ColorDetection(Mat img, Scalar colorMin, Scalar colorMax, double alpha, int beta)
{
initModule_features2d();
initModule_nonfree();
//Define matrices
Mat contrast_img = constrastImage(img, alpha, beta);
Mat imgThresh;
Mat blob;
//Threshold based on color ranges (Blue/Green/Red scalars)
inRange(contrast_img, colorMin, colorMax, imgThresh); //BGR range
//Apply Blur effect to make blobs more coherent
GaussianBlur(imgThresh, imgThresh, Size(3,3), 0);
//Set SimpleBlobDetector parameters
SimpleBlobDetector::Params params;
params.filterByArea = false;
params.filterByCircularity = false;
params.filterByConvexity = false;
params.filterByInertia = false;
params.filterByColor = true;
params.blobColor = 255;
params.minArea = 100.0f;
params.maxArea = 500.0f;
SimpleBlobDetector blobDetector(params);
blobDetector.create("SimpleBlob");
//Vector to store keypoints (center points for a blob)
vector<KeyPoint> keypoints;
//Try blob detection
threshold(imgThresh, imgThresh, 100, 255, 0);
blobDetector.detect(imgThresh, keypoints);
//Draw resulting keypoints
drawKeypoints(img, keypoints, blob, CV_RGB(255,255,0), DrawMatchesFlags::DEFAULT);
return blob;
}
Try using it that way:
Ptr<SimpleBlobDetector> sbd = SimpleBlobDetector::create(params);
vector<cv::KeyPoint> keypoints;
sbd->detect(imgThresh, keypoints);
Related
I am trying to extract features of an image using SIFT in opencv 4.5.1, but when I try to check the result by using drawKeypoints() I keep getting this cryptic error:
OpenCV(4.5.1) Error: Assertion failed (!fixedType() || ((Mat*)obj)->type() == mtype) in cv::debug_build_guard::_OutputArray::create, file C:\build\master_winpack-build-win64-vc14\opencv\modules\core\src\matrix_wrap.cpp, line 1147
D:\School\IP2\OpenCVApplication-VS2019_OCV451_basic\x64\Debug\OpenCVApplication.exe (process 6140) exited with code -1.
To automatically close the console when debugging stops, enable Tools->Options->Debugging->Automatically close the console when debugging stops.
The problem seems to be with the drawKeypoints() function but I'm not sure what causes the problem.
The function:
vector<KeyPoint> extractFeatures(String path) {
Mat_<uchar> source = imread(path, 0);
Mat_<uchar> output(source.rows, source.cols);
vector<KeyPoint> keypoints;
Ptr<SIFT> sift = SIFT::create();
sift->detect(source, keypoints);
drawKeypoints(source, keypoints, output);
imshow("sift_result", output);
return keypoints;
}
You are getting a exception because output argument of drawKeypoints must be 3 channels colored image, and you are initializing output to 1 channel (grayscale) image.
When using: Mat output(source.rows, source.cols); or Mat output;, the drawKeypoints function creates a new colored matrix automatically.
When using the derived template matrix class Mat_<uchar>, the function drawKeypoints raises an exception!
You may replace: Mat_<uchar> output(source.rows, source.cols); with:
Mat_<Vec3b> output(source.rows, source.cols); //Create 3 color channels image (matrix).
Note:
You may also use Mat instead of Mat_:
Mat output; //The matrix is going to be dynamically allocated inside drawKeypoints function.
Note:
My current OpenCV version (4.2.0) has no SIFT support, so I used ORB instead (for testing).
Here is the code sample used for testing:
#include "opencv2/opencv.hpp"
using namespace cv;
using namespace std;
int main()
{
Mat_<uchar> source = imread("graf.png", 0);
Mat_<Vec3b> output(source.rows, source.cols); //Create 3 color channels image (matrix).
vector<KeyPoint> keypoints;
Ptr<ORB> orb = ORB::create();
orb->detect(source, keypoints);
drawKeypoints(source, keypoints, output);
imshow("orb_result", output);
waitKey(0);
destroyAllWindows();
return 0;
}
Result:
By using OpenCV version 4.2.0 in c++ (VS 2019) I created project which performs face detection on the given image. I used Opencv's DNN face detector which uses res10_300x300_ssd_iter_140000_fp16.caffemodel model to detect faces. Below is the code of that function:
//variables which are used in function
const double inScaleFactor = 1.0;
const cv::Scalar meanVal = cv::Scalar(104.0, 177.0, 123.0);
const size_t inWidth = 300;
const size_t inHeight = 300;
std::vector<FaceDetectionResult> namespace_name::FaceDetection::detectFaceByOpenCVDNN(std::string filename, FaceDetectionModel model)
{
Net net;
cv::Mat frame = cv::imread(filename);
cv::Mat inputBlob;
std::vector<FaceDetectionResult> vec;
if (frame.empty())
throw std::exception("provided image file is not found or unable to open.");
int frameHeight = frame.rows;
int frameWidth = frame.cols;
if (model == FaceDetectionModel::CAFFE)
{
net = cv::dnn::readNetFromCaffe(caffeConfigFile, caffeWeightFile);
inputBlob = cv::dnn::blobFromImage(frame, inScaleFactor, cv::Size(inWidth, inHeight), meanVal, false, false);
}
else
{
net = cv::dnn::readNetFromTensorflow(tensorflowWeightFile, tensorflowConfigFile);
inputBlob = cv::dnn::blobFromImage(frame, inScaleFactor, cv::Size(inWidth, inHeight), meanVal, true, false);
}
net.setInput(inputBlob, "data");
cv::Mat detection = net.forward("detection_out");
cv::Mat detectionMat(detection.size[2], detection.size[3], CV_32F, detection.ptr<float>());
for (int i = 0; i < detectionMat.rows; i++)
{
if (detectionMat.at<float>(i, 2) >= 0.5)
{
FaceDetectionResult res;
res.faceDetected = true;
res.confidence = detectionMat.at<float>(i, 2);
res.x1 = static_cast<int>(detectionMat.at<float>(i, 3) * frameWidth);
res.y1 = static_cast<int>(detectionMat.at<float>(i, 4) * frameHeight);
res.x2 = static_cast<int>(detectionMat.at<float>(i, 5) * frameWidth);
res.y2 = static_cast<int>(detectionMat.at<float>(i, 6) * frameHeight);
vec.push_back(res);
}
#ifdef aDEBUG
else
{
cout << detectionMat.at<float>(i, 2) << endl;
}
#endif
}
return vec;
}
In the above code, after face detection I assign confidence and co-ordinates of face detected in custom class FaceDetectionResult, which a simple class having bool and int,float members as required.
Function detect faces in the given image, but while playing with this I am doing comparison with dlib's HOG+SVM face detector, So first I am doing face detection by dlib and then same image path is passed to this function.
I found some images where dlib can easily find faces in the image but opencv didn't find a single face, for example look at below image:
As you can see HOG+SVM detected 46 faces in approx 3 sec., If I pass this same image to above function then opencv did not detect a single face in it. Why? Do I need any enhancements in above code? I am not saying that function does not detect faces for any image, it does, but for some images (like above) it could not not.
For ref:
I used https://pastebin.com/9rt9reNY this python program to detect faces using dlib.
After a deep search, unfortunately I couldn't find a good explanation to this problem. The reason why I tried to crop image is that I assumed there can be a maximum detected face number limit. It is also not about occlusion.
I tried some image examples which includes more than 20(appx.) faces and the results were the same but when I cropped those images(decrease the number of faces), program was able to find the faces.This is also not about the resolution(sizes) of the image because the images I tried had different sizes.
I also changed and tried the all parameters(iteration number, confidentThreshold etc.) but the result still wasn't the desired one.
My assumption but not the answer:
The program doesn't let to find the faces if image includes more than a maximum number(approximately 20)
As a solution for this question, we can divide the source image into 2 parts and find the rectangles for each one then can be pasted to source image.
Note: After digging deeply on the internet, I couldnt find a topic related to this problem. I am also curious about the main reason causes this issue so any help will be appreciated. This post only includes my experiences and assumptions.
change this line :
inputBlob = cv::dnn::blobFromImage(frame, inScaleFactor, cv::Size(inWidth, inHeight), meanVal, false, false);
by this line :
frameHeightinputBlob = cv::dnn::blobFromImage(frame, inScaleFactor, cv::Size(inWidth, inHeight), meanVal, false, false);
I have a simple task for OpenCV SimpleBlobDetector
cv::SimpleBlobDetector::Params params;
cv::Ptr<cv::SimpleBlobDetector> detector = cv::SimpleBlobDetector::create(params);
std::vector<cv::KeyPoint> keypoints;
detector->detect(crop, keypoints);
drawKeypoints(crop, keypoints, crop, cv::Scalar(0, 0, 255), cv::DrawMatchesFlags::DRAW_RICH_KEYPOINTS);
cv::imshow("crop", crop);
cv::waitKey(0);
It is not detecting half of the blobs in my image.
Please see picture below,
I tried adding parameters and varying them, at no point has it ever detected every single blob.
Blob detection is a simple and straightforward algorithm that should be completely refined in every image processing API. Is this not the case with OpenCV?
//params.minThreshold = 0;
//params.maxThreshold = 255;
//params.filterByArea = true;
//params.minArea = 1000;
//params.maxArea = 5000;
//params.filterByCircularity = true;
//params.minCircularity = 0.4;
//params.filterByConvexity = true;
//params.minConvexity = 0.87;
//params.filterByInertia = true;
//params.minInertiaRatio = 0.71;
I'm using either OpenCV 3.3 or 3.2, I can't seem to find the version number in the sources
Im not sure if this is properly going to answer my question, but I had to write my own blob detection, it appears that OpenCV SimpleBlobDetector is not so simple.
I'm using OpenCV 3.1 to do some blob detection using SimpleBlobDetector but I'm having no luck and no tutorial has been able to solve this. My environment is XCode on x64.
I'm starting out with this image:
Then I'm turning it into greyscale:
Finally I turn it into a binary image and doing the blob detection on this:
I've included "iostream" and "opencv2/opencv.hpp".
using namespace cv;
using namespace std;
Mat img_rgb;
Mat img_gray;
Mat img_keypoints;
Ptr<SimpleBlobDetector> detector = SimpleBlobDetector::create();
vector<KeyPoint> keypoints;
img_rgb = imread("summertriangle.jpg");
//Convert to greyscale
cvtColor(img_rgb, img_gray, CV_RGB2GRAY);
imshow("Grey Scale", img_gray);
// Start by creating the matrix that will allocate the new image
Mat img_bw(img_gray.size(), img_gray.type());
// Apply threshhold to convert into binary image and save to new matrix
threshold(img_gray, img_bw, 100, 255, THRESH_BINARY);
// Extract cordinates of blobs at their centroids, save to keypoints variable.
detector->detect(img_bw, keypoints);
cout << "The size of keypoints vector is: " << keypoints.size();
The keypoints vector is always empty. Nothing I've tried works.
So I solved this, did not read the fine print on the docs. Thanks Dai for the heads up on the Params, made me give the docs a closer look.
Default values of parameters are tuned to extract dark circular blobs.
I had to simply do this when creating the SimpleBlobDetector object:
SimpleBlobDetector::Params params;
params.filterByArea = true;
params.minArea = 1;
params.maxArea = 1000;
params.filterByColor = true;
params.blobColor = 255;
Ptr<SimpleBlobDetector> detector = SimpleBlobDetector::create(params);
This did it.
I've perused this site for an explanation but to no avail...hopefully someone knows the answer.
I'm using simpleBlobDetector to track some blobs. I would like to specify a mask via the detect method, but for some reason the mask doesn't seem to work - my keypoints show up for the whole image. Here are some snippets of my code:
Mat currFrame;
Mat mask;
Mat roi;
cv::Ptr<cv::FeatureDetector> blob_detector = new cv::SimpleBlobDetector(params);//custom set of params I've left out for legibility
blob_detector->create("SimpleBlob");
vector<cv::KeyPoint> myblob;
while(true)
{
captured >> currFrame; // get a new frame from camera >> is grab and retrieve in one go, note grab does not allow frame to be modified but edges can be
// do nothing if frame is empty
if(currFrame.empty())
{
break;
}
/******************** make mask***********************/
mask = Mat::zeros(currFrame.size(),CV_8U);
roi = Mat(mask,Rect(400,400,400,400));
roi = 255;
/******************** image cleanup with some filters*/
GaussianBlur(currFrame,currFrame, Size(5,5), 1.5, 1.5);
cv::medianBlur(currFrame,currFrame,3);
blob_detector->detect(fgMaskMOG,myblob,mask);//fgMaskMOG is currFrame after some filtering and background subtraction
cv::drawKeypoints(fgMaskMOG,myblob,fgMaskMOG,Scalar::all(-1), DrawMatchesFlags::DRAW_RICH_KEYPOINTS );
imshow("mogForeground", fgMaskMOG);
imshow("original", currFrame);
imshow("mask",mask);
if(waitKey(1) != -1)
break;
}
The thing is, I confirmed that my mask is correctly made by using SurfFeatureDetector as described here (OpenCV: howto use mask parameter for feature point detection (SURF)) If anyone can see whats wrong with my mask, I'd really appreciate the help. Sorry about the messy code!
I had the same issue and couldn't find the solution, so I solved it by checking the mask myself:
blob_detector->detect(img, keypoints);
std::vector<cv::KeyPoint> keypoints_in_range;
for (cv::KeyPoint &kp : keypoints)
if (mask.at<char>(kp.pt) > 0)
keypoints_in_range.push_back(kp)
I found i opencv2.4.8 this code:
void SimpleBlobDetector::detectImpl(const cv::Mat& image, std::vector<cv::KeyPoint>& keypoints, const cv::Mat&) const
{
//TODO: support mask
keypoints.clear();
Mat grayscaleImage;
which means that this option is not supported yet.
Solution with filtering keyPoints is not quite good, because it is time taking ( you have to detect blobs in whole image ).
Better workaround is to cut ROI before detection and move each KeyPoint after detection:
int x = 500;
int y = 200;
int width = 700;
int height = 700;
Mat roi = frame(Rect(x,y,width,height));
blob_detector.detect(roi, keypoints);
for (KeyPoint &kp : keypoints)
{
kp.pt.x +=x;
kp.pt.y +=y;
}
drawKeypoints(frame, keypoints, frame,Scalar::all(-1), DrawMatchesFlags::DRAW_RICH_KEYPOINTS);