I'm working with OpenCV SURF implementation.
I have found that keypoints, that are obtained by using if the SurfFeatureDetector with default parameters never found at octave 0 independently of the image.
Can somebody explain this result?
It looks like a bug in the following function (it called from SURF::operator() in surf.cpp):
static int getPointOctave(const CvSURFPoint& kpt, const CvSURFParams& params)
{
int octave = 0, layer = 0, best_octave = 0;
float min_diff = FLT_MAX;
for( octave = 1; octave < params.nOctaves; octave++ ) <---- octave counter starts from 1
for( layer = 0; layer < params.nOctaveLayers; layer++ )
{
float diff = std::abs(kpt.size - (float)((HAAR_SIZE0 + HAAR_SIZE_INC*layer) << octave));
if( min_diff > diff )
{
min_diff = diff;
best_octave = octave;
if( min_diff == 0 )
return best_octave;
}
}
return best_octave;
}
At the code that is written above, we can see that octave counter starts from 1. In accordance to the initialization of min_diff = FLT_MAX, the diff variable is obviously less than min_diff.
So, in case of nOctaves > 1, result best_octave will never be 0 even if we actually found keypoint at octave 0.
EDIT: This issue is fixed in the next release of the OpenCV (2.4)
Related
PROBLEM: The documentation for cv::LMSolver at opencv.org is very thin, to say the least. Finding some useful examples on the internet, also, was not possible.
APPROACH: So, I did write some simple code:
#include <opencv2/calib3d.hpp>
#include <iostream>
using namespace cv;
using namespace std;
struct Easy : public LMSolver::Callback {
Easy() = default;
virtual bool compute(InputArray f_param, OutputArray f_error, OutputArray f_jacobian) const override
{
Mat param = f_param.getMat();
if( f_error.empty() ) f_error.create(1, 1, CV_64F); // dim(error) = 1
Mat error = f_error.getMat();
vector<double> x{param.at<double>(0,0), param.at<double>(1,0)}; // dim(param) = 2
double error0 = calc(x);
error.at<double>(0,0) = error0;
if( ! f_jacobian.needed() ) return true;
else if( f_jacobian.empty() ) f_jacobian.create(1, 2, CV_64F);
Mat jacobian = f_jacobian.getMat();
double e = 1e-10; // estimate derivatives in epsilon environment
jacobian.at<double>(0, 0) = (calc({x[0] + e, x[1] }) - error0) / e; // d/dx0 (error)
jacobian.at<double>(0, 1) = (calc({x[0], x[1] + e}) - error0) / e; // d/dx1 (error)
return true;
}
double calc(const vector<double> x) const { return x[0]*x[0] + x[1]*x[1]; }
};
int main(int argc, char** argv)
{
Ptr<Easy> callback = makePtr<Easy>();
Ptr<LMSolver> solver = LMSolver::create(callback, 100000, 1e-37);
Mat parameters = (Mat_<double>(2,1) << 5, 100);
solver->run(parameters);
cout << parameters << endl;
}
QUESTIONS:
What does the return value of LMSolver::Callback::compute() report to the caller?
Currently, it finds the minimum at (-9e-07,4e-5), instead of the expected (0.0, 0.0). How can the precision be improved?
What does the return value of LMSolver::Callback::compute() report to the caller?
Thankfully, opencv is opensource, so we might be able to figure this out simply by checking out the code.
Looking at the source code on Github, I found that all of the calls to compute() look like:
if( !cb->compute(x, r, J) )
return -1;
Returning false simply causes the solver to bail out. So it seems that the return value of the callback's compute() is simply whether the generation of the jacobian was successful or not.
Currently, it finds the minimum at (-9e-07,4e-5). How can the precision be improved?
If anything, you should at least compare the return value of run() against your maximum iteration count to make sure that it did, in fact, converge as much as it could.
I suppose OP wants to minimize x^2 + y^2 with respect to [x, y].
Because Levenberg-Marquardt method solves a least square problem, the error should be defined as [x, y] so that it minimizes || [x, y] ||^2 = x^2 + y^2.
Another suggestion is that the Jacobian matrix should be provided analytically whenever possible, although this is not crucial in this particular case.
struct Easy : public LMSolver::Callback {
Easy() = default;
virtual bool compute(InputArray f_param, OutputArray f_error, OutputArray f_j
acobian) const override
{
Mat param = f_param.getMat();
if( f_error.empty() ) f_error.create(2, 1, CV_64F); // dim(error) = 2
Mat error = f_error.getMat();
vector<double> x{param.at<double>(0,0), param.at<double>(1,0)}; // dim(param) = 2
error.at<double>(0, 0) = x[0];
error.at<double>(1, 0) = x[1];
if( ! f_jacobian.needed() ) return true;
else if( f_jacobian.empty() ) f_jacobian.create(2, 2, CV_64F);
Mat jacobian = f_jacobian.getMat();
jacobian.at<double>(0, 0) = 1;
jacobian.at<double>(0, 1) = 0;
jacobian.at<double>(1, 0) = 0;
jacobian.at<double>(1, 1) = 1;
return true;
}
};
I want to train a decision tree with hog features.
When I try to train my cv::ml::DTrees model, I receive the following error:
OpenCV(3.4.3) Error: Assertion failed (n >= 0) in cv::ml::TrainDataImpl::getValues, file C:\build\3_4_winpack-build-win64-vc14\opencv\modules\ml\src\data.cpp, line 890
I understand, that the number of samples(features) have to match the number of responses(labels) and both have to be >=0.
My cv::Mat feats and cv::Mat labels have the same number of rows (388).
This is how I try to do it:
cv::Ptr<cv::ml::DTrees> model = cv::ml::DTrees::create();
cv::Ptr<cv::ml::TrainData> data = cv::ml::TrainData::create(feats, cv::ml::ROW_SAMPLE, labels);
model->train(data);
model->train(data) fails, when it calls this function:
void getValues( int vi, InputArray _sidx, float* values ) const CV_OVERRIDE
{
Mat sidx = _sidx.getMat();
int i, n = sidx.checkVector(1, CV_32S), nsamples = getNSamples();
CV_Assert( 0 <= vi && vi < getNAllVars() );
CV_Assert( n >= 0 );
const int* s = n > 0 ? sidx.ptr<int>() : 0;
if( n == 0 )
n = nsamples;
}
}
Can anyone point me in the right direction to fix this problem?
EDIT:
If I set the MinSampleCount to 388 (the number of samples) I don't get an error, but the prediction won't work correctly (it always returns label 5).
Let say, A and B are matrices of the same size.
In Matlab, I could use simple indexing as below.
idx = A>0;
B(idx) = 0
How can I do this in OpenCV? Should I just use
for (i=0; ... rows)
for(j=0; ... cols)
if (A.at<double>(i,j)>0) B.at<double>(i,j) = 0;
something like this? Is there a better (faster and more efficient) way?
Moreover, in OpenCV, when I try
Mat idx = A>0;
the variable idx seems to be a CV_8U matrix (not boolean but integer).
You can easily convert this MATLAB code:
idx = A > 0;
B(idx) = 0;
// same as
B(A>0) = 0;
to OpenCV as:
Mat1d A(...)
Mat1d B(...)
Mat1b idx = A > 0;
B.setTo(0, idx) = 0;
// or
B.setTo(0, A > 0);
Regarding performance, in C++ it's usually faster (it depends on the enabled optimizations) to work on raw pointers (but is less readable):
for (int r = 0; r < B.rows; ++r)
{
double* pA = A.ptr<double>(r);
double* pB = B.ptr<double>(r);
for (int c = 0; c < B.cols; ++c)
{
if (pA[c] > 0.0) pB[c] = 0.0;
}
}
Also note that in OpenCV there isn't any boolean matrix, but it's a CV_8UC1 matrix (aka a single channel matrix of unsigned char), where 0 means false, and any value >0 is true (typically 255).
Evaluation
Note that this may vary according to optimization enabled with OpenCV. You can test the code below on your PC to get accurate results.
Time in ms:
my results my results #AdrienDescamps
(OpenCV 3.0 No IPP) (OpenCV 2.4.9)
Matlab : 13.473
C++ Mask: 640.824 5.81815 ~5
C++ Loop: 5.24414 4.95127 ~4
Note: I'm not entirely sure about the performance drop with OpenCV 3.0, so I just remark: test the code below on your PC to get accurate results.
As #AdrienDescamps stated in comments:
It seems that the performance drop with OpenCV 3.0 is related to the OpenCL option, that is now enabled in the comparison operator.
C++ Code
#include <opencv2/opencv.hpp>
#include <iostream>
using namespace std;
using namespace cv;
int main()
{
// Random initialize A with values in [-100, 100]
Mat1d A(1000, 1000);
randu(A, Scalar(-100), Scalar(100));
// B initialized with some constant (5) value
Mat1d B(A.rows, A.cols, 5.0);
// Operation: B(A>0) = 0;
{
// Using mask
double tic = double(getTickCount());
B.setTo(0, A > 0);
double toc = (double(getTickCount()) - tic) * 1000 / getTickFrequency();
cout << "Mask: " << toc << endl;
}
{
// Using for loop
double tic = double(getTickCount());
for (int r = 0; r < B.rows; ++r)
{
double* pA = A.ptr<double>(r);
double* pB = B.ptr<double>(r);
for (int c = 0; c < B.cols; ++c)
{
if (pA[c] > 0.0) pB[c] = 0.0;
}
}
double toc = (double(getTickCount()) - tic) * 1000 / getTickFrequency();
cout << "Loop: " << toc << endl;
}
getchar();
return 0;
}
Matlab Code
% Random initialize A with values in [-100, 100]
A = (rand(1000) * 200) - 100;
% B initialized with some constant (5) value
B = ones(1000) * 5;
tic
B(A>0) = 0;
toc
UPDATE
OpenCV 3.0 uses IPP optimization in the function setTo. If you have that enabled (you can check with cv::getBuildInformation()), you'll have a faster computation.
The answer of Miki is very good, but i just want to add some clarification about the performance problem to avoid any confusion.
It is true that the best way to implement an image filter (or any algorithm) with OpenCV is to use the raw pointers, as shown in the second C++ example of Miki (C++ Loop).
Using the at function is also correct, but significantly slower.
However, most of the time, you don't need to worry about that, and you can simply use the high level functions of OpenCV (first example of Miki , C++ Mask). They are well optimized, and will usually be almost as fast as a low level loop on pointers, or even faster.
Of course, there are exceptions (we just found one), and you should always test for your specific problem.
Now, regarding this specific problem :
The example here where the high level function was much slower (100x slower) than the low level loop is NOT a normal case, as it is demonstrated by the timings with other version/configuration of OpenCV, that are much lower.
The problem seems to be that when OpenCV3.0 is compiled with OpenCL, there is a huge overhead the first time a function that uses OpenCL is called. The simplest solution is to disable OpenCL at compile time, if you use OpenCV3.0 (see also here for other possible solutions if you are interested).
so I am struggling to understand why I am getting this assertion error from opencv when accessing a pointer in the next col/row of an image. Let me tell you what is happening and provide some code.
I am taking a ROI from the image, which is a cv::Mat or a header to a section of a bigger cv::Mat.
I constructed some pointers to access the value of my ROI. So lets say my ROI is filled with values of pixels and is a 3x3 Mat
with the following Dimensions (index starting at 0,0)
---------
| 1 | 2 | 3 |
| 4 | 5 | 6 |
| 7 | 8 | 9 |
first of all I need to initialize my pointers to point to their positions respectively. I took the ptr function of the cv::Mat and their location in the grid via cv::Point.
Problem faced:
When I try to access the pixel of the next neighbor, I get an assertion error.
Diagnostics by me:
I thought it might be the range, but I made sure that wouldn't be the case by defining the for loop conditions according to my dimensions.
The item I am trying to access doesn't exist, but how I understand it when I go through the ROI, i already have the values in a new Matrix and I should be able to access all values around my desired pixels.
PART OF THE CODE:
cv::Mat ROI =disTrafo(cv::Rect(cv::Point(x,y),cv::Size(3,3)));
cv::minMaxLoc(ROI,&minVal,&maxVal,&minCoord,&maxCoord);
auto* maxPtr_x = &maxCoord.x;
auto* maxPtr_y = &maxCoord.y;
auto* maxPtr_value = &maxVal;
uchar diff1 = 0;
uchar diff2= 0;
uchar diff3 = 0;
uchar diff4 = 0;
uchar max_diff = 0;
for(int j = 1; j < ROI.rows ; j++){
auto current = ROI.ptr<uchar>(maxCoord.y);
auto neighbor_down = ROI.ptr<uchar>(maxCoord.y+1); //THE PROB IS HERE according to debugging
auto neighbor_up = ROI.ptr<uchar>(maxCoord.y-1);
cv::Point poi ; //point of interest
for(int i= 0; i < ROI.cols; i++){
switch(maxCoord.x){ //PROOF FOR LOGIC
case 0:
if(maxCoord.y == 0){ //another switch statement maybe ??
diff1 = std::abs(current[maxCoord.x+1] - current[maxCoord.x]);
diff2 = std::abs(neighbor_down[maxCoord.x] - current[maxCoord.x]);
if(diff2 > diff1){
cv::Point(maxCoord.x,maxCoord.y+1) = poi;
} else {
cv::Point(maxCoord.x+1,maxCoord.y) = poi;
}
};
ASSERTION FAILED when running it: OpenCV Error: Assertion failed(y == 0|| < data && dims >= 1 && (unsigned)y < (unsigned)size.p[0])) in cv::Mat::ptr, file //... indicates path to header file mat.hpp// line 428
I can't put my finger on the problem, could you please be of assistance. And please give me some knowledge when working with pointers and pixels in case I misunderstood something.
Thank you
try this
for(int j = 1; j < ROI.rows ; j++){
auto current = ROI.ptr<uchar>(maxCoord.y);
auto neighbor_down = ROI.ptr<uchar>(maxCoord.y+1); //THE PROB IS HERE according to debugging
auto neighbor_up = ROI.ptr<uchar>(maxCoord.y-1);
cv::Point poi ; //point of interest
cv::Point bordess(Point(0,0));
for(int i= 0; i < ROI.cols; i++){
switch(maxCoord.x){ //PROOF FOR LOGIC
case 0:
if(maxCoord.y == 0){ //another switch statement maybe ??
diff1 = std::abs(current[maxCoord.x+1] - current[maxCoord.x]);
diff2 = std::abs(neighbor_down[maxCoord.x] - current[maxCoord.x]);
if(diff2 > diff1){
cv::Point(maxCoord.x,maxCoord.y+1) = poi & bordess;
} else {
cv::Point(maxCoord.x+1,maxCoord.y) = poi & bordess;
}
};
Ok so basically I figured out that my pointer definition was wrong due to the nature of the input image. I have done some preprocessing with the image and the ranges of the values inside changed from ucharto some other value. When I changed the auto neighbor_down = ROI.ptr<uchar>(maxCoord.y+1);for example to auto neighbor_down = ROI.ptr<float>(maxCoord.y+1); everything ran normal.
I have a client/server application, my server manages the opencv library to do for example disparity mapping, the application works fine with stereoSGBM, but with stereoBM I have random crash with ctrl + f5 release, so launching it without debugging.
The crash is random, with a try/catch sometimes I can get bad allocation memory, failed to allocate 1k bytes. Instead with the call stack I'm not able to catch anything relevant because the crash is not always in the same point, sometimes is in a imgread, sometimes is a malloc, a free a mat.release, so every time is different, but always involves memory in some way.
The code is pretty simple:
void disparity_mapping(std::vector<std::string> & _return, const StereoBmValue& BmValue, const ClientShowSelection& clientShowSelection, const std::string& filenameL, const std::string& filenameR)
{
int ch;
alg = BmValue.algorithmSelection;
if((filenameL == "0" || filenameR == "0"))
_return.push_back("0");
if((filenameL != "0" && filenameR != "0"))
{
imgL = imread(filenameL , CV_LOAD_IMAGE_GRAYSCALE );
imgR = imread(filenameR, CV_LOAD_IMAGE_GRAYSCALE );
_return.push_back("1");
ch = imgL.channels();
setAlgValue(BmValue, methodSelection, ch); //Setting the value for StereoBM or SGBM
disp = calculateDisparity(imgL, imgR, alg); //calculating disparity
normalize(disp, disp8, 0, 255, CV_MINMAX, CV_8U);
string matAsStringL(imgL.begin<unsigned char>(), imgL.end<unsigned char>());
_return.push_back(matAsStringL);
string matAsStringR(imgR.begin<unsigned char>(), imgR.end<unsigned char>());
_return.push_back(matAsStringR);
string matAsStringD(disp8.begin<unsigned char>(), disp8.end<unsigned char>());
_return.push_back(matAsStringD);
}
the two function that are called:
void setAlgValue(const StereoBmValue BmValue, int methodSelection, int ch)
{
if (initDisp)
initDisparity(methodSelection); //inizializing bm.init(...) and find remap informations from steroRect, etc.
//Select 0 == StereoSGBM, 1 == StereoBM
int alg = BmValue.algorithmSelection;
//storing alg value.
stereoValue.minDisparity = BmValue.minDisparity;
stereoValue.disp12MaxDiff = BmValue.disp12MaxDiff;
stereoValue.SADWindowSize = BmValue.SADWindowSize;
stereoValue.textureThreshold = BmValue.textureThreshold;
stereoValue.uniquenessRatio = BmValue.uniquenessRatio;
stereoValue.numberOfDisparities = BmValue.numberOfDisparities;
stereoValue.preFilterCap = BmValue.preFilterCap;
stereoValue.speckleWindowSize = BmValue.speckleWindowSize;
stereoValue.speckleRange = BmValue.speckleRange;
stereoValue.preFilterSize = BmValue.preFilterSize;
if (alg == 1) //set of the values in the bm state
{
bm.state->roi1 = roi1;
bm.state->roi2 = roi2;
bm.state->preFilterCap = stereoValue.preFilterCap;
bm.state->SADWindowSize = stereoValue.SADWindowSize;
bm.state->minDisparity = stereoValue.minDisparity;
bm.state->numberOfDisparities = stereoValue.numberOfDisparities;
bm.state->textureThreshold = stereoValue.textureThreshold;
bm.state->uniquenessRatio = stereoValue.uniquenessRatio;
bm.state->speckleWindowSize = stereoValue.speckleWindowSize;
bm.state->speckleRange = stereoValue.speckleRange;
bm.state->disp12MaxDiff = stereoValue.disp12MaxDiff;
bm.state->preFilterSize = stereoValue.preFilterSize;
}
else if(alg == 0) //same for SGBM
{
sgbm.P1 = 8*ch*sgbm.SADWindowSize*sgbm.SADWindowSize;
sgbm.P2 = 32*ch*sgbm.SADWindowSize*sgbm.SADWindowSize;
sgbm.preFilterCap = stereoValue.preFilterCap;
sgbm.SADWindowSize = stereoValue.SADWindowSize;
sgbm.minDisparity = stereoValue.minDisparity;
sgbm.numberOfDisparities = stereoValue.numberOfDisparities;
sgbm.uniquenessRatio = stereoValue.uniquenessRatio;
sgbm.speckleWindowSize = stereoValue.speckleWindowSize;
sgbm.speckleRange = stereoValue.speckleRange;
sgbm.disp12MaxDiff = stereoValue.disp12MaxDiff;
}
}
and the other one:
Mat calculateDisparity(Mat& imgL, Mat& imgR, int alg)
{
Mat disparity;
//remap for rectification
remap(imgL, imgL, map11, map12, INTER_LINEAR,BORDER_CONSTANT, Scalar());
remap(imgR, imgR, map21, map22, INTER_LINEAR,BORDER_CONSTANT, Scalar());
//disp
if (alg == 1)
bm( imgL , imgR , disparity);
else if (alg == 0)
sgbm(imgL, imgR, disparity);
return disparity;
}
So as you can see the code is really simple, but using bm make all crash. I'm using the last OpenCV library build for VS9 updated. Is also linked with thrift apache, pcl, eigen, vtk and boost. The bm/sgbm value are controlled by the client and are correct, i don't get any error in debug/release with debug.
What can be? Why one works and the other one make the entire application to crash? Why just in release without debug?
I was having this same issue, and just found out that with high values of bm.state->textureThreshold it would crash. Values from ~50+ are crashing for me.