Multi-Channel Back Projection Assertion (j < nimages) - c++

Attempting to do histogram back-projection on a three-channel image results in the following error:
OpenCV Error: Assertion failed (j < nimages) in histPrepareImages, file ../modules/imgproc/src/histogram.cpp, line 148
The code which fails:
cv::Mat _refImage; //contains reference image of type CV_8UC3
cv::Mat output; //contains image data of type CV_8UC3
int histSize[] = {16, 16, 16};
int channels[] = {0, 1, 2};
const float hRange[] = {0.f, 256.f};
const float* ranges[] = {hRange, hRange, hRange};
int nChannels = 3;
cv::Mat hist;
cv::calcHist(&_refImage, 1, channels, cv::noArray(), hist, nChannels, histSize, ranges);
cv::calcBackProject(&output, 1, channels, hist, output, ranges); //This line causes assertion failure
Running nearly identical code on a single-channeled image works. According to the documentation, multi-channel images are also supported. Why won't this code work?

The short answer is that cv::calcBackProject() does not support in-place operation, although this is not mentioned in the documentation.
Explanation
Digging into the OpenCV source yields the following snippet:
void calcBackProject( const Mat* images, int nimages, const int* channels,
InputArray _hist, OutputArray _backProject,
const float** ranges, double scale, bool uniform )
{
//Some code...
_backProject.create( images[0].size(), images[0].depth() );
Mat backProject = _backProject.getMat();
assert(backProject.type() == CV_8UC1);
histPrepareImages( images, nimages, channels, backProject, dims, hist.size, ranges,
uniform, ptrs, deltas, imsize, uniranges );
//More code...
}
The line which causes the problem is:
_backProject.create( images[0].size(), images[0].depth() );
which, if the source and destination are the same, reallocates the input image data. images[0].depth() evaluates to CV_8U, which is numerically equivalent to the type specifier CV_8UC1. Thus, the data is created as a single-channel image.
This is a problem because histPrepareImages still expects the input image to have 3 channels, and the assertion is thrown.
Solution
Fortunately, the workaround is simple. The output parameter must be different from the input, like so:
cv::Mat result;
cv::calcBackProject(&output, 1, channels, hist, result, ranges);

Related

Why do these histogram functions differ, and why is one nondeterministic?

NOTE: This is a homework problem and the professor explicitly forbids soliciting answers from StackOverflow, so please limit your response to the specific question I have asked and do not attempt to provide a working solution.
I am asked to implement a function that computes the histogram of a single-channel 8-bit image represented as an OpenCV Mat with type CV_U8.
In this case, the histogram uses 256 uniformly-distributed buckets. This is the reference we are intended to replicate (using OpenCV 3.4):
Mat reference;
/// Establish the number of bins
int histSize = 256;
/// Set the ranges ( for B,G,R) )
float range[] = { 0, 256 } ;
const float* histRange = { range };
bool uniform = true;
bool accumulate = false;
cv::calcHist(&bgr_planes[0], 1, 0, Mat(), reference, 1, &histSize, &histRange,
uniform, accumulate);
// reference now contains the canonical histogram of the input image's
// blue channel
I wrote the following function to calculate the histogram, which produces the correct results 45-69% of the time (p<0.05, n=66). Once when it failed, I examined the results and found no discernable pattern. All trials were conducted on the same test image.
Mat myCalcHist(const Mat& input) {
assert(input.isContinuous());
Mat res(256, 1, CV_32F);
for (const uint8_t* it = input.datastart; it != input.dataend; ++it) {
++res.at<float>(*it);
}
return res;
}
The following function, on the other hand, more closely matches OpenCV's internal implementation in that it uses the idiomatic accessors and converts the float result from an int work matrix, but in n=66 trials it did not produce the correct result a single time. Again, I found no discernable pattern in the data.
Mat myCalcHist(const Mat& input) {
Mat ires(256, 1, CV_32S);
for (int i = 0; i < input.total(); ++i) {
++ires.at<int>(input.at<uint8_t>(i));
}
Mat res(256, 1, CV_32F);
ires.convertTo(res, CV_32F);
return res;
}
Why are the results for my first implementation different than those from my second implementation, and where is nondeterminism introduced to the first implementation?
initializing the histogram matrix should work:
Mat myCalcHist(const Mat& input)
{
Mat ires = cv::Mat::zeros(256, 1, CV_32S);
for (int i = 0; i < input.total(); ++i)
{
++ires.at<int>(input.at<uint8_t>(i));
}
Mat res(256, 1, CV_32F);
ires.convertTo(res, CV_32F);
return res;
}

'compareHist' not working for similar images

I have been trying to find matched image from sample image using histogram matching. for most of the cases my code is working fine. The range of used method, Bhattacharyya, is 0 <= method <= 1.
normally using Bhattacharyya method the output result will close to 0, in case of matched cases. but i have come to a case where both images are almost similar, though there could be some contrast difference.
which is why this procedure is giving higher result...
can anyone help me why this comparison is giving so much bigger value?
src image and test image
int main(){
src_base = imread("images/src.jpg",-1);
src_test1 = imread("images/test.png",-1);
double base_test1 = hsvToHist(src_base, src_test1,3);
cout<< " Bhattacharyya template Base-Test(1) : "<< base_test1<<endl;
return 0;
}
double hsvToHist( Mat src_base, Mat src_test1, int method){
Mat hsv_base, hsv_test1;
cvtColor( src_base, hsv_base, COLOR_BGR2HSV );
cvtColor( src_test1, hsv_test1, COLOR_BGR2HSV );
/// initialization to calculate histograms (Using 50 bins for hue, 60 for saturation)
int h_bins = 50; int s_bins = 60;
int histSize[] = { h_bins, s_bins };
float h_ranges[] = { 0, 180 };
float s_ranges[] = { 0, 256 };
const float* ranges[] = { h_ranges, s_ranges };
int channels[] = { 0, 1 };
/// Histograms
Mat hist_base, hist_test1;
/// Calculate the histograms for the HSV images
calcHist( &hsv_base, 1, channels, Mat(), hist_base, 2, histSize, ranges, true, false );
normalize( hist_base, hist_base, 0, 1, NORM_MINMAX, -1, Mat() );
calcHist( &hsv_test1, 1, channels, Mat(), hist_test1, 2, histSize, ranges, true, false );
normalize( hist_test1, hist_test1, 0, 1, NORM_MINMAX, -1, Mat() );
///'3' for Bhattacharyya
double base_test1 = compareHist( hist_base, hist_test1, method );
return base_test1;
}
The PNG and JPEG images will have different histograms even though they appear the same, because the JPEG is compressed which means information has been removed and the histogram has been essentially filtered and smoothed. Also, the PNG will have a larger range of values than the JPEG. You may get better results with different bin sizes, but it's hard to tell without testing.
The Bhattacharyya distance has an N^2 term in the denominator where N is the number of pixels. In general, this allows similar values for different sizes of images. However, for the icons that you are comparing, the divisor is much smaller. You could scale the metric by a factor related to the image size.
Alternately, you could use the HISTCMP_CORREL method, which produces lower absolute values if the differences between pixels are less significant. This method produces larger values if more pixels are compared.
When you want similar results independent of differences in image size you could compute both metrics and consider the images equal if one of them passes a tight threshold for similarity. Actual thresholds will vary depending on whether you are comparing color or grayscale images, and whether you have pre-processed the images using histogram equalization (see cv::equalizeHist).

Shouldn't GaussianBlur be symmetric?

I expected a Gaussian Blur operation to be symmetric, but using the OpenCV 2.4.11 GaussianBlur I am getting differences.
Here's an example. I apply a GaussianBlur to an image, and to a flipped version of the image. I've separately verified the flip operation doesn't change the image pixel values (not shown). When I flip the blurred image back, I expected it to be the same as the blur of the original, but the diff shows a lot of small differences (between 0.0 and 6.103515625e-005). I know that's small, but it has a knock-on effect in my subsequent processing.
The Gaussian Kernel is symmetric, so the result should be the same. Is this simply a rounding error in the implementation?
int main(int, char **)
{
// e.g. 2008_005541.jpg from VOC2012 dataset
char const * const filename = "...";
float const sig_diff = 1.24899971f;
cv::Mat image = cv::imread(filename, cv::IMREAD_GRAYSCALE);
cv::Mat gray_fpt;
image.convertTo(gray_fpt, cv::DataType<float>::type, 1, 0);
GaussianBlur(gray_fpt, gray_fpt, cv::Size(), sig_diff, sig_diff);
cv::Mat mirror;
flip(image, mirror, 1);
cv::Mat mirror_gray_fpt;
mirror.convertTo(mirror_gray_fpt, cv::DataType<float>::type, 1, 0);
GaussianBlur(mirror_gray_fpt, mirror_gray_fpt, cv::Size(), sig_diff, sig_diff);
flip(mirror_gray_fpt, mirror_gray_fpt, 1);
cv::Mat diff = abs(gray_fpt - mirror_gray_fpt);
double minval, maxval;
minMaxLoc(diff, &minval, &maxval);
// minval = 0.0;
// maxval = 6.103515625e-005;
// easier to visualise the differences with this:
normalize(diff, diff, 0.0, 1.0, cv::NORM_MINMAX, CV_32FC1);
return 0;
}
EDIT: I changed the type from cv::DataType<float>::type to cv::DataType<double>::type and now the max error is 1.1368683772161603e-013, so rounding seems to be the problem.
Changing the code above to call gaussian_blur (below) instead of GaussianBlur produces no differences in the example images I've tested so far.
From this, if the working type of the Gaussian operation is double precision, then the output in floating point precision yields no error. That seems like a nice solution to my problem.
// Perform gaussian blur in double precision and convert back
void gaussian_blur(
cv::Mat const &src, cv::Mat &dst,
cv::Size ksize, double sigmaX, double sigmaY=0,
int borderType=cv::BORDER_DEFAULT)
{
cv::Mat src_dp;
src.convertTo(src_dp, cv::DataType<double>::type, SIFT_FIXPT_SCALE, 0);
cv::Mat dst_dp;
GaussianBlur(src_dp, dst_dp, ksize, sigmaX, sigmaY, borderType);
dst_dp.convertTo(dst, src.type(), SIFT_FIXPT_SCALE, 0);
}

Unwieldy CvMat* in ANN using OpenCV

I'm trying to use OpenCV to train a neural network in C++.
I can't convert between cv::Mat* (or Mat*, if namespace cv is used) to CvMat*, and I would appreciate some help with this.
Let me elaborate:
I've got two data structures of cv::Mat* type. The first is the set of feature vectors and the second is the set of expected output.
cv::Mat *feat = new cv::Mat(3000, 100, CV_32F, featureData);
cv::Mat *op = new cv::Mat(3000, 2, CV_32F, expectedOutput);
(These are 3000 data points of feature vector length = 100 and output state = 2)
These two matrices had been populated with data of correct dimensions and seem to be working fine when sample data were printed on the console.
The neural network has been initialized as:
int layers_array[] = {100,200,2}; //hidden layer nodes = 200
CvMat* layer = cvCreateMatHeader(1, 3, CV_32SC1);
cvInitMatHeader(layer, 1,3,CV_32SC1, layers_array);
CvANN_MLP nnetwork;
nnetwork.create(layer, CvANN_MLP::SIGMOID_SYM, SIGMOID_ALPHA, SIGMOID_BETA);
Now, the train method of ANN is of the following template:
virtual int train( const CvMat* inputs, const CvMat* outputs,
const CvMat* sampleWeights, const CvMat* sampleIdx=0,
CvANN_MLP_TrainParams params = CvANN_MLP_TrainParams(),
int flags=0 );
I tried to convert between cv::Mat * and CvMat * using the following code:
CvMat featMat,opMat;
(&featMat)->cols = feat->cols;
(&featMat)->rows = feat->rows;
(&featMat)->type = CV_32F;
(&featMat)->data.fl = (float *)feat->data;
(&opMat)->cols = op->cols;
(&opMat)->rows = op->rows;
(&opMat)->type = CV_32F;
(&opMat)->data.fl = (float *)op->data;
//setting up the ANN training parameters
int iterations = network.train(&featMat, &opMat, NULL, NULL, trainingParams);
When I run this code, I get the following error message in my console:
**OpenCV Error: Bad argument (input training data should be a floating-point matrix withthe number of rows equal to the number of training samples and the number
of columns equal to the size of 0-th (input) layer) in CvANN_MLP::prepare_to_train, file ..\..\OpenCV-2.3.0-win-src\OpenCV-2.3.0\modules\ml\src\ann_mlp.cpp,
line 694**
I understand the error message. However, to the best of my knowledge, I believe I haven't made a mess of the number of nodes in the input/output layer.
Can you please help me understand what is going wrong?
please try to avoid pointers to cv::Mat as well as CvMat* in general.
luckily, there's an overload to CvANN_MLP::train that takes cv::Mat as args, so use that instead:
cv::Mat feat = cv::Mat(3000, 100, CV_32F, featureData);
cv::Mat op = cv::Mat(3000, 2, CV_32F, expectedOutput);
int layers_array[] = {100,200,2}; //hidden layer nodes = 200
cv::Mat layers = cv::Mat (3, 1, CV_32SC1, layers_array );
CvANN_MLP nnetwork;
nnetwork.create(layers, CvANN_MLP::SIGMOID_SYM, SIGMOID_ALPHA, SIGMOID_BETA);
int iterations = nnetwork.train(feat, op, cv::Mat(), cv::Mat(), CvANN_MLP_TrainParams());

OpenCV cv::Mat to short* (avoiding memcpy)

I have a C++ function that is to be called from someone else's C# application. As input my function is given an array of signed short integers, the dimensions of the image it represents, and memory allocated for the returning data, namely another array of signed short integers. This would represent my function's header:
my_function (short* input, int height, int width, short* output)
Inside my function I create a cv::Mat from input, like this:
cv::Mat mat_in = cv::Mat (height, width, CV_16S, input);
This mat_in is then converted to CV_32F and processed by OpenCV's cv::bilateralFilter. After it returns cv::Mat mat_out, I convert the data back to CV_16S (bilateralFilter only accepts CV_8U and CV_32F). Now I need to convert this cv::Mat mat_out back to an array of short integers so that it may be returned to the calling function. This is my code:
my_function (short* input, int height, int width, short* output)
{
Mat mat_in_16S = Mat (height, width, CV_16S, input);
Mat mat_in_32F = Mat (height, width, CV_32F);
Mat mat_out_CV_32F = Mat (height, width, CV_32F);
mat_in_16S.convertTo (mat_in_32F, CV_32F);
bilateralFilter (mat_in_32F, mat_out_32F, 5, 160, 2);
Mat mat_out_16S = Mat (mat_in_16S.size(), mat_in_16S.type());
mat_out_32F.convertTo (mat_out_16S, CV_16S);
return 0;
}
Obviously, somewhere there at the end I need to get the data that is in mat_out_16S into output. My first try was to return a reference:
output = &mat_out_16S.at<short>(0,0);
but of course I realised that this was a silly idea, as mat_out_16S goes out of scope as soon as the function returns, leaving output pointing at emptiness. Currently my best attempt is as follows (from this question):
memcpy ((short*)output, (short*)mat_out_16S.data, height*width*sizeof(short));
Now I would like to know, is there a better way? It feels kind of inefficient to copy all this data, but I don't see what else I can do. I can't return a cv::Mat unfortunately. If there is no better way, is my current memcpy method safe at least? My data are all 2-byte signed short integers, so I don't think there should be issues with padding, but I don't want to run into any unpleasant surprises.
You can use this constructor for your mat_out_16S:
Mat::Mat(Size size, int type, void* data, size_t step=AUTO_STEP)
So your function will be:
my_function (short* input, int height, int width, short* output)
{
Mat mat_in_16S = Mat (height, width, CV_16S, input);
Mat mat_in_32F = Mat (height, width, CV_32F);
Mat mat_out_CV_32F = Mat (height, width, CV_32F);
mat_in_16S.convertTo (mat_in_32F, CV_32F);
bilateralFilter (mat_in_32F, mat_out_32F, 5, 160, 2);
Mat mat_out_16S = Mat (mat_in_16S.size(), mat_in_16S.type(), output);
mat_out_32F.convertTo (mat_out_16S, CV_16S);
return 0;
}