Assertion failed with accumulateWeighted in OpenCV - c++

I am using openCV and trying to calculate a moving average of the background, then taking the current frame and subtracting the background to determine movement (of some sort).
However, when running the program I get:
OpenCV Error: Assertion failed (func != 0) in accumulateWeighted, file /home/sebbe/projekt/opencv/trunk/opencv/modules/imgproc/src/accum.cpp, line 431
terminate called after throwing an instance of 'cv::Exception'
what(): /home/sebbe/projekt/opencv/trunk/opencv/modules/imgproc/src/accum.cpp:431: error: (-215) func != 0 in function accumulateWeighted
I cant possibly see what arguments are wrong to accumulateWeighted.
Code inserted below:
#include <stdio.h>
#include <stdlib.h>
#include "cv.h"
#include "highgui.h"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "cxcore.h"
using namespace cv;
int main( int argc, char **argv )
{
Mat colourFrame;
Mat frame;
Mat greyFrame;
Mat movingAverage;
Mat difference;
Mat temp;
int key = 0;
VideoCapture cap(0);
/* always check */
if ( !cap.isOpened() ) {
fprintf( stderr, "Cannot open initialize webcam!\n" );
return 1;
}
namedWindow("Camera Window", 0);
// Initialize
cap >> movingAverage;
while( key != 'q' ) {
/* get a frame */
cap >> colourFrame;
/* Create a running average of the motion and convert the scale */
accumulateWeighted(colourFrame, movingAverage, 0.02, Mat() );
/* Take the difference from the current frame to the moving average */
absdiff(colourFrame, movingAverage, difference);
/* Convert the image to grayscale */
cvtColor(difference, greyFrame, CV_BGR2GRAY);
/* Convert the image to black and white */
threshold(greyFrame, greyFrame, 70, 255, CV_THRESH_BINARY);
/* display current frame */
imshow("Camera Window",greyFrame);
/* exit if user press 'q' */
key = cvWaitKey( 1 );
}
return 0;
}

Looking at the OpenCV sources, specifically at modules/imgproc/src/accum.cpp line 431, the lines that precede this assertion are:
void cv::accumulateWeighted( InputArray _src, CV_IN_OUT InputOutputArray _dst,
double alpha, InputArray _mask )
{
Mat src = _src.getMat(), dst = _dst.getMat(), mask = _mask.getMat();
int sdepth = src.depth(), ddepth = dst.depth(), cn = src.channels();
CV_Assert( dst.size == src.size && dst.channels() == cn );
CV_Assert( mask.empty() || (mask.size == src.size && mask.type() == CV_8U) );
intfidx = getAccTabIdx(sdepth, ddepth);
AccWFunc func = fidx >= 0 ? accWTab[fidx] : 0;
CV_Assert( func != 0 ); // line 431
What's happening in your case is that getAccTabIdx() is returning -1, which in turn makes func be ZERO.
For accumulateWeighted() to work properly, the depth of colourFrame and movingAverage must be one of the following options:
colourFrame.depth() == CV_8U && movingAverage.depth() == CV_32F
colourFrame.depth() == CV_8U && movingAverage.depth() == CV_64F
colourFrame.depth() == CV_16U && movingAverage.depth() == CV_32F
colourFrame.depth() == CV_16U && movingAverage.depth() == CV_64F
colourFrame.depth() == CV_32F && movingAverage.depth() == CV_32F
colourFrame.depth() == CV_32F && movingAverage.depth() == CV_64F
colourFrame.depth() == CV_64F && movingAverage.depth() == CV_64F
Anything different than that will make getAccTabIdx() return -1 and trigger the exception at line 431.

From the documentation on OpenCV API you can see that the output image from accumulateWeighted is
dst – Accumulator image with the same number of channels as input image, 32-bit or 64-bit floating-point.
So your initialization is wrong. You should retrieve the colourFrame size first and then do this:
cv::Mat movingAverage = cv::Mat::zeros(colourFrame.size(), CV_32FC3);

On Python a working solution is to initiate movingAverage using FIRSTcolourFrame.copy().astype("float").
I found the solution on this website

Related

Copying cv::Mat to another creates "assertion failed 0 <= _colRange.start && .."

A pretty simple concept, I have a 640x480 Mat and a 800x480 screen, so I am trying to copy the original image to the center of a black 800x480 image so the aspect ratio is maintained but the whole screen is used.
I followed this post and tried both solutions (direct copy to and region of interest) and get the same error:
OpenCV Error: Assertion failed (0 <= _colRange.start && _colRange.start <= _colRange.end && _colRange.end <= m.cols) in Mat, file /home/pi/opencv-3.0.0/modules/core/src/matrix.cpp, line 464
terminate called after throwing an instance of 'cv::Exception'
what(): /home/pi/opencv-3.0.0/modules/core/src/matrix.cpp:464: error: (-215) 0 <= _colRange.start && _colRange.start <= _colRange.end && _colRange.end <= m.cols in function Mat
Aborted
The offending code:
cv::Mat displayimage = cv::Mat(800, 480, CV_16U, cv::Scalar(0));
modimage1.copyTo(displayimage.rowRange(1,480).colRange(81,720));
I first attempted it with start/end range/row of (0,480) and (80,720), but then the error made it sound like it couldn't start at 0, so then of course I thought I was off by 1 and I started at 1 with the same results. But in actuality, the error is for the COLUMNS and not the ROWS, and with the columns being off by 1 wouldn't even matter. So what doesn't it like about where I'm trying to copy this image to?
Duh, this one was easier than I thought. The cv::Mat() arguments are height THEN width, not width then heigth. Tricky. But I also ran into an error with the wrong number of channels for my mat type, so to make the code bulletproof I just initialized it as the same image type of the image that would be copied to it, so the code below works fine:
cv::Mat displayimage = cv::Mat(480, 800, modimage1.type(), cv::Scalar(0));
modimage1.copyTo(displayimage.rowRange(0,480).colRange(80,720));
you can use cv::copyMakeBorder
#include "opencv2/imgproc.hpp"
#include "opencv2/highgui.hpp"
#include "iostream"
using namespace cv;
using namespace std;
int main(int argc, char* argv[])
{
Mat src = imread(argv[1]);
if (src.empty())
{
cout << endl
<< "ERROR! Unable to read the image" << endl
<< "Press a key to terminate";
cin.get();
return 0;
}
imshow("Source image", src);
Mat dst;
Size dst_dims = Size(800,480);
int top = ( dst_dims.height - src.rows ) / 2;
int bottom = ( (dst_dims.height + 1) - src.rows ) / 2;
int left = ( dst_dims.width - src.cols ) / 2;
int right = ( ( dst_dims.width + 1 ) - src.cols ) / 2;
copyMakeBorder(src, dst, top, bottom, left, right, BORDER_CONSTANT, Scalar(0,0,0));
imshow("New image", dst);
waitKey();
return 0;
}

How to interpret c++ opencv Assertion error messages due to an error in cvtColor function?

Following is an Assertion Error report (displayed on console) when calling cvtColor() function in opencv giving the argument CV_GRAY2BGR on a Mat object which is already a BGR image. I want to know how to interpret this error message by a person who yet doesn't know what the error here. (Hope some erudites won't vote to close this question as off topic, as I know there is a big value in learning to read Assertion or any other error messages for newbees for c++. ) And as I guess this might be most probably a opencv issue on reading assertion errors.
OpenCV Error: Assertion failed (scn == 1 && (dcn == 3 ||
dcn == 4)) in cv::cvtColor, file C:\builds\2_4_PackSlave-win32-vc12-shared\open
cv\modules\imgproc\src\color.cpp, line 3791
I know 2 conditions are tested here,
(scn == 1)
(dcn == 3 || dcn == 4)
and one of them should have failed which has caused the Assertion error. How to distinct and clarify the failed condition? May be I might have to seek into the cvtColor function source code and that would be no problem. (actually I did it but I couldn't find variables with names scn or dcn there in that improc.cpp class there)
This snippet
#include <opencv2\opencv.hpp>
using namespace cv;
int main(int argc, char** argv)
{
// Just a small green BGR image
Mat3b img(10,10,Vec3b(0,255,0));
Mat1b gray;
cvtColor(img, gray, CV_GRAY2BGR); // WARNING: this won't work (on purpose)
return 0;
}
will produce your exact error:
OpenCV Error: Assertion failed (scn == 1 && (dcn == 3 || dcn == 4)) in cv::cvtCo
lor, file C:\builds\2_4_PackSlave-win32-vc12-static\opencv\modules\imgproc\src\c
olor.cpp, line 3789
This code is obviuosly wrong, because you're trying to convert an BGR image from GRAY.
OpenCV is telling you:
Since you're using the code CV_GRAY2BGR, I'm expecting to convert from GRAY (1 channel) source image to a BGR (3 channel) destination image. (I'll allow also BGRA (4 channels) as destination image, even if CV_GRAY2BGRA would be more appropriate in this case.)
In the documentation OpenCV is telling you:
src: input image: 8-bit unsigned, 16-bit unsigned ( CV_16UC... ), or single-precision floating-point.
dst: output image of the same size and depth as src.
code: color space conversion code (see the description below).
dstCn: number of channels in the destination image; if the parameter is 0, the number of the channels is derived automatically from src and code .
In C++, OpenCV say this as
CV_Assert( scn == 1 && (dcn == 3 || dcn == 4));
where scn stands for "Source Channels Number", and dcn stands for "Destination Channels Number".
Now the last point, where do scn and dcn comes from? If you use a debugger and follow the execution path, you'll see in function void cv::cvtColor( InputArray _src, OutputArray _dst, int code, int dcn ) in color.cpp that (comments added by me):
void cv::cvtColor( InputArray _src /* source image*/,
OutputArray _dst /* destination image*/,
int code, /* here is CV_GRAY2BGR*/
int dcn /* defaults to -1*/ )
{
Mat src = _src.getMat(), dst;
...
int scn = src.channels(); // scn is the number of channels of the source image
...
switch( code ) {
...
case CV_GRAY2BGR: case CV_GRAY2BGRA:
if( dcn <= 0 ) dcn = (code==CV_GRAY2BGRA) ? 4 : 3;
// destination channels are set to 3 because of the code CV_GRAY2BGR
// Check that the input arguments are correct
CV_Assert( scn == 1 && (dcn == 3 || dcn == 4));
...
}
calling cvtColor() function in opencv giving the argument CV_GRAY2BGR on a Mat object which is already a BGR image
You have already answered your own question here. The assertion will have originally have been something like:
CV_Assert( scn == 1 && (dcn == 3 || dcn == 4));
Since you're using a BGR Mat, scn - which is the number of channels in the source Mat - will be 3 causing the whole expression to evaluate to false, and the assertion to fail.
The operation you are performing makes no sense. Omit it, and your code will probably work.

Template matching with Orb : error with batchDistance method

I'm trying to detect whether a template image (logo) is present in a pdf document. The document can be either a scan encapsulated in a pdf or a "pure" pdf document, but this is completely random.
First, I convert the pdf document to a png image using ImageMagick's convert tool, then I cut the output images in half because they're so big, and after that I try to match a logo from a database with any of the shapes present in the half-cut image.
To do so, I use an Orb Feature Detector with an Orb Descriptor, and a RobustMatcher (sort of improved BruteForce matcher, source code available here). Here is a snippet of code from my adaptation of it :
// Read input images
Mat image1 = imread(argv[1], CV_LOAD_IMAGE_GRAYSCALE);
Mat image2 = imread(argv[2], CV_LOAD_IMAGE_GRAYSCALE);
if (!image1.data || !image2.data) {
std::cout << " --(!) Error reading images " << std::endl;
exit(1);
}
// Setting up values for ORB Detector
int nfeatures = 800;
//float scaleFactor = 1.10;
int nlevels = 8;
int edgeThreshold = 12;
int firstLevel = 0;
int WTA_K = 2;
int scoreType = 0;
int patchSize = 31;
// Prepare the matcher
RobustMatcher rmatcher;
rmatcher.setConfidenceLevel(0.98);
rmatcher.setMinDistanceToEpipolar(1.0);
rmatcher.setRatio(0.80f);
cv::Ptr<cv::FeatureDetector> pfd = new cv::OrbFeatureDetector(nfeatures, scaleFactor, nlevels, edgeThreshold, firstLevel, WTA_K, scoreType, patchSize);
rmatcher.setFeatureDetector(pfd);
cv::Ptr<cv::DescriptorExtractor> pde = new cv::OrbDescriptorExtractor();
rmatcher.setDescriptorExtractor(pde);
// Match the two images
std::vector<cv::DMatch> matches;
std::vector<cv::KeyPoint> keypoints1, keypoints2;
cv::Mat fundemental = rmatcher.match(image1, image2, matches, keypoints1, keypoints2);
// If nothing could be matched, stop here
if(matches.size() < 4){
exit(2);
}
The code works great on some examples that I chose carefully, with a highly-recognizable logo and a clean image, with certain proportions... etc. But when I try to apply the process to random pdf files, I start to get this error from OpenCV :
OpenCV Error: Assertion failed (type == src2.type() && src1.cols == src2.cols && (type == CV_32F || type == CV_8U)) in batchDistance, file /home/das/Downloads/opencv-2.4.5/modules/core/src/stat.cpp, line 1797
terminate called after throwing an instance of 'cv::Exception'
what(): /home/das/Downloads/opencv-2.4.5/modules/core/src/stat.cpp:1797: error: (-215) type == src2.type() && src1.cols == src2.cols && (type == CV_32F || type == CV_8U) in function batchDistance
Aborted (core dumped)
I checked for this error, and it appeared that src1.cols != src2.cols, and a quick fix for it would be to test the condition before trying to match the images. The problem is that I miss a lot of images doing so, and this would be OK only if I were working on a video stream... but I'm not, and the next image has nothing in common with the previous one, and I can't determine whether my logo was present or not in the document.
Here is the code from stat.cpp, lines 1789 to 1826 : (assertion is at the beginning on line 1797)
void cv::batchDistance( InputArray _src1, InputArray _src2,
OutputArray _dist, int dtype, OutputArray _nidx,
int normType, int K, InputArray _mask,
int update, bool crosscheck )
{
Mat src1 = _src1.getMat(), src2 = _src2.getMat(), mask = _mask.getMat();
int type = src1.type();
CV_Assert( type == src2.type() && src1.cols == src2.cols &&
(type == CV_32F || type == CV_8U));
CV_Assert( _nidx.needed() == (K > 0) );
if( dtype == -1 )
{
dtype = normType == NORM_HAMMING || normType == NORM_HAMMING2 ? CV_32S : CV_32F;
}
CV_Assert( (type == CV_8U && dtype == CV_32S) || dtype == CV_32F);
K = std::min(K, src2.rows);
_dist.create(src1.rows, (K > 0 ? K : src2.rows), dtype);
Mat dist = _dist.getMat(), nidx;
if( _nidx.needed() )
{
_nidx.create(dist.size(), CV_32S);
nidx = _nidx.getMat();
}
if( update == 0 && K > 0 )
{
dist = Scalar::all(dtype == CV_32S ? (double)INT_MAX : (double)FLT_MAX);
nidx = Scalar::all(-1);
}
if( crosscheck )
{
CV_Assert( K == 1 && update == 0 && mask.empty() );
Mat tdist, tidx;
batchDistance(src2, src1, tdist, dtype, tidx, normType, K, mask, 0, false);
So I'm wondering what does this assertion mean ? What are exactly the src1 and src2 files in stat.cpp ? Why do they need to have the same number of columns ?
I tried changing to a Surf detector and extractor, but I still get the error.
If anyone has an idea, do not hesitate to post, I welcome any advice or notice !
Thanks in advance.
EDIT
I have a more precise question now : how do I ensure that src1.cols == src2.cols ? To answer that question, I think I should know what are the transformations applied to my cv::Mat image1 and image2 before batchDistance(...) is called, in order to find a condition on image1 and image2 which will ensure that src1.cols == src2.cols, so my code would look like this :
// Match the two images
std::vector<cv::DMatch> matches;
std::vector<cv::KeyPoint> keypoints1, keypoints2;
if( CONDITION_ON_IMAGE1&IMAGE2_TO_ENSURE_SRC1.COLS==SRC2.COLS ){
cv::Mat fundemental = rmatcher.match(image1, image2, matches, keypoints1, keypoints2);
}
To get rid of errors, You can play with copying and pasting the images into empty one of required size, but this is only a quick and dirty solution for the assertion.
To make detector and descriptor work properly, You might have to get to know how it works. Maybe then You will be able to get images that will work. After reading this article, it looks that ORB will have problems with scaling (they mention it in the conclusion section). This means You will need to find a workaround for it (like image pyramids, or another way to check the image at multiple scales) or use another extractor and descriptor, which is scale and rotation invariant.

Setting up depth for a cv::Mat

I want to test a function that is looking for specific mat depth && number of channels
It has a test...
if (image.channels() == 1 && image.depth() == 8) ...
else if (image.channels() == 1 && image.depth() == 16) ...
else if (image.channels() == 1 && image.depth() == 32) ...
else
{
if ((image.channels() != 3) || (image.depth() != 8))
{printf("Expecting rgb24 input image"); return false;}
...
}
I prefer to test with a made-up mat, to avoid using local resources:
cv::Mat M(255, 255, CV_8UC3, cv::Scalar(0,0,255));
printf("M: %d %d \n", M.channels(), M.depth());
cv::Mat M1(255, 255, CV_32F, cv::Scalar(0,0,255));
cv::Mat M2(255, 255, CV_32FC3, cv::Scalar(0,0,255));
cv::Mat M2(255, 255, CV_8SC3, cv::Scalar(0,0,255));
I have experimented with all kinds of combinations, but if I print, I get 0 depth.
I have also tried to load a png or a jpg file - with same result (I prefer not to use outside files... but I see no reason why this doesn't work)
cv::Mat M3 = cv::imread( "c:/my_image.png", CV_LOAD_IMAGE_COLOR );
cv::Mat M3 = cv::imread( "c:/my_image.jpg", CV_LOAD_IMAGE_COLOR );
They all seem to have depth = 0 ?
Is there something else I have to do ? I can't see anything in documentation.
When you call depth() on Mat, it returns depth values as defined below instead of number of bits:
#define CV_8U 0
#define CV_8S 1
#define CV_16U 2
#define CV_16S 3
#define CV_32S 4
#define CV_32F 5
#define CV_64F 6
And you can use cv::DataDepth::value to figure out which one is which. For example,
cv::DataDepth<unsigned char>::value == CV_8U;
cv::DataDepth<float>::value == CV_32F;
So you should get 0 on all CV_8UCX matrix, and when you load an image, it is usually loaded as CV_8UC3, so you will get 0 as well. But I am not sure why you got 0 on cv::Mat M(255, 255, CV_32FC3), I tested it on my computer, it returned 5.

sYSMALLOc: Assertion Failed error in opencv

The code compiles successfully but I am getting the following error when I try to execute the code with some images.
malloc.c:3096: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
Aborted
My code is:
#include "opencv2/modules/imgproc/include/opencv2/imgproc/imgproc.hpp"
#include "opencv2/modules/highgui/include/opencv2/highgui/highgui.hpp"
#include <stdlib.h>
#include <stdio.h>
using namespace cv;
/// Global variables
int const min_BINARY_value = 0;
int const max_BINARY_value = 255;
Mat src, src_gray, new_image;
const char* window_name = "Web Safe Colors";
/**
* #function main
*/
int main( int argc, char** argv )
{
double sum=0, mean=0;
/// Load an image
src = imread( argv[1], 1 );
/// Convert the image to Gray
cvtColor( src, src_gray, CV_RGB2GRAY );
/// Create new image matrix
new_image = Mat::ones( src_gray.size(), src_gray.type() );
/// Calculate sum of pixels
for( int y = 0; y < src_gray.rows; y++ )
{
for( int x = 0; x < src_gray.cols; x++ )
{
sum = sum + src_gray.at<Vec3b>(y,x)[0];
}
}
/// Calculate mean of pixels
mean = sum / (src_gray.rows * src_gray.cols);
/// Perform conversion to binary
for( int y = 0; y < src_gray.rows; y++ )
{
for( int x = 0; x < src_gray.cols; x++ )
{
if(src_gray.at<Vec3b>(y,x)[0] <= mean)
new_image.at<Vec3b>(y,x)[0] = min_BINARY_value;
else
new_image.at<Vec3b>(y,x)[0] = max_BINARY_value;
}
}
/// Create a window to display results
namedWindow( window_name, CV_WINDOW_AUTOSIZE );
imshow( window_name, new_image );
/// Wait until user finishes program
while(true)
{
int c;
c = waitKey( 20 );
if( (char)c == 27 )
{ break; }
}
}
Can you please help me identify the problem?
I cannot reproduce the exact error message you get. On my computer your program stopped with a segmentation fault.
The reason for this was, that you are accessing the pixels of your gray value images as if they were rgb images. So instead of
new_image.at<Vec3b>(y,x)[0]
you need to use
new_image.at<uchar>(y,x)
Because in a gray scale image every pixel only has a single value instead of a vector of 3 values (red, green and blue). After I applied this changes your program ran without errors and produced the expected output of an thresholded binary image.
It is possible that because of this you are overwriting some other memory opencv currently used and that this memory corruption then lead to your error message.