Opencv convolution matrix gives unusual results - c++

So I have a program that is trying to apply a simple 3x3 convolution matrix to an image.
This is the function that is doing the work:
Mat process(Mat image) {
int x = 2;
int y = 2;
Mat nimage(image); //just a new mat to put the resulting image on
while (y < image.rows-2) {
while (x < image.cols-2) {
nimage.at<uchar>(y,x) = //apply matrix to pixel
image.at<char>(y-1,x-1)*matrix[0]+
image.at<char>(y-1,x)*matrix[1]+
image.at<char>(y-1,x+1)*matrix[2]+
image.at<char>(y,x-1)*matrix[3]+
image.at<char>(y,x)*matrix[4]+
image.at<char>(y,x+1)*matrix[5]+
image.at<char>(y+1,x-1)*matrix[6]+
image.at<char>(y+1,x)*matrix[7]+
image.at<char>(y+1,x+1)*matrix[8];
//if (total < 0) total = 0;
//if (total > 255) total = 255;
//cout << (int)total << ": " << x << "," << y << endl;
x++;
}
x = 0;
y++;
}
cout << "done" << endl;
return nimage;
}
And the matrix looks like this
double ar[9] = {-1,0,0,
0,2,0,
0,0,0};
And the image that is used as input looks like this:
The desired output (I ran the same matrix on the input image in GIMP):
And the result is... weird:
I think this has to do with the data type I use when I set a pixel of the new image (nimage.at<uchar>(y,x) = ...), because whenever I change it I get a different, yet still incorrect result.

From the OpenCV documentation about the copy constructor of Mat, emphasis mine:
m – Array that (as a whole or partly) is assigned to the constructed matrix. No data is copied by these constructors. Instead, the header pointing to m data or its sub-array is constructed and associated with it. The reference counter, if any, is incremented. So, when you modify the matrix formed using such a constructor, you also modify the corresponding elements of m. If you want to have an independent copy of the sub-array, use Mat::clone().
So
Mat nimage(image); //just a new mat to put the resulting image on
doesn't actually create a new matrix; it creates a new Mat object, but that object still refers to the same matrix. From then on nimage.at(y,x) acts like image.at(y,x).
To copy the image, use
Mat nimage(image.clone()); //just a new mat to put the resulting image on

Related

C++ : Create 3D array out of stacking 2D arrays

In Python I normally use functions like vstack, stack, etc to easily create a 3D array by stacking 2D arrays one onto another.
Is there any way to do this in C++?
In particular, I have loaded a image into a Mat variable with OpenCV like:
cv::Mat im = cv::imread("image.png", 0);
I would like to make a 3D array/Mat of N layers by stacking copies of that Mat variable.
EDIT: This new 3D matrix has to be "travellable" by adding an integer to any of its components, such that if I am in the position (x1,y1,1) and I add +1 to the last component, I arrive to (x1,y1,2). Similarly for any of the coordinates/components of the 3D matrix.
SOLVED: Both answers from #Aram and #Nejc do exactly what expected. I set #Nejc 's answer as the correct one for his shorter code.
The Numpy function vstack returns a contiguous array. Any C++ solution that produces vectors or arrays of cv::Mat objects does not reflect the behaviour of vstack in this regard, becase separate "layers" belonging to individual cv::Mat objects will not be stored in contiguous buffer (unless a careful allocation of underlying buffers is done in advance of course).
I present the solution that copies all arrays into a three-dimensional cv::Mat object with a contiguous buffer. As far as the idea goes, this answer is similar to Aram's answer. But instead of assigning pixel values one by one, I take advantage of OpenCV functions. At the beginning I allocate the matrix which has a size N X ROWS X COLS, where N is the number of 2D images I want to "stack" and ROWS x COLS are dimensions of each of these images.
Then I make N steps. On every step, I obtain the pointer to the location of the first element along the "outer" dimension. I pass that pointer to the constructor of temporary Mat object that acts as a kind of wrapper around the memory chunk of size ROWS x COLS (but no copies are made) that begins at the address that is pointed-at by pointer. I then use copyTo method to copy i-th image into that memory chunk. Code for N = 2:
cv::Mat img0 = cv::imread("image0.png", CV_IMREAD_GRAYSCALE);
cv::Mat img1 = cv::imread("image1.png", CV_IMREAD_GRAYSCALE);
cv::Mat images[2] = {img0, img1}; // you can also use vector or some other container
int dims[3] = { 2, img0.rows, img0.cols }; // dimensions of new image
cv::Mat joined(3, dims, CV_8U); // same element type (CV_8U) as input images
for(int i = 0; i < 2; ++i)
{
uint8_t* ptr = &joined.at<uint8_t>(i, 0, 0); // pointer to first element of slice i
cv::Mat destination(img0.rows, img0.cols, CV_8U, (void*)ptr); // no data copy, see documentation
images[i].copyTo(destination);
}
This answer is in response to the question above of:
In Python I normally use functions like vstack, stack, etc to easily create a 3D array by stacking 2D arrays one onto another.
This is certainly possible, you can add matrices into a vector which would be your "stack"
For instance you could use a
std::vector<cv::Mat>>
This would give you a vector of mats, which would be one slice, and then you could "layer" those by adding more slices vector
If you then want to have multiple stacks you can add that vector into another vector:
std::vector<std::vector<cv::Mat>>
To add matrix to an array you do:
myVector.push_back(matrix);
Edit for question below
In such case, could I travel from one position (x1, y1, z1) to an immediately upper position doing (x1,y1,z1+1), such that my new position in the matrix would be (x1,y1,z2)?
You'll end up with something that looks a lot like this. If you have a matrix at element 1 in your vector, it doesn't really have any relationship to the element[2] except for the fact that you have added it into that point. If you want to build relationships then you will need to code that in yourself.
You can actually create a 3D or ND mat with opencv, you need to use the constructor that takes the dimensions as input. Then copy each matrix into (this case) the 3D array
#include <opencv2/opencv.hpp>
using namespace cv;
using namespace std;
int main() {
// Dimensions for the constructor... set dims[0..2] to what you want
int dims[] = {5, 5, 5}; // 5x5x5 3d mat
Mat m = Mat::zeros(5, 5, CV_8UC1);
for (size_t i = 0; i < 5; i++) {
for (size_t k = 0; k < 5; k++) {
m.at<uchar>(i, k) = i + k;
}
}
// Mat with constructor specifying 3 dimensions with dimensions sizes in dims.
Mat 3DMat = Mat(3, dims, CV_8UC1);
// We fill our 3d mat.
for (size_t i = 0; i < m2.size[0]; i++) {
for (size_t k = 0; k < m2.size[1]; k++) {
for (size_t j = 0; j < m2.size[2]; j++) {
3DMat.at<uchar>(i, k, j) = m.at<uchar>(k, j);
}
}
}
// We print it to show the 5x5x5 array.
for (size_t i = 0; i < m2.size[0]; i++) {
for (size_t k = 0; k < m2.size[1]; k++) {
for (size_t j = 0; j < m2.size[2]; j++) {
std::cout << (int) 3DMat.at<uchar>(i, k, j) << " ";
}
std::cout << endl;
}
std::cout << endl;
}
return 0;
}
Based on the question and comments, I think you are looking for something like this:
std::vector<cv::Mat> vec_im;
//In side for loop:
vec_im.push_back(im);
Then, you can access it by:
Scalar intensity_1 = vec_im[z1].at<uchar>(y, x);
Scalar intensity_2 = vec_im[z2].at<uchar>(y, x);
This assumes that the image is single channel.

Mat cells set to NULL in OpenCV?

Quick summary:
I create a cv::Mat by
cv::Mat m = cv::Mat::zeros(MAP_HEIGHT, MAP_WIDTH, CV_8UC1)
My approach after this is to see if i have any polygons in a list of polygons, and if i do, fill them in, and lastly i assign m to my public cv::Mat map (defined in the header-file).
What happens is basically:
cv::Mat m = cv::Mat::zeros(MAP_HEIGHT, MAP_WIDTH, CV_8UC1);
// possibly fill polygons with 1's. Nothing happens if there are no polygons
map = m;
The logic of my program is that position x,y is allowed if a 0 is occupying the cell. So no polygons => all map should be 'legit'.
I have defined this method to check whether a given x-y coordinate is allowed.
bool Map::isAllowed(bool res, int x, int y) {
unsigned char allowed = 0;
res = (map.ptr<unsigned char>(y)[x] == allowed);
}
Now the mystery begins.
cout << cv::countNonZero(map) << endl; // prints 0, meaning all cells are 0
for(int i = 0; i < MAP_HEIGHT; i++) {
unsigned char* c = map.ptr<unsigned char>(i);
for(int j = 0; j < MAP_WIDTH; j++) {
cout << c[j] << endl;
}
} // will print nothing, only outputs empty lines, followed by a newline.
If i print (c[j] == NULL) it prints 1.
If i print the entire Mat i see only 0's flashing over my screen, so they are clearly there.
Why does isAllowed(bool, x, y) return false for (0,0), when there is clearly a 0 there?
Let me know if any more information is needed, thanks!
Problem is solved now, here are my mistakes for future reference:
1: When printing, #Miki pointed out that unsigned characters -> ASCII value gets printed, not numerical representation.
2: in isAllowedPosition(bool res, int x, int y), res has a primitive type. Aka this is pushed on the stack and not a reference to a memorylocation. When writing to it, i write to the local copy and not to the one passed in as an argumet.
Two possible fixes, either pass in a pointer to a memorylocation and write to that, or simply return the result.
Since your data type is uchar (aka unsigned char), you're printing the ASCII value. Use
cout << int(c[j]) << endl;
to print the actual value.
Also map.ptr<unsigned char>(y)[x] can be rewritten simply as map.at<uchar>(y,x), or if you use Mat1b as map(y,x)

How does the return value "res" is updated? (ConcativeMat Con NN)

I have a questions about a for loop and its return value. This is C++ code, and I'm using openCV 2.4V.
Input to this function is max value of 600 images with pooling.
600 images << pooling << max value points.
The size of "res" matrix is 600x128 and vec.size() = 600.
For me, within the for loop, the res never get updated, however return value is not zeros.
I suspected
"ptmat.copyTo(subView)"
because, I thought that is not necessary line. However when I took that out, res did not get updated(being zero like initial Mat). Can anybody explain how does the res value get updated?
Also why does this function is called concatenate..?
Mat
concatenateMat(vector<vector<Mat> > &vec) {
int subFeatures = vec[0][0].rows * vec[0][0].cols;
int height = vec[0].size() * subFeatures;
int width = vec.size();
Mat res = Mat::zeros(height, width, CV_64FC1);
for (int i = 0; i<vec.size(); i++) {
for (int j = 0; j<vec[i].size(); j++) {
Rect roi = Rect(i, j * subFeatures, 1, subFeatures);
Mat subView = res(roi);
Mat ptmat = vec[i][j].reshape(0, subFeatures);
ptmat.copyTo(subView);
}
}
return res;
}
According to OpenCV documentation, the Mat::operator() does not make a copy of matrix data, thus any change to subView matrix object in the loop will be reflected in res matrix object as well. That's the line you've mentioned:
ptmat.copyTo(subView);
It's called concatenate because it concatenates 2D vector of Mat objects into a single one.

Why do I get different values when using different datatypes when accessing pixels in a matrix?

I have a single channel grayscale image (slice).
cout << "num" << slice.channels() << ends; //outputs 1
for(int x = 0;x<=slice.cols;x++){
for(int y = 0;y<=slice.rows;y++){
Vec3b currentPoint = slice.at<Vec3b>(x,y);
cout << currentPoint;
}
}
however, when I try to access a pixel and expect currentPoint to be a single int as it is a single channel image. However, i get [32, 36, 255] which is odd, as it implies three channels. I appreciate I am using a type that says vec3b, but even so, where is it getting the other two elements from?
So I replace Vec3b with uchar, then i get lots of \377. That is even more confusing.
Even when I do have a 3 channel image, I get odd outputs when trying to access a single element of Vec3b (i get more \377).
How can this make sense? I must be mis understanding how the at() method is used.
Firstly, how do I get a single output for each pixel (0-255)?
Also, where am I going wrong when i see \377?
A lot of stuff for a few lines of code...
Since your image is a grayscale image, you should access it with at<uchar>.
Pay attention that the at<> function accepts (rows, cols), which is the opposite of (x,y).
It's faster to scan by line, since the matrix is stored row-wise in memory.
To print out the value of a uchar, you need to cast to int, or you get the ASCII coded character.
The loops should not be <=, but instead <, or you go out of bounds.
So:
for(int y = 0; y < slice.rows; y++) {
for(int x = 0; x < slice.cols; x++) {
uchar currentPoint = slice.at<uchar>(y,x);
cout << int(currentPoint) << " ";
}
cout << "\n";
}

opencv C++ neural network predict() function throws "Bad argument" error

I have managed to train a neural network to recognize numbers in an image and have saved the network parameters to an .xml file.
However, when testing the network against a new image the code fails at the predict() stage with the error:
OpenCV Error: Bad argument (Both input and output must be floating-point matrices of the same type and have the same number of rows) in CvANN_MLP::predict, file ........\opencv\modules\ml\src\ann_mlp.cpp, line 279.
ann_mlp.cpp line 279 is:
if( !CV_IS_MAT(_inputs) || !CV_IS_MAT(_outputs) ||
!CV_ARE_TYPES_EQ(_inputs,_outputs) ||
(CV_MAT_TYPE(_inputs->type) != CV_32FC1 &&
CV_MAT_TYPE(_inputs->type) != CV_64FC1) ||
_inputs->rows != _outputs->rows )
CV_Error( CV_StsBadArg, "Both input and output must be floating-point matrices "
"of the same type and have the same number of rows" );
I have checked input rows by running this code:
cv::Size s = newVec.size();
int rows = s.height;
int cols = s.width;
cout << "newVec dimensions: " << rows << " x " << cols << endl;
...and it comes out with the expected 1 x 900 vector / matrix.
I have set the input and output matrices to be CV_32FC1 as per the error dialog like this:
Input matrix
cv::Mat newVec(1, 900, CV_32FC1);
newVec = crop_img.reshape(0, 1); //reshape / unroll image to vector
CvMat n = newVec;
newVec = cv::Mat(&n);
Output matrix
cv::Mat classOut = cvCreateMatHeader(1, CLASSES, CV_32FC1);
And I try to run the prediction function like this:
CvANN_MLP* nnetwork = new CvANN_MLP;
nnetwork->load("nnetwork.xml", "nnetwork");
int maxIndex = 0;
cv::Mat classOut = cvCreateMatHeader(1, CLASSES, CV_32FC1);
//prediction
nnetwork->predict(newVec, classOut);
float value;
float maxValue = classOut.at<float>(0, 0);
for (int index = 1; index<CLASSES; index++)
{
value = classOut.at<float>(0, index);
if (value>maxValue)
{
maxValue = value;
maxIndex = index;
}
}
Any ideas? Much appreciated...
I suspect the problem is your input, not your output.
First it's important to understand that OpenCV deserves a lot of the blame for this, not you. Their C++ API is quite mediocre, and it caused major confusion to you.
See, normally in C++ when you define a 1x900 matrix of floats, it stays a matrix of floats. C++ has strong type safety.
OpenCV does not. If you assign a matrix of bytes to a matrix of floats, the latter will change its type (!).
Your code initializes newVec to such a matrix of floats, then assigns a second matrix, and then yet another matrix. I suspect that crop_img is still an image, i.e. 8 bits. Reshaping it will make it 1x900, but not floating point. That's the job of .convertTo.