I am trying to make a visual odometry algorithm work in real time (using my stereo camera). The camera feed gets returned as a single image (i420 pixel format), where I have to manually split the image into a left and right frame. One of the problems that I am running into is when I call cv::triangulatePoints. The function gives me an error saying that the input matrices (meaning the left and right frame) are not continuous.
When I receive the input image from the camera, using:
// Read camera feed
IMAGE_FORMAT fmt = {IMAGE_ENCODING_I420, 50};
BUFFER *buffer = arducam_capture(camera_instance, &fmt, 3000);
if (!buffer)
return -1;
// Store feed in image
cv::Mat image = cv::Mat(cv::Size(width,(int)(height * 1.5)), CV_8UC1, buffer->data);
arducam_release_buffer(buffer);
// Change image to grayscale (grayscale increases FPS)
cv::cvtColor(image, image, cv::COLOR_YUV2GRAY_I420);
if (!image.isContinuous())
std::cout << "image is not continuous" << std::endl;
The image passes the continuity check fine (meaning the image is continuous).
However, after I resize and split the image into a left and right frame, using:
double scale_factor = 640.0 / width;
int custom_width = int(width * scale_factor);
int custom_height = int(height * scale_factor);
// OpenCV resize
cv::Mat frame = cv::Mat(cv::Size(custom_width, (int)(custom_height * 1.5)), CV_8UC1);
cv::resize(image, frame, frame.size(), 0, 0);
// Split image into left and right frame
cv::Mat frame_left = frame(cv::Rect(0, 0, custom_width / 2, (int)(custom_height * 1.5)));
cv::Mat frame_right = frame(cv::Rect(custom_width / 2, 0, custom_width / 2, (int)(custom_height * 1.5)));
if (!frame.isContinuous())
std::cout << "frame is not continuous" << std::endl;
if (!frame_right.isContinuous())
std::cout << "right frame is not continuous" << std::endl;
if (!frame_left.isContinuous())
std::cout << "left frame is not continuous" << std::endl;
The resized image (frame) is continuous, but the left and right frames fail the continuity check (meaning they are not continuous).
So I guess my question is how can I split the image into two different images, while keeping them continuous?
The solution to this problem is actually quite simple:
if (!frame_right.isContinuous()) {
frame_right = frame_right.clone();
if (!frame_left.isContinuous()) {
frame_left = frame_left.clone();
By using the clone() function, you can copy the image and OpenCV will consider it to be a new image. This way the right and left frames will retain continuity (or be set to continuous status).
So splitting the image destroys continuity and cloning will restore continuity.
I have a dicom 3D image which is [512,512,5] (rows, cols, slices). I want to read it with DCMTK toolkit and convert it to a OpenCV Mat object. The image is 16 bits unsigned int.
My question is: Does anyone know the correct way to convert this dicom image into a Mat object? How to properly read all the slices with the method getOutputData?
Based on the comments of #Alan Birtles, there is the possibility to specify the frame you want to read on the getOutputData method. After reading each frame, you simply merge the Mat objects into a single Mat.
I wrote this code to get the whole volume:
DicomImage *image = new DicomImage(file);
// Get the information
unsigned int nRows = image->getHeight();
unsigned int nCols = image->getWidth();
unsigned int nImgs = image->getFrameCount();
vector <Mat> slices(nImgs);
// Loop for each slice
for(int k = 0; k<nImgs; k++){
(Uint16 *) pixelData = (Uint16 *)(image->getOutputData(16 /* bits */,k /* slice */));
slices[k] = Mat(nRows, nCols, CV_16U, pixelData).clone();
}
Mat img;
// Merge the slices in a single img
merge(slices,img);
cout << img.size() << endl;
cout << img.channels() << endl;
// Output:
// [512 x 512]
// 5
I am novice in OpenCV. Recently, I have troubles finding OpenCV functions to convert from Mat to Array. I researched with .ptr and .at methods available in OpenCV APIs, but I could not get proper data. I would like to have direct conversion from Mat to Array(if available, if not to Vector). I need OpenCV functions because the code has to be undergo high level synthesis in Vivado HLS. Please help.
If the memory of the Mat mat is continuous (all its data is continuous), you can directly get its data to a 1D array:
std::vector<uchar> array(mat.rows*mat.cols*mat.channels());
if (mat.isContinuous())
array = mat.data;
Otherwise, you have to get its data row by row, e.g. to a 2D array:
uchar **array = new uchar*[mat.rows];
for (int i=0; i<mat.rows; ++i)
array[i] = new uchar[mat.cols*mat.channels()];
for (int i=0; i<mat.rows; ++i)
array[i] = mat.ptr<uchar>(i);
UPDATE: It will be easier if you're using std::vector, where you can do like this:
std::vector<uchar> array;
if (mat.isContinuous()) {
// array.assign(mat.datastart, mat.dataend); // <- has problems for sub-matrix like mat = big_mat.row(i)
array.assign(mat.data, mat.data + mat.total()*mat.channels());
} else {
for (int i = 0; i < mat.rows; ++i) {
array.insert(array.end(), mat.ptr<uchar>(i), mat.ptr<uchar>(i)+mat.cols*mat.channels());
}
}
p.s.: For cv::Mats of other types, like CV_32F, you should do like this:
std::vector<float> array;
if (mat.isContinuous()) {
// array.assign((float*)mat.datastart, (float*)mat.dataend); // <- has problems for sub-matrix like mat = big_mat.row(i)
array.assign((float*)mat.data, (float*)mat.data + mat.total()*mat.channels());
} else {
for (int i = 0; i < mat.rows; ++i) {
array.insert(array.end(), mat.ptr<float>(i), mat.ptr<float>(i)+mat.cols*mat.channels());
}
}
UPDATE2: For OpenCV Mat data continuity, it can be summarized as follows:
Matrices created by imread(), clone(), or a constructor will always be continuous.
The only time a matrix will not be continuous is when it borrows data (except the data borrowed is continuous in the big matrix, e.g. 1. single row; 2. multiple rows with full original width) from an existing matrix (i.e. created out of an ROI of a big mat).
Please check out this code snippet for demonstration.
Can be done in two lines :)
Mat to array
uchar * arr = image.isContinuous()? image.data: image.clone().data;
uint length = image.total()*image.channels();
Mat to vector
cv::Mat flat = image.reshape(1, image.total()*image.channels());
std::vector<uchar> vec = image.isContinuous()? flat : flat.clone();
Both work for any general cv::Mat.
Explanation with a working example
cv::Mat image;
image = cv::imread(argv[1], cv::IMREAD_UNCHANGED); // Read the file
cv::namedWindow("cvmat", cv::WINDOW_AUTOSIZE );// Create a window for display.
cv::imshow("cvmat", image ); // Show our image inside it.
// flatten the mat.
uint totalElements = image.total()*image.channels(); // Note: image.total() == rows*cols.
cv::Mat flat = image.reshape(1, totalElements); // 1xN mat of 1 channel, O(1) operation
if(!image.isContinuous()) {
flat = flat.clone(); // O(N),
}
// flat.data is your array pointer
auto * ptr = flat.data; // usually, its uchar*
// You have your array, its length is flat.total() [rows=1, cols=totalElements]
// Converting to vector
std::vector<uchar> vec(flat.data, flat.data + flat.total());
// Testing by reconstruction of cvMat
cv::Mat restored = cv::Mat(image.rows, image.cols, image.type(), ptr); // OR vec.data() instead of ptr
cv::namedWindow("reconstructed", cv::WINDOW_AUTOSIZE);
cv::imshow("reconstructed", restored);
cv::waitKey(0);
Extended explanation:
Mat is stored as a contiguous block of memory, if created using one of its constructors or when copied to another Mat using clone() or similar methods. To convert to an array or vector we need the address of its first block and array/vector length.
Pointer to internal memory block
Mat::data is a public uchar pointer to its memory.
But this memory may not be contiguous. As explained in other answers, we can check if mat.data is pointing to contiguous memory or not using mat.isContinous(). Unless you need extreme efficiency, you can obtain a continuous version of the mat using mat.clone() in O(N) time. (N = number of elements from all channels). However, when dealing images read by cv::imread() we will rarely ever encounter a non-continous mat.
Length of array/vector
Q: Should be row*cols*channels right?
A: Not always. It can be rows*cols*x*y*channels.
Q: Should be equal to mat.total()?
A: True for single channel mat. But not for multi-channel mat
Length of the array/vector is slightly tricky because of poor documentation of OpenCV. We have Mat::size public member which stores only the dimensions of single Mat without channels. For RGB image, Mat.size = [rows, cols] and not [rows, cols, channels]. Mat.total() returns total elements in a single channel of the mat which is equal to product of values in mat.size. For RGB image, total() = rows*cols. Thus, for any general Mat, length of continuous memory block would be mat.total()*mat.channels().
Reconstructing Mat from array/vector
Apart from array/vector we also need the original Mat's mat.size [array like] and mat.type() [int]. Then using one of the constructors that take data's pointer, we can obtain original Mat. The optional step argument is not required because our data pointer points to continuous memory. I used this method to pass Mat as Uint8Array between nodejs and C++. This avoided writing C++ bindings for cv::Mat with node-addon-api.
References:
Create memory continuous Mat
OpenCV Mat data layout
Mat from array
Here is another possible solution assuming matrix have one column( you can reshape original Mat to one column Mat via reshape):
Mat matrix= Mat::zeros(20, 1, CV_32FC1);
vector<float> vec;
matrix.col(0).copyTo(vec);
None of the provided examples here work for the generic case, which are N dimensional matrices. Anything using "rows" assumes theres columns and rows only, a 4 dimensional matrix might have more.
Here is some example code copying a non-continuous N-dimensional matrix into a continuous memory stream - then converts it back into a Cv::Mat
#include <iostream>
#include <cstdint>
#include <cstring>
#include <opencv2/opencv.hpp>
int main(int argc, char**argv)
{
if ( argc != 2 )
{
std::cerr << "Usage: " << argv[0] << " <Image_Path>\n";
return -1;
}
cv::Mat origSource = cv::imread(argv[1],1);
if (!origSource.data) {
std::cerr << "Can't read image";
return -1;
}
// this will select a subsection of the original source image - WITHOUT copying the data
// (the header will point to a region of interest, adjusting data pointers and row step sizes)
cv::Mat sourceMat = origSource(cv::Range(origSource.size[0]/4,(3*origSource.size[0])/4),cv::Range(origSource.size[1]/4,(3*origSource.size[1])/4));
// correctly copy the contents of an N dimensional cv::Mat
// works just as fast as copying a 2D mat, but has much more difficult to read code :)
// see http://stackoverflow.com/questions/18882242/how-do-i-get-the-size-of-a-multi-dimensional-cvmat-mat-or-matnd
// copy this code in your own cvMat_To_Char_Array() function which really OpenCV should provide somehow...
// keep in mind that even Mat::clone() aligns each row at a 4 byte boundary, so uneven sized images always have stepgaps
size_t totalsize = sourceMat.step[sourceMat.dims-1];
const size_t rowsize = sourceMat.step[sourceMat.dims-1] * sourceMat.size[sourceMat.dims-1];
size_t coordinates[sourceMat.dims-1] = {0};
std::cout << "Image dimensions: ";
for (int t=0;t<sourceMat.dims;t++)
{
// calculate total size of multi dimensional matrix by multiplying dimensions
totalsize*=sourceMat.size[t];
std::cout << (t>0?" X ":"") << sourceMat.size[t];
}
// Allocate destination image buffer
uint8_t * imagebuffer = new uint8_t[totalsize];
size_t srcptr=0,dptr=0;
std::cout << std::endl;
std::cout << "One pixel in image has " << sourceMat.step[sourceMat.dims-1] << " bytes" <<std::endl;
std::cout << "Copying data in blocks of " << rowsize << " bytes" << std::endl ;
std::cout << "Total size is " << totalsize << " bytes" << std::endl;
while (dptr<totalsize) {
// we copy entire rows at once, so lowest iterator is always [dims-2]
// this is legal since OpenCV does not use 1 dimensional matrices internally (a 1D matrix is a 2d matrix with only 1 row)
std::memcpy(&imagebuffer[dptr],&(((uint8_t*)sourceMat.data)[srcptr]),rowsize);
// destination matrix has no gaps so rows follow each other directly
dptr += rowsize;
// src matrix can have gaps so we need to calculate the address of the start of the next row the hard way
// see *brief* text in opencv2/core/mat.hpp for address calculation
coordinates[sourceMat.dims-2]++;
srcptr = 0;
for (int t=sourceMat.dims-2;t>=0;t--) {
if (coordinates[t]>=sourceMat.size[t]) {
if (t==0) break;
coordinates[t]=0;
coordinates[t-1]++;
}
srcptr += sourceMat.step[t]*coordinates[t];
}
}
// this constructor assumes that imagebuffer is gap-less (if not, a complete array of step sizes must be given, too)
cv::Mat destination=cv::Mat(sourceMat.dims, sourceMat.size, sourceMat.type(), (void*)imagebuffer);
// and just to proof that sourceImage points to the same memory as origSource, we strike it through
cv::line(sourceMat,cv::Point(0,0),cv::Point(sourceMat.size[1],sourceMat.size[0]),CV_RGB(255,0,0),3);
cv::imshow("original image",origSource);
cv::imshow("partial image",sourceMat);
cv::imshow("copied image",destination);
while (cv::waitKey(60)!='q');
}
Instead of getting image row by row, you can put it directly to an array. For CV_8U type image, you can use byte array, for other types check here.
Mat img; // Should be CV_8U for using byte[]
int size = (int)img.total() * img.channels();
byte[] data = new byte[size];
img.get(0, 0, data); // Gets all pixels
byte * matToBytes(Mat image)
{
int size = image.total() * image.elemSize();
byte * bytes = new byte[size]; //delete[] later
std::memcpy(bytes,image.data,size * sizeof(byte));
}
You can use iterators:
Mat matrix = ...;
std::vector<float> vec(matrix.begin<float>(), matrix.end<float>());
cv::Mat m;
m.create(10, 10, CV_32FC3);
float *array = (float *)malloc( 3*sizeof(float)*10*10 );
cv::MatConstIterator_<cv::Vec3f> it = m.begin<cv::Vec3f>();
for (unsigned i = 0; it != m.end<cv::Vec3f>(); it++ ) {
for ( unsigned j = 0; j < 3; j++ ) {
*(array + i ) = (*it)[j];
i++;
}
}
Now you have a float array. In case of 8 bit, simply change float to uchar, Vec3f to Vec3b and CV_32FC3 to CV_8UC3.
If you know that your img is 3 channel, than you can try this code
Vec3b* dados = new Vec3b[img.rows*img.cols];
for (int i = 0; i < img.rows; i++)
for(int j=0;j<img.cols; j++)
dados[3*i*img.cols+j] =img.at<Vec3b>(i,j);
If you wanna check the (i,j) vec3b you can write
std::cout << (Vec3b)img.at<Vec3b>(i,j) << std::endl;
std::cout << (Vec3b)dados[3*i*img.cols+j] << std::endl;
Since answer above is not very accurate as mentioned in its comments but its "edit queue is full", I have to add correct one-liners.
Mat(uchar, 1 channel) to vector(uchar):
std::vector<uchar> vec = (image.isContinuous() ? image : image.clone()).reshape(1, 1); // data copy here
vector(any type) to Mat(the same type):
Mat m(vec, false); // false(by default) -- do not copy data
I am working with images in C++ with OpenCV.
I wrote code with an uchar array of two dimensions where I can read pixel values of an image, uploaded with imread in grayscale using .at< uchar>(i,j).
However I would like to do the same thing for color images. Since I know that to access the pixels values I now need .at< Vec3b>(i,j)[0], .at< Vec3b>(i,j)[1] and .at< Vec3b>(i,j)[2], I made a similar Vec3b 2d arrays.
But I don't know how to fill this array with the pixel values. It has to be a 2D array.
I tried:
array[width][height].val[0]=img.at< Vec3b>(i,j)[0]
but that didn't work.
Didn't find an answer on the OpenCV doc or here neither.
Anybody has an idea?
I've included some of my code. I need an array because I already have my whole algorithm working, using an array, for the images in grayscale with only one channel.
The grayscale code is like that:
for(int i=0;i<height;i++){
for(int j=0;j<width;j++){
image_data[i*width+j]=all_images[nb_image-1].at< uchar>(i,j);
}
}
Where I read from:
std::vector< cv::Mat> all_images
each image (I have a long sequence), retrieves the pixel values in the uchar array image_data, and processes them.
I want now to do the same but for RGB images, and I can't manage to read the data pixel of each channel and put them in an array.
This time image_data is a Vec3b array, and the code I'm trying looks like this:
for(int i=0;i<height;i++){
for(int j=0;j<width;j++){
image_data[0][i*width+j]=all_images[nb_image-1].at<cv::Vec3b>(i,j)[2];
image_data[1][i*width+j]=all_images[nb_image-1].at<cv::Vec3b>(i,j)[1];
image_data[2][i*width+j]=all_images[nb_image-1].at<cv::Vec3b>(i,j)[0];
}
}
But this doesn't work, so I am now at loss I don't know how to succeed to fill the image_data array with the values of all three channels, without changing the code structure as this array is then used on my image processing algorithm.
I don't understand exactly what you are trying to do.
You can directly read a color image with:
cv::Mat img = cv::imread("image.jpeg",1);
Your matrix (img) type will be CV_8UC3, then you can access to each pixel like you said using:
img.at<cv::Vec3b>(row,col)[channel].
If you have a 2D array of Vec3b as Vec3b myArray[n][m];
You can access the values like that:
myArray[i][j](k) where k={1,2,3} since Vec3b is a row matrix.
Here is the code I just tested, and it works.
#include <iostream>
#include <cstdlib>
#include <opencv/cv.h>
#include <opencv/highgui.h>
int main(int argc, char**argv){
cv::Mat img = cv::imread("image.jpg",1);
cv::imshow("image",img);
cv::waitKey(0);
cv::Vec3b firstline[img.cols];
for(int i=0;i<img.cols;i++){
// access to matrix
cv::Vec3b tmp = img.at<cv::Vec3b>(0,i);
std::cout << (int)tmp(0) << " " << (int)tmp(1) << " " << (int)tmp(2) << std::endl;
// access to my array
firstline[i] = tmp;
std::cout << (int)firstline[i](0) << " " << (int)firstline[i](0) << " " << (int)firstline[i](0) << std::endl;
}
return EXIT_SUCCESS;
}
In you edited first message, this line is strange:
image_data[0][i*width+j]=all_images[nb_image-1].at<cv::Vec3b>(i,j)[2];
If image data is your colored image, then it should be written like this:
image_data[i][j] = all_images[nb_image-1].at<cv::Vec3b>(i,j);
CompVision once again, I'm working with jpeg images in my application. Just because I'm a bit familiar with MFC and ATL, I used CImage to access pixel values.
For my needs I calculate brightness matrix for the image during initialization. Function goes like this (Image is the name of my own class, unimportant, bright is float[][]):
void Image::fillBrightnessMatrix(){
COLORREF val;
for(int i=0;i<width;i++){
for(int j=0; j<height;j++){
val=src.GetPixel(i,j);
bright[i][j]=rgb_to_L(val);
}
}
}
Where src is an instance of CImage class, rgb_to_L - some function that calculates brightness of the color.
Examining the performance of my app, I discovered that GetPixel is the most expensive operation, and it significantly (really, ~700 times slower than any other operation) slows down the whole initializing of image. The question is, which library can you suggest for fast access to single pixel values? I don't need any other operations but loading jpeg image and accessing single pixels. Performance is important, because my application works with set of ~3000 images and I can't wait for hours to get results.
Use CBitmap::GetBits() to get a raw pointer to the pixel data. You can now directly party on the pixels without going through the expensive GetPixel() method. There are a number of things you need to be careful with when you do this:
You have to use CBitmap::GetPitch() to calculate the offset to the start of a line. The pitch is not the same as the width.
Lines in the bitmap are stored upside-down
You have to deal with the pixel format yourself. A 24bpp image stores 3 bytes per pixel. An indexed format like 8bpp requires looking up the color in the color table. 32bpp is the easy one, 4 bytes per pixel and the pitch is always the same as the width.
I always recommend OpenCV.
This is a humble code snippet to get you started:
IplImage* pRGBImg = cvLoadImage("c:\\test.jpg", CV_LOAD_IMAGE_UNCHANGED);
if (!pRGBImg)
{
std::cout << "!!! cvLoadImage failed !!!" << std::endl;
exit(1);
}
int width = pRGBImg->width;
int height = pRGBImg->height;
int bpp = pRGBImg->nChannels;
for (int i=0; i < width*height*bpp; i+=bpp)
{
if (!(i % (width*bpp))) // print empty line for better readability
std::cout << std::endl;
std::cout << std::dec << "R:" << (int) pRGBImg->imageData[i] <<
" G:" << (int) pRGBImg->imageData[i+1] <<
" B:" << (int) pRGBImg->imageData[i+2] << " ";
}
You should probably extract the jpeg to raw data, then access the raw data instead of GetPixel.