OpenCV reads an image in NCHW format (Number of samples x Channels x Height x Width), I need to convert it to NHWC format (move 2nd dimension of array to last). Is there an efficient way to convert from NCHW to NHWC in C++? I can do this with a 3 for loops, but obviously this is not efficient at all.
This straightforward solution worked for me with OpenCV C++:
static void hwc_to_chw(cv::InputArray src, cv::OutputArray dst) {
std::vector<cv::Mat> channels;
cv::split(src, channels);
// Stretch one-channel images to vector
for (auto &img : channels) {
img = img.reshape(1, 1);
}
// Concatenate three vectors to one
cv::hconcat( channels, dst );
}
With OpenCV >= 4.6, you can use transposeND (from opencv2/core.hpp) for such kind of convertion:
std::vector<int> order = {0, 2, 3, 1};
Mat inp, out; // inp: NCHW, out: NHWC
transposeND(inp, order, out);
Related
I'm trying to implement color conversion from RGB-LMS and LMS-RGB back and using reshape for multiplication matrix, following answer from this question : Fastest way to apply color matrix to RGB image using OpenCV 3.0?
My ori Mat object is from an image with 3 channel (RGB), and I need to multiply them with matrix of 1 channel (lms), it seems like I have an issue with the matrix type. I've read reshape docs and questions related to this issue, like Issues multiplying Mat matrices, and I believe I have followed the instructions.
Here's my code : [UPDATED : Convert into flat image]
void test(const Mat &forreshape, Mat &output, Mat &pic, int rows, int cols)
{
Mat lms(3, 3, CV_32FC3);
Mat rgb(3, 3, CV_32FC3);
Mat intolms(rows, cols, CV_32F);
lms = (Mat_<float>(3, 3) << 1.4671, 0.1843, 0.0030,
3.8671, 27.1554, 3.4557,
4.1194, 45.5161 , 17.884 );
/* switch the order of the matrix according to the BGR order of color on OpenCV */
Mat transpose = (3, 3, CV_32F, lms).t(); // this will do transpose from matrix lms
pic = forreshape.reshape(1, rows*cols);
Mat flatFloatImage;
pic.convertTo(flatFloatImage, CV_32F);
rgb = flatFloatImag*transpose;
output = rgb.reshape(3, cols);
}
I define my Mat object, and I have converted it into float using convertTo
Mat ori = imread("ori.png", CV_LOAD_IMAGE_COLOR);
int rows = ori.rows;
int cols = ori.cols;
Mat forreshape;
ori.convertTo(forreshape, CV_32F);
Mat pic(rows, cols, CV_32FC3);
Mat output(rows, cols, CV_32FC3);
Error is :
OpenCV Error: Assertion failed (type == B.type() && (type == CV_32FC1 || type == CV_64FC1 || type == CV_32FC2 || type == CV_64FC2)) ,
so it's the type issue.
I tried to change all type into either 32FC3 of 32FC1, but doesn't seem to work. Any suggestion ?
I believe what you need is to convert your input to a flat image and than multiply them
float lms [] = {1.4671, 0.1843, 0.0030,
3.8671, 27.1554, 3.4557,
4.1194, 45.5161 , 17.884};
Mat lmsMat(3, 3, CV_32F, lms );
Mat flatImage = ori.reshape(1, ori.rows * ori.cols);
Mat flatFloatImage;
flatImage.convertTo(flatFloatImage, CV_32F);
Mat mixedImage = flatFloatImage * lmsMat;
Mat output = mixedImage.reshape(3, imData.rows);
I might have messed up with lms matrix there, but I guess you will catch up from here.
Also see 3D matrix multiplication in opencv for RGB color mixing
EDIT:
Problem with distortion is that you got overflow after float to 8U conversion. This would do the trick:
rgb = flatFloatImage*transpose;
rgb.convertTo(pic, CV_32S);
output = pic.reshape(3, rows)
Output:
;
Also I'm not sure but quick google search gives me different matrix for LMS see here. Also note that opencv stores colors in B-G-R format instead of RGB so change your mix mtraixes recordingly.
How can I permute dimensions in cv::Mat from CxWxH to WxHxC (Width x Height x Channels)?
I.e. how can I convert Mat:
from cv::Mat frame (1000, 1000, CV_8UC3) = cv::imread("image.jpg", -1);
with dimesions: channels, width, height
to Mat with dimesions: width, height, channels
int sizes_inputs[] = { 3, 1000, 1000 };
cv::Mat out_image(3, sizes_inputs, CV_8UC);
Is there in OpenCV a ready-made fast function for such a conversion? Or should I implement this algorithm myself?
If you wish to treat the underlying data differently, then you can use reshape. The data is stored as BGRBGRBGR....
Otherwise you will have to shuffle the data yourself.
cv::reshape and cv::minChannels may be handy for this.
You can emulate it. I have not tested the code but something like this should do:
cv::Mat frame (1000, 1000, CV_8UC3) = cv::imread("image.jpg", -1);
int sizes_inputs[] = { 3, 1000, 1000 };
cv::Mat out_image(3, sizes_inputs, CV_8UC);
cv::Mat slices[] = {cv::Mat(1000, 1000, CV_8UC, out_image.ptr()),
cv::Mat(1000, 1000, CV_8UC, out_image.ptr() + 1000*1000),
cv::Mat(1000, 1000, CV_8UC, out_image.ptr() + 1000*1000*2)};
cv::split(frame, slices);
Is there a way to convert from RGB to YUYV (YUY 4:2:2) format? I noted that OpenCV has reverse operation, but not RGB to YUYV for some reason. Maybe someone can point to code which does that (even outside of OpenCV library)?
UPDATE
I found libyuv library which may work for this purpose by doing BGR to ARGB conversion and then ARGB to YUY2 format (hopefully this is the same as YUYV 4:2:2). But it doesn't seem to work. Do you happen to know what yuyv buffer dimensions/type should look like? What its stride?
To clarify YUYV and YUY2 are the same formats if it helps.
UPDATE 2
Here is my code of using libyuv library:
Mat frame;
// Convert original image im from BGR to BGRA for further use in libyuv
cvtColor(im, frame, CVX_BGR2BGRA);
// Actually libyuv requires ARGB (i.e. reverse of BGRA), so I swap channels here
int from_to[] = { 0,3, 1,2, 2,1, 3,0 };
mixChannels(&frame, 1, &frame, 1, from_to, 4);
// This is the most confusing part. Not sure what argb_stride suppose to be - length of a row in bytes or size of single value in the array?
const uint8_t* argb_data = frame.data;
int argb_stride = 8;
// Also it is not clear what size of yuyv frame should be since we duplicate one Y
Mat yuyv(frame.rows, frame.cols, CVX_8UC2);
uint8_t* yuyv_data = yuyv.data;
int yuyv_stride = 16;
// Do actual conversion
libyuv::ARGBToYUY2(argb_data, argb_stride, yuyv_data, yuyv_stride,
frame.cols, frame.rows);
// Then I feed yuyv_data to video stream buffer and see green or purple image instead of video stream.
UPDATE 3
Mat frame;
cvtColor(im, frame, CVX_BGR2BGRA);
// ARGB
int from_to[] = { 0,3, 1,2, 2,1, 3,0 };
Mat rgba(frame.size(), frame.type());
mixChannels(&frame, 1, &rgba, 1, from_to, 4);
const uint8_t* argb_data = rgba.data;
int argb_stride = rgba.cols*4;
Mat yuyv(rgba.rows, rgba.cols, CVX_8UC2);
uint8_t* yuyv_data = yuyv.data;
int yuyv_stride = width * 2;
int res = libyuv::ARGBToYUY2(argb_data, argb_stride, yuyv_data, yuyv_stride, rgba.cols, rgba.rows);
It appears that although method is called ARGBToYUY2 it requires BGRA order of channels (not reverse).
I'am using OpenCV in a C++ project. I have obtained a depthmap from a stereovision camera and would like to filter it with a median filter. My Depthmap is a cv::Mat_< double>. (I can not change the format, only can convert it).
Is there a possibility to achieve the functionality of the median filter for a cv::Mat_< double> input?
medianBlur(cv::Mat_<double> src, cv::Mat_<double> dst, int ksize)
In the documentation I only found support for CV_8U, CV_16U, or CV_32F images.
http://docs.opencv.org/modules/imgproc/doc/filtering.html?highlight=medianblur#medianblur
Is there a solution for my problem availible in the OpenCV library.
Should I make my own median filter? If I have to, do you have any good reference?
You can do it like:
cv::Mat_<double> input;
cv::Mat_<double> output; // or cv::Mat output;
cv::medianBlur(input, output, 3);
PS: you have to make sure that int ksize can only be 3 or 5 for your input (double mat). As said in the documentation:
Parameters:
src – input 1-, 3-, or 4-channel image; when ksize is 3 or 5, the image depth should be CV_8U, CV_16U, or CV_32F, for larger aperture sizes, it can only be CV_8U.
...
ksize – aperture linear size; it must be odd and greater than 1, for example: 3, 5, 7 ...
can used and cv::UMat for operate with OpenCL/CUDA
cv::UMat input;
cv::UMat output;
for ksize >5
cv::UMat tmp;
if (ksize>5){
switch (input.type()) {
case CV_16UC1:
case CV_16UC3:
case CV_16UC4:
input.convertTo(tmp,CV_8U,1./255.);
break;
case CV_32FC1:
case CV_32FC3:
case CV_32FC4:
input.convertTo(tmp,CV_8U,255.);
break;
default:
input.assignTo(tmp);
break;
}
}else{
input.assignTo(tmp);
}
try{
cv::medianBlur(tmp, output, ksize);
}
catch (cv::Exception& ex) {
qWarning()<<"medianBlurImage error"<<ex.what();
}
I have a, for example, 4*5*6 3D matrix. I would like to divide it into 6 2D matrix. The purpose is to make data operation on these 2D matrix and get results.
Tried row(), rowRange(), I got errors. No clues right now. Anyone throw any better ideas?
Thanks~
Remember that last index varies fastest, so maybe you mean you have a 6*5*4 Mat and would like to divide it into six 5x4 Mats. According to the documentation "3-dimensional matrices are stored plane-by-plane".
However, assuming your 3D Mat was created like this:
int dims[] = {4, 5, 6};
Mat m3(3, dims, CV_8UC1, data);
You can do something like this to do what you asked (but possibly not what you actually want):
Mat m2(4, 30, CV_8UC1, m3.data);
Mat m2x6 = m2.reshape(6);
std::vector<cv::Mat> channels;
cv::split(m2x6, channels);
However, to get out 4 images from m3 that have 5 rows x 6 cols:
Mat p0(5, 6, CV_8UC1, m3.data + m3.step[0] * 0);
Mat p1(5, 6, CV_8UC1, m3.data + m3.step[0] * 1);
Mat p2(5, 6, CV_8UC1, m3.data + m3.step[0] * 2);
Mat p3(5, 6, CV_8UC1, m3.data + m3.step[0] * 3);
Because support for 3D Mats in OpenCV is not great, avoid using them if you can.
An alternative would be to use a 2D Mat that has multiple channels. That is often much easier to handle.