How can I permute dimensions in cv::Mat from CxWxH to WxHxC (Width x Height x Channels)?
I.e. how can I convert Mat:
from cv::Mat frame (1000, 1000, CV_8UC3) = cv::imread("image.jpg", -1);
with dimesions: channels, width, height
to Mat with dimesions: width, height, channels
int sizes_inputs[] = { 3, 1000, 1000 };
cv::Mat out_image(3, sizes_inputs, CV_8UC);
Is there in OpenCV a ready-made fast function for such a conversion? Or should I implement this algorithm myself?
If you wish to treat the underlying data differently, then you can use reshape. The data is stored as BGRBGRBGR....
Otherwise you will have to shuffle the data yourself.
cv::reshape and cv::minChannels may be handy for this.
You can emulate it. I have not tested the code but something like this should do:
cv::Mat frame (1000, 1000, CV_8UC3) = cv::imread("image.jpg", -1);
int sizes_inputs[] = { 3, 1000, 1000 };
cv::Mat out_image(3, sizes_inputs, CV_8UC);
cv::Mat slices[] = {cv::Mat(1000, 1000, CV_8UC, out_image.ptr()),
cv::Mat(1000, 1000, CV_8UC, out_image.ptr() + 1000*1000),
cv::Mat(1000, 1000, CV_8UC, out_image.ptr() + 1000*1000*2)};
cv::split(frame, slices);
Related
OpenCV reads an image in NCHW format (Number of samples x Channels x Height x Width), I need to convert it to NHWC format (move 2nd dimension of array to last). Is there an efficient way to convert from NCHW to NHWC in C++? I can do this with a 3 for loops, but obviously this is not efficient at all.
This straightforward solution worked for me with OpenCV C++:
static void hwc_to_chw(cv::InputArray src, cv::OutputArray dst) {
std::vector<cv::Mat> channels;
cv::split(src, channels);
// Stretch one-channel images to vector
for (auto &img : channels) {
img = img.reshape(1, 1);
}
// Concatenate three vectors to one
cv::hconcat( channels, dst );
}
With OpenCV >= 4.6, you can use transposeND (from opencv2/core.hpp) for such kind of convertion:
std::vector<int> order = {0, 2, 3, 1};
Mat inp, out; // inp: NCHW, out: NHWC
transposeND(inp, order, out);
I am trying to convert a float image that I get from a simulated depth camera to CV_16UC1. The camera publishes the depth in CV_32FC1 format. I tried many ways but the result was not reasonable.
cv::Mat depth_cv(512, 512, CV_32FC1, depth);
cv::Mat depth_converted;
depth_cv.convertTo(depth_converted,CV_16UC1);
The result is a black image. If I use a scale factor, the image will be white.
I also tried to do it this way:
float depthValueF [512*512];
for (int i=0;i<resolution[1];i++){ // go through the rows (y)
for (int j=0;j<resolution[0];j++){ // go through the columns (x)
depthValueOfPixel=depth[i*resolution[0]+j]; // this is location j/i, i.e. x/y
depthValueF[i*resolution[0]+j] = (depthValueOfPixel) * (65535.0f);
}
}
It was not successful either.
Try using cv::normalize instead, which will not only convert the image into the proper data type, but it will properly do the scaling for you under the hood.
Therefore:
cv::Mat depth_cv(512, 512, CV_32FC1, depth);
cv::Mat depth_converted;
cv::normalize(depth_cv, depth_converted, 0, 65535, NORM_MINMAX, CV_16UC1);
I want to convert pytorch tensors to opencv mat and vice versa in C++. I have these two functions:
cv::Mat TensorToCVMat(torch::Tensor tensor)
{
std::cout << "converting tensor to cvmat\n";
tensor = tensor.squeeze().detach().permute({1, 2, 0});
tensor = tensor.mul(255).clamp(0, 255).to(torch::kU8);
tensor = tensor.to(torch::kCPU);
int64_t height = tensor.size(0);
int64_t width = tensor.size(1);
cv::Mat mat(width, height, CV_8UC3);
std::memcpy((void *)mat.data, tensor.data_ptr(), sizeof(torch::kU8) * tensor.numel());
return mat.clone();
}
torch::Tensor CVMatToTensor(cv::Mat mat)
{
std::cout << "converting cvmat to tensor\n";
cv::cvtColor(mat, mat, cv::COLOR_BGR2RGB);
cv::Mat matFloat;
mat.convertTo(matFloat, CV_32F, 1.0 / 255);
auto size = matFloat.size();
auto nChannels = matFloat.channels();
auto tensor = torch::from_blob(matFloat.data, {1, size.height, size.width, nChannels});
return tensor.permute({0, 3, 1, 2});
}
In my code I load two images (image1 and image2) and I want to convert them to pytorch tensors and then back to opencv mat to check if it works. The problem is that I get an memory access error on the first call of TensorToCVMat and I cant figure out whats wrong as I do not have much experience with C++ programming.
cv::Mat image1;
image1 = cv::imread(argv[1]);
if (!image1.data)
{
std::cout << "no image data\n";
return -1;
}
cv::Mat image2;
image2 = cv::imread(argv[2]);
if (!image2.data)
{
std::cout << "no image data\n";
return -1;
}
torch::Tensor tensor1 = CVMatToTensor(image1);
cv::Mat new_image1 = TensorToCVMat(tensor1); // <<< this is where the memory access error is thrown
torch::Tensor tensor2 = CVMatToTensor(image2);
cv::Mat new_image2 = TensorToCVMat(tensor2);
It would be great if you could give me hints or an explanation to solve this problem.
Not sure if the error is happening at the memcpy step. But you can use the void* data variant of the Mat constructor
Mat (int rows, int cols, int type, void *data, size_t step=AUTO_STEP)
and you can skip the memcpy step
tensor = uint8_tensor //shape: (h, w, 3)
cv::Mat mat = cv::Mat(height, width, CV_8UC3, tensor.data_ptr());
return mat;
I am using torch>=1.7.0.
For a tensor of dtype=float and size [1, 3, height, width]
this is what worked for me
cv::Mat torchTensortoCVMat(torch::Tensor& tensor)
{
tensor = tensor.squeeze().detach();
tensor = tensor.permute({1, 2, 0}).contiguous();
tensor = tensor.mul(255).clamp(0, 255).to(torch::kU8);
tensor = tensor.to(torch::kCPU);
int64_t height = tensor.size(0);
int64_t width = tensor.size(1);
cv::Mat mat = cv::Mat(cv::Size(width, height), CV_8UC3, tensor.data_ptr<uchar>());
return mat.clone();
}
My tensor shape was 500x500x3, I have to add tensor.reshape({width * height * 3}) to get the actual image
cv::Mat TensorToCVMat(torch::Tensor tensor)
{
// torch.squeeze(input, dim=None, *, out=None) → Tensor
// Returns a tensor with all the dimensions of input of size 1 removed.
// tensor.detach
// Returns a new Tensor, detached from the current graph.
// permute dimension, 3x700x700 => 700x700x3
tensor = tensor.detach().permute({1, 2, 0});
// float to 255 range
tensor = tensor.mul(255).clamp(0, 255).to(torch::kU8);
// GPU to CPU?, may not needed
tensor = tensor.to(torch::kCPU);
// shape of tensor
int64_t height = tensor.size(0);
int64_t width = tensor.size(1);
// Mat takes data form like {0,0,255,0,0,255,...} ({B,G,R,B,G,R,...})
// so we must reshape tensor, otherwise we get a 3x3 grid
tensor = tensor.reshape({width * height * 3});
// CV_8UC3 is an 8-bit unsigned integer matrix/image with 3 channels
cv::Mat imgbin(cv::Size(width, height), CV_8UC3, tensor.data_ptr());
return imgbin;
}
I have written a function to take a Mat image in and transpose it onto the center of a blank image three times the size. I have written function to do so but I feel it can be improved in terms of efficiency.
void transposeFrame( cv::Mat &frame){
Mat new_frame( frame.rows * 3, frame.cols * 3, CV_8UC3, Scalar(0,0,255));
Rect dim = Rect( frame.rows, frame.cols, frame.rows * 2, frame.cols * 2);
Mat subview = new_frame(dim);
frame.copyTo(subview);
frame = subview;
}
Is there a better to preform this operation?
I'd use something like :-
Mat frame_tpsed;
cv::Mat::transpose(frame, frame_tpsed);
new_frame(Rect(frame.rows, frame.cols, frame.rows*2, frame.cols*2)) = frame_tpsed
Transpose on opencvDocs : http://docs.opencv.org/modules/core/doc/operations_on_arrays.html#void transpose(InputArray src, OutputArray dst)
Didn't try it out. Forgot to mention it.
You can use the copyMakeBorder function. However, I don't think it will work considerably faster.
I have a C++ function that is to be called from someone else's C# application. As input my function is given an array of signed short integers, the dimensions of the image it represents, and memory allocated for the returning data, namely another array of signed short integers. This would represent my function's header:
my_function (short* input, int height, int width, short* output)
Inside my function I create a cv::Mat from input, like this:
cv::Mat mat_in = cv::Mat (height, width, CV_16S, input);
This mat_in is then converted to CV_32F and processed by OpenCV's cv::bilateralFilter. After it returns cv::Mat mat_out, I convert the data back to CV_16S (bilateralFilter only accepts CV_8U and CV_32F). Now I need to convert this cv::Mat mat_out back to an array of short integers so that it may be returned to the calling function. This is my code:
my_function (short* input, int height, int width, short* output)
{
Mat mat_in_16S = Mat (height, width, CV_16S, input);
Mat mat_in_32F = Mat (height, width, CV_32F);
Mat mat_out_CV_32F = Mat (height, width, CV_32F);
mat_in_16S.convertTo (mat_in_32F, CV_32F);
bilateralFilter (mat_in_32F, mat_out_32F, 5, 160, 2);
Mat mat_out_16S = Mat (mat_in_16S.size(), mat_in_16S.type());
mat_out_32F.convertTo (mat_out_16S, CV_16S);
return 0;
}
Obviously, somewhere there at the end I need to get the data that is in mat_out_16S into output. My first try was to return a reference:
output = &mat_out_16S.at<short>(0,0);
but of course I realised that this was a silly idea, as mat_out_16S goes out of scope as soon as the function returns, leaving output pointing at emptiness. Currently my best attempt is as follows (from this question):
memcpy ((short*)output, (short*)mat_out_16S.data, height*width*sizeof(short));
Now I would like to know, is there a better way? It feels kind of inefficient to copy all this data, but I don't see what else I can do. I can't return a cv::Mat unfortunately. If there is no better way, is my current memcpy method safe at least? My data are all 2-byte signed short integers, so I don't think there should be issues with padding, but I don't want to run into any unpleasant surprises.
You can use this constructor for your mat_out_16S:
Mat::Mat(Size size, int type, void* data, size_t step=AUTO_STEP)
So your function will be:
my_function (short* input, int height, int width, short* output)
{
Mat mat_in_16S = Mat (height, width, CV_16S, input);
Mat mat_in_32F = Mat (height, width, CV_32F);
Mat mat_out_CV_32F = Mat (height, width, CV_32F);
mat_in_16S.convertTo (mat_in_32F, CV_32F);
bilateralFilter (mat_in_32F, mat_out_32F, 5, 160, 2);
Mat mat_out_16S = Mat (mat_in_16S.size(), mat_in_16S.type(), output);
mat_out_32F.convertTo (mat_out_16S, CV_16S);
return 0;
}