I faced a problem with C++ blobFromImage function in OpenCV. I trained a CNN-network in Keras which takes a 4-d blob as input (common practice, nothing special). The problem is that my blob order is NHWC (where Channle size is always 6) but blobFromImage returns only NCHW. There is no any trouble to reshape numpy-blob in python but I haven't found any solution for C++.
Input data is two 3-channel images stitched together (at channel axis) in one blob. For example, if images resolution is 1280x720 than blob shape will be (1, 720, 1280, 6)
Is there any way to create blob of NHWC in C++ or reshape blobFromImage result to NHWC?
It seems like you received an answer to your question in the OpenCV forum
assuming, you have 2 (float) images A and B of equal size, 3 channels each, you could first merge them like this:
vector<Mat> v = {A,B};
Mat C;
merge(v, C);
now C has 6 interleaved channels, and we need to add the "batch" dimension:
int sz[] = {1, A.rows, A.cols, 6};
Mat blob(4, sz, CV_32F, C.data);
but careful ! your blob does NOT hold a deep copy of the data, so C has to be kept alive during the processing !
EDIT: due to public demand, here's the reverse operation ;)
// extract 2d, 6chan Mat
Mat c2(blob.size[2], blob.size[3], CV_32FC(6), blob.ptr(0));
// split into channels
vector<Mat> v2;
split(c2,v2);
// merge back into 2 images
Mat a,b;
merge(vector<Mat>(v2.begin(), v2.begin()+3), a);
merge(vector<Mat>(v2.begin()+3, v2.end()), b);
Related
I want to convert BGR image to ABGR/ARGB.There are conversion BGR2RGBA in opencv but not BGR2ABGR or BGR2ARGB.
It is possible with opencv or using any other method?
The required operation can be accomplished by swapping the image channels using cv::mixChannels as follows:
cv::Mat bgr, bgra;
//bgr initialization code here...
//.
//.
//.
cv::cvtColor(bgr, bgra, cv::COLOR_BGR2BGRA);
cv::Mat abgr(bgra.size(), bgra.type());
int from_to[] = { 0,3, 1,1, 2,2, 3,0 };
cv::mixChannels(&bgra,1,&abgr,1,from_to,4);
from_to array is the mapping function which specifies which channels from source will be copied to which channels of the destination image. The pairs indicate that channel number 0 of the input will be copied to channel number 3 of the output, 1 to 1, 2 to 2, and channel number 3 will be copied to channel number 0 of the output.
Alternatively, we can split the image channels, swap the required channels and merge again. It can be done as follows:
cv::cvtColor(bgr, bgra, cv::COLOR_BGR2BGRA);
std::vector<cv::Mat> channels_bgra;
cv::split(bgra, channels_bgra);
std::vector<cv::Mat> channels_abgr = { channels_bgra[3], channels_bgra[1], channels_bgra[2], channels_bgra[0] };
cv::merge(channels_abgr, abgr);
OpenCV doesn't support ARGB or ABGR formats, so you will not be able to display it or use some of the functions on it... However, it is possible to create them with split and merge functions of OpenCV. Here is some code to explain what I mean.
cv::Mat src, final_image;
// fill src as you prefer
std::vector<cv::Mat> channels;
cv::split(src, channels); // this will put each channel in a mat in the vector
// swap or add channels in the vector
cv::Mat alpha(src.rows, src.cols, CV_8U, cv::Scalar(255));
channels.push_back(alpha);
std::reverse(channels.begin(), channels.end()); //needs <algorithm>
// merge the channels in one new image
cv::merge(channels, final_image);
This can be done faster (maybe it will be just shorter) with the function mixChannels, but I will say that this one is a little bit more confusing.
I am new to Open CV, so please forgive me if my question sounds stupid. So, i was studying about this new concept of splitting a BGR channel to individual channels using the split function. I was reading this article(http://answers.opencv.org/question/37132/i-want-to-split-and-show-r-b-g-pictures-why-does-it-not-work/) and i could not understand the code. So, please can anyone explain me the following line of code as i am really wanted to understand the concept.
I did not understand the create blue channel part at all. Please can anyone explain me a bit?
src = imread("pic.png");
vector<mat> spl(3);
split(src,spl);
Mat empty_image = Mat::zeros(src.rows, src.cols, CV_8UC1);
Mat result_blue(src.rows, src.cols, CV_8UC3); // notice the 3 channels here!
// Create blue channel
Mat in1[] = { spl[0], empty_image, empty_image };
int from_to1[] = { 0,0, 1,1, 2,2 };
mixChannels( in1, 3, &result_blue, 1, from_to1, 3 );
imshow("blue image", result_blue);
What the code does is split the color image into 3 grayscale images, holding the intensities of the red, green and blue.
Then the code takes the blue channel and constructs a color image with a zero red and green, so that when you show it, it will show as bluish, and not just grayscale.
Th OpenCV Split Function:
cv:split(src, spl)
Takes a 3 channel 24bit (8 bit for each channel) of type CV_8UC3 and split it into type CV_8UC1 (GrayScale), that is a single channel image or type R, G and B separately.
If you take as an example a Green channel image from spl1 vector and do cv::imshow, you should notice that the green color from the original RGB image will appear as high intensity value in only Green channel image. Likewise for any other channel.
mixChannels( in1, 3, &result_blue, 1, from_to1, 3 );
mixChannel is basically, a function that copies some channel of source image to the new destination image. When using the function you need to specify the the number of channel you need to mix which in your case is 3.
cv::Mat result_blue(src.rows, src.cols, CV_8UC3);
Note that this variable of type cv::Mat is memory to hold the output image where the channels from src are mixed to the destination image.
1 specifies the number of matrix or image you want in the final output image, that is in cv::Mat result_blue.
int from_to1[] = { 0,0, 1,1, 2,2 };
This array specifies which channel from src needs to be mixed with which channel in the destination. That is 0 will be copied to 0 and so on.
The last param 3 in mixChannel specifies the number of channel pairs in the destination image.
You may also look at this function of merging channels.
The cv::merge() function on the other than takes multiple single channel image and merge it to produce a high level. Note that cv::merge also takes a int, the second param which specifies the number of channels you need to merge.
Lastly, I would suggest that you play with the function to understand them well.
OpenCV docs say A.mul(B) is per-element multiplication. Yet the following code produces the following output, and then gives this error:
OpenCV Error: Sizes of input arguments do not match
.
cout << laplacian_pyramids[i][numLevels - 1 - l].rows << endl;
cout << gaussian_weight_pyramids[i][l].rows << endl;
cout << laplacian_pyramids[i][numLevels - 1 - l].cols << endl;
cout << gaussian_weight_pyramids[i][l].cols << endl;
Gives:
339
339
571
571
Then:
Mat prod = gaussian_weight_pyramids[i][l].mul(laplacian_pyramids[i][numLevels - 1 - l]);
gives the error. I tried Mat::multiply to a similar effect.
I would recommend converting single channel to three channels:
Mat A = Mat::zeros(100, 200, CV_32FC1);
Mat B = Mat::zeros(100, 200, CV_32FC3);
// Mat C = A.mul(B); // Sizes of input arguments do not match
Mat Afc3;
Mat t[] = {A, A, A};
merge(t, 3, Afc3);
Mat C = Afc3.mul(B); // now Afc3 has 3 channels ans it is type of 32_FC3
// we can multiply each elem in B by the same coef from A
But if B it is a CV_8UC3 type, it does not work because opencv would not allow to multiply Mats which have different types of pixels. In that case, convert 8UC3 to 32FC3 remebering to scale each pixel by 1/255.0 beacuse each pixel in 32FC3 has a value between 0.0 and 1.0 (and of course each pixel in 8UC3 has a value between 0 and 255).
Mat A = Mat::zeros(100, 200, CV_32FC1);
Mat B = Mat::zeros(100, 200, CV_8UC3);
// Mat C = A.mul(B);
Mat Afc3, Bfc3;
Mat t[] = {A, A, A};
merge(t, 3, Afc3);
B.convertTo(Bfc3, CV_32FC3, 1/255.0);
Mat C = Afc3.mul(Bfc3);
There can be 2 reasons for such error: different number of channels or different type of data (for example if first matrix contain unsigned char and second matrix contain unsigned short). Of course there can be both reasons. In general there 3 types of solutions for problems like the one you encountered:
1) Write your own 'for' loop that will do the operation you need. You won't benefit from optimizations that might be present in OpenCV functions but other solutions will have their own overheads. You can see this tutorial about how to access pixels in efficient way.
2) Use functions like 'merge' or 'convertTo' in order to create input of same type and number of channels. See answer posted by #marol for code example. In this solution the main overhead is copy of data. That means extra time and space. This is reasonable solution if you are going to perform multiple operations with both images. But if all you need is simple multiplication it won't be very effective.
3) Use workarounds. For example if your matrices have same type but differ in number of channels you can use reshape function:
// two matrices of same size but different number of channels
Mat laplac(100, 200, CV_32FC3);
Mat gauss(100, 200, CV_32FC1);
// turn them into single channel matrices. they have NxM rows and 1 or 3 columns.
// note that there no copy of data. any change in them will affect original matrices
Mat laplac2 = laplac.reshape( 1, laplac.rows*laplac.cols );
Mat gauss2 = gauss.reshape( 1, gauss.rows*gauss.cols ;
// perform multiplication
laplac2.col(0) = laplac2.col(0).mul(gauss2);
laplac2.col(1) = laplac2.col(1).mul(gauss2);
laplac2.col(2) = laplac2.col(2).mul(gauss2);
This way you are using only OpenCV build-in functions without copy overhead. But I doubt that this will be any faster than solution-1, because solution-1 is more efficient in terms of memory access.
In any case you won't have nice and clean operation that takes exactly one line :(
suppose that I have a
vector<unsigned char>a
which is the raster information of a geotiff image extracted by the RasterIO function of the GDAL library( an opensource library for geographic information systems)
my image is a 7697x7309 one so the vector has 56257373 members.
How can I apply a 5x5 gaussian filter on this vector and then gain the result as another 56257373 members vector of the type unsigned char to be able to save the vector as another geotiff image using GDAL library.
My main question is the above but if it's not possible tell me if I have a geotiff file how can I apply filters on it using opencv at run-time. I mean I don't want to convert the format to another one like bitmap and tiff on the hard and then read data from hard to apply processes on that, suppose that I have data in GDAL format in one part of the memory and want to convert it to opencv compatible data in another part and apply filters on it?
I think this is what you are asking for:
// 1. Convert vector to Mat
cv::Mat amat(7309, 7697, CV_8UC1, &a[0]);
// 2. Apply 5x5 Gaussian filter
cv::Mat bmat; // blurred output, sigma=1.4 assumed below
cv::GaussianBlur(amat, bmat, cv::Size(5,5), 1.4);
// 3. Convert Mat to vector
cv::Mat cmat = bmat.reshape(1, 1); // make the Mat one big long row
std::vector<unsigned char>b = cmat;
A simpler than the vector< > way to convert from GDAL to OpenCV raster data:
//Region of Interest to be read
cv::Rect roi(x, y, w, h);
//Mat allocation to store the data
cv::Mat mat;
mat.create(roi.size(),CV_32F);
//data is stored directly in the mat passing the mat.data pointer to RasterIO
band->RasterIO( GF_Read, roi.x, roi.y, roi.width, roi.height, mat.data,
roi.width, roi.height, GDT_Float32, 0, 0);
You just have to be sure that OpenCV datatype fit the GDAL datatype and that ROI dimensions are ok for the raster size
I used it:
Mat map( img.size(), CV_8UC3, CV_RGB(0,0,0) );
but it seems not create any matrix with 3 dimensions!
Could anyone help me?
The CV_8UC3 flag means that you are creating an image that has three channels where each pixel in each channel is represented as an unsigned character. You should be able to confirm the multiple channels (or 3rd dimension) by seeing the output of
map.channels();
which will return how large the matrix is in the third dimension. If you require more channels, then use something like:
map.create(100,60,CV_8UC(15));
where 15 is the number of channels.
The good way to do that is to use the appropriated constructor :
Mat::Mat(int ndims, const int* sizes, int type)
For example if you want to create a 100x60x15 matrix :
int sz[] = {100, 60, 15};
Mat map(3, sz, CV_8U);