Im working on resizing bitmap image and converting bitmap image to 8-bit (grayscale). But I have the problem that when I convert 32-bit image to 8-bit image, the result has another color overlay while it works perfectly on 24-bit. I guess the cause is in the alpha color. but I dont know where the problem exactly is.
This is my code to generate 8-bit palette color and write it after DIB part:
char* palette = new char[1024];
for (int i = 0; i < 256; i++) {
palette[i * 4] = palette[i * 4 + 1] = palette[i * 4 + 2] = (char)i;
palette[i * 4 + 3] = 255;
}
fout.write(palette, 1024);
delete[] palette;
As I said, my code works perfectly on 24-bit. In 32-bit the color is still kept after resizing, but when converting to 8-bit, it will look like this:
expected image (when converted from 24-bit) //
unexpected image (when converted from 32-bit)
This is how I get the colors and save it to srcPixel[]:
int i = 0;
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
int index = getIndex(width, x, y);
srcPixel[index].A = srcBMP.pImageData[i];
i += alpha;
srcPixel[index].B = srcBMP.pImageData[i++];
srcPixel[index].G = srcBMP.pImageData[i++];
srcPixel[index].R = srcBMP.pImageData[i++];
}
i += padding;
}
And this is the code I converted it by getting average of 4 colors A, B, G and R from that srcPixel[]:
int i = 0;
for (int y = 0; y < dstHeight; y++) {
for (int x = 0; x < dstWidth; x++) {
int index = getIndex(dstWidth, x, y);
dstBMP.pImageData[i++] = (srcPixel[index].A + srcPixel[index].B + srcPixel[index].G + srcPixel[index].R) / 4;
}
i += dstPadding;
}
If I remove and skip all alpha bytes in my code, when converting my image is still like that and I will have another problem is when resizing, my image will have another color overlay like the problem when converting to 8-bit: resizing without alpha channel.
If I skip the alpha channel while getting average (change into dstBMP.pImageData[i++] = (srcPixel[index].B + srcPixel[index].G + srcPixel[index].R) / 3, there is almost nothing different, the overlay still exists.
If I remove palette[i * 4 + 3] = 255; or doing anything with it, the result is still not affected.
Thank you very much.
You add alpha channel to the color and that's why it becomes brighter. From here I found that opaque is 255 and transparent 0 - therefore you add another channel which is set to 'white' to your result.
Remove alpha channel from your equation and see if I'm right.
I have a distorted image in YUY2 data form, YUY2 comes under the family of YUV 4:2:2 (not 4:2:0).
And I have mapx and mapy (height-720, width-1280), which I obtained from
cv::fisheye::initUndistortRectifyMap(K, D, cv::Mat::eye(3, 3, CV_64F), new_K, Size, CV_32FC1, mapx, mapy);
How can I have undistorted YUY2?
My final goal is to have undistorted YUY2 (not BGR).
I thought to perform below steps:
cv::cvtColor(YUY, BGR, cv::COLOR_YUV2BGR_YUY2);
\\ then perform remapping
\\ and convert back to YUY
But there is no conversion from BGR2YUY_YUY2.
Is there is any smarter way?
You may convert from YUV 4:2:2 to YUV 4:4:4, undistorted the 4:4:4, and convert back to 4:2:2.
Illustration of the conversion stages:
YUV422 -> YUV444 -> remap(YUV444) -> YUV422
I could not find an OpenCV function for converting from YUV 4:2:2 to YUV 4:4:4.
Implementing the conversion by simple for loops is quite straight forward:
//Convert YUYV to YUV (y,u,v,y,u,v,y,u,v...)
//The conversion is performed by duplicating each U and V element twice (equivalent to resize with nearest neighbor interpolation).
//The input type is CV_8UC1 (considered to be Grayscale image).
//The output type is CV_8UC3 (considered to be colored image with 3 channels).
static cv::Mat convertYuyv422toYuv444(const cv::Mat yuyv)
{
int rows = yuyv.rows;
int src_cols = yuyv.cols;
size_t src_step = yuyv.step;
const unsigned char *I = (unsigned char*)yuyv.data; //Pointer to source image.
int dst_cols = src_cols / 2;
cv::Mat yuv = cv::Mat(rows, dst_cols, CV_8UC3);
size_t dst_step = yuv.step;
unsigned char *J = (unsigned char*)yuv.data; //Pointer to destination image.
for (int y = 0; y < rows; y++)
{
const unsigned char *I0 = I + y*src_step; //Points the beginning for the source row.
unsigned char *J0 = J + y*dst_step; //Points the beginning for the destination row.
int srcx = 0;
int dstx = 0;
//yuyv -> yuvyuv
//Convert 2 pixels per iteration
for (int x = 0; x < src_cols / 2; x += 2)
{
unsigned char y0 = I0[srcx];
unsigned char u0 = I0[srcx + 1];
unsigned char y1 = I0[srcx + 2];
unsigned char v0 = I0[srcx + 3];
J0[dstx] = y0;
J0[dstx + 1] = u0;
J0[dstx + 2] = v0;
J0[dstx + 3] = y1;
J0[dstx + 4] = u0; //Duplicate U
J0[dstx + 5] = v0; //Duplicate V
srcx += 4; //Source has 2 elements per pixel
dstx += 6; //Destination has 3 elements per pixel
}
}
return yuv;
}
The conversion simply duplicate every U and V element twice.
It's not the best way, but it's assumed to be good enough.
Duplicating U and V is equivalent to resize with Nearest Neighbor interpolation.
For converting YUV 4:4:4 back to YUV 4:2:2, you may use the code sample from (my) following post:
Convert YUV4:4:4 to YUV4:2:2 images.
There are existing optimized libraries that support all sorts of color format conversions.
libswscale for example, but I think it's an overkill for your needs...
Testing:
For testing, I used the input form your previous post (with my answer):
How to undistort I420 image data? Efficiently
Since I don't have a YUYV image, I used FFmpeg (command line) for creating one:
ffmpeg -i input_image.jpg -codec rawvideo -pix_fmt yuyv422 input_image_yuyv.yuv
I used MATLAB code for converting the raw input_image_yuyv.yuv to PNG.
The MATLAB implementation convert 4:2:2 to 4:4:4 in two ways and verify that duplicating U and V is is equivalent to resize with Nearest Neighbor interpolation.
The MATLAB code is also used for validating the correctness of the C++ implementation.
I = imread('input_image.jpg');
[rows, cols, ch] = size(I); % rows = 1280, cols = 720
% Read the YUYV to 2560x720 matrix from a binary file
f = fopen('input_image_yuyv.yuv', 'r');
YUYV = fread(f, [cols*2, rows], '*uint8')';
fclose(f);
% Write YUYV to PNG image - to be used as C++ input.
imwrite(YUYV, 'YUYV.png');
%figure;imshow(YUYV);title('YUYV');impixelinfo
Y = YUYV(:, 1:2:end); % 1280x720
U = YUYV(:, 2:4:end); % 640x720
V = YUYV(:, 4:4:end); % 640x720
% figure;imshow(Y);title('in Y');impixelinfo
% figure;imshow(U);title('in U');impixelinfo
% figure;imshow(V);title('in V');impixelinfo
% Convert U and V to 4:4:4 format using imresize with Nearest Neighbor interpolation method (used as reference).
refU2 = imresize(U, [rows, cols], 'nearest');
refV2 = imresize(V, [rows, cols], 'nearest');
% figure;imshow(U2);title('reference inU full');impixelinfo
% figure;imshow(V2);title('reference inV full');impixelinfo
% Resize x2 in the horizontal axis by simple duplication:
U2 = zeros(rows, cols, 'uint8');
U2(:, 1:2:end) = U;
U2(:, 2:2:end) = U;
V2 = zeros(rows, cols, 'uint8');
V2(:, 1:2:end) = V;
V2(:, 2:2:end) = V;
% Verify that the simple duplication is equivalent to resize with Nearest Neighbor interpolation:
% display(isequal(U2, refU2) && isequal(V2, refV2)) % Equal!!!
% Build YUV444 3840x720 matrix:
YUV444 = zeros(rows, cols*3, 'uint8');
YUV444(:, 1:3:end) = Y;
YUV444(:, 2:3:end) = U2;
YUV444(:, 3:3:end) = V2;
%figure;imshow(YUV444);title('YUV444');impixelinfo
% Write the YUV444 image to binary file (used as reference for C++ implementation)
f = fopen('image_yuv444.yuv', 'w');
fwrite(f, YUV444', 'uint8');
fclose(f);
imwrite(YUV444, 'matlabYUV444.png');
% Test output (after executing C++ code).
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
c_YUV444 = imread('yuv444.png');
display(isequal(YUV444, c_YUV444));
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
The following code is the main() of the C++ implementation:
int main()
{
cv::Mat yuyv = cv::imread("YUYV.png", cv::IMREAD_GRAYSCALE); //Read YUYV.png (created using MATLAB) as Grayscale
cv::Mat yuv = convertYuyv422toYuv444(yuyv); //Convet yuyv to yuv (y,u,v,y,u,v...)
//cv::imshow("yuyv", yuyv);
cv::imwrite("yuv444.png", yuv); //Store YUV image for testing.
//https://stackoverflow.com/questions/59876539/how-to-undistort-i420-image-data-efficiently
//remap the YUV 4:4:4
///////////////////////////////////////////////////////////////////////////////
int W = 1280, H = 720; //Assume resolution of Y plane is 1280x720
cv::Mat mapx;
cv::Mat mapy;
cv::Mat dst_yuv;
cv::Matx33d K = cv::Matx33d(541.2152931632737, 0.0, 661.7479652584254,
0.0, 541.0606969363056, 317.4524205037745,
0.0, 0.0, 1.0);
cv::Vec4d D = cv::Vec4d(-0.042166406281296365, -0.001223961942208027, -0.0017036710622692108, 0.00023929900459453295);
cv::Size newSize = cv::Size(3400, 1940);
cv::Matx33d new_K;
cv::fisheye::estimateNewCameraMatrixForUndistortRectify(K, D, cv::Size(W, H), cv::Mat::eye(3, 3, CV_64F), new_K, 1, newSize); // W,H are the distorted image size
cv::fisheye::initUndistortRectifyMap(K, D, cv::Mat::eye(3, 3, CV_64F), new_K, newSize, CV_16SC2, mapx, mapy);
cv::remap(yuv, dst_yuv, mapx, mapy, cv::INTER_LINEAR, cv::BORDER_CONSTANT, cv::Scalar(0, 128, 128));
///////////////////////////////////////////////////////////////////////////////
//Convert for BGR - just for dispaly
cv::Mat dst_bgr;
cv::cvtColor(dst_yuv, dst_bgr, cv::COLOR_YUV2BGR);
cv::imshow("yuv", yuv);
cv::imshow("dst_yuv", dst_yuv);
cv::imshow("dst_bgr", dst_bgr);
cv::waitKey(0);
cv::destroyAllWindows();
cv::imwrite("dst_bgr.png", dst_bgr); //Store BGR image for testing.
return 0;
}
Note:
Use remap with cv::BORDER_CONSTANT and cv::Scalar(0, 128, 128):
cv::remap(yuv, dst_yuv, mapx, mapy, cv::INTER_LINEAR, cv::BORDER_CONSTANT, cv::Scalar(0, 128, 128));
C++ Result (after converting to BGR):
I tried modification in mapx & mapy to make it suitable for YUV422. Result is very good in terms of computation time. Just one remapping is required in real time. But the quality is not the best.
Then I tried YUV422 -> YUV444 -> remap(YUV444) -> YUV422 through libswscale, but again YUV conversion was taking time.
Finally I developed cuda kernels for YUV conversions. I attached below.
// nvcc -c -o colorConversion.o colorConversion.cu `pkg-config --libs --cflags opencv4`
// /usr/bin/g++ -g -O3 /home/jai/vscode/opencvCUDA/cuda3.cpp -o /home/jai/vscode/opencvCUDA/cuda3 colorConversion.o `pkg-config --libs --cflags opencv4` `pkg-config --libs --cflags gstreamer-1.0` `pkg-config --libs --cflags cuda-11.3` `pkg-config --libs --cflags cudart-11.3`
#include "colorConversion.h"
__global__ void kernel_YUY422toYUY(cv::cuda::PtrStepSz<uchar2> YUV422, cv::cuda::PtrStepSz<uchar3> YUV)
{
int i = blockIdx.y; // row
int j = blockDim.x * blockIdx.x + threadIdx.x; // col
if (threadIdx.x & 1) { // odd 1,3,5
// YUV[i * step3 + 3 * j] = YUV422[i * step2 + 2 * j]; // Y0
// YUV[i * step3 + 3 * j + 1] = YUV422[i * step2 + 2 * j - 1]; // Y0
// YUV[i * step3 + 3 * j + 2] = YUV422[i * step2 + 2 * j + 1]; // Y0
YUV(i, j).x = YUV422(i, j).x;
YUV(i, j).y = YUV422(i, j - 1).y;
YUV(i, j).z = YUV422(i, j).y;
} else { // even 0,2,4,
// YUV[i * step3 + 3 * j] = YUV422[i * step2 + 2 * j]; // Y0
// YUV[i * step3 + 3 * j + 1] = YUV422[i * step2 + 2 * j + 1]; // U0
// YUV[i * step3 + 3 * j + 2] = YUV422[i * step2 + 2 * j + 3]; // V0
YUV(i, j).x = YUV422(i, j).x;
YUV(i, j).y = YUV422(i, j).y;
YUV(i, j).z = YUV422(i, j+1).y;
}
}
void YUY422toYUY(const cv::cuda::GpuMat &YUV422gpu, cv::cuda::GpuMat &YUVgpu)
{
kernel_YUY422toYUY<<<dim3(2, YUVgpu.rows), dim3(YUVgpu.cols / 2)>>>(YUV422gpu, YUVgpu);
//cudaSafeCall(cudaGetLastError());
}
__global__ void kernel_YUYtoYUY422(cv::cuda::PtrStepSz<uchar3> YUV, cv::cuda::PtrStepSz<uchar2> YUV422)
{
int i = blockIdx.x; // row
int j = threadIdx.x*2; // col
YUV422(i, j).x = YUV(i, j).x;
YUV422(i, j).y = (YUV(i, j).y + YUV(i, j+1).y)/2;
YUV422(i, j+1).x = YUV(i, j+1).x;
YUV422(i, j+1).y = (YUV(i, j).z + YUV(i, j+1).z)/2;
}
void YUYtoYUY422(const cv::cuda::GpuMat &YUVgpu, cv::cuda::GpuMat &YUV422gpu)
{
kernel_YUYtoYUY422<<<dim3(YUV422gpu.rows), dim3(YUV422gpu.cols / 2)>>>(YUVgpu, YUV422gpu);
//cudaSafeCall(cudaGetLastError());
}
And then I do remapping using CUDA again with following lines of code:
YUV422GPU.upload(YUV422); // YUV422 #channel = 2
YUV1.create(H, W, CV_8UC3);
YUV2.create(H, W, CV_8UC3);
YUY422toYUY(YUV422GPU, YUV1);
cv::cuda::remap(YUV1, YUV2, mapxGPU, mapyGPU, interpolationMethod); // YUV remap
YUYtoYUY422(YUV2, YUV422GPU);
YUV422GPU.download(dst); // dst is the final YUV422. 2 channel image
I have a image with 4 channels that i need to overlay it over a bunch of pictures. Over the pictures with 3 channels, the overlaying works great, but over the pictures that have an alpha channel, the background of the picture changes to black.
Original picture: http://img.blog.csdn.net/20130610074054484
Overlayed picture: http://imgur.com/mlVAN0A
This is the code that does the overlaying:
void overlayImage(const cv::Mat &background, const cv::Mat &foreground,
cv::Mat &output, cv::Point2i location)
{
background.copyTo(output);
for(int y = std::max(location.y , 0); y < background.rows; ++y)
{
int fY = y - location.y;
if(fY >= foreground.rows)
break;
for(int x = std::max(location.x, 0); x < background.cols; ++x)
{
int fX = x - location.x;
if(fX >= foreground.cols)
break;
double opacity = ((double)foreground.data[fY * foreground.step + fX * foreground.channels() + 3]) / 255.;
for(int c = 0; opacity > 0 && c < output.channels(); ++c)
{
unsigned char foregroundPx = foreground.data[fY * foreground.step + fX * foreground.channels() + c];
unsigned char backgroundPx = background.data[y * background.step + x * background.channels() + c];
output.data[y*output.step + output.channels()*x + c] =
backgroundPx * (1.-opacity) + foregroundPx * opacity;
}
}
}
}
This is because you use 1-opacity for the background image. If the opacity of the forground image is 0, the opacity of your backgroundpixel will be 1 instead of 0 which it is before.
You have to calc the result opacity fpr both images which can be 0 for both too.
Claus
I am trying to implement image warping in C++ and OpenCV. My code is as follows:
Mat input = imread("Lena.jpg",CV_LOAD_IMAGE_GRAYSCALE);
Mat out;
double xo, yo;
input.convertTo(input, CV_32FC1);
copyMakeBorder(input, input, 3, 3, 3, 3, 0);
int height = input.rows;
int width = input.cols;
out = Mat(height, width, input.type());
for(int j = 0; j < height; j++){
for(int i =0; i < width; i++){
xo = (8.0 * sin(2.0 * PI * j / 128.0));
yo = (8.0 * sin(2.0 * PI * i / 128.0));
out.at<float>(j,i) = (float)input.at<float>(((int)(j+yo+height)%height),((int)(i+xo+width)%width));
}
}
normalize(out, out,0,255,NORM_MINMAX,CV_8UC1);
imshow("output", out);
This produces the following image:
As it is clearly visible, the values near the border are non-zero. Can anyone tell me how do I get black border as shown in the following image instead of artifacts that I get from my code?
Only the black border of this image should be considered, i.e the image should be wavy (sinusoidal) but without artifacts.
Thanks...
Here:
xo = (8.0 * sin(2.0 * PI * j / 128.0));
yo = (8.0 * sin(2.0 * PI * i / 128.0));
out.at<float>(j,i) = (float)input.at<float>(((int)(j+yo+height)%height),((int)(i+xo+width)%width));
You calculate the location of the source pixel, but you take the mod with width/height to ensure it's within the image. This results in pixels wrapping around at the edge. Instead you need to set any pixel outside of the image to black (or, if your source image has a black border, clamp to the edge).
As you have a border already, you could just clamp the coordinates, like this:
int ix = min(width-1, max(0, (int) (i + xo)));
int iy = min(height-1, max(0, (int) (j + yo)));
out.at<float>(j,i) = (float)input.at<float>(iy,ix);
I have searched internet and stackoverflow thoroughly, but I haven't found answer to my question:
How can I get/set (both) RGB value of certain (given by x,y coordinates) pixel in OpenCV? What's important-I'm writing in C++, the image is stored in cv::Mat variable. I know there is an IplImage() operator, but IplImage is not very comfortable in use-as far as I know it comes from C API.
Yes, I'm aware that there was already this Pixel access in OpenCV 2.2 thread, but it was only about black and white bitmaps.
EDIT:
Thank you very much for all your answers. I see there are many ways to get/set RGB value of pixel. I got one more idea from my close friend-thanks Benny! It's very simple and effective. I think it's a matter of taste which one you choose.
Mat image;
(...)
Point3_<uchar>* p = image.ptr<Point3_<uchar> >(y,x);
And then you can read/write RGB values with:
p->x //B
p->y //G
p->z //R
Try the following:
cv::Mat image = ...do some stuff...;
image.at<cv::Vec3b>(y,x); gives you the RGB (it might be ordered as BGR) vector of type cv::Vec3b
image.at<cv::Vec3b>(y,x)[0] = newval[0];
image.at<cv::Vec3b>(y,x)[1] = newval[1];
image.at<cv::Vec3b>(y,x)[2] = newval[2];
The low-level way would be to access the matrix data directly. In an RGB image (which I believe OpenCV typically stores as BGR), and assuming your cv::Mat variable is called frame, you could get the blue value at location (x, y) (from the top left) this way:
frame.data[frame.channels()*(frame.cols*y + x)];
Likewise, to get B, G, and R:
uchar b = frame.data[frame.channels()*(frame.cols*y + x) + 0];
uchar g = frame.data[frame.channels()*(frame.cols*y + x) + 1];
uchar r = frame.data[frame.channels()*(frame.cols*y + x) + 2];
Note that this code assumes the stride is equal to the width of the image.
A piece of code is easier for people who have such problem. I share my code and you can use it directly. Please note that OpenCV store pixels as BGR.
cv::Mat vImage_;
if(src_)
{
cv::Vec3f vec_;
for(int i = 0; i < vHeight_; i++)
for(int j = 0; j < vWidth_; j++)
{
vec_ = cv::Vec3f((*src_)[0]/255.0, (*src_)[1]/255.0, (*src_)[2]/255.0);//Please note that OpenCV store pixels as BGR.
vImage_.at<cv::Vec3f>(vHeight_-1-i, j) = vec_;
++src_;
}
}
if(! vImage_.data ) // Check for invalid input
printf("failed to read image by OpenCV.");
else
{
cv::namedWindow( windowName_, CV_WINDOW_AUTOSIZE);
cv::imshow( windowName_, vImage_); // Show the image.
}
The current version allows the cv::Mat::at function to handle 3 dimensions. So for a Mat object m, m.at<uchar>(0,0,0) should work.
uchar * value = img2.data; //Pointer to the first pixel data ,it's return array in all values
int r = 2;
for (size_t i = 0; i < img2.cols* (img2.rows * img2.channels()); i++)
{
if (r > 2) r = 0;
if (r == 0) value[i] = 0;
if (r == 1)value[i] = 0;
if (r == 2)value[i] = 255;
r++;
}
const double pi = boost::math::constants::pi<double>();
cv::Mat distance2ellipse(cv::Mat image, cv::RotatedRect ellipse){
float distance = 2.0f;
float angle = ellipse.angle;
cv::Point ellipse_center = ellipse.center;
float major_axis = ellipse.size.width/2;
float minor_axis = ellipse.size.height/2;
cv::Point pixel;
float a,b,c,d;
for(int x = 0; x < image.cols; x++)
{
for(int y = 0; y < image.rows; y++)
{
auto u = cos(angle*pi/180)*(x-ellipse_center.x) + sin(angle*pi/180)*(y-ellipse_center.y);
auto v = -sin(angle*pi/180)*(x-ellipse_center.x) + cos(angle*pi/180)*(y-ellipse_center.y);
distance = (u/major_axis)*(u/major_axis) + (v/minor_axis)*(v/minor_axis);
if(distance<=1)
{
image.at<cv::Vec3b>(y,x)[1] = 255;
}
}
}
return image;
}