I am receiving a video frame in a GstVideoFrame structure of a gstreamer element which is written in C++.
The frame is in YUV420 NV12 format
In this gstreamer element, I am trying to copy y-frame and uv-frame in separate buffers.
According to videolan.org, the YUV420 NV12 data is stored as following in the incoming frame buffer : (info copied from website)
NV12:
Related to I420, NV12 has one luma "luminance" plane Y and one plane with U and V values interleaved.
In NV12, chroma planes (blue and red) are subsampled in both the horizontal and vertical dimensions by a factor of 2.
For a 2×2 group of pixels, you have 4 Y samples and 1 U and 1 V sample.
It can be helpful to think of NV12 as I420 with the U and V planes interleaved.
Here is a graphical representation of NV12. Each letter represents one bit:
For 1 NV12 pixel: YYYYYYYY UVUV
For a 2-pixel NV12 frame: YYYYYYYYYYYYYYYY UVUVUVUV
For a 50-pixel NV12 frame: Y×8×50 (UV)×2×50
For a n-pixel NV12 frame: Y×8×n (UV)×2×n
but I cant seem to calculate the offset of y-data and uv-data in the buffer.
Update_1 : I have height and width of the frame to calculate the size of y-data and uv-data as:
y_size = width * height;
uv_size = y_size / 2;
any help or comments regarding this will be appreciated.
thanks
Thanks to #Ext3h, this is how I was able to separate y-data and uv-data from incoming YUV frame.
y_data = (u8 *)GST_VIDEO_FRAME_PLANE_DATA (in_frame, 0);
uv_data =(u8 *)GST_VIDEO_FRAME_PLANE_DATA (in_frame, 1);
Related
I have a Mipi camera that captures frames and stores them into the struct buffer that you can see below. Once the frame is stored I want to convert it into a cv::Mat, the thing is that the Mat ends up looking like the first pic.
The var buf.index is just part of the V4L2 API, useful to understand which buffer I'm using.
//The structure where the data is stored
struct buffer{
void *start;
size_t length;
};
struct buffer *buffers;
//buffer->mat
cv::Mat im = cv::Mat(cv::Size(width, height), CV_8UC3, ((uint8_t*)buffers[buf.index].start));
At first I thought that the data might be corrupted but storing the image with lodepng results in a nice image without any distortion.
unsigned char* out_buf = (unsigned char*)malloc( width * height * 3);
for(int pix = 0; pix < width*height; ++pix) {
memcpy(out_buf + pix*3, ((uint8_t*)buffers[buf.index].start)+4*pix+1, 3);
}
lodepng_encode24_file(filename, out_buf, width, height);
I bet it's something really silly.
the picture you post has oddly colored pixels and the patterns look like there's more information than simply 24 bits per pixel.
after inspecting the data, it appears that V4L gives you four bytes per pixel, and the first byte is always 0xFF (let's call that X). further, the channel order seems to be XRGB.
create a cv::Mat using 8UC4 to contain the data.
to use the picture in OpenCV, you need BGR order. cv::split the received data into its four color planes which are X,R,G,B. use cv::merge to reassemble the B,G,R planes into a picture that OpenCV can handle, or reassemble into R,G,B to create a Mat for other purposes (that other library you seem to use).
I use OpenGL shaders to do color conversion from YUV to RGB. For example, on YUV420P, I create 3 textures (one for Y, one for U, one for V) and use the texture GLSL call to get each texture. Then I use matrix multiplication to get the RGB value. Each of thes textures have the format GL_RED, because they store only 1 component.
This all works on C++. Now I'm using the safe OpenGL Rust library glium. I'm creating a texture like this:
let mipmap = glium::texture::MipmapsOption::NoMipmap;
let format = glium::texture::UncompressedFloatFormat::U8;
let y_texture = glium::texture::texture2d::Texture2d::empty_with_format(&display, format, mipmap, width as u32, height as u32).unwrap();
let u_texture = glium::texture::texture2d::Texture2d::empty_with_format(&display, format, mipmap, (width as u32)/2, (height as u32)/2).unwrap();
let v_texture = glium::texture::texture2d::Texture2d::empty_with_format(&display, format, mipmap, (width as u32)/2, (height as u32)/2).unwrap();
See that the sizes of the U and V textures are 1/4 of the Y texture, as expected for YUV420P.
as you see, for YUV420P I've chosen glium::texture::UncompressedFloatFormat::U8, which I think is the same as GL_RED.
The problem is that I don't know how to fill this texture with data. Its write method expect something that can be converted into a RawImage2D. However, all the filling for methods for RawImage2D expect an RGB image.
I need a method to fill only Y to the first texture, then only U to the second, and only V to the third.
I have a stereo camera that gives an image in the format YUYV with a a resolution of 320 x 480, where in each pixel (16 bits) has encoded 2 pixels of 8 bits. I'm using OpenCV in order to get the image but when I try to get the real resolution of the image I wont get good results. I guess I missing how to properly split the 16 bits in two.
Using this and this I'm able to reconstruct an image but still is not the real one.
Mat frame;
unsigned int width= cap.get(CV_CAP_PROP_FRAME_WIDTH);
unsigned int height= cap.get(CV_CAP_PROP_FRAME_HEIGHT);
m_pDepthImgBuf = (unsigned char*)calloc(width*height*2, sizeof(unsigned char));
...
cap >> frame; // get a new frame from camera
imshow("YUVY 320x480", frame);
memcpy( (void*)m_pDepthImgBuf, (void*)frame.data, width*height*2 * sizeof(unsigned char) );
cv::Mat depth(height,width*2,CV_8UC1,(void*)m_pDepthImgBuf);
camera properties:
SDL information:
Video driver: x11
Device information:
Device path: /dev/video2
Stream settings:
Frame format: YUYV (MJPG is not supported by device)
Frame size: 320x480
Frame rate: 30 fps
v4l2-ctl -d /dev/video1 --list-formats
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: 'YUYV'
Name : YUV 4:2:2 (YUYV)
In the following image you can see in green the initial 320x480 image and in gray scale the depth that I'm trying to extract.
The expected result should be:
I want to use the CImg library (http://cimg.sourceforge.net/) to rotate an image with an arbitrary angle (the image is read by Qt which should not perform the rotation):
QImage img("sample_with_alpha.png");
img = img.convertToFormat(QImage::Format_ARGB32);
float angle = 45;
cimg_library::CImg<uint8_t> src(img.bits(), img.width(), img.height(), 1, 4);
cimg_library::CImg<uint8_t> out = src.get_rotate(angle);
// Further processing:
// Data: out.data(), out.width(), out.height(), Stride: out.width() * 4
The final data in "out.data()" is ok when the the angle is set to 0. But for other angles the output data is distorted. I assume that the CImg library changes the output format and/or stride during rotation?
Regards,
CImg does not store the pixel buffer of an image in interleaved mode, as RGBARGBARGBA... but uses a channel by channel structure RRRRRRRR.....GGGGGGGGG.......BBBBBBBBB.....AAAAAAAAA.
I assume your img.bits() pointer points to pixels with interleaved channels, so if you want to pass this to CImg, you'll need to permute the buffer structure before you can apply any of the CImg method.
Try this :
cimg_library::CImg<uint8_t> src(img.bits(), 4,img.width(), img.height(), 1);
src.permute_axes("yzcx");
cimg_library::CImg<uint8_t> out = src.get_rotate(angle);
// Here, the out image should be OK, try displaying it with out.display();
// But you still need to go back to an interleaved image pointer if you want to
// get it back in Qt.
out.permute_axes("cxyz"); // Do the inverse permutation.
const uint8_t *p_out = out.data(); // Interleaved result.
I guess this should work as expected.
I'm trying to get the Y value of pixel from a frame that's in Ycbcr color mode.
here what I' wrote:
cv::Mat frame, Ycbcrframe, helpframe;
........
cvtColor(frame,yCbCrFrame,CV_RGB2YCrCb); // converting to Ycbcr
Vec3b intensity =yCbCrFrame.at<uchar>(YPoint);
uchar yv = intensity.val[0]; // I thought it's my Y value but its not, coz he gives me I think the Blue channel of RGB color space
any Idea how what the correct way to do that
what about the following code?
Vec3f Y_pix = YCbCrframe.at<Vec3f>(rows, cols);
int pixelval = Y_pix[0];
(P.S. I havent tried it yet)
You need to know both the depth (numerical format and precision of channel sample) as well as the channel count (typically 3, but can also be 1 (monochrome) or 4 (alpha-containing)), ahead of time.
For 3-channel, 8-bit unsigned integer (a.k.a. byte or uchar) pixel format, each pixel can be accessed with
mat8UC3.at<cv::Vec3b>(pt);