I use OpenGL shaders to do color conversion from YUV to RGB. For example, on YUV420P, I create 3 textures (one for Y, one for U, one for V) and use the texture GLSL call to get each texture. Then I use matrix multiplication to get the RGB value. Each of thes textures have the format GL_RED, because they store only 1 component.
This all works on C++. Now I'm using the safe OpenGL Rust library glium. I'm creating a texture like this:
let mipmap = glium::texture::MipmapsOption::NoMipmap;
let format = glium::texture::UncompressedFloatFormat::U8;
let y_texture = glium::texture::texture2d::Texture2d::empty_with_format(&display, format, mipmap, width as u32, height as u32).unwrap();
let u_texture = glium::texture::texture2d::Texture2d::empty_with_format(&display, format, mipmap, (width as u32)/2, (height as u32)/2).unwrap();
let v_texture = glium::texture::texture2d::Texture2d::empty_with_format(&display, format, mipmap, (width as u32)/2, (height as u32)/2).unwrap();
See that the sizes of the U and V textures are 1/4 of the Y texture, as expected for YUV420P.
as you see, for YUV420P I've chosen glium::texture::UncompressedFloatFormat::U8, which I think is the same as GL_RED.
The problem is that I don't know how to fill this texture with data. Its write method expect something that can be converted into a RawImage2D. However, all the filling for methods for RawImage2D expect an RGB image.
I need a method to fill only Y to the first texture, then only U to the second, and only V to the third.
Related
I investigated and stripped down my previous question (Is there a way to avoid conversion from YUV to BGR?). I want to overlay few images (format is YUV) on the resulting, bigger image (think about it like it is a canvas) and send it via network library (OPAL) forward without converting it to to BGR.
Here is the code:
Mat tYUV;
Mat tClonedYUV;
Mat tBGR;
Mat tMergedFrame;
int tMergedFrameWidth = 1000;
int tMergedFrameHeight = 800;
int tMergedFrameHalfWidth = tMergedFrameWidth / 2;
tYUV = Mat(tHeader->height * 1.5f, tHeader->width, CV_8UC1, OPAL_VIDEO_FRAME_DATA_PTR(tHeader));
tClonedYUV = tYUV.clone();
tMergedFrame = Mat(Size(tMergedFrameWidth, tMergedFrameHeight), tYUV.type(), cv::Scalar(0, 0, 0));
tYUV.copyTo(tMergedFrame(cv::Rect(0, 0, tYUV.cols > tMergedFrameWidth ? tMergedFrameWidth : tYUV.cols, tYUV.rows > tMergedFrameHeight ? tMergedFrameHeight : tYUV.rows)));
tClonedYUV.copyTo(tMergedFrame(cv::Rect(tMergedFrameHalfWidth, 0, tYUV.cols > tMergedFrameHalfWidth ? tMergedFrameHalfWidth : tYUV.cols, tYUV.rows > tMergedFrameHeight ? tMergedFrameHeight : tYUV.rows)));
namedWindow("merged frame", 1);
imshow("merged frame", tMergedFrame);
waitKey(10);
The result of above code looks like this:
I guess the image is not correctly interpreted, so the pictures stay black/white (Y component) and below them, we can see the U and V component. There are images, which describes the problem well (http://en.wikipedia.org/wiki/YUV):
and: http://upload.wikimedia.org/wikipedia/en/0/0d/Yuv420.svg
Is there a way for these values to be correctly read? I guess I should not copy the whole images (their Y, U, V components) straight to the calculated positions. The U and V components should be below them and in the proper order, am I right?
First, there are several YUV formats, so you need to be clear about which one you are using.
According to your image, it seems your YUV format is Y'UV420p.
Regardless, it is a lot simpler to convert to BGR work there and then convert back.
If that is not an option, you pretty much have to manage the ROIs yourself. YUV is commonly a plane-format where the channels are not (completely) multiplexed - and some are of different sizes and depths. If you do not use the internal color conversions, then you will have to know the exact YUV format and manage the pixel copying ROIs yourself.
With a YUV image, the CV_8UC* format specifier does not mean much beyond the actual memory requirements. It certainly does not specify the pixel/channel muxing.
For example, if you wanted to only use the Y component, then the Y is often the first plane in the image so the first "half" of whole image can just be treated as a monochrome 8UC1 image. In this case using ROIs is easy.
I want to use the CImg library (http://cimg.sourceforge.net/) to rotate an image with an arbitrary angle (the image is read by Qt which should not perform the rotation):
QImage img("sample_with_alpha.png");
img = img.convertToFormat(QImage::Format_ARGB32);
float angle = 45;
cimg_library::CImg<uint8_t> src(img.bits(), img.width(), img.height(), 1, 4);
cimg_library::CImg<uint8_t> out = src.get_rotate(angle);
// Further processing:
// Data: out.data(), out.width(), out.height(), Stride: out.width() * 4
The final data in "out.data()" is ok when the the angle is set to 0. But for other angles the output data is distorted. I assume that the CImg library changes the output format and/or stride during rotation?
Regards,
CImg does not store the pixel buffer of an image in interleaved mode, as RGBARGBARGBA... but uses a channel by channel structure RRRRRRRR.....GGGGGGGGG.......BBBBBBBBB.....AAAAAAAAA.
I assume your img.bits() pointer points to pixels with interleaved channels, so if you want to pass this to CImg, you'll need to permute the buffer structure before you can apply any of the CImg method.
Try this :
cimg_library::CImg<uint8_t> src(img.bits(), 4,img.width(), img.height(), 1);
src.permute_axes("yzcx");
cimg_library::CImg<uint8_t> out = src.get_rotate(angle);
// Here, the out image should be OK, try displaying it with out.display();
// But you still need to go back to an interleaved image pointer if you want to
// get it back in Qt.
out.permute_axes("cxyz"); // Do the inverse permutation.
const uint8_t *p_out = out.data(); // Interleaved result.
I guess this should work as expected.
I'm working on a NES emulator right now and I'm having trouble figuring out how to render the pixels. I am using a 3 dimensional array to hold the RGB value of each pixel. The array definition looks like this for the 256 x 224 screen size:
byte screenData[224][256][3];
For example, [0][0][0] holds the blue value, [0][0][1] holds the green values and [0][0][2] holds the red value of the pixel at screen position [0][0].
When the vblank flag goes high, I need to render the screen. When SDL goes to render the screen, the screenData array will be full of the RGB values for each pixel. I was able to find a function named SDL_CreateRGBSurfaceFrom that looked like it may work for what I want to do. However, all of the examples I have seen use 1 dimensional arrays for the RGB values and not a 3 dimensional array.
What would be the best way for me to render my pixels? It would also be nice if the function allowed me to resize the surface somehow so I didn't have to use a 256 x 224 window size.
You need to store the data as an unidimensional char array:
int channels = 3; // for a RGB image
char* pixels = new char[img_width * img_height * channels];
// populate pixels with real data ...
SDL_Surface *surface = SDL_CreateRGBSurfaceFrom((void*)pixels,
img_width,
img_height,
channels * 8, // bits per pixel = 24
img_width * channels, // pitch
0x0000FF, // red mask
0x00FF00, // green mask
0xFF0000, // blue mask
0); // alpha mask (none)
In 2.0, use SDL_Texture + SDL_TEXTUREACCESS_STREAMING + SDL_RenderCopy, it's faster than SDL_RenderPoint.
See:
official example: http://hg.libsdl.org/SDL/file/e12c38730512/test/teststreaming.c
my derived example which does not require blob data and compares both methods: https://github.com/cirosantilli/cpp-cheat/blob/0607da1236030d2e1ec56256a0d12cadb6924a41/sdl/plot2d.c
Related: Why do I get bad performance with SDL2 and SDL_RenderCopy inside a double for loop over all pixels?
I have *.png files and I want to get different 8x8 px parts from textures and place them on bitmap (SDL_Surface, I guess, but maybe not), smth like this:
Now I'm rendering that without bitmap, i.e. I call each texture and draw part directly on screen each frame, and it's too slow. I guess I need to load each *.png to separate bitmap and use them passing video memory, then call just one big bitmap, but maybe I'm wrong. I need the fastest way of doing that, I need code of this (SDL 2, not SDL 1.3).
Also maybe I need to use clear OpenGL here?
Update:
Or maybe I need to load *.png's to int arrays somehow and use them just like usual numbers and place them to one big int array, and then convert it to SDL_Surface/SDL_Texture? It seems this is the best way, but how to write this?
Update 2:
Colors of pixels in each block are not the same as it presented at the picture and also can they be transparent. Picture is just an example.
Assumming you already have your bitmaps loaded up as SDL_Texture(s), composing them into a different texture is done via SDL_SetRenderTarget .
SDL_SetRenderTarget(renderer, target_texture);
SDL_RenderCopy(renderer, texture1, ...);
SDL_RenderCopy(renderer, texture2, ...);
...
SDL_SetRenderTarget(renderer, NULL);
Every render operation you perform between setting your Render Target and resetting it (by calling SDL_SetRenderTarget with a NULL texture parameter) will be renderer to the designated texture. You can then use this texture as you would use any other.
Ok so, when I asked about "solid colour", I meant - "in that 8x8 pixel area in the .png that you are copying from, do all 64 pixels have the same identical RGB value?" It looks that way in your diagram, so how about this:
How about creating an SDL_Surface, and directly painting 8x8 pixel areas of the memory pointed to by the pixels member of that SDL_Surface with the values read from the original .png.
And then when you're done, convert that surface to an SDL_Texture and render that?
You would avoid all the SDL_UpdateTexture() calls.
Anyway here is some example code. Let's say that you create a class called EightByEight.
class EightByEight
{
public:
EightByEight( SDL_Surface * pDest, Uint8 r, Uint8 g, Uint8 b):
m_pSurface(pDest),
m_red(r),
m_green(g),
m_blue(b){}
void BlitToSurface( int column, int row );
private:
SDL_Surface * m_pSurface;
Uint8 m_red;
Uint8 m_green;
Uint8 m_blue;
};
You construct an object of type EightByEight by passing it a pointer to an SDL_Surface and also some values for red, green and blue. This RGB corresponds to the RGB value taken from the particular 8x8 pixel area of the .png you are currently reading from. You will paint a particular 8x8 pixel area of the SDL_Surface pixels with this RGB value.
So now when you want to paint an area of the SDL_Surface, you use the function BlitToSurface() and pass in a column and row value. For example, if you divided the SDL_Surface into 8x8 pixel squares, BlitToSurface(3,5) means the paint the square at the 4th column, and 5th row with the RGB value that I set on construction.
The BlitToSurface() looks like this:
void EightByEight::BlitToSurface(int column, int row)
{
Uint32 * pixel = (Uint32*)m_pSurface->pixels+(row*(m_pSurface->pitch/4))+column;
// now pixel is pointing to the first pixel in the correct 8x8 pixel square
// of the Surface's pixel memory. Now you need to paint a 8 rows of 8 pixels,
// but be careful - you need to add m_pSurface->pitch - 8 each time
for(int y = 0; y < 8; y++)
{
// paint a row
for(int i = 0; i < 8; i++)
{
*pixel++ = SDL_MapRGB(m_pSurface->format, m_red, m_green, m_blue);
}
// advance pixel pointer by pitch-8, to get the next "row".
pixel += (m_pSurface->pitch - 8);
}
}
I'm sure you could probably speed things up further by pre-calculating an RGB value on construction. Or if you're reading a pixel from the texture, you could probably dispense with the SDL_MapRGB() (but it's just there in case the Surface has different pixel format to the .png).
memcpy is probably faster than 8 individual assignments to the RGB value - but I just want to demonstrate the technique. You could experiment.
So, all the EightByEight objects you create, all point to the same SDL_Surface.
And then, when you're done, you just convert that SDL_Surface to an SDL_Texture and blit that.
Thanks to everyone who took part, but we solved it with my friends. So here is an example (source code is too big and unnecessary here, I'll just describe the main idea):
int pitch, *pixels;
SDL_Texture *texture;
...
if (!SDL_LockTexture(texture, 0, (void **)&pixels, &pitch))
{
for (/*Conditions*/)
memcpy(/*Params*/);
SDL_UnlockTexture(texture);
}
SDL_RenderCopy(renderer, texture, 0, 0);
I am learning OpenGL NeHe Production.When I read lesson22 Bump-MappingăMulti-texture,I got a problem.
When I load logo bmp file,I need to load two bmp files:one stores color information ,and another stores alpha information.
here is the two bmp files:
OpenGL_Alpha.bmp:
and OpenGL.bmp :
Here is the code:
if (Image=auxDIBImageLoad("Data/OpenGL_ALPHA.bmp")) {
alpha=new char[4*Image->sizeX*Image->sizeY];
for (int a=0; a<Image->sizeX*Image->sizeY; a++)
alpha[4*a+3]=Image->data[a*3]; //???????
if (!(Image=auxDIBImageLoad("Data/OpenGL.bmp"))) status=false;
for (a=0; a<Image->sizeX*Image->sizeY; a++) {
alpha[4*a]=Image->data[a*3];//??????????
alpha[4*a+1]=Image->data[a*3+1];
alpha[4*a+2]=Image->data[a*3+2];
}
glGenTextures(1, &glLogo);
glBindTexture(GL_TEXTURE_2D, glLogo);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA8, Image->sizeX, Image->sizeY, 0, GL_RGBA, GL_UNSIGNED_BYTE, alpha);
delete alpha;
}
My question is :why the index of Image->data is a*3???
Could someone interpret for me ?
I am learning OpenGL NeHe Production.When I read lesson22 Bump-Mapping
Why? The NeHe tutorials are terribly outdated, and the Bump Mapping technique outlined there completely obsolete. It's been superseeded by shader based normal mapping for well over 13 years (until 2003 texture combiners were used instead of shaders).
Also instead of BMPs you should use a image file format better suited for textures (with alpha channel). Like:
TGA
PNG
OpenEXR
Also the various compressed DX texture formats are a good choice for several applications.
My question is :why the index of Image->data is a*3???
Extracting the red channel of a RGB DIB.
It's the channel offset. The RGB data is stored as three consecutive bytes. Here 'a' represents which pixel (group of 3 bytes, one for R, one for G, one for B).
Think of a*3 as a pointer to an array of 3 bytes:
char* myPixel = Image->data + (a*3);
char red = myPixel[0];
char green = myPixel[1];
char blue = myPixel[2];