The scenario I'm dealing with is actually as follow: I need to get the screen generated by OpenGL and send it through HDMI to a FPGA component while keeping the alpha channel. But right now the data that is being sent through HDMI is only RGB (24bit without alpha channel) So i need a way to force sending the Alpha bits through this port somehow.
See image: http://i.imgur.com/hhlcbb9.jpg
One solution i could think of is to convert the screen buffer from RGBA mode to RGB while mixing the Alpha channels within the RGB buffer.
For example:
The original buffer: [R G B A][R G B A][R G B A]
The output i want: [R G B][A R G][B A R][G B A]
The point is not having to go through every single pixels.
But I'm not sure if it's possible at all using OpenGL or any technology (VideoCore kernel?)
opengl frame buffer
Do you actually mean a framebuffer, or some kind of texture? Because framebuffers cannot be resized, and the size of this resulting image will be larger in the number of pixels by 25%. You can't actually do that.
You could do it with a texture, but only by resizing it. You would have to get the texel data with glGetTexImage into some buffer, then upload the texel data to another texture with glTexImage2D. You would simply change the pixel transfer format and texture width appropriately. The read would use GL_RGBA, and the write would use GL_RGB, with an internal format of GL_RGB8.
The performance of this will almost certainly not be very good. But it should work.
But no, there is no simple call you can make to cause this to happen.
You may be able to send the alpha channel separately in opengl via different rgb or hdmi video output on your video card.
So your pc now outputs RGB down one cable, and then the Alpha down the other cable. Your alpha probably needs to be converted to grey scale so that it is still 24 bits.
You then select the which signal is the key in your fpga.
I'm presuming your fpga supports chroma key.
I have done something similar before but using a program called Brainstorm which uses a specific video card that supports SDI out and it splits the RGB, and the Alpha into separate video channels(cables), and then the vision mixer does the keying.
Today, however, I have created a new solution which mixes the multiple video channels first on the pc, and then outputs the final mixed video directly to either a RTMP streaming server or to a DVI to SDI scan converter.
Related
What't the best data-pass from usb-camera to opengl texture?
The only way I know is usb-camera -> (cv.capture()) cv_image -> glGenTexture(image.bytes)
Since CPU would parse the image for each frame, frame rate is lower.
Is there any better way?
I'm using nvidia jetson tx2, is there some way relative to the environment?
Since USB frames must be reassembled anyway by the USB driver and UVC protocol handler, the data is passing through the CPU anyway. The biggest worry is having redundant copy operations.
If the frames are transmitted in M-JPEG format (which almost all UVC compliant cameras do support), then you must decode it on the CPU anyway, since GPU video decoding acceleration HW usually doesn't cover JPEG (also JPEG is super easy to decode).
For YUV color formats it is advisable to create two textures, one for the Y channel, one for the UV channels. Usually YUV formats are planar (i.e. images of a single component per pixel each), so you'd make the UV texture a 2D array with two layers. Since chroma components may be subsampled you need the separate textures to support the different resolutions.
RGB data goes in is a regular 2D texture.
Use a pixel buffer object (PBO) for transfer. By mapping the PBO into host memory (glMapBuffer) you can decode the images coming from the camera directly into that staging PBO. After unmapping a call to glTexSubImage2D will then transfer the image to the GPU memory – in the case of a unified memory architecture this "transfer" might be as simple as shuffling around a few internal buffer references.
Since you didn't mention the exact API used to access the video device, it's difficult to give more detailed information.
I'm using IPP to resize image efficiently, but the really curious part of IPP is 'Does IPP image resize function care about alpha value position?'
There's really lot of pixel type of image, but there's well-used two pixel type: 'ARGB, RGBA'. and the ippiResizeNearest_8u_C4R function resize four-channel-image according to Intel IPP function naming documentation.
And as you can see at documentation, we can distinguish function caring position of alpha channel like: A, A0. but ippiResizeNearest_8u_C4R function doesn't have any of alpha channel descriptor. so I can't even expect which color-order IPP resizes.
so my question is: Do Intel IPP image resize function care about position of alpha value? if not, what's the default pixel type for ippiResizeNearest_8u_C4R function?
I am no IPP expert but a quick read in the documentation seems to confirm what I believe to be correct. Anything else wouldn't make sense to me.
Resizing a multi channel image is usually done for each channel separately because interpolation between channels just doesn't make sense (given that we are talking about 2d images). It would completely mess up colours and transparencies.
And if we do it for every channel separately, IPP should not care about how you call your channels. It will resize ARGB the same way as RGBA. It will not change the channel order so if you put ARGB in you'll get a resized ARGB out. In case of resizing it does not matter what you store in those 4 channels.
Just try it out. Create a test image with different values in each channel. The resized image should still have those values.
We are currently switching to WebP for textures in a video game. We've come across a problem where the areas in an image that have the alpha channel set to zero end up losing all the detail. You can see this effect in the following example:
Original Image (left is color channels, right is alpha channel)
After saving as WebP
As you can see, the zero-alpha areas have lost their detail.
This optimization makes sense when the alpha channel is being used as transparency. However, in our game we are using the alpha for something else and need to maintain the color channel integrity independently from the alpha channel. How do I disable this effect in the encoder so the color channel encodes normally?
I should mention I'm using libwebp in C++, calling the function WebPEncodeRGBA.
Thanks!
https://developers.google.com/speed/webp/docs/cwebp
In this documentation, the -exact parameter is documented.
-exact
Preserve RGB values in transparent area. The default is off, to help compressibility.
Found the solution. After tracing through libwebp code I discovered an undocumented option in WebPConfig called "exact". Setting this to 1 will prevent the library from optimizing zero-alpha areas when encoding.
My requirements: Overlay graphics (with alpha/antialiasing) onto a UYVY image as fast as possible. The rendering must take place in UYVY preferably because I need to to both render and encode (H.264 with ffmpeg).
What framework (perferablly cross-platform, but Windows only is OK) should I use to render the image to later render/encode?
I looked at openvc, and it seems the drawing happens in BGR, which would requirement to convert each frame from UYVY (2-channel) to BGR (3-channel), and then back again.
I looked at SDL, which uses hardware acceleration. It supports multiple textures with different color spaces. However, the method SDL_RenderReadPixels, which I would need to get the resulting composited image, mentions in the documentation "warning This is a very slow operation, and should not be used frequently."
Is there a framework that can draw onto a BYTE array of YUV, possible with alpha blending/anti-aliasing?
You also can convert YUV to BGRA. And then perform drawing operation with using of the format. BGRA is more convenient then BGR for drawing because every its pixel is equal to 32-bit integer. Naturally after drawing you have to convert backward BGRA to YUV.
There is a fast cross-platform C++ library which can perform these manipulations.
I am writing a custom video rendering filter for Directshow. My renderer assumes the incoming pixels are organized one row of pixels at a time (correct assumption?) and blits them to another DirectX display elsewhere using a DirectX texture.
This approach works with webcams as input, but when I use an analog capture board, the samples the renderer receives are not in any expected order (see left image below). When I render the capture using the stock DirectShow video renderer, it looks fine (see right image below). So the directshow renderer must be doing something extra that my renderer is not. Any idea what it is?
Some more details:
The capture card is NTSC, I'm not sure if that would matter.
As input to the custom renderer, I am accepting only MEDIASUBTYPE_RGB24, so I do not think that this is a YUV issue (is it?).
It's a bit hard to see, but the second image below is my filter graph. My custom renderer connects to the color space converter on the far right.
I assume that the pixels coming into my renderer, are all organized one row of pixels at a time. Is this a correct assumption?
Maybe texture is padded to keep rows aligned at (multiply of) 32 bytes per row? Mind you that I did not ever use DirectShow but that's what I would expect in D3D.In other words, your input might have different stride than you think. Unfortunately I do not know DS so I can only assume that something that computes input / output coordinates should have different stride factor e.g. something in code that looks like this offset = y * stride + x.