How do I remove (or apply) transparency on a gdk-pixbuf? - c++

I have a c++ program in which a gdk-pixbuf is created. I want to output it as an image, so I call gdk_pixbuf_save_to_stream(pixbuf,stream,type,NULL,&err,NULL). This works fine when "type" is png or tiff, but with jpeg or bmp it just produces a black square. The original pixbuf consists of black-on-transparent (and gdk_pixbuf_get_has_alpha returns true) so I'm guessing that the problem is with the alpha mask.
GdkPixbuf has a function to add an alpha channel, but I can't see one that removes it again, or (which might be as good) to invert it.
Is there a simple way to get the jpeg and bmp formats to work properly?
(I should say that I'm very new to proper programming like this.)

JPEG doesn't have any notion of an alpha channel, or transparency at all. The alpha channel is stripped during the conversion to JPEG. BMP has the same restriction.
Since transparency is important to you, your program should stick to generating PNGs.
As far as the question you've posed in the title, removing an alpha channel can be done manually. The trick is understanding how the data in a GdkPixbuf is stored. When you have an RGB pixbuf with an alpha channel (also called RGBA), the pixels are stored as 32-bit values: 4 bytes, one byte per color, the fourth being the alpha channel. RGB pixbufs are stored as 24-bit values, one byte per color.
So, if you create a temporary byte buffer and copy over the first three bytes of each RGBA pixel and drop the fourth, that temporary buffer is then pure RGB. To diagram it a little:
[R][G][B][A][R][G][B][A]... => [R][G][B][R][G][B]...
Note that you have to pack the temporary buffer; there's no spare byte between the [B] byte and the next [R] byte.
You then create a new GdkPixbuf by handing it this RGB buffer, and you've removed the alpha channel.
See gdk_pixbuf_get_pixels() to access the RGBA buffer and gdk_pixbuf_new_from_data() to create the RGB pixbuf. See here for more discussion on how the packed data is stored inside a GdkPixbuf.

Here is (rather inefficient and ugly) Vala application that removes transparency from an image and saves it in the format specified. NOTE: There is a small bug in vala binding for gdk_pixbuf_new_from_data() that causes corruption of the resulting image. I'm going to fix that soon but this is meanly for demonstration purposes for now (besides the question was about C++):
public static int main (string[] args) {
if (args.length < 4) {
print ("Usage: %s SOURCE DESTINATION FORMAT\n", args[0]);
return -1;
}
var src_path = args[1];
var destination_path = args[2];
var dest_type = args[3];
var pixbuf = new Gdk.Pixbuf.from_file_at_scale (src_path, 48, 48, false);
// Remove alpha channel
if (pixbuf.get_has_alpha () && pixbuf.get_n_channels () == 4 && pixbuf.get_bits_per_sample () == 8) {
var width = pixbuf.get_width ();
var height = pixbuf.get_height ();
var rowstride = pixbuf.get_rowstride ();
unowned uint8[] orig_pixels = pixbuf.get_pixels ();
var pixels = new uint8[rowstride * height];
for (var i = 0; i < height; i++) {
for (var j = 0, k = 0; j < width * 4; j += 4, k += 3) {
var orig_index = rowstride * i + j;
var index = rowstride * i + k;
if (orig_pixels[orig_index] == 0 &&
orig_pixels[orig_index + 1] == 0 &&
orig_pixels[orig_index + 2] == 0 &&
orig_pixels[orig_index + 3] == 0) {
pixels[index] = 0xFF;
pixels[index + 1] = 0xFF;
pixels[index + 2] = 0xFF;
} else {
pixels[index] = orig_pixels[orig_index];
pixels[index + 1] = orig_pixels[orig_index + 1];
pixels[index + 2] = orig_pixels[orig_index + 2];
}
}
}
pixbuf = new Gdk.Pixbuf.from_data (pixels,
pixbuf.get_colorspace (),
false,
8,
width,
height,
rowstride,
null);
}
pixbuf.save (destination_path, dest_type);
return 0;
}

Related

Why my bitmap image have another color overlay after converting 32-bit to 8-bit

Im working on resizing bitmap image and converting bitmap image to 8-bit (grayscale). But I have the problem that when I convert 32-bit image to 8-bit image, the result has another color overlay while it works perfectly on 24-bit. I guess the cause is in the alpha color. but I dont know where the problem exactly is.
This is my code to generate 8-bit palette color and write it after DIB part:
char* palette = new char[1024];
for (int i = 0; i < 256; i++) {
palette[i * 4] = palette[i * 4 + 1] = palette[i * 4 + 2] = (char)i;
palette[i * 4 + 3] = 255;
}
fout.write(palette, 1024);
delete[] palette;
As I said, my code works perfectly on 24-bit. In 32-bit the color is still kept after resizing, but when converting to 8-bit, it will look like this:
expected image (when converted from 24-bit) //
unexpected image (when converted from 32-bit)
This is how I get the colors and save it to srcPixel[]:
int i = 0;
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
int index = getIndex(width, x, y);
srcPixel[index].A = srcBMP.pImageData[i];
i += alpha;
srcPixel[index].B = srcBMP.pImageData[i++];
srcPixel[index].G = srcBMP.pImageData[i++];
srcPixel[index].R = srcBMP.pImageData[i++];
}
i += padding;
}
And this is the code I converted it by getting average of 4 colors A, B, G and R from that srcPixel[]:
int i = 0;
for (int y = 0; y < dstHeight; y++) {
for (int x = 0; x < dstWidth; x++) {
int index = getIndex(dstWidth, x, y);
dstBMP.pImageData[i++] = (srcPixel[index].A + srcPixel[index].B + srcPixel[index].G + srcPixel[index].R) / 4;
}
i += dstPadding;
}
If I remove and skip all alpha bytes in my code, when converting my image is still like that and I will have another problem is when resizing, my image will have another color overlay like the problem when converting to 8-bit: resizing without alpha channel.
If I skip the alpha channel while getting average (change into dstBMP.pImageData[i++] = (srcPixel[index].B + srcPixel[index].G + srcPixel[index].R) / 3, there is almost nothing different, the overlay still exists.
If I remove palette[i * 4 + 3] = 255; or doing anything with it, the result is still not affected.
Thank you very much.
You add alpha channel to the color and that's why it becomes brighter. From here I found that opaque is 255 and transparent 0 - therefore you add another channel which is set to 'white' to your result.
Remove alpha channel from your equation and see if I'm right.

How to convert CMSampleBufferRef/CIImage/UIImage into pixels e.g. uint8_t[]

I have input from captured camera frame as CMSampleBufferRef and I need to get the raw pixels preferably in C type uint8_t[].
I also need to find the color scheme of the input image.
I know how to convert CMSampleBufferRef to UIImage and then to NSData with png format but I dont know how to get the raw pixels from there. Perhaps I could get it already from CMSampleBufferRef/CIImage`?
This code shows the need and the missing bits.
Any thoughts where to start?
int convertCMSampleBufferToPixelArray (CMSampleBufferRef sampleBuffer)
{
// inputs
CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
CIImage *ciImage = [CIImage imageWithCVPixelBuffer:imageBuffer];
CIContext *imgContext = [CIContext new];
CGImageRef cgImage = [imgContext createCGImage:ciImage fromRect:ciImage.extent];
UIImage *uiImage = [UIImage imageWithCGImage:cgImage];
NSData *nsData = UIImagePNGRepresentation(uiImage);
// Need to fill this gap
uint8_t* data = XXXXXXXXXXXXXXXX;
ImageFormat format = XXXXXXXXXXXXXXXX; // one of: GRAY8, RGB_888, YV12, BGRA_8888, ARGB_8888
// sample showing expected data values
// this routine converts the image data to gray
//
int width = uiImage.size.width;
int height = uiImage.size.height;
const int size = width * height;
std::unique_ptr<uint8_t[]> new_data(new uint8_t[size]);
for (int i = 0; i < size; ++i) {
new_data[i] = uint8_t(data[i * 3] * 0.299f + data[i * 3 + 1] * 0.587f +
data[i * 3 + 2] * 0.114f + 0.5f);
}
return 1;
}
Some pointers you can use to search for more info. It's nicely documented and you shouldn't have an issue.
int convertCMSampleBufferToPixelArray (CMSampleBufferRef sampleBuffer) {
CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
if (imageBuffer == NULL) {
return -1;
}
// Get address of the image buffer
CVPixelBufferLockBaseAddress(imageBuffer, 0);
uint8_t* data = CVPixelBufferGetBaseAddress(imageBuffer);
// Get size
size_t width = CVPixelBufferGetWidth(imageBuffer);
size_t height = CVPixelBufferGetHeight(imageBuffer);
// Get bytes per row
size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);
// At `data` you have a bytesPerRow * height bytes of the image data
// To get pixel info you can call CVPixelBufferGetPixelFormatType, ...
// you can call CVImageBufferGetColorSpace and inspect it, ...
// When you're done, unlock the base address
CVPixelBufferUnlockBaseAddress(imageBuffer, 0);
return 0;
}
There're couple of things you should be aware of.
First one is that it can be planar. Check the CVPixelBufferIsPlanar, CVPixelBufferGetPlaneCount, CVPixelBufferGetBytesPerRowOfPlane, etc.
Second one is that you have to calculate pixel size based on CVPixelBufferGetPixelFormatType. Something like:
CVPixelBufferGetPixelFormatType(imageBuffer)
size_t pixelSize;
switch (pixelFormat) {
case kCVPixelFormatType_32BGRA:
case kCVPixelFormatType_32ARGB:
case kCVPixelFormatType_32ABGR:
case kCVPixelFormatType_32RGBA:
pixelSize = 4;
break;
// + other cases
}
Let's say that the buffer is not planar and:
CVPixelBufferGetWidth returns 200 (pixels)
Your pixelSize is 4 (calcuated bytes per row is 200 * 4 = 800)
CVPixelBufferGetBytesPerRow can return anything >= 800
In other words, the pointer you have is not a pointer to a contiguous buffer. If you need row data you have to do something like this:
uint8_t* data = CVPixelBufferGetBaseAddress(imageBuffer);
// Get size
size_t width = CVPixelBufferGetWidth(imageBuffer);
size_t height = CVPixelBufferGetHeight(imageBuffer);
size_t pixelSize = 4; // Let's pretend it's calculated pixel size
size_t realRowSize = width * pixelSize;
size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);
for (int row = 0 ; row < height ; row++) {
// bytesPerRow acts like an offset where the next row starts
// bytesPerRow can be >= realRowSize
uint8_t *rowData = data + row * bytesPerRow;
// realRowSize = how many bytes are available for this row
// copy them somewhere
}
You have to allocate a buffer and copy these row data there if you'd like to have contiguous buffer. How many bytes to allocate? CVPixelBufferGetDataSize.

Convert FreeType GlyphSlot Bitmap To Vulkan BGRA

I'm trying to convert a FreeType GlyphSlot Bitmap to Vulkan BGRA format.
void DrawText(const std::string &text) {
// WIDTH & HEIGHT == dst image dimensions
FT_GlyphSlot Slot = face->glyph;
buffer.resize(WIDTH*HEIGHT*4);
int dst_Pitch = WIDTH * 4;
for (auto c : text) {
FT_Error error = FT_Load_Char(face, c, FT_LOAD_RENDER);
if (error) {
printf("FreeType: Load Char Error\n");
continue;
}
auto char_width = Slot->bitmap.width;
auto char_height = Slot->bitmap.rows;
uint8_t* src = Slot->bitmap.buffer;
uint8_t* startOfLine = src;
for (int y = 0; y < char_height; ++y) {
src = startOfLine;
for (int x = 0; x < char_width; ++x) {
// y * dst_Pitch == Destination Image Row
// x * 4 == Destination Image Column
int dst = (y*dst_Pitch) + (x*4);
// Break if we have no more space to draw on our
// destination texture.
if (dst + 4 > buffer.size()) { break; }
auto value = *src;
src++;
buffer[dst] = 0xff; // +0 == B
buffer[dst+1] = 0xff; // +1 == G
buffer[dst+2] = 0xff; // +2 == R
buffer[dst+3] = value; // +3 == A
}
startOfLine += Slot->bitmap.pitch;
}
}
}
This is giving me garbled output. I'm not sure what I need to do to properly convert to Vulkan B8G8R8A8. I feel like moving from left to right in the buffer we write to our Vulkan texture is incorrect and maybe Vulkan is expecting I add the pixels into the buffer in a different way?
I understand this code will write each letter on top of one another, I will implement taking advantage of Slot->advance after I can properly draw at least a single letter.
One problem is that you resize buffer with every character (which will leave the previous data at the start of the newly allocated space) but when storing the data for the new character c you overwrite the start of the buffer since dst is 0. You probably want to set dst the buffer.size() from before the resize call.
int dst = /*previous buffer size*/;
The issue was due to the fact that I had VkImageCreateInfo tiling set to VK_IMAGE_TILING_OPTIMAL. After changing it to VK_IMAGE_TILING_LINEAR I received the correct output.
Taken straight from https://www.khronos.org/registry/vulkan/specs/1.1-extensions/man/html/VkImageTiling.html
VK_IMAGE_TILING_OPTIMAL specifies optimal tiling (texels are laid out
in an implementation-dependent arrangement, for more optimal memory
access).
VK_IMAGE_TILING_LINEAR specifies linear tiling (texels are laid out in
memory in row-major order, possibly with some padding on each row).
While I may not be rendering garbage now, my letters are still backwards and seemingly drawing from right to left instead of left to right.
You can see the green 'the' in the top right corner.

ImageMagick C++ Version 7 Modify Pixel Value in Blank Image

I have the following code that creates a blank black image and then attempts to write to that image by modifying each pixel to red.
Magick::Image image(Magick::Geometry(1024, 1024),
Magick::Color(std::uint8_t(0), std::uint8_t(0), std::uint8_t(0)));
assert(image.channels() == 3 && "Created wrong image format.");
image.type(Magick::TrueColorType);
image.fillColor("black");
std::size_t w = image.columns();
std::size_t h = image.rows();
assert(image.columns() == 1024 && image.rows() == 1024);
Magick::Quantum *mpixels = image.setPixels(0, 0, w, h);
for (int row = 0; row < h - 1; ++row) {
for (int col = 0; col < w - 1; ++col) {
std::size_t offset = (w * row + col);
std::size_t moffset = image.channels() * offset;
mpixels[moffset + 0] = 255;
mpixels[moffset + 1] = 0;
mpixels[moffset + 2] = 0;
}
}
image.syncPixels();
image.write(out.c_str());
However, after inspecting the image it is still all black after changing the pixel values. What do I need to change to modify the pixel values?
I suspect that you are using the Q16 version of ImageMagick which means that each pixel channel value will be in the range 0-65535 and you are using 255 for the red channel which is really close to black. I think the following will fix your issue:
mpixels[moffset + 0] = 65535;
You could also decide to switch to the Q8 version of ImageMagick if channels in the range 0-255 would be sufficient for you.

How to compress YUYV raw data to JPEG using libjpeg?

I'm looking for an example of how to save a YUYV format frame to a JPEG file using the libjpeg library.
In typical computer APIs, "YUV" actually means YCbCr, and "YUYV" means "YCbCr 4:2:2" stored as Y0, Cb01, Y1, Cr01, Y2 ...
Thus, if you have a "YUV" image, you can save it to libjpeg using the JCS_YCbCr color space.
When you have a 422 image (YUYV) you have to duplicate the Cb/Cr values to the two pixels that need them before writing the scanline to libjpeg. Thus, this write loop will do it for you:
// "base" is an unsigned char const * with the YUYV data
// jrow is a libjpeg row of samples array of 1 row pointer
cinfo.image_width = width & -1;
cinfo.image_height = height & -1;
cinfo.input_components = 3;
cinfo.in_color_space = JCS_YCbCr;
jpeg_set_defaults(&cinfo);
jpeg_set_quality(&cinfo, 92, TRUE);
jpeg_start_compress(&cinfo, TRUE);
unsigned char *buf = new unsigned char[width * 3];
while (cinfo.next_scanline < height) {
for (int i = 0; i < cinfo.image_width; i += 2) {
buf[i*3] = base[i*2];
buf[i*3+1] = base[i*2+1];
buf[i*3+2] = base[i*2+3];
buf[i*3+3] = base[i*2+2];
buf[i*3+4] = base[i*2+1];
buf[i*3+5] = base[i*2+3];
}
jrow[0] = buf;
base += width * 2;
jpeg_write_scanlines(&cinfo, jrow, 1);
}
jpeg_finish_compress(&cinfo);
delete[] buf;
Use your favorite auto-ptr to avoid leaking "buf" if your error or write function can throw / longjmp.
Providing YCbCr to libjpeg directly is preferrable to converting to RGB, because it will store it directly in that format, thus saving a lot of conversion work. When the image comes from a webcam or other video source, it's also usually most efficient to get it in YCbCr of some sort (such as YUYV.)
Finally, "U" and "V" mean something slightly different in analog component video, so the naming of YUV in computer APIs that really mean YCbCr is highly confusing.
libjpeg also has a raw data mode, whereby you can directly supply the raw downsampled data (which is almost what you have in the YUYV format). This is more efficient than duplicating the UV values only to have libjpeg downscale them again internally.
To do so, you use jpeg_write_raw_data instead of jpeg_write_scanlines, and by default it will process exactly 16 scanlines at a time. JPEG expects the U and V planes to be 2x downsampled by default. YUYV format already has the horizontal dimension downsampled but not the vertical, so I skip U and V every other scanline.
Initialization:
cinfo.image_width = /* width in pixels */;
cinfo.image_height = /* height in pixels */;
cinfo.input_components = 3;
cinfo.in_color_space = JCS_YCbCr;
jpeg_set_defaults(&cinfo);
cinfo.raw_data_in = true;
JSAMPLE y_plane[16][cinfo.image_width];
JSAMPLE u_plane[8][cinfo.image_width / 2];
JSAMPLE v_plane[8][cinfo.image_width / 2];
JSAMPROW y_rows[16];
JSAMPROW u_rows[8];
JSAMPROW v_rows[8];
for (int i = 0; i < 16; ++i)
{
y_rows[i] = &y_plane[i][0];
}
for (int i = 0; i < 8; ++i)
{
u_rows[i] = &u_plane[i][0];
}
for (int i = 0; i < 8; ++i)
{
v_rows[i] = &v_plane[i][0];
}
JSAMPARRAY rows[] { y_rows, u_rows, v_rows };
Compressing:
jpeg_start_compress(&cinfo, true);
while (cinfo.next_scanline < cinfo.image_height)
{
for (JDIMENSION i = 0; i < 16; ++i)
{
auto offset = (cinfo.next_scanline + i) * cinfo.image_width * 2;
for (JDIMENSION j = 0; j < cinfo.image_width; j += 2)
{
y_plane[i][j] = image.data[offset + j * 2 + 0];
y_plane[i][j + 1] = image.data[offset + j * 2 + 2];
if (i % 2 == 0)
{
u_plane[i / 2][j / 2] = image_data[offset + j * 2 + 1];
v_plane[i / 2][j / 2] = image_data[offset + j * 2 + 3];
}
}
}
jpeg_write_raw_data(&cinfo, rows, 16);
}
jpeg_finish_compress(&cinfo);
I was able to get about a 33% decrease in compression time with this method compared to the one in #JonWatte's answer. This solution isn't for everyone though; some caveats:
You can only compress images with dimensions that are a multiple of 8. If you have different-sized images, you will have to write code to pad in the edges. If you're getting the images from a camera though, they will most likely be this way.
The quality is somewhat impaired by the fact that I simply skip color values for alternating scanlines instead of something fancier like averaging them. For my application though, speed was more important than quality.
The way it's written right now it allocates a ton of memory on the stack. This was acceptable for me because my images were small (640x480) and enough memory was available.
Documentation for libjpeg-turbo: https://raw.githubusercontent.com/libjpeg-turbo/libjpeg-turbo/master/libjpeg.txt