I have a image with 4 channels that i need to overlay it over a bunch of pictures. Over the pictures with 3 channels, the overlaying works great, but over the pictures that have an alpha channel, the background of the picture changes to black.
Original picture: http://img.blog.csdn.net/20130610074054484
Overlayed picture: http://imgur.com/mlVAN0A
This is the code that does the overlaying:
void overlayImage(const cv::Mat &background, const cv::Mat &foreground,
cv::Mat &output, cv::Point2i location)
{
background.copyTo(output);
for(int y = std::max(location.y , 0); y < background.rows; ++y)
{
int fY = y - location.y;
if(fY >= foreground.rows)
break;
for(int x = std::max(location.x, 0); x < background.cols; ++x)
{
int fX = x - location.x;
if(fX >= foreground.cols)
break;
double opacity = ((double)foreground.data[fY * foreground.step + fX * foreground.channels() + 3]) / 255.;
for(int c = 0; opacity > 0 && c < output.channels(); ++c)
{
unsigned char foregroundPx = foreground.data[fY * foreground.step + fX * foreground.channels() + c];
unsigned char backgroundPx = background.data[y * background.step + x * background.channels() + c];
output.data[y*output.step + output.channels()*x + c] =
backgroundPx * (1.-opacity) + foregroundPx * opacity;
}
}
}
}
This is because you use 1-opacity for the background image. If the opacity of the forground image is 0, the opacity of your backgroundpixel will be 1 instead of 0 which it is before.
You have to calc the result opacity fpr both images which can be 0 for both too.
Claus
Related
As context, I'm working with building a topographic program which needs relatively extreme detail. I do not expect the files to be small, and they do not formally need to be viewed on a monitor, they just need to have very high resolution.
I know that most image formats are limited to 8 bpp, on account of the standard limits on both monitors (at a reasonable price) and on human perception. However, 2⁸ is just 256 possible values, which induces plateauing artifacts in a reconstructed displacement. 2¹⁶ may be close enough at 65,536 possible values, which I have achieved.
I'm using FreeImage and DLang to construct the data, currently on a Linux Mint machine.
However, when I went on to 2³², software support seemed to fade on me. I tried a TIFF of this form and nothing seemed to be able to interpret it, either showing a completely (or mostly) transparent image (remembering that I didn't expect any monitor to really support 2³² shades of a channel) or complaining about being unable to decode the RGB data. I imagine that it's because it was assumed to be an RGB or RGBA image.
FreeImage is reasonably well documented for most purposes, but I'm now wondering, what is the highest-precision single-channel format I can export, and how would I do it? Can anyone provide an example? Am I really limited, in any typical and not-home-rolled image format, to 16-bit? I know that's high enough for, say, medical imaging, but I'm sure I'm not the first person to try to aim higher and we science-types can be pretty ambitious about our precision-level…
Did I make a glaring mistake in my code? Is there something else I should try instead for this kind of precision?
Here's my code.
The 16-bit TIFF that worked
void writeGrayscaleMonochromeBitmap(const double width, const double height) {
FIBITMAP *bitmap = FreeImage_AllocateT(FIT_UINT16, cast(int)width, cast(int)height);
for(int y = 0; y < height; y++) {
ubyte *scanline = FreeImage_GetScanLine(bitmap, y);
for(int x = 0; x < width; x++) {
ushort v = cast(ushort)((x * 0xFFFF)/width);
ubyte[2] bytes = nativeToLittleEndian(cast(ushort)(x/width * 0xFFFF));
scanline[x * ushort.sizeof + 0] = bytes[0];
scanline[x * ushort.sizeof + 1] = bytes[1];
}
}
FreeImage_Save(FIF_TIFF, bitmap, "test.tif", TIFF_DEFAULT);
FreeImage_Unload(bitmap);
}
The 32-bit TIFF that didn't really work
void writeGrayscaleMonochromeBitmap32(const double width, const double height) {
FIBITMAP *bitmap = FreeImage_AllocateT(FIT_UINT32, cast(int)width, cast(int)height);
writeln(width, ", ", height);
writeln("Width: ", FreeImage_GetWidth(bitmap));
for(int y = 0; y < height; y++) {
ubyte *scanline = FreeImage_GetScanLine(bitmap, y);
writeln(y, ": ", scanline);
for(int x = 0; x < width; x++) {
//writeln(x, " < ", width);
uint v = cast(uint)((x/width) * 0xFFFFFFFF);
writeln("V: ", v);
ubyte[4] bytes = nativeToLittleEndian(v);
scanline[x * uint.sizeof + 0] = bytes[0];
scanline[x * uint.sizeof + 1] = bytes[1];
scanline[x * uint.sizeof + 2] = bytes[2];
scanline[x * uint.sizeof + 3] = bytes[3];
}
}
FreeImage_Save(FIF_TIFF, bitmap, "test32.tif", TIFF_NONE);
FreeImage_Unload(bitmap);
}
Thanks for any pointers.
For a single channel, the highest available from FreeImage is 32-bit, as FIT_UINT32. However, the file format must be capable of this, and as of the moment, only TIFF appears to be up to the task (See page 104 of the Stanford Documentation). Additionally, most monitors are incapable of representing more than 8-bits-per-sample, 12 in extreme cases, so it is very difficult to read data back out and have it render properly.
A unit test involving comparing bytes before marshaling to the bitmap, and sampled from the same bitmap afterward, show that the data is in fact being encoded.
To imprint data to a 16-bit gray scale (currently supported by J2K, JP2, PGM, PGMRAW, PNG and TIF), you would do something like this:
void toFreeImageUINT16PNG(string fileName, const double width, const double height, double[] data) {
FIBITMAP *bitmap = FreeImage_AllocateT(FIT_UINT16, cast(int)width, cast(int)height);
for(int y = 0; y < height; y++) {
ubyte *scanline = FreeImage_GetScanLine(bitmap, y);
for(int x = 0; x < width; x++) {
//This magic has to happen with the y-coordinate in order to keep FreeImage from following its default behavior, and generating
//the image upside down.
ushort v = cast(ushort)(data[cast(ulong)(((height - 1) - y) * width + x)] * 0xFFFF); //((x * 0xFFFF)/width);
ubyte[2] bytes = nativeToLittleEndian(v);
scanline[x * ushort.sizeof + 0] = bytes[0];
scanline[x * ushort.sizeof + 1] = bytes[1];
}
}
FreeImage_Save(FIF_PNG, bitmap, fileName.toStringz);
FreeImage_Unload(bitmap);
}
Of course you would want to make adjustments for your target file type. To export as 48-bit RGB16, you would do this.
void toFreeImageColorPNG(string fileName, const double width, const double height, double[] data) {
FIBITMAP *bitmap = FreeImage_AllocateT(FIT_RGB16, cast(int)width, cast(int)height);
uint pitch = FreeImage_GetPitch(bitmap);
uint bpp = FreeImage_GetBPP(bitmap);
for(int y = 0; y < height; y++) {
ubyte *scanline = FreeImage_GetScanLine(bitmap, y);
for(int x = 0; x < width; x++) {
ulong offset = cast(ulong)((((height - 1) - y) * width + x) * 3);
ushort r = cast(ushort)(data[(offset + 0)] * 0xFFFF);
ushort g = cast(ushort)(data[(offset + 1)] * 0xFFFF);
ushort b = cast(ushort)(data[(offset + 2)] * 0xFFFF);
ubyte[6] bytes = nativeToLittleEndian(r) ~ nativeToLittleEndian(g) ~ nativeToLittleEndian(b);
scanline[(x * 3 * ushort.sizeof) + 0] = bytes[0];
scanline[(x * 3 * ushort.sizeof) + 1] = bytes[1];
scanline[(x * 3 * ushort.sizeof) + 2] = bytes[2];
scanline[(x * 3 * ushort.sizeof) + 3] = bytes[3];
scanline[(x * 3 * ushort.sizeof) + 4] = bytes[4];
scanline[(x * 3 * ushort.sizeof) + 5] = bytes[5];
}
}
FreeImage_Save(FIF_PNG, bitmap, fileName.toStringz);
FreeImage_Unload(bitmap);
}
Lastly, to encode a UINT32 greyscale image (limited purely to TIFF at the moment), you would do this.
void toFreeImageTIF32(string fileName, const double width, const double height, double[] data) {
FIBITMAP *bitmap = FreeImage_AllocateT(FIT_UINT32, cast(int)width, cast(int)height);
//DEBUG
int xtest = cast(int)(width/2);
int ytest = cast(int)(height/2);
uint comp1a = cast(uint)(data[cast(ulong)(((height - 1) - ytest) * width + xtest)] * 0xFFFFFFFF);
writeln("initial: ", nativeToLittleEndian(comp1a));
for(int y = 0; y < height; y++) {
ubyte *scanline = FreeImage_GetScanLine(bitmap, y);
for(int x = 0; x < width; x++) {
//This magic has to happen with the y-coordinate in order to keep FreeImage from following its default behavior, and generating
//the image upside down.
ulong i = cast(ulong)(((height - 1) - y) * width + x);
uint v = cast(uint)(data[i] * 0xFFFFFFFF);
ubyte[4] bytes = nativeToLittleEndian(v);
scanline[x * uint.sizeof + 0] = bytes[0];
scanline[x * uint.sizeof + 1] = bytes[1];
scanline[x * uint.sizeof + 2] = bytes[2];
scanline[x * uint.sizeof + 3] = bytes[3];
}
}
//DEBUG
ulong index = cast(ulong)(xtest * uint.sizeof);
writeln("Final: ", FreeImage_GetScanLine(bitmap, ytest)
[index .. index + uint.sizeof]);
FreeImage_Save(FIF_TIFF, bitmap, fileName.toStringz);
FreeImage_Unload(bitmap);
}
I've yet to find a program, built by anyone else, which will readily render a 32-bit gray-scale image on a monitor's available palette. However, I left my checking code in which will consistently write out the same array both at the top DEBUG and the bottom one, and that's consistent enough for me.
Hopefully this will help someone else out in the future.
I am rendering the Viewport with a resolution of something like 1920x1080 multiplied by a Oversampling value like 4. Now i need to downsample from the rendered Resolution 7680x4320 back to the 1920x1080.
Are there any functions in Unreal I could use for that ? Or any Library (windows only) which handle this nicely ?
Or what would be a propper way of writing this my own ?
We tried to implement a downsampling but it only works if SnapshotScale is 2, when its higher than 2 it doesn't seem to have an effect regarding image quality.
UTexture2D* AAVESnapShotManager::DownsampleTexture(UTexture2D* Texture)
{
UTexture2D* Result = UTexture2D::CreateTransient(RenderSettings.imageWidth, RenderSettings.imageHeight, PF_B8G8R8A8);
void* TextureDataVoid = Texture->PlatformData->Mips[0].BulkData.Lock(LOCK_READ_ONLY);
void* ResultDataVoid = Result->PlatformData->Mips[0].BulkData.Lock(LOCK_READ_WRITE);
FColor* TextureData = (FColor*)TextureDataVoid;
FColor* ResultData = (FColor*)ResultDataVoid;
int32 WindowSize = RenderSettings.resolutionScale / 2;
for (int x = 0; x < Result->GetSizeX(); ++x)
{
for (int y = 0; y < Result->GetSizeY(); ++y)
{
const uint32 ResultIndex = y * Result->GetSizeX() + x;
uint32_t R = 0, G = 0, B = 0, A = 0;
int32 Samples = 0;
for (int32 dx = -WindowSize; dx < WindowSize; ++dx)
{
for (int32 dy = -WindowSize; dy < WindowSize; ++dy)
{
int32 PosX = (x * RenderSettings.resolutionScale + dx);
int32 PosY = (y * RenderSettings.resolutionScale + dy);
if (PosX < 0 || PosX >= Texture->GetSizeX() || PosY < 0 || PosY >= Texture->GetSizeY())
{
continue;
}
size_t TextureIndex = PosY * Texture->GetSizeX() + PosX;
FColor& Color = TextureData[TextureIndex];
R += Color.R;
G += Color.G;
B += Color.B;
A += Color.A;
++Samples;
}
}
ResultData[ResultIndex] = FColor(R / Samples, G / Samples, B / Samples, A / Samples);
}
}
Texture->PlatformData->Mips[0].BulkData.Unlock();
Result->PlatformData->Mips[0].BulkData.Unlock();
Result->UpdateResource();
return Result;
}
I expect a high quality oversampled Texture output, working with any positive int value in SnapshotScale.
I have a suggestion. It's not really direct, but it involves no writing of image filtering or importing of libraries.
Make an unlit Material with nodes TextureObject->TextureSample-> connect to Emissive.
Use the texture you start with in your function to populate the Texture Object on a Material Instance Dynamic of the material.
Use the "Draw Material to Render Target" function to draw the Material Instance Dynamic to a Render Target that is pre-set with your target resolution.
In my iOS Project i am adding small PNG Images including alpha channel as overlay on a JPEG Picture. The result on my device in DEBUG mode is as expected, the tears are drawn correctly.
When i run the same code on Simulator or when i archive and export the App in RELEASE mode i get random artifacts in alpha channel.
The underlying cv::Mat all contain header infos and a valid data section. Even on green background the error is reproducible.
The behaviour seem to be totally random as from time to time no artifacts are drawn (image 3: right tear, image 4: left tear).
Ideas, anybody?
const char *cpath1 = [#"" cStringUsingEncoding:NSUTF8StringEncoding];//overlay image path , within #"" pass your image path which is in NSString
const char *cpath = [#"" cStringUsingEncoding:NSUTF8StringEncoding];//underlay imagepath
cv::Mat overlay = cv::imread(cpath1,-1);//-1 is for read .png images
cv::Mat underlay = cv::imread(cpath,-1);
//convert mat image in to RGB channel
cv::Mat overlayAlpha;
std::vector<Mat> channels1;
split(overlay, channels1);
channels1[3].copyTo(overlayAlpha);
cv::Mat underlayAlpha;
std::vector<Mat> channels2;
split(underlay, channels2);
channels2[3].copyTo(underlayAlpha);
overlayImage( &underlay, &overlay,cv::Point(10,10);
convert final image to RGB channel
cv::split(underlay,channels1);
std::swap(channels1[0],channels1[2]);// swap B and R channels.
cv::merge(channels1,underlay);//merge channels
MatToUIImage(background); //display your final image, it returns cv::Mat image
and overlay function is like below
overlay function referenced from : http://answers.opencv.org/question/73016/how-to-overlay-an-png-image-with-alpha-channel-to-another-png/
void overlayImage(Mat* src, Mat* overlay, const cv::Point& location){
for (int y = max(location.y, 0); y < src->rows; ++y)
{
int fY = y - location.y;
if (fY >= overlay->rows)
break;
for (int x = max(location.x, 0); x < src->cols; ++x)
{
int fX = x - location.x;
if (fX >= overlay->cols)
break;
double opacity = ((double)overlay->data[fY * overlay->step + fX * overlay->channels() + 3]) / 255;
for (int c = 0; opacity > 0 && c < src->channels(); ++c)
{
unsigned char overlayPx = overlay->data[fY * overlay->step + fX * overlay->channels() + c];
unsigned char srcPx = src->data[y * src->step + x * src->channels() + c];
src->data[y * src->step + src->channels() * x + c] = srcPx * (1. - opacity) + overlayPx * opacity;
}
}
}
}
I am using opencv to achieve object tracking. I read that YUV image is better option to use than RGB image. My problem is that I fail to understand about the YUV format although i spend much time read notes. Y is the brightness which i believe is calculated from the combination of R, G, B component.
My main problem is how can I access and manipulate the pixels in YUV image format. In RGB format its easy to access the component and therefore change it using simple operatin like
src.at<Vec3b>(j,i).val[0] = 0; for example
But this is not the case in YUV. I need help in accessing and changing the pixel values in YUV image. For example if pixel in RGB is red, then I want to only keep the corresponding pixel in YUV and the rest is removed. Please help me with this.
I would suggest operating on your image in HSV or LAB rather than RGB.
The raw image from the camera will be in YCbCr (sometimes called YUV, which I think is incorrect, but I may be wrong), and laid out in a way that resembles something like YUYV (repeating), so if you can convert directly from that to HSV, you will avoid additional copy and conversion operations which will save you some time. That may only matter to you if you're processing video or batches of images however.
Here's some C++ code for converting between YCbCr and RGB (one uses integer math, the other floating point):
Colour::bgr Colour::YCbCr::toBgrInt() const
{
int c0 = 22987;
int c1 = -11698;
int c2 = -5636;
int c3 = 29049;
int y = this->y;
int cb = this->cb - 128;
int cr = this->cr - 128;
int b = y + (((c3 * cb) + (1 << 13)) >> 14);
int g = y + (((c2 * cb + c1 * cr) + (1 << 13)) >> 14);
int r = y + (((c0 * cr) + (1 << 13)) >> 14);
if (r < 0)
r = 0;
else if (r > 255)
r = 255;
if (g < 0)
g = 0;
else if (g > 255)
g = 255;
if (b < 0)
b = 0;
else if (b > 255)
b = 255;
return Colour::bgr(b, g, r);
}
Colour::bgr Colour::YCbCr::toBgrFloat() const
{
float y = this->y;
float cb = this->cb;
float cr = this->cr;
int r = y + 1.40200 * (cr - 0x80);
int g = y - 0.34414 * (cb - 0x80) - 0.71414 * (cr - 0x80);
int b = y + 1.77200 * (cb - 0x80);
if (r < 0)
r = 0;
else if (r > 255)
r = 255;
if (g < 0)
g = 0;
else if (g > 255)
g = 255;
if (b < 0)
b = 0;
else if (b > 255)
b = 255;
return Colour::bgr(b, g, r);
}
And a conversion from BGR to HSV:
Colour::hsv Colour::bgr2hsv(bgr const& in)
{
Colour::hsv out;
int const hstep = 255 / 3; // Hue step size between red -> green -> blue
int min = in.r < in.g ? in.r : in.g;
min = min < in.b ? min : in.b;
int max = in.r > in.g ? in.r : in.g;
max = max > in.b ? max : in.b;
out.v = max; // v
int chroma = max - min;
if (max > 0)
{
out.s = 255 * chroma / max; // s
}
else
{
// r = g = b = 0 // s = 0, v is undefined
out.s = 0;
out.h = 0;
out.v = 0; // it's now undefined
return out;
}
if (chroma == 0)
{
out.h = 0;
return out;
}
const int chroma2 = chroma * 2;
int offset;
int diff;
if (in.r == max)
{
offset = 3 * hstep;
diff = in.g - in.b;
}
else if (in.g == max)
{
offset = hstep;
diff = in.b - in.r;
}
else
{
offset = 2 * hstep;
diff = in.r - in.g;
}
int h = offset + (diff * (hstep + 1)) / chroma2;
// Rotate such that red has hue 0
if (h >= 255)
h -= 255;
assert(h >= 0 && h < 256);
out.h = h;
return out;
Unfortunately I do not have code to do this in one step.
You can also use the built-in OpenCV functions for colour conversion.
cvtColor(img, img, CV_BGR2HSV);
Also the U and V components are calculated as linear combinations of RGB values. Then it means, that different intensities of red (R,0,0) are mapped to some (y*R + a,u*R + b, v*R + c), which again means that to detect "red" in YUV one can calculate if the distance of the pixel to that line determined by y,u,v,a,b,c (some of which are redundant) is close to zero. That's achievable with a single dot product. Then set the remaining pixels to the (0,128,128) in YUV space (I think that's R=0,G=0,B=0 in almost all varieties of YCrCb, YUV and such).
There are several YUV formats, but the common ones keep Y at the same resolution as the original image, but U and V are half size, and are saved as separate or interlaced planes/channels after the single channel Y image buffer.
This allows you to efficiently access Y as a 1-channel 8-bit greyscale image.
Access and manipulate pixels does not know the colorformat so the same code applies for color components Y U and V. If you need to access in RGB mode, best is probably calling cv::cvtColor for your region of interest first.
I can find many examples on how to do this in managed c++ but none for unmanaged.
I want to get all the pixel data as efficiently as possible, but some of the scan0 stuff I would need more info about so I can properly iterate through the pixel data and get each rgba value from it.
right now I have this:
Bitmap *b = new Bitmap(filename);
if(b == NULL)
{
return 0;
}
UINT w,h;
w = b->GetWidth();
h = b->GetHeight();
Rect *r = new Rect(0,0,w,h);
BitmapData *lockdat;
b->LockBits(r,ImageLockModeRead,PixelFormatDontCare,lockdat);
delete(r);
if(w == 0 && h == 0)
{
return 0;
}
Color c;
std::vector<GLubyte> pdata(w * h * 4,0.0);
for (unsigned int i = 0; i < h; i++) {
for (unsigned int j = 0; j < w; j++) {
b->GetPixel(j,i,&c);
pdata[i * 4 * w + j * 4 + 0] = (GLubyte) c.GetR();
pdata[i * 4 * w + j * 4 + 1] = (GLubyte) c.GetG();
pdata[i * 4 * w + j * 4 + 2] = (GLubyte) c.GetB();
pdata[i * 4 * w + j * 4 + 3] = (GLubyte) c.GetA();
}
}
delete(b);
return CreateTexture(pdata,w,h);
How do I use lockdat to do the equivalent of getpixel?
Thanks
lockdat->Scan0 is a pointer to the pixel data of the bitmap. Note that you really do care what pixel format you ask for, PixelFormatDontCare won't do. Because how you use the pointer is affected by the pixel format. PixelFormat32bppARGB is the easiest, one pixel will be the size of an int, 4 bytes representing alpha, red, green and blue. And the stride will be equal to the width of the bitmap. Making it likely that a simple memcpy() will get the job done. Beware the bitmaps are stored upside-down.
Bitmap *m_image = new Bitmap(...) // a 24-bit RGB bitmap
BitmapData bmData;
Rect rect(0, 0, m_image->GetWidth(), m_image->GetHeight());
m_image->LockBits(&rect , ImageLockModeRead , PixelFormat24bppRGB,&bmData );
memcpy(your_bytes_buffer, bmData.Scan0, min(bmData.Height * bmData.Stride, your_buffer_size));
m_image->UnlockBits(&bmData);