Read From Media Buffer - Pointer Arithmetic C++ Syntax - c++

This may well have come up before but the following code is taken from an MSDN example I am modifying. I want to know how I can iterate through the contents of the buffer which contains data about a bitmap and print out the colors. Each pixel is 4 bytes of data so I am assuming the R G B values account for 3 of these bytes, and possibly A is the 4th.
What is the correct C++ syntax for the pointer arithmetic required (ideally inside a loop) that will store the value pointed to during that iteration in to a local variable that I can use, eg. print to the console.
Many thanks
PS. Is this safe? Or is there a safer way to read the contents of an IMFMediaBuffer? I could not find an alternative.
Here is the code:
hr = pSample->ConvertToContiguousBuffer(&pBuffer); // this is the BitmapData
// Converts a sample with multiple buffers into a sample with a single IMFMediaBuffer which we Lock in memory next...
// IMFMediaBuffer represents a block of memory that contains media data
hr = pBuffer->Lock(&pBitmapData, NULL, &cbBitmapData); // pBuffer is IMFMediaBuffer
/* Lock method gives the caller access to the memory in the buffer, for reading or writing:
pBitmapData - receives a pointer to start of buffer
NULL - receives the maximum amount of data that can be written to the buffer. This parameter can be NULL.
cbBitmapData - receives the length of the valid data in the buffer, in bytes. This parameter can be NULL.
*/

I solved the problem myself and thought it best to add the answer here so that it formats correctly and maybe others will benefit from it. Basically in this situation we use 32 bits for the image data and what is great is that we are reading raw from memory so there is not yet a Bitmap header to skip because this is just raw color information.
NOTE: Across these 4 bytes we have (from bit 0 - 31) B G R A, which we can verify by using my code:
int x = 0;
while(x < cbBitmapData){
Console::Write("B: {0}", (*(pBitmapData + x++)));
Console::Write("\tG: {0}", (*(pBitmapData + x++)));
Console::Write("\tR: {0}", (*(pBitmapData + x++)));
Console::Write("\tA: {0}\n", (*(pBitmapData + x++)));
}
From the output you will see that the A value is 0 for each pixel because there is no concept of transparency or depth here, which is what we expect.
Also to verify that all we have in the buffer is raw image data and no other data I used this calculation which you may also find of use:
Console::Write("no of pixels in buffer: {0} \nexpected no of pixels based on dimensions:{1}", (cbBitmapData/4), (m_format.imageWidthPels * m_format.imageHeightPels) );
Where we divide the value of cbBitmapData by 4 because it is a count of the bytes, and as aforementioned for each pixel we have a width of 4 bytes (32-bit DWORDS in actual fact because the length of a byte is not always strictly uniform across hardware apparently!?). We compare this to the image width multiplied by its height. They are equal and thus we have just pixel color information in the buffer.
Hope this helps someone.

Related

How to get a pixel array from TBitmap?

In a camera application bitmap pixel arrays are retrieved from a streaming camera.
The pixel arrays are captured by writing them to a named pipe, where on the other end of the pipe, ffmpeg retrieves them and creates an AVI file.
I will need to create one custom frame (with custom text on), and pipe its pixels as the first frame in the resulting movie.
The question is how can I use a TBitmap (for convenience) to
Create a X by Y monochrome (8 bit) bitmap from scratch, with
custom text on. I want the background to be white, and the text to
be black. (Mostly figured this step out, see below.)
Retrieve the pixel array that I can send/write to the pipe
Step 1: The following code creates a TBitmap and writes text on it:
int w = 658;
int h = 492;
TBitmap* bm = new TBitmap();
bm->Width = w;
bm->Height = h;
bm->HandleType = bmDIB;
bm->PixelFormat = pf8bit;
bm->Canvas->Font->Name = "Tahoma";
bm->Canvas->Font->Size = 8;
int textY = 10;
string info("some Text");
bm->Canvas->TextOut(10, textY, info.c_str());
The above basically concludes step 1.
The writing/piping code expects a byte array with the bitmaps pixels; e.g.
unsigned long numWritten;
WriteFile(mPipeHandle, pImage, size, &numWritten, NULL);
where pImage is a pointer to a unsigned char buffer (the bitmaps pixels), and the size is the length of this buffer.
Update:
Using the generated TBitmap and a TMemoryStream for transferring data to the ffmpeg pipeline does not generate the proper result. I get a distorted image with 3 diagonal lines on it.
The buffersize for the camera frame buffers that I receive are are exactly 323736, which is equal to the number of pixels in the image, i.e. 658x492.
NOTE I have concluded that this 'bitmap' is not padded. 658 is not divisible by four.
The buffersize I get after dumping my generated bitmap to a memory stream, however, has the size 325798, which is 2062 bytes larger than it is supposed to be. As #Spektre pointed out below, this discrepancy may be padding?
Using the following code for getting the pixel array;
ByteBuffer CustomBitmap::getPixArray()
{
// --- Local variables --- //
unsigned int iInfoHeaderSize=0;
unsigned int iImageSize=0;
BITMAPINFO *pBitmapInfoHeader;
unsigned char *pBitmapImageBits;
// First we call GetDIBSizes() to determine the amount of
// memory that must be allocated before calling GetDIB()
// NB: GetDIBSizes() is a part of the VCL.
GetDIBSizes(mTheBitmap->Handle,
iInfoHeaderSize,
iImageSize);
// Next we allocate memory according to the information
// returned by GetDIBSizes()
pBitmapInfoHeader = new BITMAPINFO[iInfoHeaderSize];
pBitmapImageBits = new unsigned char[iImageSize];
// Call GetDIB() to convert a device dependent bitmap into a
// Device Independent Bitmap (a DIB).
// NB: GetDIB() is a part of the VCL.
GetDIB(mTheBitmap->Handle,
mTheBitmap->Palette,
pBitmapInfoHeader,
pBitmapImageBits);
delete []pBitmapInfoHeader;
ByteBuffer buf;
buf.buffer = pBitmapImageBits;
buf.size = iImageSize;
return buf;
}
So final challenge seem to be to get a bytearray that has the same size as the ones coming from the camera. How to find and remove the padding bytes from the TBitmap code??
TBitmap has a PixelFormat property to set the bit depth.
TBitmap has a HandleType property to control whether a DDB or a DIB is created. DIB is the default.
Since you are passing BMPs around between different systems, you really should be using DIBs instead of DDBs, to avoid any corruption/misinterpretation of the pixel data.
Also, this line of code:
Image1->Picture->Bitmap->Handle = bm->Handle;
Should be changed to this instead:
Image1->Picture->Bitmap->Assign(bm);
// or:
// Image1->Picture->Bitmap = bm;
Or this:
Image1->Picture->Assign(bm);
Either way, don't forget to delete bm; afterwards, since the TPicture makes a copy of the input TBitmap, it does not take ownership.
To get the BMP data as a buffer of bytes, you can use the TBitmap::SaveToStream() method, saving to a TMemoryStream. Or, if you just want the pixel data, not the complete BMP data (ie, without BMP headers - see Bitmap Storage), you can use the Win32 GetDiBits() function, which outputs the pixels in DIB format. You can't obtain a byte buffer of the pixels for a DDB, since they depend on the device they are rendered to. DDBs are only usable in-memory in conjunction with HDCs, you can't pass them around. But you can convert a DIB to a DDB once you have a final device to render it to.
In other words, get the pixels from the camera, save them to a DIB, pass that around as needed (ie, over the pipe), and then do whatever you need with it - save to a file, convert to DDB to render onscreen, etc.
This is just an addon to existing answer (with additional info after the OP edit)
Bitmap file-format has align bytes on each row (so there usually are some bytes at the end of each line that are not pixels) up to some ByteLength (present in bmp header). Those create the skew and diagonal like lines. In your case the size discrepancy is 4 bytes per row:
(xs + align)*ys + header = size
(658+ 4)*492 + 94 = 325798
but beware the align size depends on image width and bmp header ...
Try this instead:
// create bmp
Graphics::TBitmap *bmp=new Graphics::TBitmap;
// bmp->Assign(???); // a) copy image from ???
bmp->SetSize(658,492); // b) in case you use Assign do not change resolution
bmp->HandleType=bmDIB;
bmp->PixelFormat=pf8bit;
// bmp->Canvas->Draw(0,0,???); // b) copy image from ???
// here render your text using
bmp->Canvas->Brush->Style=bsSolid;
bmp->Canvas->Brush->Color=clWhite;
bmp->Canvas->Font->Color=clBlack;
bmp->Canvas->Font->Name = "Tahoma";
bmp->Canvas->Font->Size = 8;
bmp->Canvas->TextOutA(5,5,"Text");
// Byte data
for (int y=0;y<bmp->Height;y++)
{
BYTE *p=(BYTE*)bmp->ScanLine[y]; // pf8bit -> BYTE*
// here send/write/store ... bmp->Width bytes from p[]
}
// Canvas->Draw(0,0,bmp); // just renfder it on Form
delete bmp; bmp=NULL;
mixing GDI winapi calls for pixel array access (bitblt etc...) with VCL bmDIB bitmap might cause problems and resource leaks (hence the error on exit) and its also slower then usage of ScanLine[] (if coded right) so I strongly advice to use native VCL functions (as I did in above example) instead of the GDI/winapi calls where you can.
for more info see:
#4. GDI Bitmap
Delphi / C++ builder Windows 10 1709 bitmap operations extremely slow
Draw tbitmap with scale and alpha channel faster
Also you mention your image source is camera. If you use pf8bit it mean its palette indexed color which is relatively slow and ugly if native GDI algo is used (to convert from true/hi color camera image) for better transform see:
Effective gif/image color quantization?
simple dithering

reinterpret_cast and use with CV_MAT_ELEM

i want to put all the data of an 8 bit input colorimage (Inputfile is a.bmp file)
in a new 16 bit Mat Array.
I do this because i need to sum up several image patches and then build the meanvalue for each pixel.
Mat img = imread(Inputfile); //there now should be values between 0 and 255
Addressing the blue value for example like follows
uchar* ptr = img.data + img.step*row;
cout << ptr[n*col] << endl;
only brings up single Letters and no values.
cout << static_cast<short>(ptr[n*col]) << endl;
Typecasting to short or bigger brings up the correct values. but a cast to unsigned char (which is the correct datatype in my opinion) brings up the same Letters than without any typecast.
Short has 2 Bytes as i know, but a color .bmp should only have 1 Byte color information per channel. As i need to sum up in worst case 81 (smaller 128=7bit) pixel values, i thought the short as a target value would be great.
Any help concerning the right way to get simple access to the 8 bit values and use them in 16bit arrays would be great.
Thank you.
The cast works correct, but if you send an unsigned char into the output stream, it will be interpreted as a character and printed as a character.
Also note that OpenCV already has functionality to convert a matrix to a different datatype. You can even read your image into a matrix of the preferred datatype:
cv::Mat3s img = cv::imread(...);
And it is disregarded to use the data pointer. Read OpenCV documentation on how to access single pixels or rows in a clean fashion (iterators, operator(), operator[]...).

C++ memcpy and happy access violation

For some reason i can't figure i am getting access violation.
memcpy_s (buffer, bytes_per_line * height, image, bytes_per_line * height);
This is whole function:
int Flip_Bitmap(UCHAR *image, int bytes_per_line, int height)
{
// this function is used to flip bottom-up .BMP images
UCHAR *buffer; // used to perform the image processing
int index; // looping index
// allocate the temporary buffer
if (!(buffer = (UCHAR *) malloc (bytes_per_line * height)))
return(0);
// copy image to work area
//memcpy(buffer, image, bytes_per_line * height);
memcpy_s (buffer, bytes_per_line * height, image, bytes_per_line * height);
// flip vertically
for (index = 0; index < height; index++)
memcpy(&image[((height - 1) - index) * bytes_per_line], &buffer[index * bytes_per_line], bytes_per_line);
// release the memory
free(buffer);
// return success
return(1);
} // end Flip_Bitmap
Whole code:
http://pastebin.com/udRqgCfU
To run this you'll need 24-bit bitmap, in your source directory.
This is a part of a larger code, i am trying to make Load_Bitmap_File function to work...
So, any ideas?
You're getting an access violation because a lot of image programs don't set biSizeImage properly. The image you're using probably has biSizeImage set to 0, so you're not allocating any memory for the image data (in reality, you're probably allocating 4-16 bytes, since most malloc implementations will return a non-NULL value even when the requested allocation size is 0). So, when you go to copy the data, you're reading past the ends of that array, which results in the access violation.
Ignore the biSizeImage parameter and compute the image size yourself. Keep in mind that the size of each scan line must be a multiple of 4 bytes, so you need to round up:
// Pseudocode
#define ROUNDUP(value, power_of_2) (((value) + (power_of_2) - 1) & (~((power_of_2) - 1)))
bytes_per_line = ROUNDUP(width * bits_per_pixel/8, 4)
image_size = bytes_per_line * height;
Then just use the same image size for reading in the image data and for flipping it.
As the comments have said, the image data is not necessarily width*height*bytes_per_pixel
Memory access is generally faster on 32bit boundaries and when dealing with images speed generally matters. Because of this the rows of an image are often shifted to start on a 4byte (32bit) boundary
If the image pixels are 32bit (ie RGBA) this isn't a problem but if you have 3bytes per pixel (24bit colour) then for certain image widths, where the number of columns * 3 isn't a multiple of 4, then extra blank bytes will be inserted at the edn of each row.
The image format probably has a "stride" width or elemsize value to tell you this.
You allocate bitmap->bitmapinfoheader.biSizeImage for image but proceed to copy bitmap->bitmapinfoheader.biWidth * (bitmap->bitmapinfoheader.biBitCount / 8) * bitmap->bitmapinfoheader.biHeight bytes of data. I bet the two numbers aren't the same.

C++ Dereferencing char-Pointer (image array) is very slow

I have some trouble getting fast access to an unsigned character array.
I want to actually copy a BGRABGRA....BGRABGRA.... linewise coded image array to the OpenCV-version which uses three layers. The code below works fine but is really slow (around 0.5 seconds for a 640*480 image). I pointed out that the dereferencing operator * makes it slow. Do you have any plan how to fix this? (Hint: BYTE is an unsigned char)
// run thorugh all pixels and copy image data
for (int y = 0; y<imHeight; y++){
BYTE* pLine= vrIm->mp_buffer + y * vrIm->m_pitch;
for (int x = 0; x<imWidth; x++){
BYTE* b= pLine++; // fast pointer operation
BYTE* g= pLine++;
BYTE* r= pLine++;
BYTE* a= pLine++; // (alpha)
BYTE bc = *b; // this is really slow!
BYTE gc = *g; // this is really slow!
BYTE rc = *r; // this is really slow!
}
}
Thanks!
Shouldn't be - there is no way that is taking 0.5sec for a 640x480 unless you are doing this on a 8086. Is there some other code you aren't showing? The destination memory doesn't currently go anywhere
ps take a look at cvCvtColor() it uses optimized SSE2/SIMD instructions to do this
What hardware is the memory you're reading located on? Perhaps that device has limited bandwidth to the memory it uses or just has slow RAM. If the memory is shared by many devices there may also be bottle necks on it's access. Try reading the entire screen(?) to local memory using memcpy(), performing your operations on it in local RAM, then writing it back using memcpy(). This will reduce the number of times you must negotiate access to it from 640*480 to 1.

GDI+ gif speed problem

I am using C++ GDI+ to open a gif
however I find the frame interval is really strange.
It is different from played it by window's pic viewer.
The code I written is as follow.
pMultiPageImg = new Bitmap(XXXXX);
int size = m_pMultiPageImg->GetPropertyItemSize(PropertyTagFrameDelay);
m_pTimeDelays = (PropertyItem*) malloc (size);
m_pMultiPageImg->GetPropertyItem(PropertyTagFrameDelay, size, m_pTimeDelays);
int frameSize = m_pMultiPageImg->GetFrameDimensionsCount();();
// the interal of frame FrameNumber:
long lPause = ((long*)m_pTimeDelays->value)[FrameNumber] * 10;
however I found some frame the lPause <= 0.
What does this mean?
And are code I listed right for get the interval?
Many thanks!
The frame duration field in the gif header is only two bytes long (interpreted as 100ths of a second - allowing values from 0 to 32.768 seconds).
You seem to be interpreting it as long, which is probably 4 bytes on your platform so you will be reading another field along with the duration. It is hard to tell from the code you provide, but I think this is the problem.
Frame delays should not be negative numbers. I think the error comes in during the array type conversion or "FrameNumber" goes out of bounds.
GetPropertyItemSize(PropertyTagFrameDelay) returns a native byte array. It'll be safer to convert it to an Int32 array instead of a "long" array. "long" is always 4 bytes long under 32-bit systems, but could be 8 bytes under some 64-bit systems.
m_pMultiPageImg->GetFrameDimensionsCount() returns the number of frame dimensions in the image, not the number of frames. The dimension of the first frame (master image) is usually used in order to get the frame count.
In your case, the code looks like
int count = m_pMultiPageImg->GetFrameDimensionsCount();
GUID* dimensionIDs = new GUID[count];
m_pMultiPageImg->GetFrameDimensionsList(dimensionIDs, count);
int frameCount = m_pMultiPageImg->GetFrameCount(&m_pDimensionIDs[0]);
Hope this helps.