I'm getting a callback from a library that gives me a raw image which is 640*480*4 R8G8B8A8
I did copying it into another buffer then I try to skip the 4th buffer because my framework works only with R8G8B8
I tried to debug and see the bytes in the variable d, but the bytes are all garbage.
-(void)onRawImageBuffer:(NSData * _Nonnull)data withWidth:(int)width andHeight:(int)height andBytesPerPixel:(int)bytesPerPixel;
{
NSUInteger len = [data length];
memcpy(m_CopyData, [data bytes], len);
for(int i =0; i < 640*480*3;i++)
{
if ( i%4 == 3 )
continue;
else{
m_FrameData[i] = m_CopyData[i];
}
}
NSData* d = [NSData dataWithBytes:(const void *)m_FrameData length:640*480*3];
Input::SensorInput::getSingleton()->setVideoData( m_FrameData );
}
You are hard coding width and height;
Your loop must still go to width * height * 4;
You have not allocated m_FrameData;
You are not considering bytesPerPixel;
continue; still advance i by one, you must use two indices;
It's very inefficient, better copy RGB together, skip A, advance src pointer by 4 and dst pointer by 3.
Related
I am writing a longer program and I found myself needing to read a .bmp file into an array in a specific way so that the rest of the program can use it without extensive rewrites. I failed to find older answers that would resolve my problem, and I am pretty much at the beginner stages.
The image I am trying to read is used to create a text font, so I want to read it character by character into an array, where the pixels belonging to one character are added in order to a 2d bool (true if pixel is not black) array [character_id] [pixel_n]. The dimensions of characters are predetermined and known, and the file is cropped so that they all appear in a single row with no unaccounted margins.
This is the specific file I am trying to read, though here it might not show up as .bmp
As an example, shown here, I want to read the pixels in the order of the yellow line, then jump to another character. For clarity each character is 5px wide and 11px high, with 1px of margin on both sides horizontally.
Based on what I was able to find, I have written a function to do it, but I fail to make it work as intended, as far as I can tell even the pixel values are not being read correctly:
void readBMP(char* filename)
{
int i;
FILE* f = fopen(filename, "rb");
unsigned char info[54];
// read the 54-byte header
fread(info, sizeof(unsigned char), 54, f);
// extract image height and width from header
int width = *(int*)&info[18];
int height = *(int*)&info[22];
// number of pixels in total
int size = 3 * width * height;
unsigned char* data = new unsigned char[size];
// number of characters to read
int counter1 = size / ((font_width + 2) * font_height) / 3 ;
// read the rest of the data at once
fread(data, sizeof(unsigned char), size, f);
fclose(f);
//loop that goes from character to character
for(int i = 0; i < counter1; i++)
{
int tmp = 0;
//loop that reads one character into font_ref array
for(int j = 0; j < font_height; j++)
{
//loop for each row of a character
for(int k = 0; k < font_width; k++)
{
int w = static_cast<int>(data[3*(j*(font_width+2)*(counter1) + i*(font_width + 2) + 1 + k + j*font_width + j)-1]);
if( w != 0 )
font_ref [i][(tmp)] = 1;
else
font_ref [i][(tmp)] = 0;
tmp++;
}
}
}
}
(bool font_ref [150][font_width*font_height]; is the array where the font is being loaded and stored)
this code reads something, but the result is a seemingly random mess and I am unable to resolve that. Here is an example of lowercase alphabet printed using another function in the program, where white pixels represent true bools. I am aware that some libraries exist to work with graphical files, however in this program I wanted to possibly avoid that to learn more lower-level things, and the goal is rather limited and specific.
Thank you in advance for any help with the issue.
The main errors are in the offset computation for a pixel in the bitmap data:
int w = static_cast<int>(data[3*(j*(font_width+2)*(counter1) + i*(font_width + 2) + 1 + k + j*font_width + j)-1]);
j*(font_width+2)*(counter1) - This doesn't take into account that
although you say the file is cropped, there is extra black space to the right of the last character cell, so the true width must be used;
(as drescherjm and user3386109 mentioned) padding bytes are appended to the rows so that their length is a multiple of four bytes.
+ j*font_width + j)-1 - This part makes no sense - perhaps you tried to compensate the above errors.
This would be correct:
int w = data[j*(3*width+3&~3)+3*(i*(font_width+2)+1+k)];
I have input from captured camera frame as CMSampleBufferRef and I need to get the raw pixels preferably in C type uint8_t[].
I also need to find the color scheme of the input image.
I know how to convert CMSampleBufferRef to UIImage and then to NSData with png format but I dont know how to get the raw pixels from there. Perhaps I could get it already from CMSampleBufferRef/CIImage`?
This code shows the need and the missing bits.
Any thoughts where to start?
int convertCMSampleBufferToPixelArray (CMSampleBufferRef sampleBuffer)
{
// inputs
CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
CIImage *ciImage = [CIImage imageWithCVPixelBuffer:imageBuffer];
CIContext *imgContext = [CIContext new];
CGImageRef cgImage = [imgContext createCGImage:ciImage fromRect:ciImage.extent];
UIImage *uiImage = [UIImage imageWithCGImage:cgImage];
NSData *nsData = UIImagePNGRepresentation(uiImage);
// Need to fill this gap
uint8_t* data = XXXXXXXXXXXXXXXX;
ImageFormat format = XXXXXXXXXXXXXXXX; // one of: GRAY8, RGB_888, YV12, BGRA_8888, ARGB_8888
// sample showing expected data values
// this routine converts the image data to gray
//
int width = uiImage.size.width;
int height = uiImage.size.height;
const int size = width * height;
std::unique_ptr<uint8_t[]> new_data(new uint8_t[size]);
for (int i = 0; i < size; ++i) {
new_data[i] = uint8_t(data[i * 3] * 0.299f + data[i * 3 + 1] * 0.587f +
data[i * 3 + 2] * 0.114f + 0.5f);
}
return 1;
}
Some pointers you can use to search for more info. It's nicely documented and you shouldn't have an issue.
int convertCMSampleBufferToPixelArray (CMSampleBufferRef sampleBuffer) {
CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
if (imageBuffer == NULL) {
return -1;
}
// Get address of the image buffer
CVPixelBufferLockBaseAddress(imageBuffer, 0);
uint8_t* data = CVPixelBufferGetBaseAddress(imageBuffer);
// Get size
size_t width = CVPixelBufferGetWidth(imageBuffer);
size_t height = CVPixelBufferGetHeight(imageBuffer);
// Get bytes per row
size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);
// At `data` you have a bytesPerRow * height bytes of the image data
// To get pixel info you can call CVPixelBufferGetPixelFormatType, ...
// you can call CVImageBufferGetColorSpace and inspect it, ...
// When you're done, unlock the base address
CVPixelBufferUnlockBaseAddress(imageBuffer, 0);
return 0;
}
There're couple of things you should be aware of.
First one is that it can be planar. Check the CVPixelBufferIsPlanar, CVPixelBufferGetPlaneCount, CVPixelBufferGetBytesPerRowOfPlane, etc.
Second one is that you have to calculate pixel size based on CVPixelBufferGetPixelFormatType. Something like:
CVPixelBufferGetPixelFormatType(imageBuffer)
size_t pixelSize;
switch (pixelFormat) {
case kCVPixelFormatType_32BGRA:
case kCVPixelFormatType_32ARGB:
case kCVPixelFormatType_32ABGR:
case kCVPixelFormatType_32RGBA:
pixelSize = 4;
break;
// + other cases
}
Let's say that the buffer is not planar and:
CVPixelBufferGetWidth returns 200 (pixels)
Your pixelSize is 4 (calcuated bytes per row is 200 * 4 = 800)
CVPixelBufferGetBytesPerRow can return anything >= 800
In other words, the pointer you have is not a pointer to a contiguous buffer. If you need row data you have to do something like this:
uint8_t* data = CVPixelBufferGetBaseAddress(imageBuffer);
// Get size
size_t width = CVPixelBufferGetWidth(imageBuffer);
size_t height = CVPixelBufferGetHeight(imageBuffer);
size_t pixelSize = 4; // Let's pretend it's calculated pixel size
size_t realRowSize = width * pixelSize;
size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);
for (int row = 0 ; row < height ; row++) {
// bytesPerRow acts like an offset where the next row starts
// bytesPerRow can be >= realRowSize
uint8_t *rowData = data + row * bytesPerRow;
// realRowSize = how many bytes are available for this row
// copy them somewhere
}
You have to allocate a buffer and copy these row data there if you'd like to have contiguous buffer. How many bytes to allocate? CVPixelBufferGetDataSize.
I'm trying to convert a FreeType GlyphSlot Bitmap to Vulkan BGRA format.
void DrawText(const std::string &text) {
// WIDTH & HEIGHT == dst image dimensions
FT_GlyphSlot Slot = face->glyph;
buffer.resize(WIDTH*HEIGHT*4);
int dst_Pitch = WIDTH * 4;
for (auto c : text) {
FT_Error error = FT_Load_Char(face, c, FT_LOAD_RENDER);
if (error) {
printf("FreeType: Load Char Error\n");
continue;
}
auto char_width = Slot->bitmap.width;
auto char_height = Slot->bitmap.rows;
uint8_t* src = Slot->bitmap.buffer;
uint8_t* startOfLine = src;
for (int y = 0; y < char_height; ++y) {
src = startOfLine;
for (int x = 0; x < char_width; ++x) {
// y * dst_Pitch == Destination Image Row
// x * 4 == Destination Image Column
int dst = (y*dst_Pitch) + (x*4);
// Break if we have no more space to draw on our
// destination texture.
if (dst + 4 > buffer.size()) { break; }
auto value = *src;
src++;
buffer[dst] = 0xff; // +0 == B
buffer[dst+1] = 0xff; // +1 == G
buffer[dst+2] = 0xff; // +2 == R
buffer[dst+3] = value; // +3 == A
}
startOfLine += Slot->bitmap.pitch;
}
}
}
This is giving me garbled output. I'm not sure what I need to do to properly convert to Vulkan B8G8R8A8. I feel like moving from left to right in the buffer we write to our Vulkan texture is incorrect and maybe Vulkan is expecting I add the pixels into the buffer in a different way?
I understand this code will write each letter on top of one another, I will implement taking advantage of Slot->advance after I can properly draw at least a single letter.
One problem is that you resize buffer with every character (which will leave the previous data at the start of the newly allocated space) but when storing the data for the new character c you overwrite the start of the buffer since dst is 0. You probably want to set dst the buffer.size() from before the resize call.
int dst = /*previous buffer size*/;
The issue was due to the fact that I had VkImageCreateInfo tiling set to VK_IMAGE_TILING_OPTIMAL. After changing it to VK_IMAGE_TILING_LINEAR I received the correct output.
Taken straight from https://www.khronos.org/registry/vulkan/specs/1.1-extensions/man/html/VkImageTiling.html
VK_IMAGE_TILING_OPTIMAL specifies optimal tiling (texels are laid out
in an implementation-dependent arrangement, for more optimal memory
access).
VK_IMAGE_TILING_LINEAR specifies linear tiling (texels are laid out in
memory in row-major order, possibly with some padding on each row).
While I may not be rendering garbage now, my letters are still backwards and seemingly drawing from right to left instead of left to right.
You can see the green 'the' in the top right corner.
I'm using dcmtk library to modify the pixel data of a multi frame compressed dicom image. So, to do that, at one stage in an for loop I take the pixel data of each decompressed frame and modify them according my wish and try to concatenate each modify pixel data in a big memory buffer frame by frame. This core process of for loop is as below.
The problem is after the first iteration it gives memory at the line of the code where I call the function getUncompressedFrame. I think it's happening because of the line memcpy(fullBuffer+(i*sizeF),newBuffer,sizeF);, as when I remove that line there's no error at that time and the whole for loop works absolutely fine.
Could you please say me if I'm making a mistake in working with memcpy? Thanks.
Uint32 sizeF=828072;// I just wrote it to show what is the data type.
Uint8 * fullBuffer = new Uint8(int(sizeF*numOfFrames));//The big memory buffer
for(int i=0;i<numOfFrames;i++)
{
Uint8 * buffer = new Uint8[int(sizeF)];//Buffer for each frame
Uint8 * newBuffer = new Uint8[int(sizeF)];//Buffer in which the modified frame data is stored
DcmFileCache * cache=NULL;
OFCondition cond=element->getUncompressedFrame(dataset,i,startFragment,buffer,sizeF,decompressedColorModel,cache);
//I get the uncompressed individual frame pixel data
if(buffer != NULL)
{
for(unsigned long y = 0; y < rows; y++)
{
for(unsigned long x = 0; x < cols; x++)
{
if(planarConfiguration==0)
{
if(x>xmin && x<xmax && y>ymin && y<ymax)
{
index=(x + y + y*(cols-1))*samplePerPixel;
if(index<sizeF-2)
{
newBuffer[index] = 0;
newBuffer[index + 1] = 0;
newBuffer[index +2] = 0;
}
}
else
{
index=(x + y + y*(cols-1))*samplePerPixel;
if(index<sizeF-2)
{
newBuffer[index] = buffer[index];
newBuffer[index + 1] = buffer[index + 1];
newBuffer[index + 2] = buffer[index + 2];
}
}
}
}
}
memcpy(fullBuffer+(i*sizeF),newBuffer,sizeF);
//concatenate the modified frame by frame pixel data
}
Change the declaration of fullBuffer to this:
Uint8 * fullBuffer = new Uint8[int(sizeF*numOfFrames)];
Your code didn't allocate an array, it allocated a single Uint8 with the value int(sizeF*numOfFrames).
Uint8 * fullBuffer = new Uint8(int(sizeF*numOfFrames));
This allocates a single byte, giving it an initial value of sizeF*numOfFrames (after truncating it first to int and then to Uint8). You want an array, and you don't want to truncate the size to int:
Uint8 * fullBuffer = new Uint8[sizeF*numOfFrames];
^ ^
or, to fix the likely memory leaks in your code:
std::vector<Uint8> fullBuffer(sizeF*numOfFrames);
If the method getUncompressedFrame is doing an inner memcpy to cache, then it makes sense why, as you are passing a null pointer as argument for the cache, with no memory allocated.
I am working with OpenCV and Qt, Opencv use BGR while Qt uses RGB , so I have to swap those 2 bytes for very big images.
There is a better way of doing the following?
I can not think of anything faster but looks so simple and lame...
int width = iplImage->width;
int height = iplImage->height;
uchar *iplImagePtr = (uchar *) iplImage->imageData;
uchar buf;
int limit = height * width;
for (int y = 0; y < limit; ++y) {
buf = iplImagePtr[2];
iplImagePtr[2] = iplImagePtr[0];
iplImagePtr[0] = buf;
iplImagePtr += 3;
}
QImage img((uchar *) iplImage->imageData, width, height,
QImage::Format_RGB888);
We are currently dealing with this issue in a Qt application. We've found that the Intel Performance Primitives to be be fastest way to do this. They have extremely optimized code. In the html help files at Intel ippiSwapChannels Documentation they have an example of exactly what you are looking for.
There are couple of downsides
Is the size of the library, but you can link static link just the library routines you need.
Running on AMD cpus. Intel libs run VERY slow by default on AMD. Check out www.agner.org/optimize/asmlib.zip for details on how do a work around.
I think this looks absolutely fine. That the code is simple is not something negative. If you want to make it shorter you could use std::swap:
std::swap(iplImagePtr[0], iplImagePtr[2]);
You could also do the following:
uchar* end = iplImagePtr + height * width * 3;
for ( ; iplImagePtr != end; iplImagePtr += 3) {
std::swap(iplImagePtr[0], iplImagePtr[2]);
}
There's cvConvertImage to do the whole thing in one line, but I doubt it's any faster either.
Couldn't you use one of the following methods ?
void QImage::invertPixels ( InvertMode mode = InvertRgb )
or
QImage QImage::rgbSwapped () const
Hope this helps a bit !
I would be inclined to do something like the following, working on the basis of that RGB data being in three byte blocks.
int i = 0;
int limit = (width * height); // / 3;
while(i != limit)
{
buf = iplImagePtr[i]; // should be blue colour byte
iplImagePtr[i] = iplImagaePtr[i + 2]; // save the red colour byte in the blue space
iplImagePtr[i + 2] = buf; // save the blue color byte into what was the red slot
// i++;
i += 3;
}
I doubt it is any 'faster' but at end of day, you just have to go through the entire image, pixel by pixel.
You could always do this:
int width = iplImage->width;
int height = iplImage->height;
uchar *start = (uchar *) iplImage->imageData;
uchar *end = start + width * height;
for (uchar *p = start ; p < end ; p += 3)
{
uchar buf = *p;
*p = *(p+2);
*(p+2) = buf;
}
but a decent compiler would do this anyway.
Your biggest overhead in these sorts of operations is going to be memory bandwidth.
If you're using Windows then you can probably do this conversion using the BitBlt and two appropriately set up DIBs. If you're really lucky then this could be done in the graphics hardware.
I hate to ruin anyone's day, but if you don't want to go the IPP route (see photo_tom) or pull in an optimized library, you might get better performance from the following (modifying Andreas answer):
uchar *iplImagePtr = (uchar *) iplImage->imageData;
uchar buf;
size_t limit = height * width;
for (size_t y = 0; y < limit; ++y) {
std::swap(iplImagePtr[y * 3], iplImagePtr[y * 3 + 2]);
}
Now hold on, folks, I hear you yelling "but all those extra multiplies and adds!" The thing is, this form of the loop is far easier for a compiler to optimize, especially if they get smart enough to multithread this sort of algorithm, because each pass through the loop is independent of those before or after. In the other form, the value of iplImagePtr was dependent on the value in previous pass. In this form, it is constant throughout the whole loop; only y changes, and that is in a very, very common "count from 0 to N-1" loop construct, so it's easier for an optimizer to digest.
Or maybe it doesn't make a difference these days because optimizers are insanely smart (are they?). I wonder what a benchmark would say...
P.S. If you actually benchmark this, I'd also like to see how well the following performs:
uchar *iplImagePtr = (uchar *) iplImage->imageData;
uchar buf;
size_t limit = height * width;
for (size_t y = 0; y < limit; ++y) {
uchar *pixel = iplImagePtr + y * 3;
std::swap(pix[0], pix[2]);
}
Again, pixel is defined in the loop to limit its scope and keep the optimizer from thinking there's a cycle-to-cycle dependency. If the compiler increments and decrements the stack pointer each time through the loop to "create" and "destroy" pixel, well, it's stupid and I'll apologize for wasting your time.
cvCvtColor(iplImage, iplImage, CV_BGR2RGB);