I'm looking for a way to convert a screen capture to a char* in an efficient way.
I thought about capturing the screen using a bitmap object, then converting the bitmap object to an array of bytes and then to a char*, but I couldn't find any way to convert then bitmap to an array of bytes.
I also thought about iterating every pixel in the screen capture and saving the RGB values in an array, but that creates an array that is too big for me.
The end goal is to return a char* object in the fastest way that doesn't return an object that is too large, and that char* contains the data of the screen captured.
You don't given enough info/code to understand what you're really trying to achieve by this. E.g. how are you planning to represent the colour coefficients? Anyway, a way to go would be to use the opencv library, e.g.:
cv::Mat mat = cv::imread("PATH/TO/YOUR/FILE.bmp");
char table[mat.rows * mat.cols * 3]; // assuming that we have 3 colour channels
unsigned int index = 0;
cv::MatIterator_<uchar> it, end;
for( it = mat.begin<uchar>(), end = mat.end<uchar>(); it != end; ++it)
table[index++] = *it;
Related
I'm using the WinAPI to grab a bitmap of your monitor and then add that data to a texture. The problem however is that the GetDIBits function writes its color-data (lpPixels parameter) in BGR-format as opposed to RGB. Thus, assigning this data to a texture, which uses a RGB-formatted byte array, makes the order wrong. One can easily just make a for/while loop and just iterate through the arrays and fix the order, however, this process is extremely slow. You could also just directly assign the BGR-array to the RGB-array but this results in a texture which has the wrong RGB-order! On the other hand, the second method is extremely fast, which is exactly what I am looking for.
I suppose there might be ways to use bitmasks within the WinAPIs BITMAPINFO struct, but I do not know how or even if that would be possible. There might even be a variable of some sort which determines the color-order used by the WinAPI, either way, I haven't been able to find such a variable. Hence, my question: How do I quickly assign a BGR-formatted array to a RGB-formatted one?
Slow way of converting BGR to RGB:
for (size_t i = 0; i < myWidth * myHeight; i++)
{
size_t tempIndex = (i * 4);
// The colors are in BGR-format (windows format)
// Hence, the order is reversed
rgbPixels[tempIndex] = bgrPixels[tempIndex + 2]; // r
rgbPixels[tempIndex + 1] = bgrPixels[tempIndex + 1]; // g
rgbPixels[tempIndex + 2] = bgrPixels[tempIndex]; // b
rgbPixels[tempIndex + 3] = 255; // a (always 255)
}
In a camera application bitmap pixel arrays are retrieved from a streaming camera.
The pixel arrays are captured by writing them to a named pipe, where on the other end of the pipe, ffmpeg retrieves them and creates an AVI file.
I will need to create one custom frame (with custom text on), and pipe its pixels as the first frame in the resulting movie.
The question is how can I use a TBitmap (for convenience) to
Create a X by Y monochrome (8 bit) bitmap from scratch, with
custom text on. I want the background to be white, and the text to
be black. (Mostly figured this step out, see below.)
Retrieve the pixel array that I can send/write to the pipe
Step 1: The following code creates a TBitmap and writes text on it:
int w = 658;
int h = 492;
TBitmap* bm = new TBitmap();
bm->Width = w;
bm->Height = h;
bm->HandleType = bmDIB;
bm->PixelFormat = pf8bit;
bm->Canvas->Font->Name = "Tahoma";
bm->Canvas->Font->Size = 8;
int textY = 10;
string info("some Text");
bm->Canvas->TextOut(10, textY, info.c_str());
The above basically concludes step 1.
The writing/piping code expects a byte array with the bitmaps pixels; e.g.
unsigned long numWritten;
WriteFile(mPipeHandle, pImage, size, &numWritten, NULL);
where pImage is a pointer to a unsigned char buffer (the bitmaps pixels), and the size is the length of this buffer.
Update:
Using the generated TBitmap and a TMemoryStream for transferring data to the ffmpeg pipeline does not generate the proper result. I get a distorted image with 3 diagonal lines on it.
The buffersize for the camera frame buffers that I receive are are exactly 323736, which is equal to the number of pixels in the image, i.e. 658x492.
NOTE I have concluded that this 'bitmap' is not padded. 658 is not divisible by four.
The buffersize I get after dumping my generated bitmap to a memory stream, however, has the size 325798, which is 2062 bytes larger than it is supposed to be. As #Spektre pointed out below, this discrepancy may be padding?
Using the following code for getting the pixel array;
ByteBuffer CustomBitmap::getPixArray()
{
// --- Local variables --- //
unsigned int iInfoHeaderSize=0;
unsigned int iImageSize=0;
BITMAPINFO *pBitmapInfoHeader;
unsigned char *pBitmapImageBits;
// First we call GetDIBSizes() to determine the amount of
// memory that must be allocated before calling GetDIB()
// NB: GetDIBSizes() is a part of the VCL.
GetDIBSizes(mTheBitmap->Handle,
iInfoHeaderSize,
iImageSize);
// Next we allocate memory according to the information
// returned by GetDIBSizes()
pBitmapInfoHeader = new BITMAPINFO[iInfoHeaderSize];
pBitmapImageBits = new unsigned char[iImageSize];
// Call GetDIB() to convert a device dependent bitmap into a
// Device Independent Bitmap (a DIB).
// NB: GetDIB() is a part of the VCL.
GetDIB(mTheBitmap->Handle,
mTheBitmap->Palette,
pBitmapInfoHeader,
pBitmapImageBits);
delete []pBitmapInfoHeader;
ByteBuffer buf;
buf.buffer = pBitmapImageBits;
buf.size = iImageSize;
return buf;
}
So final challenge seem to be to get a bytearray that has the same size as the ones coming from the camera. How to find and remove the padding bytes from the TBitmap code??
TBitmap has a PixelFormat property to set the bit depth.
TBitmap has a HandleType property to control whether a DDB or a DIB is created. DIB is the default.
Since you are passing BMPs around between different systems, you really should be using DIBs instead of DDBs, to avoid any corruption/misinterpretation of the pixel data.
Also, this line of code:
Image1->Picture->Bitmap->Handle = bm->Handle;
Should be changed to this instead:
Image1->Picture->Bitmap->Assign(bm);
// or:
// Image1->Picture->Bitmap = bm;
Or this:
Image1->Picture->Assign(bm);
Either way, don't forget to delete bm; afterwards, since the TPicture makes a copy of the input TBitmap, it does not take ownership.
To get the BMP data as a buffer of bytes, you can use the TBitmap::SaveToStream() method, saving to a TMemoryStream. Or, if you just want the pixel data, not the complete BMP data (ie, without BMP headers - see Bitmap Storage), you can use the Win32 GetDiBits() function, which outputs the pixels in DIB format. You can't obtain a byte buffer of the pixels for a DDB, since they depend on the device they are rendered to. DDBs are only usable in-memory in conjunction with HDCs, you can't pass them around. But you can convert a DIB to a DDB once you have a final device to render it to.
In other words, get the pixels from the camera, save them to a DIB, pass that around as needed (ie, over the pipe), and then do whatever you need with it - save to a file, convert to DDB to render onscreen, etc.
This is just an addon to existing answer (with additional info after the OP edit)
Bitmap file-format has align bytes on each row (so there usually are some bytes at the end of each line that are not pixels) up to some ByteLength (present in bmp header). Those create the skew and diagonal like lines. In your case the size discrepancy is 4 bytes per row:
(xs + align)*ys + header = size
(658+ 4)*492 + 94 = 325798
but beware the align size depends on image width and bmp header ...
Try this instead:
// create bmp
Graphics::TBitmap *bmp=new Graphics::TBitmap;
// bmp->Assign(???); // a) copy image from ???
bmp->SetSize(658,492); // b) in case you use Assign do not change resolution
bmp->HandleType=bmDIB;
bmp->PixelFormat=pf8bit;
// bmp->Canvas->Draw(0,0,???); // b) copy image from ???
// here render your text using
bmp->Canvas->Brush->Style=bsSolid;
bmp->Canvas->Brush->Color=clWhite;
bmp->Canvas->Font->Color=clBlack;
bmp->Canvas->Font->Name = "Tahoma";
bmp->Canvas->Font->Size = 8;
bmp->Canvas->TextOutA(5,5,"Text");
// Byte data
for (int y=0;y<bmp->Height;y++)
{
BYTE *p=(BYTE*)bmp->ScanLine[y]; // pf8bit -> BYTE*
// here send/write/store ... bmp->Width bytes from p[]
}
// Canvas->Draw(0,0,bmp); // just renfder it on Form
delete bmp; bmp=NULL;
mixing GDI winapi calls for pixel array access (bitblt etc...) with VCL bmDIB bitmap might cause problems and resource leaks (hence the error on exit) and its also slower then usage of ScanLine[] (if coded right) so I strongly advice to use native VCL functions (as I did in above example) instead of the GDI/winapi calls where you can.
for more info see:
#4. GDI Bitmap
Delphi / C++ builder Windows 10 1709 bitmap operations extremely slow
Draw tbitmap with scale and alpha channel faster
Also you mention your image source is camera. If you use pf8bit it mean its palette indexed color which is relatively slow and ugly if native GDI algo is used (to convert from true/hi color camera image) for better transform see:
Effective gif/image color quantization?
simple dithering
I am creating a program that allows you to view fractals like the Mandelbrot or Julia set. I would like to render them as quickly as possible. I would love a way to put an array of uint8_t pixel values onto the screen. The array is formatted like this...
{r0,g0,b0,r1,g1,b1,...}
(A one dimensional array or RGB color values)
I know I have the proper data because before I just set individual points and it worked...
for(int i = 0;i < height * width;++i) {
//setStroke and point are functions that I made that together just draw a colored point
r.setStroke(data[i*3],data[i*3+1],data[i*3+2]);
r.point(i % r.window.w,i / r.window.w);
}
This is a pretty slow operation especially if the screen is big (which I would like it to be)
Is there any faster way to just put all the data onto the screen.
I tried doing something like this
void* pixels;
int pitch;
SDL_Texture* img = SDL_CreateTexture(ren,
SDL_GetWindowPixelFormat(win),SDL_TEXTUREACCESS_STREAMING,window.w,window.h);
SDL_LockTexture(img, NULL, &pixels, &pitch);
memcpy(pixels, data, window.w * 3 * window.h);
SDL_UnlockTexture(img);
SDL_RenderCopy(ren,img,NULL,NULL);
SDL_DestroyTexture(img);
I have no idea what I'm doing so please have mercy
Edit (thank you for comments :))
So here is what I do now
SDL_Texture* img = SDL_CreateTexture(ren, SDL_PIXELFORMAT_RGB888,SDL_TEXTUREACCESS_STREAMING,window.w,window.h);
SDL_UpdateTexture(img,NULL,&data[0],window.w * 3);
SDL_RenderCopy(ren,img,NULL,NULL);
SDL_DestroyTexture(img);
But I get this Image... which is not what it should look like
I am thinking that my data is just formatted wrong, right now it is formatted as an array of uint8_t in RGB order. Is there another way I should be formatting it (note I do not need an alpha channel)
I am trying to display an image in Qt that is coming in as data from another function.
That function stores the rows of an image as an unsigned char**
I read somewhere I could somehow store it as a QByteArray and then create a QPixMap then set the QPixMap of a label to display it, but I am not having much luck.
This is what I had:
unsigned char* fullCharArray = new unsigned char[imheight * imwidth];
for (int i = 0 ; i < imheight ; i++)
for (int j = 0 ; j < imwidth ; j++)
fullCharArray[i*j+j] = imageData[i ][j];
QPixmap *p = new QPixmap(reinterpret_cast<const char *>(fullCharArray));
ui->viewLabel->setPixMap(p);
But this seems to give me an error, and may be the wrong thing anyway. When i tried to call setPixMap(p[]) the error goes away, but the image does not get displayed in the label.
imageData is the 2D array that is populated by my other function. Best I figured in order to create a new QPixMap I had to convert that to a 1D array, and do the index calculations manually there. That is what the double For loop is doing.
So is there a better way of displaying this image data in Qt?
Use QImage. This constructor should help you: http://doc.qt.io/qt-5/qimage.html#QImage-4
Pixmaps are not 2D arrays of pixels. They are actually defined as an array of character strings describing the image.
For example, a 4x4 pixels image with center 4 pixels black, and 4 corner pixels red is:
/* XPM */
static char * sample_xpm[] = {
"4 4 3 1",
" c #FF0000",
". c #FFFFFF",
"+ c #000000",
" .. ",
".++.",
".++.",
" .. "};
It is a convenient file format to store small images (e.g. for toolbar buttons) that can be integrated as is into a C source file using a standard #include directive, to be compiled along with the rest of the program.
You use QPixmap if the data is already in an image format like PNG or jpeg with a header.
To create an image from pixel values use Qimage, then convert this to a pixmap (Qpixmap.fromImage() - if needed to save it or display it.
I need some C++/pointer help. When I create an RGB IplImage and I want to access i,j I use the following C++ class taken from: http://www.cs.iit.edu/~agam/cs512/lect-notes/opencv-intro/opencv-intro.html
template<class T> class Image
{
private:
IplImage* imgp;
public:
Image(IplImage* img=0) {imgp=img;}
~Image(){imgp=0;}
void operator=(IplImage* img) {imgp=img;}
inline T* operator[](const int rowIndx) {
return ((T *)(imgp->imageData + rowIndx*imgp->widthStep));}
};
typedef struct{
unsigned char b,g,r;
} RgbPixel;
typedef struct{
float b,g,r;
} RgbPixelFloat;
typedef Image<RgbPixel> RgbImage;
typedef Image<RgbPixelFloat> RgbImageFloat;
typedef Image<unsigned char> BwImage;
typedef Image<float> BwImageFloat;
I've been working with CUDA so sometimes I have to put all the data into an array, I like to keep every channel in its own array, seems easier to handle the data that way. So I would usually do something like this:
IplImage *image = cvLoadImage("whatever.tif");
RgbImageFloat img(image);
for(int i = 0; i < exrIn->height; i++)
{
for(int j = 0; j < exrIn->width; j++)
{
hostr[j*data->height+i] = img[i][j].r;
hostg[j*data->height+i] = img[i][j].g;
hostb[j*data->height+i] = img[i][j].b;
}
}
I would then copy my data to the device, do some stuff with it, get it back to the host and then loop, yet again, through the array assigning the data back to the IplImage and saving my results.
It seems like I'm looping to much there has to be a faster way to do this with pointers but I'm lost, there has to be a more efficient way to do it. Is there a way I can simply use a pointer for every channel? I tried doing something like this but it didn't work:
float *hostr = &img[0][0].r
float *hostg = &img[0][0].b
float *hostb = &img[0][0].g
Any suggestions? Thanks!
EDIT:
Thanks everyone for answering. Maybe I wasn't very clear on my question. I am familiar on how to access channels and their data. What I am interested is in increasing the performance and efficiency of completely copying data off the IplImage to a standard array, more along the lines of what csl said so far. The problem I see is that the way data in an IplImage is arranged is "rgbrgbrgbrgb".
Firstly, if you're comfortable with C++, you should consider using OpenCV 2.0 which does away with different data types for images and matrices (IplImage* and CvMat*) and uses one structure (Mat) to handle both. Apart from automatic memory management and a truckload of useful routines to handle channels, etc. and some MATLAB-esque ones as well, it's really fun to use.
For your specific problem, you access the channels of an IplImage* with Mat, like this:
IplImage *image = cvLoadImage("lena.bmp");
Mat Lena(image);
vector<Mat> Channels;
split(Lena,Channels);
namedWindow("LR",CV_WINDOW_AUTOSIZE);
imshow("LR",Channels[0]);
waitKey();
Now you have the copies of each channel in the vector Channels.
If you don't want to use OpenCV2.0 and extract channels, note the following. OpenCV orders multi-channel images in the following manner:
x(1,1,1) x(1,1,2) x(1,1,3) x(1,2,1) x(1,2,2) x(1,2,3) ...
where x(i,j,k) = an element in row i of column j in channel k
Also, OpenCV pads it's images .. so don't forget to jump rows with widthStep which accounts for these padding gaps. And along the lines of what csl said, increase your row pointer in the outer loop (using widthStep) and increment this pointer to access elements in a row.
NOTE:
Since you're using 2.0 now, you can bypass IplImage* with Mat Lena = imread("Lena.bmp");.
There is room for a lot of improvement here. So much, that you should read up on how people access bitmaps.
First of all, increase memory locality as much as possible. This will increase cache hits, and performance. I.e., don't use three separate arrays for each color channel. Store each together, since you probably will be working mostly on pixels.
Secondly, don't do that y*width calculation for every pixel. When done in an inner loop, it consumes a lot of cycles.
Lastly, if you just want a complete copy of the image, then you could simply do a memcpy(), which is very fast. I couldn't deduce if you converted from floats to integers, but if not, use memcpy() for non-overlapping regions.
If you wonder how you can do this with pointers (kind of pseudo-code, and also not tested):
float *dst = &hostg[0][0];
RgbPixelFloat *src = &img[0][0];
RgbPixelFloat *end = &img[HEIGHT][WIDTH] + 1;
// copy green channel of whole image
while ( src != end ) {
*dst = src->g;
++dst;
++src;
}