How to turn a screenshot bitmap to a cv::Mat - c++

Question

I currently am trying to take a screenshot of the screen, and then get it into a format editable by OpenCV. The code I'm using is from the microsoft website, https://learn.microsoft.com/en-gb/windows/win32/gdi/capturing-an-image. The code uses the "Windows.h" library. The easiest way of doing it is obviously to just save the bitmap as a .bmp, then open it using opencv. However, I would like it to be more efficient than that, and I don't know how to. When I used the code, it outputted a char pointer, which I don't know how to convert to a cv::Mat. The code is below:
cv::Mat * Capture::GetMat()
{
cv::Mat * mat1;
MemoryHandle = NULL;
BitmapHandle = NULL;
// Find the handle for the device context of the entire screen, and the specific window specified.
ScreenHandle = GetDC(NULL);
WindowHandle = GetDC(hwnd);
//Make the compatible DC (Device Context) for storing the data in memory.
MemoryHandle = CreateCompatibleDC(WindowHandle);
//Make a compatible DC for the bitmap to be stored in.
BitmapHandle = CreateCompatibleBitmap(WindowHandle, width, height);
//Select the correct bitmap, and put it into memory using the memory handle.
SelectObject(MemoryHandle, BitmapHandle);
//Transfer the actual bitmap into the compatible memory DC.
BitBlt(MemoryHandle, 0, 0, 1920, 1080, WindowHandle, 0, 0, SRCCOPY);
//Get the bitmap from the handle, and ready it to be filed.
GetObject(BitmapHandle, sizeof(BITMAP), &Bitmap);
//Cofinguring INFO details.
bmpInfoHeader.biSize = sizeof(BITMAPINFOHEADER);
bmpInfoHeader.biWidth = Bitmap.bmWidth;
bmpInfoHeader.biHeight = Bitmap.bmHeight;
bmpInfoHeader.biPlanes = 1;
bmpInfoHeader.biBitCount = 32;
bmpInfoHeader.biCompression = BI_RGB;
bmpInfoHeader.biSizeImage = 0;
bmpInfoHeader.biXPelsPerMeter = 0;
bmpInfoHeader.biYPelsPerMeter = 0;
bmpInfoHeader.biClrUsed = 0;
bmpInfoHeader.biClrImportant = 0;
bmpSize = ((Bitmap.bmWidth * bmpInfoHeader.biBitCount + 31) / 32) * 4 * Bitmap.bmHeight;
memhnd = GlobalAlloc(GHND, bmpSize);
mat1 = (cv::Mat *)GlobalLock(memhnd);
std::cout << GetLastError() << std::endl;
return mat1;
}
int Capture::save_mat(cv::Mat * mat)
{
std::string FileName("P:/Photos/capture");
FileName += std::to_string(image_count_mat);
FileName += (const char*)(".jpg");
cv::Mat mat2 = *mat;
cv::imwrite(FileName.c_str(), mat2);
image_count_mat++;
return 0;
}
The class has these attributes:
private:
HWND hwnd;
HDC hdc;
int image_count_bitmap = 0;
int image_count_mat = 0;
int height;
int width;
HDC ScreenHandle;
HDC WindowHandle;
HDC MemoryHandle = NULL;
HBITMAP BitmapHandle = NULL;
BITMAP Bitmap;
BITMAPFILEHEADER bmpFileHeader;
BITMAPINFOHEADER bmpInfoHeader;
DWORD bmpSize;
HANDLE memhnd;
The GetMat() function works fine and doesn't output an error, although I have no idea how to check if the outputted cv::Mat is correct. When I run the save_mat() function however, the program crashes.

Answer 1

It is not recommended to store device context handles. For example a call to GetDC should be followed by ReleaseDC as soon as you are finished with the handle. You can store bitmap handles and memory dc, but in most cases it is not necessary.
Once you have copied the image in to bitmap, use GetDIBits to copy the bits in to cv::Mat as shown in the example below.
Note that your application needs DPI compatibility, for example with SetProcessDPIAware to find the correct desktop size.
This example uses 32-bit bitmap with CV_8UC4, but these GDI functions are 24-bit. You can also use 24-bit bitmap with CV_8UC3.
void screenshot()
{
auto w = GetSystemMetrics(SM_CXFULLSCREEN);
auto h = GetSystemMetrics(SM_CYFULLSCREEN);
auto hdc = GetDC(HWND_DESKTOP);
auto hbitmap = CreateCompatibleBitmap(hdc, w, h);
auto memdc = CreateCompatibleDC(hdc);
auto oldbmp = SelectObject(memdc, hbitmap);
BitBlt(memdc, 0, 0, w, h, hdc, 0, 0, SRCCOPY);
cv::Mat mat(h, w, CV_8UC4);
BITMAPINFOHEADER bi = { sizeof(bi), w, -h, 1, 32, BI_RGB };
GetDIBits(hdc, hbitmap, 0, h, mat.data, (BITMAPINFO*)&bi, DIB_RGB_COLORS);
cv::imwrite("screenshot.png", mat);
SelectObject(memdc, oldbmp);
DeleteDC(memdc);
DeleteObject(hbitmap);
ReleaseDC(HWND_DESKTOP, hdc);
}