Acquiring frames from Chrome (simple game) and displaying them with OpenCV - c++

Question

I've searched for ways to display a real-time simple game in a small window, and got to understand that OpenCV can handle it for me. But to accomplish that I need to acquire the frames in each iteration, just before showing it in the desired window.
This is the current solution that I am using:
HDC hwindowDC;
HDC hwindowCompatibleDC; // Memory for given DC
HBITMAP hbwindow;
int height, width;
// handles information about DIB (device-independent bitmap)
// https://learn.microsoft.com/en-us/windows/desktop/gdi/device-independent-bitmaps
BITMAPINFOHEADER bi;
// retrieces a handle to DC for the client area for entire screen
hwindowDC = GetDC(NULL);
// creates memory DC for the entire screen
hwindowCompatibleDC = CreateCompatibleDC(hwindowDC);
// enables strecn mode which has low costs - because of the grayscale game
SetStretchBltMode(hwindowCompatibleDC, COLORONCOLOR);
// size of the game window - according to the inspect of the game within chrome
height = 150;
width = 600;
// creates matrix with four color channels and values in the range 0 to 255
// - https://stackoverflow.com/questions/39543580/what-is-the-purpose-of-cvtype-cv-8uc4
frame.create(height, width, CV_8UC4);
// create a bitmap for the window
hbwindow = CreateCompatibleBitmap(hwindowDC, width, height);
bi.biSize = sizeof(BITMAPINFOHEADER);
bi.biWidth = width;
bi.biHeight = -height; //the minus sign makes it draw upside down or not
bi.biPlanes = 1;
bi.biBitCount = 32;
bi.biCompression = BI_RGB;
bi.biSizeImage = 0;
bi.biXPelsPerMeter = 0;
bi.biYPelsPerMeter = 0;
bi.biClrUsed = 0;
bi.biClrImportant = 0;
// attach the bitmap with the memory DC for our window
SelectObject(hwindowCompatibleDC, hbwindow);
// copy from the window device context to the bitmap device context
StretchBlt(hwindowCompatibleDC, 0, 0, width, height, hwindowDC, 650, 130, 620, 150, SRCCOPY); //change SRCCOPY to NOTSRCCOPY for wacky colors !
// Gets the "bits" from the bitmap and copies them into src.data
GetDIBits(hwindowCompatibleDC, hbwindow, 0, height, frame.data, (BITMAPINFO *)&bi, DIB_RGB_COLORS); //copy from hwindowCompatibleDC to hbwindow
// avoid memory leak // avoid memory leak
DeleteObject(hbwindow);
DeleteDC(hwindowCompatibleDC); // for any DC that was created by calling the CreateDC function
ReleaseDC(hwnd, hwindowDC); // releasing the DC for use of other applications (MUST)
img2 = frame;
//img1 = frame;
return frame;
After trying to figure out how to make the game running smoother, I've tried to implement screen recorder with the DirectX API, but without any success. This code results in a black screen.
IDirect3DSurface9* pSurface;
d3ddev->CreateOffscreenPlainSurface(100, 100,
D3DFMT_A8R8G8B8, D3DPOOL_SCRATCH, &pSurface, NULL);
d3ddev->GetFrontBufferData(NULL , pSurface);
/*D3DLOCKED_RECT lockedRect;
ZeroMemory(&lockedRect, sizeof(D3DLOCKED_RECT));
pSurface->LockRect(&lockedRect, 0, D3DLOCK_READONLY);
*/
LPD3DXBUFFER buffer;
D3DXSaveSurfaceToFileInMemory(&buffer, D3DXIFF_BMP, pSurface, NULL, NULL);
DWORD imSize = buffer->GetBufferSize();
void* imgBuffer = buffer->GetBufferPointer();
//**** OpenCV
Mat D3DSurface(100, 100, CV_8UC4, imgBuffer);
imshow("D3DSurface", D3DSurface);
pSurface->Release();
return D3DSurface;
Can anyone can help me figure out how to accomplish this task in a better way?
I just think that there is something that I'm not quite understanding in this process, mirroring a live moving game to a desired window with minimal lags.
My purpose is to make a bot for an online game.