Video recording of the window on OpenCV C++ - c++

Question

I am trying to write a program for recording windows, but for some reason, after the program finishes, I get a corrupted .avi file.
I don't understand what the problem is. The hwnd2mat() and windowNames() functions work correctly, the error is clearly not in it. The code looks massive, but in fact, most of the code is occupied by the translation of the image from the HWND to the Mat. Also it should be noted that the resulting image after recording, always has the same size (irrespective of the recording time).
#include <iostream>
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
#include <opencv2/videoio.hpp>
#include <Windows.h>
BOOL CALLBACK windowNames(HWND hwnd, LPARAM lParam) {
const DWORD TITLE_SIZE = 1024;
WCHAR windowTitle[TITLE_SIZE];
GetWindowTextW(hwnd, windowTitle, TITLE_SIZE);
int length = ::GetWindowTextLength(hwnd);
std::wstring title(&windowTitle[0]);
if (!IsWindowVisible(hwnd) || length == 0 || title == L"Program Manager") {
return TRUE;
}
// Retrieve the pointer passed into this callback, and re-'type' it.
// The only way for a C API to pass arbitrary data is by means of a void*.
std::vector<std::wstring>& titles = *reinterpret_cast<std::vector<std::wstring>*>(lParam);
titles.push_back(title);
return TRUE;
}
cv::Mat hwnd2mat(HWND hwnd)
{
HDC hwindowDC, hwindowCompatibleDC;
int height, width, srcheight, srcwidth;
HBITMAP hbwindow;
cv::Mat src;
BITMAPINFOHEADER bi;
HBITMAP bi2;
hwindowDC = GetDC(hwnd);
hwindowCompatibleDC = CreateCompatibleDC(hwindowDC);
SetStretchBltMode(hwindowCompatibleDC, COLORONCOLOR);
RECT windowsize; // get the height and width of the screen
GetClientRect(hwnd, &windowsize);
srcheight = windowsize.bottom;
srcwidth = windowsize.right;
height = windowsize.bottom / 1; //change this to whatever size you want to resize to
width = windowsize.right / 1;
// create a bitmap
hbwindow = CreateCompatibleBitmap(hwindowDC, width, height);
bi.biSize = sizeof(BITMAPINFOHEADER); //http://msdn.microsoft.com/en-us/library/windows/window/dd183402%28v=vs.85%29.aspx
bi.biWidth = width;
bi.biHeight = -height; //this is the line that makes it draw upside down or not
bi.biPlanes = 1;
bi.biBitCount = 32;
bi.biCompression = BI_RGB;
bi.biSizeImage = 0;
bi.biXPelsPerMeter = 1;
bi.biYPelsPerMeter = 2;
bi.biClrUsed = 3;
bi.biClrImportant = 4;
// use the previously created device context with the bitmap
SelectObject(hwindowCompatibleDC, hbwindow);
// copy from the window device context to the bitmap device context
StretchBlt(hwindowCompatibleDC, 0, 0, width, height, hwindowDC, 0, 0, srcwidth, srcheight, SRCCOPY); //change SRCCOPY to NOTSRCCOPY for wacky colors !
src.create(height, width, CV_8UC4);
GetDIBits(hwindowCompatibleDC, hbwindow, 0, height, src.data, (BITMAPINFO*)&bi, DIB_RGB_COLORS); //copy from hwindowCompatibleDC to hbwindow
// avoid memory leak
DeleteObject(hbwindow);
DeleteDC(hwindowCompatibleDC);
ReleaseDC(hwnd, hwindowDC);
return src;
}
int main(int argc, char** argv)
{
std::vector<std::wstring> titles; // we use std::wstring in place of std::string. This is necessary so that the entire character set can be represented.
EnumWindows(windowNames, reinterpret_cast<LPARAM>(&titles));
HWND hwndDesktop = GetDesktopWindow();
size_t number = 0;
int i = 0;
for (const auto& title : titles)
std::wcout << L"Title: " << i++ << title << std::endl;
std::cin >> number;
HWND hwndWindow = FindWindow(NULL, titles[number].c_str());
cv::namedWindow("output", cv::WINDOW_NORMAL);
cv::Mat src = hwnd2mat(/*hwndDesktop*/hwndWindow);
cv::VideoWriter outputVideo("output.avi", cv::VideoWriter::fourcc('M', 'J', 'P', 'G'), 1, cv::Size(src.cols, src.rows));
outputVideo.write(src);
int key = 0;
while (key != 27)
{
src = hwnd2mat(hwndWindow);
outputVideo.write(src);
cv::imshow("output", src);
key = cv::waitKey(60); //press ESC to end
}
return 0;
}

Answer 1

The problem lies in the fact that we need to transfer the Mat from BGRA to BGR during the transfer of the frame to the VideoWriter object.
For correct operation, it is necessary to write
Mat bgrImg; cvtColor(src, bgrImg, COLOR_BGRA2BGR);
in the range before sending a frame and send bgrImg as a frame.

Answer 2

VideoWriter instances need to be closed using the release() method. that finalizes the video container file.

Answer 3

OpenCV has no support for modern screen capture AFAIK. You’ll need to use platform-specific means of doing this (maybe encapsulated in some library). The problem here is that the screen data is already in the GPU, and by using OpenCV you’re forcing it to be copied to the main memory and processed with a relatively slow CPU. It won’t perform well. Instead, a platform-specific approach will process the data on the GPU, using it to both extract the window’s frames and encode them. It’ll be very efficient both in terms of speed as well as energy consumption (you’ll vastly improve battery life while the capture is running, and will prevent the fans from being annoying on notebooks).