Related
I want to write a screencasting program for the Windows platform, but am unsure of how to capture the screen. The only method I'm aware of is to use GDI, but I'm curious whether there are other ways to go about this, and, if there are, which incurs the least overhead? Speed is a priority.
The screencasting program will be for recording game footage, although, if this does narrow down the options, I'm still open for any other suggestions that fall out of this scope. Knowledge isn't bad, after all.
Edit: I came across this article: Various methods for capturing the screen. It has introduced me to the Windows Media API way of doing it and the DirectX way of doing it. It mentions in the Conclusion that disabling hardware acceleration could drastically improve the performance of the capture application. I'm curious as to why this is. Could anyone fill in the missing blanks for me?
Edit: I read that screencasting programs such as Camtasia use their own capture driver. Could someone give me an in-depth explanation on how it works, and why it is faster? I may also need guidance on implementing something like that, but I'm sure there is existing documentation anyway.
Also, I now know how FRAPS records the screen. It hooks the underlying graphics API to read from the back buffer. From what I understand, this is faster than reading from the front buffer, because you are reading from system RAM, rather than video RAM. You can read the article here.
This is what I use to collect single frames, but if you modify this and keep the two targets open all the time then you could "stream" it to disk using a static counter for the file name. - I can't recall where I found this, but it has been modified, thanks to whoever!
void dump_buffer()
{
IDirect3DSurface9* pRenderTarget=NULL;
IDirect3DSurface9* pDestTarget=NULL;
const char file[] = "Pickture.bmp";
// sanity checks.
if (Device == NULL)
return;
// get the render target surface.
HRESULT hr = Device->GetRenderTarget(0, &pRenderTarget);
// get the current adapter display mode.
//hr = pDirect3D->GetAdapterDisplayMode(D3DADAPTER_DEFAULT,&d3ddisplaymode);
// create a destination surface.
hr = Device->CreateOffscreenPlainSurface(DisplayMde.Width,
DisplayMde.Height,
DisplayMde.Format,
D3DPOOL_SYSTEMMEM,
&pDestTarget,
NULL);
//copy the render target to the destination surface.
hr = Device->GetRenderTargetData(pRenderTarget, pDestTarget);
//save its contents to a bitmap file.
hr = D3DXSaveSurfaceToFile(file,
D3DXIFF_BMP,
pDestTarget,
NULL,
NULL);
// clean up.
pRenderTarget->Release();
pDestTarget->Release();
}
EDIT: I can see that this is listed under your first edit link as "the GDI way". This is still a decent way to go even with the performance advisory on that site, you can get to 30fps easily I would think.
From this comment (I have no experience doing this, I'm just referencing someone who does):
HDC hdc = GetDC(NULL); // get the desktop device context
HDC hDest = CreateCompatibleDC(hdc); // create a device context to use yourself
// get the height and width of the screen
int height = GetSystemMetrics(SM_CYVIRTUALSCREEN);
int width = GetSystemMetrics(SM_CXVIRTUALSCREEN);
// create a bitmap
HBITMAP hbDesktop = CreateCompatibleBitmap( hdc, width, height);
// use the previously created device context with the bitmap
SelectObject(hDest, hbDesktop);
// copy from the desktop device context to the bitmap device context
// call this once per 'frame'
BitBlt(hDest, 0,0, width, height, hdc, 0, 0, SRCCOPY);
// after the recording is done, release the desktop context you got..
ReleaseDC(NULL, hdc);
// ..delete the bitmap you were using to capture frames..
DeleteObject(hbDesktop);
// ..and delete the context you created
DeleteDC(hDest);
I'm not saying this is the fastest, but the BitBlt operation is generally very fast if you're copying between compatible device contexts.
For reference, Open Broadcaster Software implements something like this as part of their "dc_capture" method, although rather than creating the destination context hDest using CreateCompatibleDC they use an IDXGISurface1, which works with DirectX 10+. If there is no support for this they fall back to CreateCompatibleDC.
To change it to use a specific application, you need to change the first line to GetDC(game) where game is the handle of the game's window, and then set the right height and width of the game's window too.
Once you have the pixels in hDest/hbDesktop, you still need to save it to a file, but if you're doing screen capture then I would think you would want to buffer a certain number of them in memory and save to the video file in chunks, so I will not point to code for saving a static image to disk.
I wrote a video capture software, similar to FRAPS for DirectX applications. The source code is available and my article explains the general technique. Look at http://blog.nektra.com/main/2013/07/23/instrumenting-direct3d-applications-to-capture-video-and-calculate-frames-per-second/
Respect to your questions related to performance,
DirectX should be faster than GDI except when you are reading from the frontbuffer which is very slow. My approach is similar to FRAPS (reading from backbuffer). I intercept a set of methods from Direct3D interfaces.
For video recording in realtime (with minimal application impact), a fast codec is essential. FRAPS uses it's own lossless video codec. Lagarith and HUFFYUV are generic lossless video codecs designed for realtime applications. You should look at them if you want to output video files.
Another approach to recording screencasts could be to write a Mirror Driver. According to Wikipedia: When video mirroring is active, each time the system draws to the primary video device at a location inside the mirrored area, a copy of the draw operation is executed on the mirrored video device in real-time. See mirror drivers at MSDN: http://msdn.microsoft.com/en-us/library/windows/hardware/ff568315(v=vs.85).aspx.
I use d3d9 to get the backbuffer, and save that to a png file using the d3dx library:
IDirect3DSurface9 *surface ;
// GetBackBuffer
idirect3ddevice9->GetBackBuffer(0, 0, D3DBACKBUFFER_TYPE_MONO, &surface ) ;
// save the surface
D3DXSaveSurfaceToFileA( "filename.png", D3DXIFF_PNG, surface, NULL, NULL ) ;
SAFE_RELEASE( surface ) ;
To do this you should create your swapbuffer with
d3dpps.SwapEffect = D3DSWAPEFFECT_COPY ; // for screenshots.
(So you guarantee the backbuffer isn't mangled before you take the screenshot).
In my Impression, the GDI approach and the DX approach are different in its nature.
painting using GDI applies the FLUSH method, the FLUSH approach draws the frame then clear it and redraw another frame in the same buffer, this will result in flickering in games require high frame rate.
WHY DX quicker?
in DX (or graphics world), a more mature method called double buffer rendering is applied, where two buffers are present, when present the front buffer to the hardware, you can render to the other buffer as well, then after the frame 1 is finished rendering, the system swap to the other buffer( locking it for presenting to hardware , and release the previous buffer ), in this way the rendering inefficiency is greatly improved.
WHY turning down hardware acceleration quicker?
although with double buffer rendering, the FPS is improved, but the time for rendering is still limited. modern graphic hardware usually involves a lot of optimization during rendering typically like anti-aliasing, this is very computation intensive, if you don't require that high quality graphics, of course you can just disable this option. and this will save you some time.
I think what you really need is a replay system, which I totally agree with what people discussed.
I wrote a class that implemented the GDI method for screen capture. I too wanted extra speed so, after discovering the DirectX method (via GetFrontBuffer) I tried that, expecting it to be faster.
I was dismayed to find that GDI performs about 2.5x faster. After 100 trials capturing my dual monitor display, the GDI implementation averaged 0.65s per screen capture, while the DirectX method averaged 1.72s. So GDI is definitely faster than GetFrontBuffer, according to my tests.
I was unable to get Brandrew's code working to test DirectX via GetRenderTargetData. The screen copy came out purely black. However, it could copy that blank screen super fast! I'll keep tinkering with that and hope to get a working version to see real results from it.
For C++ you can use: http://www.pinvoke.net/default.aspx/gdi32/BitBlt.html
This may hower not work on all types of 3D applications/video apps. Then this link may be more useful as it describes 3 different methods you can use.
Old answer (C#):
You can use System.Drawing.Graphics.Copy, but it is not very fast.
A sample project I wrote doing exactly this: http://blog.tedd.no/index.php/2010/08/16/c-image-analysis-auto-gaming-with-source/
I'm planning to update this sample using a faster method like Direct3D: http://spazzarama.com/2009/02/07/screencapture-with-direct3d/
And here is a link for capturing to video: How to capture screen to be video using C# .Net?
You want the Desktop Duplication API (available since Windows 8). That is the officially recommended way of doing it, and it's also the most CPU efficient.
One nice feature it has for screencasting is that it detects window movement, so you can transmit block deltas when windows get moved around, instead of raw pixels. Also, it tells you which rectangles have changed, from one frame to the next.
The Microsoft example code is quite complex, but the API is actually simple and easy to use. I've put together an example project that is much simpler:
Simplified Sample Code
WindowsDesktopDuplicationSample
Microsoft References
Desktop Duplication API
Official example code (my example above is a stripped down version of this)
A few things I've been able to glean: apparently using a "mirror driver" is fast though I'm not aware of an OSS one.
Why is RDP so fast compared to other remote control software?
Also apparently using some convolutions of StretchRect are faster than BitBlt
http://betterlogic.com/roger/2010/07/fast-screen-capture/comment-page-1/#comment-5193
And the one you mentioned (fraps hooking into the D3D dll's) is probably the only way for D3D applications, but won't work with Windows XP desktop capture. So now I just wish there were a fraps equivalent speed-wise for normal desktop windows...anybody?
(I think with aero you might be able to use fraps-like hooks, but XP users would be out of luck).
Also apparently changing screen bit depths and/or disabling hardware accel. might help (and/or disabling aero).
https://github.com/rdp/screen-capture-recorder-program includes a reasonably fast BitBlt based capture utility, and a benchmarker as part of its install, which can let you benchmark BitBlt speeds to optimize them.
VirtualDub also has an "opengl" screen capture module that is said to be fast and do things like change detection http://www.virtualdub.org/blog/pivot/entry.php?id=290
You can try the c++ open source project WinRobot #git, a powerful screen capturer
CComPtr<IWinRobotService> pService;
hr = pService.CoCreateInstance(__uuidof(ServiceHost) );
//get active console session
CComPtr<IUnknown> pUnk;
hr = pService->GetActiveConsoleSession(&pUnk);
CComQIPtr<IWinRobotSession> pSession = pUnk;
// capture screen
pUnk = 0;
hr = pSession->CreateScreenCapture(0,0,1280,800,&pUnk);
// get screen image data(with file mapping)
CComQIPtr<IScreenBufferStream> pBuffer = pUnk;
Support :
UAC Window
Winlogon
DirectShowOverlay
Screen Recording can be done in C# using VLC API. I have done a sample program to demonstrate this. It uses LibVLCSharp and VideoLAN.LibVLC.Windows libraries. You could achieve many more features related to video rendering using this cross platform API.
For API documentation see: LibVLCSharp API Github
using System;
using System.IO;
using System.Reflection;
using System.Threading;
using LibVLCSharp.Shared;
namespace ScreenRecorderNetApp
{
class Program
{
static void Main(string[] args)
{
Core.Initialize();
using (var libVlc = new LibVLC())
using (var mediaPlayer = new MediaPlayer(libVlc))
{
var media = new Media(libVlc, "screen://", FromType.FromLocation);
media.AddOption(":screen-fps=24");
media.AddOption(":sout=#transcode{vcodec=h264,vb=0,scale=0,acodec=mp4a,ab=128,channels=2,samplerate=44100}:file{dst=testvlc.mp4}");
media.AddOption(":sout-keep");
mediaPlayer.Play(media);
Thread.Sleep(10*1000);
mediaPlayer.Stop();
}
}
}
}
This might not be the fastest method, but it is leightweight and easy to use. The image is returned as an integer array containing the RGB colors.
#define WIN32_LEAN_AND_MEAN
#define VC_EXTRALEAN
#include <Windows.h>
int* screenshot(int& width, int& height) {
HDC hdc = GetDC(NULL); // get the desktop device context
HDC cdc = CreateCompatibleDC(hdc); // create a device context to use yourself
height = (int)GetSystemMetrics(SM_CYVIRTUALSCREEN); // get the width and height of the screen
width = 16*height/9; // only capture left monitor for dual screen setups, for both screens use (int)GetSystemMetrics(SM_CXVIRTUALSCREEN);
HBITMAP hbitmap = CreateCompatibleBitmap(hdc, width, height); // create a bitmap
SelectObject(cdc, hbitmap); // use the previously created device context with the bitmap
BITMAPINFOHEADER bmi = { 0 };
bmi.biSize = sizeof(BITMAPINFOHEADER);
bmi.biPlanes = 1;
bmi.biBitCount = 32;
bmi.biWidth = width;
bmi.biHeight = -height; // flip image upright
bmi.biCompression = BI_RGB;
bmi.biSizeImage = 3*width*height;
BitBlt(cdc, 0, 0, width, height, hdc, 0, 0, SRCCOPY); // copy from desktop device context to bitmap device context
ReleaseDC(NULL, hdc);
int* image = new int[width*height];
GetDIBits(cdc, hbitmap, 0, height, image, (BITMAPINFO*)&bmi, DIB_RGB_COLORS);
DeleteObject(hbitmap);
DeleteDC(cdc);
return image;
}
The above code combines this answer and this answer.
Example on how to use it:
int main() {
int width=0, height=0;
int* image = screenshot(width, height);
// access pixel colors for position (x|y)
const int x=0, y=0;
const int color = image[x+y*width];
const int red = (color>>16)&255;
const int green = (color>> 8)&255;
const int blue = color &255;
delete[] image;
}
i myself do it with directx and think it's as fast as you would want it to be. i don't have a quick code sample, but i found this which should be useful. the directx11 version should not differ a lot, directx9 maybe a little more, but thats the way to go
DXGI Desktop Capture
Project that captures the desktop image with DXGI duplication. Saves the captured image to the file in different image formats (*.bmp; *.jpg; *.tif).
This sample is written in C++. You also need some experience with DirectX (D3D11, D2D1).
What the Application Can Do
If you have more than one desktop monitor, you can choose.
Resize the captured desktop image.
Choose different scaling modes.
You can show or hide the mouse icon in the output image.
You can rotate the image for the output picture, or leave it as default.
I realize the following suggestion doesn't answer your question, but the simplest method I have found to capture a rapidly-changing DirectX view, is to plug a video camera into the S-video port of the video card, and record the images as a movie. Then transfer the video from the camera back to an MPG, WMV, AVI etc. file on the computer.
Windows.Graphics.Capture
Enables apps to capture environments, application windows, and displays in a secure, easy to use way with the use of a system picker UI control.
https://blogs.windows.com/windowsdeveloper/2019/09/16/new-ways-to-do-screen-capture/
In my Windows MFC application, in its View class I created an OpenGL context using View's DC:
HANDLE * hdc = GetDC()->m_hdc;
int nPixelFormat;
static PIXELFORMATDESCRIPTOR pfd = {
sizeof(PIXELFORMATDESCRIPTOR), // Size of this structure
1, // Version of this structure
PFD_DRAW_TO_WINDOW | // Draw to window (not bitmap)
PFD_SUPPORT_OPENGL | // Support OpenGL calls
PFD_DOUBLEBUFFER, // Double -buffered mode
PFD_TYPE_RGBA, // RGBA Color mode
24, // Want 24bit color
0,0,0,0,0,0, // Not used to select mode
0,0, // Not used to select mode
0,0,0,0,0, // Not used to select mode
32, // Size of depth buffer
0, // Not used to select mode
0, // Not used to select mode
PFD_MAIN_PLANE, // Draw in main plane
0, // Not used to select mode
0,0,0 }; // Not used to select mode
// Choose a pixel format that best matches that described in pfd
nPixelFormat = ChoosePixelFormat(hdC, &pfd);
// Set the pixel format for the device context
assert(SetPixelFormat(hdC, nPixelFormat, &pfd));
HGLRC m_hrc = wglCreateContext(hdc);
assert(m_hrc);
wglMakeCurrent(m_hdc,m_hrc);
All the code above works all right, and I can do OpenGL drawings as expected.
But , What I need now is to change the DC to memory dc instead of window DC . To be exact, how can I use the 'hmemDC' bellow to create an OpenGL context like the way I did above with window DC:
CRect rct;
GetClientRect(&rct);
HDC hmemDC = CreateCompatibleDC(pDC->m_hDC);
HBITMAP hBmp = CreateCompatibleBitmap(pDC->m_hDC,rct.Width(),rct.Height());
with the same pixel format constructed above, I came across the "Invalid pixel format" error in calling wglCreateContext() , can not success in getting the correct OpenGL context.
I googled a lot , and tryed to change some of the values of pixel format, the result was the same.
Is it possible to create OpenGL context with Windows Memory DC? and if it is how should I do it?
Edit:
This is why I need a bitmap ( or Memory DC ): I created a 2d map rendering library which uses OpenGL. The client want to use this library to render background map, and draw its own symbols on top of it . But, they prefer to use Windows GDI other than OpenGL to draw their symbols. So I thought if I can provide them with a bitmap or a Mmeory DC, they could to what they want. Any better solutions? Am I in the right direction? Or it is a total bad idea to provide such a 2d library in OpenGL backend.
This can't be done in a useful way.
You can in principle render to a bitmap by using the PFD_DRAW_TO_BITMAP flag instead of PFD_DRAW_TO_WINDOW in the PIXELFORMATDESCRIPTOR.
However, doing so will disable all hardware accelearated rendering. This will fall back to Microsofts default OpenGL 1.1 implementation.
If you want hw-accleration and/or modern GL, you either need a window or some offscreen buffer like a pbuffer, which is available via the WGL_ARB_pbuffer extension. However, in modern GL, you are probably better off creating a window which is just never shown, and using a Frambeuffer Object as the offscreen render target.
In either case, you will have to copy the data back to the CPU, if you need it as some bitmap there.
Put in few words: You can't create arbitrary OpenGL contexts for MemDCs. At least no kind of OpenGL context you'd actually want to use.
If your goal is off-screen rendering either create a PBuffer-DC; which requires to create a OpenGL context first which in turn required to create a window and setting its pixel format. Or you can just create a window and a OpenGL context for it and use a framebuffer object.
I am implementing a plug-in inside a 3rd party program in C++ on Windows.
The 3rd party program has a window that displays 3D graphics using OpenGL.
However I need the plug-in to create another window that also displays 3D graphics using OpenGL.
Do I need to create a new OpenGL rendering context for my window or is there some way that I can "reuse" the OpenGL rendering context used by the 3rd party program?
I assumed that I had to create a new OpenGL rendering context and tried the following:
// create a rendering context
hglrc = wglCreateContext (hdc);
// make it the calling thread's current rendering context
wglMakeCurrent (hdc, hglrc);
However the last function failed.
Reading the documentation of wglMakeCurrent I notice that
A thread can have one current rendering context. A process can have multiple rendering contexts by means of multithreading.
Does this mean that my window need to run in a separate thread from the 3rd party program?
You didn't post error code generated by wglMakeCurrent(), so I won't be guessing the reason. It's not the binding itself, however. Sentence 'A thread can have one current rendering context' means, that new context will 'replace' the old one and become the current. I don't know why are you trying to set two contexts as current (or run another thread), but it's not the way to go. Avoid multithreading in rendering unless it's absolutely necessary.
So, answering your question:
Yes, you CAN 'reuse' OpenGL rendering context.
Why, you may ask? Rendering context is created for specific device context (HDC), which is exclusive property of each window (HWND)! How is this possible, then?!
Well, it seems somehow impossible because of function prototypes:
HWND my_window = CreateWindow(...);
HDC my_dc = GetDC(my_new_window);
//Setup pixel format for 'my_dc'...
HGLRC my_rc = wglCreateContext(my_dc);
wglMakeCurrent(my_dc, my_rc);
This really lets you think that rendering context is bound to this specific device context and valid only for it. But it's not.
The critical part is the comment (setup pixel format). Rendering context is created for specific CLASS of DCs, to be more precise: for DCs with the same pixel format. So code below is perfectly valid:
//window_1 = main window, window_2 = your window
HDC dc_1 = GetDC(window_1);
Set_pixel_format_for_dc_1(); //Usual stuff
HGLRC rc = wglCreateContext(dc_1);
wglMakeCurrent(dc_1, rc);
ultra_super_draw();
//.....
HDC dc_2 = GetDC(window_2);
//Get dc_1's PF to make sure it's compatible with rc.
int pf_index = GetPixelFormat(dc_1);
PIXELFORMATDESCRIPTOR pfd;
ZeroMemory(&pfd, sizeof(PIXELFORMATDESCRIPTOR));
DescribePixelFormat(dc_1, pf_index, sizeof(PIXELFORMATDESCRIPTOR), &pfd);
SetPixelFormat(dc_2, pf_index, &pfd);
wglMakeCurrent(dc_2, rc);
another_awesome_render();
wglMakeCurrent(NULL, NULL);
If you are still not convinced, MSDN:
wglMakeCurrent(hdc, hglrc): The hdc parameter must refer to a drawing surface supported by OpenGL. It need not be the same hdc that was passed to wglCreateContext when hglrc was created, but it must be on the same device and have the same pixel format.
I guess you are already familiar with these calls. Now, I don't know what are the conditions that your rendering must meet, but without additional requirements, I don't see any difficulties from this point:
HDC my_dc = Create_my_DC();
//...
void my_new_render
{
//Probably you want to save current binding:
HDC current_dc = wglGetCurrentDC();
HGLRC current_context = wglGetCurrentContext();
wglMakeCurrent(my_dc, current_context);
MyUltraSuperRender(...);
wglMakeCurrent(current_dc, current_context);
}
Hope this helps :)
First things first, you actually should create a separate OpenGL context for your plugin, for the simple reason that it gives you a separate state space that doesn't interfere with the main programs OpenGL context.
You misunderstood the part about multiple rendering contexts though. It's perfectly possible to have an arbitrary number of OpenGL contexts for a process. But each thread of the process can bind only one context at a time. That one binding also includes the window DC the context is bound to. It is however perfectly legal change a context binding at any time. Either you change the window a given context is bound to, or you switch the context or you do both at the same time.
So in your situation I suggest you create a custom context for your plug-in, that you use for all the windows your plug-in creates.
That your simple context "creation" code fails has one simple reason: Your window will most likely not have a pixel format descriptor set.
I suggest you use the following method to create your new windows and contexts:
/* first get hold of the HDC/HRC of the parent */
HDC parentDC = wglGetCurrentDC();
HRC parentRC = wglGetCurrentContext();
int pixelformatID = GetPixelFormat(parentDC);
/* we use the same PFD as the parent */
PIXELFORMATDESCRIPTOR pixelformat;
memset(pixelformat, 0, sizeof(pixelformat);
DescribePixelFormat(parentDC, pixelformatID, sizeof(pixelformat), &pixelformat);
/* create a window and set it's pixelformat to the parent one's */
HWND myWND = create_my_window();
HDC myDC = GetDC(myWND);
SetPixelFormat(myDC, pixelformatID, &pixelformat);
/* finally we can create a rendering context
* it doesn't matter if we create it against
* the parent or our own DC.
*/
HRC myRC = wglCreateContext(myDC);
/* we're done here... */
Now whenever your plugin wants to render something it should bind its own context, do its thing and bind the context that was bound before:
HDC prevDC = wglGetCurrentDC();
HRC prevRC = wglGetCurrentContext();
wglMakeCurrent(myDC, myRC);
/* do OpenGL stuff */
wglMakeCurrent(prevDC, prevRC);
I have a task to implement offscreen OpenGL renderer both for Window and Linux in C++.I have such a version already written in Java using LWJGL lib.There I used PBuffer object ,which under hood creates Pbuffers based on the used OS.First I thought to re-implement the full PBuffer creation logic just as it i done in native source of LWJGL.Then I read this post on StackOverflow.com where it is suggested using the standard context creation ,let's say using GLFW (which is cross platform) but not to create the actual window.Is it the right way to go? What are pros and cons vs using Pbuffer in such a case?
Update:
I just want to emphasize that I use FBOs to render the frames so my problem here is not how to render in offscreen mode but how to create a context without window both in Windows and Linux OSs.
I'd highly recommend not to use PBuffers anymore but to use Frame Buffer Objects (FBOs) instead. FBOs offer much better performance as using them does not require a context switch and they have several other advantages.
LWJGL supports FBOs, but GLFW is "just" for cross-platform setup of OpenGL and not for rendering. For convenient cross-platform FBO usage I'd recommend to use a library like OGLplus on top of GLFW. See here for a render-to-texture example.
The Simple DirectMedia Layer (SDL) library is worth a try. It simplifies cross-platform OpenGL context creation, with the ability to use memory surfaces for off-screen rendering.
The only thing you would have to do extra, is to include your OpenGL and SDL headers from different locations, depending on your platform. This can be done with simple pre-processor directives.
As far as I know there is no cross-platform way to create contexts, you will have to create your own abstraction and then implement it for each platform.
On windows I have used the following code to create a second context to do loading of content in a background thread (this program used GLFW but that should not matter):
void Program::someFunction()
{
HDC hdc = wglGetCurrentDC();
HGLRC hglrc = wglGetCurrentContext();
HGLRC hglrc_new = wglCreateContext(hdc);
wglShareLists(hglrc, hglrc_new);
loadThread = boost::thread(&Program::loadFunc, this, hdc, hglrc_new);
}
/**
* Imports all our assets. Supposed to run in its own thread with its own OpenGL context.
* #param hdc The current device context.
* #param hglrc A OpenGL context thats shares its display list with the main rendering context.
*/
void Program::loadFunc(HDC hdc, HGLRC hglrc)
{
wglMakeCurrent(hdc, hglrc);
//Do stuff...
wglMakeCurrent(NULL, NULL);
wglDeleteContext(hglrc);
}
I want to write a screencasting program for the Windows platform, but am unsure of how to capture the screen. The only method I'm aware of is to use GDI, but I'm curious whether there are other ways to go about this, and, if there are, which incurs the least overhead? Speed is a priority.
The screencasting program will be for recording game footage, although, if this does narrow down the options, I'm still open for any other suggestions that fall out of this scope. Knowledge isn't bad, after all.
Edit: I came across this article: Various methods for capturing the screen. It has introduced me to the Windows Media API way of doing it and the DirectX way of doing it. It mentions in the Conclusion that disabling hardware acceleration could drastically improve the performance of the capture application. I'm curious as to why this is. Could anyone fill in the missing blanks for me?
Edit: I read that screencasting programs such as Camtasia use their own capture driver. Could someone give me an in-depth explanation on how it works, and why it is faster? I may also need guidance on implementing something like that, but I'm sure there is existing documentation anyway.
Also, I now know how FRAPS records the screen. It hooks the underlying graphics API to read from the back buffer. From what I understand, this is faster than reading from the front buffer, because you are reading from system RAM, rather than video RAM. You can read the article here.
This is what I use to collect single frames, but if you modify this and keep the two targets open all the time then you could "stream" it to disk using a static counter for the file name. - I can't recall where I found this, but it has been modified, thanks to whoever!
void dump_buffer()
{
IDirect3DSurface9* pRenderTarget=NULL;
IDirect3DSurface9* pDestTarget=NULL;
const char file[] = "Pickture.bmp";
// sanity checks.
if (Device == NULL)
return;
// get the render target surface.
HRESULT hr = Device->GetRenderTarget(0, &pRenderTarget);
// get the current adapter display mode.
//hr = pDirect3D->GetAdapterDisplayMode(D3DADAPTER_DEFAULT,&d3ddisplaymode);
// create a destination surface.
hr = Device->CreateOffscreenPlainSurface(DisplayMde.Width,
DisplayMde.Height,
DisplayMde.Format,
D3DPOOL_SYSTEMMEM,
&pDestTarget,
NULL);
//copy the render target to the destination surface.
hr = Device->GetRenderTargetData(pRenderTarget, pDestTarget);
//save its contents to a bitmap file.
hr = D3DXSaveSurfaceToFile(file,
D3DXIFF_BMP,
pDestTarget,
NULL,
NULL);
// clean up.
pRenderTarget->Release();
pDestTarget->Release();
}
EDIT: I can see that this is listed under your first edit link as "the GDI way". This is still a decent way to go even with the performance advisory on that site, you can get to 30fps easily I would think.
From this comment (I have no experience doing this, I'm just referencing someone who does):
HDC hdc = GetDC(NULL); // get the desktop device context
HDC hDest = CreateCompatibleDC(hdc); // create a device context to use yourself
// get the height and width of the screen
int height = GetSystemMetrics(SM_CYVIRTUALSCREEN);
int width = GetSystemMetrics(SM_CXVIRTUALSCREEN);
// create a bitmap
HBITMAP hbDesktop = CreateCompatibleBitmap( hdc, width, height);
// use the previously created device context with the bitmap
SelectObject(hDest, hbDesktop);
// copy from the desktop device context to the bitmap device context
// call this once per 'frame'
BitBlt(hDest, 0,0, width, height, hdc, 0, 0, SRCCOPY);
// after the recording is done, release the desktop context you got..
ReleaseDC(NULL, hdc);
// ..delete the bitmap you were using to capture frames..
DeleteObject(hbDesktop);
// ..and delete the context you created
DeleteDC(hDest);
I'm not saying this is the fastest, but the BitBlt operation is generally very fast if you're copying between compatible device contexts.
For reference, Open Broadcaster Software implements something like this as part of their "dc_capture" method, although rather than creating the destination context hDest using CreateCompatibleDC they use an IDXGISurface1, which works with DirectX 10+. If there is no support for this they fall back to CreateCompatibleDC.
To change it to use a specific application, you need to change the first line to GetDC(game) where game is the handle of the game's window, and then set the right height and width of the game's window too.
Once you have the pixels in hDest/hbDesktop, you still need to save it to a file, but if you're doing screen capture then I would think you would want to buffer a certain number of them in memory and save to the video file in chunks, so I will not point to code for saving a static image to disk.
I wrote a video capture software, similar to FRAPS for DirectX applications. The source code is available and my article explains the general technique. Look at http://blog.nektra.com/main/2013/07/23/instrumenting-direct3d-applications-to-capture-video-and-calculate-frames-per-second/
Respect to your questions related to performance,
DirectX should be faster than GDI except when you are reading from the frontbuffer which is very slow. My approach is similar to FRAPS (reading from backbuffer). I intercept a set of methods from Direct3D interfaces.
For video recording in realtime (with minimal application impact), a fast codec is essential. FRAPS uses it's own lossless video codec. Lagarith and HUFFYUV are generic lossless video codecs designed for realtime applications. You should look at them if you want to output video files.
Another approach to recording screencasts could be to write a Mirror Driver. According to Wikipedia: When video mirroring is active, each time the system draws to the primary video device at a location inside the mirrored area, a copy of the draw operation is executed on the mirrored video device in real-time. See mirror drivers at MSDN: http://msdn.microsoft.com/en-us/library/windows/hardware/ff568315(v=vs.85).aspx.
I use d3d9 to get the backbuffer, and save that to a png file using the d3dx library:
IDirect3DSurface9 *surface ;
// GetBackBuffer
idirect3ddevice9->GetBackBuffer(0, 0, D3DBACKBUFFER_TYPE_MONO, &surface ) ;
// save the surface
D3DXSaveSurfaceToFileA( "filename.png", D3DXIFF_PNG, surface, NULL, NULL ) ;
SAFE_RELEASE( surface ) ;
To do this you should create your swapbuffer with
d3dpps.SwapEffect = D3DSWAPEFFECT_COPY ; // for screenshots.
(So you guarantee the backbuffer isn't mangled before you take the screenshot).
In my Impression, the GDI approach and the DX approach are different in its nature.
painting using GDI applies the FLUSH method, the FLUSH approach draws the frame then clear it and redraw another frame in the same buffer, this will result in flickering in games require high frame rate.
WHY DX quicker?
in DX (or graphics world), a more mature method called double buffer rendering is applied, where two buffers are present, when present the front buffer to the hardware, you can render to the other buffer as well, then after the frame 1 is finished rendering, the system swap to the other buffer( locking it for presenting to hardware , and release the previous buffer ), in this way the rendering inefficiency is greatly improved.
WHY turning down hardware acceleration quicker?
although with double buffer rendering, the FPS is improved, but the time for rendering is still limited. modern graphic hardware usually involves a lot of optimization during rendering typically like anti-aliasing, this is very computation intensive, if you don't require that high quality graphics, of course you can just disable this option. and this will save you some time.
I think what you really need is a replay system, which I totally agree with what people discussed.
I wrote a class that implemented the GDI method for screen capture. I too wanted extra speed so, after discovering the DirectX method (via GetFrontBuffer) I tried that, expecting it to be faster.
I was dismayed to find that GDI performs about 2.5x faster. After 100 trials capturing my dual monitor display, the GDI implementation averaged 0.65s per screen capture, while the DirectX method averaged 1.72s. So GDI is definitely faster than GetFrontBuffer, according to my tests.
I was unable to get Brandrew's code working to test DirectX via GetRenderTargetData. The screen copy came out purely black. However, it could copy that blank screen super fast! I'll keep tinkering with that and hope to get a working version to see real results from it.
For C++ you can use: http://www.pinvoke.net/default.aspx/gdi32/BitBlt.html
This may hower not work on all types of 3D applications/video apps. Then this link may be more useful as it describes 3 different methods you can use.
Old answer (C#):
You can use System.Drawing.Graphics.Copy, but it is not very fast.
A sample project I wrote doing exactly this: http://blog.tedd.no/index.php/2010/08/16/c-image-analysis-auto-gaming-with-source/
I'm planning to update this sample using a faster method like Direct3D: http://spazzarama.com/2009/02/07/screencapture-with-direct3d/
And here is a link for capturing to video: How to capture screen to be video using C# .Net?
You want the Desktop Duplication API (available since Windows 8). That is the officially recommended way of doing it, and it's also the most CPU efficient.
One nice feature it has for screencasting is that it detects window movement, so you can transmit block deltas when windows get moved around, instead of raw pixels. Also, it tells you which rectangles have changed, from one frame to the next.
The Microsoft example code is quite complex, but the API is actually simple and easy to use. I've put together an example project that is much simpler:
Simplified Sample Code
WindowsDesktopDuplicationSample
Microsoft References
Desktop Duplication API
Official example code (my example above is a stripped down version of this)
A few things I've been able to glean: apparently using a "mirror driver" is fast though I'm not aware of an OSS one.
Why is RDP so fast compared to other remote control software?
Also apparently using some convolutions of StretchRect are faster than BitBlt
http://betterlogic.com/roger/2010/07/fast-screen-capture/comment-page-1/#comment-5193
And the one you mentioned (fraps hooking into the D3D dll's) is probably the only way for D3D applications, but won't work with Windows XP desktop capture. So now I just wish there were a fraps equivalent speed-wise for normal desktop windows...anybody?
(I think with aero you might be able to use fraps-like hooks, but XP users would be out of luck).
Also apparently changing screen bit depths and/or disabling hardware accel. might help (and/or disabling aero).
https://github.com/rdp/screen-capture-recorder-program includes a reasonably fast BitBlt based capture utility, and a benchmarker as part of its install, which can let you benchmark BitBlt speeds to optimize them.
VirtualDub also has an "opengl" screen capture module that is said to be fast and do things like change detection http://www.virtualdub.org/blog/pivot/entry.php?id=290
You can try the c++ open source project WinRobot #git, a powerful screen capturer
CComPtr<IWinRobotService> pService;
hr = pService.CoCreateInstance(__uuidof(ServiceHost) );
//get active console session
CComPtr<IUnknown> pUnk;
hr = pService->GetActiveConsoleSession(&pUnk);
CComQIPtr<IWinRobotSession> pSession = pUnk;
// capture screen
pUnk = 0;
hr = pSession->CreateScreenCapture(0,0,1280,800,&pUnk);
// get screen image data(with file mapping)
CComQIPtr<IScreenBufferStream> pBuffer = pUnk;
Support :
UAC Window
Winlogon
DirectShowOverlay
Screen Recording can be done in C# using VLC API. I have done a sample program to demonstrate this. It uses LibVLCSharp and VideoLAN.LibVLC.Windows libraries. You could achieve many more features related to video rendering using this cross platform API.
For API documentation see: LibVLCSharp API Github
using System;
using System.IO;
using System.Reflection;
using System.Threading;
using LibVLCSharp.Shared;
namespace ScreenRecorderNetApp
{
class Program
{
static void Main(string[] args)
{
Core.Initialize();
using (var libVlc = new LibVLC())
using (var mediaPlayer = new MediaPlayer(libVlc))
{
var media = new Media(libVlc, "screen://", FromType.FromLocation);
media.AddOption(":screen-fps=24");
media.AddOption(":sout=#transcode{vcodec=h264,vb=0,scale=0,acodec=mp4a,ab=128,channels=2,samplerate=44100}:file{dst=testvlc.mp4}");
media.AddOption(":sout-keep");
mediaPlayer.Play(media);
Thread.Sleep(10*1000);
mediaPlayer.Stop();
}
}
}
}
This might not be the fastest method, but it is leightweight and easy to use. The image is returned as an integer array containing the RGB colors.
#define WIN32_LEAN_AND_MEAN
#define VC_EXTRALEAN
#include <Windows.h>
int* screenshot(int& width, int& height) {
HDC hdc = GetDC(NULL); // get the desktop device context
HDC cdc = CreateCompatibleDC(hdc); // create a device context to use yourself
height = (int)GetSystemMetrics(SM_CYVIRTUALSCREEN); // get the width and height of the screen
width = 16*height/9; // only capture left monitor for dual screen setups, for both screens use (int)GetSystemMetrics(SM_CXVIRTUALSCREEN);
HBITMAP hbitmap = CreateCompatibleBitmap(hdc, width, height); // create a bitmap
SelectObject(cdc, hbitmap); // use the previously created device context with the bitmap
BITMAPINFOHEADER bmi = { 0 };
bmi.biSize = sizeof(BITMAPINFOHEADER);
bmi.biPlanes = 1;
bmi.biBitCount = 32;
bmi.biWidth = width;
bmi.biHeight = -height; // flip image upright
bmi.biCompression = BI_RGB;
bmi.biSizeImage = 3*width*height;
BitBlt(cdc, 0, 0, width, height, hdc, 0, 0, SRCCOPY); // copy from desktop device context to bitmap device context
ReleaseDC(NULL, hdc);
int* image = new int[width*height];
GetDIBits(cdc, hbitmap, 0, height, image, (BITMAPINFO*)&bmi, DIB_RGB_COLORS);
DeleteObject(hbitmap);
DeleteDC(cdc);
return image;
}
The above code combines this answer and this answer.
Example on how to use it:
int main() {
int width=0, height=0;
int* image = screenshot(width, height);
// access pixel colors for position (x|y)
const int x=0, y=0;
const int color = image[x+y*width];
const int red = (color>>16)&255;
const int green = (color>> 8)&255;
const int blue = color &255;
delete[] image;
}
i myself do it with directx and think it's as fast as you would want it to be. i don't have a quick code sample, but i found this which should be useful. the directx11 version should not differ a lot, directx9 maybe a little more, but thats the way to go
DXGI Desktop Capture
Project that captures the desktop image with DXGI duplication. Saves the captured image to the file in different image formats (*.bmp; *.jpg; *.tif).
This sample is written in C++. You also need some experience with DirectX (D3D11, D2D1).
What the Application Can Do
If you have more than one desktop monitor, you can choose.
Resize the captured desktop image.
Choose different scaling modes.
You can show or hide the mouse icon in the output image.
You can rotate the image for the output picture, or leave it as default.
I realize the following suggestion doesn't answer your question, but the simplest method I have found to capture a rapidly-changing DirectX view, is to plug a video camera into the S-video port of the video card, and record the images as a movie. Then transfer the video from the camera back to an MPG, WMV, AVI etc. file on the computer.
Windows.Graphics.Capture
Enables apps to capture environments, application windows, and displays in a secure, easy to use way with the use of a system picker UI control.
https://blogs.windows.com/windowsdeveloper/2019/09/16/new-ways-to-do-screen-capture/