How to capture DirectX animations as videos - c++

Question

I am trying to record an animation that I created using DirectX 11 as a video, which I can present whenever needed (without re-rendering). I am still learning about DirectX and windows API.
This's what I've done so far, I was able to capture animation frames using DirectXTk and following this post. After that I'm using OpenCV to collect frames from disk, and create a video. Is there a way to merge this process? That way I'd be able to append frames into a video file right after img capture.
Code for animation capture:
static int Frame_Number;
void D3D::screenCapture() {
//For each Call to Present() do the following:
//Get Device
//ID3D11Device* baseDevice;
HRESULT gd = m_swapChain->GetDevice(__uuidof(ID3D11Device), (void**)&m_device);
assert(gd == S_OK);
//Get context
//ID3D11DeviceContext* context;
m_device->GetImmediateContext(&m_deviceContext);
//get pointer to back buffer
ID3D11Texture2D* backbufferTex;
HRESULT gb = m_swapChain->GetBuffer(0, __uuidof(ID3D11Texture2D), (LPVOID*)&backbufferTex);
assert(gb == S_OK);
//Set-up Directory
std::wstringstream Image_Directory;
Image_Directory << L"path to directory/screenShots" << Frame_Number << L".JPG";
//Capture Frame
REFGUID GUID_ContainerFormatJpeg{ 0x19e4a5aa, 0x5662, 0x4fc5, 0xa0, 0xc0, 0x17, 0x58, 0x2, 0x8e, 0x10, 0x57 };
HRESULT hr = DirectX::SaveWICTextureToFile(m_deviceContext, backbufferTex, GUID_ContainerFormatJpeg, Image_Directory.str().c_str());
assert(hr == S_OK);
Frame_Number = Frame_Number + 1;
}
I call this function after I present the rendered scene to the screen. After that I use a python script to create a video from the captured frames.
This's not optimal, especially in the case of rendering many animations. It will take forever, I would like to eliminate the reading and writing to disk. Is there a way to get frames from
SaveWICTextureToFile
that I can push in a video in a sequential manner.
How could one accomplish this?
I would really appreciate any help or pointers.

Answer 1

Possible, but relatively hard, many pages of code.
Here’s a tutorial written by Microsoft. You gonna need to change following there.
Integrate with Direct3D. To do that, call MFCreateDXGIDeviceManager, then IMFDXGIDeviceManager.ResetDevice, then pass that IMFDXGIDeviceManager interface in MF_SINK_WRITER_D3D_MANAGER attribute when creating the sink writer. Also, don’t forget to set MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS to TRUE, you don’t want software encoders, they are way too slow.
Replace the video codec, use MFVideoFormat_H264 instead of MFVideoFormat_WMV3, and *.mp4 extension for the output file.
The sample code encodes video frames provided in system memory. Instead, you should supply your video frames in VRAM. Every time your D3D app renders a frame, create a new D3D texture, copy your render target into that new texture with CopyResource, then call MFCreateDXGISurfaceBuffer. This will create IMFMediaBuffer object referencing a video frame in VRAM. You can then submit that video frame to the sink writer, and it should do the rest of the things automagically.
If you’ll manage to implement that correctly, Media Foundation framework gonna use a proprietary hardware transform to convert RGBA textures into NV12 textures on GPU, then use another proprietary hardware transform to encode NV12 into h.264, download encoded video samples from VRAM to system RAM as soon as ready, and append these encoded samples into mpeg-4 container file on disk.
Both of the above transforms are implemented by GPU, in hardware. All 3 GPU vendors have hardware for that, and they ship media foundation transform DLLs to use their hardware as a part of their GPU drivers.