Trying to use a MFT in Media Foundation Encoding - c++

Question

The idea is to use a Media Foundation Transform, such as the Video Stabilization MFT while transcoding a video with Media Foundation.
When not using an MFT, the code works fine.
Create IMFSourceReader for the source file - OK
Create IMFSinkWriter for the target file - OK
Add a stream to the writer describing the Video - OK
Add the audio stream - OK
Set input types for video and audio - OK
Loop to read samples and send them to the sink writer, OK.
When using the MFT, these are the facts. To create the MFT (error checking removed):
CComPtr<IMFTransform> trs;
trs.CoCreateInstance(CLSID_CMSVideoDSPMFT);
std::vector<DWORD> iids;
std::vector<DWORD> oods;
DWORD is = 0, os = 0;
hr = trs->GetStreamCount(&is, &os);
iids.resize(is);
oods.resize(os);
hr = trs->GetStreamIDs(is, iids.data(), os, oods.data());
CComPtr<IMFMediaType> ptype;
CComPtr<IMFMediaType> ptype2;
MFCreateMediaType(&ptype);
MFCreateMediaType(&ptype2);
SourceVideoType->CopyAllItems(ptype);
SourceVideoType->CopyAllItems(ptype2);
ptype->SetUINT32(MF_VIDEODSP_MODE, MFVideoDSPMode_Stabilization);
// LogMediaType(ptype);
ptype2->SetUINT32(MF_VIDEODSP_MODE, MFVideoDSPMode_Stabilization);
// LogMediaType(ptype2);
hr = trs->SetInputType(iids[0], ptype, 0);
auto hr2 = trs->SetOutputType(oods[0], ptype2, 0);
if (SUCCEEDED(hr) && SUCCEEDED(hr2))
{
VideoStabilizationMFT = trs;
}
This code works - the MFT is successfully configured. However, in my sample processing loop:
// pSample = sample got from the reader
CComPtr<IMFSample> pSample2;
LONGLONG dur = 0, tim = 0;
pSample->GetSampleDuration(&dur);
pSample->GetSampleTime(&tim);
trs->ProcessInput(0, pSample, 0);
MFT_OUTPUT_STREAM_INFO si = {};
trs->GetOutputStreamInfo(0, &si);
// Create pSample2
MFCreateSample(&pSample2);
CComPtr<IMFMediaBuffer> bb;
MFCreateMemoryBuffer(si.cbSize, &bb);
pSample2->AddBuffer(bb);
DWORD st = 0;
hr = trs->ProcessOutput(0, 1, &db, &st);
This last call fails initially with MF_E_TRANSFORM_NEED_MORE_INPUT, I can understand that the MFT needs more than one sample to achieve stabilizization, so I skip this sample for the writer.
When the call succeeds, I get a sample with no time or duration. Even If I set the time and duration manually, the sink writer fails with E_INVALIDARG.
What do I miss?

Answer 1

With this source code I provide, the sink writer returns S_OK :
VideoStabilizationMFT
if Microsoft is reading this, what are these guids from CLSID_CMSVideoDSPMFT ?
Guid : 44A4AB4B-1D0C-4181-9293-E2F37680672E : VT_UI4 = 4
Guid : 8252D735-8CB3-4A2E-A296-894E7B738059 : VT_R8 = 0.869565
Guid : 9B2DEAFE-37EC-468C-90FF-024E22BD6BC6 : VT_UI4 = 0
Guid : B0052692-FC62-4F21-A1DD-B9DFE1CEB9BF : VT_R8 = 0.050000
Guid : C8DA7888-14AA-43AE-BDF2-BF9CC48E12BE : VT_UI4 = 4
Guid : EF77D08F-7C9C-40F3-9127-96F760903367 : VT_UI4 = 0
Guid : F67575DF-EA5C-46DB-80C4-CEB7EF3A1701 : VT_UI4 = 1
Microsoft, are you serious ?
according to this documentation : https://learn.microsoft.com/en-us/windows/win32/medfound/video-stabilization-mft
On Win10 :
Video Stabilization MFT is MF_SA_D3D11_AWARE, the documentation does not speak about this
Video Stabilization MFT can fallback to software mode, the documentation does not speak about this (see MF_SA_D3D11_AWARE)
Video Stabilization MFT has dynamic format change, the MFT_SUPPORT_DYNAMIC_FORMAT_CHANGE is not present on IMFTransform::GetAttributes
Video Stabilization MFT implements IMFGetService/IMFRealTimeClientEx/IMFShutdown, it is not in the documentation
Video Stabilization MFT only handles MFVideoFormat_NV12, the documentation speak about MEDIASUBTYPE_YUY2
The documentation tells to include Camerauicontrol.h, seriously...
Having said that, this MTF is really good doing stabilization...

Answer 2

This is strange, you set attribute on IMFMediaType, not on Video Stabilization MFT :
ptype->SetUINT32(MF_VIDEODSP_MODE, MFVideoDSPMode_Stabilization);
ptype2->SetUINT32(MF_VIDEODSP_MODE, MFVideoDSPMode_Stabilization);
Should be :
Call IMFTransform::GetAttributes on the video stabilization MFT to get an IMFAttributes pointer.
Call IMFAttributes::SetUINT32 to set the attribute.
MF_VIDEODSP_MODE attribute