Low quality H.265 Media Foundation encoding? - c++

Question

I'm trying to encode video with MF H.265, and no matter what I try, the quality is always lower than the same-settings video procuded by non MF encoders, like what VideoPad uses (say, ffmpeg) at the same 4000 bitrate.
Videopad produces this video of a swimming boy. My app produces this video. The sky in my app is clearly worse at a 6K bitrate, where the VideoPad is at 1K.
pMediaTypeOutVideo->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video);
pMediaTypeOutVideo->SetGUID(MF_MT_SUBTYPE, MFVideoFormat_HEVC);
pMediaTypeOutVideo->SetUINT32(MF_MT_AVG_BITRATE, 4000000);
MFSetAttributeSize(pMediaTypeOutVideo, MF_MT_FRAME_SIZE, 1920,1080);
MFSetAttributeRatio(pMediaTypeOutVideo, MF_MT_FRAME_RATE, 25, 1);
MFSetAttributeRatio(pMediaTypeOutVideo, MF_MT_PIXEL_ASPECT_RATIO, 1, 1);
pMediaTypeOutVideo->SetUINT32(MF_MT_INTERLACE_MODE, MFVideoInterlace_Progressive);
pMediaTypeOutVideo->SetUINT32(MF_MT_VIDEO_NOMINAL_RANGE, MFNominalRange_Wide);
CComPtr<ICodecAPI> ca;
hr = pSinkWriter->GetServiceForStream(OutVideoStreamIndex, GUID_NULL, __uuidof(ICodecAPI), (void**)&ca);
if (ca)
{
if (true)
{
VARIANT v = {};
v.vt = VT_BOOL;
v.boolVal = VARIANT_FALSE;
ca->SetValue(&CODECAPI_AVLowLatencyMode, &v);
}
if (true)
{
VARIANT v = {};
v.vt = VT_UI4;
v.ulVal = 100;
hr = ca->SetValue(&CODECAPI_AVEncCommonQualityVsSpeed, &v);
}
if (true)
{
VARIANT v = {};
v.vt = VT_UI4;
v.ulVal = eAVEncCommonRateControlMode_Quality;
ca->SetValue(&CODECAPI_AVEncCommonRateControlMode, &v);
if (true)
{
VARIANT v = {};
v.vt = VT_UI4;
v.ulVal = 100;
ca->SetValue(&CODECAPI_AVEncCommonQuality, &v);
}
}
}
No matter what, the quality at 4000k remains inferior to what ffmpeg produces. Also the eAVEncCommonRateControlMode_Quality and CODECAPI_AVEncCommonQuality does not seem to take effect (it works in H.264). The only way to see better quality is to raise the bitrate.
Also, the speed parameter does not seem to affect the quality or the encoder speed.
Even at 1000k Videopad produced video does not have pixelizing in the sky. Of course, its speed is 1/100.
Is the Media Foundation encoders worse than ffmpeg's? What am I missing?
Edit: Rendering with software (MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS to FALSE) is also equally bad.
Update: Tried it ot my laptop with an AMD hardware encoder. Similar problem, when the bitrate is low the quality is awful.

Answer 1

I checked the two videos with MediaInfo and it is obvious that they use different HEVC profile, which should be the main reason that affects the quality of the NVidia encoded video.
Here is the comparison screenshot:
You can try setting the MF_MT_VIDEO_PROFILE in your input IMFMediaType to eAVEncH265VProfile_Main_420_8. Additionally the MF_MT_MPEG2_LEVEL should be set accordingly as well. For instance to eAVEncH265VLevel4_1.
You might also consider using IClassFactory approach in order to guarantee the correct order of calling the ICodecAPI methods.

Answer 2

Perhaps simply because software encoders are better than hardware encoders.
If i take a look at this : https://www.techspot.com/article/1131-hevc-h256-enconding-playback/page7.html, I also would confirm hardware nvidia encoder is bad, comparing to x265 (date 2016).
I can't investigate a bit more, but from what I see about your Post :
pMediaTypeOutVideo->SetUINT32(MF_MT_VIDEO_NOMINAL_RANGE, MFNominalRange_Wide); -> Why not MFNominalRange_Normal ?
Are there others ICodecAPI from NVidia encoder, than CODECAPI_AVLowLatencyMode/CODECAPI_AVEncCommonQualityVsSpeed/CODECAPI_AVEncCommonRateControlMode...
Where is the 2 passes encoding parameter ?
I found at least three forums saying NVidia HEVC encoder blurs images. And you confirm this, so... Fake news or not, (date 2018/2019)..
From NVidia https://developer.nvidia.com/nvidia-video-codec-sdk#NVENCFeatures (date undefined).
I don't understand nothing about this diagram, but NVidia seems to pretend they are the best... So Fake news or not.
EDIT
Rendering with software (MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS to FALSE) is also equally bad.
Can you confirm in software mode, the H.265 / HEVC Video Encoder used ?
If so, did you play with Codec Properties ?
CODECAPI_AVEncCommonRateControlMode
CODECAPI_AVEncCommonMeanBitRate
CODECAPI_AVEncCommonBufferSize
CODECAPI_AVEncCommonMaxBitRate
CODECAPI_AVEncMPVGOPSize
CODECAPI_AVLowLatencyMode
CODECAPI_AVEncCommonQualityVsSpeed
CODECAPI_AVEncVideoForceKeyFrame
CODECAPI_AVEncVideoEncodeQP
CODECAPI_AVEncVideoMinQP
CODECAPI_AVEncVideoMaxQP
CODECAPI_VideoEncoderDisplayContentType
CODECAPI_AVEncNumWorkerThreads