Why codecs x264/x265 ignores pts and dts of input frame? - c++

Question

I'm trying to encode images from a webcam with libx265 (libx264 tried earlier) ...The webcam can not shoot with stable FPS because of the different amount of light entering the matrix and, as a result, different delays. Therefore, I count the fps and dts of the incoming frame and set these values for the corresponding parameters of the x265_image object, and init the encoder fpsNum with 1000 and fpsDenom with 1 (for millisecond timebase).
The problem is that the encoder ignores pts and dts of input image and encodes at 1000 fps! The same trick with timebase produces smooth record with libvpx. Why it does not work with x264/x265 codecs?
Here is parameters initialization:
...
error = (x265_param_default_preset(param, "fast", "zerolatency") != 0);
if(!error){
param->sourceWidth = width;
param->sourceHeight = height;
param->frameNumThreads = 1;
param->fpsNum = 1000;
param->fpsDenom = 1;
// Intra refres:
param->keyframeMax = 15;
param->intraRefine = 1;
// Rate control:
param->rc.rateControlMode = X265_RC_CQP;
param->rc.rfConstant = 12;
param->rc.rfConstantMax = 48;
// For streaming:
param->bRepeatHeaders = 1;
param->bAnnexB = 1;
encoder = x265_encoder_open(param);
...
}
...
Here is frame adding function:
bool hevc::Push(unsigned char *data){
if(!error){
std::lock_guard<std::mutex> lock(m_framestack);
if( timer > 0){
framestack.back()->dts = clock() - timer;
timer+= framestack.back()->dts;
}
else{timer = clock();}
x265_picture *picture = x265_picture_alloc();
if( picture){
x265_picture_init(param, picture);
picture->height = param->sourceHeight;
picture->stride[0] = param->sourceWidth;
picture->stride[1] = picture->stride[2] = picture->stride[0] / 2;
picture->planes[0] = new char[ luma_size];
picture->planes[1] = new char[chroma_size];
picture->planes[2] = new char[chroma_size];
colorspaces::BGRtoI420(param->sourceWidth, param->sourceHeight, data, (byte*)picture->planes[0], (byte*)picture->planes[1], (byte*)picture->planes[2]);
picture->pts = picture->dts = 0;
framestack.emplace_back(picture);
}
else{error = true;}
}
return !error;
}
Global PTS is increasing right after x265_encoder_encode call:
pts+= pic_in->dts; and sets as a pts of new image from framestack queue when it comes to encoder.
Can the x265/x264 codecs encode at variable fps? How to configure it if yes?

Answer 1

I don't know about x265 but in x264 to encode variable frame rate (VFR) video you should enable x264_param_t.b_vfr_input option which was disabled by your zerolatency tuning (VFR encode need 1 frame latency). Also at least in x264 timebase should be in i_timebase_num/i_timebase_den and i_fps_num/i_fps_den to be average fps (or keep default 25/1 if you don't know fps) or you will broke ratecontrol.