I'm trying to play vc1 coded video in matroska container.
For that I'm using ffmpeg's av_read_frame function and a sertain video driver, which requires AVPacket's data to be prefixed by PES header.
In AVPacket only dts field is valid, pts is AV_NOPTS_VALUE. I write dts value into PES header instead of pts.
Video driver logs constant framerate change from 23976 to 24000 and vice versa. The video jerks. Although I put framerate into PES header (value 23976 is what ffmpeg's probing gives), but apparently, it's changing according to current packet's pts.
I tried to look at AVCodecParserContext's pts_dts_delta and dts_ref_dts_delta but they are of AV_NOPTS_VALUE, its pts and dts are the same as of AVPacket
Please advise how to get proper pts values, or what to do to solve it.
Thanks.
EDIT:
I saw in ffplay.c they use av_frame_get_best_effort_timestamp but that's after decoding by ffmpeg's means, which I cannot afford.
Related
I have the MKV file. I am taking an audio stream from it and using ffmpeg and its opus decoder to decode audio packets into audio frames.
The problem is that the resulting frames have unexpected presentation timestamps (pts): during decoding, ffmpeg just copies pkt->pts to frame->pts, ignoring the fact that the time base for pkt->pts is 1/1000, while for frame->pts it is 1/48000 (which means they definitely cannot be the same).
Setting pkt_timebase of the audio decoder to 1/1000 does not help.
I also tried using libopus audio decoder instead of opus (which is considered experimental), but the issue is still there.
How to tell ffmpeg that the time base conversion is needed? Or setting frame->pts manually after decoding is the only option?
I have tried the same test with a really old ffmpeg version (3.0). It sets frame->pts to AV_NOPTS_VALUE, which seems more correct behavior than the current one. Can it be a bug in ffmpeg introduced sometime after version 3.0 and the frame->pts should always be AV_NOPTS_VALUE if the decoder cannot assign it by itself (both opus and libopus cannot, according to their caps_internal)?
The project to reproduce the issue is here. Here it prints pkt->pts (before decoding) and frame->pts (after decoding) and they happen to be the same.
I've a 16bit greyscale video stream from a LWIR (thermal camera) and I want to forward the stream over RTP without any compression.
gstreamer format is: video/x-raw,format=GRAY16_LE,width=640,height=520,framerate=9/1
But I can't find any plugin to transmit the data over RTP.
https://gstreamer.freedesktop.org/documentation/rtp/index.html?gi-language=c
Do you have an idea?
Thanks, Martin
Check for the specs of uncompressed video data over RTP:
https://www.rfc-editor.org/rfc/rfc4175
As you will notice your specific format is not covered by the specification.
Question: What does the Libav/FFmpeg decoding pipeline need in order to produce valid presentation timestamps (PTS) in the decoded AVFrames?
I'm decoding an H264 stream received via RTSP. I use Live555 to parse H264 and feed the stream to my LibAV decoder. Decoding and displaying is working fine, except I'm not using timestamp info and get some stuttering.
After getting a frame with avcodec_decode_video2, the presentation timestamp (PTS) is not set.
I need the PTS in order to find out for how long each frame needs to be displayed, and avoid any stuttering.
Notes on my pipeline
I get the SPS/PPS information via Live555, I copy these values to my AVCodecContext->extradata.
I also send the SPS and PPS to my decoder as NAL units, with the appended {0,0,0,1} startcode.
Live555 provides presentation timestamps for each packet, these are in most cases not monotonically increasing. The stream contains B-frames.
My AVCodecContext->time_base is not valid, value is 0/2.
Unclear:
Where exactly should I set the NAL PTS coming from my H264 sink (Live555)? As the AVPacket->dts, pts, none, or both?
Why is my time_basevalue not valid? Where is this information?
According to the RTP payload spec. It seems that
The RTP timestamp is set to the sampling timestamp of the content. A 90 kHz clock rate MUST be used.
Does this mean that I must always asume a 1/90000 timebase for the decoder? What if some other value is specified in the SPS?
Copy the live555 pts into the avpacket pts. Process the packet with avcodec_decode_video2, and then retrieve the pts from avframe->pkt_pts, these will be monotonically increasing.
There is no need to set anything in the codec context, apart from setting the SPS and PPS in the AVCodecContex extradata
You can find a good example in VLC's github:
Setting AVPacket pts: https://github.com/videolan/vlc/blob/master/modules/codec/avcodec/video.c#L983
Decoding AVPacket into AVFrame: https://github.com/videolan/vlc/blob/master/modules/codec/avcodec/video.c#L1014
Retrieving from AVFrame pts:
https://github.com/videolan/vlc/blob/master/modules/codec/avcodec/video.c#L1078
avcodec_decode_video2() reorders the frames so that decode order and presentation order is the same.
Even if you somehow convince ffmpeg to give you PTS on the decoded frame it should be the same as DTS.
//
// decode a video frame
//
avcodec_decode_video2
(
ctxt->video_st->codec,
frame,
&is_finished,
buffer
);
if (buffer->dts != AV_NOPTS_VALUE)
{
//
// you should end up here
//
pts = buffer->dts;
}
else
{
pts = 0;
}
//
// adjust time base
//
pts *= av_q2d(ctxt->video_st->time_base);
I received some video data via RTP / RTSP / SIP, the data is encoded by H264 and sent by a IP camera. I would like to convert H264 keyframe data into a picture and analyze whether it contains faces. I do not want to use FFMPEG such a huge library, just use libx264 and opencv can do it? How?
Thanks.
No, not possible. X264 can not decode (it is a h264 encoder only). It also can not encode jpeg/png. Ffmpeg is what you need. If it is too large, custom compile including only the features you need. And static link so unused functions are striped out.
I use live555 to receive RTP video frame (frame encoded in H264). I use Live555 open my local .sdp file to receive frame data. I just saw DummySink::afterGettingFrame was called ceaselessly。 if fReceiveBuffer in DummySink is correct, Why FFMPEG cannot decode the frame? My code is wrong?
Here is my Code Snippet:
http://paste.ubuntu.com/12529740/
the function avcodec_decode_video2 is always return failed , its value less than zero
fReceiveBuffer is present one video frame？
Oh, Here is my FFMPEG init code need to open related video decoder:
http://paste.ubuntu.com/12529760/
I read the document related H264 again， I found out that I-frame(IDR) need SPS/PPS separated by 0x00000001 insert into the header and decoder have a capacity to decode the frame correctly. Here is a related solution
FFmpeg can't decode H264 stream/frame data
Decoding h264 frames from RTP stream
and now, My App works fine, it can decode the frame and convert it to OSD Image for displaying to screen .