Reading out specific video frame using FFMPEG API - c++

I read frames from video stream in FFMPEG using this loop:
while(av_read_frame(pFormatCtx, &packet)>=0) {
// Is this a packet from the video stream?
if(packet.stream_index==videoStream) {
// Decode video frame
avcodec_decode_video2(pCodecCtx,pFrame,&frameFinished,&packet);
// Did we get a video frame?
if(frameFinished) {
sws_scale(img_convert_context ,pFrame->data,pFrame->linesize,0,
pCodecCtx->height, pFrameRGBA->data, pFrameRGBA->linesize);
printf("%s\n","Frame read finished ");
ExportFrame(pFrameRGBA->data[0]);
break;
}
}
// Save the frame to disk
}
printf("%s\n","Read next frame ");
// Free the packet that was allocated by av_read_frame
av_free_packet(&packet);
}
So in this way the stream is read sequentially.What I want is to have a random access to the frame to be able reading a specific frame (by frame number).How is it done?

You may want to look
int av_seek_frame(AVFormatContext *s, int stream_index, int64_t timestamp,
int flags);
The above api will seek to the keyframe at give timestamp. After seeking you can read the frame. Also the tutorial below explain conversion between position and timestamp.
http://dranger.com/ffmpeg/tutorial07.html

Since most frames in a video depend on previous and next frames, in general, accessing random frames in a video is not straightforward. However, some frames, are encoded independently of any other frames, and occur regularly throughout the video. These frames are known as I-frames. Accessing these frames is straightforward through seeking.
If you want to "randomly" access any frame in the video, then you must:
Seek to the previous I-frame
Read the frames one by one until you get to the frame number that you want
You've already got the code for the second point, so all you need to do is take care of the first point and you're done. Here's an updated version of the Dranger tutorials that people often refer to -- it may be of help.

Related

Retrieve specific frame from video | Opencv VideoCapture | C++

I have a program that extracts every n-th frame of a video. The current way of doing it is to iterate trough the entire video and when we get to a desired frame we extract it a process it. But this is slow as we are going trough every frame when if we could just jump to the ones we desire it would be much faster. I have searched for solutions but I couldn't find one.
current slow code
while (true) {
cv::Mat frame;
vid >> frame;
if (frame.empty()) {
break;
}
//check if we need the frame
//process frame
}
Also tried other versions using stuff like vid.set(CAP_PROP_POS_FRAMES,frame_num)
but that also just seems to do the same thing like the code above but sets back the current frame to the original one so its even slower.

How do we skip frame, calculate number of frames and get current frame with video reader

I am using Opencv for video decoding now. I can play the video with no problem. However, I could not find a way to skip frame, get total number of frames and current frame number. Is there a way to use VideoReader like VideoCapture? I have gone through their api and could not get anything.
OpenCV is Open Source library. If you use VideoCapture then the object of this class has any methods:
To skip a frame you should read 2 times a frame.
To get total number of frames as follows:
cv::VideoCapture m_capture;
....
int totalFrames = m_capture.get(cv::CAP_PROP_FRAME_COUNT)
To get a current frame number:
int currentFrame = m_capture.get(cv::CAP_PROP_POS_FRAMES)
For detailed information visit the OpenCV documentation https://docs.opencv.org/3.1.0/d8/dfe/classcv_1_1VideoCapture.html.

FFmpeg resample audio while decoding

I am having a task to build a decoder that generates exactly 1 raw audio frame for 1 raw video frame, from an encoded mpegts network stream, so that users can use the API by calling getFrames() and receive exactly these two frames.
Currently I am reading with av_read_frame in a thread, decode as packets come, audio or video; collect until a video packet is hit. Problem is generally multiple audio packets are received before video is seen.
av_read_frame is blocking, returns when certain amount of audio data is collected (1152 samples for mp2); and decoding that packet gives a raw AVFrame having duration of T (depends on samplerate); whereas the video frame generally has duration bigger than T (depends on fps), so multiple audio frames are received before it.
I was guessing I have to find a way to merge collected audio frames into 1 single frame just when video is hit. Also resampling and setting timestamp to align with video is needed I guess. I don't know if this is even valid though.
What is the smoothest way to sync video and audio in this manner ?

How to insert a key frame(Iframe) to a h.264 video stream in ffmpeg C++ api?

I have a real time video stream, and want to cut some video clips from it by accurate timestamp(pts).
When I receiver an avpacket, I decode it, and do something and cache the avpacket. I don't want to re-encode all avpackets, it cost cpu resource.
There are many gop structure in H.264 stream, usually we should cut the video begin at the key frame, and end at the key frame. Otherwise the front some frames in the video clip would display error.
Now I use av_write_frame to make avpacket to video. But sometimes the length of gop is very long, such as it could be 250, 8.3s(30 frame per second). It means the distance between two I-frame could be 250 frames. The video clip is short, I don't want to add too many unused frames.
How should I do? I think i should insert a i-frame at the start position of video clip. Could I change a p-frame to i-frame?
Thanks your reading!
This is not possible in the generic case, but may be in specific cases. Even then, there are no open source/free tools to do this, and I am unaware of any commercial tools. The reason I say it is not possible in the generic case is each frame can reference up to 16 other frames. So you can not just replace a single frame, You will need to replace all referenced frames. Doing this will likely take almost as much CPU as encoding the whole GOP.

Synchronizing FFMPEG video frames using PTS

I'm attempting to synchronize the frames decoded from an MP4 video. I'm using the FFMPEG libraries. I've decoded and stored each frame and successfully displayed the video over an OPENGL plane.
I've started a timer just before cycling through the frames; the aim being to synchronize the Video correctly. I then compare the PTS of each frame against this timer. I stored the PTS received from the packet during decoding.
What is displayed within my application does not seem to play at the rate I expect. It plays faster than the original video file would within a media player.
I am inexperienced with FFMPEG and programming video in general. Am I tackling this the wrong way?
Here is an example of what I'm attempting to do
FrameObject frameObject = frameQueue.front();
AVFrame frame = *frameObject.pFrame;
videoClock += dt;
if(videoClock >= globalPTS)
{
//Draw the Frame to a texture
DrawFrame(&frame, frameObject.m_pts);
frameQueue.pop_front();
globalPTS = frameObject.m_pts;
}
Please note I'm using C++, Windows, Opengl, FFMPEG and the VS2010 IDE
First off, Use int64_t pts = av_frame_get_best_effort_timestamp(pFrame) to get the pts. Second you must make sure both streams you are syncing use the same time base. The easiest way to do this is convert everything to AV_TIME_BASE_Q. pts = av_rescale_q ( pts, formatCtx->streams[videoStream]->time_base, AV_TIME_BASE_Q ); In this format, pts is in nanoseconds.