ffmpeg how to efficiently decode the video frame? - c++

Question

Here is the code I used to decode a rtsp stream in a worker thread:
while(1)
{
// Read a frame
if(av_read_frame(pFormatCtx, &packet)<0)
break; // Frame read failed (e.g. end of stream)
if(packet.stream_index==videoStream)
{
// Is this a packet from the video stream -> decode video frame
int frameFinished;
avcodec_decode_video2(pCodecCtx,pFrame,&frameFinished,&packet);
// Did we get a video frame?
if (frameFinished)
{
if (LastFrameOk == false)
{
LastFrameOk = true;
}
// Convert the image format (init the context the first time)
int w = pCodecCtx->width;
int h = pCodecCtx->height;
img_convert_ctx = ffmpeg::sws_getCachedContext(img_convert_ctx, w, h, pCodecCtx->pix_fmt, w, h, ffmpeg::PIX_FMT_RGB24, SWS_BICUBIC, NULL, NULL, NULL);
if (img_convert_ctx == NULL)
{
printf("Cannot initialize the conversion context!\n");
return false;
}
ffmpeg::sws_scale(img_convert_ctx, pFrame->data, pFrame->linesize, 0, pCodecCtx->height, pFrameRGB->data, pFrameRGB->linesize);
// Convert the frame to QImage
LastFrame = QImage(w, h, QImage::Format_RGB888);
for (int y = 0; y < h; y++)
memcpy(LastFrame.scanLine(y), pFrameRGB->data[0] + y*pFrameRGB->linesize[0], w * 3);
LastFrameOk = true;
} // frameFinished
} // stream_index==videoStream
av_free_packet(&packet); // Free the packet that was allocated by av_read_frame
}
I followed the ffmpeg's tutorial and used a while loop to read the packet and decode the video.
But is there a more efficient way to do this, like a event-triggered function when there is packet received?

Answer 1

I haven't seen any event driven approach for reading frames, but what is the purpose of reading RTSP stream? But I can give some recommendations for improving performance. First of all, you may add a very short sleep in your loop (e.g. Sleep(1);). In your program, if your purpose is to:
Display images to the user: Don't use conversion to RGB, after decoding, the resulting frame is in YUV420P format which can be directly displayed to the user using GPU without any CPU usage. Almost all graphics cards support YUV420P (or YV12) format. Conversion to RGB is a highly CPU-consuming operation, especially for large images.
Record (save) to disk: I you want to record the stream to play it later, there is no need to decode the frames. You may use OpenRTSP to record directly to the disk without any CPU usage.
Process realtime images: You may find alternative algorithms to process on YUV420P format instead of RGB. The Y plane in YUV420P is actually a grayscale version of the colored RGB images.