FFMPEG corrupted frame when streaming in UDP (MPEGTS) - c++

Question

I am working on a screen streaming program, till now, I can capture and encode my screen into a video perfectly.
However, whenever I try to stream it over LAN, and play using Mplayer, it is able to run in simple desktop (text document, etc...) but when I try to play video the frame stated to corrupted until I quit the video (this video cause the most corrupted frame https://www.youtube.com/watch?v=1La4QzGeaaQ&t=230s)
I am not really sure why when streaming, I got corrupted frame with the youtube video but when save it to a file (change URL from udp://192.168.1.xxx:YYYY to file name C:\Test.ts) there were no corrupted frame at all.
My Mplayer log:
`V: 0.0 1563/1563 51% 43% 0.0% 0 0
[h264 # 0000000001d04940]Invalid NAL unit 0, skipping.
[h264 # 0000000001d04940]error while decoding MB 23 51, bytestream -64
[h264 # 0000000001d04940]concealing 2066 DC, 2066 AC, 2066 MV errors in I frame
V: 0.0 1564/1564 51% 43% 0.0% 0 0
[h264 # 0000000001d04940]concealing 7598 DC, 7598 AC, 7598 MV errors in P frame
V: 0.0 1652/1652 50% 43% 0.0% 0 0
[h264 # 0000000001d04940]Invalid NAL unit 0, skipping.
[h264 # 0000000001d04940]error while decoding MB 26 49, bytestream -55
[h264 # 0000000001d04940]concealing 2303 DC, 2303 AC, 2303 MV errors in I frame
V: 0.0 1653/1653 50% 43% 0.0% 0 0
[h264 # 0000000001d04940]concealing 7727 DC, 7727 AC, 7727 MV errors in P frame
V: 0.0 1741/1741 49% 43% 0.0% 0 0
[h264 # 0000000001d04940]Invalid NAL unit 0, skipping.
[h264 # 0000000001d04940]error while decoding MB 65 62, bytestream -57
[h264 # 0000000001d04940]concealing 704 DC, 704 AC, 704 MV errors in I frame
V: 0.0 1742/1742 49% 43% 0.0% 0 0`
Stream initialize code
static void Stream_ini(const char *Url, AVFormatContext *&ofmt_ctx, AVCodec *codec, AVCodecContext *c, AVStream *&out_stream)
{
int ret;
avformat_alloc_output_context2(&ofmt_ctx, NULL, "mpegts", Url);
out_stream = avformat_new_stream(ofmt_ctx, codec)
out_stream->codec = c;
av_dump_format(ofmt_ctx, 0, Url, 1);
if (ofmt_ctx->oformat->flags & AVFMT_GLOBALHEADER)
out_stream->codec->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
if (!(ofmt_ctx->flags & AVFMT_NOFILE))
{
ret = avio_open(&ofmt_ctx->pb, Url, AVIO_FLAG_WRITE);
if (ret < 0)
{
printf("Could not open output URL '%s'", Url);
return;
}
}
ret = avformat_write_header(ofmt_ctx, NULL);
if (ret < 0)
{
printf("Error occurred when opening output URL\n");
return;
}
}
this code will get the captured screen and send it to encoder:
ScreenCap.load_screen_data(inFrame,timmer) // this will capture screen and return the time and AVFrame for encoding
sws_scale(ctx, inFrame->data, inFrame->linesize, 0,c->height, frame->data, frame->linesize);
frame->pts = (timmer - first_frame_time) / fps;
EncodeToPkT(c, frame, pkt, ofmt_ctx,out_stream);
av_frame_free(&inFrame);
the AVFrame will then be send to encoder using avcodec_send_frame() to get Packet data and av_interleaved_write_frame() to stream it over LAN.
All error check are removed for simplicity
Also, this is my AVCodecContex setting for the encoder:
c->bit_rate = 15000000;
c->width = 1920;
c->height = 1080;
c->time_base = AVRational{ 1, 90 };
c->framerate = AVRational{ 90, 1 };
c->gop_size = 90;
c->max_b_frames = 0;
c->pix_fmt = AV_PIX_FMT_RGB0;
I also notice it only happen when I increase encoder bitrate (15 MBIT) but when lower it down to (10 MBIT) it happen less. And when lower it down to 2 MBIT the corrupted frame not happen anymore however the quality is really bad.
I test the streaming in LAN using: PC -> cable ->laptop, PC->wireless->laptop, PC->Virtual PC, PC only (enter PC IP address in both program and Mplayer), all got the same result.
I also test my DXGICap.load_screen_data function by make it output raw image and there were no corrupted image at all
Does anybody have an idea why.
Thank Nam.

Answer 1

It's not just a case of using UDP, ie. you'll receive packets out-of-order and the decoding will then fail/complain? Try using TCP and see what happens.