Problems converting .flv to mp3 using FFmpeg SDK - c++

I'm using the FFmpeg SDK to programmatically convert videos to mp3.
I read the audio frames of the video this way:
while(av_read_frame(pFromCtx, &pkt) >= 0)
{
if(pkt.stream_index == audioStreamIndex)
{
avcodec_get_frame_defaults(frame);
got_frame = 0;
ret = avcodec_decode_audio4(pCodecCtx, frame, &got_frame, &pkt);
if (ret < 0) {
av_log(NULL, AV_LOG_ERROR, "Error decoding audio frame.\n");
continue;
}
if(got_frame)
{
// Write the decoded audio frame
write_audio_frame(pToCtx, pToCtx->streams[pToCtx->nb_streams-1], frame);
}
}
av_free_packet(&pkt);
}
Decoding the audio from the input video file works fine. The problem occurs when I try to encode a mp3 frame:
static void write_audio_frame(AVFormatContext *oc, AVStream *st, AVFrame *frame)
{
AVCodecContext *enc = st->codec;
AVPacket pkt;
int got_packet = 0;
int ret = 0;
av_init_packet(&pkt);
pkt.data = NULL;
pkt.size = 0;
ret = avcodec_encode_audio2(enc, &pkt, frame, &got_packet);
if (ret < 0) {
// PROBLEM
fprintf(stderr, "Error encoding audio frame. \n");
exit(1);
}
}
I get the following console output:
[libmp3lame] inadequate AVFrame plane padding
The only happens with .flv files, the code works fine for .mp4 files. Any clue what the error message means?
Thanks

The source code containing the error message is here: http://ffmpeg.org/doxygen/trunk/libmp3lame_8c-source.html. The relevant source says:
if (frame->linesize[0] < 4 * FFALIGN(frame->nb_samples, 8)) {
av_log(avctx, AV_LOG_ERROR, "inadequate AVFrame plane padding\n");
return AVERROR(EINVAL);
}
FFALIGN is defined as
#define FFALIGN (x,a)(((x)+(a)-1)&~((a)-1))

Related

Get video from webcam using FFmpeg Libav

I am trying to record webcam video using FFmpeg C libraries (libav), on a Mac computer. I made changes to the transcode.c example so that it opens a device instead of a file. However, for some reason the code only receives a single packet and then closes.
static int open_input_source(const char *dev_name) {
int ret;
unsigned int i;
AVDictionary *p_av_options = NULL;
AVInputFormat *p_av_input_format = av_find_input_format("avfoundation");
av_dict_set(&p_av_options, "framerate", "30", 0);
ifmt_ctx = NULL;
if ((ret = avformat_open_input(&ifmt_ctx, dev_name, p_av_input_format, &p_av_options) < 0)) {
av_log(NULL, AV_LOG_ERROR, "Cannot open input file\n");
return ret;
}
if ((ret = avformat_find_stream_info(ifmt_ctx, NULL)) < 0) {
av_log(NULL, AV_LOG_ERROR, "Cannot find stream information\n");
return ret;
}
stream_ctx = av_calloc(ifmt_ctx->nb_streams, sizeof(*stream_ctx));
if (!stream_ctx)
return AVERROR(ENOMEM);
for (i = 0; i < ifmt_ctx->nb_streams; i++) {
AVStream *stream = ifmt_ctx->streams[i];
const AVCodec *dec = avcodec_find_decoder(stream->codecpar->codec_id);
AVCodecContext *codec_ctx;
if (!dec) {
av_log(NULL, AV_LOG_ERROR, "Failed to find decoder for stream #%u\n", i);
return AVERROR_DECODER_NOT_FOUND;
}
codec_ctx = avcodec_alloc_context3(dec);
if (!codec_ctx) {
av_log(NULL, AV_LOG_ERROR, "Failed to allocate the decoder context for stream #%u\n", i);
return AVERROR(ENOMEM);
}
ret = avcodec_parameters_to_context(codec_ctx, stream->codecpar);
if (ret < 0) {
av_log(NULL, AV_LOG_ERROR, "Failed to copy decoder parameters to input decoder context "
"for stream #%u\n", i);
return ret;
}
/* Reencode video & audio and remux subtitles etc. */
if (codec_ctx->codec_type == AVMEDIA_TYPE_VIDEO
|| codec_ctx->codec_type == AVMEDIA_TYPE_AUDIO) {
if (codec_ctx->codec_type == AVMEDIA_TYPE_VIDEO)
codec_ctx->framerate = av_guess_frame_rate(ifmt_ctx, stream, NULL);
/* Open decoder */
ret = avcodec_open2(codec_ctx, dec, NULL);
if (ret < 0) {
av_log(NULL, AV_LOG_ERROR, "Failed to open decoder for stream #%u\n", i);
return ret;
}
}
stream_ctx[i].dec_ctx = codec_ctx;
stream_ctx[i].dec_frame = av_frame_alloc();
if (!stream_ctx[i].dec_frame)
return AVERROR(ENOMEM);
}
av_dump_format(ifmt_ctx, 0, dev_name, 0);
return 0;
}
I have looked for other code examples but they are all deprecated and no longer compile in updated FFmpeg.
Is there some missing setting in my open_input_source function? Alternatively, is the problem in using transcoding is my basis? Should I try to use some other example?
In general, is there is a C source code reference which fulfills my requirements?
Thanks
This is a pretty fleshed out example that might meet your requirements:
https://github.com/argvk/ffmpeg-examples/blob/master/dshow_capture_video.c
I don't think you need nearly as much code as is include there, but you might be able to just update lines 260 with how long you want it to run (that example is 300 frames) and line 83 to open your webcam (sounds like you've already successfully done this with ret = avformat_open_input(&ifmt_ctx, dev_name, p_av_input_format, &p_av_options) in your code.
There are lots of other options there which you might want to remove, keep, or change depending on the details which are not provided here. Unfortunately I don't have a simplified code sample I'm able to share on here, but this dshow example is doing everything I expect.
Hope it helps some.

An Unhandled exception: YUV Packed to JPEG Conversion On Button Press VC++2010 Form Application

i have YUV packed images which i have first convert to planar and then jpeg format (on button press) for Displaying in a picture box in VC++ 2010, using command at the end of entire code (after jpeg conversion done)
pictureBox1->Image = Image::FromFile("d:\\encoded_pic_420.jpg");
i am able to store and format convert these received images from YUV packed to YUV planar First and then Second from "Planar to JPEG format" (below Code First & Second).
on first time it Convert and Display image in picture box sucessfully, but when i press button second time it generate an error (in this below code section) SECOND PART OF CODE (PLANAR TO JPEG CONVERSION
if (got_picture==1)
{
pkt.stream_index = video_st->index;
ret = av_write_frame(pFormatCtx, &pkt);
}
error mesage is at Code Line (above) ret = av_write_frame (pFormatCtx,&pkt);
message is :
"An unhandled exception of type accured System.AccessViolationException , Additional information: Attempted to read or write protected memory, this is often an indication that other memory is corrupted"
the Complete code which i am using for YUV packet to Planar first and then second Planar to Jpeg conversion is below one.
First: YUV PACKED to PLANAR CONVERSION
FILE *in_file = NULL; //fopen("myHexFile.yuv","rb"); input PACKED
FILE *out_file = NULL; //Output File Planar format
int in_width = 2448; //YUV's width
int in_height = 2050; //YUV's heigh
int out_width = 2448; //YUV's width
int out_height = 2050; //YUV's heigh
int in_linesize[4];
int out_linesize[4];
uint8_t *in_data[4], *out_data[4];
unsigned long int out_bufsize,in_bufsize;
in_file = fopen("myHexFile.yuv","rb"); //This is YUV422-UYVY Input packed image
if(in_file == NULL)
{
this->Print2TextBox1(L"Input File Opening error...!");
exit(1);
}
out_file = fopen("d:\\myHexFile_Planar.yuv", "wb"); //Source Input File
if(out_file == NULL)
{
this->Print2TextBox1(L"toutput File Opening error...!!");
exit(1);
}
else { this->Print2TextBox1(L"Output File Created...!!\n"); }
//-Loads the whole database of available codecs and formats-------
av_register_all();
this->Print2TextBox1(L"Codac database Loaded...\n");
//---Create scaling context------------------------sws_getContex
this->Print2TextBox1(L"Creating Scaling context..\n");
sws_ctx = sws_getContext( in_width, in_height, src_pix_fmt,
out_width,out_height,dst_pix_fmt,
SWS_BICUBIC, NULL, NULL, NULL);
if(!sws_ctx) { this->Print2TextBox1(L"Context Error..\n"); }
//--Allocate Source Image Buffer--------------------------
this->Print2TextBox1(L"Allocate Source Image Buffer...\n");
AVFrame *RawPic = av_frame_alloc();
if(!RawPic)
{
this->Print2TextBox1(L"Could not allocate Raw Image frame\n");
exit(1);
}
RawPic->format = src_pix_fmt;
RawPic->width = in_width;
RawPic->height = in_height;
int num_bytes1 = avpicture_get_size(src_pix_fmt,in_width,in_height);
uint8_t* RawPic_Buffer = (uint8_t*)av_malloc(num_bytes1*sizeof(int8_t));
ret =av_image_alloc(RawPic->data,in_linesize,in_width,in_height,src_pix_fmt, 1);
if(ret < 0)
{
this->Print2TextBox1(L"Could not allocate raw picture buffer\n");
exit(1);
}
in_bufsize = ret;
//------Reading Input Image and Store in RawPic->Data Pointer---
fread(RawPic->data[0],1,in_bufsize,in_file);
//----Allocate Desitnation Image Buffer-------------------
this->Print2TextBox1(L"Allocate Destination Image Buffer...\n");
AVFrame *ScalePic = av_frame_alloc();
if(!ScalePic)
{
this->Print2TextBox1(L"Could not allocate Scale Image frame\n");
exit(1);
}
ScalePic->format = dst_pix_fmt;//pCodecCtx->pix_fmt;
ScalePic->width = out_width;
ScalePic->height = out_height;
int num_bytes2 = avpicture_get_size(dst_pix_fmt,out_width,out_height);
uint8_t* ScalePic_Buffer = (uint8_t *)av_malloc(num_bytes2*sizeof(int8_t));
ret = av_image_alloc(ScalePic->data,out_linesize,out_width,out_height,dst_pix_fmt, 1); //16
if(ret < 0) { this->Print2TextBox1(L"Could not allocate Scale picture buffer\n"); exit(1);}
out_bufsize = ret;
//-Create scaling context-OR CONVERTED TO DESTINATION FORMAT-----sws_scale
this->Print2TextBox1(L"Creating Scaling context...sws_scale\n");
sws_scale(sws_ctx, RawPic->data, in_linesize, 0, ScalePic->height, ScalePic->data, out_linesize);
//-----Write Scale Image to outputfile-
this->Print2TextBox1(L"Write Scale Image to outputfile..\n");
fwrite(ScalePic->data[0],1,out_bufsize,out_file);
//---Release all memory and close file--
fclose(in_file);
fclose(out_file);
av_freep(&RawPic->data[0]);
av_freep(&ScalePic->data[0]);
av_frame_free(&ScalePic);
av_frame_free(&RawPic);
SECOND--CONVERT to PLANAR TO JPEG FORMAT------(in Continuation to Above Code)
const char* myJpeg_file = "d:\\encoded_pic_444.jpg"; //Output JPEG
in_file = fopen("d:\\myHexFile_Planar.yuv", "rb"); //Input Planar File
if(in_file == NULL)
{
this->Print2TextBox1(L"File Opening error...!!");
exit(1);
}
else this->Print2TextBox1(L"YUV File Open Sucessfully...!!\n\n");
av_register_all(); // Loads the whole database of available codecs and formats.
pFormatCtx = avformat_alloc_context();
fmt = NULL;
fmt = av_guess_format("mjpeg",NULL,NULL);
pFormatCtx->oformat = fmt;
if (avio_open(&pFormatCtx->pb,myJpeg_file, AVIO_FLAG_READ_WRITE) < 0)
{
this->Print2TextBox1(L"Couldn't open output file.");
}
video_st = avformat_new_stream(pFormatCtx, 0);
if (video_st==NULL)
{
this->Print2TextBox1(L"avformat_new_stream.");
}
pCodecCtx = video_st->codec;
pCodecCtx->codec_id = fmt->video_codec;
pCodecCtx->codec_type = AVMEDIA_TYPE_VIDEO;
pCodecCtx->pix_fmt = AV_PIX_FMT_YUVJ420P;
pCodecCtx->width = in_width;
pCodecCtx->height = in_height;
pCodecCtx->time_base.num = 1;
pCodecCtx->time_base.den = 1;//25;
this->Print2TextBox1(L"Conversion start\n");
//Output some information
av_dump_format(pFormatCtx, 0, myJpeg_file, 1);
// Determine if desired video encoder is installed
pCodec = avcodec_find_encoder(pCodecCtx->codec_id);
if (!pCodec)
{
this->Print2TextBox1(L"Codec not found.");
//return -1;
}
this->Print2TextBox1(L"Codec Identified done\n");
if (avcodec_open2(pCodecCtx, pCodec,NULL) < 0){
this->Print2TextBox1(L"Could not open codec.\n");
//return -1;
}
this->Print2TextBox1(L"Codec Open done\n");
//-----------------------------------------------
picture = av_frame_alloc();
size = avpicture_get_size(pCodecCtx->pix_fmt, pCodecCtx->width, pCodecCtx->height);
picture_buf = (uint8_t *)av_malloc(size);
if (!picture_buf)
{ this->Print2TextBox1(L"Size Allocation error\n");
//return -1;
}
avpicture_fill((AVPicture *)picture, picture_buf, pCodecCtx->pix_fmt, pCodecCtx->width, pCodecCtx->height);
this->Print2TextBox1(L"Write Header..");
avformat_write_header(pFormatCtx,NULL);
y_size = pCodecCtx->width * pCodecCtx->height;
av_new_packet(&pkt,y_size*3);
//-------------------------------------------------------420 Format
//Read YUV
if (fread(picture_buf, 1, y_size*3/2, in_file) <=0)
{
this->Print2TextBox1(L"Could not read input file.");
//return -1;
}
//--------------------------------------------input image format UYVY
picture->data[0] = picture_buf; // Y
picture->data[1] = picture_buf+ y_size; // U
picture->data[2] = picture_buf+ y_size*5/4; // V
this->Print2TextBox1(L" Encode the image..\n");
ret = avcodec_encode_video2(pCodecCtx, &pkt,picture, &got_picture);
if(ret < 0)
{
this->Print2TextBox1(L"Encode Error.\n");
//return -1;
}
if (got_picture==1)
{
pkt.stream_index = video_st->index;
//#### PROBLEM IN THIS LINE BELOW WHEN RE-EXECUTE THE CODE ###
ret = av_write_frame(pFormatCtx, &pkt);
}
av_free_packet(&pkt);
//Write Trailer
av_write_trailer(pFormatCtx);
this->Print2TextBox1(L"Encode Successful.\n");
if (video_st)
{
avcodec_close(video_st->codec);
av_free(picture);
av_free(picture_buf);
}
avio_close(pFormatCtx->pb);
avformat_free_context(pFormatCtx);
fclose(in_file);
it seems that some of memory is not yet free or when i am trying to re-use this above code in second time in a loop,
plz suggest/guide me where i am doing wrong and not freeing up the memory..?
i am trying to Display Image (Current/updated) on every button press in VC++2010
You are allocating AVFrame
picture = av_frame_alloc();
, but casting it to deprecated AVPicture later:
avpicture_fill((AVPicture *)picture, picture_buf, pCodecCtx->pix_fmt, pCodecCtx->width, pCodecCtx->height);
solved by renaming the "encoded_pic_420.jpg" file with some other name and display it on PictureBox
https://msdn.microsoft.com/en-us/library/windows/desktop/aa363851(v=vs.85).aspx

Audio/Video encoding with ffmpeg

Audio/Video encoding with ffmpeg:
I am trying to create an avi file with encoded video and audio, using ffmpeg.
First, I create the file:
//define BITRATE 10000000
//define GOP 300
//define FPS 60
//define VIDEOTYPE "avi"
if (!encoder_->createFile(QFileInfo(*(videoFile_.data())).absoluteFilePath(), targetRect.width(), targetRect.height(), BITRATE*(1000 / FPS), GOP, 1000))
The buffers are initialized as:
audio_outbuf_size = 44100 * 0.005 * 16; //5ms of audio should be encoded, each time this function is called
audio_outbuf = new uint8_t[audio_outbuf_size];
outbuf_size = getWidth()*getHeight() * 3;
outbuf = new uint8_t[outbuf_size];
Then add audio and video streams (audio: CODEC_ID_PCM_S16LE, 16000 kb/s and 44100 Hz, video: PIX_FMT_YUV420P)
void MediaMuxer::addAudioStream(QString fileName, ffmpeg::CodecID codec_id)
{
// Add the audio stream
ffmpeg::AVCodec *encoder = avcodec_find_encoder(codec_id);
pAudioStream_ = ffmpeg::av_new_stream(pOutputFormatCtx_, 0);
if (!pAudioStream_) {
printf("Could not allocate stream\n");
return;
}
pAudioCodecCtx_ = pAudioStream_->codec;
pAudioCodecCtx_->codec_id = codec_id;
pAudioCodecCtx_->codec_type = ffmpeg::AVMEDIA_TYPE_AUDIO;
pAudioCodecCtx_->sample_fmt = ffmpeg::AV_SAMPLE_FMT_S16;
pAudioCodecCtx_->sample_fmt = encoder->sample_fmts[0];
pAudioCodecCtx_->bit_rate = 16000;
//pAudioCodecCtx_->bit_rate = 64000;
pAudioCodecCtx_->sample_rate = N;
pAudioCodecCtx_->channels = 1;
pAudioCodecCtx_->time_base.den = FPS;
pAudioCodecCtx_->time_base.num = 1;
avcodec_thread_init(pAudioCodecCtx_, 10);
// some formats want stream headers to be separate
if (pOutputFormatCtx_->oformat->flags & AVFMT_GLOBALHEADER)
pAudioCodecCtx_->flags |= CODEC_FLAG_GLOBAL_HEADER;
if (av_set_parameters(pOutputFormatCtx_, NULL) < 0)
{
printf("Invalid output format parameters\n");
return;
}
//ffmpeg::dump_format(pOutputFormatCtx_, 0, fileName.toStdString().c_str(), 1);
// open_video
// find the audio encoder
pAudioCodec_ = avcodec_find_encoder(pAudioCodecCtx_->codec_id);
if (!pAudioCodec_)
{
printf("codec not found\n");
return;
}
// open the codec
if (avcodec_open(pAudioCodecCtx_, pAudioCodec_) < 0)
{
printf("could not open codec\n");
return;
}
// Allocate memory for output
if (!initAudioOutputBuf())
{
printf("Can't allocate memory for audio output bitstream\n");
return;
}
// Allocate the audio frame
if (!initAudioFrame())
{
printf("Can't init audio frame\n");
return;
}
if (url_fopen(&pOutputFormatCtx_->pb, fileName.toStdString().c_str(), URL_WRONLY) < 0)
{
printf("Could not open '%s'\n", fileName.toStdString().c_str());
return;
}
av_write_header(pOutputFormatCtx_);
}
void MediaMuxer::addVideoStream(QString fileName)
{
// Add the video stream
pVideoStream_ = ffmpeg::av_new_stream(pOutputFormatCtx_, 0);
if (!pVideoStream_)
{
printf("Could not allocate stream\n");
return;
}
pVideoCodecCtx_ = pVideoStream_->codec;
pVideoCodecCtx_->codec_id = pOutputFormat_->video_codec;
pVideoCodecCtx_->codec_type = ffmpeg::AVMEDIA_TYPE_VIDEO;
pVideoCodecCtx_->bit_rate = Bitrate;
pVideoCodecCtx_->width = getWidth();
pVideoCodecCtx_->height = getHeight();
pVideoCodecCtx_->time_base.den = FPS;
pVideoCodecCtx_->time_base.num = 1;
pVideoCodecCtx_->gop_size = Gop;
pVideoCodecCtx_->pix_fmt = ffmpeg::PIX_FMT_YUV420P;
avcodec_thread_init(pVideoCodecCtx_, 10);
// some formats want stream headers to be separate
if (pOutputFormatCtx_->oformat->flags & AVFMT_GLOBALHEADER)
pVideoCodecCtx_->flags |= CODEC_FLAG_GLOBAL_HEADER;
if (av_set_parameters(pOutputFormatCtx_, NULL) < 0)
{
printf("Invalid output format parameters\n");
return;
}
//ffmpeg::dump_format(pOutputFormatCtx_, 0, fileName.toStdString().c_str(), 1);
// open_video
// find the video encoder
pVideoCodec_ = avcodec_find_encoder(pVideoCodecCtx_->codec_id);
if (!pVideoCodec_)
{
printf("codec not found\n");
return;
}
// open the codec
if (avcodec_open(pVideoCodecCtx_, pVideoCodec_) < 0)
{
printf("could not open codec\n");
return;
}
// Allocate memory for output
if (!initOutputBuf())
{
printf("Can't allocate memory for output bitstream\n");
return;
}
// Allocate the YUV frame
if (!initFrame())
{
printf("Can't init frame\n");
return;
}
if (url_fopen(&pOutputFormatCtx_->pb, fileName.toStdString().c_str(), URL_WRONLY) < 0)
{
printf("Could not open '%s'\n", fileName.toStdString().c_str());
return;
}
av_write_header(pOutputFormatCtx_);
}
Finally, I call alternatively encodeVideo/encodeAudio to encode video and PCM audio frames at specific recording times(pts):
int MediaMuxer::encodeVideo(const QImage &img, unsigned pts)
{
convertImage_sws(img); // SWS conversion
pVideoCodecCtx_->coded_frame->pts = pts; // Set the time stamp
int out_size = ffmpeg::avcodec_encode_video(pVideoCodecCtx_, outbuf, outbuf_size, ppicture);
pVideoCodecCtx_->coded_frame->pts = pts; // Set the time stamp
if (out_size > 0)
{
ffmpeg::av_init_packet(&pkt);
if (pVideoCodecCtx_->coded_frame->pts != (0x8000000000000000LL))
pkt.pts = av_rescale_q(pVideoCodecCtx_->coded_frame->pts, pVideoCodecCtx_->time_base, pVideoStream_->time_base);
if (pVideoCodecCtx_->coded_frame->key_frame)
pkt.flags |= AV_PKT_FLAG_KEY;
pkt.stream_index = pVideoStream_->index;
pkt.data = outbuf;
pkt.size = out_size;
int ret = ffmpeg::av_interleaved_write_frame(pOutputFormatCtx_, &pkt);
if (ret<0)
return -1;
}
return out_size;
}
int MediaMuxer::encodeAudio(unsigned pts)
{
pAudioCodecCtx_->coded_frame->pts = pts; // Set the time stamp
// simple sound encoding
int16_t samples[220] = { 0 }; // buffer
int n; // buffer index
double Fs = 44100.0; // sampling frequency
// Generate audio data
for (n = 0; n < 220; ++n) //220 samples (44100*.005sec as the interval between 2 video frames is 10ms)
samples[n] = 16383.0 * sin(n*1000.0*2.0*M_PI / Fs); //sine wav
int out_size = ffmpeg::avcodec_encode_audio(pAudioCodecCtx_, audio_outbuf, audio_outbuf_size, (const short*)samples);
pAudioCodecCtx_->coded_frame->pts = pts; // Set the time stamp
if (out_size>0)
{
// Packet
ffmpeg::AVPacket pkt = { 0 };
av_init_packet(&pkt);
pkt.data = NULL; // packet data will be allocated by the encoder
pkt.size = 0;
if (pAudioCodecCtx_->coded_frame->pts != (0x8000000000000000LL))
pkt.pts = av_rescale_q(pAudioCodecCtx_->coded_frame->pts, pAudioCodecCtx_->time_base, pAudioStream_->time_base);
if (pAudioCodecCtx_->coded_frame->key_frame)
pkt.flags |= AV_PKT_FLAG_KEY;
pkt.stream_index = pAudioStream_->index;
pkt.data = audio_outbuf;
pkt.size = out_size;
int ret = av_interleaved_write_frame(pOutputFormatCtx_, &pkt);
if (ret<0)
return -1;
av_free_packet(&pkt);
}
//end simple sound encoding
return pkt.size;
}
The result is a nice video with some audio behind (either a regular beeping sound at regular intervals but ending way earlier than the video or a continuous longer sound that also last shorter than the video).
I want to generate a beeping sound each time the function encodeAudio() is called - at non-regular intervals. I have tried to modify the sampling rate, the buffer size, the pkt size and the number of samples but without any success. I also tried to set the pts at different times but it did not get me where I want to be. Could someone please help?

ffmpeg hangs in avcodec_encode_video2

I encode the video stream using ffmeg. Shell QT Creator. OC Windows 7. The compiler MinGW. H264 codec.
There is a function, which I give to the input frame, and I must return an encoded byte array. Within this function it hangs when calling avcodec_encode_video2, but not on the first call, and at random.
That is, an arbitrary coding frame hangs (100 to 10000).
QByteArray VideoEncoder::createFrameVideoFromImage(QImage picture)
{
AVFrame* frame_source = av_frame_alloc();
avpicture_fill((AVPicture*)frame_source, picture.bits(), AV_PIX_FMT_RGB24, this->width_frame, this->height_frame);
AVFrame* frame_dst = av_frame_alloc();
avpicture_fill((AVPicture*)frame_dst, (uint8_t*)this->inbuffer, AV_PIX_FMT_YUV420P, this->width_frame, this->height_frame);
sws_scale(this->convert_rgb_yuv, frame_source->data, frame_source->linesize,
0, this->ctx_codec_in->height, frame_dst->data, frame_dst->linesize);
AVPacket packet;
av_init_packet(&packet);
packet.data = NULL;
packet.size = 0;
packet.pts = packet.dts = AV_NOPTS_VALUE;
int nOutputSize = 0;
if (avcodec_encode_video2(this->ctx_codec_in, &packet, frame_dst, &nOutputSize) < 0)
{
qDebug() << "VideoEncoder error";
}
this->traffic += packet.size;
QByteArray data_frame = QByteArray((char*)packet.data, packet.size);
av_frame_free(&frame_source);
av_frame_free(&frame_dst);
av_free_packet(&packet);
return data_frame;
}
Thank you in advance!

How do I dump the buffer when encoding H264 with FFMPEG?

I'm using a c++ library to write images captured from a webcam to an libx264 encoded mp4 file.
The encoding is working properly but when it starts it writes 40 frames to the buffer. When I close the file these frames aren't flushed so about 6 seconds of video are left unwritten (cam is about 6fps).
So i'm calling:
out_size = libffmpeg::avcodec_encode_video( codecContext, data->VideoOutputBuffer,data->VideoOutputBufferSize, data->VideoFrame );
// if zero size, it means the image was buffered
if ( out_size > 0 )
{
//... write to file
}
I can't see a way of accessing the images that are left in the buffer. Any ideas?
I've got this working using the following code to flush the buffer. Seems that I was searching for the wrong term - should have been "delayed frames"...
void VideoFileWriter::Flush(void)
{
if ( data != nullptr )
{
int out_size = 0;
int ret = 0;
libffmpeg::AVCodecContext* c = data->VideoStream->codec;
/* get the delayed frames */
while (1) {
libffmpeg::AVPacket packet;
libffmpeg::av_init_packet(&packet);
out_size = libffmpeg::avcodec_encode_video(c, data->VideoOutputBuffer, data->VideoOutputBufferSize, NULL);
if (out_size < 0) {
//fprintf(stderr, "Error encoding delayed frame %d\n", out_size);
break;
}
if (out_size == 0) {
break;
}
if (c->coded_frame->pts != AV_NOPTS_VALUE) {
packet.pts = av_rescale_q(c->coded_frame->pts,
c->time_base,
data->VideoStream->time_base);
//fprintf(stderr, "Video Frame PTS: %d\n", (int)packet.pts);
} else {
//fprintf(stderr, "Video Frame PTS: not set\n");
}
if (c->coded_frame->key_frame) {
packet.flags |= AV_PKT_FLAG_KEY;
}
packet.stream_index = data->VideoStream->index;
packet.data = data->VideoOutputBuffer;
packet.size = out_size;
ret = libffmpeg::av_interleaved_write_frame( data->FormatContext, &packet );
if (ret != 0) {
//fprintf(stderr, "Error writing delayed frame %d\n", ret);
break;
}
}
libffmpeg::avcodec_flush_buffers(data->VideoStream->codec);
}
}
Here is a tutorial regarding ffmpeg with avcodec, stating that avcodec uses some internal buffers which need to be flushed. There is also some code showing how flushing of these buffers is done ("Flushing our buffers").