FFmpeg memory leak - c++

I have developed just a simple library modifing a library that I found on the internet.
What scares me, is that, when I play an avi, it plays and free the memory when the video ends, but when I play the video, it's like a memory leak! It grows to 138mb although the video has ended and the FreeAll method (A function that deletes the context, etc...) has been called.
Here is the code of the method that is causing the memory leak:
int VideoGL::NextVideoFrame(){
int frameDone = 0;
int result = 0;
double pts = 0;
if(!this->ended){
if (!_started) return 0;
AVPacket* packet;
// Get the number of milliseconds passed and see if we should display a new frame
int64_t msPassed = (1000 * (clock() - _baseTime)) / CLOCKS_PER_SEC;
if (msPassed >= _currentPts)
{
// If this is not the current frame, copy it to the buffer
if (_currentFramePts != _currentPts){
_currentFramePts = _currentPts;
memcpy(buffer_a,buffer, 3 * _codec_context_video->width * _codec_context_video->height);
result = 1;
}
// Try to load a new frame from the video packet queue
bool goodop=false;
AVFrame *_n_frame = avcodec_alloc_frame();
while (!frameDone && (packet = this->DEQUEUE(VIDEO)) != NULL)
{
if (packet == (AVPacket*)-1) return -1;
goodop=true;
_s_pts = packet->pts;
avcodec_decode_video2(_codec_context_video, _n_frame, &frameDone, packet);
av_free_packet(packet);
if (packet->dts == AV_NOPTS_VALUE)
{
if (_n_frame->opaque && *(uint64_t*)_n_frame->opaque != AV_NOPTS_VALUE) pts = (double) *(uint64_t*)_n_frame->opaque;
else pts = 0;
}
else pts = (double) packet->dts;
pts *= av_q2d(_codec_context_video->time_base);
}
if (frameDone)
{
// if a frame was loaded scale it to the current texture frame buffer, but also set the pts so that it won't be copied to the texture until it's time
sws_scale(sws_ctx,_n_frame->data, _n_frame->linesize, 0, _codec_context_video->height, _rgb_frame->data, _rgb_frame->linesize);
double nts = 1.0/av_q2d(_codec_context_video->time_base);
_currentPts = (uint64_t) (pts*nts);
}
avcodec_free_frame(&_n_frame);
av_free(_n_frame);
if(!goodop){
ended=true;
}
}
}
return result;
}
I'll be waiting for answers, thanks.

I had a memory leak problem either. For me, the deallocation worked when I included the following commands:
class members:
AVPacket avpkt;
AVFrame *frame;
AVCodecContext *avctx;
AVCodec *codec;
constructor:
av_init_packet(&avpkt);
avcodec_open2(avctx, codec, NULL);
frame = avcodec_alloc_frame();
destructor:
av_free_packet(&avpkt);
avcodec_free_frame(&frame);
av_free(frame);
avcodec_close(avctx);

i also had the same problem. According to the ffplay.c
you should call
av_frame_unref(pFrame);
avcodec_get_frame_defaults(pFrame);
after every sw_scale call. this will free up all malloc during decode.

I had similar routine using FFmpeg that would leak memory. I found a resolution by deallocating memory for the frame and packet objects for each call to avcodec_decode_video2.
In your code the packet object is freed, however the frame is not. Adding the following lines before avcodec_decode_video2 should resolve the memory leak. I found that it's safe to call avcodec_free_frame on a frame object that is already deallocated. You could remove the allocation of the frame before the while loop.
avcodec_free_frame(&_n_frame);
_n_frame = avcodec_alloc_frame();
avcodec_decode_video2(_codec_context_video, _n_frame, &frameDone, packet);

Related

SDL2 & SMPEG2 - Empty sound buffer trying to read a MP3

I'm trying to load a MP3 in a buffer using the SMPEG2 library, which comes with the SDL2. Every SMPEG function calls returns without error, but when I'm done, the sound buffer is full of zeros.
Here's the code :
bool LoadMP3(char* filename)
{
bool success = false;
const Uint32 Mp3ChunkLen = 4096;
SMPEG* mp3;
SMPEG_Info infoMP3;
Uint8 * ChunkBuffer;
Uint32 MP3Length = 0;
// Allocate a chunk buffer
ChunkBuffer = (Uint8*)malloc(Mp3ChunkLen);
SDL_RWops *mp3File = SDL_RWFromFile(filename, "rb");
if (mp3File != NULL)
{
mp3 = SMPEG_new_rwops(mp3File, &infoMP3, 1, 0);
if(mp3 != NULL)
{
if(infoMP3.has_audio)
{
Uint32 readLen;
// Inform the MP3 of the output audio specifications
SMPEG_actualSpec(mp3, &asDeviceSpecs); // static SDL_AudioSpec asDeviceSpecs; containing valid values after a call to SDL_OpenAudioDevice
// Enable the audio and disable the video.
SMPEG_enableaudio(mp3, 1);
SMPEG_enablevideo(mp3, 0);
// Play the MP3 once to get the size of the needed finale buffer
SMPEG_play(mp3);
while ((readLen = SMPEG_playAudio(mp3, ChunkBuffer, Mp3ChunkLen)) > 0)
{
MP3Length += readLen;
}
SMPEG_stop(mp3);
if(MP3Length > 0)
{
// Reallocate the buffer with the new length (if needed)
if (MP3Length != Mp3ChunkLen)
{
ChunkBuffer = (Uint8*)realloc(ChunkBuffer, MP3Length);
}
// Replay the entire MP3 into the new ChunkBuffer.
SMPEG_rewind(mp3);
SMPEG_play(mp3);
bool readBackSuccess = (MP3Length == SMPEG_playAudio(mp3, ChunkBuffer, MP3Length));
SMPEG_stop(mp3);
if(readBackSuccess)
{
// !!! Here, ChunkBuffer contains only zeros !!!
success = true;
}
}
}
SMPEG_delete(mp3);
mp3 = NULL;
}
SDL_RWclose(mp3File);
mp3File = NULL;
}
free(ChunkBuffer);
return success;
}
The code's widely based on SDL_Mixer, which I cannot use for my projet, based on its limitations.
I know Ogg Vorbis would be a better choice of file format, but I'm porting a very old project, and it worked entirely with MP3s.
I'm sure the sound system is initialized correctly because I can play WAV files just fine. It's intialized with a frequency of 44100, 2 channels, 1024 samples, and the AUDIO_S16SYS format (the latter which is, as I understood from the SMPEG source, mandatory).
I've calculated the anticipated buffer size, based on the bitrate, the amount of data in the MP3 and the OpenAudioDevice audio specs, and everything is consistent.
I cannot figure why everything but the buffer data seems to be working.
UPDATE #1
Still trying to figure out what's wrong, I thought the support for MP3 might not be working, so I created the following function :
SMPEG *mpeg;
SMPEG_Info info;
mpeg = SMPEG_new(filename,&info, 1);
SMPEG_play(mpeg);
do { SDL_Delay(50); } while(SMPEG_status(mpeg) == SMPEG_PLAYING);
SMPEG_delete(mpeg);
The MP3 played. So, the decoding should actually be working. But that's not what I need ; I really need the sound buffer data so I can send it to my mixer.
After much tinkering, research and digging through the SMPEG source code, I realized that I had to pass 1 as the SDLAudio parameter to SMPEG_new_rwops function.
The comment found in smpeg.h is misleading :
The sdl_audio parameter indicates if SMPEG should initialize the SDL audio subsystem. If not, you will have to use the SMPEG_playaudio() function below to extract the decoded data.
Since the audio subsystem was already initialized and I was using the SMPEG_playaudio() function, I had no reason to think I needed this parameter to be non-zero. In SMPEG, this parameter triggers the audio decompression at opening time, but even though I called SMPEG_enableaudio(mp3, 1); the data is never reparsed. This might be a bug/a shady feature.
I had another problem with the freesrc parameter which needed to be 0, since I freed the SDL_RWops object myself.
For future reference, once ChunkBuffer has the MP3 data, it needs to pass through SDL_BuildAudioCVT/SDL_ConvertAudio if it's to be played through an already opened audio device.
The final working code is :
// bool ReadMP3ToBuffer(char* filename)
bool success = false;
const Uint32 Mp3ChunkLen = 4096;
SDL_AudioSpec mp3Specs;
SMPEG* mp3;
SMPEG_Info infoMP3;
Uint8 * ChunkBuffer;
Uint32 MP3Length = 0;
// Allocate a chunk buffer
ChunkBuffer = (Uint8*)malloc(Mp3ChunkLen);
memset(ChunkBuffer, 0, Mp3ChunkLen);
SDL_RWops *mp3File = SDL_RWFromFile(filename, "rb"); // filename is a char* passed to the function.
if (mp3File != NULL)
{
mp3 = SMPEG_new_rwops(mp3File, &infoMP3, 0, 1);
if(mp3 != NULL)
{
if(infoMP3.has_audio)
{
Uint32 readLen;
// Get the MP3 audio specs for later conversion
SMPEG_wantedSpec(mp3, &mp3Specs);
SMPEG_enablevideo(mp3, 0);
// Play the MP3 once to get the size of the needed buffer in relation with the audio specs
SMPEG_play(mp3);
while ((readLen = SMPEG_playAudio(mp3, ChunkBuffer, Mp3ChunkLen)) > 0)
{
MP3Length += readLen;
}
SMPEG_stop(mp3);
if(MP3Length > 0)
{
// Reallocate the buffer with the new length (if needed)
if (MP3Length != Mp3ChunkLen)
{
ChunkBuffer = (Uint8*)realloc(ChunkBuffer, MP3Length);
memset(ChunkBuffer, 0, MP3Length);
}
// Replay the entire MP3 into the new ChunkBuffer.
SMPEG_rewind(mp3);
SMPEG_play(mp3);
bool readBackSuccess = (MP3Length == SMPEG_playAudio(mp3, ChunkBuffer, MP3Length));
SMPEG_stop(mp3);
if(readBackSuccess)
{
SDL_AudioCVT convertedSound;
// NOTE : static SDL_AudioSpec asDeviceSpecs; containing valid values after a call to SDL_OpenAudioDevice
if(SDL_BuildAudioCVT(&convertedSound, mp3Specs.format, mp3Specs.channels, mp3Specs.freq, asDeviceSpecs.format, asDeviceSpecs.channels, asDeviceSpecs.freq) >= 0)
{
Uint32 newBufferLen = MP3Length*convertedSound.len_mult;
// Make sure the audio length is a multiple of a sample size to avoid sound clicking
int sampleSize = ((asDeviceSpecs.format & 0xFF)/8)*asDeviceSpecs.channels;
newBufferLen &= ~(sampleSize-1);
// Allocate the new buffer and proceed with the actual conversion.
convertedSound.buf = (Uint8*)malloc(newBufferLen);
memcpy(convertedSound.buf, ChunkBuffer, MP3Length);
convertedSound.len = MP3Length;
if(SDL_ConvertAudio(&convertedSound) == 0)
{
// Save convertedSound.buf and convertedSound.len_cvt for future use in your mixer code.
// Dont forget to free convertedSound.buf once it's not used anymore.
success = true;
}
}
}
}
}
SMPEG_delete(mp3);
mp3 = NULL;
}
SDL_RWclose(mp3File);
mp3File = NULL;
}
free(ChunkBuffer);
return success;
NOTE : Some MP3 files I tried lost a few milliseconds and cutoff too early during playback when I resampled them with this code. Some others didn't. I could reproduce the same behaviour in Audacity, so I'm not sure what's going on. There may still have a bug with my code, a bug in SMPEG, or it maybe a known issue with the MP3 format itself. If someone can provide and explanation in the comments, that would be great!

How to new & delete AVPacket?

I'm working with a FFmpeg project right now, I have to store the data of AVPacket which from av_read_frame, and fill data to new AVPacket for following decoding.
Here is my problem: when I try to new & free an AVPacket, memory leaks always happen.
I am just doing a simple testing:
for(;;) {
AVPacket pkt;
av_new_packet(&pkt, 1000);
av_init_packet(&pkt);
av_free_packet(&pkt);
}
What am I doing wrong?
av_new_packet creates a packet and allocates data
av_init_packet
sets all packet members to default, and sets data pointer to NULL,
the leak is here
av_free_packet clears all visible members, but
your data is already leaking
If you want FFmpeg to allocate the data for you, do not call av_init_packet. If you want to handle the data yourself, allocate the packet object on the stack and set its data yourself (and free it yourself):
AVPacket pkt;
av_init_packet(&pkt);
pkt.data = dataBuffer;
pkt.size = dataBufferSize;
// use your packet
// free your dataBuffer
I just read the FFmpeg 2.2 AVPacket.c source code.
int av_new_packet(AVPacket *pkt, int size) {
AVBufferRef *buf = NULL;
int ret = packet_alloc(&buf, size);
if (ret < 0)
return ret;
av_init_packet(pkt);
pkt->buf = buf;
pkt->data = buf->data;
pkt->size = size;
#if FF_API_DESTRUCT_PACKET
FF_DISABLE_DEPRECATION_WARNINGS
pkt->destruct = dummy_destruct_packet;
FF_ENABLE_DEPRECATION_WARNINGS
#endif
return 0;
}
void av_init_packet(AVPacket *pkt) {
pkt->pts = AV_NOPTS_VALUE;
pkt->dts = AV_NOPTS_VALUE;
pkt->pos = -1;
pkt->duration = 0;
pkt->convergence_duration = 0;
pkt->flags = 0;
pkt->stream_index = 0;
#if FF_API_DESTRUCT_PACKET
FF_DISABLE_DEPRECATION_WARNINGS
pkt->destruct = NULL;
FF_ENABLE_DEPRECATION_WARNINGS
#endif
pkt->buf = NULL;
pkt->side_data = NULL;
pkt->side_data_elems = 0;
}
I don't really know about the defines FF_API_DESTRUCT_PACKET, FF_DISABLE_DEPRECATION_WARNINGS, FF_ENABLE_DEPRECATION_WARNINGS
Some reason makes destruct of av_free_packet leak
According the source code, av_init_packet is called in av_new_packet and av_new_packet already allocates the AVBuffer, so if you want to set the data to new AVPacket.
Just memory copy to data of AVPacket, and call av_free_packet when you are done.

FFmpeg + OpenAL - playback streaming sound from video won't work

I am decoding an OGG video (theora & vorbis as codecs) and want to show it on the screen (using Ogre 3D) while playing its sound. I can decode the image stream just fine and the video plays perfectly with the correct frame rate, etc.
However, I cannot get the sound to play at all with OpenAL.
Edit: I managed to make the playing sound resemble the actual audio in the video at least somewhat. Updated sample code.
Edit 2: I was able to get "almost" correct sound now. I had to set OpenAL to use AL_FORMAT_STEREO_FLOAT32 (after initializing the extension) instead of just STEREO16. Now the sound is "only" extremely high pitched and stuttering, but at the correct speed.
Here is how I decode audio packets (in a background thread, the equivalent works just fine for the image stream of the video file):
//------------------------------------------------------------------------------
int decodeAudioPacket( AVPacket& p_packet, AVCodecContext* p_audioCodecContext, AVFrame* p_frame,
FFmpegVideoPlayer* p_player, VideoInfo& p_videoInfo)
{
// Decode audio frame
int got_frame = 0;
int decoded = avcodec_decode_audio4(p_audioCodecContext, p_frame, &got_frame, &p_packet);
if (decoded < 0)
{
p_videoInfo.error = "Error decoding audio frame.";
return decoded;
}
// Frame is complete, store it in audio frame queue
if (got_frame)
{
int bufferSize = av_samples_get_buffer_size(NULL, p_audioCodecContext->channels, p_frame->nb_samples,
p_audioCodecContext->sample_fmt, 0);
int64_t duration = p_frame->pkt_duration;
int64_t dts = p_frame->pkt_dts;
if (staticOgreLog)
{
staticOgreLog->logMessage("Audio frame bufferSize / duration / dts: "
+ boost::lexical_cast<std::string>(bufferSize) + " / "
+ boost::lexical_cast<std::string>(duration) + " / "
+ boost::lexical_cast<std::string>(dts), Ogre::LML_NORMAL);
}
// Create the audio frame
AudioFrame* frame = new AudioFrame();
frame->dataSize = bufferSize;
frame->data = new uint8_t[bufferSize];
if (p_frame->channels == 2)
{
memcpy(frame->data, p_frame->data[0], bufferSize >> 1);
memcpy(frame->data + (bufferSize >> 1), p_frame->data[1], bufferSize >> 1);
}
else
{
memcpy(frame->data, p_frame->data, bufferSize);
}
double timeBase = ((double)p_audioCodecContext->time_base.num) / (double)p_audioCodecContext->time_base.den;
frame->lifeTime = duration * timeBase;
p_player->addAudioFrame(frame);
}
return decoded;
}
So, as you can see, I decode the frame, memcpy it to my own struct, AudioFrame. Now, when the sound is played, I use these audio frame like this:
int numBuffers = 4;
ALuint buffers[4];
alGenBuffers(numBuffers, buffers);
ALenum success = alGetError();
if(success != AL_NO_ERROR)
{
CONSOLE_LOG("Error on alGenBuffers : " + Ogre::StringConverter::toString(success) + alGetString(success));
return;
}
// Fill a number of data buffers with audio from the stream
std::vector<AudioFrame*> audioBuffers;
std::vector<unsigned int> audioBufferSizes;
unsigned int numReturned = FFMPEG_PLAYER->getDecodedAudioFrames(numBuffers, audioBuffers, audioBufferSizes);
// Assign the data buffers to the OpenAL buffers
for (unsigned int i = 0; i < numReturned; ++i)
{
alBufferData(buffers[i], _streamingFormat, audioBuffers[i]->data, audioBufferSizes[i], _streamingFrequency);
success = alGetError();
if(success != AL_NO_ERROR)
{
CONSOLE_LOG("Error on alBufferData : " + Ogre::StringConverter::toString(success) + alGetString(success)
+ " size: " + Ogre::StringConverter::toString(audioBufferSizes[i]));
return;
}
}
// Queue the buffers into OpenAL
alSourceQueueBuffers(_source, numReturned, buffers);
success = alGetError();
if(success != AL_NO_ERROR)
{
CONSOLE_LOG("Error queuing streaming buffers: " + Ogre::StringConverter::toString(success) + alGetString(success));
return;
}
}
alSourcePlay(_source);
The format and frequency I give to OpenAL are AL_FORMAT_STEREO_FLOAT32 (it is a stereo sound stream, and I did initialize the FLOAT32 extension) and 48000 (which is the sample rate of the AVCodecContext of the audio stream).
And during playback, I do the following to refill OpenAL's buffers:
ALint numBuffersProcessed;
// Check if OpenAL is done with any of the queued buffers
alGetSourcei(_source, AL_BUFFERS_PROCESSED, &numBuffersProcessed);
if(numBuffersProcessed <= 0)
return;
// Fill a number of data buffers with audio from the stream
std::vector<AudiFrame*> audioBuffers;
std::vector<unsigned int> audioBufferSizes;
unsigned int numFilled = FFMPEG_PLAYER->getDecodedAudioFrames(numBuffersProcessed, audioBuffers, audioBufferSizes);
// Assign the data buffers to the OpenAL buffers
ALuint buffer;
for (unsigned int i = 0; i < numFilled; ++i)
{
// Pop the oldest queued buffer from the source,
// fill it with the new data, then re-queue it
alSourceUnqueueBuffers(_source, 1, &buffer);
ALenum success = alGetError();
if(success != AL_NO_ERROR)
{
CONSOLE_LOG("Error Unqueuing streaming buffers: " + Ogre::StringConverter::toString(success));
return;
}
alBufferData(buffer, _streamingFormat, audioBuffers[i]->data, audioBufferSizes[i], _streamingFrequency);
success = alGetError();
if(success != AL_NO_ERROR)
{
CONSOLE_LOG("Error on re- alBufferData: " + Ogre::StringConverter::toString(success));
return;
}
alSourceQueueBuffers(_source, 1, &buffer);
success = alGetError();
if(success != AL_NO_ERROR)
{
CONSOLE_LOG("Error re-queuing streaming buffers: " + Ogre::StringConverter::toString(success) + " "
+ alGetString(success));
return;
}
}
// Make sure the source is still playing,
// and restart it if needed.
ALint playStatus;
alGetSourcei(_source, AL_SOURCE_STATE, &playStatus);
if(playStatus != AL_PLAYING)
alSourcePlay(_source);
As you can see, I do quite heavy error checking. But I do not get any errors, neither from OpenAL nor from FFmpeg.
Edit: What I hear somewhat resembles the actual audio from the video, but VERY high pitched and stuttering VERY much. Also, it seems to be playing on top of TV noise. Very strange. Plus, it is playing much slower than the correct audio would.
Edit: 2 After using AL_FORMAT_STEREO_FLOAT32, the sound plays at the correct speed, but is still very high pitched and stuttering (though less than before).
The video itself is not broken, it can be played fine on any player. OpenAL can also play *.way files just fine in the same application, so it is also working.
Any ideas what could be wrong here or how to do this correctly?
My only guess is that somehow, FFmpeg's decode function does not produce data OpenGL can read. But this is as far as the FFmpeg decode example goes, so I don't know what's missing. As I understand it, the decode_audio4 function decodes the frame to raw data. And OpenAL should be able to work with RAW data (or rather, doesn't work with anything else).
So, I finally figured out how to do it. Gee, what a mess. It was a hint from a user on the libav-users mailing list that put me on the correct path.
Here are my mistakes:
Using the wrong format in the alBufferData function. I used AL_FORMAT_STEREO16 (as that is what every single streaming example with OpenAL uses). I should have used AL_FORMAT_STEREO_FLOAT32, as the video I stream is Ogg and vorbis is stored in floating points. And using swr_convert to convert from AV_SAMPLE_FMT_FLTP to AV_SAMPLE_FMT_S16 just crashes. No idea why.
Not using swr_convert to convert the decoded audio frame to the target format. After I was trying to use swr_convert to convert from FLTP to S16, and it would simply crash without a reason given, I assumed it was broken. But after figuring out my first mistake, I tried again, converting from FLTP to FLT (non-planar) and then it worked! So OpenAL uses interleaved format, not planar. Good to know.
So here is the decodeAudioPacket function that is working for me with Ogg video, vorbis audio stream:
int decodeAudioPacket( AVPacket& p_packet, AVCodecContext* p_audioCodecContext, AVFrame* p_frame,
SwrContext* p_swrContext, uint8_t** p_destBuffer, int p_destLinesize,
FFmpegVideoPlayer* p_player, VideoInfo& p_videoInfo)
{
// Decode audio frame
int got_frame = 0;
int decoded = avcodec_decode_audio4(p_audioCodecContext, p_frame, &got_frame, &p_packet);
if (decoded < 0)
{
p_videoInfo.error = "Error decoding audio frame.";
return decoded;
}
if(decoded <= p_packet.size)
{
/* Move the unread data to the front and clear the end bits */
int remaining = p_packet.size - decoded;
memmove(p_packet.data, &p_packet.data[decoded], remaining);
av_shrink_packet(&p_packet, remaining);
}
// Frame is complete, store it in audio frame queue
if (got_frame)
{
int outputSamples = swr_convert(p_swrContext,
p_destBuffer, p_destLinesize,
(const uint8_t**)p_frame->extended_data, p_frame->nb_samples);
int bufferSize = av_get_bytes_per_sample(AV_SAMPLE_FMT_FLT) * p_videoInfo.audioNumChannels
* outputSamples;
int64_t duration = p_frame->pkt_duration;
int64_t dts = p_frame->pkt_dts;
if (staticOgreLog)
{
staticOgreLog->logMessage("Audio frame bufferSize / duration / dts: "
+ boost::lexical_cast<std::string>(bufferSize) + " / "
+ boost::lexical_cast<std::string>(duration) + " / "
+ boost::lexical_cast<std::string>(dts), Ogre::LML_NORMAL);
}
// Create the audio frame
AudioFrame* frame = new AudioFrame();
frame->dataSize = bufferSize;
frame->data = new uint8_t[bufferSize];
memcpy(frame->data, p_destBuffer[0], bufferSize);
double timeBase = ((double)p_audioCodecContext->time_base.num) / (double)p_audioCodecContext->time_base.den;
frame->lifeTime = duration * timeBase;
p_player->addAudioFrame(frame);
}
return decoded;
}
And here is how I initialize the context and the destination buffer:
// Initialize SWR context
SwrContext* swrContext = swr_alloc_set_opts(NULL,
audioCodecContext->channel_layout, AV_SAMPLE_FMT_FLT, audioCodecContext->sample_rate,
audioCodecContext->channel_layout, audioCodecContext->sample_fmt, audioCodecContext->sample_rate,
0, NULL);
int result = swr_init(swrContext);
// Create destination sample buffer
uint8_t** destBuffer = NULL;
int destBufferLinesize;
av_samples_alloc_array_and_samples( &destBuffer,
&destBufferLinesize,
videoInfo.audioNumChannels,
2048,
AV_SAMPLE_FMT_FLT,
0);

Segmentation fault while avcodec_encode_video2

I have some problems while trying to encode a AVFrame to a packet.
Before reading the whole code, the input stuff is working, I tested it. The output stuff is from an example here. I think there is the problem. But the segmentation fault occurs in the loop near the end.
Here is my reduced code:
void nmain() {
// input stuff
AVFormatContext *formatCtxIn=0;
AVInputFormat *formatIn=0;
AVCodecContext *codecCtxIn=0;
AVCodec *codecIn;
AVPacket *pktIn;
av_register_all();
avdevice_register_all();
avcodec_register_all();
formatIn = av_find_input_format("dshow");
if(!formatIn)
return;
AVDictionary *avoption=0;
av_dict_set(&avoption, "rtbufsize", "1000000000", NULL);
if(avformat_open_input(&formatCtxIn, "video=Integrated Camera", formatIn, &avoption)!=0)
return;
if(avformat_find_stream_info(formatCtxIn, NULL)<0)
return;
codecCtxIn = formatCtxIn->streams[0]->codec;
codecIn = avcodec_find_decoder(codecCtxIn->codec_id);
if(avcodec_open2(codecCtxIn, codecIn, NULL)<0)
return;
// end input stuff
//------------------------------------------------------------------------------
// output stuff
AVOutputFormat *formatOut=0;
AVFormatContext *formatCtxOut=0;
AVStream *streamOut=0;
AVFrame *frame=0;
AVCodec *codecOut=0;
AVPacket *pktOut;
const char *filename = "test.mpeg";
formatOut = av_guess_format(NULL, filename, NULL);
if(!formatOut)
formatOut = av_guess_format("mpeg", NULL, NULL);
if(!formatOut)
return;
formatCtxOut = avformat_alloc_context();
if(!formatCtxOut)
return;
formatCtxOut->oformat = formatOut;
sprintf(formatCtxOut->filename, "%s", filename);
if(formatOut->video_codec != AV_CODEC_ID_NONE) {
AVCodecContext *ctx;
codecOut = avcodec_find_encoder(formatOut->video_codec);
if(!codecOut)
return;
streamOut = avformat_new_stream(formatCtxOut, codecOut);
if(!streamOut)
return;
ctx = streamOut->codec;
ctx->bit_rate = 400000;
ctx->width = 352;
ctx->height = 288;
ctx->time_base.den = 25;
ctx->time_base.num = 1;
ctx->gop_size = 12;
ctx->pix_fmt = AV_PIX_FMT_YUV420P;
if(ctx->codec_id == AV_CODEC_ID_MPEG2VIDEO)
ctx->max_b_frames = 2;
if(ctx->codec_id == AV_CODEC_ID_MPEG1VIDEO)
ctx->mb_decision = 2;
if(formatCtxOut->oformat->flags & AVFMT_GLOBALHEADER)
ctx->flags |= CODEC_FLAG_GLOBAL_HEADER;
}
if(streamOut) {
AVCodecContext *ctx;
ctx = streamOut->codec;
if(avcodec_open2(ctx, codecOut, NULL) < 0)
return;
}
if(!(formatCtxOut->flags & AVFMT_NOFILE))
if(avio_open(&formatCtxOut->pb, filename, AVIO_FLAG_WRITE) < 0)
return;
avformat_write_header(formatCtxOut, NULL);
// doit
pktIn = new AVPacket;
pktOut = new AVPacket;
av_init_packet(pktOut);
pktOut->data = 0;
frame = avcodec_alloc_frame();
if(!frame)
return;
for(;;) {
if(av_read_frame(formatCtxIn, pktIn) >= 0) {
av_dup_packet(pktIn);
int fff;
if(avcodec_decode_video2(codecCtxIn, frame, &fff, pktIn) < 0)
std::cout << "bad frame" << std::endl;
if(!fff)
return; // ok
static int counter=0;
SaveFrame(frame, codecCtxIn->width, codecCtxIn->height, counter++); // work fine
// here a segmentation fault is occured.
if(avcodec_encode_video2(streamOut->codec, pktOut, frame, &fff) < 0)
std::cout << "bad frame" << std::endl;
}
}
}
// only for testing
// add to ensure frame is valid
void SaveFrame(AVFrame *pFrame, int width, int height, int iFrame) {
FILE *pFile;
char szFilename[32];
int y;
// Open file
sprintf(szFilename, "frame%d.ppm", iFrame);
pFile=fopen(szFilename, "wb");
if(pFile==NULL)
return;
// Write header
fprintf(pFile, "P6\n%d %d\n255\n", width, height);
// Write pixel data
for(y=0; y<height; y++)
fwrite(pFrame->data[0]+y*pFrame->linesize[0], 1, width*3, pFile);
// Close file
fclose(pFile);
}
What am i doing wrong?
While debugging i didn't found any problems with the parameters. streamOut->codec is filled. pktOut is allocated and frame is filled with the picture encoded before.
I think the problem is while creating the output codec but watching the example and looking to the doxypages it seems to be correct.
The trace route is from QT using msvc11 and framework 5.
I also tried to run with dr. memory and get this:
Error #26: UNADDRESSABLE ACCESS: reading 0x00000000-0x00000004 4 byte(s)
# 0 replace_memcpy [d:\derek\drmemory\withwiki\trunk\drmemory\replace.c:203]
# 1 avcodec-54.dll!ff_dct_quantize_c +0xd463 (0x6a482364 <avcodec-54.dll+0x3c2364>)
# 2 avcodec-54.dll!avcodec_encode_video2+0xb7 (0x6a56a5b8 <avcodec-54.dll+0x4aa5b8>)
# 3 nmain [d:\prg\tests\recording system-qt\libav\recsys\main.cpp:610]
# 4 main [d:\prg\tests\recording system-qt\libav\recsys\main.cpp:182]
Note: #0:00:06.318 in thread 5312
Note: instruction: mov (%edx) -> %ebx
It seems like the reading process while memcpy is going wrong.
Version:
I've forgot to mention the version of libav/ffmpeg i'm using:
libavutil 51. 76.100 / 51. 76.100
libavcodec 54. 67.100 / 54. 67.100
libavformat 54. 33.100 / 54. 33.100
libavdevice 54. 3.100 / 54. 3.100
libavfilter 3. 19.103 / 3. 19.103
libswscale 2. 1.101 / 2. 1.101
libswresample 0. 16.100 / 0. 16.100
libpostproc 52. 1.100 / 52. 1.100
Addendum:
Function SafeFrame is copied from tutorial 1.
Finally i solved my problem.
The problem is (apart from the documentation of libav) avpacket is not a (real) copy of the picture in the packet. it just points to the data of the packet. You have to make a copy, or better you have to let it libav do.
So first i created a new avframe for the output and a buffer on which the output avframe is pointing to.
AVFrame *outpic = avcodec_alloc_frame();
nbytes = avpicture_get_size(codecCtxOut->pix_fmt, codecCtxOut->width, codecCtxOut->height);
uint8_t* outbuffer = (uint8_t*)av_malloc(nbytes);
This buffer is used for the conversion from input to output. Then in the loop i have to fillup the outpic (avframe) with the buffer.
I have found in the code that this function is filling up the plane pointers with the buffer.
see here
avpicture_fill((AVPicture*)outpic, outbuffer, AV_PIX_FMT_YUV420P, codecCtxOut->width, codecCtxOut->height);
Then i converted the inpic to outpic using sws_scale. But first you have to setup the swscontext.
SwsContext* swsCtx_ = sws_getContext(codecCtxIn->width, codecCtxIn->height,
codecCtxIn->pix_fmt,
codecCtxOut->width, codecCtxOut->height,
codecCtxOut->pix_fmt,
SWS_BICUBIC, NULL, NULL, NULL);
sws_scale(swsCtx_, inpic->data, inpic->linesize, 0, codecCtxIn->height, outpic->data, outpic->linesize);
Then you can encode the outpic into pktout (avpacket for output). But first do free the output packet, otherwise you will get an error and a leak... see here
av_free_packet(pktOut);
if(avcodec_encode_video2(streamOut->codec, pktOut, outpic, &fff) < 0) {
std::cout << "shit frame" << std::endl;
continue;
}
// and write it to the file
formatOut->write_packet(formatCtxOut, pktOut);
So now it works (nearly fine) for me. Still a small memory leak, but this i can spot later.
I see at least two problems with that transcoding loop:
1) You're not checking whether the decoder produced a frame. Many decoders have a delay between input and output, so a decode call won't necessarily produce a frame even if no error occurs. You just have to keep passing packets to the decoder until it starts returning frames (and then flush it with NULL packets at the end, as described in the documentation).
The result is that you're passing an uninitialized frame to the encoder, which is probably the reason for the crash.
2) Another problem I see is that you're initing the ouput packet only once. As the documentation says
The user can supply an output buffer by setting avpkt->data and avpkt->size prior to calling the function, but if the size of the user-provided data is not large enough, encoding will fail. All other AVPacket fields will be reset by the encoder using av_init_packet(). If avpkt->data is NULL, the encoder will allocate it. The encoder will set avpkt->size to the size of the output packet. The returned data (if any) belongs to the caller, he is responsible for freeing it.
So if you only initialize it once before starting the transcode loop, on each iteration after the first it will contain the old data. The encoder will think you want to use that buffer for encoding and overwrite it. This will end in tears if you've already passed that packet to a muxer or something like that. So make sure to init packet data and size to NULL/0 before each encode call.

ffmpeg(libavcodec). memory leaks in avcodec_encode_video

I'm trying to transcode a video with help of libavcodec.
On transcoding big video files(hour or more) i get huge memory leaks in avcodec_encode_video. I have tried to debug it, but with different video files different functions produce leaks, i have got a little bit confused about that :). Here FFMPEG with QT memory leak is the same issue that i have, but i have no idea how did that person solve it. QtFFmpegwrapper seems to do the same i do(or i missed something).
my method is lower. I took care about aFrame and aPacket outside with av_free and av_free_packet.
int
Videocut::encode(
AVStream *anOutputStream,
AVFrame *aFrame,
AVPacket *aPacket
)
{
AVCodecContext *outputCodec = anOutputStream->codec;
if (!anOutputStream ||
!aFrame ||
!aPacket)
{
return 1;
/* NOTREACHED */
}
uint8_t * buffer = (uint8_t *)malloc(
sizeof(uint8_t) * _DefaultEncodeBufferSize
);
if (NULL == buffer) {
return 2;
/* NOTREACHED */
}
int packetSize = avcodec_encode_video(
outputCodec,
buffer,
_DefaultEncodeBufferSize,
aFrame
);
if (packetSize < 0) {
free(buffer);
return 1;
/* NOTREACHED */
}
aPacket->data = buffer;
aPacket->size = packetSize;
return 0;
}
The first step would be to try to reproduce your problem under Valgrind on a Linux box, if you can.
ffmpeg encoders and decoders usually not dynamically allocate memory; they reuse buffers between calls. Leaks are usually going to be in the frames somewhere.
Note that av_free_packet will only free your dynamically allocated buffer if the packet has a destructor function!
Look at how the function is defined in libavcodec/avpacket.c:
void av_free_packet(AVPacket *pkt)
{
if (pkt) {
if (pkt->destruct) pkt->destruct(pkt);
pkt->data = NULL; pkt->size = 0;
pkt->side_data = NULL;
pkt->side_data_elems = 0;
}
}
If there is no pkt->destruct function, no clean up takes place!