I have 16 bit PCM data with stereo setup which I grab from a microphone.
Once I get the data I encode it using the following encoder settings
AVCodec* audio_codec = avcodec_find_encoder(AV_CODEC_ID_MP2);
AVCodecContext* audio_codec_ctx = avcodec_alloc_context3(audio_codec);
audio_codec_ctx->bit_rate = 64000;
audio_codec_ctx->channels = 2;
audio_codec_ctx->channel_layout = AV_CH_LAYOUT_STEREO;
audio_codec_ctx->sample_rate = 44100;
audio_codec_ctx->sample_fmt = AV_SAMPLE_FMT_S16;
When I pass the audio data into the encoder I see that it takes 4608 bytes of data each time and encodes it correctly to MP2 data. PCM data grabbed by the microphone is 88320 bytes and the encoder takes 4608 bytes each time and compresses it.
If I take each 4608 byte section that has been encoded and pass it through a decoder with the same settings as above but with a decoder.
AVCodecID audio_codec_id = AV_CODEC_ID_MP2;
AVCodec * audio_decodec = avcodec_find_decoder(audio_codec_id);
audio_decodecContext = avcodec_alloc_context3(audio_decodec);
audio_decodecContext->bit_rate = 64000;
audio_decodecContext->channels = 2;
audio_decodecContext->channel_layout = AV_CH_LAYOUT_STEREO;
audio_decodecContext->sample_rate = 44100;
audio_decodecContext->sample_fmt = AV_SAMPLE_FMT_S16;
The decoding works and is successful but when I look at the data size it is exactly half 2034 of what was encoded. I dont understand why that would be. I would of imagined I would get 4608 considering the encoder and decoder are the same.
Can anyone shed some light into why this would be happening. Anything I should be setting?
The requested decoder sample format should be set using audio_decodecContext->request_sample_fmt. sample_fmt is set by the decoder itself, and may be different, in which case you should use libswresample to convert between sample formats.
Related
I need to record frames in real time. To test this situation, I make pts non-linear (since frames may be lost), thus:
// AVFrame
video_frame->pts = prev_pts + 2;
I use libavformat to write to a file. Parameters AVCodecContext and AVStream:
#define STREAM_FRAME_RATE 25
#define CODEC_PIX_FMT AV_PIX_FMT_YUV420P
#define FRAME_WIDTH 1440
#define FRAME_HEIGHT 900
// AVCodecContext
cc->codec_id = video_codec->id;
cc->bit_rate = 400000;
cc->width = FRAME_WIDTH;
cc->height = FRAME_HEIGHT;
cc->gop_size = 12;
cc->pix_fmt = CODEC_PIX_FMT;
// AVStream
video_stream->time_base = AVRational{ 1, STREAM_FRAME_RATE };
cc->time_base = video_stream->time_base;
cc->framerate = AVRational{ STREAM_FRAME_RATE , 1 };
Write to file:
static int write_frame(AVFormatContext *fmt_ctx, const AVRational *time_base, AVStream *st, AVPacket *pkt)
{
/* rescale output packet timestamp values from codec to stream timebase */
//av_packet_rescale_ts(pkt, *time_base, st->time_base);
pkt->pts = av_rescale_q(pkt->pts, *time_base, st->time_base);
pkt->dts = av_rescale_q(pkt->dts, *time_base, st->time_base);
pkt->stream_index = st->index;
/* Write the compressed frame to the media file. */
//log_packet(fmt_ctx, pkt);
//return av_write_frame(fmt_ctx, pkt);
return av_interleaved_write_frame(fmt_ctx, pkt);
}
If you use the avi container, then the information on the number of frames per second is indicated correctly in the file: 25 fps
If you use the mp4 container, then the file information about the number of frames per second is indicated incorrectly: 12.5 fps
Tell me, please, what other settings need to be added?
MP4s do not store framerate, AVIs do.
In MP4s, only timing info for packets is stored. Since your pts expr is video_frame->pts = prev_pts + 2 and stream time base is 1/25, frames are spaced 80ms apart and hence ffmpeg probes the frame rate to be 12.5 fps (correctly).
AVIs do not have per-frame timing. Instead, they write the user-supplied frame rate. Should a packet timing be greater than the pervious frame pts by 1/fps, the muxer will write skip frame(s) which are empty packets, to maintain the frame rate.
Insights to encode/decode video with ffmpeg h264 (lossless)
So I got something working on the encoding part, encode an avi in 264 however VLC wont play it, however Totem will.
Decoding the same file proves troublesome. (I want the exact same data/frame going in as going out), I get these ;
saving frame 5
Video decoding
[h264 # 0x1d19880] decode_slice_header error
frame :6
saving frame 6
Video decoding
[h264 # 0x1d19880] error while decoding MB 15 7, bytestream -27
[h264 # 0x1d19880] concealing 194 DC, 194 AC, 194 MV errors in I frame
frame :7
saving frame 7
Video decoding
[h264 # 0x1d19880] decode_slice_header error
and ultimatly this
[H264 Decoder # 0x7f1320766040] frame :11
Broken frame packetizing
[h264 # 0x1d19880] SPS changed in the middle of the frame
[h264 # 0x1d19880] decode_slice_header error
[h264 # 0x1d19880] no frame!
Error while decoding frame 11
GAME OVER.
Now I suspect that I have to go back to 1. the encoding part, there is problary a good reason VLC wont play it!
I encode like this.
void encode(char *Y,char *U,char *V){
av_init_packet(&pkt);
pkt.data = NULL; // packet data will be allocated by the encoder
pkt.size = 0;
fflush(stdout);
frame->data[0] = (uint8_t*)Y;
frame->data[1] = (uint8_t*)U;
frame->data[2] = (uint8_t*)V;
frame->pts = ++i;
ret = avcodec_encode_video2(c, &pkt, frame, &got_output);
if (ret < 0) {
fprintf(stderr, "Error encoding frame\n");
exit (EXIT_FAILURE);
}
if (got_output) {
printf("Write frame %3d (size=%5d)\n", i, pkt.size);
fwrite(pkt.data, 1, pkt.size, f);
av_free_packet(&pkt);
}
}
And the codec is setup like this:
AVCodecID dasd = AV_CODEC_ID_H264;
codec = avcodec_find_encoder(dasd);
c = avcodec_alloc_context3(codec);
c->bit_rate = 400000;
c->width = 320;
c->height = 240;
c->time_base= (AVRational){1,25};
c->gop_size = 10;
c->max_b_frames=1;
c->pix_fmt = AV_PIX_FMT_YUV420P;
av_opt_set(c->priv_data, "preset", "slow", 0);
avcodec_open2(c, codec, NULL);
Since I am going for lossless i am not dealing with delayed frames(is this a correct assumption?)
I may not actually be encoding lossless, it seems like I may have to go with something like
AVDictionary *param;
av_dict_set(¶m, "qp", "0", 0);
And then open...
So I guess me questions is these :
What are the correct codec params for lossless encoding (and advice if h264 is a terrible idea in this regard).
Do I need to handle delayed frames when going for lossless?
Why is VLC mad at me?
Thanks.
To achieve lossless: av_dict_set(¶m, "crf", "0", 0);
Delayed frames (B-frames) has nothing to do with lossless. If you need low-delay, then don't use B-frames.
Some thing is seriously wrong in your encoding. The error "MV errors in I frame" is odd one here, there shouldn't be any MVs in I-frame. It seems the header parsing it-self gone wrong. Please share the bit-stream & more details for VLC failure
You're writing raw annexb frames into a file without any container wrapping. Use a container like mp4 or matroska and VLC should be happy.
I want to write silence/zeroed audio sampled data into mov media container file inside audio data. My audio data is G711 linear PCM-mulaw encoded data with one channel. Currently my code looks like:
AVFrame* pSilentData = av_frame_alloc();
memset(&pSilentData->data[0], 0, iDataSize);
pkt.data = (uint8_t*) pSilentData;
pkt.size = iDataSize;
// ...
av_freep(&pSilentData->data[0]);
av_frame_free(&pSilentData);
But this sounds noise like dot dot instead of silence. What's the problem?
For ยต-law audio the zero value is represented as 0xff, so change:
memset(&pSilentData->data[0], 0, iDataSize);
to:
memset(&pSilentData->data[0], 0xff, iDataSize);
I'm trying to encode images into an H264 MP4 video. The issues I'm having is that some of the images are skipped or at the end of the video simply missing. I need the video to play every single image I encode since it is an animation.
Any help setting the encoder properly would be greatly appreciated!
Encoder settings:
AVCodecContext *c;
...
c->codec_id = AV_CODEC_ID_H264;
c->bit_rate = mOutputWidth*mOutputHeight*4;//400000;
/* Resolution must be a multiple of two. */
c->width = mOutputWidth;
c->height = mOutputHeight;
/* timebase: This is the fundamental unit of time (in seconds) in terms
* of which frame timestamps are represented. For fixed-fps content,
* timebase should be 1/framerate and timestamp increments should be
* identical to 1. */
c->time_base.den = mFps;
c->time_base.num = 1;
c->gop_size = 12; /* emit one intra frame every twelve frames at most */
c->pix_fmt = AV_PIX_FMT_YUV420P;
...
av_dict_set(&pOptions, "preset", "medium", 0);
av_dict_set(&pOptions, "tune", "animation", 0);
/* open the codec */
ret = avcodec_open2(c, codec, &pOptions);
if (ret < 0) {
LOGE("Could not open video codec: %s", av_err2str(ret));
return -1;
}
Update 07/24/13:
I was able to achieve a better video by setting the gop_size=FPS and writing the last video frame repeatedly FPS+1 times seemed to resolve all issues. To me it seems odd to do that but might be something standard in the video encoding world? Any tips feedback about this?
From what I understand, you have a set of images and you want to make a video out of them. If this is the case and you don't care about the size of the video, you can try to disable inter prediction. Maybe the encoder finds that some of the images are not required and skips them.
Inter frame prediction can be disabled by setting gop_size to 0.
I'm trying ton encode video from set of jpeg images to h264, using ffmpeg + x264 for it. I init AVCodecContext in such way:
_outputCodec = avcodec_find_encoder(AV_CODEC_ID_H264);
_outputCodecContext = avcodec_alloc_context3(_outputCodec);
avcodec_get_context_defaults3(_outputCodecContext, _outputCodec);
_outputCodecContext->width = _currentWidth;
_outputCodecContext->height = _currentHeight;
_outputCodecContext->pix_fmt = AV_PIX_FMT_YUV420P;
_outputCodecContext->time_base.num = 1;
_outputCodecContext->time_base.den = 25;
_outputCodecContext->profile =FF_PROFILE_H264_BASELINE;
_outputCodecContext->level = 50;
avcodec_open return no errors, anything is OK, but when I call avcodec_encode_video2() I get such messages (I think it's from x264):
using mv_range_thread = %d
%s
profile %s, level %s
And then app crashs. My be there are more neccessary settings for codec context, when use x264 &&
Without a full version of your code it is hard to see what the actual problem is.
Firstly here is a working example of the FFMPEG library encoding RGB frames to a H264 video:
http://www.imc-store.com.au/Articles.asp?ID=276
You could expand on this example by using CImage to load in your JPGs and pass the RGB data to the FFMPEG Class to encode to a H264 video.
A few thoughts on your example though:
Have you called register all like below?
avcodec_register_all();
av_register_all();
Also I'd re-write your code to be something like below:
AVStream *st;
m_video_codec = avcodec_find_encoder(AV_CODEC_ID_H264);
st = avformat_new_stream(_outputCodec, m_video_codec);
_outputCodecContext = st->codec;
_outputCodecContext->codec_id = m_fmt->video_codec;
_outputCodecContext->bit_rate = m_AVIMOV_BPS; //Bits Per Second
_outputCodecContext->width = m_AVIMOV_WIDTH; //Note Resolution must be a multiple of 2!!
_outputCodecContext->height = m_AVIMOV_HEIGHT; //Note Resolution must be a multiple of 2!!
_outputCodecContext->time_base.den = m_AVIMOV_FPS; //Frames per second
_outputCodecContext->time_base.num = 1;
_outputCodecContext->gop_size = m_AVIMOV_GOB; // Intra frames per x P frames
_outputCodecContext->pix_fmt = AV_PIX_FMT_YUV420P;//Do not change this, H264 needs YUV format not RGB
And then you need to convert the RGB JPG picture to the YUV format using swscale as pogorskiy said.
Have a look at the linked example, I tested it on VC++2010 and it works perfectly and you can send it an RGB char array.