Programmatically read fMP4 using ffmpeg - c++

Question

All of my attempts to parse fmp4 have failed with the following errors:
avcodec_send_packet returned with -10949955291 and printed the following errors:
[h264 # 0x105001600] No start code is found.
[h264 # 0x105001600] Error splitting the input into NAL units.
What did i do?
Downloaded a perfectly playable h.264 file and fragmented it using the following cmd
ffmpeg -i long.mp4 -an -sn -vcodec libx264 -force_key_frames "expr:gte(t,n_forced*4)" -sc_threshold 0 -r 25 -f hls -hls_time 4 -hls_list_size 99999 -start_number 1 -hls_segment_type fmp4 -hls_fmp4_init_filename init.mp4 -t 30 -threads 0 big_bunny.m3u8
and used the following AVIO read example
#include <fstream>
#include <iterator>
#include <algorithm>
#include <libavcodec/avcodec.h>
std::vector<unsigned char> b1;
std::vector<unsigned char> b2;
bool initialized = false;
int bytesread = 0;
static int read_packet(void* opaque, uint8_t* buf, int buf_size)
{
if (b1.size() && !initialized) {
size_t bytesToRead = std::min(buf_size, (int)b1.size());
::memcpy(buf, b1.data(), bytesToRead);
bytesread += bytesToRead;
if (bytesread >= b1.size()) {
initialized = true;
bytesread = 0;
}
return bytesToRead;
}
::memcpy(buf, b2.data() + bytesread, buf_size);
bytesread += buf_size;
return buf_size;
}
int main(int argc, char** argv)
{
AVFormatContext* fmt_ctx = NULL;
AVIOContext* avio_ctx = NULL;
uint8_t *buffer = NULL, *buffer2 = nullptr, *avio_ctx_buffer = NULL;
size_t buffer_size, buffer_size2, avio_ctx_buffer_size = 4096;
char* input_filename = NULL;
int ret = 0;
struct buffer_data bd = {0};
std::ifstream input1("/Users/x/Downloads/fmp4/init.mp4", std::ios::binary);
std::ifstream input2("/Users/x/Downloads/fmp4/big_bunny1.m4s", std::ios::binary);
b1 = std::vector<unsigned char>((std::istreambuf_iterator<char>(input1)), std::istreambuf_iterator<char>());
b2 = std::vector<unsigned char>((std::istreambuf_iterator<char>(input2)), std::istreambuf_iterator<char>());
avio_ctx_buffer = (uint8_t*)av_malloc(avio_ctx_buffer_size);
fmt_ctx = avformat_alloc_context();
avio_ctx = avio_alloc_context(avio_ctx_buffer, avio_ctx_buffer_size, 0, nullptr, &read_packet, NULL, NULL);
fmt_ctx->pb = avio_ctx;
AVDictionary* opts = NULL;
// av_dict_set(&opts, "movflags", "frag_keyframe+empty_moov", 0);
ret = avformat_open_input(&fmt_ctx, NULL, NULL, &opts);
ret = avformat_find_stream_info(fmt_ctx, NULL);
AVCodec* decoder = nullptr;
decoder = avcodec_find_decoder(fmt_ctx->streams[0]->codecpar->codec_id);
AVCodecContext* decoderCtx = avcodec_alloc_context3(decoder);
ret = avcodec_open2(decoderCtx, decoder, nullptr);
AVPacket pkt;
av_init_packet(&pkt);
AVFrame* frame = av_frame_alloc();
while (true) {
ret = av_read_frame(fmt_ctx, &pkt);
ret = avcodec_send_packet(decoderCtx, &pkt);
if (ret != 0)
assert(false);
for (;;) {
ret = avcodec_receive_frame(decoderCtx, frame);
if (ret < 0) {
break;
}
int g;
g = 0;
}
}
I'm not even sure that this is the right way to process fmp4 types. but for the clarity of this example, i simply loaded the init file into the first buffer, and the actual media file on the second buffer, and switch between the buffers respectfully to buf_size's value.

Answer 1

I don't think that in a case where you use avio_alloc_context demuxer get's all the needed info - see Opening a media file, but this is a wild guess since your output shows no errors from the mp4 demuxer but only from the H.264 bitstream parser, so I am leaning that the probed file format could be raw H.264, but it is hard to tell. But if I am mistaken, you have not provided a seek callback, and demuxing an mp4 would certainly require seeking to jump around the boxes and then seek to the sample data. So, I'd first try to provide a seek callback, and see if that gets called. But from the example it is not sure why you even need the I/O callbacks, since the data reside in a file not in a memory location, so I'd suggest trying to use the standard avformat_open_input way.