Programmatically creating a video using FFmpeg, using SDL's sprite screenshot BMP - c++

Question

I have an animation/sprite developed in C++ on SDL2 libs (based on this answer). The bitmaps are saved to a certain path. They are of dimensions 640x480 and format is given by the SDL constant SDL_PIXELFORMAT_ARGB8888.
I have a second program written in C on top of FFmpeg libs, which reads one image from the above path (just one for the time being, will read the whole series when it works for just one).
This does the following (in gist - skipping validation & comments for conciseness)
AVCodec *codec;
AVCodecContext *c = NULL;
int i, ret, x, y, got_output;
FILE *f;
AVFrame *frame;
AVPacket pkt;
uint8_t endcode[] = { 0, 0, 1, 0xb7 };
codec = avcodec_find_encoder(codec_id);
c = avcodec_alloc_context3(codec);
c->bit_rate = 400000;
/* resolution must be a multiple of two */
c->width = 640;
c->height = 480;
c->time_base = (AVRational ) { 1, 25 };
c->gop_size = 5;
c->max_b_frames = 1;
c->pix_fmt = AV_PIX_FMT_YUV420P;
av_opt_set(c->priv_data, "preset", "slow", 0);
avcodec_open2(c, codec, NULL);
fopen(filename, "wb");
frame = av_frame_alloc();
av_image_alloc(frame->data, frame->linesize, c->width, c->height, c->pix_fmt, 32);
for (i = 0; i < 25; ++i) {
readSingleFile("/tmp/alok1/ss099.bmp", &frame->data);//Read the saved BMP into frame->data
frame->pts = i;
frame->width = 640;
frame->height = 480;
frame->format = -1;
av_init_packet(&pkt);
pkt.data = NULL; // packet data will be allocated by the encoder
pkt.size = 0;
ret = avcodec_encode_video2(c, &pkt, frame, &got_output);
if (got_output) {
printf("Write frame %3d (size=%5d)\n", i, pkt.size);
fwrite(pkt.data, 1, pkt.size, f);
}
av_packet_unref(&pkt);
}
for (got_output = 1; got_output; i++) {
fflush(stdout);
ret = avcodec_encode_video2(c, &pkt, NULL, &got_output);
if (ret < 0) {
fprintf(stderr, "Error encoding frame\n");
exit(1);
}
if (got_output) {
printf("[DELAYED]Write frame %3d (size=%5d)\n", i, pkt.size);
fwrite(pkt.data, 1, pkt.size, f);
av_packet_unref(&pkt);
}
}
fwrite(endcode, 1, sizeof(endcode), f);
//cleanup
As a result of the above code(which compiles without trouble), I can get a video which plays for 1 second - this part is working as expected. Problem is that the image seen is a green full screen like below.
The image that is being read using the readSingleImage(...) function is rendered by image viewer(linux, gwenview and okular) as follows:
Any pointers as to what could be going wrong?

Answer 1

To sum up the comments:
Encoder expects raw image data in format specified upon opening; it will not try to convert anything
Colorspace/format conversion has to be done manually; use swscale
if you are using Windows API to load image: it uses BGR, not RGB
BMP files usually but not always store image bottom-up as opposed to top-down used by FFmpeg; if it is bottom-up then it has to be flipped (there might be a way to do that without much or any performance hit if combined with colorspace conversion).
Also keep an eye on linesizes. Each line in an image can occupy more bytes than its width. This applies both to images allocated by ffmpeg and to images loaded from BMP - one has to be careful to always provide valid linesizes to each API call.
In addition to the above, below are "must-reads" for the final solution:
1. Taking a screenshot with SDL
2. RGB to YUV conversion
3. FFmpeg-related source code was written with this as base