What is AVHWAccel, and how can I use it? - c++

Question

I want to make use of hardware acceleration for decoding an h264 encoded MP4 file.
My computing environment:
Hardware: MacPro (2015 model)
Software: FFmpeg (installed by brew)
Here is the output of FFmpeg command:
$ffmpeg -hwaccels
Hardware acceleration methods:
vda
videotoolbox
According to this document, there are two options for my environment, that is, VDA and VideoToolBox. I tried VDA in C++:
Codec = avcodec_find_decoder_by_name("h264_vda");
It kind of worked, but the output of the pixel format is UYVY422 which I have trouble to deal with (any suggestion on how to render UYVY422 in C++? The ideal format is yuv420p)
So I want to try VideotoolBox, but there is no such simple thing like (it may work in the case of encoding though)
Codec = avcodec_find_decoder_by_name("h264_videotoolbox");
It seems I should use AVHWAccel, but what is AVHWAccel and how to use it?
Part of My C++ code:
for( unsigned int i = 0; i < pFormatCtx->nb_streams; i++ ){
if(pFormatCtx->streams[i]->codec->codec_type == AVMEDIA_TYPE_VIDEO){
pCodecCtx = pFormatCtx->streams[i]->codec;
video_stream = pFormatCtx->streams[i];
if( pCodecCtx->codec_id == AV_CODEC_ID_H264 ){
//pCodec = avcodec_find_decoder(pCodecCtx->codec_id);
pCodec = avcodec_find_decoder_by_name("h264_vda");
break;
}
}
}
// open codec
if( pCodec ){
if((ret=avcodec_open2(pCodecCtx, pCodec, NULL)) < 0) {
....

Answer 1

It's nothing to do with the decoder for which pix format to choose.
Your video pix format is UYVY422, so you got this format after you decode the frame.
Like the answer #halfelf mentioned, you can perform a swscale after you decode a frame, to convert the pix format to your ideal format yuv420p, then render it.
Meanwhile, if you are sure it's the format UYVY422, SDL2 can handle the render directly for you.
In the example below, my format is yuv420p, and I use swscale to convert to UYVY422 to render to SDL2
// prepare swscale context, AV_PIX_FMT_UYVY422 is my destination pix format
SwsContext *swsCtx = sws_getContext(codecCtx->width, codecCtx->height, codecCtx->pix_fmt,
codecCtx->width, codecCtx->height, AV_PIX_FMT_UYVY422,
SWS_FAST_BILINEAR, NULL, NULL, NULL);
SDL_Init(SDL_INIT_EVERYTHING);
SDL_Window *window;
SDL_Renderer *render;
SDL_Texture *texture;
SDL_CreateWindowAndRenderer(codecCtx->width,
codecCtx->height, SDL_WINDOW_OPENGL, &window, &render);
texture = SDL_CreateTexture(render, SDL_PIXELFORMAT_UYVY, SDL_TEXTUREACCESS_STREAMING,
codecCtx->width, codecCtx->height);
// ......
// decode the frame
// ......
AVFrame *frameUYVY = av_frame_alloc();
av_image_alloc(frameUYVY->data, frameUYVY->linesize, codecCtx->width, codecCtx->height, AV_PIX_FMT_UYVY422, 32);
SDL_LockTexture(texture, NULL, (void **)frameUYVY->data, frameUYVY->linesize);
// convert the decoded frame to destination frameUYVY (yuv420p -> uyvy422)
sws_scale(swsCtx, frame->data, frame->linesize, 0, frame->height,
frameUYVY->data, frameUYVY->linesize);
SDL_UnlockTexture(texture);
// performa render
SDL_RenderClear(render);
SDL_RenderCopy(render, texture, NULL, NULL);
SDL_RenderPresent(render);
In your example, if your pix format is uyvy422, you can skip the swscale part, and perform the render directly after decode from ffmpeg.

Answer 2

Decoders won't choose which pixel format the output is, it is determined by the video itself. swscale lib is used to convert one pixel format to another.
auto sws_ctx = sws_getContext(src_width, src_height, AV_PIX_FMT_UYUV422, dst_width, dst_height, AV_PIX_FMT_YUV420P, 0,0,0,0);
av_image_alloc(new_data, new_linesize, dst_width, dst_height, AV_PIX_FMT_BGR24, FRAME_ALIGN);
sws_scale(sws_ctx, frame->data, frame->linesize, 0, src_height, new_data, new_linesize);
And there is no h264_videotoolbox decoder, only encoder. To list decoders/encoders available:
ffmpeg -encoders
ffmpeg -decoders
The decoder/encoder names is written in the source, for example, at the end of libavcodec/vda_h264_dec.c and libavcodec/videotoolboxenc.c.