How to read alpha channel from .webm video using ffmpeg in c++ - c++

Background
I have a .webm file (pix_fmt: yuva420p) converted from .mov video file in order to reduce file size and I would like to read the video data using c++, so I followed using this repo as a reference.
This works perfectly on .mov video.
Problem
By using same repo, however, there is no alpha channel data (pure zeros on that channel) for .webm video but I can get the alpha data from .mov video.
Apparently many people already noticed that after the video conversion, ffmpeg somehow detect video as yuv420p + alpha_mode : 1 and thus alpha channel is not used but there is no one discuss about workaround of this.
I tried forcing pixel format during this part to use yuva420p but that just broke the whole program.
// Set up sws scaler
if (!sws_scaler_ctx) {
auto source_pix_fmt = correct_for_deprecated_pixel_format(av_codec_ctx->pix_fmt);
sws_scaler_ctx = sws_getContext(width, height, source_pix_fmt,
width, height, AV_PIX_FMT_RGB0,
SWS_BILINEAR, NULL, NULL, NULL);
}
I also verified my video that it contains alpha channel using other source so I am sure there is alpha channel in my webm video but I cannot fetch the data using ffmpeg.
Is there a way to fetch the alpha data? Other video format or using other libraries work as well as long as it does have some file compression but I need to access the data in c++.
Note: This is the code I used for converting video to webm
ffmpeg -i input.mov -c:v libvpx-vp9 -pix_fmt yuva420p output.webm

You have to force the decoder.
Set the following before avformat_open_input()
AVCodec *vcodec;
vcodec = avcodec_find_decoder_by_name("libvpx-vp9");
av_fmt_ctx->video_codec = vcodec;
av_fmt_ctx->video_codec_id = vcodec->id;
You don't need to set pixel format or any scaler args.
This assumes that your libavcodec is linked with libvpx.

Related

Encoding uncompressed avi using RAWVIDEO codec and RGB24

I coded an encoder using FFMPEG (c++). The requirements for this encoder are:
The output format should be uncompressed avi,
Preferably using RGB24/YUV444 pixel format since we do not want chroma subsampling.
Most standard players should support the format (windows media player (WMP), VLC)
Using the encoder I wrote, I can write a number of file types right now:
Lossless H.264 encoded video using the YUV420p pixel format and AVI container. (Obviously not uncompressed and chroma subsampled, however both WMP and VLC play without any problem.)
MPEG4 encoded video using the YUV420p pixel format and AVI container.(Obviously not uncompressed and chroma subsampled, however both WMP and VLC play without any problem.)
AYUV encoded video using the YUVA444P pixel format. (uncompressed as far as I understand and not chroma subsampled. However, VLC does not play this.)
FFV1 encoded video using the YUV444P pixel format. (lossless, and not chroma subsampled. However, WMP does not play this.)
The above is derived from this very usefull post.
So I am now looking into the RAWVIDEO encoder from FFMPEG. I can't get this to work and neither can I find an example in the FFMPEG documentation on how to use this encoder for writing video. Can somebody point me in the right direction or supply sample code for this?
Also, if there is another direction I should follow to meet my requirements feel free to point me to it.
Thanks in advance

Setting bitrate of video in FFmpeg

I use FFmpeg to record videos from a RTSP stream (the codec is H.264). It works. But I face a problem with the bitrate value. First, I set bitrate like below, but it doesn't work:
AVCodecContext *m_c;
m_c->bit_rate = bitrate_value;
Following this question I can set bitrate manually with this command:
av_opt_set(m_c->priv_data, "crf", "39", AV_OPT_SEARCH_CHILDREN);
But I have to test several times to choose value '39', which creates acceptable video quality. It's hard to do it again if I use another camera setting (image width, height, etc). Is there a way to set bitrate more easily, and adaptively?

Writing variable framerate videos in openCV

The steps I follow for writing a video file in openCV are as follows:
CvVideoWriter *writer =cvCreateVideoWriter(fileName, Codec ID, frameRate, frameSize); // Create Video Writer
cvWriteFrame(writer, frame); // Write frame
cvReleaseVideoWriter(&writer); // Release video writer
The above code snippet writes at a fixed frame rate. I need to write out variable frame rate videos. The approach I had used earlier with libx264 involved writing individual timestamps to each frame.
So, the question is how do I write timestamps to a frame in openCV - what is the specific API ? More generally, how do I create variable frame rate videos ?
I don't think it is possible to do this with OpenCV directly without modifying the code to give access under the hood. You would need to use a different library like libvlc to do so using the imem to get your raw RGB frames in OpenCV into a file. This link provides an example using imem with raw images loaded from OpenCV. You would just need to change the :sout options to save to the file you want using your preferred codec.

converting a UYVY FFmpeg

I want to read and show a video using opencv. I've recorded with Direct-show, the Video has UYVY (4:2:2) codec, since opencv can't read that format, I want to convert the codec to an RGB color model, I readed about ffmpeg and I want to know if it's possible to get this done with it ? if not if you a suggestion I'll be thankful.
As I explained to you before, OpenCV can read some formats of YUV, including UYVY (thanks to FFmpeg/GStreamer). So I believe the cv::Mat you get from the camera is already converted to the BGR color space which is what OpenCV uses by default.
I modified my previous program to store the first frame of the video as PNG:
cv::Mat frame;
if (!cap.read(frame))
{
return -1;
}
cv::imwrite("mat.png", frame);
for(;;)
{
// ...
And the image is perfect. Executing the command file on mat.png reveals:
mat.png: PNG image data, 1920 x 1080, 8-bit/color RGB, non-interlaced
A more accurate test would be to dump the entire frame.data() to the disk and open it with an image editor. If you do that keep in mind that the R and B channels will be switched.

How to write YUV 420 video frames from RGB data using OpenCV or other image processing library?

I have an array of rgb data generated from glReadPixels().
Note that RGB data is pixel packed (r1,g1,b1,r2,g2,b2,...).
How can I quickly write a YUV video frame using OpenCV or another C++ library, so that I can stream them to FFMPEG? Converting RGB pixel to YUV pixel is not a problem, as there are many conversion formula available online. However writing the YUV frame is the main problem for me. I have been trying to write the YUV video frame since the last few days and were not successful in doing that.
This is one of my other question about writing YUV frame and the issues that I encountered: Issue with writing YUV image frame in C/C++
I don't know what is wrong with my current approach in writing the YUV frame to a file.
So right now I may want to use existing library (if any), that accepts an RGB data, and convert them to YUV and write the YUV frame directly to a file or to a pipe. Of course it would be much better if I can fix my existing program to write the YUV frame, but you know, there is also a deadline in every software development project, so time is also a priority for me and my project team members.
FFmpeg will happily receive RGB data in. You can see what pixel formats FFmpeg supports by running:
ffmpeg -pix_fmts
Any entry with an I in the first column can be used as an input.
Since you haven't specified the pixel bit depth, I am going to assume it's 8-bit and use the rgb8 pixel format. So to get FFmpeg to read rgb8 data from stdin you would use the following command (I am cating data in but you would be supplying via your pipe):
cat data.rgb | ffmpeg -f rawvideo -pix_fmt rgb8 -s WIDTHxHEIGHT -i pipe:0 output.mov
Since it is a raw pixel format with no framing, you need to subsitite WIDTH and HEIGHT for the appropriate values of your image dimensions so that FFmpeg knows how to frame the data.
I have specifed the output as a MOV file but you would need to configure your FFmpeg/Red5 output accordingly.
OpenCV does not support the YUV format directly, as you know, so it's really up to you to find a way to do RGB <-> YUV conversions.
This is a very interesting post as it shows how to load and create YUV frames on the disk, while storing the data as IplImage.
ffmpeg will write an AVI file with YUV but as karl says there isn't direct support for it in openCV.
Alternatively (and possibly simpler) you can just write the raw UYVY values to a file and then use ffmpeg to convert it to an AVI/MP4 in any format you want. It's also possible to write directly to a pipe and call ffmpeg directly from your app avoiding the temporary yuv file
eg. to convert an HD yuv422 stream to a h264 MP4 file at 30fps
ffmpeg -pix_fmt yuyv422 -s 1920x1080 -i input.yuv -vcodec libx264 -x264opts -r 30 output.mp4