Writing a video file using H.264 compression in OpenCV - c++

How do I write a video using H.264 compression with the VideoWriter class in OpenCV? I basically want to get a video from the webcam and save it after a character is pressed. The ouput video file is huge when using MPEG4 Part 2 compression.

You can certainly use the VideoWriter class, but you need to use the correct FourCC code that represents the the H264 standard. FourCC stands for Four Character Code, which is an identifier for a video codec, compression format, colour or pixel format used in media files.
Specifically, when you create a VideoWriter object, you specify the FourCC code when constructing it. Consult the OpenCV docs for more details: http://docs.opencv.org/trunk/modules/highgui/doc/reading_and_writing_images_and_video.html#videowriter-videowriter
I'm assuming you're using C++, and so the definition of the VideoWriter constructor is:
VideoWriter::VideoWriter(const String& filename, int fourcc,
double fps, Size frameSize, bool isColor=true)
filename is the output of the video file, fourcc is the FourCC code for the code you wish to use, fps is the desired frame rate, frameSize is the desired dimensions of the video, and isColor specifies whether or not you want the video to be in colour. Even though FourCC uses four characters, OpenCV has a utility that parses FourCC and outputs a single integer ID which is used as a lookup to be able to write the correct video format to file. You use the CV_FOURCC function, and specify four single characters - each corresponding to a single character in the FourCC code of the codec you want. Note that CV_FOURCC is for OpenCV 2.x. It is recommended you use cv::Videowriter::fourcc for OpenCV 3.x and beyond.
Specifically, you would call it like this:
int fourcc = CV_FOURCC('X', 'X', 'X', 'X');
int fourcc = VideoWriter::fourcc('X', 'X', 'X', 'X');
Replace X with each character that belongs to the FourCC (in order). Because you want the H264 standard, you would create a VideoWriter object like so:
#include <iostream> // for standard I/O
#include <string> // for strings
#include <opencv2/core/core.hpp> // Basic OpenCV structures (cv::Mat)
#include <opencv2/highgui/highgui.hpp> // Video write
using namespace std;
using namespace cv;
int main()
{
VideoWriter outputVideo; // For writing the video
int width = ...; // Declare width here
int height = ...; // Declare height here
Size S = Size(width, height); // Declare Size structure
// Open up the video for writing
const string filename = ...; // Declare name of file here
// Declare FourCC code - OpenCV 2.x
// int fourcc = CV_FOURCC('H','2','6','4');
// Declare FourCC code - OpenCV 3.x and beyond
int fourcc = VideoWriter::fourcc('H','2','6','4');
// Declare FPS here
double fps = ...;
outputVideo.open(filename, fourcc, fps, S);
// Put your processing code here
// ...
// Logic to write frames here... see below for more details
// ...
return 0;
}
Alternatively, you could simply do this when declaring your VideoWriter object:
VideoWriter outputVideo(filename, fourcc, fps, S);
If you use the above, it's not required that you call open as this will automatically open up the writer for writing frames to file.
If you're not sure if H.264 is supported on your computer, specify -1 as the FourCC code, and a window should pop up when you run the code that displays all of the available video codecs that are on your computer. I'd like to mention that this only works for Windows. Linux or Mac OS doesn't have this window popping out when you specify -1. In other words:
VideoWriter outputVideo(filename, -1, fps, S);
You can choose which one is most suitable should H.264 not exist on your computer. Once that is done, OpenCV will create the right FourCC code to be input into the VideoWriter constructor so that you will get a VideoWriter instance that represents a VideoWriter that will write that type of video to file.
Once you have a frame ready, stored in frm for writing to the file, you can do either:
outputVideo << frm;
OR
outputVideo.write(frm);
As a bonus, here's a tutorial on how to read/write videos in OpenCV: http://docs.opencv.org/3.0-beta/doc/py_tutorials/py_gui/py_video_display/py_video_display.html - However, it's written for Python, but what is good to know is near the bottom of the link, there is a list of FourCC codes that are known to work for each operating system. BTW, the FourCC code they specify for the H264 standard is actually 'X','2','6','4', so if 'H','2','6','4' doesn't work, replace H with X.
Another small note. If you are using Mac OS, then what you need to use is 'A','V','C','1' or 'M','P','4','V'. From experience, 'H','2','6','4'or 'X','2','6','4'when trying to specify the FourCC code doesn't seem to work.

As rayryeng states, you need to pass the correct FourCC code to VideoWriter.
For H.264 most people use AVC, which would look like this:
cv2.VideoWriter_fourcc('a','v','c','1')
mp4v also seems to work for many. If they both don't, check your list of available codecs.

Related

How to read alpha channel from .webm video using ffmpeg in c++

Background
I have a .webm file (pix_fmt: yuva420p) converted from .mov video file in order to reduce file size and I would like to read the video data using c++, so I followed using this repo as a reference.
This works perfectly on .mov video.
Problem
By using same repo, however, there is no alpha channel data (pure zeros on that channel) for .webm video but I can get the alpha data from .mov video.
Apparently many people already noticed that after the video conversion, ffmpeg somehow detect video as yuv420p + alpha_mode : 1 and thus alpha channel is not used but there is no one discuss about workaround of this.
I tried forcing pixel format during this part to use yuva420p but that just broke the whole program.
// Set up sws scaler
if (!sws_scaler_ctx) {
auto source_pix_fmt = correct_for_deprecated_pixel_format(av_codec_ctx->pix_fmt);
sws_scaler_ctx = sws_getContext(width, height, source_pix_fmt,
width, height, AV_PIX_FMT_RGB0,
SWS_BILINEAR, NULL, NULL, NULL);
}
I also verified my video that it contains alpha channel using other source so I am sure there is alpha channel in my webm video but I cannot fetch the data using ffmpeg.
Is there a way to fetch the alpha data? Other video format or using other libraries work as well as long as it does have some file compression but I need to access the data in c++.
Note: This is the code I used for converting video to webm
ffmpeg -i input.mov -c:v libvpx-vp9 -pix_fmt yuva420p output.webm
You have to force the decoder.
Set the following before avformat_open_input()
AVCodec *vcodec;
vcodec = avcodec_find_decoder_by_name("libvpx-vp9");
av_fmt_ctx->video_codec = vcodec;
av_fmt_ctx->video_codec_id = vcodec->id;
You don't need to set pixel format or any scaler args.
This assumes that your libavcodec is linked with libvpx.

OpenCV (C++): write produces big file compared to input

I am trying to read a video file using OpenCV (C++), apply a filter to each frame and write a new modified frame into an output file.
The crucial parts of the code are these:
int out_format = CV_FOURCC('M','P','4','2'); // can be another one
double fps = media.get(CV_CAP_PROP_FPS),
width = media.get(CV_CAP_PROP_FRAME_WIDTH),
height = media.get(CV_CAP_PROP_FRAME_HEIGHT);
// On Linux FFMPEG is used to write videos
VideoWriter writer("./" + outputname + ".mkv", out_format, fps, Size(width, height) );
.......
.......
writer.write(newFrame); // Mat newFrame
The fact is, I am not proficient with codecs and video output formats, thus I would like to know why a file which is around 280 MB produces an output of 2 GB.
Is that a codec problem? I have tried DIVX, MPEG and others. Moreover, some output formats must go together with specific codecs.
Try using the 'M','P', '4', 'V' codec with the .mp4 container, that is, set the extension to your filename as something.mp4
For the mp4 container, you could also try, MPEG, MJPG, FMP4

c++ opencv get encoded webcam stream

I am currently work on a project that capture video from webcam and send the encoded stream via UDP to do a real time streaming.
#include "opencv2/highgui/highgui.hpp"
#include <iostream>
using namespace cv;
using namespace std;
int main(int argc, char* argv[])
{
VideoCapture cap(0); // open the video camera no. 0
double dWidth = cap.get(CV_CAP_PROP_FRAME_WIDTH); //get the width of frames of the video
double dHeight = cap.get(CV_CAP_PROP_FRAME_HEIGHT); //get the height of frames of the video
while (1)
{
Mat frame;
bool bSuccess = cap.read(frame); // read a new frame from video
if (!bSuccess) //if not success, break loop
{
cout << "Cannot read a frame from video stream" << endl;
break;
}
return 0;
}
Some people say that the frame get from cap.read(frame) is already the decoded frame,I have no idea how and when does that happen. And what I want is the encoded frame or stream. What should I do to get it? Should I encoded it back again?
According to the docs, calling VideoCapture::read() is equivalent to calling VideoCapture::grab() then VideoCapture::retrieve().
The docs for the Retrieve function say it does indeed decode the frame.
Why not just use the decoded frame; presumably you'd be decoding it at the far end in any case?
OpenCV API does not give access to the encoded frames.
You will have to use a more low-level library, probably device and platform dependent. If your OS is Linux, Video4Linux2 may be an option, there must be equivalent libraries for Windows/MacOS. You may also have a look at mjpg-streamer, which does something very similar to what you want to achieve (on linux only).
Note that the exact encoding of the image will depend on your webcam, some usb webcam support mjpeg compression (or even h264), but other are only able to send raw data (usually in yuv colorspace).
Another option is to grab the decoded image wit Opencv, and reencode it, for example with imencode. It has the advantages of simplicity and portability, but image reencoding will use more resource.

Writing variable framerate videos in openCV

The steps I follow for writing a video file in openCV are as follows:
CvVideoWriter *writer =cvCreateVideoWriter(fileName, Codec ID, frameRate, frameSize); // Create Video Writer
cvWriteFrame(writer, frame); // Write frame
cvReleaseVideoWriter(&writer); // Release video writer
The above code snippet writes at a fixed frame rate. I need to write out variable frame rate videos. The approach I had used earlier with libx264 involved writing individual timestamps to each frame.
So, the question is how do I write timestamps to a frame in openCV - what is the specific API ? More generally, how do I create variable frame rate videos ?
I don't think it is possible to do this with OpenCV directly without modifying the code to give access under the hood. You would need to use a different library like libvlc to do so using the imem to get your raw RGB frames in OpenCV into a file. This link provides an example using imem with raw images loaded from OpenCV. You would just need to change the :sout options to save to the file you want using your preferred codec.

Size in kbytes of AVI of video in C++/OpenCV

Is there a way to know the size of an avi video in OpenCV? There are some videos that are blank and instead of processing them I want to get rid of them by comparing their size to a threshold. For example, if the size of the video is 200 kB or less, then I skip processing that video.
Update:
CvCapture *capture = cvCreateFileCapture(path);
IplImage *img = cvQueryFrame( capture );
All videos have the same number of frames but some videos have all blank frames.
You can get number of frames by getting CV_CAP_PROP_FRAME_COUNT video property using GetCaptureProperty:
CV_CAP_PROP_FRAME_COUNT Number of frames in the video file.
To get size of file (video) there is another API - depends on your system (WinAPI, POSIX etc).
For example on Windows it's GetFileSizeEx() function.
Also look at this SO discussion.
If you want to get the actual file size, you will have to use fopen() or stat as OpenCV doesn't store this information in a cvCapture object. However, you can detect blank videos like so:
CvCapture *capture = cvCaptureFromAVI( argv[1] );
int numFrames = (int) cvGetCaptureProperty(&capture, CV_CAP_PROP_FRAME_COUNT);
Then just decide if it's blank based on the number of frames.
Relevant Documentation:
http://opencv.willowgarage.com/documentation/reading_and_writing_images_and_video.html?highlight=cvcapture#getcaptureproperty
If the file size is different for the blank videos, but the files have the same number of frames, it's due to video compression via keyframing. That means the best way to detect blank frames is via getting the file size, which is outside of the scope of OpenCV and can easily be accomplished using plain C/C++:
#include <sys/stat.h>
...
struct stat st;
stat(avi_filename, &st);
size = st.st_size;