I'm trying to compare 3 videos that are encoded by H.264, H.265, and VP9.
All of them are made by a same YUV video.
I want to use OpenCV's function to read each frame of the video and do some comparison:
VideoCapture vCap1, vCap2, vCap3;
vCap1.open("h264.mp4");
vCap2.open("h265.mp4");
vCap3.open("vp9.webm");
Mat frame1, frame2, frame3;
while (vCap1.read(frame1) && vCap2.read(frame2) && vCap3.read(frame3))
{
//do something
}
The vCap1 opened successfully, but vCap2 and vCap3 won't open.
Did I miss something to include to make it work?
Or OpenCV even not support the other 2 formats?
After using google :-) I found that
http://answers.opencv.org/question/10741/videocapture-format-supported-by-opencv/
Especially you have the needed codecs installed on your system. You can visit also
http://www.fourcc.org/codecs.php
for codecs.
The documentation from OpenCV is indeed not very helpful. :-)
What I would try if you are running under linux:
strace -xfo dump
and take a look in the system calls. Maybe you can find some hints of missing codec files, used configuration files and or other failed system function calls. If so, you have a startpoint.
Related
I am trying to use libav to convert an MP4 file to an MP3 file. Basically trying to achieve what ffmpeg -i filename.mp4 filename.mp3 does. I've found this official example. But when I run it with an input MP4 and an output MP3 I get an error:
Invalid audio stream. Exactly one MP3 audio stream is required.
I am not at all familiar with this library but I think I have narrowed the problem down to this line:
ret = avcodec_parameters_copy(out_stream->codecpar, in_codecpar);
It seems to copy all streams for a video file but we only need one for the MP3 file? I am not sure. There doesn't seem to be a function to copy only the parameters relevant to audio. I checked the sources, avcodec_parameters_copy does a simple memcpy.
Questions:
Is this the actual problem?
How do I solve it?
Am I on the right track to achieve the goal of extracting audio from a video file? I've seen this question (and other similar questions like this and this) on here but none seem to have a complete code example. The C API documentation for this library is also a little lacking.
You can have multiple audio tracks in mp4 file, but only one such track in an mp3 file. The easiest fix for the remuxing example would be to replace lines 101-103:
if (in_codecpar->codec_type != AVMEDIA_TYPE_AUDIO ||
stream_index != 0) {
This, naturally, is relevant only if the output is mp3.
PS, make sure that your input mp4 uses the MP3 audio codec. If it does not (and most have AAC or AC3 these days), it's not enough to remux the file, you also need to decode and re-encode the audio stream.
I am using flycapture sdk sample program to capture image form the flycapture.
My problem is that when i capture the image using the flycapture installed application the size of image is about 1.3 - 1.5 Mb. But when the take the same image using my program which consist of flycapture sample program. The size of the image is about 340K to 500K(max).Image format is .tiff
There is reduction in the quality of the image due to which my program is not able to get any valuable information form the image.
Using the following approach to save the image:
FlyCapture2::Camera camera;
FlyCapture2::Image image;
camera.RetrieveBuffer(&image);
ostringstream saveImage;
saveImage << "Image-" << "-" << i << ".tiff";
image.Save(saveImage.str().c_str());
And using the windows application following the approach mentioned in the link:
http://www.ptgrey.com/Content/Images/uploaded/FlyCapture2Help/flycapture/03demoprogram/saving%20images_flycap2.html
Please let me of any other details required
I am not 100% sure about this, since the documentation I found was for Java and not c++, but it is probably very similar.
You are using :
image.Save(saveImage.str().c_str());
to save your image, but are you sure it is saved as a tiff? the documentation (the java one), doesn't go deep into this, I am not sure if it is like OpenCV's imwrite that it automatically deduces the type and does it or not. So you should check that. There was one overload that you can pass the ImageFileFormat... this should be set to the TIFF one.
Another overload let's you specify the TIFF Options... in here you may tune it to have a different compression method. Notice that there is JPEG compression method... which would make something wayyy lighter but lossy... You may try with None, or the one that OpenCV uses LZW.
I am trying to get at specific frame from a video file using OpenCV 2.4.11.
I have tried to follow the documentation and online tutorials of how to do it correctly and have now tested two approaches:
1) The first method is brute force reading each frame using the video.grab() until I reach the specific frame (timestamp) I want. This method is slow if the specific frame is late in the video sequence!
string videoFile(videoFilename);
VideoCapture video(videoFile);
double videoTimestamp = video.get(CV_CAP_PROP_POS_MSEC);
int videoFrameNumber = static_cast<int>(video.get(CV_CAP_PROP_POS_FRAMES));
while (videoTimestamp < targetTimestamp)
{
videoTimestamp = video.get(CV_CAP_PROP_POS_MSEC);
videoFrameNumber = static_cast<int>(video.get(CV_CAP_PROP_POS_FRAMES));
// Grabe frame (but don't decode the frame as we are only "Fast forwarding")
video.grab();
}
// Get and save frame
if (video.retrieve(frame))
{
char txtBuffer[100];
sprintf(txtBuffer, "Video1Frame_Target_%f_TS_%f_FN_%d.png", targetTimestamp, videoTimestamp, videoFrameNumber);
string imgName = txtBuffer;
imwrite(imgName, frame);
}
2) The second method I uses the video.set(...). This method is faster and doesn't seem to be any slower if the specific frame is late in the video sequence.
string videoFile(videoFilename);
VideoCapture video2(videoFile);
videoTimestamp = video2.get(CV_CAP_PROP_POS_MSEC);
videoFrameNumber = static_cast<int>(video2.get(CV_CAP_PROP_POS_FRAMES));
video2.set(CV_CAP_PROP_POS_MSEC, targetTimestamp);
while (videoTimestamp < targetTimestamp)
{
videoTimestamp = video2.get(CV_CAP_PROP_POS_MSEC);
videoFrameNumber = (int)video2.get(CV_CAP_PROP_POS_FRAMES);
// Grabe frame (but don't decode the frame as we are only "Fast forwarding")
video2.grab();
}
// Get and save frame
if (video2.retrieve(frame))
{
char txtBuffer[100];
sprintf(txtBuffer, "Video2Frame_Target_%f_TS_%f_FN_%d.png", targetTimestamp, videoTimestamp, videoFrameNumber);
string imgName = txtBuffer;
imwrite(imgName, frame);
}
Problem) Now the issue is that using the two methods does end up with the same frame number of the content of the target image frame is not equal?!?
I am tempted to conclude that Method 1 is the correct one and there is something wrong with the OpenCV video.set(...) method. But if I use the VLC player finding the approximate target frame position it is actually Method 2 that is closest to a "correct" result?
As some extra info: I have tested the same video sequence but in two different video files being encoded with respectively 'avc1' MPG4 and 'wmv3' WMV codec.
Using the WMV file the two found frames are way off?
Using the MPG4 file the two found frames are only slightly off?
Is there anybody having some experience with this, can explain my findings and tell me the correct way to get a specific frame from a video file?
Obviously there's still a bug in opencv/ ffmpeg.
ffmpeg doesn't deliver the frames that are wanted and/or opencv doesn't handles this. See here and here.
[Edit:
Until that bug is fixed (either in ffmpeg or (as a work-around in opencv)) the only way to get exact frame by number is to "fast forward" as you did.
(Concerning VLC-player: I suspect that it uses that buggy set ()-interface. As for a player it is usually not too important to seek frame-exact. But for an editor it is).]
I think that OpenCV uses FFmpeg for video decoding.
We once had a similar problem but used FFmpeg directly. By default, random (but exact) frame access isn't guaranteed. The WMV decoder was particularly fuzzy.
Newer versions of FFmpeg allow you access to lower-level routines which can be used to build a retrieval function for frames. This solution was a little involved and nothing I can remember off my head right now. I try to find some more details later.
As a quick work-around, I would suggest to decode your videos off-line and then work on sequences off images. Though, this increases the amount of storage needed, it guarantees exact random frame access. You can use FFmpeg to convert your video file in to a sequence of images like this:
ffmpeg -i "input.mov" -an -f image2 "output_%05d.png"
In OpenCV, I see imread() and VideoCapture() both take a string to a file path of multiple extensions. Is there a way to get a list of extensions that are supported by them? For example, getting a list of "jpg", "png", "mov", "mpg", etc.? I assume this is system dependent and others have needed to query this at runtime.
Furthermore, how is support determined? If have something like the below code and the Mat I get back always seems partially corrupted (I can see a bit of the image). It doesn't seem to change regardless of the frame number I ask for. I can play this video in my video player "totem", but I'm not even sure if totem and OpenCV are even using the same codec for this file.
Mat fromVideo(std::string _videoPath, int frame) {
VideoCapture capture(_videoPath);
Mat f;
for (int i = 0; i < frame; i++) {
capture >> f;
}
return f;
}
For imread() (more info here):
Windows bitmaps - *.bmp, *.dib (always supported)
JPEG files - *.jpeg, *.jpg, *.jpe (see the Notes section)
JPEG 2000 files - *.jp2 (see the Notes section)
Portable Network Graphics - *.png (see the Notes section)
Portable image format - *.pbm, *.pgm, *.ppm (always supported)
Sun rasters - *.sr, *.ras (always supported)
TIFF files - *.tiff, *.tif (see the Notes section)
For VideoCapture():
AVI files - *.avi
It seems that AVI is the only format with decent cross-platform support. See here for more info.
Use the method cv::VideoCapture::isOpened() to make sure that the constructor was successful in initializing the VideoCapture object.
Note that even if it was possible to get a list of supported container formats from OpenCV (AVI, MKV for instance) with their typical filename extensions, you would still need to know the exact list of supported codecs (and even then the exact file you want to open might be corrupted, etc...). So a list of filename extensions is not enough to accurately describe what is internally supported by OpenCV, and the simplest solution at the OpenCV API level is this isOpened() method.
Just update:
cv::VideoCapture cap("D:\\test.mp4")
works for me.
I want to create a program, which gets a video-file from Qt, converts that video file to TIFF-files and sends them to an algorithm which handles these TIFF-Files.
My questions:
is it possible with ffmpeg or avcodec not to convert a video-file to TIFF-files first on harddrive and send them to the algorithm after that, but to convert frame for frame and send it to the algorithm right away?
The more important question: Is it possible to do that not with an external process with ffmpeg.exe, but with ffmpeg.dll? Or is it only possible with avcodec.dll? (It doesn't have to be "on-the-fly" like at my point above) How can I create a ffmpeg.dll with header and lib?
for exporting tif :
http://www.repaire.net/forums/cinema-numerique/215306-projet-dencodage-dcp.html
Creating a tiff from second 29 in a mpeg, using ffmpeg dd201110 can be done like this:
ffmpeg -i 'test.mpg' -vframes 1 -compression_level 0 -ss 29 'test.tiff'
YMMV :-D
If you dont want to store the image as a file, take a look at ffmpeg-php
http://ffmpeg-php.sourceforge.net/
$movie->getFrame([Integer framenumber])
Returns a frame from the movie as an ffmpeg_frame object.
$frame->toGDImage()
Returns a truecolor GD image of the frame.
There may be C code underneath you can reuse..