I'm developing a C++ app that uses FFmpeg to play audio/video. Now I want to enhance the application to allow the users to extract audio from video. How can FFmpeg can be used for this? I searched a lot about this but I was not able to find a tutorial regarding it.
You need to
Open the input context [ avformat_open_input ]
Get the stream information [ avformat_find_stream_info ]
Get the audio stream:
if (inputFormatContext->streams[index]->codec->codec_type ==
AVMEDIA_TYPE_AUDIO) {
inputAudioStream = inputFormatContext->streams[index];
}
Read each packet.
AVPacket packet;
int ret = av_read_frame(inputFormatContext, &packet);
if (ret == 0) {
if (packet.stream_index == inputAudioStream->index) {
// packet.data will have encoded audio data.
}
}
This seems like a simple scripting task... why do you want to use the heavy artillery (C/C++) to swat a fly?
I use Applescript to build/run an ffmpeg command line via a Bash shell. The only reason I involve Applescript is so I can invoke it as a droplet (ie drag-and-drop the file(s) onto the app and have it run without interaction.)
I get that you're probably on Windows, meaning no Applescript and no Bash. But surely something lighter than C can build/run an ffmpeg command line. It's really as simple as:
ffmpeg -i infile.mp4 -b 160k outfile.mp3
Related
I am trying to use libav to convert an MP4 file to an MP3 file. Basically trying to achieve what ffmpeg -i filename.mp4 filename.mp3 does. I've found this official example. But when I run it with an input MP4 and an output MP3 I get an error:
Invalid audio stream. Exactly one MP3 audio stream is required.
I am not at all familiar with this library but I think I have narrowed the problem down to this line:
ret = avcodec_parameters_copy(out_stream->codecpar, in_codecpar);
It seems to copy all streams for a video file but we only need one for the MP3 file? I am not sure. There doesn't seem to be a function to copy only the parameters relevant to audio. I checked the sources, avcodec_parameters_copy does a simple memcpy.
Questions:
Is this the actual problem?
How do I solve it?
Am I on the right track to achieve the goal of extracting audio from a video file? I've seen this question (and other similar questions like this and this) on here but none seem to have a complete code example. The C API documentation for this library is also a little lacking.
You can have multiple audio tracks in mp4 file, but only one such track in an mp3 file. The easiest fix for the remuxing example would be to replace lines 101-103:
if (in_codecpar->codec_type != AVMEDIA_TYPE_AUDIO ||
stream_index != 0) {
This, naturally, is relevant only if the output is mp3.
PS, make sure that your input mp4 uses the MP3 audio codec. If it does not (and most have AAC or AC3 these days), it's not enough to remux the file, you also need to decode and re-encode the audio stream.
I am using the official encode_video.c example to test if FFmpeg works correctly for me.
I got the pre-built windows edition from ffmpeg.zeranoe.com/builds. It is built already with libx264 and other external libraries. I got both dev and shared editions and added the DLLs, header files and libs accordingly in Visual Studio.
Now the encode_video.c example does not work correctly.
What I tried:
I compiled the example and run it on many different file formats and codecs such as the following.
First I tried all of these file formats (.mp4, .m4v, .h264, .x264, .avi, .flv) with codec name as libx264. The code executed without errors but the output video file did not play in VLC or Windows 10 default player.
Next, I tried all of those above file formats but with codec name as mpeg4. The code executed without errors but the output video file played only for .m4v in VLC.
What is expected:
All of those combinations should have produced a video file which could be played in VLC. None of them worked except for .m4v as file format and mpeg4 as codec name.
Please tell me how to make this work for h264. I mainly want it to work for h264 as that is only important for now.
I am running the code like ./encode_video.exe test.mp4 libx264 where first argument is output filename and second argument is codec name.
This is the output for test.mp4 and libx264 as command line arguments https://imgur.com/a/AHLQwuK
It seems that in the encode function, it goes over the below code and returns because of AVERROR(EAGAIN) or AVERROR_EOF. Please tell me what is happening.
while (ret >= 0) {
ret = avcodec_receive_packet(enc_ctx, pkt);
if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF)
return;
else if (ret < 0) {
fprintf(stderr, "Error during encoding\n");
exit(1);
}
printf("Write packet %3"PRId64" (size=%5d)\n", pkt->pts, pkt->size);
fwrite(pkt->data, 1, pkt->size, outfile);
av_packet_unref(pkt);
}
I used DepenciesGUI to find out the DLLs linked and it shows that the DLLs are correctly linked. Please help me figure out what the problem is now!!
I found this question because I had the same issue (couldn't watch video generated by that example). Regarding your concern:
It seems that in the encode function, it goes over the below code and
returns because of AVERROR(EAGAIN) or AVERROR_EOF. Please tell me what
is happening.
While stepping through the code in debugger, I too noticed those occasional "errors". However, those frames are eventually processed either in subsequent loop iteration or when the encoder is flashed via:
/* flush the encoder */
encode(c, NULL, pkt, f);
I too tried to use VLC, and also QuickTime, with no luck.
Then I noticed ffplay tool in the ffmpeg's bin folder, that would play all videos produced by that example, with different codecs.
My point is - the issue might be with the viewer, not with the video file.
just change:
./encode_video.exe test.mp4 libx264
to:
./encode_video.exe test.264 libx264
and the result file "test.264" can be played by vlc player.
I'm running ffmpeg's encode_video.c demo on my Mac and Apple's QuickTime Player seems not supporting this format.
You want to use the muxing.c example instead.
The encode_video example doesn't produce an MPEG compliant file.
I searched on web for this problem but I didn't find a solution.
I'm starting to use samples of OpenNI to develop my application. I have not much experience with C++: I am a beginner.
I'm using SimpleSkeleton sample of OpenNI 1.5.4 which returns head position. I need it but using an ONI file which I got recording using NiViewer, and not the stream data from kinect.
Searching on web I found:
xn::Context context;
xn::Player player;
nRetVal = context.Init();
nRetVal=context.OpenFileRecording("Myfile.oni",player);
if (nRetVal != XN_STATUS_OK)
{
printf("Can't open recording file: %s\n",xnGetStatusString(nRetVal));
return 1;
}
My code enter the if loop, print 'Can't open my file.oni ...' and doesn't continue running.
I think the problem is to set player object. How can I do? Or what have I to do?
Without the lines:
nRetVal=context.OpenFileRecording("Myfile.oni",player);
if (nRetVal != XN_STATUS_OK)
{
printf("Can't open recording file: %s\n",xnGetStatusString(nRetVal));
return 1;
}
it runs correctly but use data stream from kinect and not from file.
Even if it seems like it's not, this is a path issue.
I had a similar problem, you're either passing it to OpenNI in a way it does no like, or the path is not valid.
How are you retrieving the path? Trying using boost to get the current-directory and then appending the .oni file name to that string
So I created simple Consol app: FFmpeg RTSP Video stream reader (using only general FFmpeg C API) But while ffmpeg reads from RTSP it shows lots of info. I did not asked for if... At least not all of it... So how can I filter what ffmpeg is outputing? I mean in all he talls user-developer there is only one important line something like: missing picture in acsess unit so how to put some filter mechanism for ffmpeg not to output all it wants and for me developer to catch the moment when message I want appeares? (In my project I write in C++ under visual studio using Boost libs)
Use av_log_set_callback, to set your function as callback:
static void avlog_cb(void *, int level, const char * szFmt, va_list varg) {
//do nothing...
}
av_log_set_callback(avlog_cb);
or, you may also use
av_log_set_level(AV_LOG_ERROR);
to print error messages only.
I would like to open a small video file and map every frames in memory (to apply some custom filter). I don't want to handle the video codec, I would rather let the library handle that for me.
I've tried to use Direct Show with the SampleGrabber filter (using this sample http://msdn.microsoft.com/en-us/library/ms787867(VS.85).aspx), but I only managed to grab some frames (not every frames!). I'm quite new in video software programming, maybe I'm not using the best library, or I'm doing it wrong.
I've pasted a part of my code (mainly a modified copy/paste from the msdn example), unfortunately it doesn't grabb the 25 first frames as expected...
[...]
hr = pGrabber->SetOneShot(TRUE);
hr = pGrabber->SetBufferSamples(TRUE);
pControl->Run(); // Run the graph.
pEvent->WaitForCompletion(INFINITE, &evCode); // Wait till it's done.
// Find the required buffer size.
long cbBuffer = 0;
hr = pGrabber->GetCurrentBuffer(&cbBuffer, NULL);
for( int i = 0 ; i < 25 ; ++i )
{
pControl->Run(); // Run the graph.
pEvent->WaitForCompletion(INFINITE, &evCode); // Wait till it's done.
char *pBuffer = new char[cbBuffer];
hr = pGrabber->GetCurrentBuffer(&cbBuffer, (long*)pBuffer);
AM_MEDIA_TYPE mt;
hr = pGrabber->GetConnectedMediaType(&mt);
VIDEOINFOHEADER *pVih;
pVih = (VIDEOINFOHEADER*)mt.pbFormat;
[...]
}
[...]
Is there somebody, with video software experience, who can advise me about code or other simpler library?
Thanks
Edit:
Msdn links seems not to work (see the bug)
Currently these are the most popular video frameworks available on Win32 platforms:
Video for Windows: old windows framework coming from the age of Win95 but still widely used because it is very simple to use. Unfortunately it supports only AVI files for which the proper VFW codec has been installed.
DirectShow: standard WinXP framework, it can basically load all formats you can play with Windows Media Player. Rather difficult to use.
Ffmpeg: more precisely libavcodec and libavformat that comes with Ffmpeg open- source multimedia utility. It is extremely powerful and can read a lot of formats (almost everything you can play with VLC) even if you don't have the codec installed on the system. It's quite complicated to use but you can always get inspired by the code of ffplay that comes shipped with it or by other implementations in open-source software. Anyway I think it's still much easier to use than DS (and much faster). It needs to be comipled by MinGW on Windows, but all the steps are explained very well here (in this moment the link is down, hope not dead).
QuickTime: the Apple framework is not the best solution for Windows platform, since it needs QuickTime app to be installed and also the proper QuickTime codec for every format; it does not support many formats, but its quite common in professional field (so some codec are actually only for QuickTime). Shouldn't be too difficult to implement.
Gstreamer: latest open source framework. I don't know much about it, I guess it wraps over some of the other systems (but I'm not sure).
All of this frameworks have been implemented as backend in OpenCv Highgui, except for DirectShow. The default framework for Win32 OpenCV is using VFW (and thus able only to open some AVI files), if you want to use the others you must download the CVS instead of the official release and still do some hacking on the code and it's anyway not too complete, for example FFMPEG backend doesn't allow to seek in the stream.
If you want to use QuickTime with OpenCV this can help you.
I have used OpenCV to load video files and process them. It's also handy for many types of video processing including those useful for computer vision.
Using the "Callback" model of SampleGrabber may give you better results. See the example in Samples\C++\DirectShow\Editing\GrabBitmaps.
There's also a lot of info in Samples\C++\DirectShow\Filters\Grabber2\grabber_text.txt and readme.txt.
I know it is very tempting in C++ to get a proper breakdown of the video files and just do it yourself. But although the information is out there, it is such a long winded process building classes to hand each file format, and make it easily alterable to take future structure changes into account, that frankly it just is not worth the effort.
Instead I recommend ffmpeg. It got a mention above, but says it is difficult, it isn't difficult. There are a lot more options than most people would need which makes it look more difficult than it is. For the majority of operations you can just let ffmpeg work it out for itself.
For example a file conversion
ffmpeg -i inputFile.mp4 outputFile.avi
Decide right from the start that you will have ffmpeg operations run in a thread, or more precisely a thread library. But have your own thread class wrap it so that you can have your own EventAgs and methods of checking the thread is finished. Something like :-
ThreadLibManager()
{
List<MyThreads> listOfActiveThreads;
public AddThread(MyThreads);
}
Your thread class is something like:-
class MyThread
{
public Thread threadForThisInstance { get; set; }
public MyFFMpegTools mpegTools { get; set; }
}
MyFFMpegTools performs many different video operations, so you want your own event
args to tell your parent code precisely what type of operation has just raised and
event.
enum MyFmpegArgs
{
public int thisThreadID { get; set; } //Set as a new MyThread is added to the List<>
public MyFfmpegType operationType {get; set;}
//output paths etc that the parent handler will need to find output files
}
enum MyFfmpegType
{
FF_CONVERTFILE = 0, FF_CREATETHUMBNAIL, FF_EXTRACTFRAMES ...
}
Here is a small snippet of my ffmpeg tool class, this part collecting information about a video.
I put FFmpeg in a particular location, and at the start of the software running it makes sure that it is there. For this version I have moved it to the Desktop, I am fairly sure I have written the path correctly for you (I really hate MS's special folders system, so I ignore it as much as I can).
Anyway, it is an example of using windowless ffmpeg.
public string GetVideoInfo(FileInfo fi)
{
outputBuilder.Clear();
string strCommand = string.Concat(" -i \"", fi.FullName, "\"");
string ffPath =
System.Environment.GetFolderPath(Environment.SpecialFolder.Desktop) + "\\ffmpeg.exe";
string oStr = "";
try
{
Process build = new Process();
//build.StartInfo.WorkingDirectory = #"dir";
build.StartInfo.Arguments = strCommand;
build.StartInfo.FileName = ffPath;
build.StartInfo.UseShellExecute = false;
build.StartInfo.RedirectStandardOutput = true;
build.StartInfo.RedirectStandardError = true;
build.StartInfo.CreateNoWindow = true;
build.ErrorDataReceived += build_ErrorDataReceived;
build.OutputDataReceived += build_ErrorDataReceived;
build.EnableRaisingEvents = true;
build.Start();
build.BeginOutputReadLine();
build.BeginErrorReadLine();
build.WaitForExit();
string findThis = "start";
int offset = 0;
foreach (string str in outputBuilder)
{
if (str.Contains("Duration"))
{
offset = str.IndexOf(findThis);
oStr = str.Substring(0, offset);
}
}
}
catch
{
oStr = "Error collecting file information";
}
return oStr;
}
private void build_ErrorDataReceived(object sender, DataReceivedEventArgs e)
{
string strMessage = e.Data;
if (outputBuilder != null && strMessage != null)
{
outputBuilder.Add(string.Concat(strMessage, "\n"));
}
}
Try using the OpenCV library. It definitely has the capabilities you require.
This guide has a section about accessing frames from a video file.
If it's for AVI files I'd read the data from the AVI file myself and extract the frames. Now use the video compression manager to decompress it.
The AVI file format is very simple, see: http://msdn.microsoft.com/en-us/library/dd318187(VS.85).aspx (and use google).
Once you have the file open you just extract each frame and pass it to ICDecompress() to decompress it.
It seems like a lot of work but it's the most reliable way.
If that's too much work, or if you want more than AVI files then use ffmpeg.
OpenCV is the best solution if video in your case only needs to lead to a sequence of pictures. If you're willing to do real video processing, so ViDeo equals "Visual Audio", you need to keep up track with the ones offered by "martjno". New windows solutions also for Win7 include 3 new possibilities additionally:
Windows Media Foundation: Successor of DirectShow; cleaned-up interface
Windows Media Encoder 9: It does not only include the programm, it also ships libraries for coding
Windows Expression 4: Successor of 2.
Last 2 are commercial-only solutions, but the first one is free. To code WMF, you need to install the Windows SDK.
I would recommend FFMPEG or GStreamer. Try and stay away from openCV unless you plan to utilize some other functionality than just streaming video. The library is a beefy build and a pain to install from source to configure FFMPEG/+GStreamer options.