For a scientific project i need to compress video data. The video however doesn't contain natural video and the quality characteristics of the compression will be different than for natural footage (preservation of hard edges for example is more important than smooth gradients or color correctness).
I'm looking for a library that can be easily integrated in an existing c++ project and that let's me experiment with different video compression algorithms.
Any suggestions?
Look at FFmpeg. It is the the most mature open source tool for video compression and decompression. It comes with a command line tool, and with libraries for codecs and muxers/demuxers that can be statically or dynamically linked.
As satuon already answered, FFmpeg is the go-to solution for all things multimedia. However, I just wanted to suggest an easier path for you than trying to hook your program up to its libraries. It would probably be far easier for you to generate a sequence of raw RGB images within your program, dump each out to disc (perhaps using a ridiculously simple format like PPM), and then use FFmpeg from the command like to compress them into a proper movie.
This workflow might cut down on your prototyping and development time.
As for the specific video codec you will want to use, you have a plethora of options available to you. One of the most important considerations will be: Who needs to be able to play your video and what software will they have available?
Related
I've been wanting to play around with audio parsing for a while now but I haven't really been able to find the correct library for what I want to do.
I basically just want to parse through a sound file and get amplitudes/frequencies and other relevant information at certain times during the song (like every 10 ms or so) so I can graph the data for example where the song speeds up a lot and where it gets really loud.
I've looked at OpenAL quite a bit but it doesn't look like it provides this ability, other than that I have not had much luck with finding out where to start. If anyone has done this or used a library which can do this a point in the right direction would be greatly appreciated. Thanks!
For parsing and decoding audio files I had good results with libsndfile, which runs on Windows/OSX/Linux and is open source (LGPL license). This library does not support mp3 (the author wants to avoid licensing issues), but it does support FLAC and Ogg/Vorbis.
If working with closed source libraries is not a problem for you, then an interesting option could be the Quicktime SDK from Apple. This SDK is available for OSX and Windows and is free for registered developers (you can register as an Apple developer for free as well). With the QT SDK you can parse all the file formats that the Quicktime Player supports, and that includes .mp3. The SDK gives you access to all the codecs installed by QuickTime, so you can read .mp3 files and have them decoded to PCM on the fly. Note that to use this SDK you have to have the free QuickTime Player installed.
As far as signal processing libraries I honestly can't recommend any, as I have written my own functions (for speech recognition, in case you are curious). There are a few open source projects that seem interesting listed in this page.
I recommend that you start simple, for example working on analyzing amplitude data, which is readily available from the PCM samples without having to do any processing. Being able to visualize the data is very useful, I have found Audacity to be an excellent visualization tool, and since it is open source you can build your own tests inside it.
Good luck!
I am working on a C++ project with openCV. It is a simple web cam application with basic features like capturing pictures and videos. I have already been able to save video (w/o audio). Since openCV doesnot support audio processing, I was wondering if there is any way I can record audio separately in a different file and later combine those together to get one video file.
While searching on the internet, I did hear something about using ffmpeg with openCV. But I just cant figure out how to do it exactly.....
Can you guys help me? I would be very grateful... Thankyou!
P.S. I have used openCV and QT (for GUI)
As you said, opencv doesn't by itself deal with audio. However once you get a separate audio and video file, you can combine them using a technique called muxing. There are many many ways to do this. I use VirtualDub for most of my muxing needs, although it is windows only (not sure if that's a problem). I know ffmpeg is also capable of muxing (via the command line interface), I can't recall what the command is. There's also mplayer and a multitude of other programs out there to do this.
as far as i know openCV is good for video/image processing. To support audio processing, you can use other libraries e.g. PortAudio or C-sound.
I've always been curious about audio conversion software, but I have never seen a proper explanation from a beginners point of view as to how to write a simple program that converts for example, a mp3 file to a wav. I'm not asking about any of the complex algorithms involved, just a small example using a simple library. Searching on SO, I came up with several names including:
Lame
The Synthesis Toolkit
OpenAL
DirectSound
But I'm unable to find a straightforward example of any of these libraries. Usually I don't mind wading through tons of code, but here I have absolutely no knowledge about the subject and so I always feel like I'm shooting in the dark.
Anyone here have a simple example / tutorial on converting a sound file using any of these libraries? My question is specifically directed towards C/C++ because those are the two languages I'm currently learning and so I'd like to continue to focus on them.
Edit: One thing I forgot to mention: I'm on a *NIX system.
Thanks everyone for the responses! I sort of cobbled them together to successfully make a small utility that converts a AIFF/WAV/etc file to an mp3 file. There seems to be some interest in this question, so here it what I did, step by step:
Step 1:
Download and install the libsndfile library as suggested by James Morris. This library is very easy to use – its only shortcoming is it won't work with mp3 files.
Step 2:
Look inside the 'examples' folder that comes with libsndfile and find generate.c. This gives a nice working example of converting any non-mp3 file to various file formats. It also gives a glimpse of the power behind libsndfile.
Step 3:
Borrowing code from generate.c, I created a c file that just converts an audio file to a .wav file. Here is my code: http://pastie.org/719546
Step 4:
Download and install the LAME encoder. This will install both the libmp3lame library and the lame command-line utility.
Step 5:
Now you can peruse LAME's API or just fork & exec a process to lame to convert your wav file to an mp3 file.
Step 6: Bring out the champagne and caviar!
If there is a better way (I'm sure there is) to do this, please let me know. I personally have never seen a step-by-step roadmap like this so I thought I'd put it out there.
For converting between various formats (except MP3) check libsndfile http://mega-nerd.com/libsndfile/
Libsndfile is a library designed to
allow the reading and writing of many
different sampled sound file formats
(such as MS Windows WAV and the
Apple/SGI AIFF format) through one
standard library interface.
During read and write operations,
formats are seamlessly converted
between the format the application
program has requested or supplied and
the file's data format. The
application programmer can remain
blissfully unaware of issues such as
file endian-ness and data format
It is also simple to use, with the API following the style of the Standard C library function names:
http://mega-nerd.com/libsndfile/api.html
And examples are included in the source distribution.
For actual audio output, another library will be needed, SDL as already mentioned might be a good place to start. While SDL can also read/write audio files, libsndfile is far superior.
If your curious about DSP and computers, take a look at the Synthesis Toolkit. It's sweet. It's designed for learning. The examples and tutorials they have on their website are straightforward and thorough. Keep in mind, the guys who wrote it, wrote it so they could create acoustic models of real instruments. As a result, they've included some instruments that are just plain wacky, but fun. It will give you a core understanding of processing PCM sound. And you'll probably be able to hack together some fun little noisemakers while your at it.
https://ccrma.stanford.edu/software/stk/
Check libmad http://mad.sourceforge.net " "M"peg "A"udio "D"ecoder library", should provide a good example.
Also for an easy cross-platform audio handling, check SDL http://www.libsdl.org/.
Hope that helps.
Please provide some pointers on how to convert real media formats to other popular media formats using some C++ sdk(I guess Helix provides one but don't know how to use it).
I am a total newbie in the above area, any help would be highly appreciated.
libavcodec (a library behind ffmpeg and other heavily-used programs) supports some common Real video formats. See this page; the tutorial itself is obsolete, but there are linked updates such as An ffmpeg and SDL Tutorial
Helix may be an option, but keep in mind the actual Real Video codecs are only available as binaries, while libavcode is fully open source.
Don't expect this to be trivial, whichever library you use.
How do you programmatically compress a WAV file to another format (PCM, 11,025 KHz sampling rate, etc.)?
I'd look into audacity... I'm pretty sure they don't have a command line utility that can do it, but they may have a library...
Update:
It looks like they use libsndfile, which is released under the LGPL. I for one, would probably just try using that.
Use sox (Sound eXchange : universal sound sample translator) in Linux:
SoX is a command line program that can convert most popular audio files to most other popular audio file formats. It can optionally
change the audio sample data type and apply one or more sound effects to the file during this translation.
If you mean how do you compress the PCM data to a different audio format then there are a variety of libraries you can use to do this, depending on the platform(s) that you want to support. If you just want to change the sample rate of the PCM data then you need a sample rate conversion algorithm instead, which is a completely different problem. Can you be more specific in your requirements?
You're asking about resampling, and more specifically downsampling, not compression. While both processes are lossy (meaning that you will suffer loss of information), downsampling works on raw samples instead of in the frequency domain.
If you are interested in doing compression, then you should look into lame or OGG vorbis libraries; you are no doubt familiar with MP3 and OGG technology, though I have a feeling from your question that you are interested in getting back a PCM file with a lower sampling rate.
In that case, you need a resampling library, of which there are a few possibilites. The most widely known is libsamplerate, which I honestly would not recommend due to quality issues not only within the generated audio files, but also of the stability of the code used in the library itself. The other non-commercial possibility is sox, as a few others have mentioned. Depending on the nature of your program, you can either exec sox as a separate process, or you can call it from your own code by using it as a library. I personally have not tried this approach, but I'm working on a product now where we use sox (for upsampling, actually), and we're quite happy with the results.
The other option is to write your own sample rate conversion library, which can be a significant undertaking, but, if you only are interested in converting with an integer factor (ie, from 44.1kHz to 22kHz, or from 44.1kHz to 11kHz), then it is actually very easy, since you only need to strip out every Nth sample.
In Windows, you can make use of the Audio Compression Manager to convert between files (the acm... functions). You will also need a working knowledge of the WAVEFORMAT structure, and WAV file formats. Unfortunately, to write all this yourself will take some time, which is why it may be a good idea to investigate some of the open source options suggested by others.
I have written a my own open source .NET audio library called NAudio that can convert WAV files from one format to another, making use of the ACM codecs that are installed on your machine. I know you have tagged this question with C++, but if .NET is acceptable then this may save you some time. Have a look at the NAudioDemo project for an example of converting files.