I'm searching for multimedia editing graph-like frameworks where you can create graph elements with C++ and where you can build graphs for your app needs from elements of framework.
So far I know there are
GStreamer
DirectShow
QuickTime
Qt Phonon (cross-platform wrapper around first 3)
I wonder to see what else is out there - are there proprietary or open source, cross-platform or platform-dependent alternatives and what are their main goals?
Microsoft Media Foundation -- more or less a replacement for DirectShow. I don't see any drastic improvements, but if you want to use (for example) Microsoft's new H.264 decoder, you're pretty much stuck with it. As you can undoubtedly guess from the name, it's Windows only. Perhaps less obviously, it's supported only on Vista and later.
Another that sort of fits the description is FFMpeg.
Related
I'm working on an application that makes analysis on video files.
Being no expert on DirectShow I used simple code for the analysis
of all frames (SampleGrabber, Callback etc.).
This works fine for all media files, even when decoded with
VideoInfoHeader2 structure (although it shouldn't, as stated everywhere).
The problem is with grabbing a single frame.
For this I used IMediaDet. And this doesn't do if there's only VideoInfoHeader2, and no VideoInfoHeader.
I tried modifications of my analysis code (OneShot, Seek), but it doesn't do.
All sources in the internet concerning this are not very helpful, as they point to SDK/ DX examples that aren't accessible anymore, or they just say that modification would be "easy".
Well, maybe for an DX expert ...
(But I need to use the car, not first to build it ... ;-)
As the matter became more & more important to me my "workaround" is to recode all videos with VideoInfoHeader2, save them with VideoInfoHeader, and do the analysis/ grabbing on that.
Very resource consuming, and the opposite of smart ...
Any help appreciated.
You outlined the necessary steps which are still the easiest solution (provided that you don't give up and use Windows API; using a third party library might be easier in comparison but this is beyond the scope of this question).
Sample Grabber and IMediaDet are parts of deprecated DirectShow Editing Services, development of which was stopped long ago. If you are not satisfied with stock API, you have to use a more flexible replacement. For example, you can grab source of similar Sample Grabber sample from older DirectX or Platform SDK and extend it to support VIDEOINFOHEADER2.
IMediaDet is nothing but COM class building its own graph internally trying to decode video. It is inflexible and almost every time your building your own graph is a more reliable solution.
Microsoft's answer to this problem is - as they abandoned DirectShow development - newer API Media Foundation. However there are reasons why this "answer" is not so good: limited OS compatibility, limited support for codecs and formats, completely new API which has little in common with DirectShow and you need to redesign your application.
All together, you either have to find a Sample Grabber replacement using one of the popular and explained methods (no matter they look not so much helpful), or switch to another API or third party library. Or, another possible solution is to use a different filter/codec which is capable of decoding into VIDEOINFOHEADER formatted media type.
What are the available options for creating applications that allow preview of a webcam and exporting the current webcam image to file?
EDIT: I've since found that MS Media Foundation offers similar features to DirectShow that work from Vista upwards. However, while apparently possible, the documentation, samples, and forums don't suggest an easy way to perform the above. On the other hand, with DirectShow you can fairly easily merge two of the sample apps to achieve it.
EDIT: Researching available camera APIs I've also found the MS MediaCapture API looks useful but this seems to be only available on windows 8 (and may be limited to managed C++/.net/javascript).
EDIT: DirectShow is supported on the Windows 8 standard desktop (For Metro style applications the video platform is Media Foundation).
Consider using directshow. You may have a look here
You basically found the answers yourself. DirectShow and Media Foundation are the APIs. Neither of the two cover all Microsoft environments: Metro supports a subset of MF, and MF is not available in legacy OSes. DirectShow is presumably easier, more flexible, with more sample code around.
I mainly code using the Qt library, which is cross-platform enough for my need. However, I found out that if I'd like to play a video file there are no good options for cross-platform video playback. The Phonon module doesn't seem to work on my machine and I am not sure if it is supported on the major platforms (Windows, OSX, Linux).
Are their any light-weight alternatives for simple video playback? I would be okay to change my video to any format as long as its playable cross-platform.
I am considering libVLC as one option. Someone have already made a wrapper but it only supports Windows and Linux.
I found the Theora playback library to be very useful and only depends on the original theora library. Here is a simple Qt project I made that encapsulate everything needed to play (only video) theora media. You just type 'qmake' and you are good to go.
According to the documentation is appears Phonon needs third party libraries to work correctly. Have you installed the appropriate backend to the platform you're on?
From the documentation:
Backends
The multimedia functionality is not implemented by Phonon itself, but
by a back end - often also referred to as an engine. This includes
connecting to, managing, and driving the underlying hardware or
intermediate technology. For the programmer, this implies that the
media nodes, e.g., media objects, processors, and sinks, are produced
by the back end. Also, it is responsible for building the graph, i.e.,
connecting the nodes. The backends of Qt use the media systems
DirectShow (which requires DirectX) on Windows, QuickTime on Mac, and
GStreamer on Linux. The functionality provided on the different
platforms are dependent on these underlying systems and may vary
somewhat, e.g., in the media formats supported.
Backends expose
information about the underlying system. It can tell which media
formats are supported, e.g., AVI, mp3, or OGG.
A user can often add
support for new formats and filters to the underlying system, by, for
instance, installing the DivX codex. We can therefore not give an
exact overview of which formats are available with the Qt backends.
QtWebKit 2.2 has support for HTML5 media. It should (at least eventually) be pretty cross-platform, but probably has the same backend issues as Phonon. But a Q(Graphics)WebView with a <video> element could do the trick for a truly play-and-forget solution.
My commercial embedded C++ Linux project requires playing wav files and tones at individual volume levels concurrently. A few examples of the sounds:
• “Click” sounds each time user presses screen played at a user-specified volume
• Warning sounds played at max-volume
• Warning tones requested by other applications at app-specified volume level (0-100%)
• Future support for MP3 player and/or video playback (with sound) at user-specified volume. All other sounds should continue while song/video is playing.
We're using Qt as our UI framework which has QtMultimedia and Phonon support. However, I heard the former has spotty sound support on Linux and the latter is an older version and may be deprecated in an upcoming Qt release.
I've done some research and here are a few APIs I've come across:
KDE Phonon
SFML
PortAudio
SDL_Mixer
OpenAL Soft
FMOD (though I'd prefer to avoid license fees)
ALSA (perhaps a bit too low-level...)
Other considerations:
Cross-platform isn't required but preferred. We'd like to limit dependencies as much as possible. There is no need for advanced features like 3D audio or special effects in the foreseeable future. My team doesn't have much audio experience so ease-of-use is important.
Are any of these overkill for my application? Which seems like the best fit?
Update:
It turns out we were already dependent on SDL for other reasons so we decided on SDL_Mixer. For other Embedded applications, however, I'd take a long at the PortAudio/libsndfile combo as well due to their minimal dependencies.
libao is simple, cross-platform, Xiphy goodness.
There's documentation too!
Usage is outlined here - simple usage goes like this:
Initialize (ao_initialize())
Call ao_open_live() or ao_open_file()
Play sound using ao_play()
Close device/file using ao_close() and then ao_shutdown() to clean up.
Go for PortAudio. For just plain audio without unneeded overhead such as complex streaming pipelines, or 3D, it is the best lib out there. In addition you have really nice cross-platform support. It is used by several professional audio programs and has really high quality.
i have used SDL_Mixer time and time again, lovely library, it should serve well for your needs, the license is flexible and its heavily documented. i have also experimented with SFML, while more modern and fairly documented, i find it a bit bulky and cumbersome to work with even tho both libraries are very similar. imo SDL_Mixer is the best.
however you might also want to check out this one i found a few weeks ago http://www.mpg123.de/, i haven't delved too much into it, but it is very lightweight and again the license is flexible.
There is a sound library called STK that would meet most of your requirements:
https://ccrma.stanford.edu/software/stk/faq.html
Don't forget about:
FFmpeg: is a complete, cross-platform solution to record, convert and stream audio and video.
GStreamer: is a library for constructing graphs of media-handling components. The applications it supports range from simple Ogg/Vorbis playback, audio/video streaming to complex audio (mixing) and video (non-linear editing) processing.
How would i make it so in my program there is a button when that button is clicked i want it to play a .wma file without opening and media player?
The C++ standard does not include this functionality. That means it depends on what your system offers.
For Windows, you can try something like PlaySound.
However, you best bet is to use a pre-existing library, like:
OpenAL
BASS
FMOD
SDL's: Sound.
Searching for C++ Sound Library brings up a lot of information.
Also, check out these three other SO topics:
What Is The Best C++ Sound API For Windows?
How to play MP3 files in C?
Learning to work with audio in C++
Although the above answer mentions it (and everything else, it doesn't give any recommendations so...), FMOD is the king of C++ sound (the most used) and works great so I'd recommend FMOD in particular.
Gstreamer is a free, cross-platform multimedia framework written in C (using GObject) that allows encoding/decoding for many types of media, including wma. Very easy to use and well documented.
KDE's Phonon is a cross-platform(!) C++ multimedia library. It supports native sound systems as backends. Worth a look if you haven't seen it before, and a simple video player can be written in a mere 83 lines of code.
For Windows, in all honesty your best bet is DirectShow. The RenderFile API allows you to play most audio file types with just a few lines of code.
The best part about DirectShow is that it's a part of the Windows platform so you don't need to bundle an external component.