I need your advice. I'd like to develop the app for audio/video splitting using Metro interface.
Usually I use DirectShow for it using the follow schema: create a grabber, add it to DS graph, capture by it the audio/video streams and pass them to my AVstream drivers for splitting. But in new program I want to use Media Foundation and insert it into UWP.
How I see my new app. It must have Metro interface for common control: choice of sources, adding parameters, changing modes and etc. I'd like to use MediaCapture class for capture of streams and rendering them too. Here I don't see any problems, MSDN has many samples for it. But I have no ideas how to insert a grabber between source and render.
Which operations a grabber will do:
Receive input stream from MediaCapture.
Stream converting : YUV -> RGB, adding effects and etc.
Send output stream for rendering (MediaCapture) and to my AVstream driver for splitting with any apps (Skype, Adobe Flash Player, Edge, ....).
How to make a grabber. In MSDN I found three ways:
Sample Grabber Sink (https://msdn.microsoft.com/en-us/library/windows/desktop/hh184779(v=vs.85).aspx). No problem to receive/control/send stream in MF dll. But I don't know how to link that dll with MediaCapture?
Source Reader (https://msdn.microsoft.com/en-us/library/windows/desktop/dd940436(v=vs.85).aspx). The same problems, plus the Source Reader doesn't work for playback.
Custom MFT? Any case MediaCapture allows to connect to MFT by AddEffectAsync().
My environment: MS Windows 10, MS Visual Studio Community 2015.
Thank you for any ideas.
This question and UWP are not actual for me at all. I found the following:
"Some apps can work intensively in background, for example it maybe video converting, online financial data processing and more.
Now UWP application will suspended when it go offscreen."
https://wpdev.uservoice.com/forums/110705-universal-windows-platform/suggestions/9950598-exclude-suspending-in-desktop
So if the user minimizes the program window, then the program stops a video stream.
Related
I am creating sample application using Windows Media Foundation.
I have used Source Reader IMFSourceReader to Read the media file and then After I am processing the samples IMFSamples using Custom MFT IMFTransform.
In the MFT I have processed IMFSamples, How can I play/display them in a windows. I don't want to use EVR for display.
Also I have read the question:
How to play IMFMediaSample in media foundation?
As per the suggestion I need to use MFPlay for playing the samples, but exactly how this can be done.
In the interface IMFPMediaPlayer I am not able to find any method where I can push the media samples.
https://msdn.microsoft.com/en-us/library/windows/desktop/dd374329(v=vs.85).aspx
IMFSample is a wrapper over raw data. If you happen to waive standard API offering for playback/presentation (such as EVR for video), you will have to extract the data from the media sample object and consume it otherwise, such as using another API at your discretion.
This does not have to be visualization exactly, you are not limited in consumption ideas: writing to file, sending via network etc. For visualization you have other Windows APIs at your choice: DirectX, DirectShow, legacy DirectDraw, GDI, GDI+, Direct2D etc.
IMFSample is not immediately accepted by other APIs right away because it is not what it is designed for. In Media Foundation API EVR is designed for presentation, and EVR is what you are supposed to use.
The video sample object is a specialized implementation of the IMFSample interface for use with the Enhanced Video Renderer (EVR)...
I have a directshow Video renderer redived from CBaseVideoRenderer. The renderer is used in a graph that receives data from a live source (BDA). It looks like the connections are established properly, but the video rendering immediately ends because there is no sample. However, audio Rendering works, ie I can hear the sound while DoRenderSample of my renderer is never called.
Stepping through the code in the debugger, I found out that in CBaseRenderer::StartStreaming, the stream ends immedately, because the member m_pMediaSample is NULL. If I replace my renderer with the EVR renderer, it shows frames, ie the stream is not ending before the first frame for the EVR renderer, but only for my renderer.
Why is that and how can I fix it? I implemented (following the sample from http://www.codeproject.com/Articles/152317/DirectShow-Filters-Development-Part-Video-Render) what I understand as the basic interface (CheckMediaType, SetMediaType and DoRenderSample), so I do not see any possibility to influence what is happening here...
Edit: This is the graph as seen from the ROT:
What I basically try to do is capturing a DVB stream that uses VIDEOINFOHEADER2, which is not supported by the standard sample grabber. Although the channel is a public German TV channel without encryption, could it be that this is a DRM issue?
Edit 2: I have attached my renderer to another source (a Blackmagic Intensity Shuttle). It seams that the source causes the issue, because I get samples in the other graph.
Edit 3: Following Roman's Suggestion, I have created a transform filter. The graph looks like
an has unfortunately the same problem, ie I do not get any sample (Transform is not called).
You supposedly chose wrong path of fetching video frames out of media pipeline. So you are implementing a "network renderer", something that terminates the pipeline to further send data to network.
A renderer which accepts the feed sounds appropriate. Implementing a custom renderer, however, is an untypical task and then there is not so much information around on this. Additionally, a fully featured renderer is typically equipped with sample scheduling part, which end of stream delivery - things relatively easy to break when you customize it through inheriting from base classes. That is, while the approach sounds good, you might want to compare it to another option you have, which is...
A combination of Sample Grabber + Null Renderer, two standard filters, which you can attach your callback to and get frames having the pipeline properly terminated. The problem here is that standard Sample Grabber does not support VIDEOINFOHEADER2. With another video decoder you could possibly have the feed decoded info VIDEOINFOHEADER, which is one option. And then improvement of Sample Grabber itself is another solution: DirectX SDK Extras February 2005 (dxsdk_feb2005_extras.exe) was the SDK which included a filter similar to standard Sample Grabber called Grabber \DirectShow\Samples\C++\DirectShow\Filters\Grabber. It is/was available in source code and provided with a good description text file. It is relatively easy to extend to allow it accept VIDEOINFOHEADER2 and make payload data available to your application this way.
The easiest way to get data out of a DirectShow graph, if youњre not going to use
MultiMedia Streaming, is probably to write your own TransInPlace filter, a sub-variety
of a Transform filter. Then connect this filter to the desired stream of data you wish to
monitor, and then run, pause, seek, or otherwise control the graph. The data, as it passes
through the transform filter, can be manipulated however you want. We call this kind of
filter, a Њsample grabberћ. Microsoft released a limited-functionality sample grabber
with DX8.0. This filter is limited because it doesnњt deal with DV Data or mediatypes
with a format of VideoInfo2. It doesnњt allow the user to receive prerolled samples.
(Whatњs a preroll sample? See the DX8.1 docs) Its ЊOneShotћ mode also has some problems.
To add to this, the Grabber sample is pretty simple itself - perhaps 1000 lines of code all together, including comments.
Looks like your decoder or splitter isn't demuxing the video frames. Look further up the chain to see what filters are supplying your renderer pin with data, chances are its only recognising audio.
Try dropping the file into Graphedit (there's a better one on the web BTW) and see what filters it creates.
Then look at the samples in the DirectShow SDK.
I'm currently building a project on C++ using Visual Studio on Windows 8. This application captures video from camera and triggers some virtual animations in real-time, with some sounds being played along with the animations.
The user has the option to record the experience in video and sound. I already am able to record video, now I want to create a audio track of the sounds that are being played by the application, to later fuse both video and audio files.
So, which is the best way to record audio output from an application in windows?
Let me stress that I do NOT want to record audio from any input devices (such as a microphone), only from the application itself.
Best regards.
There is no recording of application output. If you generate audio on your own, you make a copy for the recording purposes, mix if you have multiple sources, and then use one of the APIs to produce a file depending on your preferences: directly writing a WAV file, Windows Media audio files (ASF/WMA), DriectShow, Media Foundation, third party libraries.
Real playback audio data is being mixed and sent for further playback. Sometimes you can enable loopback recording to capture fully mixed output (not just of specific application through) as if it is a capture from realtime audio input device.
MPEG is a really nice format, specially because it really compress the file to unimaginable sizes. A 140Mb raw AVI is now only 4Mb and the quality is still very good. With the Animation Control Windows provides I can play only raw AVI but I would really like to play a MPEG instead, due to the the sizes of the video file.
Now, how would I do that with C++ and WINAPI? Do I have to use some ActiveX components? How do I make sure other users can run my application without being harassed about missing plug-ins/codecs/third-party programs? Can I use the Animation Control someway for displaying the MPEG video?
Thanks
I took a look at the MSDN documentation and it looks like you can not use the Animation Control to play MPEG video, you seem to have two choices:
1. DirectShow.
2 The newer Microsoft Media Foundation.
Both choices based on COM (and not ActiveX as I stated earlier).
As for making sure your users can run your application, see this page on Building DirectShow Applications which answers that questions for DirectShow. For Microsoft Media Foundation your users need to be running MS Vista or later.
I've stumbled through some code to enumerate my microphone devices (with some help), and am able to grab the "friendly name" and "clsid" information from each device.
I've done some tinkering with GraphEd.exe to try and figure out how I can take audio from directshow and write it to a file (I'm not currently concerned about the format, wav should be fine), and can't seem to find the right combination.
One of the articles I've read linked to this Windows SDK sample, but when I examined the code, I ended up getting pretty confused at how to use that code, ie. setting the output file, or specifying which audio capture device to use.
I also came across a codeguru article that has a nicely featured audio recorder, but it does not have an interface for selecting the audio device, and I can't seem to find where it statically picks which recording device to use.
I think I'd be most interested in figuring out how to use the Windows SDK sample, but any explanation on either of the two approaches would be fantastic.
Edit: I should mention my knowledge and ability as a win32 COM programmer is very low on the scale, so if this is easy, just explain it to me like I'm five, please.
Recording audio into file with DirectShow needs you to build the right filter graph, as you should have figured out already. The parts include:
The device itself, which you instantiate via moniker (not CLSID!), it is typically PCM format
Multiplexer component that converts streams into container format
File Writer Filter that takes file-compatible stream and writes into a file
The tricky moment is #2 since there is not standard component available. Windows SDK samples however contains the missing part - WavDest Filter Sample. Building it and making it ready for use, you can build a graph that records from device into .WAV file.
Your graph will look like this, and it's built easily programmatically as well:
I noticed that I have a variation of WavDest installed with Google Earth - for the case you have troubles building it yourself and you will be looking for prebuilt binary.
You can instruct ffmpeg to record from a directshow device, and output to a file.