It's hard to find anything on the web reliable for audio rendering (there is capture, record and save to file, but rendering to speakers are rare using Media Foundation).
The problem I have is that the tutorial of audio rendering provided by Microsoft are... all for desktop application.
There is one that could be used for UWP as it is using WASAPI:
But... the uuid of lines fails on UWP:
const CLSID CLSID_MMDeviceEnumerator = __uuidof(MMDeviceEnumerator);
const IID IID_IMMDeviceEnumerator = __uuidof(IMMDeviceEnumerator);
const IID IID_IAudioClient = __uuidof(IAudioClient);
const IID IID_IAudioRenderClient = __uuidof(IAudioRenderClient);
Someone seemed to have the same problem:
WASAPI + windows store app initialization
Error: operand of __uuidof must have a class or enum type for which __declspec(uuid('..')) has been specified.
Nobody really answered that as well and even more the link he provides is not reachable (or at least not anymore). I Can't figure out how to activate the output audio device and stream audio to it on Universal Windows Platform.
I am using Media Foundation for a specific scenario and I use a Source Reader to get asynchronously audio decoded PCM samples.
While there are samples that should work on Standalone desktop builds, I can't find anywhere something that is compatible with Universal Windows Platform...
Anyone has done something similar (audio rendering using Media Foundation or WASAPI with C++ for UWP?) or at least got better links that would help me?
Thanks for your help!
Related
I am developing an C++ application which should use an USB camera to capture high resolution photos. It should have same behavior as the Camera application in Windows 10. I am trying to use DirectShow for doing it. Now I am only able to take high resolution photo which is delayed or take a photo in time but low resolution. Also I am very confused from MS documentation, lot of things are deprecated and nowhere mentioned what replaces them. I'll describe my hopeless steps awaiting there will be somebody who could be able to show me a way.
Let's start from beginning...
Knowing nothing about video capturing in Window I started by searching suitable library. After some googling I found there are four main libraries for capturing video in Windows.
Video for Windows
DirectShow
Windows Media Foundation
OpenCV
Let's observe:
Video for Windows
This library is unfortunately marked as deprecated but it seems it still works. I have written "unfortunately", because I think this is the only which is easy to use. There are only a few lines of code needed for seeing video from camera. The only think I miss here is a "TakePhoto" function. You can use VFW for capture a video or single frames to an avi file. Or am I missing something?
DirectShow
This is much more complicated library. You need hundreds of lines of code to see a video preview. But you can obtain this code on MS Docs. Ok, now I have a video preview and I need only to take a photo. One would expect this should be just one function call. But where is the function? I did not find it.
You can simply use GetCurrentImage from IVMRWindowlessControl but this takes only one frame from preview with low resolution. If you set a higher resolution for preview the video is not fluent.
Best approach I could achieve is from an article called "Capturing an Image From a Still Image Pin" available here https://learn.microsoft.com/en-us/windows/desktop/directshow/capturing-an-image-from-a-still-image-pin. When I had found this site I thought I won and my task was almost finished. But it wasn't.
The first advice which the article gives you is not to use it: "The recommended way to get still images from the device is to use the Windows Image Acquisition (WIA) APIs. For more information, see "Windows Image Acquisition" in the Platform SDK documentation. However, you can also use DirectShow to capture an image." I tried to explore the WIA. But this stopped to work on Vista. I continued to study the article.
Everything seems to be clear but you need to implement your class which inherits ISampleGrabberCB marked as deprecated here https://learn.microsoft.com/en-us/windows/desktop/directshow/isamplegrabbercb. Why???? Where to find some alternative?
I found an acceptable solution here https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/2ab5c212-5824-419d-b5d9-7f5db82f57cd/qedith-missing-in-current-windows-sdk-v70?forum=windowsdirectshowdevelopment. You need to add header file from elder SDK. (BTW This is an advice almost ten years old.) After I compiled the application with this header I was able to read high resolution picture but I need to wait a few seconds which is unacceptable. I know the problem is not in camera, because in the it works in the Camera application. Furthermore the image is obtained in function SampleCB instead of BufferCB and is in some strange format. I can save it as jpg but it is not compressed enough.
Windows Media Foundation
I think MS doesn't like programmers and that's why it released WMF. I understand nothing. I found this tutorial https://www.dreamincode.net/forums/topic/347938-a-new-webcam-api-tutorial-in-c-for-windows/. It works but it only stores one frame from preview and this is not what I want.
Next I explored some WMF interfaces on MS Docs. IMFCapturePhotoSink interface should do the stuff. But how implement it. The documentation is useless.
OpenCV
During my research I found also this library. But again I'm not able to take a high resolution photo. It only stores one frame from preview.
Could someone tell me what should I focus on? I believe it cannot be so difficult. There are tens and hundreds of applications for webcams. How could other programmers implement them? What's wrong with me? I'd like to find an easy way to implement an easy task. Thank a lot for any help.
You question is not related to the topic - the question must be related to the code - but I faced with the similar problem many years ago and I had found solution:
DirectShow is declared as deprecated for Windows 10 and it has problem with supporting of the USB web cam. In Windows 10 there is USB Video Class which is supported only by Media Foundation.
So, I have wrote a simple C++ wrapper around Media Foundation code which simplify getting of the raw images Capturing Video from Web-camera on Windows 7 and 8 by using Media Foundation
Also, there is project CaptureManager SDK - it is DLL COM component with the simple interfaces, huge functionality and with many demo programs on C++, Python, C#, Java.
Thanks to Evgeny.
Recapitulation:
Download the CaptureEngine video capture sample
Edit CaptureManager::TakePhoto method. Add the code to find highest resolution media type just before CreatePhotoMediaType(pMediaType, &pMediaType2); line
Extra code for setup the photo stream to highest resolution:
DWORD dwMediaTypeIndex = 0;
UINT32 maxSize = 0;
DWORD maxSizeIndex = 0;
while (1) {
IMFMediaType* pMediaType = NULL;
hr = pSource->GetAvailableDeviceMediaType((DWORD)MF_CAPTURE_ENGINE_PREFERRED_SOURCE_STREAM_FOR_PHOTO, dwMediaTypeIndex, &pMediaType);
if (hr == MF_E_NO_MORE_TYPES)
break;
UINT32 w, h;
MFGetAttributeSize(pMediaType, MF_MT_FRAME_SIZE, &w, &h);
UINT32 size = w * h;
if (size > maxSize) {
maxSize = size;
maxSizeIndex = dwMediaTypeIndex;
}
SafeRelease(&pMediaType);
dwMediaTypeIndex++;
}
SafeRelease(&pMediaType);
pSource->GetAvailableDeviceMediaType((DWORD)MF_CAPTURE_ENGINE_PREFERRED_SOURCE_STREAM_FOR_PHOTO, maxSizeIndex, &pMediaType);
I am creating sample application using Windows Media Foundation.
I have used Source Reader IMFSourceReader to Read the media file and then After I am processing the samples IMFSamples using Custom MFT IMFTransform.
In the MFT I have processed IMFSamples, How can I play/display them in a windows. I don't want to use EVR for display.
Also I have read the question:
How to play IMFMediaSample in media foundation?
As per the suggestion I need to use MFPlay for playing the samples, but exactly how this can be done.
In the interface IMFPMediaPlayer I am not able to find any method where I can push the media samples.
https://msdn.microsoft.com/en-us/library/windows/desktop/dd374329(v=vs.85).aspx
IMFSample is a wrapper over raw data. If you happen to waive standard API offering for playback/presentation (such as EVR for video), you will have to extract the data from the media sample object and consume it otherwise, such as using another API at your discretion.
This does not have to be visualization exactly, you are not limited in consumption ideas: writing to file, sending via network etc. For visualization you have other Windows APIs at your choice: DirectX, DirectShow, legacy DirectDraw, GDI, GDI+, Direct2D etc.
IMFSample is not immediately accepted by other APIs right away because it is not what it is designed for. In Media Foundation API EVR is designed for presentation, and EVR is what you are supposed to use.
The video sample object is a specialized implementation of the IMFSample interface for use with the Enhanced Video Renderer (EVR)...
I need your advice. I'd like to develop the app for audio/video splitting using Metro interface.
Usually I use DirectShow for it using the follow schema: create a grabber, add it to DS graph, capture by it the audio/video streams and pass them to my AVstream drivers for splitting. But in new program I want to use Media Foundation and insert it into UWP.
How I see my new app. It must have Metro interface for common control: choice of sources, adding parameters, changing modes and etc. I'd like to use MediaCapture class for capture of streams and rendering them too. Here I don't see any problems, MSDN has many samples for it. But I have no ideas how to insert a grabber between source and render.
Which operations a grabber will do:
Receive input stream from MediaCapture.
Stream converting : YUV -> RGB, adding effects and etc.
Send output stream for rendering (MediaCapture) and to my AVstream driver for splitting with any apps (Skype, Adobe Flash Player, Edge, ....).
How to make a grabber. In MSDN I found three ways:
Sample Grabber Sink (https://msdn.microsoft.com/en-us/library/windows/desktop/hh184779(v=vs.85).aspx). No problem to receive/control/send stream in MF dll. But I don't know how to link that dll with MediaCapture?
Source Reader (https://msdn.microsoft.com/en-us/library/windows/desktop/dd940436(v=vs.85).aspx). The same problems, plus the Source Reader doesn't work for playback.
Custom MFT? Any case MediaCapture allows to connect to MFT by AddEffectAsync().
My environment: MS Windows 10, MS Visual Studio Community 2015.
Thank you for any ideas.
This question and UWP are not actual for me at all. I found the following:
"Some apps can work intensively in background, for example it maybe video converting, online financial data processing and more.
Now UWP application will suspended when it go offscreen."
https://wpdev.uservoice.com/forums/110705-universal-windows-platform/suggestions/9950598-exclude-suspending-in-desktop
So if the user minimizes the program window, then the program stops a video stream.
I am trying to write a program which captures your speech applies my effect on the captured audio and play it back immediately. I am writing this app to the windows store in c++. After suffering with MediaCapture I decided to use WASAPI to do that. I am using an example from the msdn to set up WASAPI. As I expected I ran into a problem very soon. So the problem is with the following two lines:
const CLSID CLSID_MMDeviceEnumerator = __uuidof(MMDeviceEnumerator);
const IID IID_IMMDeviceEnumerator = __uuidof(IMMDeviceEnumerator);
The visual studio says that:
Error: operand of __uuidof must have a class or enum type for which __declspec(uuid('..')) has been specified.
I haven't got any idea what it means. I would appreciate some help here.
To be able to use __uuidof() a uuid must have been declared for the target. When compiling for Windows 8 Store Apps it seems that MMDeviceEnumerator and IMMDeviceEnumerator don't have any uuid declaration and you should instead use the Windows::Devices::Enumeration namespace to enumerate the devices and then use IActivateAudioInterfaceAsyncOperation to initialize and activate the audio device.
Here is a link to a fully working example that captures pcm data and writes it to a wav-file. It uses the systems' default audio capture device but enumerating the available devices and using any of them instead is a piece of cake. :)
WASAPI sample for Windows 8
Simply put, I want my C++/CX XAML Windows 8 app to output continuous synthesized sound (not sound effects). However I've been looking all over the Web and I cannot figure out how to get the system feed it buffers of PCM samples (or better, have it ask me for them through callbacks) for them to be played. I would use the old waveOut* APIs, however they are banned in Store app development.
So, what is the simplest way to do this? Please note that I am not interested in playing media files (.wav, .mp3) or web audio streaming.
Thanks in advance.
You need to use WASAPI which is enabled in Windows Store apps. This article will get you started with how to use the API to render audio. One annoyance is that WASAPI devices generally don't resample for you so you'll have to be willing to go with what the device is using (probably 44.1kHz or 48kHz) or do the resampling yourself (for which you can make use of the Resampler Media Foundation transform).