I am trying to write a program which captures your speech applies my effect on the captured audio and play it back immediately. I am writing this app to the windows store in c++. After suffering with MediaCapture I decided to use WASAPI to do that. I am using an example from the msdn to set up WASAPI. As I expected I ran into a problem very soon. So the problem is with the following two lines:
const CLSID CLSID_MMDeviceEnumerator = __uuidof(MMDeviceEnumerator);
const IID IID_IMMDeviceEnumerator = __uuidof(IMMDeviceEnumerator);
The visual studio says that:
Error: operand of __uuidof must have a class or enum type for which __declspec(uuid('..')) has been specified.
I haven't got any idea what it means. I would appreciate some help here.
To be able to use __uuidof() a uuid must have been declared for the target. When compiling for Windows 8 Store Apps it seems that MMDeviceEnumerator and IMMDeviceEnumerator don't have any uuid declaration and you should instead use the Windows::Devices::Enumeration namespace to enumerate the devices and then use IActivateAudioInterfaceAsyncOperation to initialize and activate the audio device.
Here is a link to a fully working example that captures pcm data and writes it to a wav-file. It uses the systems' default audio capture device but enumerating the available devices and using any of them instead is a piece of cake. :)
WASAPI sample for Windows 8
Related
I can not seem to find any tutorial on the internet for my question.
All the simple guide is not suitable for UWP.
For Example,
To use WASAPI there are these steps
enumerate devices
capture audio
play (render) audio back
But the enumerating step, The client must call CoCreateInstance. But from my understanding this function is not support in UWP. Also I failed at Line 30 when following this code.
So, I try to understand This, C++ UWP using WASAPI, But I can't find any Enumerate part and this project is very complicate for me.
It include a lot of other files (DeviceState.h, common.h)
And I failed to extract the code to create my own application.
My question is how can I capture audio on c++ UWP app with WASAPI?
If this question is too board, I will change my question to How to enumerate audio device in c++ UWP application?.
And the reason why I use WASAPI is because I want to access the data stored in the Buffer.
Edit:
For enumerating.
https://github.com/Microsoft/Windows-universal-samples/blob/7c7832e1f144e4fc836603fd70e1352024a5fe1a/Samples/WindowsAudioSession/cpp/Scenario1.xaml.cpp#L85
Yes, you can use WASAPI to do audio capturing in UWP and this is what is done in the sample you have referenced (https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/WindowsAudioSession).
For the enumeration, the main function is DeviceInformation::FindAllAsync with this selector MediaDevice::GetAudioCaptureSelector it will allow you to list the capture devices.
For the stream capturing, the main function you need is
ActivateAudioInterfaceAsync, it will allow you to create an IAudioClient from a device id (specific device) or a device class (render or capture) if you just need to use the default device.
Once you have this IAudioClient you can use it to get an IAudioClientCapture, basically the things that you have seen in the sample.
I am developing an C++ application which should use an USB camera to capture high resolution photos. It should have same behavior as the Camera application in Windows 10. I am trying to use DirectShow for doing it. Now I am only able to take high resolution photo which is delayed or take a photo in time but low resolution. Also I am very confused from MS documentation, lot of things are deprecated and nowhere mentioned what replaces them. I'll describe my hopeless steps awaiting there will be somebody who could be able to show me a way.
Let's start from beginning...
Knowing nothing about video capturing in Window I started by searching suitable library. After some googling I found there are four main libraries for capturing video in Windows.
Video for Windows
DirectShow
Windows Media Foundation
OpenCV
Let's observe:
Video for Windows
This library is unfortunately marked as deprecated but it seems it still works. I have written "unfortunately", because I think this is the only which is easy to use. There are only a few lines of code needed for seeing video from camera. The only think I miss here is a "TakePhoto" function. You can use VFW for capture a video or single frames to an avi file. Or am I missing something?
DirectShow
This is much more complicated library. You need hundreds of lines of code to see a video preview. But you can obtain this code on MS Docs. Ok, now I have a video preview and I need only to take a photo. One would expect this should be just one function call. But where is the function? I did not find it.
You can simply use GetCurrentImage from IVMRWindowlessControl but this takes only one frame from preview with low resolution. If you set a higher resolution for preview the video is not fluent.
Best approach I could achieve is from an article called "Capturing an Image From a Still Image Pin" available here https://learn.microsoft.com/en-us/windows/desktop/directshow/capturing-an-image-from-a-still-image-pin. When I had found this site I thought I won and my task was almost finished. But it wasn't.
The first advice which the article gives you is not to use it: "The recommended way to get still images from the device is to use the Windows Image Acquisition (WIA) APIs. For more information, see "Windows Image Acquisition" in the Platform SDK documentation. However, you can also use DirectShow to capture an image." I tried to explore the WIA. But this stopped to work on Vista. I continued to study the article.
Everything seems to be clear but you need to implement your class which inherits ISampleGrabberCB marked as deprecated here https://learn.microsoft.com/en-us/windows/desktop/directshow/isamplegrabbercb. Why???? Where to find some alternative?
I found an acceptable solution here https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/2ab5c212-5824-419d-b5d9-7f5db82f57cd/qedith-missing-in-current-windows-sdk-v70?forum=windowsdirectshowdevelopment. You need to add header file from elder SDK. (BTW This is an advice almost ten years old.) After I compiled the application with this header I was able to read high resolution picture but I need to wait a few seconds which is unacceptable. I know the problem is not in camera, because in the it works in the Camera application. Furthermore the image is obtained in function SampleCB instead of BufferCB and is in some strange format. I can save it as jpg but it is not compressed enough.
Windows Media Foundation
I think MS doesn't like programmers and that's why it released WMF. I understand nothing. I found this tutorial https://www.dreamincode.net/forums/topic/347938-a-new-webcam-api-tutorial-in-c-for-windows/. It works but it only stores one frame from preview and this is not what I want.
Next I explored some WMF interfaces on MS Docs. IMFCapturePhotoSink interface should do the stuff. But how implement it. The documentation is useless.
OpenCV
During my research I found also this library. But again I'm not able to take a high resolution photo. It only stores one frame from preview.
Could someone tell me what should I focus on? I believe it cannot be so difficult. There are tens and hundreds of applications for webcams. How could other programmers implement them? What's wrong with me? I'd like to find an easy way to implement an easy task. Thank a lot for any help.
You question is not related to the topic - the question must be related to the code - but I faced with the similar problem many years ago and I had found solution:
DirectShow is declared as deprecated for Windows 10 and it has problem with supporting of the USB web cam. In Windows 10 there is USB Video Class which is supported only by Media Foundation.
So, I have wrote a simple C++ wrapper around Media Foundation code which simplify getting of the raw images Capturing Video from Web-camera on Windows 7 and 8 by using Media Foundation
Also, there is project CaptureManager SDK - it is DLL COM component with the simple interfaces, huge functionality and with many demo programs on C++, Python, C#, Java.
Thanks to Evgeny.
Recapitulation:
Download the CaptureEngine video capture sample
Edit CaptureManager::TakePhoto method. Add the code to find highest resolution media type just before CreatePhotoMediaType(pMediaType, &pMediaType2); line
Extra code for setup the photo stream to highest resolution:
DWORD dwMediaTypeIndex = 0;
UINT32 maxSize = 0;
DWORD maxSizeIndex = 0;
while (1) {
IMFMediaType* pMediaType = NULL;
hr = pSource->GetAvailableDeviceMediaType((DWORD)MF_CAPTURE_ENGINE_PREFERRED_SOURCE_STREAM_FOR_PHOTO, dwMediaTypeIndex, &pMediaType);
if (hr == MF_E_NO_MORE_TYPES)
break;
UINT32 w, h;
MFGetAttributeSize(pMediaType, MF_MT_FRAME_SIZE, &w, &h);
UINT32 size = w * h;
if (size > maxSize) {
maxSize = size;
maxSizeIndex = dwMediaTypeIndex;
}
SafeRelease(&pMediaType);
dwMediaTypeIndex++;
}
SafeRelease(&pMediaType);
pSource->GetAvailableDeviceMediaType((DWORD)MF_CAPTURE_ENGINE_PREFERRED_SOURCE_STREAM_FOR_PHOTO, maxSizeIndex, &pMediaType);
It's hard to find anything on the web reliable for audio rendering (there is capture, record and save to file, but rendering to speakers are rare using Media Foundation).
The problem I have is that the tutorial of audio rendering provided by Microsoft are... all for desktop application.
There is one that could be used for UWP as it is using WASAPI:
But... the uuid of lines fails on UWP:
const CLSID CLSID_MMDeviceEnumerator = __uuidof(MMDeviceEnumerator);
const IID IID_IMMDeviceEnumerator = __uuidof(IMMDeviceEnumerator);
const IID IID_IAudioClient = __uuidof(IAudioClient);
const IID IID_IAudioRenderClient = __uuidof(IAudioRenderClient);
Someone seemed to have the same problem:
WASAPI + windows store app initialization
Error: operand of __uuidof must have a class or enum type for which __declspec(uuid('..')) has been specified.
Nobody really answered that as well and even more the link he provides is not reachable (or at least not anymore). I Can't figure out how to activate the output audio device and stream audio to it on Universal Windows Platform.
I am using Media Foundation for a specific scenario and I use a Source Reader to get asynchronously audio decoded PCM samples.
While there are samples that should work on Standalone desktop builds, I can't find anywhere something that is compatible with Universal Windows Platform...
Anyone has done something similar (audio rendering using Media Foundation or WASAPI with C++ for UWP?) or at least got better links that would help me?
Thanks for your help!
I need your advice. I'd like to develop the app for audio/video splitting using Metro interface.
Usually I use DirectShow for it using the follow schema: create a grabber, add it to DS graph, capture by it the audio/video streams and pass them to my AVstream drivers for splitting. But in new program I want to use Media Foundation and insert it into UWP.
How I see my new app. It must have Metro interface for common control: choice of sources, adding parameters, changing modes and etc. I'd like to use MediaCapture class for capture of streams and rendering them too. Here I don't see any problems, MSDN has many samples for it. But I have no ideas how to insert a grabber between source and render.
Which operations a grabber will do:
Receive input stream from MediaCapture.
Stream converting : YUV -> RGB, adding effects and etc.
Send output stream for rendering (MediaCapture) and to my AVstream driver for splitting with any apps (Skype, Adobe Flash Player, Edge, ....).
How to make a grabber. In MSDN I found three ways:
Sample Grabber Sink (https://msdn.microsoft.com/en-us/library/windows/desktop/hh184779(v=vs.85).aspx). No problem to receive/control/send stream in MF dll. But I don't know how to link that dll with MediaCapture?
Source Reader (https://msdn.microsoft.com/en-us/library/windows/desktop/dd940436(v=vs.85).aspx). The same problems, plus the Source Reader doesn't work for playback.
Custom MFT? Any case MediaCapture allows to connect to MFT by AddEffectAsync().
My environment: MS Windows 10, MS Visual Studio Community 2015.
Thank you for any ideas.
This question and UWP are not actual for me at all. I found the following:
"Some apps can work intensively in background, for example it maybe video converting, online financial data processing and more.
Now UWP application will suspended when it go offscreen."
https://wpdev.uservoice.com/forums/110705-universal-windows-platform/suggestions/9950598-exclude-suspending-in-desktop
So if the user minimizes the program window, then the program stops a video stream.
I developed a video capture application in C++ with Qt on Windows 7, using the videoInput library (which uses DirectShow). My application works fine with all the devices I previously tried (some inboard capture devices, webcams, a Pinnacle Dazzle DVC 100, a Pinnacle Studio Movie Box, and even a troubled PixelView PlayTV USB Hybrid), but now I bought an AVerMedia DarkCrystal HD Capture Station (in order to support HD capturing) and it simply won't work.
According to the AVerMedia website, it is totally DirectShow-compatible and I don't even need to use its SDK. I followed the manufacturer's installation instructions and I'm able to capture video using its own bundled software (TotalMedia Studio MV), so I suppose it's working properly.
I tried to run my videoInput-based application and also AmCap (DirectShow sample from Microsoft) and both of them behave the same way. They are able to find the device, retrieve its friendly name, but nothing else. AmCap says Error 80004002: Cannot find VCapture:IAMStreamConfig.
I was debugging AmCap and found out the ICaptureGraphBuilder2::FindInterface method always returns E_NOINTERFACE, i.e. no such interface supported, in lines like this:
hr = gcap.pBuilder->FindInterface(&PIN_CATEGORY_CAPTURE,
&MEDIATYPE_Video, gcap.pVCap,
IID_IAMStreamConfig, (void **)&gcap.pVSC);
Here, gcap.pBuilder is a pointer to ICaptureGraphBuilder2, gcap.pVCap is a pointer to IBaseFilter, and gcap.pVSC is a pointer to IAMStreamConfig and should receive the return value.
I suppose there's some workaround that could help me to capture from this device, since I could sucessfully retrieve the device name (using the ICreateDevEnum::CreateClassEnumerator method, based on the CLSID_VideoInputDeviceCategory class identifier).
A video capture device which does not implement IAMStreamConfig is a possible scenario, though it's not something expected. Actually it more looks like a bug on video device or underlying driver, however you might want to update your app to use default media type in case of missing IAMStreamConfig (you can still use GraphEdit to check if the device is operational).