I need to do some basic video processing from the Webcam in Windows Phone 8.1.
I cannot find any examples of how to access the webcam preview buffer. The Microsoft examples (very few) all have a video preview frame activate, I can find none that show how to subscribe to a 'new frame ready' event, or where the buffer is.
The MediaCapture, and CaptureDevice appear to be the main ways of reaching the camera in this api.
Can anyone point to a specific example? For instance, a QR code reader, or maybe a program that adds video effects, like greyscale, would need to attach an event to every frame.
Thanks.
It sounds like you need to access the buffer from the preview stream of the phone cameras. If you absolutely need to target 8 and 8.1, then you should look into GetPreviewBufferArgb(out int[] pixels). See here: https://msdn.microsoft.com/en-us/library/windows.phone.media.capture.cameracaptureframe.getpreviewbufferargb.aspx
But if you can instead target Windows 10, you get the benefit of fully universal APIs, and you don't take a dependency on the likely soon to be deprecated (if not already) Windows.Phone.Media.Capture namespace. I would strongly recommend doing this instead, as the APIs will be easier to work with, and any 8.1 device can upgrade to 10 for free.
Here is a fully working Windows 10 sample: http://aka.ms/2015buildgetpreviewframesample, which was shown off at the last Build conference (video here: http://channel9.msdn.com/Events/Build/2015/2-730).
Related
I am developing an C++ application which should use an USB camera to capture high resolution photos. It should have same behavior as the Camera application in Windows 10. I am trying to use DirectShow for doing it. Now I am only able to take high resolution photo which is delayed or take a photo in time but low resolution. Also I am very confused from MS documentation, lot of things are deprecated and nowhere mentioned what replaces them. I'll describe my hopeless steps awaiting there will be somebody who could be able to show me a way.
Let's start from beginning...
Knowing nothing about video capturing in Window I started by searching suitable library. After some googling I found there are four main libraries for capturing video in Windows.
Video for Windows
DirectShow
Windows Media Foundation
OpenCV
Let's observe:
Video for Windows
This library is unfortunately marked as deprecated but it seems it still works. I have written "unfortunately", because I think this is the only which is easy to use. There are only a few lines of code needed for seeing video from camera. The only think I miss here is a "TakePhoto" function. You can use VFW for capture a video or single frames to an avi file. Or am I missing something?
DirectShow
This is much more complicated library. You need hundreds of lines of code to see a video preview. But you can obtain this code on MS Docs. Ok, now I have a video preview and I need only to take a photo. One would expect this should be just one function call. But where is the function? I did not find it.
You can simply use GetCurrentImage from IVMRWindowlessControl but this takes only one frame from preview with low resolution. If you set a higher resolution for preview the video is not fluent.
Best approach I could achieve is from an article called "Capturing an Image From a Still Image Pin" available here https://learn.microsoft.com/en-us/windows/desktop/directshow/capturing-an-image-from-a-still-image-pin. When I had found this site I thought I won and my task was almost finished. But it wasn't.
The first advice which the article gives you is not to use it: "The recommended way to get still images from the device is to use the Windows Image Acquisition (WIA) APIs. For more information, see "Windows Image Acquisition" in the Platform SDK documentation. However, you can also use DirectShow to capture an image." I tried to explore the WIA. But this stopped to work on Vista. I continued to study the article.
Everything seems to be clear but you need to implement your class which inherits ISampleGrabberCB marked as deprecated here https://learn.microsoft.com/en-us/windows/desktop/directshow/isamplegrabbercb. Why???? Where to find some alternative?
I found an acceptable solution here https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/2ab5c212-5824-419d-b5d9-7f5db82f57cd/qedith-missing-in-current-windows-sdk-v70?forum=windowsdirectshowdevelopment. You need to add header file from elder SDK. (BTW This is an advice almost ten years old.) After I compiled the application with this header I was able to read high resolution picture but I need to wait a few seconds which is unacceptable. I know the problem is not in camera, because in the it works in the Camera application. Furthermore the image is obtained in function SampleCB instead of BufferCB and is in some strange format. I can save it as jpg but it is not compressed enough.
Windows Media Foundation
I think MS doesn't like programmers and that's why it released WMF. I understand nothing. I found this tutorial https://www.dreamincode.net/forums/topic/347938-a-new-webcam-api-tutorial-in-c-for-windows/. It works but it only stores one frame from preview and this is not what I want.
Next I explored some WMF interfaces on MS Docs. IMFCapturePhotoSink interface should do the stuff. But how implement it. The documentation is useless.
OpenCV
During my research I found also this library. But again I'm not able to take a high resolution photo. It only stores one frame from preview.
Could someone tell me what should I focus on? I believe it cannot be so difficult. There are tens and hundreds of applications for webcams. How could other programmers implement them? What's wrong with me? I'd like to find an easy way to implement an easy task. Thank a lot for any help.
You question is not related to the topic - the question must be related to the code - but I faced with the similar problem many years ago and I had found solution:
DirectShow is declared as deprecated for Windows 10 and it has problem with supporting of the USB web cam. In Windows 10 there is USB Video Class which is supported only by Media Foundation.
So, I have wrote a simple C++ wrapper around Media Foundation code which simplify getting of the raw images Capturing Video from Web-camera on Windows 7 and 8 by using Media Foundation
Also, there is project CaptureManager SDK - it is DLL COM component with the simple interfaces, huge functionality and with many demo programs on C++, Python, C#, Java.
Thanks to Evgeny.
Recapitulation:
Download the CaptureEngine video capture sample
Edit CaptureManager::TakePhoto method. Add the code to find highest resolution media type just before CreatePhotoMediaType(pMediaType, &pMediaType2); line
Extra code for setup the photo stream to highest resolution:
DWORD dwMediaTypeIndex = 0;
UINT32 maxSize = 0;
DWORD maxSizeIndex = 0;
while (1) {
IMFMediaType* pMediaType = NULL;
hr = pSource->GetAvailableDeviceMediaType((DWORD)MF_CAPTURE_ENGINE_PREFERRED_SOURCE_STREAM_FOR_PHOTO, dwMediaTypeIndex, &pMediaType);
if (hr == MF_E_NO_MORE_TYPES)
break;
UINT32 w, h;
MFGetAttributeSize(pMediaType, MF_MT_FRAME_SIZE, &w, &h);
UINT32 size = w * h;
if (size > maxSize) {
maxSize = size;
maxSizeIndex = dwMediaTypeIndex;
}
SafeRelease(&pMediaType);
dwMediaTypeIndex++;
}
SafeRelease(&pMediaType);
pSource->GetAvailableDeviceMediaType((DWORD)MF_CAPTURE_ENGINE_PREFERRED_SOURCE_STREAM_FOR_PHOTO, maxSizeIndex, &pMediaType);
This is going to be my first question in StackOverflow after several days looking for an explanation. Please, be gentle with me for asking because I know my problem is a bit bizarre to be a general problem.
I made a MF capture video application, based in the Microsoft example 'CaptureToFile'. It did work on Windows 7 x64. I upgraded to Visual Studio 2013 without problems. Problems arose when I try to put all the development on a Windows 8.1 x64 machine.
The app compiles and executes without error, but it's UNABLE to capture samples by using m_pReader->ReadSample() in asynchronous mode; only the first two samples arrive to OnReadSample method; and there must be 'control' samples, because the IMFSample is null in all of them. After that, the app gets 'hanged' waiting for data.
I've tried the original MFCaptureToFile sample with the same sad results.
Of course, I think hardware and software are similar (the same capture card with the same driver version, both are desktop PC...)
Do you know any possible reason for this behaviour? in Win7 everything is working flawless! Or at least, if you could light me a bit about new paths for finding what's happening
Thanks in advance
UPDATE: There is another 'player' in the game. Looking into the threads, I see that a worker thread is in 'RTWorkQ.dll', the real-time work queue container, specific only for Windows 8. I will go on investigating. In the meantime, if you have any idea, something to share, I'll be glad to hear you.
UPDATE 2: I've modified the sample MFCaptureToFile to get the video samples synchronously, because I thought the problem could be due to the asynchronous behaviour; related with the queues. I've to say that the problem persist even with this change. The second time it tries to read a sample, the application gets 'hanged' waiting for a reading that doesn't never arrives.
UPDATE 3: I've tried with the CaptureEngine sample application that uses another MF way to capture video (MFCaptureEngine). It builds and runs flawlessly but doesn't show any images when starting the 'preview' and doesn't record any useful, only non-playable files.
UPDATE 4: I've installed Visual Studio 2010 Ultimate in Windows 8 PRO. The sample MFCaptureToFile fails again in the sample. It's unable to read a 2nd sample from the frame grabber. I'm starting to think that can be an incompatibility between the capture card (Datapath VisionRGB-E1S) and Windows 8 PRO despite the driver assures it works fine in this platform and the test program shows images. Tomorrow I'm going to try the test with an external USB webcam.
Finally, I have figured out the reason of this problem.
With Windows 8.1 release Microsoft has introduced New AVStream Interfaces for Windows 8.1
There is a small but very important change in KS_FRAME_INFO structure - the new FrameCompletionNumber member.
An identifying sequence number for the frame in the completed queue.
This number is used to verify proper frame order. When this value is
0, the frame was cancelled. This member is available starting with
Windows 8.1.
DirectShow doesn't care about this number. And MediaFoundation cares.
So, you cannot just fix that on your user-mode side. The manufacture developers must release an update. Btw, I have two webcams - Logitech C270 and Creative Live Socialize HD. Logitech supports Metro while Creative does not.
I have successfully updated my driver with only a few lines of code (to set up FrameCompletionNumber properly).
UPD. similar thread http://www.osronline.com/showthread.cfm?link=255004
It must be a problem of the frame grabber Datapath VisionRGB-E1S. I've tried with the brand-new USB webcam LifeCam Studio, and everything worked fine.
I will left for other future thread why this unpaired behaviour between Windows 8 and Windows 7, but it could be something related to the User-mode access...
I had the same kind of issue:
IMFSourceReader was obtained successfully
reader->SetCurrentMediaType() reported no error.
reader->ReadSample() was successful.
then OnReadSample() was called only once and the hrStatus argument 0x80070491
For me, the issue was that I modified the video subtype IMFMediaType, then applied to the reader as current media type.
We have an "MC1362 Camera" and an "Inspecta-5" frame grabber in our lab. There is program in LABVIEW11 which gets the data from a frame grabber, however as the Labview is slow my supervisor has told me to write a program in c++ to get the data from the frame grabber. I have no idea how to write a c++ program to connect to a frame grabber and do the data acquisition. I know how to write software in c++, but have never tried programming to connect to hardware and read data from it. Is there any specific library or framework which can help me, or any tutorial?
Please, if anybody knows, help me in this matter.
Update:just to add, we are doing medical image analysis, and a laser illuminate a subject, so camera will take pictures and pass it to the computer. I need to grab the pictures and analysis them.
You basically have a couple of options,
1 see if there is an SDK for the grabber card, if there is this is usually easier then option 2 but is of course restricted to work with that grabber or familly of grabber cards, we do it this way with the eurysys grabber cards.
2 assuming you are running on a windows platform, implement a DirectShow filtergraph and write your own ouput filter to get the data, the SDK for DirectShow is quiet good and has many examples. This approach is far more flexible and you should be able to use a number of grabber but its also alot more complex, we do it this way for USB / some other inbuilt grabbers.
Our software is done in Delphi 7 but its just importing DLLs, for C++ should be no problem and most SDK's are written round C++ anyway.
I know its not much but its a place to start.
Update
Just done a quick Google search and there is an SDK for that Grabber and on first looks its seams fairly straight forward.
In our project, we would like to access the webcam image programmatically.
The main problem we have is that the webcam automatically adjusts the sensitivity depending on the brightness of the captured image.
Is there any (platform-independent) way to change this kind of parameters for the webcam (preferably any model)?
We're currently using Ubuntu 10.04, Microsoft Windows XP & 7. Programming language is C/C++.
Any idea is appreciated.
Thanks and regards
Tobias
There most likely won't be any platform independent way to do what you need. If there is, it's probably by using some high level language, which likely won't suit.
I don't know about the linux platform, but I'm a C++/windows/COM/DirectShow developer who works on internet based video applications.
On the Windows platform, capture devices are communicated with via COM and DirectShow.
For a general overview of video capture on windows, see the Video Capture section of MSDN.
Have a look at Selecting a Capture Device for information on how to enumerate the capture devices on your system. You'll need to enumerate the devices in the CLSID_VideoInputDeviceCategory, in order to discover (programmatically) the webcam as a video input device - there may be many devices in this category.
Video capture devices have a "FriendlyName" to help identify your webcam that you can store and retrieve the device for later use.
Once you've got the device, your query said you'd wanted to configure the device. Check out the Configuring a Video Capture Device for this.
DirectShow is one of Microsoft's most comprehensive (and difficult) APIs to learn. The MSDN developer forum on DirectShow is very active and beginner friendly and I highly recommend you check it out.
Finally, capture graphs aren't the easiest thing to build in DirectShow, I'd start off with a simple playback graph - e.g. playback a media file from disk and progress from there to capture graphs.
The VLC project is open source and cross-platform and it uses DirectShow for playback on the windows platform.
Good luck!
When I run this code:
MIXERLINE MixerLine;
memset( &MixerLine, 0, sizeof(MIXERLINE) );
MixerLine.cbStruct = sizeof(MIXERLINE);
MixerLine.dwComponentType = MIXERLINE_COMPONENTTYPE_SRC_WAVEOUT;
mmResult = mixerGetLineInfo( (HMIXEROBJ)m_dwMixerHandle, &MixerLine, MIXER_GETLINEINFOF_COMPONENTTYPE );
Under XP MixerLine.cChannels comes back as the number of channels that the sound card supports. Often 2, these days often many more.
Under Vista MixerLine.cChannels comes back as one.
I have been then getting a MIXERCONTROL_CONTROLTYPE_VOLUME control and setting the volume for each channel that is supported, and setting the volumne control to different levels on different channels so as to pan music back and forth between the speakers (left to right).
Obviously under Vista this approach isn't working since there is only one channel. I can set the volume and it is for both channels at the same time.
I tried to get a MIXERCONTROL_CONTROLTYPE_PAN for this device, but that was not a valid control.
So, the question for all you MMSystem experts is this: what type of control do I need to get to adjust the left/right balance? Alternately, is there a better way? I would like a solution that works with both XP and Vista.
Computer Details: Running Vista Ultimta 32 bit SP1 and all latest patches. Audio is provided by a Creative Audigy 2 ZS card with 4 speakers attached which can all be properly addressed (controlled) through Vista's sound panel. Driver is latest on Creative's site (SBAX_PCDRV_LB_2_18_0001). The Vista sound is not set to mono, and all channels are visable and controlable from the sound panel.
Running the program in "XP Compatibility Mode" does not change the behaviour of this problem.
If you run your application in "XP compatibility" mode, the mixer APIs should work much closer to the way they did in XP.
If you're not running in XP mode, then the mixer APIs reflect the mix format - if your PC's audio solution is configured for mono, then you'll see only one channel, but if you're machine is configured for multichannel output the mixer APIs should reflect that.
You can run the speaker tuning wizard to determine the # of channels configured for your audio solution.
Long time Microsoftie Larry Osterman has a blog where he discusses issues like this because he was on the team that redid all the audio stuff in Vista.
In the comments to this blog post he seems to indicate that application controlled balance is not something they see the need for:
CN, actually we're not aware of ANY situations in which it's appropriate for an application to control its balance. Having said that, we do support individual channel volumes for applications, but it is STRONGLY recommended that apps don't use it.
He also indicates that panning the sound from one side to the other can be done, but it is dependent on whether the hardware supports it:
Joku, we're exposing the volume controls that the audio solution implements. If it can do pan, we do pan (we actually expose separate sliders for the left and right channels).
So that explains why the MIXERCONTROL_CONTROLTYPE_PAN thing failed -- the audio hardware on your system does not support it.