I'm creating a Windows video capture application and am using DirectShow for capture. As each frame comes in, I want to grab it as a raw RGB bitmap into a buffer, at which point my code will do whatever processing I need.
I've been searching for samples similar to what I want to do, and everywhere I look online, people recommend using either the IMediaDet and/or the ISampleGrabber interface to do frame-by-frame capture. Unfortunately, both are deprecated and aren't even in the newest version of the Windows SDK.
What is the best (modern) way to do frame-by-frame capture in DirectShow? If there is none, is there a different library I should use that will give me frame-by-frame capture?
Sample Grabber was deprecated a few years ago, which was a few years after DirectShow development actually stopped. That is, use Sample Grabber as you read as suggested method and it is going to work great for you.
The only thing you will additionally need is to copy definitions int your source code, see details:
Alternative for ISampleGrabber
Sample Grabber replacement
ISampleGrabber deprecated: where can I find alternatives?
Related
I am developing an C++ application which should use an USB camera to capture high resolution photos. It should have same behavior as the Camera application in Windows 10. I am trying to use DirectShow for doing it. Now I am only able to take high resolution photo which is delayed or take a photo in time but low resolution. Also I am very confused from MS documentation, lot of things are deprecated and nowhere mentioned what replaces them. I'll describe my hopeless steps awaiting there will be somebody who could be able to show me a way.
Let's start from beginning...
Knowing nothing about video capturing in Window I started by searching suitable library. After some googling I found there are four main libraries for capturing video in Windows.
Video for Windows
DirectShow
Windows Media Foundation
OpenCV
Let's observe:
Video for Windows
This library is unfortunately marked as deprecated but it seems it still works. I have written "unfortunately", because I think this is the only which is easy to use. There are only a few lines of code needed for seeing video from camera. The only think I miss here is a "TakePhoto" function. You can use VFW for capture a video or single frames to an avi file. Or am I missing something?
DirectShow
This is much more complicated library. You need hundreds of lines of code to see a video preview. But you can obtain this code on MS Docs. Ok, now I have a video preview and I need only to take a photo. One would expect this should be just one function call. But where is the function? I did not find it.
You can simply use GetCurrentImage from IVMRWindowlessControl but this takes only one frame from preview with low resolution. If you set a higher resolution for preview the video is not fluent.
Best approach I could achieve is from an article called "Capturing an Image From a Still Image Pin" available here https://learn.microsoft.com/en-us/windows/desktop/directshow/capturing-an-image-from-a-still-image-pin. When I had found this site I thought I won and my task was almost finished. But it wasn't.
The first advice which the article gives you is not to use it: "The recommended way to get still images from the device is to use the Windows Image Acquisition (WIA) APIs. For more information, see "Windows Image Acquisition" in the Platform SDK documentation. However, you can also use DirectShow to capture an image." I tried to explore the WIA. But this stopped to work on Vista. I continued to study the article.
Everything seems to be clear but you need to implement your class which inherits ISampleGrabberCB marked as deprecated here https://learn.microsoft.com/en-us/windows/desktop/directshow/isamplegrabbercb. Why???? Where to find some alternative?
I found an acceptable solution here https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/2ab5c212-5824-419d-b5d9-7f5db82f57cd/qedith-missing-in-current-windows-sdk-v70?forum=windowsdirectshowdevelopment. You need to add header file from elder SDK. (BTW This is an advice almost ten years old.) After I compiled the application with this header I was able to read high resolution picture but I need to wait a few seconds which is unacceptable. I know the problem is not in camera, because in the it works in the Camera application. Furthermore the image is obtained in function SampleCB instead of BufferCB and is in some strange format. I can save it as jpg but it is not compressed enough.
Windows Media Foundation
I think MS doesn't like programmers and that's why it released WMF. I understand nothing. I found this tutorial https://www.dreamincode.net/forums/topic/347938-a-new-webcam-api-tutorial-in-c-for-windows/. It works but it only stores one frame from preview and this is not what I want.
Next I explored some WMF interfaces on MS Docs. IMFCapturePhotoSink interface should do the stuff. But how implement it. The documentation is useless.
OpenCV
During my research I found also this library. But again I'm not able to take a high resolution photo. It only stores one frame from preview.
Could someone tell me what should I focus on? I believe it cannot be so difficult. There are tens and hundreds of applications for webcams. How could other programmers implement them? What's wrong with me? I'd like to find an easy way to implement an easy task. Thank a lot for any help.
You question is not related to the topic - the question must be related to the code - but I faced with the similar problem many years ago and I had found solution:
DirectShow is declared as deprecated for Windows 10 and it has problem with supporting of the USB web cam. In Windows 10 there is USB Video Class which is supported only by Media Foundation.
So, I have wrote a simple C++ wrapper around Media Foundation code which simplify getting of the raw images Capturing Video from Web-camera on Windows 7 and 8 by using Media Foundation
Also, there is project CaptureManager SDK - it is DLL COM component with the simple interfaces, huge functionality and with many demo programs on C++, Python, C#, Java.
Thanks to Evgeny.
Recapitulation:
Download the CaptureEngine video capture sample
Edit CaptureManager::TakePhoto method. Add the code to find highest resolution media type just before CreatePhotoMediaType(pMediaType, &pMediaType2); line
Extra code for setup the photo stream to highest resolution:
DWORD dwMediaTypeIndex = 0;
UINT32 maxSize = 0;
DWORD maxSizeIndex = 0;
while (1) {
IMFMediaType* pMediaType = NULL;
hr = pSource->GetAvailableDeviceMediaType((DWORD)MF_CAPTURE_ENGINE_PREFERRED_SOURCE_STREAM_FOR_PHOTO, dwMediaTypeIndex, &pMediaType);
if (hr == MF_E_NO_MORE_TYPES)
break;
UINT32 w, h;
MFGetAttributeSize(pMediaType, MF_MT_FRAME_SIZE, &w, &h);
UINT32 size = w * h;
if (size > maxSize) {
maxSize = size;
maxSizeIndex = dwMediaTypeIndex;
}
SafeRelease(&pMediaType);
dwMediaTypeIndex++;
}
SafeRelease(&pMediaType);
pSource->GetAvailableDeviceMediaType((DWORD)MF_CAPTURE_ENGINE_PREFERRED_SOURCE_STREAM_FOR_PHOTO, maxSizeIndex, &pMediaType);
I need to add webcam video capture to a legacy MFC C++ application. The video needs to be saved as MP4. Did a bit of googling but didn't come across anything that looked promising. Any suggestions on the best approach?
EDIT:
Windows platform.
EDIT:
Must be compatible with XP
There are a few popular options to choose from:
DirectShow API - it does not have stock MPEG-4 compressors for video and audio, neither it has a stock multiplexor for .MP4 format, though there is an excellent free multiplexor from GDCL: http://www.gdcl.co.uk/mpeg4/. Also there is decent documentation, a lot of samples
Media Foundation API - it has everything you need (codecs, multiplexor) but only in Windows 7 (maybe even not all edtions)
FFmpeg and libavcodec/libavformat are definitely relevant, however H.264 encoder is only available under GPL license, not sure about video capture part right there, and you might have hard time looking for documentation and samples.
I'd say look at OpenCV as a library, hook into their video capture for that aspect, it can write out to mp4 but you'll need a couple of other libs for handling the output stream (on Linux I'd say ffmpeg and x264), that should get the buffer into the container with a reasonable amount of hassle.
I want to capture a video from a real webcam, apply filters with openCv and write the filtered video on a virtual webcam, to stream it on web.
I don't have problem with the first 2 points, but I don't know how I can write on a virtual webcam.
It's possible?
How can I do it?
I use openCv with C++ on Debian.
Thanks
Well, actually this is possible. A quick and dirty way to do this is to use WebcamStudio.
That will create a new video device (e.g., /device/video2) that other programs see as a normal video device and can take its input from desktop, so you just set it up to capture a part of the screen that OpenCV's output is shown there.
A better but more technical way is to use the V4L2 loop back module. This way you can simply pipe the output of OpenCV to the module which is seen as a regular video device by the other programs. See the readme at the bottom of this page:
https://github.com/umlaeute/v4l2loopback
and the wiki page:
https://github.com/umlaeute/v4l2loopback/wiki
for more information.
Hope that helps.
You can also use a combination of v4l2loopback, OBS Studio and obs-v4l2sink.
Use OBS Studio to capture video from your device, then obs-v4l2sink is a small plugin that writes output into /dev/video* of your choice.
https://github.com/umlaeute/v4l2loopback/wiki/OBS-Studio
https://github.com/CatxFish/obs-v4l2sink
We have an "MC1362 Camera" and an "Inspecta-5" frame grabber in our lab. There is program in LABVIEW11 which gets the data from a frame grabber, however as the Labview is slow my supervisor has told me to write a program in c++ to get the data from the frame grabber. I have no idea how to write a c++ program to connect to a frame grabber and do the data acquisition. I know how to write software in c++, but have never tried programming to connect to hardware and read data from it. Is there any specific library or framework which can help me, or any tutorial?
Please, if anybody knows, help me in this matter.
Update:just to add, we are doing medical image analysis, and a laser illuminate a subject, so camera will take pictures and pass it to the computer. I need to grab the pictures and analysis them.
You basically have a couple of options,
1 see if there is an SDK for the grabber card, if there is this is usually easier then option 2 but is of course restricted to work with that grabber or familly of grabber cards, we do it this way with the eurysys grabber cards.
2 assuming you are running on a windows platform, implement a DirectShow filtergraph and write your own ouput filter to get the data, the SDK for DirectShow is quiet good and has many examples. This approach is far more flexible and you should be able to use a number of grabber but its also alot more complex, we do it this way for USB / some other inbuilt grabbers.
Our software is done in Delphi 7 but its just importing DLLs, for C++ should be no problem and most SDK's are written round C++ anyway.
I know its not much but its a place to start.
Update
Just done a quick Google search and there is an SDK for that Grabber and on first looks its seams fairly straight forward.
I'm trying to stream a video from a camera ( Sony HVR-Z1E ) over FireWire to my Computer. The incoming pictures/stream shall be processed further by some functions which expect the CVMat format ( from openCV ).
Well my problem is now that I have no idea how to grab the stream. Okay openCV 2.1 offers me some methods ( cvCapturefromCam ) , but no matter which parameter I give him, it always gets the stream from the webcam of the laptop and not from the firewire. I heard I need to switch the primary cam in the DirectShow API ( with the Windows SDK ). But I actually don't know how to do that either.
So any suggestions how to do this?
See my related answer here. OpenCV cannot capture video from Firewire cameras natively. You will either need to use the CMU1394 driver, or the Sony driver (if an SDK is available for it) to capture video from that camera, and then pass it to OpenCV.
Years ago, I've made something like this using DirectShow. The main limitation was fact, that image acquired via DShow was in standard PAL resolution. HD Image grabbing was not possible (it was one of the first pro-consumer HD camcorders from Sony, don't remember exact model now). Good thing was - this method didn't need anything except bare DirectShow - no additional drivers and so on. And it was VERY fast.
In general, method was something like this:
building media render graph (of course, You have to enumerate video devices in that stage)
inserting into it custom class which inherited from ISampleGrabberCB.
How it worked:
it used BufferCB() virtual method from ISampleGrabberCB - which you have to write in your inherited class.
in mentioned method, you have to leave data in global struct, and from main thread - take care of them.
I know, that's a bit fuzzy description, but I hope You'll find Your info (googling for "ISampleGrabberCB" should be a good starting point, there should be a lot of sample code).