I'm trying to stream a video from a camera ( Sony HVR-Z1E ) over FireWire to my Computer. The incoming pictures/stream shall be processed further by some functions which expect the CVMat format ( from openCV ).
Well my problem is now that I have no idea how to grab the stream. Okay openCV 2.1 offers me some methods ( cvCapturefromCam ) , but no matter which parameter I give him, it always gets the stream from the webcam of the laptop and not from the firewire. I heard I need to switch the primary cam in the DirectShow API ( with the Windows SDK ). But I actually don't know how to do that either.
So any suggestions how to do this?
See my related answer here. OpenCV cannot capture video from Firewire cameras natively. You will either need to use the CMU1394 driver, or the Sony driver (if an SDK is available for it) to capture video from that camera, and then pass it to OpenCV.
Years ago, I've made something like this using DirectShow. The main limitation was fact, that image acquired via DShow was in standard PAL resolution. HD Image grabbing was not possible (it was one of the first pro-consumer HD camcorders from Sony, don't remember exact model now). Good thing was - this method didn't need anything except bare DirectShow - no additional drivers and so on. And it was VERY fast.
In general, method was something like this:
building media render graph (of course, You have to enumerate video devices in that stage)
inserting into it custom class which inherited from ISampleGrabberCB.
How it worked:
it used BufferCB() virtual method from ISampleGrabberCB - which you have to write in your inherited class.
in mentioned method, you have to leave data in global struct, and from main thread - take care of them.
I know, that's a bit fuzzy description, but I hope You'll find Your info (googling for "ISampleGrabberCB" should be a good starting point, there should be a lot of sample code).
Related
I want to make a virtual web cam application. so I find out all pages that had written before 2010.
But there was no result. All the projects are old even I open and some parts of methods of codeproject's sources is doesn't useful now.
So I'm looking for a replacement of DirectShow. I tried following directshow guide even the getting started guide in docs.ms didn't work.
What should I do?...
So this is an AB question: you need one thing and instead you ask for another.
Yes, there is replacement for DirectShow. DirectShow is a previous multimedia API in Windows, and its successor API and current API in Windows is Media Foundation. This will however get you not an inch closer to the solution of your original problem.
The concept of virtual camera is abstract: Windows operating system does not offer a solid extensibility point to add virtual video input devices such that they are visible to application equally to real cameras. To achieve this you are basically supposed to develop and fully featured driver which is not what you want to do.
I covered details in multiple questions here on StackOverflow, I will just mention a few to start reading from:
64 bit Vivek's Virtual Camera
Registering a network video stream as a virtual camera
Virtual webcam input as byte stream
If you want some application designed to work with cameras to "see" your virtual camera as if it was a real camera, you are interested in finding out what API the application is using and if you could submit your virtual implementation as another camera option.
If the application is DirectShow based, you are rather lucky and you can take the path of virtual DirectShow camera (see links above). If the application uses Media Foundation, you are pretty much in a dead end and hooking or API detouring is probably your best option.
Or you have an option to develop a camera driver, which is most likely is not realistic due to required effort (even though such implementations do exist).
Long story short if your target application is consuming cameras via DirectShow API, you are lucky enough and you don't need DirectShow replacement. Virtual camera code and knowledge accumulated over last 20 years is still in good standing.
I am developing an C++ application which should use an USB camera to capture high resolution photos. It should have same behavior as the Camera application in Windows 10. I am trying to use DirectShow for doing it. Now I am only able to take high resolution photo which is delayed or take a photo in time but low resolution. Also I am very confused from MS documentation, lot of things are deprecated and nowhere mentioned what replaces them. I'll describe my hopeless steps awaiting there will be somebody who could be able to show me a way.
Let's start from beginning...
Knowing nothing about video capturing in Window I started by searching suitable library. After some googling I found there are four main libraries for capturing video in Windows.
Video for Windows
DirectShow
Windows Media Foundation
OpenCV
Let's observe:
Video for Windows
This library is unfortunately marked as deprecated but it seems it still works. I have written "unfortunately", because I think this is the only which is easy to use. There are only a few lines of code needed for seeing video from camera. The only think I miss here is a "TakePhoto" function. You can use VFW for capture a video or single frames to an avi file. Or am I missing something?
DirectShow
This is much more complicated library. You need hundreds of lines of code to see a video preview. But you can obtain this code on MS Docs. Ok, now I have a video preview and I need only to take a photo. One would expect this should be just one function call. But where is the function? I did not find it.
You can simply use GetCurrentImage from IVMRWindowlessControl but this takes only one frame from preview with low resolution. If you set a higher resolution for preview the video is not fluent.
Best approach I could achieve is from an article called "Capturing an Image From a Still Image Pin" available here https://learn.microsoft.com/en-us/windows/desktop/directshow/capturing-an-image-from-a-still-image-pin. When I had found this site I thought I won and my task was almost finished. But it wasn't.
The first advice which the article gives you is not to use it: "The recommended way to get still images from the device is to use the Windows Image Acquisition (WIA) APIs. For more information, see "Windows Image Acquisition" in the Platform SDK documentation. However, you can also use DirectShow to capture an image." I tried to explore the WIA. But this stopped to work on Vista. I continued to study the article.
Everything seems to be clear but you need to implement your class which inherits ISampleGrabberCB marked as deprecated here https://learn.microsoft.com/en-us/windows/desktop/directshow/isamplegrabbercb. Why???? Where to find some alternative?
I found an acceptable solution here https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/2ab5c212-5824-419d-b5d9-7f5db82f57cd/qedith-missing-in-current-windows-sdk-v70?forum=windowsdirectshowdevelopment. You need to add header file from elder SDK. (BTW This is an advice almost ten years old.) After I compiled the application with this header I was able to read high resolution picture but I need to wait a few seconds which is unacceptable. I know the problem is not in camera, because in the it works in the Camera application. Furthermore the image is obtained in function SampleCB instead of BufferCB and is in some strange format. I can save it as jpg but it is not compressed enough.
Windows Media Foundation
I think MS doesn't like programmers and that's why it released WMF. I understand nothing. I found this tutorial https://www.dreamincode.net/forums/topic/347938-a-new-webcam-api-tutorial-in-c-for-windows/. It works but it only stores one frame from preview and this is not what I want.
Next I explored some WMF interfaces on MS Docs. IMFCapturePhotoSink interface should do the stuff. But how implement it. The documentation is useless.
OpenCV
During my research I found also this library. But again I'm not able to take a high resolution photo. It only stores one frame from preview.
Could someone tell me what should I focus on? I believe it cannot be so difficult. There are tens and hundreds of applications for webcams. How could other programmers implement them? What's wrong with me? I'd like to find an easy way to implement an easy task. Thank a lot for any help.
You question is not related to the topic - the question must be related to the code - but I faced with the similar problem many years ago and I had found solution:
DirectShow is declared as deprecated for Windows 10 and it has problem with supporting of the USB web cam. In Windows 10 there is USB Video Class which is supported only by Media Foundation.
So, I have wrote a simple C++ wrapper around Media Foundation code which simplify getting of the raw images Capturing Video from Web-camera on Windows 7 and 8 by using Media Foundation
Also, there is project CaptureManager SDK - it is DLL COM component with the simple interfaces, huge functionality and with many demo programs on C++, Python, C#, Java.
Thanks to Evgeny.
Recapitulation:
Download the CaptureEngine video capture sample
Edit CaptureManager::TakePhoto method. Add the code to find highest resolution media type just before CreatePhotoMediaType(pMediaType, &pMediaType2); line
Extra code for setup the photo stream to highest resolution:
DWORD dwMediaTypeIndex = 0;
UINT32 maxSize = 0;
DWORD maxSizeIndex = 0;
while (1) {
IMFMediaType* pMediaType = NULL;
hr = pSource->GetAvailableDeviceMediaType((DWORD)MF_CAPTURE_ENGINE_PREFERRED_SOURCE_STREAM_FOR_PHOTO, dwMediaTypeIndex, &pMediaType);
if (hr == MF_E_NO_MORE_TYPES)
break;
UINT32 w, h;
MFGetAttributeSize(pMediaType, MF_MT_FRAME_SIZE, &w, &h);
UINT32 size = w * h;
if (size > maxSize) {
maxSize = size;
maxSizeIndex = dwMediaTypeIndex;
}
SafeRelease(&pMediaType);
dwMediaTypeIndex++;
}
SafeRelease(&pMediaType);
pSource->GetAvailableDeviceMediaType((DWORD)MF_CAPTURE_ENGINE_PREFERRED_SOURCE_STREAM_FOR_PHOTO, maxSizeIndex, &pMediaType);
I have a directshow Video renderer redived from CBaseVideoRenderer. The renderer is used in a graph that receives data from a live source (BDA). It looks like the connections are established properly, but the video rendering immediately ends because there is no sample. However, audio Rendering works, ie I can hear the sound while DoRenderSample of my renderer is never called.
Stepping through the code in the debugger, I found out that in CBaseRenderer::StartStreaming, the stream ends immedately, because the member m_pMediaSample is NULL. If I replace my renderer with the EVR renderer, it shows frames, ie the stream is not ending before the first frame for the EVR renderer, but only for my renderer.
Why is that and how can I fix it? I implemented (following the sample from http://www.codeproject.com/Articles/152317/DirectShow-Filters-Development-Part-Video-Render) what I understand as the basic interface (CheckMediaType, SetMediaType and DoRenderSample), so I do not see any possibility to influence what is happening here...
Edit: This is the graph as seen from the ROT:
What I basically try to do is capturing a DVB stream that uses VIDEOINFOHEADER2, which is not supported by the standard sample grabber. Although the channel is a public German TV channel without encryption, could it be that this is a DRM issue?
Edit 2: I have attached my renderer to another source (a Blackmagic Intensity Shuttle). It seams that the source causes the issue, because I get samples in the other graph.
Edit 3: Following Roman's Suggestion, I have created a transform filter. The graph looks like
an has unfortunately the same problem, ie I do not get any sample (Transform is not called).
You supposedly chose wrong path of fetching video frames out of media pipeline. So you are implementing a "network renderer", something that terminates the pipeline to further send data to network.
A renderer which accepts the feed sounds appropriate. Implementing a custom renderer, however, is an untypical task and then there is not so much information around on this. Additionally, a fully featured renderer is typically equipped with sample scheduling part, which end of stream delivery - things relatively easy to break when you customize it through inheriting from base classes. That is, while the approach sounds good, you might want to compare it to another option you have, which is...
A combination of Sample Grabber + Null Renderer, two standard filters, which you can attach your callback to and get frames having the pipeline properly terminated. The problem here is that standard Sample Grabber does not support VIDEOINFOHEADER2. With another video decoder you could possibly have the feed decoded info VIDEOINFOHEADER, which is one option. And then improvement of Sample Grabber itself is another solution: DirectX SDK Extras February 2005 (dxsdk_feb2005_extras.exe) was the SDK which included a filter similar to standard Sample Grabber called Grabber \DirectShow\Samples\C++\DirectShow\Filters\Grabber. It is/was available in source code and provided with a good description text file. It is relatively easy to extend to allow it accept VIDEOINFOHEADER2 and make payload data available to your application this way.
The easiest way to get data out of a DirectShow graph, if youњre not going to use
MultiMedia Streaming, is probably to write your own TransInPlace filter, a sub-variety
of a Transform filter. Then connect this filter to the desired stream of data you wish to
monitor, and then run, pause, seek, or otherwise control the graph. The data, as it passes
through the transform filter, can be manipulated however you want. We call this kind of
filter, a Њsample grabberћ. Microsoft released a limited-functionality sample grabber
with DX8.0. This filter is limited because it doesnњt deal with DV Data or mediatypes
with a format of VideoInfo2. It doesnњt allow the user to receive prerolled samples.
(Whatњs a preroll sample? See the DX8.1 docs) Its ЊOneShotћ mode also has some problems.
To add to this, the Grabber sample is pretty simple itself - perhaps 1000 lines of code all together, including comments.
Looks like your decoder or splitter isn't demuxing the video frames. Look further up the chain to see what filters are supplying your renderer pin with data, chances are its only recognising audio.
Try dropping the file into Graphedit (there's a better one on the web BTW) and see what filters it creates.
Then look at the samples in the DirectShow SDK.
We have an "MC1362 Camera" and an "Inspecta-5" frame grabber in our lab. There is program in LABVIEW11 which gets the data from a frame grabber, however as the Labview is slow my supervisor has told me to write a program in c++ to get the data from the frame grabber. I have no idea how to write a c++ program to connect to a frame grabber and do the data acquisition. I know how to write software in c++, but have never tried programming to connect to hardware and read data from it. Is there any specific library or framework which can help me, or any tutorial?
Please, if anybody knows, help me in this matter.
Update:just to add, we are doing medical image analysis, and a laser illuminate a subject, so camera will take pictures and pass it to the computer. I need to grab the pictures and analysis them.
You basically have a couple of options,
1 see if there is an SDK for the grabber card, if there is this is usually easier then option 2 but is of course restricted to work with that grabber or familly of grabber cards, we do it this way with the eurysys grabber cards.
2 assuming you are running on a windows platform, implement a DirectShow filtergraph and write your own ouput filter to get the data, the SDK for DirectShow is quiet good and has many examples. This approach is far more flexible and you should be able to use a number of grabber but its also alot more complex, we do it this way for USB / some other inbuilt grabbers.
Our software is done in Delphi 7 but its just importing DLLs, for C++ should be no problem and most SDK's are written round C++ anyway.
I know its not much but its a place to start.
Update
Just done a quick Google search and there is an SDK for that Grabber and on first looks its seams fairly straight forward.
I'm creating a Windows video capture application and am using DirectShow for capture. As each frame comes in, I want to grab it as a raw RGB bitmap into a buffer, at which point my code will do whatever processing I need.
I've been searching for samples similar to what I want to do, and everywhere I look online, people recommend using either the IMediaDet and/or the ISampleGrabber interface to do frame-by-frame capture. Unfortunately, both are deprecated and aren't even in the newest version of the Windows SDK.
What is the best (modern) way to do frame-by-frame capture in DirectShow? If there is none, is there a different library I should use that will give me frame-by-frame capture?
Sample Grabber was deprecated a few years ago, which was a few years after DirectShow development actually stopped. That is, use Sample Grabber as you read as suggested method and it is going to work great for you.
The only thing you will additionally need is to copy definitions int your source code, see details:
Alternative for ISampleGrabber
Sample Grabber replacement
ISampleGrabber deprecated: where can I find alternatives?