Media Foundation: Cannot change a FPS on webcam - c++

I try to replace codes with Directshow ("DS") on Media Foundation ("MF") in my app and met one problem - cannot set a needed fps using MF on a webcam. MF allowed me to set only 30 fps. If I try to set 25 fps, I always get the error 0xc00d5212 on SetCurrentMediaType(). In DS I could change that parameter.
My codes:
ASSERT(m_pReader); //IMFSourceReader *m_pReader;
IMFMediaType *pNativeType = NULL;
IMFMediaType *pType = NULL;
UINT32 w = 1280;
UINT32 h = 720;
UINT32 fps = 25; // or 30
DWORD dwStreamIndex = MF_SOURCE_READER_FIRST_VIDEO_STREAM;
// Find the native format of the stream.
HRESULT hr = m_pReader->GetNativeMediaType(dwStreamIndex, 0, &pNativeType);
if (FAILED(hr))
{
//error
}
GUID majorType, subtype;
// Find the major type.
hr = pNativeType->GetGUID(MF_MT_MAJOR_TYPE, &majorType);
if (FAILED(hr))
{
//error
}
// Define the output type.
hr = MFCreateMediaType(&pType);
if (FAILED(hr))
{
//error
}
hr = pType->SetGUID(MF_MT_MAJOR_TYPE, majorType);
if (FAILED(hr))
{
//error
}
// Select a subtype.
if (majorType == MFMediaType_Video)
{
subtype= MFVideoFormat_RGB24;
}
else
{
//error
}
hr = pType->SetGUID(MF_MT_SUBTYPE, subtype);
if (FAILED(hr))
{
//error
}
hr = MFSetAttributeSize(pType, MF_MT_FRAME_SIZE, w, h);
if (FAILED(hr))
{
//error
}
hr = MFSetAttributeSize(pType, MF_MT_FRAME_RATE, fps, 1);
if (FAILED(hr))
{
//error
}
hr = m_pReader->SetCurrentMediaType(dwStreamIndex, NULL, pType);
if (FAILED(hr))
{// hr = 0xc00d5212
//!!!!!error - if fps == 25
}
return hr;
Thanks for any help.

It might so happen that the camera does not support flexible frame rate values, and can work with only among the supported set, for example: 10, 15, 20, 24, 30 fps. You should be able to enumerate supported media types and choose the one that works for you - those media types typically include frame rate options.
Even though Media Foundation and DirectShow video capture eventually ends up in the same backend, there might be discrepancies in behavior. Specifically, you are working with Media Foundation higher level API that internally interfaces to a media source, and it might so happens that frame rate leads to 0xC00D5212 MF_E_TOPO_CODEC_NOT_FOUND "No suitable transform was found to encode or decode the content" confusion even though technically the driver can capture in respective mode.
See also:
Get all supported FPS values of a camera in Microsoft Media Foundation
Media Foundation Video/Audio Capture Capabilities

I've added the timer for fps control imitation into the codes. So at the start I set 30 fps , then by fps scale I skip some frames for my app.
Thank you for help.

Related

Playback speed of video encoded with IMFSinkWriter changes based on width

I'm making a screen recorder (without audio) using Win32s Sink Writer to encode a series of bitmaps into an MP4 file.
For some reason, the video playback speed increases (seemingly) proportionally with the video width.
From this post, I've gathered that it's most likely because I'm calculating the buffer size incorrectly. The difference here is that their video playback issue was fixed once the calculation for the audio buffer size was correct, but since I don't encode any audio at all, I'm not sure what to take from it.
I've also tried to read about how the buffer works, but I'm really at a loss as to exactly how the buffer size is causing different playback speeds.
Here is a pastebin for the entirity of the code, I really can't track down the problem any more than the buffer size and/or the frame index/duration.
i.e.:
Depending on the width of the member variable m_width (measured in pixels), the playback speed changes. That is; the higher the width, the faster the video plays, and vice versa.
Here are two video examples:
3840x1080 and 640x1080, notice the system clock.
Imugr does not retain the original resolution of the files, but I double checked before uploading, and the program does indeed create files of the claimed resolutions.
rtStart and rtDuration are defined as such, and are both private members of the MP4File class.
LONGLONG rtStart = 0;
UINT64 rtDuration;
MFFrameRateToAverageTimePerFrame(m_FPS, 1, &rtDuration);
This is where rtStart is updated, and the individual bits of the bitmap is passed to the frame writer.
Moved the LPVOID object to private members to hopefully increase performance. Now there's no need for heap allocation every time a frame is appended.
HRESULT MP4File::AppendFrame(HBITMAP frame)
{
HRESULT hr = NULL;
if (m_isInitialFrame)
{
hr = InitializeMovieCreation();
if (FAILED(hr))
return hr;
m_isInitialFrame = false;
}
if (m_hHeap && m_lpBitsBuffer) // Makes sure buffer is initialized
{
BITMAPINFO bmpInfo;
bmpInfo.bmiHeader.biBitCount = 0;
bmpInfo.bmiHeader.biSize = sizeof(BITMAPINFOHEADER);
// Get individual bits from bitmap and loads it into the buffer used by `WriteFrame`
GetDIBits(m_hDC, frame, 0, 0, NULL, &bmpInfo, DIB_RGB_COLORS);
bmpInfo.bmiHeader.biCompression = BI_RGB;
GetDIBits(m_hDC, frame, 0, bmpInfo.bmiHeader.biHeight, m_lpBitsBuffer, &bmpInfo, DIB_RGB_COLORS);
hr = WriteFrame();
if (SUCCEEDED(hr))
{
rtStart += rtDuration;
}
}
return m_writeFrameResult = hr;
}
And lastly, the frame writer which actually loads the bits into the buffer, and then writes to the Sink Writer.
HRESULT MP4File::WriteFrame()
{
IMFSample *pSample = NULL;
IMFMediaBuffer *pBuffer = NULL;
const LONG cbWidth = 4 * m_width;
const DWORD cbBufferSize = cbWidth * m_height;
BYTE *pData = NULL;
// Create a new memory buffer.
HRESULT hr = MFCreateMemoryBuffer(cbBufferSize, &pBuffer);
// Lock the buffer and copy the video frame to the buffer.
if (SUCCEEDED(hr))
{
hr = pBuffer->Lock(&pData, NULL, NULL);
}
if (SUCCEEDED(hr))
{
hr = MFCopyImage(
pData, // Destination buffer.
cbWidth, // Destination stride.
(BYTE*)m_lpBitsBuffer, // First row in source image.
cbWidth, // Source stride.
cbWidth, // Image width in bytes.
m_height // Image height in pixels.
);
}
if (pBuffer)
{
pBuffer->Unlock();
}
// Set the data length of the buffer.
if (SUCCEEDED(hr))
{
hr = pBuffer->SetCurrentLength(cbBufferSize);
}
// Create a media sample and add the buffer to the sample.
if (SUCCEEDED(hr))
{
hr = MFCreateSample(&pSample);
}
if (SUCCEEDED(hr))
{
hr = pSample->AddBuffer(pBuffer);
}
// Set the time stamp and the duration.
if (SUCCEEDED(hr))
{
hr = pSample->SetSampleTime(rtStart);
}
if (SUCCEEDED(hr))
{
hr = pSample->SetSampleDuration(rtDuration);
}
// Send the sample to the Sink Writer and update the timestamp
if (SUCCEEDED(hr))
{
hr = m_pSinkWriter->WriteSample(m_streamIndex, pSample);
}
SafeRelease(&pSample);
SafeRelease(&pBuffer);
return hr;
}
A couple details about the encoding:
Framerate: 30FPS
Bitrate: 15 000 000
Output encoding format: H264 (MP4)
To me, this behavior makes sense.
See https://github.com/mofo7777/Stackoverflow/tree/master/ScreenCaptureEncode
My program uses DirectX9 instead of GetDIBits, but the behaviour is the same. Try this program with different screen resolutions, to confirm this behaviour.
And I confirm, with my program, the video playback speed increases proportionally with the video width (and also with the video height).
Why ?
More data to copy, more time to pass. And wrong sample time/sample duration.
Using 30 FPS, means one frame each 33.3333333 ms :
Do GetDIBits, MFCopyImage, WriteSample end exactly at 33.3333333 ms... no.
Do you write each frame exactly at 33.3333333 ms... no.
So just doing rtStart += rtDuration is wrong, because you don't capture and write screen exactly at this time. And GetDIBits/DirectX9 are not able to process at 30 FPS, trust me. And why Microsoft provided Windows Desktop Duplication (for windows 8/10 only) ?
The key is latency.
Do you know how long GetDIBits, MFCopyImage and WriteSample take ? You should know, to understand the problem. Usually, it takes more than 33.3333333 ms. But it is variable.
You must know it to adjust the correct FPS to the encoder. But you also will need to WriteSample at the right time.
If you use MF_MT_FRAME_RATE with 5-10 FPS instead of 30 FPS, You will see it is more realistic, but not optimal.
For example, use an IMFPresentationClock to handle the correct WriteSample time.

Convert IMFSample* to ID3D11ShaderResourceView*

I am new to DirectX an I am trying to do a simple application that reads a video and display it on a Quad.
I read the video using Windows Media Foundation (IMFSourceReader), that sends me a callback when a sample is decoded (IMFSample).
I want to convert this IMFSample* to a ID3D11ShaderResourceView* in order to use it as a texture to draw my quad, however the conversion fails.
Here is what I do (I removed non relevant error checks):
HRESULT SourceReaderCB::OnReadSample(HRESULT hrStatus, DWORD dwStreamIndex, DWORD dwStreamFlags, LONGLONG llTimestamp, IMFSample *pSample)
{
...
DWORD NumBuffers = 0;
hr = pSample->GetBufferCount(&NumBuffers);
if (FAILED(hr) || NumBuffers < 1)
{
...
}
IMFMediaBuffer* SourceMediaPtr = nullptr;
hr = pSample->GetBufferByIndex(0, &SourceMediaPtr);
if (FAILED(hr))
{
...
}
ComPtr<IMFMediaBuffer> _pInputBuffer = SourceMediaPtr;
ComPtr<IMF2DBuffer2> _pInputBuffer2D2;
bool isVideoFrame = (_pInputBuffer.As(&_pInputBuffer2D2) == S_OK);
if (isVideoFrame)
{
IMFDXGIBuffer* pDXGIBuffer = NULL;
ID3D11Texture2D* pSurface = NULL;
hr = _pInputBuffer->QueryInterface(__uuidof(IMFDXGIBuffer), (LPVOID*)&pDXGIBuffer);
if (FAILED(hr))
{
SafeRelease(&SourceMediaPtr);
goto done;
}
hr = pDXGIBuffer->GetResource(__uuidof(ID3D11Texture2D), (LPVOID*)&pSurface);
if (FAILED(hr))
{
...
}
ID3D11ShaderResourceView* resourceView;
if (pSurface)
{
D3D11_TEXTURE2D_DESC textureDesc;
pSurface->GetDesc(&textureDesc);
D3D11_SHADER_RESOURCE_VIEW_DESC shaderResourceViewDesc;
shaderResourceViewDesc.Format = DXGI_FORMAT_R8_UNORM;
shaderResourceViewDesc.ViewDimension = D3D11_SRV_DIMENSION_TEXTURE2D;
shaderResourceViewDesc.Texture2D.MostDetailedMip = 0;
shaderResourceViewDesc.Texture2D.MipLevels = 1;
ID3D11ShaderResourceView* resourceView;
hr = d3d11device->CreateShaderResourceView(pSurface, &shaderResourceViewDesc, &resourceView);
if (FAILED(hr))
{
... // CODE FAILS HERE
}
...
}
}
}
My first issue is that I set the shaderResourceViewDesc.Format as DXGI_FORMAT_R8_UNORM which will probably just give me red image (I will have to investigate this later).
The second and blocking issue I am facing ius that the conversion of ID3D11Texture2D to ID3D11ShaderResourceView fails with following error message:
ID3D11Device::CreateShaderResourceView: A ShaderResourceView cannot be created of a Resource that did not specify the D3D11_BIND_SHADER_RESOURCE BindFlag. [ STATE_CREATION ERROR #129: CREATESHADERRESOURCEVIEW_INVALIDRESOURCE]
I understand that there is a flag missing at the creation of the texture that prevents me to do what I want to do, but as the data buffer is created by WMF, I am not sure what I am supposed to do to fix this issue.
Thanks for your help
I see you code, and I can say that your way is wrong - no offense. Firstly, video decoder creates simple texture - in you situation DirectX11 texture - it is a regular texture - it is not shader resource, as a result it cannot be used in shader code. In my view, there are two way for resolving of your task:
Research - Walkthrough: Using MF to render video in a Direct3D app - this link present way for "Walkthrough: Using Microsoft Media Foundation for Windows Phone 8" - from your code I see that you try write solution for WindowsStore - UWP and code for Windows Phone is workable - this code needs MediaEnginePlayer - The MediaEnginePlayer class serves as a helper class that wraps the MF APIs;
Find on GitHub Windows-classic-samples and find in that DX11VideoRenderer - this is full code of Media Foundation renderer with DirectX11 - it includes very good example for using of DirectX11 Video Processor which does blitting of regular video texture from decoder into the rendering video texture of swap-chain:
2.1. Get rendering texture from Swap Chain:
// Get Backbuffer
hr = m_pSwapChain1->GetBuffer(0, __uuidof(ID3D11Texture2D), (void**)&pDXGIBackBuffer);
if (FAILED(hr))
{
break;
}
2.2. Create from rendering texture output view of video processor:
//
// Create Output View of Output Surfaces.
//
D3D11_VIDEO_PROCESSOR_OUTPUT_VIEW_DESC OutputViewDesc;
ZeroMemory( &OutputViewDesc, sizeof( OutputViewDesc ) );
if (m_b3DVideo && m_bStereoEnabled)
{
OutputViewDesc.ViewDimension = D3D11_VPOV_DIMENSION_TEXTURE2DARRAY;
}
else
{
OutputViewDesc.ViewDimension = D3D11_VPOV_DIMENSION_TEXTURE2D;
}
OutputViewDesc.Texture2D.MipSlice = 0;
OutputViewDesc.Texture2DArray.MipSlice = 0;
OutputViewDesc.Texture2DArray.FirstArraySlice = 0;
if (m_b3DVideo && 0 != m_vp3DOutput)
{
OutputViewDesc.Texture2DArray.ArraySize = 2; // STEREO
}
QueryPerformanceCounter(&lpcStart);
hr = m_pDX11VideoDevice->CreateVideoProcessorOutputView(pDXGIBackBuffer, m_pVideoProcessorEnum, &OutputViewDesc, &pOutputView);
2.3. Create from regular decoder video texture input view for video processor:
D3D11_VIDEO_PROCESSOR_INPUT_VIEW_DESC InputLeftViewDesc;
ZeroMemory( &InputLeftViewDesc, sizeof( InputLeftViewDesc ) );
InputLeftViewDesc.FourCC = 0;
InputLeftViewDesc.ViewDimension = D3D11_VPIV_DIMENSION_TEXTURE2D;
InputLeftViewDesc.Texture2D.MipSlice = 0;
InputLeftViewDesc.Texture2D.ArraySlice = dwLeftViewIndex;
hr = m_pDX11VideoDevice->CreateVideoProcessorInputView(pLeftTexture2D, m_pVideoProcessorEnum, &InputLeftViewDesc, &pLeftInputView);
if (FAILED(hr))
{
break;
}
2.4. Do blitting of regular decoder video texture on rendering texture from Swap Chain:
D3D11_VIDEO_PROCESSOR_STREAM StreamData;
ZeroMemory( &StreamData, sizeof( StreamData ) );
StreamData.Enable = TRUE;
StreamData.OutputIndex = 0;
StreamData.InputFrameOrField = 0;
StreamData.PastFrames = 0;
StreamData.FutureFrames = 0;
StreamData.ppPastSurfaces = NULL;
StreamData.ppFutureSurfaces = NULL;
StreamData.pInputSurface = pLeftInputView;
StreamData.ppPastSurfacesRight = NULL;
StreamData.ppFutureSurfacesRight = NULL;
if (m_b3DVideo && MFVideo3DSampleFormat_MultiView == m_vp3DOutput && pRightTexture2D)
{
StreamData.pInputSurfaceRight = pRightInputView;
}
hr = pVideoContext->VideoProcessorBlt(m_pVideoProcessor, pOutputView, 0, 1, &StreamData );
if (FAILED(hr))
{
break;
}
Yes, they are sections of complex code, and it needs research whole DX11VideoRenderer project for understanding of it - it will take huge amount of time.
Regards,
Evgeny Pereguda
Debug output suggests that the texture is not compatible, as it was created without D3D11_BIND_SHADER_RESOURCE flag (specified in BindFlag field of D3D11_TEXTURE2D_DESC structure.
You read the texture already created by Media Foundation primitive. In some cases you can alter the creation flags, however the general case is that you need to create a compatible texture on your own, copy the data between the textures, and then call CreateShaderResourceView method with your texture as an argument rather than original texture.

DirectShow: Video-Preview and Image (with working code)

Questions / Issues
If someone can recommend me a good free hosting site I can provide the whole project file.
As mentioned in the text below the TakePicture() method is not working properly on the HTC HD 2 device. It would be nice if someone could look at the code below and tell me if it is right or wrong what I'm doing.
Introduction
I recently asked a question about displaying a video preview, taking camera image and rotating a video stream with DirectShow. The tricky thing about the topic is, that it's very hard to find good examples and the documentation and the framework itself is very hard to understand for someone who is new to windows programming and C++ in general.
Nevertheless I managed to create a class that implements most of this features and probably works with most mobile devices. Probably because the DirectShow implementation depends a lot on the device itself. I could only test it with the HTC HD and HTC HD2, which are known as quite incompatible.
HTC HD
Working: Video preview, writing photo to file
Not working: Set video resolution (CRASH), set photo resolution (LOW quality)
HTC HD 2
Working: Set video resolution, set photo resolution
Problematic: Video Preview rotated
Not working: Writing photo to file
To make it easier for others by providing a working example, I decided to share everything I have got so far below. I removed all of the error handling for the sake of simplicity. As far as documentation goes, I can recommend you to read the MSDN documentation, after that the code below is pretty straight forward.
void Camera::Init()
{
CreateComObjects();
_captureGraphBuilder->SetFiltergraph(_filterGraph);
InitializeVideoFilter();
InitializeStillImageFilter();
}
Dipslay a video preview (working with any tested handheld):
void Camera::DisplayVideoPreview(HWND windowHandle)
{
IVideoWindow *_vidWin;
_filterGraph->QueryInterface(IID_IMediaControl,(void **) &_mediaControl);
_filterGraph->QueryInterface(IID_IVideoWindow, (void **) &_vidWin);
_videoCaptureFilter->QueryInterface(IID_IAMVideoControl,
(void**) &_videoControl);
_captureGraphBuilder->RenderStream(&PIN_CATEGORY_PREVIEW,
&MEDIATYPE_Video, _videoCaptureFilter, NULL, NULL);
CRect rect;
long width, height;
GetClientRect(windowHandle, &rect);
_vidWin->put_Owner((OAHWND)windowHandle);
_vidWin->put_WindowStyle(WS_CHILD | WS_CLIPSIBLINGS);
_vidWin->get_Width(&width);
_vidWin->get_Height(&height);
height = rect.Height();
_vidWin->put_Height(height);
_vidWin->put_Width(rect.Width());
_vidWin->SetWindowPosition(0,0, rect.Width(), height);
_mediaControl->Run();
}
HTC HD2: If set SetPhotoResolution() is called FindPin will return E_FAIL. If not, it will create a file full of null bytes. HTC HD: Works
void Camera::TakePicture(WCHAR *fileName)
{
CComPtr<IFileSinkFilter> fileSink;
CComPtr<IPin> stillPin;
CComPtr<IUnknown> unknownCaptureFilter;
CComPtr<IAMVideoControl> videoControl;
_imageSinkFilter.QueryInterface(&fileSink);
fileSink->SetFileName(fileName, NULL);
_videoCaptureFilter.QueryInterface(&unknownCaptureFilter);
_captureGraphBuilder->FindPin(unknownCaptureFilter, PINDIR_OUTPUT,
&PIN_CATEGORY_STILL, &MEDIATYPE_Video, FALSE, 0, &stillPin);
_videoCaptureFilter.QueryInterface(&videoControl);
videoControl->SetMode(stillPin, VideoControlFlag_Trigger);
}
Set resolution: Works great on HTC HD2. HTC HD won't allow SetVideoResolution() and only offers one low resolution photo resolution:
void Camera::SetVideoResolution(int width, int height)
{
SetResolution(true, width, height);
}
void Camera::SetPhotoResolution(int width, int height)
{
SetResolution(false, width, height);
}
void Camera::SetResolution(bool video, int width, int height)
{
IAMStreamConfig *config;
config = NULL;
if (video)
{
_captureGraphBuilder->FindInterface(&PIN_CATEGORY_PREVIEW,
&MEDIATYPE_Video, _videoCaptureFilter, IID_IAMStreamConfig,
(void**) &config);
}
else
{
_captureGraphBuilder->FindInterface(&PIN_CATEGORY_STILL,
&MEDIATYPE_Video, _videoCaptureFilter, IID_IAMStreamConfig,
(void**) &config);
}
int resolutions, size;
VIDEO_STREAM_CONFIG_CAPS caps;
config->GetNumberOfCapabilities(&resolutions, &size);
for (int i = 0; i < resolutions; i++)
{
AM_MEDIA_TYPE *mediaType;
if (config->GetStreamCaps(i, &mediaType,
reinterpret_cast<BYTE*>(&caps)) == S_OK )
{
int maxWidth = caps.MaxOutputSize.cx;
int maxHeigth = caps.MaxOutputSize.cy;
if(maxWidth == width && maxHeigth == height)
{
VIDEOINFOHEADER *info =
reinterpret_cast<VIDEOINFOHEADER*>(mediaType->pbFormat);
info->bmiHeader.biWidth = maxWidth;
info->bmiHeader.biHeight = maxHeigth;
info->bmiHeader.biSizeImage = DIBSIZE(info->bmiHeader);
config->SetFormat(mediaType);
DeleteMediaType(mediaType);
break;
}
DeleteMediaType(mediaType);
}
}
}
Other methods used to build the filter graph and create the COM objects:
void Camera::CreateComObjects()
{
CoInitialize(NULL);
CoCreateInstance(CLSID_CaptureGraphBuilder, NULL, CLSCTX_INPROC_SERVER,
IID_ICaptureGraphBuilder2, (void **) &_captureGraphBuilder);
CoCreateInstance(CLSID_FilterGraph, NULL, CLSCTX_INPROC_SERVER,
IID_IGraphBuilder, (void **) &_filterGraph);
CoCreateInstance(CLSID_VideoCapture, NULL, CLSCTX_INPROC,
IID_IBaseFilter, (void**) &_videoCaptureFilter);
CoCreateInstance(CLSID_IMGSinkFilter, NULL, CLSCTX_INPROC,
IID_IBaseFilter, (void**) &_imageSinkFilter);
}
void Camera::InitializeVideoFilter()
{
_videoCaptureFilter->QueryInterface(&_propertyBag);
wchar_t deviceName[MAX_PATH] = L"\0";
GetDeviceName(deviceName);
CComVariant comName = deviceName;
CPropertyBag propertyBag;
propertyBag.Write(L"VCapName", &comName);
_propertyBag->Load(&propertyBag, NULL);
_filterGraph->AddFilter(_videoCaptureFilter,
L"Video Capture Filter Source");
}
void Camera::InitializeStillImageFilter()
{
_filterGraph->AddFilter(_imageSinkFilter, L"Still image filter");
_captureGraphBuilder->RenderStream(&PIN_CATEGORY_STILL,
&MEDIATYPE_Video, _videoCaptureFilter, NULL, _imageSinkFilter);
}
void Camera::GetDeviceName(WCHAR *deviceName)
{
HRESULT hr = S_OK;
HANDLE handle = NULL;
DEVMGR_DEVICE_INFORMATION di;
GUID guidCamera = { 0xCB998A05, 0x122C, 0x4166, 0x84, 0x6A, 0x93, 0x3E,
0x4D, 0x7E, 0x3C, 0x86 };
di.dwSize = sizeof(di);
handle = FindFirstDevice(DeviceSearchByGuid, &guidCamera, &di);
StringCchCopy(deviceName, MAX_PATH, di.szLegacyName);
}
Full header file:
#ifndef __CAMERA_H__
#define __CAMERA_H__
class Camera
{
public:
void Init();
void DisplayVideoPreview(HWND windowHandle);
void TakePicture(WCHAR *fileName);
void SetVideoResolution(int width, int height);
void SetPhotoResolution(int width, int height);
private:
CComPtr<ICaptureGraphBuilder2> _captureGraphBuilder;
CComPtr<IGraphBuilder> _filterGraph;
CComPtr<IBaseFilter> _videoCaptureFilter;
CComPtr<IPersistPropertyBag> _propertyBag;
CComPtr<IMediaControl> _mediaControl;
CComPtr<IAMVideoControl> _videoControl;
CComPtr<IBaseFilter> _imageSinkFilter;
void GetDeviceName(WCHAR *deviceName);
void InitializeVideoFilter();
void InitializeStillImageFilter();
void CreateComObjects();
void SetResolution(bool video, int width, int height);
};
#endif
Unfortunately I can't share the solution here, because of legal reasons.
Nevertheless I can tell you that video and image capturing with full resolution support is possible on the HTC HD 2 without using the HTC HD specific libraries.
Hint: You will probably need a NULL renderer.
I recently ran into a problem using an approach like this, where the snapshot would work the first time around, but failed the second. The problem was similar to yours, where setting the resolution would cause the FindPin to fail, but it only did it the second time.
The fix was to release the config object at the end of SetResolution!
config->Release();
After that, it worked every time.

IMovieControl::Run fails on Windows XP?

Actually, it only fails the second time it's called. I'm using a windowless control to play video content, where the video being played could change while the control is still on screen. Once the graph is built the first time, we switch media by stopping playback, replacing the SOURCE filter, and running the graph again. This works fine under Vista, but when running on XP, the second call to Run() returns E_UNEXPECTED.
The initialization goes something like this:
// Get the interface for DirectShow's GraphBuilder
mGB.CoCreateInstance(CLSID_FilterGraph, NULL, CLSCTX_INPROC_SERVER);
// Create the Video Mixing Renderer and add it to the graph
ATL::CComPtr<IBaseFilter> pVmr;
pVmr.CoCreateInstance(CLSID_VideoMixingRenderer9, NULL, CLSCTX_INPROC);
mGB->AddFilter(pVmr, L"Video Mixing Renderer 9");
// Set the rendering mode and number of streams
ATL::CComPtr<IVMRFilterConfig9> pConfig;
pVmr->QueryInterface(IID_IVMRFilterConfig9, (void**)&pConfig);
pConfig->SetRenderingMode(VMR9Mode_Windowless);
pVmr->QueryInterface(IID_IVMRWindowlessControl9, (void**)&mWC);
And here's what we do when we decide to play a movie. RenderFileToVideoRenderer is borrowed from dshowutil.h in the DirectShow samples area.
// Release the source filter, if it exists, so we can replace it.
IBaseFilter *pSource = NULL;
if (SUCCEEDED(mpGB->FindFilterByName(L"SOURCE", &pSource)) && pSource)
{
mpGB->RemoveFilter(pSource);
pSource->Release();
pSource = NULL;
}
// Render the file.
hr = RenderFileToVideoRenderer(mpGB, mPlayPath.c_str(), FALSE);
// QueryInterface for DirectShow interfaces
hr = mpGB->QueryInterface(&mMC);
hr = mpGB->QueryInterface(&mME);
hr = mpGB->QueryInterface(&mMS);
// Read the default video size
hr = mpWC->GetNativeVideoSize(&lWidth, &lHeight, NULL, NULL);
if (hr != E_NOINTERFACE)
{
if (FAILED(hr))
{
return hr;
}
// Play video at native resolution, anchored at top-left corner.
RECT r;
r.left = 0;
r.top = 0;
r.right = lWidth;
r.bottom = lHeight;
hr = mpWC->SetVideoPosition(NULL, &r);
}
// Run the graph to play the media file
if (mMC)
{
hr = mMC->Run();
if (FAILED(hr))
{
// We get here the second time this code is executed.
return hr;
}
mState = Running;
}
if (mME)
{
mME->SetNotifyWindow((OAHWND)m_hWnd, WM_GRAPHNOTIFY, 0);
}
Anybody know what's going on here?
Try calling IMediaControl::StopWhenReady before removing the source filter.
When are you calling QueryInterface directly? you can use CComQIPtr<> to warp the QI for you. This way you won't have to call Release as it will be called automatically.
The syntax look like this: CComPtr<IMediaControl> mediaControl = pGraph;
In FindFilterByName() instead of passing live pointer pass a CComPtr, again so you won't have to call release explicitly.
Never got a resolution on this. The production solution was to just call IGraphBuilder::Release and rebuild the entire graph from scratch. There's a CPU spike and a slight redraw delay when switching videos, but it's less pronounced than we'd feared.

DirectDraw question - running the application as a regular Windows application

I am developing an application for video recording and I want to overlay the video preview with a logo and recording timer.
I tried to run the full-screen application and everything worked fine. Then I tried to run the application as a regular Windows application and it returned an error.
Could anyone take a look at the code below if there's a way to modify it to run the application as a regular Windows app?
HRESULT CViewfinderRenderer::OnStartStreaming()
{
HRESULT hr = S_OK;
DDSURFACEDESC ddsd;
m_pDD = NULL;
//full screen settings
hr = DirectDrawCreate(NULL, &m_pDD, NULL);
hr = m_pDD->SetCooperativeLevel(m_hWnd, DDSCL_FULLSCREEN);
ddsd.dwSize = sizeof(ddsd);
ddsd.dwFlags = DDSD_CAPS | DDSD_BACKBUFFERCOUNT;
ddsd.ddsCaps.dwCaps = DDSCAPS_FLIP | DDSCAPS_PRIMARYSURFACE;
ddsd.dwBackBufferCount = 1;
//end full screen settings
//normal settings
/*hr = DirectDrawCreate(NULL, &m_pDD, NULL);
hr = m_pDD->SetCooperativeLevel(m_hWnd, DDSCL_NORMAL);
ddsd.dwSize = sizeof(ddsd);
ddsd.dwFlags = DDSD_BACKBUFFERCOUNT;
ddsd.dwBackBufferCount = 1;*/
//end normal settings
hr = m_pDD->CreateSurface(&ddsd, &m_pSurface, NULL);
if (hr != DD_OK) {
return hr;
}
// Get backsurface
hr = m_pSurface->EnumAttachedSurfaces(&m_pBackSurface, EnumFunction);
return S_OK;
}
Even when running windowed, you need to create a primary surface, only it is not a flippable surface.
//full screen settings
hr = DirectDrawCreate(NULL, &m_pDD, NULL);
hr = m_pDD->SetCooperativeLevel(m_hWnd, DDSCL_NORMAL);
ddsd.dwSize = sizeof(ddsd);
ddsd.dwFlags = DDSD_CAPS;
ddsd.ddsCaps.dwCaps = DDSCAPS_PRIMARYSURFACE;
Besides of creating a surface, most likely you will want to create a clipper for the window. For a complete sample see paragraph Running windowed in this GameDev article.
What error did it return?
Also try this instead:
ddsd.dwFlags = DDSD_CAPS;
ddsd.ddsCaps.dwCaps = DDSCAPS_PRIMARYSURFACE;