DirectShow DVD playback - c++

Question

I have created a custom allocator/presenter that works fine for playback of normal media files. However, when I use the following code to try to playback a DVD, it fails with a stack overflow exception.
vmr9_ap = new vmr9ap();
HMONITOR monitor = MonitorFromWindow(hwnd, NULL);
IGraphBuilder *graph;
IBaseFilter *filter;
IDvdGraphBuilder *builder;
CoCreateInstance(CLSID_DvdGraphBuilder, NULL, CLSCTX_INPROC_SERVER, IID_IDvdGraphBuilder, reinterpret_cast<void**>(&builder));
CoCreateInstance(::CLSID_VideoMixingRenderer9, NULL, CLSCTX_INPROC_SERVER, IID_IBaseFilter, reinterpret_cast<void**>(&filter));
builder->GetDvdInterface(IID_IVMRFilterConfig9, (void**)&vmr9_config);
vmr9_ap->Initialize(g_pd3dDevice, monitor, vmr9_config);
HRESULT hr = builder->RenderDvdVideoVolume(L"G:\\VIDEO_TS", AM_DVD_SWDEC_PREFER | AM_DVD_VMR9_ONLY, &status);
builder->GetFiltergraph(&graph);
IDvdControl2 *dvdControl;
builder->GetDvdInterface(::IID_IDvdControl2, (void**)&dvdControl);
graph->QueryInterface(::IID_IMediaControl, (void**)&control);
HRESULT h = control->Run();
The stack overflow happens immediately after the call to control->Run(). It's driving me nuts, as I'm sure I'm just forgetting something really really simple.
Thanks.

Answer 1

Your graph should look something like this. Make sure there aren't any buggy filters in your graph.
Because you are using a custom allocator, I would look there for the issue and set some breakpoints there. You code you pasted might be incomplete as I do not see you configure the VMR9 with the custom allocator, nor do I see it being added to the graph. I avoid using the DVDGraphBuilder as I had too difficult of a time getting it RenderVolume correctly with my VMR9+Allocator. I would build the graph a little more manually.
I have a custom allocator in my open source project, along w/ a DVD player. You can check that out for reference, though there is a lot of code noise in there due to me needing to hack a few things in there for WPF compatiblity. http://wpfmediakit.codeplex.com
What you are seeing should NOT be a DRM issue.
alt text http://img29.imageshack.us/img29/7798/capturelu.jpg

Answer 2

Could it be a form of DRM protection? Decoders in DVD graphs will typically try to prevent you building graphs that get access to the uncompressed data as you do here. Normally they do that by a cleaner method, such as refusing to connect to unauthorised renderers, but it's possible that this might be caused by something like that -- certainly there are mpeg-2 decoders which use deliberate crashes to prevent reverse engineering.
G

Answer 3

Thanks to the code Jeremiah Morrill pointed me to, I managed to get playback mostly working.
It works fine, as long as you don't try to resize the D3DImage that it's played in. Devil's in the details, I suppose.
Thanks to all the answers. DVD playback doesn't work with a debugger attached, which according to Google, is not DRM, but is an anti-reverse engineering measure. Could be particular to the DVD codec I'm using too.
extern "C" __declspec(dllexport) LPDIRECT3DSURFACE9 InitializeDvd(HWND hWnd)
{
CoInitialize(NULL);
IPin *dvdVideoOut;
IPin *vmr9VideoIn;
HRESULT hr = S_OK;
hr = CoCreateInstance(CLSID_FilterGraph, NULL, CLSCTX_INPROC_SERVER,
IID_IGraphBuilder, (void **)&graph);
if(graph)
{
hr = CoCreateInstance(CLSID_DVDNavigator, NULL, CLSCTX_INPROC_SERVER,
IID_IBaseFilter, (void **)&dvdNavigator);
if(dvdNavigator)
{
hr = graph->AddFilter(dvdNavigator, L"DVD Navigator");
if(SUCCEEDED(hr))
{
hr = CoCreateInstance(CLSID_VideoMixingRenderer9, NULL, CLSCTX_INPROC_SERVER,
IID_IBaseFilter, (void **)&vmr9);
if(vmr9)
{
hr = vmr9->QueryInterface(IID_IVMRFilterConfig9, reinterpret_cast<void**>(&p_fConfig));
p_Ap = new VMR9AllocatorPresenter();
p_Dh = new DeviceHandler();
p_device = p_Dh->Initialize(hWnd);
p_fConfig->SetRenderingMode(VMR9Mode_Renderless);
p_fConfig->SetNumberOfStreams(1);
p_Ap->Initialize(hWnd, p_device, p_fConfig);
if(SUCCEEDED(hr))
{
hr = graph->AddFilter(vmr9, L"Video Mixing Renderer 9");
if(p_fConfig)
{
dvdNavigator->FindPin(L"Video", &dvdVideoOut);
if(dvdVideoOut)
{
hr = graph->Render(dvdVideoOut);
}
hr = graph->QueryInterface(IID_IMediaControl, reinterpret_cast<void**>(&control));
if(control)
{
control->Run();
}
}
}
}
}
}
}
return p_Dh->g_surface9;
}