Actually, it only fails the second time it's called. I'm using a windowless control to play video content, where the video being played could change while the control is still on screen. Once the graph is built the first time, we switch media by stopping playback, replacing the SOURCE filter, and running the graph again. This works fine under Vista, but when running on XP, the second call to Run() returns E_UNEXPECTED.
The initialization goes something like this:
// Get the interface for DirectShow's GraphBuilder
mGB.CoCreateInstance(CLSID_FilterGraph, NULL, CLSCTX_INPROC_SERVER);
// Create the Video Mixing Renderer and add it to the graph
ATL::CComPtr<IBaseFilter> pVmr;
pVmr.CoCreateInstance(CLSID_VideoMixingRenderer9, NULL, CLSCTX_INPROC);
mGB->AddFilter(pVmr, L"Video Mixing Renderer 9");
// Set the rendering mode and number of streams
ATL::CComPtr<IVMRFilterConfig9> pConfig;
pVmr->QueryInterface(IID_IVMRFilterConfig9, (void**)&pConfig);
pConfig->SetRenderingMode(VMR9Mode_Windowless);
pVmr->QueryInterface(IID_IVMRWindowlessControl9, (void**)&mWC);
And here's what we do when we decide to play a movie. RenderFileToVideoRenderer is borrowed from dshowutil.h in the DirectShow samples area.
// Release the source filter, if it exists, so we can replace it.
IBaseFilter *pSource = NULL;
if (SUCCEEDED(mpGB->FindFilterByName(L"SOURCE", &pSource)) && pSource)
{
mpGB->RemoveFilter(pSource);
pSource->Release();
pSource = NULL;
}
// Render the file.
hr = RenderFileToVideoRenderer(mpGB, mPlayPath.c_str(), FALSE);
// QueryInterface for DirectShow interfaces
hr = mpGB->QueryInterface(&mMC);
hr = mpGB->QueryInterface(&mME);
hr = mpGB->QueryInterface(&mMS);
// Read the default video size
hr = mpWC->GetNativeVideoSize(&lWidth, &lHeight, NULL, NULL);
if (hr != E_NOINTERFACE)
{
if (FAILED(hr))
{
return hr;
}
// Play video at native resolution, anchored at top-left corner.
RECT r;
r.left = 0;
r.top = 0;
r.right = lWidth;
r.bottom = lHeight;
hr = mpWC->SetVideoPosition(NULL, &r);
}
// Run the graph to play the media file
if (mMC)
{
hr = mMC->Run();
if (FAILED(hr))
{
// We get here the second time this code is executed.
return hr;
}
mState = Running;
}
if (mME)
{
mME->SetNotifyWindow((OAHWND)m_hWnd, WM_GRAPHNOTIFY, 0);
}
Anybody know what's going on here?
Try calling IMediaControl::StopWhenReady before removing the source filter.
When are you calling QueryInterface directly? you can use CComQIPtr<> to warp the QI for you. This way you won't have to call Release as it will be called automatically.
The syntax look like this: CComPtr<IMediaControl> mediaControl = pGraph;
In FindFilterByName() instead of passing live pointer pass a CComPtr, again so you won't have to call release explicitly.
Never got a resolution on this. The production solution was to just call IGraphBuilder::Release and rebuild the entire graph from scratch. There's a CPU spike and a slight redraw delay when switching videos, but it's less pronounced than we'd feared.
Related
I'm building an application that is used for taking and sharing screenshots in real time between multiple clients over network.
I'm using the MS Desktop Duplication API to get the image data and it's working smoothly except in some edge cases.
I have been using four games as test applications in order to test how the screencapture behaves in fullscreen and they are Heroes of the Storm, Rainbow Six Siege, Counter Strike and PlayerUnknown's Battlegrounds.
On my own machine which has a GeForce GTX 1070 graphics card; everything works fine both in and out of fullscreen for all test applications. On two other machines that runs a GeForce GTX 980 however; all test applications except PUBG works. When PUBG is running in fullscreen, my desktop duplication instead produces an all black image and I can't figure out why as the
Desktop Duplication Sample works fine for all test machines and test applications.
What I'm doing is basically the same as the sample except I'm extracting the pixel data and creating my own SDL(OpenGL) texture from that data instead of using the acquired ID3D11Texture2D directly.
Why is PUBG in fullscreen on GTX 980 the only test case that fails?
Is there something wrong with the way I'm getting the frame, handling the "DXGI_ERROR_ACCESS_LOST" error or how I'm copying the data from the GPU?
Declarations:
IDXGIOutputDuplication* m_OutputDup = nullptr;
Microsoft::WRL::ComPtr<ID3D11Device> m_Device = nullptr;
ID3D11DeviceContext* m_DeviceContext = nullptr;
D3D11_TEXTURE2D_DESC m_TextureDesc;
Initialization:
bool InitializeScreenCapture()
{
HRESULT result = E_FAIL;
if (!m_Device)
{
D3D_FEATURE_LEVEL featureLevels = D3D_FEATURE_LEVEL_11_0;
D3D_FEATURE_LEVEL featureLevel;
result = D3D11CreateDevice(
nullptr,
D3D_DRIVER_TYPE_HARDWARE,
nullptr,
0,
&featureLevels,
1,
D3D11_SDK_VERSION,
&m_Device,
&featureLevel,
&m_DeviceContext);
if (FAILED(result) || !m_Device)
{
Log("Failed to create D3DDevice);
return false;
}
}
// Get DXGI device
ComPtr<IDXGIDevice> DxgiDevice;
result = m_Device.As(&DxgiDevice);
if (FAILED(result))
{
Log("Failed to get DXGI device);
return false;
}
// Get DXGI adapter
ComPtr<IDXGIAdapter> DxgiAdapter;
result = DxgiDevice->GetParent(__uuidof(IDXGIAdapter), &DxgiAdapter);
if (FAILED(result))
{
Log("Failed to get DXGI adapter);
return false;
}
DxgiDevice.Reset();
// Get output
UINT Output = 0;
ComPtr<IDXGIOutput> DxgiOutput;
result = DxgiAdapter->EnumOutputs(Output, &DxgiOutput);
if (FAILED(result))
{
Log("Failed to get DXGI output);
return false;
}
DxgiAdapter.Reset();
ComPtr<IDXGIOutput1> DxgiOutput1;
result = DxgiOutput.As(&DxgiOutput1);
if (FAILED(result))
{
Log("Failed to get DXGI output1);
return false;
}
DxgiOutput.Reset();
// Create desktop duplication
result = DxgiOutput1->DuplicateOutput(m_Device.Get(), &m_OutputDup);
if (FAILED(result))
{
Log("Failed to create output duplication);
return false;
}
DxgiOutput1.Reset();
DXGI_OUTDUPL_DESC outputDupDesc;
m_OutputDup->GetDesc(&outputDupDesc);
// Create CPU access texture description
m_TextureDesc.Width = outputDupDesc.ModeDesc.Width;
m_TextureDesc.Height = outputDupDesc.ModeDesc.Height;
m_TextureDesc.Format = outputDupDesc.ModeDesc.Format;
m_TextureDesc.ArraySize = 1;
m_TextureDesc.BindFlags = 0;
m_TextureDesc.MiscFlags = 0;
m_TextureDesc.SampleDesc.Count = 1;
m_TextureDesc.SampleDesc.Quality = 0;
m_TextureDesc.MipLevels = 1;
m_TextureDesc.CPUAccessFlags = D3D11_CPU_ACCESS_FLAG::D3D11_CPU_ACCESS_READ;
m_TextureDesc.Usage = D3D11_USAGE::D3D11_USAGE_STAGING;
return true;
}
Screen capture:
void TeamSystem::CaptureScreen()
{
if (!m_ScreenCaptureInitialized)
{
Log("Attempted to capture screen without ScreenCapture being initialized");
return false;
}
HRESULT result = E_FAIL;
DXGI_OUTDUPL_FRAME_INFO frameInfo;
ComPtr<IDXGIResource> desktopResource = nullptr;
ID3D11Texture2D* copyTexture = nullptr;
ComPtr<ID3D11Resource> image;
int32_t attemptCounter = 0;
DWORD startTicks = GetTickCount();
do // Loop until we get a non empty frame
{
m_OutputDup->ReleaseFrame();
result = m_OutputDup->AcquireNextFrame(1000, &frameInfo, &desktopResource);
if (FAILED(result))
{
if (result == DXGI_ERROR_ACCESS_LOST) // Access may be lost when changing from/to fullscreen mode(any application); when this happens we need to reaquirce the outputdup
{
m_OutputDup->ReleaseFrame();
m_OutputDup->Release();
m_OutputDup = nullptr;
m_ScreenCaptureInitialized = InitializeScreenCapture();
if (m_ScreenCaptureInitialized)
{
result = m_OutputDup->AcquireNextFrame(1000, &frameInfo, &desktopResource);
}
else
{
Log("Failed to reinitialize screen capture after access was lost");
return false;
}
}
if (FAILED(result))
{
Log("Failed to acquire next frame);
return false;
}
}
attemptCounter++;
if (GetTickCount() - startTicks > 3000)
{
Log("Screencapture timed out after " << attemptCounter << " attempts");
return false;
}
} while(frameInfo.TotalMetadataBufferSize <= 0 || frameInfo.LastPresentTime.QuadPart <= 0); // This is how you wait for an image containing image data according to SO (https://stackoverflow.com/questions/49481467/acquirenextframe-not-working-desktop-duplication-api-d3d11)
Log("ScreenCapture succeeded after " << attemptCounter << " attempt(s)");
// Query for IDXGIResource interface
result = desktopResource->QueryInterface(__uuidof(ID3D11Texture2D), reinterpret_cast<void**>(©Texture));
desktopResource->Release();
desktopResource = nullptr;
if (FAILED(result))
{
Log("Failed to acquire texture from resource);
m_OutputDup->ReleaseFrame();
return false;
}
// Copy image into a CPU access texture
ID3D11Texture2D* stagingTexture = nullptr;
result = m_Device->CreateTexture2D(&m_TextureDesc, nullptr, &stagingTexture);
if (FAILED(result) || stagingTexture == nullptr)
{
Log("Failed to copy image data to access texture);
m_OutputDup->ReleaseFrame();
return false;
}
D3D11_MAPPED_SUBRESOURCE mappedResource;
m_DeviceContext->CopyResource(stagingTexture, copyTexture);
m_DeviceContext->Map(stagingTexture, 0, D3D11_MAP_READ, 0, &mappedResource);
void* copy = malloc(m_TextureDesc.Width * m_TextureDesc.Height * 4);
memcpy(copy, mappedResource.pData, m_TextureDesc.Width * m_TextureDesc.Height * 4);
m_DeviceContext->Unmap(stagingTexture, 0);
stagingTexture->Release();
m_OutputDup->ReleaseFrame();
// Create a new SDL texture from the data in the copy varialbe
free(copy);
return true;
}
Some notes:
I have modified my original code to make it more readable so some cleanup and logging in the error handling is missing.
None of the error or timeout cases(except DXGI_ERROR_ACCESS_LOST) trigger in any testing scenario.
The "attemptCounter" never goes above 2 in any testing scenario.
The test cases are limited since I don't have access to a computer which produces the black image case.
The culprit was CopyResource() and how I created the CPU access texture.
CopyResource() returns void and that is why I didn't look into it before; I didn't think it could fail in any significant way since I expected it to return bool or HRESULT if that was the case.
In the documentation of CopyResource() does however disclose a couple of fail cases.
This method is unusual in that it causes the GPU to perform the copy operation (similar to a memcpy by the CPU). As a result, it has a few restrictions designed for improving performance. For instance, the source and destination resources:
Must be different resources.
Must be the same type.
Must have identical dimensions (including width, height, depth, and size as appropriate).
Must have compatible DXGI formats, which means the formats must be identical or at least from the same type group.
Can't be currently mapped.
Since the initialization code runs before the test application enters fullscreen, the CPU access texture description is set up using the desktop resolution, format etc. This caused CopyResouce() to fail silently and simply now write anything to stagingTexture in the test cases where a non native resoltuion was used for the test application.
In conclusion; I just moved the m_TextureDescription setup to CaptureScreen() and used the description of copyTexture to get the variables I didn't want to change between the textures.
// Create CPU access texture
D3D11_TEXTURE2D_DESC copyTextureDesc;
copyTexture->GetDesc(©TextureDesc);
D3D11_TEXTURE2D_DESC textureDesc;
textureDesc.Width = copyTextureDesc.Width;
textureDesc.Height = copyTextureDesc.Height;
textureDesc.Format = copyTextureDesc.Format;
textureDesc.ArraySize = copyTextureDesc.ArraySize;
textureDesc.BindFlags = 0;
textureDesc.MiscFlags = 0;
textureDesc.SampleDesc = copyTextureDesc.SampleDesc;
textureDesc.MipLevels = copyTextureDesc.MipLevels;
textureDesc.CPUAccessFlags = D3D11_CPU_ACCESS_FLAG::D3D11_CPU_ACCESS_READ;
textureDesc.Usage = D3D11_USAGE::D3D11_USAGE_STAGING;
ID3D11Texture2D* stagingTexture = nullptr;
result = m_Device->CreateTexture2D(&textureDesc, nullptr, &stagingTexture);
While this solved the issues I was having; I still don't know why the reinitialization in the handling of DXGI_ERROR_ACCESS_LOST didn't resolve the issue anyway. Does the DesctopDuplicationDescription not use the same dimensions and format as the copyTexture?
I also don't know why this didn't fail in the same way on computers with newer graphics cards. I did however notice that these machines were able to capture fullscreen applications using a simple BitBlt() of the desktop surface.
I try to replace codes with Directshow ("DS") on Media Foundation ("MF") in my app and met one problem - cannot set a needed fps using MF on a webcam. MF allowed me to set only 30 fps. If I try to set 25 fps, I always get the error 0xc00d5212 on SetCurrentMediaType(). In DS I could change that parameter.
My codes:
ASSERT(m_pReader); //IMFSourceReader *m_pReader;
IMFMediaType *pNativeType = NULL;
IMFMediaType *pType = NULL;
UINT32 w = 1280;
UINT32 h = 720;
UINT32 fps = 25; // or 30
DWORD dwStreamIndex = MF_SOURCE_READER_FIRST_VIDEO_STREAM;
// Find the native format of the stream.
HRESULT hr = m_pReader->GetNativeMediaType(dwStreamIndex, 0, &pNativeType);
if (FAILED(hr))
{
//error
}
GUID majorType, subtype;
// Find the major type.
hr = pNativeType->GetGUID(MF_MT_MAJOR_TYPE, &majorType);
if (FAILED(hr))
{
//error
}
// Define the output type.
hr = MFCreateMediaType(&pType);
if (FAILED(hr))
{
//error
}
hr = pType->SetGUID(MF_MT_MAJOR_TYPE, majorType);
if (FAILED(hr))
{
//error
}
// Select a subtype.
if (majorType == MFMediaType_Video)
{
subtype= MFVideoFormat_RGB24;
}
else
{
//error
}
hr = pType->SetGUID(MF_MT_SUBTYPE, subtype);
if (FAILED(hr))
{
//error
}
hr = MFSetAttributeSize(pType, MF_MT_FRAME_SIZE, w, h);
if (FAILED(hr))
{
//error
}
hr = MFSetAttributeSize(pType, MF_MT_FRAME_RATE, fps, 1);
if (FAILED(hr))
{
//error
}
hr = m_pReader->SetCurrentMediaType(dwStreamIndex, NULL, pType);
if (FAILED(hr))
{// hr = 0xc00d5212
//!!!!!error - if fps == 25
}
return hr;
Thanks for any help.
It might so happen that the camera does not support flexible frame rate values, and can work with only among the supported set, for example: 10, 15, 20, 24, 30 fps. You should be able to enumerate supported media types and choose the one that works for you - those media types typically include frame rate options.
Even though Media Foundation and DirectShow video capture eventually ends up in the same backend, there might be discrepancies in behavior. Specifically, you are working with Media Foundation higher level API that internally interfaces to a media source, and it might so happens that frame rate leads to 0xC00D5212 MF_E_TOPO_CODEC_NOT_FOUND "No suitable transform was found to encode or decode the content" confusion even though technically the driver can capture in respective mode.
See also:
Get all supported FPS values of a camera in Microsoft Media Foundation
Media Foundation Video/Audio Capture Capabilities
I've added the timer for fps control imitation into the codes. So at the start I set 30 fps , then by fps scale I skip some frames for my app.
Thank you for help.
I'm trying to use the SaveWICTextureToFile method from DirectXTK to grab a screenshot in my Windows Store app (Windows 8.1). I'm using XAML with a SwapChainPanel element. Unfortunately, the method always saves a rectangle of the screen size filled with solid color rather than the current screen content. Every time I call the SaveWICTextureToFile method it saves a different color.
This is my code (simplified):
void DirectXPage::SaveButton_Click(Platform::Object^ sender, Windows::UI::Xaml::RoutedEventArgs^ e)
{
critical_section::scoped_lock lock(m_main->GetCriticalSection());
auto folder = Windows::Storage::ApplicationData::Current->TemporaryFolder;
WCHAR fname[_MAX_PATH];
wcscpy_s(fname, folder->Path->Data());
wcscat_s(fname, L"\\screenshot.png");
auto context = m_deviceResources->GetD3DDeviceContext();
auto swapChain = m_deviceResources->GetSwapChain();
ID3D11Texture2D* backBuffer = nullptr;
HRESULT hr = swapChain->GetBuffer(0, __uuidof(*backBuffer), (LPVOID*)&backBuffer);
if (SUCCEEDED(hr))
{
HRESULT hr = SaveWICTextureToFile(context, backBuffer, GUID_ContainerFormatPng, fname);
DX::ThrowIfFailed(hr);
// ... mode code for FileSavePicker etc.
}
}
What am I doing wrong?
Thanks,
Leszek
I am embedding a window media player activex control in a dialog based mfc application. Now all play/pause button is getting disabled. I want to enable them.
As I have put put_uiMode(_T("mini"));
and when writingthese lines
hr = m_spWMPPlayer->put_fullScreen(VARIANT_TRUE)
hr = m_spWMPPlayer->get_fullScreen(&b); // returned b = VARIANT_FALSE
value of b is coming FALSE.
What could be the error? Any one knows this?
Thank You
BOOL CLuminPlayerDlg::OnInitDialog()
{
CDialogEx::OnInitDialog();
CComPtr<IWMPPlayer> m_spWMPPlayer;
CComPtr<IWMPRemoteMediaServices> m_spServices;
CComPtr<IWMPControls> m_spControls;
// Set the icon for this dialog. The framework does this automatically
// when the application's main window is not a dialog
SetIcon(m_hIcon, TRUE); // Set big icon
SetIcon(m_hIcon, FALSE); // Set small icon
// TODO: Add extra initialization here
HRESULT hr = NULL;
if(SUCCEEDED(hr))
{
hr = CoCreateInstance(__uuidof(WindowsMediaPlayer), 0, CLSCTX_INPROC_SERVER, IID_IOleObject, (void**)&m_spServices);
if(SUCCEEDED(hr))
{
BSTR str = NULL;
VARIANT_BOOL b ;
hr = m_spServices->QueryInterface(__uuidof(IWMPPlayer), (void**)&m_spWMPPlayer);
if(SUCCEEDED(hr))
{
hr = m_spWMPPlayer->get_enabled(&b); // returned b= VARIANT_TRUE
hr = m_spWMPPlayer->get_enableContextMenu(&b); //returned b = VARIANT_TRUE
hr = m_spWMPPlayer->get_uiMode(&str); // returned str = L"full"
hr = m_spWMPPlayer->put_uiMode(_T("mini")); //hr = S_OK
hr = m_spWMPPlayer->get_uiMode(&str); // str = L"mini"
hr = m_spWMPPlayer->put_fullScreen(VARIANT_TRUE);
hr = m_spWMPPlayer->get_fullScreen(&b); // returned b = VARIANT_FALSE
hr = m_spWMPPlayer->put_URL(_T("C:\\Visual Studio 2012\\Projects\\Lumin-Player\\Debug\\abc.mp4")); //returned hr = S_OK
hr = m_spServices->QueryInterface(__uuidof(IWMPControls), (void**)&m_spControls); // returned hr = S_OK
if(SUCCEEDED(hr))
{
hr = m_spControls->play(); //returned hr = S_OK
}
}
}
}
return TRUE; // return TRUE unless you set the focus to a control
}
MSDN says:
For full-screen mode to work properly when embedding the Windows Media Player control, the video display area must have a height and width of at least one pixel. If the BSTR specified in IWMPPlayer::put_uiMode is set to "mini" or "full", the height of the control itself must be 65 pixels or greater to accommodate the video display area in addition to the user interface.
This assumes that the player is already properly initialized as ActiveX control. In your code, you simply create a COM object without doing any ActiveX Control initialization. Presumably the player detects this and reports error.
Your hr in respective put_fullScreen call should have 0xC00D0FD2 NS_E_WMPOCX_NO_ACTIVE_CORE "The requested method or property is not available because the Windows Media Player ActiveX control has not been properly activated." to indicate the problem.
I am developing an application for video recording and I want to overlay the video preview with a logo and recording timer.
I tried to run the full-screen application and everything worked fine. Then I tried to run the application as a regular Windows application and it returned an error.
Could anyone take a look at the code below if there's a way to modify it to run the application as a regular Windows app?
HRESULT CViewfinderRenderer::OnStartStreaming()
{
HRESULT hr = S_OK;
DDSURFACEDESC ddsd;
m_pDD = NULL;
//full screen settings
hr = DirectDrawCreate(NULL, &m_pDD, NULL);
hr = m_pDD->SetCooperativeLevel(m_hWnd, DDSCL_FULLSCREEN);
ddsd.dwSize = sizeof(ddsd);
ddsd.dwFlags = DDSD_CAPS | DDSD_BACKBUFFERCOUNT;
ddsd.ddsCaps.dwCaps = DDSCAPS_FLIP | DDSCAPS_PRIMARYSURFACE;
ddsd.dwBackBufferCount = 1;
//end full screen settings
//normal settings
/*hr = DirectDrawCreate(NULL, &m_pDD, NULL);
hr = m_pDD->SetCooperativeLevel(m_hWnd, DDSCL_NORMAL);
ddsd.dwSize = sizeof(ddsd);
ddsd.dwFlags = DDSD_BACKBUFFERCOUNT;
ddsd.dwBackBufferCount = 1;*/
//end normal settings
hr = m_pDD->CreateSurface(&ddsd, &m_pSurface, NULL);
if (hr != DD_OK) {
return hr;
}
// Get backsurface
hr = m_pSurface->EnumAttachedSurfaces(&m_pBackSurface, EnumFunction);
return S_OK;
}
Even when running windowed, you need to create a primary surface, only it is not a flippable surface.
//full screen settings
hr = DirectDrawCreate(NULL, &m_pDD, NULL);
hr = m_pDD->SetCooperativeLevel(m_hWnd, DDSCL_NORMAL);
ddsd.dwSize = sizeof(ddsd);
ddsd.dwFlags = DDSD_CAPS;
ddsd.ddsCaps.dwCaps = DDSCAPS_PRIMARYSURFACE;
Besides of creating a surface, most likely you will want to create a clipper for the window. For a complete sample see paragraph Running windowed in this GameDev article.
What error did it return?
Also try this instead:
ddsd.dwFlags = DDSD_CAPS;
ddsd.ddsCaps.dwCaps = DDSCAPS_PRIMARYSURFACE;