What causes an access violation in "D3D11CreateDeviceAndSwapChain" in fullscreen mode? - c++

Question

I am trying to create a swapchain from a dll.
My process creates a window, initializes Direct3D11 and loads the dll.
It works perfectly fine when the process creates a non fullscreen window.
But as soon as the process creates a fullscreen window I get an access violation when calling D3D11CreateDeviceAndSwapChain from the dll.
The call occurs in a separate thread. (not from the dllmain thread.)
Now unfortunately I can't even debug properly since breakpoints freeze the fullscreen window and it is on top of my debugger window.
What causes the access violation?
EDIT: I have done some static analysis of dxgi.dll and found that when creating the swapchain it checks for a series of extended and basic window styles. Also I noticed that it should print a debug message before crashing so I hooked up a remote debugger to capture it. The message just says what I already knew:
DXGI ERROR: IDXGIFactory::CreateSwapChain: The specified output window has a style that is incompatible with DXGI. Output windows must have at least one of:
WS_POPUP
WS_BORDER
WS_CAPTION
WS_SYSMENU
WS_MAXIMIZEBOX
WS_MINIMIZEBOX
WS_THICKFRAME
WS_EX_CLIENTEDGE
WS_EX_CONTEXTHELP
WS_EX_DLGMODALFRAME
WS_EX_TOOLWINDOW
WS_EX_WINDOWEDGE
or not have WS_EX_TOPMOST. [ MISCELLANEOUS ERROR #78: ]
The error code is: DXGI_ERROR_INVALID_CALL
So why does intializing direct3d work from my executable but using the window it creates in my dll doesn't?
EDIT2: Apparently calling D3D11CreateDeviceAndSwapChain with DXGI_SWAP_CHAIN_DESC::Windowed set to FALSE in the executable changes the window styles causing the next call to D3D11CreateDeviceAndSwapChain to fail.
CODE SNIPPET:
IDXGISwapChain* swapChain;
ID3D11Device* device;
ID3D11DeviceContext* deviceContext;
DXGI_SWAP_CHAIN_DESC desc;
ZeroMemory(&desc, sizeof(desc));
desc.BufferCount = 1;
desc.BufferDesc.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
desc.BufferDesc.Scaling = DXGI_MODE_SCALING_UNSPECIFIED;
desc.BufferDesc.ScanlineOrdering = DXGI_MODE_SCANLINE_ORDER_UNSPECIFIED;
desc.BufferUsage = DXGI_USAGE_RENDER_TARGET_OUTPUT;
desc.Flags = DXGI_SWAP_CHAIN_FLAG_ALLOW_MODE_SWITCH;
desc.OutputWindow = FindWindowW(L"Window Title", NULL);
desc.SampleDesc.Count = 1;
desc.Windowed = ((GetWindowLongPtr(desc.OutputWindow, GWL_STYLE) & WS_POPUP) != 0) ? false : true;
desc.BufferDesc.ScanlineOrdering = DXGI_MODE_SCANLINE_ORDER_UNSPECIFIED;
desc.BufferDesc.Scaling = DXGI_MODE_SCALING_UNSPECIFIED;
desc.SwapEffect = DXGI_SWAP_EFFECT_DISCARD;
desc.BufferDesc.Width = 1;
desc.BufferDesc.Height = 1;
desc.BufferDesc.RefreshRate.Numerator = 0;
desc.BufferDesc.RefreshRate.Denominator = 1;
if (!desc.OutputWindow)
{
return nullptr;
}
D3D_FEATURE_LEVEL featureLevel;
const D3D_FEATURE_LEVEL featureLevelArray[] = { D3D_FEATURE_LEVEL_11_0, D3D_FEATURE_LEVEL_10_1, };
if (FAILED(D3D11CreateDeviceAndSwapChain(NULL, D3D_DRIVER_TYPE_HARDWARE, NULL, D3D11_CREATE_DEVICE_DEBUG, featureLevelArray, ARRAYSIZE(featureLevelArray), D3D11_SDK_VERSION, &desc, &swapChain, &device, &featureLevel, &deviceContext)))
{
return nullptr;
}

Answer 1

Calling D3D11CreateDeviceAndSwapChain with DXGI_SWAP_CHAIN_DESC::Windowed set to FALSE in the executable changes the window styles causing the next call to D3D11CreateDeviceAndSwapChain to fail.