How to block mouse input from transparent window with Win32 API? - c++

Question

I have a main window in a process that is not owned by the program I'm creating. I'm using a Windows Hook to inject a DLL into this process for the purpose of adding a child window to this main window.
My end goal was to create a WS_EX_LAYERED window that allows me to create an internal colored border but allow the center portion to be transparent and allow mouse clicks through. This part works perfectly.
WNDCLASS wndClass = {};
wndClass.style = CS_HREDRAW | CS_VREDRAW;
wndClass.lpfnWndProc = OverlayProc;
wndClass.hInstance = g_TargetInstance;
wndClass.hbrBackground = (HBRUSH)CreateSolidBrush(RGB(0, 255, 255));
wndClass.lpszClassName = "OVERLAY";
RegisterClass(&wndClass);
g_Window = CreateWindowEx(WS_EX_LAYERED | WS_EX_TRANSPARENT, "OVERLAY", nullptr,
WS_CHILDWINDOW, rect.left, rect.top, rect.right+1, rect.bottom+1, data->hwnd, nullptr, g_TargetInstance, nullptr);
SetLayeredWindowAttributes(g_Window, RGB(0, 255, 255), 0, LWA_COLORKEY);
ShowWindow(g_Window, SW_SHOW);
UpdateWindow(g_Window);
The 2nd part to this is a I wanted to conditionally block all mouse input to the parent window. I couldn't do this with the transparent portion of the WS_EX_LAYERED window so I tried creating a 2nd transparent STATIC control as a child of the main window but this doesn't block mouse input either.
I'm also sending simulated mouse clicks to the parent window through calls to PostMessage, passing WM_LBUTTONDOWN and WM_LBUTTONUP. How could I block all mouse input to the parent window via a transparent window?

Answer 1

It appears this is not possible to do with a simple transparent window drawn over sibling controls. What I ended up doing was using SetWindowHookEx to add a WH_GETMESSAGE hook into the process from which I use to replace the main window's WndProc function and intercept mouse messages. I tag my simulated mouse messages with a specific value in the wParam argument so the proc will now it was simulated and removes that value, passing it along to the parent window.
If it does not detect my "tag" value in the click message, it will swallow the mouse message and not pass it along to the original WndProc function.
Injected WndProc replacement
LRESULT CALLBACK WndProc(HWND hWnd, UINT uMsg, WPARAM wParam, LPARAM lParam, UINT_PTR uIdSubclass, DWORD_PTR dwRefData)
{
switch (uMsg)
{
case WM_LBUTTONDOWN:
wParam -= 11141008;
if (wParam != MK_LBUTTON && !g_Paused)
return 0;
break;
case WM_LBUTTONUP:
wParam -= 11141008;
if (wParam != 0 && !g_Paused)
return 0;
break;
case WM_MOUSEHOVER:
case WM_MOUSEMOVE:
if (!g_Paused)
return 0;
}
return DefSubclassProc(hWnd, uMsg, wParam, lParam);
}
Snippet from Windows Hook function
//...
switch (data->message)
{
case (WM_USER + 1):
{
g_Paused = FALSE;
//...
SetWindowSubclass(data->hwnd, WndProc, 1, 0);
break;
}
case (WM_USER + 2):
{
RemoveWindowSubclass(data->hwnd, WndProc, 1);
//...
break;
}
}
//...
The code inside the window hook function is used to subclass the main process window and inject my own WndProc function which in turn processes mouse input the way I want.
This is the code used to "simulate" mouse clicks without physically clicking in the window. Note the added value to wParam to identify this click as simulated and not generated by the user.
void Window::LeftClick(DWORD x, DWORD y, DWORD delayMillis)
{
LPARAM lparam = MAKELPARAM(x, y);
lock_guard<mutex> lock(this->m_ClickMutex);
PostMessage(this->m_Window, WM_LBUTTONDOWN, 11141008 + MK_LBUTTON, lparam);
this_thread::sleep_for(std::chrono::milliseconds(delayMillis));
PostMessage(this->m_Window, WM_LBUTTONUP, 11141008, lparam);
}
Also, just for the person in the comments who was ridiculing my choice of the word simulated and the added criticism for using PostMessage to simulate keyboard input, here is my keyboard input test method which (for my purposes) works flawlessly and very reliably
void GameWindow::KeyPress(UINT vkCode) const
{
UINT scanCode = MapVirtualKey(vkCode, MAPVK_VK_TO_VSC);
LPARAM lparam1 = MAKELPARAM(1, scanCode);
LPARAM lparam2 = MAKELPARAM(1, 0xC000 | scanCode);
PostMessage(this->m_Window, WM_KEYDOWN, vkCode, lparam1);
this_thread::sleep_for(chrono::milliseconds(25));
PostMessage(this->m_Window, WM_KEYUP, vkCode, lparam2);
}