Crashes during UnhookWindowsHookEx() - c++

Question

I made hook for calculator and want to get messages which calculator receives. To do that, I set my own window procedure, but during unhooking if I use SetWindowLong(..) to recover the old window procedure program crushes.
DLL code:
#define EXPORT_API extern "C" __declspec(dllexport)
EXPORT_API void InstallHook();
EXPORT_API void UninstallHook();
#pragma data_seg("Shared")
HHOOK g_hHook = NULL;
WNDPROC g_OldWndProc = NULL;
#pragma data_seg()
#pragma comment(linker, "/section:Shared,rws")
HWND GetTargetWindowHwnd()
{
return ::FindWindowA(0, "Calculator");
}
// my new wnd procedure to catch messages
LRESULT CALLBACK NewWndProc(HWND hwnd, UINT uMsg, WPARAM wParam, LPARAM lParam)
{
LRESULT lResult = 0;
switch(uMsg)
{
case WM_CLOSE:
{
MessageBoxA(0, "Here we are!", "", 0);
}
break;
default:
lResult = CallWindowProc(g_OldWndProc, hwnd, uMsg, wParam, lParam);
break;
}
lResult = CallWindowProc(g_OldWndProc, hwnd, uMsg, wParam, lParam);
return lResult;
}
// hook procedure
LRESULT CALLBACK GetMsgProc(int nCode, WPARAM wParam, LPARAM lParam)
{
MSG *pMsg = (MSG *)lParam;
HWND hWnd = GetTargetWindowHwnd();
bool flagIn = false;
if( hWnd == pMsg->hwnd )
{// if messege was sent to my target window
if(g_OldWndProc == NULL)
{
// save the adress of old wnd procedure to recover it later
g_OldWndProc = (WNDPROC)GetWindowLong(hWnd, GWL_WNDPROC);
// set my wnd procedure
SetWindowLong(hWnd, GWL_WNDPROC, (LONG)NewWndProc);
}
}
return CallNextHookEx(g_hHook, nCode, wParam, lParam);
}
EXPORT_API void InstallHook()
{
try
{
g_hHook = SetWindowsHookEx(WH_GETMESSAGE, (HOOKPROC)GetMsgProc, g_hInstance, 0);
}
catch(...)
{
MessageBoxA(0, "Hook error", "Error", 0);
}
}
EXPORT_API void UninstallHook()
{
if(g_OldWndProc)
{
// recovering old wnd proc
HWND hWnd = GetTargetWindowHwnd();
SetWindowLong(hWnd, GWL_WNDPROC, (LONG)g_OldWndProc);
g_OldWndProc = NULL;
}
if (g_hHook)
{
UnhookWindowsHookEx(g_hHook);
g_hHook = NULL;
}
}
BOOL APIENTRY DllMain( HMODULE hModule,
DWORD ul_reason_for_call,
LPVOID lpReserved
)
{
switch (ul_reason_for_call)
{
case DLL_PROCESS_ATTACH:
g_hInstance = (HINSTANCE) hModule;
break;
case DLL_THREAD_ATTACH:
case DLL_THREAD_DETACH:
case DLL_PROCESS_DETACH:
break;
}
return TRUE;
}
EXE CODE:
void CHookTestDlg::OnBnClickedBtnInstall()
{
InstallHook();
}
void CHookTestDlg::OnBnClickedBtnUninstall()
{
UninstallHook();
}
If I don't use my wnd procedure it works normal. If I use SetWindowLong(..) to recover the old window procedure program crushes during unhook. What is wrong?

Answer 1

The problem is that you are setting the window proc on the target window from within the target process (calc), and in that case, it is succeeding. But when you call UninstallHook, that code runs in your own exe's process; and in that case, SetWindowLong will fail.
(Putting the hook values in shared memory won't help; SetWindowLong will still refuse to change the window proc across a process boundary - see MSDN for details.)
To get this to work, you would need to communicate to the hooked instance of the DLL and ask it to reset the wndproc from within that target process, and once that is done, then unhook the hook.
(atzz's advice on unhooking is also valid. Hooking windows that you don't own is generally best avoided.)

Answer 2

When unsubclassing, always check that the window was not subclassed by someone else after you. I.e. before restoring WindowProc, you should read it again and compare against expected value (NewWndProc). If it is different, you must not unload the DLL, because another subclasser has a pointer to your DLL code stored, and this pointer will become dangling as soon as your DLL is unloaded.