Variable keeps getting set to NULL after function in DLL - c++

I need valid data to be in the global variable QObject *p. However, assigning anything to this variable inside of a function works within the scope of the function, but after the function returns, p is set back to NULL, even though p is global. Here is my code:
#include ... // various includes
// p is NULL
QObject *p;
HHOOK hhk;
BOOL WINAPI DllMain(__in HINSTANCE hinstDLL, __in DWORD fdwReason, __in LPVOID lpvReserved)
{
return TRUE;
}
LRESULT CALLBACK MouseProc(int nCode, WPARAM wParam, LPARAM lParam)
{
MOUSEHOOKSTRUCT *mouseInfo = (MOUSEHOOKSTRUCT*)lParam;
QMouseEvent::Type type;
QPoint pos = QPoint(mouseInfo->pt.x, mouseInfo->pt.y);
Qt::MouseButton bu;
Qt::MouseButtons bus;
Qt::KeyboardModifiers md = Qt::NoModifier;
... // very large switch statement
// here is where i need some valid data in p
QCoreApplication::postEvent(p, new QMouseEvent(type, pos, bu, bus, md));
return CallNextHookEx(NULL, nCode, wParam, lParam);
}
// note: MOUSEHOOKSHARED_EXPORT is the same as __declspec(dllexport)
// this function is called by the application that loads the dll
extern "C" MOUSEHOOKSHARED_EXPORT void install(QObject *mParent, DWORD threadID, HINSTANCE hInst)
{
p = mParent; // p is assigned here and keeps the value of mParent untill the function returns
hhk = SetWindowsHookEx(WH_MOUSE, MouseProc, hInst, threadID);
} // after this function returns, p is NULL
extern "C" MOUSEHOOKSHARED_EXPORT void uninstall()
{
UnhookWindowsHookEx(hhk);
}
I have tried various data structure "workarounds" such as using struct typedef etc... i can't seem to get this to work. All i need is for p to retain the value of mParent.
EDIT:
Here is where i execute install()
// EarthWidget is a class that is derived from QWidget(which is derived from QObject)
void EarthWidget::LoadAll()
{
HINSTANCE DLLinst = LoadLibrary("MouseHook.dll");
... // get functions in DLL using GetProcAddress & typedefs, etc...
// i pass in 'this' as mParent
install(this, GetWindowThreadProcessId((HWND)earthplugin->GetRenderHwnd(), NULL), DLLinst);
// note that GetWindowThreadProcessId does work and does return a valid thread id, so no problem there
}
EDIT:
Found out what was wrong. The this pointer becomes out of scope when install is executed, therefore mParent, being a QObject, initializes itself to NULL. Thus p becomes NULL. this comes back into scope when install returns, however, at a completely different memory address. The solution, after extensive debugging and headaches, would be to create a class member function that takes a QObject as a parameter and passes that into install instead of this. Whatever you pass into that function must last as long as you need the DLL to last. That, or, you can create your own copy constructor that performs a deep copy.

Are you exporting the global in the DLL and then importing it in the program? If you are not, or not doing it correctly, you probably have two p objects: one in the DLL and one in the program. You can confirm this by checking the address of p.
Rather than a global variable consider using an exported function in the DLL that returns the needed reference/pointer. At the very least name it something more descriptive (unless it was just renamed for the purpose of asking this question).

Is your call to install actually happening in the dll address space (in the debugger, step into the call and check the addresses in the callstack)? Is install defined in a header file or is that extract from a source file? If in a header, it's been inlined into your exe so the dll version of p is never set. This would happen without any linker warning since there are two independent binaries using the same source.
Is MOUSEHOOKSHARED_EXPORT defined in your app? Probably needs to be MOUSEHOOKSHARED_IMPORT for the app (but not the dll).

You're creating a shallow copy of that parameter mParent. At some point, that pointer must be getting set to null (or you're passing it as null), which will result in p also becoming null.

Related

C++ Windows: Access same variable from main process and CallWndProc process

I have a single-file DLL that receives and stores a _bstr_t as a global variable, then sets a Windows hook for the WH_CALLWNDPROC procedure.
In the CallWndProc function, I attempt to read the _bstr_t, but it has no value.
I printed out the variable's address from both functions and they are different.
This is not surprising as I think that the CallWndProc function is called in a different process' thread.
My question is, what is the easiest and best way to share the variable between them?
I am trying to avoid having to use ATL COM to store it for inter-process access.
Example code:
// foo.cpp
#include <comutil.h>
static HHOOK g_hook = NULL;
static _bstr_t shared = "";
static LRESULT WINAPI CallWndProc(int nHookCode, WPARAM wParam, LPARAM lParam) {
if (nHookCode == 12345) {
// Do something with '_bstr_t shared'
shared += " bar";
return 0;
}
return CallNextHookEx(g_hook, nHookCode, wParam, lParam);
}
extern "C" __declspec(dllexport) void _stdcall Do(char* someStr, long handle) {
shared = someStr;
DWORD threadId = GetWindowThreadProcessId((HWND) handle, &process);
HINSTANCE hInst = GetModuleHandle("foo.dll");
g_hook = SetWindowsHookEx(WH_CALLWNDPROC, CallWndProc, hInst, thread);
// Check value of '_bstr_t shared', or call a COM function to store it somewhere.
// However, the value of 'shared' does not include the string appended in CallWndProc.
}
Extra info:
DLL is called by Java using JNI.
DLL is built as a multi-threaded DLL.
I went with the ATL COM approach to act as a global store of values.
Not my ideal solution, but a workable one.
However, I think Hans Passant's comment on the question is a better way of doing it since it does not require a separate process to be running.

global variable in dll inconsistent?

I am creating a (temporary) log file from a dll. But the global variable I defined seem to be inconsistent.
Here is how I define variables in dll's main cpp file.
char * g_bfr;
__declspec(dllexport) CMemFile memFile;
Then in DllMain function:
extern "C" int APIENTRY
DllMain(HINSTANCE hInstance, DWORD dwReason, LPVOID lpReserved)
{
// Remove this if you use lpReserved
UNREFERENCED_PARAMETER(lpReserved);
if (dwReason == DLL_PROCESS_ATTACH)
{
TRACE0("UTLADO.DLL Initializing!\n");
g_bfr = new char[1000]();
memFile.Attach((BYTE*)g_bfr, 1000 );
// Extension DLL one-time initialization
if (!AfxInitExtensionModule(AcnDll, hInstance))
return 0;
new CDynLinkLibrary(AcnDll);
}
else if (dwReason == DLL_PROCESS_DETACH)
{
TRACE0("UTLADO.DLL Terminating!\n");
delete[] g_bfr;
// Terminate the library before destructors are called
AfxTermExtensionModule(AcnDll);
}
return 1; // ok
}
The problem is when I use the memFile in the dll to write log to memory, somewhere down the road, it becomes bad as if newly declared (uninitialized). See the where file positions/size are all reset.
What makes thing weirder is when I set breakpoint in DllMain, inside case DLL_PROCESS_ATTACH, it never breaks there (like never called) but the initialization does work! Breakpoint in case DLL_PROCESS_DETACH does work and is called only when I close application.
So, in a nutshell, it appears that memFile gets created another time during course of the application but should it? How can I make sure I only have one instance of the global variable in the dll?
After you have finished using the DLL, try using the FreeLibrary function.
However, this method has not been considered for concurrent use.
enter link description here

Easyhook: unmanaged hooking, how to call original function / change return status?

So I have a hook function at winspool.drv!WritePrinter, which is successfully hooked with unmanaged C++ remotely injected to spoolsv.exe.
Currently, the hook seems to either replace original function, or corrupt the stack in an undetectable way: after hooking, WritePrinter calls result in no printer activity outside the hook.
I've figured out there's at least one way to call original function, so-called LhGetOldProc. However, using it leads to crashes, don't sure if this is easyhook-related error or it's just bad casting.
So, how do I properly call original function in Easyhook unmanaged version?
Hook callback with LhGetOldProc:
UCHAR *uc = NULL;
LhGetOldProc(hhW, &uc);
typedef BOOL (*wp)(_In_ HANDLE, _In_ LPVOID, _In_ DWORD cbBuf, _Out_ LPDWORD);
wp my_wp = reinterpret_cast<wp>(reinterpret_cast<long>(uc)); // http://stackoverflow.com/questions/1096341/function-pointers-casting-in-c
BOOL res ;
if (my_wp == 0x0) {
return -1;
} else {
res = my_wp(hPrinter, pBuf, cbBuf, pcWritten); // crash
}
Hook code:
HMODULE hSpoolsv = LoadLibraryA("winspool.drv");
TRACED_HOOK_HANDLE hHook = new HOOK_TRACE_INFO();
NTSTATUS NtStatus;
UNICODE_STRING* NameBuffer = NULL;
HANDLE hRemoteThread;
FORCE(LhInstallHook(GetProcAddress(hSpoolsv, "WritePrinter"), WritePrinterHookA, 0x0, hHook));
ULONG ACLEntries[1] = { (ULONG) - 1 };
FORCE(LhSetExclusiveACL(ACLEntries, 1, hHook));
hhW = hHook;
TIL: in 2013, CodePlex (where EasyHook discussion list is) doesn't accept third level domains for e-mail when registering with Microsoft account. Not going to use Firebug to bypass the form.
The stack gets corrupted because your function pointer has the wrong calling convention.
The default calling convention is __cdecl which expects the caller to clean the stack.
typedef BOOL (* wp)(_In_ HANDLE ....);
equals:
typedef BOOL (__cdecl* wp)(_In_ HANDLE ...);
but the winapi functions use __stdcall calling convention which expects the callee to clean the stack.
you will have to typedef a __stdcall function:
typedef BOOL (__stdcall* wp)(_In_ HANDLE ....);

Unitialized Class Calling Function, No exception. What is going on here?

Is this normal behavior; this never happened to me before. I assume it would cause an exception but why doesn't it here? Take a look.
CWindowsApplication::MsgProc(HWND hWnd, UINT message, WPARAM wParam, LPARAM lParam)
{
static CWindowApplication* pApp = NULL;
if (message == WM_NCCREATE)
{
//// retrieve Window instance from window creation data and associate
//pApp = reinterpret_cast(((LPCREATESTRUCT)lParam)->lpCreateParams);
//::SetWindowLong(hWnd, GWL_USERDATA, reinterpret_cast(pApp));
//pApp = reinterpret_cast(::GetWindowLong(hWnd, GWL_USERDATA));
}
pApp->WndProc(hWnd, message, wParam, lParam); // pApp = NULL, but it still works? I expected a exception of some sort.
}
But, when I change the class to something else I get the exception I was expecting.
What is going on here? Never in my 10+ years as an enthusiastic programmer have I ever came across something like this.
As long as WndProc is not virtual, the pointer technically doesn't need to be dereferenced at all in order to make the call. That's not to say it won't crash and burn when you try to use this (including calling any virtual function with an implicit this) inside WndProc, but non-virtual calls go by the type of the pointer, and don't need to touch the vtable (or any other instance member).
All you're doing is invoking undefined behaviour. That means that it can appear to work, it can crash, or whatever the compiler feels like making it do.

ESP error when sending window messages between threads

I have an Observer class and a Subscriber class.
For testing purposes, the observer creates a thread that generates fake messages and calls CServerCommandObserver::NotifySubscribers(), which looks like this:
void CServerCommandObserver::NotifySubscribers(const Command cmd, void const * const pData)
{
// Executed in worker thread //
for (Subscribers::const_iterator it = m_subscribers.begin(); it != m_subscribers.end(); ++it)
{
const CServerCommandSubscriber * pSubscriber = *it;
const HWND hWnd = pSubscriber->GetWindowHandle();
if (!IsWindow(hWnd)) { ASSERT(FALSE); continue; }
SendMessage(hWnd, WM_SERVERCOMMAND, cmd, reinterpret_cast<LPARAM>(pData));
}
}
The subscriber is a CDialog derived class, that also inherits from CServerCommandSubscriber.
In the derived class, I added a message map entry, that routes server commands to the subscriber class handler.
// Derived dialog class .cpp
ON_REGISTERED_MESSAGE(CServerCommandObserver::WM_SERVERCOMMAND, HandleServerCommand)
// Subscriber base class .cpp
void CServerCommandSubscriber::HandleServerCommand(const WPARAM wParam, const LPARAM lParam)
{
const Command cmd = static_cast<Command>(wParam);
switch (cmd)
{
case something:
OnSomething(SomethingData(lParam)); // Virtual method call
break;
case // ...
};
}
The problem is, that I see strange crashes in the HandleServerCommand() method:
It looks something like this:
Debug Error!
Program: c:\myprogram.exe
Module:
File: i386\chkesp.c
Line: 42
The value of ESP was not properly
saved across a function call. This is
usually the result of calling a
function declared with one calling
convention with a function pointer
declared with a different calling
convention.
I checked the function pointer that AfxBeginThread() wants to have:
typedef UINT (AFX_CDECL *AFX_THREADPROC)(LPVOID); // AFXWIN.H
static UINT AFX_CDECL MessageGeneratorThread(LPVOID pParam); // My thread function
To me, this looks compatible, isn't it?
I don't know, what else I have to look for. Any ideas?
I made another strange observation, that might be related:
In the NotifySubscribersmethod, I call IsWindow() to check if the window to which the handle points, exists. Apparently it does. But calling CWnd::FromHandlePermanent() returns a NULL pointer.
From afxmsg_.h:
// for Registered Windows messages
#define ON_REGISTERED_MESSAGE(nMessageVariable, memberFxn) \
{ 0xC000, 0, 0, 0, (UINT_PTR)(UINT*)(&nMessageVariable), \
/*implied 'AfxSig_lwl'*/ \
(AFX_PMSG)(AFX_PMSGW) \
(static_cast< LRESULT (AFX_MSG_CALL CWnd::*)(WPARAM, LPARAM) > \
(memberFxn)) },
So the signature is LRESULT ClassName::FunctionName(WPARAM, LPARAM), while yours is void ClassName::FunctionName(const WPARAM, const LPARAM). This should not compile, at least under VS2008 it doesn't.
What is your HandleServerCommand declaration in the CServerCommandSubscriber class (in the header file)?
To me, this looks compatible, isn't
it?
Syntactically it looks that way.
I don't know, what else I have to look
for. Any ideas?
Yes: I've had the same problem when compiling a plugin library with debug settings and used in a Release-compiled application.
Basically, the problem looks like a stack corruption.
Since you're running NotifySubscribers in a separate thread, consider using PostMessage (or PostThreadMessage) instead of SendMessage.
This may not be the actual cause of the crash, but the change should be made anyway (as you're switching threading contexts by using SendMessage with no guarding of the data whatsoever.
I eventually decided to do it without window messages and am now posting my workaround here. Maybe it will help someone else.
Instead of letting the observer post window messages to its subscribers, I let the observer put data into synchronized subscriber buffers. The dialog class subscriber uses a timer to periodically check its buffers and call the apropriate handlers if those aren't empty.
There are some disadvantages:
It's more coding effort because for each data type, a buffer member needs to be added to the subscriber.
It's also more space consuming, as the data exists for each subscriber and not just once during the SendMessage() call.
One also has to do the synchronization manually instead of relying on the observer thread being suspended while the messages are handled.
A - IMO - huge advantage is that it has better type-safety. One doesn't have to cast some lParam values into pointers depending on wParam's value. Because of this, I think this workaround is very acceptable if not even superior to my original approach.