How to get post-call parameters of hooked winapi function? - c++

For this example i am talking about hooking BeginPaint() which i hooked fine and i can normally access everything pre-call...
HDC WINAPI Mine_BeginPaint(HWND hWnd, LPPAINTSTRUCT lpPaint)
{
// do stuff here, inspecting PRE-CALL lppaint struct etc...
return fpBeginPaint(hWnd, lpPaint);
}
I am looking for a way to inspect this lpPaint struct post-call, how could i do this?

HDC WINAPI Mine_BeginPaint(HWND hWnd, LPPAINTSTRUCT lpPaint){
// do stuff here, inspecting PRE-CALL lppaint struct etc...
HDC result = fpBeginPaint(hWnd, lpPaint);
//inspect here whatever you want.
return result;
}

Related

Port Win32/WinAPI application to wxWidgets

I'm currently using Windows 10, MSVC v142 (with VS2019) and wxWidgets 3.1.3. I have an old Windows C++ application that uses WinAPI for its GUI features, i.e Windows message loop, using "CreateWindow", and having to "manually" create all window procedures and event handling.
I want to improve this application by gradually replacing the UI using wxWidgets so I don't have to start over from scratch. I would implement new, independent UI features in wxWidgets (e.g specific dialogs), and then work my way back and replace all the old UI code with a wxWidgets implementation, without having to break the app along the way.
Below is a skeleton of how my app is currently set up:
LRESULT CALLBACK MainWndProc(HWND hwnd, UINT uMsg, WPARAM wParam, LPARAM lParam)
{
// Handling window messages, e.g menus, buttons, etc.
}
int __stdcall WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, char* lpCmdLine, int nCmdShow)
{
// Initialize resources, register main window class using MainWndProc, etc.
// ...
HWND mainwnd = CreateWindow(/* CreateWindow args... */);
do
{
MSG msg = {};
while (PeekMessage(&msg, nullptr, 0, 0, PM_REMOVE))
{
TranslateMessage(&msg);
DispatchMessage(&msg);
}
// Per-frame application logic
// ...
} while (msg.message != WM_QUIT);
// Clean up resources
// ...
return 0;
}
How would I need to modify this so that all the WinAPI objects continue to function, but I can now also create windows using wxWidgets? I previously tried replacing the message loop above by initializing wxWidgets through a custom class derived from wxApp, but my application kept crashing during the cleanup code (which it sometimes wouldn't even reach).
EDIT: I managed to make it work, updated skeleton can be found below.
// Main WinAPI window procedure
LRESULT CALLBACK MainWndProc(HWND hwnd, UINT uMsg, WPARAM wParam, LPARAM lParam)
{
switch (uMsg)
{
// Code to handle other messages
// ...
case WM_CLOSE:
// Closing this window should shut down the app
PostQuitMessage(0);
break;
}
return DefWindowProc(hWnd, uMsg, wParam, lParam);
}
class MyApp : public wxApp
{
public:
virtual bool OnInit() override
{
if(!Old_Init())
{
// Perform cleanup in case something goes wrong
Old_Exit();
return false;
}
// Wrap the WinAPI window in a dummy wxWindow
m_dummyMainWindow = new wxWindow();
m_dummyMainWindow->SetHWND(m_mainWnd);
return true;
}
int OnExit() override
{
// Unset the dummy window HWND and delete it (is this necessary?)
m_dummyMainWindow->SetHWND(NULL);
delete m_dummyMainWindow;
// Clean up everything else
return Old_Exit();
}
private:
bool Old_Init()
{
// Perform the old initialization
// ...
m_mainWnd = CreateWindow(/* CreateWindow args... */);
if(m_mainWnd)
{
return true;
}
else
{
return false;
}
}
int Old_Exit()
{
// Perform the old cleanup (previously done after exiting the Windows message loop)
// ...
return 0;
}
HWND m_mainWnd;
wxWindow* m_dummyMainWindow;
};
wxIMPLEMENT_APP_NO_MAIN(MyApp);
// App entrypoint
int __stdcall WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, char* lpCmdLine, int nCmdShow)
{
if (wxEntry())
{
// Exit if something goes wrong (this might not be the correct way to do it?)
wxExit();
}
return 0;
}
This does seem to function, all the old UI elements are working as before, but I am not 100% sure this is a stable solution. My debugger is warning me about memory leaks, which appear to increase in number when I activate UI elements (e.g open and close dialogs). I suspect WinAPI resources might not be cleaned up correctly. Am I missing anything?
EDIT2: I did some more debugging, and the code in my original app (i.e without wxWidgets) causes those "memory leaks" as well, and I can't replicate it in a minimal working example, so I suspect the issue is not related to wxWidgets at all. I am therefore confident that the approach described above should solve my problem, but I would not mind a second opinion.
I recommend looking at the MFC sample to see a working example of something close to what you want to do. MFC is, of course, not quite the same as Win32 API, but it should still be a good starting point.
Notably it shows how to pass the messages to wx event loop when you're running your own one (this part is in wxMFCApp class in include/wx/msw/mfc.h). Of course, if you can switch to running wx event loop, it would be even simpler.
You may also find it helpful to know that you can wrapp HWNDs you create into wxWindows using AssociateHandle() and handle their messages by overriding MSWHandleMessage() or even just using wxNativeWindow directly.
Good luck!

How to use EnumWindows with a COM wrapper in VBScript

I am writing a Windows API COM wrapper to be used in VBScript, and came up with a problem with callback controlled functions such as EnumWindows.
This is what I have currently:
STDMETHODIMP CWinAPI::WinAPI_EnumWindows(BSTR lpEnumFunc, int lParam, int *Result)
{
*Result = int(EnumWindows(WNDENUMPROC(lpEnumFunc), lParam))
return S_OK;
}
But, as I expected this didn't work and when I use this with VBScript, it crashes.
How can I export EnumWindows from my COM wrapper to use in VBScript and will VBScript support using it? If so, how can I use EnumWindowsProc callback function in VBScript?
UPDATE
BSTR WindowClassName = SysAllocString(L"");
WCHAR WindowTitle[8192];
BOOL CALLBACK EnumWindowsProc(
__in HWND hWnd,
__in LPARAM lParam
)
{
RealGetWindowClass(hWnd, WindowClassName, 100);
GetWindowText(hWnd, WindowTitle, 8192);
if (0 == wcscmp(WindowClassName, L"#32768"))
{
//<< NOW, HWND need to be caught by WSH and then show it. (I.E. WScript.Echo HWND) >>
}
return TRUE;
}
STDMETHODIMP CWinAPI::WinAPI_EnumWindows(int lParam)
{
EnumWindows(EnumWindowsProc, lParam);
return S_OK;
}
Dim WINAPI: Set WINAPI = WScript.CreateObject("WinAPIWrapperLib.WINAPI")
WINAPI.WinAPI_EnumWindows 0
Now I did above, but instead of this, I need WSH to know if a handle to specified window type is found and return HWND internally and show it from WSH, not directly from C++.(such as via a Message Box).
This function's lParam parameter can be set from VBScript, but how can I set EnumWindowsProc? Can it be done using GetRef or something similar in VBSript? But GeRef allows only one parameter. I want this EnumWindowsProc callback function also to be declared in VBScript, otherwise wrapping this function becomes useless.
I read this post from #Alex.K's comment and found it very useful, but I can't understand how to use it with EnumWindows's callback EnumWindowsProc.
Please help me more on this.

Do CustomDialogProc have to be static. WinAPI

I've been looking at creating a custom control with WinApi for my application, and I have made a class which contains the CustomDialogProc and CreateWindowEx and RegisterClass() functions.
I can set a breakpoint inside the CustomDialogProc and it hits, so the class is registered correctly.
However, I have to declare the CustomDialogProc function as static int he header of my class
static LRESULT CALLBACK CustomDialogProc(HWND hWnd, UINT uMsg, WPARAM wParam,LPARAM lParam);
If I don't set it to static, I get the error
Error C3867 'CustomControl::CustomDialogProc': non-standard syntax; use '&' to create a pointer to member
IS this necessary, this requires all my controls created within this control to be static as well. What if I want multiple instances of this control?
How can I get around this? The main MsgProc doesn't seem to be a static function. Neither is the UpDownDialogProc in the first link shown below
Below is my code for CustomControl.h in case anyone needs it.
Put together from code found at:
https://msdn.microsoft.com/en-us/library/windows/desktop/hh298353(v=vs.85).aspx
https://www.codeproject.com/Articles/485767/True-Windows-control-subclassing
Thanks,
#pragma once
#include <windows.h>
#include <commctrl.h>
#pragma comment(lib, "comctl32.lib")
class CustomControl
{
public:
CustomControl();
~CustomControl();
LRESULT CALLBACK CustomDialogProc(HWND hWnd, UINT uMsg, WPARAM wParam,LPARAM lParam)
{
switch (message)
{
case WM_CREATE:
//DO STUFF HERE
break;
}
}
bool CreateControl(HWND hwnd, HINSTANCE* m_hApp_instance)
{
g_hInst = m_hApp_instance;
RegisterSubClass(*g_hInst, WC_LISTBOX, TEXT("CustomControl"), CustomDialogProc);
HWND hwndCustom = CreateWindow(TEXT("CustomControl"), NULL, WS_CHILD | WS_VISIBLE,
0, 0, 0, 0, hwnd, (HMENU)100, *g_hInst, NULL);
return true;
}
private:
HINSTANCE* g_hInst;
WNDPROC RegisterSubClass(HINSTANCE hInstance, LPCTSTR ParentClass, LPCTSTR ChildClassName, WNDPROC ChildProc) {
WNDCLASSEX twoWayStruct;
WNDPROC parentWndProc;
if (GetClassInfoEx(NULL, ParentClass, &twoWayStruct)) {
parentWndProc = twoWayStruct.lpfnWndProc; // save the original message handler
twoWayStruct.cbSize = sizeof(WNDCLASSEX); // does not always get filled properly
twoWayStruct.hInstance = hInstance;
twoWayStruct.lpszClassName = ChildClassName;
twoWayStruct.lpfnWndProc = ChildProc;
/* Register the window class, and if it fails return 0 */
if (RegisterClassEx(&twoWayStruct))
return parentWndProc; // returns the parent class WndProc pointer;
// subclass MUST call it instead of DefWindowProc();
// if you do not save it, this function is wasted
}
return 0;
}
};
The most common way is to use SetWindowLongPtr to store a pointer to the object associated with the window handle.
HWND hWnd = CreateWindow(...);
SetWindowLongPtr(hWnd, GWLP_USERDATA, (LONG_PTR) this);
And then in your dialog proc, get that pointer and call into your class:
// this static method is registered with your window class
static LRESULT CALLBACK CustomDialogProcStatic(HWND hWnd, UINT uMsg, WPARAM wParam,LPARAM lParam)
{
auto pThis = (CustomControl*) GetWindowLongPtr(hWnd, GWLP_USERDATA);
if (pThis != NULL)
return pThis->CustomDialogProcInstance(hWnd, uMsg, wParam, lParam);
return DefWindowProc(hWnd, uMsg, wParam, lParam);
}
// this instance method is called by the static method
LRESULT CustomDialogProcInstance(HWND hWnd, UINT uMsg, WPARAM wParam, LPARAM lParam)
{
...
}
Make sure you manage your window and class life cycle appropriately to prevent the window proc from calling a deleted object instance. In many cases, this is as simple as ensuring DestroyWindow is called if your class is destructed.
The Windows API is C language based. It knows nothing about C++, non-static member functions, objects, etc.
So yes, all of your functions that will communicate with the Windows API directly must be static class member functions, or non-class / global / "free" functions.
That doesn't preclude you from writing a C++ wrapper for a C API. That's what libraries such as Microsoft's MFC or the old Borland OWL libraries accomplish. Other independent groups have also written wrappers for the Windows API.
Note that these differing libraries accomplish the goal of hooking a C based API to C++ in different ways. One is to use the SetWindowLongPtr method mentioned in the answer given by #MichaelGunter. Another method is to use maps to associate window handles and static Window procedures.
I would suggest before you try this on your own (creating a wrapper), you investigate how others have done this already, and choose the best approach that fits. Another suggestion is that before you even create a wrapper, you should know the C based API on much more than a cursory level. You need advanced to expert knowledge of any C API you plan to create a C++ wrapper for if you want the wrapper to work relatively flawless under different scenarios.

Is it possible to give a top-level function access to an object's members in C++?

So I'm writing some wrapper classes for GUI programming in Win32. I'm starting with a Window class, and so far it contains a MainLoop method that is basically a clone of the standard Win32 WinMain function. That way, one can do something like this:
int WINAPI WinMain(HINSTANCE hInst, HINSTANCE hPrev, LPSTR szCmdLine, int nCmdShow) {
Window *win = new Window();
// Do all your widget creation and add it to the window object...
return win->MainLoop(hInst, hPrev, szCmdLine, nCmdShow);
}
Inside the window object's MainLoop method, it must create the new Win32 window by setting its lpfnWndProc member. This member, as any Win32 programmer knows, is a function pointer to a specifically defined WndProc function. The problem is, if I were to create a WndProc function, I would need access to that window object's members (so that it knew what to draw on the window, etc.). This leaves me two options (that I know of):
I can define WndProc at the top level, but that cuts off access to the object's members.
I can define it as a class method, but then it's not the exact function type that lpfnWndProc asks for, so I can't set it!
Can anyone help me unravel this catch-22?
You could also make it a static member function. :)
Anyways, a solution depends on if you need only one window or if you need multiple windows.
First a solution for single windows:
// in .h
class Window{
public:
static LRESULT WINAPI MessageProc(HWND hWnd, UINT msg, WPARAM wParam, LPARAM lParam);
LRESULT InternalMessageProc(HWND hWnd, UINT msg, WPARAM wParam, LPARAM lParam);
// ...
};
// in .cpp
#include "Window.h"
Window* global_window = 0;
Window::Window(/*...*/){
if(!global_window)
global_window = this;
else
// error or exception... or something else
}
LRESULT WINAPI Window::MessageProc(HWND hWnd, UINT msg, WPARAM wParam, LPARAM lParam){
return global_window->InternalMessageProc(hWnd, msg, wParam, lParam);
}
Now if you want to allow multiple windows, use a std::map (or if your compiler supports std::unordered_map).
Edit: This solution comes with some subtle problems. As #Ben Voigt points out in his comment, you get a chicken and egg problem as the MessageProc is called inside of CreateWindow(Ex), but only after the CreateWindow(Ex) call you have the window handle. Here's a solution based on Ben's next comment (thanks!):
// Window.h stays the same
// in .cpp
#include "Window.h"
#include <map>
std::map<HWND, Window*> window_map;
Window* currently_created_window = 0;
Window::Window(){
currently_created_window = this;
window_handle = CreateWindow(/*...*/);
}
LRESULT WINAPI Window::MessageProc(HWND hWnd, UINT msg, WPARAM wParam, LPARAM lParam){
// if the key 'hWnd' doesn't exist yet in the map
// a new key-value pair gets created and the value gets value-initialized
// which, in case of a pointer, is 0
if(window_map[hWnd] == 0){
// window doesn't exist yet in the map, add it
window_map[hWnd] = currently_created_window;
}
window_map[hWnd]->InternalMessageProc(hWnd, msg, wParam, lParam);
}
Be cautious though, as the above example isn't thread-safe. You need to mutex-lock the creation of the window:
Window::Window(/*...*/){
Lock lock_it(your_mutex);
currently_created_window = this;
window_handle = CreateWindow(/*...*/);
lock_it.release();
// rest of the initialization
}
The above should do for the thread-safety (I hope).
You need to create window map and when you create new window just add it to this global map. You can use simple linked list instead of course.
map<HWND, Window *> wndmap;
LRESULT CALLBACK WndProc(HWND hwnd, UINT Message, WPARAM wparam, LPARAM lparam)
{
Window *pWnd = wndmap [hwnd];
....
}
WndProc cannot be an instance member function, because Windows will not pass any hidden this parameter. It can be namespace scope or a static member.
One simple solution is to use a map<HWND, Window*> to find the object, and then forward parameters to a method on the object.
Note that WndProc can maintain the map itself, since CreateWindow provides an opaque user parameter that shows up in WM_CREATE which is useful for carrying the Window *, and then you remove the entry in WM_DESTROY.
Define your WndProc as a static class member - this will then be compatible (for all compilers I'm aware of) with non-member function pointer, such as those used in Win32 programming.
But I have to say that this is a bit of a waste of time - there are a zillion Windows class libraries out there, and I don't think the world really needs another one.
Sounds you need to declare the function without defining it. That's what prototypes are for.
class Object;
void f(Object* o);
class Object {
public:
...
void some_method() {
... &f ...
}
void another_method() {
...
}
...
};
void f(Object* o) {
...
o->another_method();
...
}
The other way around might also be possible.
class Object {
public:
...
void some_method();
void another_method();
...
};
void f(Object* o) {
...
o->another_method();
...
}
void Object::some_method() {
... &f ...
}
void Object::another_method() {
...
}

Pointer to a callback in a DLL

I'm calling a Dialog that has it's resources and dlg-procedure in a DLL.
I'm not using a DEF file or LIB file. The function names are known, the function args are known, I'm using GetProcAddress to get a pointer to the functions I'm interested in.
'Why' I'm doing it this way is of no consequence, it's an experiment in 'learning'.
This works on _cdecl functions, but on the CALLBACK (_stdcall), I can't get a pointer to the actual dialog procedure (it returns 0).
Here's how I'm doing my pointers:
//////////////////// DLL TEST STUFF ////////////////////////////
#define DLLPATH "../../testdll/release/testdll.dll"
//typedef some function pointers
typedef HINSTANCE (__cdecl *INSTPROC )(void);
typedef int (__cdecl *VOIDPROC )(void);
typedef LRESULT (__stdcall *DLGROC )(HWND, UINT, WPARAM, LPARAM );
///////////////////////////////////////////////////////////////
As I said, any function that is NOT a callback returns a valid result, additionally, the dialog pops up as expected, no problems with DLL's HINSTANCE.
But without a pointer to it's dlgproc, no button messages have anywhere to go.
Here's some more test code:
//use DLL funcs without lib or def files
//this works
GetInst = (INSTPROC)GetProcAddress(Loadme,"getinst");
//this works
GetID = (VOIDPROC)GetProcAddress(Loadme,"getid");
//this doesn't work, rets 0
DlgProc = (DLGPROC) GetProcAddress(Loadme,"dllProc");
//test for result
dllid =(GetID)();
dllinst=(GetInst)();
//compare hinst OK
wsprintf(buf,"dllinst=%x Loadme=%x",dllinst, Loadme);
MessageBox(hwnd,buf,"",MB_OK);
//check resOurce ID OK
wsprintf(buf,"GetID returned: %d",dllid);
MessageBox(hwnd,buf,"",MB_OK);
//check dllProc addr NOGO, ret=0
wsprintf(buf,"dllProc=%x",DlgProc);
MessageBox(hwnd,buf,"",MB_OK);
// DLL instance, resource ID, parent, dlgproc
DialogBox(Loadme , MAKEINTRESOURCE(dllid), hwnd, DlgProc);
//dialog loads and shows, can't get dlgproc addr
FreeLibrary(Loadme);
///////////////// END DLL TEST STUFF ///////////////
On the DLL side of things, it looks like this:
__declspec(dllexport) LRESULT CALLBACK dllProc(
HWND hwnd,
UINT Message,
WPARAM wParam,
LPARAM lParam
)
{
hpwnd=hwnd;
switch (Message){
case WM_INITDIALOG:
MessageBox(hwnd,"At DlgProc...","",MB_OK);
return TRUE;
case WM_COMMAND:
switch(LOWORD(wParam)){
case IDEND:
case IDCANEND:
EndDialog(hwnd,0);
return TRUE;
}
return TRUE;
}
return FALSE;
}
Which really doesn't matter at this point, since I can't get a pointer to the callback in the first place.
If anyone happens to know the fix for getting a pointer to the callback, I would very much appreciate your input.
Thanks for your patience.
You need to declare your dllProc inside an extern "C" block to prevent its name being decorated when exported, so that when you ask for a function called "dllProc" such a function is found (instead of "dllProc#blahmoocow" or whatever madness the C++ decoration produces. :-)).
Place this declaration before the dllProc definition:
extern "C"
{
__declspec(dllexport) LRESULT CALLBACK dllProc(
HWND hwnd, UINT Message, WPARAM wParam, PARAM lParam);
}
You can also then drop the __declspec(dllexport) from the dllProc definition.