GetWindowLongPtr returns garbage [closed] - c++

Question

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 6 years ago.
Improve this question
I am developing a Win32 dialog box wrapper.
.h file
class dlg {
static INT_PTR CALLBACK DlgProcTmp(HWND hwnd,
UINT wm, WPARAM wp, LPARAM lp);
INT_PTR CALLBACK DlgProc(HWND hwnd, UINT wm, WPARAM wp, LPARAM lp);
bool ismodal;
protected:
HWND hwndDlg;
int id;
public:
virtual void oncreate(const widget &w) { }
virtual void oncmd(const widget &w, int code, int idCntrl, HWND hwnd) { }
virtual void onclose(const widget &w) { }
dlg() { }
dlg(int id);
INT_PTR domodal(HWND hwndOwner = nullptr);
widget domodeless(HWND hwndOwner = nullptr, int cmdshow = SW_SHOW);
};
.cpp file
INT_PTR dlg::DlgProcTmp(HWND hwnd, UINT wm, WPARAM wp, LPARAM lp)
{
dlg *This;
if (wm == WM_INITDIALOG) {
This = (dlg *) lp;
setwinlong(hwnd, DWLP_USER, This);
This->oncreate(widget(hwnd));
return TRUE;
}
if ((This = getwinlong<dlg *>(hwnd, DWLP_USER)) != nullptr)
return This->DlgProc(hwnd, wm, wp, lp);
return FALSE;
}
INT_PTR dlg::DlgProc(HWND hwnd, UINT wm, WPARAM wp, LPARAM lp)
{
this->hwndDlg = hwnd;
switch (wm) {
case WM_COMMAND:
this->oncmd(widget(hwnd), HIWORD(wp), LOWORD(wp), (HWND) lp);
if (this->ismodal)
EndDialog(hwnd, LOWORD(wp));
else
DestroyWindow(hwnd);
return TRUE;
case WM_DESTROY:
this->onclose(widget(hwnd));
return TRUE;
}
return FALSE;
}
The dialog is created in domodal with a DialogBoxParam call. The last argument is the this pointer, which I then retrieve from the lparam of the WM_INITDIALOG message. To allow this to be used between different messages, I save this with the HWND in WM_INITDIALOG. However, whenever a message not WM_INITDIALOG arrives, and I get the this pointer with GetWindowLongPtr, it returns a garbage dlg class, whose vtable is corrupt. I use the vtable to call the correct handler function. As a result, my code crashes on the first line of the WM_COMMAND handler. Here is what the debugger shows as the value of this:
Why is GetWindowLongPtr returning garbage?
BTW, getwinlong is a wrapper for GetWindowLongPtr, and setwinlong is a wrapper for SetWindowLongPtr. Here are their implementations:
template <class TYPE> static TYPE getwinlong(HWND hwnd, int idx)
{
return (TYPE) GetWindowLongPtr(hwnd, idx);
}
template <class TYPE> static void setwinlong(HWND hwnd, int idx, TYPE val)
{
SetWindowLongPtr(hwnd, idx, (LONG_PTR) val);
}
I have seen the numerous posts about how GetWindowLongPtr will fail on Win64, if you cast to LONG instead of LONG_PTR, like https://blogs.msdn.microsoft.com/oldnewthing/20131226-00/?p=2263. I believe that my problem is different.
Edit: #andlabs wanted the code that creates the dialog:
INT_PTR dlg::domodal(HWND hwndOwner)
{
this->ismodal = true;
return DialogBoxParam(gethinst(hwndOwner),
MAKEINTRESOURCE(id), hwndOwner, dlg::DlgProcTmp,
(LPARAM) this);
}

Answer 1

I pieced together the code that you've shown us, and was able to create a working sample:
dlg.h
#pragma once
#include <Windows.h>
class dlg {
static INT_PTR CALLBACK DlgProcTmp(HWND hwnd,
UINT wm, WPARAM wp, LPARAM lp);
INT_PTR CALLBACK DlgProc(HWND hwnd, UINT wm, WPARAM wp, LPARAM lp);
bool ismodal;
protected:
HWND hwndDlg;
int id;
public:
//virtual void oncreate(const widget &w) { }
//virtual void oncmd(const widget &w, int code, int idCntrl, HWND hwnd) { }
//virtual void onclose(const widget &w) { }
dlg() { }
dlg(int id) : id(id) { }
INT_PTR domodal(HWND hwndOwner = nullptr);
//widget domodeless(HWND hwndOwner = nullptr, int cmdshow = SW_SHOW);
};
dlg.cpp
#include "dlg.h"
template <class TYPE> static TYPE getwinlong(HWND hwnd, int idx)
{
return (TYPE) GetWindowLongPtr(hwnd, idx);
}
template <class TYPE> static void setwinlong(HWND hwnd, int idx, TYPE val)
{
SetWindowLongPtr(hwnd, idx, (LONG_PTR) val);
}
INT_PTR dlg::DlgProcTmp(HWND hwnd, UINT wm, WPARAM wp, LPARAM lp)
{
dlg *This;
if (wm == WM_INITDIALOG) {
This = (dlg *) lp;
setwinlong(hwnd, DWLP_USER, This);
//This->oncreate(widget(hwnd));
return TRUE;
}
if ((This = getwinlong<dlg *>(hwnd, DWLP_USER)) != nullptr)
return This->DlgProc(hwnd, wm, wp, lp);
return FALSE;
}
INT_PTR dlg::DlgProc(HWND hwnd, UINT wm, WPARAM wp, LPARAM lp)
{
this->hwndDlg = hwnd;
switch (wm) {
case WM_COMMAND:
//this->oncmd(widget(hwnd), HIWORD(wp), LOWORD(wp), (HWND) lp);
if (this->ismodal)
EndDialog(hwnd, LOWORD(wp));
else
DestroyWindow(hwnd);
return TRUE;
case WM_DESTROY:
return TRUE;
}
return FALSE;
}
INT_PTR dlg::domodal(HWND hwndOwner)
{
this->ismodal = true;
return DialogBoxParam(GetModuleHandle(NULL),
MAKEINTRESOURCE(id), hwndOwner, dlg::DlgProcTmp,
(LPARAM) this);
}
main.cpp
int APIENTRY _tWinMain(HINSTANCE, HINSTANCE, LPTSTR, int)
{
dlg myDialog(IDD_DIALOG);
myDialog.domodal();
return 0;
}
You'll notice that there's a fair bit of the code that has been commented out—in particular, the declarations for and calls to the virtual message-handling functions (onXXX) have been omitted. I did this because I didn't have the definition for the widget type, and I didn't think it was germane to your actual problem.
Turns out it must be, since the code works fine so long as those virtual message-handling functions are not called.
I then added in a stub class for widget as follows:
class widget
{
public:
widget(HWND hWnd)
: m_hWnd(hWnd)
{ }
HWND m_hWnd;
};
and uncommented the oncmd function as well as the call to it in the WM_COMMAND handler. Again, the code compiles and works correctly. So I have no idea what problem you're experiencing. It must be in the widget class or some other code you have not shown to us.
There is at least one tweak I would make to the code. Assign the hwndDlg member variable inside of the DlgProcTemp function, rather than waiting until the DlgProc function. So, do this:
INT_PTR dlg::DlgProcTmp(HWND hwnd, UINT wm, WPARAM wp, LPARAM lp)
{
dlg *This;
if (wm == WM_INITDIALOG) {
This = (dlg *) lp;
setwinlong(hwnd, DWLP_USER, This);
This->hwndDlg = hwnd;
This->oncreate(widget(hwnd));
return TRUE;
}
if ((This = getwinlong<dlg *>(hwnd, DWLP_USER)) != nullptr)
return This->DlgProc(hwnd, wm, wp, lp);
return FALSE;
}
INT_PTR dlg::DlgProc(HWND hwnd, UINT wm, WPARAM wp, LPARAM lp)
{
switch (wm) {
case WM_COMMAND:
this->oncmd(widget(hwnd), HIWORD(wp), LOWORD(wp), (HWND) lp);
if (this->ismodal)
EndDialog(hwnd, LOWORD(wp));
else
DestroyWindow(hwnd);
return TRUE;
case WM_DESTROY:
return TRUE;
}
return FALSE;
}
Also strongly consider using C++-style casts in preference to C-style casts.
If you are still having a problem getting the code to work, you will have to edit into your question a Minimal, Complete, and Verifiable example, just like I have done here.