Ending nested message loop of a custom dialog - c++

Question

I have a dialog class made from dialog template in memory which runs it's own nested message loop (in addition to main app msg loop).
To create a dialog I use CreateDialogIndirectParamW which returns handle to the dialog window,
Since the dialog is completely customized, all the message processing and creation is done manually.
I omitted Dialog procedure, but basically it calls the below relevant message handlers, once the the dialog is closing down.
Note that, we can't use EndDialog function to destroy the dialog and end message loop because for CreateDialogIndirectParamW we must use DestroyWindow explicitly.
I'm looking for efficient way to end the dialog loop.
I tried to implement this loop but problem is that this approach will either consume too much CPU resources since the for loop in the sample code will just run silly until there is a message, or if for is omited then PeekMessage will stop the loop immediately, which is not what I wan't.
Relevant class declaration:
class Dialog :
public ContainerWindow,
public MessageWindow,
public SuperClassWindow
{
// ...
public:
/** Process messages for dialog window */
[[nodiscard]] int RunMessageLoop() override;
protected:
/** Handler for WM_NCDESTROY */
std::uint32_t OnNcDestroy(const UINT& uMsg, const WPARAM& wParam, const LPARAM& lParam) override;
/** Handler for WM_DESTROY */
inline void OnDestroy() const noexcept override;
/** Handler for WM_CLOSE */
inline virtual bool OnClose() noexcept;
// ...
protected:
HWND mhWnd; /** Window handle of derived component */
}
Simplified class definition:
std::uint32_t Dialog::OnNcDestroy(
[[maybe_unused]] const UINT& uMsg,
[[maybe_unused]] const WPARAM& wParam,
[[maybe_unused]] const LPARAM& lParam)
{
// ...
delete this; // note we can't set this pointer to nullptr!
// ...
return count;
}
void Dialog::OnDestroy() const noexcept
{
PostQuitMessage(0);
}
bool Dialog::OnClose() noexcept
{
return DestroyWindow(mhWnd);
}
And here is the message loop for a dialog:
I need to add some checking code into the loop to check if dialog is valid object, that is stop the loop somehow if the Dialog object has been deleted
Once the OnNcDestroy handler is called, IsDialogMessageW bellow will fail, see comment.
Looks like GetMessageW will continue running after WM_NCDESTROY is dispatched, the loop is still waiting for WM_QUIT sent by OnDestroy handler so the msg loop will continue running once the Dialog object is deleted, and that will make IsDialogMessageW(mhWnd, &msg) bellow fail. since mhWnd does not exist any more.
int Dialog::RunMessageLoop()
{
EnableWindow(mhWndParent, FALSE);
MSG msg{ };
BOOL result = FALSE;
while ((result = GetMessageW(&msg, nullptr, 0, 0)) != FALSE)
{
if (result == -1)
{
ShowError(ERR_BOILER); // error checking function.
break;
}
// once OnNcDestroy is called "mhWnd" is invalid memory
// and this will off course cause access violation!
if (!IsDialogMessageW(mhWnd, &msg))
{
TranslateMessage(&msg);
DispatchMessageW(&msg);
}
}
EnableWindow(mhWndParent, TRUE);
return static_cast<int>(msg.wParam);
}
note that we can't if (this) because this is not nullptr and can't be set to nullptr in OnNcDestroy handler.

Answer 1

OK, after some time experimenting I come out with the following change to the code which makes everything works as it should! (see comments for more info)
btw. I will leave this question open, hopefully there is more efficient way for this, currently IsWindow is called for every message which is maybe a performance issue.
int Dialog::RunMessageLoop()
{
EnableWindow(mhWndParent, FALSE);
MSG msg{ };
BOOL result = FALSE;
// first we save the handle to the window
HWND hwnd = mhWnd;
if (!IsWindow(hwnd)) // make sure the handle is valid dialog window
std::abort();
while ((result = GetMessageW(&msg, nullptr, 0, 0)) != FALSE)
{
if (result == -1)
{
ShowError(ERR_BOILER);
break;
}
// if object is deleted the handle is no longer valid!
// because prior to OnNcDestroy handler DestroyWindow has been called
if (!IsWindow(hwnd))
goto invalid;
if (!IsDialogMessageW(mhWnd, &msg))
{
invalid: // non dialog message processing
TranslateMessage(&msg);
DispatchMessageW(&msg);
}
}
// NOTE that we removed EnableWindow(mHwndParent) here!
return static_cast<int>(msg.wParam);
}
This on it's own will work but one additional step is to enable owner window (main window) before the dialog is destroyed, so update the handler:
bool Dialog::OnClose() noexcept
{
// otherwise first window in Z order (such as Desktop) will get focus!
EnableWindow(mhWndParent, TRUE);
return DestroyWindow(mhWnd);
}