SendNotifyMessage do not send correct messages - c++

From one thread I send the message to main thread in window procedure.
But it is unsuccessful. When I send messages from the same thread - all is ok
include "stdafx.h"
#include <Windows.h>
#include <atlbase.h>
#define MAX_THREADS 1
HWND m_wnd;
enum
{
EVENT_CALL = (WM_APP + 0x30),
};
static LRESULT CALLBACK function_call()
{
//some code
int test = 0;
return 0;
}
static LRESULT CALLBACK http_message_proc(HWND hWnd, UINT uMsg, WPARAM wParam, LPARAM lParam)
{
switch (uMsg)
{
case EVENT_CALL:
function_call();
return 0;
}
return ::DefWindowProc(hWnd, uMsg, wParam, lParam);
}
void CreateNotifyWnd()
{
WNDCLASSEX w = { 0 };
w.cbSize = sizeof(w);
w.hInstance = (HINSTANCE)&__ImageBase;
w.lpszClassName = L"uistone_http_event_wnd";
w.lpfnWndProc = http_message_proc;
::RegisterClassEx(&w);
int error = GetLastError();
m_wnd = ::CreateWindowEx(0, w.lpszClassName, NULL, 0, 0, 0, 0, 0, HWND_MESSAGE, NULL, w.hInstance, 0);
error = GetLastError();
}
DWORD WINAPI SendThread(void* request_param)
{
::SendNotifyMessage(m_wnd, EVENT_CALL, 11, 12);
int error = GetLastError();
return 0;
}
int main()
{
CreateNotifyWnd();
HANDLE hThreadArray[MAX_THREADS];
hThreadArray[0] = CreateThread(nullptr, 0, SendThread, nullptr, 0, nullptr);
//::SendNotifyMessage(m_wnd, EVENT_CALL, 11, 12);
WaitForMultipleObjects(MAX_THREADS, hThreadArray, TRUE, INFINITE);
return 0;
}
Why I can not catch messages from another thread?
Thanks.

This is documented behavior. This is the relevant part from the SendNotifyMessage documentation:
If the window was created by the calling thread, SendNotifyMessage calls the window procedure for the window and does not return until the window procedure has processed the message. If the window was created by a different thread, SendNotifyMessage passes the message to the window procedure and returns immediately; it does not wait for the window procedure to finish processing the message.
This appears to work when used with a window created on the same thread, because when you call SendNotifyMessage, the function synchronously calls into the window procedure associated with the target window before returning.
If the call crosses threads, on the other hand, you'd have to run a message loop for the - now queued - message to get picked up and passed to the window procedure1). Your application doesn't run a message loop, and it exits before the message ever reaches the target window.
To fix this you'd have to run a message loop. This may or may not be the right approach to your problem. Since we don't know, what problem you are trying to solve, we cannot suggest potentially superior approaches and solutions.
1) See About Messages and Message Queues: Message Routing.

Related

WinAPI: repeatable check in a separate thread that notifies UI thread when check fails

My application has a separate thread that repeatably performs some check. If the check fails, the UI thread is notified (a MessageBox is displayed that requires user action what to do next).
Unfortunately, I have to use C++03 compiler (Visual Studio 2010 SP1) and boost library usage is prohibited. Therefore, I cannot use <thread>, <atomic>, <chrono>, etc. Thats why I have to use CreateThread, PostMessage and other WinAPI functions.
Here is my UI thread code (simplified). My main window is CMDIFrameWnd (from MFC):
//a struct with all parameters that is needed for a repeatable check
struct RepeatFunctionParameters
{
unsigned int repeatDelayInMilliseconds;
HWND checkIsFailedPostMessageWindowHandler;
UINT checkIsFailedPostMessageMessageId;
HANDLE checkIsPausedMutexHandle;
RepeatFunctionParameters(unsigned int _repeatDelayInMilliseconds, HWND _checkIsFailedPostMessageWindowHandler,
UINT _checkIsFailedPostMessageMessageId, HANDLE _haspSerialCheckIsPausedMutexHandle)
: repeatDelayInMilliseconds(_repeatDelayInMilliseconds), checkIsFailedPostMessageWindowHandler(_checkIsFailedPostMessageWindowHandler),
checkIsFailedPostMessageMessageId(_checkIsFailedPostMessageMessageId), haspSerialCheckIsPausedMutexHandle(_haspSerialCheckIsPausedMutexHandle)
{}
};
----------------------------
//creating a mutex to pause repeatable checks (whe Messagebox is displayed in UI thread)
HANDLE haspSerialCheckIsPausedMutexHandle = CreateMutex(NULL, FALSE, NULL);
//starting a separate thread with a check that repeats every 5000 milliseconds
auto params = new RepeatFunctionParameters(5000, myApp_hWnd, WM_USER_HASP_CHECK_FAILED, haspSerialCheckIsPausedMutexHandle);
CreateThread(NULL, 0, RepeatFunction, params, 0, NULL);
----------------------------
//special message that is sended when check is failed
#define WM_USER_HASP_CHECK_FAILED (WM_USER+0x150)
//mapping message handling function to that message
ON_MESSAGE( WM_USER_HASP_CHECK_FAILED, OnUserHaspCheckFailed)
//message handling function definition
afx_msg LRESULT OnUserHaspCheckFailed(WPARAM wParam, LPARAM lParam);
//message handling function body
LRESULT CMainWnd::OnUserHaspCheckFailed(WPARAM wParam, LPARAM lParam)
{
//capturing a mutex that signals to pause repeatable checks
WaitForSingleObject(haspSerialCheckIsPausedMutexHandle, INFINITE);
//show a messagebox that requires user action what to do next
if (::MessageBox(myApp_hWnd, ("Check is failed! Retry or cancel?").c_str(),
myApp_name, MB_RETRYCANCEL | MB_ICONERROR | MB_SYSTEMMODAL) == IDCANCEL)
//closing main windows if user clicks Cancel
pWnd->SendMessage(WM_CLOSE, 0x00010000, 0);
//releasing a mutex that signals to pause repeatable checks
ReleaseMutex(haspSerialCheckIsPausedMutexHandle);
return 0;
}
//WM_CLOSE handling function body
LRESULT CMainWnd::OnClose( WPARAM wParam, LPARAM lParam)
{
----------------------------
if( haspSerialCheckIsPausedMutexHandle != NULL)
CloseHandle( haspSerialCheckIsPausedMutexHandle);
----------------------------
CMDIFrameWnd::OnClose();
return NULL;
}
Here is my separate thread with repeatable check code (simplified):
DWORD WINAPI RepeatFunction(LPVOID parameters)
{
//getting parameters struct from a pointer
auto temp = static_cast<RepeatFunctionParameters*>(parameters);
//make a struct local copy (Further, all work goes only with it, regardless of the
state of the object, the pointer to which came as a function parameter)
auto params = *temp;
//deleting the structure, the pointer to which came as a function parameter
delete temp;
//repeatable check
while (true)
{
//checking a mutex that signals to pause repeatable checks. if it is free
//then there is no messagebox in UI thread and we can perform a check.
//if it is captured - wait until user clicks some button in that messagebox
WaitForSingleObject(params.haspSerialCheckIsPausedMutexHandle, INFINITE);
//and releasing it immediately
ReleaseMutex(params.haspSerialCheckIsPausedMutexHandle);
auto startMilliseconds = GetTickCount();
//performing a check
BOOL success = PerformACheck();
unsigned long defaultSleepDelay = 1000;
//if PerformACheck() will last longer than params.repeatDelayInMilliseconds,
//then check will be repeated after 1000 milliseconds, otherwise -
//after params.repeatDelayInMilliseconds minus PerformACheck() call time
auto endMilliseconds = GetTickCount();
if ((endMilliseconds - startMilliseconds) < params.repeatDelayInMilliseconds)
sleepDelay = params.repeatDelayInMilliseconds - (endMilliseconds - startMilliseconds);
//if check is failed
if (!success)
{
//sending a message with an identifier params.checkIsFailedPostMessageMessageId
//to a HWND params.checkIsFailedPostMessageWindowHandler so in it's
//handling function a messagebox with will be displayed and a mutex
//params.haspSerialCheckCanRunMutexHandle will be captured until
//user click some button in that messagebox
PostMessage(params.checkIsFailedPostMessageWindowHandler, params.checkIsFailedPostMessageMessageId, 0, 0);
//if check is failed then next check always repeats after 1000 milliseconds
sleepDelay = 1000;
}
Sleep(sleepDelay);
}
}
The result is that the main window becomes unresponsive after some time. It looks like my code has some logical mistake, or a memory leak.
I'm a newbie to C++ (and especially to outdated standards).

PeekMessage always returns FALSE

I have written a small test application that inserts files (with hardcoded paths) into the currently active folder/application via delayed rendering. It works as expected. But I have a question - why does PeekMessage always return FALSE? But if you remove the PeekMessage call, Wndproc will never be called. I read a similar post, but I'm creating a window in the same thread in which I'm trying to process messages.
Code:
static LRESULT CALLBACK WindProc(HWND hWnd, UINT Msg, WPARAM wParam, LPARAM lParam) {
switch (Msg) {
case WM_RENDERALLFORMATS: {
OpenClipboard(hWnd);
EmptyClipboard();
}
case WM_RENDERFORMAT: {
printf("WM_RENDERFORMAT received");
<Here the file paths are copied to the clipboard>
if (Msg == WM_RENDERALLFORMATS)
CloseClipboard();
return 0;
}
case WM_DESTROYCLIPBOARD:
return 0;
}
return DefWindowProc(hWnd, Msg, wParam, lParam);
}
HWND hwnd_;
void thread_(void* ignored) {
WNDCLASSEX wcx = { 0 };
wcx.cbSize = sizeof(WNDCLASSEX);
wcx.lpfnWndProc = WindProc;
wcx.hInstance = GetModuleHandle(NULL);
wcx.lpszClassName = TEXT("my_class");
RegisterClassEx(&wcx);
hwnd_ = CreateWindowEx(0, TEXT("my_class"), TEXT(""), 0, 0, 0, 0, 0, HWND_MESSAGE, NULL, GetModuleHandle(NULL), NULL);
MSG msg;
while (true) {
if (PeekMessage(&msg, NULL, 0, 0, PM_NOREMOVE)) {
printf("PeekMessage returned TRUE\n");
TranslateMessage(&msg);
DispatchMessage(&msg);
break;
}
Sleep(1000);
}
}
void main() {
CloseHandle((HANDLE)_beginthread(thread_, 0, NULL));
// let's give some time to thread to create msg window
Sleep(100);
if (OpenClipboard(hwnd_)) {
EmptyClipboard();
SetClipboardData(CF_HDROP, NULL);
CloseClipboard();
}
while (true) {
Sleep(100);
}
}
PeekMessage() (and GetMessage()) only returns messages that are posted via PostMessage() or PostThreadMessage() to the calling thread's message queue. PeekMessage() returns FALSE when there is no posted message available at the moment it is called. GetMessage() blocks until a posted message becomes available.
However, the messages in question (WM_RENDERFORMAT and WM_RENDERALLFORMATS) are instead being sent via SendMessage...() directly to the target window. But, the messages are being sent by another thread, so PeekMessage() (or GetMessage()) is still needed, as they internally dispatch messages that are sent across thread boundaries.
This is stated in PeekMessage()'s documentation:
Dispatches incoming nonqueued messages, checks the thread message queue for a posted message, and retrieves the message (if any exist).
...
During this call, the system dispatches (DispatchMessage) pending, nonqueued messages, that is, messages sent to windows owned by the calling thread using the SendMessage, SendMessageCallback, SendMessageTimeout, or SendNotifyMessage function. Then the first queued message that matches the specified filter is retrieved. The system may also process internal events. If no filter is specified, messages are processed in the following order:
Sent messages
Posted messages
Input (hardware) messages and system internal events
Sent messages (again)
WM_PAINT messages
WM_TIMER messages
As well as the GetMessage() documentation:
Retrieves a message from the calling thread's message queue. The function dispatches incoming sent messages until a posted message is available for retrieval.
...
During this call, the system delivers pending, nonqueued messages, that is, messages sent to windows owned by the calling thread using the SendMessage, SendMessageCallback, SendMessageTimeout, or SendNotifyMessage function. Then the first queued message that matches the specified filter is retrieved. The system may also process internal events. If no filter is specified, messages are processed in the following order:
Sent messages
Posted messages
Input (hardware) messages and system internal events
Sent messages (again)
WM_PAINT messages
WM_TIMER messages
And SendMessage()'s documentation:
If the specified window was created by the calling thread, the window procedure is called immediately as a subroutine. If the specified window was created by a different thread, the system switches to that thread and calls the appropriate window procedure. Messages sent between threads are processed only when the receiving thread executes message retrieval code. The sending thread is blocked until the receiving thread processes the message. However, the sending thread will process incoming nonqueued messages while waiting for its message to be processed. To prevent this, use SendMessageTimeout with SMTO_BLOCK set. For more information on nonqueued messages, see Nonqueued Messages.
With that said, your handling of the WM_RENDERALLFORMATS message is wrong. For one thing, it must not call EmptyClipboard(), and for another, it is not checking whether your app is still the clipboard owner before rendering its data. See What is the proper handling of WM_RENDERFORMAT and WM_RENDERALLFORMATS? for why these points are important.
Also, you have a race condition in your main thread. Before it calls OpenClipboard(), it sleeps for only 100ms to wait for the window to be created first, but there is no guarantee that hwnd_ will have been assigned within that 100ms. It could take that long just for the worker thread to start running, for instance.
A better option is to have the worker thread signal an event when the window is created, and then have the main thread wait on that event (even better would be to simply move the initial SetClipboardData() call into the worker thread itself after creating the window, but you have dismissed that option).
Try something more like this instead:
static LRESULT CALLBACK WindProc(HWND hWnd, UINT uMsg, WPARAM wParam, LPARAM lParam) {
switch (uMsg) {
/*
case WM_CREATE:
if (OpenClipboard(hWnd)) {
EmptyClipboard();
SetClipboardData(CF_HDROP, NULL);
CloseClipboard();
}
return 0;
}
*/
case WM_RENDERALLFORMATS: {
if (OpenClipboard(hWnd)) {
if (GetClipboardOwner() == hWnd) {
SendMessage(hWnd, WM_RENDERFORMAT, CF_HDROP, 0);
}
CloseClipboard();
}
return 0;
}
case WM_RENDERFORMAT: {
printf("WM_RENDERFORMAT received");
if (wParam == CF_HDROP) {
// <Here the file paths are copied to the clipboard>
}
return 0;
}
case WM_DESTROYCLIPBOARD:
return 0;
}
return DefWindowProc(hWnd, Msg, wParam, lParam);
}
HWND hwnd_ = NULL;
void thread_(void* arg) {
HANDLE hEvent = (HANDLE)arg;
WNDCLASSEX wcx = { 0 };
wcx.cbSize = sizeof(WNDCLASSEX);
wcx.lpfnWndProc = WindProc;
wcx.hInstance = GetModuleHandle(NULL);
wcx.lpszClassName = TEXT("my_class");
RegisterClassEx(&wcx);
hwnd_ = CreateWindowEx(0, TEXT("my_class"), TEXT(""), 0, 0, 0, 0, 0, HWND_MESSAGE, NULL, GetModuleHandle(NULL), NULL);
SetEvent(hEvent);
if (hwnd_ == NULL) {
return 0;
}
MSG msg;
while (GetMessage(&msg, NULL, 0, 0)) {
printf("GetMessage returned a message\n");
TranslateMessage(&msg);
DispatchMessage(&msg);
}
}
int main() {
HANDLE hEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
if (!hEvent)
return 1;
uintptr_t res = _beginthread(thread_, 0, hEvent);
if (res == -1L) {
CloseHandle(hEvent);
return 1;
}
WaitForSingleObject(hEvent, INFINITE);
CloseHandle(hEvent);
if (hwnd_ != NULL) {
if (OpenClipboard(hwnd_)) {
EmptyClipboard();
SetClipboardData(CF_HDROP, NULL);
CloseClipboard();
}
}
HANDLE hThread = (HANDLE)res;
WaitForSingleObject(hThread, INIFINTE);
CloseHandle(hThread);
return 0;
}
why does PeekMessage always return FALSE?
Assuming that the window handle that you pass to PeekMessage is valid, then the reason for PeekMessage returning FALSE is simply that there are no messages in the queue.
That this is the case can be discerned from the documentation which says:
If a message is available, the return value is nonzero.
If no messages are available, the return value is zero.

Deadlock in multi-threaded Windows GUI application

I develop a DAW application for Windows 10. It's a x64 application written in C++ and built by Visual Studio 2019.
The application uses a custom GUI that does not use any Windows APIs but it also has to load VST 2.4 plugins that do use standard Win32 GUI and I open them in modeless popup (non-child) windows.
The problem I've been trying to solve is a deadlock -- see below.
Disclaimer: I know the code isn't perfect and optimized -- it's a work in progress please.
======== main.cpp =============================
// ...
void winProcMsgRelay ()
{
MSG msg;
CLEAR_STRUCT (msg);
while (PeekMessage(&msg, NULL, 0, 0, PM_REMOVE))
{
TranslateMessage (&msg);
DispatchMessage (&msg);
};
}
// ...
int CALLBACK WinMain (HINSTANCE hInst, HINSTANCE hPrevInst, LPSTR lpCmdL, int nCmdShw)
{
// ...
}
=================================================
1) The WinMain function creates a new thread that will handle our custom GUI (which does not use any Windows API).
2) The WinMain thread uses the standard Windows GUI API and it handles all window messages delivered to our main application window.
The WinMain thread creates our main window by calling CreateWindowEx (with a WNDPROC window procedure callback):
{
WNDCLASSEX wc;
window_menu = CreateMenu ();
if (!window_menu)
{
// Handle error
// ...
}
wc.cbSize = sizeof (wc);
wc.style = CS_BYTEALIGNCLIENT | CS_HREDRAW | CS_VREDRAW;
wc.lpfnWndProc = mainWndProc;
wc.cbClsExtra = 0;
wc.cbWndExtra = 0;
wc.hInstance = hInst;
wc.hIcon = LoadIcon (NULL, IDI_APP);
wc.hCursor = NULL;
wc.hbrBackground = NULL;
wc.lpszMenuName = mainWinName;
wc.lpszClassName = mainWinName;
wc.hIconSm = LoadIcon (NULL, IDI_APP);
RegisterClassEx (&wc);
mainHwnd = CreateWindowEx (WS_EX_APPWINDOW | WS_EX_OVERLAPPEDWINDOW | WS_EX_CONTEXTHELP,
mainWinName, mainWinTitle,
WS_OVERLAPPEDWINDOW | WS_VISIBLE,
CW_USEDEFAULT, 0,
0, 0,
NULL, NULL, hInst, NULL);
// ...
// Then the WinMain thread keeps executing a standard window message processing loop
// ...
while (PeekMessage (&msg, NULL, 0, 0, PM_NOREMOVE) != 0
&& ! requestQuit)
{
if (GetMessage (&msg, NULL, 0, 0) == 0)
{
requestQuit = true;
}
else
{
TranslateMessage(&msg);
DispatchMessage(&msg);
}
if (! requestQuit)
{
WaitMessage ();
}
}
// ...
}
3) Our custom-GUI thread (spawned above), in addition to its other functions, does the following:
a) Loads a VST audio plugin from a DLL file by calling LoadLibrary.
b) Creates a new thread for the DLL plugin (let's call it "plugin thread") to create a new instance of it (there may be multiple instances of a loaded DLL plugin):
vst_instance_thread_handle = (HANDLE) _beginthreadex (NULL, _stack_size, redirect, (void *) this, 0, NULL);
c) After some time that the plugin instance has been running on its own thread, our custom-GUI thread (in response to a user action in our custom GUI) creates a new thread for the plugin GUI window:
vst_gui_thread_handle = (HANDLE) _beginthreadex (NULL, _stack_size, redirect, (void *) this, 0, NULL);
(Note that the DLL plugin uses standard Win32 GUI.)
When the new plugin GUI thread is being spawned, the function VSTGUI_open_vst_gui is called on the plugin instance thread -- see below:
============ vst_gui.cpp: ====================
// ...
struct VSTGUI_DLGTEMPLATE: DLGTEMPLATE
{
WORD e[3];
VSTGUI_DLGTEMPLATE ()
{
memset (this, 0, sizeof (*this));
};
};
static INT_PTR CALLBACK VSTGUI_editor_proc_callback (HWND hwnd, UINT msg, WPARAM wParam, LPARAM lParam);
thread_local AEffect * volatile Vst_instance_ptr = 0;
thread_local volatile int Vst_instance_index = -1;
thread_local volatile UINT_PTR Vst_timer_id_ptr = 0;
thread_local volatile HWND Vst_gui_handle = NULL;
void VSTGUI_open_vst_gui (int vst_instance_index)
{
AEffect *vst_instance = VST_instances [vst_instance_index].vst->pEffect;
Vst_instance_index = vst_instance_index;
Vst_instance_ptr = vst_instance;
VSTGUI_DLGTEMPLATE t;
t.style = WS_POPUPWINDOW | WS_MINIMIZEBOX | WS_DLGFRAME | WS_VISIBLE |
DS_MODALFRAME | DS_CENTER;
t.cx = 100; // We will set an appropriate size later
t.cy = 100;
VST_instances [vst_instance_index].vst_gui_open_flag = false;
Vst_gui_handle = CreateDialogIndirectParam (GetModuleHandle (0), &t, 0, (DLGPROC) VSTGUI_editor_proc_callback, (LPARAM) vst_instance);
if (Vst_gui_handle == NULL)
{
// Handle error
// ...
}
else
{
// Wait for the window to actually open and initialize -- that will set the vst_gui_open_flag to true
while (!VST_instances [vst_instance_index].vst_gui_open_flag)
{
winProcMsgRelay ();
Sleep (1);
}
// Loop here processing window messages (if any), because otherwise (1) VST GUI window would freeze and (2) the GUI thread would immediately terminate.
while (VST_instances [vst_instance_index].vst_gui_open_flag)
{
winProcMsgRelay ();
Sleep (1);
}
}
// The VST GUI thread is about to terminate here -- let's clean up after ourselves
// ...
return;
}
// The plugin GUI window messages are handled by this function:
INT_PTR CALLBACK VSTGUI_editor_proc_callback (HWND hwnd, UINT msg, WPARAM wParam, LPARAM lParam)
{
AEffect* vst_instance = Vst_instance_ptr;
int instance_index = Vst_instance_index;
if (VST_instances [instance_index].vst_gui_window_handle == (HWND) INVALID_HANDLE_VALUE)
{
VST_instances [instance_index].vst_gui_window_handle = hwnd;
}
switch(msg)
{
case WM_INITDIALOG:
{
SetWindowText (hwnd, String (tmp_str) + VST_get_best_vst_name (instance_index, false));
if (vst_instance)
{
ERect* eRect = 0;
vst_instance->dispatcher (vst_instance, effEditGetRect, 0, 0, &eRect, 0);
if (eRect)
{
// ...
SetWindowPos (hwnd, HWND_TOP, x, y, width, height, SWP_SHOWWINDOW);
}
vst_instance->dispatcher (vst_instance, effEditOpen, 0, 0, hwnd, 0);
}
}
VST_instances [instance_index].vst_gui_open_flag = true;
if (SetTimer (hwnd, (UINT_PTR) Vst_instance_ptr, 1, 0) == 0)
{
logf ("Error: Could not obtain a timer object for external VST GUI editor window.\n");
}
return 1;
case WM_PAINT:
{
PAINTSTRUCT ps;
BeginPaint (hwnd, &ps);
EndPaint (hwnd, &ps);
}
return 0;
case WM_MOVE:
if (Vst_instance_index >= 0)
{
VST_instances [Vst_instance_index].vst_gui_win_pos_x = VST_get_vst_gui_win_pos_x (Vst_instance_index);
VST_instances [Vst_instance_index].vst_gui_win_pos_y = VST_get_vst_gui_win_pos_y (Vst_instance_index);
}
return 0;
case WM_SIZE:
if (Vst_instance_index >= 0)
{
VST_instances [Vst_instance_index].vst_gui_win_width = VST_get_vst_gui_win_width (Vst_instance_index);
VST_instances [Vst_instance_index].vst_gui_win_height = VST_get_vst_gui_win_height (Vst_instance_index);
}
return 0;
case WM_TIMER:
if (vst_instance != NULL)
{
vst_instance->dispatcher (vst_instance, effEditIdle, 0, 0, 0, 0);
}
return 0;
case WM_CLOSE:
// ...
return 0;
case WM_NCCALCSIZE:
return 0;
default:
return (DefWindowProc (hwnd, msg, wParam, lParam));
}
return 0;
=================================================
Our custom-GUI thread, too, periodically calls winProcMsgRelay (); Sleep (1); in a loop.
Why multi-threaded? Because: 1) this is a real-time audio-processing application where near-zero latencies are required, and 2) we need to set CPU priorities and stack sizes independently for each thread, based on their real needs. Also, 3) having multi-threaded GUI allows our DAW app to remain responsive when the plugin or its GUI becomes unresponsive and 4) we make us of multi-core CPUs.
Everything is working well. I can open multiple instances of multiple plugins. Their GUI windows can even spawn other windows showing progress bars, all that without any deadlock.
However, the problem is that I get a deadlock when I click the app logo in a plugin GUI window (Absynth 5 and Kontakt 6 by Native Instruments), which apparently creates a child modal window, which, by the way, displays correctly and fully.
But both this modal window and the parent GUI window stop responding to user actions and window messages -- they "hang" (our custom GUI keeps working well, though). The same thing happens when the plugin GUI displays a standard Windows modal MessageBox on error, where the MessageBox is completely "frozen".
When I set a debugger breakpoint in VSTGUI_open_vst_gui in the second loop that calls winProcMsgRelay, I can determine that this is the place where it hangs, because when I get the deadlock state, that breakpoint is never triggered.
I know that modal dialogs have their own message loop that might block ours, but how should I redesign my code to accommodate for that?
I also know that SendMessage and the like are blocking until they get response. That's why I use the asynchronous PostMessage, instead.
I confirmed that the deadlock occurs in 32-bit builds of the application, too.
I've been trying to trace the cause for several weeks. I believe I've done all my homework and I honestly don't know what else to try. Any help would be greatly appreciated.
There is a lot of code not appearing here (e.g. winProcMsgRelay) and I will admit I'm finding it difficult to get a mental picture of how this works, but let me offer you some general advice and some things to keep in mind.
First of all, modal dialogs have their own message loop. As long as they are up, your message loop will not run.
Second of all, windows functions like SetWindowPos SetWindowText actually send a message to the window. Are you calling those from the thread that that created the window? Because if not, that means that the calling thread will block while the OS sends the message to the window and waits for a response. If the thread that created those windows is busy, the sending thread will remain blocked until it is not.
If I were attempting to debug this, I would simply wait until it deadlocks, then break into the debugger and bring up the threads and call stacks windows next to each other. Switch context among the threads in the threads windows (double click on them) and look at the resulting thread call stacks. You should be able to spot the problem.
Ok, I was able to resolve the deadlock myself. The solution was to rewrite the code so as to unify the window proc handlers (VST GUI messages are handled by the same callback function as the main window messages). Moreover, unlike the official VST SDK, which uses DialogBoxIndirectParam to create the plugin window, I now use CreateWindowEx, instead (not sure if this contributed to solving the deadlock issue, though). Thanks for the comments.

How to Use PostThreadMessage() (Unexpected Results)

First, I'm new to multithreading. So, if there is a better way than my approach I would like to hear.
I'm injecting my code into another process. I created a CBT hook to get notified when new windows are created. When my target window created, I get notification by CBT hook and create a thread, and at the end this thread calls PostThreadMessage() function to send data/notification to another thread of me (which is inside same/target application, too).
But this doesn't work as expected. My receiving thread gets that message 4 times before I even send it first. Then after I send it, it doesn't get it this time.
Let's come to code.
This is how I create receiver thread.
#define MM_MY_MESSAGE (WM_APP)
// Also tried with different macros
// #define MM_MY_MESSAGE (WM_APP + 999)
// #define MM_MY_MESSAGE (WM_USER)
// #define MM_MY_MESSAGE (WM_USER + 999)
gReceiverThreadHandle = CreateThread(NULL, 0, ThreadReceiver, NULL, 0, &gReceiverThreadId);
// global variables
This is my receiver thread, which also uses WM_COPYDATA messages.
DWORD WINAPI ThreadReceiver(LPVOID lpParam) {
MSG msg;
HWND hwndReceiver = CreateReceiverWindow();
// If necessary I can post this function later
// This window is created for WM_COPYDATA messages
// Not important for PostThreadMessage()
if (!hwndReceiver) {
std::cout << "Receiver window can not be created!" << std::endl;
return 0;
}
while (GetMessage(&msg, NULL, 0, 0))
{
TranslateMessage(&msg);
DispatchMessage(&msg);
}
return 0;
}
This is WndProc for WM_COPYDATA and MM_MY_MESSAGE.
LRESULT CALLBACK ReceiverWndProc(HWND hwnd, UINT message, WPARAM wParam, LPARAM lParam) {
switch(message) {
case WM_COPYDATA:
// do something
break;
case MM_MY_MESSAGE:
std::cout << "PostThreadMessage received" << std::endl;
return 0;
}
return DefWindowProc(hwnd, message, wParam, lParam);
}
This is how I call PostThreadMessage():
DWORD WINAPI AnotherThread(LPVOID lpParam) {
// This thread gets created inside CBT hook which is unrelated
// etc.
PostThreadMessage(gReceiverThreadId, MM_MY_MESSAGE, NULL, NULL);
std::cout << "PostThreadMessage send" << std::endl;
// etc
return 0;
}
After I run my code, output shows,
PostThreadMessage received
PostThreadMessage received
PostThreadMessage received
PostThreadMessage received
PostThreadMessage send
This is clearly not what I wanted. What is wrong here?

Closing Window and Thread Generates Invalid Window Handle Error - C++

My console application spawns a new (invisible) window in its own thread. Prior to exiting the application, it attempts to clean up, and the last call to GetMessage in the window's message pump fails. GetLastError returns 1400, "Invalid window handle."
This is how the clean up proceeds in the application thread:
if ( s_hNotifyWindowThread != NULL )
{
ASSERT(s_pobjNotifyWindow != NULL);
::PostMessage( s_pobjNotifyWindow->m_hWnd, WM_CLOSE, 0, 0 );
::WaitForSingleObject( s_hNotifyWindowThread, 50000L ); // Step 1: breakpoint here
::CloseHandle( s_hNotifyWindowThread ); // Step 4: breakpoint here
s_hNotifyWindowThread = NULL;
}
This WndProc exists in the new thread created for the window:
static LRESULT CALLBACK WndProc( HWND hWnd, UINT uMsg, WPARAM wParam, LPARAM lParam )
{
switch ( uMsg )
{
case WM_CLOSE:
::DestroyWindow( hWnd ); // Step 2: breakpoint here
break;
case WM_DESTROY:
::PostQuitMessage( 0 ); // Step 3: breakpoint here
break;
case WM_DEVICECHANGE:
/* Handle device change. */
break;
default:
// Do nothing.
break;
}
return ::DefWindowProc( hWnd, uMsg, wParam, lParam );
}
This is in the window's thread function where the new window is created and my message pump is located:
s_pobjNotifyWindow = new CNotifyWindow( pParam );
while ( (bRetVal = ::GetMessage(
&msg, // message structure
s_pobjNotifyWindow->m_hWnd, // handle to window whose messages are to be retrieved
0, // lowest message value to retrieve
0 // highest message value to retrieve
)) != 0 )
{
switch ( bRetVal )
{
case -1: // Error generated in GetMessage.
TRACE(_T("NotifyWindowThreadFn : Failed to get notify window message.\r\n\tError: %d\r\n\tFile: %s\r\n\tLine: %d\r\n"), ::GetLastError(), __WFILE__, __LINE__);
return ::GetLastError(); // Step 5: breakpoint here: Returns error 1400
break;
default: // Other message received.
::TranslateMessage( &msg );
::DispatchMessage( &msg );
break;
}
}
What am I doing wrong? Thanks.
You are passing an invalid window handle to GetMessage, which is why it's failing and reporting that a window handle is invalid.
You'll see the same error if you run this code with a made-up window handle:
MSG msg = {0};
BOOL b = ::GetMessage(&msg, (HWND)(0x123000), 0, 0);
if (b == -1)
{
DWORD dwErr = ::GetLastError();
wprintf(L"%lu\n", dwErr);
}
The problem is that you are still using the window handle after the window has been destroyed. As soon as a window is destroyed its handle is invalid (or, worse, re-used by some other window). The message pump won't exit until it processes the quit-message. Since the quit-message is posted during window destruction it will be processed after window destruction is complete. i.e. Your message pump keeps running for a short time after your window is destroyed.
Just pass NULL to GetMessage for the window argument, so that it retrieves messages for all windows on that thread. Since the thread only exists for that one window it's only going to get messages for that window anyway. (Plus the quit-message posted to the thread itself, and potentially messages for other windows which things like COM creates if you use COM on that thread... You'd definitely want to process those messages, so telling GetMessage to filter by window is doing nothing at best and could prevent something from working at worst, in addition to the error you're seeing GetMessage return.)