Regarding:
PeekMessage(&msg, NULL, 0, 0, PM_REMOVE)
If hWnd is NULL, PeekMessage retrieves
messages for any window that belongs
to the current thread, and any
messages on the current thread's
message queue whose hwnd value is NULL
(see the MSG structure). Therefore if
hWnd is NULL, both window messages and
thread messages are processed.
Are messages received via a named pipe included in window messages and thread messages?
Definitely not. Named pipes do not send window messages.
The thread messages in this context are special and have nothing to do with named pipes.
Use MsgWaitForMultipleObjects instead.
CODE SAMPLE:
void MessageLoop(HANDLE hNamedPipe)
{
do {
DWORD res = MsgWaitForMultipleObjects(1, &hNamedPipe, INFINITE, QS_ALLEVENTS, MWMO_INPUTAVAILABLE);
if (res == WAIT_OBJECT_0) {
/* Handle named pipe -- at this point ReadFile will not block */
} else if (res == WAIT_OBJECT_0 + 1) {
MSG msg;
if (!GetMessage(&msg, NULL, 0, 0))
break; /* WM_QUIT */
TranslateMessage(&msg);
DispatchMessage(&msg);
}
} while (1);
}
No, Windows messages and named pipes are completely unrelated. You would need to use the MsgWaitForMultipleObjectsEx function to wait for either an incoming message or a message on the named pipe.
Note that MsgWaitForMultipleObjectsEx doesn't actually retrieve the message; check its return value to see if there's a Windows message or data on the named pipe, then use GetMessage or ReadFile as appropriate.
Related
I want to watch for changes done with a file (the event i'm waiting for is change contents event, i.e. last modified date is updated)
I have a code like this (minimalized example of actual code)
I expect that each iteration of the while loop the event gets reset and is available to be fired again but that doesn't happen
Why it fires change event only once?
int main()
{
const wchar_t *dir_path = L"C:\\Users\\IC\\AppData\\Roaming\\JetBrains\\CLion2021.3\\scratches\\";
HANDLE hDir = ::CreateFileW(
dir_path,
FILE_LIST_DIRECTORY,
FILE_SHARE_READ,
NULL,
OPEN_EXISTING,
FILE_FLAG_BACKUP_SEMANTICS | FILE_FLAG_OVERLAPPED,
NULL
);
FILE_NOTIFY_INFORMATION fni;
OVERLAPPED overlapped;
overlapped.hEvent = ::CreateEventA(NULL, FALSE, FALSE, NULL);
::ReadDirectoryChangesW(
hDir,
&fni,
sizeof(fni),
TRUE,
FILE_NOTIFY_CHANGE_LAST_WRITE,
NULL,
&overlapped,
NULL
);
while (true)
{
std::vector<HANDLE> all_job_event_handles;
if (::ResetEvent(overlapped.hEvent) == FALSE)
{
printf("ResetEvent failed\n");
fflush(stdout);
return 1;
}
all_job_event_handles.push_back(overlapped.hEvent);
DWORD result = ::WaitForMultipleObjects(all_job_event_handles.size(), all_job_event_handles.data(), FALSE, INFINITE);
if (result == WAIT_FAILED)
{
printf("WaitForMultipleObjects failed\n");
fflush(stdout);
return 1;
}
if (result >= WAIT_OBJECT_0 && result < WAIT_OBJECT_0 + all_job_event_handles.size())
{
printf("file changed\n");
fflush(stdout);
}
}
}
Because that's just not how ReadDirectoryChanges works. It doesn't continuously send you changes. It sends you one batch of changes. You process them. You call the function again to tell the system that you want more changes.
I found a correct usage example of the function here: https://gist.github.com/nickav/a57009d4fcc3b527ed0f5c9cf30618f8
Some side notes:
You don't check whether ReadDirectoryChanges succeeds. This is bad; if it failed, you will hang on the Wait call forever.
You don't zero-initialize the OVERLAPPED structure.
You create the event as an auto-reset event (second parameter is FALSE). ResetEvent on such an event does nothing.
All the event handles you add into your vector are the same event object. You just have an ever-growing list of the same event repeatedly that you pass to WaitForMultipleObjects. This does nothing at best, but will eventually fail because WFMO doesn't allow more than MAXIMUM_WAIT_OBJECTS handles, and this number is fairly low (32, I think).
You probably want a more permissive share mode on the directory you open.
I'm getting a crash sometimes in RegisterWaitForSingleObject (1 out of 10). It seems that although RegisterWaitForSingleObject returns, the internal thread pool is not yet ready.
HANDLE processHandle = OpenProcess (PROCESS_ALL_ACCESS, FALSE, processID);
// CRASH IN INTERNAL SOMETIMES
RegisterWaitForSingleObject (&hWaitForChild_,processHandle,OnChildProcessExit, 0,INFINITE,WT_EXECUTEONLYONCE);
// If I sleep here, then it seems ok.
//std::this_thread::sleep_for (std::chrono::milliseconds (10));
CloseHandle (processHandle);
I can replicate this with a simple sample here. 1 in 10 times, it will crash. How should I be synchronizing it properly without resorting to sleep hack.
https://filedn.com/l3TGy7Y83c247u0RDYa9fkp/temp/stackoverflow/testregister.cpp
based on your code spinet:
// THIS CRASHS HERE SOMETIMES
if (! RegisterWaitForSingleObject (
&hWaitForChild_
,processHandle
, OnChildProcessExit
, 0 //this
, INFINITE
, WT_EXECUTEONLYONCE))
{
LogDebug ("RegisterWaitForSingleObject failed");
}
// If this is enabled, then it won't crash
//std::this_thread::sleep_for (std::chrono::milliseconds (10));
if (! CloseHandle (processHandle)) // !!!
LogDebug ("RegisterWaitForSingleObject Closehandle failed");
so you close processHandle just after you call RegisterWaitForSingleObject for this handle. however if read about RegisterWaitForSingleObject:
If this handle is closed while the wait is still pending, the
function's behavior is undefined.
if look more deep - try understand - how is RegisterWaitForSingleObject worked internally ? it pass processHandle to some worker thread. and this thread begin wait for this handle. but this is (pass handle to another thread) is asynchronous operation - say for example internally can be started new thread with this handle as argument, or it can be passed to already existing working thread via some signal. but anyway - worked thread got this handle and begin wait some later. from another side - you just close processHandle after RegisterWaitForSingleObject return control. so here race - what will be first - or worked thread begin wait on handle (in this case all will be work) or you close this handle. in case you close this handle first - worked thread will be try wait on already invalid handle and raise exception - STATUS_THREADPOOL_HANDLE_EXCEPTION.
// If this is enabled, then it won't crash
//std::this_thread::sleep_for (std::chrono::milliseconds (10));
of course - by sleep you give time for worked thread to begin wait on handle. in this case he begin wait before you close handle.
solution - you must not close handle, until WAITORTIMERCALLBACK Callback will be not called. you need allocate some context, where place processHandle and pass this context to RegisterWaitForSingleObject. and when you callback will be called - you got pointer to your context back and here and close handle.
also note, that you not need open separate, second, handle for child process, but can use process handle returned by CreateProcess
Ok, I managed to solve it by keeping the handle around until I call Unregisterwait. It seems to be stable. Thanks to the answers.
I met the same problem like you that an exception occurred while debugging in Visual Studio. I tried many time and finally found the reason. If you close the handle of the process newly created, the program would crash. I tried to close the handles in the callback function, it works perfectly, like this:
typedef struct {
LPTSTR pszCmdLine;
HANDLE hEvent;
} THREAD_PARAM;
typedef struct {
TCHAR szCmdLine[1024];
HANDLE hWaitObject;
DWORD dwProcessId;
HANDLE hProcess;
DWORD dwThreadId;
HANDLE hThread;
} OBJECT_PARAM;
static void CALLBACK WaitObjectCallback(LPVOID lpParam, BOOLEAN TimerOrWaitFired)
{
OBJECT_PARAM *pobp = static_cast<OBJECT_PARAM *>(lpParam);
TCHAR szInfo[1024] = { 0 };
DWORD dwExitCode = 0;
GetExitCodeProcess(pobp->hProcess, &dwExitCode);
wnsprintf(szInfo, ARRAYSIZE(szInfo), _T("process %u [%s] exit: %u\n"), pobp->dwProcessId, pobp->szCmdLine, dwExitCode);
OutputDebugString(szInfo);
//
// unregister the wait object handle and close the process handle finally
//
UnregisterWait(pobp->hWaitObject);
CloseHandle(pobp->hProcess);
CloseHandle(pobp->hThread);
GlobalFree(lpParam);
}
static DWORD CALLBACK ThreadFunction(LPVOID lpParam)
{
THREAD_PARAM *pthp = static_cast<THREAD_PARAM *>(lpParam);
STARTUPINFO si = { sizeof(si) };
PROCESS_INFORMATION pi = { 0 };
BOOL bResult;
bResult = CreateProcess(nullptr, pthp->pszCmdLine, nullptr, nullptr, FALSE, 0, nullptr, nullptr, &si, &pi);
if (bResult)
{
OBJECT_PARAM *pobp = static_cast<OBJECT_PARAM *>(GlobalAlloc(GPTR, sizeof(OBJECT_PARAM)));
// make copy of the command line and other informations of the newly created process
lstrcpyn(pobp->szCmdLine, pthp->pszCmdLine, ARRAYSIZE(pobp->szCmdLine));
pobp->dwProcessId = pi.dwProcessId;
pobp->hProcess = pi.hProcess;
pobp->dwThreadId = pi.dwThreadId;
pobp->hThread = pi.hThread;
bResult = RegisterWaitForSingleObject(&pobp->hWaitObject, pi.hProcess, WaitObjectCallback, pobp, INFINITE, WT_EXECUTEONLYONCE);
// once it failed...
if (!bResult)
{
CloseHandle(pi.hProcess);
CloseHandle(pi.hThread);
}
}
// Notify the thread creator that the process is created successfully
SetEvent(pthp->hEvent);
return 0;
}
I've seen a question exactly like this already exists: Redirect stdout to an edit control (Win32)
However, the solution given requires the programmer to implement a my_printf function that does a {printf; read from pipe to edit control}. I cannot do that because my printf's are in a external library.
Ideally, I'm thinking of:
redirect app's stdout to edit control
run app and voila
But if edit control's API only allows you to write a string to it, I would think of something like:
1 - dup'ing stdout to a pipe out descriptor
3 - read from pipe in descriptor into a buffer
4 - write from buffer to edit control
However, there is a missing step 2 there:
2 - getting a signal of when a write to that pipe out descriptor is done.
How could I automate that part. Could I use something like a socket select here?
[EDIT]
So, according to David Heffernan's comments, I would have something like:
#define MYPRINT 1
#define WM_MYMESSAGE (WM_USER+1)
INT_PTR CALLBACK Foo::DialogProc(
...
case WM_COPYDATA:
{
PCOPYDATASTRUCT pMyCDS = (PCOPYDATASTRUCT) lParam;
LPCSTR szString = (LPCSTR)(pMyCDS->lpData);
AppendLog(szString);
}
break;
...
}
/* static */
void Foo::MainThread()
{
// Create worker thread
DWORD dwThreadId = 0;
m_hRedirectStdoutThread = CreateThread(
// default security
NULL,
// default stack size
0,
// routine to execute
(LPTHREAD_START_ROUTINE) &CTracesConsole::RedirectStdoutThreadRun,
// thread parameter
NULL,
// immediately run the thread
0,
// thread Id
&dwThreadId);
if (NULL == m_hRedirectStdoutThread)
{
printf("Error creating stdin thread\n");
return;
}
// Loop forever
MSG msg;
while (GetMessage(&msg, NULL, 0, 0) > 0)
{
TranslateMessage(&msg);
DispatchMessage(&msg);
}
}
/* static */
void Foo::RedirectStdoutThreadRun()
{
// Redirect stdout to pipe
int fds[2];
_pipe(fds, 1024, O_TEXT);
_dup2(fds[1], 1); // 1 is stdout
char buffer[1024];
for (;;)
{
// Need to flush the pipe
_flushall();
// Read stdout from pipe
DWORD dwNumberOfBytesRead = 0;
dwNumberOfBytesRead = _read(fds[0], buffer, 1024 - 1);
buffer[dwNumberOfBytesRead] = 0;
// Send data as a message
COPYDATASTRUCT myCDS;
myCDS.dwData = MYPRINT;
myCDS.cbData = dwNumberOfBytesRead + 1;
myCDS.lpData = buffer;
PostMessage(g_hWindow,
WM_MYMESSAGE,
0,
(LPARAM)(LPVOID) &myCDS);
}
}
Where AppendLog writes a string to the edit control.
[EDIT]
This code works properly now. With the little inconvenience that, when I redirect stdout traces from libcurl, libcurl stops working :) But that's another story...
Windows supports asynchronous I/O. That makes it easy:
dup'ing stdout to a pipe out descriptor
Issue async read from pipe in descriptor into a buffer
Wait for message or event (MsgWaitForMultipleObjects).
If the wait ends with one or more messages, call PeekMessage(PM_REMOVE) to remove them.
If the pipe event is signalled, copy text from buffer to edit control.
As far as I'm aware you can't get 'notifications' with a pipe. If you do want to do that maybe you should use WM_COPYDATA instead which would also provide a simpler solution. You will get a message when text is posted to your window which you can then append to the edit control.
I've done a lot with multithreading in the past, but I'm fairly new to COM. Anyway, here's my issue:
I create a worker thread, which registers as an STA, and creates a COM object. Then the worker thread and the main thread try to communicate with each other. Using CoMarshalInterThreadInterfaceInStream and CoGetInterfaceAndReleaseStream, I can get the threads to call methods on the COM objects in the other thread.
Here's what the worker thread looks like:
void workerThread()
{
CoInitialize(NULL);
MyLib::IFooPtr foo = ...; // create my COM object
// Marshall it so the main thread can talk to it
HRESULT hr = CoMarshalInterThreadInterfaceInStream(foo.GetIID(),
foo.GetInterfacePtr(),
&m_stream);
if (FAILED(hr)) {
// handle failure
}
// begin message loop, to keep this STA alive
MSG msg;
BOOL bRet;
while( (bRet = GetMessage( &msg, NULL, 0, 0 )) != 0)
{
if (bRet == -1) break;
DispatchMessage(&msg);
}
}
In the main thread:
// launch the thread
m_worker = boost::thread (&workerThread);
// get the interface proxy
MyLib::IFooPtr foo;
LPVOID vp (NULL);
HRESULT hr = CoGetInterfaceAndReleaseStream(m_stream, foo.GetIID(), &vp);
if (SUCCEEDED(hr)) foo.Attach(static_cast<MyLib::IFoo*>(vp));
This creates the object (which takes a while to initialize), and allows the main thread to talk to it, and everything is properly synchronized with the COM Apartment stuff. As far as I can tell from reading msdn, this seems to be the right way to do things. Now the main thread can use its proxy to call methods on my COM object, and the worker thread will receive those calls over the message queue, properly dispatching them.
However, what about synchronizing these threads?
Obviously in this case I want the main thread to wait to call CoGetInterfaceAndReleaseStream until after the worker thread has created that stream via CoMarshalInterThreadInterfaceInStream. But how can I safely do that?
From MSDN, I should be using something like MsgWaitForMultipleObjects, so I can wait for my_condition OR new_message_arrived, and then I can do something like:
// verbatim from msdn
while (TRUE)
{
// wait for the event and for messages
DWORD dwReturn = ::MsgWaitForMultipleObjects(1,
&m_hDoneLoading, FALSE, INFINITE, QS_ALLINPUT);
// this thread has been reawakened. Determine why
// and handle appropriately.
if (dwReturn == WAIT_OBJECT_0)
// our event happened.
break ;
else if (dwReturn == WAIT_OBJECT_0 + 1)
{
// handle windows messages to maintain
// client liveness
MSG msg ;
while(::PeekMessage(&msg, NULL, 0, 0, PM_REMOVE))
::DispatchMessage(&msg) ;
}
}
But how do I mix boost::thread.join() and boost::condition.wait() with MsgWaitForMultipleObjects? Is that even possible, or do I have to do something else to avoid a race condition?
Your main thread has a message queue (must be, since is an STA host), why not simply post a message to it, PostThreadMessage? Post an user message (WM_USER +X) and your normal main thread message pump can handle this user message as a notification that the COM object has marshaled the interface into the stream and the main thread is safe to call CoGetInterfaceAndReleaseStream.
I must call out though that with your current design your worker thread does basically nothing more than just run an additional message pump. Any call to any method on your interface from the main thread will block, wait for the worker thread to pick up the message from its message queue, process the call, respond, and then the main thread will resume. All operations will be at least as slow as having the COM object hosted in the main thread, plus the overhead of COM marshaling back and forth between the two STAs. Basically there is no concurrency whatsoever between the two threads, because of how COM STA works. Are you sure is this what you want?
Edit
(omitting a bunch of details like number of threads, timeout handling, assignment of a stream/IID/CLSID for each worker etc etc)
in the .h:
HANDLE m_startupDone;
volatile int m_threadStartCount;
worker thread:
void workerThread()
{
CoInitialize(NULL);
MyLib::IFooPtr foo = ...; // create my COM object
// Marshall it so the main thread can talk to it
HRESULT hr = CoMarshalInterThreadInterfaceInStream(foo.GetIID(),
foo.GetInterfacePtr(),
&m_stream);
if (FAILED(hr)) {
// handle failure
// remember to decrement and signal *even on failure*
}
if (0 == InterlockedDecrement(&m_threadStartCount))
{
SetEvent (m_startupDone);
}
// begin message loop, to keep this STA alive
MSG msg;
BOOL bRet;
while( (bRet = GetMessage( &msg, NULL, 0, 0 )) != 0)
{
if (bRet == -1) break;
DispatchMessage(&msg);
}
}
in the main thread:
m_startupDone = CreateEvent (NULL, FALSE, FALSE, NULL);
m_threadStartCount = <number of workerthreads>
// launch the thread(s)
m_worker = boost::thread (&workerThread);
m_worker2 = boost::thread (&workerThread);
...
// now wait for tall the threads to create the COM object(s)
if (WAIT_OBJECT0 != WaitForSingleObject(m_startupDone, ...))
{
// handle failure like timeout
}
// By now all COM objects are guaranteed created and marshaled, unmarshall them all in main
// here must check if all threads actually succeeded (could be as simple as m_stream is not NULL)
// get the interface proxy
MyLib::IFooPtr foo;
LPVOID vp (NULL);
HRESULT hr = CoGetInterfaceAndReleaseStream(m_stream, foo.GetIID(), &vp);
if (SUCCEEDED(hr)) foo.Attach(static_cast<MyLib::IFoo*>(vp));
I have a long-running console-based application Sender that sends simple text to STDOUT using non-buffered output such as cout << "Message" << flush(). I want to create an MFC dialog-based application (named Receiver) that starts Sender and can read it's output. Receiver should also be able to detect when Sender has died, or be able to kill Sender if it wants to. Sender knows nothing of Reciever, and I can't change Sender's code.
I have asked a separate question about the best way to do this. My first attempt was to create pipes with redirected STDIN and STDOUT for the child process and use asynchronous ReadFileEx calls to read in Sender's data. This isn't working correctly, because the ReadFileEx function only fires once, and only with zero bytes transferred even though I know for a fact that Sender is sending data.
I am creating 2 pipes with redirected STDIN and STDOUT, ala this MS example:
// allow the child process to inherit handles
SECURITY_ATTRIBUTES sa = {0};
sa.nLength = sizeof(sa);
sa.bInheritHandle = 1;
// create pipes with rerouted stdin & stdout
CreatePipe(&handles[h_Child_StdOut_Read], &handles[h_Child_StdOut_Write], &sa, 0);
SetHandleInformation(handles[h_Child_StdOut_Read], HANDLE_FLAG_INHERIT, 0);
CreatePipe(&handles[h_Child_StdIn_Read], &handles[h_Child_StdIn_Write], &sa, 0);
SetHandleInformation(handles[h_Child_StdIn_Read], HANDLE_FLAG_INHERIT, 0);
...Receiver then goes on to start Sender via CreateProcess():
// create child process
PROCESS_INFORMATION pi = {0};
STARTUPINFO si = {0};
si.cb = sizeof(si);
si.hStdOutput = handles[h_Child_StdOut_Write];
si.hStdInput = handles[h_Child_StdIn_Read];
si.dwFlags |= STARTF_USESTDHANDLES;
CreateProcess( 0, "Sender.EXE", 0, 0, 1, 0, 0, 0, &si, &pi);
handles[h_Child_Process] = pi.hProcess;
handles[h_Child_Thread] = pi.hThread;
My main loop is based on WaitForObjectsEx, placed in to an alertable wait state to support the asynch file read. I am waiting on two handles: one that signals when Sender dies prematurely, and one that signals when Receiver's main thread wants Sender to die. Before starting the loop, I kick off an overlapped (asynchronous) file read operation on Sender's STDOUT. Ignore the obvious memory leaks and other hacks -- this is illustrative:
vector<HANDLE> wait_handles;
wait_handles.push_back(handles[h_Die_Sig]);
wait_handles.push_back(handles[h_Child_Process]);
for( bool cont = true; cont; )
{
IO* io = new IO;
memset(io, 0, sizeof(IO));
io->buf_size_ = 16 * 1024;
io->buf_ = new char[io->buf_size_];
memset(io->buf_, 0, io->buf_size_);
io->thread_ = ¶m;
io->file_ = handles[h_Child_StdOut_Read];
if( !ReadFileEx(io->file_, io->buf_, io->buf_size_, io, OnFileRead) )
{
DWORD err = GetLastError();
string err_msg = util::strprintwinerr(err);
}
DWORD rc = WaitForMultipleObjectsEx(wait_handles.size(), &wait_handles[0], FALSE, INFINITE, TRUE);
// ...
}
The IO object above is derived publicly from OVERLAPPED:
struct IO : public OVERLAPPED
{
char* buf_;
DWORD buf_size_;
DWORD read_;
ThreadParam* thread_;
HANDLE file_;
};
When the overlapped Read function completes, I read the incoming data and generate a string:
void CALLBACK OnFileRead(DWORD err, DWORD bytes, OVERLAPPED* ovr)
{
IO* io = static_cast<IO*>(ovr);
string msg(io->buf_, bytes);
}
Sender knows nothing of Receiver, and it sends text to the console using very simple, but non-buffered means.
The problem: I know that Sender is sending data to its STDOUT, but my OnFileRead function is called only once, and only with zero bytes transferred.
Why can't I receive Sender's output this way? Do I have a bug, or am I doing something wrong?
Besides the error pointed out by #DyP, you are assuming that CreatePipe opened the handle in overlapped mode. Your assumption is incorrect. Microsoft documents it:
Asynchronous (overlapped) read and write operations are not supported
by anonymous pipes. This means that you cannot use the ReadFileEx and
WriteFileEx functions with anonymous pipes. In addition, the
lpOverlapped parameter of ReadFile and WriteFile is ignored when these
functions are used with anonymous pipes.
(Indeed, if you look inside kernel32.dll, on Windows XP for example, CreatePipe does not set the lower bit on the seventh parameter to NtCreateNamedPipeFile; that bit is set when CreateNamedPipe is called with FILE_FLAG_OVERLAPPED.)
Look for Dave Hart's MyCreatePipeEx implementation; it can be used as a drop-in replacement for CreatePipe when overlapped I/O is needed. Simply change PipeSerialNumber++ to InterlockedIncrement(&PipeSerialNumber) to avoid race conditions in MT code.
I think you have a typo:
CreatePipe(&handles[h_Child_StdOut_Read], &handles[h_Child_StdOut_Write], &sa, 0);
SetHandleInformation(handles[h_Child_StdOut_Read], HANDLE_FLAG_INHERIT, 0);
CreatePipe(&handles[h_Child_StdIn_Read], &handles[h_Child_StdIn_Write], &sa, 0);
SetHandleInformation(handles[h_Child_StdIn_Read], HANDLE_FLAG_INHERIT, 0);
change the last one to
SetHandleInformation(handles[h_Child_StdIn_Write], HANDLE_FLAG_INHERIT, 0);
that's also what they do at the MSDN example.