Application Crashes on Thread Exit - C++ - c++

My application is crashing when it exits the thread function. This is how my thread is initialized:
LPTHREAD_START_ROUTINE pThreadStart = (LPTHREAD_START_ROUTINE)NotifyWindowThreadFn;
void * pvThreadData = reinterpret_cast<void *>(_pobjSerialPort);
// Create the exit notify window thread event handle.
_hNotifyWindowThreadExitEvent = ::CreateEvent(
NULL, // No security
TRUE, // Create a manual-reset event object
FALSE, // Initial state is non-signaled
NULL // No name specified
);
if ( _hNotifyWindowThreadExitEvent == NULL )
{
TRACE(_T("CreateNotifyWindow : Failed to get a handle for the exit message-only window event.\r\n\tError: %d\r\n\tFile: %s\r\n\tLine: %d\r\n"), ::GetLastError(), __WFILE__, __LINE__);
return ::GetLastError();
}
// Create the notify window thread to begin execution on its own.
_hNotifyWindowThread = ::CreateThread(
NULL, // No security attributes.
0, // Use default initial stack size.
pThreadStart, // Function to execute in new thread.
pvThreadData, // Thread parameters.
0, // Use default creation settings.
NULL // Thread ID is not needed.
);
if ( _hNotifyWindowThread == NULL )
{
TRACE(_T("CreateNotifyWindow : Failed to create handle for message-only window thread.\r\n\tError: %d\r\n\tFile: %s\r\n\tLine: %d\r\n"), ::GetLastError(), __WFILE__, __LINE__);
return ::GetLastError();
}
This is the portion of my thread function that gets executed:
DWORD NotifyWindowThreadFn( void * pParam )
{
static CNotifyWindow * pobjNotifyWindow = NULL;
CSerialPort * pobjSerialPort = reinterpret_cast<CSerialPort *>(pParam);
// Create notify window to handle surprize removal/insertion events...
try
{
pobjNotifyWindow = new CNotifyWindow();
}
catch ( DWORD error )
{
return error; // 1. PC gets here
}
catch ( long error )
{
return error;
}
catch ( ... )
{
return ERROR_CANNOT_MAKE;
}
/* Other stuff that is not executed due to return. */
} // 2. PC then gets here
When the application crashes, Visual Studio gives me this error message:
Windows has triggered a breakpoint in CppTestConsole.exe.
This may be due to a corruption of the heap, which indicates a bug in CppTestConsole.exe or any of the DLLs it has loaded.
This may also be due to the user pressing F12 while CppTestConsole.exe has focus.
The output window may have more diagnostic information.
The output window doesn't have anything especially useful. Only...
The thread 'NotifyWindowThreadFn' (0x414) has exited with code 0 (0x0).
Then it shows that a bunch of DLLs are unloaded. When I click the Break button, the PC is at the end of _CrtIsValidHeapPointer in dbgheap.c. Does anyone have any ideas as to why my application is crashing when the thread exits? Should I not be returning directly from within a threaded function? Thanks.

I might be wrong, but it seems like you're trying to create a window from a worker thread. Don't do this. Windows need the message pump in order to function, and there's only one message pump in your application -- it's in the main thread.

you should declare and define you function as : DWORD WINAPI NotifyWindowThreadFn( void * pParam )

Try using _beginthreadex instead of CreateThread:
A thread in an executable that calls
the C run-time library (CRT) should
use the _beginthreadex and
_endthreadex functions for thread management rather than CreateThread
and ExitThread; this requires the use
of the multi-threaded version of the
CRT. If a thread created using
CreateThread calls the CRT, the CRT
may terminate the process in
low-memory conditions.

Related

Access violation when unloading injected DLL via CreateRemoteThread

I'm trying to cleanly inject DLL into foreground window's process, call simple function from this DLL and then cleanly unload the DLL. My injection code:
HWND fgwnd = GetForegroundWindow();
DWORD cur_thread = GetCurrentThreadId();
DWORD fg_pid = 0;
DWORD fg_thread = GetWindowThreadProcessId(fgwnd, &fg_pid);
BOOL res = 0;
const char* inj_path = "C:\\Users\\pc\\source\\repos\\hothook\\x64\\Debug\\fground_injector.dll";
// Get process handle to victim
HANDLE victim = OpenProcess(PROCESS_ALL_ACCESS, FALSE, fg_pid);
// Find exact adress of LoadLibraryA function from text space of kernel32.dll loaded by the OS
// and used by victim
PVOID llib = (LPVOID)GetProcAddress(GetModuleHandle(L"kernel32.dll"), "LoadLibraryA");
PVOID flib = (LPVOID)GetProcAddress(GetModuleHandle(L"kernel32.dll"), "FreeLibrary");
// Allocate memory inside victim's address space
LPVOID inj_path_victim = (LPVOID)VirtualAllocEx(victim, NULL, strlen(inj_path)+1,
MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE);
// Write inject dll's adress into victim
SIZE_T written;
res = WriteProcessMemory(victim, inj_path_victim, inj_path, strlen(inj_path)+1, &written);
// Finally, inject DLL into victim!
// Spawn thread in remote process ================================================================
HANDLE inj_llib_thread = CreateRemoteThread(
/*I*/ victim, // Handle to process where thread will be created
/*I*/ NULL, // SECURITY_ATTRIBUTES for new thread
/*I*/ 0, // Initial stacks size, bytes. 0 -> default size
/*I*/ (LPTHREAD_START_ROUTINE)llib, // User defined callback LPTHREAD_START_ROUTINE
/*I*/ inj_path_victim, // Ptr to variable to be sent as func parameter
/*I*/ 0, // Creation control flags. 0 -> immediate start
/*O|O*/ NULL); // Ptr to variable that recieves thread ID
// ===============================================================================================
// CANNOT WAIT FOR THREAD IN OTHER PROCESS.... OR CAN I?! I CAN!
// Wait for DLL to get properly injected into victim
res = WaitForSingleObject(inj_llib_thread, INFINITE);
// Get executable base address of the loaded DLL
DWORD llib_exit;
res = GetExitCodeThread(inj_llib_thread, &llib_exit);
// Free previously allocated remote memory
res = VirtualFreeEx(victim, inj_path_victim, 0, MEM_RELEASE);
// Call injected DLL's function
HMODULE fg_inj = LoadLibraryA(inj_path);
PVOID inj_t_proc = (LPVOID)GetProcAddress(fg_inj, "injectThread");
PVOID ulib = NULL;
// Spawn thread in remote process ================================================================
HANDLE inj_thread = CreateRemoteThread(
/*I*/ victim, // Handle to process where thread will be created
/*I*/ NULL, // SECURITY_ATTRIBUTES for new thread
/*I*/ 0, // Initial stacks size, bytes. 0 -> default size
/*I*/ (LPTHREAD_START_ROUTINE)inj_t_proc, // User defined callback LPTHREAD_START_ROUTINE
/*I*/ NULL, // Ptr to variable to be sent as func parameter
/*I*/ 0, // Creation control flags. 0 -> immediate start
/*O|O*/ NULL); // Ptr to variable that recieves thread ID
// ===============================================================================================
// Wait before injected DLL's thread finishes before extraction
res = WaitForSingleObject(inj_thread, INFINITE);
// Extract injected DLL from victim
// Spawn thread in remote process ================================================================
HANDLE inj_flib_thread = CreateRemoteThread(
/*I*/ victim, // Handle to process where thread will be created
/*I*/ NULL, // SECURITY_ATTRIBUTES for new thread
/*I*/ 0, // Initial stacks size, bytes. 0 -> default size
/*I*/ (LPTHREAD_START_ROUTINE)flib, // User defined callback LPTHREAD_START_ROUTINE
/*I*/ (LPVOID)llib_exit, // Ptr to variable to be sent as func parameter
/*I*/ 0, // Creation control flags. 0 -> immediate start
/*O|O*/ NULL); // Ptr to variable that recieves thread ID
// ===============================================================================================
// Wait untill injected DLL is fully extracted from the victim
res = WaitForSingleObject(inj_flib_thread, INFINITE);
DWORD flib_exit;
res = GetExitCodeThread(inj_flib_thread, &flib_exit);
// Extract injection DLL from host
FreeLibrary(fg_inj);
// ^^^ This actually does not unload DLL from host app, even if I call it 100 times in a loop...
// Clean up by closing all utilised handles
CloseHandle(victim);
CloseHandle(inj_llib_thread);
CloseHandle(inj_thread);
CloseHandle(inj_flib_thread);
My injection DLL is truly bare-bones:
BOOL APIENTRY DllMain(HMODULE hm,
DWORD call_reas,
LPVOID reserved)
{
switch(call_reas)
{
case DLL_PROCESS_ATTACH:
{
DisableThreadLibraryCalls(hm);
// I thought that threads calling DllMain with case 2-3 cause crash
// But no, same crash with or without this call
break;
case DLL_THREAD_ATTACH:
break;
case DLL_THREAD_DETACH:
break;
case DLL_PROCESS_DETACH:
break;
}
return TRUE;
}
DWORD injectThread(LPVOID p)
{
//AllowSetForegroundWindow(ASFW_ANY);
// This is what it originally suppose to do
// Let the host "steal" foreground window status from victim
// But even commenting this out results in the same crash
return 0;
}
The crash happens after I create FreeLibrary remote thread to unload injected DLL. This is as much crash details as I could get, as it happens in the system kernel it seems:
Exception thrown at 0x00007FFE598C7170 in maudswch.exe: 0xC0000005: Access violation executing location 0x00007FFE598C7170.
00007ffe598c7170()
kernel32.dll!00007ffe9ac47034()
ntdll.dll!00007ffe9c682651()
Not Flagged 50864 0 Main Thread Main Thread win32u.dll!00007ffe99e51104
Not Flagged 47548 0 Worker Thread ntdll.dll thread ntdll.dll!00007ffe9c6d0794
Not Flagged 41096 0 Worker Thread ntdll.dll thread ntdll.dll!00007ffe9c6d0794
Not Flagged 43456 0 Worker Thread ntdll.dll thread ntdll.dll!00007ffe9c6d0794
Not Flagged > 29820 0 Worker Thread Win64 Thread 00007ffe598c7170
I tested this on many "victims". CMD, Notepad, my other custom GUI program. The result is the same. DLL gets successfully injected, DLL function gets executed, but when unloading it crashes "victim". One more detail, is that FreeLibrary remote thread takes quite long time to finish, about 1.5 seconds sometimes even 7 seconds!
I'm trying to solve this puzzle for days now, I searched all the Internet, but all other similar cases seems to be non-related... And at this point I have no idea what is going on and how to fix this. Of course, I can just bite the bullet and never try to unload my injected DLL, after all it works. But I don't want to leave useless DLLs in other processes after it finished it's job.
Edit:
Some debug prints. As you can see, hm: and func addr: is outputted in DllMain PROCESS_ATTACH. As you can see, I'm lucky to have base DLL address and function address to be exactly the same in host and victim processes. However, passing full base address to FreeLibrary external thread results in exactly the same crash, so truncated base address was not the real reason behind the crash...
Using GetExitCodeThread() to retrieve the HMODULE returned by LoadLibraryA() in the target process will work only if the victim is a 32bit process. Otherwise, the HMODULE will be too large to fit in the remote thread's exit code, so you will have to use a different mechanism to get the DLL's base address in the target process, such as using EnumProcessModules()+GetModuleFileNameEx() or CreateToolhelp32Snapshot()+Module32(First|Next)(), or even injecting stub code that calls LoadLibraryA() and saves the HMODULE into a variable that the injector allocates and can read from via ReadProcessMemory().
More importantly, the way you are calling the DLL's injectThread() function in the target process is wrong. You are using LoadLibraryA()+GetProcAddress() to get a pointer to the injectThread() function within the injector's process, and then you are assuming that the function is located at the same address in the target process. But the DLL in the target process may have been loaded at a different base address than the DLL in the injector process (especially in light of technologies like ALSR, DLL rebasing, etc).
The correct solution is to get the offset of the injectThread() function within the DLL (which the injector can calculate by subtracting its DLL's base address from the function's address), and then add that offset to the base address of the DLL in the target process.
Also, make sure your injectThread() function has the correct signature that CreateRemoteThread() is expecting. What you showed is missing a calling convention specified, so it will use whatever the compiler's default convention is, which is usually __cdecl. But CreateRemoteThread() requires __stdcall instead.

I want to write a kernel driver program where when a file is created I need to be notified

I used the reference of this project where it creates a handler for IRP_MJ_CREATE. which displays all the files which are created or opened the system.
The documentation of IRP_MJ_CREATE is this:
The I/O Manager sends an IRP_MJ_CREATE request when a new file or
directory is being created, or when an existing file, device,
directory, or volume is being opened.
Normally this IRP is sent on behalf of a user-mode application that
has called a Microsoft Win32 function such as CreateFile or on behalf
of a kernel-mode component that has called a function such as
IoCreateFile, IoCreateFileSpecifyDeviceObjectHint, ZwCreateFile, or
ZwOpenFile.
If the create request is completed successfully, the application or
kernel-mode component receives a handle to the file object.
This program below prints all the files or volumes which are opened, created.
main.c
for (i = 0; i <= IRP_MJ_MAXIMUM_FUNCTION; ++i)
{
DriverObject->MajorFunction[i] = FsFilterDispatchPassThrough;
}
//creating handle for IRP_MJ_CREATE.
DriverObject->MajorFunction[IRP_MJ_CREATE] = FsFilterDispatchCreate;
// IRP_MJ_CREATE IRP Handler
NTSTATUS FsFilterDispatchCreate(
__in PDEVICE_OBJECT DeviceObject,
__in PIRP Irp
)
{
PFILE_OBJECT pFileObject = IoGetCurrentIrpStackLocation(Irp)->FileObject;
DbgPrint("%wZ\n", &pFileObject->FileName);
return FsFilterDispatchPassThrough(DeviceObject, Irp);
}
I just need the driver to print only when a file or directory is created.
Your filter driver is being called before the actual file system driver. However you want to check the result after the file system driver handled the irp. according to this article, you can simply check irp->IoStatus.Information == FILE_CREATED.
Now you only need to let the file system driver know that you have more processing to do with the irp, so it won't call IoCompleteRequest and free the irp. To do so we can use scenario 2 out of the Cheat Sheet. It will end up something like this:
NTSTATUS
CompletionRoutine_2(
IN PDEVICE_OBJECT DeviceObject,
IN PIRP Irp,
IN PVOID Context
)
{
UNREFERENCED_PARAMETER(DeviceObject);
if (Irp->PendingReturned == TRUE) {
//
// You will set the event only if the lower driver has returned
// STATUS_PENDING earlier. This optimization removes the need to
// call KeSetEvent unnecessarily and improves performance because the
// system does not have to acquire an internal lock.
//
KeSetEvent ((PKEVENT) Context, IO_NO_INCREMENT, FALSE);
}
// This is the only status you can return.
return STATUS_MORE_PROCESSING_REQUIRED;
}
NTSTATUS
FsFilterDispatchCreate(
IN PDEVICE_OBJECT DeviceObject,
IN PIRP Irp
)
{
KEVENT event;
NTSTATUS status;
KeInitializeEvent(&event, NotificationEvent, FALSE);
//
// You are setting completion routine, so you must copy
// current stack location to the next. You cannot skip a location
// here.
//
IoCopyCurrentIrpStackLocationToNext(Irp);
IoSetCompletionRoutine(Irp,
CompletionRoutine_2,
&event,
TRUE,
TRUE,
TRUE
);
PFSFILTER_DEVICE_EXTENSION pDevExt = (PFSFILTER_DEVICE_EXTENSION)DeviceObject->DeviceExtension;
status = IoCallDriver(pDevExt->AttachedToDeviceObject, Irp);
if (status == STATUS_PENDING) {
KeWaitForSingleObject(&event,
Executive, // WaitReason
KernelMode, // must be Kernelmode to prevent the stack getting paged out
FALSE,
NULL // indefinite wait
);
status = Irp->IoStatus.Status;
}
// Your logic
if (Irp->IoStatus.Information == FILE_CREATED) {
PFILE_OBJECT pFileObject = IoGetCurrentIrpStackLocation(Irp)->FileObject;
DbgPrint("%wZ\n", &pFileObject->FileName);
}
//
// Because you stopped the completion of the IRP in the CompletionRoutine
// by returning STATUS_MORE_PROCESSING_REQUIRED, you must call
// IoCompleteRequest here.
//
IoCompleteRequest (Irp, IO_NO_INCREMENT);
return status;
}
When using the example you based on, you can simply change the function to this.

RegisterWaitForSingleObject crash sometime if handle is closed immediately

I'm getting a crash sometimes in RegisterWaitForSingleObject (1 out of 10). It seems that although RegisterWaitForSingleObject returns, the internal thread pool is not yet ready.
HANDLE processHandle = OpenProcess (PROCESS_ALL_ACCESS, FALSE, processID);
// CRASH IN INTERNAL SOMETIMES
RegisterWaitForSingleObject (&hWaitForChild_,processHandle,OnChildProcessExit, 0,INFINITE,WT_EXECUTEONLYONCE);
// If I sleep here, then it seems ok.
//std::this_thread::sleep_for (std::chrono::milliseconds (10));
CloseHandle (processHandle);
I can replicate this with a simple sample here. 1 in 10 times, it will crash. How should I be synchronizing it properly without resorting to sleep hack.
https://filedn.com/l3TGy7Y83c247u0RDYa9fkp/temp/stackoverflow/testregister.cpp
based on your code spinet:
// THIS CRASHS HERE SOMETIMES
if (! RegisterWaitForSingleObject (
&hWaitForChild_
,processHandle
, OnChildProcessExit
, 0 //this
, INFINITE
, WT_EXECUTEONLYONCE))
{
LogDebug ("RegisterWaitForSingleObject failed");
}
// If this is enabled, then it won't crash
//std::this_thread::sleep_for (std::chrono::milliseconds (10));
if (! CloseHandle (processHandle)) // !!!
LogDebug ("RegisterWaitForSingleObject Closehandle failed");
so you close processHandle just after you call RegisterWaitForSingleObject for this handle. however if read about RegisterWaitForSingleObject:
If this handle is closed while the wait is still pending, the
function's behavior is undefined.
if look more deep - try understand - how is RegisterWaitForSingleObject worked internally ? it pass processHandle to some worker thread. and this thread begin wait for this handle. but this is (pass handle to another thread) is asynchronous operation - say for example internally can be started new thread with this handle as argument, or it can be passed to already existing working thread via some signal. but anyway - worked thread got this handle and begin wait some later. from another side - you just close processHandle after RegisterWaitForSingleObject return control. so here race - what will be first - or worked thread begin wait on handle (in this case all will be work) or you close this handle. in case you close this handle first - worked thread will be try wait on already invalid handle and raise exception - STATUS_THREADPOOL_HANDLE_EXCEPTION.
// If this is enabled, then it won't crash
//std::this_thread::sleep_for (std::chrono::milliseconds (10));
of course - by sleep you give time for worked thread to begin wait on handle. in this case he begin wait before you close handle.
solution - you must not close handle, until WAITORTIMERCALLBACK Callback will be not called. you need allocate some context, where place processHandle and pass this context to RegisterWaitForSingleObject. and when you callback will be called - you got pointer to your context back and here and close handle.
also note, that you not need open separate, second, handle for child process, but can use process handle returned by CreateProcess
Ok, I managed to solve it by keeping the handle around until I call Unregisterwait. It seems to be stable. Thanks to the answers.
I met the same problem like you that an exception occurred while debugging in Visual Studio. I tried many time and finally found the reason. If you close the handle of the process newly created, the program would crash. I tried to close the handles in the callback function, it works perfectly, like this:
typedef struct {
LPTSTR pszCmdLine;
HANDLE hEvent;
} THREAD_PARAM;
typedef struct {
TCHAR szCmdLine[1024];
HANDLE hWaitObject;
DWORD dwProcessId;
HANDLE hProcess;
DWORD dwThreadId;
HANDLE hThread;
} OBJECT_PARAM;
static void CALLBACK WaitObjectCallback(LPVOID lpParam, BOOLEAN TimerOrWaitFired)
{
OBJECT_PARAM *pobp = static_cast<OBJECT_PARAM *>(lpParam);
TCHAR szInfo[1024] = { 0 };
DWORD dwExitCode = 0;
GetExitCodeProcess(pobp->hProcess, &dwExitCode);
wnsprintf(szInfo, ARRAYSIZE(szInfo), _T("process %u [%s] exit: %u\n"), pobp->dwProcessId, pobp->szCmdLine, dwExitCode);
OutputDebugString(szInfo);
//
// unregister the wait object handle and close the process handle finally
//
UnregisterWait(pobp->hWaitObject);
CloseHandle(pobp->hProcess);
CloseHandle(pobp->hThread);
GlobalFree(lpParam);
}
static DWORD CALLBACK ThreadFunction(LPVOID lpParam)
{
THREAD_PARAM *pthp = static_cast<THREAD_PARAM *>(lpParam);
STARTUPINFO si = { sizeof(si) };
PROCESS_INFORMATION pi = { 0 };
BOOL bResult;
bResult = CreateProcess(nullptr, pthp->pszCmdLine, nullptr, nullptr, FALSE, 0, nullptr, nullptr, &si, &pi);
if (bResult)
{
OBJECT_PARAM *pobp = static_cast<OBJECT_PARAM *>(GlobalAlloc(GPTR, sizeof(OBJECT_PARAM)));
// make copy of the command line and other informations of the newly created process
lstrcpyn(pobp->szCmdLine, pthp->pszCmdLine, ARRAYSIZE(pobp->szCmdLine));
pobp->dwProcessId = pi.dwProcessId;
pobp->hProcess = pi.hProcess;
pobp->dwThreadId = pi.dwThreadId;
pobp->hThread = pi.hThread;
bResult = RegisterWaitForSingleObject(&pobp->hWaitObject, pi.hProcess, WaitObjectCallback, pobp, INFINITE, WT_EXECUTEONLYONCE);
// once it failed...
if (!bResult)
{
CloseHandle(pi.hProcess);
CloseHandle(pi.hThread);
}
}
// Notify the thread creator that the process is created successfully
SetEvent(pthp->hEvent);
return 0;
}

Winapi Timer callback thread, never returns

I've got to debug some code which is not from me.
This code implement a timer API using winapi Timer interface.
I'm not very used to this Winapi functionality, so i could use your help :)
From what I understand this code is done like this :
=> Init()
timerQueue = CreateTimerQueue();
=> CreateTimer()
CreateTimerQueueTimer(timerHandle, timerQueue, timerCallback, ..., WT_EXECUTEDEFAULT);
=> timerCallback()
DeleteTimerQueueTimer(timerQueue , timerHandle, NULL));
calback() //Launch user-defined callback
=> CleanUp() // to be called at the end
DeleteTimerQueueEx(timerQueue , INVALID_HANDLE_VALUE);
When we test that, user-defined callback are executed successfully after the desired amount of time. But after that timerCallback threads keep pending and never return, preventing the all process to returns. Using VS debugger I can see those threads (named TppWorkerThread#4) on the thread...
Perhaps we miss something to make callback returns properly or we created some sort of deadlocks... However I cannot figure it out ...
Please let me know if I forgot some relevant information.
Thank you for your help.
EDIT:
Further information :
- Blocking thread are at this state at the end of the process :
* Category :Worker Thread
* Name : _TppWorkerThread#4
* Location : _ZwWaitForWorkViaWorkerFactory#8
* Priotity : Normal
EDIT2:
Having some more time to work on that strange behavior, I am now able to reproduce it in a standalone code.
#include <windows.h>
#include <stdio.h>
HANDLE gDoneEvent;
HANDLE hTimer[5];
HANDLE hTimerQueue = NULL;
HANDLE g_threadHandle;
void PeriodicCallback(void)
{
printf("Periodic routine called.\n");
}
void SingleCallback(void)
{
printf("Single routine called.\n");
if (!DeleteTimerQueueTimer(hTimerQueue, hTimer[2], NULL))
printf("DeleteTimerQueueTimer() fail. Return value is %d.\n", GetLastError());
}
void CALLBACK CommonCallback(PVOID lpParam, BOOLEAN TimerOrWaitFired)
{
printf("Common routine called. Parameter is %d.\n", *(int *)lpParam);
((void (*)(void))lpParam)();
}
void MainTest(void)
{
// Use an event object to track the TimerRoutine execution
gDoneEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
if (NULL == gDoneEvent)
{
printf("CreateEvent failed (%d)\n", GetLastError());
return -1;
}
if(0 == SetThreadPriority(GetCurrentThread(), THREAD_PRIORITY_BELOW_NORMAL))
{
printf("SetThreadPriority failed (%d)\n", GetLastError());
return -2;
}
// Create the timer queue.
hTimerQueue = CreateTimerQueue();
if (NULL == hTimerQueue)
{
printf("CreateTimerQueue failed (%d)\n", GetLastError());
return -3;
}
/*
if (!CreateTimerQueueTimer( &hTimer[2], hTimerQueue,
(WAITORTIMERCALLBACK)CommonCallback, &SingleCallback, 1000, 0, WT_EXECUTEDEFAULT))
{
printf("CreateTimerQueueTimer failed (%d)\n", GetLastError());
return -4;
}
*/
if (!CreateTimerQueueTimer( &hTimer[4], hTimerQueue,
(WAITORTIMERCALLBACK)CommonCallback, &PeriodicCallback, 10, 500, WT_EXECUTEDEFAULT))
{
printf("CreateTimerQueueTimer failed (%d)\n", GetLastError());
return -5;
}
// TODO: Do other useful work here
printf("Call timer routine in 10 seconds...\n");
Sleep(4000);
CloseHandle(gDoneEvent);
if (!DeleteTimerQueueTimer(hTimerQueue, hTimer[4], INVALID_HANDLE_VALUE))
printf("DeleteTimerQueueTimer failed (%d)\n", GetLastError());
// Delete all timers in the timer queue.
if (!DeleteTimerQueueEx(hTimerQueue, INVALID_HANDLE_VALUE))
printf("DeleteTimerQueue failed (%d)\n", GetLastError());
Sleep(1000);
ExitThread(0);
}
int main(int argc, char **argv[])
{
if(g_threadHandle == CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)MainTest, NULL, 0, NULL))
printf("Creation fail");
ExitThread(0);
}
I'm compiling this code on VisualStudio 2010 Professional.
It appears that event after calling DeleteTimerQueueTimer() some threads remain pending on the threads pool, preventing my process to shutdown. I still cannot figured it out ...
When you call DeleteTimerQueueEx with an INVALID_HANDLE_VALUE as its second parameter it will block until all callbacks which are running have completed. The Error may be in one of your callback functions which never returns.
You are calling DeleteTimerQueueTimer(timerQueue , timerHandle, NULL); with NULL as the third parameter, this will not wait for the callback to complete if one is running at the time you delete the timer. I suggest using DeleteTimerQueueTimer(timerQueue , timerHandle, INVALID_HANDLE_VALUE) which will block until the call back completes (if one is running). Calling cleanUp() without using the blocking version of DeleteTimerQueueTimer is likely a bug as you may be cleaning up at the same time as the callback is executing.
It could also be a problem of calling DeleteTimerQueueEx or DeleteTimerQueueTimer from within a callback, which is forbidden. Break on execution of DeleteTimerQueueEx and look at what thread you are in, if its a TppWorkerThread than you have found your bug.
EDIT:
In your comment you say you do call DeleteTimerQueueTimer from within the callback but don't use INVALID_HANDLE_VALUE, reading the documentation again from http://msdn.microsoft.com/en-us/library/windows/desktop/ms682569%28v=vs.85%29.aspx this does seem to be legal but I distinctly remember us making design decisions to avoid this, I'm sorry this is so vague, I hope someone can give authoritative advice on this.
We send an event/message to the queue of a non timer thread which then removes the timer, you could even have a dedicated thread for this but that is probably overkill. At the end of the day you need to be sure that the timer is removed before doing cleanup so you have to either block on removal or have some other thread do it upon signaling of an event.
After some work on that issue I think I got to an answer.
I appears that this timerQueue API is coded on top of threadPool winAPI, and when we ask to create a timerQueue Windows create a thread pool from where all callback will be launched.
Until here no problem, but, when we ask for timerQueue deletion, it appears that this thread pool is not deleted...
This result in some thread keeping pending waiting to be used and preventing the process to returns.
After some time (timeout??) those threads returns and the process exit.
I don't really get why this pool is not closed... but, now, I use a workaround :
exit(0);
At the end of my program, it's a bit brutal but it does the job (ie: killing my process, whatever threads are still pending or not)

Safely synchronizing a COM thread

I've done a lot with multithreading in the past, but I'm fairly new to COM. Anyway, here's my issue:
I create a worker thread, which registers as an STA, and creates a COM object. Then the worker thread and the main thread try to communicate with each other. Using CoMarshalInterThreadInterfaceInStream and CoGetInterfaceAndReleaseStream, I can get the threads to call methods on the COM objects in the other thread.
Here's what the worker thread looks like:
void workerThread()
{
CoInitialize(NULL);
MyLib::IFooPtr foo = ...; // create my COM object
// Marshall it so the main thread can talk to it
HRESULT hr = CoMarshalInterThreadInterfaceInStream(foo.GetIID(),
foo.GetInterfacePtr(),
&m_stream);
if (FAILED(hr)) {
// handle failure
}
// begin message loop, to keep this STA alive
MSG msg;
BOOL bRet;
while( (bRet = GetMessage( &msg, NULL, 0, 0 )) != 0)
{
if (bRet == -1) break;
DispatchMessage(&msg);
}
}
In the main thread:
// launch the thread
m_worker = boost::thread (&workerThread);
// get the interface proxy
MyLib::IFooPtr foo;
LPVOID vp (NULL);
HRESULT hr = CoGetInterfaceAndReleaseStream(m_stream, foo.GetIID(), &vp);
if (SUCCEEDED(hr)) foo.Attach(static_cast<MyLib::IFoo*>(vp));
This creates the object (which takes a while to initialize), and allows the main thread to talk to it, and everything is properly synchronized with the COM Apartment stuff. As far as I can tell from reading msdn, this seems to be the right way to do things. Now the main thread can use its proxy to call methods on my COM object, and the worker thread will receive those calls over the message queue, properly dispatching them.
However, what about synchronizing these threads?
Obviously in this case I want the main thread to wait to call CoGetInterfaceAndReleaseStream until after the worker thread has created that stream via CoMarshalInterThreadInterfaceInStream. But how can I safely do that?
From MSDN, I should be using something like MsgWaitForMultipleObjects, so I can wait for my_condition OR new_message_arrived, and then I can do something like:
// verbatim from msdn
while (TRUE)
{
// wait for the event and for messages
DWORD dwReturn = ::MsgWaitForMultipleObjects(1,
&m_hDoneLoading, FALSE, INFINITE, QS_ALLINPUT);
// this thread has been reawakened. Determine why
// and handle appropriately.
if (dwReturn == WAIT_OBJECT_0)
// our event happened.
break ;
else if (dwReturn == WAIT_OBJECT_0 + 1)
{
// handle windows messages to maintain
// client liveness
MSG msg ;
while(::PeekMessage(&msg, NULL, 0, 0, PM_REMOVE))
::DispatchMessage(&msg) ;
}
}
But how do I mix boost::thread.join() and boost::condition.wait() with MsgWaitForMultipleObjects? Is that even possible, or do I have to do something else to avoid a race condition?
Your main thread has a message queue (must be, since is an STA host), why not simply post a message to it, PostThreadMessage? Post an user message (WM_USER +X) and your normal main thread message pump can handle this user message as a notification that the COM object has marshaled the interface into the stream and the main thread is safe to call CoGetInterfaceAndReleaseStream.
I must call out though that with your current design your worker thread does basically nothing more than just run an additional message pump. Any call to any method on your interface from the main thread will block, wait for the worker thread to pick up the message from its message queue, process the call, respond, and then the main thread will resume. All operations will be at least as slow as having the COM object hosted in the main thread, plus the overhead of COM marshaling back and forth between the two STAs. Basically there is no concurrency whatsoever between the two threads, because of how COM STA works. Are you sure is this what you want?
Edit
(omitting a bunch of details like number of threads, timeout handling, assignment of a stream/IID/CLSID for each worker etc etc)
in the .h:
HANDLE m_startupDone;
volatile int m_threadStartCount;
worker thread:
void workerThread()
{
CoInitialize(NULL);
MyLib::IFooPtr foo = ...; // create my COM object
// Marshall it so the main thread can talk to it
HRESULT hr = CoMarshalInterThreadInterfaceInStream(foo.GetIID(),
foo.GetInterfacePtr(),
&m_stream);
if (FAILED(hr)) {
// handle failure
// remember to decrement and signal *even on failure*
}
if (0 == InterlockedDecrement(&m_threadStartCount))
{
SetEvent (m_startupDone);
}
// begin message loop, to keep this STA alive
MSG msg;
BOOL bRet;
while( (bRet = GetMessage( &msg, NULL, 0, 0 )) != 0)
{
if (bRet == -1) break;
DispatchMessage(&msg);
}
}
in the main thread:
m_startupDone = CreateEvent (NULL, FALSE, FALSE, NULL);
m_threadStartCount = <number of workerthreads>
// launch the thread(s)
m_worker = boost::thread (&workerThread);
m_worker2 = boost::thread (&workerThread);
...
// now wait for tall the threads to create the COM object(s)
if (WAIT_OBJECT0 != WaitForSingleObject(m_startupDone, ...))
{
// handle failure like timeout
}
// By now all COM objects are guaranteed created and marshaled, unmarshall them all in main
// here must check if all threads actually succeeded (could be as simple as m_stream is not NULL)
// get the interface proxy
MyLib::IFooPtr foo;
LPVOID vp (NULL);
HRESULT hr = CoGetInterfaceAndReleaseStream(m_stream, foo.GetIID(), &vp);
if (SUCCEEDED(hr)) foo.Attach(static_cast<MyLib::IFoo*>(vp));