Why win32 thread doesn't exit automatically? - c++

Background:
In my application written in C++, I create a worker thread which in turn creates two threads using CreateThread(). The two threads which worker thread creates, talk to WCF Service through a client which is implemented using Windows Web Services API which offers C/C++ application programming interface (API) for building SOAP based web services and clients to them. My application implements only the client using this API.
Problem:
The problem I'm facing is that all other threads exit gracefully, except the worker thread, as you can see yourself, in the image below that WorkerThreadProc uses no CPU cycles yet it doesn't exit. There are also few other threads running which are not created by me, but by the runtime.
The thread states are as follows (as reported by ProcessExplorer):
WorkerThreadProc is in Wait:WrUserRequest state.
wWinMainCRTStartup is in Wait:UserRequest state.
All TpCallbackIndependent are in Wait:WrQueue state.
What are they waiting for? What could be possible causes that I need to look into? Also, what is the difference between WrUserRequest and UserRequest? And what does WrQueue mean? I've absolutely no idea what is going on here.
Here is my WorkerThreadProc code. I've removed all the logging statements except the last one at the bottom of the function:
DWORD WINAPI WorkerThreadProc(PVOID pVoid)
{
//Initialize GDI+
GdiplusStartupInput gdiplusStartupInput;
ULONG_PTR gdiplusToken;
Status status = GdiplusStartup(&gdiplusToken, &gdiplusStartupInput, NULL);
if ( status != Status::Ok )
{
return 1;
}
GuiThreadData *pGuiData = (GuiThreadData*)pVoid;
auto patternIdRequestQueue= new PatternIdRequestQueue();
auto resultQueue = new ResultQueue();
auto patternManager = new PatternManager(patternIdRequestQueue);
LocalScheduler *pScheduler = new LocalScheduler(resultQueue, patternManager);
bool bInitializationDone = pScheduler->Initialize(pGuiData->m_lpCmdLine);
if ( !bInitializationDone )
{
return 0;
}
//PatternIdThread
PatternIdThread patternIdThread(patternIdRequestQueue);
DWORD dwPatternIdThreadId;
HANDLE hPatternIdThread = CreateThread(NULL, 0, PatternIdThreadProc, &patternIdThread, 0, &dwPatternIdThreadId);
ResultPersistence resultPersistence(resultQueue);
DWORD dwResultPersistenceThreadId;
HANDLE hResultPersistenceThread = CreateThread(NULL, 0, ResultPersistenceThreadProc, &resultPersistence, 0, &dwResultPersistenceThreadId);
pScheduler->ScheduleWork(pGuiData->m_hWnd, pGuiData->m_hInstance, ss.str());
pScheduler->WaitTillDone();
patternIdThread.Close();
resultPersistence.Close();
delete pScheduler;
//Uninitialize GDI+
GdiplusShutdown(gdiplusToken);
dwRet = WaitForSingleObject(hPatternIdThread, INFINITE);
CloseHandle(hPatternIdThread);
dwRet = WaitForSingleObject(hResultPersistenceThread,INFINITE);
CloseHandle(hResultPersistenceThread);
SendMessage(pGuiData->m_hWnd, WM_CLOSE, 0, 0);
//IMPORTANT : this verbose message is getting logged!
T_VERBOSE(EvtSrcInsightAnalysis, 0, 0, "After sending message to destroy window");
delete patternManager;
delete patternIdRequestQueue;
delete resultQueue;
return 0;
}
Please see the T_VERBOSE macro, it is used to log verbose message. I see the message is getting logged, yet the thread doesn't exit!
EDIT:
I just commented the following line in my WorkerThreadProc, then worker thread exits gracefully!
SendMessage(pGuiData->m_hWnd, WM_CLOSE, 0, 0);
Does it mean that SendMessage is the culprit? Why would it block the thread the calling thread?

If we look at the docs for SendMessage, you can see this little quote:
To send a message and return immediately, use the SendMessageCallback
or SendNotifyMessage function. To post a message to a thread's message
queue and return immediately, use the PostMessage or PostThreadMessage
function.
and this:
Messages sent between threads are processed only when the receiving
thread executes message retrieval code. The sending thread is blocked
until the receiving thread processes the message. However, the sending
thread will process incoming nonqueued messages while waiting for its
message to be processed. To prevent this, use SendMessageTimeout with
SMTO_BLOCK set. For more information on nonqueued messages, see
Nonqueued Messages.
so from this we can see SendMessage will block till the message is processed, which may somehow lead to a deadlock in your code, as the msgproc doesn't reside in your worker thread, leading to a context switch (which is only triggered when the thread's queue is pumped for messages). Try using PostMessage, which immediately returns.
EDIT: there is also a nice little piece of info here on message deadlocks from SendMessage

Related

How to keep my camera alive when I use Cannon EDSDK in my C++ code?

I think it is essential to send a message to the camera per some minutes ,so I initialize my camera and send message to camera every once in a while in my main thread , and in the other thread I open the liveview to process my other jobs, but in the liveview thread , it wait time out when sending this message :
EdsSetPropertyData(theCamera, kEdsPropID_Evf_Mode, 0, sizeof(evfMode), &evfMode)
I never met this when all jobs handled in just one thread, I don't know why it happends like this , could someone help me ? Here is some of my code.
my main thread : (I don't send keepalive message in the thread,but also timeout when starting liveview !)
CanonCameraWrapper& wrapper = param->wrapper;
bool setup_ok = wrapper.setup(0);
if (!setup_ok)
{
wrapper.destroy();
}
wrapper.setDownloadPath("img");
pthread_t camera_thread;
pthread_create(&camera_thread, NULL, camera_thread_func, (void *)(param));
pthread_join(camera_thread, NULL);
the other thread
void * camera_thread_func(void * arg)
{
global_param* param = (global_param*)arg;
CanonCameraWrapper& wrapper = param->wrapper;
wrapper.beginLiveView();//**//it wait time out here!!!**
...
}
I believe there are two things you have to be aware of. One is this notice in the Canon SDK documentation:
When creating applications that run under Windows, a COM initialization is required for each thread in order to
access a camera from a thread other than the main thread.
To create a user thread and access the camera from that thread, be sure to execute CoInitializeEx( NULL,
COINIT_APARTMENTTHREADED ) at the start of the thread and CoUnInitialize() at the end.
Sample code is shown below. This is the same when controlling EdsVolumeRef or EdsDirectoryItemRef objects
from another thread, not just with EdsCameraRef
The other thing is: you cannot access (most of) the SDK at the same time. So if you are using multiple threads you have to be careful about this

Invalid handle error in TerminateThread

I have an application, which has registered handle routine via SetConsoleCtrlHandler . Thing is on break signal I have to do some cleanup before exiting, and that is when problems begin. Because, other threads keeps on running and using some of the resources which should freed I got access violation during cleanup. I tried to save thread instances which are using these resources and stop them before calling cleanup routine, but this did not help. I constantly get ERROR_INVALID_HANDLE. I tried to get thread instance via DuplicateHandle also I tried to save only thread id and then get thread handle via OpenThread function. But still invalid handle error is displayed.
Also, when I enumerate all thread instance and trying to terminate them I get the same error. It seems to me like there are some "magic" around handle routine, which does not allow to terminate other threads from it.
Listing for enumerating all threads
HANDLE h = CreateToolhelp32Snapshot(TH32CS_SNAPTHREAD, 0);
if (h != INVALID_HANDLE_VALUE) {
THREADENTRY32 te;
te.dwSize = sizeof(te);
if (Thread32First(h, &te)) {
do {
if (te.dwSize >= FIELD_OFFSET(THREADENTRY32, th32OwnerProcessID) +
sizeof(te.th32OwnerProcessID)) {
HANDLE thread = OpenThread(THREAD_TERMINATE, FALSE, te.th32ThreadID);
if (TerminateThread(thread , NUM_SIG_ACTION_1) == 0)
{
// printf("terminate failed (%d)\n", GetLastError());
if( SuspendThread(thread ) == -1)
{
// printf("suspend failed (%d)\n", GetLastError());
}
}
}
te.dwSize = sizeof(te);
} while (Thread32Next(h, &te));
}
CloseHandle(h);
}
Listing for get thread via DuplicateHandle
//code in working thread
DuplicateHandle(GetCurrentProcess(), GetCurrentThread(), GetCurrentProcess(), &g_hWorkerThread ,
DUPLICATE_SAME_ACCESS, FALSE, DUPLICATE_SAME_ACCESS);
//code in handler routine
if (TerminateThread( g_hWorkerThread , NUM_SIG_ACTION_1 ) == 0)
{
printf("Terminate thread failed (%d)\n", GetLastError());
}
Listing for OpenThread
//code from worker thread
g_iWorkerThreadId = GetThreadId(GetCurrentThread());
//code in routine handler
HANDLE handle = OpenThread(THREAD_TERMINATE, FALSE, g_iWorkerThreadId);
if (TerminateThread( handle , NUM_SIG_ACTION_1 ) == 0)
{
printf("Terminate thread failed (%d)\n", GetLastError());
}
Is there anyway to force handle routine run in to current worker thread?
Any help will be appreciated.
Thanks on advance.
You're trying to do something yourself that the OS already does. Don't fight the system.
Let me suggest another way of working, that could simplify your code considerably.
When your main application thread gets the Ctrl signal, it wants to shut down the application.
You can have Windows do that for you, but you might leave resources in an undefined state, so that's why you want to shut down gracefully.
Likely, each thread already runs an event loop - that's the basic Windows applications paradigm. Upon the ctrl-signal, post events to your other threads and catch them in each thread's event loop. Add custom event code that will make them gracefully end themselves. You now only have to wait for all the thread handles to ensure all threads have ended.
Using TerminateThread is not a good idea, it is exactly the opposite of the graceful end of a thread.
How one can sure calling TerminateThread will free the resource that is in use?
Only killing a process guarantees freeing of kernel resources used by the process.
In my humble experiences, I didn't found any multi-thread program runs good and stably without the clear and explicit exit mechanism.
Each thread must have an exit condition, when exiting it must free the resources, and the main thread (or controlling thread) must trigger the exit conditions and wait until all the other thread are exit.

Invalid HANDLE from AfxBeginThread

I am writing a multi-threaded networked application, and I'm using a separate thread with a blocking socket to receive data asynchronously from the server.
When I need to shutdown the socket I use a function which checks if the receive thread is still running and if it is calls TerminateThread to end it as follows:
DWORD dwExitCode = 0;
if( GetExitCodeThread( theApp.m_hRecvThread, &dwExitCode ) && dwExitCode == STILL_ACTIVE )
TerminateThread( theApp.m_hRecvThread, 0 );
However, GetExitCodeThread returns FALSE, and when polling GetLastError() it returns 6 (ERROR_INVALID_HANDLE). Which suggests that I do not have the THREAD_QUERY_INFORMATION or THREAD_QUERY_LIMITED_INFORMATION access rights on the m_hRecvThread handle.
My m_hRecvThread handle is set when creating the thread like so:
m_hRecvThread = AfxBeginThread( RecvThread, hWndMainFrame );
This successfully creates the thread, and the thread is running fine and exhibiting expected functionality. The TerminateThread and GetExitCodeThread are being called from the same thread which created the Receive thread in the first place.
My understanding was that when using AfxBeginThread, the HANDLE returned had THREAD_ALL_ACCESS access rights, is this the case, and if so, why am I still getting ERROR_INVALID_HANDLE?
Thanks in advance!
The thread created using:
m_hRecvThread = AfxBeginThread( RecvThread, hWndMainFrame )
will return a pointer to winthread. But GetExitCodeThread() requires the handle to the thread for example, you can pass RecvThread->m_hThread, which will solve the issue

Internal message loop

In my main dialog I have a function that creates a process and waits for it to finish. It might take up to 15-20 seconds. If I simply wait using WaitForSingleObject my dialog becomes unresponsive.
I want to use a combination of EnableWindow(FALSE), and an internal message loop to make my dialog block, but without looking like the app freezes, the way MessageBox and DoModal do. But I'm not sure how to do that an internal message loop.
I'm afraid your approach won't work. Your app is single-threaded, or at least your UI is. After you call WaitForSingleObject your thread is put to sleep and it won't process windows messages. The fact that you have an internal message loop won't matter. You should probably start a new thread and use it to wait for the process to finish, then notify your UI thread and exit. Or something along those lines.
Running internal message loop is rather trivial coding.
Something like below is all:
EnableWindow(FALSE);
while ( /* check for my exit condition */ )
{
MSG msg;
if(::PeekMessage(&msg, NULL, 0, 0, PM_NOREMOVE))
{
if( !AfxGetApp()->PumpMessage() )
{
::PostQuitMessage(0);
}
}
}
EnableWindow(TRUE);
To wait for the process exit, you could use very short(<30ms) timeout WaitForSingleObject call in the message loop. Or MsgWaitForMultipleObjects. Or GetExitCodeProcess.
I'd like to recommend another approach.
1) Show new modal popup
2) Start the process in OnInitDialog handler and start a timer
3) Check if the process is still running in OnTimer handler, by GetExitCodeProcess
4) Call EndDialog when the process is no longer running
Try MsgWaitForMultipleObjects function; it can process Windows messages while waiting for the event object.
You could:
(a bit complicated) use MsgWaitForMultipleObjects (or MsgWaitForMultipleObjectsEx) to wait for the process to finish or for a message to arrive (processing it in the normal way).
(simple) use RegisterWaitForSingleObject to register a callback that is called in a separate thread when the process exits (and perhaps have that callback just post a message to your window).
(fairly simple) create your own thread to do the waiting in.
I'd go with the 2nd option.
DWORD ec;
if(CreateProcess( NULL, // No module name (use command line).
szExe, // Command line.
NULL, // Process handle not inheritable.
NULL, // Thread handle not inheritable.
FALSE, // Set handle inheritance to FALSE.
procFlags, // No creation flags.
NULL, // Use parent's environment block.
NULL, // Use parent's starting directory.
&si, // Pointer to STARTUPINFO structure.
&pi ) // Pointer to PROCESS_INFORMATION structure.
)
{
while(GetExitCodeProcess(pi.hProcess, &ec) && ec == STILL_ACTIVE)
{
MSG msg;
while(::PeekMessage(&msg, NULL, 0, 0, PM_NOREMOVE))
{
if(!AfxGetApp()->PumpMessage())
{
::PostQuitMessage(0);
break;
}
}
// let MFC do its idle processing
LONG lIdle = 0;
while(AfxGetApp()->OnIdle(lIdle++))
;
}
}
if(ec)
{
CloseHandle(pi.hProcess);
}

EnterCriticalSection Deadlock

Having what appears to be a dead-lock situation with a multi-threaded logging application.
Little background:
My main application has 4-6 threads running. The main thread responsible for monitoring health of various things I'm doing, updating GUIs, etc... Then I have a transmit thread and a receive thread. The transmit and receive threads talk to physical hardware. I sometimes need to debug the data that the transmit and receive threads are seeing; i.e. print to a console without interrupting them due to their time critical nature of the data. The data, by the way, is on a USB bus.
Due to the threading nature of the application, I want to create a debug console that I can send messages to from my other threads. The debug consule runs as a low priority thread and implements a ring buffer such that when you print to the debug console, the message is quickly stored to a ring buffer and sets and event. The debug console's thread sits WaitingOnSingleObject events from the in bound messages that come in. When an event is detected, console thread updates a GUI display with the message. Simple eh? The printing calls and the console thread use a critical section to control access.
NOTE: I can adjust the ring buffer size if I see that I am dropping messages (at least that's the idea).
In a test application, the console works very well if I call its Print method slowly via mouse clicks. I have a button that I can press to send messages to the console and it works. However, if I put any sort of load (many calls to Print method), everything dead-locks. When I trace the dead-lock, my IDE's debugger traces to EnterCriticalSection and sits there.
NOTE: If I remove the Lock/UnLock calls and just use Enter/LeaveCriticalSection (see the code) I sometimes work but still find myself in a dead-lock situation. To rule out deadlocks to stack push/pops, I call Enter/LeaveCriticalSection directly now but this did not solve my issue.... What's going on here?
Here is one Print statement, that allows me to pass in a simple int to the display console.
void TGDB::Print(int I)
{
//Lock();
EnterCriticalSection(&CS);
if( !SuppressOutput )
{
//swprintf( MsgRec->Msg, L"%d", I);
sprintf( MsgRec->Msg, "%d", I);
MBuffer->PutMsg(MsgRec, 1);
}
SetEvent( m_hEvent );
LeaveCriticalSection(&CS);
//UnLock();
}
// My Lock/UnLock methods
void TGDB::Lock(void)
{
EnterCriticalSection(&CS);
}
bool TGDB::TryLock(void)
{
return( TryEnterCriticalSection(&CS) );
}
void TGDB::UnLock(void)
{
LeaveCriticalSection(&CS);
}
// This is how I implemented Console's thread routines
DWORD WINAPI TGDB::ConsoleThread(PVOID pA)
{
DWORD rVal;
TGDB *g = (TGDB *)pA;
return( g->ProcessMessages() );
}
DWORD TGDB::ProcessMessages()
{
DWORD rVal;
bool brVal;
int MsgCnt;
do
{
rVal = WaitForMultipleObjects(1, &m_hEvent, true, iWaitTime);
switch(rVal)
{
case WAIT_OBJECT_0:
EnterCriticalSection(&CS);
//Lock();
if( KeepRunning )
{
Info->Caption = "Rx";
Info->Refresh();
MsgCnt = MBuffer->GetMsgCount();
for(int i=0; i<MsgCnt; i++)
{
MBuffer->GetMsg( MsgRec, 1);
Log->Lines->Add(MsgRec->Msg);
}
}
brVal = KeepRunning;
ResetEvent( m_hEvent );
LeaveCriticalSection(&CS);
//UnLock();
break;
case WAIT_TIMEOUT:
EnterCriticalSection(&CS);
//Lock();
Info->Caption = "Idle";
Info->Refresh();
brVal = KeepRunning;
ResetEvent( m_hEvent );
LeaveCriticalSection(&CS);
//UnLock();
break;
case WAIT_FAILED:
EnterCriticalSection(&CS);
//Lock();
brVal = false;
Info->Caption = "ERROR";
Info->Refresh();
aLine.sprintf("Console error: [%d]", GetLastError() );
Log->Lines->Add(aLine);
aLine = "";
LeaveCriticalSection(&CS);
//UnLock();
break;
}
}while( brVal );
return( rVal );
}
MyTest1 and MyTest2 are just two test functions that I call in response to a button press. MyTest1 never causes a problem no matter how fast I click the button. MyTest2 dead locks nearly everytime.
// No Dead Lock
void TTest::MyTest1()
{
if(gdb)
{
// else where: gdb = new TGDB;
gdb->Print(++I);
}
}
// Causes a Dead Lock
void TTest::MyTest2()
{
if(gdb)
{
// else where: gdb = new TGDB;
gdb->Print(++I);
gdb->Print(++I);
gdb->Print(++I);
gdb->Print(++I);
gdb->Print(++I);
gdb->Print(++I);
gdb->Print(++I);
gdb->Print(++I);
}
}
UPDATE:
Found a bug in my ring buffer implementation. Under heavy load, when buffer wrapped, I didn't detect a full buffer properly so buffer was not returning. I'm pretty sure that issue is now resolved. Once I fixed the ring buffer issue, performance got much better. However, if I decrease the iWaitTime, my dead lock (or freeze up issue) returns.
So after further tests with a much heavier load it appears my deadlock is not gone. Under super heavy load I continue to deadlock or at least my app freezes up but no where near it use to since I fixed ring buffer problem. If I double the number of Print calls in MyTest2 I easily can lock up every time....
Also, my updated code is reflected above. I know make sure my Set & Reset event calls are inside critical section calls.
With those options closed up, I would ask questions about this "Info" object. Is it a window, which window is it parented to, and which thread was it created on?
If Info, or its parent window, was created on the other thread, then the following situation might occur:
The Console Thread is inside a critical section, processing a message.
The Main thread calls Print() and blocks on a critical section waiting for the Console Thread to release the lock.
The Console thread calls a function on Info (Caption), which results in the system sending a message (WM_SETTEXT) to the window. SendMessage blocks because the target thread is not in a message alertable state (isn't blocked on a call to GetMessage/WaitMessage/MsgWaitForMultipleObjects).
Now you have a deadlock.
This kind of #$(%^ can happen whenever you mix blocking routines with anything that interacts with windows. The only appropriate blocking function to use on a GUI thread is MSGWaitForMultipleObjects otherwise SendMessage calls to windows hosted on the thread can easily deadlock.
Avoiding this involves two possible approaches:
Never doing any GUI interaction in worker threads. Only use PostMessage to dispatch non blocking UI update commands to the UI thread, OR
Use kernel Event objects + MSGWaitForMultipleObjects (on the GUI thread) to ensure that even when you are blocking on a resource, you are still dispatching messages.
Without knowing where it is deadlocking this code is hard to figure out. Two comments tho:
Given that this is c++, you should be using an Auto object to perform the lock and unlock. Just in case it ever becomes non catastrophic for Log to throw an exception.
You are resetting the event in response to WAIT_TIMEOUT. This leaves a small window of opportunity for a 2nd Print() call to set the event while the worker thread has returned from WaitForMultiple, but before it has entered the critical section. Which will result in the event being reset when there is actually data pending.
But you do need to debug it and reveal where it "Deadlocks". If one thread IS stuck on EnterCriticalSection, then we can find out why. If neither thread is, then the incomplete printing is just the result of an event getting lost.
I would strongly recommend a lockfree implementation.
Not only will this avoid potential deadlock, but debug instrumentation is one place where you absolutely do not want to take a lock. The impact of formatting debug messages on timing of a multi-threaded application is bad enough... having locks synchronize your parallel code just because you instrumented it makes debugging futile.
What I suggest is an SList-based design (The Win32 API provides an SList implementation, but you can build a thread-safe template easily enough using InterlockedCompareExchange and InterlockedExchange). Each thread will have a pool of buffers. Each buffer will track the thread it came from, after processing the buffer, the log manager will post the buffer back to the source thread's SList for reuse. Threads wishing to write a message will post a buffer to the logger thread. This also prevents any thread from starving other threads of buffers. An event to wake the logger thread when a buffer is placed into the queue completes the design.