I used to call CreateThread() for all my threads, and WaitForMultipleObjects(), an leave the routine.
To get somewhat faster code, I'd like to do a kind of thread pool. My thread pools are sometimes created, later used multiple times, and later destroyed (ie., there is not a single pool created at the begining of the program). Each thread in my thread pool call the same routine with different parameters, the number of threads is constant, and they always need to be launched at the same time.
What I do is as follows :
DWORD WINAPI runFunction(LPVOID p) {
Thread* = (Thread*) p;
while(true) {
WaitForSingleObject(thread->awakeEvenHandle, INFINITE);
thread->run();
SetEvent(thread->SleepingEventHandle);
SuspendThread(thread->handle);
}
return 0;
}
void ExecuteThreads(std::vector<Thread*> &threads) {
HANDLE* waitingEvents = new HANDLE[threads.size()];
for (int i=0; i<threads.size(); i++) {
if (threads[i]->handle == NULL) {
threads[i]->AwakeEventHandle = CreateEvent(NULL, true, false, "Awake");
threads[i]->SleepingEventHandle = CreateEvent(NULL, true, false, "Sleeping");
threads[i]->handle = CreateThread(NULL, 0. runFunction, (void*) threads[i], CREATE_SUSPENDED, NULL);
}
ResumeThread(threads[i]->handle);
ResetEvent(threads[i]->SleepingEventHandle);
SetEvent(threads[i]->AwakeEventHandle);
waitingEvents[i] = threads[i]->SleepingEventHandle;
}
WaitForMultipleObjects( threads.size(), waitingEvents, TRUE, INFINITE);
}
My class Thread has a destructor which calls CloseHandle for the HANDLEs SleepingEventHandle and AwakeEventHandle, and for the thread handle. The function Thread::run() is pure virtual, and it's up to the coder to inherit the Thread for an actual run() implementation.
As it, the code doesn't work . One reason is that when I don't need this pool anymore, the destructors of the Threads are called, but the runFunction cannot exits and this crashes (the pointer "thread" has been destroyed but is still used the the function). There are probably many other problems with my code.
How would you do it, in a simple manner ? Is-there an easy fix ? What problems will I encounter with this code ?
Thanks!
Why do you have to deal with such low level api functions? Have a look at boost::thread and boost::thread_group. Also there is a thread pool implementation works with boost::thread.
Now if your threads work for a short period of time, your system will have remarkable overhead with creating and signaling all those threads and events. ppl Task Parallelism or tbb::task are definitely the ways to go.
Related
In the following example(not all the code included just the necessary portions):
class A
{
public:
void FlushToDisk(char* pData, unsigned int uiSize)
{
char* pTmp = new char[uiSize];
memcpy(pTmp, pData, uiSize);
m_Thread = boost::thread(&CSimSwcFastsimExporter::WriteToDisk, this, pTmp, uiSize);
}
void WriteToDisk(char* pData, unsigned int uiSize)
{
m_Mtx.lock();
m_ExportFile.write(pData, uiSize);
delete[] pData;
m_Mtx.unlock();
}
boost::thread m_Thread;
boost::mutex m_Mtx
}
is it safe to use the m_Thread that way since the FlushToDisk method can be called while the created thread is executing the WriteToDisk method.
Or should I do something like:
m_Thread.join();
m_Thread = boost::thread(&CSimSwcFastsimExporter::WriteToDisk, this, pTmp, uiSize);
Would this second solution be slower than the first?
From what i saw at http://www.boost.org/doc/libs/1_59_0/doc/html/thread/thread_management.html#thread.thread_management.tutorial
"When the boost::thread object that represents a thread of execution is destroyed the thread becomes detached. Once a thread is detached, it will continue executing until the invocation of the function or callable object supplied on construction has completed, or the program is terminated".
So in my case the threads should not be interrupted or?
Thanks in advance.
The second solution will pause the main thread to wait until the writer thread completes. You would be able to remove mutex if you go this way. You are guaranteed to have one file writing thread.
The first solution is going to allow main thread to continue, and will create an uncontrolled writing thread - serialized on the mutex. While you might believe this is better (main thread will not wait) I do not like this solution for several reasons.
First, you do not have any control over the number of created threads. If the function is called often, and the operation is slow, you can easily run out of threads! Second, and much more important, you will accumulate a backlog of detached threads waiting on mutex. If your main application decides to exit, all those threads will be silently killed and the updates will be lost.
In my Windows service application I may resort to calling TerminateThread API on some of the threads in my process. (Note that I do so only as a last resort measure when a thread fails to quit in a "normal fashion" using signaling mechanism and thread synchronization techniques.) What I've noticed in the event log submitted by a customer is that very rarely TerminateThread may throw the STATUS_INVALID_THREAD exception, which happens only when that API is called on a thread belonging to a threadpool.
Since I know for sure that none of my threads are started from a threadpool, the thread that my call to TerminateThread attempts to close must be coming from another process. This could happen only due to a race condition where my thread handle is closed first and then is passed again to the TerminateThread API while the OS reuses it for some other thread in another process.
So my question is, since my service is running with sufficiently high privileges (as localService) can TerminateThread API in this situation inadvertently terminate some thread belonging to another process? And if yes, how can I prevent this (apart from finding the race-condition, that I'm doing now)?
Let's let the docs speak for themselves:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms686717(v=vs.85).aspx
Do not use TerminateThread():
TerminateThread is a dangerous function that should only be used in the most extreme cases. You should call TerminateThread only if you know exactly what the target thread is doing, and you control all of the code that the target thread could possibly be running at the time of the termination. For example, TerminateThread can result in the following problems:
[...]
You can terminate any thread, as long as you have a handle with sufficient privileges:
A thread cannot protect itself against TerminateThread, other than by controlling access to its handles. The thread handle returned by the CreateThread and CreateProcess functions has THREAD_TERMINATE access, so any caller holding one of these handles can terminate your thread.
Note that I do so only as a last resort measure when a thread fails to quit in a "normal fashion" using signaling mechanism and thread synchronization techniques.
That is a case where you cannot call TerminateThread. You can only call TerminateThread if you have precise control over the thread you are terminating and its full cooperation. If the thread fails to quit in the normal fashion, then you have lost control over the thread, the very opposite of the required conditions under which you can call TerminateThread.
If a process has lost control over one of its threads, the process cannot be saved. This is a fundamental property of threads -- they do not provide isolation of anything but flow control.
If you must do this you can do it like this.
All you need to start is the thread handle.
You call the first function with thread handle as input using the "undocumented" NtQueryInformationThread() function with the ThreadQuerySetWin32StartAddress argument, you get the StartAddress of the thread. More reading # NTInternals.
It calls NTQueryInformationThread by function address after getting the address via GetProcAddress. Then it calls it with the ThreadQuerySetWin32StartAddress argument, getting the StartAddress of the thread.
Then you call the second function which loops through all the threads via CreateToolHelp32Snapshot and compares against the supplied StartAddress. It calls TerminateThread once it finds it.
enum THREADINFOCLASS
{
ThreadQuerySetWin32StartAddress = 9,
};
typedef NTSTATUS(__stdcall * f_NtQueryInformationThread)(HANDLE, THREADINFOCLASS, void*, ULONG_PTR, ULONG_PTR*);
ULONG_PTR GetThreadStartAddress(HANDLE hThread)
{
auto NtQueryInformationThread = reinterpret_cast<f_NtQueryInformationThread>(GetProcAddress(GetModuleHandleA("ntdll.dll"), "NtQueryInformationThread"));
if (!NtQueryInformationThread)
return 0;
ULONG_PTR ulStartAddress = 0;
NTSTATUS Ret = NtQueryInformationThread(hThread, ThreadQuerySetWin32StartAddress, &ulStartAddress, sizeof(ULONG_PTR), nullptr);
if (NT_FAIL(Ret))
return 0;
return ulStartAddress;
}
bool TerminateThreadByStartaddress(ULONG_PTR StartAddress)
{
HANDLE hSnap = CreateToolhelp32Snapshot(TH32CS_SNAPTHREAD, 0);
if (hSnap == INVALID_HANDLE_VALUE)
return false;
THREADENTRY32 TE32 = { 0 };
TE32.dwSize = sizeof(THREADENTRY32);
BOOL Ret = Thread32First(hSnap, &TE32);
while (Ret)
{
HANDLE hTempThread = OpenThread(THREAD_ALL_ACCESS, FALSE, TE32.th32ThreadID);
if (!hTempThread)
{
Ret = Thread32Next(hSnap, &TE32);
continue;
}
if (StartAddress == GetThreadStartAddress(hTempThread))
{
TerminateThread(hTempThread, 0);
CloseHandle(hTempThread);
CloseHandle(hSnap);
return true;
}
CloseHandle(hTempThread);
Ret = Thread32Next(hSnap, &TE32);
}
CloseHandle(hSnap);
return false;
}
Credits to my friend Broihon, I didn't write this code but have used it before.
Use undocumented NTSYSAPI NTSTATUS NTAPI NtTerminateThread(IN HANDLE ThreadHandle, IN NTSTATUS ExitStatus); from http://undocumented.ntinternals.net where ThreadHandle is result from OpenThread MS function and ExitStatus set to whatever you want.
let's pretend there are no libraries that provide semaphores for C++. I wrote this:
#include <vector>
#include <Windows.h>
class Semaphore {
HANDLE mutexS; // provides mutex in semaphore rutines
std::vector<HANDLE> queue; // provides FIFO queue for blocked threads
int value; // semaphore's value
public:
Semaphore(int init=1);
~Semaphore();
void wait();
void signal();
};
Semaphore::Semaphore(int init) {
value = init;
queue = std::vector<HANDLE>();
mutexS = CreateMutex(0,0,0);
}
Semaphore::~Semaphore() {
CloseHandle(mutexS);
}
void Semaphore::signal() {
WaitForSingleObject(mutexS, INFINITE);
if (++value <= 0) {
HANDLE someOldThread = queue.front();
ResumeThread(someOldThread);
queue.erase(queue.begin());
CloseHandle(someOldThread);
}
ReleaseMutex(mutexS);
}
I would like to know why this implementation of wait() doesn't work:
void Semaphore::wait() {
WaitForSingleObject(mutexS, INFINITE);
if (--value < 0) {
HANDLE thisThread = GetCurrentThread();
queue.push_back(thisThread);
ReleaseMutex(mutexS);
SuspendThread(thisThread );
}
else
ReleaseMutex(mutexS);
}
And this one works:
void Semaphore::wait() {
WaitForSingleObject(mutexS, INFINITE);
if (--value < 0) {
HANDLE thisThread = GetCurrentThread();
HANDLE alsoThisThread;
DuplicateHandle(GetCurrentProcess(), thisThread, GetCurrentProcess(), &alsoThisThread, 0, 0, DUPLICATE_SAME_ACCESS);
queue.push_back(alsoThisThread);
ReleaseMutex(mutexS);
SuspendThread(alsoThisThread);
}
else
ReleaseMutex(mutexS);
}
What exactly happens in each case? I've been banging my head over it for a lot of time now. The first implementation of wait, which doesn't work, makes my program block (well, it probably blocks some thread forever). The 2nd implementation works like a charm. What gives ? Why do I need to duplicate thread handles and block the duplicate ?
MSDN helps a lot here ;)
GetCurrentThread returns a pseudo-handle which is a constant for "the current thread":
A pseudo handle is a special constant that is interpreted as the current thread handle.
So when you push it in the queue, you are always pushing a constant that says "the current thread", which is obviously not what you want.
To get a real handle, you have to use DuplicateHandle
If hSourceHandle is a pseudo handle returned by GetCurrentProcess or GetCurrentThread, DuplicateHandle converts it to a real handle to a process or thread, respectively.
A final note: I suppose you are implementing this as a "test" right? Because there are several potential problems.. A very good learning exercise would be to dig them out. But you should not use this in production code.
Out of curiosity: if you want to experiment a little more, the "canonical" way of implementing semaphore with mutexes is to use two mutexes: see here
MSDN documentation for GetCurrentThread has the answer (accents are mine):
The return value is a pseudo handle for the current thread.
A pseudo handle is a special constant that is interpreted as the current thread handle. The calling thread can use this handle to specify itself whenever a thread handle is required.
...
The function cannot be used by one thread to create a handle that can be used by other threads to refer to the first thread. The handle is always interpreted as referring to the thread that is using it. A thread can create a "real" handle to itself that can be used by other threads, or inherited by other processes, by specifying the pseudo handle as the source handle in a call to the DuplicateHandle function.
These days I'm trying to learn more things about threads in windows. I thought about making this practical application:
Let's say there are several threads started when a button "Start" is pressed. Assume these threads are intensive (they keep running / have always something to work on).
This app would also have a "Stop" button. When this button is pressed all the threads should close in a nice way: free resources and abandon work and return the state they were before the "Start" button was pressed.
Another request of the app is that the functions runned by the threads shouldn't contain any instruction checking if the "Stop" button was pressed. The function running in the thread shouldn't care about the stop button.
Language: C++
OS: Windows
Problems:
WrapperFunc(function, param)
{
// what to write here ?
// if i write this:
function(param);
// i cannot stop the function from executing
}
How should I construct the wrapper function so that I can stop the thread properly?
( without using TerminateThread or some other functions )
What if the programmer allocates some memory dynamically? How can I free it before closing
the thread?( note that when I press "Stop button" the thread is still processing data)
I though about overloading the new operator or just imposing the usage of a predefined
function to be used when allocating memory dynamically. This, however, means
that the programmer who uses this api is constrained and it's not what I want.
Thank you
Edit: Skeleton to describe the functionality I'd like to achieve.
struct wrapper_data
{
void* (*function)(LPVOID);
LPVOID *params;
};
/*
this function should make sure that the threads stop properly
( free memory allocated dynamically etc )
*/
void* WrapperFunc(LPVOID *arg)
{
wrapper_data *data = (wrapper_data*) arg;
// what to write here ?
// if i write this:
data->function(data->params);
// i cannot stop the function from executing
delete data;
}
// will have exactly the same arguments as CreateThread
MyCreateThread(..., function, params, ...)
{
// this should create a thread that runs the wrapper function
wrapper_data *data = new wrapper_data;
data->function = function;
data->params = params;
CreateThread(..., WrapperFunc, (LPVOID) wrapper_data, ...);
}
thread_function(LPVOID *data)
{
while(1)
{
//do stuff
}
}
// as you can see I want it to be completely invisible
// to the programmer who uses this
MyCreateThread(..., thread_function, (LPVOID) params,...);
One solution is to have some kind of signal that tells the threads to stop working. Often this can be a global boolean variable that is normally false but when set to true it tells the threads to stop. As for the cleaning up, do it when the threads main loop is done before returning from the thread.
I.e. something like this:
volatile bool gStopThreads = false; // Defaults to false, threads should not stop
void thread_function()
{
while (!gStopThreads)
{
// Do some stuff
}
// All processing done, clean up after my self here
}
As for the cleaning up bit, if you keep the data inside a struct or a class, you can forcibly kill them from outside the threads and just either delete the instances if you allocated them dynamically or let the system handle it if created e.g. on the stack or as global objects. Of course, all data your thread allocates (including files, sockets etc.) must be placed in this structure or class.
A way of keeping the stopping functionality in the wrapper, is to have the actual main loop in the wrapper, together with the check for the stop-signal. Then in the main loop just call a doStuff-like function that does the actual processing. However, if it contains operations that might take time, you end up with the first problem again.
See my answer to this similar question:
How do I guarantee fast shutdown of my win32 app?
Basically, you can use QueueUserAPC to queue a proc which throws an exception. The exception should bubble all the way up to a 'catch' in your thread proc.
As long as any libraries you're using are reasonably exception-aware and use RAII, this works remarkably well. I haven't successfully got this working with boost::threads however, as it's doesn't put suspended threads into an alertable wait state, so QueueUserAPC can't wake them.
If you don't want the "programmer" of the function that the thread will execute deal with the "stop" event, make the thread execute a function of "you" that deals with the "stop" event and when that event isn't signaled executes the "programmer" function...
In other words the "while(!event)" will be in a function that calls the "job" function.
Code Sample.
typedef void (*JobFunction)(LPVOID params); // The prototype of the function to execute inside the thread
struct structFunctionParams
{
int iCounter;
structFunctionParams()
{
iCounter = 0;
}
};
struct structJobParams
{
bool bStop;
JobFunction pFunction;
LPVOID pFunctionParams;
structJobParams()
{
bStop = false;
pFunction = NULL;
pFunctionParams = NULL;
}
};
DWORD WINAPI ThreadProcessJob(IN LPVOID pParams)
{
structJobParams* pJobParams = (structJobParams*)pParams;
while(!pJobParams->bStop)
{
// Execute the "programmer" function
pJobParams->pFunction(pJobParams->pFunctionParams);
}
return 0;
}
void ThreadFunction(LPVOID pParams)
{
// Do Something....
((structFunctionParams*)pParams)->iCounter ++;
}
int _tmain(int argc, _TCHAR* argv[])
{
structFunctionParams stFunctionParams;
structJobParams stJobParams;
stJobParams.pFunction = &ThreadFunction;
stJobParams.pFunctionParams = &stFunctionParams;
DWORD dwIdThread = 0;
HANDLE hThread = CreateThread(
NULL,
0,
ThreadProcessJob,
(LPVOID) &stJobParams, 0, &dwIdThread);
if(hThread)
{
// Give it 5 seconds to work
Sleep(5000);
stJobParams.bStop = true; // Signal to Stop
WaitForSingleObject(hThread, INFINITE); // Wait to finish
CloseHandle(hThread);
}
}
Read the problem carefully first.
There is a worker thread which gets spawned from a CreateInstance of CTest class. Here is the prototype of the class. hThread is the handle to thread and hEventShutdown is the event used to shutdown thread when program exits. WaitForShutdown is the public function which is used to signal hEventShutdown and wait on handle to thread till thread exits gracefully. WaitForShutdown is invoked from Exit of Application.
//pseudocode
CTest
{
public:
CTest* CreateInstance();
static threadproc(void *pv);
void WaitForShutdown();
public:
HANDLE hThread;
HANDLE hEventShutdown;
}
void CTest::CTest* CreateInstance()
{
// spawn a thread, pass 'this' pointer to thread , use beginthreadex
hThread = beginthreadex ( threadproc, this );
}
unsigned int CTest::threadproc( void *pv)
{
Ctest *ptest = (Ctest*)pv;
do
{
HANDLES hArray[2] = { pv->hEventShutdown, someotherhandle }
dwResult = Waitformultipleobjects( hArrary , 2);
if ( dwResult == WAIT_OBJECT_0)
delete pTest; // since it is allocated dynamically ( This is required due to some known reasons in my code)
if(dwResult == WAIT_OBJECT_0 + 1)
Doprocessing(); //DoProcessing when other thread signal someotherhandle
}while (1)
void CTest::WaitForShutdown()
{
SetEvent ( hEventShutdown);
WaitForSingleObject ( hThread,INFINITE);
}
void CTest::~CTest()
{
Closehandle(hThread)
Closehandle(hEventShutdown);
}
Now if you look at the code carefully, you will find that event is signaled from WaitForShutdown function, thread comes out of WaitForMultipleOjbects and deletes pointer of CTest. It means destructor of CTest is invoked which will obviously close thread handle ( hThread). But WaitForSingleObject from WaitForShutdown is actually waiting on thread handle. So here behavior will be undefined ( I think so, you can correct me if I am wrong). Another problem is destructor of Ctest is invoked when WaitForSingleObject is waiting on its member hThread which is not correct. I can not remove delete pTest from thread since it has to be there due to some reasons.
How will you suggest the solution to the above ?
Couple of Solution which I can think of :
I can keep thread handle in another map but I dont want to do it.
I can copy thread handle to some local variable before WaitForSingleObject in WaitForShutdown and will wait on it. Don;t know is it right ? you tell me.
Or I will use Duplicatehandle API to get reference of existing thread handle before WaitForSingleObject and wait on it. Dont know is it correct. Dont know if will the duplicate handle be alive after CloseHandle on original.
I will keep thread ID, get thread handle from thread ID and keep waiting on thread handle in WaitForShutdown. This looks more elegant but I do not know is there any way to get handle from thread id.
Correct me.
Your feedback appreciated.
The simplest way to handle this would be to simply delete the thread from WaitForShutdown, after WaitForSingleObject returns. This ensures all the handles you need - and more importantly, the object itself - remain alive to the end.
I have run this piece as is. Seems it works, and it doesn't crash.
Strangely that we can call CloseHandle(hthread), before we go out of WaitforSingleObject(hThread,INFINITE).
Of course, "academic" way to join thread is firstly WaitForSingleObject(hThread,INFINITE) than CloseHandle(hThread). So that is my suggestion - do in this way.
I don't have to add anymore.