How should a thread pool be implemented in C? - c++

Question

I'm programming in C++, but I'm only using pthread.h, no boost or C++11 threads.
So I'm trying to use threads but based on one of my previous questions (link), this doesn't seem feasible since threads terminate right after completion of its task, and one of the more prevalent reasons to use a thread-pool implementation is to reduce thread-creation overhead by reusing these threads for multiple tasks.
So is the only other way to implement this in C to use fork(), and create a pipe from the main to child processes? Or is there a way to set up a pipe between threads and their parent that I don't know about?
Many thanks in advance!

Answer 1

Yes, you can create a thread-safe queue between the threads. Then the threads in the pool will sit in a loop retrieving an item from the queue, executing whatever it needs, then going back and getting another.
That's generally a bit easier/simpler in C++ because it's a little easier to agree on some of the interface (e.g., overload operator() to execute the code for a task), but at a fundamental level you can do all the same things in C (e.g., each task struct you put in the queue will contain a pointer to a function to carry out the work for that task).
In your case, since you are using C++, it's probably easier to use an overload of operator() to do the work though. The rest of the task struct (or whatever you choose to call it) will contain any data needed, etc.

Answer 2

From the POSIX standard:
int pthread_create(pthread_t *restrict thread,
const pthread_attr_t *restrict attr,
void *(*start_routine)(void*), void *restrict arg);
(...) The thread is created executing start_routine with arg as its sole argument.
So, you should create a bunch of threads with this function, and have them all execute a function that goes something like
void *consumer(void *arg)
{
WorkQueue *queue = static_cast<WorkQueue *>(arg);
for (task in queue) {
if (task == STOP_WORKING)
break;
do work;
}
return WHATEVER;
}
(At the end of input, push n STOP_WORKING items to the queue where n is the number of threads.)
Mind you, pthreads is a very low-level API that offers very little type-safety (all data is passed as void pointers). If you're trying to parallelize CPU-intensive tasks, you might want to look at OpenMP instead.

Answer 3

'doesn't seem feasible since threads terminate right after completion of its task' what??
for(;;){
Task *myTask=theCommonProducerConsumerQueue->pop();
myTask->run();
}
.. never return anything, in fact, never return.

Answer 4

You may find it helpful to look at the source code for libdispatch, which is the basis for Apple's Grand Central Dispatch and uses thread pools.

Answer 5

I would suggest using Threaded Building Blocks from Intel to accomplish work-queue/threadpool like tasks. A fairly contrived example using TBB 3.0:
class PoorExampleTask : public tbb::task {
PoorExampleTask(int foo, tbb::concurrent_queue<float>& results)
: _bar(foo), _results(results)
{ }
tbb::task* execute() {
_results.push(pow(2.0, foo));
return NULL;
}
private:
int _bar;
tbb::concurrent_queue<float>& _results;
}
Used later on like so:
tbb::concurrent_queue<float> powers;
for (int ww = 0; ww < LotsOfWork; ++ww) {
PoorExampleTask* tt
= new (tbb::task::allocate_root()) PoorExampleTask(ww, powers);
tbb::task::enqueue(*tt);
}

Answer 6

http://people.clarkson.edu/~jmatthew/cs644.archive/cs644.fa2001/proj/locksmith/code/ExampleTest/threadpool.c
I used google a couple months ago, you should try it.
Edit: it seems maybe you want a group instead. I was able to create one with some minor alteration of the above so that the worker didn't perform work, but just joined threads.