How to wait for completion of all tasks in this ThreadPool? - c++

Question

I am trying to write a ThreadPool class
class ThreadPool {
public:
ThreadPool(size_t numberOfThreads):isAlive(true) {
for(int i =0; i < numberOfThreads; i++) {
workerThreads.push_back(std::thread(&ThreadPool::doJob, this));
}
#ifdef DEBUG
std::cout<<"Construction Complete"<<std::endl;
#endif
}
~ThreadPool() {
#ifdef DEBUG
std::cout<<"Destruction Start"<<std::endl;
#endif
isAlive = false;
conditionVariable.notify_all();
waitForExecution();
#ifdef DEBUG
std::cout<<"Destruction Complete"<<std::endl;
#endif
}
void waitForExecution() {
for(std::thread& worker: workerThreads) {
worker.join();
}
}
void addWork(std::function<void()> job) {
#ifdef DEBUG
std::cout<<"Adding work"<<std::endl;
#endif
std::unique_lock<std::mutex> lock(lockListMutex);
jobQueue.push_back(job);
conditionVariable.notify_one();
}
private:
// performs actual work
void doJob() {
// try {
while(isAlive) {
#ifdef DEBUG
std::cout<<"Do Job"<<std::endl;
#endif
std::unique_lock<std::mutex> lock(lockListMutex);
if(!jobQueue.empty()) {
#ifdef DEBUG
std::cout<<"Next Job Found"<<std::endl;
#endif
std::function<void()> job = jobQueue.front();
jobQueue.pop_front();
job();
}
conditionVariable.wait(lock);
}
}
// a vector containing worker threads
std::vector<std::thread> workerThreads;
// a queue for jobs
std::list<std::function<void()>> jobQueue;
// a mutex for synchronized insertion and deletion from list
std::mutex lockListMutex;
std::atomic<bool> isAlive;
// condition variable to track whether or not there is a job in queue
std::condition_variable conditionVariable;
};
I am adding work to this thread pool from my main thread. My problem is calling waitForExecution() results in forever waiting main thread. I need to be able to terminate threads when all work is done and continue main thread execution from there. How should I proceed here?

Answer 1

The first step when writing a robust thread pool is to split the queue from the management of threads. A thread-safe queue is hard enough to write by its own, and managing threads similarly.
A thread safe queue looks like:
template<class T>
struct threadsafe_queue {
boost::optional<T> pop() {
std::unique_lock<std::mutex> l(m);
cv.wait(l, [&]{ aborted || !data.empty(); } );
if (aborted) return {};
return data.pop_front();
}
void push( T t )
{
std::unique_lock<std::mutex> l(m);
if (aborted) return;
data.push_front( std::move(t) );
cv.notify_one();
}
void abort()
{
std::unique_lock<std::mutex> l(m);
aborted = true;
data = {};
cv.notify_all();
}
~threadsafe_queue() { abort(); }
private:
std::mutex m;
std::condition_variable cv;
std::queue< T > data;
bool aborted = false;
};
where pop returns nullopt when the queue is aborted.
Now our thread pool is easy:
struct threadpool {
explicit threadpool(std::size_t n) { add_threads(n); }
threadpool() = default;
~threadpool(){ abort(); }
void add_thread() { add_threads(1); }
void add_threads(std::size_t n)
{
for (std::size_t i = 0; i < n; ++i)
threads.push_back( std::thread( [this]{ do_thread_work(); } ) );
}
template<class F>
auto add_task( F && f )
{
using R = std::result_of_t< F&() >;
auto pptr = std::make_shared<std::promise<R>>();
auto future = pptr.get_future();
tasks.push([pptr]{ (*pptr)(); });
return future;
}
void abort()
{
tasks.abort();
while (!threads.empty()) {
threads.back().join();
threads.pop_back();
}
}
private:
threadsafe_queue< std::function<void()> > tasks;
std::vector< std::thread > threads;
void do_thread_work() {
while (auto f = tasks.pop()) {
(*f)();
}
}
};
note that if you abort, outstanding future's are filled with a broken promise exception.
Worker threads stop running when the queue they are feeding from is aborted. The main thread on abort() will wait for the worker threads to finish (as is wise).
This does mean that worker thread tasks must also terminate, or the main thread will hang. There is no way to avoid this; often, your worker threads' tasks need to cooperate to get a message saying they should abort early.
Boost has a thread pool that integrates with its threading primitives and permits a less cooperative abort; in it, all mutex type operations implicitly check for an abort flag, and if they see it the operation throws.

Answer 2

How should I proceed here?
Well, you should learn to use your debugger, which should show you exactly where each of the threads you want to join is stopped.
I'm going to tell you what looks wrong, but strongly encourage you to do that first. It's invaluable.
OK, now: your condition variable loop is wrong.
The correct pattern is the one that behaves like the second form, with the predicate argument, here:
while (!pred()) {
wait(lock);
}
Specifically, if your predicate is true, you must not call wait. You may never be woken again, because the predicate never became false in the first place!
Try
// wait until we have something to do
while(jobQueue.empty() && isAlive) {
conditionVariable.wait(lock);
}
// unless we're exiting, we must have a job
if (isAlive) {
#ifdef DEBUG
std::cout<<"Next Job Found"<<std::endl;
#endif
std::function<void()> job = jobQueue.front();
jobQueue.pop_front();
job();
}
Imagine your thread is running a job when you call notify_all - it will call wait after the notification has already happened, and it isn't coming again. Since it doesn't check isAlive between finishing the job and calling wait, it's going to wait forever.
Even without the shutdown problem it would be wrong, because it should keep consuming jobs while there is work to do, instead of blocking every time it finishes one. Which reminds me of the last issue - you should probably unlock the mutex while executing the job (and re-lock it afterwards) - otherwise your pool is single-threaded.