How can I tell when my ThreadPool is finished with its tasks? - c++

Question

In c++11, I have a ThreadPool object which manages a number of threads that are enqueued via a single lambda function. I know how many rows of data I have to work on and so I know ahead of time that I will need to queue N jobs. What I am not sure about is how to tell when all of those jobs are finished, so I can move on to the next step.
This is the code to manage the ThreadPool:
#include <cstdlib>
#include <vector>
#include <deque>
#include <iostream>
#include <atomic>
#include <thread>
#include <mutex>
#include <condition_variable>
class ThreadPool;
class Worker {
public:
Worker(ThreadPool &s) : pool(s) { }
void operator()();
private:
ThreadPool &pool;
};
class ThreadPool {
public:
ThreadPool(size_t);
template<class F>
void enqueue(F f);
~ThreadPool();
void joinAll();
int taskSize();
private:
friend class Worker;
// the task queue
std::deque< std::function<void()> > tasks;
// keep track of threads
std::vector< std::thread > workers;
// sync
std::mutex queue_mutex;
std::condition_variable condition;
bool stop;
};
void Worker::operator()()
{
std::function<void()> task;
while(true)
{
{ // acquire lock
std::unique_lock<std::mutex>
lock(pool.queue_mutex);
// look for a work item
while ( !pool.stop && pool.tasks.empty() ) {
// if there are none wait for notification
pool.condition.wait(lock);
}
if ( pool.stop ) {// exit if the pool is stopped
return;
}
// get the task from the queue
task = pool.tasks.front();
pool.tasks.pop_front();
} // release lock
// execute the task
task();
}
}
// the constructor just launches some amount of workers
ThreadPool::ThreadPool(size_t threads)
: stop(false)
{
for (size_t i = 0;i<threads;++i) {
workers.push_back(std::thread(Worker(*this)));
}
//workers.
//tasks.
}
// the destructor joins all threads
ThreadPool::~ThreadPool()
{
// stop all threads
stop = true;
condition.notify_all();
// join them
for ( size_t i = 0;i<workers.size();++i) {
workers[i].join();
}
}
void ThreadPool::joinAll() {
// join them
for ( size_t i = 0;i<workers.size();++i) {
workers[i].join();
}
}
int ThreadPool::taskSize() {
return tasks.size();
}
// add new work item to the pool
template<class F>
void ThreadPool::enqueue(F f)
{
{ // acquire lock
std::unique_lock<std::mutex> lock(queue_mutex);
// add the task
tasks.push_back(std::function<void()>(f));
} // release lock
// wake up one thread
condition.notify_one();
}
And then I distribute my job among threads like this:
ThreadPool pool(4);
/* ... */
for (int y=0;y<N;y++) {
pool->enqueue([this,y] {
this->ProcessRow(y);
});
}
// wait until all threads are finished
std::this_thread::sleep_for( std::chrono::milliseconds(100) );
Waiting for 100 milliseconds works just because I know those jobs can complete in less time than 100ms, but obviously its not the best approach. Once it has completed N rows of processing it needs to go through another 1000 or so generations of the same thing. Obviously, I want to begin the next generation as soon as I can.
I know there must be some way to add code into my ThreadPool so that I can do something like this:
while ( pool->isBusy() ) {
std::this_thread::sleep_for( std::chrono::milliseconds(1) );
}
I've been working on this for a couple nights now and I find it hard to find good examples of how to do this. So, what would be the proper way to implementat my isBusy() method?

Answer 1

I got it!
First of all, I introduced a few extra members to the ThreadPool class:
class ThreadPool {
/* ... exisitng code ... */
/* plus the following */
std::atomic<int> njobs_pending;
std::mutex main_mutex;
std::condition_variable main_condition;
}
Now, I can do better than checking some status every X amount of time. Now, I can block the Main loop until no more jobs are pending:
void ThreadPool::waitUntilCompleted(unsigned n) {
std::unique_lock<std::mutex> lock(main_mutex);
main_condition.wait(lock);
}
As long as I manage what's pending with the following bookkeeping code, at the head of the ThreadPool.enqueue() function:
njobs_pending++;
and right after I run the task in the Worker::operator()() function:
if ( --pool.njobs_pending == 0 ) {
pool.main_condition.notify_one();
}
Then the main thread can enqueue whatever tasks are necessary and then sit and wait until all calculations are completed with:
for (int y=0;y<N;y++) {
pool->enqueue([this,y] {
this->ProcessRow(y);
});
}
pool->waitUntilCompleted();

Answer 2

You may need to create an internal structure of threads associated with a bool variable flag.
class ThreadPool {
private:
// This Structure Will Keep Track Of Each Thread's Progress
struct ThreadInfo {
std::thread thread;
bool isDone;
ThreadInfo( std::thread& threadIn ) :
thread( threadIn ), isDone(false)
{}
}; // ThredInfo
// This Vector Should Be Populated In The Constructor Initially And
// Updated Anytime You Would Add A New Task.
// This Should Also Replace // std::vector<std::thread> workers
std::vector<ThreadInfo> workers;
public:
// The rest of your class would appear to be the same, but you need a
// way to test if a particular thread is currently active. When the
// thread is done this bool flag would report as being true;
// This will only return or report if a particular thread is done or not
// You would have to set this variable's flag for a particular thread to
// true when it completes its task, otherwise it will always be false
// from moment of creation. I did not add in any bounds checking to keep
// it simple which should be taken into consideration.
bool isBusy( unsigned idx ) const {
return workers[idx].isDone;
}
};

Answer 3

If you have N jobs and they have to be awaited for by calling thread sleep, then the most efficient way would be to create somewhere a variable, that would be set by an atomic operation to N before scheduling jobs and inside each job when done with computation, there would be atomic decrement of the variable. Then you can use atomic instruction to test if the variable is zero.
Or locked decrement with wait handles, when the variable would decrement to zero.
I just have to say, I do not like this idea you are asking for:
while ( pool->isBusy() ) {
std::this_thread::sleep_for( std::chrono::milliseconds(1) );
}
It just does not fit well, it won't be 1ms almost never, it is using resources needlessly etc...
The best way would be to decrement some variable atomically, and test atomically the variable if all done and the last job will simply based on atomic test set WaitForSingleObject.
And if you must, the waiting will be on WaitForSingleObject, and would woke up after completion, not many times.
WaitForSingleObject