Wake up of the thread is time consuming - c++

Question

#ifndef THREADPOOL_H
#define THREADPOOL_H
#include <iostream>
#include <deque>
#include <functional>
#include <thread>
#include <condition_variable>
#include <mutex>
#include <atomic>
#include <vector>
//thread pool
class ThreadPool
{
public:
ThreadPool(unsigned int n = std::thread::hardware_concurrency())
: busy()
, processed()
, stop()
{
for (unsigned int i=0; i<n; ++i)
workers.emplace_back(std::bind(&ThreadPool::thread_proc, this));
}
template<class F> void enqueue(F&& f)
{
std::unique_lock<std::mutex> lock(queue_mutex);
tasks.emplace_back(std::forward<F>(f));
cv_task.notify_one();
}
void waitFinished()
{
std::unique_lock<std::mutex> lock(queue_mutex);
cv_finished.wait(lock, [this](){ return tasks.empty() && (busy == 0); });
}
~ThreadPool()
{
// set stop-condition
std::unique_lock<std::mutex> latch(queue_mutex);
stop = true;
cv_task.notify_all();
latch.unlock();
// all threads terminate, then we're done.
for (auto& t : workers)
t.join();
}
unsigned int getProcessed() const { return processed; }
private:
std::vector< std::thread > workers;
std::deque< std::function<void()> > tasks;
std::mutex queue_mutex;
std::condition_variable cv_task;
std::condition_variable cv_finished;
unsigned int busy;
std::atomic_uint processed;
bool stop;
void thread_proc()
{
while (true)
{
std::unique_lock<std::mutex> latch(queue_mutex);
cv_task.wait(latch, [this](){ return stop || !tasks.empty(); });
if (!tasks.empty())
{
// got work. set busy.
++busy;
// pull from queue
auto fn = tasks.front();
tasks.pop_front();
// release lock. run async
latch.unlock();
// run function outside context
fn();
++processed;
latch.lock();
--busy;
cv_finished.notify_one();
}
else if (stop)
{
break;
}
}
}
};
#endif // THREADPOOL_H
I have the above thread pool implementation using a latch. However, every time I add a task through the enqueue call, the overhead is quite large, it takes about 100 micro seconds.
How can I improve the performance of the threadpool?

Answer 1

Your code looks fine. The comments above in your question about compiling with release optimizations on are probably correct and all you need to do.
Disclaimer: Always measure code first with appropriate tools to identify where the bottlenecks are before attempting to improve it's performance. Otherwise, you might not get the improvements you seek.
But a couple of potential micro-optimizations I see are this.
Change this in your thread_proc function
while (true)
{
std::unique_lock<std::mutex> latch(queue_mutex);
cv_task.wait(latch, [this](){ return stop || !tasks.empty(); });
if (!tasks.empty())
To this:
std::unique_lock<std::mutex> latch(queue_mutex);
while (!stop)
{
cv_task.wait(latch, [this](){ return stop || !tasks.empty(); });
while (!tasks.empty() && !stop)
And then remove the else if (stop) block and the end of the function.
The main impact this has is that it avoids the extra "unlock" and "lock" on queue_mutex as a result of latch going out of scope on each iteration of the while loop. The changing of if (!tasks.empty()) to while (!tasks.empty()) might save a cycle or two as well by letting the currently executing thread which has the quantum keep the lock and try to deque the next work item.
<opinion>
One final thing. I'm always of the opinion that the notify should be outside the lock. That way, there's no lock contention when the other thread is woken up by the thread that just updated the queue. But I've never actually measured this assumption, so take it with a grain of salt:
template<class F> void enqueue(F&& f)
{
queue_mutex.lock();
tasks.emplace_back(std::forward<F>(f));
queue_mutex.unlock();
cv_task.notify_one();
}