I have a thread pool which executes tasks asynchronously. But I need to wait for a certain task to complete before proceeding (running the task in current thread is not allowed, the task must be run by a worker thread).
What's the easiest way to achieve this using C++11 or Boost?
pool.enqueue([]() {
std::this_thread::sleep_for(2s); // task 1
// notify task 1 completion???
std::this_thread::sleep_for(2s); // task 2
});
// wait until task 1 is complete???
If you have a thread pool, either the pool should handle the dependencies or
you should chain the continuation task from the first task directly.
Otherwise, the pool can deadlock. Imagine just for example a pool with 1 thread. It would block indefinitely. Same can occur with many threads given enough task inter dependencies.
Use std::condition_variable:
std::mutex m;
bool task1_done=false;
std::condition_variable cond_var;
pool.enqueue([&cond_var, &task1_done]() {
std::this_thread::sleep_for(2s); // task 1
// notify task 1 completion
task1_done=true;
cond_var.notify_one();
std::this_thread::sleep_for(2s); // task 2
});
// wait until task 1 is complete
std::unique_lock<std::mutex> lock(m);
while( !task1_done ) {
cond_var.wait(lock);
}
You can use mutex and wait_for/wait_until
You can look example
Going to answer my own question.
I ended up using a future:
std::packaged_task<int()> task1([]() {
std::this_thread::sleep_for(2s); // task 1
return 1;
});
std::future<int> task1result = task1.get_future();
std::thread thread1([&]() {
task1();
std::this_thread::sleep_for(2s); // task 2
});
int rc1 = task1result.get();
printf("task1 complete: %d\n", rc1);
thread1.join();
printf("thread complete\n");
And no, there is no chance for a deadlock since there is no cyclic dependency between the threads (the waiting thread is not part of the pool).
Related
I need a thread to perform processing every one second accurately. Suppose if the worker thread is busy on some operation that takes more than one second, I want the worker thread to miss the 1s expiry notification and perform the processing in the next cycle.
I am trying to implement this using two threads. One thread is a worker thread, another thread sleeps for one second and notifies the worker thread via condition variable.
Code is shown below
Worker thread
while(!threadExit){
std::unique_lock<std::mutex> lock(mutex);
// Block until a signal is received
condVar_.wait(lock, [this](){return (threadExit || performProc);)});
if(threadExit_){
break;
}
// Perform the processing
..............
}
Timer thread
while(!threadExit)
{
{
std::unique_lock<std::mutex> lock(mutex);
performProc= false;
}
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
if(threadExit){
break;
}
{
std::unique_lock<std::mutex> lock(mutex);
performProc= true;
}
condVar.notify_one();
}
Please note the variable threadExit is set by the main thread under the mutex lock and notified to worker thread. The timer thread can see this flag when it wakes up(which should be fine for my implementation)
Do you think performProc may set to false again before the worker thread sees it as true? If yes, can you please throw some light on how to tackle this problem? Thanks!
Unless threadExit is atomic, the code exhibits undefined behavior (race condition). All accesses to threadExit must be protected by a mutex, so also reads in while(!threadExit) and if(threadExit)....
But there's no need to do any of this. You can run everything in the same thread if you use sleep_until (and a steady clock) instead of sleep_for.
#include <chrono>
#include <iostream>
#include <thread>
using namespace std::literals;
void do_work() {
std::cout << "Work # " << std::chrono::system_clock::now() << std::endl;
}
int main() {
while (true) {
auto t = ceil<std::chrono::seconds>(std::chrono::steady_clock::now() + 600ms);
std::this_thread::sleep_until(t);
do_work();
}
}
Output:
Work # 2022-03-04 09:56:51.0148904
Work # 2022-03-04 09:56:52.0134687
Work # 2022-03-04 09:56:53.0198704
Work # 2022-03-04 09:56:54.0010437
Work # 2022-03-04 09:56:55.0148975
. . .
I'm having a problem where I'm having a few condition_variable's get stuck in their wait phase even though they've been notified. Each one even has a predicate that's being set just in case they miss the notify call from the main thread.
Here's the code:
unsigned int notifyCount = 10000;
std::atomic<int> threadCompletions = 0;
for (unsigned int i = 0; i < notifyCount; i++)
{
std::atomic<bool>* wakeUp = new std::atomic<bool>(false);
std::condition_variable* condition = new std::condition_variable();
// Worker thread //
std::thread([&, condition, wakeUp]()
{
std::mutex mutex;
std::unique_lock<std::mutex> lock(mutex);
condition->wait(lock, [wakeUp] { return wakeUp->load(); });
threadCompletions++;
}).detach();
// Notify //
*wakeUp = true;
condition->notify_one();
}
Sleep(5000); // Sleep for 5 seconds just in case some threads are taking a while to finish executing
// Check how many threads finished (threadCompletions should be equal to notifyCount)
Unless I'm mistaken, after the for loop is done, threadCompletions should always be equal to notifyCount. Very often though, it is not.
When running in release, I'll sometimes get just one or two out of 10000 threads that never finished, but when running in debug, I'll get 20 or more.
I thought maybe the wait call in the thread is happening after the main thread's notify_one call (meaning it missed it's notification to wake up), so I passed a predicate into wait to insure that it doesn't get stuck waiting. But it still does in some cases.
Does anyone know why this is happening?
You are assuming the call to wait() is atomic. I don't believe it is. That is why it requires the use of a mutex and a lock.
Consider the following:
Main Thread. Child Thread
// This is your wait unrolled.
while (!wakeUp->load()) {
// This is atomic
// But already checked in the
// thread.
*wakeUp = true;
// Child has not yet called wait
// So this notify_one is wasted.
condition->notify_one();
// The previous call to notify_one
// is not recorded and thus the
// thread is now locked in this wait
// never to be let free.
wait(lock);
}
// Your race condition.
Calls to notify_one() and wait() should be controlled via the same mutext to make sure they don't overlap like this.
for (unsigned int i = 0; i < notifyCount; i++)
{
std::atomic<bool>* wakeUp = new std::atomic<bool>(false);
std::mutex* mutex = new std::mutex{};
std::condition_variable* condition = new std::condition_variable();
// Worker thread //
std::thread([&]()
{
std::unique_lock<std::mutex> lock(*mutex);
condition->wait(lock, [&wakeUp] { return wakeUp->load(); });
threadCompletions++;
}).detach();
// Notify //
*wakeUp = true;
std::unique_lock<std::mutex> lock(*mutex);
condition->notify_one();
}
// Don't forget to clean up the new structures correctly/.
You have data racing. Consider following scenario:
Worker Thread: condition variable tests for whether wakeup is true - it isn't
Main Thread: wakeup is set to true and condition variable is getting notified
Worker Thread: condition_variable triggers wait but it happens after notification already occurred - impling that notification misses and the thread might never wake up.
Normally, synchronization of condition variables is done via mutexes - atomics aren't too helpful here. In C++20 there will be special mechanism for waiting/notifying in atomics.
I'm trying to do this with the C++11 concurrency support.
I have a sort of thread pool of worker threads that all do the same thing, where a master thread has an array of condition variables (one for each thread, they need to 'start' synchronized, ie not run ahead one cycle of their loop).
for (auto &worker_cond : cond_arr) {
worker_cond.notify_one();
}
then this thread has to wait for a notification of each thread of the pool to restart its cycle again. Whats the correct way of doing this? Have a single condition variable and wait on some integer each thread that isn't the master is going to increase? something like (still in the master thread)
unique_lock<std::mutex> lock(workers_mtx);
workers_finished.wait(lock, [&workers] { return workers = cond_arr.size(); });
I see two options here:
Option 1: join()
Basically instead of using a condition variable to start the calculations in your threads, you spawn a new thread for every iteration and use join() to wait for it to be finished. Then you spawn new threads for the next iteration and so on.
Option 2: locks
You don't want the main-thread to notify as long as one of the threads is still working. So each thread gets its own lock, which it locks before doing the calculations and unlocks afterwards. Your main-thread locks all of them before calling the notify() and unlocks them afterwards.
I see nothing fundamentally wrong with your solution.
Guard workers with workers_mtx and done.
We could abstract this with a counting semaphore.
struct counting_semaphore {
std::unique_ptr<std::mutex> m=std::make_unique<std::mutex>();
std::ptrdiff_t count = 0;
std::unique_ptr<std::condition_variable> cv=std::make_unique<std::condition_variable>();
counting_semaphore( std::ptrdiff_t c=0 ):count(c) {}
counting_semaphore(counting_semaphore&&)=default;
void take(std::size_t n = 1) {
std::unique_lock<std::mutex> lock(*m);
cv->wait(lock, [&]{ if (count-std::ptrdiff_t(n) < 0) return false; count-=n; return true; } );
}
void give(std::size_t n = 1) {
{
std::unique_lock<std::mutex> lock(*m);
count += n;
if (count <= 0) return;
}
cv->notify_all();
}
};
take takes count away, and blocks if there is not enough.
give adds to count, and notifies if there is a positive amount.
Now the worker threads ferry tokens between two semaphores.
std::vector< counting_semaphore > m_worker_start{count};
counting_semaphore m_worker_done{0}; // not count, zero
std::atomic<bool> m_shutdown = false;
// master controller:
for (each step) {
for (auto&& starts:m_worker_start)
starts.give();
m_worker_done.take(count);
}
// master shutdown:
m_shutdown = true;
// wake up forever:
for (auto&& starts:m_worker_start)
starts.give(std::size_t(-1)/2);
// worker thread:
while (true) {
master->m_worker_start[my_id].take();
if (master->m_shutdown) return;
// do work
master->m_worker_done.give();
}
or somesuch.
live example.
This code is simplification of real project code. Main thread create worker thread and wait with std::condition_variable for worker thread really started. In code below std::condition_variable wakes up after current_thread_state becomes "ThreadState::Stopping" - this is the second notification from worker thread, that is the main thread did not wake up after the first notification, when current_thread_state becomes "ThreadState::Starting". The result was deadlock. Why this happens? Why std::condition_variable not wake up after first thread_event.notify_all()?
int main()
{
std::thread thread_var;
struct ThreadState {
enum Type { Stopped, Started, Stopping };
};
ThreadState::Type current_thread_state = ThreadState::Stopped;
std::mutex thread_mutex;
std::condition_variable thread_event;
while (true) {
{
std::unique_lock<std::mutex> lck(thread_mutex);
thread_var = std::move(std::thread([&]() {
{
std::unique_lock<std::mutex> lck(thread_mutex);
cout << "ThreadFunction() - step 1\n";
current_thread_state = ThreadState::Started;
}
thread_event.notify_all();
// This code need to disable output to console (simulate some work).
cout.setstate(std::ios::failbit);
cout << "ThreadFunction() - step 1 -> step 2\n";
cout.clear();
{
std::unique_lock<std::mutex> lck(thread_mutex);
cout << "ThreadFunction() - step 2\n";
current_thread_state = ThreadState::Stopping;
}
thread_event.notify_all();
}));
while (current_thread_state != ThreadState::Started) {
thread_event.wait(lck);
}
}
if (thread_var.joinable()) {
thread_var.join();
current_thread_state = ThreadState::Stopped;
}
}
return 0;
}
Once you call the notify_all method, your main thread and your worker thread (after doing its work) both try to get a lock on the thread_mutex mutex. If your work load is insignificant, like in your example, the worker thread is likely to get the lock before the main thread and sets the state back to ThreadState::Stopped before the main thread ever reads it. This results in a dead lock.
Try adding a significant work load, e.g.
std::this_thread::sleep_for( std::chrono::seconds( 1 ) );
to the worker thread. Dead locks are far less likely now. Of course, this is not a fix for your problem. This is just for illustrating the problem.
You have two threads racing: one writes values of current_thread_state twice, another reads the value of current_thread_state once.
It is indeterminate whether the sequence of events is write-write-read or write-read-write as you expect, both are valid executions of your application.
I am running the following chunk of the code. This code is going to create 5 slave threads and 1 main thread. All slave threads are waited for the main thread to make the data ready and when the data gets ready, all slaves will notify to start processing.
My question is, it is possible that before the slave threads start waiting for the conditional_variable, the main thread make the data ready and notify the waited threads. In this case, some threads which were waited will get the notification and start processing but the ones which were not waited, will starting waiting for a notification which will NEVER come.
If you run this example, this case won't happen but I am looking for a way to make sure that all the slave threads are waiting for the notification, then notifying them. Do you know how can I do that?
/*
Condition Variables - Many waiting threads
Shows how one condition variable can be used to notify multiple threads
that a condition has occured.
* Part of "Threading with Boost - Part IV: Condition Variables", published at:
http://antonym.org/boost
Copyright (c) 2015 Gavin Baker <gavinb#antonym.org>
Published under the MIT license, see LICENSE for details
*/
#include <cstdio>
#include <boost/thread.hpp>
boost::condition_variable data_ready_cond;
boost::mutex data_ready_mutex;
bool data_ready = false;
void master_thread()
{
printf("+++ master thread\n");
// Pretend to work
printf(" master sleeping...\n");
boost::chrono::milliseconds sleepDuration(750);
boost::this_thread::sleep_for(sleepDuration);
// Let other threads know we're done
printf(" master notifying...\n");
data_ready = true;
data_ready_cond.notify_all();
printf("--- master thread\n");
}
void slave_thread(int id)
{
printf("+++ slave thread: %d\n", id);
boost::unique_lock<boost::mutex> lock(data_ready_mutex);
while (!data_ready)
{
data_ready_cond.wait(lock);
}
printf("--- slave thread: %d\n", id);
}
int main()
{
printf("Spawning threads...\n");
boost::thread slave_1(slave_thread, 1);
boost::thread slave_2(slave_thread, 2);
boost::thread slave_3(slave_thread, 3);
boost::thread slave_4(slave_thread, 4);
boost::thread master(master_thread);
printf("Waiting for threads to complete...\n");
slave_1.join();
slave_2.join();
slave_3.join();
slave_4.join();
master.join();
printf("Done\n");
return 0;
}
You have race condition - setting flag and notifying slave threads is not atomic. So you just have to lock data_ready_mutex before you are modifying data_ready flag in main thread. This will eliminate race condition, slave thread either will see data_ready false and go to wait on condition variable and will be notified, or it will acquire mutex lock only after data_ready is set to true and so it will not wait at all.