Implementing a "temporarily suspendable" concurrent loop in C++ - c++

Question

I'm writing a program whose main thread spawns a worker thread that performs some work, sleeps for a set amount of time in an infinite loop, i.e. the worker thread executes:
void do_work() {
for (;;) {
// do some work
std::this_thread::sleep_for(100ms);
}
}
Now, I would additionally like to be able to temporarily completely disable this worker thread from the main thread, i.e. I would like to write the following functions:
disable_worker(): disable the worker thread
enable_worker(): enable the worker thread again
What I've come up with is the following:
#include <chrono>
#include <condition_variable>
#include <mutex>
#include <thread>
using namespace std::literals::chrono_literals;
bool enabled;
std::mutex mtx;
std::condition_variable cond;
void disable_worker() {
std::lock_guard<std::mutex> lock(mtx);
enabled = false;
}
void enable_worker() {
{
std::lock_guard<std::mutex> lock(mtx);
enabled = true;
}
cond.notify_one();
}
void do_work() {
for (;;) {
std::unique_lock<std::mutex> lock(mtx);
cond.wait(lock, []{ return enabled; });
// ... do some work ...
std::this_thread::sleep_for(100ms);
}
}
int main() {
std::thread t(do_work);
// ... enable/disable t as necessary ...
}
I suppose this works (at least I can't spot any issues), however, I would also like to guarantee that when either of enable_worker and disable_worker return (in the main thread), the working thread is guaranteed to be either blocking on the condition variable or sleeping, i.e. not performing any work. How can I implement this without any race conditions?

Answer 1

Here is an API for a concurrent door with a queue counter and a the idea of using it "sleepily".
struct SleepyDoorQueue {
void UseDoor() {
auto l = lock();
++queue_size;
cv.notify_all();
cv.wait( l, [&]{ return open; } );
--queue_size;
}
// sleeps for a while, then tries to open the door.
// considered in queue while sleeping.
template<class Rep, class Period>
void SleepyUseDoor( const std::chrono::duration<Rep, Period>& rel_time ) {
{
auto l = lock();
++queue_size;
cv.notify_all();
}
std::this_thread::sleep_for(rel_time);
auto l = lock();
cv.wait( l, [&]{ return open; } );
--queue_size;
}
void CloseDoor() {
auto l = lock();
open = false;
}
void OpenDoor() {
auto l = lock();
open = true;
cv.notify_all();
}
void WaitForQueueSize(std::size_t n) const {
auto l = lock();
cv.wait(l, [&]{ return queue_size >= n; } );
}
explicit SleepyDoorQueue( bool startOpened = true ):open(startOpened) {}
private:
std::condition_variable cv;
mutable std::mutex m;
std::size_t queue_size = 0;
bool open = true;
auto lock() const { return std::unique_lock(m); }
};
the main thread closes the door, and waits for a queue size of 1 to ensure that the worker thread isn't working.
The worker thread does a SleepyUseDoor to try to open it after sleeping for 100ms.
When the worker thread can do work, the main thread just opens the door.
This will be inefficient with a large number of worker and controller threads, as I use the same cv for both the queue and door opening message. So one will cause the other threads to wake up spuriously. With one worker and one controller thread, the messages won't be spurious to any significant degree.
I only notify on queue size increase and door opening, but I do more than 1 notification on purpose (if there is someone waiting for a queue size change and a door opener eats it, that would suck).
You could probably implement this with two doors actually.
struct Door {
// blocks until the door is open
void UseDoor() const {
auto l = lock();
cv.wait(l, [&]{ return open; });
}
// opens the door. Notifies blocked threads trying to use the door.
void OpenDoor() {
auto l = lock();
open = true;
cv.notify_all();
}
// closes the door.
void CloseDoor() {
auto l = lock();
open = false;
}
explicit Door(bool startOpen=true):open(startOpen) {}
private:
std::condition_variable cv;
mutable std::mutex m;
bool open = true;
auto lock() const { return std::unique_lock(m); }
};
The worker thread does this:
Door AmNotWorking(true);
Door CanWork(true);
void work() {
for(;;) {
canWork.UseDoor()
AmNotWorking.CloseDoor();
// work
AmNotWorking.OpenDoor();
std::this_thread::sleep_for(100ms);
}
}
the controller thread does:
void preventWork() {
CanWork.CloseDoor();
AmNotWorking.UseDoor();
}
void allowWork() {
CanWork.OpenDoor();
}
but I see a race condition there; between CanWork.UseDoor() and AmNotWorking.OpenDoor(); someone could close the CanWork door then read the AmNotWorking door. We need that to be atomic.
// Goes through the door when it is open.
// atomically runs the lambda passed in while the
// mutex is locked with checking the door state.
// WARNING: this can cause deadlocks if you do the
// wrong things in the lambda.
template<class F>
void UseDoor(F atomicWhenOpen) const {
auto l = lock();
cv.wait(l, [&]{ return open; });
atomicWhenOpen();
}
that does an atomic operation when we successfully use the door. A bit dangerous, but the worker thread can now:
void work() {
for(;;) {
canWork.UseDoor([]{AmNotWorking.CloseDoor();});
// work
AmNotWorking.OpenDoor();
std::this_thread::sleep_for(100ms);
}
}
this guarantees we have the "AmNotWorking" door closed in the same lock as we verified the "CanWork" door is open.
void preventWork() {
CanWork.CloseDoor();
AmNotWorking.UseDoor();
}
If the "use can work and close am working" operation happens before the CanWork.CloseDoor(), we won't be able to AmNotWorking.UseDoor() until the worker thread finishes their work.
If it happens after CanWork.CloseDoor(), then the AmNotWorking.UseDoor() is closed, so we again wait until the worker thread is not working.
We can't CanWork.CloseDoor() between the can work door being used and the AmNotWorking being closed, which is what that extra atomic lambda callback gives us.
We can probably make a less dangerous primitive, but I'm not sure how to do it elegantly.
Maybe a simple semaphore?
template<class T = std::ptrdiff_t>
struct Semaphore {
void WaitUntilExactValue( T t ) const {
auto l = lock();
cv.wait( l, [&]{ return value==t; }
}
void WaitUntilAtLeastValue( T t ) const {
auto l = lock();
cv.wait( l, [&]{ return value>=t; }
}
void WaitUntilAtMostValue( T t ) const {
auto l = lock();
cv.wait( l, [&]{ return value<=t; }
}
void Increment() {
auto l = lock();
++value;
cv.notify_all();
}
void BoundedIncrement(T ceil) {
auto l = lock();
cv.wait(l, [&]{ return value+1 <= ceil; });
++value;
cv.notify_all();
}
void Decrement() {
auto l = lock();
--value;
cv.notify_all();
}
void BoundedDecrement(T floor) {
auto l = lock();
cv.wait(l, [&]{ return value-1 >= floor; });
--value;
cv.notify_all();
}
explicit Semaphore( T in = 0 ):value(std::forward<T>(in)) {}
private:
std::condition_variable cv;
mutable std::mutex m;
T value = 0;
auto lock() const; // see above
};
then
Semaphore workLimit(1);
void work() {
for(;;) {
workLimit.BoundedDecrement(0);
// work
workLimit.Increment();
std::this_thread::sleep_for(100ms);
}
}
void preventWork() {
workLimit.Decrement();
workLimit.WaitUntilExactValue(0);
}
void allowWork() {
workLimit.Increment();
}
Here, the workLimit is how many more workers are permitted to be working at this point. It is 1 to start with.
When a worker is working but not allowed to, it is -1. When it is working and allowed to, it is 0. When it is sleeping and allowed to work, it is 1. When it is sleeping (either because it is in sleep for, or bounded decrement) and not allowed to work, it is 0.