Avoiding lost-wakeup when condition update is a blocking function - c++

Question

I'm writing an event loop that goes to sleep when there's no work to do by waiting on a "work to do" condition variable (work_to_do). This condition variable could be notified by different threads based on various events. When an event happens in another thread it notifies on the condition variable, waking up the event loop which then checks the conditions that could have triggered the notify, loops until there's no more work to do and then waits again. One of the conditions is set by a blocking function (WaitForMessage()).
The event loop thread:
std::lock_guard<std::mutex> lock(work_to_do_lock);
for (;;) {
if (condition1) {
// Act on condition 1.
} else if (condition2) {
// Act on condition 2.
} else if (HasMessage()) {
// Act on receiving message.
} else {
work_to_do.wait(lock);
}
}
The thread that handles the notify from the blocking function:
for (;;) {
// Wait for message to be received (blocking). Once it returns you are
// guaranteed that HasMessage() will return true.
WaitForMessage();
// Wake-up the main event loop.
work_to_do.notify_one();
}
The main thread acquires a lock on the mutex guarding the condition variable (work_to_do_lock) before entering the event loop, and passes it into the wait() call when there's no work to do. To avoid lost-wakeups, the common advice is that all notifiers must hold the lock while updating their condition states. However, if you were to guard the WaitForMessage() call with work_to_do_lock you could prevent other signals from waking up the event loop.
The solution I came up with is to acquire and release the lock after WaitForMessage() but before notify_one():
for (;;) {
// Wait for message to be received (blocking). Once it returns you are
// guaranteed that HasMessage() will return true.
WaitForMessage();
{
std::lock_guard<std::mutex> lock(work_to_do_lock);
}
// Wake-up the main event loop.
work_to_do.notify_one();
}
This should avoid the lost-wakeup issue, as it is no longer possible for both the condition to become true (WaitForMessage() to return) and the notify_one() to occur in-between the condition check (HasMessage()) and the wait().
An alternative approach is to not rely on HasMessage() and just update a shared variable, which we could guard with the lock:
for (;;) {
// Wait for message to be received (blocking). Once it returns you are
// guaranteed that HasMessage() will return true.
WaitForMessage();
{
std::lock_guard<std::mutex> lock(work_to_do_lock);
has_message = true;
}
// Wake-up the main event loop.
work_to_do.notify_one();
}
Corresponding event loop that checks new condition predicate:
std::lock_guard<std::mutex> lock(work_to_do_lock);
for (;;) {
if (condition1) {
// Act on condition 1.
} else if (condition2) {
// Act on condition 2.
} else if (has_message) {
has_message = false;
// Act on receiving message.
} else {
work_to_do.wait(lock);
}
}
I've never seen the former approach before, so I was wondering if there was a flaw with the design or a reason it's typically avoided? It seems that you could use this approach as a general replacement for locking the condition variable lock before the condition state update, assuming that the specific condition state write/read itself is protected by some mutual exclusion mechanism.

Answer 1

Your approach works, but it’s less efficient than one which reuses whatever synchronization makes it safe to call WaitForMessage and HasMessage concurrently (or, put differently, takes your work_to_do_lock to update the HasMessage value rather than (say) using an atomic for it). Of course, if that’s inaccessible to this code, this is about the best you can do, since you need mutual exclusion for the other conditions.