I have two condition variables:
CondVar1
CondVar2
Used in two threads like this (pseudo-code):
// thread1 starts in 'waiting' mode, and then Thread2 signals
void Thread1()
{
CondVar1->Wait();
CondVar2->Signal();
}
void Thread2()
{
CondVar1->Signal();
CondVar2->Wait();
}
Can this cause a deadlock? meaning, thread1 waits, thread2 signals, and then can thread1 signals before thread2 enters Wait(), meaning thread2 will never return?
Thanks
You don't usually just wait on a condition variable. The common use pattern is holding a lock, checking a variable that determines whether you can proceed or not and if you cannot wait in the condition:
// pseudocode
void push( T data ) {
Guard<Mutex> lock( m_mutex ); // Hold a lock on the queue
while (m_queue.full()) // [1]
m_cond1.wait(lock); // Wait until a consumer leaves a slot for me to write
// insert data
m_cond2.signal_one(); // Signal consumers that might be waiting on an empty queue
}
Some things to note: most libraries allow for spurious wakes in condition variables. While it is possible to implement a condition variable that avoid spurious wakes, the cost of the operations would be higher, so it is considered a lesser evil to require users to recheck the state before continuing (while loop in [1]).
Some libraries, notably C++11, allow you to pass a predicate, and will implement the loop internally: cond.wait(lock, [&queue](){ return !queue.full(); } );
There are two situations that could lead to a deadlock here:
In normal execution, the one you described. It is possible that the variable is signaled before the thread reaches the call to Wait, so the signal is lost.
A spurious wake-up could happen, causing the first thread to leave the call to Wait before actually being signaled, hence signaling Thread 2 who is not yet waiting.
You should design your code as follows when using signaling mechanisms:
bool thread1Waits = true;
bool thread2Waits = true;
void Thread1()
{
while(thread1Waits) CondVar1->Wait();
thread2Waits = false;
CondVar2->Signal();
}
void Thread2()
{
thread1Waits = false;
CondVar1->Signal();
while(thread2Waits) CondVar2->Wait();
}
Of course, this assumes there are locks protecting the condition variables and that additionally thread 1 runs before thread 2.
Related
We have implemented TaskRunner whose functions will be called by different threads to start, stop and post tasks. TaskRunner will internally create a thread and if the queue is not empty, it will pop the task from queue and executes it. Start() will check if the thread is running. If not creates a new thread. Stop() will join the thread. The code is as below.
bool TaskRunnerImpl::PostTask(Task* task) {
tasks_queue_.push_back(task);
return true;
}
void TaskRunnerImpl::Start() {
std::lock_guard<std::mutex> lock(is_running_mutex_);
if(is_running_) {
return;
}
is_running_ = true;
runner_thread_ = std::thread(&TaskRunnerImpl::Run, this);
}
void TaskRunnerImpl::Run() {
while(is_running_) {
if(tasks_queue_.empty()) {
continue;
}
Task* task_to_run = tasks_queue_.front();
task_to_run->Run();
tasks_queue_.pop_front();
delete task_to_run;
}
}
void TaskRunnerImpl::Stop() {
std::lock_guard<std::mutex> lock(is_running_mutex_);
is_running_ = false;
if(runner_thread_.joinable()) {
runner_thread_.join();
}
}
We want to use conditional variables now otherwise the thread will be continuously checking whether the task queue is empty or not. We implemented as below.
Thread function (Run()) will wait on condition variable.
PostTask() will signal if some one posts a task.
Stop() will signal if some one calls stop.
Code is as below.
bool TaskRunnerImpl::PostTask(Task* task) {
std::lock_guard<std::mutex> taskGuard(m_task_mutex);
tasks_queue_.push_back(task);
m_task_cond_var.notify_one();
return true;
}
void TaskRunnerImpl::Start() {
std::lock_guard<std::mutex> lock(is_running_mutex_);
if(is_running_) {
return;
}
is_running_ = true;
runner_thread_ = std::thread(&TaskRunnerImpl::Run, this);
}
void TaskRunnerImpl::Run() {
while(is_running_) {
Task* task_to_run = nullptr;
{
std::unique_lock<std::mutex> mlock(m_task_mutex);
m_task_cond_var.wait(mlock, [this]() {
return !(is_running_ && tasks_queue_.empty());
});
if(!is_running_) {
return;
}
if(!tasks_queue_.empty()) {
task_to_run = tasks_queue_.front();
task_to_run->Run();
tasks_queue_.pop_front();
}
}
if(task_to_run)
delete task_to_run;
}
}
void TaskRunnerImpl::Stop() {
std::lock_guard<std::mutex> lock(is_running_mutex_);
is_running_ = false;
m_task_cond_var.notify_one();
if(runner_thread_.joinable()) {
runner_thread_.join();
}
}
I have couple of questions as below. Can some one please help me to understand these.
Condition variable m_task_cond_var is linked with mutex m_task_mutex. But Stop() already locks mutex is_running_mutex to gaurd 'is_running_'. Do I need to lock m_task_mutex before signaling? Here I am not convinced why to lock m_task_mutex as we are not protecting any thing related to task queue.
In Thread function(Run()), we are reading is_running_ without locking is_running_mutex. Is this correct?
Do I need to lock m_task_mutex before signaling [In Stop]?
When the predicate being tested in condition_variable::wait method depends on something happening in the signaling thread (which is almost always), then you should obtain the mutex before signaling. Consider the following possibility if you are not holding the m_task_mutex:
The watcher thread (TaskRunnerImpl::Run) wakes up (via spurious wakeup or being notified from elsewhere) and obtains the mutex.
The watcher thread checks its predicate and sees that it is false.
The signaler thread (TaskRunnerImpl::Stop) changes the predicate to return true (by setting is_running_ = false;).
The signaler thread signals the condition variable.
The watcher thread waits to be signaled (bad)
the signal has already come and gone
the predicate was false, so the watcher begins waiting, possibly indefinitely.
The worst that can happen if you are holding the mutex when you signal is that, the blocked thread (TaskRunnerImpl::Run) wakes up and is immediately blocked when trying to obtain the mutex. This can have some performance implications.
In [TaskRunnerImpl::Run] , we are reading is_running_ without locking is_running_mutex. Is this correct?
In general no. Even if it's of type bool. Because a boolean is typically implemented as a single byte, it's possible that one thread is writing to the byte while you are reading, resulting in a partial read. In practice, however, it's safe. That said, you should obtain the mutex before you read (and then release immediately afterwards).
In fact, it may be preferable to use std::atomic<bool> instead of a bool + mutex combination (or std::atomic_flag if you want to get fancy) which will have the same effect, but be easier to work with.
Do I need to lock m_task_mutex before signaling [In Stop]?
Yes you do. You must change condition under the same mutex and send signal either after the mutex is locked or unlocked after the change. If you do not use the same mutex, or send signal before that mutex is locked you create race condition that std::condition_variable is created to solve.
Logic is this:
Watching thread locks mutex and checks watched condition. If it did not happen it goes to sleep and unlocks the mutex atomically. So signaling thread lock the mutex, change condition and signal. If signalling thread does that before watching one locks the mutex, then watchiong one would see condition happen and would not go to sleep. If it locks before, it would go to sleep and woken when signalling thread raise the signal.
Note: you can signal condition variable before or after mutex is unlocked, both cases is correct but may affect performance. But it is incorrect to signal before locking the mutex.
Condition variable m_task_cond_var is linked with mutex m_task_mutex. But Stop() already locks mutex is_running_mutex to gaurd 'is_running_'. Do I need to lock m_task_mutex before signaling? Here I am not convinced why to lock m_task_mutex as we are not protecting any thing related to task queue.
You overcomlicated your code and made things worse. You should use only one mutex in this case and it would work as intended.
In Thread function(Run()), we are reading is_running_ without locking is_running_mutex. Is this correct?
On x86 hardware it may "work", but from language point of view this is UB.
Suppose I have two threads and one shared c++ 11 condition variable.
What whould happen if thread1 call notify and after that thread2 call wait?
Will thread2 block forever or it will continue it's work due to call of notify by thread1?
Edit:
enum bcLockOperation
{
bcLockOperation_Light = -1,
bcLockOperation_Medium = 50,
bcLockOperation_Heavy = 1
}
class BC_COREDLL_EXP bcCustomMutex
{
private:
bcCustomMutex(const bcCustomMutex&);
bcCustomMutex& operator=(const bcCustomMutex&);
protected:
bcAtomic<int> mFlag;
bcMutex mMutex;
bcConditionVariable mCond;
public:
bcCustomMutex() { bcAtomicOperation::bcAtomicInit(mFlag, 0); };
~bcCustomMutex() {};
void lock(bcLockOperation pLockOperation = bcLockOperation_Medium)
{
bcINT32 lNewLoopCount = static_cast<bcINT32>(pLockOperation);
bcINT32 lLoopCounter = 0;
bcINT32 lExpected = 0;
bcINT32 lLoopCount = bcAtomicOperation::bcAtomicLoad(mFlag, bcMemoryOrder_Relaxed);
while (true)
{
while(bcAtomicOperation::bcAtomicLoad(mFlag, bcMemoryOrder_Relaxed) != 0 && lLoopCounter != lLoopCount)
++lLoopCounter;
bcAtomicOperation::bcAtomicCompareExchangeStrong(
mFlag,
&lExpected,
lNewLoopCount,
bcMemoryOrder_Acquire,
bcMemoryOrder_Relaxed);
if(lExpected == 0)
{
return;
}
else if(lLoopCounter == lLoopCount)
{
bcLockGuard<bcMutex> lGuard(mMutex);
mCond.wait(mMutex);
}
else
{
continue;
}
}
void UnLock()
{
bcAtomicOperation::bcAtomicStore(mFlag, 0, bcMemoryOrder_Relaxed);
bcUniqueLock<bcMutex> lGuard(mMutex);
mCond.notifyOne();
}
bcBOOL TryLock()
{
};
};
I want to write a custom mutex such that each thread can provide an argument that represents the complexity of operations that the current thread wants to execute.
If the complexity of the operation is low other threads will be in a loop like a spin lock but if the complexity of the operation is medium each thread will iterate 50 times and then will sleep by condition variable and if operation is very complex other threads will go to sleep directly.
now assume thread1 locks this mutex and thread2 goes for waiting due to its loopCounter reaching its end and right before locking the condition variable's mutex, thread1 calls notify on the condition variable. Now thread2 will sleep until another thread locks the custom mutex and then calls unlock on it.
I am new to multithreading and I want to learn. I know that my class may contain errors or may be completely wrong, but is there any way to correct this problem or a good algorithm to write such a mutex.
Thread2 will block until someone calls notify. Calls to notify release threads that are waiting at the time of the call. If there are no threads waiting, they do nothing. They aren't saved.
Usually both the code that decides to wait and the code that decides to notify share the same mutex. So thread2 will never "miss" the notify from thread1.
Here's the classic lock-based concurrent queue example:
void push(int x)
{
lock_guard<mutex> guard{queue_mutex};
thequeue.push(x);
not_empty_condition.notify_one();
}
int pop()
{
unique_lock<mutex> guard{queue_mutex};
not_empty_condition.wait(guard, []{ return !thequeue.empty(); } );
int x = thequeue.front();
thequeue.pop();
return x;
}
Assume thread1 and thread2 are running push() and pop() respectively. Only one of them will be in the critical section at a time.
If thread2 has the lock, either it never waits because the queue is not empty (so "losing" a notify is harmless), or it sits there waiting for a notify (which won't be lost).
If thread1 got the lock, it will put an element in the queue; if thread2 was waiting, it will get notified properly; if thread2 was still waiting for the mutex, it will never wait, as there is at least one element on the queue, so losing a notify is harmless.
In this manner, a notify is only lost if it was not needed in the first place.
Now, if you have a different usage for condition variables in mind, where "losing" a notification has any consequence, I believe you either have a race condition, or are using the wrong tool altogether.
I would like to write a class that wraps around std::thread and behaves like a std::thread but without actually allocating a thread every time I need to process something async. The reason is that I need to use multi threading in a context where I'm not allow to dynamically allocate and I also don't want to have the overhead of creating a std::thread.
Instead, I want a thread to run in a loop and wait until it can start processing. The client calls invoke which wakes up the thread. The Thread locks a mutex, does it's processing and falls asleep again. A function join behaves like std::thread::join by locking until the thread frees the lock (i.e. falls asleep again).
I think I got the class to run but because of a general lack of experience in multi threading, I would like to ask if anybody can spot race conditions or if the approach I used is considered "good style". For example, I'm not sure if temporary locking the mutex is a decent way to "join" the thread.
EDIT
I found another race condition: when calling join directly after invoke, there is no reason the thread already locked the mutex and thus locks the caller of join until the thread goes to sleep. To prevent this, I had to add a check for the invoke counter.
Header
#pragma once
#include <thread>
#include <atomic>
#include <mutex>
class PersistentThread
{
public:
PersistentThread();
~PersistentThread();
// set function to invoke
// locks if thread is currently processing _func
void set(const std::function<void()> &f);
// wakes the thread up to process _func and fall asleep again
// locks if thread is currently processing _func
void invoke();
// mimics std::thread::join
// locks until the thread is finished with it's loop
void join();
private:
// intern thread loop
void loop(bool *initialized);
private:
bool _shutdownRequested{ false };
std::mutex _mutex;
std::unique_ptr<std::thread> _thread;
std::condition_variable _cond;
std::function<void()> _func{ nullptr };
};
Source File
#include "PersistentThread.h"
PersistentThread::PersistentThread()
{
auto lock = std::unique_lock<std::mutex>(_mutex);
bool initialized = false;
_thread = std::make_unique<std::thread>(&PersistentThread::loop, this, &initialized);
// wait until _thread notifies, check bool initialized to prevent spurious wakeups
_cond.wait(lock, [&] {return initialized; });
}
PersistentThread::~PersistentThread()
{
{
std::lock_guard<std::mutex> lock(_mutex);
_func = nullptr;
_shutdownRequested = true;
// wake up and let join
_cond.notify_one();
}
// join thread,
if (_thread->joinable())
{
_thread->join();
}
}
void PersistentThread::set(const std::function<void()>& f)
{
std::lock_guard<std::mutex> lock(_mutex);
this->_func = f;
}
void PersistentThread::invoke()
{
std::lock_guard<std::mutex> lock(_mutex);
_cond.notify_one();
}
void PersistentThread::join()
{
bool joined = false;
while (!joined)
{
std::lock_guard<std::mutex> lock(_mutex);
joined = (_invokeCounter == 0);
}
}
void PersistentThread::loop(bool *initialized)
{
std::unique_lock<std::mutex> lock(_mutex);
*initialized = true;
_cond.notify_one();
while (true)
{
// wait until we get the mutex again
_cond.wait(lock, [this] {return _shutdownRequested || (this->_invokeCounter > 0); });
// shut down if requested
if (_shutdownRequested) return;
// process
if (_func) _func();
_invokeCounter--;
}
}
You are asking about potential race conditions, and I see at least one race condition in the shown code.
After constructing a PersistentThread, there is no guarantee that the new thread will acquire its initial lock in its loop() before the main execution thread returns from the constructor and enters invoke(). It is possible that the main execution thread enters invoke() immediately after the constructor is complete, ends up notifying nobody, since the internal execution thread hasn't locked the mutex yet. As such, this invoke() will not result in any processing taking place.
You need to synchronize the completion of the constructor with the execution thread's initial lock acquisition.
EDIT: your revision looks right; but I also spotted another race condition.
As documented in the description of wait(), wait() may wake up "spuriously". Just because wait() returned, doesn't mean that some other thread has entered invoke().
You need a counter, in addition to everything else, with invoke() incrementing the counter, and the execution thread executing its assigned duties only when the counter is greater than zero, decrementing it. This will guard against spurious wake-ups.
I would also have the execution thread check the counter before entering wait(), and enter wait() only if it is 0. Otherwise, it decrements the counter, executes its function, and loops back.
This should plug up all the potential race conditions in this area.
P.S. The spurious wake-up also applies to the initial notification, in your correction, that the execution thread has entered the loop. You'll need to do something similar for that situation, too.
I don't understand what you're trying to ask exactly. It's a nice style you used.
It would be much safer using bools and check the single routines because void returns nothing so you could be maybe stuck caused by bugs. Check everything you can since the thread runs under the hood. Make sure the calls are running correctly, if the process had really success. Also you could read some stuff about "Thread Pooling".
In a distributed job system written in C++11 I have implemented a fence (i.e. a thread outside the worker thread pool may ask to block until all currently scheduled jobs are done) using the following structure:
struct fence
{
std::atomic<size_t> counter;
std::mutex resume_mutex;
std::condition_variable resume;
fence(size_t num_threads)
: counter(num_threads)
{}
};
The code implementing the fence looks like this:
void task_pool::fence_impl(void *arg)
{
auto f = (fence *)arg;
if (--f->counter == 0) // (1)
// we have zeroed this fence's counter, wake up everyone that waits
f->resume.notify_all(); // (2)
else
{
unique_lock<mutex> lock(f->resume_mutex);
f->resume.wait(lock); // (3)
}
}
This works very well if threads enter the fence over a period of time. However, if they try to do it almost simultaneously, it seems to sometimes happen that between the atomic decrementation (1) and starting the wait on the conditional var (3), the thread yields CPU time and another thread decrements the counter to zero (1) and fires the cond. var (2). This results in the previous thread waiting forever in (3), because it starts waiting on it after it has already been notified.
A hack to make the thing workable is to put a 10 ms sleep just before (2), but that's unacceptable for obvious reasons.
Any suggestions on how to fix this in a performant way?
Your diagnose is correct, this code is prone to lose condition notifications in the way you described. I.e. after one thread locked the mutex but before waiting on the condition variable another thread may call notify_all() so that the first thread misses that notification.
A simple fix is to lock the mutex before decrementing the counter and while notifying:
void task_pool::fence_impl(void *arg)
{
auto f = static_cast<fence*>(arg);
std::unique_lock<std::mutex> lock(f->resume_mutex);
if (--f->counter == 0) {
f->resume.notify_all();
}
else do {
f->resume.wait(lock);
} while(f->counter);
}
In this case the counter need not be atomic.
An added bonus (or penalty, depending on the point of view) of locking the mutex before notifying is (from here):
The pthread_cond_broadcast() or pthread_cond_signal() functions may be called by a thread whether or not it currently owns the mutex that threads calling pthread_cond_wait() or pthread_cond_timedwait() have associated with the condition variable during their waits; however, if predictable scheduling behavior is required, then that mutex shall be locked by the thread calling pthread_cond_broadcast() or pthread_cond_signal().
Regarding the while loop (from here):
Spurious wakeups from the pthread_cond_timedwait() or pthread_cond_wait() functions may occur. Since the return from pthread_cond_timedwait() or pthread_cond_wait() does not imply anything about the value of this predicate, the predicate should be re-evaluated upon such return.
In order to keep the higher performance of an atomic operation instead of a full mutex, you should change the wait condition into a lock, check and loop.
All condition waits should be done in that way. The condition variable even has a 2nd argument to wait which is a predicate function or lambda.
The code might look like:
void task_pool::fence_impl(void *arg)
{
auto f = (fence *)arg;
if (--f->counter == 0) // (1)
// we have zeroed this fence's counter, wake up everyone that waits
f->resume.notify_all(); // (2)
else
{
unique_lock<mutex> lock(f->resume_mutex);
while(f->counter) {
f->resume.wait(lock); // (3)
}
}
}
I am a bit confused about the use of std::condition_variable. I understand I have to create a unique_lock on a mutex before calling condition_variable.wait(). What I cannot find is whether I should also acquire a unique lock before calling notify_one() or notify_all().
Examples on cppreference.com are conflicting. For example, the notify_one page gives this example:
#include <iostream>
#include <condition_variable>
#include <thread>
#include <chrono>
std::condition_variable cv;
std::mutex cv_m;
int i = 0;
bool done = false;
void waits()
{
std::unique_lock<std::mutex> lk(cv_m);
std::cout << "Waiting... \n";
cv.wait(lk, []{return i == 1;});
std::cout << "...finished waiting. i == 1\n";
done = true;
}
void signals()
{
std::this_thread::sleep_for(std::chrono::seconds(1));
std::cout << "Notifying...\n";
cv.notify_one();
std::unique_lock<std::mutex> lk(cv_m);
i = 1;
while (!done) {
lk.unlock();
std::this_thread::sleep_for(std::chrono::seconds(1));
lk.lock();
std::cerr << "Notifying again...\n";
cv.notify_one();
}
}
int main()
{
std::thread t1(waits), t2(signals);
t1.join(); t2.join();
}
Here the lock is not acquired for the first notify_one(), but is acquired for the second notify_one(). Looking though other pages with examples I see different things, mostly not acquiring the lock.
Can I choose myself to lock the mutex before calling notify_one(), and why would I choose to lock it?
In the example given, why is there no lock for the first notify_one(), but there is for subsequent calls. Is this example wrong or is there some rationale?
You do not need to be holding a lock when calling condition_variable::notify_one(), but it's not wrong in the sense that it's still well defined behavior and not an error.
However, it might be a "pessimization" since whatever waiting thread is made runnable (if any) will immediately try to acquire the lock that the notifying thread holds. I think it's a good rule of thumb to avoid holding the lock associated with a condition variable while calling notify_one() or notify_all(). See Pthread Mutex: pthread_mutex_unlock() consumes lots of time for an example where releasing a lock before calling the pthread equivalent of notify_one() improved performance measurably.
Keep in mind that the lock() call in the while loop is necessary at some point, because the lock needs to be held during the while (!done) loop condition check. But it doesn't need to be held for the call to notify_one().
2016-02-27: Large update to address some questions in the comments about whether there's a race condition if the lock isn't held for the notify_one() call. I know this update is late because the question was asked almost two years ago, but I'd like to address #Cookie's question about a possible race condition if the producer (signals() in this example) calls notify_one() just before the consumer (waits() in this example) is able to call wait().
The key is what happens to i - that's the object that actually indicates whether or not the consumer has "work" to do. The condition_variable is just a mechanism to let the consumer efficiently wait for a change to i.
The producer needs to hold the lock when updating i, and the consumer must hold the lock while checking i and calling condition_variable::wait() (if it needs to wait at all). In this case, the key is that it must be the same instance of holding the lock (often called a critical section) when the consumer does this check-and-wait. Since the critical section is held when the producer updates i and when the consumer checks-and-waits on i, there is no opportunity for i to change between when the consumer checks i and when it calls condition_variable::wait(). This is the crux for a proper use of condition variables.
The C++ standard says that condition_variable::wait() behaves like the following when called with a predicate (as in this case):
while (!pred())
wait(lock);
There are two situations that can occur when the consumer checks i:
if i is 0 then the consumer calls cv.wait(), then i will still be 0 when the wait(lock) part of the implementation is called - the proper use of the locks ensures that. In this case the producer has no opportunity to call the condition_variable::notify_one() in its while loop until after the consumer has called cv.wait(lk, []{return i == 1;}) (and the wait() call has done everything it needs to do to properly 'catch' a notify - wait() won't release the lock until it has done that). So in this case, the consumer cannot miss the notification.
if i is already 1 when the consumer calls cv.wait(), the wait(lock) part of the implementation will never be called because the while (!pred()) test will cause the internal loop to terminate. In this situation it doesn't matter when the call to notify_one() occurs - the consumer will not block.
The example here does have the additional complexity of using the done variable to signal back to the producer thread that the consumer has recognized that i == 1, but I don't think this changes the analysis at all because all of the access to done (for both reading and modifying) are done while in the same critical sections that involve i and the condition_variable.
If you look at the question that #eh9 pointed to, Sync is unreliable using std::atomic and std::condition_variable, you will see a race condition. However, the code posted in that question violates one of the fundamental rules of using a condition variable: It does not hold a single critical section when performing a check-and-wait.
In that example, the code looks like:
if (--f->counter == 0) // (1)
// we have zeroed this fence's counter, wake up everyone that waits
f->resume.notify_all(); // (2)
else
{
unique_lock<mutex> lock(f->resume_mutex);
f->resume.wait(lock); // (3)
}
You will notice that the wait() at #3 is performed while holding f->resume_mutex. But the check for whether or not the wait() is necessary at step #1 is not done while holding that lock at all (much less continuously for the check-and-wait), which is a requirement for proper use of condition variables). I believe that the person who has the problem with that code snippet thought that since f->counter was a std::atomic type this would fulfill the requirement. However, the atomicity provided by std::atomic doesn't extend to the subsequent call to f->resume.wait(lock). In this example, there is a race between when f->counter is checked (step #1) and when the wait() is called (step #3).
That race does not exist in this question's example.
As others have pointed out, you do not need to be holding the lock when calling notify_one(), in terms of race conditions and threading-related issues. However, in some cases, holding the lock may be required to prevent the condition_variable from getting destroyed before notify_one() is called. Consider the following example:
thread t;
void foo() {
std::mutex m;
std::condition_variable cv;
bool done = false;
t = std::thread([&]() {
{
std::lock_guard<std::mutex> l(m); // (1)
done = true; // (2)
} // (3)
cv.notify_one(); // (4)
}); // (5)
std::unique_lock<std::mutex> lock(m); // (6)
cv.wait(lock, [&done]() { return done; }); // (7)
}
void main() {
foo(); // (8)
t.join(); // (9)
}
Assume there is a context switch to the newly created thread t after we created it but before we start waiting on the condition variable (somewhere between (5) and (6)). The thread t acquires the lock (1), sets the predicate variable (2) and then releases the lock (3). Assume there is another context switch right at this point before notify_one() (4) is executed. The main thread acquires the lock (6) and executes line (7), at which point the predicate returns true and there is no reason to wait, so it releases the lock and continues. foo returns (8) and the variables in its scope (including cv) are destroyed. Before thread t could join the main thread (9), it has to finish its execution, so it continues from where it left off to execute cv.notify_one() (4), at which point cv is already destroyed!
The possible fix in this case is to keep holding the lock when calling notify_one (i.e. remove the scope ending in line (3)). By doing so, we ensure that thread t calls notify_one before cv.wait can check the newly set predicate variable and continue, since it would need to acquire the lock, which t is currently holding, to do the check. So, we ensure that cv is not accessed by thread t after foo returns.
To summarize, the problem in this specific case is not really about threading, but about the lifetimes of the variables captured by reference. cv is captured by reference via thread t, hence you have to make sure cv stays alive for the duration of the thread's execution. The other examples presented here do not suffer from this issue, because condition_variable and mutex objects are defined in the global scope, hence they are guaranteed to be kept alive until the program exits.
Situation
Using vc10 and Boost 1.56 I implemented a concurrent queue pretty much like this blog post suggests. The author unlocks the mutex to minimize contention, i.e., notify_one() is called with the mutex unlocked:
void push(const T& item)
{
std::unique_lock<std::mutex> mlock(mutex_);
queue_.push(item);
mlock.unlock(); // unlock before notificiation to minimize mutex contention
cond_.notify_one(); // notify one waiting thread
}
Unlocking the mutex is backed by an example in the Boost documentation:
void prepare_data_for_processing()
{
retrieve_data();
prepare_data();
{
boost::lock_guard<boost::mutex> lock(mut);
data_ready=true;
}
cond.notify_one();
}
Problem
Still this led to the following erratic behaviour:
while notify_one() has not been called yet cond_.wait() can still be interrupted via boost::thread::interrupt()
once notify_one() was called for the first time cond_.wait() deadlocks; the wait cannot be ended by boost::thread::interrupt() or boost::condition_variable::notify_*() anymore.
Solution
Removing the line mlock.unlock() made the code work as expected (notifications and interrupts end the wait). Note that notify_one() is called with the mutex still locked, it is unlocked right afterwards when leaving the scope:
void push(const T& item)
{
std::lock_guard<std::mutex> mlock(mutex_);
queue_.push(item);
cond_.notify_one(); // notify one waiting thread
}
That means that at least with my particular thread implementation the mutex must not be unlocked before calling boost::condition_variable::notify_one(), although both ways seem correct.
#Michael Burr is correct. condition_variable::notify_one does not require a lock on the variable. Nothing prevents you to use a lock in that situation though, as the example illustrates it.
In the given example, the lock is motivated by the concurrent use of the variable i. Because the signals thread modifies the variable, it needs to ensure that no other thread is access it during that time.
Locks are used for any situation requiring synchronization, I don't think we can state it in a more general way.
In some case, when the cv may be occupied(locked) by other threads. You needs to get lock and release it before notify_*().
If not, the notify_*() maybe not executed at all.
Just adding this answer because I think the accepted answer might be misleading. In all cases you will need to lock the mutex, prior to calling notify_one() somewhere for your code to be thread-safe, although you might unlock it again before actually calling notify_*().
To clarify, you MUST take the lock before entering wait(lk) because wait() unlocks lk and it would be Undefined Behavior if the lock wasn't locked. This is not the case with notify_one(), but you need to make sure you won't call notify_*() before entering wait() and having that call unlock the mutex; which obviously only can be done by locking that same mutex before you call notify_*().
For example, consider the following case:
std::atomic_int count;
std::mutex cancel_mutex;
std::condition_variable cancel_cv;
void stop()
{
if (count.fetch_sub(1) == -999) // Reached -1000 ?
cv.notify_one();
}
bool start()
{
if (count.fetch_add(1) >= 0)
return true;
// Failure.
stop();
return false;
}
void cancel()
{
if (count.fetch_sub(1000) == 0) // Reached -1000?
return;
// Wait till count reached -1000.
std::unique_lock<std::mutex> lk(cancel_mutex);
cancel_cv.wait(lk);
}
Warning: this code contains a bug.
The idea is the following: threads call start() and stop() in pairs, but only as long as start() returned true. For example:
if (start())
{
// Do stuff
stop();
}
One (other) thread at some point will call cancel() and after returning from cancel() will destroy objects that are needed at 'Do stuff'. However, cancel() is supposed not to return while there are threads between start() and stop(), and once cancel() executed its first line, start() will always return false, so no new threads will enter the 'Do stuff' area.
Works right?
The reasoning is as follows:
1) If any thread successfully executes the first line of start() (and therefore will return true) then no thread did execute the first line of cancel() yet (we assume that the total number of threads is much smaller than 1000 by the way).
2) Also, while a thread successfully executed the first line of start(), but not yet the first line of stop() then it is impossible that any thread will successfully execute the first line of cancel() (note that only one thread ever calls cancel()): the value returned by fetch_sub(1000) will be larger than 0.
3) Once a thread executed the first line of cancel(), the first line of start() will always return false and a thread calling start() will not enter the 'Do stuff' area anymore.
4) The number of calls to start() and stop() are always balanced, so after the first line of cancel() is unsuccessfully executed, there will always be a moment where a (the last) call to stop() causes count to reach -1000 and therefore notify_one() to be called. Note that can only ever happen when the first line of cancel resulted in that thread to fall through.
Apart from a starvation problem where so many threads are calling start()/stop() that count never reaches -1000 and cancel() never returns, which one might accept as "unlikely and never lasting long", there is another bug:
It is possible that there is one thread inside the 'Do stuff' area, lets say it is just calling stop(); at that moment a thread executes the first line of cancel() reading the value 1 with the fetch_sub(1000) and falling through. But before it takes the mutex and/or does the call to wait(lk), the first thread executes the first line of stop(), reads -999 and calls cv.notify_one()!
Then this call to notify_one() is done BEFORE we are wait()-ing on the condition variable! And the program would indefinitely dead-lock.
For this reason we should not be able to call notify_one() until we called wait(). Note that the power of a condition variable lies there in that it is able to atomically unlock the mutex, check if a call to notify_one() happened and go to sleep or not. You can't fool it, but you do need to keep the mutex locked whenever you make changes to variables that might change the condition from false to true and keep it locked while calling notify_one() because of race conditions like described here.
In this example there is no condition however. Why didn't I use as condition 'count == -1000'? Because that isn't interesting at all here: as soon as -1000 is reached at all, we are sure that no new thread will enter the 'Do stuff' area. Moreover, threads can still call start() and will increment count (to -999 and -998 etc) but we don't care about that. The only thing that matters is that -1000 was reached - so that we know for sure that there are no threads anymore in the 'Do stuff' area. We are sure that this is the case when notify_one() is being called, but how to make sure we don't call notify_one() before cancel() locked its mutex? Just locking cancel_mutex shortly prior to notify_one() isn't going to help of course.
The problem is that, despite that we're not waiting for a condition, there still is a condition, and we need to lock the mutex
1) before that condition is reached
2) before we call notify_one.
The correct code therefore becomes:
void stop()
{
if (count.fetch_sub(1) == -999) // Reached -1000 ?
{
cancel_mutex.lock();
cancel_mutex.unlock();
cv.notify_one();
}
}
[...same start()...]
void cancel()
{
std::unique_lock<std::mutex> lk(cancel_mutex);
if (count.fetch_sub(1000) == 0)
return;
cancel_cv.wait(lk);
}
Of course this is just one example but other cases are very much alike; in almost all cases where you use a conditional variable you will need to have that mutex locked (shortly) before calling notify_one(), or else it is possible that you call it before calling wait().
Note that I unlocked the mutex prior to calling notify_one() in this case, because otherwise there is the (small) chance that the call to notify_one() wakes up the thread waiting for the condition variable which then will try to take the mutex and block, before we release the mutex again. That's just slightly slower than needed.
This example was kinda special in that the line that changes the condition is executed by the same thread that calls wait().
More usual is the case where one thread simply wait's for a condition to become true and another thread takes the lock before changing the variables involved in that condition (causing it to possibly become true). In that case the mutex is locked immediately before (and after) the condition became true - so it is totally ok to just unlock the mutex before calling notify_*() in that case.
As I understand notify_one calls pthread_cond_signal.
If so, then what do think about this?
For predictable scheduling behavior and to prevent lost wake-ups, the mutex should be held when signaling a condition variable.
https://www.unix.com/man-page/hpux/3T/pthread_cond_signal/
All of the threads waiting on the condition variable are suspended until another thread uses the signal function:
pthread_cond_signal(&myConVar);
In this case the mutex has to be locked before calling the function and unlocked after it.
https://www.i-programmer.info/programming/cc/12288-fundamental-c-condition-variables.html
I personally had cases when notifications were missed because notify_one was called without locking the mutex.