I have some code like this:
std::queue<myData*> _qDatas;
std::mutex _qDatasMtx;
Generator function
void generator_thread()
{
std::this_thread::sleep_for(std::chrono::milliseconds(1));
{
std::lock_guard<std::mutex> lck(_qDatasMtx);
_qData.push(new myData);
}
//std::lock_guard<std::mutex> lck(cvMtx); //need lock here?
cv.notify_one();
}
Consumer function
void consumer_thread()
{
for(;;)
{
std::unique_lock lck(_qDatasMtx);
if(_qDatas.size() > 0)
{
delete _qDatas.front();
_qDatas.pop();
}
else
cv.wait(lck);
}
}
So if I have dozens of generator threads and one consumer thread, do I need a mutex lock when calling cv.notify_one() in each thread?
And is std::condition_variable thread-safe?
do I need a mutex lock when calling cv.notify_one() in each thread?
No
is std::condition_variable thread-safe?
Yes
When calling wait, you pass a locked mutex which is immediately unlocked for use. When calling notify you don't lock it with the same mutex, since what will happen is (detailed on links):
Notify
Wakeup thread that is waiting
Lock mutex for use
From std::condition_variable:
execute notify_one or notify_all on the std::condition_variable (the lock does not need to be held for notification)
and from std::condition_variable::notify_all:
The notifying thread does not need to hold the lock on the same mutex as the one held by the waiting thread(s);
With regards to your code snippet:
//std::lock_guard<std::mutex> lck(cvMtx); //need lock here?
cv.notify_one();
No you do not need a lock there.
notify_one can be called from multiple threads without a lock.
However, in order for normal operation to work and no chance of notifications "slipping through", you must hold the lock the condition variable is guarded by between the point where you modify the spurious wakeup guard and/or package the condition variable checks on read, and the point where you notify one.
Your code appears to pass that test. However this version is more clear:
std::unique_lock lck(_qDatasMtx);
for(;;) {
cv.wait(lck,[&]{ return _qDatas.size()>0; });
delete _qDatas.front();
_qDatas.pop();
}
and it reduces spurious unlocking/relocking.
Related
For simplicity, let's assume that we have only one conditional variable to match a single condition that is reflected by a boolean.
1) Why does std::condition_variable::wait(...) locks the mutex again after a "notify" has been sent to un-sleep it?
2) Seeing the behaviour in "1)", does that mean that when you do std::condition_variable::notify_all it only makes it so that all of the waiting threads are unblocked/woken up... but in order instead of all at once? If so, what can be done to do it all at once?
3) If I only care about threads sleeping until a condition is met and not care a single bit for any mutex acquisition, what can I do? Is there an alternative or should current std::condition_variable::wait(...) approach(es) be hacked around this?
If "hackery" is to be used, will this function work for unblocking all waiting threads on a condition and can it be called from any(per thread) threads:
//declared somehwere and modified before sending "notify"(ies)
std::atomic<bool> global_shared_condition_atomic_bool;
//the single(for simplicity in our case) condition variable matched with the above boolean result
std::condition_variable global_shared_condition_variable;
static void MyClass:wait()
{
std::mutex mutex;
std::unique_lock<std::mutex> lock(mutex);
while (!global_shared_condition_atomic_bool) global_shared_condition_variable.wait(lock);
}
it would have been called from random "waiting" threads like so:
void random_thread_run()
{
while(someLoopControlValue)
{
//random code...
MyClass:wait(); //wait for whatever condition the class+method is for.
//more random code...
}
}
Edit:
Gate class
#ifndef Gate_Header
#define Gate_Header
#include <mutex>
#include <condition_variable>
class Gate
{
public:
Gate()
{
gate_open = false;
}
void open()
{
m.lock();
gate_open = true;
m.unlock();
cv.notify_all();
}
void wait()
{
std::unique_lock<std::mutex> lock(m);
while (!gate_open) cv.wait(lock);
}
void close()
{
m.lock();
gate_open = false;
m.unlock();
}
private:
std::mutex m;
std::condition_variable cv;
bool gate_open;
};
#endif
Condition variables wake things up spuriously.
You must have a mutex and it must guard a message of some kind for them to work, or you have zero guarantee that any such wakeup occurred.
This was done, presumably, because efficient implementations of a non-spurious version end up being implemeneted in terms of such a spurious version anyhow.
If you fail to guard the message editing with a mutex (ie, no synchronization on it, the state of the message is undefined behavior. This can cause compilers to optimize the read from memory to skip it after the first read.
Even excluding that undefined behavior (imagine you use atomics), there are race conditions where a message is set, a notification occurs, and nobody waiting on the notification sees the message being set if you fail to have the mutex acquired in the time between the variable being set and the condition variable being notified.
Barring extreme cases, you usually want to use the lambda version of wait.
Auditing condition variable code is not possible unless you audit both the notification code and the wait code.
struct gate {
bool gate_open = false;
mutable std::condition_variable cv;
mutable std::mutex m;
void open_gate() {
std::unique_lock<std::mutex> lock(m);
gate_open=true;
cv.notify_all();
}
void wait_at_gate() const {
std::unique_lock<std::mutex> lock(m);
cv.wait( lock, [this]{ return gate_open; } );
}
};
or
void open_gate() {
{
std::unique_lock<std::mutex> lock(m);
gate_open=true;
}
cv.notify_all();
}
No, your code will not work.
The mutex protects modifications to the shared variable. As such, all of the waiting threads and the signaling thread must lock that specific mutex instance. With what you've written, each thread has its own mutex instance.
The main reason for all of this mutex stuff is due to the concept of spurious wakeup, an unfortunate aspect of OS implementations of condition variables. Threads waiting on them sometimes just start running even though the condition hasn't been satisfied yet.
The mutex-bound check of the actual variable allows the thread to test whether it was spuriously awoken or not.
wait atomically releases the mutex and starts waiting on the condition. When wait exits, the mutex is atomically reacquired as part of the wakeup process. Now, consider a race between a spurious wakeup and the notifying thread. The notifying thread can be in one of 2 states: about to modify the variable, or after modifying it and about to notify everyone to wake up.
If the spurious wakeup happens when the notifying thread is about to modify the varaible, then one of them will get to the mutex first. So the spuriously awoken thread will either see the old value or the new value. If it sees the new, then it has been notified and will go do its business. If it sees the old, then it will wait on the condition again. But if it saw the old, then it blocked the notifying thread from modifying that variable, so it had to wait until the spurious thread went back to sleep.
Why does std::condition_variable::wait(...) locks the mutex again after a "notify" has been sent to un-sleep it?
Because the mutex locks access to the condition variable. And the first thing you have to do after waking up from a wait call is to check the condition variable. As such, that must be done under the protection of the mutex.
The signalling thread must be prevented from modifying the variable while other threads are reading it. That's what the mutex is for.
Seeing the behaviour in "1)", does that mean that when you do std::condition_variable::notify_all it only makes it so that all of the waiting threads are unblocked/woken up... but in order instead of all at once?
The order they wake up in is not specified. However, by the time notify_all returns, all threads are guaranteed to have been unblocked.
If I only care about threads sleeping until a condition is met and not care a single bit for any mutex acquisition, what can I do?
Nothing. condition_variable requires that access to the actual variable you're checking is controlled via a mutex.
I started using std::mutexes to stop a thread and wait for another thread to resume it. It works like this:
Thread 1
// Ensures the mutex will be locked
while(myWaitMutex.try_lock());
// Locks it again to pause this thread
myWaitMutex.lock();
Thread 2
// Executed when thread 1 should resume processing:
myWaitMutex.unlock();
However I am not sure if this is correct and will work without problems on all platforms. If this is not correct, what is the correct way to implement this in C++11?
The problems with the code
// Ensures the mutex will be locked
while(myWaitMutex.try_lock());
.try_lock() tries to aquire the lock and returns true if successful, i.e., the code says "if we aquire the lock then retry to lock it again and again until we fail". We can never "fail" as we currently own the lock ourselves that we are waiting on, and so this will be an infinite loop. Also, attempting to lock using a std::mutex that the caller have already aquired a lock on is UB, so this is guaranteed to be UB. If not successful, .try_lock() will return false and the while loop will be exited. In other words, this will not ensure that the mutex will be locked.
The correct way to ensure the mutex will be locked is simply:
myWaitMutex.lock();
This will cause the current thread to block (indefinitely) until it can aquire the lock.
Next, the other thread tries to unlock a mutex it does not have a lock on.
// Executed when thread 1 should resume processing:
myWaitMutex.unlock();
This won't work as it's UB to .unlock() on a std::mutex that you don't already have a lock on.
Using locks
When using mutex locks, it's easier to use a RAII ownership-wrapper object such as std::lock_guard. The usage pattern of std::mutex is always: "Lock -> do something in critical section -> unlock". A std::lock_guard will lock the mutex in its constructor, and unlock it in its destructor. No need to worry about when to lock and unlock and such low-level stuff.
std::mutex m;
{
std::lock_guard<std::mutex> lk{m};
/* We have the lock until we exit scope. */
} // Here 'lk' is destroyed and will release lock.
A simple lock might not be the best tool for the job
If what you want is to be able to signal a thread to wake up, then there's the wait and notify structure using std::condition_variable. The std::condition_variable allows any caller to send a signal to waiting threads without holding any locks.
#include <atomic>
#include <condition_variable>
#include <iostream>
#include <mutex>
#include <thread>
using namespace std::literals;
int main() {
std::mutex m;
std::condition_variable cond;
std::thread t{[&] {
std::cout << "Entering sleep..." << std::endl;
std::unique_lock<std::mutex> lk{m};
cond.wait(lk); // Will block until 'cond' is notified.
std::cout << "Thread is awake!" << std::endl;
}};
std::this_thread::sleep_for(3s);
cond.notify_all(); // Notify all waiting threads.
t.join(); // Remember to join thread before exit.
}
However, to further complicate things there's this thing called spurious wakeups that mean that any waiting threads may wake up at any time for unknown reasons. This is a fact on most systems and has to do with the inner workings of thread scheduling. Also, we probably need to check that waiting is really needed as we're dealing with concurrency. If, for example, the notifying thread happens to notify before we start waiting, then we might wait forever unless we have a way to first check this.
To handle this we need to add a while loop and a predicate that tells when we need to wait and when we're done waiting.
int main() {
std::mutex m;
std::condition_variable cond;
bool done = false; // Flag for indicating when done waiting.
std::thread t{[&] {
std::cout << "Entering sleep..." << std::endl;
std::unique_lock<std::mutex> lk{m};
while (!done) { // Wait inside loop to handle spurious wakeups etc.
cond.wait(lk);
}
std::cout << "Thread is awake!" << std::endl;
}};
std::this_thread::sleep_for(3s);
{ // Aquire lock to avoid data race on 'done'.
std::lock_guard<std::mutex> lk{m};
done = true; // Set 'done' to true before notifying.
}
cond.notify_all();
t.join();
}
There are additional reasons why it's a good idea to wait inside a loop and use a predicate such as "stolen wakeups" as mentioned in the comments by #David Schwartz.
It sounds to me that you are looking for condition variable. In the end there should always be a way to make it work through mutexes, but condition variable is the current C++ idiomatic way to handle the `block and wait until something happens' scenario.
The behavior of a mutex when a thread that holds it attempts to lock it is undefined. The behavior of a mutex when a thread that doesn't hold it attempts to unlock it is undefined. So your code might do anything at all on various platforms.
Instead, use a mutex together with a condition variable and a predicate boolean. In pseudo-code:
To block:
Acquire the mutex.
While the predicate is false, block on the condition variable.
If you want to re-arm here, set the predicate to false.
Release the mutex.
To release:
Acquire the mutex.
Set the predicate to true.
Signal the condition variable.
Release the mutex.
To rearm:
Acquire the mutex.
Set the predicate to false.
Release the mutex.
Please check this code....
std::mutex m_mutex;
std::condition_variable m_cond_var;
void threadOne(){
std::unique_lock<std::mutex> lck(mtx);
while (!ready){
m_cond_var.wait(lck);
}
m_cond_var.notify_all();
}
void threadTwo(){
std::unique_lock<std::mutex> lck(mtx);
read = true;
m_cond_var.notify_all();
}
I hope you will get the solution. And it is very proper!!
For simplicity, let's assume that we have only one conditional variable to match a single condition that is reflected by a boolean.
1) Why does std::condition_variable::wait(...) locks the mutex again after a "notify" has been sent to un-sleep it?
2) Seeing the behaviour in "1)", does that mean that when you do std::condition_variable::notify_all it only makes it so that all of the waiting threads are unblocked/woken up... but in order instead of all at once? If so, what can be done to do it all at once?
3) If I only care about threads sleeping until a condition is met and not care a single bit for any mutex acquisition, what can I do? Is there an alternative or should current std::condition_variable::wait(...) approach(es) be hacked around this?
If "hackery" is to be used, will this function work for unblocking all waiting threads on a condition and can it be called from any(per thread) threads:
//declared somehwere and modified before sending "notify"(ies)
std::atomic<bool> global_shared_condition_atomic_bool;
//the single(for simplicity in our case) condition variable matched with the above boolean result
std::condition_variable global_shared_condition_variable;
static void MyClass:wait()
{
std::mutex mutex;
std::unique_lock<std::mutex> lock(mutex);
while (!global_shared_condition_atomic_bool) global_shared_condition_variable.wait(lock);
}
it would have been called from random "waiting" threads like so:
void random_thread_run()
{
while(someLoopControlValue)
{
//random code...
MyClass:wait(); //wait for whatever condition the class+method is for.
//more random code...
}
}
Edit:
Gate class
#ifndef Gate_Header
#define Gate_Header
#include <mutex>
#include <condition_variable>
class Gate
{
public:
Gate()
{
gate_open = false;
}
void open()
{
m.lock();
gate_open = true;
m.unlock();
cv.notify_all();
}
void wait()
{
std::unique_lock<std::mutex> lock(m);
while (!gate_open) cv.wait(lock);
}
void close()
{
m.lock();
gate_open = false;
m.unlock();
}
private:
std::mutex m;
std::condition_variable cv;
bool gate_open;
};
#endif
Condition variables wake things up spuriously.
You must have a mutex and it must guard a message of some kind for them to work, or you have zero guarantee that any such wakeup occurred.
This was done, presumably, because efficient implementations of a non-spurious version end up being implemeneted in terms of such a spurious version anyhow.
If you fail to guard the message editing with a mutex (ie, no synchronization on it, the state of the message is undefined behavior. This can cause compilers to optimize the read from memory to skip it after the first read.
Even excluding that undefined behavior (imagine you use atomics), there are race conditions where a message is set, a notification occurs, and nobody waiting on the notification sees the message being set if you fail to have the mutex acquired in the time between the variable being set and the condition variable being notified.
Barring extreme cases, you usually want to use the lambda version of wait.
Auditing condition variable code is not possible unless you audit both the notification code and the wait code.
struct gate {
bool gate_open = false;
mutable std::condition_variable cv;
mutable std::mutex m;
void open_gate() {
std::unique_lock<std::mutex> lock(m);
gate_open=true;
cv.notify_all();
}
void wait_at_gate() const {
std::unique_lock<std::mutex> lock(m);
cv.wait( lock, [this]{ return gate_open; } );
}
};
or
void open_gate() {
{
std::unique_lock<std::mutex> lock(m);
gate_open=true;
}
cv.notify_all();
}
No, your code will not work.
The mutex protects modifications to the shared variable. As such, all of the waiting threads and the signaling thread must lock that specific mutex instance. With what you've written, each thread has its own mutex instance.
The main reason for all of this mutex stuff is due to the concept of spurious wakeup, an unfortunate aspect of OS implementations of condition variables. Threads waiting on them sometimes just start running even though the condition hasn't been satisfied yet.
The mutex-bound check of the actual variable allows the thread to test whether it was spuriously awoken or not.
wait atomically releases the mutex and starts waiting on the condition. When wait exits, the mutex is atomically reacquired as part of the wakeup process. Now, consider a race between a spurious wakeup and the notifying thread. The notifying thread can be in one of 2 states: about to modify the variable, or after modifying it and about to notify everyone to wake up.
If the spurious wakeup happens when the notifying thread is about to modify the varaible, then one of them will get to the mutex first. So the spuriously awoken thread will either see the old value or the new value. If it sees the new, then it has been notified and will go do its business. If it sees the old, then it will wait on the condition again. But if it saw the old, then it blocked the notifying thread from modifying that variable, so it had to wait until the spurious thread went back to sleep.
Why does std::condition_variable::wait(...) locks the mutex again after a "notify" has been sent to un-sleep it?
Because the mutex locks access to the condition variable. And the first thing you have to do after waking up from a wait call is to check the condition variable. As such, that must be done under the protection of the mutex.
The signalling thread must be prevented from modifying the variable while other threads are reading it. That's what the mutex is for.
Seeing the behaviour in "1)", does that mean that when you do std::condition_variable::notify_all it only makes it so that all of the waiting threads are unblocked/woken up... but in order instead of all at once?
The order they wake up in is not specified. However, by the time notify_all returns, all threads are guaranteed to have been unblocked.
If I only care about threads sleeping until a condition is met and not care a single bit for any mutex acquisition, what can I do?
Nothing. condition_variable requires that access to the actual variable you're checking is controlled via a mutex.
I have following scenario:
condition_variable cv;
mutex mut;
// Thread 1:
void run() {
while (true) {
mut.lock();
// create_some_data();
mut.unlock();
cv.notify_all();
}
}
// Thread 2
void thread2() {
mutex lockMutex;
unique_lock<mutex> lock(lockMutex);
while (running) {
cv.wait(lock);
mut.lock();
// copy data
mut.unlock();
// process data
}
}
// Thread 3, 4... - same as Thread 2
I run thread 1 all the time to get new data. Other threads wait with condition_variable until new data is available, then copy it and do some work on it. Work perfomed by threads differs in time needed to finish, the idea is that threads will get new data only when they finished with the old one. Data got in meantime is allowed to be "missed". I don't use shared mutex (only to access data) because I don't want threads to depend on each other.
Above code works fine on Windows, but now I run it on Ubuntu and I noticed that only one thread is being notified when notify_all() is called and the other ones just hangs on wait().
Why is that? Does Linux require different approach for using condition_variable?
Your code exhibits UB immediately as it relocks the unique lock that the cv has relocked when it exits wait.
There are other problems, like not detecting spurious wakeups.
Finally cv notify all onky notified currently waiting threads. If a thread shows up later, no dice.
It's working by luck.
The mutex and the condition variable are two parts of the same construct. You can't mix and match mutexes and cvs.
try this:
void thread2() {
unique_lock<mutex> lock(mut); // use the global mutex
while (running) {
cv.wait(lock);
// mutex is already locked here
// test condition. wakeups can be spurious
// copy data
lock.unlock();
// process data
lock.lock();
}
}
Per this documentation:
Any thread that intends to wait on std::condition_variable has to
acquire a std::unique_lock, on the same mutex as used to
protect the shared variable
execute wait, wait_for, or wait_until. The wait operations atomically release the mutex and suspend the execution of the
thread.
When the condition variable is notified, a timeout expires, or a spurious wakeup occurs, the thread is awakened, and the mutex is
atomically reacquired. The thread should then check the condition
and resume waiting if the wake up was spurious.
This code
void thread2() {
mutex lockMutex;
unique_lock<mutex> lock(lockMutex);
while (running) {
doesn't do that.
boost::condition_variable cond;
boost::mutex mut;
//thread1
{
"read_socket()"
cond.notify_one();
}
//thread2
{
for(;;)
{
...
boost::unique_lock<boost::mutex> lock(mut);
cond.wait(lock);
}
}
versus
boost::condition_variable cond;
boost::mutex mut;
//thread1
{
"read_socket()"
boost::unique_lock<boost::mutex> lock(mut);
cond.notify_one();
}
//thread2
{
for(;;)
{
...
boost::unique_lock<boost::mutex> lock(mut);
cond.wait(lock);
}
Is there an impact if I omit the lock before calling cond.notify_one() ?
The C++11 standard does not state any requirement for notify_one and notify_all; so not holding the lock when you signal a condition_variable is fine. However, it's often necessary for the signaling thread to hold the lock until it sets the condition checked by the waiting thread after it's woken up. If it does not, the program may contain races. For an example, see this SO question: Boost synchronization.
When thread2 is waking, it will attempt to re-aquire the lock. If thread1 is holding the lock, thread2 will block until thread1 releases the lock.
In the code shown here, this doesn't significantly impact behavior. If you were to add any behavior in thread1 after cond.notify_one();, that behavior would be guaranteed to execute before thread2 proceeds in the second code block only.
Alternatively, you could construct the unique lock in thread2 before entering the for loop, rather than just before waiting on the condition variable. This would ensure that thread1 blocks when trying to construct its own unique lock until thread2 is waiting for a signal, provided that thread1 is not executing before thread2 has initialized itself and entered the loop. This would allow you to guarantee that thread1 doesn't send any notifications that thread2 isn't waiting for.