This question already has an answer here:
std::condition_variable::notify_one() called several times without context switching
(1 answer)
Closed 10 years ago.
How many waiting threads will wake up if I call std::condition_variable::notify_one() twice without any time interval, like this:
{
std::unique_lock<std::mutex> lock(condvar_mutex);
condvar.notify_one();
condvar.notify_one();
}
Is there any guarantee that these notifications will be delivered to different threads, not the same thread several times?
§30.5.1.7: If any threads are blocked waiting for *this, unblocks one
of those threads.
There is no guarantee that it's different threads, just that it's one thread. During the time between the two notify_one calls it is possible that the same thread that was waked with the first notify_one is re-blocked.
e.g. In the following example it is not guaranteed whether thread 3 will wake or not (disregard spurious wakes for this example).
- thread 1:
1 condvar.notify_one();
- 3 and 4 could run here.
2 condvar.notify_one();
- thread 2:
3 condvar.wait(/*...*/);
4 condvar.wait(/*...*/);
- thread 3:
5 condvar.wait(/*...*/);
I assume that condvar_mutex is the correct mutex for the condvar.
It's not possible for both notifications to be delivered to the same thread. The reason is that you call notify_one twice while holding the mutex. So whichever thread is unblocked first, there's no way it can acquire the mutex, so it can't return from wait. It can't even throw an exception from wait_for without first acquiring the mutex.
Since it can't get out of its wait, there's no way it can get back onto the list of waiters before the second notify_one is called.
So, your code unblocks up to two threads blocked on the condvar, if there are that many. If there are fewer then the additional notifications have no effect.
Related
While implementing a thread pool pattern in C++ based on this, I came across a few questions.
Let's assume minimal code sample:
std::mutex thread_mutex;
std::condition_variable thread_condition;
void thread_func() {
std::unique_lock<std::mutex> lock(thread_mutex);
thread_condition.wait(lock);
lock.unlock();
}
std::thread t1 = std::thread(thread_func);
Regarding cppreference.com about conditon_variable::wait(), wait() causes the current thread to block. What is locking the mutex then for when I only need one thread at all using wait() to get notified when something is to do?
unique_lock will block the thread when the mutex already has been locked by another thread. But this wouldn't be neccesary as long as wait() blocks anyway or what do I miss here?
Adding a few lines at the bottom...
std::thread t2 = std::thread(thread_func);
thread_condition.notify_all()
When unique_lock is blocking the thread, how will notify_all() reach both threads when one of them is locked by unique_lock and the other is blocked by wait()? I understand that blocking wait() will be freed by notify_all() which afterwards leads to unlocking the mutex and that this gives chance to the other thread for locking first the mutex and blocking thread by wait() afterwards. But how is this thread notified than?
Expanding this question by adding a loop in thread_func()...
std::mutex thread_mutex;
std::condition_variable thread_condition;
void thread_func() {
while(true) {
std::unique_lock<std::mutex> lock(thread_mutex);
thread_condition.wait(lock);
lock.unlock();
}
}
std::thread t1 = std::thread(thread_func);
std::thread t2 = std::thread(thread_func);
thread_condition.notify_all()
While reading documentation, I would now expect both threads running endlessly. But they do not return from wait() lock. Why do I have to use a predicate for expected behaviour like this:
bool wakeup = false;
//[...]
thread_condition.wait(lock, [] { return wakeup; });
//[...]
wakeup = !wakeup;
thread_condition.notify_all();
Thanks in advance.
This is really close to being a duplicate, but it's actually that question that answers this one; we also have an answer that more or less answers this question, but the question is distinct. I think that an independent answer is needed, even though it's little more than a (long) definition.
What is a condition variable?
The operational definition is that it's a means for a thread to block until a message arrives from another thread. A mutex alone can't possibly do this: if all other threads are busy with unrelated work, a mutex can't block a thread at all. A semaphore can block a lone thread, but it's tightly bound to the notion of a count, which isn't always appropriate to the nature of the message to receive.
This "channel" can be implemented in several ways. Very low-tech is to use a pipe, but that involves expensive system calls. Windows provides the Event object which is fundamentally a boolean on whose truth a thread may wait. (C++20 provides a similar feature with atomic_flag::wait.)
Condition variables take a different approach: their structural definition is that they are stateless, but have a special connection to a corresponding mutex type. The latter is necessitated by the former: without state, it is impossible to store a message, so arrangements must be made to prevent sending a message during some interval between a thread recognizing the need to wait (by examining some other state: perhaps that the queue from which it wants to pop is empty) and it actually being blocked. Of course, after the thread is blocked it cannot take any action to allow the message to be sent, so the condition variable must do so.
This is implemented by having the thread take a mutex before checking the condition and having wait release that mutex only after the thread can receive the message. (In some implementations, the mutex is also used to protect the workings of the condition variable, but C++ does not do so.) When the message is received, the mutex is re-acquired (which may block the thread again for a time), as is necessary to consult the external state again. wait thus acts like an everted std::unique_lock: the mutex is unlocked during wait and locked again afterwards, with possibly arbitary changes having been made by other threads in the meantime.
Answers
Given this understanding, the individual answers here are trivial:
Locking the mutex allows the waiting thread to safely decide to wait, given that there must be some other thread affecting the state in question.
If the std::unique_lock blocks, some other thread is currently updating the state, which might actually obviate the need for wait.
Any number of threads can be in wait, since each unlocks the mutex when it calls it.
Waiting on a condition variable, er, unconditionally is always wrong: the state you're after might already apply, with no further messages coming.
Looking at several videos and the documentation example, we unlock the mutex before calling the notify_all(). Will it be better to instead call it after?
The common way:
Inside the Notifier thread:
//prepare data for several worker-threads;
//and now, awaken the threads:
std::unique_lock<std::mutex> lock2(sharedMutex);
_threadsCanAwaken = true;
lock2.unlock();
_conditionVar.notify_all(); //awaken all the worker threads;
//wait until all threads completed;
//cleanup:
_threadsCanAwaken = false;
//prepare new batches once again, etc, etc
Inside one of the worker threads:
while(true){
// wait for the next batch:
std::unique_lock<std::mutex> lock1(sharedMutex);
_conditionVar.wait(lock1, [](){return _threadsCanAwaken});
lock1.unlock(); //let sibling worker-threads work on their part as well
//perform the final task
//signal the notifier that one more thread has completed;
//loop back and wait until the next task
}
Notice how the lock2 is unlocked before we notify the condition variable - should we instead unlock it after the notify_all() ?
Edit
From my comment below: My concern is that, what if the worker spuriously awakes, sees that the mutex is unlocked, super-quickly completes the task and loops back to the start of while. Now the slow-poke Notifier finally calls notify_all(), causing the worker to loop an additional time (excessive and undesired).
There are no advantages to unlocking the mutex before signaling the condition variable unless your implementation is unusual. There are two disadvantages to unlocking before signaling:
If you unlock before you signal, the signal may wake a thread that choose to block on the condition variable after you unlocked. This can lead to a deadlock if you use the same condition variable to signal more than one logical condition. This kind of bug is hard to create, hard to diagnose, and hard to understand. It is trivially avoided by always signaling before unlocking. This ensures that the change of shared state and the signal are an atomic operation and that race conditions and deadlocks are impossible.
There is a performance penalty for unlocking before signaling that is avoided by unlocking after signaling. If you signal before you unlock, a good implementation will know that your signal cannot possibly render any thread ready-to-run because the mutex is held by the calling thread and any thread affects by the condition variable necessarily cannot make forward progress without the mutex. This permits a significant optimization (often called "wait morphing") that is not possible if you unlock first.
So signal while holding the lock unless you have some unusual reason to do otherwise.
should we instead unlock it after the notify_all() ?
It is correct to do it either way but you may have different behavior in different situations. It is quite difficult to predict how it will affect performance of your program - I've seen both positive and negative effects for different applications. So it is better you profile your program and make decision on your particular situation based on profiling.
As mentioned here : cppreference.com
The notifying thread does not need to hold the lock on the same mutex
as the one held by the waiting thread(s); in fact doing so is a
pessimization, since the notified thread would immediately block
again, waiting for the notifying thread to release the lock.
That said, documentation for wait
At the moment of blocking the thread, the function automatically calls
lck.unlock(), allowing other locked threads to continue.
Once notified (explicitly, by some other thread), the function
unblocks and calls lck.lock(), leaving lck in the same state as when
the function was called. Then the function returns (notice that this
last mutex locking may block again the thread before returning).
so when notified wait will re-attempt to gain the lock and in that process it will get blocked again till original notifying thread releases the lock.
So I'll suggest that release the lock before calling notify. As done in example on cppreference.com and most importantly
Don't be Pessimistic.
David's answer seems to me wrong.
First, assuming the simple case of two threads, one waiting for the other on a condition variable, unlocking first by the notifier will not waken the other waiting thread, as the signal has not arrived. Then the notify call will immediately waken the waiting thread. You do not need any special optimizations.
On the other hand, signalling first has the potential of waking up a thread and making it sleep immediately again, as it cannot hold the lock—unless wait morphing is implemented.
Wait morphing does not exist in Linux at least, according to the answer under this StackOverflow question: Which OS / platforms implement wait morphing optimization?
The cppreference example also unlocks first before signalling: https://en.cppreference.com/w/cpp/thread/condition_variable/notify_all
It explicit says:
The notifying thread does not need to hold the lock on the same mutex as the one held by the waiting thread(s). Doing so may be a pessimization, since the notified thread would immediately block again, waiting for the notifying thread to release the lock, though some implementations recognize the pattern and do not attempt to wake up the thread that is notified under lock.
should we instead unlock it after the notify_all() ?
After reading several related posts, I've formed the opinion that it's purely a performance issue. If OS supports "wait morphing", unlock after; otherwise, unlock before.
I'm adding an answer here to augment that of #DavidSchwartz 's. Particularly, I'd like to clarify his point 1.
If you unlock before you signal, the signal may wake a thread that choose to block on the condition variable after you unlocked. This can lead to a deadlock if you use the same condition variable to signal more than one logical condition. This kind of bug is hard to create, hard to diagnose, and hard to understand. It is trivially avoided by always signaling before unlocking. This ensures that the change of shared state and the signal are an atomic operation and that race conditions and deadlocks are impossible.
The 1st thing I said is that, because it's a CV and not a Mutex, a better term for the so-called "deadlock" might be "sleep paralysis" - a mistake some programs make is that
a thread that's supposed to wake
went to sleep due to not rechecking the condition it's been waiting for before wait'ng again.
The 2nd thing is that, when waking some other thread(s),
the default choice should be broadcast/notify_all (broadcast is the POSIX term, which is equivalent to its C++ counterpart).
signal/notify is an optimized special case used for when there's only 1 other thread is waiting.
Finally 3rd, David is adamant that
it's better to unlock after notify,
because it can avoid the "deadlock" which I've been referring to as "sleep paralysis".
If it's unlock then notify, then there's a window where another thread (let's call this the "wrong" thread) may i.) acquire the mutex, ii.)going into wait, and iii.) wake up. The steps i. ii. and iii. happens too quickly, consumed the signal, leaving the intended (let's call it "correct") thread in sleep.
I discussed this extensively with David, he clarified that only when all 3 points are violated ( 1. condvar associated with several separate conditions and/or didn't check it before waiting again; 2. signal/notify only 1 thread when there're more than 1 other threads using the condvar; 3. unlock before notify creating a window for race condition ), the "sleep paralysis" would occur.
Finally, my recommendation is that, point 1 and 2 are essential for correctness of the program, and fixing issues associated with 1 and 2 should be prioritized over 3, which should only be a augmentative "last resort".
For the purpose of providing reference, manpage for signal/broadcast and wait contains some info from version 3 of Single Unix Specification that gave some explanations on point 1 and 2, and partly 3. Although specified for POSIX/Unix/Linux in C, it's concepts are applicable to C++.
As of this writing (2023-01-31), the 2018 edition of version 4 of Single Unix Specification is released, and the drafting of version 5 is underway.
Is there a way to give a certain thread priority if multiple threads try to aquire the same mutex at one time
For example you have 2 threads both started at the same time and they sleep then try to acquire lock
In main thread
pthread_mutex_t mut = PTHREAD_MUTEX_INITIALIZER;
pthread_t thrd1, thrd2;
pthread_create( &thrd1, NULL, someFunc, NULL);
pthread_create( &thrd2, NULL, someFunc, NULL);
and someFunc() would have
usleep(1000);
pthread_mutex_lock(&mut);
I thought this had to do with threading priority but that appears to be a different topic.
Is there a way to guarantee thread 2 got the lock on mut first?
Your problem is underspecified, because you do not provide any parameters on use case or how you have determined a choice is to be made. You seem to believe that priority alone will be sufficient to get you the behavior you want, but that is a fallacy.
Priority alone is insufficient.
Priority alone does not let a higher priority thread wrest control of a mutex away from a lower priority thread that has acquired a lock. The lower priority thread must voluntarily yield the lock for the sake of safely handling the protected critical section. If the higher priority thread was allowed to yank the mutex away from the lower priority thread, the state in the critical section may be left inconsistent by the lower priority thread, and this could cause problems for the higher priority thread.
Here are some options.
Option 1: Start thread 2 first.
This is the simplest solution. Just launch thread 2 first, and let it notify the main thread when it has acquired the lock. Then, the main thread can start thread 1.
If you really just want each thread to figure it out without intervention from the main thread, then you have the specify what you mean by "guarantee thread 2 got the lock first". If thread 1 truly wakes up first, and goes for the lock without any contention, what do you expect it to do about that? And, how does thread 1 know that it is thread 1 and not thread 2?
Option 2: Why care about which thread is which?
If there is no specific reason that thread 2 is thread 2, then you could just let which ever thread gets the lock first do whatever it is that thread 2 is supposed to do, and the subsequent thread does whatever thread 1 is supposed to do.
This reduces the problem to only needing to know if any thread has gone yet or not. However, if you really do care that thread 2 is supposed to go first, then you still need to handle the case that thread 1 gets the lock out of turn. So, it needs to somehow know that it is thread 1, and it needs to know somehow that thread 2 has not done its thing yet.
Option 3: Make thread 1 wait for thread 2 (somehow).
When the thread starts, it first determines which thread it is. If it is thread 1, it waits for an indication from thread 2 that it can acquire the lock. If it is thread 2, it first acquires the lock, then delivers an indication to thread 1 that it may acquire the lock.
Options 1 and 3 have one thread notifying another thread. There are many variations on how this notification can be delivered. You could use a semaphore, or a condition wait, or a pipe, or even a spin lock. There are many other options, it just depends on what you believe is the right choice for your application.
I want to use pthreads and pthread_mutexes in a C++ program. I don't have any actual code yet, just a question on how mutexes work. If I have thread 1 lock a mutex, then n number of other threads attempt to lock that same mutex, what will the behavior be when thread 1 unlocks the mutex? If thread 1 acquires the lock, then thread 2 attempts to lock, then thread 3 attempts to lock, will thread 2 have priority over thread 3 when the lock is released?
Here is a more organized timeline of the locking would look:
thread 1 acquires lock
thread 2 attempts to lock
thread 3 attempts to lock
thread 4 attempts to lock
thread 1 unlocks mutex
??
in other words, I'd like the threads to execute in the order in which they attempt to acquire the lock/are created. if there is a better way to do this, I'm more than open to suggestions.
No - there will be no such guaranteed ordering. After thread 1 releases the mutex thread 3 could get or thread 2 could get it. You can't predict that.
Read this for an explanation.
I figured I'd post my solution so others could see a potential fix:
My main thread is the thread that creates worker threads by locking a creation mutex and putting each thread into a std::queue.
at the start of main, I create a manager thread and that is in an infinite while loop which checks to see if the queue is empty every cycle. when the manager thread sees the queue is nonempty
(if(! queue::empty())), it waits to acquire the lock and then creates the front thread, pops it from the queue, executes a pthread_join() on the thread and loops back around for the next thread.
How many waiting threads will wake up in this example:
1st thread:
void wakeUp2Threads()
{
std::unique_lock<std::mutex> lock(condvar_mutex);
condvar.notify_one();
condvar.notify_one();
}
2nd thread:
{
std::unique_lock<std::mutex> lock(condvar_mutex);
condvar.wait(lock); <- 2nd thread has entered here before 1st thread entered wakeUp2Threads.
}
3rd thread (the same as 2nd):
{
std::unique_lock<std::mutex> lock(condvar_mutex);
condvar.wait(lock); <- 3rd thread has entered here before 1st thread entered wakeUp2Threads.
}
Is there any guarantee that in this example both notifications will be delivered to different threads, not the same thread several times?
I.e what does notify_one() mean:
1) notify one thread, no matter has it been already notified (but has not been woken up yet), or not. (* see note)
or
2) notify one thread, but only this one, which has not been notified yet.
(*) Pay attention! I'm not talking here about scenario "waiting thread had already been notified somewhere in the past, woken up, done some stuff and entered condvar.wait() again" - of course, in this case several notify_one() routines can wake up the same thread over and over again.
I'm talking about another case:
notify_one() has notified waiting thread about wake-up, but BEFORE this waiting thread has received time slot form the kernel scheduler and continued execution - another notify_one() has been called again. Is it possible this second notification will be delivered to the same thread again, while it hasn't been woken up from first notification yet?
The notify_one call atomically unblocks one thread. That means that when called a second time it can't unblock the same thread as it's no longer blocked.
This is specified in the standard at section 30.5/3 and 30.5.1/7.