Let's say I have something like this:
bool signalled = false;
std::condition_variable cv;
void thread1() {
while (true) {
std::unique_lock l(mutex);
cv.wait_until(l, [] { return signalled; });
return;
}
}
void thread2...N() {
signalled = true;
cv.notify_all();
}
Is that considered thread-safe? The boolean may be set to true in many threads to interrupt thread1.
Edit: If not thread-safe I’m looking for a description of what the race condition is so I can better understand the underlying issue and fill in a knowledge gap.
Writing to a non-atomic variable without synchronization is UB. However, making signalled an atomic<bool> will not solve the problem.
C++ reference on std::condition_variable reads:
Even if the shared variable is atomic, it must be modified under the mutex in order to correctly publish the modification to the waiting thread.
You should do this:
bool signalled = false;
void thread2...N()
{
std::unique_lock l(mutex);
signalled = true;
cv.notify_all();
}
Related questions:
Mutex protecting std::condition_variable
Shared atomic variable is not properly published if it is not modified under mutex
Why do I need to acquire a lock to modify a shared "atomic" variable before notifying condition_variable
Related
Consider the following quote about std::condition_variable from cppreference:
The condition_variable class is a synchronization primitive that can be used to block a thread, or multiple threads at the same time, until another thread both modifies a shared variable (the condition), and notifies the condition_variable.
The thread that intends to modify the variable has to
acquire a std::mutex (typically via std::lock_guard)
perform the modification while the lock is held
execute notify_one or notify_all on the std::condition_variable (the lock does not need to be held for notification)
Even if the shared variable is atomic, it must be modified under the mutex in order to correctly publish the modification to the waiting thread.
An exemplary typical scenario of this approach is as follows:
// shared (e.d., global) variables:
bool proceed = false;
std::mutex m;
std::condition_variable cv;
// thread #1:
{
std::unique_lock<std::mutex> l(m);
while (!proceed) cv.wait(l); // or, cv.wait(l, []{ return proceed; });
}
// thread #2:
{
std::lock_guard<std::mutex> l(m);
proceed = true;
}
cv.notify_one();
However, #Tim in this question came up with (kind-of an academic) alternative:
std::atomic<bool> proceed {false}; // atomic to avoid data race
std::mutex m;
std::condition_variable cv;
// thread #1:
{
std::unique_lock<std::mutex> l(m);
while (!proceed) cv.wait(l);
}
// thread #2:
proceed = true; // not protected with mutex
{ std::lock_guard<std::mutex> l(m); }
cv.notify_one();
It obviously does not meet the requirements of the above cppreference quote since proceed shared variable (atomic in this case) is not modified under the mutex.
The question is if this code is correct. In my opinion, it is, since:
First option is that !proceed in while (!proceed) is evaluated as false. In that case, cv.wait(l); is not invoked at all.
Second option is that !proceed in while (!proceed) is evaluated as true. In that case, cv.notify_one() cannot happen until thread #1 enters cv.wait(l);.
Am I missing something or is cppreference wrong in this regard? (For instance, are some reordering issues involved?)
And, what if proceed = true; would be changed to proceed.store(true, std::memory_order_relaxed);?
I've implemented a "Ticket" class which is shared as a shared_ptr between multiple threads.
The program flow is like this:
parallelQuery() is called to start a new query job. A shared instance of Ticket is created.
The query is split into multiple tasks, each task is enqueued on a worker thread (this part is important, otherwise I'd just join threads and done). Each task gets the shared ticket.
ticket.wait() is called to wait for all tasks of the job to complete.
When one task is done it calls the done() method on the ticket.
When all tasks are done the ticket is unlocked, result data from the task aggregated and returned from parallelQuery()
In pseudo code:
std::vector<T> parallelQuery(std::string str) {
auto ticket = std::make_shared<Ticket>(2);
auto task1 = std::make_unique<Query>(ticket, str+"a");
addTaskToWorker(task1);
auto task2 = std::make_unique<Query>(ticket, str+"b");
addTaskToWorker(task2);
ticket->waitUntilDone();
auto result = aggregateData(task1, task2);
return result;
}
My code works. But I wonder if it is theoretically possible that it can lead to a deadlock in case when unlocking the mutex is executed right before it gets locked again by the waiter thread calling waitUntilDone().
Is this a possibility, and how to avoid this trap?
Here is the complete Ticket class, note the execution order example comments related to the problem description above:
#include <mutex>
#include <atomic>
class Ticket {
public:
Ticket(int numTasks = 1) : _numTasks(numTasks), _done(0), _canceled(false) {
_mutex.lock();
}
void waitUntilDone() {
_doneLock.lock();
if (_done != _numTasks) {
_doneLock.unlock(); // Execution order 1: "waiter" thread is here
_mutex.lock(); // Execution order 3: "waiter" thread is now in a dealock?
}
else {
_doneLock.unlock();
}
}
void done() {
_doneLock.lock();
_done++;
if (_done == _numTasks) {
_mutex.unlock(); // Execution order 2: "task1" thread unlocks the mutex
}
_doneLock.unlock();
}
void cancel() {
_canceled = true;
_mutex.unlock();
}
bool wasCanceled() {
return _canceled;
}
bool isDone() {
return _done >= _numTasks;
}
int getNumTasks() {
return _numTasks;
}
private:
std::atomic<int> _numTasks;
std::atomic<int> _done;
std::atomic<bool> _canceled;
// mutex used for caller wait state
std::mutex _mutex;
// mutex used to safeguard done counter with lock condition in waitUntilDone
std::mutex _doneLock;
};
One possible solution which just came to my mind when editing the question is that I can put _done++; before the _doneLock(). Eventually, this should be enough?
Update
I've updated the Ticket class based on the suggestions provided by Tomer and Phil1970. Does the following implementation avoid mentioned pitfalls?
class Ticket {
public:
Ticket(int numTasks = 1) : _numTasks(numTasks), _done(0), _canceled(false) { }
void waitUntilDone() {
std::unique_lock<std::mutex> lock(_mutex);
// loop to avoid spurious wakeups
while (_done != _numTasks && !_canceled) {
_condVar.wait(lock);
}
}
void done() {
std::unique_lock<std::mutex> lock(_mutex);
// just bail out in case we call done more often than needed
if (_done == _numTasks) {
return;
}
_done++;
_condVar.notify_one();
}
void cancel() {
std::unique_lock<std::mutex> lock(_mutex);
_canceled = true;
_condVar.notify_one();
}
const bool wasCanceled() const {
return _canceled;
}
const bool isDone() const {
return _done >= _numTasks;
}
const int getNumTasks() const {
return _numTasks;
}
private:
std::atomic<int> _numTasks;
std::atomic<int> _done;
std::atomic<bool> _canceled;
std::mutex _mutex;
std::condition_variable _condVar;
};
Don't write your own wait methods but use std::condition_variable instead.
https://en.cppreference.com/w/cpp/thread/condition_variable.
Mutexes usage
Generally, a mutex should protect a given region of code. That is, it should lock, do its work and unlock. In your class, you have multiple method where some lock _mutex while other unlock it. This is very error-prone as if you call the method in the wrong order, you might well be in an inconsistant state. What happen if a mutex is lock twice? or unlocked when already unlocked?
The other thing to be aware with mutex is that if you have multiple mutexes, it that you can easily have deadlock if you need to lock both mutexes but don't do it in consistant order. Suppose that thread A lock mutex 1 first and the mutex 2, and thread B lock them in the opposite order (mutex 2 first). There is a possibility that something like this occurs:
Thread A lock mutex 1
Thread B lock mutex 2
Thread A want to lock mutex 2 but cannot as it is already locked.
Thread B want to lock mutex 1 but cannot as it is already locked.
Both thread will wait forever
So in your code, you should at least have some checks to ensure proper usage. For example, you should verify _canceled before unlocking the mutex to ensure cancel is called only once.
Solution
I will just gave some ideas
Declare a mutux and a condition_variable to manage the done condition in your class.
std::mutex doneMutex;
std::condition_variable done_condition;
Then waitUntilDone would look like:
void waitUntilDone()
{
std::unique_lock<std::mutex> lk(doneMutex);
done_condition.wait(lk, []{ return isDone() || wasCancelled();});
}
And done function would look like:
void done()
{
std::lock_guard<std::mutex> lk(doneMutex);
_done++;
if (_done == _numTasks)
{
doneCondition.notify_one();
}
}
And cancel function would become
void done()
{
std::lock_guard<std::mutex> lk(doneMutex);
_cancelled = true;
doneCondition.notify_one();
}
As you can see, you only have one mutex now so you basically eliminate the possibility of a deadlock.
Variable naming
I suggest you to not use lock in the name of you mutex since it is confusing.
std::mutex someMutex;
std::guard_lock<std::mutex> someLock(someMutex); // std::unique_lock when needed
That way, it is far easier to know which variable refer to the mutex and which one to the lock of the mutex.
Good reading
If you are serious about multithreading, then you should buy that book:
C++ Concurrency in Action
Practical Multithreading
Anthony Williams
Code Review (added section)
Essentially same code has beed posted to CODE REVIEW: https://codereview.stackexchange.com/questions/225863/multithreading-ticket-class-to-wait-for-parallel-task-completion/225901#225901.
I have put an answer there that include some extra points.
You not need to use mutex for operate with atomic values
UPD
my answer to mainn question was wrong. I deleted one.
You can use simple (non atomic) int _numTasks; also. And you not need shared pointer - just create Task on the stack and pass pointer
Ticket ticket(2);
auto task1 = std::make_unique<Query>(&ticket, str+"a");
addTaskToWorker(task1);
or unique ptr if you like
auto ticket = std::make_unique<Ticket>(2);
auto task1 = std::make_unique<Query>(ticket.get(), str+"a");
addTaskToWorker(task1);
because shared pointer can be cut by Occam's razor :)
In their example usage of std::condition_variable they have essentially
std::mutex m;
std::condition_variable cv;
bool ready = false;
void worker_thread()
{
// Wait until main() sends data
std::unique_lock<std::mutex> lk(m);
cv.wait(lk, []{return ready;});
// more ...
}
int main()
{
std::thread worker(worker_thread);
data = "Example data";
// send data to the worker thread
{
std::lock_guard<std::mutex> lk(m);
ready = true;
}
cv.notify_one();
// more...
}
My problem now is the variable ready which is not declared std::atomic*.
Why doesn't this introduce a race condition if spurious wakeup occurs?
No, there is no race condition.
Even if the condition variable spuriously wakes up, it must re-acquire the lock. hence, two things happen:
no thread can touch ready while the lock is held, as a lock protects it.
by re-acquiring the lock, the boolean must be synchronized, because the lock enforce memory order acquire, which causes ready to have the latest value.
So there is no way for race condition to happen.
I have some code like this:
std::queue<myData*> _qDatas;
std::mutex _qDatasMtx;
Generator function
void generator_thread()
{
std::this_thread::sleep_for(std::chrono::milliseconds(1));
{
std::lock_guard<std::mutex> lck(_qDatasMtx);
_qData.push(new myData);
}
//std::lock_guard<std::mutex> lck(cvMtx); //need lock here?
cv.notify_one();
}
Consumer function
void consumer_thread()
{
for(;;)
{
std::unique_lock lck(_qDatasMtx);
if(_qDatas.size() > 0)
{
delete _qDatas.front();
_qDatas.pop();
}
else
cv.wait(lck);
}
}
So if I have dozens of generator threads and one consumer thread, do I need a mutex lock when calling cv.notify_one() in each thread?
And is std::condition_variable thread-safe?
do I need a mutex lock when calling cv.notify_one() in each thread?
No
is std::condition_variable thread-safe?
Yes
When calling wait, you pass a locked mutex which is immediately unlocked for use. When calling notify you don't lock it with the same mutex, since what will happen is (detailed on links):
Notify
Wakeup thread that is waiting
Lock mutex for use
From std::condition_variable:
execute notify_one or notify_all on the std::condition_variable (the lock does not need to be held for notification)
and from std::condition_variable::notify_all:
The notifying thread does not need to hold the lock on the same mutex as the one held by the waiting thread(s);
With regards to your code snippet:
//std::lock_guard<std::mutex> lck(cvMtx); //need lock here?
cv.notify_one();
No you do not need a lock there.
notify_one can be called from multiple threads without a lock.
However, in order for normal operation to work and no chance of notifications "slipping through", you must hold the lock the condition variable is guarded by between the point where you modify the spurious wakeup guard and/or package the condition variable checks on read, and the point where you notify one.
Your code appears to pass that test. However this version is more clear:
std::unique_lock lck(_qDatasMtx);
for(;;) {
cv.wait(lck,[&]{ return _qDatas.size()>0; });
delete _qDatas.front();
_qDatas.pop();
}
and it reduces spurious unlocking/relocking.
Suppose I have producer and consumer threads synchronized using a condition variable.
// Approach 1 - Using condition variable
mutex mmutex;
condition_variable mcond;
queue<string> mqueue;
void producer(){
while (true) {
unique_lock<mutex> lck(mmutex);
mqueue.push_back("Hello Hi");
mcond.notify_one();
}
}
void consumer{
while (true){
unique_lock<mutex> lck(mmutex); // locks mmutex
mcond.wait(); // releases mmutex;
string s = mqueue.front(); // locks mmutex again
mqueue.pop();
mmutex.unlock();
}
}
How does the above code compare for synchronization using simple atomic types, as follows -
// Approach 2 - using atomics
atomic_bool condition = false;
condition_variable mcond;
queue<string> mqueue;
void producer(){
unique_lock<mutex> lck(mmutex);
mqueue.push_back("Hello Hi");
condition = true;
}
void consumer{
while (1) {
if (condition == true) {
condition = false;
unique_lock<mutex> lck(mmutex);
mqueue.front();
mqueue.pop();
lck.unlock();
}
}
}
Since, condition variables exist, I assume they are the preferred means of achieving synchronization in cases like these. If indeed so, why are condition variables a better alternative to the simple atomic_bool (or atomic_int if you need more then two states) based synchronization? If not, then what is the best way to achieve synchronization in this case?
The difference is that a condition variable doesn't consume CPU cycles while another thread is waiting for it. If you use an atomic variable for synchronization you have to keep checking its value in a loop.