The Test class is used in a multithreaded enviroment. ThreadA asks if he has to wait for ThreadB by calling the hasToWait method (ITestWaiter). When ThreadB has done his work he notify`s all waiters by calling the Test::notify method.
Could you tell me if there is a possible deadlock situation in the wait() method - between the part, which is locked by the mutex and the call to the semaphore acquire method?
struct Semaphore {
bool acquire() { return WaitForSingleObject(sem, INFINITE); }
private:
Handle sem;
};
struct Test
{
bool wait(std::mutex mutex, const ITestWaiter *obj);
bool notify(std::mutex mutex);
private:
std::vector<Semaphore> waiters;
};
bool Test::wait(std::mutex mutex, const ITestWaiter *obj) {
Semaphore* sem;
{
std::unique_lock<std::mutex> mlock(mutex);
if (!obj->hasToWait())
return false;
sem = createSemaphoreAndPushBackToVector();
}
try {
sem->acquire();
}
catch (std::exception e) {}
return true;
}
bool Test::notify(std::mutex mutex) {
std::unique_lock<std::mutex> mlock(mutex);
//notify waiters by releasing the semaphore
return true;
}
From the code you posted, there shouldn't be a problem: In both cases, you do not block during the time you hold the lock; you just do some small actions (once modify the vector, once iterate over it) instead. But there's code you didn't show!
First, there's how you are going to notify. I assume you use CreateEvent to get the handle and SetEvent for notification – if so, no problem either.
Then, there's the hasToWait function. Suspicious: You are calling it while already holding the lock! Is there any reason for? Does hasToWait some locking, too? Does the other thread possibly try to lock the same facility? Then risk of deadlock exists if both threads do not acquire the locks in the same order.
If there's no separate locking involved, but hasToWait needs to access some resources that need to be protected by the same mutex, then the code as is is fine, too.
If there's no locking and no access to shared resources, then locking the mutex first is in vain and just requires time; in this case, first checking is more efficient:
if (obj->hasToWait())
{
Semaphore* sem;
{
std::unique_lock<std::mutex> mlock(mutex);
sem = createSemaphoreAndPushBackToVector();
}
try
{
sem->acquire();
}
catch (std::exception e)
{ }
}
Related
I've implemented a "Ticket" class which is shared as a shared_ptr between multiple threads.
The program flow is like this:
parallelQuery() is called to start a new query job. A shared instance of Ticket is created.
The query is split into multiple tasks, each task is enqueued on a worker thread (this part is important, otherwise I'd just join threads and done). Each task gets the shared ticket.
ticket.wait() is called to wait for all tasks of the job to complete.
When one task is done it calls the done() method on the ticket.
When all tasks are done the ticket is unlocked, result data from the task aggregated and returned from parallelQuery()
In pseudo code:
std::vector<T> parallelQuery(std::string str) {
auto ticket = std::make_shared<Ticket>(2);
auto task1 = std::make_unique<Query>(ticket, str+"a");
addTaskToWorker(task1);
auto task2 = std::make_unique<Query>(ticket, str+"b");
addTaskToWorker(task2);
ticket->waitUntilDone();
auto result = aggregateData(task1, task2);
return result;
}
My code works. But I wonder if it is theoretically possible that it can lead to a deadlock in case when unlocking the mutex is executed right before it gets locked again by the waiter thread calling waitUntilDone().
Is this a possibility, and how to avoid this trap?
Here is the complete Ticket class, note the execution order example comments related to the problem description above:
#include <mutex>
#include <atomic>
class Ticket {
public:
Ticket(int numTasks = 1) : _numTasks(numTasks), _done(0), _canceled(false) {
_mutex.lock();
}
void waitUntilDone() {
_doneLock.lock();
if (_done != _numTasks) {
_doneLock.unlock(); // Execution order 1: "waiter" thread is here
_mutex.lock(); // Execution order 3: "waiter" thread is now in a dealock?
}
else {
_doneLock.unlock();
}
}
void done() {
_doneLock.lock();
_done++;
if (_done == _numTasks) {
_mutex.unlock(); // Execution order 2: "task1" thread unlocks the mutex
}
_doneLock.unlock();
}
void cancel() {
_canceled = true;
_mutex.unlock();
}
bool wasCanceled() {
return _canceled;
}
bool isDone() {
return _done >= _numTasks;
}
int getNumTasks() {
return _numTasks;
}
private:
std::atomic<int> _numTasks;
std::atomic<int> _done;
std::atomic<bool> _canceled;
// mutex used for caller wait state
std::mutex _mutex;
// mutex used to safeguard done counter with lock condition in waitUntilDone
std::mutex _doneLock;
};
One possible solution which just came to my mind when editing the question is that I can put _done++; before the _doneLock(). Eventually, this should be enough?
Update
I've updated the Ticket class based on the suggestions provided by Tomer and Phil1970. Does the following implementation avoid mentioned pitfalls?
class Ticket {
public:
Ticket(int numTasks = 1) : _numTasks(numTasks), _done(0), _canceled(false) { }
void waitUntilDone() {
std::unique_lock<std::mutex> lock(_mutex);
// loop to avoid spurious wakeups
while (_done != _numTasks && !_canceled) {
_condVar.wait(lock);
}
}
void done() {
std::unique_lock<std::mutex> lock(_mutex);
// just bail out in case we call done more often than needed
if (_done == _numTasks) {
return;
}
_done++;
_condVar.notify_one();
}
void cancel() {
std::unique_lock<std::mutex> lock(_mutex);
_canceled = true;
_condVar.notify_one();
}
const bool wasCanceled() const {
return _canceled;
}
const bool isDone() const {
return _done >= _numTasks;
}
const int getNumTasks() const {
return _numTasks;
}
private:
std::atomic<int> _numTasks;
std::atomic<int> _done;
std::atomic<bool> _canceled;
std::mutex _mutex;
std::condition_variable _condVar;
};
Don't write your own wait methods but use std::condition_variable instead.
https://en.cppreference.com/w/cpp/thread/condition_variable.
Mutexes usage
Generally, a mutex should protect a given region of code. That is, it should lock, do its work and unlock. In your class, you have multiple method where some lock _mutex while other unlock it. This is very error-prone as if you call the method in the wrong order, you might well be in an inconsistant state. What happen if a mutex is lock twice? or unlocked when already unlocked?
The other thing to be aware with mutex is that if you have multiple mutexes, it that you can easily have deadlock if you need to lock both mutexes but don't do it in consistant order. Suppose that thread A lock mutex 1 first and the mutex 2, and thread B lock them in the opposite order (mutex 2 first). There is a possibility that something like this occurs:
Thread A lock mutex 1
Thread B lock mutex 2
Thread A want to lock mutex 2 but cannot as it is already locked.
Thread B want to lock mutex 1 but cannot as it is already locked.
Both thread will wait forever
So in your code, you should at least have some checks to ensure proper usage. For example, you should verify _canceled before unlocking the mutex to ensure cancel is called only once.
Solution
I will just gave some ideas
Declare a mutux and a condition_variable to manage the done condition in your class.
std::mutex doneMutex;
std::condition_variable done_condition;
Then waitUntilDone would look like:
void waitUntilDone()
{
std::unique_lock<std::mutex> lk(doneMutex);
done_condition.wait(lk, []{ return isDone() || wasCancelled();});
}
And done function would look like:
void done()
{
std::lock_guard<std::mutex> lk(doneMutex);
_done++;
if (_done == _numTasks)
{
doneCondition.notify_one();
}
}
And cancel function would become
void done()
{
std::lock_guard<std::mutex> lk(doneMutex);
_cancelled = true;
doneCondition.notify_one();
}
As you can see, you only have one mutex now so you basically eliminate the possibility of a deadlock.
Variable naming
I suggest you to not use lock in the name of you mutex since it is confusing.
std::mutex someMutex;
std::guard_lock<std::mutex> someLock(someMutex); // std::unique_lock when needed
That way, it is far easier to know which variable refer to the mutex and which one to the lock of the mutex.
Good reading
If you are serious about multithreading, then you should buy that book:
C++ Concurrency in Action
Practical Multithreading
Anthony Williams
Code Review (added section)
Essentially same code has beed posted to CODE REVIEW: https://codereview.stackexchange.com/questions/225863/multithreading-ticket-class-to-wait-for-parallel-task-completion/225901#225901.
I have put an answer there that include some extra points.
You not need to use mutex for operate with atomic values
UPD
my answer to mainn question was wrong. I deleted one.
You can use simple (non atomic) int _numTasks; also. And you not need shared pointer - just create Task on the stack and pass pointer
Ticket ticket(2);
auto task1 = std::make_unique<Query>(&ticket, str+"a");
addTaskToWorker(task1);
or unique ptr if you like
auto ticket = std::make_unique<Ticket>(2);
auto task1 = std::make_unique<Query>(ticket.get(), str+"a");
addTaskToWorker(task1);
because shared pointer can be cut by Occam's razor :)
Since I have recently started coding multi threaded programs this might be a stupid question. I found out about the awesome mutex and condition variable usage. From as far as I can understand there use is:
Protect sections of code/shared resources from getting corrupted by multiple threads access. Hence lock that portion thus one can control which thread will be accessing.
If a thread is waiting for a resource/condition from another thread one can use cond.wait() instead of polling every msec
Now Consider the following class example:
class Queue {
private:
std::queue<std::string> m_queue;
boost::mutex m_mutex;
boost::condition_variable m_cond;
bool m_exit;
public:
Queue()
: m_queue()
, m_mutex()
, m_cond()
, m_exit(false)
{}
void Enqueue(const std::string& Req)
{
boost::mutex::scoped_lock lock(m_mutex);
m_queue.push(Req);
m_cond.notify_all();
}
std::string Dequeue()
{
boost::mutex::scoped_lock lock(m_mutex);
while(m_queue.empty() && !m_exit)
{
m_cond.wait(lock);
}
if (m_queue.empty() && m_exit) return "";
std::string val = m_queue.front();
m_queue.pop();
return val;
}
void Exit()
{
boost::mutex::scoped_lock lock(m_mutex);
m_exit = true;
m_cond.notify_all();
}
}
In the above example, Exit() can be called and it will notify the threads waiting on Dequeue that it's time to exit without waiting for more data in the queue.
My question is since Dequeue has acquired the lock(m_mutex), how can Exit acquire the same lock(m_mutex)? Isn't unless the Dequeue releases the lock then only Exit can acquire it?
I have seen this pattern in Destructor implementation too, using same class member mutex, Destructor notifies all the threads(class methods) thats it time to terminate their respective loops/functions etc.
As Jarod mentions in the comments, the call
m_cond.wait(lock)
is guaranteed to atomically unlock the mutex, releasing it for the thread, and starts listening to notifications of the condition variable (see e.g. here).
This atomicity also ensures any code in the thread is executed after the listening is set up (so no notify calls will be missed). This assumes of course that the thread first locks the mutex, otherwise all bets are off.
Another important bit to understand is that condition variables may suffer from "spurious wakeups", so it is important to have a second boolean condition (e.g. here, you could check the emptiness of your queue) so that you don't end up awoken with an empty queue. Something like this:
m_cond.wait(lock, [this]() { return !m_queue.empty() || m_exit; });
i have a problem. i want to use a mutex for my program. so what happens is this:
i am constructing an object that holds a std::timed_mutex. on creation this object locks the mutex because it should be unlocked later on. the same thread that created the mutex should now wait for that mutex while some other thread does work in the background. joining the thread is no option.
class A{
std::timed_mutex mutex;
A(){
mutex.lock();
}
bool waitForIt(int timeout){
if(mutex.try_lock_for(std::chrono::milliseconds(timeout))){
mutex.unlock();
return true;
}else{
return false;
}
}
}
when calling waitForIt from the same thread the program just goes through and instantly gets a false, totally ignoring the timeout.(yes its intended to unlock the mutex afterwards. it should mime something like an event so every thread waiting gets through)
so in the documentation it says this mutex has a nonrecursive behaviour. but testing revealed that for example i can use the .lock() multiple times from the same thread without getting blocked. i also can use try_lock_for multiple times and every time get true!!! if i once use lock before the try_lock_fors i always get false. sadly i need something that also blocks the same thread that locked the mutex. and i have no idea what to use. im programming on linux btw. so maybe there is a native solution?
also i didnt find a semaphore in the std libs.i could use that instead of the mutex. using my own implementation would be possible but i dont know how to make my own semaphore. any ideas?
as people dont seems to understand that its not that simple:
class IObservable : public IInterface{
private:
std::list<std::shared_ptr<IObserver>> observers;
public:
virtual ~IObservable(){}
void AddObserver(std::shared_ptr<IObserver> observer);
void RemoveObserver(std::shared_ptr<IObserver> observer);
void ClearObservers();
void TellCompleted(bool wasCanceled = false, std::shared_ptr<void> status = 0);
TYPEIDHASHFUNC(IObservable)
};
IObservable is the thing that threads can add observers to. the thing deriving from IObservable calls the method TellCompleted at the end of its actions.
class IObserver : public IInterface{
public:
virtual ~IObserver(){}
virtual CompleteResult Complete(bool wasCanceled, std::shared_ptr<void> status) = 0;
virtual bool WaitForCompletion(int timeoutInMs) = 0;
virtual bool IsCompleted() const = 0;
virtual bool WasCanceled() const = 0;
virtual std::shared_ptr<void> GetStatus() const = 0;
virtual void Reset() = 0;
TYPEIDHASHFUNC(IObserver)
};
IObserver are the observer that can be added to a IObservable. if IObservable completes the method Complete gets called on each observer that was added to the observable
class BasicObserver : public IObserver{
private:
bool isCompleted;
bool wasCanceled;
CompleteResult completeResult;
std::shared_ptr<void> status;
std::timed_mutex mutex;
public:
BasicObserver(CompleteResult completeResult);
~BasicObserver();
CompleteResult Complete(bool wasCanceled, std::shared_ptr<void> status);
bool WaitForCompletion(int timeoutInMs);
bool IsCompleted() const;
bool WasCanceled() const;
std::shared_ptr<void> GetStatus() const;
void Reset();
TYPEIDHASHFUNC(BasicObserver)
};
this is one implementation of an observer. it holds the mutex and implements the WaitForCompletion with the timeout. WaitForCompletion should block. when complete is being called its mutex should be unlocked. when the timeout runs WaitForCompletion returns false
BasicObserver::BasicObserver(CompleteResult completeResult):
isCompleted(false),
wasCanceled(false),
completeResult(completeResult)
{
std::thread createThread([this]{
this->mutex.lock();
});
createThread.join();
}
BasicObserver::~BasicObserver(){
}
CompleteResult BasicObserver::Complete(bool wasCanceled, std::shared_ptr<void> status){
this->wasCanceled = wasCanceled;
this->status = status;
isCompleted = true;
mutex.unlock();
return completeResult;
}
bool BasicObserver::WaitForCompletion(int timeoutInMs){
std::chrono::milliseconds time(timeoutInMs);
if(mutex.try_lock_for(time)){
mutex.unlock();
return true;
}else{
return false;
}
}
bool BasicObserver::IsCompleted() const{
return isCompleted;
}
bool BasicObserver::WasCanceled() const{
return wasCanceled;
}
std::shared_ptr<void> BasicObserver::GetStatus() const{
return status;
}
void BasicObserver::Reset(){
isCompleted = false;
wasCanceled = false;
status = 0;
std::chrono::milliseconds time(250);
mutex.try_lock_for(time); //if this fails it might be already resetted
}
//edit: solved by using a semaphore instead (sem_t from semaphore.h)
You could use a condation_variable, specifically wait_until or wait_for.
I would consider a redesign of your locking structure.
Why not have the lock held by the main thread, and when event x happens you unlock it. If you need to block for a duration I would just make the thread sleep.
Have all working threads blocking on the mutex trying to acquire the lock, if they need to run concurrently have them immediately release the lock once they acquire it.
maybe use a second mutex to emulate event x.
i want to setup the lock from thread 1 then start a thread 2 that
does something (wait for input from hardware in this case) and then
wait for the mutex in thread 1. thread 2 then unlocks the mutex when i
press the switch on the hardware. im using some kind of observer
pattern. so i have something observable where i add an observer to(in
this case the class A is the observer). at some point the observable
tells all added observers that it completed its task and thus unlocks
the mutex. as we have hardware here it could be that the hardware
locks up or a sensor doesnt work. so i NEED a timeout. – fredlllll 3
mins ago
EDIT - Maybe this would work?
Hold lock in thread 1, after thread 2 gets input block on that lock. Have thread 1 release the lock after timeout duration, maybe sleep a little to allow threads through then acquire the lock again. Have thread 2 release lock 1 then begin blocking on a second mutex after acquiring mutex 1, have hardware switch unlock mutex 2 which causes thread 2 to lock mutex2 then unlock mutex 2. Have hardware switch acquire mutex 2 again.
class MyClass
{
public:
void PushMessage(MyMessage m) // Thread 1 calls this
{
boost::mutex::scoped_lock lock(mMutex);
mQueue.push_back(m);
mCondition.notify_one();
}
MyMessage PopMessage()
{
boost::mutex::scoped_lock lock(mMutex);
while(mQueue.empty())
mCondition.wait(lock);
MyMessage message = mQueue.front();
mQueue.pop_front();
return message;
}
void foo() // thread 2 is running this loop, and supposed to get messages
{
for(;;)
{
MyMessage message = PopMessage();
do_something(message);
}
}
private:
std::deque<MyMessage> mQueue;
boost::mutex mMutex;
boost::condition mCondition;
};
When I run the code, PushMessage is called, and foo() is waiting on PopMessage(), but PopMessage never returns.
What does do_something here is not irrelevant I think.
What am I doing wrong here?
Strangely, the above code worked fine under mac, but I'm having trouble on linux.
boost version is 1.44.0
Thank you
Rather than letting the scope of the lock object expire before it unlocks, you could try to manually unlock the mutex in PushMessage() before you unblock the waiting thread, i.e.,
void PushMessage(MyMessage m) // Thread 1 calls this
{
boost::mutex::scoped_lock lock(mMutex);
mQueue.push_back(m);
lock.unlock(); // <== manually unlock
mCondition.notify_one();
}
That way when thread 2 unblocks, there will be no "cross-over" time where thread 1 contains the lock, and thread 2 is trying to obtain a lock on your mutex. I don't see why that would create problems, but again, at least you won't have thread 2 trying to call lock.lock() while thread 1 still contains the lock.
I think you need 2 mutex objects, one is for synchronizing method call in different threads, one is for condition wait. You mixed them.
I'm looking for a way to wait for multiple condition variables.
ie. something like:
boost::condition_variable cond1;
boost::condition_variable cond2;
void wait_for_data_to_process()
{
boost::unique_lock<boost::mutex> lock(mut);
wait_any(lock, cond1, cond2); //boost only provides cond1.wait(lock);
process_data();
}
Is something like this possible with condition variables. And if not are there alternative solutions?
Thanks
I don't believe you can do anything like this with boost::thread. Perhaps because POSIX condition variables don't allow this type of construct. Of course, Windows has WaitForMultipleObjects as aJ posted, which could be a solution if you're willing to restrict your code to Windows synchronization primitives.
Another option would to use fewer condition variables: just have 1 condition variable that you fire when anything "interesting" happens. Then, any time you want to wait, you run a loop that checks to see if your particular situation of interest has come up, and if not, go back to waiting on the condition variable. You should be waiting on those condition variables in such a loop anyways, as condition variable waits are subject to spurious wakeups (from boost::thread docs, emphasis mine):
void wait(boost::unique_lock<boost::mutex>& lock)
...
Effects:
Atomically call lock.unlock() and blocks the current thread. The thread will unblock when notified by a call to this->notify_one() or this->notify_all(), or spuriously. ...
As Managu already answered, you can use the same condition variable and check for multiple "events" (bool variables) in your while loop. However, concurrent access to these bool variables must be protected using the same mutex that the condvar uses.
Since I already went through the trouble of typing this code example for a related question, I'll repost it here:
boost::condition_variable condvar;
boost::mutex mutex;
bool finished1 = false;
bool finished2 = false;
void longComputation1()
{
{
boost::lock_guard<boost::mutex> lock(mutex);
finished1 = false;
}
// Perform long computation
{
boost::lock_guard<boost::mutex> lock(mutex);
finished1 = true;
}
condvar.notify_one();
}
void longComputation2()
{
{
boost::lock_guard<boost::mutex> lock(mutex);
finished2 = false;
}
// Perform long computation
{
boost::lock_guard<boost::mutex> lock(mutex);
finished2 = true;
}
condvar.notify_one();
}
void somefunction()
{
// Wait for long computations to finish without "spinning"
boost::lock_guard<boost::mutex> lock(mutex);
while(!finished1 && !finished2)
{
condvar.wait(lock);
}
// Computations are finished
}
alternative solutions?
I am not sure of Boost library but you can use WaitForMultipleObjects Function to wait for multiple kernel objects. Just check if this helps.
As Managu points out using multiple conditions might not be a good solution in the first place. What you want to do should be possible to be implemented using Semaphores.
Using the same condition variable for multiple events technically works, but it doesn't allow encapsulation. So I had an attempt at making a class that supports it. Not tested yet! Also it doesn't support notify_one() as I haven't worked out how to implement that.
#pragma once
#include <condition_variable>
#include <unordered_set>
// This is like a `condition_variable` but you can wait on multiple `multi_condition_variable`s.
// Internally it works by creating a new `condition_variable` for each `wait_any()` and registering
// it with the target `multi_condition_variable`s. When `notify_all()` is called, the main `condition_variable`
// is notified, as well as all the temporary `condition_variable`s created by `wait_any()`.
//
// There are two caveats:
//
// 1. You can't call the destructor if any threads are `wait()`ing. This is difficult to get around but
// it is the same as `std::wait_condition` anyway.
//
// 2. There is no `notify_one()`. You can *almost* implement this, but the only way I could think to do
// it was to add an `atomic_int` that indicates the number of waits(). Unfortunately there is no way
// to atomically increment it, and then wait.
class multi_condition_variable
{
public:
multi_condition_variable()
{
}
// Note that it is only safe to invoke the destructor if no thread is waiting on this condition variable.
~multi_condition_variable()
{
}
// Notify all threads calling wait(), and all wait_any()'s that contain this instance.
void notify_all()
{
_condition.notify_all();
for (auto o : _others)
o->notify_all();
}
// Wait for notify_all to be called, or a spurious wake-up.
void wait(std::unique_lock<std::mutex>& loc)
{
_condition.wait(loc);
}
// Wait for any of the notify_all()'s in `cvs` to be called, or a spurious wakeup.
static void wait_any(std::unique_lock<std::mutex>& loc, std::vector<std::reference_wrapper<multi_condition_variable>> cvs)
{
std::condition_variable c;
for (multi_condition_variable& cv : cvs)
cv.addOther(&c);
c.wait(loc);
for (multi_condition_variable& cv : cvs)
cv.removeOther(&c);
}
private:
void addOther(std::condition_variable* cv)
{
std::lock_guard<std::mutex> lock(_othersMutex);
_others.insert(cv);
}
void removeOther(std::condition_variable* cv)
{
// Note that *this may have been destroyed at this point.
std::lock_guard<std::mutex> lock(_othersMutex);
_others.erase(cv);
}
// The condition variable.
std::condition_variable _condition;
// When notified, also notify these.
std::unordered_set<std::condition_variable*> _others;
// Mutex to protect access to _others.
std::mutex _othersMutex;
};
// Example use:
//
// multi_condition_variable cond1;
// multi_condition_variable cond2;
//
// void wait_for_data_to_process()
// {
// unique_lock<boost::mutex> lock(mut);
//
// multi_condition_variable::wait_any(lock, {cond1, cond2});
//
// process_data();
// }