Is there a way to have a boost mutex throw an exception on any waiting threads? I have a problem where an object is deleted but do to the nature of the software library it is possible threads are still waiting at a mutex within the object and a rather nasty exception is thrown when the mutex is closed. I guess I could use a multiple mutex counter but that could cause performance degradation. What I'd like to have happen is the mutex throw an exception on any threads that are waiting when it is closed so that the stack is unrolled. Is there a clean way to do this that is platform-independent?
Such a concept of a mutex that throws when it is destroyed seems innocuous enough, but when it comes time to implement it, it reveals a flaw in how you are thinking about mutexes.
Let's take a look at some example code to get an idea of the pitfalls of such an approach.
Note: Please do not use the code below, it will cause nothing but endless hours of torment and suffering debugging synchronization problems.
class throwing_mutex
{
private:
mutex m_;
condition_variable cv_;
bool destroyed_;
bool locked_;
public:
void lock()
{
std::unique_lock<std::mutex> lock(m_);
cv_.wait(lock, [&]() {return !locked_ || destroyed_;}); // Wait until the mutex is unlocked or destroyed.
if (destroyed_) throw runtime_error("The exception was terminated while waiting.");
locked_ = true;
}
void unlock()
{
std::unique_lock<std::mutex> lock(m_);
locked_ = false;
lock.unlock();
cv_.notify_one();
}
~throwing_mutex()
{
std::unique_lock<std::mutex> lock(m_);
destroyed_ = true;
lock.unlock();
cv_.notify_all(); // Let all waiters know we are dead.
}
};
Upon destruction, everyone waiting on the throwing_mutex will have an exception thrown. But this opens up a pretty big race condition.
We've handled the case where everyone is waiting for the mutex -- they will safely throw. What we have not handled is the case where anyone is on their way to calling lock() but not quite there yet. When they finally get to the point where they can call lock(), the throwing_mutex has already been destroyed. The bug we've just introduced by means of our flawed methodology is called use-after-free. If we are lucky, the error will present itself early and clearly, but sometimes we aren't so lucky and we will be tormented for hours or days. There is no way that our throwing_mutex class can ever solve that problem and any code that would need such a class does not have well thought out ownership semantics.
So, how do we solve this problem if it isn't by means of a mutex that throws? We fix the lifetime of the mutex and the object[s] that are locked by it.
Presumably, this is mutex is a member of a class. If that is the case, it means delaying destruction until everyone who depends on the object is done with it. This is conveyed with the use of a shared_ptr. Without getting into the nitty-gritty of ownership-semantics, that is the best this can be answered. Hopefully I've changed your way of thinking of the problem enough to stray you away from your original plan and toward something that will work more reliably.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 1 year ago.
Improve this question
Youtube details
I have been browsing youtube to try and develop my understanding of C++ multithread support with mutex and condition variables.
I came across this video. Skip to time 6:30 to see what I am currently looking at. (A page of code.)
https://www.youtube.com/watch?v=eh_9zUNmTig
I believe there is a mistake in the code, but I wanted to check. It could just as well be that I don't understand something.
Question
The author states that std::unique_lock locks the mutex on creation. Meaning that there is no need to call
unique_lock<mutex> lock(m)
lock.lock(); // this is wrong, because unique_lock already locked the mutex
after creating a unique_lock object.
I assume although I do not know for certain, that unique_lock will release the mutex lock on destruction. (Aka when it goes out of scope.)
Can it also be unlocked manually by calling
lock.unlock()
? From the documentation it appears there is no such unlock function. It looks like unique_lock is therefore the same as scoped_lock? But again, I'm assuming this isn't the case and there's some other information I am missing.
Continuing... The author has a function which looks like this:
void addMoney(int money)
{
std::lock_guard<mutex> lg(m); // lock_guard being used interchangably with unique_lock - why?
balance += money; // adding to global variable
cv.notify_one(); // error here
// the lock_guard is still in scope
// the mutex is still locked
// calling notify_one() may cause the sleeping thread to wake up
// check if the mutex is still locked (which it might be if the
// destructor for lg hasn't finished running)
// and then go back to sleep
// meaning this line of code may have no effect
// it is undefined behaviour
}
I have anotated where I believe there is an error. I think this function causes undefined behaviour, because the lock_guard is still in scope, and therefore the mutex might be locked.
Effectively it is a race condition:
If addMoney() ends before the other function begins, we are ok
If the other function withdrawMoney() checks the lock (cv.wait()) before addMoney() exits then the program breaks, and remains in a locked state
For completeness here is the other function:
void withdrawMoney(int money)
{
std::unique_lock<mutex> ul(m); // unique_lock instead of scoped_lock? why?
cv.wait(ul, []{return balance != 0;});
// some more stuff omitted
}
Summary
There are a couple of points I have raised
Most importantly the race condition
Of secondary importance, why are two different things (lock_guard and unique_lock) being used to do what appears to be the same thing (performing the same function)
That comment
// calling notify_one() may cause the sleeping thread to wake up
// check if the mutex is still locked (which it might be if the
// destructor for lg hasn't finished running)
// and then go back to sleep
is incorrect. There are two separate control mechanisms here: the condition variable and the mutex. Waking up on a notification to a condition variable means, simply, waking up. After waking up, the thread blocks waiting for the mutex. When the mutex is released by the thread that called notify_one(), the blocked thread (or perhaps some other thread, but eventually, the blocked thread) gets the mutex and continues execution. It does not go back to waiting for the condition variable.
For some more explanation on std::unique_lock vs std::lock_guard, see this question.
There is no undefined behavior when sending the notification while the mutex is still locked. It might cause an unnecessary thread-switch especially if the receiving thread has a higher priority, but this is just a small performance hit. It is also not needed to have the mutex locked to send the notification, so the function may be written as:
void addMoney(int money)
{
{
std::lock_guard<mutex> lg(m);
balance += money;
}
cv.notify_one();
}
You have to make sure that the resources for the condition are protected when changing and when checked.
Someone please help me solve deadlock issues in c++ if possible with reference or examples.
Scenario would be like below.
Thread1 is locked by mutex and doing some operation, thread2 and thread3 are in waiting state for thread1 to unlock to access the resource.
Some abort/unexpected thing happened -- thread1 was terminated and didn't get the unlock, thread2 and thread3 are still waiting.
How to save the main thread (mean nothing should happen to main thread) in such situations.
Please throw some light how to solve such issues in c++.
Thanks,
Sheik
Some abort/unexpected thing happened
Use s.th. like std::lock_guard to prevent 'hanging' locks due to exceptions or forgotten/unexpected, but necessary unlock() operations.
The principle is pretty simple and you can easily implement it for any mechanism that uses a pair of methods that correspond together in a 'lock/unlock' manner:
class LockObject // E.g. mutex or alike
{
public:
// ...
void lock();
void unlock();
};
Bind the guard classes constructor to a reference to the lock object's instance and call lock() in the constructor and unlock() in the destructor:
template<typename T>
class LockGuard
{
public:
LockGuard(T& lockObject)
: lockObject_(lockObject)
{
lockObject_.lock();
}
~LockGuard()
{
lockObject_.unlock();
}
private:
T& lockObject_;
};
Use LockGuard like this:
// Some scope providing 'LockObject lockObject'
{ LockGuard<LockObject> lock(lockObject)
// Do s.th. when lockObject is locked
} // Call of lockObject.unlock() is guaranteed at least here, no matter what
// (exception, goto, break, etc.) caused leaving the block's scope.
Generally threads should not terminate unexpectedly. You may try using try/catch blocks. If you still want to free resources when a thread terminates unexpectedly, you may create a monitor thread that waits for the termination of the first thread.
On Windows, you can use something as ::WaitForSingleObject(m_htThread, INFINITE).
Once the 1st thread had been terminated, you may proceed with freeing abandoned locks.
Maybe you'll want to add some flag which indicates if the termination was graceful.
You'll probably also have to remember which thread is locking which object.
As said, I wouldn't recommend using such method, but on extreme cases.
The way to solve deadlocks in any language or platform is always the same.
Always acquire the locks in the same order.
EDIT: However you have misdescribed your problem. This is not a deadlock. A deadlock is a circular chain of locks. This is simply an unreleased lock, i.e. a lock leak. The solution is the same as any other resource leak: don't. In C++ that means releasing resources in destructors, and ensuring that destructors are called. Somehow your thread has terminated without doing that. Find that problem and fix it.
I have some code which need to be thread safe and exception safe. The code below is a very simplified version of my problem :
#include <mutex>
#include <thread>
std::mutex mutex;
int n=0;
class Counter{
public:
Counter(){
std::lock_guard<std::mutex>guard(mutex);
n++;}
~Counter(){
std::lock_guard<std::mutex>guard(mutex);//How can I protect here the underlying code to mutex.lock() ?
n--;}
};
void doSomething(){
Counter counter;
//Here I could do something meaningful
}
int numberOfThreadInDoSomething(){
std::lock_guard<std::mutex>guard(mutex);
return n;}
I have a mutex that I need to lock in the destructor of an object. The problem is that my destructor should not throw exceptions.
What can I do ?
0) I cannot replace n with an atomic variable (of course it would do the trick here but that is not the point of my question)
1) I could replace my mutex with a spin lock
2) I could try and catch the locking into an infinite loop until I eventualy acquire the lock with no exception raised
None of those solution seems very appealing. Did you have the same problem ? How did you solve it ?
As suggested by Adam H. Peterson, I finally decided to write a no throw mutex :
class NoThrowMutex{
private:
std::mutex mutex;
std::atomic_flag flag;
bool both;
public:
NoThrowMutex();
~NoThrowMutex();
void lock();
void unlock();
};
NoThrowMutex::NoThrowMutex():mutex(),flag(),both(false){
flag.clear(std::memory_order_release);}
NoThrowMutex::~NoThrowMutex(){}
void NoThrowMutex::lock(){
try{
mutex.lock();
while(flag.test_and_set(std::memory_order_acquire));
both=true;}
catch(...){
while(flag.test_and_set(std::memory_order_acquire));
both=false;}}
void NoThrowMutex::unlock(){
if(both){mutex.unlock();}
flag.clear(std::memory_order_release);}
The idea is to have two mutex instead of only one. The real mutex is the spin mutex implemented with an std::atomic_flag. This spin mutex is protected by a std::mutex which could throw.
In a normal situation, the standard mutex is acquired and the flag is set with a cost of only one atomic operation. If the standard mutex cannot be locked immediately, the thread is going to sleep.
If for any reason the standard mutex throws, the mutex will enter its spin mode. The thread where the exception occured will then loop until it can set the flag. As no other thread is aware that this thread bybassed completely the standard mutex, they could spin too.
In the worst case scenario, this locking mechanism degrades to a spin lock. Most of the time it reacts just like a normal mutex.
This is a bad situation to be in. Your destructor is doing something that might fail. If failure to update this counter will irrecoverably corrupt your application, you may want to simply let the destructor throw. This will crash your application with a call to terminate, but if your application is corrupted, it may be better to kill the process and rely on some higher-level recovery scheme (such as a watchdog for a daemon or retrying execution for another utility). If failure to decrement the counter is recoverable, you should absorb the exception with a try{}catch() block and recover (or potentially save information for some other operation to eventually recover). If it's not recoverable, but it's not fatal, you may want to catch and absorb the exception and log the failure (being sure to log in an exception-safe way, of course).
It would be ideal if the code could be restructured such that the destructor doesn't do anything that can't fail. However, if your code is otherwise correct, failure while acquiring a lock is probably rare except in cases of constrained resources, so either absorbing or aborting on failure may very well be acceptable. For some mutexes, lock() is probably a no-throw operation (such as a spinlock using atomic_flag), and if you can use such a mutex, you can expect lock_guard to never throw. Your only worry in that situation would be deadlock.
I am a bit confused about the role of std::unique_lock when working with std::condition_variable. As far as I understood the documentation, std::unique_lock is basically a bloated lock guard, with the possibility to swap the state between two locks.
I've so far used pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex) for this purpose (I guess that's what the STL uses on posix). It takes a mutex, not a lock.
What's the difference here? Is the fact that std::condition_variable deals with std::unique_lock an optimization? If so, how exactly is it faster?
so there is no technical reason?
I upvoted cmeerw's answer because I believe he gave a technical reason. Let's walk through it. Let's pretend that the committee had decided to have condition_variable wait on a mutex. Here is code using that design:
void foo()
{
mut.lock();
// mut locked by this thread here
while (not_ready)
cv.wait(mut);
// mut locked by this thread here
mut.unlock();
}
This is exactly how one shouldn't use a condition_variable. In the regions marked with:
// mut locked by this thread here
there is an exception safety problem, and it is a serious one. If an exception is thrown in these areas (or by cv.wait itself), the locked state of the mutex is leaked unless a try/catch is also put in somewhere to catch the exception and unlock it. But that's just more code you're asking the programmer to write.
Let's say that the programmer knows how to write exception safe code, and knows to use unique_lock to achieve it. Now the code looks like this:
void foo()
{
unique_lock<mutex> lk(mut);
// mut locked by this thread here
while (not_ready)
cv.wait(*lk.mutex());
// mut locked by this thread here
}
This is much better, but it is still not a great situation. The condition_variable interface is making the programmer go out of his way to get things to work. There is a possible null pointer dereference if lk accidentally does not reference a mutex. And there is no way for condition_variable::wait to check that this thread does own the lock on mut.
Oh, just remembered, there is also the danger that the programmer may choose the wrong unique_lock member function to expose the mutex. *lk.release() would be disastrous here.
Now let's look at how the code is written with the actual condition_variable API that takes a unique_lock<mutex>:
void foo()
{
unique_lock<mutex> lk(mut);
// mut locked by this thread here
while (not_ready)
cv.wait(lk);
// mut locked by this thread here
}
This code is as simple as it can get.
It is exception safe.
The wait function can check lk.owns_lock() and throw an exception if it is false.
These are technical reasons that drove the API design of condition_variable.
Additionally, condition_variable::wait doesn't take a lock_guard<mutex> because lock_guard<mutex> is how you say: I own the lock on this mutex until lock_guard<mutex> destructs. But when you call condition_variable::wait, you implicitly release the lock on the mutex. So that action is inconsistent with the lock_guard use case / statement.
We needed unique_lock anyway so that one could return locks from functions, put them into containers, and lock/unlock mutexes in non-scoped patterns in an exception safe way, so unique_lock was the natural choice for condition_variable::wait.
Update
bamboon suggested in the comments below that I contrast condition_variable_any, so here goes:
Question: Why isn't condition_variable::wait templated so that I can pass any Lockable type to it?
Answer:
That is really cool functionality to have. For example this paper demonstrates code that waits on a shared_lock (rwlock) in shared mode on a condition variable (something unheard of in the posix world, but very useful nonetheless). However the functionality is more expensive.
So the committee introduced a new type with this functionality:
`condition_variable_any`
With this condition_variable adaptor one can wait on any lockable type. If it has members lock() and unlock(), you are good to go. A proper implementation of condition_variable_any requires a condition_variable data member and a shared_ptr<mutex> data member.
Because this new functionality is more expensive than your basic condition_variable::wait, and because condition_variable is such a low level tool, this very useful but more expensive functionality was put into a separate class so that you only pay for it if you use it.
It's essentially an API design decision to make the API as safe as possible by default (with the additional overhead being seen as negligible). By requiring to pass a unique_lock instead of a raw mutex users of the API are directed towards writing correct code (in the presence of exceptions).
In recent years the focus of the C++ language has shifted towards making it safe by default (but still allowing users to shoot themselves into their feet if they want to and try hard enough).
In the following blogpost:
http://www.justsoftwaresolutions.co.uk/threading/implementing-a-thread-safe-queue-using-condition-variables.html
There is a 'push' method defined as follows:
void push(Data const& data)
{
boost::mutex::scoped_lock lock(the_mutex);
the_queue.push(data);
lock.unlock();
the_condition_variable.notify_one();
}
My questions are:
why is there an explicit 'lock.unlock()' being called upon the scoped_lock variable?
What is its purpose?
Can it be safely removed, resulting in the 'notify_one' method call be within the scope of the scoped_mutex?
The unlock is not necessary. It might reduce the time that the mutex is locked for slightly, however.
Keeping it or removing it will not affect thread safety or result in deadlocks.
EDIT: As the article mentions, however, leaving the unlock in there can result in less contention on the mutex. You might as well leave it in. Alternatively, use scope around the mutex, which I personally find highlights the scope of the mutex better if glancing at the code.
The explicit lock is so the that waiting thread isn't woken up by the notification only to have to block on the mutex. This is explained earlier in the article that you linked to.
The reason to still use a scoped_lock is to ensure that the mutex is properly unlocked in case the push fails and throws an exception.
Points 1 and 2 can be answered by the principle that you should hold your locks for the least time possible. As soon as the data has been pushed onto the queue, you no longer need to lock it.
You can remove the unlock as you've suggested, at the expense of keeping the lock in place for longer.
Alternatively, the unlock itself can be removed by adding a new scope, like so:
void push(Data const& data)
{
{ // new scope
boost::mutex::scoped_lock lock(the_mutex);
the_queue.push(data);
} // scope ends, lock gets destroyed
the_condition_variable.notify_one();
}