There is a set of problems in which mutexes themselves (without additional stuff like condition variables) can be used to synchronize threads.
For example, let's say I want a background thread to perform some possibly-time-consuming initialization and then to perform its main job, but the main job should start not earlier than I give a signal from the main thread:
std::thread backgroundThread ([]() {
initialize ();
waitForSignalQ ();
doMainJob ();
});
…;
emitSignalQ ();
Yes, I know that I can use mutex + boolean variable + condition variable to implement waitForSignalQ and emitSignalQ. Or, I can use void-returning future + void-returning promice for the same purpose, as Scott Meyers suggests.
But the simpler approach here seems to be using just a single mutex:
std::mutex myMutex;
std::unique_lock <std::mutex> backgroundThreadCantStartItsMainJob (myMutex);
std::thread backgroundThread ([&myMutex]() {
initialize ();
// This line won't return until myMutex is unlocked:
std::lock_quard <std::mutex> (myMutex);
doMainJob ();
});
…;
// This line will unlock myMutex:
backgroundThreadCantStartItsMainJob.unlock();
But is this approach valid? Or are there some drawbacks (for example: OS-level mutex-locking time limit, false positives from software analysis tools, etc)?
P. S.:
• Yes, I know that the problems will arise if myMutex isn't unlocked (due to an exception before backgroundThreadCantStartItsMainJob.unlock() or etc), but the question is not about that (with condvar and future approaches we get the same problem if the main thread "forgets" to emit the signal Q).
• Yes, I know that in practice it might make sense to have several variants of signal Q (like "proceed to the main job" vs. "cancel"), the question is not about that (as well as the condvar and future approaches, the discussed approach allows passing extra data too).
There's no specific downsides to locking a mutex for a long time.
If I understood correctly, you have a thread that will lock a mutex during its init. Another thread will try to get that mutex (to be sure the init of the other thread is finished) and then get the mutex for the rest of the program runtime.
That is fine. There's no runtime cost with having the mutex. The OS will not time it out or anything. The challenge is a software engineering one: Make it clear, readable and understandable for the person that will have to maintain it. Apart from that, the architecture is really up to you.
Related
I apologise in advance if my question is a duplicate, but I was not able to find a satisfying answer to my question.
I am dealing with the following (maybe silly) issue: I am trying to synchronise two threads (A and B), and I want to block the thread A until a condition is set to true by the thread B.
The "special" thing is that the condition is checked on a thread-safe object (for instance, let's consider it to be a std::atomic_bool).
My naive approach was the following:
// Shared atomic object
std::atomic_bool condition{false};
// Thread A
// ... does something
while(!condition.load()) ; // Do nothing
// Condition is met, proceed with the job
// Thread B
// ... does something
condition.store(true); // Unlock Thread A
but, as far as I have understood, the while implies an active wait which is undesirable.
So, I thought about having a small sleep_for as the body of the while to reduce the frequency of the active wait, but then the issue becomes finding the right waiting time that does not cause waste of time in case the condition unlocks while thread A is sleeping and, at the same time, does not make the loop to execute too often.
My feeling is that this is very much dependant on the time that thread B spends before setting the condition to true, which may be not predictable.
Another solution I have found looking on other SO topics is to use a condition variable, but that would require the introduction of a mutex that is not really needed.
I am perhaps overthinking the problem, but I'd like to know if there are alternative "standard" solutions to follow (bearing in mind that I am limited to C++11), and what would be the best approach in general.
Many thanks in advance for the help.
Your use case is simple and there are many ways to implement that.
The first recommendation would be to make use of condition variable. But it
seems from your question that you would like to avoid that because of mutex.
I don't have any profiling data for your use case, but mutex isn't costly for your use case.
In a multi-threaded environment, at some point of time, you would need some techniques to protect shared access and modification of data. You would probably need mutexes for that.
You could go for condition variable approach.
It is by the standard, and it also provides function to notify all the threads as well, if your use case scales in future.
Also, as you mentioned about "time", condition_variable also comes with variations of wait* functions where the condition could be in terms of "time". It can wait_for or wait_until a certain time as well.
About the while loop and a sleep_for approach, blocking a thread from execution and then rescheduling it again isn't that cheap if we are counting in terms of milliseconds. The condition variable approach would be better suited in this case, rather than having the while loop and an explicit call to sleep_for.
Sorry, condition variables are the way to go here.
The mutex is being used as a part of the condition variable, not as a traditional mutex. And barring some strange priority inversion situation, it shouldn't have much cost.
Here is a simple "farm gate". It starts shut, and can be opened. Once opened, it can never be shut again.
struct gate {
void open_gate() {
auto l = lock();
gate_is_open = true;
cv.notify_all();
}
void wait_on_gate() const {
auto l = lock();
cv.wait(l, [&]{ return gate_is_open; });
}
private:
auto lock() const { return std::unique_lock{m}; }
mutable std::mutex m;
bool gate_is_open = false;
std::condition_variable cv;
};
which you'd use like this:
// Shared gate
gate condition;
// Thread A
// ... does something
condition.wait_on_gate(); // Do nothing
// Condition is met, proceed with the job
// Thread B
// ... does something
condition.open_gate(); // Unlock Thread A
and there we have it.
In c++20 there is std::latch. Start the counter at 1, decrement it when the gate opens, and the other thread waits on the latch.
How about using some sort of a sentinel value to check if the conditions of thread B are true to unlock thread A and synchronize both of them once the condition is met.
I am trying to use read/write lock in C++ using shared_mutex
typedef boost::shared_mutex Lock;
typedef boost::unique_lock< Lock > WriteLock;
typedef boost::shared_lock< Lock > ReadLock;
class Test {
Lock lock;
WriteLock writeLock;
ReadLock readLock;
Test() : writeLock(lock), readLock(lock) {}
readFn1() {
readLock.lock();
/*
Some Code
*/
readLock.unlock();
}
readFn2() {
readLock.lock();
/*
Some Code
*/
readLock.unlock();
}
writeFn1() {
writeLock.lock();
/*
Some Code
*/
writeLock.unlock();
}
writeFn2() {
writeLock.lock();
/*
Some Code
*/
writeLock.unlock();
}
}
The code seems to be working fine but I have a few conceptual questions.
Q1. I have seen the recommendations to use unique_lock and shared_lock on http://en.cppreference.com/w/cpp/thread/shared_mutex/lock, but I don't understand why because shared_mutex already supports lock and lock_shared methods?
Q2. Does this code have the potential to cause write starvation? If yes then how can I avoid the starvation?
Q3. Is there any other locking class I can try to implement read write lock?
Q1: use of a mutex wrapper
The recommendation to use a wrapper object instead of managing the mutex directly is to avoid unfortunate situation where your code is interrupted and the mutex is not released, leaving it locked forever.
This is the principle of RAII.
But this only works if your ReadLock or WriteLock are local to the function using it.
Example:
readFn1() {
boost::unique_lock< Lock > rl(lock);
/*
Some Code
==> imagine exception is thrown
*/
rl.unlock(); // this is never reached if exception thrown
} // fortunately local object are destroyed automatically in case
// an excpetion makes you leave the function prematurely
In your code this won't work if one of the function is interupted, becaus your ReadLock WriteLock object is a member of Test and not local to the function setting the lock.
Q2: Write starvation
It is not fully clear how you will invoke the readers and the writers, but yes, there is a risk:
as long as readers are active, the writer is blocked by the unique_lock waiting for the mutex to be aquirable in exclusive mode.
however as long as the wrtier is waiting, new readers can obtain access to the shared lock, causing the unique_lock to be further delayed.
If you want to avoid starvation, you have to ensure that waiting writers do get the opportunity to set their unique_lock. For example att in your readers some code to check if a writer is waiting before setting the lock.
Q3 Other locking classes
Not quite sure what you're looking for, but I have the impression that condition_variable could be of interest for you. But the logic is a little bit different.
Maybe, you could also find a solution by thinking out of the box: perhaps there's a suitable lock-free data structure that could facilitate coexistance of readers and writers by changing slightly the approach ?
The types for the locks are ok but instead of having them as member functions create then inside the member functions locktype lock(mymutex). That way they are released on destruction even in the case of an exception.
Q1. I have seen the recommendations to use unique_lock and shared_lock on http://en.cppreference.com/w/cpp/thread/shared_mutex/lock, but I don't understand why because shared_mutex already supports lock and lock_shared methods?
Possibly because unique_lock has been around since c++11 but shared_lock is coming onboard with c++17. Also, [possibly] unique_lock can be more efficient. Here's the original rationale for shared_lock [by the creator] http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2406.html and I defer to that.
Q2. Does this code have the potential to cause write starvation? If yes then how can I avoid the starvation?
Yes, absolutely. If you do:
while (1)
writeFn1();
You can end up with a time line of:
T1: writeLock.lock()
T2: writeLock.unlock()
T3: writeLock.lock()
T4: writeLock.unlock()
T5: writeLock.lock()
T6: writeLock.unlock()
...
The difference T2-T1 is arbitrary, based on amount of work being done. But, T3-T2 is near zero. This is the window for another thread to acquire the lock. Because the window is so small, it probably won't get it.
To solve this, the simplest method is to insert a small sleep (e.g. nanosleep) between T2 and T3. You could do this by adding it to the bottom of writeFn1.
Other methods can involve creating a queue for the lock. If a thread can't get the lock, it adds itself to a queue and the first thread on the queue gets the lock when the lock is released. In the linux kernel, this is implemented for a "queued spinlock"
Q3. Is there any other locking class I can try to implement read write lock?
While not a class, you could use pthread_mutex_lock and pthread_mutex_unlock. These implement recursive locks. You could add your own code to implement the equivalent of boost::scoped_lock. Your class can control the semantics.
Or, boost has its own locks.
I am a bit confused about the role of std::unique_lock when working with std::condition_variable. As far as I understood the documentation, std::unique_lock is basically a bloated lock guard, with the possibility to swap the state between two locks.
I've so far used pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex) for this purpose (I guess that's what the STL uses on posix). It takes a mutex, not a lock.
What's the difference here? Is the fact that std::condition_variable deals with std::unique_lock an optimization? If so, how exactly is it faster?
so there is no technical reason?
I upvoted cmeerw's answer because I believe he gave a technical reason. Let's walk through it. Let's pretend that the committee had decided to have condition_variable wait on a mutex. Here is code using that design:
void foo()
{
mut.lock();
// mut locked by this thread here
while (not_ready)
cv.wait(mut);
// mut locked by this thread here
mut.unlock();
}
This is exactly how one shouldn't use a condition_variable. In the regions marked with:
// mut locked by this thread here
there is an exception safety problem, and it is a serious one. If an exception is thrown in these areas (or by cv.wait itself), the locked state of the mutex is leaked unless a try/catch is also put in somewhere to catch the exception and unlock it. But that's just more code you're asking the programmer to write.
Let's say that the programmer knows how to write exception safe code, and knows to use unique_lock to achieve it. Now the code looks like this:
void foo()
{
unique_lock<mutex> lk(mut);
// mut locked by this thread here
while (not_ready)
cv.wait(*lk.mutex());
// mut locked by this thread here
}
This is much better, but it is still not a great situation. The condition_variable interface is making the programmer go out of his way to get things to work. There is a possible null pointer dereference if lk accidentally does not reference a mutex. And there is no way for condition_variable::wait to check that this thread does own the lock on mut.
Oh, just remembered, there is also the danger that the programmer may choose the wrong unique_lock member function to expose the mutex. *lk.release() would be disastrous here.
Now let's look at how the code is written with the actual condition_variable API that takes a unique_lock<mutex>:
void foo()
{
unique_lock<mutex> lk(mut);
// mut locked by this thread here
while (not_ready)
cv.wait(lk);
// mut locked by this thread here
}
This code is as simple as it can get.
It is exception safe.
The wait function can check lk.owns_lock() and throw an exception if it is false.
These are technical reasons that drove the API design of condition_variable.
Additionally, condition_variable::wait doesn't take a lock_guard<mutex> because lock_guard<mutex> is how you say: I own the lock on this mutex until lock_guard<mutex> destructs. But when you call condition_variable::wait, you implicitly release the lock on the mutex. So that action is inconsistent with the lock_guard use case / statement.
We needed unique_lock anyway so that one could return locks from functions, put them into containers, and lock/unlock mutexes in non-scoped patterns in an exception safe way, so unique_lock was the natural choice for condition_variable::wait.
Update
bamboon suggested in the comments below that I contrast condition_variable_any, so here goes:
Question: Why isn't condition_variable::wait templated so that I can pass any Lockable type to it?
Answer:
That is really cool functionality to have. For example this paper demonstrates code that waits on a shared_lock (rwlock) in shared mode on a condition variable (something unheard of in the posix world, but very useful nonetheless). However the functionality is more expensive.
So the committee introduced a new type with this functionality:
`condition_variable_any`
With this condition_variable adaptor one can wait on any lockable type. If it has members lock() and unlock(), you are good to go. A proper implementation of condition_variable_any requires a condition_variable data member and a shared_ptr<mutex> data member.
Because this new functionality is more expensive than your basic condition_variable::wait, and because condition_variable is such a low level tool, this very useful but more expensive functionality was put into a separate class so that you only pay for it if you use it.
It's essentially an API design decision to make the API as safe as possible by default (with the additional overhead being seen as negligible). By requiring to pass a unique_lock instead of a raw mutex users of the API are directed towards writing correct code (in the presence of exceptions).
In recent years the focus of the C++ language has shifted towards making it safe by default (but still allowing users to shoot themselves into their feet if they want to and try hard enough).
I understand how to use condition variables (crummy name for this construct, IMO, as the cv object neither is a variable nor indicates a condition). So I have a pair of threads, canonically set up with Boost.Thread as:
bool awake = false;
boost::mutex sync;
boost::condition_variable cv;
void thread1()
{
boost::unique_lock<boost::mutex> lock1(sync);
while (!awake)
cv.wait(lock1);
lock1.unlock(); // this line actually not canonical, but why not?
// proceed...
}
void thread2()
{
//...
boost::unique_lock<boost::mutex> lock2;
awake = true;
lock2.unlock();
cv.notify_all();
}
My question is: does thread2 really need to be protecting the assignment to awake? It seems to me the notify_all() call should be sufficient. If the data being manipulated and checked against were more than a simple "ok to proceed" flag, I see the value in the mutex, but here it seems like overkill.
A secondary question is that asked in the code fragment: Why doesn't the Boost documentation show the lock in thread1 being unlocked before the "process data" step?
EDIT: Maybe my question is really: Is there a cleaner construct than a CV to implement this kind of wait?
does thread2 really need to be protecting the assignment to awake?
Yes. Modifying an object from one thread and accessing it from another without synchronisation gives undefined behaviour. Even if it's just a bool.
For example, on some multiprocessor systems the write might only affect local memory; without an explicit synchronisation operation, other threads might never see the change.
Why doesn't the Boost documentation show the lock in thread1 being unlocked before the "process data" step?
If you unlocked the mutex before clearing the flag, then you might miss another signal.
Is there a cleaner construct than a CV to implement this kind of wait?
In Boost and the standard C++ library, no; a condition variable is flexible enough to handle arbitrary shared state and not particularly over-complicated for this simple case, so there's no particular need for anything simpler.
More generally, you could use a semaphore or a pipe to send a simple signal between threads.
Formally, you definitely need the lock in both threads: if any thread
modifies an object, and more than one thread accesses it, then all
accesses must be synchronized.
In practice, you'll probably get away with it without the lock; it's
almost certain that notify_all will issue the necessary fence or
membar instructions to ensure that the memory is properly synchronized.
But why take the risk?
As to the absense of the unlock, that's the whole point of the scoped
locking pattern: the unlock is in the destructor of the object, so
that the mutex will be unlocked even if an exception passes through.
I'm using the C++ boost::thread library, which in my case means I'm using pthreads. Officially, a mutex must be unlocked from the same thread which locks it, and I want the effect of being able to lock in one thread and then unlock in another. There are many ways to accomplish this. One possibility would be to write a new mutex class which allows this behavior.
For example:
class inter_thread_mutex{
bool locked;
boost::mutex mx;
boost::condition_variable cv;
public:
void lock(){
boost::unique_lock<boost::mutex> lck(mx);
while(locked) cv.wait(lck);
locked=true;
}
void unlock(){
{
boost::lock_guard<boost::mutex> lck(mx);
if(!locked) error();
locked=false;
}
cv.notify_one();
}
// bool try_lock(); void error(); etc.
}
I should point out that the above code doesn't guarantee FIFO access, since if one thread calls lock() while another calls unlock(), this first thread may acquire the lock ahead of other threads which are waiting. (Come to think of it, the boost::thread documentation doesn't appear to make any explicit scheduling guarantees for either mutexes or condition variables). But let's just ignore that (and any other bugs) for now.
My question is, if I decide to go this route, would I be able to use such a mutex as a model for the boost Lockable concept. For example, would anything go wrong if I use a boost::unique_lock< inter_thread_mutex > for RAII-style access, and then pass this lock to boost::condition_variable_any.wait(), etc.
On one hand I don't see why not. On the other hand, "I don't see why not" is usually a very bad way of determining whether something will work.
The reason I ask is that if it turns out that I have to write wrapper classes for RAII locks and condition variables and whatever else, then I'd rather just find some other way to achieve the same effect.
EDIT:
The kind of behavior I want is basically as follows. I have an object, and it needs to be locked whenever it is modified. I want to lock the object from one thread, and do some work on it. Then I want to keep the object locked while I tell another worker thread to complete the work. So the first thread can go on and do something else while the worker thread finishes up. When the worker thread gets done, it unlocks the mutex.
And I want the transition to be seemless so nobody else can get the mutex lock in between when thread 1 starts the work and thread 2 completes it.
Something like inter_thread_mutex seems like it would work, and it would also allow the program to interact with it as if it were an ordinary mutex. So it seems like a clean solution. If there's a better solution, I'd be happy to hear that also.
EDIT AGAIN:
The reason I need locks to begin with is that there are multiple master threads, and the locks are there to prevent them from accessing shared objects concurrently in invalid ways.
So the code already uses loop-level lock-free sequencing of operations at the master thread level. Also, in the original implementation, there were no worker threads, and the mutexes were ordinary kosher mutexes.
The inter_thread_thingy came up as an optimization, primarily to improve response time. In many cases, it was sufficient to guarantee that the "first part" of operation A, occurs before the "first part" of operation B. As a dumb example, say I punch object 1 and give it a black eye. Then I tell object 1 to change it's internal structure to reflect all the tissue damage. I don't want to wait around for the tissue damage before I move on to punch object 2. However, I do want the tissue damage to occur as part of the same operation; for example, in the interim, I don't want any other thread to reconfigure the object in such a way that would make tissue damage an invalid operation. (yes, this example is imperfect in many ways, and no I'm not working on a game)
So we made the change to a model where ownership of an object can be passed to a worker thread to complete an operation, and it actually works quite nicely; each master thread is able to get a lot more operations done because it doesn't need to wait for them all to complete. And, since the event sequencing at the master thread level is still loop-based, it is easy to write high-level master-thread operations, as they can be based on the assumption that an operation is complete (more precisely, the critical "first part" upon which the sequencing logic depends is complete) when the corresponding function call returns.
Finally, I thought it would be nice to use inter_thread mutex/semaphore thingies using RAII with boost locks to encapsulate the necessary synchronization that is required to make the whole thing work.
man pthread_unlock (this is on OS X, similar wording on Linux) has the answer:
NAME
pthread_mutex_unlock -- unlock a mutex
SYNOPSIS
#include <pthread.h>
int
pthread_mutex_unlock(pthread_mutex_t *mutex);
DESCRIPTION
If the current thread holds the lock on mutex, then the
pthread_mutex_unlock() function unlocks mutex.
Calling pthread_mutex_unlock() with a mutex that the
calling thread does not hold will result in
undefined behavior.
...
My counter-question would be - what kind of synchronization problem are you trying to solve with this? Most probably there is an easier solution.
Neither pthreads nor boost::thread (built on top of it) guarantee any order in which a contended mutex is acquired by competing threads.
Sorry, but I don't understand. what will be the state of your mutex in line [1] in the following code if another thread can unlock it?
inter_thread_mutex m;
{
m.lock();
// [1]
m.unlock();
}
This has no sens.
There's a few ways to approach this. Both of the ones I'm going to suggest are going to involve adding an additional piece of information to the object, rather adding a mechanism to unlock a thread from a thread other than the one that owns it.
1) you can add some information to indicate the object's state:
enum modification_state { consistent, // ready to be examined or to start being modified
phase1_complete, // ready for the second thread to finish the work
};
// first worker thread
lock();
do_init_work(object);
object.mod_state = phase1_complete;
unlock();
signal();
do_other_stuff();
// second worker thread
lock()
while( object.mod_state != phase1_complete )
wait()
do_final_work(obj)
object.mod_state = consistent;
unlock()
signal()
// some other thread that needs to read the data
lock()
while( object.mod_state != consistent )
wait();
read_data(obj)
unlock()
Works just fine with condition variables, because obviously you're not writing your own lock.
2) If you have a specific thread in mind, you can give the object an owner.
// first worker
lock();
while( obj.owner != this_thread() ) wait();
do_initial_work(obj);
obj.owner = second_thread_id;
unlock()
signal()
...
This is pretty much the same solution as my first solution, but more flexible in the adding/removing of phases, and less flexible in the adding/removing of threads.
To be honest, I'm not sure how inter thread mutex would help you here. You'd still need a semaphore or condition variable to signal the passing of the work to the second thread.
Small modification to what you already have: how about storing the id of the thread which you want to take the lock, in your inter_thread_whatever? Then unlock it, and send a message to that thread, saying "I want you execute whatever routine it is that tries to take this lock".
Then the condition in lock becomes while(locked || (desired_locker != thisthread && desired_locker != 0)). Technically you've "released the lock" in the first thread, and "taken it again" in the second thread, but there's no way that any other thread can grab it in between, so it's as if you've transferred it directly from one to the other.
There's a potential problem, that if a thread exits or is killed, while it's the desired locker of your lock, then that thread deadlocks. But you were already talking about the first thread waiting for a message from the second thread to say that it has successfully acquired the lock, so presumably you already have a plan in mind for what happens if that message is never received. To that plan, add "reset the desired_locker field on the inter_thread_whatever".
This is all very hairy, though, I'm not convinced that what I've proposed is correct. Is there a way that the "master" thread (the one that's directing all these helpers) can just make sure that it doesn't order any more operations to be performed on whatever is protected by this lock, until the first op is completed (or fails and some RAII thing notifies you)? You don't need locks as such, if you can deal with it at the level of the message loop.
I don't think it is a good idea to say that your inter_thread_mutex (binary_semaphore) can be seen as a model of Lockable. The main issue is that the main feature of your inter_thread_mutex defeats the Locakble concept. If inter_thread_mutex was a model of lockable you will expect in In [1] that the inter_thread_mutex m is locked.
// thread T1
inter_thread_mutex m;
{
unique_lock<inter_thread_mutex> lk(m);
// [1]
}
But as an other thread T2 can do m.unlock() while T1 is in [1], the guaranty is broken.
Binary semaphores can be used as Lockables as far as each thread tries to lock before unlocking. But the main goal of your class is exactly the contrary.
This is one of the reason semaphores in Boost.Interprocess don't use lock/unlock to name the functions, but wait/notify. Curiously these are the same names used by conditions :)
A mutex is a mechanism for describing mutually exclusive blocks of code. It does not make sense for these blocks of code to cross thread boundaries. Trying to use such a concept in such an counter intuitive way can only lead to problems down the line.
It sounds very much like you're looking for a different multi-threading concept, but without more detail it's hard to know what.