I have a class that opens transactions, adds operations to a queue, then closes the transaction. Across the open->close lifetime I would like to employ a recursive mutex, so that only one thread can have a transaction open at any time. All other threads are blocked until the current transaction is ended.
class MyObject
{
void beginTransaction()
{
// acquire mutex
}
void endTransaction()
{
// release mutex
}
boost::recursive_mutex m_mutex;
}
I am having difficulty determining how I can use a recursive_mutex in this case, as the lock would exist longer than the scope of a single method. Can anyone suggest how I might apply locking here?
Boost (and the standard library) provide mutexes, which can be locked and unlocked, and locks, which lock mutexes in their constructor and unlocks them in their destructor. Locks are really just lightweight wrappers that are used to ensure a lock is released when leaving scope.
In your case, you would just use the mutex directly, without wrapping it in a lock.
You can call m_mutex.lock() in beginTransaction to lock the mutex, then m_mutex.unlock() in endTransaction to unlock the mutex. If another thread attempts to call m_mutex.lock() between calls, it will block until the mutex is unlocked by the owning thread in endTransaction
Related
Please consider this classical approach, I have simplified it to highlight the exact question:
#include <iostream>
#include <mutex>
using namespace std;
class Test
{
public:
void modify()
{
std::lock_guard<std::mutex> guard(m_);
// modify data
}
private:
/// some private data
std::mutex m_;
};
This is the classical approach of using std::mutex to avoid data races.
The question is why are we keeping an extra std::mutex in our class? Why can't we declare it every time before the declaration of std::lock_guard like this?
void modify()
{
std::mutex m_;
std::lock_guard<std::mutex> guard(m_);
// modify data
}
Lets say two threads are calling modify in parallel. So each thread gets its own, new mutex. So the guard has no effect as each guard is locking a different mutex. The resource you are trying to protect from race-conditions will be exposed.
The misunderstanding comes from what the mutex is and what the lock_guard is good for.
A mutex is an object that is shared among different threads, and each thread can lock and release the mutex. That's how synchronization among different threads works. So you can work with m_.lock() and m_.unlock() as well, yet you have to be very careful that all code paths (including exceptional exits) in your function actually unlocks the mutex.
To avoid the pitfall of missing unlocks, a lock_guard is a wrapper object which locks the mutex at wrapper object creation and unlocks it at wrapper object destruction. Since the wrapper object is an object with automatic storage duration, you will never miss an unlock - that's why.
A local mutex does not make sense, as it would be local and not a shared ressource. A local lock_guard perfectly makes sense, as the autmoatic storage duration prevents missing locks / unlocks.
Hope it helps.
This all depends on the context of what you want to prevent from being executed in parallel.
A mutex will work when multiple threads try to access the same mutex object. So when 2 threads try to access and acquire the lock of a mutex object, only one of them will succeed.
Now in your second example, if two threads call modify(), each thread will have its own instance of that mutex, so nothing will stop them from running that function in parallel as if there's no mutex.
So to answer your question: It depends on the context. The mission of the design is to ensure that all threads that should not be executed in parallel will hit the same mutex object at the critical part.
Synchronization of threads involves checking if there is another thread executing the critical section. A mutex is the objects that holds the state for us to check if it was "locked" by a thread. lock_guard on the other hand is a wrapper that locks the mutex on initialization and unlocks it during destruction.
Having realized that, it should be clearer why there has to be only one instance of the mutex that all lock_guards need access to - they need to check if it's clear to enter the critical section against the same object. In the second snippet of your question each function call creates a separate mutex that is seen and accessible only in its local context.
You need a mutex at class level. Otherwise, each thread has a mutex for itself, and therefore the mutex has no effect.
If for some reason you don't want your mutex to be stored in a class attribute, you could use a static mutex as shown below.
void modify()
{
static std::mutex myMutex;
std::lock_guard<std::mutex> guard(myMutex);
// modify data
}
Note that here there is only 1 mutex for all the class instances. If the mutex is stored in an attribute, you would have one mutex per class instance. Depending on your needs, you might prefer one solution or the other.
I'm trying to solve the producer consumer problem in C++11.
I have an object which holds resources, and multiple threads can
add or consume those resources. My problem is when I try to implement
a "consume when available" method on that object.
Please assume that insert/remove operations are of trivial complexity.
A little explanation of the logic in code.
struct ResourceManager{
std::mutex mux;
std::unique_lock lock{mux};
std::condition_variable bell;
void addResource(/*some Resource*/){
lock.lock();
//add resource
lock.unlock();
bell.notify_one(); //notifies waiting consumer threads to consume
}
T getResource(){
while(true){
lock.lock();
if(/*resource is available*/){
//remove resource from the object
lock.unlock();
return resource;
}else{
//new unique lock mutex object wmux creation
lock.unlock(); //problem line
bell.wait(wmux); //waits until addResource rings the bell
continue;
}
}
}
};
Suppose the following scenario:
-Two threads, T1, T2, call addResource, getResource almost simultaneously.
-T2 locks the mutex, and finds out there are no more resources available,
so it has to block until a new resource is available.
So it unlocks the mutex and sets up the bell waiting.
-T1 runs match faster. When the mutex is unlocked,
it immediately adds the resource, and before T2 sets up the waiting bell,
T1 has already rang the bell, which notifies no one.
-T2 indefinitely waits for the bell to ring, but no further resources are added.
I'm making the assumption that a thread locking a mutex, may be the only one
to unlock it. So if I tried calling bell.wait before unlocking the mutex,
the mutex could never be unlocked.
I'd like to use no time-waiting or multiple times checking solution if possible.
So which way could I solve this problem in C++11?
lock.unlock(); //problem line
bell.wait(wmux); //waits until addResource rings the bell
Yes, this is the problem line, indeed.
To correctly use a condition variable as designed, you do not unlock a mutex before wait()ing on its related condition variable. wait()ing on a condition variable atomically unlocks it for the duration of the wait, and reacquires the mutex once the thread has been notify()-ed. Both the unlock-and-wait, and wake-up-after-being-notified-and-lock, are atomic operations.
Allnotify()s should be issued while the mutex is locked. All wait() are also done while the mutex is fully locked. Given that notify(), as I mentioned, is atomic, this results in all mutex-related operations being atomic and fully sequenced, including managing the resources protected by the mutex, and thread notification via condition variables, which is now protected by a mutex as well.
There are design patterns that can be implemented that notify condition variables without using mutex protection. But they're much harder to implement correctly and still achieve thread-safe semantics. Having all condition variable operations also protected by the mutex, in addition to everything else that the mutex protects, is much simpler to implement.
std::condition_variable::wait needs to be passed a locked std::unique_lock on your mutex. wait will unlock the mutex as part of its operation, and will re-lock it before it returns.
The normal way to use lock guards like std::lock_guard and std::unique_lock is to construct them locally and let their constructor lock your mutex and their destructor unlock it.
Also, you can avoid the external while loop in your original code by providing a predicate to std::condition_variable::wait.
struct ResourceManager {
std::mutex mux;
std::condition_variable bell;
void addResource(T resource)
{
std::lock_guard<std::mutex> lock{mux};
// Add the resource
bell.notify_one();
}
T getResource()
{
std::unique_lock<std::mutex> lock{mux};
bell.wait(lock, [this](){ return resourceIsAvailable(); });
return // the ressource
}
};
I've been trying to learn how to multithread and came up with the following understanding. I was wondering if I'm correct or far off and, if I'm incorrect in any way, if someone could give me advice.
To create a thread, first you need to utilize a library such as <thread> or any alternative (I'm using boost's multithreading library to get cross-platform capabilities). Afterwards, you can create a thread by declaring it as such (for std::thread)
std::thread thread (foo);
Now, you can use thread.join() or thread.detach(). The former will wait until the thread finishes, and then continue; while, the latter will run the thread alongside whatever you plan to do.
If you want to protect something, say a vector std::vector<double> data, from threads accessing simultaneously, you would use a mutex.
Mutex's would be declared as a global variable so that they may access the thread functions (OR, if you're making a class that will be multithreaded, the mutex can be declared as a private/public variable of the class). Afterwards, you can lock and unlock a thread using a mutex.
Let's take a quick look at this example pseudo code:
std::mutex mtx;
std::vector<double> data;
void threadFunction(){
// Do stuff
// ...
// Want to access a global variable
mtx.lock();
data.push_back(3.23);
mtx.unlock();
// Continue
}
In this code, when the mutex locks down on the thread, it only locks the lines of code between it and mtx.unlock(). Thus, other threads will still continue on their merry way until they try accessing data (Note, we would likely through a mutex in the other threads as well). Then they would stop, wait to use data, lock it, push_back, unlock it and continue. Check here for a good description of mutex's.
That's about it on my understanding of multithreading. So, am I horribly wrong or accurate?
Your comments refer to "locking the whole thread". You can't lock part of a thread.
When you lock a mutex, the current thread takes ownership of the mutex. Conceptually, you can think of it as the thread places its mark on the mutex (stores its threadid in the mutex data structure). If any other thread comes along and attempts to acquire the same mutex instance, it sees that the mutex is already "claimed" by somebody else and it waits until the first thread has released the mutex. When the owning thread later releases the mutex, one of the threads that is waiting for the mutex can wake up, acquire the mutex for themselves, and carry on.
In your code example, there is a potential risk that the mutex might not be released once it is acquired. If the call to data.push_back(xxx) throws an exception (out of memory?), then execution will never reach mtx.unlock() and the mutex will remain locked forever. All subsequent threads that attempt to acquire that mutex will drop into a permanent wait state. They'll never wake up because the thread that owns the mutex is toast.
For this reason, acquiring and releasing critical resources like mutexes should be done in a manner that will guarantee they will be released regardless of how execution leaves the current scope. In other languages, this would mean putting the mtx.unlock() in the finally section of a try..finally block:
mtx.lock();
try
{
// do stuff
}
finally
{
mtx.unlock();
}
C++ doesn't have try..finally statements. Instead, C++ leverages its language rules for automatic disposal of locally defined variables. You construct an object in a local variable, the object acquires a mutex lock in its constructor. When execution leaves the current function scope, C++ will make sure that the object is disposed, and the object releases the lock when it is disposed. That's the RAII others have mentioned. RAII just makes use of the existing implicit try..finally block that wraps every C++ function body.
I want to use lock-free algorithm for shared memory for avoiding mutex. I have some processes that share data use shared memory. If a process is locking mutex and crashes, all other processes also crash.
I read some papers which implement lock-free algorithm with linked list. But in my shared memory, I can't define data structure for using on this block of memory. I have just a pointer to this block.
So I don't have any ideas for apply lock-free algorithm in my situation. I need some helps from you. Thanks and sorry if my English is very bad.
If a process is locking mutex and crashes, all other processes also crash.
Specifically for this use case there are robust mutexes:
PTHREAD_MUTEX_ROBUST
If the process containing the owning thread of a robust mutex terminates while holding the mutex lock, the next thread that acquires the mutex shall be notified about the termination by the return value [EOWNERDEAD] from the locking function. If the owning thread of a robust mutex terminates while holding the mutex lock, the next thread that acquires the mutex may be notified about the termination by the return value [EOWNERDEAD]. The notified thread can then attempt to mark the state protected by the mutex as consistent again by a call to pthread_mutex_consistent(). After a subsequent successful call to pthread_mutex_unlock(), the mutex lock shall be released and can be used normally by other threads. If the mutex is unlocked without a call to pthread_mutex_consistent(), it shall be in a permanently unusable state and all attempts to lock the mutex shall fail with the error [ENOTRECOVERABLE]. The only permissible operation on such a mutex is pthread_mutex_destroy().
I read this answer on SO:
Because the recursive mutex has a sense of ownership, the thread that grabs the mutex must be the same thread that releases the mutex. In the case of non-recursive mutexes, there is no sense of ownership and any thread can usually release the mutex no matter which thread originally took the mutex.
I'm confused by the last statement. Can one thread lock a mutex and another different thread unlock that mutex? I thought that same thread should be the only one able to unlock the mutex? Or is there any specific mutex that allows this? I hope someone can clarify.
Non-recursive mutex
Most mutexes are (or at least should be) non-recursive. A mutex is an object which can be acquired or released atomically, which allows data which is shared between multiple threads to be protected against race conditions, data corruption, and other nasty things.
A single mutex should only ever be acquired once by a single thread within the same call chain. Attempting to acquire (or hold) the same mutex twice, within the same thread context, should be considered an invalid scenario, and should be handled appropriately (usually via an ASSERT as you are breaking a fundamental contract of your code).
Recursive mutex
This should be considered a code smell, or a hack. The only way in which a recursive mutex differs from a standard mutex is that a recursive mutex can be acquired multiple times by the same thread.
The root cause of the need for a recursive mutex is lack of ownership and no clear purpose or delineation between classes. For example, your code may call into another class, which then calls back into your class. The starting class could then try to acquire the same mutex again, and because you want to avoid a crash, you implement this as a recursive mutex.
This kind of topsy-turvy class hierarchy can lead to all sorts of headaches, and a recursive mutex provides only a band-aid solution for a more fundamental architectural problem.
Regardless of the mutex type, it should always be the same thread which acquires and releases the same mutex. The general pattern that you use in code is something like this:
Thread 1
Acquire mutex A
// Modify or read shared data
Release mutex A
Thread 2
Attempt to acquire mutex A
Block as thread 1 has mutex A
When thread 1 has released mutex A, acquire it
// Modify or read shared data
Release mutex A
It gets more complicated when you have multiple mutexes that can be acquired simultaneously (say, mutex A and B). There is the risk that you'll run into a deadlock situation like this:
Thread 1
Acquire mutex A
// Access some data...
*** Context switch to thread 2 ***
Thread 2
Acquire mutex B
// Access some data
*** Context switch to thread 1 ***
Attempt to acquire mutex B
Wait for thread 2 to release mutex B
*** Context switch to thread 2 ***
Attempt to acquire mutex A
Wait for thread 1 to release mutex A
*** DEADLOCK ***
We now have a situation where each thread is waiting for the other thread to release the other lock -- this is known as the ABBA deadlock pattern.
To prevent this situation, it is important that each thread always acquires the mutexes in the same order (e.g. always A, then B).
Recursive mutexes are thread-specific by design (the same thread locking again = recursion, another thread locking meanwhile = block). Regular mutexes do not have this design and thus they can in fact be locked and unlocked in different threads.
I think this covers all your questions. Straight from linux man pages for pthreads:
If the mutex type is PTHREAD_MUTEX_NORMAL, deadlock detection shall not be provided. Attempting to relock the mutex causes deadlock. If a thread
attempts to unlock a mutex that it has not locked or a mutex which is unlocked, undefined behavior results.
If the mutex type is PTHREAD_MUTEX_ERRORCHECK, then error checking shall be provided. If a thread attempts to relock a mutex that it has already
locked, an error shall be returned. If a thread attempts to unlock a mutex that it has not locked or a mutex which is unlocked, an error shall be
returned.
If the mutex type is PTHREAD_MUTEX_RECURSIVE, then the mutex shall maintain the concept of a lock count. When a thread successfully acquires a
mutex for the first time, the lock count shall be set to one. Every time a thread relocks this mutex, the lock count shall be incremented by one.
Each time the thread unlocks the mutex, the lock count shall be decremented by one. When the lock count reaches zero, the mutex shall become
available for other threads to acquire. If a thread attempts to unlock a mutex that it has not locked or a mutex which is unlocked, an error shall
be returned.
If the mutex type is PTHREAD_MUTEX_DEFAULT, attempting to recursively lock the mutex results in undefined behavior. Attempting to unlock the mutex
if it was not locked by the calling thread results in undefined behavior. Attempting to unlock the mutex if it is not locked results in undefined
behavior.