I have a test program that I wrote to try and debug a GMutex issue that I am having and I cannot seem to figure it out. I am using the class below to lock and unlock a mutex within a scoped context. This is similar to BOOST's guard.
/// #brief Helper class used to create a mutex.
///
/// This helper Mutex class will lock a mutex upon creation and unlock when deleted.
/// This class may also be referred to as a guard.
///
/// Therefore this class allows scoped access to the Mutex's locking and unlocking operations
/// and is good practice since it ensures that a Mutex is unlocked, even if an exception is thrown.
///
class cSessionMutex
{
GMutex* apMutex;
/// The object used for logging.
mutable cLog aLog;
public:
cSessionMutex (GMutex *ipMutex) : apMutex(ipMutex), aLog ("LOG", "->")
{
g_mutex_lock(apMutex);
aLog << cLog::msDebug << "MUTEX LOCK " << apMutex << "," << this << cLog::msEndL;
}
~cSessionMutex ()
{
aLog << cLog::msDebug << "MUTEX UNLOCK " << apMutex << "," << this << cLog::msEndL;
g_mutex_unlock(apMutex);
}
};
Using this class, I call it as follows:
bool van::cSessionManager::RegisterSession(const std::string &iSessionId)
{
cSessionMutex lRegistryLock (apRegistryLock);
// SOME CODE
}
where apRegistryLock is a member variable of type GMutex* and is initialized using g_mutex_new() before I ever call RegisterSession.
With this said, when I run the application with several threads, I sometimes notice at the beginning, when RegisterSession is called for the first few times that the log (from the constructor above)
[DEBUG] LOG.-> - MUTEX LOCK 0x26abb40,0x7fc14ad7ae10
[DEBUG] LOG.-> - MUTEX LOCK 0x26abb40,0x7fc14af7ce10
is logged twice in a row with the same mutex but different instance; therefore, suggesting that the mutex is being locked twice or the second lock is simply being ignored - which is seriously bad.
Moreover, it is worth noting that I also check to see if these logs were initiated from the same thread using the g_thread_self() function, and this returned two separate thread identifiers; thus, suggesting that the mutex was locked twice from separate threads.
So my question is, how is it possible for this to occur?
If it's called twice in the same call chain in the same thread this could happen. The second lock is typically (although not always) ignored. At least in pthreads it's possible to configure multiple locks as counted.
What was happening in my case was there was another thread that was calling g_cond_timed_wait function with the same mutex, but with the mutex unlocked. In this case, g_cond_timed_wait function unlocks a mutex that is not locked and leaves the mutex in an undefined state, which explains why I was seeing the behaviour explained in this question: the mutex being locked twice.
Related
I have the following code:
#include <chrono>
#include <iostream>
#include <mutex>
#include <thread>
int shared_var {0};
std::mutex shared_mutex;
void task_1()
{
while (true)
{
shared_mutex.lock();
const auto temp = shared_var;
std::this_thread::sleep_for(std::chrono::seconds(1));
if(temp == shared_var)
{
//do something
}
else
{
const auto timenow = std::chrono::system_clock::to_time_t(std::chrono::system_clock::now());
std::cout << ctime(&timenow) << ": Data race at task_1: shared resource corrupt \n";
std::cout << "Actual value: " << shared_var << "Expected value: " << temp << "\n";
}
shared_mutex.unlock();
}
}
void task_2()
{
while (true)
{
std::this_thread::sleep_for(std::chrono::seconds(2));
++shared_var;
}
}
int main()
{
auto task_1_thread = std::thread(task_1);
auto task_2_thread = std::thread(task_2);
task_1_thread.join();
task_2_thread.join();
return 0;
}
shared_var is protected in task_1 but not protected in task_2
What is expected:
I was expecting else branch is not entered in task_1 as the shared resource is locked.
What actually happens:
Running this code will enter else branch in task_1.
Expected outcome is obtained when replace shared_mutex.lock(); with std::lock_guard<std::mutex> lock(shared_mutex); and shared_mutex.unlock(); with std::lock_guard<std::mutex> unlock(shared_mutex);
Questions:
What is the problem in my current approach?
Why does it work with loack_guard?
I am running the code on:
https://www.onlinegdb.com/online_c++_compiler
Suppose you have a room with two entries. One entry has a door the other not. The room is called shared_var. There are two guys that want to enter the room, they are called task_1 and task_2.
You now want to make sure somehow that only one of them is inside the room at any time.
taks_2 can enter the room freely through the entry without a door. task_1 uses the door called shared_mutex.
Your question is now: Can achieve that only one guy is in the room by adding a lock to the door at the first entry?
Obviously no, because the second door can still be entered and left without you having any control over it.
If you experiment you might observe that without the lock it happens that you find both guys in the room while after adding the lock you don't find both guys in the room. Though this is pure luck (bad luck actually, because it makes you beleive that the lock helped). In fact the lock did not change much. The guy called task_2 can still enter the room while the other guy is inside.
The solution would be to make both go through the same door. They lock the door when going inside and unlock it when leaving the room. Putting an automatic lock on the door can be nice, because then the guys cannot forget to unlock the door when they leave.
Oh sorry, i got lost in telling a story.
TL;DR: In your code it does not matter if you use the lock or not. Actually also the mutex in your code is useless, because only one thread un/locks it. To use the mutex properly, both threads need to lock it before reading/writing shared memory.
With UB (as data race), output is undetermined, you might see "expected" output, or strange stuff, crash, ...
What is the problem in my current approach?
In first sample, you have data race as you write (non-atomic) shared_var in one thread without synchronization and read in another thread.
Why does it work with loack_guard?
In modified sample, you lock twice the same (non-recursive) mutex, which is also UB
From std::mutex::lock:
If lock is called by a thread that already owns the mutex, the behavior is undefined
You just have 2 different behaviours for 2 different UB (when anything can happen for both cases).
A mutex lock does not lock a variable, it just locks the mutex so that other code cannot lock the same mutex at the same time.
In other words, all accesses to a shared variable need to be wrapped in a mutex lock on the same mutex to avoid multiple simultaneous accesses to the same variable, it's not in any way automatic just because the variable is wrapped in a mutex lock in another place in the code.
You're not locking the mutex at all in task2, so there is a race condition.
The reason it seems to work when you wrap the mutex in a std::lock_guard is that the lock guard holds the mutex lock until the end of the scope which in this case is the end of the function.
Your function first locks the mutex with the lock lock_guard to later in the same scope try to lock the same mutex with the unlock lock_guard. Since the mutex is already locked by the lock lock_guard, execution stops and there is no output because the program is in effect not running anymore.
If you output "ok" in your code at the point of the "//do something" comment, you'll see that you get the output once and then the program stops all output.
Note; as of this behaviour being guaranteed, see #Jarod42s answer for much better info on that. As with most unexpected behaviour in C++, there is probably an UB involved.
I've reached a point in my project that requires communication between threads on resources that very well may be written to, so synchronization is a must. However I don't really understand synchronization at anything other than the basic level.
Consider the last example in this link: http://www.bogotobogo.com/cplusplus/C11/7_C11_Thread_Sharing_Memory.php
#include <iostream>
#include <thread>
#include <list>
#include <algorithm>
#include <mutex>
using namespace std;
// a global variable
std::list<int>myList;
// a global instance of std::mutex to protect global variable
std::mutex myMutex;
void addToList(int max, int interval)
{
// the access to this function is mutually exclusive
std::lock_guard<std::mutex> guard(myMutex);
for (int i = 0; i < max; i++) {
if( (i % interval) == 0) myList.push_back(i);
}
}
void printList()
{
// the access to this function is mutually exclusive
std::lock_guard<std::mutex> guard(myMutex);
for (auto itr = myList.begin(), end_itr = myList.end(); itr != end_itr; ++itr ) {
cout << *itr << ",";
}
}
int main()
{
int max = 100;
std::thread t1(addToList, max, 1);
std::thread t2(addToList, max, 10);
std::thread t3(printList);
t1.join();
t2.join();
t3.join();
return 0;
}
The example demonstrates how three threads, two writers and one reader, accesses a common resource(list).
Two global functions are used: one which is used by the two writer threads, and one being used by the reader thread. Both functions use a lock_guard to lock down the same resource, the list.
Now here is what I just can't wrap my head around: The reader uses a lock in a different scope than the two writer threads, yet still locks down the same resource. How can this work? My limited understanding of mutexes lends itself well to the writer function, there you got two threads using the exact same function. I can understand that, a check is made right as you are about to enter the protected area, and if someone else is already inside, you wait.
But when the scope is different? This would indicate that there is some sort of mechanism more powerful than the process itself, some sort of runtime environment blocking execution of the "late" thread. But I thought there were no such things in c++. So I am at a loss.
What exactly goes on under the hood here?
Let’s have a look at the relevant line:
std::lock_guard<std::mutex> guard(myMutex);
Notice that the lock_guard references the global mutex myMutex. That is, the same mutex for all three threads. What lock_guard does is essentially this:
Upon construction, it locks myMutex and keeps a reference to it.
Upon destruction (i.e. when the guard's scope is left), it unlocks myMutex.
The mutex is always the same one, it has nothing to do with the scope. The point of lock_guard is just to make locking and unlocking the mutex easier for you. For example, if you manually lock/unlock, but your function throws an exception somewhere in the middle, it will never reach the unlock statement. So, doing it the manual way you have to make sure that the mutex is always unlocked. On the other hand, the lock_guard object gets destroyed automatically whenever the function is exited – regardless how it is exited.
myMutex is global, which is what is used to protect myList. guard(myMutex) simply engages the lock and the exit from the block causes its destruction, dis-engaging the lock. guard is just a convenient way to engage and dis-engage the lock.
With that out of the way, mutex does not protect any data. It just provides a way to protect data. It is the design pattern that protects data. So if I write my own function to modify the list as below, the mutex cannot protect it.
void addToListUnsafe(int max, int interval)
{
for (int i = 0; i < max; i++) {
if( (i % interval) == 0) myList.push_back(i);
}
}
The lock only works if all pieces of code that need to access the data engage the lock before accessing and disengage after they are done. This design-pattern of engaging and dis-engaging the lock before and after every access is what protects the data (myList in your case)
Now you would wonder, why use mutex at all, and why not, say, a bool. And yes you can, but you will have to make sure that the bool variable will exhibit certain characteristics including but not limited to the below list.
Not be cached (volatile) across multiple threads.
Read and write will be atomic operation.
Your lock can handle situation where there are multiple execution pipelines (logical cores, etc).
There are different synchronization mechanisms that provide "better locking" (across processes versus across threads, multiple processor versus, single processor, etc) at a cost of "slower performance", so you should always choose a locking mechanism which is just about enough for your situation.
Just to add onto what others here have said...
There is an idea in C++ called Resource Acquisition Is Initialization (RAII) which is this idea of binding resources to the lifetime of objects:
Resource Acquisition Is Initialization or RAII, is a C++ programming technique which binds the life cycle of a resource that must be acquired before use (allocated heap memory, thread of execution, open socket, open file, locked mutex, disk space, database connection—anything that exists in limited supply) to the lifetime of an object.
C++ RAII Info
The use of a std::lock_guard<std::mutex> class follows the RAII idea.
Why is this useful?
Consider a case where you don't use a std::lock_guard:
std::mutex m; // global mutex
void oops() {
m.lock();
doSomething();
m.unlock();
}
in this case, a global mutex is used and is locked before the call to doSomething(). Then once doSomething() is complete the mutex is unlocked.
One problem here is what happens if there is an exception? Now you run the risk of never reaching the m.unlock() line which releases the mutex to other threads.
So you need to cover the case where you run into an exception:
std::mutex m; // global mutex
void oops() {
try {
m.lock();
doSomething();
m.unlock();
} catch(...) {
m.unlock(); // now exception path is covered
// throw ...
}
}
This works but is ugly, verbose, and inconvenient.
Now lets write our own simple lock guard.
class lock_guard {
private:
std::mutex& m;
public:
lock_guard(std::mutex& m_):(m(m_)){ m.lock(); } // lock on construction
~lock_guard() { t.unlock(); }} // unlock on deconstruction
}
When the lock_guard object is destroyed, it will ensure that the mutex is unlocked.
Now we can use this lock_guard to handle the case from before in a better/cleaner way:
std::mutex m; // global mutex
void ok() {
lock_guard lk(m); // our simple lock guard, protects against exception case
doSomething();
} // when scope is exited our lock guard object is destroyed and the mutex unlocked
This is the same idea behind std::lock_guard.
Again this approach is used with many different types of resources which you can read more about by following the link on RAII.
This is precisely what a lock does. When a thread takes the lock, regardless of where in the code it does so, it must wait its turn if another thread holds the lock. When a thread releases a lock, regardless of where in the code it does so, another thread may acquire that lock.
Locks protect data, not code. They do it by ensuring all code that accesses the protected data does so while it holds the lock, excluding other threads from any code that might access that same data.
I was reading through a Boost Mutex tutorial on drdobbs.com, and found this piece of code:
#include <boost/thread/thread.hpp>
#include <boost/thread/mutex.hpp>
#include <boost/bind.hpp>
#include <iostream>
boost::mutex io_mutex;
void count(int id)
{
for (int i = 0; i < 10; ++i)
{
boost::mutex::scoped_lock
lock(io_mutex);
std::cout << id << ": " <<
i << std::endl;
}
}
int main(int argc, char* argv[])
{
boost::thread thrd1(
boost::bind(&count, 1));
boost::thread thrd2(
boost::bind(&count, 2));
thrd1.join();
thrd2.join();
return 0;
}
Now I understand the point of a Mutex is to prevent two threads from accessing the same resource at the same time, but I don't see the correlation between io_mutex and std::cout. Does this code just lock everything within the scope until the scope is finished?
Now I understand the point of a Mutex is to prevent two threads from accessing the same resource at the same time, but I don't see the correlation between io_mutex and std::cout.
std::cout is a global object, so you can see that as a shared resource. If you access it concurrently from several threads, those accesses must be synchronized somehow, to avoid data races and undefined behavior.
Perhaps it will be easier for you to notice that concurrent access occurs by considering that:
std::cout << x
Is actually equivalent to:
::operator << (std::cout, x)
Which means you are calling a function that operates on the std::cout object, and you are doing so from different threads at the same time. std::cout must be protected somehow. But that's not the only reason why the scoped_lock is there (keep reading).
Does this code just lock everything within the scope until the scope is finished?
Yes, it locks io_mutex until the lock object itself goes out of scope (being a typical RAII wrapper), which happens at the end of each iteration of your for loop.
Why is it needed? Well, although in C++11 individual insertions into cout are guaranteed to be thread-safe, subsequent, separate insertions may be interleaved when several threads are outputting something.
Keep in mind that each insertion through operator << is a separate function call, as if you were doing:
std::cout << id;
std::cout << ": ";
std::cout << i;
std::cout << endl;
The fact that operator << returns the stream object allows you to chain the above function calls in a single expression (as you have done in your program), but the fact that you are having several separate function calls still holds.
Now looking at the above snippet, it is more evident that the purpose of this scoped lock is to make sure that each message of the form:
<id> ": " <index> <endl>
Gets printed without its parts being interleaved with parts from other messages.
Also, in C++03 (where insertions into cout are not guaranteed to be thread-safe) , the lock will protect the cout object itself from being accessed concurrently.
A mutex has nothing to do with anything else in the program
(except a conditional variable), at least at a higher level.
A mutex has two effeccts: it controls program flow, and prevents
multiple threads from executing the same block of code
simultaneously. It also ensures memory synchronization. The
important issue here, is that mutexes aren't associated with
resources, and don't prevent two threads from accessing the same
resource at the same time. A mutex defines a critical section
of code, which can only be entered by one thread at a time. If
all of the use of a particular resource is done in critical
sections controled by the same mutex, then the resource is
effectively protected by the mutex. But the relationship is
established by the coder, by ensuring that all use does take
place in the critical sections.
boost::timed_lock
void wait(int seconds)
{
boost::this_thread::sleep(boost::posix_time::seconds(seconds));
}
boost::timed_mutex mutex;
void thread()
{
for (int i = 0; i < 5; ++i)
{
wait(1);
boost::unique_lock<boost::timed_mutex> lock(mutex, boost::try_to_lock);
if (!lock.owns_lock())
lock.timed_lock(boost::get_system_time() + boost::posix_time::seconds(1));//<<<<
std::cout << "Thread " << boost::this_thread::get_id() << ": " << i << std::endl;
boost::timed_mutex *m = lock.release();
m->unlock();
}
}
timed_lock
Question> I have problems to understand the following lines:
if (!lock.owns_lock())
lock.timed_lock(boost::get_system_time() +
boost::posix_time::seconds(1));//<<<<
Here is my understanding. Assume lock.owns_lock() returns false which means the current object DOES NOT own the lock on the lockable object. So next line will be executed. If after the specified time lapsed and the object still cannot get the lock, then the boost::timed_lock will return false. So the following line will be executed???
std::cout << "Thread " << boost::this_thread::get_id() << ": " << i << std::endl;
Is this idea correct? I think the purpose of the code is to make sure the above line is executed iff the object has the lock. But based on my understanding(i guess is NOT correct), the above line always gets run!
Where is the problem?
You are right and the example does NOT guarantee the lock is always properly acquired before executing the protected code.
Given the explanation below the example:
The above program passes boost::try_to_lock as the second parameter to the constructor of boost::unique_lock. Whether or not the mutex has been acquired can be checked via the owns_lock() method afterwards. In case it has not - owns_lock() returns false - another function provided by boost::unique_lock is used: timed_lock() waits for a certain time to acquire the mutex. The given program waits for up to one second which should be more than enough time to acquire the mutex.
The example actually shows the three fundamental ways of acquiring a mutex: lock() waits until the mutex has been acquired. try_lock() does not wait but acquires the mutex if it is available at the time of the call and returns false otherwise. Finally, timed_lock() tries to acquire the mutex within a given period of time. As with try_lock(), success or failure is indicated by the return value of type bool.
the authors seem aware of the problem (given that the document the return value of timed_lock) but did not think a re-test if the lock had been acquired was needed (as demonstrated by them saying "waits for up to one second which should be more than enough time to acquire the mutex").
One error in your understanding:
If after the specified time lapsed and the object still cannot get the lock, then the boost::timed_lock will return false.
This is not true. timed_lock will 'continuously' try to obtain the lock, but give up if the specified time has expired.
You are right. The example doesn't properly handle the state of when the mutex fails to lock. If you read closely just below that example you'll see this quoted there:
The above example uses various methods to illustrate some of the features provided by boost::unique_lock. Certainly, the usage of these features does not necessarily make sense for the given scenario; the usage of boost::lock_guard in the previous example was already adequate. This example is rather meant to demonstrate the possibilities offered by boost::unique_lock.
I'm supposed to implement my own logging class for use in a program with two threads, the main one and a joinable processing thread. I don't want either thread to use the logger while the other is using it, so I figured I'd use a wxMutex. The logger has to act like a C++ ostream with operator<<, except that a stream manipulator function (std::endl, for example) will indicate the end of the logger's output. I think I need a recursive mutex so that the same thread can continue outputting when the mutex is locked.
The problem (I think) is that the mutex doesn't get fully unlocked, so when the other thread tries to output to the logger, it deadlocks.
Is there something in my code that I'm missing?
class Logger{
private:
std::ostream* out;
wxMutex* mut;
....
public:
template<class T>
Logger& operator<<(T str){ // accept any type of input and lock outputting until finished
if(out){
if(mut->TryLock() == wxMUTEX_BUSY){ // if we didn't get lock
mut->Lock(); // we block
}
if(out){
(*out) << str;
out->flush();
}
//mut->Unlock(); // leave locked until done outputting (use std::endl or similar)
}
return *this;
}
// accept stream manipulators and unlock stream output
Logger& operator<<(std::ostream& (*pf) (std::ostream&)){
if(out){
if(mut->TryLock() == wxMUTEX_BUSY){
mut->Lock();
}
(*out) << pf;
out->flush();
while(mut->Unlock()!= wxMUTEX_UNLOCKED);
}
return *this;
}
};
If you are worried about threading issues, you could instead make a macro that makes sure the mutex is acquired before the output and released after the output.
Something like:
#define LOG(logger, output) \
do { logger.lock(); logger << output; logger.unlock(); } while (0)
Logger my_logger;
int some_integer = 5;
LOG(my_logger, "Hello world!" << some_integer << std::endl);
First of all, in wxWidgets 2.9 wxLog itself is MT-safe and you can have independent log targets for each thread so perhaps you could just use it instead of writing your own.
Second, using TryLock() is suspicious, if you want to be able to re-lock the mutex already belonging to the current thread, you should use wxMUTEX_RECURSIVE when creating the mutex and simply use Lock() nevertheless. However personally I believe that using recursive mutexes is a bad idea because it makes your MT-code less clear and more difficult to reason about and this is almost invariably catastrophic.
Finally, the whole idea of relying on someone to call << endl is just wrong. It's too easy to forget to do it somewhere and leave the mutex locked preventing all the other threads from continuing. You absolutely should create a proxy object locking the mutex in its ctor and unlocking it in its dtor and use it to ensure that the mutex is always unlocked at the end of statement doing the logging. I think that by using this technique you should avoid the need for recursive mutexes too.
I might be wrong but it looks to me that you are trying to lock the mutex when it is in use already.
if(mut->TryLock() == wxMUTEX_BUSY) // if TryLock() returns wxMUTEX_BUSY another thread is using it.
{
mut->Lock(); // A dead lock situation could be detected here because Lock() also returns an error code.
}
TryLock() usually attempts to acquire the mutex without blocking. So, if the mutex is already busy on that call, it means that another thread is using it.
You could have a look to this link from wxMutex which explains how each functions works. Their function returns values, so you could use that to see what's going wrong in your program.
See the doc from their website below.
wxMutex::Lock
wxMutexError Lock()
Locks the mutex object.
Return value
One of:
wxMUTEX_NO_ERROR There was no error.
wxMUTEX_DEAD_LOCK A deadlock situation was detected.