I need to use read-write lock, but I would also like to be sure that the lock is recursive in both read and write mode, so that calling function that uses it on the same thread doesn't lock.
I found nothing on MSDN docs. Maybe it's hidden somewhere and I didn't find it?
SRWLocks are not recursive, from the docs:
An SRW lock is the size of a pointer. The advantage is that it is fast to update the lock state. The disadvantage is that very little state information can be stored, so SRW locks cannot be acquired recursively. In addition, a thread that owns an SRW lock in shared mode cannot upgrade its ownership of the lock to exclusive mode.
Related
How is C++ mutex implemented under the hood ? Does it only use Decker's, Peterson's or other algorithms to enforce mutual exclusion or does it also use hardware support such as interrupt ad compare-and-swap (CAS).
Is it possible to implement an entire multi-threading library consisting of mutex and condition variable without any hardware support ?
Standard requires that there are atomic operations. Not necessarily CAS, but at least exchange. std::atomic_flag is required to be true atomic, although CAS is superfluous for it, simple exchange is satisfactory.
Note that any algorithm such as Decker's, Peterson's or other would at least require atomic load and atomic store, and they will not work if load and store are non-atomic. Also non-atomic types do not enforce memory ordering, which is implied by those algorithms.
It is not really required that std::mutex will be based on operating system calls, and that there are operating system calls at all.
In theory, std::mutex could be spinning only mutex.
It could also be blocking only mutex.
In practice a good implementation would first try to handle blocking with atomic, but when it is resorted to wait, it would do OS wait.
It asks the operating system to do it.
On Linux that'll probably be using pthreads (How does pthread implemented in linux kernel 3.2?).
On Windows it's with the Windows API. You can try asking Microsoft about that one.
The standard library implementation won't be making direct hardware access. This is what operating systems are for.
On Linux, when the mutex is uncontended, the lock and unlock happen in user-space only using atomic instructions, such as compare-and-swap. In a contended case a call into the kernel is required to wait on the mutex till it's been unlocked (on lock) or to wake up waiters (on unlock). Locking user-space mutexes does not disable interrupts.
Here is the source code for low-level mutex primitives in glibc, the comments are instructive:
/* Low-level locks use a combination of atomic operations (to acquire and
release lock ownership) and futex operations (to block until the state
of a lock changes). A lock can be in one of three states:
0: not acquired,
1: acquired with no waiters; no other threads are blocked or about to block
for changes to the lock state,
>1: acquired, possibly with waiters; there may be other threads blocked or
about to block for changes to the lock state.
We expect that the common case is an uncontended lock, so we just need
to transition the lock between states 0 and 1; releasing the lock does
not need to wake any other blocked threads. If the lock is contended
and a thread decides to block using a futex operation, then this thread
needs to first change the state to >1; if this state is observed during
lock release, the releasing thread will wake one of the potentially
blocked threads.
Much of this code takes a 'private' parameter. This may be:
LLL_PRIVATE: lock only shared within a process
LLL_SHARED: lock may be shared across processes.
Condition variables contain an optimization for broadcasts that requeues
waiting threads on a lock's futex. Therefore, there is a special
variant of the locks (whose name contains "cond") that makes sure to
always set the lock state to >1 and not just 1.
Robust locks set the lock to the id of the owner. This allows detection
of the case where the owner exits without releasing the lock. Flags are
OR'd with the owner id to record additional information about lock state.
Therefore the states of robust locks are:
0: not acquired
id: acquired (by user identified by id & FUTEX_TID_MASK)
The following flags may be set in the robust lock value:
FUTEX_WAITERS - possibly has waiters
FUTEX_OWNER_DIED - owning user has exited without releasing the futex. */
And pthread_mutex_lock code.
Mutex implementation requires multiple threads to agree on a value of a shared variable - the mutex, whether it is locked or unlocked. This is also known as consensus problem and it is not possible to solve it using only atomic loads and stores, a compare-and-swap is required.
I called a pthread_mutex_lock(&th) in a thread then I want to unlock the mutex in another thread pthread_mutex_unlock(&th)
Is it possible to do that?
Or the mutex should be unlocked in the same thread ?
It should be unlocked in the same thread. From the man page: "If a thread attempts to unlock a mutex that it has not locked or a mutex which is unlocked, undefined behavior results." (http://pubs.opengroup.org/onlinepubs/009604499/functions/pthread_mutex_lock.html)
I just wanted to add to Guijt's answer:
When a thread locks a mutex, it is assumed it is inside a critical section. If we allow another thread to unlock that mutex, the first thread might still be inside the critical section, resulting in problems.
I can see several solutions to your problem:
Option 1: Rethink your algorithm
Try to understand why you need to unlock from a different thread, and see if you can get the unlocking to be done within the locking thread. This is the best solution, as it typically produces the code that is simplest to understand, and simplest to prove it is actually doing what you believe it is doing. With multithreaded programming being so complicated, the price it is worth paying for such simplicity should be quite high.
Option 2: Synchronize the threads with an event
One might argue it is just a method to implement option 1 above. The idea is that when the locking thread finishes with the critical section, it does not go out to do whatever, but waits on an event. When the second thread wishes to release the lock, it instead signals the event. The first thread then releases the lock.
This procedure has the advantage that thread 2 cannot inadvertently release the lock too soon.
Option 3: Don't use a mutex
If neither one of the above options work for you, you most likely are not using the mutex for mutual exclusion, but for synchronizations. If such is the case, you are likely using the wrong construct.
The construct most resembling a mutex is a semaphore. In fact, for years the Linux kernel did not have a mutex, claiming that it's just a semaphore with a maximal value of 1. A semaphore, unlike a mutex, does not require that the same thread lock and release.
RTFM on sem_init and friends for how to use it.
Please be reminded that you must first model your problem, and only then choose the correct synchronization construct to use. If you do it the other way around, you are almost certain to introduce lots of bugs that are really really really difficult to find and fix.
Whole purpose of using Mutex is achieve mutual exclusion in a critical section with ownership being tracked by the kernel. So mutex has to be unlocked in the same thread which has acquired it
If I need to synchronize two threads that both call a function with send() on a specific socket, would it be more useful to warp a critical section on the send() function or look into using a mutex? (since a socket is a kernel object)
Assuming Windows platform (that's where we have a choice between critical sections and mutexes).
Mutex (of CreateMutex) is way slower: locking and unlocking is always a system call, even if there is no contention. The cost of send, though, is likely to be enough to make this difference unnoticeable.
As pointed by another answer, mutexes can be shared between processes (if named/reopened or inherited), and critical sections are process-local.
I am assuming that this is about Windows (can't recall seeing critical section elsewhere).
It doesn't matter really which synchronization object you use if all the locking is within one process. If you want to lock across process boundary, then you should use mutex because critical section only works within single process, but named mutex can be shared between many processes.
I think, mutex should work faster.
I seem to be having an issue with boost::interprocess::file_lock
I have process 1 that is essentially
boost::interprocess::file_lock test_lock("testfile.csv");
test_lock.lock();
sleep(1000);
test_lock.unlock();
When I run a second process while the first process is sleeping, I find that I am still able to read testfile.csv. What's worse, I can even overwrite it.
Am I misinterpreting how file_lock works? I was under the impression that calling .lock() gives it exclusive lock over the file and prevents any other process from read/modifying the file.
file_lock is not for locking a file. It is a mutual exclusion object that uses a file as its backing technology. The contents of the file are basically irrelevant; what is relevant is that all instances of a file_lock pointing to that file will respect the locking characteristics of the lock.
As with any mutex-type object, the lock itself is there to protect or otherwise meter access to some other resource.
It has nothing to do with filesystem protection of files.
Reference
To ensure portability the kind of lock you are expecting doesn't exist in boost. There is no kernel level enforceable lock on the Unix/Linux/OSX OS's that boost supports.
see:
http://www.boost.org/doc/libs/1_51_0/doc/html/interprocess/synchronization_mechanisms.html#interprocess.synchronization_mechanisms.file_lock
Because of this the boost interprocess lock is an advisory or cooperative lock. You can accomplish what you are trying to do with boost::interprocess::file_lock but, you must also use the interprocess::file_lock in other processes that might try to read/write the file. Simply try to acquire the lock in your other process before accessing the file.
This is how interprocess::file_lock was designed to be used. You can do it some OS specific way that enforces kernel level locking if you are on some OS's, but if you use the boost::interprocess::file_lock, your code will be portable.
Its not a filesystem lock.
Simply locking in one process is not going to prevent another process from accessing the file. Think of it "like" a mutex. To have your second process honor the file_lock, you need to acquire the lock in the 2nd process as well. Then you will see that proc2 will block, waiting for proc1 to release the lock.
For example the c++0x interfaces
I am having a hard time figuring out when to use which of these things (cv, mutex and lock).
Can anyone please explain or point to a resource?
Thanks in advance.
On the page you refer to, "mutex" is the actual low-level synchronizing primitive. You can take a mutex and then release it, and only one thread can take it at any single time (hence it is a synchronizing primitive). A recursive mutex is one which can be taken by the same thread multiple times, and then it needs to be released as many times by the same thread before others can take it.
A "lock" here is just a C++ wrapper class that takes a mutex in its constructor and releases it at the destructor. It is useful for establishing synchronizing for C++ scopes.
A condition variable is a more advanced / high-level form of synchronizing primitive which combines a lock with a "signaling" mechanism. It is used when threads need to wait for a resource to become available. A thread can "wait" on a CV and then the resource producer can "signal" the variable, in which case the threads who wait for the CV get notified and can continue execution. A mutex is combined with CV to avoid the race condition where a thread starts to wait on a CV at the same time another thread wants to signal it; then it is not controllable whether the signal is delivered or gets lost.
I'm not too familiar w/ C++0x so take this answer w/ a grain of salt.
re: Mutex vs. locks: From the documentation you posted, it looks like a mutex is an object representing an OS mutex, whereas a lock is an object that holds a mutex to facilitate the RAII pattern.
Condition variables are a handy mechanism to associate a blocking/signaling mechanism (signal+wait) with a mutual exclusion mechanism, yet keep them decoupled in the OS so that you as system programmer can choose the association between condvar and mutex. (useful for dealing with multiple sets of concurrently-accessed objects) Rob Krten has some good explanations on condvars in one of the online chapters of his book on QNX.
As far as general references: This book (not out yet) looks interesting.
This question has been answered. I just add this that may help to decide WHEN to use these synchronization primitives.
Simply, the mutex is used to guarantee mutual access to a shared resource in the critical section of multiple threads. The luck is a general term but a binary mutex can be used as a lock. In modern C++ we use lock_guard and similar objects to utilize RAII to simplify and make safe the mutex usage. The conditional variable is another primitive that often combined with a mutex to make something know as a monitor.
I am having a hard time figuring out when to use which of these things
(cv, mutex and lock). Can anyone please explain or point to a
resource?
Use a mutex to guarantee mutual exclusive access to something. It's the default solution for a broad range of concurrency problems. Use lock_guard if you have a scope in C++ that you want to guard it with a mutex. The mutex is handled by the lock_guard. You just create a lock_guard in the scope and initialize it with a mutex and then C++ does the rest for you. The mutex is released when the scope is removed from the stack, for any reason including throwing an exception or returning from a function. It's the idea behind RAII and the lock_guard is another resource handler.
There are some concurrency issues that are not easily solvable by only using a mutex or a simple solution can lead to complexity or inefficiency. For example, the produced-consumer problem is one of them. If we want to implement a consumer thread reading items from a buffer shared with a producer, we should protect the buffer with a mutex but, without using a conditional variable we should lock the mutex, check the buffer and read an item if it's not empty, unlock it and wait for some time period, lock it again and go on. It's a waste of time if the buffer is often empty (busy waiting) and also there will be lots of locking and unlocking and sleeps.
The solution we need for the producer-consumer problem must be simpler and more efficient. A monitor (a mutex + a conditional variable) helps us here. We still need a mutex to guarantee mutual exclusive access but a conditional variable lets us sleep and wait for a certain condition. The condition here is the producer adding an item to the buffer. The producer thread notifies the consumer thread that there is and item in the buffer and the consumer wakes up and gets the item. Simply, the producer locks the mutex, puts something in the buffer, notifies the consumer. The consumer locks the mutex, sleeps while waiting for a condition, wake s up when there is something in the buffer and gets the item from the buffer. It's a simpler and more efficient solution.
The next time you face a concurrency problem think this way: If you need mutual exclusive access to something, use a mutex. Use lock_guard if you want to be safer and simpler. If the problem has a clue of waiting for a condition that must happen in another thread, you MIGHT need a conditional variable.
As a general rule of thumb, first, analyze your problem and try to find a famous concurrency problem similar to yours (for example, see classic problems of synchronization section in this page). Read about the solutions proposed for the well-known solution to peak the best one. You may need some customization.