I have a scenario where the shared memory area is exclusively accessed by two different processes. When I launch the processes, the first process successfully locks the mutex, updates the memory and unlock the mutex. But I observe that when the second process try to lock it, it is still in deadlock state, waiting for mutex to unlock.
Time difference between the mutex lock is 10s for first and second process.
I am using the std::mutex. Please tell me what I am missing.
Despite the citations that others have made to documentation, the classes std::mutex and std::shared_mutex actually do work across processes when used in shared memory on Linux. I've checked with GCC 8.3.1 and Clang 6.0.1.
Standard C++ implementations of these on Linux use pthreads. The pthreads provision of PTHREAD_PROCESS_SHARED and PTHREAD_PROCESS_PRIVATE as attributes for pthread_mutex_t and pthread_rwlock_t are abstracted out and defaulted
by std::mutex and std::shared_mutex. Even though POSIX documentation says that the default is PRIVATE, a pthread_mutex_t and pthread_rwlock_t allocated in shared memory will block competing processes when locked. This is because the actual implementation of pthreads on Linux uses futexes, and they are intended for use in shared memory even in cases where mapped addresses may differ across processes.
It's possible that the PTHREADS_PROCESS_PRIVATE behaviour is actually more difficult to implement given the strategy of using futexes to implement mutexes, and as such, is silently not implemented.
If you did want your mutexes to be process-private, just avoid putting them in shared memory. On the other hand if you do want to share them, be cautious that this standards discrepancy could be subject to change.
For reliability, use Boost Interprocess.
An std::mutex instance is only scoped to a single process; it is not capable of interprocess synchronization/concurrency. It is only capable of synchronizing child threads within a parent process.
Look to use Boost or an interprocess synchronization library instead.
std::mutex does not support interprocess operation but pthread library has interprocess mutex that you can use. Example here.
Related
In one of the projects I am working on migration from C to C++. A lot of the C code uses pthread and mutex associated with this library for multithreading. I want to perform possibly step by step, incremental migration. I wanted to start with the data structures as they are most obvious but they would need to be synchronized using pthread mutex. Is usage of pthread mutex safe or only the standard library threading infrastructure (like std::mutex) can guarantee proper memory interthread memory consistency?
Thread library in C++ provides higher level abstractions and other useful synchronization mechanisms, If your compiler supports C++11 or newever C++ standard features, In my opinion you should use C++11 thread library and experience beauty of RAII when managing shared resources.
I don't have much experience with pthreads, but pthread_mutex has same semantics as std::mutex and to answer your question
Is usage of pthread mutex safe or only the standard library threading infrastructure (like std::mutex) can guarantee proper memory inter thread memory consistency
As I mentioned semantically they are same and you can build a locable abstraction (class with try_lock, unlock, lock methods) on top of pthread_mutex easily, it comes down to how you use both, for example
If you are causing data races or not
If you are using lock_guard,
unique_lock or not, Because if you don't use RAII in addition to similar problems as in C for resource management,
C++ statements can throw exception which will cause threads to
deadlock if mutex is not unlocked.
Besides this there are bunch of other useful stuff available like std::recursive_mutex, std::shared_mutex, std::futures, std::promises etc.
I have a scenario where the shared memory area is exclusively accessed by two different processes. When I launch the processes, the first process successfully locks the mutex, updates the memory and unlock the mutex. But I observe that when the second process try to lock it, it is still in deadlock state, waiting for mutex to unlock.
Time difference between the mutex lock is 10s for first and second process.
I am using the std::mutex. Please tell me what I am missing.
Despite the citations that others have made to documentation, the classes std::mutex and std::shared_mutex actually do work across processes when used in shared memory on Linux. I've checked with GCC 8.3.1 and Clang 6.0.1.
Standard C++ implementations of these on Linux use pthreads. The pthreads provision of PTHREAD_PROCESS_SHARED and PTHREAD_PROCESS_PRIVATE as attributes for pthread_mutex_t and pthread_rwlock_t are abstracted out and defaulted
by std::mutex and std::shared_mutex. Even though POSIX documentation says that the default is PRIVATE, a pthread_mutex_t and pthread_rwlock_t allocated in shared memory will block competing processes when locked. This is because the actual implementation of pthreads on Linux uses futexes, and they are intended for use in shared memory even in cases where mapped addresses may differ across processes.
It's possible that the PTHREADS_PROCESS_PRIVATE behaviour is actually more difficult to implement given the strategy of using futexes to implement mutexes, and as such, is silently not implemented.
If you did want your mutexes to be process-private, just avoid putting them in shared memory. On the other hand if you do want to share them, be cautious that this standards discrepancy could be subject to change.
For reliability, use Boost Interprocess.
An std::mutex instance is only scoped to a single process; it is not capable of interprocess synchronization/concurrency. It is only capable of synchronizing child threads within a parent process.
Look to use Boost or an interprocess synchronization library instead.
std::mutex does not support interprocess operation but pthread library has interprocess mutex that you can use. Example here.
Problem (in short):
I'm using POSIX Shared Memory and currently just used POSIX semaphores and i need to control multiple readers, multiple writers. I need help with what variables/methods i can use to control access within the limitations described below.
I've found an approach that I want to implement but i'm unsure of what methodology i can use to implement it when using POSIX Shared memory.
What I've Found
https://stackoverflow.com/a/28140784
This link has the algorithm i'd like to use but i'm unsure how to implement it with shared memory. Do i store the class in shared memory somehow? This is where I need help please.
The reason I'm unsure is a lot of my research, points towards keeping shared memory to primitives only to avoid addressing problems and STL objects can't be used.
NOTE:
For all my multi-threading i'm using C++11 features. This shared memory will be completely seperate program executables using C++11 std::threads from which any thread of any process/executable will want access. I have avoided the Linux pthread for any of my multi-threading and will continue to do so (except if its just control variable not actual pThreads).
Solution Parameters aimed for
Must be shareable between 2+ processes which will be running multiple C++11 std::thread that may wish access. I.e. Multiple Writers (exclusive one at a time) while allowing multiple simultaneous readers when no writer wants access.
Not using BOOST libraries. Ideally native C++11 or built in linux libraries, something that will work without the need to install abstract libraries.
Not using pThread actual threads but could use some object from there that will work with C++11 std::thread.
Ideally can handle a process crash while in operation. E.g. Using POSIX semaphore if a process crashes while it has the semaphore, everyone is screwed. I have seen people using file locks?
Thanks in advance
keeping shared memory to primitives only to avoid addressing problems
You can use pointers in and to shared memory objects across programs, so long as the memory is mmaped to the same address. This is actually a straightforward proposition, especially on 64 bit. See this open source C library I wrote for implementation details: rszshm - resizable pointer-safe shared memory.
Using POSIX semaphore if a process crashes while it has the semaphore, everyone is screwed.
If you want to use OS mediated semaphores, the SysV semaphores have SEM_UNDO, which recovers in this case. OTOH pthread offers robust mutexes that can be embedded and shared in shared memory. This can be used to build more sophisticated mechanisms.
The SysV scheme of providing multiple semaphores in a semaphore set, where a group of actions must all succeed, or the call blocks, permits building sophisticated mechanism too. A read/write lock can be made with a set of three semaphores.
Currently, I have 2 processes that communicate using the message_queue and shared_memory form boost. Everything work as attended.
Now I need to make one of this process multi threaded (thanks to boost again), and I was wondering if I need to use protection mechanism between the threads (such as mutex), or if the boost::interprocess library already provides a protection mechanism ?
I did not find any info on that on the boost documentation. By the way I'm using boost 1.40.
Thanks in advance.
The shared resources from boost::interprocess (shared memory, etc) require that you provide the necessary synchronization. The reason for this is that you may not require synchronization, and it usually a somewhat expensive operation performance wise.
Say for example that you had a process that wrote to shared memory the current statistics of something in 32-bit integer format, and a few processes that read those values. Since the values are integers (and therefore on your platform the reads and writes are atomic) and you have a single process writing them and a few process reading them, no synchronization is needed for this design.
However in some examples you will require synchronization, like if the above example had multiple writers, or if instead of integers you were using string data. There are various synchronization mechanisms inside of boost (as well as non-boost ones, but since your already using boost), described here:
[Boost info for stable version: 1.48]
http://www.boost.org/doc/libs/1_48_0/doc/html/interprocess/synchronization_mechanisms.html
[Boost info for version your using: 1.40]
http://www.boost.org/doc/libs/1_40_0/doc/html/interprocess/synchronization_mechanisms.html
With shared memory it is a common practice to place the synchronization mechanism at the base of the shared memory segment, where it can be anonymous (meaning the OS kernel does not provide access to it by name). This way all the processes know how to lock the shared memory segment, and you can associate locks with their segments (if you had multiple for example)
Remember that a mutex requires the same thread of execution (inside a process) to unlock it that locked it. If you require locking and unlocking a synchronization object from different threads of execution, you need a semaphore.
Please be sure that if you choose to use a mutex that it is an interprocess mutex (http://www.boost.org/doc/libs/1_48_0/doc/html/boost/interprocess/interprocess_mutex.html) as opposed to the mutex in the boost thread library which is for a single process with multiple threads.
You have to make sure you lock the shared resources.
You can find examples in boost documentation. For example:
http://www.boost.org/doc/libs/1_40_0/doc/html/interprocess/synchronization_mechanisms.html#interprocess.synchronization_mechanisms.mutexes.mutexes_scoped_lock
I found this:
Fast interprocess synchronization method
I used to believe that a pthread mutex can only be shared between two threads in the same address space.
The question / answers there seems to imply:
If I have two separate proceses A & B. They have a shared memory region M. I can put a pThread mutex in M, lock in A, lock in B, unlock in A; and B will no longer block on the mutex. Is this correct? Can pThread mutexes be shared in two separate processes?
Edit: I'm using C++, on MacOSX.
You need to tell the mutex to be process-shared when it's inited:
http://www.opengroup.org/onlinepubs/007908775/xsh/pthread_mutexattr_setpshared.html
Note in particular, "The default value of the attribute is PTHREAD_PROCESS_PRIVATE", meaning that accessing it from different processes is undefined behaviour.
If your C/pthread library is conforming, you should be able to tell if it supports mutexes shared across multiple process by checking if the _POSIX_THREAD_PROCESS_SHARED feature test macro is defined to a value other than -1 or by querying the system configuration at run-time using sysconf(_SC_THREAD_PROCESS_SHARED) if that feature test macro is undefined.
EDIT: As Steve pointed out, you'll need to explicitly configure the mutex for sharing across processes assuming the platform supports that feature as I described above.
I was concerned that there might be a condition where a mutex in shared memory might fail to behave properly, so I did some digging and came up with some documents which treat the issue like a no-brainer:
https://computing.llnl.gov/tutorials/pthreads/
Further digging, however, showed that older versions of glibc suffered issues in shared memory mutexes: (This is an ancient change, but it illustrates the point.)
in linuxthreads/mutex.c
int __pthread_mutexattr_setpshared(...) {
/* For now it is not possible to shared a conditional variable. */
if (pshared != PTHREAD_PROCESS_PRIVATE)
return ENOSYS;
}
Without more detail on what implementation of pthread you're using, it's difficult to say whether you're safe or not.
My cause for concern is that many implementations (and some entire languages, like perl, python, and ruby) have a global lock object that manages access to shared objects. That object would not be shared between processes and therefore, while your mutexes would probably work most of the time, you might find yourself having two processes simultaneously manipulating the mutex at the same time.
I know that this flies in the face of the definition of a mutex but it is possible:
If two threads are operating at the same time in different processes, it implies that they are on different cores. Both acquire their global lock object and go to manipulate the mutex in shared memory. If the pthread implementation forces the update of the mutex through the caches, both threads could end up updating at the same time, both thinking they hold the mutex. This is just a possible failure vector that comes to mind. There could be any number of others. What are the specifics of your situation - OS, pthreads version, etc.?