I'm porting a Windows application to Linux and I have a synchronization problem.
In Windows I'm using a system-level named mutex to sync access to a shared memory block.
How do I emulate that in Linux? I've created a SystemV semaphore, using semget. The problem is that it is not reentrant, if I already hold it it will block, unlike on Windows. I could add a reference count to it, but then I would need to synchronize access to that, which means another (this time for the current process only) mutex.
Is there a class somewhere which provides a reentrant interprocess lock (maybe in Boost)?
BTW, using a file lock is not acceptable since it will probably be too slow (I need ultra-low latency communication between the two processes).
You can just use a shared (interprocess), recursive pthread_mutex_t. Create a normal pthread_mutex (stored in shared memory) and set its attributes using pthread_mutexattr_settype with the PTHREAD_MUTEX_RECURSIVE flag, and then call pthread_mutexattr_setpshared with the PTHREAD_MUTEX_SHARED flag.
That will give you a reentrant, interprocess lock.
You could try building your own out of futexes. See usersem.c in this tarball.
Related
I have a scenario where the shared memory area is exclusively accessed by two different processes. When I launch the processes, the first process successfully locks the mutex, updates the memory and unlock the mutex. But I observe that when the second process try to lock it, it is still in deadlock state, waiting for mutex to unlock.
Time difference between the mutex lock is 10s for first and second process.
I am using the std::mutex. Please tell me what I am missing.
Despite the citations that others have made to documentation, the classes std::mutex and std::shared_mutex actually do work across processes when used in shared memory on Linux. I've checked with GCC 8.3.1 and Clang 6.0.1.
Standard C++ implementations of these on Linux use pthreads. The pthreads provision of PTHREAD_PROCESS_SHARED and PTHREAD_PROCESS_PRIVATE as attributes for pthread_mutex_t and pthread_rwlock_t are abstracted out and defaulted
by std::mutex and std::shared_mutex. Even though POSIX documentation says that the default is PRIVATE, a pthread_mutex_t and pthread_rwlock_t allocated in shared memory will block competing processes when locked. This is because the actual implementation of pthreads on Linux uses futexes, and they are intended for use in shared memory even in cases where mapped addresses may differ across processes.
It's possible that the PTHREADS_PROCESS_PRIVATE behaviour is actually more difficult to implement given the strategy of using futexes to implement mutexes, and as such, is silently not implemented.
If you did want your mutexes to be process-private, just avoid putting them in shared memory. On the other hand if you do want to share them, be cautious that this standards discrepancy could be subject to change.
For reliability, use Boost Interprocess.
An std::mutex instance is only scoped to a single process; it is not capable of interprocess synchronization/concurrency. It is only capable of synchronizing child threads within a parent process.
Look to use Boost or an interprocess synchronization library instead.
std::mutex does not support interprocess operation but pthread library has interprocess mutex that you can use. Example here.
Problem (in short):
I'm using POSIX Shared Memory and currently just used POSIX semaphores and i need to control multiple readers, multiple writers. I need help with what variables/methods i can use to control access within the limitations described below.
I've found an approach that I want to implement but i'm unsure of what methodology i can use to implement it when using POSIX Shared memory.
What I've Found
https://stackoverflow.com/a/28140784
This link has the algorithm i'd like to use but i'm unsure how to implement it with shared memory. Do i store the class in shared memory somehow? This is where I need help please.
The reason I'm unsure is a lot of my research, points towards keeping shared memory to primitives only to avoid addressing problems and STL objects can't be used.
NOTE:
For all my multi-threading i'm using C++11 features. This shared memory will be completely seperate program executables using C++11 std::threads from which any thread of any process/executable will want access. I have avoided the Linux pthread for any of my multi-threading and will continue to do so (except if its just control variable not actual pThreads).
Solution Parameters aimed for
Must be shareable between 2+ processes which will be running multiple C++11 std::thread that may wish access. I.e. Multiple Writers (exclusive one at a time) while allowing multiple simultaneous readers when no writer wants access.
Not using BOOST libraries. Ideally native C++11 or built in linux libraries, something that will work without the need to install abstract libraries.
Not using pThread actual threads but could use some object from there that will work with C++11 std::thread.
Ideally can handle a process crash while in operation. E.g. Using POSIX semaphore if a process crashes while it has the semaphore, everyone is screwed. I have seen people using file locks?
Thanks in advance
keeping shared memory to primitives only to avoid addressing problems
You can use pointers in and to shared memory objects across programs, so long as the memory is mmaped to the same address. This is actually a straightforward proposition, especially on 64 bit. See this open source C library I wrote for implementation details: rszshm - resizable pointer-safe shared memory.
Using POSIX semaphore if a process crashes while it has the semaphore, everyone is screwed.
If you want to use OS mediated semaphores, the SysV semaphores have SEM_UNDO, which recovers in this case. OTOH pthread offers robust mutexes that can be embedded and shared in shared memory. This can be used to build more sophisticated mechanisms.
The SysV scheme of providing multiple semaphores in a semaphore set, where a group of actions must all succeed, or the call blocks, permits building sophisticated mechanism too. A read/write lock can be made with a set of three semaphores.
There's a well-known algorithm that employs readers/writer lock synchronization between threads of a single process on a Windows platform using pure WinAPIs/C++:
Here's an example
In my case I need to do this between several processes, i.e. the writer is in one process, and readers are in other processes. Any idea how to do that?
PS. I need this for an already developed project, so I can't use anything other than C++/MFC or pure WinAPIs. In other words I can't use Boost or the like extensions.
You can use the same algorithm but instead of CriticalSection you can use Mutexes from the WinAPI.
If you use the same name for your Mutex objects you can use them in several processes.
Currently, I have 2 processes that communicate using the message_queue and shared_memory form boost. Everything work as attended.
Now I need to make one of this process multi threaded (thanks to boost again), and I was wondering if I need to use protection mechanism between the threads (such as mutex), or if the boost::interprocess library already provides a protection mechanism ?
I did not find any info on that on the boost documentation. By the way I'm using boost 1.40.
Thanks in advance.
The shared resources from boost::interprocess (shared memory, etc) require that you provide the necessary synchronization. The reason for this is that you may not require synchronization, and it usually a somewhat expensive operation performance wise.
Say for example that you had a process that wrote to shared memory the current statistics of something in 32-bit integer format, and a few processes that read those values. Since the values are integers (and therefore on your platform the reads and writes are atomic) and you have a single process writing them and a few process reading them, no synchronization is needed for this design.
However in some examples you will require synchronization, like if the above example had multiple writers, or if instead of integers you were using string data. There are various synchronization mechanisms inside of boost (as well as non-boost ones, but since your already using boost), described here:
[Boost info for stable version: 1.48]
http://www.boost.org/doc/libs/1_48_0/doc/html/interprocess/synchronization_mechanisms.html
[Boost info for version your using: 1.40]
http://www.boost.org/doc/libs/1_40_0/doc/html/interprocess/synchronization_mechanisms.html
With shared memory it is a common practice to place the synchronization mechanism at the base of the shared memory segment, where it can be anonymous (meaning the OS kernel does not provide access to it by name). This way all the processes know how to lock the shared memory segment, and you can associate locks with their segments (if you had multiple for example)
Remember that a mutex requires the same thread of execution (inside a process) to unlock it that locked it. If you require locking and unlocking a synchronization object from different threads of execution, you need a semaphore.
Please be sure that if you choose to use a mutex that it is an interprocess mutex (http://www.boost.org/doc/libs/1_48_0/doc/html/boost/interprocess/interprocess_mutex.html) as opposed to the mutex in the boost thread library which is for a single process with multiple threads.
You have to make sure you lock the shared resources.
You can find examples in boost documentation. For example:
http://www.boost.org/doc/libs/1_40_0/doc/html/interprocess/synchronization_mechanisms.html#interprocess.synchronization_mechanisms.mutexes.mutexes_scoped_lock
Is it possible to use fcntl() system call on a file to achieve thread/process synchronization (instead of semaphoress)?
Yes. Unix fcntl locks (and filesystem resources in general) are system-wide, so any two threads of execution (be they separate processes or not) can use them. Whether that's a good idea or not is context-dependent.
That's one way of synchronizing between processes, but if you don't want to use semaphores, you could use process shared mutexes, such as mutexes and condition variables created with the PTHREAD_PROCESS_SHARED attribute on POSIX based platforms (see pthread_mutexattr_setpshared() and pthread_condattr_setpshared()). Another option is to use an event based IPC (sockets, etc) mechanism that blocks until an event you define is demultiplexed (e.g. via select()). There are several other shared memory based options as well.
However, since you're using C++ I'd recommend using a C++ framework that greatly simplifies this sort of interprocess synchronization across multiple platforms like boost.interprocess or ACE.
The fcntl and flock are not for thread, but for process, so they cannot be used for thread synchronization.