I used boost::interprocess to create a boost::multi_index data structure in shared memory. There are many client processes that will access this data structure. When accessing, I will lock the data structure. The problem I encountered is Once the client process is accessing the data structure and crashes without releasing the occupied lock, then all other client processes cannot access the data structure. I use boost::interprocess::named_mutex, I Know that boost::interprocess::file_lock can be automatically released when the process crashes, but because he has a lot of restrictions, so I have no use, I don't know if there is any good way to solve this problem, thank you!
Do not place a mutex in shared memory. The boost documentation for named_mutex says:
https://www.boost.org/doc/libs/1_70_0/doc/html/boost/interprocess/named_mutex.html
A mutex with a global name, so it can be found from different processes. This mutex can't be placed in shared memory, and each process should have it's own named_mutex.
The whole point of using a named mutex is that multiple processes can create their own local mutex objects using the same name and they will share an underlying mutex that they can sync on. If a given process locks the mutex and then crashes, the underlying shared mutex will be released automatically by the OS, allowing another process to lock it (depending on OS, the underlying mutex API may report that the mutex had been unlocked abnormally).
I guess you can try to access the mutex with timed_lock and, if you get a timeout, forcibly delete the mutex with remove.
Related
Let's say I am on CentOS 7 x86_64 + GCC 7.
I would like to create a ringbuffer in shared memory.
If I have two processes Producer and Consumer, and both share a named shared memory, which is created/accessed through shm_open() + mmap().
If Producer writes something like:
struct Data {
uint64_t length;
char data[100];
}
to the shared memory at a random time, and the Consumer is constantly polling the shared memory to read. Will I have some sort of synchronization issue that the member length is seen but the member data is still in the progress of writing? If yes, what's the most efficient technique to avoid the issue?
I see this post:
Shared-memory IPC synchronization (lock-free)
But I would like to get a deeper, more low level of understanding what's required to synchronize between two processes efficiently.
Thanks in advance!
To avoid this, you would want to make the structure std::atomic and access it with acquire-release memory ordering. On most modern processors, the instructions this inserts are memory fences, which guarantee that the writer wait for all loads to complete before it begins writing, and that the reader wait for all stores to complete before it begins reading.
There are, in addition, locking primitives in POSIX, but the <atomic> header is newer and what you probably want.
What the Standard Says
From [atomics.lockfree], emphasis added:
Operations that are lock-free should also be address-free. That is, atomic operations on the same memory location via two different addresses will communicate atomically. The implementation should not depend on any per-process state. This restriction enables communication by memory that is mapped into a process more than once and by memory that is shared between two processes.
For lockable atomics, the standard says in [thread.rec.lockable.general], emphasis added:
An execution agent is an entity such as a thread that may perform work in parallel with other execution agents. [...] Implementations or users may introduce other kinds of agents such as processes [....]
You will sometimes see the claim that the standard supposedly makes no mention of using the <atomic> primitives with memory shared between processes, only threads. This is incorrect.
However, passing pointers to the other process through shared memory will not work, as the shared memory may be mapped to different parts of the address space, and of course a pointer to any object not in shared memory is right out. Indices and offsets of objects within shared memory will. (Or, if you really need pointers, Boost provides IPC-safe wrappers.)
Yes, you will ultimately run into data races, not only length being written and read before data is written, but also parts of those members will be written out of sync of your process reading it.
Although lock-free is the new trend, I'd suggest to go for a simpler tool as your first IPC sync job: the semaphore. On linux, the following man pages will be useful:
sem_init
sem_wait
sem_post
The idea is to have both processes signal the other one it is currently reading or writing the shared memory segment. With a semaphore, you can write inter-process mutexes:
Producer:
while true:
(opt) create resource
lock semaphore (sem_wait)
copy resource to shm
unlock semaphore (sem_post)
Consumer:
while true:
lock semaphore (sem_wait)
copy resource to local memory
or crunch resource
unlock semaphore (sem_post)
If for instance Producer is writing into shm while Consumer calls sem_wait, Consumer will block until after Producer will call sem_post, but, you have no guarantee Producer won't go for another loop, writing two times in a row before Consumer will be woke up. You have to build a mechanism unsure Producer & Consumer do work alternatively.
Does Windows offer any kind of mutex that can be placed in a memory mapped file and used across multiple processes?
Ideally it must be completely self contained such that it can survive by itself in the file, even across a reboot.
Also, no resources should be leaked if I simply remove the file manually while no processes are running.
If possible the solution should also offer the accompanying 'condition' concept which should also be an object that can sit in a shared memory mapped file.
In short, I need something similar to a PTHREADS mutex with the SHARED attribute.
As far as I understand, simply using a PTHREADS mutex is not possible because the SHARED attribute is unsupported in the Windows port of PTHREADS.
To share a synchronization object, give it a name and use the same name in each process when you Create the object.
The following synchronization objects can be shared between process that way :
Mutex
Semaphore
Event
Critical sections cannot be shared, but are faster.
Testing or waiting on those objects is done with the wait family of functions, often WaitForMultipleObjects.
Use the file as its own mutex: Use the LockFileEx function and have everybody agree to lock byte 0 of the file when they want to claim the mutex.
That's not possible. The mutex object itself lives in kernel space to protect it from user code messing with its state. The handle you acquired to it is only valid for the process that acquired it. Technically you could use DuplicateHandle() and put the returned handle in the mmf, but only if you have a handle to the other process that accesses the memory section. That's fairly brittle.
This is why you can specify a name for the mutex in the CreateMutex() function. The other process gets to it by using the same name in the OpenMutex call.
I have created two processes which are accessing same global shared memory. For synchronization purpose, I have used global semaphore.
Can we find out without debugging(using any windows tool) which process had acquired semaphore?
Print a message in your program each time the semaphore is acquired. Why don't you want you/can't you debug?
Really, without more information about what you're trying to do, this is all that can be said.
I am working on application where several multi-threaded processes are attempting to acquire a lock on the shared memory files managed via boost::interprocess (scoped_lock on named_mutex).
It appears that if one of these processes dumps a core while holding a lock on the shared memory file, all the other processes end up waiting for this lock to be released.
Any ideas on how this can be overcome? I want to ensure that when a process that is holding the lock dies, it should ensure that the lock is released.
One problem I see is that I am not invoking the destructor of the named_mutex object. Could this be the cause of this issue? Is the destructor guaranteed to be called when the process core dumps?
Thanks!
I seem to be having an issue with boost::interprocess::file_lock
I have process 1 that is essentially
boost::interprocess::file_lock test_lock("testfile.csv");
test_lock.lock();
sleep(1000);
test_lock.unlock();
When I run a second process while the first process is sleeping, I find that I am still able to read testfile.csv. What's worse, I can even overwrite it.
Am I misinterpreting how file_lock works? I was under the impression that calling .lock() gives it exclusive lock over the file and prevents any other process from read/modifying the file.
file_lock is not for locking a file. It is a mutual exclusion object that uses a file as its backing technology. The contents of the file are basically irrelevant; what is relevant is that all instances of a file_lock pointing to that file will respect the locking characteristics of the lock.
As with any mutex-type object, the lock itself is there to protect or otherwise meter access to some other resource.
It has nothing to do with filesystem protection of files.
Reference
To ensure portability the kind of lock you are expecting doesn't exist in boost. There is no kernel level enforceable lock on the Unix/Linux/OSX OS's that boost supports.
see:
http://www.boost.org/doc/libs/1_51_0/doc/html/interprocess/synchronization_mechanisms.html#interprocess.synchronization_mechanisms.file_lock
Because of this the boost interprocess lock is an advisory or cooperative lock. You can accomplish what you are trying to do with boost::interprocess::file_lock but, you must also use the interprocess::file_lock in other processes that might try to read/write the file. Simply try to acquire the lock in your other process before accessing the file.
This is how interprocess::file_lock was designed to be used. You can do it some OS specific way that enforces kernel level locking if you are on some OS's, but if you use the boost::interprocess::file_lock, your code will be portable.
Its not a filesystem lock.
Simply locking in one process is not going to prevent another process from accessing the file. Think of it "like" a mutex. To have your second process honor the file_lock, you need to acquire the lock in the 2nd process as well. Then you will see that proc2 will block, waiting for proc1 to release the lock.