I'm using boost::interprocess to share data between processes via managed shared memory. Clients may read or write to the shared memory.
Clients first attempt to open an existing shared memory:
managed_shared_memory(open_only, "MySharedMemory");
If the open fails then the memory is created:
managed_shared_memory(create_only, "MySharedMemory");
Once the shared memory has been opened or created the client increments the client count (integer stored in the shared memory).
When a client's destructor is called the client count is decremented. If client count == 0 then the shared memory is removed:
shared_memory_object::remove("MySharedMemory");
So far so good. However, if a process crashes then it can't decrement the client count so the memory isn't properly removed.
At this point a new client may successfully open the shared memory in whatever state it was left in instead of a fresh default state. This is problematic.
So my question is. What is the best way to manage the lifetime of shared memory?
Not crashing is a good idea but I'm working in a plugin environment where something beyond my control could take everything down and clients come and go continuously.
Another idea is to use pipes or sockets to verify that clients are still valid (e.g. ping them when the memory is opened and cleanup manually if there is no response) but this feels like overkill.
Related
i have a boost interprocess managed_shared_memory on windows and i have a boost interprocess vector stored in it. The vector is created or opened by
auto* vec = shm.find_or_construct< MyVector >( "Data" )( shmAllocator );
as stated in the boost interprocess examples. My point is that i now constructed or opened an Object vec referencing the object inside the shared memory. I checked that the d'tor of vec is only called when i use shm.destroy<MyVector>("Data") and if i call delete vec the application crashes.
Now how do i properly release the object "vec" without destoying the underlying data?
The complete Scenario:
Two users are running my software, sharing data via shared memory (in windows emulated using a file)
One user exits the software and if i do not call destroy i have a memory leak, if i do call it as stated in the boost docs:
In Windows operating systems, current version supports an usually acceptable emulation of the UNIX unlink behaviour: the file is renamed with a random name and marked as to be deleted when the last open handle is closed
Another users starts the software and it tries to share the memory, but as the file has been renamed, it is unable to share the memory with the other running instance of my software.
The vector is created or opened by
This is slightly mixing up concepts. It's looked up, and constructed if necessary. (open_or_create applies to actual shareable objects like memory maps or shared memory objects).
I checked that the d'tor of vec is only called when i use shm.destroy<MyVector>("Data") and if i call delete vec the application crashes.
That's both by design.
One user exits the software and if i do not call destroy i have a memory leak,
Not really. If you don't destroy the vector, it still exists in the managed segment. This means that you can reopen the shared memory segment and still find it there.
To remove the shared segment, use remove()
The docs say this related to this:
When the managed_mapped_file object is destroyed, the file is automatically unmapped, and all the resources are freed. To remove the file from the filesystem you could use standard C std::remove or Boost.Filesystem's remove() functions, but file removing might fail if any process still has the file mapped in memory or the file is open by any process.
To obtain a more portable behaviour, use file_mapping::remove(const char *) operation, which will remove the file even if it's being mapped. However, removal will fail in some OS systems if the file (eg. by C++ file streams) and no delete share permission was granted to the file. But in most common cases file_mapping::remove is portable enough.
And here:
~basic_managed_mapped_file();
Destroys *this and indicates that the calling process is finished using the resource. The destructor function will deallocate any system resources allocated by the system for use by this process for this resource. The resource can still be opened again calling the open constructor overload. To erase the resource from the system use remove()
Additional info
If you really just want the vector gone after the last user releases it, use the interprocess shared_pointer: http://www.boost.org/doc/libs/1_64_0/doc/html/interprocess/interprocess_smart_ptr.html#interprocess.interprocess_smart_ptr.shared_ptr
I'm using Boost shared memory to share vectors across different processes. However, on some occasions, the consumer of the shared memory throws up this exception:
Unexpected exception: The volume for a file has been externally altered so that the opened file is no longer valid.
I have the proper Synchronization mechanism set in place. What could this error indicate?
SOLVED Size of the memory hadn't been properly allocated upon creation by one of the processes.
When a shared memory object is created, its size is 0. To set the size of the shared memory, the user must use the truncate function call, in a shared memory that has been opened with read-write attributes
Source - Boost shared memory
It means the volume for a file has been externally altered. Look for other processes writing the file.
In other words, it means you do not have proper synchronization in place.
Do you use bip::managed_mapped_file::grow by any chance? The documentation states it only allows offline growing.
I'm using Boost Shared Memory to share a vector across processes.
In the client, how can I, once I try and open the shared memory and read a vector off it, realize if the memory is not valid, or is not what I'm looking for.
Will the Open_Only fail if the memory segment does not exist, and if so, how do I catch this failure?
Also, the shared memory segment is supposed to be removed, if there are no references to it. However, in my case, even when both the client and server are shut down, and nothing else is accessing the shared memory, the segment remains in Boost Interprocess folder in Program data, with some data. So the next time client starts up, it has no problem opening up the segment, and so thinks it is accessing correct data when in fact, there is no data to be shared.
Kindly advise. Thank you.
Speaking from experience with the underlying shm api--and not as a Boost expert...
To determine validity, one technique is to figure out if the current process is the one that is creating the shared memory (the first time). You can do this by getting the size after creating (fstat) and seeing if the size is zero. If it is zero, the process is creating it. Once you know that you can initialize it. Also, when you call truncate() to set the size here, that size is set for all other processes.
To ensure removal, you can call shm_unlink() to remove the shared memory file from the system. I believe in Boost there is a remove() api that will do that.
Can any one look into this and suggest me with an API.
We have APIs for a process which can create and/or attach a shared memory to its own process. But I don't find an API to attach a shared memory to one process by other process(for e.g., process A should call one API(like shmat()) to attach the shared memory to process B).
Shared memory doesn't belong to any particular process (unless you create it with a private IPC_PRIVATE key). It belongs to the system.
So, when you use shmget with a non-private key (and the IPC_CREAT flag), you will either create a shared memory block or attach to an existing one.
You need a way for both processes to use the same IPC key and this is often done by using ftok which uses a file specification and an identifier to give you an IPC key for use in the shmget call (and other IPC type calls, such as msgget or semget).
For example, in the programs pax1 and pax2, you may have a code segment like:
int getMyShMem (void) {
key_t mykey = ftok ("/var/pax.cfg", 0); // only one shm block so use id of 0
if (mykey == (key_t)-1) // no go.
return -1;
return shmget (mykey, 1024, IPC_CREAT); // get (or make) a 1K block.
}
By having both processes use the same file specification and ID, they'll get the same shared memory block.
You can use different IDs to give you distinct shared memory blocks all based on the same file (you may, for example, want one for a configuration shared memory block and another for storing shared state).
And, given that it's your configuration file the IPC key is based on, the chances of other programs using it is minuscule (I think it may be zero but I'm not 100% sure).
You can't forcefully inject shared memory into a process from outside that process (well, you may be able to but it would be both dangerous and require all sorts of root-level permissions). That would break the protected process model and turn you system into something about as secure as MS-DOS :-)
Let's see, allow one process to force a shared memory segment on to another? What is the receiver going to do with it? How will it know it now has mapped this block in - what is expected of it.
You're thinking about the problem the wrong way - simply hoisting a block of memory on to a second process is not going to allow you to do what you want. You need to notify the second process also that it has now mapped this block and so can start doing stuff with it. I suggest you take a step back and really look at your design and what you are doing. My recommended approach would be
A connects to B via some other IPC (say socket)
A informs B that it should attach with the details (name etc.)
B then attaches - and now B is aware of it and can start doing stuff with it. (say for example once the attach completes, B confirms to A, and then they can start talking over the shared memory block).
As for wrapping shared memory in a nice library - consider boost::interprocess.
You are asking to attach the process memory of other process, right?
Just open(2) the file /proc/<pid>/mem and use it. Check the /proc/<pid>/map for the list of usable address in the file.
I'm using boost::interprocess::shared_memory_object between the processes.
A process (server) creates a shared memory object, and other processes (clients) open that shared memory object. But, how can client processes determine if that shared memory object is removed by server process?
As boost documentation states, when the shared memory is requested to be removed, it won't be deleted until all other references de-refer it. So, even if the server process (tries to) delete the shared memory object, that shared memory object is not deleted, and, so other client processes cannot know that was deleted or not.
I need this kind of delete-detection because of the reference caching. I mean, client processes stores the reference to the shared memory, and reuse it whenever needed. But, when the server process delete the shared memory and recreate one using the same name, I just want my client processes to detect this recreation and update the reference to the newly created shared memory object.
The command
ipcs -m
will display all existing shared memory segments. By shmid you should be able to identify your segment. There's also nattch column showing number of attached processes to the segment.
You could include a flag in your shared memory object that indicates if your server process is still using it.
Set this flag to false before your server process attempts to delete the shared memory. When the client processes see that the flag is false they can close their references to the object.
Also, I don't think the server process will be permitted to recreate a shared object with the same name until it has been deleted, as I am sure the names must be unique.