I have read the article about Memory-Mapped Files and the example about CreateFileMapping.
My question is: Can I pass a pointer to a struct or a object between two processes using memory-mapped file?
Since there are some answers that it is possible, here is struct that I want to pass:
// First Process
struct OtherStruct{};
struct MyStruct
{
unsigned long handleObject;
unsigned long *phandleObject;
OtherStruct someData;
OtherStruct *pData;
}
MyStruct dataSend = { ... };
WriteToMappedFile(data);
// Second Process
MyStruct dataReceived = ReadFromMappedFile()
As the other answers already stated, you must either rely on the address of the memory-mapped areas to be equal, or you must move from absolute addresses in your pointers to relative addressing.
One possible implementation I stumbled across recently is the offset_ptr in the Boost library, which seems to fit your use case perfectly.
The answer depends on what you want to achieve. Passing a pointer in shared memory is easy, but the other process may not be able to use it in the way you expect.
Note that a pointer contains a virtual address of the data structure it points to. Such a virtual address is only valid within the process that holds the pointed-to data structure. If you pass the pointer to another process, the other process will have its own virtual address space, and the passed pointer loses its validity.
So the answer to your question is: Yes, you can pass the pointer, but without further actions, you won't be able to successfully use this pointer in the receiving process. Specifically, you will most probably not be able to use it for accessing the struct or object it points to.
If you want to access the struct or object within the other process, you need to do the following:
Put the object itself into shared memory.
Convert the pointer to the object into an offset relative to the beginning of the memory mapped file.
Pass this offset to the other process
In the other process, use the offset to convert back to a pointer.
boost::offset_ptr can help you with part of that.
Assuming the Pointer is to a struct that is part of the same memory mapped region, yes that can make sense. but then you will have to ensure that the memory mapped region is mapped to the same virtual address, this is not always guaranteed and is a bad way to design things.
You can pass the offset instead and deal with relative offsets everywhere fpor structures present in this memory region.
If the pointer you want to pass into the memory-mapped-file is not allocated by GlobalAlloc, and not locked by GlobalLock, it can't. However, you have already the memory allocated to pass the data. So you can re-write the memory on the memory-mapped-file.
Related
In this topic Two variables in one shared memory , you can see how to put 2 variables into one shared memory in C.
However, I want to put multiple variables into one shared memory using MapViewOfFile function.
But there's no example or solution I've found.
Types of variables are different so I can't use std::vector nor array.
How should I set arguments of CreateFileMapping and MapViewOfFile to do this?
How can I get and set an address in FileMapping which I want to read and write a variable?
It shouldn't be needed that I have to create a FileMapping for each variables, right?
Use a struct as davetherock pointed out.
You have to specify the total size of all the members for "CreateFileMapping" and for "MapViewOfFile". Then you can cast the return void pointer of MapViewOfFile to the structure pointer type.
For example, if you have a struct like
typedef struct combinedStruct {
int intVariable;
float floatVariable;
}CombinedStruct;
You have to create a pointer of type CombinedStruct and cast the return of MapViewOfFile.
CombinedStruct * structPtr = (CombinedStruct*)(MapViewOfFile(...));
Then you can access the individual variables like structPtr->intVariable and structPtr->floatVariable
This may be better suited as comment but i do not have enough reputation to comment.
I am also looking for the answer for this same question and i did as said by #Manoj AV , with one change-
In the CopyMemory Instead of retrieving it as structPtr->intVariable and structPtr->floatVariable i used structPtrfor first variable and structPtr+1 to retrieve the second variable in the structure and so on.And it runs successfully.
The easiest way to have two variables back-to-back in the same chunk of memory is to use a Struct: typedef struct vs struct definitions.
If you'd like to use shmget, you simply must adjust the size of the memory allocated
shmid = shmget (shmkey, sizeof(var1) + sizeof(var2), 0644 | IPC_CREAT);
Then, adjust the pointers as necessary.
I have 2 applications that are using a shared memory segment to pass data from a writing process to a reading process. At the top of the shared memory segment, I want to declare a single header struct to contain general segment data such as buffer count values, read/write offsets, pthread mutex/condition variables, etc. The header struct does NOT contain any pointers. Then the rest of the segment is used for the actual data being passed between the 2 processes.
Since the shared memory segment is attached to the process, I am assured that the segment is indeed valid/accessible to the process. It is easy to simply cast the base segment pointer to a pointer of the header struct type and that works very well for regular data types (like ints, etc) but does not work for the non-integral data types in the struct like pthread mutex/condition variables since they have not actually been declared and allocated.
So, how do I declare and allocate my header struct to start at the specific address of my already allocated shared memory segment?
The following does not appear to work:
SmHeader header = *((SmHeader *)m_pBaseAddress);
m_pHeader = &header;
where SmHeader is a typedef of my header struct and m_pBaseAddress is the base address of the attached shared memory segment.
thanks,
-Andres
I think it doesn't work because of the way you're creating it:
// this creates a copy of the SmHeader struct, on the stack
SmHeader header = *((SmHeader *)m_pBaseAddress);
// this sets m_pHeader to address of the local variable, not
// m_pBaseAddress
m_pHeader = &header;
Try this instead:
m_pHeader = (SmHeader *) m_pBaseAddress;
Say I write address of a struct in myFile.txt. Will I be able to read that address into my program? Will that address be useful?
A pointer is just a memory address. You can write it just fine, but when you read it, it is still just a memory address. Unless the object that it was pointing to is at the exact same memory location when you read it, you will be "reading" a pointer to random data, which you cannot convert to the class of the object it was pointing to before.
It's like storing the location (lat/long) of a butterfly, then trying to find that butterfly just from that position. The butterfly is most likley in a completely different place now.
1)You can write it.
2)You can read it.
3)If you read it back into the same instance of the program that wrote it, it has a real chance of being meaningful, depending on whether you have deleted the memory.
But generally it will be no use at all. Memory layout is free to changed from run to run, and memory you malloc/new on one run is not still waiting around when next you run the program.
I doubt it will be useful. When a program runs, it is loaded in memory. So now you write a text file of a struct. When the program terminates, the memory will be freed. Unless you are running two programs at the same time (one is writing the address and one other is reading), it will be useless, and may produce exception.
If you intend to use the struct value some other time, I think it is better to write the content of the struct and save it to a file, rather than the address. To write the content of a struct, you can try looking at how to write a binary file.
You can do it. Whether it will be useful or not depends on whether that address still contains any useful information. Generally speaking, it should be avoided.
Yes, but not for anything purposeful.
You can save it however you like, then read it back immediately, in the same process from which it came, as long as you haven't done something to invalidate the pointer. That includes:
Closing the process
Freeing the memory the pointer points, if it's on the heap (from new, malloc, HeapAlloc, etc)
Returning from the function, if it's on the stack (a local variable or parameter)
In other words, you can't do anything with it by saving it in a file that you couldn't do just by having the pointer to save in the first place.
Use files for persistent storage (i.e. state you need to keep when the process closes, invalidating all pointers) or as an abstraction for streams that share data between processes (i.e. can't share pointers anyway) and computers (i.e. can't share pointers anyway). If you need to restore a structure from a previous run, save the structure in a file. If you need to share it between processes, you can copy it by value via a file, or you are looking at a totally different set of kernel functions. If you need to share it between computers, you have to copy it.
If you don't need any of that, just use the pointer.
The structure or class should write its members to the file.
If the structure contains a pointer to another class, the container structure needs to tell the element structure to write itself to the file.
For example:
class Element
{
public:
ostream& write_to_file(ostream& output_file);
};
class Container
{
public:
ostream& write_to_file(ostream& output_file)
{
return contained_element.write_to_file(output_file);
}
private:
Element contained_element;
};
The rule is that a class knows the members it has. The class is a good candidate for writing its members to a file; not some external mechanism.
I am trying to solve the following issue: having a custom data container that manages a generic type, I need to allow for other application components to retrieve the container's internal pointer and use it as if it were a simple T* array region (without treating it as a more intelligent array holder). The problem is that this memory is, in a very special case, moved somewhere else and erased. So there are a plethora of components that are aware of the old data pointer and will use that one to access their required information.
The setup looks, pseudo-codeish, something like this:
container<T>
{
T* ptr;
public:
ContainerInterfaceCode..
}
Hypothesis:
T* ptr is a pseudo-address (may I call it "virtual"?) which is mapped in a physical space A.
When an event is risen, T* ptr's mapping will be set for another physical space, B.
Any component that uses T* ptr is then oblivious of the change of physical location, "thinking" its data is stored at that virtual address.
Conclusion:
I would therefore like to know whether there is a mechanism involving memory mapping (virtual to physical) that will allow to juggle with the mapping of the T* ptr, thus leaving other application components untouched. Simply put, T* ptr should point to a memory region that gets mapped in a certain part, and, upon request, that same pointer will be mapped in another place (where the underlying data is to be copied for consistency). This must allow seamless transitions.
Note: I can't use wrappers, smart pointers, handles, etc. for the simple fact that it means modifying a huge codebase just for one, rather minor, modification.
As I haven't found enough resources dealing with this scenario, can anyone, perhaps, present a short webography with some relevant reading material on the subject?
In linux you can used the shared memory.The shared memory is a mechanism which allow two process access a same area of memory, it is a kind of IPC method. You can find some more
details here http://en.wikipedia.org/wiki/Shared_memory.
How would you locate an object in memory, lets say that you have a struct defined as:
struct POINT {
int x;
int y;
};
How would I scan the memory region of my app to find instances of this struct so that I can read them out?
Thanks R.
You can't without adding type information to the struct. In memory a struct like that is nothing else than 2 integers so you can't recognize them any better than you could recognize any other object.
You can't. Structs don't store any type information (unless they have virtual member functions), so you can't distinguish them from any other block of sizeof(POINT) bytes.
Why don't you store your points in a vector or something?
You can't. You have to know the layout to know what section of memory have to represent a variable. That's a kind of protocol and that's why we use text based languages instead raw values.
You don't - how would you distinguish two arbitrary integers from random noise?
( but given a Point p; in your source code, you can obtain its address using the address-of operator ... Point* pp = &p;).
Short answer: you can't. Any (appropriately aligned) sequence of 8 bytes could potentially represent a POINT. In fact, an array of ints will be indistinguishable from an array of POINTS. In some cases, you could take advantage of knowledge of the compiler implementation to do better. For instance, if the struct had virtual functions, you could look for the correct vtable pointer - but there could also be false positives.
If you want to keep track of objects, you need to register them in their constructor and unregister them in their destructor (and pay the performance penalty), or give them their own allocator.
There's no way to identify that struct. You need to put the struct somewhere it can be found, on the stack or on the heap.
Sometimes data structures are tagged with identifying information to assist with debugging or memory management. As a means of data organization, it is among the worst possible approaches.
You probably need to a lot of general reading on memory management.
There is no standard way of doing this. The platform may specify some APIs which allow you to access the stack and the free store. Further, even if you did, without any additional information how would you be sure that you are reading a POINT object and not a couple of ints? The compiler/linker can read this because it deals with (albeit virtual) addresses and has some more information (and control) than you do.
You can't. Something like that would probably be possible on some "tagged" architecture that also supported tagging objects of user-defined types. But on a traditional architecture it is absolutely impossible to say for sure what is stored in memory simply by looking at the raw memory content.
You can come closer to achieving what you want by introducing a unique signature into the type, like
struct POINT {
char signature[8];
int x;
int y;
};
and carefully setting it to some fixed and "unique" pattern in each object of POINT type, and then looking for that pattern in memory. If it is your application, you can be sure with good degree of certainty that each instance of the pattern is your POINT object. But in general, of course, there will never be any guarantee that the pattern you found belongs to your object, as opposed to being there purely accidentally.
What everyone else has said is true. In memory, your struct is just a few bytes, there's nothing in particular to distinguish it.
However, if you feel like a little hacking, you can look up the internals of your C library and figure out where memory is stored on the heap and how it appears. For example, this link shows how stuff gets allocated in one particular system.
Armed with this knowledge, you could scan your heap to find allocated blocks that were sizeof(POINT), which would narrow down the search considerably. If you look at the table you'll notice that the file name and line number of the malloc() call are being recorded - if you know where in your source code you're allocating POINTs, you could use this as a reference too.
However, if your struct was allocated on the stack, you're out of luck.