Sharing heap memory in a dll between two separate applications - c++

Sorry if this question has been answered before; however all of the questions that are similar seem to be related to global or static variables in a DLL and sharing of those.
Is it possible to have one instance of a dll shared between two separate applications?
I have two applications (appA, appB) and a DLL (theDLL).
I am seeing if it is possible for appA to call a function in theDLL which then creates a variable on the heap and stores that in theDLL. At a later time, I would like to have appB connect to theDLL and be able to access that variable created earlier. Again, sorry if this answer is the same as static and global variables in dlls.
Here is some psuedo code:
(theDLL)
class VariableHolder
{
public:
void StoreVariable(int x)
{
mInt = new int(x);
}
int GetVariable()
{
return mInt;
}
private:
int mInt;
}
(appA)
int main()
{
...
(assuming we got access to a VariableHolder singleton created in theDLL)
theVarialbeHolder.StoreVariable(5);
...
}
(appB)
int main()
{
...
(assuming we got access to a VariableHolder singleton created in theDLL)
if (theVarialbeHolder.GetVariable() == 5)
{
cout << "Hurray, how did you know?";
}
...
}

This exactly is not possible - as the address spaces for the two processes are different (because they're virtual, having been created by the kernel), so a valid pointer in one won't work within the other. However, you can use shared memory to transport raw scalar data (strings, integers) between processes - here's how.

Yes, this is possible using shared memory. It doesn't need to use a shared DLL though.
Depending on the operating, the approaches are somewhat different:
On Windows, a shared file is used on mapped into memory (see Creating Named Shared Memory).
On Linux and Unix, there are direct functions to create shared memory areas, e.g. System V IPC. Just google for it.

Shared libraries on almost any modern operating system are implemented by shared read-only executable and data pages, mapped simultaneously into the address space of any process that uses the given library. On Windows though (in contrast to most Unix system) this sharing can also be extended to read-write data segments in DLLs, so it is possible to have global variables in a DLL, shared among all images that have the DLL loaded. To achieve this, there is a two-step process involved. First you tell the compiler to put the shared variables in a new named data section:
#pragma data_seg (".myshared")
int iscalar = 0;
int iarray[10] = { 0 };
#pragma data_seg ()
It is important to have all those variables statically intialised otherwise they will end up into the .bss section instead. Then you have to tell the linker that you'd like to have the .myshared section with shared read-write attributes using the /SECTION:.myshared,RWS option.
This mechanism is much simpler than creating and binding to named shared memory objects but it only allows to share statically allocated global variables - you cannot use it to share data on the heap as the heap is private to the process. For anything more complex you should use shared memory mappings, i.e. as shown on the MSDN page, linked in the answer from H2CO3.

This is not possible. The DLL can be shared in the 2 process but the data isn't. It's the code or program image (i.e. the logic or instructions) that is shared and not the data. Every Dll is mapped into the virtual address space of the process that loads it so the data either is on the data section of the process or on stack if it is local to the function. When a process is executing the address of the other process data is not visible.
You need to do some reading on virtual memory and how memory management unit(MMU) works. The OS, CPU, MMU works together to make it possible. The reliable way to do this is inter process communication. You can use shared memory where each process has a copy of data in form of virtual address but it is eventually mapped to same location into the real memory i.e the real address. The OS makes it possible.

This as #H2CO3 pointed out, is not possible because of different address spaces.
However, from your problem, it looks like you need either a surrogate process around that DLL or a Service and then different processes can connect to that surrogate process/exe and use the shared memory.

You must use shared memory (as was written above).
I recommend to use boost interprocess library. See documentation about shared memory - Shared memory between processes

Related

How are global variables in shared libraries linked?

Suppose I have shared library with this function where "i" is some global variable.
int foo() {
return i++;
}
When I call this function from multiple processes the value of "i" in each process is independent on the other processes.
This behavior is quite expected.
I was just wondering how is usually this behavior implemented by the linker? From my understanding the code is shared between processes, so the variable has to have the same virtual address in all address spaces of every program that uses this library. That condition seems quite difficult to accomplish to me so I guess I am missing something here and it is done differently.
Can I get some more detailed info on this subject?
The dynamic linking process at run time (much the same as the static linking process), allocates separate data (and bss) segments for each process, and maps those into the process address space. Only the text segments are shared between processes. This way, each process gets its own copy of static data.
the code is shared between processes, so the variable has to have the
same virtual address in all address spaces of every program that uses
this library
The code is not shared the way you think. Yes the dynamic shared object is loaded only once but the memory references or the stack or the heap that code in the so uses is not shared. Only the section that contains the code is shared.
Each process has it's own unique address space, so when a process access the variable it can have different values then the other process. If the process should share the same memory, they would have to specifically set this up. A shared library is not enough for that.

Two applications(*.exe) using same data pointers from data DLL?

I have a huge program (A) which uses about 30 (most of my own, some 3rd party) dll´s. It uses ActiveX, ATL and MFC to do different stuff. Now i want to use wxWidgets for some special tasks and will call the wxWidgets dialogs from within the program. I can do this with a special designed DLL which takes the wxW.. parts. But to run the special tasks with or without the A programm i would to like to put the wxW.. stuff in an exe (B) and these exe should address the same data from the A program. As far as i know each *.exe has its own process and so i can not share the same pointer address.
I can put in some shared data block in one of the DLLs.
#pragma data_seg("SHARED")
CClassA *g_ClassAPointer=NULL;
#pragma data_seg()
#pragma comment(linker, "/section:SHARED,RWS")
If the A is running and starts B, i can get the pointer g_ClassAPointer with the address within A. Is there a way to get the address or get an offset to reach this address within B ?
Thanks in advance,
Howie
BTW: We also want to use wxWidgets to fade all the MFC stuff more and more to cross platform code otherwise i would stick to MFC or use the wxW - DLL within a wrapper *.exe.
You're looking for shared memory, and the usual way to create that is via CreateFileMapping. This can create shared memory backed by a named file, or backed by the paging file. (Memory allocated by GlobalAlloc is also backed by the paging file, so that's no unusual thing).
In either case, the memory block from CreateFileMapping is named, so another process can access the shared memory block by calling OpenFileMapping with the same name.
Keep in mind that the shared memory block might reside at different offsets in memory. Also, if you put CClassA in shared memory, there's no automatic mechanism to ensure that all pointers inside CClassA point to the same shared memory block. E.g. putting a std::string or MFC CString in shared memory is unlikely to achieve what you intended.

Shared memory API, where a process can attach shared memory to other process

Can any one look into this and suggest me with an API.
We have APIs for a process which can create and/or attach a shared memory to its own process. But I don't find an API to attach a shared memory to one process by other process(for e.g., process A should call one API(like shmat()) to attach the shared memory to process B).
Shared memory doesn't belong to any particular process (unless you create it with a private IPC_PRIVATE key). It belongs to the system.
So, when you use shmget with a non-private key (and the IPC_CREAT flag), you will either create a shared memory block or attach to an existing one.
You need a way for both processes to use the same IPC key and this is often done by using ftok which uses a file specification and an identifier to give you an IPC key for use in the shmget call (and other IPC type calls, such as msgget or semget).
For example, in the programs pax1 and pax2, you may have a code segment like:
int getMyShMem (void) {
key_t mykey = ftok ("/var/pax.cfg", 0); // only one shm block so use id of 0
if (mykey == (key_t)-1) // no go.
return -1;
return shmget (mykey, 1024, IPC_CREAT); // get (or make) a 1K block.
}
By having both processes use the same file specification and ID, they'll get the same shared memory block.
You can use different IDs to give you distinct shared memory blocks all based on the same file (you may, for example, want one for a configuration shared memory block and another for storing shared state).
And, given that it's your configuration file the IPC key is based on, the chances of other programs using it is minuscule (I think it may be zero but I'm not 100% sure).
You can't forcefully inject shared memory into a process from outside that process (well, you may be able to but it would be both dangerous and require all sorts of root-level permissions). That would break the protected process model and turn you system into something about as secure as MS-DOS :-)
Let's see, allow one process to force a shared memory segment on to another? What is the receiver going to do with it? How will it know it now has mapped this block in - what is expected of it.
You're thinking about the problem the wrong way - simply hoisting a block of memory on to a second process is not going to allow you to do what you want. You need to notify the second process also that it has now mapped this block and so can start doing stuff with it. I suggest you take a step back and really look at your design and what you are doing. My recommended approach would be
A connects to B via some other IPC (say socket)
A informs B that it should attach with the details (name etc.)
B then attaches - and now B is aware of it and can start doing stuff with it. (say for example once the attach completes, B confirms to A, and then they can start talking over the shared memory block).
As for wrapping shared memory in a nice library - consider boost::interprocess.
You are asking to attach the process memory of other process, right?
Just open(2) the file /proc/<pid>/mem and use it. Check the /proc/<pid>/map for the list of usable address in the file.

Using shared memory under Windows. How to pass different data

I currently try to implement some interprocess communication using the Windows CreateFileMapping mechanism. I know that I need to create a file mapping object with CreateFileMapping first and then create a pointer to the actual data with MapViewOfFile. The example then puts data into the mapfile by using CopyMemory.
In my application I have an image buffer (1 MB large) which I want to send to another process. So now I inquire a pointer to the image and then copy the whole image buffer into the mapfile. But I wonder if this is really necessary. Isn't it possible to just copy an actual pointer in the shared memory which points to the image buffer data? I tried a bit but didn't succeed.
Different processes have different address spaces. If you pass a valid pointer in one process to another process, it will probably point to random data in the second process. So you will have to copy all the data.
I strongly recommend you use Boost::interprocess. It has lots of goodies to manage this kind of stuff & even includes some special Windows-only functions in case you need to interoperate w/ other processes that use particular Win32 features.
The most important thing is to use offset pointers rather than regular pointers. Offset pointers are basically relative pointers (they store the difference between where the pointer is and where the thing pointed to is). This means that even if the two pointers are mapped to different address spaces, as long as the mappings are identical in structure then you are fine.
I've used all kinds of complicated data structures with offset smart pointers and it worked like a charm.
Shared Memory doesn't mean sending and receiving of Data. Its a memory created for number of processes without violation. For that you have to follow some mechanisms like locks so that the data will not corrupt.
In process 1 :
CreateFileMapping() : It will create the Shared Memory Block, with the name provided in last parameter, if it is not already present and returns back a handle (you may call it a pointer), if successful.
MapViewOfFile() : It maps (includes) this shared block in the process address space and returns a handle (again u can say a pointer).
With this pointer returned by MapViewOfFile() only you can access that shared block.
In process 2 :
OpenFileMapping() : If the shared memory block is successfully created by CreateFileMapping(), you can use it with the same name (name used to create the shared memory block).
UnmapViewOfFile() : It will unmap (you can remove the shared memory block from that process address space). When you are done using the shared memory (i.e. access, modification etc) call this function .
Closehandle() : finally to detach the shared memory block from process , call this with argument,handle returned by OpenFileMapping() or CreateFileMapping().
Though these functions look simple, the behaviour is tricky if the flags are not selected properly.
If you wish to read or write shared memory, specify PAGE_EXECUTE_READWRITE in CreateFileMapping().
Whenever you wish to access shared memory after creating it successfully, use FILE_MAP_ALL_ACCESS in MapViewOfFile().
It is better to specify FALSE (do not inherit handle from parent process) in OpenFileMapping() as it will avoid confusion.
You CAN get shared memory to use the same address over 2 processes for Windows. It's achieveable with several techniques.
Using MapViewOfFileEx, here's the significant experpt from MSDN.
If a suggested mapping address is
supplied, the file is mapped at the
specified address (rounded down to the
nearest 64K-boundary) if there is
enough address space at the specified
address. If there is not enough
address space, the function fails.
Typically, the suggested address is
used to specify that a file should be
mapped at the same address in multiple
processes. This requires the region of
address space to be available in all
involved processes. No other memory
allocation can take place in the
region that is used for mapping,
including the use of the VirtualAlloc
or VirtualAllocEx function to reserve
memory.
If the lpBaseAddress parameter
specifies a base offset, the function
succeeds if the specified memory
region is not already in use by the
calling process. The system does not
ensure that the same memory region is
available for the memory mapped file
in other 32-bit processes.
Another related technique is to use a DLL with a section marked Read + Write + Shared. In this case, the OS will pretty much do the MapViewOfFileEx call for you and for any other process which loads the DLL.
You may have to mark your DLL to a FIXED load address, not relocateable etc.. naturally.
You can use Marshalling of pointers.
If it's possible, it would be best to have the image data loaded/generated directly into the shared memory area. This eliminates the memory copy and puts it directly where it needs to be. When it's ready you can signal the other process, giving it the offset into your shared memory where the data begins.

Creating an object in shared memory inside a Shared Lib (so) in C++

Is it possible to share a single 'god' instance among everyone that links to this code, to be placed in a shared object?
god* _god = NULL;
extern "C"
{
int set_log_level(int level)
{
if(_god == NULL) return -1;
_stb->log_level(level);
return 0;
}
int god_init(){
if(_god == NULL){
_god = new god(); //Magic happens here
}
}
}
Provided that I perform a lock synchronization at the beginning of every function, and considering that God itself can new/malloc other things, but those things will never be returned themselves to the caller (God mallocs only for internal use), what is the simplest way of doing this, if possible.
How can that be extended to an arbitrary number of programs linked to this shared library?
Boost Interprocess library has high(er) level, portable shared memory objects.
This isn't the correct approach at all. By doing what you suggest, the variable, yes, is global to the library, and thus the program, but the data is private to the actual running process. You won't be able to share the values across running programs. #grieve is referring to a global accessed by multiple threads, but threads share the same parent process instance.
Across actual processes, you need to break out to an OS specific shared memory facility.
Take a look at Shared Memory for details. It's a doable issue, but it's not particularly trivial to pull off. You'll also need a interprocess synchronization system like Semaphores as well to coordinate usage.
I have feeling that god will be a server of some kind. Consider using a proper client/server architecture, so as to keep god away from the masses.