I'm trying to implement IPC for a school assignment by sharing memory.
I made a class called SharedMemoryBuffer to deal with creating file mappings and views.
My Init() function looks like this:
BYTE * SharedMemoryBuffer::Init(const wchar_t * name, size_t bufferSize)
{
FileMapHandle = OpenFileMapping(
FILE_MAP_ALL_ACCESS, // read/write access
FALSE, // do not inherit the name
name); // name of mapping object
if (FileMapHandle == NULL)
{
FileMapHandle =
CreateFileMapping(INVALID_HANDLE_VALUE,
NULL,
PAGE_READWRITE,
0,
bufferSize,
name);
pBuf = (BYTE*)MapViewOfFile(FileMapHandle, // handle to map object
FILE_MAP_ALL_ACCESS, // read/write permission
0,
0,
bufferSize);
}
else
{
pBuf = (BYTE*)MapViewOfFile(
FileMapHandle,
FILE_MAP_ALL_ACCESS,
0,
0,
bufferSize
);
}
return this->GetBuffer();
}
Essentially, I pass it a name and size and it tries to open a mapping with this name. If it fails, it creates it instead.
I call it like so
this->ringBuffer.Init(widestr.c_str(), buffSize);
After this is done (I call Init 4 times for 2 buffers, from the same process) I print out the addresses of the buffers (pBuf from Init()) but theyre all different addresses.
I cant for the love of my life figure out why the addresses would be different!
I have made sure that the second time i call Init() with the same name that it does indeed open the file mapping successfully.
source: https://github.com/RavioliFinoli/SharedMemory
You are mapping the same region twice in your process. You will get two distinct addresses, but they are backed by the same physical memory. Writing into the buffer pointed by the first address modifies the buffer pointed to by the second address, since they are really the same memory.
I've been attempting to use ReadFileScatter today in my code (which sounds like exactly what I need), so far without a lot of luck. Google'ing the internet for what goes wrong doesn't give me much insight.
The documentation states:
The array must contain enough elements to store nNumberOfBytesToRead bytes of data, plus one element for the terminating NULL. For example, if there are 40 KB to be read and the page size is 4 KB, the array must have 11 elements that includes 10 for the data and one for the NULL.
Each buffer must be at least the size of a system memory page and must be aligned on a system memory page size boundary. The system reads one system memory page of data into each buffer.
The function stores the data in the buffers in sequential order. For example, it stores data into the first buffer, then into the second buffer, and so on until each buffer is filled and all the data is stored, or there are no more buffers.
So far I've been attempting to do just that. I allocated a bunch of bytes using VirtualAlloc (which ensures the page boundary constraint), add a terminator NULL to the list, ensure the data on disk is on the system boundary (and implicitly a disk sector size boundary) as well and issue the call.
Without further due, here's the minimum test case in C++:
// Setup: c:\tmp\test.dat is a file with at least 12K of stuff.
// I attempt to read page 2/3, e.g. offset [4096-4096+8192>
// TEST:
SYSTEM_INFO systemInfo;
GetSystemInfo(&systemInfo);
auto pageSize = systemInfo.dwPageSize;
std::cout << "Page size: "<< pageSize << std::endl;
// Allocate 2 pages that are aligned with one in the middle:
auto buffer = reinterpret_cast<char*>(VirtualAlloc(NULL, pageSize * 3, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE));
// Create read buffer:
std::vector<FILE_SEGMENT_ELEMENT> elements;
{
FILE_SEGMENT_ELEMENT element1;
element1.Buffer = buffer;
elements.push_back(element1);
}
{
FILE_SEGMENT_ELEMENT element2;
element2.Buffer = buffer + pageSize * 2;
elements.push_back(element2);
}
{
FILE_SEGMENT_ELEMENT terminator;
terminator.Buffer = NULL;
elements.push_back(terminator);
}
// [..] Physical sector size is normally checked as well. In my case it's 512 bytes,
// so I guess that's irrelevant here.
//
// Open file:
auto fileHandle = CreateFile(
"c:\\tmp\\test.dat",
GENERIC_READ | GENERIC_WRITE,
FILE_SHARE_READ,
NULL,
OPEN_ALWAYS,
FILE_FLAG_NO_BUFFERING | FILE_FLAG_WRITE_THROUGH,
NULL);
auto err = GetLastError();
if (err != ERROR_ALREADY_EXISTS && err != ERROR_SUCCESS)
{
throw std::exception(); // FIXME.
}
OVERLAPPED overlapped;
memset(&overlapped, 0, sizeof(OVERLAPPED));
LARGE_INTEGER tmp;
tmp.QuadPart = 4096; // Read from disk page 1
overlapped.Offset = tmp.LowPart;
overlapped.OffsetHigh = tmp.HighPart;
overlapped.hEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
auto succes = ReadFileScatter(fileHandle, elements.data(), pageSize * 2, NULL, &overlapped);
err = GetLastError();
if (!succes && err != ERROR_IO_PENDING && err != ERROR_SUCCESS)
{
throw std::exception(); // The call always ends up here with error 87: Invalid parameter
}
WaitForSingleObject(overlapped.hEvent, INFINITE);
std::cout << "Call succeeded!" << std::endl;
// FIXME: Proper exception handling.
// Clean up:
VirtualFree(buffer, pageSize * 3, MEM_DECOMMIT | MEM_RELEASE);
CloseHandle(overlapped.hEvent);
CloseHandle(fileHandle);
In the code, the error is noted with the comment // The call always ends up here with error 87: Invalid parameter. However, as far as I can see, I check all the boxes that they describe on MSDN... so...
What am I doing wrong here?
I come to you because I have some problems when I trying to write a driver supporting the QL355P from TTI. I already wrote the driver in VB6, but now, I am trying to write it in C++. I use the ReadFile/WriteFile function to communicate with the interface. The problem is when I send the identification command (*IDN?) and the endchar (.), I have no response (I have Statut_timeout).
My code is:
DCB l_dcb;
BOOL l_Success;
COMMTIMEOUTS l_TimeOuts;
const char *command = "*IDN?.";
// open an access to serial port
m_ComPort = CreateFile(L"COM2",
GENERIC_READ | GENERIC_WRITE,
0, // comm devices must be opened w/exclusive-access
NULL, // no security attrs
OPEN_EXISTING, // comm devices must use OPEN_EXISTING
0, // not overlapped I/O
NULL // hTemplate must be NULL for comm devices
);
// get the current configuration
l_Success = GetCommState(m_ComPort, &l_dcb);
// fill in the DCB with configuation:
// baud=19200, 8 data bits, no parity, 1 stop bit
l_dcb.BaudRate = 19200;
l_dcb.ByteSize = 8;
l_dcb.Parity = NOPARITY;
l_dcb.StopBits = ONESTOPBIT;
// configure the port
l_Success = SetCommState(m_ComPort, &l_dcb);
// get the current time-outs configuration
l_Success = GetCommTimeouts(m_ComPort, &l_TimeOuts);
// change the time-outs
l_TimeOuts.ReadIntervalTimeout=50;
l_TimeOuts.ReadTotalTimeoutMultiplier=50;
l_TimeOuts.ReadTotalTimeoutConstant=1000;
// sets the time-outs configuration
l_Success = SetCommTimeouts(m_ComPort, &l_TimeOuts);
// first purge the input buffer
PurgeComm(m_ComPort,PURGE_RXCLEAR);
// send the command
DWORD l_NumBytesTransfered=0;
DWORD l_NumBytesToTransfer=strlen(command);
l_Success = WriteFile(m_ComPort,command,l_NumBytesToTransfer,&l_NumBytesTransfered,NULL);
//Try to read
char l_Char='*';
int i=0;
char l_Pointer[100];
while(l_Char !=0x0a && i<100)
{
l_Success = ReadFile(m_ComPort,&l_Char,1,&l_NumBytesTransfered,NULL);
l_Pointer[i] = l_Char;
i++;
}
CloseHandle(m_ComPort); //close the handle
ReadFile can return 0 in case of a timeout, or various possible errors. Your code snippet ignores that possibility and puts garbage into the input array even when ReadFile fails.
I am interested in the prospect of using memory mapped IO, preferably
exploiting the facilities in boost::interprocess for cross-platform
support, to map non-contiguous system-page-size blocks in a file into
a contiguous address space in memory.
A simplified concrete scenario:
I've a number of 'plain-old-data' structures, each of a fixed length
(less than the system page size.) These structures are concatenated
into a (very long) stream with the type & location of structures
determined by the values of those structures that proceed them in the
stream. I'm aiming to minimize latency and maximize throughput in a
demanding concurrent environment.
I can read this data very effectively by memory-mapping it in blocks
of at least twice the system-page-size... and establishing a new
mapping immediately having read a structure extending beyond the
penultimate system-page-boundary. This allows the code that interacts
with the plain-old-data structures to be blissfully unaware that these
structures are memory mapped... and, for example, could compare two
different structures using memcmp() directly without having to care
about page boundaries.
Where things get interesting is with respect to updating these data
streams... while they're being (concurrently) read. The strategy I'd
like to use is inspired by 'Copy On Write' on a system-page-size
granularity... essentially writing 'overlay-pages' - allowing one
process to read the old data while another reads the updated data.
While managing which overlay pages to use, and when, isn't necessarily
trivial... that's not my main concern. My main concern is that I may
have a structure spanning pages 4 and 5, then update a
structure wholly contained in page 5... writing the new page in
location 6... leaving page 5 to be 'garbage collected' when it is
determined to be no-longer reachable. This means that, if I map page
4 into location M, I need to map page 6 into memory location
M+page_size... in order to be able to reliably process structures that
cross page boundaries using existing (non-memory-mapping-aware) functions.
I'm trying to establish the best strategy, and I'm hampered by
documentation I feel is incomplete. Essentially, I need to decouple
allocation of address space from memory mapping into that address
space. With mmap(), I'm aware that I can use MAP_FIXED - if I wish to
explicitly control the mapping location... but I'm unclear how I
should reserve address space in order to do this safely. Can I map
/dev/zero for two pages without MAP_FIXED, then use MAP_FIXED twice to
map two pages into that allocated space at explicit VM addresses? If
so, should I call munmap() three times too? Will it leak resources
and/or have any other untoward overhead? To make the issue even more
complex, I'd like comparable behaviour on Windows... is there any way
to do this? Are there neat solutions if I were to compromise my
cross-platform ambitions?
--
Thanks for your answer, Mahmoud... I've read, and think I've understood that code... I've compiled it under Linux and it behaves as you suggest.
My main concerns are with line 62 - using MAP_FIXED. It makes some assumptions about mmap, which I've been unable to confirm when I read the documentation I can find. You're mapping the 'update' page into the same address space as mmap() returned initially - I assume that this is 'correct' - i.e. not something that just happens to work on Linux? I'd also need to assume that it works cross-platform for file-mappings as well as anonymous mappings.
The sample definitely moves me forwards... documenting that what I ultimately need is probably achievable with mmap() on Linux - at least. What I'd really like is a pointer to documentation that shows that the MAP_FIXED line will work as the sample demonstrates... and, idealy, a transformation from the Linux/Unix specific mmap() to a platform independent (Boost::interprocess) approach.
Your question is a little confusing. From what I understood, this code will do what you need:
#define PAGESIZE 4096
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include <errno.h>
#include <sys/types.h>
#include <fcntl.h>
#include <unistd.h>
#include <assert.h>
struct StoredObject
{
int IntVal;
char StrVal[25];
};
int main(int argc, char **argv)
{
int fd = open("mmapfile", O_RDWR | O_CREAT | O_TRUNC, (mode_t) 0600);
//Set the file to the size of our data (2 pages)
lseek(fd, PAGESIZE*2 - 1, SEEK_SET);
write(fd, "", 1); //The final byte
unsigned char *mapPtr = (unsigned char *) mmap(0, PAGESIZE * 2, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
struct StoredObject controlObject;
controlObject.IntVal = 12;
strcpy(controlObject.StrVal, "Mary had a little lamb.\n");
struct StoredObject *mary1;
mary1 = (struct StoredObject *)(mapPtr + PAGESIZE - 4); //Will fall on the boundary between first and second page
memcpy(mary1, &controlObject, sizeof(StoredObject));
printf("%d, %s", mary1->IntVal, mary1->StrVal);
//Should print "12, Mary had a little lamb.\n"
struct StoredObject *john1;
john1 = mary1 + 1; //Comes immediately after mary1 in memory; will start and end in the second page
memcpy(john1, &controlObject, sizeof(StoredObject));
john1->IntVal = 42;
strcpy(john1->StrVal, "John had a little lamb.\n");
printf("%d, %s", john1->IntVal, john1->StrVal);
//Should print "12, Mary had a little lamb.\n"
//Make sure the data's on the disk, as this is the initial, "read-only" data
msync(mapPtr, PAGESIZE * 2, MS_SYNC);
//This is the inital data set, now in memory, loaded across two pages
//At this point, someone could be reading from there. We don't know or care.
//We want to modify john1, but don't want to write over the existing data
//Easy as pie.
//This is the shadow map. COW-like optimization will take place:
//we'll map the entire address space from the shared source, then overlap with a new map to modify
//This is mapped anywhere, letting the system decide what address we'll be using for the new data pointer
unsigned char *mapPtr2 = (unsigned char *) mmap(0, PAGESIZE * 2, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
//Map the second page on top of the first mapping; this is the one that we're modifying. It is *not* backed by disk
unsigned char *temp = (unsigned char *) mmap(mapPtr2 + PAGESIZE, PAGESIZE, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_FIXED | MAP_ANON, 0, 0);
if (temp == MAP_FAILED)
{
printf("Fixed map failed. %s", strerror(errno));
}
assert(temp == mapPtr2 + PAGESIZE);
//Make a copy of the old data that will later be changed
memcpy(mapPtr2 + PAGESIZE, mapPtr + PAGESIZE, PAGESIZE);
//The two address spaces should still be identical until this point
assert(memcmp(mapPtr, mapPtr2, PAGESIZE * 2) == 0);
//We can now make our changes to the second page as needed
struct StoredObject *mary2 = (struct StoredObject *)(((unsigned char *)mary1 - mapPtr) + mapPtr2);
struct StoredObject *john2 = (struct StoredObject *)(((unsigned char *)john1 - mapPtr) + mapPtr2);
john2->IntVal = 52;
strcpy(john2->StrVal, "Mike had a little lamb.\n");
//Test that everything worked OK
assert(memcmp(mary1, mary2, sizeof(struct StoredObject)) == 0);
printf("%d, %s", john2->IntVal, john2->StrVal);
//Should print "52, Mike had a little lamb.\n"
//Now assume our garbage collection routine has detected that no one is using the original copy of the data
munmap(mapPtr, PAGESIZE * 2);
mapPtr = mapPtr2;
//Now we're done with all our work and want to completely clean up
munmap(mapPtr2, PAGESIZE * 2);
close(fd);
return 0;
}
My modified answer should address your safety concerns.
Only use MAP_FIXED on the second mmap call (like I have above). The cool thing about MAP_FIXED is that it lets you overwrite an existing mmap address section. It'll unload the range you're overlapping and replace it with your new mapped content:
MAP_FIXED
[...] If the memory
region specified by addr and len overlaps pages of any existing
mapping(s), then the overlapped part of the existing mapping(s) will be
discarded. [...]
This way, you let the OS take care of finding a contiguous memory block of hundreds of megs for you (never call MAP_FIXED on address you don't know for sure isn't available). Then you call MAP_FIXED on a subsection of that now-mapped huge space with the data that you will be modifying. Tada.
On Windows, something like this should work (I'm on a Mac at the moment, so untested):
int main(int argc, char **argv)
{
HANDLE hFile = CreateFile(L"mmapfile", GENERIC_READ | GENERIC_WRITE, FILE_SHARE_READ, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
//Set the file to the size of our data (2 pages)
SetFilePointer(hFile, PAGESIZE*2 - 1, 0, FILE_BEGIN);
DWORD bytesWritten = -1;
WriteFile(hFile, "", 1, &bytesWritten, NULL);
HANDLE hMap = CreateFileMapping(hFile, NULL, PAGE_READWRITE, 0, PAGESIZE * 2, NULL);
unsigned char *mapPtr = (unsigned char *) MapViewOfFile(hMap, FILE_MAP_READ | FILE_MAP_WRITE, 0, 0, PAGESIZE * 2);
struct StoredObject controlObject;
controlObject.IntVal = 12;
strcpy(controlObject.StrVal, "Mary had a little lamb.\n");
struct StoredObject *mary1;
mary1 = (struct StoredObject *)(mapPtr + PAGESIZE - 4); //Will fall on the boundary between first and second page
memcpy(mary1, &controlObject, sizeof(StoredObject));
printf("%d, %s", mary1->IntVal, mary1->StrVal);
//Should print "12, Mary had a little lamb.\n"
struct StoredObject *john1;
john1 = mary1 + 1; //Comes immediately after mary1 in memory; will start and end in the second page
memcpy(john1, &controlObject, sizeof(StoredObject));
john1->IntVal = 42;
strcpy(john1->StrVal, "John had a little lamb.\n");
printf("%d, %s", john1->IntVal, john1->StrVal);
//Should print "12, Mary had a little lamb.\n"
//Make sure the data's on the disk, as this is the initial, "read-only" data
//msync(mapPtr, PAGESIZE * 2, MS_SYNC);
//This is the inital data set, now in memory, loaded across two pages
//At this point, someone could be reading from there. We don't know or care.
//We want to modify john1, but don't want to write over the existing data
//Easy as pie.
//This is the shadow map. COW-like optimization will take place:
//we'll map the entire address space from the shared source, then overlap with a new map to modify
//This is mapped anywhere, letting the system decide what address we'll be using for the new data pointer
unsigned char *reservedMem = (unsigned char *) VirtualAlloc(NULL, PAGESIZE * 2, MEM_RESERVE, PAGE_READWRITE);
HANDLE hMap2 = CreateFileMapping(hFile, NULL, PAGE_READWRITE, 0, PAGESIZE, NULL);
unsigned char *mapPtr2 = (unsigned char *) MapViewOfFileEx(hMap2, FILE_MAP_READ | FILE_MAP_WRITE, 0, 0, PAGESIZE, reservedMem);
//Map the second page on top of the first mapping; this is the one that we're modifying. It is *not* backed by disk
unsigned char *temp = (unsigned char *) MapViewOfFileEx(hMap2, FILE_MAP_READ | FILE_MAP_WRITE, 0, 0, PAGESIZE, reservedMem + PAGESIZE);
if (temp == NULL)
{
printf("Fixed map failed. 0x%x\n", GetLastError());
return -1;
}
assert(temp == mapPtr2 + PAGESIZE);
//Make a copy of the old data that will later be changed
memcpy(mapPtr2 + PAGESIZE, mapPtr + PAGESIZE, PAGESIZE);
//The two address spaces should still be identical until this point
assert(memcmp(mapPtr, mapPtr2, PAGESIZE * 2) == 0);
//We can now make our changes to the second page as needed
struct StoredObject *mary2 = (struct StoredObject *)(((unsigned char *)mary1 - mapPtr) + mapPtr2);
struct StoredObject *john2 = (struct StoredObject *)(((unsigned char *)john1 - mapPtr) + mapPtr2);
john2->IntVal = 52;
strcpy(john2->StrVal, "Mike had a little lamb.\n");
//Test that everything worked OK
assert(memcmp(mary1, mary2, sizeof(struct StoredObject)) == 0);
printf("%d, %s", john2->IntVal, john2->StrVal);
//Should print "52, Mike had a little lamb.\n"
//Now assume our garbage collection routine has detected that no one is using the original copy of the data
//munmap(mapPtr, PAGESIZE * 2);
mapPtr = mapPtr2;
//Now we're done with all our work and want to completely clean up
//munmap(mapPtr2, PAGESIZE * 2);
//close(fd);
return 0;
}
but I'm unclear how I should reserve address space in order to do this safely
That's going to vary by OS, but a little digging on msdn for mmap (I started with "xp mmap" on the msdn search) shows Microsoft have their usual VerboseAndHelpfullyCapitalizedNames for (the many) functions that implement pieces of mmap. Both the file- and anonymous- mappers can handle fixed-address requests just the same as any POSIX-2001 system can, i.e. if something else in your address space is talking to the kernel, you get to sort it out. No way I'm going to touch "safely", there's no such thing as "safely" with code you're wanting to port to unspecified platforms. You're going to have to build your own pool of pre-mapped anonymous memory that you can unmap and parcel out later under your own control.
I tested the windows code from #Mahmoud, well actually I tested the following similar code, and it doesn't work (the Linux code works.) If you uncomment VirtualFree, it will work. As mentioned in my comment above, on windows you can reserve the address space with VirtualAlloc, but you can't use MapViewOfFileEx with an already mapped address, so you need to VirtualFree it first. Then there's a race condition where another thread can grab the memory address before you do, so you have to do everything in a loop, e.g. try up to 1000 times and then give up.
package main
import (
"fmt"
"os"
"syscall"
)
func main() {
const size = 1024 * 1024
file, err := os.Create("foo.dat")
if err != nil {
panic(err)
}
if err := file.Truncate(size); err != nil {
panic(err)
}
const MEM_COMMIT = 0x1000
addr, err := virtualAlloc(0, size, MEM_COMMIT, protReadWrite)
if err != nil {
panic(err)
}
fd, err := syscall.CreateFileMapping(
syscall.Handle(file.Fd()),
nil,
uint32(protReadWrite),
0,
uint32(size),
nil,
)
//if err := virtualFree(addr); err != nil {
// panic(err)
//}
base, err := mapViewOfFileEx(fd, syscall.FILE_MAP_READ|syscall.FILE_MAP_WRITE, 0, 0, size, addr)
if base == 0 {
panic("mapViewOfFileEx returned 0")
}
if err != nil {
panic(err)
}
fmt.Println("success!")
}
type memProtect uint32
const (
protReadOnly memProtect = 0x02
protReadWrite memProtect = 0x04
protExecute memProtect = 0x20
protAll memProtect = 0x40
)
var (
modkernel32 = syscall.MustLoadDLL("kernel32.dll")
procMapViewOfFileEx = modkernel32.MustFindProc("MapViewOfFileEx")
procVirtualAlloc = modkernel32.MustFindProc("VirtualAlloc")
procVirtualFree = modkernel32.MustFindProc("VirtualFree")
procVirtualProtect = modkernel32.MustFindProc("VirtualProtect")
)
func mapViewOfFileEx(handle syscall.Handle, prot memProtect, offsetHigh uint32, offsetLow uint32, length uintptr, target uintptr) (addr uintptr, err error) {
r0, _, e1 := syscall.Syscall6(procMapViewOfFileEx.Addr(), 6, uintptr(handle), uintptr(prot), uintptr(offsetHigh), uintptr(offsetLow), length, target)
addr = uintptr(r0)
if addr == 0 {
if e1 != 0 {
err = error(e1)
} else {
err = syscall.EINVAL
}
}
return addr, nil
}
func virtualAlloc(addr, size uintptr, allocType uint32, prot memProtect) (mem uintptr, err error) {
r0, _, e1 := syscall.Syscall6(procVirtualAlloc.Addr(), 4, addr, size, uintptr(allocType), uintptr(prot), 0, 0)
mem = uintptr(r0)
if e1 != 0 {
return 0, error(e1)
}
return mem, nil
}
func virtualFree(addr uintptr) error {
const MEM_RELEASE = 0x8000
_, _, e1 := syscall.Syscall(procVirtualFree.Addr(), 3, addr, 0, MEM_RELEASE)
if e1 != 0 {
return error(e1)
}
return nil
}