Use of fread(), fwrite() and malloc in C++ - c++

I'm having amazing difficulties using the fread and fwrite functions in C++.
The project is writing a rudimentary FAT16 file system and we are restricted to using fread and fwrite.
When I am initially writing the file my code looks like this:
directoryTable = (directoryEntry *)malloc(clusterSize);
for (int i = 0; i < bootRecord[0] / 128 ; ++i){
directoryEntry * newEntry = (directoryEntry *)malloc(sizeof(directoryEntry));
newEntry->name = (char *)malloc(112);
newEntry->name[0] = 'a'
write(1, &(newEntry->name[1]), 1);
newEntry->size = 0;
newEntry->type = 0;
newEntry->creation = 0x0000;
newEntry->index = 0;
fwrite(newEntry->name, 112, 1, fp);
fwrite(&newEntry->size, sizeof(int), 1, fp);
fwrite(&newEntry->type, sizeof(int), 1, fp);
fwrite(&newEntry->creation, sizeof(int), 1, fp);
fwrite(&newEntry->index, sizeof(int), 1, fp);
directoryTable[i] = *newEntry;
}
When I'm assigning the first character of directoryEntry->name to 'a', my intent is actually to assign it the value 0x00 so I can check later if it's null. I simply am using a right for debugging purposes. My problem is when I read it, I get nothing back.
And when I'm reading my code looks like this:
fseek(fp, clusterSize * root, SEEK_SET);
for(int i = 0; i < clusterSize / 128; ++i){
directoryEntry * newEntry = (directoryEntry *) malloc(128);
newEntry->name = (char *) malloc(112);
fread(newEntry->name, 112, 1, fp);
write(1, newEntry->name[0],1);
fread(&newEntry->size, sizeof(int), 1, fp);
fread(&newEntry->type, sizeof(int), 1, fp);
fread(&newEntry->creation, sizeof(int), 1, fp);
fread(&newEntry->index, sizeof(int), 1, fp);
directoryTable[i] = *newEntry;
}
It should be noted that the values of clusterSize and root are also read in using similar methods. I've already checked and their values are accurate in both situations. Since I was able to read those in without a problem, I have no idea why I'm having such a big problem now. I feel my use of malloc is not quite right, I've never worked with it before.
Also, here is a definition of directoryTable if needed:
typedef struct{
char * name;
unsigned int index;
unsigned int size;
unsigned int type;
unsigned int creation;
} directoryEntry;
Thank you guys for you time and if you need me to clarify on anything I'd be happy to.

I see a lot of little problems:
You do not zero out the memory you malloc'd for name. Try memset(newentry->name, 0, 112);
You are writing the second element of the name array when you write it (index 1), and the first element of the name array when you read it (index 0): write(1, &(newEntry->name[1]), 1); vs. write(1, newEntry->name[0],1);
Additionally in the first write you are taking the address of it, and in the second write you are not, so it looks likes the second one should read: write(1, &(newEntry->name[0]),1);
The call to fseek looks correct, but as a reader of the question I have no way of knowing if clusterSize * root is the correct offset into the file you are reading from. You also do not seek to that offset when you start writing the file (or at least don't show it), so it becomes murkier.
What exactly does the 128 you keep dividing by in your for loops represent? Also in one spot you divide clusterSize by it and in another bootrecord[0]. I assume those are the same size, but...
In the read loop you malloc 128 bytes of memory, but in the write loop you malloc (sizeof(directoryEntry). Those are two different sizes, and malloc'ing the 128 bytes is wrong, since the structure contains a pointer to the 112 bytes, and not the actual 112 bytes for name.
I am not sure if any of these are causing the problem you are seeing, which you don't actually specify by the way, but it may at least get you pointed in the right direction.

Related

file size and buffer overshoot

I have a function which opens a file from an SD card, uses the file size to set the size of a buffer, writes a block of information to that buffer, then does something with that information, as shown in this code:
char filename = "filename.txt";
uint16_t duration;
uint16_t pixel;
int q = 0;
int w = 0;
bool largefile;
File f;
int readuntil;
long large_buffer;
f = SD.open(filename);
if(f.size() > 3072) {
w = 3072;
} else {
w = f.size();
}
uint8_t buffer[w];
while(f.available()) {
f.read(buffer, sizeof(buffer));
while(q < sizeof(buffer)) {
doStuffWithInformation(buffer[q++]);
}
q=0;
}
f.close;
This works great with smaller file sizes, but anything over the hard limit buffer size of 3072 (which I arrived at empirically, its just the amount of memory that can be safely committed to this function), runs into a problem. Larger files read fine until they hit the last loop of while(f.available()), where they read the end of the file, but then continue reading the buffer, the tail end of which is filled with data from the last loop, that wasn't overwritten by the latest f.read(). How can I make sure that the last loop of the while(f.available()) function only works with the information that was written to the buffer during the current loop? My only idea right now is to solve for factors of the file size, and set the buffer size as the largest factor less than 3072, but this seems intensive to run every time this function is called. Is there an elegant solution staring me in the face?
Your program is not behaving correctly because f.read() is not guaranteed to read the whole buffer. Moreover, it is bound to happen when you read the last chunk of the file, unless the file size is a factor of buffer size (3072 in your case).
While Arduino specification (https://www.arduino.cc/en/Reference/FileRead) doesn't say so, SD.read function returns the number of bytes read. See code of the library here: https://github.com/arduino-libraries/SD/blob/master/src/utility/SdFile.cpp, int16_t SdFile::read(void* buf, uint16_t nbyte)
Knowing that, you should change your loop as following (while also rewriting it as a for loop for better readability and removing q definition above):
while(f.available()) {
uint16_t sz = f.read(buffer, sizeof(buffer));
for (uint16_t q = 0; q < sz; ++q) {
doStuffWithInformation(buffer[q]);
}
}
On a side note, now, when you have this logic in place, it would make sense for you to do away with variable length array and use a fixed buffer of size 512 - the standard sector size on the SD card. Most likely, it will yield the same performance in regards to read, and slightly better performance in regards to sizeof, which will becomes a compile-time constant rather than a run-time calculation. This also makes your program simpler. This makes for following code:
f = SD.open(filename);
...
uint8_t buffer[512];

List of functions exported by a DLL in WinAPI [duplicate]

I found similar questions but no answer to what I am looking for. So here goes:
For a native Win32 dll, is there a Win32 API to enumerate its export function names?
dumpbin /exports is pretty much what you want, but that's a developer tool, not a Win32 API.
LoadLibraryEx with DONT_RESOLVE_DLL_REFERENCES is heavily cautioned against, but happens to be useful for this particular case – it does the heavy lifting of mapping the DLL into memory (but you don't actually need or want to use anything from the library), which makes it trivial for you to read the header: the module handle returned by LoadLibraryEx points right at it.
#include <winnt.h>
HMODULE lib = LoadLibraryEx("library.dll", NULL, DONT_RESOLVE_DLL_REFERENCES);
assert(((PIMAGE_DOS_HEADER)lib)->e_magic == IMAGE_DOS_SIGNATURE);
PIMAGE_NT_HEADERS header = (PIMAGE_NT_HEADERS)((BYTE *)lib + ((PIMAGE_DOS_HEADER)lib)->e_lfanew);
assert(header->Signature == IMAGE_NT_SIGNATURE);
assert(header->OptionalHeader.NumberOfRvaAndSizes > 0);
PIMAGE_EXPORT_DIRECTORY exports = (PIMAGE_EXPORT_DIRECTORY)((BYTE *)lib + header->
OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_EXPORT].VirtualAddress);
assert(exports->AddressOfNames != 0);
BYTE** names = (BYTE**)((int)lib + exports->AddressOfNames);
for (int i = 0; i < exports->NumberOfNames; i++)
printf("Export: %s\n", (BYTE *)lib + (int)names[i]);
Totally untested, but I think it's more or less correct. (Famous last words.)
try this:
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
void EnumExportedFunctions (char *, void (*callback)(char*));
int Rva2Offset (unsigned int);
typedef struct {
unsigned char Name[8];
unsigned int VirtualSize;
unsigned int VirtualAddress;
unsigned int SizeOfRawData;
unsigned int PointerToRawData;
unsigned int PointerToRelocations;
unsigned int PointerToLineNumbers;
unsigned short NumberOfRelocations;
unsigned short NumberOfLineNumbers;
unsigned int Characteristics;
} sectionHeader;
sectionHeader *sections;
unsigned int NumberOfSections = 0;
int Rva2Offset (unsigned int rva) {
int i = 0;
for (i = 0; i < NumberOfSections; i++) {
unsigned int x = sections[i].VirtualAddress + sections[i].SizeOfRawData;
if (x >= rva) {
return sections[i].PointerToRawData + (rva + sections[i].SizeOfRawData) - x;
}
}
return -1;
}
void EnumExportedFunctions (char *szFilename, void (*callback)(char*)) {
FILE *hFile = fopen (szFilename, "rb");
if (hFile != NULL) {
if (fgetc (hFile) == 'M' && fgetc (hFile) == 'Z') {
unsigned int e_lfanew = 0;
unsigned int NumberOfRvaAndSizes = 0;
unsigned int ExportVirtualAddress = 0;
unsigned int ExportSize = 0;
int i = 0;
fseek (hFile, 0x3C, SEEK_SET);
fread (&e_lfanew, 4, 1, hFile);
fseek (hFile, e_lfanew + 6, SEEK_SET);
fread (&NumberOfSections, 2, 1, hFile);
fseek (hFile, 108, SEEK_CUR);
fread (&NumberOfRvaAndSizes, 4, 1, hFile);
if (NumberOfRvaAndSizes == 16) {
fread (&ExportVirtualAddress, 4, 1, hFile);
fread (&ExportSize, 4, 1, hFile);
if (ExportVirtualAddress > 0 && ExportSize > 0) {
fseek (hFile, 120, SEEK_CUR);
if (NumberOfSections > 0) {
sections = (sectionHeader *) malloc (NumberOfSections * sizeof (sectionHeader));
for (i = 0; i < NumberOfSections; i++) {
fread (sections[i].Name, 8, 1, hFile);
fread (&sections[i].VirtualSize, 4, 1, hFile);
fread (&sections[i].VirtualAddress, 4, 1, hFile);
fread (&sections[i].SizeOfRawData, 4, 1, hFile);
fread (&sections[i].PointerToRawData, 4, 1, hFile);
fread (&sections[i].PointerToRelocations, 4, 1, hFile);
fread (&sections[i].PointerToLineNumbers, 4, 1, hFile);
fread (&sections[i].NumberOfRelocations, 2, 1, hFile);
fread (&sections[i].NumberOfLineNumbers, 2, 1, hFile);
fread (&sections[i].Characteristics, 4, 1, hFile);
}
unsigned int NumberOfNames = 0;
unsigned int AddressOfNames = 0;
int offset = Rva2Offset (ExportVirtualAddress);
fseek (hFile, offset + 24, SEEK_SET);
fread (&NumberOfNames, 4, 1, hFile);
fseek (hFile, 4, SEEK_CUR);
fread (&AddressOfNames, 4, 1, hFile);
unsigned int namesOffset = Rva2Offset (AddressOfNames), pos = 0;
fseek (hFile, namesOffset, SEEK_SET);
for (i = 0; i < NumberOfNames; i++) {
unsigned int y = 0;
fread (&y, 4, 1, hFile);
pos = ftell (hFile);
fseek (hFile, Rva2Offset (y), SEEK_SET);
char c = fgetc (hFile);
int szNameLen = 0;
while (c != '\0') {
c = fgetc (hFile);
szNameLen++;
}
fseek (hFile, (-szNameLen)-1, SEEK_CUR);
char* szName = calloc (szNameLen + 1, 1);
fread (szName, szNameLen, 1, hFile);
callback (szName);
fseek (hFile, pos, SEEK_SET);
}
}
}
}
}
fclose (hFile);
}
}
example:
void mycallback (char* szName) {
printf ("%s\n", szName);
}
int main () {
EnumExportedFunctions ("C:\\Windows\\System32\\user32.dll", mycallback);
return 0;
}
output:
ActivateKeyboardLayout
AddClipboardFormatListener
AdjustWindowRect
AdjustWindowRectEx
AlignRects
AllowForegroundActivation
AllowSetForegroundWindow
AnimateWindow
AnyPopup
AppendMenuA
AppendMenuW
ArrangeIconicWindows
AttachThreadInput
BeginDeferWindowPos
BeginPaint
BlockInput
BringWindowToTop
BroadcastSystemMessage
BroadcastSystemMessageA
BroadcastSystemMessageExA
BroadcastSystemMessageExW
BroadcastSystemMessageW
BuildReasonArray
CalcMenuBar
.....etc
Go over to Microsoft research and grab the Detours Library. One of its examples does exactly what you are asking. The whole library basically makes detouring/rerouting win32 function calls extremely easy. Its pretty cool stuff.
Detours
Edit: Also note that if you just want to look at the export table, you can (at least in visual studios) set your project properties to print out the export/import tables. I can't remember the exact option but should be easy to google.
**Edit2:**The option is Project Properties->Linker->Debugging->Generate MapFile ->Yes(/MAP)
While ephemient is correct that LoadLibraryEx with DONT_RESOLVE_DLL_REFERENCES can simplify this task a great deal, you can make it even simpler than he shows. Instead of finding and enumerating the DLL's export directory yourself, you can use SymEnumerateSymbols to list the symbols for you.
Although only marginally simpler than his code (without the asserts, his is only half a dozen lines of code) this at least theoretically gives a little extra flexibility in case Microsoft should someday decide to change the executable format a bit, and/or change exactly what the HMODULE points at, so his no longer works (since most of these details aren't officially documented anyway).
If you don't want to go to the trouble of writing your own code and would rather use a DLL that already exists for this purpose, I recommend PE File Format DLL. Comes with source code so that you can modify if you wish. No GPL to worry about.
Also available is a GUI application that shows how to use the DLL.
If you're just looking for a way to find out what functions are exported in a DLL, you can use Microsoft's dependency walker (depends.exe). This wont help you if you actually need to discover the exports programmatically, though.
I may be wrong, and I haven't double checked to be honest, but I believe there may be some compatibility issues with using ephemient's code on a module that is built under a different architecture than that of your process. (Again, I may be speaking completely out of my ass right now)
There's a project on github, called dll2def that uses the same technique (though it loads the file into memory on its own), but seems to have some checks in place to find the exports depending on the architecture of the binary. The code you'd most likely be interested in is in this file.
It's been 12 years since this was asked, but wanted to point out some problems with the solutions presented.
None of them account for ordinals (exports without a name string).
Complicated by potential gaps between ordinal indexes. Ordinals have a starting base (the "Base" field of IMAGE_EXPORT_DIRECTORY) but there is no guarantee that ordinal numbers are sequential.
Don't want to spend the time to code it up, but one way to do it is iterate by index 0 to NumberOfFunctions.
Then in a 2nd (inner) loop match that index up in 0 to NumberOfNames into the AddressOfNameOrdinals array.
If you match the function index to the AddressOfNameOrdinals array index, that is your index into AddressOfNames array (an offset you have to resolve). If you don't get a match (over NumberOfNames indexes) then it's an ordinal export.
If the function index in the AddressOfFunctions entry is 0 then it's just an ordinal gap and you skip to the next index.
To get the actual ordinal number (for printing out as a string) you add "Base" to the NumberOfFunctions loop index.

C++ Optimal Block Size For Reading From A File

I have a program that generates files containing random distributions of the character A - Z. I have written a method that reads these files (and counts each character) using fread with different buffer sizes in an attempt to determine the optimal block size for reads. Here is the method:
int get_histogram(FILE * fp, long *hist, int block_size, long *milliseconds, long *filelen)
{
char *buffer = new char[block_size];
bzero(buffer, block_size);
struct timeb t;
ftime(&t);
long start_in_ms = t.time * 1000 + t.millitm;
size_t bytes_read = 0;
while (!feof(fp))
{
bytes_read += fread(buffer, 1, block_size, fp);
if (ferror (fp))
{
return -1;
}
int i;
for (i = 0; i < block_size; i++)
{
int j;
for (j = 0; j < 26; j++)
{
if (buffer[i] == 'A' + j)
{
hist[j]++;
}
}
}
}
ftime(&t);
long end_in_ms = t.time * 1000 + t.millitm;
*milliseconds = end_in_ms - start_in_ms;
*filelen = bytes_read;
return 0;
}
However, when I plot bytes/second vs. block size (buffer size) using block sizes of 2 - 2^20, I get an optimal block size of 4 bytes -- which just can't be correct. Something must be wrong with my code but I can't find it.
Any advice is appreciated.
Regards.
EDIT:
The point of this exercise is to demonstrate the optimal buffer size by recording the read times (plus computation time) for different buffer sizes. The file pointer is opened and closed by the calling code.
There are many bugs in this code:
It uses new[], which is C++.
It doesn't free the allocated memory.
It always loops over block_size bytes of input, not bytes_read as returned by fread().
Also, the actual histogram code is rather inefficient, since it seems to loop over each character to determine which character it is.
UPDATE: Removed claim that using feof() before I/O is wrong, since that wasn't true. Thanks to Eric for pointing this out in a comment.
You're not stating what platform you're running this on, and what compile time parameters you use.
Of course, the fread() involves some overhead, leaving user mode and returning. On the other hand, instead of setting the hist[] information directly, you're looping through the alphabet. This is unnecessary and, without optimization, causes some overhead per byte.
I'd re-test this with hist[j-26]++ or something similar.
Typically, the best timing would be achieved if your buffer size equals the system's buffer size for the given media.

writing Bytes into a .Bin file

I have a vector in C++ that I want to write it to a .bin file.
this vector's type is byte, and the number of bytes could be huge, maybe millions.
I am doing it like this:
if (depthQueue.empty())
return;
FILE* pFiledep;
pFiledep = fopen("depth.bin", "wb");
if (pFiledep == NULL)
return;
byte* depthbuff = (byte*) malloc(depthQueue.size() * 320 * 240 * sizeof(byte));
if(depthbuff)
{
for(int m = 0; m < depthQueue.size(); m++)
{
byte b = depthQueue[m];
depthbuff[m] = b;
}
fwrite(depthbuff, sizeof(byte),
depthQueue.size() * 320 * 240 * sizeof(byte), pFiledep);
fclose(pFiledep);
free(depthbuff);
}
depthQueue is my vector which contains bytes and lets say its size is 100,000.
Sometimes I don't get this error, but the bin file is empty.
Sometime I get heap error.
Somtimes when I debug this, it seems that malloc doesn't allocate the space.
Is the problem is with space?
Or is chunk of sequential memory is so long and it can't write in bin?
You don't need hardly any of that. vector contents are guaranteed to be contiguous in memory, so you can just write from it directly:
fwrite(&depthQueue[0], sizeof (Byte), depthQueue.size(), pFiledep);
Note a possible bug in your code: if the vector is indeed vector<Byte>, then you should not be multiplying its size by 320*240.
EDIT: More fixes to the fwrite() call: The 2nd parameter already contains the sizeof (Byte) factor, so don't do that multiplication again in the 3rd parameter either (even though sizeof (Byte) is probably 1 so it doesn't matter).

Mapping large files using MapViewOfFile

I have a very large file and I need to read it in small pieces and then process each piece. I'm using MapViewOfFile function to map a piece in memory, but after reading first part I can't read the second. It throws when I'm trying to map it.
char *tmp_buffer = new char[bufferSize];
LPCWSTR input = L"input";
OFSTRUCT tOfStr;
tOfStr.cBytes = sizeof tOfStr;
HANDLE inputFile = (HANDLE)OpenFile(inputFileName, &tOfStr, OF_READ);
HANDLE fileMap = CreateFileMapping(inputFile, NULL, PAGE_READONLY, 0, 0, input);
while (offset < fileSize)
{
long k = 0;
bool cutted = false;
offset -= tempBufferSize;
if (fileSize - offset <= bufferSize)
{
bufferSize = fileSize - offset;
}
char *buffer = new char[bufferSize + tempBufferSize];
for(int i = 0; i < tempBufferSize; i++)
{
buffer[i] = tempBuffer[i];
}
char *tmp_buffer = new char[bufferSize];
LPCWSTR input = L"input";
HANDLE inputFile;
OFSTRUCT tOfStr;
tOfStr.cBytes = sizeof tOfStr;
long long offsetHigh = ((offset >> 32) & 0xFFFFFFFF);
long long offsetLow = (offset & 0xFFFFFFFF);
tmp_buffer = (char *)MapViewOfFile(fileMap, FILE_MAP_READ, (int)offsetHigh, (int)offsetLow, bufferSize);
memcpy(&buffer[tempBufferSize], &tmp_buffer[0], bufferSize);
UnmapViewOfFile(tmp_buffer);
offset += bufferSize;
offsetHigh = ((offset >> 32) & 0xFFFFFFFF);
offsetLow = (offset & 0xFFFFFFFF);
if (offset < fileSize)
{
char *next;
next = (char *)MapViewOfFile(fileMap, FILE_MAP_READ, (int)offsetHigh, (int)offsetLow, 1);
if (next[0] >= '0' && next[0] <= '9')
{
cutted = true;
}
UnmapViewOfFile(next);
}
ostringstream path_stream;
path_stream << tempPath << splitNum;
ProcessChunk(buffer, path_stream.str(), cutted, bufferSize);
delete buffer;
cout << (splitNum + 1) << " file(s) sorted" << endl;
splitNum++;
}
One possibility is that you're not using an offset that's a multiple of the allocation granularity. From MSDN:
The combination of the high and low offsets must specify an offset within the file mapping. They must also match the memory allocation granularity of the system. That is, the offset must be a multiple of the allocation granularity. To obtain the memory allocation granularity of the system, use the GetSystemInfo function, which fills in the members of a SYSTEM_INFO structure.
If you try to map at something other than a multiple of the allocation granularity, the mapping will fail and GetLastError will return ERROR_MAPPED_ALIGNMENT.
Other than that, there are many problems in the code sample that make it very difficult to see what you're trying to do and where it's going wrong. At a minimum, you need to solve the memory leaks. You seem to be allocating and then leaking completely unnecessary buffers. Giving them better names can make it clear what they are actually used for.
Then I suggest putting a breakpoint on the calls to MapViewOfFile, and then checking all of the parameter values you're passing in to make sure they look right. As a start, on the second call, you'd expect offsetHigh to be 0 and offsetLow to be bufferSize.
A few suspicious things off the bat:
HANDLE inputFile = (HANDLE)OpenFile(inputFileName, &tOfStr, OF_READ);
Every cast should make you suspicious. Sometimes they are necessary, but make sure you understand why. At this point you should ask yourself why every other file API you're using requires a HANDLE and this function returns an HFILE. If you check OpenFile documentation, you'll see, "This function has limited capabilities and is not recommended. For new application development, use the CreateFile function." I know that sounds confusing because you want to open an existing file, but CreateFile can do exactly that, and it returns the right type.
long long offsetHigh = ((offset >> 32) & 0xFFFFFFFF);
What type is offset? You probably want to make sure it's an unsigned long long or equivalent. When bitshifting, especially to the right, you almost always want an unsigned type to avoid sign-extension. You also have to make sure that it's a type that has more bits than the amount you're shifting by--shifting a 32-bit value by 32 (or more) bits is actually undefined in C and C++, which allows the compilers to do certain types of optimizations.
long long offsetLow = (offset & 0xFFFFFFFF);
In both of these statements, you have to be careful about the 0xFFFFFFFF value. Since you didn't cast it or give it a suffix, it can be hard to predict whether the compiler will treat it as an int or unsigned int. In this case,
it'll be an unsigned int, but that won't be obvious to many people. In fact,
I got this wrong when I first wrote this answer. [This paragraph corrected 16-MAY-2017] With bitwise operations, you almost always want to make sure you're using unsigned values.
tmp_buffer = (char *)MapViewOfFile(fileMap, FILE_MAP_READ, (int)offsetHigh, (int)offsetLow, bufferSize);
You're casting offsetHigh and offsetLow to ints, which are signed values. The API actually wants DWORDs, which are unsigned values. Rather than casting in the call, I would declare offsetHigh and offsetLow as DWORDs and do the casting in the initialization, like this:
DWORD offsetHigh = static_cast<DWORD>((offset >> 32) & 0xFFFFFFFFul);
DWORD offsetLow = static_cast<DWORD>( offset & 0xFFFFFFFFul);
tmp_buffer = reinterpret_cast<const char *>(MapViewOfFile(fileMap, FILE_MAP_READ, offsetHigh, offsetLow, bufferSize));
Those fixes may or may not resolve your problem. It's hard to tell what's going on from the incomplete code sample.
Here's a working sample you can compare to:
// Calls ProcessChunk with each chunk of the file.
void ReadInChunks(const WCHAR *pszFileName) {
// Offsets must be a multiple of the system's allocation granularity. We
// guarantee this by making our view size equal to the allocation granularity.
SYSTEM_INFO sysinfo = {0};
::GetSystemInfo(&sysinfo);
DWORD cbView = sysinfo.dwAllocationGranularity;
HANDLE hfile = ::CreateFileW(pszFileName, GENERIC_READ, FILE_SHARE_READ,
NULL, OPEN_EXISTING, 0, NULL);
if (hfile != INVALID_HANDLE_VALUE) {
LARGE_INTEGER file_size = {0};
::GetFileSizeEx(hfile, &file_size);
const unsigned long long cbFile =
static_cast<unsigned long long>(file_size.QuadPart);
HANDLE hmap = ::CreateFileMappingW(hfile, NULL, PAGE_READONLY, 0, 0, NULL);
if (hmap != NULL) {
for (unsigned long long offset = 0; offset < cbFile; offset += cbView) {
DWORD high = static_cast<DWORD>((offset >> 32) & 0xFFFFFFFFul);
DWORD low = static_cast<DWORD>( offset & 0xFFFFFFFFul);
// The last view may be shorter.
if (offset + cbView > cbFile) {
cbView = static_cast<int>(cbFile - offset);
}
const char *pView = static_cast<const char *>(
::MapViewOfFile(hmap, FILE_MAP_READ, high, low, cbView));
if (pView != NULL) {
ProcessChunk(pView, cbView);
}
}
::CloseHandle(hmap);
}
::CloseHandle(hfile);
}
}
You have a memory leak in your code:
char *tmp_buffer = new char[bufferSize];
[ ... ]
while (offset < fileSize)
{
[ ... ]
char *tmp_buffer = new char[bufferSize];
[ ... ]
tmp_buffer = (char *)MapViewOfFile(fileMap, FILE_MAP_READ, (int)offsetHigh, (int)offsetLow, bufferSize);
[ ... ]
}
You're never delete what you allocate via new char[] during every iteration there. If your file is large enough / you do enough iterations of this loop, the memory allocation will eventually fail - that's then you'll see a throw() done by the allocator.
Win32 API calls like MapViewOfFile() are not C++ and never throw, they return error codes (the latter NULL on failure). Therefore, if you see exceptions, something's wrong in you C++ code. Likely the above.
I also had some troubles with memory mapped files.
Basically I just wanted to share memory (1Mo) between 2 apps on the same Pc.
- Both apps where written in Delphi
- Using Windows8 Pro
At first one application (the first one launched) could read and write the memoryMappedFile, but the second one could only read it (error 5 : AccessDenied)
Finally after a lot of testing It suddenly worked when both application where using CreateFileMapping. I even tried to create my on security descriptor, nothing helped.
Just before my applications where first calling OpenFileMapping and then CreateFileMapping if the first one failed
Another thing that misleaded me is that the handles , although visibly referencing the same MemoryMappedFile where different in both applications.
One last thing, after this correction my application seemed to work all right, but after a while I had error_NotEnough_Memory. when calling MapViewOfFile.
It was just a beginner's mistake of my part, I was not always calling UnmapViewOfFile.