I want to read status information that an application provides via shared memory. I want to use C++ in order to read the content of that named shared memory and then call it with pinvoke from a C#-class.
From the software I know that it has a certain file structure: A struct STATUS_DATA with an array of four structs of SYSTEM_CHARACTERISTICS.
I'm not (yet) familiar with C++, so I tried to follow msdn basically. To find the size of the file to be mapped, I added the sizes of the struct members as to be seen in the code below. This results in a ACCESS DENIED, so I figured, that the result based on the structs is too high. When I use sizeof(STATUS_DATA) (I added the struct to my source), it still ends up in an ACCESS DENIED. If I try something lower, like 1024 Bytes, only thing I can see in pbuf is a <, while debugging.
This is what I got so far:
#include <windows.h>
#include <stdio.h>
#include <conio.h>
#include <tchar.h>
#include <iostream>
#pragma comment(lib, "user32.lib")
using namespace std;
signed int BUF_SIZE = 4 * (10368 + 16 + 4 + 16 + 4 + 16 + 4 + 1 + 4); // sizeof(STATUS_DATA);
TCHAR szName[]=TEXT("ENGINE_STATUS");
int main()
{
HANDLE hMapFile;
unsigned char* pBuf;
hMapFile = OpenFileMapping(
FILE_MAP_READ, // read access
FALSE, // do not inherit the name
szName); // name of mapping object
if (hMapFile == NULL)
{
_tprintf(TEXT("Could not open file mapping object (%d).\n"),
GetLastError());
return 1;
}
pBuf = (unsigned char*) MapViewOfFile(hMapFile, // handle to map object
FILE_MAP_READ, // read/write permission
0,
0,
BUF_SIZE); // 1024);
if (pBuf == NULL)
{
_tprintf(TEXT("Could not map view of file (%d).\n"),
GetLastError());
CloseHandle(hMapFile);
return 1;
}
UnmapViewOfFile(pBuf);
CloseHandle(hMapFile);
return 0;
}
I also made sure that this Shared Mem "is there" by following that hint. Can somebody give me a hint, what I'm missing? Thanks!
The last parameter to MapViewOfFile (dwNumberOfBytesToMap) must be less than the maximum size specified when the mapping was created. Since we don't know what that size is, it seems fair to assume that BUF_SIZE is exceeding it and 1024 isn't. Specifying 0 for this parameter is an easy way to map the entire file into a single view.
Most (all?) C++ debuggers will assume that a pointer to char is a null-terminated string, so when you try and view the mapped data it will only display up until the first byte that is zero. Depending on what data is in the file mapping, this could well be the second byte, which explains why you aren't seeing much information. You would be better to cast the returned pointer to STATUS_DATA* and viewing the individual members.
In short:
Specify zero (0) for dwNumberOfBytesToMap
Cast the returned pointer to STATUS_DATA* instead of unsigned char*
Related
In the following simple program:
# include <sys/mman.h>
# include <fcntl.h>
# include <cstdlib>
# include <cassert>
struct rgn_desc
{
size_t end_;
char data[];
};
int main(int argc, const char *argv[])
{
int fd = open("foo.mm", O_RDWR|O_CREAT|O_TRUNC, (mode_t)0700);
assert(fd != -1);
void * ptr = mmap(NULL, 4096, PROT_READ | PROT_WRITE,
MAP_SHARED | MAP_POPULATE, fd, 0);
assert(ptr != (void*) -1);
rgn_desc * rgn_ptr = (rgn_desc*) ptr;
rgn_ptr->end_ = 0; // <-- bus error
}
Basically, I want to manage a simple mmaped arena allocator and store as first part of the mapping the bytes that I have allocated. So, when I recover from a file, I get how many bytes were allocated.
However, the last line is giving me a bus error. Could someone explain why, and if possible, to suggest to me a way for avoiding it. I am running Linux on a 32 bits pentium and using clang++ compiler
According to the doc, a sig bus can trigger if:
SIGBUS
Attempted access to a portion of the buffer that does not
correspond to the file (for example, beyond the end of the
file, including the case where another process has truncated
the file).
In your snipped your file size don't match with your mmap() size (0, 4096), so you could use ftruncate() to increase the size of your file.
ftruncate(fd, 4096);
I am working on some software making use of a encryption library, the underlying mechanics of which I can't change. When the program runs it takes about 10 minuets to generate a public and private key, which is extremely frustrating when trying to debug other parts of the software. I would like to write the keys to a file and read them back to save time.
They keys are void pointers:
Enc_Key_T secKey = nullptr;
Where Enc_Key_T is defined as typedef void* Enc_Key_T
The code I have used to attempt to read and write keys is as follows (only attempted to write the secret key so far):
#ifdef WriteKey
generate_Keys(parameters, &sec_key, &prv_key);
FILE * pFile;
pFile = fopen("sk.bin", "wb");
fwrite(&sec_key, sizeof(Enc_Key_T), sizeof(&sec_key), pFile);
fclose(pFile);
#else
FILE * pFile;
long lSize;
char * buffer;
pFile = fopen("sk.bin", "rb");
if(pFile == NULL)
fileError();
fseek (pFile, 0 , SEEK_END);
lsize = ftell (pFile);
rewind (pFile);
buffer = (char *) malloc (sizeof(char)*lSize);
if(buffer == NULL)
memError();
sec_key = (void *) fread(buffer, 1, lSize, pFile);
fclose(pFile);
free(buffer);
#endif
When I write the file it comes out as 64 byte file, but reading it back in the secret key pointer gets set to a low value memory address, which makes me think I am doing something wrong. I can't find any examples of how this can be done. I'm not even sure it can, as I won't be creating any of the underling structures, so I am just trying to allocate some memory at a location provided by a pointer.
Is there anyway this can be done without having to touch the underlying library that generates the keys?
Short answer: in general, you cannot do that correctly.
Long answer:
The thing that you're missing is the structure, you have no guarantee that Enc_Key_T can be serialized by simply writing memory contents. In addition, even if it just raw random data, there is no known length of it.
In addition, there is no guarantee that the library does not have its own state, bound to the generated keys.
Code issues:
When writing, you have no known length of data. Data that is written is a pointer and then something bogus.
When reading, you don't need a secondary buffer; besides, fread returns number of bytes read, not a pointer to the data. So instead of:
buffer = (char *) malloc (sizeof(char)*lSize);
if(buffer == NULL)
memError();
sec_key = (void *) fread(buffer, 1, lSize, pFile);
you can write:
sec_key = (void *) malloc (lSize);
if(sec_key == NULL)
memError();
if ( 0 == fread(sec_key, 1, lSize, pFile) ) {
// error
}
I will assume you are going to debug intensively this application such that you may invest 1 hour building those wrappers I am suggesting.
As you long as you do not know the underlying structure and this library is a black box you have to do the following:
void generate_KeysEx(...){
#if DEBUG
// return dummy keys
#else
// call this API
#endif
}
void EncryptEx(...){
#if DEBUG
// return cipher same as plain text
#else
// call this library API
#endif
}
From discussions in the comments it doesn't sound like this is possible without capturing the structure in the library, which would require significant code changes.
A very hacky solution I found was to change a few key generation parameters which drastically reduces run time.
I am trying to figure out a (hopefully easy) way to read a large, unstructured file without bumping into the edge of a buffer. An example is helpful here.
Imagine you are trying to do some data-recovery of a 16GB flash-drive and have saved a dump of the drive to a 16GB file. You want to scan through the image, looking for certain items of interest. If the file were smaller, you could read the entire thing into a memory buffer (let’s say 1MB) and do a simple scan through the buffer. However, because it is too big to read in all at once, you need to read it in chunks. The problem is that an item of interest may not be perfectly aligned so as to fall within a single 1MB buffer. In other words, it may end up straddling the edge of the buffer so that it starts at the end of the buffer during one read, and ends in the next one (or even further).
At one time in the past, I dealt with this by using two buffers and copying the second one to the first one to create a sort of sliding window, however I imagine that this should be a common enough scenario that there are better, existing solutions. I looked into memory-mapped files, thinking that they let you read the file by simply increasing the array index/pointer, but I ended up in the exact same situation as before due to the limit of the map view size. I tried looking for some practical examples of using MapViewOfFile with offsets, but all I could find were contrived examples that skipped that.
How is this situation normally handled?
If you are running in a 64 bit environment, I would just use memory mapped files. There is no (reasonable) memory limit for a process. You can read the file in, even jump around, and the OS will swap memory to and from disk.
Here's some basic information:
http://msdn.microsoft.com/en-us/library/ms810613.aspx
And an example of a file viewer here:
http://www.catch22.net/tuts/memory-techniques-part-1
This case works on a 2.8GB file in x64, but fails in win32 because it cannot allocate more than 2GB per process. It is very fast since it touches only the first and last byte in the pBuf array. Modifying the method to traverse the buffer and count the number of 'zero' bytes works as expected. You can watch the memory footprint go up as it does it but that memory is only virtually allocated.
#include "stdafx.h"
#include <string>
#include <Windows.h>
TCHAR szName[] = TEXT( pathToFile );
int _tmain(int argc, _TCHAR* argv[])
{
HANDLE hMapFile;
char* pBuf;
HANDLE file = CreateFile( szName, GENERIC_READ, FILE_SHARE_READ, 0, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);
if ( file == NULL )
{
_tprintf(TEXT("Could not open file object (%d).\n"),
GetLastError());
return 1;
}
unsigned int length = GetFileSize(file, 0);
printf( "Length = %u\n", length );
hMapFile = CreateFileMapping( file, 0, PAGE_READONLY, 0, 0, 0 );
if (hMapFile == NULL)
{
_tprintf(TEXT("Could not create file mapping object (%d).\n"), GetLastError());
return 1;
}
pBuf = (char*) MapViewOfFile(hMapFile, FILE_MAP_READ, 0,0, length);
if (pBuf == NULL)
{
_tprintf(TEXT("Could not map view of file (%d).\n"), GetLastError());
CloseHandle(hMapFile);
return 1;
}
printf("First Byte: 0x%02x\n", pBuf[0] );
printf("Last Byte: 0x%02x\n", pBuf[length-1] );
UnmapViewOfFile(pBuf);
CloseHandle(hMapFile);
return 0;
}
the code:
#include <windows.h>
#include <stdio.h>
int main() {
system("mode 128");
int range = 0xCFE2;
char* buf = new char[range+1];
DWORD dwChars;
if (!ReadConsoleOutputCharacter(
GetStdHandle(STD_OUTPUT_HANDLE),
buf, // Buffer where store symbols
range, // Read len chars
{0,0}, // Read from row=8, column=6
&dwChars // How many symbols stored
)) {
printf("GetLastError: %lu\n", GetLastError());
}
system("pause");
return 0;
}
Console screen buffers cannot be larger than 64K. Each character in the buffer requires 2 bytes, one for the character code and another for the color attributes. It therefore never makes any sense to try to read more than 32K chars with ReadConsoleOutputCharacter().
You don't have a real problem.
The documentation for WriteConsole() says:
If the total size of the specified number of characters exceeds the available heap, the function fails with ERROR_NOT_ENOUGH_MEMORY.
ReadConsoleOutputCharacter() probably has a similar restriction if you try to read too much, even though it is not documented. Try using GetConsoleScreenBufferInfo() or similar function to determine how many rows and columns there are, and then don't read more than that.
I am trying to constantly read data into a buffer of type unsigned char* from different files. However, I can't seem to set the buffer to NULL prior to reading in the next file.
Here is only the relevant code:
#include <stdio.h>
#include <fstream>
int
main (int argc, char** argv) {
FILE* dataFile = fopen("C:\\File1.txt", "rb");
unsigned char *buffer = NULL;
buffer = (unsigned char*)malloc(1000);
fread(buffer,1,1000,dataFile);
fclose(dataFile);
dataFile = fopen("C:\\File2.txt", "rb");
buffer = NULL;
fread(buffer,1,1000,dataFile);
fclose(dataFile);
system("pause");
return 0;
}
The error I run into is at the second occurrence of this line: fread(buffer,1,1000,dataFile);
The error I get is:
Debug Assertion Failed!
Expression: (buffer != NULL)
It points me to Line 147 of fread.c which is basically:
/* validation */
_VALIDATE_RETURN((buffer != NULL), EINVAL, 0);
if (stream == NULL || num > (SIZE_MAX / elementSize))
{
if (bufferSize != SIZE_MAX)
{
memset(buffer, _BUFFER_FILL_PATTERN, bufferSize);
}
_VALIDATE_RETURN((stream != NULL), EINVAL, 0);
_VALIDATE_RETURN(num <= (SIZE_MAX / elementSize), EINVAL, 0);
}
I did Google for ways to get the buffer pointer to NULL and tried the various suggestions, but none seem to work. Anyone can clarify what is the right way to set it to NULL?
Your buffer is a pointer.
When you do this:
buffer = (unsigned char*)malloc(1000);
you allocate some space in memory, and assign its starting position to buffer. Remember, buffer holds the address of the beginning of the space, that's all. When you do this:
buffer = NULL;
you have thrown away that address.
EDIT:
C++ style, without dynamic memory:
#include <fstream>
using std:: string;
using std:: ifstream;
void readFromFile(string fname)
{
char buffer[1000];
ifstream fin(fname.c_str());
fin.read(buffer, sizeof(buffer));
// maybe do things with the data
}
int main ()
{
readFromFile("File1.txt");
readFromFile("File2.txt");
return 0;
}
There's no need to erase the contents of the buffer. If the cost of allocating and deallocating the buffer with each call is too much, just add static:
static char buffer[1000];
It will be overwritten each time.
You can't say buffer = NULL because fread wil try to dereference it. Dereferencing NULL is one of the things that are certainly and completely illegal in C++. In effect you're losing what you got from malloc. Perhaps you're looking for memset and trying to zero the buffer:
memset(buffer, 0, 1000);
However, you don't need to do this before calling fread. There's simply no reason since fread will write the buffer anyway: it doesn't care if it's zeroed or not.
As a side note: you're writing very C-ish code in what I suspect is C++ (given your fstream header). There are better-suited I/O options for C++.