The program uses ReadProcessMemory to scan through chunks of memory for a certain value. Unfortunately when I call ReadProcessMemory I get error 299.
void update_memblock(MEMBLOCK *mb)
{
//variables
static unsigned char tempbuf[128*1024];
size_t bytes_left;
size_t total_read;
size_t bytes_to_read;
size_t bytes_read;
size_t sizeMem;
size_t MemoryBase;
bytes_left = mb->size;
total_read = 0;
while (bytes_left)
{
bytes_to_read = (bytes_left > sizeof(tempbuf)) ? sizeof(tempbuf) : bytes_left;
ReadProcessMemory(mb->hProc ,mb->addr + total_read,mb->buffer, bytes_to_read, (SIZE_T*)&bytes_read);
if (bytes_read != bytes_to_read)break;
memcpy(mb->buffer + total_read, tempbuf,bytes_read);
bytes_left -= bytes_read;
total_read += bytes_read;
}
mb->size = total_read;
}
Erorr Code 299 (0x12B) ERROR_PARTIAL_COPY
"Only part of a ReadProcessMemory or WriteProcessMemory request was completed"
You're receiving this error because you're trying to read memory from a page that isn't "allocated".
You want to use VirtualQueryEx() on each page of memory which yields a MEMORY_BASIC_INFORMATION structure, which contains 2 variables of note:
State: can be MEM_COMMIT, MEM_FREE or MEM_RESERVE
Protect: can be any of the Memory Protection Constants
You want to loop through all the pages of memory, call VirtualQueryEx() on them and skip any pages that are bad. I like to skip all pages/regions in which state != MEM_COMMIT and Protect == PAGE_NOACCESS
Here is a psuedo code example:
MEMORY_BASIC_INFORMATION mbi = { 0 };
while (LoopingThroughTheMemories.bat)
{
if (!VirtualQueryEx(hProc, currentMemoryAddress, &mbi, sizeof(mbi))) continue
if (mbi.State != MEM_COMMIT || mbi.Protect == PAGE_NOACCESS) continue;
//good mem region, do ReadProcessMemory() stuffs
}
Related
I have the following function, written in C++ using WinAPI:
The function receives a handle to a process, and should return the size of the memory used by the process.
DWORD GetProcessCommitedMemorySize(const HANDLE hProcess)
{
MEMORY_BASIC_INFORMATION mbi;
DWORD totalMemory = 0;
DWORD currAddr = 0;
//for checking repetion of pages base addresses
std::unordered_set<DWORD> tempAddresses;
while (VirtualQueryEx(hProcess, (LPVOID)currAddr, &mbi, (SIZE_T)sizeof(mbi)) == (SIZE_T)sizeof(mbi))
{
if (tempAddresses.find((DWORD)mbi.BaseAddress) == tempAddresses.end()) // not found
{
currAddr = (DWORD)mbi.BaseAddress + mbi.RegionSize;
tempAddresses.insert((DWORD)mbi.BaseAddress);
if (mbi.State == MEM_COMMIT && mbi.Type == MEM_PRIVATE)
{
totalMemory += mbi.RegionSize;
}
}
else
{
break;
}
}
return totalMemory;
}
The function always returns a number which is bigger than the amount of memory which is shown to be used on the 'details' page of the Task Manager.
for example, 86155 KB as the function's output, and 56924 KB on the Task Manager.
The program still isn't complete so I need to hard-code the PID, so I know I'm looking at the right one.
Does this issue occur because the number I'm seeking with this function and the number on the Task Manager have different meanings, or is my code just not doing what's it supposed to do?
Thanks.
I've been attempting to use ReadFileScatter today in my code (which sounds like exactly what I need), so far without a lot of luck. Google'ing the internet for what goes wrong doesn't give me much insight.
The documentation states:
The array must contain enough elements to store nNumberOfBytesToRead bytes of data, plus one element for the terminating NULL. For example, if there are 40 KB to be read and the page size is 4 KB, the array must have 11 elements that includes 10 for the data and one for the NULL.
Each buffer must be at least the size of a system memory page and must be aligned on a system memory page size boundary. The system reads one system memory page of data into each buffer.
The function stores the data in the buffers in sequential order. For example, it stores data into the first buffer, then into the second buffer, and so on until each buffer is filled and all the data is stored, or there are no more buffers.
So far I've been attempting to do just that. I allocated a bunch of bytes using VirtualAlloc (which ensures the page boundary constraint), add a terminator NULL to the list, ensure the data on disk is on the system boundary (and implicitly a disk sector size boundary) as well and issue the call.
Without further due, here's the minimum test case in C++:
// Setup: c:\tmp\test.dat is a file with at least 12K of stuff.
// I attempt to read page 2/3, e.g. offset [4096-4096+8192>
// TEST:
SYSTEM_INFO systemInfo;
GetSystemInfo(&systemInfo);
auto pageSize = systemInfo.dwPageSize;
std::cout << "Page size: "<< pageSize << std::endl;
// Allocate 2 pages that are aligned with one in the middle:
auto buffer = reinterpret_cast<char*>(VirtualAlloc(NULL, pageSize * 3, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE));
// Create read buffer:
std::vector<FILE_SEGMENT_ELEMENT> elements;
{
FILE_SEGMENT_ELEMENT element1;
element1.Buffer = buffer;
elements.push_back(element1);
}
{
FILE_SEGMENT_ELEMENT element2;
element2.Buffer = buffer + pageSize * 2;
elements.push_back(element2);
}
{
FILE_SEGMENT_ELEMENT terminator;
terminator.Buffer = NULL;
elements.push_back(terminator);
}
// [..] Physical sector size is normally checked as well. In my case it's 512 bytes,
// so I guess that's irrelevant here.
//
// Open file:
auto fileHandle = CreateFile(
"c:\\tmp\\test.dat",
GENERIC_READ | GENERIC_WRITE,
FILE_SHARE_READ,
NULL,
OPEN_ALWAYS,
FILE_FLAG_NO_BUFFERING | FILE_FLAG_WRITE_THROUGH,
NULL);
auto err = GetLastError();
if (err != ERROR_ALREADY_EXISTS && err != ERROR_SUCCESS)
{
throw std::exception(); // FIXME.
}
OVERLAPPED overlapped;
memset(&overlapped, 0, sizeof(OVERLAPPED));
LARGE_INTEGER tmp;
tmp.QuadPart = 4096; // Read from disk page 1
overlapped.Offset = tmp.LowPart;
overlapped.OffsetHigh = tmp.HighPart;
overlapped.hEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
auto succes = ReadFileScatter(fileHandle, elements.data(), pageSize * 2, NULL, &overlapped);
err = GetLastError();
if (!succes && err != ERROR_IO_PENDING && err != ERROR_SUCCESS)
{
throw std::exception(); // The call always ends up here with error 87: Invalid parameter
}
WaitForSingleObject(overlapped.hEvent, INFINITE);
std::cout << "Call succeeded!" << std::endl;
// FIXME: Proper exception handling.
// Clean up:
VirtualFree(buffer, pageSize * 3, MEM_DECOMMIT | MEM_RELEASE);
CloseHandle(overlapped.hEvent);
CloseHandle(fileHandle);
In the code, the error is noted with the comment // The call always ends up here with error 87: Invalid parameter. However, as far as I can see, I check all the boxes that they describe on MSDN... so...
What am I doing wrong here?
So I have this litte code, It loops through memory regions, saves them to a byte array, then uses it and finally deletes it (deallocate it). This all happens in a non-main thread, therefore the use of CriticalSections.
Code looks like this:
SIZE_T addr_min = (SIZE_T)sysInfo.lpMinimumApplicationAddress;
SIZE_T addr_max = (SIZE_T)sysInfo.lpMaximumApplicationAddress;
while (addr_min < addr_max)
{
MEMORY_BASIC_INFORMATION mbi = { 0 };
if (!::VirtualQueryEx(hndl, (LPCVOID)addr_min, &mbi, sizeof(mbi)))
{
continue;
}
if (mbi.State == MEM_COMMIT && ((mbi.Protect & PAGE_GUARD) == 0) && ((mbi.Protect & PAGE_NOACCESS) == 0))
{
SIZE_T region_size = mbi.RegionSize;
PVOID Base_Address = mbi.BaseAddress;
BYTE * dump = new BYTE[region_size + 1];
EnterCriticalSection(...);
memset(dump, 0x00, region_size + 1);
//this is where it crashes, same thing with memcpy
//Access violation reading "dump"'s address:
//memmove(unsigned char * dst=0x42aff024, unsigned char *
//src=0x7a768000, unsigned long count=1409024)
std::memmove(dump, Base_Address, region_size);
LeaveCriticalSection(...);
//Do Stuff with dump, that only involves reading from it
if (dump){
delete[] dump;
dump = NULL;
}
}
addr_min += mbi.RegionSize;
}
Code works fine most of the time. But sometimes it just crashes in memcpy/memmove. Under the Visual Studio Debugger it shows that the crash is because there is a error reading "dump", how is that possible if I just define and allocated memory for it. Thanks!
Also, could it be because memory can change in the middle of memcpy?
Need to write an application in C/C++ on Linux that receives a stream of bytes from a socket and process them. The total bytes could be close to 1TB. If I have unlimited amount memory, I will just put it all in the memory, so my application can easily process data. It's much easy to do many things on flat memory space, such as memmem(), memcmp() ... On a circular buffer, the application has to be extra smart to be aware of the circular buffer.
I have about 8G of memory, but luckily due to locality, my application never needs to go back by more than 1GB from the latest data it received. Is there a way to have a 1TB buffer, with only the latest 1GB data mapped to physical memory? If so, how to do it?
Any ideas? Thanks.
Here's an example. It sets up a full terabyte mapping, but initially inaccessible (PROT_NONE). You, the programmer, maintain a window that can only extend and move upwards in memory. The example program uses a one and a half gigabyte window, advancing it in steps of 1,023,739,137 bytes (the mapping_use() makes sure the available pages cover at least the desired region), and does actually modify every page in every window, just to be sure.
#define _GNU_SOURCE
#define _POSIX_C_SOURCE 200809L
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <errno.h>
#include <string.h>
#include <stdio.h>
typedef struct mapping mapping;
struct mapping {
unsigned char *head; /* Start of currently accessible region */
unsigned char *tail; /* End of currently accessible region */
unsigned char *ends; /* End of region */
size_t page; /* Page size of this mapping */
};
/* Discard mapping.
*/
void mapping_free(mapping *const m)
{
if (m && m->ends > m->head) {
munmap(m->head, (size_t)(m->ends - m->head));
m->head = NULL;
m->tail = NULL;
m->ends = NULL;
m->page = 0;
}
}
/* Move the accessible part up in memory, to [from..to).
*/
int mapping_use(mapping *const m, void *const from, void *const to)
{
if (m && m->ends > m->head) {
unsigned char *const head = ((unsigned char *)from <= m->head) ? m->head :
((unsigned char *)from >= m->ends) ? m->ends :
m->head + m->page * (size_t)(((size_t)((unsigned char *)from - m->head)) / m->page);
unsigned char *const tail = ((unsigned char *)to <= head) ? head :
((unsigned char *)to >= m->ends) ? m->ends :
m->head + m->page * (size_t)(((size_t)((unsigned char *)to - m->head) + m->page - 1) / m->page);
if (head > m->head) {
munmap(m->head, (size_t)(head - m->head));
m->head = head;
}
if (tail > m->tail) {
#ifdef USE_MPROTECT
mprotect(m->tail, (size_t)(tail - m->tail), PROT_READ | PROT_WRITE);
#else
void *result;
do {
result = mmap(m->tail, (size_t)(tail - m->tail), PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_FIXED | MAP_PRIVATE | MAP_NORESERVE, -1, (off_t)0);
} while (result == MAP_FAILED && errno == EINTR);
if (result == MAP_FAILED)
return errno = ENOMEM;
#endif
m->tail = tail;
}
return 0;
}
return errno = EINVAL;
}
/* Initialize a mapping.
*/
int mapping_create(mapping *const m, const size_t size)
{
void *base;
size_t page, truesize;
if (!m || size < (size_t)1)
return errno = EINVAL;
m->head = NULL;
m->tail = NULL;
m->ends = NULL;
m->page = 0;
/* Obtain default page size. */
{
long value = sysconf(_SC_PAGESIZE);
page = (size_t)value;
if (value < 1L || (long)page != value)
return errno = ENOTSUP;
}
/* Round size up to next multiple of page. */
if (size % page)
truesize = size + page - (size % page);
else
truesize = size;
/* Create mapping. */
do {
errno = ENOTSUP;
base = mmap(NULL, truesize, PROT_NONE, MAP_ANONYMOUS | MAP_PRIVATE | MAP_NORESERVE, -1, (off_t)0);
} while (base == MAP_FAILED && errno == EINTR);
if (base == MAP_FAILED)
return errno;
/* Success. */
m->head = base;
m->tail = base;
m->ends = (unsigned char *)base + truesize;
m->page = page;
errno = 0;
return 0;
}
static void memtouch(void *const ptr, const size_t size)
{
if (ptr && size > 0) {
unsigned char *mem = (unsigned char *)ptr;
const size_t step = 2048;
size_t n = size / (size_t)step - 1;
mem[0]++;
mem[size-1]++;
while (n-->0) {
mem += step;
mem[0]++;
}
}
}
int main(void)
{
const size_t size = (size_t)1024 * (size_t)1024 * (size_t)1024 * (size_t)1024;
const size_t need = (size_t)1500000000UL;
const size_t step = (size_t)1023739137UL;
unsigned char *base;
mapping map;
size_t i;
if (mapping_create(&map, size)) {
fprintf(stderr, "Cannot create a %zu-byte mapping: %m.\n", size);
return EXIT_FAILURE;
}
printf("Have a %zu-byte mapping at %p to %p.\n", size, (void *)map.head, (void *)map.ends);
fflush(stdout);
base = map.head;
for (i = 0; i <= size - need; i += step) {
printf("Requesting %p to %p .. ", (void *)(base + i), (void *)(base + i + need));
fflush(stdout);
if (mapping_use(&map, base + i, base + i + need)) {
printf("Failed (%m).\n");
fflush(stdout);
return EXIT_FAILURE;
}
printf("received %p to %p.\n", (void *)map.head, (void *)map.tail);
fflush(stdout);
memtouch(base + i, need);
}
mapping_free(&map);
return EXIT_SUCCESS;
}
The approach is twofold. First, an inaccessible (PROT_NONE) mapping is created to reserve the necessary virtual contiguous address space. If we omit this step, it would make it possible for a malloc() call or similar to acquire pages within this range, which would defeat the entire purpose; a single terabyte-long mapping.
Second, when the accessible window extends into the region, either mprotect() (if USE_MPROTECT is defined), or mmap() is used to make the required pages accessible. Pages no longer needed are completely unmapped.
Compile and run using
gcc -Wall -Wextra -std=c99 example.c -o example
time ./example
or, to use mmap() only once and mprotect() to move the window,
gcc -DUSE_MPROTECT=1 -Wall -Wextra -std=c99 example.c -o example
time ./example
Note that you probably don't want to run the test if you don't have at least 4GB of physical RAM.
On this particular machine (i5-4200U laptop with 4GB of RAM, 3.13.0-62-generic kernel on Ubuntu x86_64), quick testing didn't show any kind of performance difference between mprotect() and mmap(), in execution speed or resident set size.
If anyone bothers to compile and run the above, and finds that one of them has a repeatable benefit/drawback (resident set size or time used), I'd very much like to know about it. Please also define your kernel and CPU used.
I'm not sure which details I should expand on, since this is pretty straightforward, really, and the Linux man pages project man 2 mmap and man 2 mprotect pages are quite descriptive. If you have any questions on this approach or program, I'd be happy to try and elaborate.
The title explains my goal very well, it should be simple but my solution just doesn't seem to be functioning properly. Currently, the function always returns NULL. When I use another program to locate the string and manually point memcmp to it, it returns 0 as it should.
DWORD findString(const char *input)
{
DWORD address;
size_t length = strlen(input);
DWORD baseAddress = (DWORD)GetModuleHandle(NULL);
DWORD maxAddress = (baseAddress + 26480640)-length;//the large # is the approximate size of the base module of the executable
for (address = baseAddress; address < maxAddress; address++)
{
if (memcmp(input, (void *)address, length) == 0)
{
return address;
}
}
OutputDebugStringA("String not found!");
return NULL;
}