CreateFileMapping error code 8 - c++

CreateFileMapping error code 8.Not enough storage is available to process this command. Im trying to create file mapping with 4 Gb (0xFFFFFFFF) on 64bit Win10 visual c++.
#define UBS_MEM_SIZE 0xffffffff
HANDLE hMapObject = CreateFileMapping(INVALID_HANDLE_VALUE, nullptr,
PAGE_READWRITE, HIWORD(UBS_MEM_SIZE), LOWORD(UBS_MEM_SIZE),
TEXT("dllmemfilemap"));
How can i solve this "ERROR 8" problem?

CreateFileMapping(..., HIWORD(UBS_MEM_SIZE), LOWORD(UBS_MEM_SIZE), ...)
The LO/HIWORD macros generate a WORD, a 16-bit value. You are asking for a 0xffff0000ffff memory-mapped file. That's 282 terabytes. Current x64 processors are limited to a 48-bit VM-address, most top out at 8 terabytes. So yes, error 8 (ERROR_NOT_ENOUGH_MEMORY) is entirely expected.
Don't use those macros. You can use LARGE_INTEGER as an alternative:
LARGE_INTEGER size;
size.QuadPart = UBS_MEM_SIZE;
HANDLE hMapObject = CreateFileMapping(..., size.HighPart, size.LowPart, ...);

The HIWORD and LOWORD macros are intended to extract the high and low 16-bit words from a 32-bit DWORD. CreateFileMapping, on the other hand, expects two DWORDs that together make up a 64-bit unsigned integer, which is the size of the mapping object.
Both HIWORD(UBS_MEM_SIZE) and LOWORD(UBS_MEM_SIZE) yield 0xffff (the two 16-bit halves), which are then converted to 32-bit unsigned integers (which is what the function expects).
So, what you're actually doing is asking for a file mapping of size 0x0000ffff0000ffff. This is more than 255 TB. Since you're using INVALID_HANDLE_VALUE, this has to be backed by RAM or the system page file; I doubt you have that much available in there.
If UBS_MEM_SIZE is always 32-bit, you can simply use
HANDLE hMapObject = CreateFileMapping(INVALID_HANDLE_VALUE, nullptr,
PAGE_READWRITE, 0, UBS_MEM_SIZE,
TEXT("dllmemfilemap"));
If you actually need to handle sizes over 4 GB, you can do something like this:
HANDLE hMapObject = CreateFileMapping(INVALID_HANDLE_VALUE, nullptr, PAGE_READWRITE,
static_cast<DWORD>(UBS_MEM_SIZE >> 32), static_cast<DWORD>(UBS_MEM_SIZE),
TEXT("dllmemfilemap"));
Make sure UBS_MEM_SIZE actually has a type larger than 32-bit (even if its value may be less than that), since otherwise shifting by 32 bits is undefined behaviour in C++. So, if you want to use the second variant above with your initial value, it will have to be something like
#define UBS_MEM_SIZE 0xFFFFFFFFull
(By the way, use const...)
To make it safer, I'd wrap the call into something like this:
inline HANDLE MyCreateMapping(unsigned long long size, LPCTSTR name)
{
return CreateFileMapping(INVALID_HANDLE_VALUE, nullptr, PAGE_READWRITE,
static_cast<DWORD>(size >> 32), static_cast<DWORD>(size), name);
}
This way, you don't need to remember any tricky details about bits, shifts and integer type sizes.

Related

Invalid Handle with NtWow64AllocateVirtualMemory64

Test code:
typedef NTSTATUS(NTAPI *ntalloc64t)(HANDLE, PULONG64, ULONG64, PULONG64, ULONG, ULONG);
#define NtCurrentProcess() ( (HANDLE)(PULONG64) -1 ) ;
int _tmain(int argc, _TCHAR* argv[])
{
ULONG64 dwSize = 0x1000;
ntalloc64t ntalloc64f = (ntalloc64t)(GetProcAddress(GetModuleHandleA("ntdll"), "NtWow64AllocateVirtualMemory64"));
PVOID pvBaseAddress;
pvBaseAddress = (PVOID)NULL;
long kk = ntalloc64f((HANDLE)GetCurrentProcess(), (PULONG64)&pvBaseAddress, 0, (PULONG64)&dwSize, MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE);
}
I am running under WOW64. This returns 0xc0000008 which means the handle is invalid. Also does not work when passing -1 as the handle, which should indicate to WinAPI to use the current process.
NtWow64AllocateVirtualMemory64 is undocumented but you can assume that its parameters are almost the same as NtAllocateVirtualMemory and MSDN says this about the base address parameter:
A pointer to a variable that will receive the base address of the allocated region of pages. If the initial value of this parameter is non-NULL, the region is allocated starting at the specified virtual address rounded down to the next host page size address boundary. If the initial value of this parameter is NULL, the operating system will determine where to allocate the region.
You are hiding a bug with your casts; (PULONG64)&pvBaseAddress points to 32 zero bits from pvBaseAddress = (PVOID)NULL and 32 undefined bits from somewhere on your stack and if these bits are not all zero then you are asking for a specific base address that is probably not available!
Remove as many casts as possible and it should start working:
typedef NTSTATUS(NTAPI *ntalloc64t)(HANDLE, PULONG64, ULONG64, PULONG64, ULONG, ULONG);
ntalloc64t ntalloc64f = (ntalloc64t) GetProcAddress(GetModuleHandleA("ntdll"), "NtWow64AllocateVirtualMemory64");
// TODO: if (!ntalloc64f) not wow64, handle error...
HANDLE hTargetProcess = OpenProcess(...);
ULONG64 base = 0, size = 0x1000;
long nts = ntalloc64f(hTargetProcess, &base, 0, &size, MEM_COMMIT|MEM_RESERVE, PAGE_READWRITE);
printf("status=%d base=%I64x size=%I64x\n", nts, base, size);
when we call NtWow64AllocateVirtualMemory64 from 32-bit ntdll.dll (it exist only in wow64 ntdll.dll) the whNtWow64AllocateVirtualMemory64 (64bit function) called inside wow64.dll. my reconstruction from win10 assembler code:
struct Wow64AllocateVirtualMemory64_Stack {
ULONG ProcessHandle;// !!! unsigned !!
ULONG BaseAddress;
ULONG64 ZeroBits;
ULONG RegionSize;
ULONG AllocationType;
ULONG Protection;
};
NTSTATUS
NTAPI
whNtWow64AllocateVirtualMemory64(Wow64AllocateVirtualMemory64_Stack* p)
{
return NtAllocateVirtualMemory(
(HANDLE)(ULONG_PTR)p->ProcessHandle,
(void**)(ULONG_PTR)p->BaseAddress,
p->ZeroBits,
(PSIZE_T)(ULONG_PTR)p->RegionSize,
p->AllocationType,
p->Protection);
}
key point here that HANDLE is 32-bit size in 32 bit code and 64-bit size in 64-bit code. as result 32-bit handle value must be extended to 64-bit handle in 64bit code. but it can be zero or sign extended. of course when we extend positive 32bit value (real process handle) - no different, result will be the same. but when we extend negative value -1 - result of zero extend will be 0xFFFFFFFF (this is invalid handle). result of sign extend - will be 0xFFFFFFFFFFFFFFFF - correct pseudo handle to current process. windows 10 use zero extend handle:
as result we can not use -1 (GetCurrentProcess()) here
win8 use sign-extend handle:
however no any sense use this api for allocate memory in wow64 process. really - if we accept any memory base address, or < 4GB - we can use NtAllocateVirtualMemory or VirtualAlloc[Ex]. so this function only have sense use in case we want allocate memory at base address >= 4Gb. but this is impossible in wow64 process. - system reserve all memory space higher than >= 4G. typical memory map for wow64bit process (with /LARGEADDRESSAWARE option)
so visible only 64bit ntdll.dll here, and all other memory is reserved.
without /LARGEADDRESSAWARE option reserved range begin from 7FFF0000. also this reserved memory can not be released - on call NtFreeVirtualMemory (from 64bit process) i got STATUS_INVALID_PAGE_PROTECTION error.
so no sense use this api for allocate inside self (and any another wow64 process). only if we want allocate memory in 64bit process and not simply allocate, but at range higher than 4GB. i even dont know for which target this can be need - why <4GB memory base, which can be allocated with usual NtAllocateVirtualMemory or VirtualAlloc[Ex] not ok. and funny that no related NtWow64FreeVirtualMemory64 api - so impossible free alocated memory. of course possible write base-independed (and as result no import) 64bit code, embedded in 32bit process, call it via 64 call gate, this code can call functions from 64bit ntdll (and only from it) and return. this is possible, but already another story

How to properly use ReadProcessMemory without raising warnings

What is the correct way to use ReadProcessMemory?
I am currently using it like this:
DWORD read_mem(DWORD addr)
{
DWORD buffer = 0x0;
if (!ReadProcessMemory(phandle, LPCVOID(addr), &buffer, sizeof(DWORD), nullptr))
{
return false;
}
return buffer;
}
This causes a warning due to addr being wrongly casted.
warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
Other example code I've seen such as this uses the same approach.
So what is the proper way to use this function without getting a warning?
"cast to pointer from integer of different size" - this means that DWORD and void* are different byte sizes, which can only happen if you are compiling your code for 64-bit, as they are the same byte size in a 32-bit compilation.
From the MSDN documentation, Windows Data Types:
DWORD
A 32-bit unsigned integer. The range is 0 through 4294967295 decimal.
A DWORD is simply not large enough to hold a 64-bit memory address (the other code you mention will similarly only work correctly in 32-bit).
Change Addr (and whatever code you are using to determine the value of Addr) to use DWORD_PTR instead:
DWORD_PTR
An unsigned long type for pointer precision. Use when casting a pointer to a long type to perform pointer arithmetic. (Also commonly used for general 32-bit parameters that have been extended to 64 bits in 64-bit Windows.)
Then Addr will be the correct byte size whether you compile for 32-bit or 64-bit.

Why bad_alloc() exception thrown in case of size_t

I am working on the below piece of code and when I'm executing this code, I'm getting a std::bad_alloc exception:
int _tmain(int argc, _TCHAR* argv[])
{
FILE * pFile;
size_t state;
pFile = fopen("C:\\shared.tmp", "rb");
if (pFile != NULL)
{
size_t rt = fread(&state, sizeof(int), 1, pFile);
char *string = NULL;
string= new char[state + 1];
fclose(pFile);
}
return 0;
}
This below line causing exception to be thrown:
string = new char[state + 1];
Why this is happening and how can I fix this?
You're passing the address of an uninitialized 64-bit (8 bytes, on modern 64-bit systems) variable, state, and tell fread to read sizeof(int) (32 bits, 4 bytes on those same systems) bytes from the file into this variable.
This will overwrite 4 bytes of the variable with the value read, but leave the other 4 uninitialized. Which 4 bytes it overwrites depends on the architecture (the least significant on Intel CPUs, the most significant on big-endian-configured ARMs), but the result will most likely be garbage either way, because 4 bytes were left uninitialized and could contain anything.
In your case, most likely they are the most significant bytes, and contain at least one non-zero bit, meaning that you then try to allocate far beyond 4GB of memory, which you don't have.
The solution is to make state a std::uint32_t (since you apparently expect the file to contain 4 bytes representing an unsigned integer; don't forget to include <cstdint>) and pass sizeof(std::uint32_t), and in general make sure that for every fread and similar call where you pass in a pointer and a size, you make sure that the thing the pointer points to actually has exactly the size you pass along. Passing a size_t* and sizeof(int) does not fulfill these requirements on 64-bit systems, and since the size of C++'s basic types is not guaranteed, you generally don't want to use them for binary I/O at all.
There are a various things which you could improve in your C++ code, but there are a number of reasons, why you end up with this behaviour:
First, the variable state is of type size_t, but your code attempts to initialize its value using fread(&state, sizeof(int), 1, pFile);. Now, if sizeof(state) != sizeof(int) then you have undefined behaviour. If sizeof(state) < sizeof(int), then the fread statement usually overwrites some arbitrary memory after the storage for variable state. This leads to undefined behaviour (e.g. state might have some random large value, and allocation fails).
Second, if sizeof(state) > sizeof(int), then state is only partially initialized and its actual value depends on both the initialized (by fread) and the uninitialized bits. So its value can be a large number and allocation may fail.
Third, the if sizeof(state) == sizeof(int) then it just might be that the the value read is too large, and allocation simply fails because you run out of memory.
Fourth, the value you read from the file might have some different encoding or endianness. For example, if value was written to the file in big-endian format, but is fread on a little-endian CPU, might cause the bytes to be incorrectly swapped. You might need to swap the bytes before using the value read.
I suggest you instead use some fixed-width integer type from <cstdint> (or <stdint.h> for pre-C++11), such as std::uint64_t for variable state, read the value using fread(&state, sizeof(state), 1, pFile);, and then byte-swap state if the endianness of your CPU doesn't match the endianness of the value stored in the file.
You should decide what the maximum number of characters you are willing to allocate is and error out if state is greater than that. Almost certainly, it is.

C++ Conversion from Float to Hex not correct when writing to Memory

I'm trying to convert a float value (0.75) to hex and write that converted value to memory.
char Actual[4];
float f = 0.75f;
int i = *(reinterpret_cast<int*>(&f));
wsprintf(Actual, "%08X", i);
MessageBox(NULL, Actual, "Float", NULL);
unsigned long OldProtection;
VirtualProtect((LPVOID)(0x01234567), 4, PAGE_EXECUTE_READWRITE, &OldProtection);
memcpy((LPVOID)0x01234567, Actual, 4);
VirtualProtect((LPVOID)(0x01234567), 4, OldProtection, NULL);
The conversion works quite well and outputs the correct value (3F400000) when using MessageBox.
But when writing the converted value to memory using memcpy the value of the target address is 30303030 and not 3F400000.
I guess I'm missing some additional step. What could be the problem?
You're writing text to Actual.
Hex 30 is the ASCII code for a zero digit.
Why would you want to do that? Is there something special at that address - is it used by some other code (other than that changing it) or you just want to see the "hex value" of the float number by inspecting the process memory at 0x01234567 with external tool? In every case you could just copy the data in the 'f' variable at the desired location.
unsigned long OldProtection;
VirtualProtect((LPVOID) (0x01234567), 4, PAGE_EXECUTE_READWRITE, &OldProtection);
memcpy((LPVOID) 0x01234567, &f, sizeof(float));
VirtualProtect((LPVOID) (0x01234567), 4, OldProtection, NULL);
The 'VirtualProtect' method calls could be mostly unneeded unless you have a special reason for this (commonly data is stored in read/writable locations).
By the way here is some useful information - the 'hex value' you are working with is an architecture specific one and it represents an encoded floating-point number in some bit format which is required by the CPU floating-point arithmetic instructions. In x86 this is IEEE 754.

Passing HRESULT as a string on command line

I have a need to pass in an HRESULT value to a program as a command line argument. I had intended to do so by passing the hex value, e.g.:
>receiver.exe 0x80048836
I'm trying to convert this string representation back into an HRESULT using wcstol, eg:
HRESULT hr = wcstol(argv[2], NULL, 16);
However, the value of the original HRESULT is usually greater than LONG_MAX, so in the line above hr ends up as 0x7fffffff.
So, two questions:
I thought HRESULTS were just 32-bit integers? So I'm not sure how I'm getting back an HRESULT greater than LONG_MAX. It seems to work fine in the originating program, though (i.e. the HRESULT doesn't overflow).
Is there a way to get around the LONG_MAX restriction of wcstol? Maybe another version of the function that matches up with whatever size integer the HRESULT actually is?
Thanks!
Check out wcstoul. http://msdn.microsoft.com/en-us/library/5k9xb7x1(v=VS.80).aspx
The HRESULT does fit in 32 bits, but with the example you gave it uses the most significant bit, which is considered to act like a sign bit for signed integers. Using wcstoul will fit it into an unsigned long.
LONG_MAX is 0x7FFFFFFF, the highest that can fit in the 31 least significant bits, leaving the top bit cleared, but ULONG_MAX goes up to 0xFFFFFFFF because it is unsigned.
0x80048836 is greater than LONG_MAX for your system (2147483647L) which is (0x7FFFFFFF). According to msdn "when the representation would cause an overflow, in which case it returns LONG_MAX or LONG_MIN"
So in your case you get LONG_MAX returned as your result.
the actual function return type is declared as long wcstol(...). long is not necessarily 32 bits in size, that will depend on your system.
In this case the return type is signed and 32 bit so the largest signed integer that will fit in 32 bits is 7FFFFFFF.
00000000 to 7FFFFFFF is positive from 0 to LONG_MAX
FFFFFFFF to 8000001 is negative from -1 to LONG_MIN
Incidentally I believe "HRESULT hr = wcstol..." would be incorrect since the return type of wcstol is (signed) long , but HRESULT is ULONG (unsigned long). This might be a problem depending on how you use that data.