How reliable is hooking for changing a single static memory address when it hits certain values?
What I'm used to doing is using read/write memory out of a basic c++ application, though I find sometimes this is not reliable for addresses that change 1000+ times per second. Often time my application cannot catch the value at the address with a case function in time enough to change it to another value. How exactly does this concept of hooking work, and does it ever miss a value change? I'm using Win 7 Ult. x86
(reusing an answer I gave to a question I thought was related, but turned out not to be.)
There are environment-specific ways to detect when a variable is changed. You can use the MMU access control flags (via mprotect or VirtualProtect) to generate an exception on the first write, and set a dirty flag from inside the handler. (Almost every modern OS does this with memory-mapped files, to find out whether it needs to be written back to disk). Or you can use a hardware breakpoint to match a write to that address (debuggers use this to implement breakpoints on variables).
Hooking can be done in many ways.
Most require you to have code inside your target process making ReadProcessMemory obsolete (just use pointers and dereference them).
If you want to hook though you can do it like this:
Find out what instruction(s) write to that address (debugger memory breakpoint), it will most likely be a function so what I usually do is just patch some bytes near the beginning to redirect execution flow to my code where it will be executed every time that function is called, what I sometimes do is also alter the return address on the stack so that I can examine and control the return value as well as execute code I want executed after the function is finished (for example, get some info from the stack because I am either too lazy to dig out the structures used to store it or if it's temporary it will be discarded and never saved).
Related
A client came to me with an application that they lost the source code to. This application is crashing seemingly at random, when loading some files. I suspect that the issue is due to a race condition in which a pointer is deleted and then either not set to NULL or not checked for validity.
When stepping through the assembly using OllyDBG, I found that the crash ALWAYS happens at the same location, so this is kind of re-enforcing my theory. This is the assembly line it sometimes crashes on, keyword sometimes.
MOV EDI,DWORD PTR DS:[EAX]
Is it possible to validate a memory address is valid and exists either through native assembly or through C++ that pulls an address through an inline assembly call (or something like this)?
There is no general standard way in C++ to validate a memory address. Nor is there such assembly instruction that I know of.
On a memory mapped system (such as any modern operating system), you may be able to check whether an address has been mapped for the process using a system specific API. An address being mapped to the process doesn't guarantee that the address is valid from the C++ point of view, but an unmapped address is definitely invalid.
Furthermore, even if could find that an address is valid, that doesn't tell you whether the object you are expecting is in that address or something else.
There are tools for validating memory accesses outside of the C++ language1. There's for example Valgrind and also compiles provide address sanitisers and memory sanitisers. Mostly, these help detect invalid accesses that wouldn't have crashed the program otherwise. But they also often can provide additional information regarding that memory.
1 If you had access to the source.
I suspect that the issue is due to a race condition
Being able to validate a memory address won't solve this problem. What you should do1, is to use a debugger to find out what object is being accessed, find all places where that object is accessed. If any of those places is not holding a mutex that is common to all other places potentially accessing the object at the same time, then there's your bug.
1 If you had access to the source.
lost the source code
If you have a massive budget, then you could try reverse-engineering it and try to figure out what it's doing. I wouldn't hold my breath; it may be best to declare this as a lost cause.
Everyone provided some great responses. Unfortunately, in this situation, the client wasn't willing to pay to have the software re-developed entirely.
I managed to kind of negate the crash... After identifying exactly where the crash occurred I used used code injection to JMP to a custom C++ function. The C++ function employs a __try __except block. Inside of the Try I placed the problem assembly code, and if the problem code causes a crash, the __except catches it. After that, it was just a matter of analyzing what the problem code was doing and when it started doing something else and then jumping to the start of that "something else" part of the problem function when an exception is caught.
It's not exactly detecting if a memory location is valid or not, but rather it's just skipping code if it causes an exception. Hope this helps someone eventually...
Yes, validating a pointer's content is possible. However, verifying that the target is the correct one is more difficult.
In order to validate a pointer you'll need:
Table of valid address ranges (addresses that are implemented and their ranges).
For example, on embedded systems addresses that are decoded may not have memory or devices at all locations.
For OS's that support paging or virtual memory, you'll need to figure out the limits of your program's memory area (as given by the OS). The OS may take portions of your executable and swap them out with code on a hard drive.
For OS's that support virtual memory, you'll have to figure out where in the "virtual" memory that your pointer is allowed to access. Read about memory mapped files.
On some platforms, the address 0x0000 is a valid address. Verify if this is the case on your target platform.
IMHO, pointer content can be validated, but the validation may be very compilicated.
Prefer to use references.
Windows 10, x64 , x86
My current knowledge
Lets say it is quad core, there will be 4 individual program counters which will point to 4 different locations of code for parallel execution.
Each of this program counters indicates where a computer is in its program sequence.
The address it points to changes after a context switch where another threads program counter gets placed onto the program counter to execute.
What I want to do:
Im in Kernel Mode my thread is running on core 1 and I want to read the current instruction pointer of core 2.
Expected Results:
0x203123 is the address of the instruction pointer and this address belongs to this thread and this thread belongs to this process... etc.
Anyone knows how to do it or can give me good book references, links etc...
Although I don't believe it's officially documented, there is a ZwGetContextThread exported from ntdll.dll. Being undocumented, things can change (and I haven't tried it in quite a while) but at least when I last tried it, you called it with a thread handle and a pointer to a CONTEXT structure, and it would return that thread's context.
I'm not certain exactly how up-to-date that is though. It's never mattered to me, so I haven't checked, but my guess would be that the IP in the CONTEXT you get is whatever was saved the last time the thread was suspended. So, if you want something (reasonably) current, you'd use ZwSuspendThread, get the context, then ZwResumeThread to start it running again.
Here I suppose I'm probably supposed to give the standard lines about undocumented function being subject to change, using them being a bad idea, and that you should generally leave all of this alone. Ah well, I been disappointing teachers and other authority figures for years, and I guess I'm not changing right now.
On the other hand, there may be a practical problem here. If you really need data that's really current, this probably isn't going to work very well for you. What it gives you will be kind of current at best. On the other hand, really current is almost a meaningless concept with information that goes out of date every clock cycle.
Anyone knows how to do it or can give me good book references, links etc...
For 80x86 hardware (regardless of operating system); there are only 3 ways to do this (that I know of):
a) send an inter-processor interrupt to the other CPU, and have an interrupt handler that stores the "return EIP" (from its stack) at a known address in memory so that your CPU can read "value of EIP immediately before interrupt" (with synchronization so that your CPU doesn't read before the value is written, etc).
b) put the other CPU into some kind of "debug mode" (single-stepping, last branch recording, ...) so that (either code in a debug exception handler or the CPU's hardware itself) is constantly writing EIP values to memory that you can read.
Of course both of these options will ruin performance, and the value you get will probably be useless (because EIP would've changed after you obtain it but before you can use the obtained value). To ensure the value is still useful; you'd need the other CPU to wait until after you've consumed the obtained value (and are ready for the next value); and to do that you'd have to resort to single-step debugging facilities (with the waiting in the debug exception handler), where you'll be lucky if you can get performance better than a thousand times slower (and can probably improve performance by simply disabling other CPUs completely).
Also note that they still won't accurately tell you EIP in all cases (e.g. if the CPU is in SMM/System Management Mode and is beyond the control of the OS); and I doubt Windows kernel supports any of it (e.g. kernel should support single-stepping of user-space processes/threads to allow debuggers to work, but won't support single-stepping of kernel and will probably lock up the computer due to various "waiting for lock to be released for 6 days" problems).
The last of the 3 options is:
c) Run the OS inside an emulator/simulator instead of running it on real hardware. In that case you can probably modify the emulator/simulator's code to inject EIP values somewhere (maybe some kind of virtual "EIP reporting device"?). This will ruin performance of the emulator/simulator, but you may be able to hide that (e.g. "virtual time inside the emulator passes at a rate of one second per 1000 seconds of real time outside the emulator").
I am a beginner C++ programmer.
I wrote a simple program that creates a char array (the size is user's choice) and reads what previous information was in it. Often you can find something that makes sense but most of it is just strange characters. I made it output into a binary file.
Why do I often find multiple copies of the alphabet?
Is it possible to find a picture inside of the RAM chunk I retrieved?
I heard about file signatures (headers), which goes before any of the data in a file, but do "trailers" go in the back after all the data?
When you read uninitialized data from memory that you allocated, you'll never see any data from another process. You only ever see data that your own process has written. That is: your code plus all the libraries that you called.
This is a security feature of your kernel: It never leaks information from a process unless it's specifically asked to transfer that information.
If you didn't load a picture in memory, you'll never see one using this method.
Assumning your computer runs Linux, Windows, MacOS or something like that, there will NEVER be any pictures in the memory your process uses - unless you loaded them into your process. For security reasons, the memory used by other processes is cleared before it gets given to YOUR process. This is the case for all modern OS's, and has been the case for multi-user OS's (Unix, VAX-VMS, etc) more or less since they were first invented in the late 1950's or early 1960's - because someone figured out that it's kind of unfun when "your" data is found by someone else who is just out there fishing for it.
Even a process that has ended will have it's memory cleared - how would you like it if your password was still stored in memory for someone to find when the program that reads the password ended? [Programs that hold highly sensitive data, such as encryption keys or passwords, often manually (as in using code, but not waiting until the OS clears it when the process ends) clear the memory used to store such, because of the below debug functionally allowing the memory content to be inspected at any time, and the shorter time, the less likely a leak of sensitive information]
Once memory has been allocated to your process, and freed again, it will contain whatever happens to be in that memory, as clearing it takes extra time, and most of the time, you'd want to fill it with something else anyway. So it contains whatever it happens to contain, and if you poke around it, you will potentially "find stuff". But it's all your own processes work.
Most OS's have a way to read what another process is doing as part of the debug functionality (if you run the "debugger" in your system, it will of course run as a separate process, but needs to be able to access your program when you debug it, so there needs to be ways to read the memory of that process), but that requires a little more effort than just calling new or malloc (and you either will need to have extra permissions (superuser, adminstrator, etc), or be the owner of the other process too).
Of course, if your computer is running DOS or CP/M, it has no such security features, and you get whatever happens to be in the memory (and you could also just make up a pointer to an arbitrary address and read it, as long as you stay within the memory range of the system).
This question already has answers here:
Watch a memory location/install 'data breakpoint' from code?
(5 answers)
Closed 9 years ago.
In Windows (both 32 and 64 bit), through program (C++) is it possible to determine if a certain memory location has changed? I am trying to extrapolate the concept that we see in Visual Studio where we can set data break point.
Use Case: I understand its a dirty hack, but the fastest to implement to be re-implemented later
I am sharing data across process boundary (read between a 32 bit client and 64 bit server). The Client allocates memory (beyond our control) and passes the address to the server. The Server allocates a storage to shadow the client memory and via various code path can update that shadowed memory location. Instead of identifying and trapping each of these location (I was trying to find an easier path), to raise an event on change and eventually write back the data through WriteProcessMemory to the client process
Whilst it's probably possible to find a solution using a combination of VirtualProtect and the Windows debug interface, I'm not sure it's a particularly good solution for this scenario. One of the problems is that you introduce a delay on every new write, and you are looking at a transfer to another process that is monitoring the program as a "debugger". That process will then have to "unprotect" that page, mark it as "updated" for the other Server (or Client, depending on which direction you are going), and "continue" the application making the update. This is quite time consuming. And of course, there is no trivial way to know when the writing process has completed a sequence of updates. You also need to know exactly where to "continue" when there is a SEH "__except" call, and it's not always entirely trivial do to that, especially if the code is in the middle of a memcpy or something like that.
When I worked with graphics, I know that both our and some competitors driver would do this, first write-protect the memory, and then by hooking into the windows own page-fault handler, look up the page-fault, see if it's the special region(s), and if so, mark that page as updated and reset it to writeable. This allowed the driver to only copy the updated regions. But in this case, there is a distinct "I want to draw this stuff" after all the updates have been made.
If you want to badly enough, you can use the debug API to set a breakpoint on your own data, which will be triggered on a write, just like the VS API does.
The basics of this are to start a thread to do the "debugging". It will:
temporarily stop the primary thread.
Get that thread's registers with GetThreadContext
set the address to break on in one of DR0 through DR 3.
Set the size of the data in question in DR 6.
Set the type of breakpoint (data write, in this case) in DR 7.
Use SetThreadContext to tell the primary thread to use the modified registers.
Restart execution of the primary thread.
That's going from memory, so although I believe it's pretty close to the basic idea, I may have gotten a detail or two wrong, or (more likely) left out a few steps.
In most cases, it's going to be easier to do something at the source level, where you overload operator= for the target in question, so you get some code executed during assignment to the data. Then in that operator you can (for example) set an Event that code in another thread waits on (and reacts appropriately).
Another possibility (especially if you want to break on any access to a whole range of addresses) is to use VirtualProtect to force an exception on any access to that block of memory. Like a debug exception, this will be triggered synchronously, so if you want asynchronous execution you'll have to accomplish it by setting an Event (or whatever) and having another thread waiting on that so it'll execute when the Event is set.
I have an address that get's writen to 1000x per second by 300 different instructions. How can I use c++ to find out the last instruction to write to an address?
I already have made it so it alerts me the instance a specific value is written to an address, but how can I make it print the last instruction address that wrote that specific value?
I would do this in a debugger but all of the debuggers I've found cannot handle doing a conditional breakpoint on an address that changes 1000x per second without freezing the program.
If I can't do this in C++, what are other ways that I can do this? I need to find what address instruction writes a specific value to a memory address that receives over 1000 writes per second from different addresses.
Update:
I am using Windows 7 x32 for those wondering.
Take a look at pin. Briefly, pin allows you to instrument your code at the x86 instruction level, allowing you to track reads and/or writes as you please. I've used it myself to model cache performance and found it fairly fast.
already have made it so it alerts me the instance a specific value is written to an address, but how can I make it print the last instruction address that wrote that specific value?
If it's just for one-off debugging, have the code that alerts system/popen pstack (http://www.linuxcommand.org/man_pages/pstack1.html) or similar - some external program that dumps your call stack. Exactly which program to use is highly OS dependent, and you've said nothing of your environment. (This is a common technique for generating call stacks from signal handlers after invalid memory accesses etc.)