I want to modify the value of a register that is in a certain program.
The only problem is, I don't know how to access it. If there is a way, how do I read/write into it? (preferred language C++)
If you want to modify a particular register one time while the program is running you can do so using a debugger, such as OllyDbg.
If you want to modify the code while the program isn't running, such that in the future when you run the program it behaves differently, you can view the Assembly using a disassembler such as IDA. But you'll also need something that can reassemble the program with your modifications, such as NAsm
You can also attach one program to another while both are running using the OpenProcess() function in windows. You can then read and write arbitrary values to the other process, including modifying it's code. This is a pretty tricky thing to set up and have work properly... It's how debuggers work, which are usually pretty complex pieces of software. Better to use an existing one than to try to write your own!
As far as I understand you correctly you want to write a program that:
Stops another running program at a certain address (breakpoint)
Waits until the breakpoint is reached
Reads the current value of a certain register
Writes another value into that register
Continues program execution after the breakpoint
Not using a breakpoint makes absolutely no sense because the assembly registers are used for completely different purposes in different parts of the program. So modifying an assembly register only makes sense when the other program is stopped at a specific position.
Under Windows, Win32 code (not .NET code), you may du this the following way:
Start the other .EXE file using CreateProcess with the DEBUG_PROCESS (and maybe - what would be more safe - the CREATE_SUSPENDED) flag set or...
...use DebugActiveProcess to debug an already running process.
Use ReadProcessMemoryand WriteProcessMemory to replace the assembler instruction (to be correct: the byte and not the instruction) at the breakpoint address by 0xCC (int3).
If you used CREATE_SUSPENDED use ResumeThread to actually start the other .EXE file
Call WaitForDebugEvent and ContinueDebugEvent until the breakpoint is reached. You may use GetThreadContext to get the location where the program is stopped.
Replace the int3 instruction by the original instruction using WriteProcessMemory
Use GetThreadContext and SetThreadContext to modify registers. Node: After an int3 instruction the EIP register must be decremented!
Use ContinueDebugEvent to let the program continue
Unfortunately your program has to wait for further debugging events of the observed EXE file and handle them (WaitForDebugEvent and ContinueDebugEvent) - even if you "just" want the program to run...
If you want to stop the program at this breakpoint multiple times it is becoming more complicated: You must set the "trace" flag in the EFlags register each time you continue after the breakpoint so a single-step is done; after that single step you have to replace the original assembler instruction by int3 again.
I myself did this years ago because I wrote some inspection program for that worked similar to "strace" in Linux...
If you are using C++.NET (or C#): There is a .NET assembly that allows you doing all this but using this assembly is not easier than the Win32 variant.
If you want to do this for educational uses only:
Unfortunately the debugging API of Windows is by far less easy to use then the debugging API (ptrace) of Linux. If you just want to do this for educational purposes (and the EXE file you want to inspect is written by you) I would do this excercice under Linux, not under Windows. However even under Linux this is not an easy job...
I'm writing a C++ Software Image processing tool.The tool works fine, but suddenly it stops and it never sends any exception or any crash or nothing that can let me which line or which area that does that crash.
How can I determine that faulty code in that situation?
There's a few things you can do:
First of all though, it sounds more like an infinite loop, deadlock, or like you're using all of your system resources and it's just slowing down and taking a very (possibly infinite) long time. If that's the case, you'll have to find it with debugging.
Things that you can try - not necessarily in this order:
Look for shared variables you're using. Is there a chance that you
have a deadlock with threads and mutexes? Think about it and try to
fix it.
Check for use of uninitialized variables/pointers. Sometimes
(rarely) you can get very strange behavior when you invoke undefined
behavior - I'm not a windows C++ dev (I work on Linux), but it
wouldn't be the first time I saw a lockup from a segmentation fault.
Add error output (std::cerr/stderror) to your processing logic so
you can see how far in it crashes. After that, set a condition to
catch it around that point so you can watch it happen in the
debugger and see the state of your variables and what might be
wrong.
Do a stack trace so you can see what calls have been hit most
recently. This will at least let you know the last function chain
that executed.
Have you used stack tracing before?
Look up MSDN documentation on how to use them. They have different type of stack trace depending on your application.
You could
Use a debugger
Add logging code to your program
Cut sections of code from your program until it starts working
Start with the first.
When I want to find a segfault or any other error which leads to a crash of a program, i would always inspect the core dump with gdb. This is quite painful when such an application runs on a computer without gdb installed.
So a few days ago I used a program (JDownloader) wich wrote a crash log file, and this file contained a stack trace. I thought this would be a great enhancement to my application. But I haven't found any information on how to write a file which contains the stacktrace just before the crash.
Is it even possible? How would I do this on Linux/Windows?
I'm using C/C++.
I believe JDownloader is written in Java. I think the language allows you to retrieve a full plain text stack trace at any point. C++ is unable to do this, because the compiled executable usually doesn't keep any information about the code used to generate it.
Windows API does allows you to catch fatal exceptions and create a dump of the process (or parts of the process, if you don't want to deal with a huge file). This dump can then be inspected with windbg, Visual Studio, or your debugger of choice.
The downside to this is that you must have the exact source code that was used to build the dumped executable, as well as the symbol database (PDB file) that was generated during the build. On top of that, some code can be optimized in ways that makes it impossible for the debugger to give you an accurate stack trace, even with the symbol data.
See MiniDumpWriteDump for details. If you're going to take this route, the best practice is to not generate the dump in the crashing process, but spawn a child process to take a dump of the parent.
There are also C and C++ libraries that can 'manually' record the call stack and give you a textual representation of it at run time, but I haven't encountered any of these that I would suggest.
I need to use a function in a obj file for which I don't have the source code. The function works well and does its job but it corrupts the stack (runtime esp was not properly saved.. error in visual studio c++ 2010), and I don't know assembly well enough to find the problem and patch it. (From what Ive read it has to do with not having an equal number of pops and pushes.) I have tried executing the function in a separate thread but since it shares the same memory as my main program, it also crashes. The function works and does everything it needs to do, expect that it corrupts the stack and make the application crash. Is there any way I can isolate it from my main application? Ive read about CreateProcess but from what I understood it executes a separate .EXE file which I don't want to do. I want to execute a function as if it was an external program, so it does not crash my main program even though it corrupts the stack. Anyway to do such a thing in C/C++? Or maybe someone has another idea to fix things up? Also, I don't care if its "dirty" or not standard compliant since im writing this program for personal use. Since the function does what it needs to do Im really trying to keep it even though it corrupts the call stack.
Thank you very much
That smells very much like a calling convention mismatch. Is it documented anywhere how the function is to be called? Do you have a header file containing a function prototype for the function? Try calling it with a different calling convention—if you're calling it as cdecl, try calling it as stdcall, or vice-versa.
See also What can go wrong when you mismatch the calling convention?.
I've stumbled onto a very interesting issue where a function (has to deal with the Windows clipboard) in my app only works properly when a breakpoint is hit inside the function. This got me wondering, what exactly does the debugger do (VS2008, C++) when it hits a breakpoint?
Without directly answering your question (since I suspect the debugger's internal workings may not really be the problem), I'll offer two possible reasons this might occur that I've seen before:
First, your program does pause when it hits a breakpoint, and often that delay is enough time for something to happen (perhaps in another thread or another process) that has to happen before your function will work. One easy way to verify this is to add a pause for a few seconds beforehand and run the program normally. If that works, you'll have to look for a more reliable way of finding the problem.
Second, Visual Studio has historically (I'm not certain about 2008) over-allocated memory when running in debug mode. So, for example, if you have an array of int[10] allocated, it should, by rights, get 40 bytes of memory, but Visual Studio might give it 44 or more, presumably in case you have an out-of-bounds error. Of course, if you DO have an out-of-bounds error, this over-allocation might make it appear to be working anyway.
Typically, for software breakpoints, the debugger places an interrupt instruction at the location you set the breakpoint at. This transfers control of the program to the debugger's interrupt handler, and from there you're in a world where the debugger can decide what to do (present you with a command prompt, print the stack and continue, what have you.)
On a related note, "This works in the debugger but not when I run without a breakpoint" suggests to me that you have a race condition. So if your app is multithreaded, consider examining your locking discipline.
It might be a timing / thread synchronization issue. Do you do any multimedia or multithreading stuff in your program?
The reason your app only works properly when a breakpoint is hit might be that you have some watches with side effects still in your watch list from previous debugging sessions. When you hit the break point, the watch is executed and your program behaves differently.
http://en.wikipedia.org/wiki/Debugger
A debugger essentially allows you to step through your source code and examine how the code is working. If you set a breakpoint, and run in debug mode, your code will pause at that break point and allow you to step into the code. This has some distinct advantages. First, you can see what the status of your variables are in memory. Second, it allows you to make sure your code is doing what you expect it to do without having to do a whole ton of print statements. And, third, it let's you make sure the logic is working the way you expect it to work.
Edit: A debugger is one of the more valuable tools in my development toolbox, and I'd recommend that you learn and understand how to use the tool to improve your development process.
I'd recommend reading the Wikipedia article for more information.
The debugger just halts execution of your program when it hits a breakpoint. If your program is working okay when it hits the breakpoint, but doesn't work without the breakpoint, that would indicate to me that you have a race condition or another threading issue in your code. The breakpoint is stopping the execution of your code, perhaps allowing another process to complete normally?
It stops the program counter for your process (the one you are debugging), and shows the current value of your variables, and uses the value of your variables at the moment to calculate expressions.
You must take into account, that if you edit some variable value when you hit a breakpoint, you are altering your process state, so it may behave differently.
Debugging is possible because the compiler inserts debugging information (such as function names, variable names, etc) into your executable. Its possible not to include this information.
Debuggers sometimes change the way the program behaves in order to work properly.
I'm not sure about Visual Studio but in Eclipse for example. Java classes are not loaded the same when ran inside the IDE and when ran outside of it.
You may also be having a race condition and the debugger stops one of the threads so when you continue the program flow it's at the right conditions.
More info on the program might help.
On Windows there is another difference caused by the debugger. When your program is launched by the debugger, Windows will use a different memory manager (heap manager to be exact) for your program. Instead of the default heap manager your program will now get the debug heap manager, which differs in the following points:
it initializes allocated memory to a pattern (0xCDCDCDCD comes to mind but I could be wrong)
it fills freed memory with another pattern
it overallocates heap allocations (like a previous answer mentioned)
All in all it changes the memory use patterns of your program so if you have a memory thrashing bug somewhere its behavior might change.
Two useful tricks:
Use PageHeap to catch memory accesses beyond the end of allocated blocks
Build using the /RTCsu (older Visual C++ compilers: /GX) switch. This will initialize the memory for all your local variables to a nonzero bit pattern and will also throw a runtime error when an unitialized local variable is accessed.