I have an application, which is running for a long time and then crash. I need to debug it several times to fix it and don't want to wait every time for an hour to reach the state, in which an error is occurred.
So, I want some tool to clone the whole process on a disk, then raise it, attach to it and debug.
I use Visual Studio 2012/2013 on (surprise) Windows.
For example:
for (int i = 0; i < 10000; ++i)
{
if (i == 9999)
throw MyExcept();
}
And I want to have a saved state of application (process) at 9998-th iteration to start debugging from it.
UPD 1: Visual Studio dump files are not admirable, because I can't get all functionality of debugger after opening it it VS. For example: I can't set breakpoints and even old ones don't work.
UPD 2: Also I need to have a possibility of duplicating this saves session of the app.
If I understood correctly, you want to accomplish two tasks:
1) Break the execution on a specific condition
You need to set a watchpoint with a condition to achieve this, or Data Breakpoint in Visual Studio terms. Have a look at this question: Can I set a breakpoint when variable is getting a specific value in .NET?
2) Dump a core file
Once you have set the watchpoint and your program reached that point, you can dump a core file. From that you can continue with the execution later on. There is an official FAQ entry entry on how to dump and load cores.
You need procdump from here
Register as the Just-in-Time (AeDebug) debugger.
Perhaps you should enable full local dumps.
Crash you program
Launch a process with procdump from full dump
But I think it is better to use DebugBreak API in your example without a crash.
Usually crash will not allow you to start further - only postmortem analysis.
Related
I'm having some difficulties determining what is causing a process to exit. I have a breakpoint in some shutdown code that I'm debugging, but, after breaking in the debugger at the breakpoint and stepping once, the whole process exits immediately. Every thread reports an exit code of -1 in the output window. There are a large number of threads in the process at that time, and the code base is quite large, making searching for the culprit difficult.
I've tried installing a std::atexit function, but this doesn't get hit. I've also tried overriding SetUnhandledExceptionFilter, in case it is caused by a crash, and it also doesn't get hit. The project has exceptions disabled (#define _HAS_EXCEPTIONS=0), so I cannot call std::set_terminate or std::set_unexpected.
Is there some other way to determine what is causing the process to exit? Some option to break in the debugger when the process is about to terminate?
Run your app with debugger and read the debug output. If the app terminates because C++ exceptions, or SEH, you’ll read it in the output window.
If you’ll see nothing interesting there, it means your app called ExitProcess/ExitThread/exit, or worse, TerminateProcess/TerminateThread/_exit.
You can put breakpoints on these. Set a breakpoint at startup, launch debugger. Ensure you have debug symbols loaded for relevant DLLs, kernel32.dll for ExitProcess and friends, some other DLL for exit, e.g. ucrtbase.dll. Press “New / Function breakpoint” in the Breakpoints window, type e.g. “ExitProcess”, press OK.
You can also try using gflags tool from Windows SDK.
If you’ll find (by reading Windows Logs > Application) the reason was self exit, you can check “Enable dump collection” in gflags, then you’ll be able to load the dump in WinDBG and get the complete call stack telling you who called what.
Unfortunately, the latest version of the tool is broken beyond repair.
But you can install older Windows SDK. You only need “Debugging tools for Windows” from there, no need to install the complete SDK.
I am trying to extract the execution sequence of my program (something like a program counter) with gdb on my local computer (windows x86) and gdbserver on a remote target (arm-linux). The idea I had was to insert breakpoints at "important" lines of my source files (i.e.: at the beginning of a specific function, and more in general before and after a conditional statement) with a high ignore count for each breakpoint, and then check if a breakpoint was hit or not. I was actually able to receive the informations with this method, but there is a problem: the application behavior I am debugging depends on real-time, and this specific method slows down the program execution too much. Do you think I could use some other method with gdb? I stumbled upon tracepoints, wich seems the exact thing I am looking for, but I was not able to find some property like a "hit counter" for them. The gdb version I am currently using is 7.5.
Thanks a lot in advance.
If your program execution must not be slowed down, you will probably need some HW tool. See these:
Keil real time trace
Lauterbach PowerDebug
(probably other similar solutions)
Question: Can I set up VS2010 so it automatically writes debug output to a file?
Motivation: I have a DirectX 9 application that I'm trying to debug. I've noticed that when my application is fullscreen, it may crash under certain conditions. Normally I would just check my logs or DirectX debug output. However, the way my program crashes prevents that. It freezes and does not respond to any my attempts to end it (including "End Process" from task manager). Moreover, it also freezes my VS2010, and so VS doesn't respond to any commands either. The only way out of this whole thing that I've found is to End VS process. This, however, also destroys the output I'd very much like to read.
Now I see two ways out of this. First is to write all the debug info to a file but I have no idea how to do it. Second is to make my application crash in a more friendly way, but this seems like a difficult task.
Probably MiniDumpWriteMiniDump(..) helps on this. You can at any time dump the current state of the process to a file. After that, you can open the dumps with Visual Studio and analyze the state of the process - this includes callstacks of every thread, variable values...
Try to identify conditions in which your process crashes and write one or more dumps.
Another try is to install the Windows Debugging Tools and use WinDbg to debug your application. This is not as comfortable as Visual Studio, but allows a deeper insight.
Edit:
If there are debug statements made with OutputDebugString(..), you can use DebugView (from Microsoft, earlier Sysinternals) to display it.
I have a native C++ program which runs more than 20 times slower when started with Debug (F5), but runs at normal speed when using start without debug (Ctrl + F5).
It does not matter whether I use a debug or release build. Also if I use WinDbg the program is a magnitude slower.
Is there some setting I did choose wrong or something?
Set the _NO_DEBUG_HEAP environment variable to 1 (as seen on http://preshing.com/20110717/the-windows-heap-is-slow-when-launched-from-the-debugger).
This can be done from inside Visual Studio, too.
Now this is just a workaround, and I'm curious to know how to refactor a program which suffers from this kind of problem. Do you have many std::map's, shared_ptr, or any other big indirections by any chance?
This is of course not caused by having the _DEBUG symbol defined or compiling the code in the debug configuration. The added debugging code runs whether or not the debugger is attached to the program.
The debugger doesn't normally affect code execution, it stays out of the way by calling WaitForDebugEvent. Which blocks it, until the operating system tells it that something noteworthy happened. That can trigger a bunch of code in the debugger that can slow down your program. You can see the events listed in the DEBUG_EVENT structure documentation.
Annotating them a bit beyond the documentation: the debugger steps in and can slow down your program when:
The program loads or unloads a DLL. Lots of stuff happens during load, the debugger goes hunting for a debug symbol file (.pdb). It may contact a symbol server to download it. Any breakpoints that were set in the DLL source code will get activated. This can be quite slow, but the effect is temporary and generally only slows down the startup. You can see the load/unload notification in the Output window.
The program raises an exception. This activates the debugger at the moment the exception is raised, a "first chance notification". Which can be very helpful, you can use the Debug + Exception, Thrown checkbox to make the debugger stop when the exception is raised. You can see the notification message in the Output window. This does slow down code that raises and catches exceptions tremendously and is quite likely the source of your slowdown. Never use exceptions for flow control.
A thread starts running or terminates. Again, a notification message is printed to the Output window. You'd have to create a lot of threads to make this slow down your program.
When your program uses OutputDebugString() for tracing purposes. Visible in the Output window. Another good candidate for a slow down, output falls in the bit bucket if no debugger is attached. You shouldn't have any trouble diagnosing this as the cause, the obvious side-effect is seeing a lot of messages in the Output window.
When the program hits a breakpoint. Not a lot of reasons to be stumped by that one. But you can set breakpoints that slow down the program a lot yet don't cause a debugger break. Particularly the Conditional breakpoint, Hit counter, Filter and the When Hit operation will be slow. Use Debug + Windows + Breakpoints to review the breakpoints that are defined.
When a process is created under the debugger, the operating system by default uses the debug heap. The debug heap does more verification of your memory, especially at de-alloc.
There are several possible options in order to disable the use of the Debug Heap:
Attach to the process soon after startup. This will allow you to speed up performance in debug mode knowingly so and being fully aware of the mode in which you are running.
Add the environment variable setting _NO_DEBUG_HEAP=1.
This can be set either globally for the machine or for a specific instance of Visual Studio.
a. Globally you would set an environment variable through the Control Panel → System → Advanced system settings → Environment Variables, and there add the variable _NO_DEBUG_HEAP=1.
Note: This will have an affect on EVERY application you debug.
b. For an instance of Visual Studio you can open a command prompt, setting the environment variable _NO_DEBUG_HEAP=1 and then open visual studio from inside that command prompt. This will influence only the processes created from
that instance of Visual Studio will inherit the environment
variable.
Append the behavior of the debugger, this is possible for VS2015. There are 2 ways to override this:
a. To modify for a specific project, go to the project properties Configuration Properties → Debugging and change the Environment property _NO_DEBUG_HEAP to 1
b. To modify for every project in Visual Studio, go to Tools → Options → Debugging and check the option: “Enable Windows debug heap allocator (Native only)”.
Note: f the _NO_DEBUG_HEAP environment variable mentioned in the 'a' is set in a project level it will override this global setting.
For me the difference in performance between debug mode and release mode is about a factor 40. After some digging, it appears several things contribute to the difference in performance, but there is one compiler option that quadruples my debug performance almost for free.
Namely, changing /ZI into /Zi. For a description, see the MSDN page.
I don't use the edit and continue feature anyway.
No one has mentioned closing unused source windows.
After closing 20+ unused windows, debug source stepping went from ~5s back down to ~.2s. This unusually slow project loads a DLL dynamically and that DLL was also the one being stepped through (and having source windows open) so it seems likely related. However this was C# (headline and tags are non-specific).
There are several things that are different if you run the debug build outside the IDE. One is that the IDE takes a while to load symbols, and if you depend on a lot of libraries then that startup time can be significant.
If you are using a symbol server (including the Microsoft public symbol server) then this may add to the startup time, so ensure you have a local symbol cache in your _NT_SYMBOL_PATH variable if that is the case.
Also the IDE runs with the debug heap enabled, but I don't think this happens if you run outside the IDE.
Debugging Visual C++ comes with lots and lots of overhead, especially in the STL. Try not defining _DEBUG, and define NDEBUG.
If my C++ app crashes on Windows I want to send useful debugging information to our server.
On Linux I would use the GNU backtrace() function - is there an equivalent for Windows?
Is there a way to extract useful debugging information after a program has crashed? Or only from within the process?
(Advice along the lines of "test you app so it doesn't crash" is not helpful! - all non-trivial programs will have bugs)
The function Stackwalk64 can be used to snap a stack trace on Windows.
If you intend to use this function, you should be sure to compile your code with FPO disabled - without symbols, StackWalk64 won't be able to properly walk FPO'd frames.
You can get some code running in process at the time of the crash via a top-level __try/__except block by calling SetUnhandledExceptionFilter. This is a bit unreliable since it requires you to have code running inside a crashed process.
Alternatively, you can just the built-in Windows Error Reporting to collect crash data. This is more reliable, since it doesn't require you to add code running inside the compromised, crashed process. The only cost is to get a code-signing certificate, since you must submit a signed binary to the service. https://sysdev.microsoft.com/en-US/Hardware/signup/ has more details.
You can use the Windows API call MiniDumpWriteDump if you wish to roll your own code. Both Windows XP and Vist automate this process and you can sign up at https://winqual.microsoft.com to gain access to the error reports.
Also check out http://kb.mozillazine.org/Breakpad and http://www.codeproject.com/KB/debug/crash_report.aspx for other solutions.
This website provides quite a detailed overview of stack retrieval on Win32 after a C++ exception:
http://www.eptacom.net/pubblicazioni/pub_eng/except.html
Of course, this will only work from within the process, so if the process gets terminated or crashes to the point where it terminates before that code is run, it won't work.
Generate a minidump file. You can then load it up in windbg or Visual Studio and inspect the entire stack where the crash occurred.
Here's a good place to start reading.
Its quite simple to dump the current stackframe addresses into a log file. All you have to do is get such a function called on program faults (i.e. a interrupt handler in Windows) or asserts. This can be done at released versions as well. The log file then can be matched with a map file resulting in a call stack with function names.
I published a article about this some years ago.
See http://www.ddj.com/architect/185300443
Let me describe how I handle crashes in my C++/WTL application.
First, in the main function, I call _set_se_translator, and pass in a function that will throw a C++ exception instead of using structured windows exceptions. This function gets an error code, for which you can get a Windows error message via FormatMessage, and a PEXCEPTION_POINTERS argument, which you can use to write a minidump (code here). You can also check the exception code for certain "meltdown" errors that you should just bail from, like EXCEPTION_NONCONTINUABLE_EXCEPTION or EXCEPTION_STACK_OVERFLOW :) (If it's recoverable, I prompt the user to email me this minidump file.)
The minidump file itself can be opened in Visual Studio like a normal project, and providing you've created a .pdb file for your executable, you can run the project and it'll jump to the exact location of the crash, together with the call stack and registers, which can be examined from the debugger.
If you want to grab a callstack (plus other good info) for a runtime crash, on a release build even on site, then you need to set up Dr Watson (run DrWtsn32.exe). If you check the 'generate crash dumps' option, when an app crashes, it'll write a mini dump file to the path specified (called user.dmp).
You can take this, combine it with the symbols you created when you built your server (set this in your compiler/linker to generate pdb files - keep these safe at home, you use them to match the dump so they can work out the source where the crash occurred)
Get yourself windbg, open it and use the menu option to 'load crash dump'. Once it's loaded everything you can type '~#kp' to get a callstack for every thread (or click the button at the top for the current thread).
There's good articles to know how to do this all over the web, This one is my favourite, and you'll want to read this to get an understanding of how to helpyourself manage the symbols really easily.
You will have to set up a dump generation framework in your application, here is how you may do it.
You may then upload the dump file to the server for further analysis using dump analyzers like windbg.
You may want to use adplus to capture the crash callstack.
You can download and install Debugging tools for Windows.
Usage of adplus is mentioned here:
Adplus usage
This creates the complete crash or hang dump. Once you have the dump, Windbg comes to the rescue. Map the correct pdbs and symbols and you are all set to analyze the dump. To start with use the command "!analyze -v"