I am building the debug version of my app, with full symbols. I set a breakpoint on the following line:
throw std::range_error( "invalid utf32" );
When the breakpoint hits, my stack looks normal. I can see all my routines. But if I run, and let the exception get thrown, I see a worthless stack. it has MyApp.exe!_threadstartex() towards the bottom, a few disabled entries labeled kernel32.dll, and the line "Frames below may be incorrect and/or missing" etc.
This really sucks! Because very often I will get an exception in my debug build, and this $5000 development environment is not even showing me my own stack! I am statically linking with everything so its not a DLL problem.
Help!
The problem is that you don't have any debug symbols loaded for the location the debugger stopped at. This means the rest of the call stack is nonsense.
Fortunately the solution is easy: Starting at the top of the call stack window, right-click on each greyed-out entry (e.g. KernelBase.dll!...) and select Load Symbols From / Microsoft Symbol Servers. After doing this for one or two entries your true call stack will be revealed.
I bet the problem is that you don't have "break on exception" enabled. I had this same problem, and couldn't believe I couldn't find useful answers to this...how did my vs2008 get configured differently? I don't know. But one day my call stacks just became useless when a debugged program crashed. Finally I went and checked all those classes of exceptions to get the program to "break on exception", and now it breaks right when the exception is thrown (e.g. bad access) and I have a useful call stack to debug with. Yay!
I think you mix up smth. here.
If you catch the exception in some catch statement or it is propagated until main your stack was unwound and you can not expect VC++ to remember the entire stack.
For example in Java stack trace is part of the exception itself. Dependent on you compiler you can write an exception class which records the stack trace if it is constructed (but not copy constructed) and carries the information. When the class is caught you can evaluate the info. If you program using MFC take a look at AfxDumpStack.
Hope that helps,
Ovanes
P.S: This DDJ article might be helpful to you: C++ Stack Traces
Probably you are watching the callstack of the wrong thread.
Go to the Tread Panel, in Debug->Windows->Threads, and then select the correct thread.
Related
I have been working on a single thread C++ program using visual studio. I have been running the "debug" version and normally If I come across some kind of crash the debugger helpfully points me to the bad line in my code. Sometimes the crash happens within a C++ library function in which case I look over to the "Call Stack" window and can quickly spot the last call from my code which called the library function.
But now I am struggling with a "Exception thrown" problem where the crash occurs in "strcat.asm", but the Call Stack does not display any code I recognise. In the call stack window I can see things like "ntdll.dll!__RtlUserThreadstart#8() - whatever that means!
How can I find what part of my code led to this crash?
UPDATE: after much pain, I finally found that the bug was essentially
value[idx] = a;
where idx had taken on the value -1. This, no doubt, had corrupted the stack.
How can I specify a separate function which will be called automatically at Run-Time error to prevent program crash?
Best thing as mentioned already is to identify the area where the crash ic occuring and then fix the piece of code. This is the idealistic approach.
In case you are unable to find that out another alternative is to do structured exception handling in the areas where you suspect crashes to occur. Once the crash occurs you capture what ever data you want and process it. Meanwhile you can change the settings in windows service manager to restart your application whenever it crashes. Hope this answers your question.
Also in case you are looking for methods to capture the crash and analyse debugdiag and windbg are some of the standard tools that people use to take the crash dumps.
If your in Windows you need to write your own custom runtime check handler. Use:
_RTC_SetErrorFunc
to install your custom function in place of
_CrtDbgReport.
Here is a good article on how to do it:
http://msdn.microsoft.com/en-us/library/40ky6s47(VS.71).aspx
If by run-time error you mean an unhandled exception, don't do it. If an unhanded exception propagates through your code, you want your app to crash. Maybe create a dumpfile on the way down, sure. But the last thing you want to do is catch the exception, do nothing, and continue running as if nothing had happened in the first place.
When an exception is generated, whatever caused the problem could have had other effects. Like blowing the stack or corrupting the heap. If you were to silently ignore these exceptions and continue running anyway, your app might run for a while and seem OK, but something underneath is unstable. Memory could be corrupt, resources might not be available, who knows. You will eventually crash. Bu ignoring unhandled exceptions and running anyway, all you do is delay the inevitable, and make it that much more difficult to diagnose the real problem, because when you do crash, the stack will be in a totally different and probably unrelated place from what caused the initial problem.
currently I used SetUnhandledExceptionFilter() to provide callback to get information when an unhandled exception was occurred, that callback will provides me with EXCEPTION_RECORD which provides ExceptionAddress.
[1]what is actually ExceptionAddress is? does it the address of function / code that gives exception, or the memory address that some function tried to access?
[2]is there any better mechanism that could give me better information when unhandled exception occured? (I can't use debug mode or add any code that affect runtime performance, since crash is rare and only on release build when code run as fast as possible)
[3]is there any way for me to get several callstack address when unhandled exception occured.
[4]suppose ExceptionAddress has address A, and I have DLL X loaded and executed at base address A-x, and some other DLL Y at A+y, is it good to assume that crash was PROBABLY caused by code on DLL X?
(1) The ExceptionAddress is the address of the code that caused the exception. In case of an access violation error (0xC0000005) one of the additional arguments of the exception record holds the address from which a read or a write was attempted and another argument tells if it was a read or a write. This is documented in the link you provide in the question.
(2) no. additionally, adding debug information to a release build doesn't affect performance. You can verify this and see for yourself.
(3) dbghelp.dll provides a complete library to investigate crashes. among athers there is StackWalk64 which allows you to get the complete stack of the crash.
(4) calling GetModuleHandleEx with the ExceptionAddress as an argument will get you the handle of the dll where the offending code resides. As to the question of which DLL CAUSED the crash, this depends on your definition of "CAUSED". A crash which occur in one dll can be the result of a bug in a completely different and unrelated dll.
Not direct answer to your question, but possibly this is what you need:
http://www.codeproject.com/KB/debug/postmortemdebug_standalone1.aspx
Post-mortem debugging is the way to find exception place when the program runs in Release build on a client computer.
Also, not a direct answer to your question, but I think it could help you:
http://www.codeproject.com/KB/applications/blackbox.aspx
It will produce a readable output to screen or file, which will give you a stack output to find the place where the exception occured along with other useful information.
It worked good for me.
There is also an improved version named "Blackbox revised". Can't find the website right now, though.
Background
I have an application with a Poof-Crash[1]. I'm fairly certain it is due to a blown stack.
The application is Multi-Threaded.
I am compiling with "Enable C++ Exceptions: Yes With SEH Exceptions (/EHa)".
I have written an SE Translator function and called _set_se_translator() with it.
I have written functions for and setup set_terminate() and set_unexpected().
To get the Stack Overflow, I must run in release mode, under heavy load, for several days. Running under a debugger is not an option as the application can't perform fast enough to achieve the runtime necessary to see the issue.
I can simulate the issue by adding infinite recursion on execution of one of the functions, and thus test the catching of the EXCEPTION_STACK_OVERFLOW exception.
I have WinDBG setup as the crash dump program, and get good information for all other crash issues but not this one. The crash dump will only contain one thread, which is 'Sleep()'ing. All other threads have exited.
The Question
None of the things I've tried has resulted in picking up the EXCEPTION_STACK_OVERFLOW exception.
Does anyone know how to guarantee getting a a chance at this exception during runtime in release mode?
Definitions
Poof-Crash: The application crashes by going "poof" and disappearing without a trace.
(Considering the name of this site, I'm kind of surprised this question isn't on here already!)
Notes
An answer was posted briefly about adjusting the stack size to potentially force the issue sooner and allow catching it with a debugger. That is a clever thought, but unfortunately, I don't believe it would help. The issue is likely caused by a corner case leading to infinite recursion. Shortening the stack would not expose the issue any sooner and would likely cause an unrelated crash in validly deep code. Nice idea though, and thanks for posting it, even if you did remove it.
Everything prior to windows xp would not (or would be harder) generally be able to trap stack overflows. With the advent of xp, you can set vectored exception handler that gets a chance at stack overflow prior to any stack-based (structured exception) handlers (this is being the very reason - structured exception handlers are stack-based).
But there's really not much you can do even if you're able to trap such an exception.
In his blog, cbrumme (sorry, do not have his/her real name) discusses a stack page neighboring the guard page (the one, that generates the stack overflow) that can potentially be used for backout. If you can squeeze your backout code to use just one stack page - you can free as much as your logic allows. Otherwise, the application is pretty much dead upon encountering stack overflow. The only other reasonable thing to do, having trapped it, is to write a dump file for later debugging.
Hope, it helps.
I'm not convinced that you're on the right track in diagnosing this as a stack overflow.
But in any case, the fact that you're getting a poof!, plus what you're seeing in WinDbg
The crash dump will only contain one thread, which is 'Sleep()'ing. All other threads have exited.
suggests to me that somebody has called the C RTL exit() function, or possibly called the Windows API TerminateProcess() directly. That could have something to do with your interrupt handlers or not. Maybe something in the exception handling logic has a re-entrance check and arbitrarily decides to exit() if it's reentered.
My suggestion is to patch your executables to put maybe an INT 3 debug at the entry point to exit (), if it's statically linked, or if it's dynamically linked, patch up the import and also patch up any imports of kernel32::TerminateProcess to throw a DebugBreak() instead.
Of course, exit() and/or TerminateProcess() may be called on a normal shutdown, too, so you'll have to filter out the false alarms, but if you can get the call stack for the case where it's just about to go proof, you should have what you need.
EDIT ADD: Just simply writing your own version of exit() and linking it in instead of the CRTL version might do the trick.
I remember code from a previous workplace that sounded similar having explicit bounds checks on the stack pointer and throwing an exception manually.
It's been a while since I've touched C++ though, and even when I did touch it I didn't know what I was doing, so caveat implementor about portability/reliability of said advice.
Have you considered ADPlus from Debugging Tools for Windows?
ADPlus attaches the CDB debugger to a process in "crash" mode and will generate crash dumps for most exceptions the process generates. Basically, you run "ADPlus -crash -p yourPIDhere", it performs an invasive attach and begins logging.
Given your comment above about running under a debugger, I just wanted to add that CDB adds virtually zero overhead in -crash mode on a decent (dual-core, 2GB RAM) machine, so don't let that hold you back from trying it.
You can generate debugging symbols without disabling optimizations. In fact, you should be doing that anyways. It just makes debugging harder.
And the documentation for _set_se_translator says that each thread has its own SE translator. Are you setting one for each thread?
set_unexpected is probably a no-op, at least according to the VS 2005 documentation. And each thread also has its own terminate handler, so you should install that per thread as well.
I would also strongly recommend NOT using SE translation. It takes hardware exceptions that you shouldn't ignore (i.e., you should really log an error and terminate) and turns them into something you can ignore (C++ exceptions). If you want to catch this kind of error, use a __try/__except handler.
Please help! I'm really at my wits' end.
My program is a little personal notes manager (google for "cintanotes").
On some computers (and of course I own none of them) it crashes with an unhandled exception just after start.
Nothing special about these computers could be said, except that they tend to have AMD CPUs.
Environment: Windows XP, Visual C++ 2005/2008, raw WinApi.
Here is what is certain about this "Heisenbug":
1) The crash happens only in the Release version.
2) The crash goes away as soon as I remove all GDI-related stuff.
3) BoundChecker has no complains.
4) Writing a log shows that the crash happens on a declaration of a local int variable! How could that be? Memory corruption?
Any ideas would be greatly appreciated!
UPDATE: I've managed to get the app debugged on a "faulty" PC. The results:
"Unhandled exception at 0x0044a26a in CintaNotes.exe: 0xC000001D: Illegal Instruction."
and code breaks on
0044A26A cvtsi2sd xmm1,dword ptr [esp+14h]
So it seems that the problem was in the "Code Generation/Enable Enhanced Instruction Set" compiler option. It was set to "/arch:SSE2" and was crashing on the machines that didn't support SSE2. I've set this option to "Not Set" and the bug is gone. Phew!
Thank you all very much for help!!
4) Writig a log shows that the crash happen on a declaration of a local int variable! how could that be? Memory corruption?
What is the underlying code in the executable / assembly? Declaration of int is no code at all, and as such cannot crash. Do you initialize the int somehow?
To see the code where the crash happened you should perform what is called a postmortem analysis.
Windows Error Reporting
If you want to analyse the crash, you should get a crash dump. One option for this is to register for Windows Error Reporting - requires some money (you need a digital code signing ID) and some form filling. For more visit https://winqual.microsoft.com/ .
Get the crash dump intended for WER directly from the customer
Another option is to get in touch witch some user who is experiencing the crash and get a crash dump intended for WER from him directly. The user can do this when he clicks on the Technical details before sending the crash to Microsoft - the crash dump file location can be checked there.
Your own minidump
Another option is to register your own exception handler, handle the exception and write a minidump anywhere you wish. Detailed description can be found at Code Project Post-Mortem Debugging Your Application with Minidumps and Visual Studio .NET article.
So it doesnnt crash when configuration is DEBUG Configuration? There are many things different than a RELEASE configruation:
1.) Initialization of globals
2.) Actual machine Code generated etc..
So first step is find out what are exact settings for each parameter in the RELEASE mode as compared to the DEBUG mode.
-AD
1) The crash happens only in the Release version.
That's usually a sign that you're relying on some behaviour that's not guaranteed, but happens to be true in the debug build. For example, if you forget to initialize your variables, or access an array out of bounds. Make sure you've turned on all the compiler checks (/RTCsuc). Also check things like relying on the order of evaluation of function parameters (which isn't guaranteed).
2) The crash goes away as soon as I remove all GDI-related stuff.
Maybe that's a hint that you're doing something wrong with the GDI related stuff? Are you using HANDLEs after they've been freed, for example?
Download the Debugging tools for Windows package. Set the symbol paths correctly, then run your application under WinDbg. At some point, it will break with an Access Violation. Then you should run the command "!analyze -v", which is quite smart and should give you a hint on whats going wrong.
Most heisenbugs / release-only bugs are due to either flow of control that depends on reads from uninitialised memory / stale pointers / past end of buffers, or race conditions, or both.
Try overriding your allocators so they zero out memory when allocating. Does the problem go away (or become more reproducible?)
Writig a log shows that the crash happens on a declaration of a local int variable! How could that be? Memory corruption?
Stack overflow! ;)
4) Writig a log shows that the crash happen on a declaration of a local int variable!how could that be? Memory corruption
I've found the cause to numerous "strange crashes" to be dereferencing of a broken this inside a member function of said object.
What does the crash say ? Access violation ? Exception ? That would be the further clue to solve this with
Ensure you have no preceeding memory corruptions using PageHeap.exe
Ensure you have no stack overflow (CBig array[1000000])
Ensure that you have no un-initialized memory.
Further you can run the release version also inside the debugger, once you generate debug symbols (not the same as creating debug version) for the process. Step through and see if you are getting any warnings in the debugger trace window.
"4) Writing a log shows that the crash happens on a declaration of a local int variable! How could that be? Memory corruption?"
This could be a sign that the hardware is in fact faulty or being pushed too hard. Find out if they've overclocked their computer.
When I get this type of thing, i try running the code through gimpels PC-Lint (static code analysis) as it checks different classes of errors to BoundsChecker. If you are using Boundschecker, turn on the memory poisoning options.
You mention AMD CPUs. Have you investigated whether there is a similar graphics card / driver version and / or configuration in place on the machines that crash? Does it always crash on these machines or just occasionally? Maybe run the System Information tool on these machines and see what they have in common,
Sounds like stack corruption to me. My favorite tool to track those down is IDA Pro. Of course you don't have that access to the user's machine.
Some memory checkers have a hard time catching stack corruption ( if it indeed that ). The surest way to get those I think is runtime analysis.
This can also be due to corruption in an exception path, even if the exception was handled. Do you debug with 'catch first-chance exceptions' turned on? You should as long as you can. It does get annoying after a while in many cases.
Can you send those users a checked version of your application? Check out Minidump Handle that exception and write out a dump. Then use WinDbg to debug on your end.
Another method is writing very detailed logs. Create a "Log every single action" option, and ask the user to turn that on and send it too you. Dump out memory to the logs. Check out '_CrtDbgReport()' on MSDN.
Good Luck!
EDIT:
Responding to your comment: An error on a local variable declaration is not surprising to me. I've seen this a lot. It's usually due to a corrupted stack.
Some variable on the stack may be running over it's boundaries for example. All hell breaks loose after that. Then stack variable declarations throw random memory errors, virtual tables get corrupted, etc.
Anytime I've seen those for a prolong period of time, I've had to go to IDA Pro. Detailed runtime disassembly debugging is the only thing I know that really gets those reliably.
Many developers use WinDbg for this kind of analysis. That's why I also suggested Minidump.
Try Rational (IBM) PurifyPlus. It catches a lot of errors that BoundsChecker doesn't.