I'm writing a plugin (basically a dll) for a 3D application and occasionally there are crashes. Sometimes these are very difficult to find and I wanted to invest some time into making (or integrating an existing) crash logger that will
Give me a stack trace.
Give me a list of local variables.
Dump these items to a file, or upload it to a given URL.
So far I've looked at Google breakpad, but have no idea how to integrate it and the documentation seems poor at best. I've tried to use it and managed to get as far as building it on windows, but some unit tests fail and there is no help about what to do at that point. Also, it may be a bit excessive for my needs.
I've found the following site which details how to get a stack trace GENERATE STACK TRACES ON CRASH PORTABLY IN C++. But I'm not sure if this will work on a remote system. I'm guessing this will need to be the debug version and be supplied the pdb file for this to work?
As for getting local variables, I've not managed to find anything yet.
Does anyone know of some resources to help?
this article, though written in 2002, is still relevant to post-mortem debugging. It shows you all the reasons and steps and design required to get it working.
Nowadays, its a little easier (though I liked Windbg!) you get your app to call SetUnhandledExceptionFilter and write the .dmp file, then just double-click it to load it into Visual Studio. You will need good symbols (.pdf files) on the debugging system to make sense of the dump, but create your own symbol server (instructions in the article, its dead easy) and it should be able to figure out which symbols are needed for any app. You have to be disciplined about saving the symbols though - wrong symbols are worse than useless.
This question is rather old, but regarding exceptions I want to note that you can get nice backtraces in a cross-platform way, using only standard C++11 and without the need for a debugger or cumbersome logging:
Use std::nested_exception and std::throw_with_nested
It is described on StackOverflow here and here, how you can get a backtrace on your exceptions inside your code by simply writing a proper exception handler which will rethrow nested exceptions.
It will, however, require that you insert try/catch statements at the functions you wish to trace.
Since you can do this with any derived exception class, you can add a lot of information to such a backtrace!
You may also take a look at my MWE on GitHub or my "trace" library, where a backtrace would look something like this:
Library API: Exception caught in function 'api_function'
Backtrace:
~/Git/mwe-cpp-exception/src/detail/Library.cpp:17 : library_function failed
~/Git/mwe-cpp-exception/src/detail/Library.cpp:13 : could not open file "nonexistent.txt"
Regarding local variables, in this approach you would have to manually place those you wish to print into your exception or exception message in some form, which would be quite cumbersome...
Writing whatever you logged to a file and uploading it somewhere is a different problem for which I don't have any specific suggestions.
Related
I am a java developer trying my hands on C-C++ code.
Basically I have some build script that uses Visual Studio components to build the application libraries - dlls. We do not use Visual Studio IDE to debug.
But whenever we face some crash in our application, we have to enter the debug statements line by line and need to check the exact line of code in which it is crashing.
Is there any API in C/C++ which would write the stack trace into a file during the crash of our program?
Some kind of event listener that would be called during a program exit and that could print the stack trace and error information into a log file.
I have seen many questions related to this but I am not able to get how to handle this in code rather than debugging tools. I feel Java is very much advanced with respect to error handling.
Thanks in advance!
The Standard does not give you any facilites for this. That said, you can use OS-specific APIs to get what you want. On windows, the easiest is probably to use StackWalker. You do need to take care of SEH oddities yourself. I suggest this article for more information.
On POSIX platforms, you can use the backtrace function from glibc.
In general, both require your program to be compiled with debug information. It is also possible to create crash dumps without symbol names (see Minidumps on Windows), but you will need to decode them (which Visual Studio does for you for example, but you mentioned in your post that you don't use it).
Also, keep in mind that writing crash dumps from a crashing process is very error-prone (since you really have no idea what state your application is at that time). You might get more reliable results if you write the crash dumps from another process. Earlier I've put together a simple example out-of-process crash handler that demonstrates how to implement that.
You can use std::set_terminate to define a function to be called when an exception leaves main() additionally you can install a signal handler for SIGSEGV to catch segmentation faults.
You can get a stack trace (on Linux) using libunwind ( http://www.nongnu.org/libunwind/ ) or use the backtrace() function from libc.
One thing I sometimes do in my own exception classes is to grab and store a stacktrace in the constructor that I can then obtain where I catch the exception and print the trace of where it was thrown from.
You could use <errno.h> in C to handle your error and log them into a log file. The standard flux for the errors is stderr.
One solution to get the error is to use the function perror included in <errno.h> that will display the exact error.
To use it, put it in an error catcher.
if (!fileIsOpen())
perror("File not opened");
I hope it helped.
First of all, thank you for taking the time to view my question and help. I noticed that a lot of questioners here show little or no appreciation, but I'm sincerely appreciative for the help and the community here :)
I wrote a C++ plugin (compromised of hundreds of source files) for an application I do not have the source code for (it's a video game). In other words, I only have the source code for my plugin, but not the game. Now, somewhere in those thousands of lines in my plugin, something causes the game engine to throw (probably an access violation) and I don't know where. By the time the debugger breaks, the stack is corrupted and all I get are hex addresses for DLLs I do not have the source for (but the exception occurs in my DLL for sure). I tried everything... I just can't seem to find where the exception occurs. Sometimes the debugger points to a "memory relocation" function (which I never used in my plugin), sometimes it points to the engine's GameFrame(), and other times it points to a damage callback (all these are just different member functions of a class).
I tried practically everything... I googled for hours trying to find out how to use other debuggers like WinDbg and Microsoft Application Verifier. I tried to comment out one or the other, or both, where the debugger points, but it still crashes. I even inserted OUTPUT("The name of the last executed function is: %s", __FUNCTION__) into EVERY function in my application hoping to painstakingly catch the last function but it seems any kind of I/O prevents the exception from occurring for some reason... And 10 minutes of debugging and the crash happens at some random last executed function.
I can't find out where this access violation is happening or where some temporary object is removed to cause these bad pointers (I check every pointer before using it), but damn, I'm reaching my limit's end here.
So, how does one debug the impossible... a random crash with a crappy debugger call stack? Thanks in advance for your patient and kind help!
My suggestion: try different debuggers (non MS), they catch different things.
My experience: a program I have source code and full debugging symbols corrupt the stack, VS nor WinDbg can help but Ollydbg comments a non-string var with the value "r for pattern.", so I had overwrote some string buffer onto this var. Also Ollydbg have option to walk the stack the hard way (not using dbghelp.dll)
From my experience, the old adage "Prevention is better than cure" is very relevant. It is best to prevent the bugs from creeping in, by following good software development practices (unit tests, regressions, code review, etc.) than to work it out later once the bugs show up.
Of course, real world is not perfect, and bugs do show up. To debug memory corruption, you have some nice tools like valgrind, which at least narrow down the problem sections for you to take a closer look at. Debugging a complex program is not easy, and if your debugger throws up, it requires a lot of persistence on your part. One technique I find useful is to selectively enable or disable certain modules, to narrow down the module has the problem.
Sometimes you need to use "referential transparency" to unload some modules. To give you a stripped down example, consider:
int foo = factorial(3);
If I suspect there's a problem in this code (and the debugger crashes before I can see the call stack), I have to try by removing this code, and see if the problem persists. However, foo may be used later, so I cannot just remove it. Instead I can replace it with int foo = 6; and continue.
Another important point is to always maintain a trace file, where your code keeps logging what it is doing. When a program crashes, the trace file can often help narrow down the problem. Of course, you disable the tracing by default, so that it doesn't cause a performance bottleneck.
We have an application that is a C/C++/MFC desktop application, with some C++/CLI assemblies that allow us to access some managed code functionality. The app is crashing at startup in release mode only with the message
An unhandled exception of type 'System.TypeInitializationException' occurred in Unknown Module.
Additional information: The type initializer for '' threw an exception.
How can I go about debugging this scenario and what are the issues in mixing managed/unmanaged code? What special steps do I have to take to make them play nicely?
Things to suspect are:
Missing unmanaged DLLs. You can use Depends (from Sysinternals) and start profiling, but I've struggled to get good results in mixed-mode.
Make a native minimal test harness which has the same dependencies, and run that through Depends - you will get definitive information about missing DLLs.
Are you using obfuscation in your release build product?
The obfuscation software we use adds a field to types in evaluation mode. We have fixed offset structs, yet the new field doesn't get an explicit offset. This is an error which would be flagged up at compile time if it was in our own code. Since the obfuscator is dynamically patching the assembly, the CLR has no option but to fail to load the invalid type at runtime.
In my opinion (based on some trouble I have had) Matt Smith's comment is the most useful answer; actually, check 'Thrown' for all types of exception. Quite often the problem is a crash in the constructor of a global object. See also answer 5 at
http://www.codeproject.com/Questions/67419/The-type-initializer-for-threw-an-exception
which says (among other things):
Click Debug--> Exceptions and check ON all the Thrown checkboxes. This
will cause the debugger to stop on all first chance exceptions and
will help you find the error under the Type Initializer error that
you're seeing. If it is related to another assembly, as mine was, you
can use Microsoft's Assembly Binding Log Viewer tool to help determine
the problem.
Today, in my C++ multi-platform code, I have a try-catch around every function. In every catch block I add the current function's name to the exception and throw it again, so that in the upmost catch block (where I finally print the exception's details) I have the complete call stack, which helps me to trace the exception's cause.
Is it a good practice, or are there better ways to get the call stack for the exception?
What you are doing is not good practice. Here's why:
1. It's unnecessary.
If you compile your project in debug mode so that debugging information gets generated, you can easily get backtraces for exception handling in a debugger such as GDB.
2. It's cumbersome.
This is something you have to remember to add to each and every function. If you happen to miss a function, that could cause a great deal of confusion, especially if that were the function that caused the exception. And anyone looking at your code would have to realize what you are doing. Also, I bet you used something like __FUNC__ or __FUNCTION__ or __PRETTY_FUNCTION__, which sadly to say are all non-standard (there is no standard way in C++ to get the name of the function).
3. It's slow.
Exception propagation in C++ is already fairly slow, and adding this logic will only make the codepath slower. This is not an issue if you are using macros to catch and rethrow, where you can easily elide the catch and rethrow in release versions of your code. Otherwise, performance could be a problem.
Good practice
While it may not be good practice to catch and rethrow in each and every function to build up a stack trace, it is good practice to attach the file name, line number, and function name at which the exception was originally thrown. If you use boost::exception with BOOST_THROW_EXCEPTION, you will get this behavior for free. It's also good to attach explanatory information to your exception that will assist in debugging and handling the exception. That said, all of this should occur at the time the exception is constructed; once it is constructed, it should be allowed to propagate to its handler... you shouldn't repeatedly catch and rethrow more than stricly necessary. If you need to catch and rethrow in a particular function to attach some crucial information, that's fine, but catching all exceptions in every function and for the purposes of attaching already available information is just too much.
No, it is deeply horrible, and I don't see why you need a call stack in the exception itself - I find the exception reason, the line number and the filename of the code where the initial exception occurred quite sufficient.
Having said that, if you really must have a stack trace, the thing to do is to generate the call stack info ONCE at the exception throw site. There is no single portable way of doing this, but using something like http://stacktrace.sourceforge.net/ combined with and a similar library for VC++ should not be too difficult.
One solution which may be more graceful is to build a Tracer macro/class. So at the top of each function, you write something like:
TRACE()
and the macro looks something like:
Tracer t(__FUNCTION__);
and the class Tracer adds the function name to a global stack on construction, and removes itself upon destruction. Then that stack is always available to logging or debugging, maintenance is much simpler (one line), and it doesn't incur exception overhead.
Examples of implementations include things like http://www.drdobbs.com/184405270, http://www.codeproject.com/KB/cpp/cmtrace.aspx, and http://www.codeguru.com/cpp/v-s/debug/tracing/article.php/c4429. Also Linux functions like this http://www.linuxjournal.com/article/6391 can do it more natively, as described by this Stack Overflow question: How to generate a stacktrace when my gcc C++ app crashes. ACE's ACE_Stack_Trace may be worth looking at too.
Regardless, the exception-handling method is crude, inflexible, and computationally expensive. Class-construction/macro solutions are much faster and can be compiled out for release builds if desired.
The answer to all your problems is a good debugger, usually http://www.gnu.org/software/gdb/ on linux or Visual Studio on Windows. They can give you stack traces on demand at any point in the program.
Your current method is a real performance and maintenance headache. Debuggers are invented to accomplish your goal, but without the overhead.
There's a nice little project that gives a pretty stack trace:
https://github.com/bombela/backward-cpp
Look at this SO Question. This might be close to what you're looking for. It isn't cross-platform but the answer gives solutions for gcc and Visual Studio.
One more project for stack-trace support: ex_diag. There are no macros, cross-platform is present, no huge code needs, tool is fast, clear and easy in use.
Here you need only wrap objects, which are need to trace, and they will be traced if exception occurs.
Linking with the libcsdbg library (see https://stackoverflow.com/a/18959030/364818 for original answer) looks like the cleanest way of getting a stack trace without modifying your source code or 3rd party source code (ie STL).
This uses the compiler to instrument the actual stack collection, which is really want you want to do.
I haven't used it and it is GPL tainted, but it looks like the right idea.
While quite a few counter-arguments have been made in the answers here, I want to note that since this question was asked, with C++11, methods have been added which allow you to get nice backtraces in a cross-platform way and without the need for a debugger or cumbersome logging:
Use std::nested_exception and std::throw_with_nested
It is described on StackOverflow here and here, how you can get a backtrace on your exceptions inside your code by simply writing a proper exception handler which will rethrow nested exceptions.
It will, however, require that you insert try/catch statements at the functions you wish to trace.
Since you can do this with any derived exception class, you can add a lot of information to such a backtrace!
You may also take a look at my MWE on GitHub or my "trace" library, where a backtrace would look something like this:
Library API: Exception caught in function 'api_function'
Backtrace:
~/Git/mwe-cpp-exception/src/detail/Library.cpp:17 : library_function failed
~/Git/mwe-cpp-exception/src/detail/Library.cpp:13 : could not open file "nonexistent.txt"
An exception that isn't handled is left for the calling function to handle. That continues until the exception is handled. This happens with or without try/catch around a function call. In other words, if a function is called that isn't in a try block, an exception that happens in that function will automatically be passed up to call stack. So, all you need to do is put the top-most function in a try block and handle the exception "..." in the catch block. That exception will catch all exceptions. So, your top-most function will look something like
int main()
{
try
{
top_most_func()
}
catch(...)
{
// handle all exceptions here
}
}
If you want to have specific code blocks for certain exceptions, you can do that too. Just make sure those occur before the "..." exception catch block.
Is there a way catch an exception like access violation and get information about on which line an exception occurred? This would be very good for debugging purposes, especially for testers..
My environment is Windows with VC++ on VS2008
An access violation is not an exception in C++ terms, so the answer is in general "no". Concieveably, your specific implementation might have a feature that turns access violations into C++ exceptions - you need to specify what compiler & platform you are using.
In case you really want to log info about C++ exceptions (AV is not one) thrown from your code, you can use macros __FILE__ and __LINE__ within constructors of your exception types.
If you catch the SEH exception using a __catch handler you can then access the thread context at the time of the exception and then use StackWalk64 to dump the call stack at the point where the exception was generated. Note that as I mentioned here: Printing the stack trace in C++ (MSVC)? StackWalker by Jochen Kalmbach [MVP VC++] and available on codeproject is probably the easiest way to do this. It wraps up all of the details of dealing with the underlying StackWalk64 API.
Alternatively you could the same solution that I proposed here: How to catch divide-by-zero error in Visual Studio 2008 C++? to convert the Windows Structured Exceptions into C++ exceptions and capture the stack trace as shown above at the point where you translate the exception. This would give you a C++ exception with a stack trace like you get in C# or Java.
I'm not sure why testers would need to know which line an exception occurred.
The developers might want to know, though. But a better approach is the following:
Include the information about class and method with each PROGRAM exception. These are exceptions that should NOT have happened.
This should be output by whatever logs your exceptions. Your program does catch and log every exception, doesn't it? If not, it should.
Make sure your methods are small enough that the above is enough information to easy track down a bug. If you need line information as well, then your methods are too large and your exceptions are not specific enough.
In MSVC you can set the debugger to break when any exception or things like an access violation happen buy going to debug->Exceptions and checking the appropriate box.
Note that access violations are not C++ exceptions and will be handled differently on different systems. In Windows you can trap them with a structured exception handler. In unixy systems it will usually core dump. You can usually get a stacktrace from that.
For msdev / Windows, during development, always have your linker generate a MAP file.
When Windows throws up an access violation, and it's your code (not library), you can use the address to match up function / data in the MAP file and get within a few lines of the offender. At a minimum, you'll know the function/method.
That's really a compiler question.
If you want to know if it is possible for a compiler to provide, the answer is yes. I know multiple Ada compilers that provide tracebacks for unhandled exceptions, so it is clearly possible. This includes the gcc-based Gnat, so if the C++ compiler uses any of the same facilities for its exceptions, the support for doing that should already be there.
On unix type systems a Access violation generates a SEGV or BUS signal. This normally causes the application to core dump. You could always write your own signal handler. From there you could probably generate a stack-dump using the glibc before core dumping.
A core dump generally give you all this for wasy analysis in gdb.
My answer refers to Windows only. You have quite a few options:
Without changing your code -
Avoid catching exceptions (at least ones you do not expect), and let your app just crash. Configure good ol' dr. watson (drwtsn32.exe) to create a crash dump, and open that dump in your debugger. You'll get the exact line of code there.
Use adPlus (from the windbg installation) to monitor your app's run, and create a dump when an exception is thrown.
From within your code, you can -
Easily walk the stack when a structured exception is thrown. See Jochen Kalmbach's stack walker for example.
Use this weird hack to walk the stack when a C++ exception is thrown.
Finally, quite a lot of questions have been asked here on this issue. Look around, and you'll get additional answers.
On one project I worked on, the root class of the exception hierarchy used captured a callstack in the constructor in case that information was needed later for debugging after the exception was caught. It was a decent idea, although if you're catching and 'handling' the exception, exactly when it was thrown shouldn't really matter all that much.
To put it another way, if you care about this info (context of "who" threw it), you probably shouldn't be using an exception to report this condition, or potentially you shouldn't be catching the exception in the first place. Uncaught exceptions cause crashes, giving you a crash dump at the point the exception was thrown.
On windows, the C runtime system function _set_se_translator() takes a simple static function with signature
void f(int, EXCEPTION_POINTERS*)
neither of which argument you totally need. In the body of f, throw your favorite exception. Call the function near the beginning of your program. The documentation is reasonable for microsoft.
You can do all manner of additional stuff with this function.
The google breakpad project has many good things. Among them are means to convert a crash address to file, line, and function name using build symbols and the debuginfo dll.