I'm building an application for Windows XP using the MinGW tool chain and it sometimes crashes unexpectedly. So, I'm trying to use a debugger (Gdb) but the program exits with code 03 before anything happens. In fact, all I see from GDB is:
[New thread 3184.0x7b8][New thread
3184.0xef8]
Program exited with code 03.
My suspicion is that there is some failed dynamic linking of a dependency (which are Qt, VTK, and ITK, all built with MinGW). However, this does not happen when I just run the program normally. Or if it happens, it appears to be intermittent and well after the program is launched and running. NOTE: I'm also using Cmake for cross compiling.
What should I do? What can I try?
Add a callback via signal(SIGABRT, <callback>) to catch the call to abort before it shuts down the process. If this happens before you hit main() you might have to resort to a static global and compiler trickery to catch it.
Code 3 is usually returned on a segfault.
Try switching to Linux and debugging the program with electric fence. It might give you some extra insight.
Related
Note: This is meant for windows, and I am using Visual Studio 2019 with C++17
I have been trying to make a dll that when injected into a program, can run some code before the program crashes. I have attempted using atexit, hooking onto exit using libMinHook, hooking onto abort using libMinHook, and hooking onto terminate using libMinHook, but it didn't work. I learned that when a program crashes it gets killed. Does anyone have any advice? (I've tried to find answers online but couldn't find any)
You have to build a debugging shell for your program and insert error exit code where you are likely to have malfunctions.
I normally use GDB (in Linux, with the Qt Creator debugger GUI) to debug. But right now I have a crash that refuses to ever happen when running under the debugger, yet happens easily when running outside of it.
How do I get a stack trace of my crash, in these circumstances?
A linux-specific solution is OK.
Note: I'm talking about running a debug build only, even when it's run outside the debugger.
The easiest way to be sure you can obtain a stacktrace after a crash is to run
ulimit -c unlimited
In your shell before starting the program.
This will ensure that the kernel is allowed to produce a "core dump" of unlimited size (for many distros the default size is 0) when a program crashes.
That core file can then be loaded into gdb as gdb programfile corefile and then the command thread apply all bt will give you stack traces for all threads for that specific crash (use just bt if you only care about the crashing thread).
You can also use the pstack program to get a stacktrace from a running program.
I have a huge code-base I am unfamiliar with, and the program terminates abnormally because a thread somewhere is calling __fastfail. This is based on the message, which ends with
... Fatal program exit requested.
The call stack has no symbols because it's inside C++ 2015 runtime (ucrtbase.dll). The call appears to be made on a thread other than my main thread. This mysterious thread only starts just before the issue happens so I can't catch the act of it starting in the debugger - I don't know what's starting it, and what causes the whole process in the first place.
I have SEH in my main() using __try/__catch, so any unhandled exceptions should be getting trapped there. Instead I am guessing something somewhere bubbles up to the runtime and results in __fastfail.
I tried peppering all my threads with SEH just like main(), tried hooking abort(), exit() and terminate(), and can't find the problem. How do I debug this, any tips?
WinDbg
I would say this is a nice task for WinDbg. WinDbg is part of the Debugging Tools for Windows and it's free. Install both versions, x64 and x86 so that you can debug any kind of application.
Start WinDbg, use the correct bitness
Run your executable under WinDbg (File/Open executable). It will stop at the initial breakpoint.
Set up the symbols, at least .symfix c:\debug\symbols and .reload. As mentioned by #James McNellis, the symbols are available and this will download them when needed.
Continue running the application with g
Reproduce the issue
When WinDbg stops,
create a crash dump with .dump /ma c:\debug\mydump.dmp so you can analyze it later
get information about the exception with .exr -1
switch to the thread that caused the exception with ~#s
look at the callstack with k
Learning WinDbg is a hard task, since most things are done via cryptic commands and not via UI, but it can do almost everything.
For more specific problems when you have some more clues, ask additional questions using the windbg tag.
Visual Studio
Also Visual Studio can download symbols (PDB files; callstack information) from Microsoft Servers.
Go to Tools | Options ... in the main menu
Select Debugging | Symbols from the options menu
Enter a directory name if you want the file to be stored in a specific directory.
Here it is how it looks like in Visual Studio 2015 Community Edition:
Hans and James are right, getting a readable call stack was essential in solving this case.
The codebase installed a pure call handler which was performing a fast fail. Pure handlers fall through SEH. One of the 40-something threads in the program had a pure call error because some other thread partially cleaned up program state which the problem thread then tried to access. Once I had the symbols Visual Studio broke into C++ runtime library at a place where it was calling the purecall handler, and I traced it back into the program where they installed their own.
The way they did fast fail was convoluted and resolved into some RtlXXX function that apparently could not be trapped with signal(SIGABRT) because I did try that before.
Thanks again for the help everyone!
There are a number of errors going on here but I'm sure they are all linked, so first off I'm getting Access violation reading location exceptions at locations (generally near 0x00000000)
but I only get these exceptions in release and not in debug, even if i setup debugging to do so. Also I found where the error happens but when I step over that it goes in to a complete unrelated function that is not called in the line or in any of the functions in that line of code. But when similar code is called it still goes to this unrelated function but doesn't fail. This all only happens in release versions. I know which piece of code is causing the error as if I comment it out its all fine. Also in the watch window in release all the numbers are wrong most of the time, which has never been a problem but it could help also a added some code to it to output the values of variables to a file and the variables are all fine, the pointers are sensible, but when the exception happens it always says their 0x00000020 but when I look at the log they are fine. This is in 32 bit , but on 64 bit windows 7 in c++. Please any help would be good!
Visual Studio's debugger will cause the program to use a debug heap that can hide uninitialized memory if you use the option "Start with Debugging" to start your program, in release or debug. You can try to run your program built in the Debug configuration without the debugger attached, and then use the Attach to Process... option in the debug menu to attach to your executable.
If this doesn't help, I recommend using windbg to debug your program. For help getting started with windbg, take a look at the answers on this question.
Once windbg is configured and symbol paths are set up, you can use the "!analyze -v" option once the program crashes to get a lot of information about the crash automatically.
I'm trying to generate a release build for a C++ application that I've written. The application runs fine (debug & release) when you run it from within VS2008; but when you run the executable it crashes nearly every single time.
Now, is there a hack so I can run this application as a standalone application without having to run through all of the code and finding the bug that is causing it?
Thanks in advance.
In short, no.
you will have to find the bug, if it works within VS, then I'd hazard a guess that it is a timing issue, possibly you're overwriting shared thread data, this would be less likely (though still possible to see) inside VS as its being run in a debug environment which slows it down a bit.
If you want help finding your bug, then tell us more. Otherwise, build your release with debug symbols (pdbs), install DrWatson as the system debugger and run it standalone. When it crashes DrWatson will create a minidump file, load this into WinDbg (my favourite) and you'll be able to see exactly where your bug is (it'll even tell you that the dump contains an exception and show you it by default. You need to add your source code path and path to your symbols in WinDbg to get it to do this correctly).
Then you will also know how to diagnose crashes when the app is run on-site too.
Are you loading external resources? If you are check that your relative paths are correct in the C++ program.
One possibility is that your program uses uninitialized heap data. Launching a program from the debugger enables the NT debug heap, which causes the heap allocator to fill new memory blocks with a fill pattern, and also enables some heap checking. Launching the same program from outside the debugger leaves the NT debug heap disabled, but if the program was linked against the debug version of the C runtime, then the CRT debug heap will still be enabled.
A much less likely possibility is that your program requires SeDebugPrivilege to be set in its process token. The debugger enables this privilege in its process token, which has the side effect that all programs launched from the debugger inherit this privilege. If your program tries to use OpenProcess()/ReadProcessMemory()/WriteProcessMemory() and doesn't handle errors correctly, it's conceivable that it could crash.
There are a few possibilities. Besides what has already been mentioned, running an app from Visual Studio will execute in the same security context as the Visual Studio instance. So if, for instance, you are working on Vista, you might be hitting an unhandled security violation if you're trying to access protected files, or the registry.
What if you build a debug version and run it standalone? Does it crash? If so, you can usually break into the debugger from there and get a call stack to see what the malfunction is.
From the details you've given, it sounds like there may be a library issue. Are you running the program on the same computer? If not then you'll also have to deploy the appropriate libraries for your application. If you are running on the same computer but outside of the dev environment, ensure that your application can see the appropriate libraries.
Best way i have found to debug in release is to create a crash dump when an crash happens and the dump then allows me to load debug symbols on my dev computer and find out whats going on. More info here: http://www.debuginfo.com/articles/effminidumps.html
You can also go to file => open in Visual Studio and open the .exe, so you are not starting it under the debugger per se. Not sure if it will help.
http://blogs.msdn.com/saraford/archive/2008/08/21/did-you-know-you-can-debug-an-executable-that-isn-t-a-part-of-a-visual-studio-project-without-using-tools-attach-to-process-296.aspx