C++ Program hangs on ntdll.dll execution - c++

When I try to run my program it never enters the main() function, but just hangs. Some how though, my program is throwing an exception in one of my classes as the first call in the stack after numerous calls to ntdll.dll. The exception seems to be the factor that is causing my program to hang with a "< bad ptr >". I am unable to trigger any breakpoints I set though, making it harder to find where the problem originates.

You need to figure out which system call you made that caused the crash. Normally this kind of thing is the result of passing garbage into some windows API.
Use a debugger and look at the call stack. The last entry that is inside your code is likely to be the cause of the problem. It probably has some bad data that is it not handling correctly.

This sounds like you are using global objects/singletons. If any of these depend on each other you are in trouble. I would look for all your globals and put a break point in the contructor of each. It sounds like the contructor for one global is using another global but the second has not been contructed yet.
Finally if this is not the cause, and your application is loading 3rd party dlls they might be clashing. I remember once orbix and ms message queue dlls classing in one was loaded before the other, everything worked if they were loaded the other way around. In the end to find this I removed every thing till I was just left with an empty main and then started adding things one by one until I found the problem.

Related

Analysing a Crash

I'm writing a C++ Software Image processing tool.The tool works fine, but suddenly it stops and it never sends any exception or any crash or nothing that can let me which line or which area that does that crash.
How can I determine that faulty code in that situation?
There's a few things you can do:
First of all though, it sounds more like an infinite loop, deadlock, or like you're using all of your system resources and it's just slowing down and taking a very (possibly infinite) long time. If that's the case, you'll have to find it with debugging.
Things that you can try - not necessarily in this order:
Look for shared variables you're using.  Is there a chance that you
have a deadlock with threads and mutexes?  Think about it and try to
fix it.
Check for use of uninitialized variables/pointers.  Sometimes
(rarely) you can get very strange behavior when you invoke undefined
behavior - I'm not a windows C++ dev (I work on Linux), but it
wouldn't be the first time I saw a lockup from a segmentation fault.
Add error output (std::cerr/stderror) to your processing logic so
you can see how far in it crashes. After that, set a condition to
catch it around that point so you can watch it happen in the
debugger and see the state of your variables and what might be
wrong.
Do a stack trace so you can see what calls have been hit most
recently. This will at least let you know the last function chain
that executed.
Have you used stack tracing before?
Look up MSDN documentation on how to use them. They have different type of stack trace depending on your application.
You could
Use a debugger
Add logging code to your program
Cut sections of code from your program until it starts working
Start with the first.

About stack cleanup and separate process

I need to use a function in a obj file for which I don't have the source code. The function works well and does its job but it corrupts the stack (runtime esp was not properly saved.. error in visual studio c++ 2010), and I don't know assembly well enough to find the problem and patch it. (From what Ive read it has to do with not having an equal number of pops and pushes.) I have tried executing the function in a separate thread but since it shares the same memory as my main program, it also crashes. The function works and does everything it needs to do, expect that it corrupts the stack and make the application crash. Is there any way I can isolate it from my main application? Ive read about CreateProcess but from what I understood it executes a separate .EXE file which I don't want to do. I want to execute a function as if it was an external program, so it does not crash my main program even though it corrupts the stack. Anyway to do such a thing in C/C++? Or maybe someone has another idea to fix things up? Also, I don't care if its "dirty" or not standard compliant since im writing this program for personal use. Since the function does what it needs to do Im really trying to keep it even though it corrupts the call stack.
Thank you very much
That smells very much like a calling convention mismatch. Is it documented anywhere how the function is to be called? Do you have a header file containing a function prototype for the function? Try calling it with a different calling convention—if you're calling it as cdecl, try calling it as stdcall, or vice-versa.
See also What can go wrong when you mismatch the calling convention?.

How do I find out where an object was instanciated using gdb?

I'm debugging an application and it segfaults at a position where it is almost impossible to determine which of the many instances causes the segfault.
I figured that if I'm able to resolve the position at which the object is created, I will know which instance is causing the problem and resolve the bug.
To be able to retrieve this information, gdb (or some other application) would of course have to override the default malloc/new/new[] implementations, but instrumenting my application with this would be alright.
One could argue that I could just put a breakpoint on the line before the one that segfaults and step into the object from there, but the problem is that this is a central message dispatcher loop which handles a lot of different messages and I'm not able to set a breakpoint condition in such a way as to trap my misbehaving object.
So, at the point where the segfault occurs, you have the object, but you don't know which of many pieces of code that create such objects created it, right?
I'd instrument all of those object-creation bits and have them log the address of each object created to a file, along with the file and line number (the __LINE__ and __FILE__ pre-defined macros can help make this easy).
Then run the app under the debugger, let it trap the segfault and look the address of the offending object up in your log to find out where it was created. Then peel the next layer of the onion.
Have you tried using a memory debugging library (e.g. dmalloc). Many of these already instrument new, etc. and records where an allocation is made. Some are easier to access from gdb than others though.
This product has a memory debugging feature that does what you want: http://www.allinea.com/index.php?page=48
I would first try using the backtrace command in gdb when the segfault occurs. If that does not give me a good clue about what is going on, I would next try to use valgrind to check if there are any memory leaks occurring. These two steps are usually sufficient, in my experience, to narrow down and find the problem spot in most of the usual cases.
Regards.

debug stack overflow in windows?

So I'm trying to debug this strange problem where a process ends without calling some destructors...
In the VS (2005) debugger, I hit 'Break all' and look around in the call stacks of the threads of the misteriously disappearing process, when I see this:
smells like SO http://img6.imageshack.us/img6/7628/95434880.jpg
This definitely looks like a SO in the making, which would explain why the process runs to its happy place without packing its suitcase first.
The problem is, the VS debugger's call stack only shows what you can see in the image.
So my question is: how can I find where the infinite recursion call starts?
I read somewhere that in Linux you can attach a callback to the SIGSEGV handler and get more info on what's going on.
Is there anything similar on Windows?
To control what Windows does in case of an access violation (SIGSEGV-equivalent), call SetErrorMode (pass it parameter 0 to force a popup in case of errors, allowing you to attach to it with a debugger.)
However, based on the stack trace you have already obtained, attaching with a debugger on fault may yield no additional information. Either your stack has been corrupted, or the depth of recursion has exceeded the maximum number of frames displayable by VS. In the latter case, you may want to decrease the default stack size of the process (use the /F switch or equivalent option in the Project properties) in order to make the problem manifest itself sooner, and make sure that VS will display all frames. You may, alternatively, want to stick a breakpoint in std::basic_filebuf<>::flush() and walk through it until the destruction phase (or disable it until just prior to the destruction phase.)
Well, you know what thread the problem is on - it might be a simple matter of tracing through it from inception to see where it goes off into the weeds.
Another option is to use one of the debuggers in the Debugging Tools for Windows package - they may be able to show more than the VS debugger (maybe), even if they are generally more complex and difficult to use (actually maybe because of that).
That does look at first glance like an infinite recursion, you could try putting a breakpoint at the line before the one that terminates the process. Does it get there ok? If it does, you've got two fairly easy ways to go.
Either you just step forward and see which destructors get called and when it gets caught up. Or you could put a printf/OutputDebugString in every relevant objects destructor (ONly ones which are globals should need this). If the message is the first thing the destructor does, then the last message you see is from the destructor which hangs things up.
On the other hand, if it doesn't get to that breakpoint I originally mentioned, then can do something similar, but it will be more annoying since the program is still "doing stuff".
I wouldn't rule out there being such a handler in Windows, but I've never heard of it.
I think the traceback that you're showing may be bogus. If you broke into the process after some kind of corruption had already occurred, then the traceback isn't necessarily valid. However, if you're lucky the bottom of the stack trace still has some clues about what's going on.
Try putting Sleep() calls into selected functions in your source that might be involved in the recursion. That should give you a better chance of breaking into the process before the stack has completely overflowed.
I agree with Dan Breslau. Your stack is bogus. may be simply because you don't have the right symbols, though.
If a program simply disappears without the WER handling kicking in, it's usually an out of memory condition. Have you gone investigated that possibility ?

Debugging a crash after exiting? (After main returned)

This is a fairly involved bug, and I've tried looking around to find other sources of help, but for reasons I don't understand, "Program Crashes in Vista" is not the most helpful query.
The issue I'm having is that the program I'm working on - a graphical, multithreaded data visualization software that uses OpenGL and the Windows API - is crashing after WinMain() returns. I've tried stepping through the shutdown routine as well as looking at a stack trace, and the last bit of code that's not assembly is _crtExitProcess, where it crashes in the actual ExitProcess(0) call. After that, the stack trace shows kernel32.dll and four ntdll.dll, which is where it actually crashes.
This bug only occurs on Vista, and the same exact code when run on XP exits normally. I really can't think of anything that would help me debug this problem, and debugging this issue is something I've never really learned. Any help would be appreciated.
I've done a little digging around, and I've found a couple of posts around that suggest you're not the only one suffering from this:
http://developer.nvidia.com/forums/index.php?showtopic=318
http://objectmix.com/xml-soap/115379-problem-latest-ms-patches-msxml4-vista.html
Particularly, the second one is of interest, where Tom Chm mentions:
We believe we have identified the root
cause of our crash, and adding a
virtual destructor to our interface
class wrapper seems to resolve our
problem. But we would like to know the
exact cause of the crash to verify
that we didn't just sweep the actual
problem under the rug.
The problem may be with a destructor somewhere, or lack thereof. If you have a way of attaching a debugger and stepping through the shutdown process, it might be of help.
You might want to read through the whole thread and see if there's something you can learn. That is, if you haven't already found these posts in your searching, of course.
Sounds a little like a problem with a destructor.
Check for objects that are destroyed on shutdown. This will mainly be global or static objects. Look carefully at their destructors for a case of accessing something that is no longer valid. In a multi-threaded environment, it might be a race condition, where an object is destroyed while another thread is still using it.
Try writing a log as objects are destroyed. e.g.
SomeClass::~SomeClass()
{
WriteLog("Begin ~SomeClass()");
// do whatever
WriteLog("End ~SomeClass()");
}
WriteLog() should open the log file, write and then close the file to make sure the file is flushed. Using a Mutex or CriticalSection to avoid conflicts would be a good idea.
Looking at the log after a crash might give you some clues as to what is going on.