I have read many articles about SEH exceptions in StackOverflow and CodeProject.net.
After I implemented SEH exceptions handling in my C++ program, I was affected by stack overflow exception, which hadn't been caught by my software.
After next part of research I understand, that it's impossible to detect such exception programmatically, because we don't have free stack address space to use, so program memory is corrupted.
I would like to ask you about your experience in handling stack overflow exception. It looks like a challenge and I'm really interested if it's not possible in unmanaged code programming languages?
Below I present a part of my sample program (C++), which reproduces stack overflow exception. It works perfectly for any SEH exception, but not stack overflow:
LONG WINAPI SehHandler(PEXCEPTION_POINTERS pExceptionPtrs)
{
cerr << "Handled SEH exception!\n";
cerr << "ContextRecord: " << pExceptionPtrs->ContextRecord << endl;
cerr << "ExceptionRecord: " << pExceptionPtrs->ExceptionRecord << endl;
// Write minidump file
CreateMiniDump(pExceptionPtrs);
// Terminate process
TerminateProcess(GetCurrentProcess(), 1);
return EXCEPTION_EXECUTE_HANDLER;
}
int fib(unsigned int n) {
if(n == 0) return 0;
if(n == 1) return 1;
return fib(n-1)+fib(n-2);
}
int main(){
SetUnhandledExceptionFilter(SehHandler);
cout << fib(1000000);
return 0;
}
Yes, you can get a minidump out of a SO crash, but never the way you are doing it now. Your SehHandler() function runs on the thread that triggered the exception. And it is in a perilous state, you have about ~7080 bytes of emergency stack space left to do what you need to do. If you consume that then the program will fail with an uncatchable access violation exception.
You cannot call MiniDumpWriteDump() and hope to survive it, that function requires more stack than you have available. So it is a hard kaboom without a minidump.
You need another thread to make that call. That could be, for example, a thread that you create at initialization and block with a WaitForMultipleObjects() call. Your SehHandler() can call SetEvent() to wake it up. After writing the PEXCEPTION_POINTERS value to a global variable. And block indefinitely to allow the thread to create the minidump and abort the process.
Fwiw, by far the best place for that thread is in another process. That also allows you to deal with the really nasty ones that completely corrupt the process state. A "guard" process that you start at initialization. With a named event to signal it and, say, a memory-mapped file to pass the PEXCEPTION_POINTERS. Don't start it in SehHandler(), the process heap is no longer reliable so CreateProcess() cannot work anymore, you have to do it early.
The answer from Hans Passant indicates that there's 7080 bytes of emergency stack. I don't know where that info comes from, and he hasn't answered #nop above, and my findings indicate that that info is incorrect. However this site won't allow me to comment above for some reason, so I'll just leave this here...
There's a function that can be used to query and set how much emergency stack is left to the stack handler: SetThreadStackGuarantee(). Note that since Windows 10 definitely (but I think it's since Windows 7 as well), in most cases this value will be 0. So there's no way to do anything complicated in the handler, by default. You may be able to signal another thread or an external process, as Hans has suggested, but that's all.
However, if you don't want to implement such a complicated solution, and can spare some slack space on the stack, it is easiest to use SetThreadStackGuarantee() to set this to a value high enough that you can continue handling the stack overflow exception just like any other. Note that you need to call this function on each thread that needs this feature, and call it before the stack overflow occurs, so preferably at thread initialization.
Related
Is it possible to catch a stack overflow exception in a recursive C++ function? If so, how?
so what will happen in this case
void doWork()
{
try() {
doWork();
}
catch( ... ) {
doWork();
}
}
I am not looking for an answer to specific OS. Just in general
It's not an exception per se, but if you just want to be able to limit your stack usage to a fixed amount, you could do something like this:
#include <stdio.h>
// These will be set at the top of main()
static char * _topOfStack;
static int _maxAllowedStackUsage;
int GetCurrentStackSize()
{
char localVar;
int curStackSize = (&localVar)-_topOfStack;
if (curStackSize < 0) curStackSize = -curStackSize; // in case the stack is growing down
return curStackSize;
}
void MyRecursiveFunction()
{
int curStackSize = GetCurrentStackSize();
printf("MyRecursiveFunction: curStackSize=%i\n", curStackSize);
if (curStackSize < _maxAllowedStackUsage) MyRecursiveFunction();
else
{
printf(" Can't recurse any more, the stack is too big!\n");
}
}
int main(int, char **)
{
char topOfStack;
_topOfStack = &topOfStack;
_maxAllowedStackUsage = 4096; // or whatever amount you feel comfortable allowing
MyRecursiveFunction();
return 0;
}
There's really no portable way to do it. An out of control recursive function will usually cause an invalid memory access when it tries to allocate a stack frame beyond the stack address space. This will usually just crash your program with a Segmentation Fault/Access Violation depending on the OS. In other words, it won't throw a c++ exception that can be handled in a standard way by the language.
Even if you can do this non-portably, as you can in Windows, it's still a very bad idea. The best strategy is to not overflow the stack in the first place. If you need isolation from some code you don't control, run that code in a different process and you can detect when it crashes. But you don't want to do that sort of thing in your own process, because you don't know what sort of nasty corruption of state the offending code is going to do, and that will make you unstable.
There's an interesting, somewhat related blog post by Microsoft's Raymond Chen about why you shouldn't try to check for valid pointers in a user mode application on Windows.
There isn't a portable way. However, there are a few nonportable solutions.
First, as others have mentioned, Windows provides a nonstandard __try and __except framework called Structured Exeption Handling (your specific answer is in the Knowledge Base).
Second, alloca -- if implemented correctly -- can tell you if the stack is about to overflow:
bool probe_stack(size_t needed_stack_frame_size)
{
return NULL != alloca(needed_stack_frame_size);
};
I like this approach, because at the end of probe_stack, the memory alloca allocated is released and available for your use. Unfortunately only a few operating systems implement alloca correctly. alloca never returns NULL on most operating systems, letting you discover that the stack has overflown with a spectacular crash.
Third, UNIX-like systems often have a header called ucontext.h with functions to set the size of the stack (or, actually, to chain several stacks together). You can keep track of where you are on the stack, and determine if you're about to overflow. Windows comes with similar abilities a la CreateFiber.
As of Windows 8, Windows has a function specifically for this (GetCurrentThreadStackLimits)
On what OS? Just for example, you can do it on Windows using Structured Exception Handling (or Vectored Exception Handling). Normally you can't do it with native C++ exception handling though, if that's what you're after.
Edit: Microsoft C++ can turn a structured exception into a C++ exception. That was enabled by default in VC++ 6. It doesn't happen by default with newer compilers, but I'm pretty sure with a bit of spelunking, you could turn it back on.
It's true that when this happens, you're out of stack space. That's part of why I mentioned vectored exception handling. Each thread gets its own stack, and a vectored exception handler can run in a separate thread from where the exception was thrown. Even SEH, however, you can handle a stack overflow exception -- it just has to manually spawn a thread to do most of the work.
I doubt so, when stack got overflow the program will not be able even to handle exception. Normally OS will close such program and report the error.
This happens mostly because of infinite recursions.
In Windows you can use structured exception handling (SEH), with __try and __except keywords to install your own exception handler routine that can catch stack overflows, access violation, etc etc.
It's pretty neat to avoid Windows' default crash dialog, and replace it with your own, if you need to.
This is done all the time by most modern operating systems. If you want to do it on your own, you'll have to know the maximum "safe" address for your stack (or likewise do some math to determine how many times you can safely call the function), but this can get very tricky if you aren't managing the call stack yourself, since the OS will usually (for good reason) be hiding this from you.
If you are programming in kernel space, this gets significantly easier, but still something I question why you're doing. If you have a stack overflow, it's probably because of a bad algorithmic decision or else an error in the code.
edit: just realized you want to "catch the exception" that results. I don't think my answer directly answers that at all (does this exception even exist? i would figure instead on a spectacular failure), but I'll leave it up for insight. If you want it removed, please let me know in the comments and I will do so.
You have to know always a level of your recursion and check it if greater than some threshold. Max level (threshold) is calclulated by ratio of stack size divided by the memory required one recursive call.
The memory required one recursive call is the memory for all arguments of the function plus the memory for all local variables plus the memory for return address + some bytes (about 4-8).
Of course, you could avoid the recursion problem by converting it to a loop.
Not sure if you're aware of this but any recursive solution can be translated to a loop-based solution, and vice-versa. It is usually desirable to use a loop based solution because it is easier to read and understand.
Regardless of use of recursion or loop, you need to make sure the exit-condition is well defined and will always be hit.
If you use Visual C++
Goto C/C++ , Code Generation
Choose "Both..." in "Basic Runtime Checks"
Then, run your application...
I'm presently moving back to C++ from Java. There are some areas of C++ where higher performance can be achieved by doing more computation on the stack.And some recursive algorithms operate more efficiently on the stack than on the heap.
Obviously the stack is a resource, and if I am going to use it, I should ensure that I do not consume too much (to the point of crashing my program).
I'm running Xcode, and wrote the following simple program:
#include <csignal>
static bool interrupted = false;
long stack_test(long limit){
if((limit>0)&&(interrupted==false))
return stack_test(limit-1)+1; // program crashes here with EXC_BAD_ACCESS...
else
return 0;
}
void signal_handler(int sig){
interrupted = true;
}
int main(char* args[]){
signal(SIGSEGV,&signal_handler);
stack_test(1000000);
signal(SIGSEGV,SIG_DFL);
}
The documentation states that running on BSD, stack limits can be checked by using getrlimit() and that when the stack limit is being reached, a SIGSEGV event is issued. I tried installing the above event handler for this event, but instead, my program stops at the next iteration with EXT_BAD_ACCESS (code=2, ...).
Am I taking the wrong approach here, or is there a better way?
This has the same problem in Java as it does in c++. You are way over-committing to the stack.
And some recursive algorithms operate more efficiently on the stack than on the heap.
Indeed, and they are commonly of the divide and conquer type.
The usefulness of recursion is to reduce the computation to a more manageable computation with each call. limit - 1 is not such a candidate.
If your question is only about the signal, I unfortunately can't offer you any advice on your system.
Your signal handler can't do much to fix the stack overflow. Setting your interrupted flag doesn't help. When your signal handler returns, the instruction that tried to write to an address beyond the end of the stack resumes and it's still going to attempt to write beyond the end of the stack. Your code won't get back to the part which checks your interrupted flag.
With great care and a lot of architecture-specific code, your signal handler could potentially change the context of the thread which encountered the signal such that, when it resumes, it will be at a different point in the code.
You could also use setjmp() and longjmp() to accomplish this at a coarser granularity.
A different approach would be to set up a thread to use a stack that your code allocated, using pthread_attr_setstackaddr() and pthread_attr_setstacksize() prior to pthread_create(). You would run your code in that secondary thread and not the main one. You could set the last page or two of the stack you allocated to be non-writable using mprotect(). Then, your signal handler could set the interrupted flag and also set those pages to be writable. That should give you enough headroom that the resumed code can execute without re-raising the signal, get far enough to check the flag, and return gracefully. Note that this is a one-time last resort, unless you can find a good point to set those guard pages non-writable again.
I was reading thinking in c++ (exceptional handling).
I didn't understand following line
C++ exceptions cannot be used to handle asynchronous events because the exception and its handler are on the same call stack.
I tried searching over web but couldn't able to under stand this line.(specially call stack part)
Can anyone help on it?
EDIT:
what does same call stack means?
Exceptions, when thrown, divert the current thread's execution path to the handling of that exception. There's no way to avoid this by, say, getting another thread to perform the exception handling. The stack is important here because the exception handling involves stack-unwinding which isn't conducive to asyncronouis event handling, or much else.
The problem is like so
try {
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
// do some really long running operation here
longFunctionToCalculate42();
// oops, some critical error!
throw std::runtime_error( "Something went wrong!" );
});
} catch ( const std::exception& e ) {
// this won't do what you think it does
std::cerr << e.what() << std::endl;
}
The asynchronous block is executed separately from the call site, and the caller of that asynchronous function (the dispatched block) cannot catch the exception thrown from it.
You actually can handle async events using exceptions. Weather or not you should is another matter. I'll only address that briefly: you usually shouldn't because there are more purpose-direct mechanisms to handle such things. Like passing messages between threads or raising some kind of event.
As to how you can accomplish this, what you have to do is catch the exception in the throw-ing thread, record the information somewhere, and have the other thread pick that up. Note that this really boils down fundamentally to passing messages between threads, with the additional complexity of stack unwinding and the like.
C++11 provides current_exception(), returning a exception_ptr, which provides the means to save the information about the exception somewhere the responding thread can pick it up. It is still up to you to build the code that actually retrieves and processes this exception_ptr up from wherever you saved it, and that's beyond the scope of this answer.
Note when thinking about this that, unless you need actual exceptions, doing this gains you nothing over simply passing messages between threads, and costs you the stack unwinding and semantic implications of throwing and catching exceptions.
It means that asynchronous events do not follow the exception's model where "exception and its handler are on the same call stack". That is -
exceptions rely on the dynamic chain of function calls on the program
s runtime stack (they have dynamic scope ), whereas asynchronous
events must be handled by completely separate code that is not part of
the normal program flow (typically, interrupt service routines or
event loops)
Note "completely separate code", which means that you'd have to rely on some other mechanism to handle asynchronous events (if you really need so).
In C++ a stack overflow usually leads to an unrecoverable crash of the program. For programs that need to be really robust, this is an unacceptable behaviour, particularly because stack size is limited. A few questions about how to handle the problem.
Is there a way to prevent stack overflow by a general technique. (A scalable, robust solution, that includes dealing with external libraries eating a lot of stack, etc.)
Is there a way to handle stack overflows in case they occur? Preferably, the stack gets unwound until there's a handler to deal with that kinda issue.
There are languages out there, that have threads with expandable stacks. Is something like that possible in C++?
Any other helpful comments on the solution of the C++ behaviour would be appreciated.
Handling a stack overflow is not the right solution, instead, you must ensure that your program does not overflow the stack.
Do not allocate large variables on the stack (where what is "large" depends on the program). Ensure that any recursive algorithm terminates after a known maximum depth. If a recursive algorithm may recurse an unknown number of times or a large number of times, either manage the recursion yourself (by maintaining your own dynamically allocated stack) or transform the recursive algorithm into an equivalent iterative algorithm
A program that must be "really robust" will not use third-party or external libraries that "eat a lot of stack."
Note that some platforms do notify a program when a stack overflow occurs and allow the program to handle the error. On Windows, for example, an exception is thrown. This exception is not a C++ exception, though, it is an asynchronous exception. Whereas a C++ exception can only be thrown by a throw statement, an asynchronous exception may be thrown at any time during the execution of a program. This is expected, though, because a stack overflow can occur at any time: any function call or stack allocation may overflow the stack.
The problem is that a stack overflow may cause an asynchronous exception to be thrown even from code that is not expected to throw any exceptions (e.g., from functions marked noexcept or throw() in C++). So, even if you do handle this exception somehow, you have no way of knowing that your program is in a safe state. Therefore, the best way to handle an asynchronous exception is not to handle it at all(*). If one is thrown, it means the program contains a bug.
Other platforms may have similar methods for "handling" a stack overflow error, but any such methods are likely to suffer from the same problem: code that is expected not to cause an error may cause an error.
(*) There are a few very rare exceptions.
You can protect against stack overflows using good programming practices, like:
Be very carefull with recursion, I have recently seen a SO resulting from badly written recursive CreateDirectory function, if you are not sure if your code is 100% ok, then add guarding variable that will stop execution after N recursive calls. Or even better dont write recursive functions.
Do not create huge arrays on stack, this might be hidden arrays like a very big array as a class field. Its always better to use vector.
Be very carefull with alloca, especially if it is put into some macro definition. I have seen numerous SO resulting from string conversion macros put into for loops that were using alloca for fast memory allocations.
Make sure your stack size is optimal, this is more important in embeded platforms. If you thread does not do much, then give it small stack, otherwise use larger. I know reservation should only take some address range - not physical memory.
those are the most SO causes I have seen in past few years.
For automatic SO finding you should be able to find some static code analysis tools.
Re: expandable stacks. You could give yourself more stack space with something like this:
#include <iostream>
int main()
{
int sp=0;
// you probably want this a lot larger
int *mystack = new int[64*1024];
int *top = (mystack + 64*1024);
// Save SP and set SP to our newly created
// stack frame
__asm__ (
"mov %%esp,%%eax; mov %%ebx,%%esp":
"=a"(sp)
:"b"(top)
:
);
std::cout << "sp=" << sp << std::endl;
// call bad code here
// restore old SP so we can return to OS
__asm__(
"mov %%eax,%%esp":
:
"a"(sp)
:);
std::cout << "Done." << std::endl;
delete [] mystack;
return 0;
}
This is gcc's assembler syntax.
C++ is a powerful language, and with that power comes the ability to shoot yourself in the foot. I'm not aware of any portable mechanism to detect and correct/abort when stack overflow occurs. Certainly any such detection would be implementation-specific. For example g++ provides -fstack-protector to help monitor your stack usage.
In general your best bet is to be proactive in avoiding large stack-based variables and careful with recursive calls.
I don't think that that would work. It would be better to push/pop esp than move to a register because you don't know if the compiler will decide to use eax for something.
Here ya go:
https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/resetstkoflw?view=msvc-160
You wouldn't catch the EXCEPTION_STACK_OVERFLOW structured exception yourself because the OS is going to catch it (in Windows' case).
Yes, you can safely recover from an structured exception (called "asynchronous" above) unlike what was indicated above. Windows wouldn't work at all if you couldn't. PAGE_FAULTs are structured exceptions that are recovered from.
I am not as familiar with how things work under Linux and other platforms.
Is it possible to catch a stack overflow exception in a recursive C++ function? If so, how?
so what will happen in this case
void doWork()
{
try() {
doWork();
}
catch( ... ) {
doWork();
}
}
I am not looking for an answer to specific OS. Just in general
It's not an exception per se, but if you just want to be able to limit your stack usage to a fixed amount, you could do something like this:
#include <stdio.h>
// These will be set at the top of main()
static char * _topOfStack;
static int _maxAllowedStackUsage;
int GetCurrentStackSize()
{
char localVar;
int curStackSize = (&localVar)-_topOfStack;
if (curStackSize < 0) curStackSize = -curStackSize; // in case the stack is growing down
return curStackSize;
}
void MyRecursiveFunction()
{
int curStackSize = GetCurrentStackSize();
printf("MyRecursiveFunction: curStackSize=%i\n", curStackSize);
if (curStackSize < _maxAllowedStackUsage) MyRecursiveFunction();
else
{
printf(" Can't recurse any more, the stack is too big!\n");
}
}
int main(int, char **)
{
char topOfStack;
_topOfStack = &topOfStack;
_maxAllowedStackUsage = 4096; // or whatever amount you feel comfortable allowing
MyRecursiveFunction();
return 0;
}
There's really no portable way to do it. An out of control recursive function will usually cause an invalid memory access when it tries to allocate a stack frame beyond the stack address space. This will usually just crash your program with a Segmentation Fault/Access Violation depending on the OS. In other words, it won't throw a c++ exception that can be handled in a standard way by the language.
Even if you can do this non-portably, as you can in Windows, it's still a very bad idea. The best strategy is to not overflow the stack in the first place. If you need isolation from some code you don't control, run that code in a different process and you can detect when it crashes. But you don't want to do that sort of thing in your own process, because you don't know what sort of nasty corruption of state the offending code is going to do, and that will make you unstable.
There's an interesting, somewhat related blog post by Microsoft's Raymond Chen about why you shouldn't try to check for valid pointers in a user mode application on Windows.
There isn't a portable way. However, there are a few nonportable solutions.
First, as others have mentioned, Windows provides a nonstandard __try and __except framework called Structured Exeption Handling (your specific answer is in the Knowledge Base).
Second, alloca -- if implemented correctly -- can tell you if the stack is about to overflow:
bool probe_stack(size_t needed_stack_frame_size)
{
return NULL != alloca(needed_stack_frame_size);
};
I like this approach, because at the end of probe_stack, the memory alloca allocated is released and available for your use. Unfortunately only a few operating systems implement alloca correctly. alloca never returns NULL on most operating systems, letting you discover that the stack has overflown with a spectacular crash.
Third, UNIX-like systems often have a header called ucontext.h with functions to set the size of the stack (or, actually, to chain several stacks together). You can keep track of where you are on the stack, and determine if you're about to overflow. Windows comes with similar abilities a la CreateFiber.
As of Windows 8, Windows has a function specifically for this (GetCurrentThreadStackLimits)
On what OS? Just for example, you can do it on Windows using Structured Exception Handling (or Vectored Exception Handling). Normally you can't do it with native C++ exception handling though, if that's what you're after.
Edit: Microsoft C++ can turn a structured exception into a C++ exception. That was enabled by default in VC++ 6. It doesn't happen by default with newer compilers, but I'm pretty sure with a bit of spelunking, you could turn it back on.
It's true that when this happens, you're out of stack space. That's part of why I mentioned vectored exception handling. Each thread gets its own stack, and a vectored exception handler can run in a separate thread from where the exception was thrown. Even SEH, however, you can handle a stack overflow exception -- it just has to manually spawn a thread to do most of the work.
I doubt so, when stack got overflow the program will not be able even to handle exception. Normally OS will close such program and report the error.
This happens mostly because of infinite recursions.
In Windows you can use structured exception handling (SEH), with __try and __except keywords to install your own exception handler routine that can catch stack overflows, access violation, etc etc.
It's pretty neat to avoid Windows' default crash dialog, and replace it with your own, if you need to.
This is done all the time by most modern operating systems. If you want to do it on your own, you'll have to know the maximum "safe" address for your stack (or likewise do some math to determine how many times you can safely call the function), but this can get very tricky if you aren't managing the call stack yourself, since the OS will usually (for good reason) be hiding this from you.
If you are programming in kernel space, this gets significantly easier, but still something I question why you're doing. If you have a stack overflow, it's probably because of a bad algorithmic decision or else an error in the code.
edit: just realized you want to "catch the exception" that results. I don't think my answer directly answers that at all (does this exception even exist? i would figure instead on a spectacular failure), but I'll leave it up for insight. If you want it removed, please let me know in the comments and I will do so.
You have to know always a level of your recursion and check it if greater than some threshold. Max level (threshold) is calclulated by ratio of stack size divided by the memory required one recursive call.
The memory required one recursive call is the memory for all arguments of the function plus the memory for all local variables plus the memory for return address + some bytes (about 4-8).
Of course, you could avoid the recursion problem by converting it to a loop.
Not sure if you're aware of this but any recursive solution can be translated to a loop-based solution, and vice-versa. It is usually desirable to use a loop based solution because it is easier to read and understand.
Regardless of use of recursion or loop, you need to make sure the exit-condition is well defined and will always be hit.
If you use Visual C++
Goto C/C++ , Code Generation
Choose "Both..." in "Basic Runtime Checks"
Then, run your application...