I am working with a program where my code calls a third party library which uses boost and shared_pointers to create a large and complex structure. This structure is created in a method that I call and at the end of the method I know that the program is finished.
For a large sample that I am handling the code to handle the processing takes 30 minutes and the boost code called automatically at exit takes many hours. Exiting the program without releasing the memory and spending all that time would be a perfectly acceptable outcome.
I tried
vector *iddListV = new vector(); // this WILL leak memory
with all the relevant structures added to the vector but this does not help.
I also tried calling exit(0); before reaching the end of the subroutine. This also causes the boost code to spend many hours trying to release pointers.
How to I get a C++ program (Microsoft C++ on Windows if that matters) to abruptly exit without calling the boost destructors.
My constraints are I can call any function before the boost structure are allocated but cannot modify the code once it starts running.
_Exit quits without calling any destructors.
If you're unconcerned about portability, you can call TerminateProcess(). But remember to take care that you are absolutely sure that your program is in a state which is ready to terminate. For example, if you terminate before I/O has had a chance to flush, then your file data and network streams may become invalid.
It is possible, in a portable manner, to do:
#include <exception>
...
std::terminate();
However, there's a big gotcha, in that, at least on linux, this may cause a core dump. (I'm really not sure what the behavior is on Windows).
It should be noted, that the behavior is implementation defined as far as whether or not destructors are called. Siting §15.5.1 P2:
In the situation where the search for a handler (15.3) encounters the
outermost block of a function with a noexcept-specification that does
not allow the exception (15.4), it is implementation-defined whether
the stack is unwound, unwound partially, or not unwound at all before
std::terminate() is called.
Additionally in §18.8.3.4 P1:
Remarks: Called by the implementation when exception handling must be
abandoned for any of several reasons (15.5.1), in effect immediately
after evaluating the throw-expression (18.8.3.1). May also be called
directly by the program.
C++11 also defines the function std::quick_exit(int status) that can be used in a similar manner (presumably without a coredump). This function is available from <cstdlib>.
Related
Copying files to a directory to another directory using this code
auto const copyOption = std::filesystem::copy_options::recursive | std::filesystem::copy_options::skip_symlinks;
std::filesystem::copy("/mnt/iso", "/mnt/usb", copyOption);
Copying big files can take a long time.
So how to check when copy is ended ?
How to check if std::filesystem::copy is ended?
The call to a function ends by either (1) returning, by (2) throwing or by (3) terminating the program.
If the execution of the program proceeds to the next statement (or next sibling expression), then you know that the function call has ended by (1) returning. If the execution proceeds to unwind and eventually enters a catch block, then you know that the function has ended by (2) throwing. If the program no longer runs, then you know that the function has (3) terminated the program (std::filesystem::copy won't do this one directly, although it can happen if it throws without being caught). If none of those have happened, then you know that the function call hasn't ended yet.
For example try to unmount an usbkey just after copy finished will take one or two minutes longer if you have big files
There is no standard way in C++ to verify when data has physically been written onto a device.
You've tagged [ubuntu], so you may be interested in fsync function in the POSIX standard. You cannot use it with conjunction with std::filesystem::copy; you'll need to use the other POSIX file manipulation functions instead. fsync should guarantee that any writes have been passed on to the device by the kernel. From then on, you rely on the hardware.
Say I have two C++ functions foo1() and foo2(), and I want to minimize the likelihood that that foo1() starts execution but foo2() is not called due to some external event. I don't mind if neither is called, but foo2() must execute if foo1() was called. Both functions can be called consecutively and do not throw exceptions.
Is there any benefit / drawback to wrapping the functions in an object and calling both in the destructor? Would things change if the application was multi-threaded (say the parent thread crashes)? Are there any other options for ensuring foo2() is called so long as foo1() is called?
I thought having them in a destructor might help with e.g. SIGINT, though I learned SIGINT will stop execution immediately, even in the middle of the destructor.
Edit:
To clarify: both foo1() and foo2() will be abstracted away, so I'm not concerned about someone else calling them in the wrong order. My concern is solely related to crashes, exceptions, or other interruptions during the execution of the application (e.g. someone pressing SIGINT, another thread crashing, etc.).
If another thread crashes (without relevant signal handler -> the whole application exits), there is not much you can do to guarantee that your application does something - it's up to what the OS does. And there are ALWAYS cases where the system will kill your app without your actual knowledge (e.g. a bug that causes "all" memory being used by your app and the OS "out of memory killer" killing your process).
The only time your destructor is guaranteed to be executed is if the object is constructed and a C++ exception is thrown. All signals and such, make no such guarantees, and contininuing to execute [in the same thread] after for example SIGSEGV or SIGBUS is well into the "undefined" parts of the world - nothing much you can do about that, since the SEGV typically means "you tried to do something to memory that doesn't exist [or that you can't access in the way you tried, e.g. write to code-memory]", and the processor would have aborted the current instruction. Attempting to continue where you were will either lead to the same instruction being executed again, or the instruction being skipped [if you continue at the next instruction - and I'm ignoring the trouble of determining where that is for now]. And of course, there are situations where it's IMPOSSIBLE to continue even if you wanted to - say for example the stack pointer has been corrupted [restored from memory that was overwritten, etc].
In short, don't spend much time trying to come up with something that tries to avoid these sort of scenarios, because it's unlikely to work. Spend your time trying to come up with schemes where you don't need to know if you completed something or not [for example transaction based programming, or "commit-based" programming (not sure if that's the right term, but basically you do some steps, and then "commit" the stuff done so far, and then do some further steps, etc - only stuff that has been "committed" is sure to be complete, uncommitted work is discarded next time around) , where something is either completely done, or completely discarded, depending on if it completed or not].
Separating "sensitive" and "not sensitive" parts of your application into separate processes can be another way to achieve some more safety.
In Ubuntu 14.04, I have a C++ API as a shared library which I am opening using dlopen, and then creating pointers to functions using dlsym. One of these functions CloseAPI releases the API from memory. Here is the syntax:
void* APIhandle = dlopen("Kinova.API.USBCommandLayerUbuntu.so", RTLD_NOW|RTLD_GLOBAL);
int (*CloseAPI) = (int (*)()) dlsym(APIhandle,"CloseAPI");
If I ensure that during my code, the CloseAPI function is always called before the main function returns, then everything seems fine, and I can run the program again the next time. However, if I Ctrl-C and interrupt the program before it has had time to call CloseAPI, then on the next time I run the program, I get a return error whenever I call any of the API functions. I have no documentation saying what this error is, but my intuition is that there is some sort of lock on the library from the previous run of the program. The only thing that allows me to run the program again, is to restart my machine. Logging in and out does not work.
So, my questions are:
1) If my library is a shared library, why am I getting this error when I would have thought a shared library can be loaded by more than one program simultaneously?
2) How can I resolve this issue if I am going to be expecting Ctrl-C to be happening often, without being able to call CloseAPI?
So, if you do use this api correctly then it requires you to do proper clean up after use (which is not really user friendly).
First of all, if you really need to use Ctrl-C, allow program to end properly on this signal: Is destructor called if SIGINT or SIGSTP issued?
Then use a technique with a stack object containing a resource pointer (to a CloseAPI function in this case). Then make sure this object will call CloseAPI in his destructor (you may want to check if CloseAPI wasn't called before). See more in "Effective C++, Chapter 3: Resource Management".
That it, even if you don't call CloseAPI, pointer container will do it for you.
p.s. you should considering doing it even if you're not going to use Ctrl-C. Imagine exception occurred and your program has to be stopped: then you should be sure you don't leave OS in an undefined state.
In certain cases - especially when an exception escapes a destructor during stack unwinding - C++ runtime calls terminate() which must do something reasonable post-mortem and then exit the program. When a question "why so harsh" arises the answer is usually "there's nothing more reasonable to do in such error situations". That sounds reasonable if the whole program is in C++.
Now what if the C++ code is in a library and the program that uses the library is not in C++? This happens quite often - for example I might have a native C++ COM component consumed by a .NET program. Once terminate() is called inside the component code the .NET program suddenly ends abnormally. The program author will first of all think "I don't care of C++, why the hell is this library make my program exit?"
How do I handle the latter scenario when developing libraries in C++? Is it reasonable that terminate() unexpectedly ends the program? Is there a better way to handle such situations?
Why is the C++ runtime calling terminate()? It doesn't do it at random, or due to circumstances which cannot be defined and/or avoided when the code is written. It does it because your code does something that is defined to result in a call to terminate(), such as throwing an exception from a destructor during stack unwinding.
There is a list in the C++ standard of all the situations which are defined to result in call to terminate(). If you don't want terminate() to be called, don't do any of those things in your code. The same applies to unexpected(), abort(), and so on.
I don't think this is any different really from the fact that you have to avoid undefined behavior, or in general avoid writing code which is wrong. You also have to avoid behavior which is defined but undesirable.
Perhaps you have a particular example where it is difficult to avoid a call to terminate(), but throwing an exception from a destructor during stack unwinding isn't it. Just don't throw exceptions out of destructors, ever. This means designing your destructors such that if they do something which might fail, the destructor catches the exception and your code continues in a defined state.
There are some situations where your system will impolitely hack your process away at the knees because of something that your C++ code does (although not by calling terminate()). For example, if the system is overcommitting memory and the VMM can't honour the promises malloc/new have made, then your process might be killed. But that's a system feature, and probably applies equally to the other language that's calling your C++. I don't think there's anything you can (or need to) do about it, as long as your caller is aware that your library might allocate memory. In that circumstance it's not your code's fault the process died, it's the defined response of the operating system to low-memory conditions.
I think the more fundamental issue isn't specifically what terminate does, but the library's design. The library may be designed to be used only with C++ in which case exceptions can be caught and handled as appropriate within the app.
If the library is intended to be used in conjunction with non-C++ apps, it needs to provide an interface that ensures no exceptions leave the library, for example an interface that does a catch(...).
Suppose you have a function in C++ called cppfunc and you are invoking it from another langugage (such as C or .NET). I suggest you create a wrapper function, let's say exportedfunc, like so:
int exportedfunc(resultype* outresult, paramtype1 param1, /* ... */)
{
try {
*outresult = cppfunc(param1,param2,/* ... */);
return 0; // indicate success
}catch( ... ) { // may want to have other handlers
/* possibly set other error status info */
return -1; // indicate failure
}
}
Basically, you need to ensure that exceptions do not cross language boundaries... so you need to wrap your C++ functions with a function that catches all exceptions and reports a status code or does something acceptable other than invoking std::terminate.
The default terminate handler will call abort. If you don't want this behavior, define your own terminate handler and set it using set_terminate.
The C++ standard provides the std::set_terminate function which lets you specify what function std::terminate should actually call. std::terminate should only get called in dire circumstances, and sure enough the situations the standard describes for when it's called are dire (e.g. an uncaught exception). When std::terminate does get called the situation seems analagous to being out of memory -- there's not really much you can sensibly do.
I've read that it can be used to make sure resources are freed -- but for the majority of resources this should be handled automatically by the OS when the process exits (e.g. file handles). Theoretically I can see a case for if say, you needed to send a server a specific message when exiting due to a crash. But the majority of the time the OS handling should be sufficient.
When is using a terminate handler the Right Thing(TM)?
Update: People interested in what can be done with custom terminate handlers might find this non-portable trick useful.
This is just optimistic:
but for the majority of resources this should be handled automatically by the OS when the process exits
About the only resources that the OS handles automatically are "File Handles" and "Memory" (And this may vary across OS's).
Practically all other resources (and if somebody has a list of resources that are automatically handled by OS's I
would love that) need to be manually released by the OS.
Your best bet is to avoid exit using terminate() and try a controlled shut down by forcing the stack to unwind correctly.
This will make sure that all destructors are called correctly and your resources are released (via destructors).
About the only thing I would do is log the problem. So that when it does happened I could go back and fix the code so that it does not happen again. I like my code to unwind the stack nicely for resource deallocation, but this is an opinion some people like abrupt halts when things go badly.
My list of when terminate is called:
In general it is called when the exception handling mechanism cannot find a handler for a thrown exception. Some specific examples are:
An exception escapes main()
Note: It is implementation defined whether the stack is unwound here.
Thus I always catch in main and then rethrow (if I do not explicitly handle).
That way I guarantee unwinding of the stack (across all platforms) and still get the benefits of the OS exception handling mechanism.
Two exceptions propagating simultaneously.
An exception escapes a desatructor while another exception is propagating.
The expression being thrown generates an exception
An exception before or after main.
If an exception escapes the constructor/destructor of a global object.
If an exception escapes the destructor of a function static variable.
(ie be careful with constructors/destructors of nonlocal static object)
An exception escapes a function registered with atexit().
A rethrow when no exception is currently propagating.
An unlisted exception escapes a method/function that has exception specifier list.
via unexpected.
Similar to a statement made in Martin York's answer, about the only thing I do in a custom terminate handler is log the problem so I can identify and correct the offending code. This is the only instance I find that using a custom terminate handler is the Right Thing.
Since it is implementation-defined whether or not the stack is unwound before std::terminate() is called, I sometimes add code to generate a backtrace in order to locate an uncaught exception1.
1) This seems to work for me when using GCC on Linux platforms.
I think the right question would be how to avoid the calls to terminate handler, rather than when to use it.