I'm trying to spawn a new process from my C++-project using fork-exec. I'm using fork-exec in order to create a bi-directional pipe to the child process. But I'm afraid my resources in the forked process won't get freed properly, since the exec-call will completely take over my process and is not going to call any destructors.
I tried circumventing this by throwing an exception and calling execl from a catch block at the end of main, but this solution doesn't destruct any singletons.
Is there any sensible way to achieve this safely? (hopefully avoiding any atExit hacks)
Ex: The following code outputs:
We are the child, gogo!
Parent proc, do nothing
Destroying object
Even though the forked process also has a copy of the singleton which needs to be destructed before I call execl.
#include <iostream>
#include <unistd.h>
using namespace std;
class Resources
{
public:
~Resources() { cout<<"Destroying object\n"; }
};
Resources& getRes()
{
static Resources r1;
return r1;
}
void makeChild(const string &command)
{
int pid = fork();
switch(pid)
{
case -1:
cout<<"Big error! Wtf!\n";
return;
case 0:
cout<<"Parent proc, do nothing\n";
return;
}
cout<<"We are the child, gogo!\n";
throw command;
}
int main(int argc, char* argv[])
{
try
{
Resources& ref = getRes();
makeChild("child");
}
catch(const string &command)
{
execl(command.c_str(), "");
}
return 0;
}
There are excellent odds that you don't need to call any destructors in between fork and exec. Yeah, fork makes a copy of your entire process state, including objects that have destructors, and exec obliterates all that state. But does it actually matter? Can an observer from outside your program -- another, unrelated process running on the same computer -- tell that destructors weren't run in the child? If there's no way to tell, there's no need to run them.
Even if an external observer can tell, it may be actively wrong to run destructors in the child. The usual example for this is: imagine you wrote something to stdout before calling fork, but it got buffered in the library and so has not actually been delivered to the operating system yet. In that case, you must not call fclose or fflush on stdout in the child, or the output will happen twice! (This is also why you almost certainly should call _exit instead of exit if the exec fails.)
Having said all that, there are two common cases where you might need to do some cleanup work in the child. One is file descriptors (do not confuse these with stdio FILEs or iostream objects) that should not be open after the exec. The correct way to deal with these is to set the FD_CLOEXEC flag on them as soon as possible after they are opened (some OSes allow you to do this in open itself, but that's not universal) and/or to loop from 3 to some large number calling close (not fclose) in the child. (FreeBSD has closefrom, but as far as I know, nobody else does, which is a shame because it's really quite handy.)
The other case is system-global thread locks, which - this is a thorny and poorly standardized area - may wind up held by both the parent and the child, and then inherited across exec into a process that has no idea it holds a lock. This is what pthread_atfork is supposed to be for, but I have read that in practice it doesn't work reliably. The only advice I can offer is "don't be holding any locks when you call fork", say sorry.
Related
I'm trying to check when the console is closed through the close button on Windows. I read about SetConsoleCtrlHandler and I thought I'd use that, but there's some cleanup I want to do in my main function. I'll make a small example describing what I want to do for my larger program.
BOOL CtrlHandler( DWORD fdwCtrlType )
{
switch( fdwCtrlType )
{
//Cleanup exit
case CTRL_CLOSE_EVENT:
bool* programIsOn = &???; //How do I pass the address to that variable in this function?
*programIsOn = false;
return( TRUE );
default:
return FALSE;
}
}
int main(){
MyObject obj = new MyObject();
bool programIsOn = true;
//How do I pass the address of programIsOn here?
if(!SetConsoleCtrlHandler( (PHANDLER_ROUTINE) CtrlHandler, TRUE )){
cout << "Could not set CtrlHandler. Exiting." << endl;
return 0;
}
while(programIsOn){
//...
}
//CLEANUP HERE
delete obj;
return 0;
}
I want to perform cleanup when my program closes via the console close event, however if I just close the console the main function doesn't terminate and is forced to stop. I thought of passing in programIsOn's address to the CtrlHandler callback but I have no idea how to do this without using a global variable.
TL;DR: Proper handling of this control signal is complicated. Don't bother with any 'clean-up' unless it's absolutely necessary.
The system creates a new thread (see the Remarks) in your application, which is then used to execute the handler function you registered. That immediately causes a few issues and forces you in a particular design direction.
Namely, your program suddenly became multi-threaded, with all the complications that brings. Just setting a 'program should stop' (global) boolean variable to true in the handler is not going to work; this has to be done in a thread-aware manner.
Another complication this handler brings is that the moment it returns the program is terminated as per a call to ExitProcess. This means that the handler should wait for the program to finish, again in a thread-aware manner. Queue the next complication, where the OS gives you only 10 seconds to respond to the handler before the program is terminated anyway.
The biggest issue here, I think, is that all these issues force your program to be designed in a very particular way that potentially permeates every nook and cranny of your code.
It's not necessary for your program to clean up any handles, objects, locks or memory it uses: these will all be cleaned up by Windows when your program exits.
Therefore, your clean-up code should consists solely of those operations that need to happen and otherwise wouldn't happen, such as write the end of a log file, delete temporary files, etc.
In fact, it is recommended to not perform such clean-up, as it only slows down the closing of the application and can be so hard to get right in 'unexpected termination' cases; The Old New Thing has a wonderful post about it that's also relevant to this situation.
There are two general choices here for the way to handle the remaining clean-up:
The handler routine does all the clean-up, or
the main application does all the clean-up.
Number 1 has the issue that it's very hard to determine what clean-up to perform (as this depends on where the main program is currently executing) and it's doing so 'while the engine is still running'. Number 2 means that every piece of code in the the main application needs to be aware of the possibility of termination and have short-circuit code to handle such.
So if you truly must, necessarily, absolutely, perform some additional clean-up, choose method 2. Add a global variable, preferably a std::atomic<bool> if C++11 is available to you, and use that to track whether or not the program should exit. Have the handler set it to true
// Shared global variable to track forced termination.
std::atomic<bool> programShouldExit = false;
// In the console handler:
BOOL WINAPI CtrlHandler( DWORD fdwCtrlType )
{
...
programShouldExit = true;
Sleep(10000); // Sleep for 10 seconds; after this returns the program will be terminated if it hasn't already.
}
// In the main application, regular checks should be made:
if (programShouldExit.load())
{
// Short-circuit execution, such as return from function, throw exception, etc.
}
Where you can pick your favourite short-circuiting method, for instance throwing an exception and using the RAII pattern to guard resources.
In the console handler, we sleep for as long as we think we can get away with (it doesn't really matter); the hope is that the main thread will have exited by then causing the application to exit. If not, either the sleep ends, the handler returns and the application is closed, or the OS became impatient and killed the process.
Conclusion: Don't bother with clean-up. Even if there is something you prefer to have done, such as deleting temporary files, I'd recommend you don't. It's truly not worth the hassle (but that's my opinion). If you really must, then use thread-safe means to notify the main thread that it must exit. Modify all longer-running code to handle the exit status and all other code to handle the failure of the longer-running code. Exceptions and RAII can be used to make this more manageable, for instance.
And this is why I feel that it's a very poor design choice, born from legacy code. Just being able to handle an 'exit request' requires you to jump through hoops.
I heard that "a modern operating system will clean up all threads created by the process on closing it" but when I return main(), I'm getting these errors:
1) This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.
2) terminate called without an active exception
My implementation looks like this (I'm writing now for example sorry for bad implementation):
void process(int id)
{
while(true) { std::this_thread::sleep_for(std::chrono::milliseconds(1); } }
}
int main()
{
std::thread thr1(process, 0);
std::thread thr2(process, 1);
//thr1.detach();
//thr2.detach();
return 0;
}
If I uncomment detach();s, there is no problem but my processing threads will be socket readers/writers and they will run infinitely (until main returns). So how to deal with it? What's wrong?
EDIT: Namely, I can't detach() every thread one-by-one because they will not be terminated normally (until the end). Oh and again, if I close my program from the DDOS window's X button, (my simple solution not works in this case) my detach(); functions being passed because program force-terminated and here is the error again :)
What happens in an application is not related to what the OS may do.
If a std::thread is destroyed, still having a joinable thread, the application calls std::terminate and that's what is showing up: http://en.cppreference.com/w/cpp/thread/thread/~thread`
With the c++11 threads, either you detach if you do not care on their completion time, or you care and need to join before the thread object is destroyed.
If my application runs out of memory, I would like to re-run it with changed parameters. I have malloc / new in various parts of the application, the sizes of which are not known in advance. I see two options:
Track all memory allocations and write a restarting procedure which deallocates all before re-running with changed parameters. (Of course, I free memory at the appropriate places if no errors occur)
Restarting the application (e.g., with WinExec() on Windows) and exiting
I am not thrilled by either solution. Did I miss an alternative maybe.
Thanks
You could embedd all the application functionality in a class. Then let it throw an expection when it runs out of memory. This exception would be catched by your application and then you could simply destroy the class, construct a new one and try again. All in one application in one run, no need for restarts. Of course this might not be so easy, depending on what your application does...
There is another option, one I have used in the past, however it requires having planned for it from the beginning, and it's not for the library-dependent programmer:
Create your own heap. It's a lot simpler to destroy a heap than to cleanup after yourself.
Doing so requires that your application is heap-aware. That means that all memory allocations have to go to that heap and not the default one. In C++ you can simply override the static new/delete operators which takes care of everything your code allocates, but you have to be VERY aware of how your libraries, even the standard library, use memory. It's not as simple as "never call a library method that allocates memory". You have to consider each library method on a case-by-case basis.
It sounds like you've already built your app and are looking for a shortcut to memory wiping. If that is the case, this will not help as you could never tack this kind of thing onto an already built application.
The wrapper-program (as proposed before) does not need to be a seperate executable. You could just fork, run your program and then test the return code of the child. This would have the additional benefit, that the operating system automatically reclaims the child's memory when it dies. (at least I think so)
Anyway, I imagined something like this (this is C, you might have to change the includes for C++):
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#define OUT_OF_MEMORY 99999 /* or whatever */
int main(void)
{
int pid, status;
fork_entry:
pid = fork();
if (pid == 0) {
/* child - call the main function of your program here */
} else if (pid > 0) {
/* parent (supervisor) */
wait(&status); /* waiting for the child to terminate */
/* see if child exited normally
(i.e. by calling exit(), _exit() or by returning from main()) */
if (WIFEXITED(status)) {
/* if so, we can get the status code */
if (WEXITSTATUS(status) == OUT_OF_MEMORY) {
/* change parameters */
goto fork_entry; /* forking again */
}
}
} else {
/* fork() error */
return 1;
}
return 0;
}
This might not be the most elegant solution/workaround/hack, but it's easy to do.
A way to accomplish this:
Define an exit status, perhaps like this:
static const int OUT_OF_MEMORY=9999;
Set up a new handler and have it do this:
exit(OUT_OF_MEMORY);
Then just wrap your program with another program that detects this
exit status. When it does then it can rerun the program.
Granted this is more of a workaround than a solution...
The wrapper program I mentioned above could be something like this:
static int special_code = 9999;
int main()
{
const char* command = "whatever";
int status = system(command);
while ( status == 9999 )
{
command = ...;
status = system(command);
}
return 0;
}
That's the basicness of it. I would use std::string instead of char* in production. I'd probably also have another condition for breaking out of the while loop, some maximum number of tries perhaps.
Whatever the case, I think the fork/exec route mentioned below is pretty solid, and I'm pretty sure a solution like it could be created for Windows using spawn and its brethren.
simplicity rules: just restart your app with different parameters.
it is very hard to either track down all allocs/deallocs and clean up the memory (just forget some minor blocks inside bigger chunks [fragmentation] and you still have problems to rerun the class), or to do introduce your own heap-management (very clever people have invested years to bring nedmalloc etc to live, do not fool yourself into the illusion this is an easy task).
so:
catch "out of memory" somehow (signals, or std::bad_alloc, or whatever)
create a new process of your app:
windows: CreateProcess() (you can just exit() your program after this, which cleans up all allocated resources for you)
unix: exec() (replaces the current process completely, so it "cleans up all the memory" for you)
done.
Be warned that on Linux, by default, your program can request more memory than the system has available. (This is done for a number of reasons, e.g. avoiding memory duplication when fork()ing a program into two with identical data, when most of the data will remain untouched.) Memory pages for this data won't be reserved by the system until you try to write in every page you've allocated.
Since there's no good way to report this (since any memory write can cause your system to run out memory), your process will be terminated by the out of memory process killer, and you won't have the information or opportunity for your process to restart itself with different parameters.
You can change the default by using the setrlimit system call, to to limit the RLIMIT_RSS which limits the total amount of memory your process can request. Only after you have done this will malloc return NULL or new throw a std::bad_alloc exception when you reach the limit that you have set.
Be aware that on a heavily loaded system, other processes can still contribute to a systemwide out of memory condition that could cause your program to be killed without malloc or new raising an error, but if you manage the system well, this can be avoided.
I'm writing a multi-threaded C++ program. I plan on killing threads. However, I am also using a ref-counted GC. I'm wondering if stack allocated objects get destructed when a thread gets killed.
The stack does not unwind when you 'kill' a thread.
Killing threads is not a robust way to operate - resources they have open, such as files, remain open until the process closes. Furthermore, if they hold open any locks at the time you close them, the lock likely remains locked. Remember, you are likely calling a lot of platform code you do not control and you can't always see these things.
The graceful robust way to close a thread is to interrupt it - typically it will poll to see if it's been told to close down periodically, or it's running a message loop and you send it a quit message.
I doubt it - pthread is a pure C api, so I doubt it would have any mechanism to unwind the stack of the thread.
It's not standardised to do this. It appears that some implementations do and some don't.
pthread_cancel() really should be avoided, if you can; it doesn't actually stop the thread until it hits a cancellation point, which is usually any other pthread_* call. In particular, on lots of platforms a cancel won't interrupt a blocking read.
#include<iostream>
#include<pthread.h>
class obj
{
public:
obj(){printf("constructor called\n");}
~obj(){printf("destructor called\n");}
};
void *runner(void *param)
{
printf("In the thread\n");
obj ob;
puts("sleep..");
sleep(4);
puts("woke up");
pthread_exit(0);
}
int main(int argc,char *argv[])
{
int i,n;
puts("testing pkill");
pthread_attr_t attr;
pthread_t tid;
//create child thread with default attributes
pthread_attr_init(&attr);
pthread_create(&tid,&attr,runner,0);
pthread_cancel(tid);
pthread_join(tid,NULL);//wait till finished
//the parent process outputs value
return 0;
}
Although not coinciding with the views above, the following code outputs
testing pkill
In the thread
constructor called
sleep..
destructor called
I have a C++ function that I'd like to call using execvp(), due to the way my program is organized.
Is this possible?
All of the exec variants including execvp() can only call complete programs visible in the filesystem. The good news is that if you want to call a function in your already loaded program, all you need is fork(). It will look something like this pseudo-code:
int pid = fork();
if (pid == 0) {
// Call your function here. This is a new process and any
// changes you make will not be reflected back into the parent
// variables. Be careful with files and shared resources like
// database connections.
_exit(0);
}
else if (pid == -1) {
// An error happened and the fork() failed. This is a very rare
// error, but you must handle it.
}
else {
// Wait for the child to finish. You can use a signal handler
// to catch it later if the child will take a long time.
waitpid(pid, ...);
}
excecvp() is meant ot start a program not a function. So you'll have to wrap that function into a compiled executable file and then have that file's main call your function.
Creating processes can be heavyweight. If you really only want to call your function in parallel why not using threads. There are many platform independent libraries available that have threading support for C++ like Boost, QT or ACE.
If you really need your function to be executed in another process you can use fork or vfork. vfork may not be available on every platform and it has it's drawbacks so make sure if you can use it. If not just use fork.