How can I measure the memory used by a child process after I call fork and exec? Basically I want to be able to write code that corresponds to the following
if (!fork()) {
// run child process
exec();
} else {
while (child active) {
print memory used by child
}
}
There are two things that I do not know here, how can I see if the child process has finished running? Will I have to use some sort of process level mutual exclusion here? If yes then what is a structure I can use? Can I just use the OS filesystem for this purpose?
Also I was looking at the answer at this link Differences between fork and exec, in paragraph 8 the author says copy on write is useful when process calls fork without calling exec. But isn't this true more in the case when the parent calls fork and does not call exec? When the parent calls exec the virtual address space of the child is wiped out and replaced with the one resulting from the new program loaded into memory correct?
Thank you!
Regarding the above comment chain which I evidently can't reply to because I don't have 50 rep:
The return value of fork in the parent if successful is the PID of the child. You should probably save the return value so you can 1. wait on the correct child (if you have more than one), and 2. see if fork fails (in which case you probably don't want to loop until the child exits ).
You could also use signals to figure out when the child dies instead of continuously trying to wait with the WNOHANG option. The process will send SIGCHLD to the parent when it terminates (or stops) and if it died then you can wait on it with waitpid and stop your loop. see:
man 7 signal
man 2 sigaction
for more information on this.
regarding memory usage, it seems you either want /proc/[pid]/statm or /proc/[pid]/stat.
man 5 proc will give you all the information about what is in those files.
Related
I'm writing a shell in cpp and I was hoping to get some advice. I have a command that will do an exec in the background, and I'm trying to keep track of which background processes are still running. I thought maybe I could keep track of the PID and do a string find on /proc/, but it seems to stay longer than it should. I'm testing it by using the sleep command, but it seems to always linger around wherever I look long after it should've finished. I'm probably just not doing the right thing to see if it is still running though.
Thanks in advance for any help.
Assuming you are spawning off the child process via fork() or forkpty(), one reasonably good way to track the child process's condition is to have the parent process create a connected-socket-pair (e.g. via socketpair()) before forking, and have the child process call dup2() to make one end of that socket-pair its stdin/stdout/stderr file descriptor, e.g.:
// Note: error-checking has been removed for clarity
int temp[2];
(void) socketpair(AF_UNIX, SOCK_STREAM, 0, temp);
pid_t pid = fork();
if (pid == 0)
{
// We are the child process!
(void) dup2(temp[1], STDIN_FILENO);
(void) dup2(temp[1], STDOUT_FILENO);
(void) dup2(temp[1], STDERR_FILENO);
// call exec() here...
}
The benefit of this is that now the parent process has a file descriptor (temp[0]) that is connected to the stdin, stdout, and stderr of the child process, and the parent process can select() on that descriptor to find out whenever the child process has written text to its stderr or stdout streams, and can then read() on that file descriptor to find out what the child process wrote (useful if you want to then display that text to the user, or if not you can just throw the read text away), and most importantly, it will know when the child process has closed its stderr and stdout streams, because then the parent process's next call to read() on that file descriptor will indicate 0 aka EOF.
Since the OS will automatically close the child process's streams whenever it exits for any reason (including crashing), this is a pretty reliable way to get notified that the child process has gone away.
The only potential gotcha is that the child process could (for whatever reason) manually call close(STDOUT_FILENO) and close(STDERR_FILENO), and yet still remain running; in that case the parent process would see the socket-pair connection closing as usual, and wrongly think the child process had gone away when in fact it hadn't. Fortunately it's pretty rare for a program to do that, so unless you need to be super-robust you can probably ignore that corner case.
On a POSIX-like system, after you create any child processes using fork, you should clean up those child processes by calling wait or waitpid from the parent process. The name "wait" is used because the functions are most commonly used when the parent has nothing to do until a child exits or is killed, but waitpid can also be used (by passing WNOHANG) to check on whether a child process is finished without making the parent process wait.
Note that at least on Linux, when a child process has exited or been killed but the parent process has not "waited" for the child, the kernel keeps some information about the child process in memory, as a "zombie process". This is done so that a later "wait" can correctly fetch the information about the child's exit code or fatal signal. These zombie processes do have entries in /proc, which may be why you see a child "stay longer than it should", if that's how you were checking.
In my process I need to start/restart another process.
Currently I use a thread with a tiny stack size and the following code:
void startAndMonitorA()
{
while(true)
{
system("myProcess");
LOG("myProcess crashed");
usleep(1000 * 1000);
}
}
I feel like that's not best practice. I have no idea about the resources the std::system() call is blocking or wasting. I'm on an embedded Linux - so in general I try to care about resources.
One problematic piece is restarting immediately: if the child process fails to start that is going to cause 100% CPU usage. It may be a transient error in the child process (e.g. cannot connect to a server). It may be a good idea to add a least one second pause before trying to restart.
What system call does on Linux is:
Sets up signals SIGINT and SIGQUIT to be ignored.
Blocks signal SIGCHLD.
fork()
Child process calls exec() shell, passing the command line to the shell.
Parent process calls waitpid() that blocks the thread till the child process terminates.
Parent process restores its signal dispositions.
If you were to re-implement the functionality of system you would probably omit step 5 (along with steps 1, 2 and 6) to avoid blocking the thread and rely on SIGCHLD to get notified when the child process has terminated and needs to be restarted.
In other words, the bare minimum would be to set up a signal handler for SIGCHLD and call fork and exec.
The code as shown would be adequate for most circumstances. If you really care about resource usage, you should be aware that you are starting (and keeping around) a thread for each process you are monitoring. If your program has an event loop anyway, that kind of thing can be avoided at the cost of some additional effort (and an increase in complexity).
Implementing this would entail the following:
Instead of calling system(), use fork() and exec() to start the external program. Store its PID in a global table.
Set a SIGCHLD handler that notifies the event loop of the exit of a child, e.g. by writing a byte to a pipe monitored by the event loop.
When a child exits, run waitpid with the WNOHANG flag in a loop that runs for as long as there are children to reap. waitpid() will return the PID of the child that exited, so that you know to remove its PID from the table, and to schedule a timeout that restarts it.
I need to execute processes with still being in control of each process.
I want to create a class which stores the threads or pids or what ever is necessary to do so.
I currently have a program which executes one external application with the C function execvp and also loads the environment from a shell script. So my current program is blocking. But I need to be able to keep it freely running and only by time I terminate a currently running or start a new external application.
My current approach would be to create a thread, which uses the execve function. But then the thread would be blocking as far as I can see.
The code which might be in the thread (with variables then):
char *argv[] = { "/bin/bash", "-c", "myApplication", 0 };
execve(argv[0], &argv[0], environment.data());
The applications called are probably not fixed in the code their names will be given by an external setup file, including parameters.
Now my actual question, is there a better way to "manage" external applications like that in c++? Some ready solution (class, library)? And if not how do I terminate the thread if this is the actual way. Using the terminate call is said to be bad practice, that's what I often read.
I hope this is now specific enough for the forum, because I do not know how to get more specific anymore. If you need more hints what I want to create here, feel free to ask in the comments.
Update:
to DBus & others:
Additional information I do not wrote all of the processes I want to start!
So it will be used to start 3rd party applications, which even if I have the code, do not want to change.
You want to fork() before you exec. fork() is a function that creates a new process identical to the original caller of fork() running as a subprocess. The difference is that the parent process gets the child's pid as a return value and the child gets 0. The gist of what you want to do is this:
pid_t pid = fork();
if( pid == 0 )
{
// we're the child process
char *argv[] = { "/bin/bash", "-c", "myApplication", 0 };
int rc = execve(argv[0], &argv[0], environment.data());
// execve only returns if there was an error
// check 'errno' and handle it here
}
else if ( pid < 0 )
{
// pid is less than zero, we didn't successfully fork,
// there is no child process.
throw "error message";
}
// do whatever processing the parent does
More info is here. The kill() function isn't bad practice per se, if you want to quickly and gracefully end the subprocess you can write signal handlers in it, but you should be using something like dbus or zeromq to do proper interprocess communication. You want to tell the program to do something, not just tell it to die (usually what you want it to do if you're killing it).
NEVER USE execv functions in threads because the execve() system call overlays the current process image with a new process image.
The correct pattern if fork-exec or better vfork-exec. Extract from the manpage:
The vfork() system call can be used to create new processes without fully
copying the address space of the old process, which is horrendously inefficient in a paged environment. It is useful when the purpose of fork(2)
would have been to create a new system context for an execve(2). The
vfork() system call differs from fork(2) in that the child borrows the
parent's memory and thread of control until a call to execve(2) or an
exit (either by a call to _exit(2) or abnormally). The parent process is
suspended while the child is using its resources.
Using vfork shortly followed with execve, you avoid the copy of the original process image, and do not erase if with the new process, so the original process has the pid of its child and cat control it, look whether it has ended, send it signals and so on.
I have a program written in C++ intended to run on a Linux OS. Ignoring much of the program, it boils down to this - it starts X number of executables after some amount of time (for simplicity sake, let's use 5 seconds).
Currently, I'm using system(path/to/executable/executable_name) to do the actual starting of the executable(s) and that works just fine for getting the executable(s) to start.
I'm also trying to maintain a status for each executable (for simplicity sake again, let's just say the status is either "UP" or "DOWN" (running or not running)). I have been able to accomplish this...somewhat...
Backing up just a tad, when my program is told to start the executable(s), the logic looks something like this:
pid = fork()
if (pid < 0) exit 0; //fork failed
if (pid == 0) {
system(path/to/executable/executable_name)
set executable's status to DOWN
} else {
verify executable started
set executable's status to UP
}
Herein lies my problem. fork() causes a child process to be spawned, which is what I thought I needed in order for the original process to continue starting additional executables. I don't want to wait for an executable to stop in order to start another.
However, the executable starts in another child process...which is separate from the parent process... and if I try to set the executable's status to DOWN in the child process when system returns, the parent process does not know about it...
I have a few ideas of what I might need to do:
use threads instead of fork: create a new thread to call system, but would the parent/main thread know about the new thread changing the status of the executable?
use fork and exec: but I'm not sure that would be any better than what I already have (I've read the man pages for fork and exec but I guess I'm still a little fuzzy on how to best utilize exec)
Any suggestions?
EDIT 1
I thought I'd better give a little more context for the logic:
void startAll() {
for each 'executable'
call startExecutable(executable_name)
}
...
void startExecutable (executable_name) {
pid = fork()
if (pid < 0) exit 0; //fork failed
if (pid == 0) {
system(path/to/executable/executable_name)
set executable's status to DOWN
exit (1); <-- this is because once the child process's system returns, I don't want it to return to the above loop and start starting executables
} else {
verify executable started
set executable's status to UP
}
}
EDIT 2
As mentioned at the beginning, this is assuming a simplified setup (a first run if you will). The plan is to handle not just an "UP" or "DOWN" state, but also a third state to handle sending a message to the executables my program has started - "STANDBY." I initially left this piece out to avoid complicating the explanation but I now see that it is imperitive to include.
You need to understand what exactly is happening when you fork. What you're doing is creating a subprocess that's an exact clone of the forking process. All variables currently in memory are copied exactly, and the subprocess has access to all of those copies of all of those variables.
But they're copies, so as you've noticed, fork and exec/system does not on its own handle inter-process communication (IPC). Setting a memory value in one of the processes doesn't alter that variable in any other process, including its parent, because the memory spaces are different.
Also, system is very similar to exec, but gives you much less control over the file descriptors and execution environment. You're effectively already doing a fork and exec, which is what you should be doing.
When you fork properly (as you do in your example), you now have two processes, and neither one is waiting for the other - they just run in completely different codepaths. What you basically want is to have the parent do nothing but sit around waiting for new programs to open, and occassionally check the status of the kids, while the kids run and play as long as they want.
There are IPC solutions such as pipes and message FIFO queues, but that's excessive in your case. In your case, you're just looking for process management. The parent is given the pid of the children. Save it and use it. You can call waitpid to wait for the child to end, but you don't want that. You just want the parent to check the status of the child. One way to do that is check if kill(childPid,0) == 0. If not, then the pid has exited, i.e. it's no longer running. You can also check /proc/childPid for all sorts of information.
If your status is less simple than your question implied, you'll want to look into piping after forking and execing. Otherwise, all you need is process monitoring.
Based on your EDIT 2, you're still within the domain of process management, instead of IPC. The kill command sends a signal to a process (if the command is non-0). What you're looking for is to have the parent kill(childPid, SIGTSTP). On the child side, you just need to make a signal handler, using the signal command. Among many other references, see http://www.yolinux.com/TUTORIALS/C++Signals.html. Basically, you want:
void sigTempStopHandler(int signum) { /* ... */ }
signal(SIGTSTP, sigTempStopHandler);
to be executed in the child code. The parent, of course, would know when this state is sent, so can change the status. You can use other signals for resuming when necessary.
When to pipe vs. signal:
Piping is the most robust IPC you could use - it lets you send any amount of data from one process to another, and can be in whichever direction you want. If you want your parent to send "You've been a very bad boy" to the child, it can, and the child can send "But I'll choose your nursing home one day" to the parent. (Less flippantly, you can pass any data, whether text or binary from one process to another - including objects that you serialize, or just the raw data for objects if it doesn't depend on memory, e.g. an int.)
So far, what you've described is sending simple command structures from the parent to the child, and kill is perfect for that. The child could send signals almost as easily - except that it would need to know the parent's pid to do that. (Not hard to do - before forking, save the pid: int pid = getPid();, now the child knows the parent.) Signals have no data, they're just very raw events, but so far, that sounds like all you're looking for.
I wrote a program that forks some processes with fork(). I want to kill all child- and the mother process if there is an error. If I use exit(EXIT_FAILURE) only the child process is killed.
I am thinking about a system("killall [program_name]") but there must be a better way...
Thank you all!
Lennart
Under UNIX, send SIGTERM, or SIGABRT, or SIGPIPE or sth. alike to the mother process. This signal will then be propagated to all clients automatically, if they do not explicitely block or ignore it.
Use getppid() to get the PID to send the signal to, and kill() to send the signal.
getppid() returns the process ID of
the parent of the calling process.
The kill() system call can be used to send any signal to any process group or process.
Remarks:
1. Using system is evil. Use internal functions to send signals.
2. killall would be even more evil. Consider several instances of your program running at once.
See How to make child process die after parent exits?
On Linux there's a prctl() call which is explicitly designed to send a signal to all of a process's children when the parent dies for whatever reason.
I need to check and can't do it where I am at the second, but I'm really not sure that ypnos' assertion about SIGPIPE, SIGTERM and SIGABRT being propagated to all children is correct.
However if you use kill(-ppid) (note the minus sign) then so long as the children are still in the parent process's process group then the kernel will deliver any signal to all of the children.
If your mother process is not started by the command line, it may not be the
process group leader, like as a deamon.
To ensure that your mother process is the process group leader, call setsid() during
your process initialization.
Then in your child process, if you want to cause all the processes to exit:
pgid = getpgid();
kill(pgid, 15);
You can also do tricks, like telling all your siblings to suspend:
kill(pgid, 20);
And resume:
kill(pgid, 18);
Consider suicidal approach - setting up an alarm() at the beginning of the process (both parent and child) with some positive number of seconds. If computation completes within that time and "there is no error", call alarm(0) to cancel the timer; otherwise the SIGALRM will kill the process (assuming you're not explicitly catching or ignoring it.)
Well, make a case against this instead of just down-voting :)