Cleaning up children processes asynchronously - c++

This is an example from <Advanced Linux Programming>, chapter 3.4.4. The programs fork() and exec() a child process. Instead of waiting for the termination of the process, I want the parent process to clean up the children process (otherwise the children process will become a zombie process) asynchronously. The can be done using the signal SIGCHLD. By setting up the signal_handler we can make the clean-up work done when the child process ends. And the code the following:
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <unistd.h>
#include <sys/wait.h>
#include <signal.h>
#include <string.h>
int spawn(char *program, char **arg_list){
pid_t child_pid;
child_pid = fork();
if(child_pid == 0){ // it is the child process
execvp(program, arg_list);
fprintf(stderr, "A error occured in execvp\n");
return 0;
}
else{
return child_pid;
}
}
int child_exit_status;
void clean_up_child_process (int signal_number){
int status;
wait(&status);
child_exit_status = status; // restore the exit status in a global variable
printf("Cleaning child process is taken care of by SIGCHLD.\n");
};
int main()
{
/* Handle SIGCHLD by calling clean_up_process; */
struct sigaction sigchld_action;
memset(&sigchld_action, 0, sizeof(sigchld_action));
sigchld_action.sa_handler = &clean_up_child_process;
sigaction(SIGCHLD, &sigchld_action, NULL);
int child_status;
char *arg_list[] = { //deprecated conversion from string constant to char*
"ls",
"-la",
".",
NULL
};
spawn("ls", arg_list);
return 0;
}
However, When I run the program in the terminal, the parent process never ends. And it seems that it doesn't execute the function clean_up_child_process (since it doesn't print out "Cleaning child process is taken care of by SIGCHLD."). What's the problem with this snippet of code?

The parent process immediately returns from main() after the child pid is returned from fork(), it never has the opportunity to wait for the child to terminate.

for GNU/Linux users
I already read this book. Although the book talked about this mechanism as a:
quote from 3.4.4 page 59 of the book:
A more elegant solution is to notify the parent process when a child terminates.
but it just said that you can use sigaction to handle this situation.
Here is a complete example of how to handle processes in this way.
First why do ever we use this mechanism? Well, since we do not want to synchronize all processes together.
real example
Imagine that you have 10 .mp4 files and you want to convert them to .mp3 files. Well, I junior user does this:
ffmpeg -i 01.mp4 01.mp3
and repeats this command 10 times. A little higher users does this:
ls *.mp4 | xargs -I xxx ffmpeg -i xxx xxx.mp3
This time, this command pipes all 10 mp4 files per line, each one-by-one to xargs and then they one by one is converted to mp3.
But I senior user does this:
ls *.mp4 | xargs -I xxx -P 0 ffmpeg -i xxx xxx.mp3
and this means if I have 10 files, create 10 processes and run them simultaneously. And there is BIG different. In the two previous command we had only 1 process; it was created then terminated and then continued to another one. But with the help of -P 0 option, we create 10 processes at the same time and in fact 10 ffmpeg commands are running.
Now the purpose of cleaning up children asynchronously becomes cleaner. In fact we want to run some new processes but the order of those process and maybe the exit status of them is not matter for us. In this way we can run them as fast as possible and reduce the time.
First you can see man sigaction for any more details you want.
Second seeing this signal number by:
T ❱ kill -l | grep SIGCHLD
16) SIGSTKFLT 17) SIGCHLD 18) SIGCONT 19) SIGSTOP 20) SIGTSTP
sample code
objective: using the SIGCHLD to clean up child process
#include <stdio.h>
#include <stdlib.h>
#include <signal.h>
#include <string.h>
#include <wait.h>
#include <unistd.h>
sig_atomic_t signal_counter;
void signal_handler( int signal_number )
{
++signal_counter;
int wait_status;
pid_t return_pid = wait( &wait_status );
if( return_pid == -1 )
{
perror( "wait()" );
}
if( WIFEXITED( wait_status ) )
{
printf ( "job [ %d ] | pid: %d | exit status: %d\n",signal_counter, return_pid, WEXITSTATUS( wait_status ) );
}
else
{
printf( "exit abnormally\n" );
}
fprintf( stderr, "the signal %d was received\n", signal_number );
}
int main()
{
// now instead of signal function we want to use sigaction
struct sigaction siac;
// zero it
memset( &siac, 0, sizeof( struct sigaction ) );
siac.sa_handler = signal_handler;
sigaction( SIGCHLD, &siac, NULL );
pid_t child_pid;
ssize_t read_bytes = 0;
size_t length = 0;
char* line = NULL;
char* sleep_argument[ 5 ] = { "3", "4", "5", "7", "9" };
int counter = 0;
while( counter <= 5 )
{
if( counter == 5 )
{
while( counter-- )
{
pause();
}
break;
}
child_pid = fork();
// on failure fork() returns -1
if( child_pid == -1 )
{
perror( "fork()" );
exit( 1 );
}
// for child process fork() returns 0
if( child_pid == 0 ){
execlp( "sleep", "sleep", sleep_argument[ counter ], NULL );
}
++counter;
}
fprintf( stderr, "signal counter %d\n", signal_counter );
// the main return value
return 0;
}
This is what the sample code does:
create 5 child processes
then goes to inner-while loop and pauses for receiving a signal. See man pause
then when a child terminates, parent process wakes up and calls signal_handler function
continue up to the last one: sleep 9
output: (17 means SIGCHLD)
ALP ❱ ./a.out
job [ 1 ] | pid: 14864 | exit status: 0
the signal 17 was received
job [ 2 ] | pid: 14865 | exit status: 0
the signal 17 was received
job [ 3 ] | pid: 14866 | exit status: 0
the signal 17 was received
job [ 4 ] | pid: 14867 | exit status: 0
the signal 17 was received
job [ 5 ] | pid: 14868 | exit status: 0
the signal 17 was received
signal counter 5
when you run this sample code, on the other terminal try this:
ALP ❱ ps -o time,pid,ppid,cmd --forest -g $(pgrep -x bash)
TIME PID PPID CMD
00:00:00 5204 2738 /bin/bash
00:00:00 2742 2738 /bin/bash
00:00:00 4696 2742 \_ redshift
00:00:00 14863 2742 \_ ./a.out
00:00:00 14864 14863 \_ sleep 3
00:00:00 14865 14863 \_ sleep 4
00:00:00 14866 14863 \_ sleep 5
00:00:00 14867 14863 \_ sleep 7
00:00:00 14868 14863 \_ sleep 9
As you can see a.out process has 5 children. And They are running simultaneously. Then whenever each of them terminates, kernel sends the signal SIGCHLD to their parent that is: a.out
NOTE
If we do not use pause or any mechanism so that the parent can wait for its children, then we will abandon the created processes and the upstart (= on Ubuntu or init) becomes parent of them. You can try it if you remove pause()

I'm using Mac, so my answer may be not quite relevant, but still. I compile without any options, so executable name is a.out.
I have the same experience with the console (the process doesn't seem to terminate), but I noticed that it's just terminal glitch, because you actually can just press Enter and your command line will be back, and actually ps executed from other terminal window doesn't show a.out, nor ls which it launched.
Also if I run ./a.out >/dev/null it finishes immediately.
So the point of the above is that everything actually terminates, just the terminal freezes for some reason.
Next, why it never prints Cleaning child process is taken care of by SIGCHLD.. Simply because the parent process terminates before child. The SIGCHLD signal can't be delivered to already terminated process, so the handler is never invoked.
In the book it's said that the parent process contiunes to do some other things, and if it really does then everything works fine, for example if you add sleep(1) after spawn().

Related

Execute binary from C++ without shell

Is there a way to execute a binary from my C++ program without a shell? Whenever I use system my command gets run via a shell.
You need to:
fork the process
call one of the "exec" functions in the child process
(if necessary) wait for it to stop
For example, this program runs ls.
#include <iostream>
#include <unistd.h>
#include <sys/wait.h>
// for example, let's "ls"
int ls(const char *dir) {
int pid, status;
// first we fork the process
if (pid = fork()) {
// pid != 0: this is the parent process (i.e. our process)
waitpid(pid, &status, 0); // wait for the child to exit
} else {
/* pid == 0: this is the child process. now let's load the
"ls" program into this process and run it */
const char executable[] = "/bin/ls";
// load it. there are more exec__ functions, try 'man 3 exec'
// execl takes the arguments as parameters. execv takes them as an array
// this is execl though, so:
// exec argv[0] argv[1] end
execl(executable, executable, dir, NULL);
/* exec does not return unless the program couldn't be started.
when the child process stops, the waitpid() above will return.
*/
}
return status; // this is the parent process again.
}
int main() {
std::cout << "ls'ing /" << std::endl;
std::cout << "returned: " << ls("/") << std::endl;
return 0;
}
And the output is:
ls'ing /
bin dev home lib lib64 media opt root sbin srv tmp var
boot etc initrd.img lib32 lost+found mnt proc run selinux sys usr vmlinuz
returned: 0
I used popen, fgets and pclose functions to execute external command line program and redirect its output.

Is it possible a kill a command launched using system api in C? If not any alternatives?

I am launching a command using system api (I am ok with using this api with C/C++). The command I pass may hang at times and hence I would like to kill after certain timeout.
Currently I am using it as:
system("COMMAND");
I want to use it something like this:
Run a command using a system independent API (I don't want to use CreateProcess since it is for Windows only) Kill the process if it does not exit after 'X' Minutes.
Since system() is a platform-specific call, there cannot be a platform-independent way of solving your problem. However, system() is a POSIX call, so if it is supported on any given platform, the rest of the POSIX API should be as well. So, one way to solve your problem is to use fork() and kill().
There is a complication in that system() invokes a shell, which will probably spawn other processes, and I presume you want to kill all of them, so one way to do that is to use a process group. The basic idea is use fork() to create another process, place it in its own process group, and kill that group if it doesn't exit after a certain time.
A simple example - the program forks; the child process sets its own process group to be the same as its process ID, and uses system() to spawn an endless loop. The parent process waits 10 seconds then kills the process group, using the negative value of the child process PID. This will kill the forked process and any children of that process (unless they have changed their process group.)
Since the parent process is in a different group, the kill() has no effect on it.
#include <unistd.h>
#include <stdlib.h>
#include <signal.h>
#include <stdio.h>
int main() {
pid_t pid;
pid = fork();
if(pid == 0) { // child process
setpgid(getpid(), getpid());
system("while true ; do echo xx ; sleep 5; done");
} else { // parent process
sleep(10);
printf("Sleep returned\n");
kill(-pid, SIGKILL);
printf("killed process group %d\n", pid);
}
exit(0);
}
There is no standard, cross-platform system API. The hint is that they are system APIs! We're actually "lucky" that we get system, but we don't get anything other than that.
You could try to find some third-party abstraction.
Check below C++ thread based attempt for linux. (not tested)
#include <iostream>
#include <string>
#include <thread>
#include <stdio.h>
using namespace std;
// execute system command and get output
// http://stackoverflow.com/questions/478898/how-to-execute-a-command-and-get-output-of-command-within-c
std::string exec(const char* cmd) {
FILE* pipe = popen(cmd, "r");
if (!pipe) return "ERROR";
char buffer[128];
std::string result = "";
while(!feof(pipe)) {
if(fgets(buffer, 128, pipe) != NULL)
result += buffer;
}
pclose(pipe);
return result;
}
void system_task(string& cmd){
exec(cmd.c_str());
}
int main(){
// system commad that takes time
string command = "find /";
// run the command in a separate thread
std::thread t1(system_task, std::ref(command));
// gives some time for the system task
std::this_thread::sleep_for(chrono::milliseconds(200));
// get the process id of the system task
string query_command = "pgrep -u $LOGNAME " + command;
string process_id = exec(query_command.c_str());
// kill system task
cout << "killing process " << process_id << "..." << endl;
string kill_command = "kill " + process_id;
exec(kill_command.c_str());
if (t1.joinable())
t1.join();
cout << "continue work on main thread" << endl;
return 0;
}
I had a similar problem, in a Qt/QML development: I want to start a bash command, while continuing to process events on the Qt Loop, and killing the bash command if it takes too long.
I came up with the following class that I'm sharing here (see below), in hope it may be of some use to people with a similar problem.
Instead of calling a 'kill' command, I call a cleanupCommand supplied by the developper. Example: if I'm to call myscript.sh and want to check that it won't last run for more than 10 seconds, I'll call it the following way:
SystemWithTimeout systemWithTimeout("myScript.sh", 10, "killall myScript.sh");
systemWithTimeout.start();
Code:
class SystemWithTimeout {
private:
bool m_childFinished = false ;
QString m_childCommand ;
int m_seconds ;
QString m_cleanupCmd ;
int m_period;
void startChild(void) {
int rc = system(m_childCommand.toUtf8().data());
if (rc != 0) SYSLOG(LOG_NOTICE, "Error SystemWithTimeout startChild: system returned %d", rc);
m_childFinished = true ;
}
public:
SystemWithTimeout(QString cmd, int seconds, QString cleanupCmd)
: m_childFinished {false}, m_childCommand {cmd}, m_seconds {seconds}, m_cleanupCmd {cleanupCmd}
{ m_period = 200; }
void setPeriod(int period) {m_period = period;}
void start(void) ;
};
void SystemWithTimeout::start(void)
{
m_childFinished = false ; // re-arm the boolean for 2nd and later calls to 'start'
qDebug()<<"systemWithTimeout"<<m_childCommand<<m_seconds;
QTime dieTime= QTime::currentTime().addSecs(m_seconds);
std::thread child(&SystemWithTimeout::startChild, this);
child.detach();
while (!m_childFinished && QTime::currentTime() < dieTime)
{
QTime then = QTime::currentTime();
QCoreApplication::processEvents(QEventLoop::AllEvents, m_period); // Process events during up to m_period ms (default: 200ms)
QTime now = QTime::currentTime();
int waitTime = m_period-(then.msecsTo(now)) ;
QThread::msleep(waitTime); // wait for the remaning of the 200 ms before looping again.
}
if (!m_childFinished)
{
SYSLOG(LOG_NOTICE, "Killing command <%s> after timeout reached (%d seconds)", m_childCommand.toUtf8().data(), m_seconds);
int rc = system(m_cleanupCmd.toUtf8().data());
if (rc != 0) SYSLOG(LOG_NOTICE, "Error SystemWithTimeout 164: system returned %d", rc);
m_childFinished = true ;
}
}
I do not know any portable way to do that in C nor C++ languages. As you ask for alternatives, I know it is possible in other languages. For example in Python, it is possible using the subprocess module.
import subprocess
cmd = subprocess.Popen("COMMAND", shell = True)
You can then test if COMMAND has ended with
if cmd.poll() is not None:
# cmd has finished
and you can kill it with :
cmd.terminate()
Even if you prefere to use C language, you should read the documentation for subprocess module because it explains that internally it uses CreateProcess on Windows and os.execvp on Posix systems to start the command, and it uses TerminateProcess on Windows and SIG_TERM on Posix to stop it.

Launching a service from c++ with execv

I'm trying to launch a linux service from a c++ and I do it successfully but one of my process is marked as "defunct" and I don't want that my parent process dies.
My code is (testRip.cpp):
int main()
{
char* zebraArg[2];
zebraArg[0] = (char *)"zebra";
zebraArg[1] = (char *)"restart";
char* ripdArg[2];
ripdArg[0] = (char *)"ripd";
ripdArg[1] = (char *)"restart";
pid_t ripPid;
pid_t zebraPid;
zebraPid = fork();
if(zebraPid == 0)
{
int32_t iExecvRes = 0;
iExecvRes = execv("/etc/init.d/zebra", zebraArg);
return 0;
if(iExecvRes == -1)
{
::syslog((LOG_LOCAL0 | LOG_ERR),
"zebra process failed \n");
}
}
else
{
while(1)
{
::syslog((LOG_LOCAL0 | LOG_ERR),
"running\n");
sleep(2);
}
}
}
The exit of ps -e command is:
9411 pts/1 00:00:00 testRip
9412 pts/1 00:00:00 testRip <defunct>
9433 ? 00:00:00 zebra
The /etc/init.d/zebra launches the service as daemon or something like that so I think this is the trick but:
Why there are 3 processes and one of them is marked as defunct?
What is wrong in my code?
How can I fix it?
Thanks in advance.
To remove zombies you the parent process must wait() its child or dies. If you need to make a non blocking wait() look at waitpid() with W_NOHANG flag.
Correctly forking a daemon process is hard in Unix and Linux because there are a lot of details to get right, and order is also important. I would suspect a combination of open file descriptors and not detatching the controlling terminal, in this case.
I would strongly suggest using a well-debugged implementation from another program - one of the reduced-functionality command line shells such as rsh or ksh may be a good choice, rather than trying to bake your own version.

fork() 2 children with pipeline, error when wait() for both

I have the following code fork()'s 2 children from a common parent and implements a pipeline between them. When I call the wait() function in the parent once only the program runs perfectly. However if I try to call the wait() function twice (to reap from both the children), the program does nothing and must be force exited.
Can someone tell me why I can't wait for both children here?
int main()
{
int status;
int pipeline[2];
pipe(pipeline);
pid_t pid_A, pid_B;
if( !(pid_A = fork()) )
{
dup2(pipeline[1], 1);
close(pipeline[0]);
close(pipeline[1]);
execl("/bin/ls", "ls", 0);
}
if( !(pid_B = fork()) )
{
dup2(pipeline[0], 0);
close(pipeline[0]);
close(pipeline[1]);
execl("/usr/bin/wc", "wc", 0);
}
wait(&status);
wait(&status);
}
You need to close both ends of the pipe in the parent after you fork the children. The problem is that output of ls is going to the parent, and the wc is waiting for input. So the first wait cleans up the ls, but the second is waiting for wc which is blocked on a pipe that's not receiving data.
Process B (wc) does not terminate until it receives end-of-file on its input stream. The other end of the pipe is shared as both the output stream of process A, and as pipeline[1] in the parent process, so you will need to close(pipeline[1]) in the parent process before waiting for process B.

Persistent IPC between Bash script and C++

Problem:
There is a C application that calls a Bash script each time an event happens. And there is also a C++ application that needs to track down those events. The C++ application is driven by a select() event loop. What would be the most simplest IPC to implement between Bash script and C++ application?
C Application ---Each time calls Bash script---> Bash application ---???---> C++ Application
Few solutions that came into my mind:
To use TCP networking sockets, but this would mean that select will have to handle events for both Listening and Actual sockets
To use Named pipes, but once the bash script terminates then the other end of the pipe is closed as well
Is there something simpler that would allow me to use only one File Descriptor in select()?
Unix datagram or UDP socket would do. The bash script would just send a datagram to that socket (you may need a helper program that does sendmsg() or sendto() on that socket, such as socat or netcat/nc). The receiver does not need to accept connections for datagram sockets, once it is ready for read there must be a datagram waiting. Subject to datagram length restrictions.
I would do it with a unnamed pipe(). Remember: In UNIX file descriptors remain open after fork() and execve() in both processes! So you can use pipe() to get a pair of file descriptors and then write into the fd using bash's echo >&FD where FD is the file descriptor number.
That is very straight forward, easy and uses less resources than anything else I assume. The use of select() is no problem, just do not block on read() is I do in my sample but select() on pfds[0].
Sample program (spawns 10 bash processes which send 'hello work, my pid: XXX', waiting 1s between spawning the processes. The sample only uses one pipe for all the children. I changed it that way because the author asked about that. In practice, I would NOT recommend it (see the note below the sample)):
#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <assert.h>
int main(int argc, char **argv) {
int pfds[2];
pid_t p;
assert(0 == pipe(pfds));
p = fork();
if (p == 0) {
unsigned int i;
char str_fd[3]; char *env[] = {NULL};
char *args[] = { "/bin/bash", "-c", "echo >&$1 hello world, my pid: $$"
, "-s", str_fd, NULL};
snprintf(str_fd, 3, "%d", pfds[1]);
str_fd[2] = 0;
for (i = 0; i < 10; i++) {
p = fork();
if(0 == p) {
assert(0 ==
execve( "/bin/bash", (char *const*)args
, (char *const*)env));
} else if (0 > p) {
perror("fork");
exit(1);
} else {
wait(NULL);
}
sleep(1);
}
} else if(p > 0) {
char *buf = malloc(100);
ssize_t sz;
printf("fd is %d, <hit Ctrl+C to exit>\n", pfds[1]);
while(0 < ( sz = read(pfds[0], buf, 100))) {
buf[99] = 0;
printf("received: '%s'\n", buf);
}
free(buf);
if (0 == sz) {
fprintf(stderr, "EOF!");
} else {
perror("read from bash failed");
}
wait(NULL);
} else {
perror("fork failed");
exit(1);
}
return 0;
}
sample program output:
$ gcc test.c && ./a.out
fd is 4, <hit Ctrl+C to exit>
received: 'hello world, my pid: 779
'
received: 'hello world, my pid: 780
'
received: 'hello world, my pid: 781
'
received: 'hello world, my pid: 782
'
received: 'hello world, my pid: 783
'
received: 'hello world, my pid: 784
'
received: 'hello world, my pid: 785
'
received: 'hello world, my pid: 786
'
received: 'hello world, my pid: 787
'
received: 'hello world, my pid: 788
'
works, bashs send 'hello world, my pid: XXX\n' to parent process all using one pipe :-).
Nevertheless that seems to work as the demo program shows (should be ok using the POSIX semantics and tested under Linux and MacOS X), I would recommend using one pipe() per child process. That will lead to fewer problems and running more than one child process at a time is possible, too. select() or epoll() (if you have MANY child processes) are your friend.
Since pipe() is very cheap, in particular compared to bash!, I would definitely not use the same pipe for more than more than one child (as my updated sample now does).
You could use a lightweight messaging queue like ZeroMQ. I think you would use its PUSH-PULL mechanism. Your C program or the bash script pushes events while the C++ application pulls them. ZeroMQ is written in C but, besides many others, there exists a C++ and a Python binding for it. See here for a PUSH-PULL example.