I am using version 0.5 of Boost.Process. Documentation can be found here. I am using Mac OS X Yosemite.
My problem: I am launching a compilation as a child process. I want to wait for the process to finish.
When my child process compiles correctly, everything is ok.
But when my child process does not compile, my code seems to crash when calling boost::process::wait_for_exit.
My user code looks like this:
EDIT: Code has been edited to match latest, more correct version (still does not work).
s::error_code ec{};
bp::child child = bp::execute(bpi::set_args(compilationCommand),
bpi::bind_stderr(outErrLog_),
bpi::bind_stdout(outErrLog_),
bpi::inherit_env(),
bpi::set_on_error(ec));
bool compilationSuccessful = true;
if (!ec) {
s::error_code ec2;
bp::wait_for_exit(child, ec2);
if (ec2)
compilationSuccessful = false;
}
The internal implementation of bp::wait_for_exit:
template <class Process>
inline int wait_for_exit(const Process &p, boost::system::error_code &ec)
{
pid_t ret;
int status;
do
{
ret = ::waitpid(p.pid, &status, 0);
} while ((ret == -1 && errno == EINTR) || (ret != -1 && !WIFEXITED(status)));
if (ret == -1) {
BOOST_PROCESS_RETURN_LAST_SYSTEM_ERROR("waitpid(2) failed");
}
else
ec.clear();
return status;
}
The code after ::waitpidis never reached when my compilation command fails. The error shown is: "child has exited; pid: xxxx; uid: yyy; exit value: 1".
Questions:
Is this a bug or I am misusing boost::process::wait_for_exit.
Any workaround for avoiding the crash I am getting that is portable?
Just looking at your code, the first thing that strikes me is that you don't actually test the "ec" variable that says whether execute() succeeded or not until after you call wait_for_exit(). If you're calling wait_for_exit() with an invalid child process, it's quite understandable that it would crash.
Start by checking "ec" before calling wait_for_exit().
So the problem was that Boost.Test modifies the signals stack in some way.
This signal stack modification has interactions with Boost.Process and code cannot be reliably tested, at least in the default Boost.Test configuration.
I rewrote the tests with a normal main and some functions and it did the job.
I've been tasked to create a program that takes a text file that contains a list of programs as input. It then needs to run valgrind on the programs (one at a time) until valgrind ends or until the program hits a max allotted time. I have the program doing everything I need it to do EXCEPT it isn't waiting for valgrind to finish. The code I'm using has this format:
//code up to this point is working properly
pid_t pid = fork();
if(pid == 0){
string s = "sudo valgrind --*options omitted*" + testPath + " &>" + outPath;
system(s.c_str());
exit(0);
}
//code after here seems to also be working properly
I'm running into an issue where the child just calls the system and moves on without waiting for valgrind to finish. As such I'm guessing that system isn't the right call to use, but I don't know what call I should be making. Can anyone tell me how to get the child to wait for valgrind to finish?
I think that you are looking for fork/execv. Here is an example:
http://www.cs.ecu.edu/karl/4630/spr01/example1.html
An other alternative could be popen.
You can fork and exec your program and then wait for it to finish. See the following example.
pid_t pid = vfork();
if(pid == -1)
{
perror("fork() failed");
return -1;
}
else if(pid == 0)
{
char *args[] = {"/bin/sleep", "5", (char *)0};
execv("/bin/sleep", args);
}
int child_status;
int child_pid = wait(&child_status);
printf("Child %u finished with status %d\n", child_pid, child_status);
I'm use fork() to create child process. From child process I am use exec() to launch new process. My code as below:
......
pid = fork();
if (pid > 0) {
WriteLog("Parent Process");
//Do something
} else if (pid == 0) {
WriteLog("Child process");
int return = execl(ShellScript);
if ( return == -1 )
WriteLog("Launch process fail");
} else {
WriteLog("Can't create child process");
}
......
Note: WriteLog function will be open file, write log, and close file. (It is flushed)
ShellScript will launch new process c/c++.
I run my program for long run and the code above is called many times. And sometime (rarely) there are problem happen that the new process can't launch successful although the child process is created successfully (I have checked carefully). And one thing is extremely misunderstand when this problem happen that the "Child process" log can't printed although the child process is created successful.
In normal case (there are not error happen) the number of times print the "Child process" and "Parent process" log are the same.
In abnormal case, they are not the same although the child process always create successfully.The "Launch process fail" and "Can't create child process" log aren't printed in this case.
Please help me for consult.
Remember that stdio(3) is buffered. Always call fflush(NULL); (see fflush(3) for more) before fork. Add a \n (newline) at end of every printf(3) format string (or else, follow them by fflush(NULL); ...).
The function execl(3) (perhaps you want execlp?) can fail (so sets errno on failure).
} else if (pid == 0) {
printf("Child process\n");
fflush(NULL);
execl("/bin/foo", "foo", "arg1", NULL);
// if we are here execl has failed
perror("Launch process fail");
}
On error, fork(2) fails by returning -1 and sets errno(3) (see also perror(3) and strerror(3)). So your last else should be
} else {
perror("Can't create child process");
fflush(NULL);
}
You might want to use strace(1) (notably as strace -f yourprog ...) to understand the involved syscalls (see syscalls(2)...)
Your WriteLog should probably use strerror (on the errno value saved at beginning of WriteLog ....). I suggest something like
void WriteLog(const char* msg) {
int e = errno;
if (e)
syslog (LOG_ERR, "%s [%s]", msg, strerrno(e));
else
syslog (LOG_ERR, "%s", msg);
}
See syslog(3).
There are limits on the number of fork-ed processes, see setrlimit(2) with RLIMIT_NPROC and the bash ulimit builtin.
Read also Advanced Linux Programming.
I have roughly created the following code to call a child process:
// pipe meanings
const int READ = 0;
const int WRITE = 1;
int fd[2];
// Create pipes
if (pipe(fd))
{
throw ...
}
p_pid = fork();
if (p_pid == 0) // in the child
{
close(fd[READ]);
if (dup2(fd[WRITE], fileno(stdout)) == -1)
{
throw ...
}
close(fd[WRITE]);
// Call exec
execv(argv[0], const_cast<char*const*>(&argv[0]));
_exit(-1);
}
else if (p_pid < 0) // fork has failed
{
throw
}
else // in th parent
{
close(fd[WRITE]);
p_stdout = new std::ifstream(fd[READ]));
}
Now, if the subprocess does not write too much to stdout, I can wait for it to finish and then read the stdout from p_stdout. If it writes too much, the write blocks and the parent waits for it forever.
To fix this, I tried to wait with WNOHANG in the parent, if it is not finished, read all available output from p_stdout using readsome, sleep a bit and try again. Unfortunately, readsome never reads anything:
while (true)
{
if (waitid(P_PID, p_pid, &info, WEXITED | WNOHANG) != 0)
throw ...;
else if (info.si_pid != 0) // waiting has succeeded
break;
char tmp[1024];
size_t sizeRead;
sizeRead = p_stdout->readsome(tmp, 1024);
if (sizeRead > 0)
s_stdout.write(tmp, sizeRead);
sleep(1);
}
The question is: Why does this not work and how can I fix it?
edit: If there is only child, simply using read instead of readsome would probably work, but the process has multiple children and needs to react as soon as one of them terminates.
As sarnold suggested, you need to change the order of your calls. Read first, wait last. Even if your method worked, you might miss the last read. i.e. you exit the loop before you read the last set of bytes that was written.
The problem might be is that ifstream is non-blocking. I've never liked iostreams, even in my C++ projects, I always liked the simplicity of C's stdio functions (i.e. FILE*, fprintf, etc). One way to get around this is to read if the descriptor is readable. You can use select to determine if there is data waiting on that pipe. You're going to need select if you are going to read from multiple children anyway, so might as well learn it now.
As for a quick isreadable function, try something like this (please note I haven't tried compiling this):
bool isreadable(int fd, int timeoutSecs)
{
struct timeval tv = { timeoutSecs, 0 };
fd_set readSet;
FD_ZERO(&readSet);
return select(fds, &readSet, NULL, NULL, &tv) == 1;
}
Then in your parent code, do something like:
while (true) {
if (isreadable(fd[READ], 1)) {
// read fd[READ];
if (bytes <= 0)
break;
}
}
wait(pid);
I'd suggest re-writing the code so that it doesn't call waitpid(2) until after read(2) calls on the pipe return 0 to signify end-of-file. Once you get the end-of-file return from your read calls, you know the child is dead, and you can finally waitpid(2) for it.
Another option is to de-couple the reading from the reaping even further and perform the wait calls in a SIGCHLD signal handler asynchronously to the reading operations.
This question already has answers here:
Can popen() make bidirectional pipes like pipe() + fork()?
(6 answers)
Closed 3 years ago.
Is it possible to read and write to a file descriptor returned by popen. I have an interactive process I'd like to control through C. If this isn't possible with popen, is there any way around it?
As already answered, popen works in one direction. If you need to read and write, You can create a pipe with pipe(), span a new process by fork() and exec functions and then redirect its input and outputs with dup2(). Anyway I prefer exec over popen, as it gives you better control over the process (e.g. you know its pid)
EDITED:
As comments suggested, a pipe can be used in one direction only. Therefore you have to create separate pipes for reading and writing. Since the example posted before was wrong, I deleted it and created a new, correct one:
#include<unistd.h>
#include<sys/wait.h>
#include<sys/prctl.h>
#include<signal.h>
#include<stdlib.h>
#include<string.h>
#include<stdio.h>
int main(int argc, char** argv)
{
pid_t pid = 0;
int inpipefd[2];
int outpipefd[2];
char buf[256];
char msg[256];
int status;
pipe(inpipefd);
pipe(outpipefd);
pid = fork();
if (pid == 0)
{
// Child
dup2(outpipefd[0], STDIN_FILENO);
dup2(inpipefd[1], STDOUT_FILENO);
dup2(inpipefd[1], STDERR_FILENO);
//ask kernel to deliver SIGTERM in case the parent dies
prctl(PR_SET_PDEATHSIG, SIGTERM);
//replace tee with your process
execl("/usr/bin/tee", "tee", (char*) NULL);
// Nothing below this line should be executed by child process. If so,
// it means that the execl function wasn't successfull, so lets exit:
exit(1);
}
// The code below will be executed only by parent. You can write and read
// from the child using pipefd descriptors, and you can send signals to
// the process using its pid by kill() function. If the child process will
// exit unexpectedly, the parent process will obtain SIGCHLD signal that
// can be handled (e.g. you can respawn the child process).
//close unused pipe ends
close(outpipefd[0]);
close(inpipefd[1]);
// Now, you can write to outpipefd[1] and read from inpipefd[0] :
while(1)
{
printf("Enter message to send\n");
scanf("%s", msg);
if(strcmp(msg, "exit") == 0) break;
write(outpipefd[1], msg, strlen(msg));
read(inpipefd[0], buf, 256);
printf("Received answer: %s\n", buf);
}
kill(pid, SIGKILL); //send SIGKILL signal to the child process
waitpid(pid, &status, 0);
}
The reason popen() and friends don't offer bidirectional communication is that it would be deadlock-prone, due to buffering in the subprocess. All the makeshift pipework and socketpair() solutions discussed in the answers suffer from the same problem.
Under UNIX, most commands cannot be trusted to read one line and immediately process it and print it, except if their standard output is a tty. The reason is that stdio buffers output in userspace by default, and defers the write() system call until either the buffer is full or the stdio stream is closed (typically because the program or script is about to exit after having seen EOF on input). If you write to such a program's stdin through a pipe, and now wait for an answer from that program's stdout (without closing the ingress pipe), the answer is stuck in the stdio buffers and will never come out - This is a deadlock.
You can trick some line-oriented programs (eg grep) into not buffering by using a pseudo-tty to talk to them; take a look at libexpect(3). But in the general case, you would have to re-run a different subprocess for each message, allowing to use EOF to signal the end of each message and cause whatever buffers in the command (or pipeline of commands) to be flushed. Obviously not a good thing performance-wise.
See more info about this problem in the perlipc man page (it's for bi-directional pipes in Perl but the buffering considerations apply regardless of the language used for the main program).
You want something often called popen2. Here's a basic implementation without error checking (found by a web search, not my code):
// http://media.unpythonic.net/emergent-files/01108826729/popen2.c
#include <sys/types.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <errno.h>
#include "popen2.h"
int popen2(const char *cmdline, struct popen2 *childinfo) {
pid_t p;
int pipe_stdin[2], pipe_stdout[2];
if(pipe(pipe_stdin)) return -1;
if(pipe(pipe_stdout)) return -1;
//printf("pipe_stdin[0] = %d, pipe_stdin[1] = %d\n", pipe_stdin[0], pipe_stdin[1]);
//printf("pipe_stdout[0] = %d, pipe_stdout[1] = %d\n", pipe_stdout[0], pipe_stdout[1]);
p = fork();
if(p < 0) return p; /* Fork failed */
if(p == 0) { /* child */
close(pipe_stdin[1]);
dup2(pipe_stdin[0], 0);
close(pipe_stdout[0]);
dup2(pipe_stdout[1], 1);
execl("/bin/sh", "sh", "-c", cmdline, NULL);
perror("execl"); exit(99);
}
childinfo->child_pid = p;
childinfo->to_child = pipe_stdin[1];
childinfo->from_child = pipe_stdout[0];
close(pipe_stdin[0]);
close(pipe_stdout[1]);
return 0;
}
//#define TESTING
#ifdef TESTING
int main(void) {
char buf[1000];
struct popen2 kid;
popen2("tr a-z A-Z", &kid);
write(kid.to_child, "testing\n", 8);
close(kid.to_child);
memset(buf, 0, 1000);
read(kid.from_child, buf, 1000);
printf("kill(%d, 0) -> %d\n", kid.child_pid, kill(kid.child_pid, 0));
printf("from child: %s", buf);
printf("waitpid() -> %d\n", waitpid(kid.child_pid, NULL, 0));
printf("kill(%d, 0) -> %d\n", kid.child_pid, kill(kid.child_pid, 0));
return 0;
}
#endif
popen() can only open the pipe in read or write mode, not both. Take a look at this thread for a workaround.
In one of netresolve backends I'm talking to a script and therefore I need to write to its stdin and read from its stdout. The following function executes a command with stdin and stdout redirected to a pipe. You can use it and adapt it to your liking.
static bool
start_subprocess(char *const command[], int *pid, int *infd, int *outfd)
{
int p1[2], p2[2];
if (!pid || !infd || !outfd)
return false;
if (pipe(p1) == -1)
goto err_pipe1;
if (pipe(p2) == -1)
goto err_pipe2;
if ((*pid = fork()) == -1)
goto err_fork;
if (*pid) {
/* Parent process. */
*infd = p1[1];
*outfd = p2[0];
close(p1[0]);
close(p2[1]);
return true;
} else {
/* Child process. */
dup2(p1[0], 0);
dup2(p2[1], 1);
close(p1[0]);
close(p1[1]);
close(p2[0]);
close(p2[1]);
execvp(*command, command);
/* Error occured. */
fprintf(stderr, "error running %s: %s", *command, strerror(errno));
abort();
}
err_fork:
close(p2[1]);
close(p2[0]);
err_pipe2:
close(p1[1]);
close(p1[0]);
err_pipe1:
return false;
}
https://github.com/crossdistro/netresolve/blob/master/backends/exec.c#L46
(I used the same code in Can popen() make bidirectional pipes like pipe() + fork()?)
Use forkpty (it's non-standard, but the API is very nice, and you can always drop in your own implementation if you don't have it) and exec the program you want to communicate with in the child process.
Alternatively, if tty semantics aren't to your liking, you could write something like forkpty but using two pipes, one for each direction of communication, or using socketpair to communicate with the external program over a unix socket.
You can't use popen to use two-way pipes.
In fact, some OSs don't support two-way pipes, in which case a socket-pair (socketpair) is the only way to do it.
popen works for me in both directions (read and write)
I have been using a popen() pipe in both directions..
Reading and writing a child process stdin and stdout with the file descriptor returned by popen(command,"w")
It seems to work fine..
I assumed it would work before I knew better, and it does.
According posts above this shouldn't work.. which worries me a little bit.
gcc on raspbian (raspbery pi debian)