I need help to get the following to work. I need to start a bash process from c++, this bash process needs to accept input from stdin and output as per normal it's output to stdout.
From a different process I need to write commands to stdin which will then actually execute in bash as per above, then I'm interested in the result from stdout.
This is what I've tried so far, but the output does not make sense to me at all...
if (pipe(pipeBashShell)) {
fprintf(stderr, "Pipe error!\n");
exit(1);
}
if ((pipePId = fork()) == -1) {
fprintf(stderr, "Fork error. Exiting.\n"); /* something went wrong */
exit(1);
}
if (pipePId == 0) { //this is the child process
dup2(pipeBashShell[0], STDIN_FILENO);
dup2(pipeBashShell[1], STDOUT_FILENO);
dup2(pipeBashShell[1], STDERR_FILENO);
static char* bash[] = {"/bin/bash", "-i", NULL};
if (execv(*bash, bash) == -1) {
fprintf(stderr, "execv Error!");
exit(1);
}
exit(0);
} else {
char buf[512];
memset(buf, 0x00, sizeof(buf));
sprintf(buf, "ls\n");
int byteswritten = write(pipeBashShell[1], buf, strlen(buf));
int bytesRead = read(pipeBashShell[0], buf, sizeof(buf));
write(STDOUT_FILENO, buf, strlen(buf));
exit(0);
}
.
The output of the result above is as follows:
' (main)
bash:: command not found gerhard#gerhard-work-pc:~/workspaces/si/si$ gerhard
orkspaces/si/si$ gerhard# gerhard-work-pc:~/workspa
....
The command i'm trying to send to bash is "ls", which should give me a directory listing
Am I missing something here?
You have created one pipe (with two ends) and you are trying to use it for bi-directional communication -- from your main process to bash and vice versa. You need two separate pipes for that.
The way you have connected the file descriptors makes bash talk to itself -- it interprets its prompt as a command which it cannot find, and then interprets the error messages as subsequend commands.
Edit:
The correct setup works as follows:
prepare two pipes:
int parent2child[2], child2parent[2];
pipe(parent2child);
pipe(child2parent);
fork()
in the parent process:
close(parent2child[0]);
close(child2parent[1]);
// write to parent2child[1], read from child2parent[0]
in the child process:
close(parent2child[1]);
close(child2parent[0]);
dup2(parent2child[0], STDIN_FILENO);
dup2(child2parent[1], STDOUT_FILENO);
Related
I'm writing a custom shell where I try to add support for input, output redirections and pipes just like standard shell. I stuck at point where I cannot do input redirection, but output redirection is perfectly working. My implementation is something like this (only related part), you can assume that (string) input is non-empty
void execute() {
... // stuff before execution and initialization of variables
int *fds;
std::string content;
std::string input = readFromAFile(in_file); // for input redirection
for (int i = 0; i < commands.size(); i++) {
fds = subprocess(commands[i]);
dprintf(fds[1], "%s", input.data()); // write to write-end of pipe
close(fds[1]);
content += readFromFD(fds[0]); // read from read-end of pipe
close(fds[0]);
}
... // stuff after execution
}
int *subprocess(std::string &cmd) {
std::string s;
int *fds = new int[2];
pipe(fds);
pid_t pid = fork();
if (pid == -1) {
std::cerr << "Fork failed.";
}
if (pid == 0) {
dup2(fds[1], STDOUT_FILENO);
dup2(fds[0], STDIN_FILENO);
close(fds[1]);
close(fds[0]);
system(cmd.data());
exit(0); // child terminates
}
return fds;
}
My thought is subprocess returns a pipe (fd_in, fd_out) and parent can write to write-end and read-from read-end afterwards. However when I try an input redirection something like sort < in.txt, the program just hangs. I think there is a deadlock because one waiting other to write, and other one to read, however, after parent writes to write-end it closes, and then read from read-end. How should I consider this case ?
When I did a bit of searching, I saw this answer, which my original thinking was similar except that in the answer it mentions creating two pipes. I did not quite understand this part. Why do we need two separate pipes ?
To expand on this, by chain piping I am referring to when I have 3 separate processes:
process 1 writes to process 2,
process 2 reads from process 1 and writes to process 3,
process 3 reads from process 2 and then finishes.
I am specifically trying to handle complex commands in a C++ written bash shell. So I would be using this to execute a set of commands like this that all communicate with each other:
ls | sort | grep "exit"
where process 1 is executing ls and its stdout is written to process 2 through a pipe, etc.
I already am writing code to solve this for a project and was just wondering if my approach is correct, as right now when just doing a 2 command call of ls | grep "exit" I am getting the bash error "grep: (standard input): Bad file descriptor"
//Block for when the userInput is a complex command
else{
if (debug)
printf("Complex command: %s\n", userInput.c_str());
vector<char*> commandsVect = splitCString(const_cast<char*>(userInput.c_str()), const_cast<char*>( delimVertPipe.c_str()));
if (debug)
printVect(commandsVect);
if (pipe(fileDescriptor) == -1){
fprintf(stderr, "Pipe failed for command %s\n", userInput.c_str());
return 1;
}
for (int i = 0; i < commandsVect.size(); ++i) {
vector<char*> tokens = splitCString(const_cast<char*>(commandsVect[i]), const_cast<char*>( delimSpace.c_str()));
printf("Commands vect size is %ld\n", commandsVect.size());
printf("Parsing command \'%s\'\n", commandsVect[i]);
if (debug) {
printVect(tokens);
}
procID = fork();
//Block for the first command
if (i == 0){
if (procID < 0){
fprintf(stderr, "Fork number %d in the complex command \'%s\' failed\n", i+1, userInput.c_str());
return 1;
}
//Child process
else if (procID == 0){
//close(fileDescriptor[READ_END]);
close(STDOUT_FILENO);
//Links the write end of the pipe to the STDOUT
dup2(fileDescriptor[WRITE_END], 1);
close(fileDescriptor[READ_END]);
close(fileDescriptor[WRITE_END]);
tokens.push_back(nullptr); //execvp() arg array needs a NULL pointer at the end
if ( execvp(tokens[0], tokens.data()) < 0 ) {
fprintf( stderr, "execvp() call failed for the command \'%s\' inside the input string \'%s\'\n", commandsVect[i], userInput.c_str() );
return 1;
}
exit(1);
}
//Parent process
else{
close(fileDescriptor[READ_END]);
close(fileDescriptor[WRITE_END]);
wait(NULL);
}
}
//Block for the very last command, which will pipe input from the previous
else if (i == commandsVect.size() - 1){
if (procID < 0){
fprintf(stderr, "Fork number %d in the complex command \'%s\' failed\n", i+1, userInput.c_str());
return 1;
}
//Child process
else if (procID == 0){
//close(fileDescriptor[WRITE_END]);
close(STDIN_FILENO);
//Links the read end of the pipe to the STDIN
dup2(fileDescriptor[READ_END], 0);
close(fileDescriptor[WRITE_END]);
close(fileDescriptor[READ_END]);
tokens.push_back(nullptr); //execvp() arg array needs a NULL pointer at the end
if ( execvp(tokens[0], tokens.data()) < 0 ) {
fprintf( stderr, "execvp() call failed for the command \'%s\' inside the input string \'%s\'\n", commandsVect[i], userInput.c_str() );
return 1;
}
exit(1);
}
//Parent process
else{
close(fileDescriptor[READ_END]);
close(fileDescriptor[WRITE_END]);
wait(NULL);
}
}
//To note for StackOverflow, this block of code is never executed since I am only ever calling a 2 chained command like ls|grep "exit"
//Block for the middle commands. (Will pipe input from previous, and output to the next)
else{
printf("GOING THROUGH BAD CODE");
continue;
if (procID < 0){
fprintf(stderr, "Fork number %d in the complex command \'%s\' failed\n", i+1, userInput.c_str());
return 1;
}
//Child process
else if (procID == 0){
exit(1);
}
//Parent process
else{
wait(NULL);
}
}
}
close(fileDescriptor[READ_END]);
close(fileDescriptor[WRITE_END]);
}
This might not be possible with your larger application, but you could simplify things by letting the shell manage the pipes. Write P1 (process one), P2, and P3 as three separate executables. In stead of doing IO on pipes, each program could read from stdin and write to stdout. Simple. To execute - let bash or whatever shell you use glue the three together by calling them as...
$P1 | P2 | P3;
Under the hood, your shell is doing pretty much what you're doing in C++ (only successfully 😉). It creates a pipe for P1, which it passes to exec as stdin to launch P1 after forking. It creates an input and output pipe for P2, and binds it stdin and stdout as appropriate in the same way - passed into exec when launching P2 after the fork. P3 gets only a stdin pipe and its stdout stream goes right to the console as normal. It's not quite as sexy as doing it all in C++, but it's very robust - pretty much guaranteed to work.
I am trying to write a program that runs an external program.
I know that I can catch stdout, and I can catch stdout and stderr together BUT the question is can I catch the stderr and stdout separated?
I mean for example, stderr in variable STDERR and stdout in variable STDOUT. I mean I want them separated.
Also I need the exit code of the external program in a variable.
On Windows you must fill STARTUPINFO for the CreateProcess to catch standart streams, and you can use GetExitCodeProcess function to get the termination status. There is an example how to redirect standart streams into the parent process http://msdn.microsoft.com/en-us/library/windows/desktop/ms682499.aspx
On Linux-like OS you probably want to use fork instead of execve, and working with a forked process is another story.
In Windows and Linux redirecting streams has general approach - you must create several pipes (one for each stream) and redirect child process streams into that pipes, and the parent process can read data from that pipes.
Sample code for Linux:
int fd[2];
if (pipe(fd) == -1) {
perror("pipe");
exit(EXIT_FAILURE);
}
pid_t cpid = fork();
if (cpid == -1) {
perror("fork");
exit(EXIT_FAILURE);
}
if (cpid == 0) { // child
dup2(fd[1], STDERR_FILENO);
fprintf(stderr, "Hello, World!\n");
exit(EXIT_SUCCESS);
} else { // parent
char ch;
while (read(fd[0], &ch, 1) > 0)
printf("%c", ch);
exit(EXIT_SUCCESS);
}
EDIT: If you need to catch streams from another program, use the same stragey as above, first fork, second - use pipes (as in code above), then execve another progrram in child process and use this code in parent process to wait an execution end and catch a return code:
int status;
if (waitpid(cpid, &status, 0) < 0) {
perror("waitpid");
exit(EXIT_FAILURE);
}
You can find more details in man pages pipe, dup2 and waitpid.
This question already has answers here:
Can popen() make bidirectional pipes like pipe() + fork()?
(6 answers)
Closed 3 years ago.
Is it possible to read and write to a file descriptor returned by popen. I have an interactive process I'd like to control through C. If this isn't possible with popen, is there any way around it?
As already answered, popen works in one direction. If you need to read and write, You can create a pipe with pipe(), span a new process by fork() and exec functions and then redirect its input and outputs with dup2(). Anyway I prefer exec over popen, as it gives you better control over the process (e.g. you know its pid)
EDITED:
As comments suggested, a pipe can be used in one direction only. Therefore you have to create separate pipes for reading and writing. Since the example posted before was wrong, I deleted it and created a new, correct one:
#include<unistd.h>
#include<sys/wait.h>
#include<sys/prctl.h>
#include<signal.h>
#include<stdlib.h>
#include<string.h>
#include<stdio.h>
int main(int argc, char** argv)
{
pid_t pid = 0;
int inpipefd[2];
int outpipefd[2];
char buf[256];
char msg[256];
int status;
pipe(inpipefd);
pipe(outpipefd);
pid = fork();
if (pid == 0)
{
// Child
dup2(outpipefd[0], STDIN_FILENO);
dup2(inpipefd[1], STDOUT_FILENO);
dup2(inpipefd[1], STDERR_FILENO);
//ask kernel to deliver SIGTERM in case the parent dies
prctl(PR_SET_PDEATHSIG, SIGTERM);
//replace tee with your process
execl("/usr/bin/tee", "tee", (char*) NULL);
// Nothing below this line should be executed by child process. If so,
// it means that the execl function wasn't successfull, so lets exit:
exit(1);
}
// The code below will be executed only by parent. You can write and read
// from the child using pipefd descriptors, and you can send signals to
// the process using its pid by kill() function. If the child process will
// exit unexpectedly, the parent process will obtain SIGCHLD signal that
// can be handled (e.g. you can respawn the child process).
//close unused pipe ends
close(outpipefd[0]);
close(inpipefd[1]);
// Now, you can write to outpipefd[1] and read from inpipefd[0] :
while(1)
{
printf("Enter message to send\n");
scanf("%s", msg);
if(strcmp(msg, "exit") == 0) break;
write(outpipefd[1], msg, strlen(msg));
read(inpipefd[0], buf, 256);
printf("Received answer: %s\n", buf);
}
kill(pid, SIGKILL); //send SIGKILL signal to the child process
waitpid(pid, &status, 0);
}
The reason popen() and friends don't offer bidirectional communication is that it would be deadlock-prone, due to buffering in the subprocess. All the makeshift pipework and socketpair() solutions discussed in the answers suffer from the same problem.
Under UNIX, most commands cannot be trusted to read one line and immediately process it and print it, except if their standard output is a tty. The reason is that stdio buffers output in userspace by default, and defers the write() system call until either the buffer is full or the stdio stream is closed (typically because the program or script is about to exit after having seen EOF on input). If you write to such a program's stdin through a pipe, and now wait for an answer from that program's stdout (without closing the ingress pipe), the answer is stuck in the stdio buffers and will never come out - This is a deadlock.
You can trick some line-oriented programs (eg grep) into not buffering by using a pseudo-tty to talk to them; take a look at libexpect(3). But in the general case, you would have to re-run a different subprocess for each message, allowing to use EOF to signal the end of each message and cause whatever buffers in the command (or pipeline of commands) to be flushed. Obviously not a good thing performance-wise.
See more info about this problem in the perlipc man page (it's for bi-directional pipes in Perl but the buffering considerations apply regardless of the language used for the main program).
You want something often called popen2. Here's a basic implementation without error checking (found by a web search, not my code):
// http://media.unpythonic.net/emergent-files/01108826729/popen2.c
#include <sys/types.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <errno.h>
#include "popen2.h"
int popen2(const char *cmdline, struct popen2 *childinfo) {
pid_t p;
int pipe_stdin[2], pipe_stdout[2];
if(pipe(pipe_stdin)) return -1;
if(pipe(pipe_stdout)) return -1;
//printf("pipe_stdin[0] = %d, pipe_stdin[1] = %d\n", pipe_stdin[0], pipe_stdin[1]);
//printf("pipe_stdout[0] = %d, pipe_stdout[1] = %d\n", pipe_stdout[0], pipe_stdout[1]);
p = fork();
if(p < 0) return p; /* Fork failed */
if(p == 0) { /* child */
close(pipe_stdin[1]);
dup2(pipe_stdin[0], 0);
close(pipe_stdout[0]);
dup2(pipe_stdout[1], 1);
execl("/bin/sh", "sh", "-c", cmdline, NULL);
perror("execl"); exit(99);
}
childinfo->child_pid = p;
childinfo->to_child = pipe_stdin[1];
childinfo->from_child = pipe_stdout[0];
close(pipe_stdin[0]);
close(pipe_stdout[1]);
return 0;
}
//#define TESTING
#ifdef TESTING
int main(void) {
char buf[1000];
struct popen2 kid;
popen2("tr a-z A-Z", &kid);
write(kid.to_child, "testing\n", 8);
close(kid.to_child);
memset(buf, 0, 1000);
read(kid.from_child, buf, 1000);
printf("kill(%d, 0) -> %d\n", kid.child_pid, kill(kid.child_pid, 0));
printf("from child: %s", buf);
printf("waitpid() -> %d\n", waitpid(kid.child_pid, NULL, 0));
printf("kill(%d, 0) -> %d\n", kid.child_pid, kill(kid.child_pid, 0));
return 0;
}
#endif
popen() can only open the pipe in read or write mode, not both. Take a look at this thread for a workaround.
In one of netresolve backends I'm talking to a script and therefore I need to write to its stdin and read from its stdout. The following function executes a command with stdin and stdout redirected to a pipe. You can use it and adapt it to your liking.
static bool
start_subprocess(char *const command[], int *pid, int *infd, int *outfd)
{
int p1[2], p2[2];
if (!pid || !infd || !outfd)
return false;
if (pipe(p1) == -1)
goto err_pipe1;
if (pipe(p2) == -1)
goto err_pipe2;
if ((*pid = fork()) == -1)
goto err_fork;
if (*pid) {
/* Parent process. */
*infd = p1[1];
*outfd = p2[0];
close(p1[0]);
close(p2[1]);
return true;
} else {
/* Child process. */
dup2(p1[0], 0);
dup2(p2[1], 1);
close(p1[0]);
close(p1[1]);
close(p2[0]);
close(p2[1]);
execvp(*command, command);
/* Error occured. */
fprintf(stderr, "error running %s: %s", *command, strerror(errno));
abort();
}
err_fork:
close(p2[1]);
close(p2[0]);
err_pipe2:
close(p1[1]);
close(p1[0]);
err_pipe1:
return false;
}
https://github.com/crossdistro/netresolve/blob/master/backends/exec.c#L46
(I used the same code in Can popen() make bidirectional pipes like pipe() + fork()?)
Use forkpty (it's non-standard, but the API is very nice, and you can always drop in your own implementation if you don't have it) and exec the program you want to communicate with in the child process.
Alternatively, if tty semantics aren't to your liking, you could write something like forkpty but using two pipes, one for each direction of communication, or using socketpair to communicate with the external program over a unix socket.
You can't use popen to use two-way pipes.
In fact, some OSs don't support two-way pipes, in which case a socket-pair (socketpair) is the only way to do it.
popen works for me in both directions (read and write)
I have been using a popen() pipe in both directions..
Reading and writing a child process stdin and stdout with the file descriptor returned by popen(command,"w")
It seems to work fine..
I assumed it would work before I knew better, and it does.
According posts above this shouldn't work.. which worries me a little bit.
gcc on raspbian (raspbery pi debian)
This question already has answers here:
How do I execute a command and get the output of the command within C++ using POSIX?
(12 answers)
Closed 7 years ago.
I'm trying to start an external application through system() - for example, system("ls"). I would like to capture its output as it happens so I can send it to another function for further processing. What's the best way to do that in C/C++?
From the popen manual:
#include <stdio.h>
FILE *popen(const char *command, const char *type);
int pclose(FILE *stream);
Try the popen() function. It executes a command, like system(), but directs the output into a new file. A pointer to the stream is returned.
FILE *lsofFile_p = popen("lsof", "r");
if (!lsofFile_p)
{
return -1;
}
char buffer[1024];
char *line_p = fgets(buffer, sizeof(buffer), lsofFile_p);
pclose(lsofFile_p);
EDIT: misread question as wanting to pass output to another program, not another function. popen() is almost certainly what you want.
System gives you full access to the shell. If you want to continue using it, you can
redirect it's output to a temporary file, by system("ls > tempfile.txt"), but choosing a secure temporary file is a pain. Or, you can even redirect it through another program: system("ls | otherprogram");
Some may recommend the popen() command. This is what you want if you can process the output yourself:
FILE *output = popen("ls", "r");
which will give you a FILE pointer you can read from with the command's output on it.
You can also use the pipe() call to create a connection in combination with fork() to create new processes, dup2() to change the standard input and output of them, exec() to run the new programs, and wait() in the main program to wait for them. This is just setting up the pipeline much like the shell would. See the pipe() man page for details and an example.
The functions popen() and such don't redirect stderr and such; I wrote popen3() for that purpose.
Here's a bowdlerised version of my popen3():
int popen3(int fd[3],const char **const cmd) {
int i, e;
int p[3][2];
pid_t pid;
// set all the FDs to invalid
for(i=0; i<3; i++)
p[i][0] = p[i][1] = -1;
// create the pipes
for(int i=0; i<3; i++)
if(pipe(p[i]))
goto error;
// and fork
pid = fork();
if(-1 == pid)
goto error;
// in the parent?
if(pid) {
// parent
fd[STDIN_FILENO] = p[STDIN_FILENO][1];
close(p[STDIN_FILENO][0]);
fd[STDOUT_FILENO] = p[STDOUT_FILENO][0];
close(p[STDOUT_FILENO][1]);
fd[STDERR_FILENO] = p[STDERR_FILENO][0];
close(p[STDERR_FILENO][1]);
// success
return 0;
} else {
// child
dup2(p[STDIN_FILENO][0],STDIN_FILENO);
close(p[STDIN_FILENO][1]);
dup2(p[STDOUT_FILENO][1],STDOUT_FILENO);
close(p[STDOUT_FILENO][0]);
dup2(p[STDERR_FILENO][1],STDERR_FILENO);
close(p[STDERR_FILENO][0]);
// here we try and run it
execv(*cmd,const_cast<char*const*>(cmd));
// if we are there, then we failed to launch our program
perror("Could not launch");
fprintf(stderr," \"%s\"\n",*cmd);
_exit(EXIT_FAILURE);
}
// preserve original error
e = errno;
for(i=0; i<3; i++) {
close(p[i][0]);
close(p[i][1]);
}
errno = e;
return -1;
}
The most efficient way is to use stdout file descriptor directly, bypassing FILE stream:
pid_t popen2(const char *command, int * infp, int * outfp)
{
int p_stdin[2], p_stdout[2];
pid_t pid;
if (pipe(p_stdin) == -1)
return -1;
if (pipe(p_stdout) == -1) {
close(p_stdin[0]);
close(p_stdin[1]);
return -1;
}
pid = fork();
if (pid < 0) {
close(p_stdin[0]);
close(p_stdin[1]);
close(p_stdout[0]);
close(p_stdout[1]);
return pid;
} else if (pid == 0) {
close(p_stdin[1]);
dup2(p_stdin[0], 0);
close(p_stdout[0]);
dup2(p_stdout[1], 1);
dup2(::open("/dev/null", O_WRONLY), 2);
/// Close all other descriptors for the safety sake.
for (int i = 3; i < 4096; ++i) {
::close(i);
}
setsid();
execl("/bin/sh", "sh", "-c", command, NULL);
_exit(1);
}
close(p_stdin[0]);
close(p_stdout[1]);
if (infp == NULL) {
close(p_stdin[1]);
} else {
*infp = p_stdin[1];
}
if (outfp == NULL) {
close(p_stdout[0]);
} else {
*outfp = p_stdout[0];
}
return pid;
}
To read output from child use popen2() like this:
int child_stdout = -1;
pid_t child_pid = popen2("ls", 0, &child_stdout);
if (!child_pid) {
handle_error();
}
char buff[128];
ssize_t bytes_read = read(child_stdout, buff, sizeof(buff));
To both write and read:
int child_stdin = -1;
int child_stdout = -1;
pid_t child_pid = popen2("grep 123", &child_stdin, &child_stdout);
if (!child_pid) {
handle_error();
}
const char text = "1\n2\n123\n3";
ssize_t bytes_written = write(child_stdin, text, sizeof(text) - 1);
char buff[128];
ssize_t bytes_read = read(child_stdout, buff, sizeof(buff));
The functions popen() and pclose() could be what you're looking for.
Take a look at the glibc manual for an example.
In Windows, instead of using system(), use CreateProcess, redirect the output to a pipe and connect to the pipe.
I'm guessing this is also possible in some POSIX way?
Actually, I just checked, and:
popen is problematic, because the process is forked. So if you need to wait for the shell command to execute, then you're in danger of missing it. In my case, my program closed even before the pipe got to do it's work.
I ended up using system call with tar command on linux. The return value from system was the result of tar.
So: if you need the return value, then not no only is there no need to use popen, it probably won't do what you want.
In this page: capture_the_output_of_a_child_process_in_c describes the limitations of using popen vs. using fork/exec/dup2/STDOUT_FILENO approach.
I'm having problems capturing tshark output with popen.
And I'm guessing that this limitation might be my problem:
It returns a stdio stream as opposed to a raw file descriptor, which
is unsuitable for handling the output asynchronously.
I'll come back to this answer if I have a solution with the other approach.
I'm not entirely certain that its possible in standard C, as two different processes don't typically share memory space. The simplest way I can think of to do it would be to have the second program redirect its output to a text file (programname > textfile.txt) and then read that text file back in for processing. However, that may not be the best way.