I'm using the following code part to perform input redirection for my custom C++ shell.
While the output redirection similar to this works well, the child process for the input redirection stays open and doesn't return, like it keeps waiting for new input.
What is the best way to 'ask' or 'force' a child process like this to return immediately after reading input?
Code for input redirection
int in_file = open(in, O_CREAT | O_RDONLY , S_IREAD | S_IWRITE);
pid_t pid = fork();
if (!pid) {
dup2(in_file, STDIN_FILENO);
if (execvp(argv[0], argv) < 0) {
cerr << "*** ERROR: exec failed: "<< argv[0] << endl;
exit(EXIT_FAILURE);
}
}
close(in_file);
Code for output redirection
out_file = open(out, O_CREAT | O_WRONLY | O_TRUNC, S_IRWXU);
pid_t pid = fork();
if (!pid) {
dup2(out_file, STDOUT_FILENO);
if (execvp(argv[0], argv) < 0) {
cerr << "*** ERROR: exec failed: "<< argv[0] << endl;
exit(EXIT_FAILURE);
}
}
close(out_file);
I used the following commands to test:
ps aux > out.txt
grep root < out.txt
The first command returns to shell after succesfully writing to out.txt. The second command reads succesfully from out.txt, but doesn't return or stop.
The child still has the in_file open. You must close it before exec:
dup2(in_file, STDIN_FILENO);
close(in_file);
(error checking omitted for brevity). Because the child still has an open file descriptor, it never sees the file as being closed, so it blocks on a read waiting for someone to write more data. The child process does not realize that it is the one holding the file descriptor open. Another option is to set the 'close-on-exec' flag for the file descriptor, so that it is closed automatically on exec. (Search for FD_CLOEXEC)
First, check the arguments to execvp(). If this code is part of main(int argc, char* argv[]), then argv[0] is your own program, not grep. This means that your program is recursively re-executing itself for ever.
Then, make sure that there is no error when opening in with:
if (in_file < 0) { perror(in); ... }
If in_file is an invalid descriptor, dup2() will fail and grep will run reading from the terminal, hence, not terminating. BTW, using O_CREAT | O_RDONLY looks fishy. Why read from a file that does not previously exist?
Related
I'm writing a custom shell where I try to add support for input, output redirections and pipes just like standard shell. I stuck at point where I cannot do input redirection, but output redirection is perfectly working. My implementation is something like this (only related part), you can assume that (string) input is non-empty
void execute() {
... // stuff before execution and initialization of variables
int *fds;
std::string content;
std::string input = readFromAFile(in_file); // for input redirection
for (int i = 0; i < commands.size(); i++) {
fds = subprocess(commands[i]);
dprintf(fds[1], "%s", input.data()); // write to write-end of pipe
close(fds[1]);
content += readFromFD(fds[0]); // read from read-end of pipe
close(fds[0]);
}
... // stuff after execution
}
int *subprocess(std::string &cmd) {
std::string s;
int *fds = new int[2];
pipe(fds);
pid_t pid = fork();
if (pid == -1) {
std::cerr << "Fork failed.";
}
if (pid == 0) {
dup2(fds[1], STDOUT_FILENO);
dup2(fds[0], STDIN_FILENO);
close(fds[1]);
close(fds[0]);
system(cmd.data());
exit(0); // child terminates
}
return fds;
}
My thought is subprocess returns a pipe (fd_in, fd_out) and parent can write to write-end and read-from read-end afterwards. However when I try an input redirection something like sort < in.txt, the program just hangs. I think there is a deadlock because one waiting other to write, and other one to read, however, after parent writes to write-end it closes, and then read from read-end. How should I consider this case ?
When I did a bit of searching, I saw this answer, which my original thinking was similar except that in the answer it mentions creating two pipes. I did not quite understand this part. Why do we need two separate pipes ?
A child process runs a bin file, which is provided by Qualcomm.
The child process is invoked by my parent process, which is developed by me.
When the child process is running, it always prints lots pieces of logs in shell command.
So, am I able to redirect Qualcomm's outstream from stdout to another file in the parent process?
As you know, it's nearly impossible to push Qualcomm to update this bin file.
The key piece here is the POSIX function dup2, which lets you essentially replace one file descriptor with another. And if you use fork (not system), you actually have control of what happens in the child process between the fork and the exec* that loads the other executable.
#include <cstdlib>
extern "C" {
#include <fcntl.h>
#include <unistd.h>
}
#include <stdexcept>
#include <iostream>
pid_t start_child(const char* program, const char* output_filename)
{
pid_t pid = fork();
if (pid < 0) {
// fork failed!
std::perror("fork");
throw std::runtime_error("fork failed");
} else if (pid == 0) {
// This code runs in the child process.
int output_fd = open(output_filename, O_WRONLY | O_CREAT | O_TRUNC);
if (output_fd < 0) {
std::cerr << "Failed to open log file " << output_filename << ":"
<< std::endl;
std::perror("open");
std::exit(1);
}
// Replace the child's stdout and stderr handles with the log file handle:
if (dup2(output_fd, STDOUT_FILENO) < 0) {
std::perror("dup2 (stdout)");
std::exit(1);
}
if (dup2(output_fd, STDERR_FILENO) < 0) {
std::perror("dup2 (stderr)");
std::exit(1);
}
if (execl(program, program, (char*)nullptr) < 0) {
// These messages will actually go into the file.
std::cerr << "Failed to exec program " << program << ":"
<< std::endl;
std::perror("execl");
std::exit(1);
}
}
return pid;
}
It is possible, for POSIX, because the POSIX shells do this. Executing a program has two steps, for POSIX. First use fork to clone the parent process to create the child process. Then have the child process use one of the exec family of system calls to execute the chosen program instead of the program of the parent. In between those two steps the code executing for the child process can do additional operations, which will affect the environment of the program to be executed. In particular, the code could open a file descriptor to the file to be redirected to, close the stdout file descriptor, then duplicate the file's file descriptor to the value (1) used for stdout.
You could create own pipes and attach them to the child process.
Create 3 pipes. they are going to replace stdin, stdout, stderr of the child.
fork()
In subprocess close() the parent end of the pipes. Close stdin,stdout and stderr.
The parent process close() the child end of the pipes.
dup2() the pipe ends in the child process that are intended to work as the new stdin,out,err
exec() the child.
Now you got all Output from the child to the pipe in the parent. Ofcourse you need to read from the pipes that come from the child or it will block on any write to the stdout/stderr. For this you could use a select(), poll(), epoll() multiplexing algorithm.
See
https://linux.die.net/man/2/pipe
https://linux.die.net/man/2/dup2
https://linux.die.net/man/2/execve
https://linux.die.net/man/2/fork
I have written the daemon in C/C++ in linux.
Now I want to get the out put of ls -l (list directory) command inside daemon and write output of command to the file.
I know how to write to the file from my daemon, but,
I don't know how to execute ls -l command and get the output in buffer.
Here is the code...
/* Create a new SID for the child process */
sid = setsid();
if (sid < 0) {
/* Log any failures here */
ofs << "set sid : fail";
ofs.close();
exit(EXIT_FAILURE);
}
ofs << "\nchdir :" << chdir(filePath) << "\n";
/* Change the current working directory */
if ((chdir(filePath)) < 0) {
/* Log any failures here */
ofs << "chdir : fail";
ofs.close();
exit(EXIT_FAILURE);
}
/* Close out the standard file descriptors */
close(STDIN_FILENO);
close(STDOUT_FILENO);
close(STDERR_FILENO);
while(1){
//here I want to execute the ls -l and get output of the command
}
You can use popen that executes a shell command and return the output as a pipe:
#include <stdio.h>
FILE* pipe = popen("ls -l", "r");
if (!pipe) return "ERROR";
You can also use system to execute any shell command:
#include <stdlib.h>
int system(const char *command);
To get the output of ls -l, forward it to a file ls -l >> myls.log than read that file.
system("ls -l >> myls.log");
I'm posting my code simply for context of my question. I'm not explicitly looking for you to help fix it, I'm more so looking to understand the dup2 system call that I'm just not picking up from the man page and the numerous other stackoverflow questions.
pid = fork();
if(pid == 0) {
if(strcmp("STDOUT", outfile)) {
if (command->getOutputFD() == REDIRECT) {
if ((outfd = open(outfile, O_CREAT | O_WRONLY | O_TRUNC)) == -1)
return false;
command->setOutputFD(outfd);
if (dup2(command->getOutputFD(), STDOUT_FILENO) == -1)
return false;
pipeIndex++;
}
else if (command->getOutputFD() == REDIRECTAPPEND) {
if ((outfd = open(outfile, O_CREAT | O_WRONLY | O_APPEND)) == -1)
return false;
command->setOutputFD(outfd);
if (dup2(command->getOutputFD(), STDOUT_FILENO) == -1)
return false;
pipeIndex++;
}
else {
if (dup2(pipefd[++pipeIndex], STDOUT_FILENO) == -1)
return false;
command->setOutputFD(pipefd[pipeIndex]);
}
}
if(strcmp("STDIN", infile)) {
if(dup2(pipefd[pipeIndex - 1], STDIN_FILENO) == -1)
return false;
command->setOutputFD(pipefd[pipeIndex - 1]);
pipeIndex++;
}
if (execvp(arguments[0], arguments) == -1) {
std::cerr << "Error!" << std::endl;
_Exit(0);
}
}
else if(pid == -1) {
return false;
}
For context to you, that code represents the execution step of a basic linux shell. The command object contains the commands arguments, IO "name", and IO descriptors (I think I might get rid of the file descriptors as fields).
What I'm having the most difficultly understanding is when and which file descriptors to close. I guess I'll just ask some questions to try and improve my understanding of the concept.
1) With my array of file descriptors used for handling pipes, the parent has a copy of all those descriptors. When are the descriptors held by the parent closed? And even more so, which descriptors? Is it all of them? All of the ones left unused by the executing commands?
2) When handling pipes within the children, which descriptors are left open by which processes? Say if I execute the command: ls -l | grep
"[username]", Which descriptors should be left open for the ls process? Just the write end of the pipe? And if so when? The same question applies to the grep command.
3) When I handle redirection of IO to a file, a new file must be opened and duped to STDOUT (I do not support input redirection). When does this descriptor get closed? I've seen in examples that it gets closed immediately after the call to dup2, but then how does anything get written to the file if the file has been closed?
Thanks ahead of time. I've been stuck on this problem for days and I'd really like to be done with this project.
EDIT I've updated this with modified code and sample output for anyone interested in offering specific help to my issue. First I have the entire for loop that handles execution. It has been updated with my calls to close on various file descriptors.
while(currCommand != NULL) {
command = currCommand->getData();
infile = command->getInFileName();
outfile = command->getOutFileName();
arguments = command->getArgList();
pid = fork();
if(pid == 0) {
if(strcmp("STDOUT", outfile)) {
if (command->getOutputFD() == REDIRECT) {
if ((outfd = open(outfile, O_CREAT | O_WRONLY | O_TRUNC)) == -1)
return false;
if (dup2(outfd, STDOUT_FILENO) == -1)
return false;
close(STDOUT_FILENO);
}
else if (command->getOutputFD() == REDIRECTAPPEND) {
if ((outfd = open(outfile, O_CREAT | O_WRONLY | O_APPEND)) == -1)
return false;
if (dup2(outfd, STDOUT_FILENO) == -1)
return false;
close(STDOUT_FILENO);
}
else {
if (dup2(pipefd[pipeIndex + 1], STDOUT_FILENO) == -1)
return false;
close(pipefd[pipeIndex]);
}
}
pipeIndex++;
if(strcmp("STDIN", infile)) {
if(dup2(pipefd[pipeIndex - 1], STDIN_FILENO) == -1)
return false;
close(pipefd[pipeIndex]);
pipeIndex++;
}
if (execvp(arguments[0], arguments) == -1) {
std::cerr << "Error!" << std::endl;
_Exit(0);
}
}
else if(pid == -1) {
return false;
}
currCommand = currCommand->getNext();
}
for(int i = 0; i < numPipes * 2; i++)
close(pipefd[i]);
for(int i = 0; i < commands->size();i++) {
if(wait(status) == -1)
return false;
}
When executing this code I receive the following output
ᕕ( ᐛ )ᕗ ls -l
total 68
-rwxrwxrwx 1 cook cook 242 May 31 18:31 CMakeLists.txt
-rwxrwxrwx 1 cook cook 617 Jun 1 22:40 Command.cpp
-rwxrwxrwx 1 cook cook 9430 Jun 8 18:02 ExecuteExternalCommand.cpp
-rwxrwxrwx 1 cook cook 682 May 31 18:35 ExecuteInternalCommand.cpp
drwxrwxrwx 2 cook cook 4096 Jun 8 17:16 headers
drwxrwxrwx 2 cook cook 4096 May 31 18:32 implementation files
-rwxr-xr-x 1 cook cook 25772 Jun 8 18:12 LeShell
-rwxrwxrwx 1 cook cook 243 Jun 5 13:02 Makefile
-rwxrwxrwx 1 cook cook 831 Jun 3 12:10 Shell.cpp
ᕕ( ᐛ )ᕗ ls -l > output.txt
ls: write error: Bad file descriptor
ᕕ( ᐛ )ᕗ ls -l | grep "cook"
ᕕ( ᐛ )ᕗ
The output of ls -l > output.txt implies that I'm closing the wrong descriptor, but closing the other related descriptor, while rendering no error, provides no output to the file. As demonstrated by ls -l, grep "cook", should generate output to the console.
With my array of file descriptors used for handling pipes, the parent
has a copy of all those descriptors. When are the descriptors held by
the parent closed? And even more so, which descriptors? Is it all of
them? All of the ones left unused by the executing commands?
A file descriptor may be closed in one of 3 ways:
You explicitly call close() on it.
The process terminates, and the operating system automatically closes every file descriptor that was still open.
When the process calls one of the seven exec() functions and the file descriptor has the O_CLOEXEC flag.
As you can see, most of the times, file descriptors will remain open until you manually close them. This is what happens in your code too - since you didn't specify O_CLOEXEC, file descriptors are not closed when the child process calls execvp(). In the child, they are closed after the child terminates. The same goes for the parent. If you want that to happen any time before terminating, you have to manually call close().
When handling pipes within the children, which descriptors are left
open by which processes? Say if I execute the command: ls -l | grep
"[username]", Which descriptors should be left open for the ls
process? Just the write end of the pipe? And if so when? The same
question applies to the grep command.
Here's a (rough) idea of what the shell does when you type ls -l | grep "username":
The shell calls pipe() to create a new pipe. The pipe file descriptors are inherited by the children in the next step.
The shell forks twice, let's call these processes c1 and c2. Let's assume c1 will run ls and c2 will run grep.
In c1, the pipe's read channel is closed with close(), and then it calls dup2() with the pipe write channel and STDOUT_FILENO, so as to make writing to stdout equivalent to writing to the pipe. Then, one of the seven exec() functions is called to start executing ls. ls writes to stdout, but since we duplicated stdout to the pipe's write channel, ls will be writing to the pipe.
In c2, the reverse happens: the pipe's write channel is closed, and then dup2() is called to make stdin point to the pipe's read channel. Then, one of the seven exec() functions is called to start executing grep. grep reads from stdin, but since we dup2()'d standard input to the pipe's read channel, grep will be reading from the pipe.
When I handle redirection of IO to a file, a new file must be opened
and duped to STDOUT (I do not support input redirection). When does
this descriptor get closed? I've seen in examples that it gets closed
immediately after the call to dup2, but then how does anything get
written to the file if the file has been closed?
So, when you call dup2(a, b), either one of these is true:
a == b. In this case, nothing happens and dup2() returns prematurely. No file descriptors are closed.
a != b. In this case, b is closed if necessary, and then b is made to refer to the same file table entry as a. The file table entry is a structure that contains the current file offset and file status flags; multiple file descriptors can point to the same file table entry, and that's exactly what happens when you duplicate a file descriptor. So, dup2(a, b) has the effect of making a and b share the same file table entry. As a consequence, writing to a or b will end up writing to the same file. So the file that is closed is b, not a. If you dup2(a, STDOUT_FILENO), you close stdout and you make stdout's file descriptor point to the same file table entry as a. Any program that writes to stdout will then be writing to the file instead, since stdout's file descriptor is pointing to the file you dupped.
UPDATE:
So, for your specific problem, here's what I have to say after briefly looking through the code:
You shouldn't be calling close(STDOUT_FILENO) in here:
if (command->getOutputFD() == REDIRECT) {
if ((outfd = open(outfile, O_CREAT | O_WRONLY | O_TRUNC)) == -1)
return false;
if (dup2(outfd, STDOUT_FILENO) == -1)
return false;
close(STDOUT_FILENO);
}
If you close stdout, you will get an error in the future when you try to write to stdout. This is why you get ls: write error: Bad file descriptor. After all, ls is writing to stdout, but you closed it. Oops!
You're doing it backwards: you want to close outfd instead. You opened outfd so that you could redirect STDOUT_FILENO to outfd, once the redirection is done, you don't really need outfd anymore and you can close it. But you most definitely don't want to close stdout because the idea is to have stdout write to the file that was referenced by outfd.
So, go ahead and do that:
if (command->getOutputFD() == REDIRECT) {
if ((outfd = open(outfile, O_CREAT | O_WRONLY | O_TRUNC)) == -1)
return false;
if (dup2(outfd, STDOUT_FILENO) == -1)
return false;
if (outfd != STDOUT_FILENO)
close(outfd);
}
Note the final if is necessary: If outfd by any chance happens to be equal to STDOUT_FILENO, you don't want to close it for the reasons I just mentioned.
The same applies to the code inside else if (command->getOutputFD() == REDIRECTAPPEND): you want to close outfd rather than STDOUT_FILENO:
else if (command->getOutputFD() == REDIRECTAPPEND) {
if ((outfd = open(outfile, O_CREAT | O_WRONLY | O_APPEND)) == -1)
return false;
if (dup2(outfd, STDOUT_FILENO) == -1)
return false;
if (outfd != STDOUT_FILENO)
close(STDOUT_FILENO);
}
This should at least get you ls -l to work as expected.
As for the problem with the pipes: your pipe management is not really correct. It's not clear from the code you showed where and how pipefd is allocated, and how many pipes you create, but notice that:
A process will never be able to read from a pipe and write to another pipe. For example, if outfile is not STDOUT and infile is not STDIN, you end up closing both the read and the write channels (and worse yet, after closing the read channel, you attempt to duplicate it). There is no way this will ever work.
The parent process is closing every pipe before waiting for the termination of the children. This provokes a race condition.
I suggest redesigning the way you manage pipes. You can see an example of a working bare-bones shell working with pipes in this answer: https://stackoverflow.com/a/30415995/2793118
I have a C++ code that calls a test. I am doing a system call to execute this test. When this test fails, it will display something like this " ERROR: One or more devices of following component type(s) could not be discovered:"
I have a C++ code that runs on Linux redhat and it is capable of detecting if the system call pass or failed. But it can not capture the error message (ERROR: One or more devices of following component type(s) could not be discovered:) and append into the log file or print it.
Can someone please tell me how to capture the error message (ERROR: One or more devices of following component type(s) could not be discovered:)?
PS: I am an intern, any help would be really nice.
#include<iostream.h>
int main ()
{
int i;
if (system(NULL))
puts ("Ok");
else
exit (1);
i=system("hpsp --discover -verbose --user Admin --oapasswd password");
printf ("The value returned was: %d.\n",i);
return false;
}
Instead of using system() use popen(). This will open a pipe capturing the standard output of the test program so that your program can read it via the pipe.
Example EDITED:
#define _BSD_SOURCE 1
#define BUFFSIZE 400
#include <stdio.h>
#include <string.h>
int main(int argc, char *argv[])
{
char *cmd = "hpsp --discover -verbose --user Admin --oapasswd password";
char buf[BUFFSIZE];
char* searchResult;
int testPassed = 0;
FILE *ptr;
if ((ptr = popen(cmd, "r")) != NULL)
while (fgets(buf, BUFFSIZE, ptr) != NULL)
{
if ((searchResult = strstr(buf, "The test passed")) != NULL )
{
testPassed = 1;
break;
}
}
if (testPassed)
printf("yea!!\n");
else
printf("boo!!\n");
pclose(ptr);
return 0;
}
You can use dup and dup2 to backup/store the stderr file descriptor to redirect to your log file. Well, I'm guessing that errors go to stderr anyways.
Here's an example if you just want to write to a log file.
//open log file, choose whatever flags you need
int logfd = open("whateveryourlogfileis", O_APPEND);
//back up stderr file descriptor
int stderr_copy = dup(STDERR_FILENO);
//redirect stderr to your opened log file
dup2(logfd, STDERR_FILENO);
//close the original file descriptor for the log file
close(logfd);
//system call here
//restore stderr
dup2(stderr_copy, STDERR_FILENO);
//close stderr copy
close(stderr_copy);
Note: dup2 closes the target file descriptor before dup2ing to it. dup just duplicates the file descriptor and returns to you the new file descriptor.