In my C++ program, I need to start a very long running new process and monitor its I/O. I cannot modify the source code of the program in question.
I was thinking of create a new thread and starting the process in it and sending the output continuously (which will be coming out asynchronously) to main thread.
My code for creating the process currently looks like this:
std::string SysExec::exec(char* cmd) {
FILE* pipe = popen(cmd, "r");
if (!pipe)
return "ERROR";
char buffer[128];
std::string result = "";
while (!feof(pipe)) {
if (fgets(buffer, 128, pipe) != NULL)
result += buffer;
}
pclose(pipe);
return result;
}
However, if called from main thread, it will make the main program stop (because of while (!feof(pipe))). How should I modify this? Or is there any better way to do this?
Related
I need to create pipe/subprocess and get output from it as it will change and pass it as string to main process. It would be great if there is some crossplatform solution to it (but right now windows is main priority)
I tried Observer pattern plus Async from std (I don't think that it's working right).
All in C++ and WinApi (if needed)
Code:
void pipe::Pipe::syncMode()
{
//some code for subproccessing in Windows
}
void pipe::Pipe::asyncMode() {
std::async(std::launch::async, &pipe::Pipe::syncMode, this);
}
Thanks!
CreateProcess, if you prioritize Windows.
Can get you:
1. Standard Output and error from the child process.
2. Return code of the child process.
3. Can set security attributes.
4. Can inherit handles from parent process. And a lot more.
If you needed a cross platform solution, popen is your way to go.
popen is not as sophisticated as createprocess. But if you only want to invoke a child process and get its standard output, it is more than enough.
Sample popen code to create a process and get its standard output
std::wstring ExecuteCmdLine(std::string cmd) // NOLINT
//! Execute given comment in std::string and returns output as std::wstring
{
try
{
printf("##### \t Inside CMcUtil::executeCMDLine \t #####\n");
printf("The commnad to execute is : \"%s\"\n", cmd.c_str());
wchar_t buffer[128];
ATL::CStringW result = "";
FILE* pipe = _wpopen(CA2W(cmd.c_str()), L"r");
if (!pipe)
{
throw std::runtime_error("popen() failed!");
}
try
{
while (!feof(pipe)) {
if (fgetws(buffer, 128, pipe) != nullptr)
result += buffer;
}
}
catch (...)
{
_pclose(pipe);
throw;
}
_pclose(pipe);
return result.GetBuffer();
}
catch (...)
{
ERRMSG("Exception caught while executing command in command prompt\n");
return L"Error";
}
}
The application I'm working on needs to execute commands. Commands can be console commands or 'GUI applications' (like notepad).
I need to get the return code in both cases, and in the case of console commands I also need to catch the output from stdin and stderr.
In order to implement this feature, I based my code on the stack overflow question 'How to execute a command and get output of command within C++ using POSIX?'.
My code:
int ExecuteCmdEx(const char* cmd, std::string &result)
{
char buffer[128];
int retCode = -1; // -1 if error ocurs.
std::string command(cmd);
command.append(" 2>&1"); // also redirect stderr to stdout
result = "";
FILE* pipe = _popen(command.c_str(), "r");
if (pipe != NULL) {
try {
while (!feof(pipe)) {
if (fgets(buffer, 128, pipe) != NULL)
result += buffer;
}
}
catch (...) {
retCode = _pclose(pipe);
throw;
}
retCode = _pclose(pipe);
}
return retCode;
}
It works perfectly with console applications, but in the case of 'GUI applications' it doesn't work as expected...
With 'GUI applications', code stops on while (!feof(pipe)) expecting to get something from pipe.
I understand that 'GUI applications' like notepad don't finish until someone interacts with them (user closes the app, kills the process, etc.),
but when I launch console applications from Windows Console, prompt comes back immediately.
I would like to obtain the same behavior from 'GUI applications'...
One possible solution would be to add the isGui variable indicating when the code should read from the pipe, but I rejected this option, as I don't want to indicate if it is a 'GUI application' or not.
Well you don't have to indicate isGui yourself but detect it by checking the subsystem of the executable (windows/console) prior to executing the command, and in case of windows skip waiting on the redirected pipes.
For example, using SHGetFileInfo with the SHGFI_EXETYPE flag:
bool isGuiApplication(const std::string& command)
{
auto it = command.find_first_of(" \t");
const std::string& executable = (it == std::string::npos ? command : command.substr(0, it));
DWORD_PTR exetype = SHGetFileInfo(executable.c_str(), 0, nullptr, 0, SHGFI_EXETYPE);
if (!exetype) {
cerr << "Executable check failed\n";
}
return ((uintptr_t)exetype & 0xffff0000);
}
Then later in the code...
if (isGuiApplication(command)) {
cout << "GUI application\n";
system(command.c_str()); // don't wait on stdin
}
else {
cout << "Console application\n";
. . .
// _popen and stuff
}
I've got a piece of code that executes a process and retrieves the result.
namespace {
FILE* really_popen(const char* cmd, const char* mode) {
#ifdef _MSC_VER
return _popen(cmd, mode);
#else
return popen(cmd, mode);
#endif
}
void really_pclose(FILE* pipe) {
#ifdef _MSC_VER
_pclose(pipe);
#else
pclose(pipe);
#endif
}
std::string ExecuteProcess(std::string cmd) {
FILE* pipe = really_popen(cmd.c_str(), "r");
if (!pipe) throw std::runtime_error("Could not invoke command " + cmd);
char buffer[128];
std::string result = "";
while(!feof(pipe)) {
if(fgets(buffer, 128, pipe) != NULL)
result += buffer;
}
really_pclose(pipe);
return result;
}
}
This works just fine for me on Linux, but on Windows, it has a terrible habit of deadlocking- seems that fgets never returns. I've looked into the CRT sources and fgets eventually delegates to ReadFile, which never returns.
If I invoke the command from the command line, it returns within a second.
How can I read the output on Windows without deadlocking the parent?
If the child hasn't exited, then the call to fgets() won't exit, so you need to resolve why the child isn't exiting. The most likely cause in this sort of situation is that the child has hung because it has no standard input.
If that's the problem, you may be able to resolve it by changing the pipe mode to "rw". You won't typically need to do anything with the extra pipe, it just has to be there.
(As you mention in the comments, the problem can also be resolved by using the command shell redirection to give the child a handle to NUL as standard input.)
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I need to execute my console program, write some value to the standard input and read the result with pipe. I tried to implement it. This code works perfectly. But I think there are other easier ways to implement it. Do you have any ideas?
pid_t pid = fork();
std::string output_data;
if(pid < 0)
{
printf("Error\n");
}
else if(!pid)
{
FILE* process = popen(program.c_str(), "r");
char temp_data[128];
if(process)
{
while(fgets(temp_data, sizeof(temp_data), process))
{
output_data.append(temp_data);
}
}
pclose(process);
exit(1);
}
else
{
FILE* process = popen(program.c_str(), "w");
std::string output_data;
char temp_data[128];
if(process)
{
fwrite("5", 1, sizeof(int), process);
}
pclose(process);
}
It looks like your code will run 4 processes: One process forks into 2, and then each of them forks using popen(). popen() will not only create a pipe but also a child process connected through that pipe, thus each of the 2 processes calling popen() will create a child process.
You should decide how to go: Either let popen() do all the work, but then don't use fork(), or let pipe() create the pipe yourself and use fork() - that is what my code below does.
There is still one big difference: popen() will not only fork, but also, in the child, execute another program. My example stays in the same executable. If you want to execute another program in the child, then popen() can be quite useful, but then you should not use fork().
int fsd[2]; // write to fds[1], read from fds[0]
pipe(fds); // creates the pipe with read+write file descriptors in fds
id_t pid = fork();
std::string output_data;
if(pid < 0)
{
printf("Error\n");
}
else if(!pid)
{
close(fds[1]); // else we'll never detect end of input
FILE* process = fdopen(fds[0], "r");
char temp_data[128];
if(process)
{
while(fgets(temp_data, sizeof(temp_data), process))
{
output_data.append(temp_data);
}
}
pclose(process);
exit(1);
}
else
{
close(fds[0]); // just to clean up
FILE* process = fdopen(fds[1], "w");
std::string output_data;
char temp_data[128];
if(process)
{
fprintf(process, "5\n"); // your fwrite("5",...) worked by luck only
}
pclose(process);
}
Finally, if you want your process to launch another executable, write to that excutable and then read from it, you need to do this:
Create 2 pipes, one used to write from the parent to child and one to write back from child to parent, and then do some magic to re-assign the file descriptors:
int p2c[2]; pipe(p2c); // parent to child
int c2p[2]; pipe(c2p); // child to parent
if(fork()) { // parent - I ignore error checks here
close(p2c[0]); // the child reads from that
close(c2p[1]); // the child writes to that
FILE * f_w = fdopen(p2c[1],"w");
... write some data to the child using f_w ...
fclose(f_w);
FILE * f_r = fdopen(c2p[0],"r");
... read some data from the child using f_r ...
fclose(f_r);
wait(0); // just to avoid a zombie
} else { // in the child, change stdin and stdout
close(p2c[1]); close(0); dup(p2c[0]); close(p2c[0]);
close(c2p[0]); close(1); dup(c2p[1]); close(c2p[1]);
... now use execv or something similar to run the other executable
}
This question already has answers here:
How do I execute a command and get the output of the command within C++ using POSIX?
(12 answers)
Closed 7 years ago.
I'm trying to start an external application through system() - for example, system("ls"). I would like to capture its output as it happens so I can send it to another function for further processing. What's the best way to do that in C/C++?
From the popen manual:
#include <stdio.h>
FILE *popen(const char *command, const char *type);
int pclose(FILE *stream);
Try the popen() function. It executes a command, like system(), but directs the output into a new file. A pointer to the stream is returned.
FILE *lsofFile_p = popen("lsof", "r");
if (!lsofFile_p)
{
return -1;
}
char buffer[1024];
char *line_p = fgets(buffer, sizeof(buffer), lsofFile_p);
pclose(lsofFile_p);
EDIT: misread question as wanting to pass output to another program, not another function. popen() is almost certainly what you want.
System gives you full access to the shell. If you want to continue using it, you can
redirect it's output to a temporary file, by system("ls > tempfile.txt"), but choosing a secure temporary file is a pain. Or, you can even redirect it through another program: system("ls | otherprogram");
Some may recommend the popen() command. This is what you want if you can process the output yourself:
FILE *output = popen("ls", "r");
which will give you a FILE pointer you can read from with the command's output on it.
You can also use the pipe() call to create a connection in combination with fork() to create new processes, dup2() to change the standard input and output of them, exec() to run the new programs, and wait() in the main program to wait for them. This is just setting up the pipeline much like the shell would. See the pipe() man page for details and an example.
The functions popen() and such don't redirect stderr and such; I wrote popen3() for that purpose.
Here's a bowdlerised version of my popen3():
int popen3(int fd[3],const char **const cmd) {
int i, e;
int p[3][2];
pid_t pid;
// set all the FDs to invalid
for(i=0; i<3; i++)
p[i][0] = p[i][1] = -1;
// create the pipes
for(int i=0; i<3; i++)
if(pipe(p[i]))
goto error;
// and fork
pid = fork();
if(-1 == pid)
goto error;
// in the parent?
if(pid) {
// parent
fd[STDIN_FILENO] = p[STDIN_FILENO][1];
close(p[STDIN_FILENO][0]);
fd[STDOUT_FILENO] = p[STDOUT_FILENO][0];
close(p[STDOUT_FILENO][1]);
fd[STDERR_FILENO] = p[STDERR_FILENO][0];
close(p[STDERR_FILENO][1]);
// success
return 0;
} else {
// child
dup2(p[STDIN_FILENO][0],STDIN_FILENO);
close(p[STDIN_FILENO][1]);
dup2(p[STDOUT_FILENO][1],STDOUT_FILENO);
close(p[STDOUT_FILENO][0]);
dup2(p[STDERR_FILENO][1],STDERR_FILENO);
close(p[STDERR_FILENO][0]);
// here we try and run it
execv(*cmd,const_cast<char*const*>(cmd));
// if we are there, then we failed to launch our program
perror("Could not launch");
fprintf(stderr," \"%s\"\n",*cmd);
_exit(EXIT_FAILURE);
}
// preserve original error
e = errno;
for(i=0; i<3; i++) {
close(p[i][0]);
close(p[i][1]);
}
errno = e;
return -1;
}
The most efficient way is to use stdout file descriptor directly, bypassing FILE stream:
pid_t popen2(const char *command, int * infp, int * outfp)
{
int p_stdin[2], p_stdout[2];
pid_t pid;
if (pipe(p_stdin) == -1)
return -1;
if (pipe(p_stdout) == -1) {
close(p_stdin[0]);
close(p_stdin[1]);
return -1;
}
pid = fork();
if (pid < 0) {
close(p_stdin[0]);
close(p_stdin[1]);
close(p_stdout[0]);
close(p_stdout[1]);
return pid;
} else if (pid == 0) {
close(p_stdin[1]);
dup2(p_stdin[0], 0);
close(p_stdout[0]);
dup2(p_stdout[1], 1);
dup2(::open("/dev/null", O_WRONLY), 2);
/// Close all other descriptors for the safety sake.
for (int i = 3; i < 4096; ++i) {
::close(i);
}
setsid();
execl("/bin/sh", "sh", "-c", command, NULL);
_exit(1);
}
close(p_stdin[0]);
close(p_stdout[1]);
if (infp == NULL) {
close(p_stdin[1]);
} else {
*infp = p_stdin[1];
}
if (outfp == NULL) {
close(p_stdout[0]);
} else {
*outfp = p_stdout[0];
}
return pid;
}
To read output from child use popen2() like this:
int child_stdout = -1;
pid_t child_pid = popen2("ls", 0, &child_stdout);
if (!child_pid) {
handle_error();
}
char buff[128];
ssize_t bytes_read = read(child_stdout, buff, sizeof(buff));
To both write and read:
int child_stdin = -1;
int child_stdout = -1;
pid_t child_pid = popen2("grep 123", &child_stdin, &child_stdout);
if (!child_pid) {
handle_error();
}
const char text = "1\n2\n123\n3";
ssize_t bytes_written = write(child_stdin, text, sizeof(text) - 1);
char buff[128];
ssize_t bytes_read = read(child_stdout, buff, sizeof(buff));
The functions popen() and pclose() could be what you're looking for.
Take a look at the glibc manual for an example.
In Windows, instead of using system(), use CreateProcess, redirect the output to a pipe and connect to the pipe.
I'm guessing this is also possible in some POSIX way?
Actually, I just checked, and:
popen is problematic, because the process is forked. So if you need to wait for the shell command to execute, then you're in danger of missing it. In my case, my program closed even before the pipe got to do it's work.
I ended up using system call with tar command on linux. The return value from system was the result of tar.
So: if you need the return value, then not no only is there no need to use popen, it probably won't do what you want.
In this page: capture_the_output_of_a_child_process_in_c describes the limitations of using popen vs. using fork/exec/dup2/STDOUT_FILENO approach.
I'm having problems capturing tshark output with popen.
And I'm guessing that this limitation might be my problem:
It returns a stdio stream as opposed to a raw file descriptor, which
is unsuitable for handling the output asynchronously.
I'll come back to this answer if I have a solution with the other approach.
I'm not entirely certain that its possible in standard C, as two different processes don't typically share memory space. The simplest way I can think of to do it would be to have the second program redirect its output to a text file (programname > textfile.txt) and then read that text file back in for processing. However, that may not be the best way.