I am spawning a process in python using subprocess and want to read output from the program using pipes. The C++ program does not seem to close the pipe though, even when explicitly telling it to close.
#include <cstdlib>
#include <ext/stdio_filebuf.h>
#include <iostream>
int main(int argc, char **argv) {
int fd = atoi(argv[1]);
__gnu_cxx::stdio_filebuf<char> buffer(fd, std::ios::out);
std::ostream stream(&buffer);
stream << "Hello World" << std::endl;
buffer.close();
return 0;
}
I invoke this small program with this python snippet:
import os
import subprocess
read, write = os.pipe()
proc = subprocess.Popen(["./dummy", str(write)])
data = os.fdopen(read, "r").read()
print data
The read() method does not return, as the fd is not closed. Opening and closing the write fd in python solves the problem. But it seems like a hack to me. Is there a way to close the fd in my C++ process?
Thanks a lot!
Spawning a child process on Linux (all POSIX OSes, really) is usually accomplished via fork and exec. After fork, both processes have the file open. The C++ process closes it, but the file remains open until the parent process closes the fd also. This is normal for code using fork, and usually is handled by a wrapper around fork. Read the man page for pipe. I guess python has no way of knowing which files are being transferred to the child, though, and therefore doesn't know what to close in the parent vs the child process.
POSIX file descriptors are local to the process. The file descriptor write from the Python tool is not valid in the C++ process.
Perhaps the easiest way would be to have the C++ process write its output to stdout (like cout <<), and Python call Popen using stdout=PIPE and read proc.stdout (or use proc.communicate() instead of using fdopen. This should work in Windows, too.
For passing the file descriptor as a command-line argument, see Ben Voigt's answer.
Related
Consider we have some_function and it prints result to stdout instead returning it.Changing it's defination is out of our scope and there's no alternative to it. We're left with option of reading it from stdout. So the question.
How to read stdout of C++ program in itself.
It is possible to get pid I searched if we can get fd of the same programm but I'm not able to find anything.
#include <unistd.h>
#include <sys/types.h>
#include <iostream>
void some_function(){
std::cout<<"Hello World";
}
int main(){
int pid = ::getpid();
string s = //What to write here.
cout<<"Printing";
some_function(); //This function prints "Hello World" to screen
cout<<s; //"PrintingHello World"
return 0;
}
How to attach pipe to same process i.e instead of creating child process.
Some might think of creating child process and call some_function in it, to be able to read its stdout in parent process, but No, some_function depends on process which calls it and hence we want to call it the very process instead of creating child process.
This isn't hard to do, but IMO it's quite a hack, and it won't work with a multithreaded program:
// make a temp file to store the function's stdout
int newStdOut = mkstemp( "/tmp/stdout.XXXXXXX" );
// save the original stdout
int tmpStdOut = dup( STDOUT_FILENO );
// clear stdout
fflush( stdout );
// now point the stdout file descriptor to the file
dup2( newStdOut, STDOUT_FILENO );
// call the function we want to collect the stdout from
some_function();
// make sure stdout is empty
fflush( stdout );
// restore original stdout
dup2( tmpStdOut, STDOUT_FILENO );
// the tmp file now contains whatever some_function() wrote to stdout
Error checking, proper headers, syncing C stdout with C++ cout, and reading from and cleaning up the temp file are left as exercises... ;-)
Note that you can't safely use a pipe - the function can write enough to fill up the pipe, and you can't read from the pipe because you've called the function.
How to read stdout of C++ program in itself?
There are very few reasons to do that and that is usually (but not always) a design bug.
Be aware of an important thing (at least in a single-threaded program). If your program is both reading from its "stdout" and writing (as usual) in it, it could be stuck in a deadlock: unable to read so not reaching any output routine, (or unable to write because the pipe is full).
So a program which both reads and writes the same thing (actually, the two sides of the same pipe(7)) should use some multiplexing call like poll(2). See also this.
Once you understand that, you'll have some event loop. And before that, you'll make a pipe(7) using pipe(2) (and dup2(2)).
However, pipe to self is a good thing in some signal(7) handling (see signal-safety(7)). That trick is even recommended in Qt Unix signal handling.
Read more about Unix system programming, e.g. ALP or some newer book. Read also intro(2) & syscalls(2).
I have looked for pipe and it requires fd
Wrong. Read much more carefully pipe(2); on success it fills an array of two file descriptors. Of course it could fail (see errno(3) & perror(3) & strerror(3))
Maybe you just need popen(3). Or std::ostringstream. Or open_memstream(3).
Consider we have some_function and it prints result to stdout instead returning it. Changing it's definition is out of our scope and there's no alternative to it
If some_function is your code, or is some free software, you could and probably should improve it to give a result somewhere....
I'm trying to write a C++ program which will start new process (some script, programm, command) like daemon and gets some info from it (stdout). I'm trying to use popen(). But subprocess finishing when main program complete. I dont know, does C++ have something easy in use like Python (subprocessing). There is my code:
#include <stdio.h>
#include <iostream>
using namespace std;
int main(int argc, char *argv[]) {
FILE *in;
char buff[512];
in = popen(argv[1], "r");
fgets(buff, sizeof(buff), in);
cout << buff;
}
P.S: & in the end of executive command doesn't helps.
fgets doesn't wait for the subprocess to complete, and neither does popen.
You need to read until the end of in:
while (fgets(buff, sizeof(buff), in))
{
cout << buff;
}
Reference for popen function: http://man7.org/linux/man-pages/man3/popen.3.html
Excerpt:
"The popen() function opens a process by creating a pipe, forking, and invoking the shell."
I think that because of the fork mechanism, when your (calling) process ends, its child process (the called one) is stopped too, at least as default behaviour.
One option is to keep reading from the pipe (and keep the caller process running) until the child process ends.
Your code is also missing a pclose(in) before exiting.
See also this other post on StackOverflow.
Have a look here https://github.com/arun11299/cpp-subprocess
This library can be exactly what you're looking for.
I am writing a baby program for practice. What I am trying to accomplish is basically a simple little GUI which displays services (for Linux); with buttons to start, stop, enable, and disable services (Much like the msconfig application "Services" tab in Windows). I am using C++ with Qt Creator on Fedora 21.
I want to create the GUI with C++, and populating the GUI with the list of services by calling bash scripts, and calling bash scripts on button clicks to do the appropriate action (enable, disable, etc.)
But when the C++ GUI calls the bash script (using system("path/to/script.sh")) the return value is only for exit success. How do I receive the output of the script itself, so that I can in turn use it to display on the GUI?
For conceptual example: if I were trying to display the output of (systemctl --type service | cut -d " " -f 1) into a GUI I have created in C++, how would I go about doing that? Is this even the correct way to do what I am trying to accomplish? If not,
What is the right way? and
Is there still a way to do it using my current method?
I have looked for a solution to this problem but I can't find information on how to return values from Bash to C++, only how to call Bash scripts from C++.
We're going to take advantage of the popen function, here.
std::string exec(char* cmd) {
FILE* pipe = popen(cmd, "r");
if (!pipe) return "ERROR";
char buffer[128];
std::string result = "";
while(!feof(pipe)) {
if(fgets(buffer, 128, pipe) != NULL)
result += buffer;
}
pclose(pipe);
return result;
}
This function takes a command as an argument, and returns the output as a string.
NOTE: this will not capture stderr! A quick and easy workaround is to redirect stderr to stdout, with 2>&1 at the end of your command.
Here is documentation on popen. Happy coding :)
You have to run the commands using popen instead of system and then loop through the returned file pointer.
Here is a simple example for the command ls -l
#include <stdio.h>
#include <stdlib.h>
int main() {
FILE *process;
char buff[1024];
process = popen("ls -l", "r");
if (process != NULL) {
while (!feof(process)) {
fgets(buff, sizeof(buff), process);
printf("%s", buff);
}
pclose(process);
}
return 0;
}
The long approach - which gives you complete control of stdin, stdout, and stderr of the child process, at the cost of fairly significant complexity - involves using fork and execve directly.
Before forking, set up your endpoints for communication - pipe works well, or socketpair. I'll assume you've invoked something like below:
int childStdin[2], childStdout[2], childStderr[2];
pipe(childStdin);
pipe(childStdout);
pipe(childStderr);
After fork, in child process before execve:
dup2(childStdin[0], 0); // childStdin read end to fd 0 (stdin)
dup2(childStdout[1], 1); // childStdout write end to fd 1 (stdout)
dup2(childStderr[1], 2); // childStderr write end to fd 2 (stderr)
.. then close all of childStdin, childStdout, and childStderr.
After fork, in parent process:
close(childStdin[0]); // parent cannot read from stdin
close(childStdout[1]); // parent cannot write to stdout/stderr
close(childStderr[1]);
Now, your parent process has complete control of the std i/o of the child process - and must safely multiplex childStdin[1], childStdout[0], and childStderr[0], while also monitoring for SIGCLD and eventually using a wait-series call to check the process termination code. pselect is particularly good for dealing with SIGCLD while dealing with std i/o asynchronously. See also select or poll of course.
If you want to merge the child's stdout and stderr, just dup2(childStdout[1], 2) and get rid of childStderr entirely.
The man pages should fill in the blanks from here. So that's the hard way, should you need it.
I'm developing a cross platform project currently. On windows i had a class that ran a process/script (using a commandline), waited for it to end, and read everything from it's stdout/stderr to a buffer. I then printed the output to a custom 'console'. Note: This was not a redirection of child stdout to parent stdout, just a pipe from child stdout to parent.
I'm new to OSX/unix-like api's but i can understand the canonical way of doing something like this is forking and piping stdouts together. However, i dont want to redirect it to stdout and i would like to capture the output.. It should work pretty much like this (pseudocode, resemblance with unix functions purely coincidental):
class program
{
string name, cmdline;
string output;
program(char * name, char * cmdline)
: name(name), cmdline(cmdline) {};
int run()
{
// run program - spawn it as a new process
int pid = exec(name, cmdline);
// wait for it to finish
wait(pid);
char buf[size];
int n;
// read output of program's stdout
// keep appending data until there's nothing left to read
while (read(pid, buf, size, &n))
output.append(buf, n);
// return exit code of process
return getexitcode(pid);
}
const string & getOutput() { return output; }
};
How would i go about doing this on OSX?
E:
Okay so i studied the relevant api's and it seems that some kind of fork/exec combo is unavoidable. Problem at hand is that my process is very large and forking it really seems like a bad idea (i see that some unix implementations can't do it if the parent process takes up 50%+ of the system ram).
Can't i avoid this scheme in any way? I see that vfork() might be a possible contender, so maybe i could try to mimic the popen() function using vfork. But then again, most man pages state that vfork might very well just be fork()
You have a library call to do just that: popen. It will provide you with a return value of a file descriptor, and you can read that descriptor till eof. It's part of stdio, so you can do that on OSX, but other systems as well. Just remember to pclose() the descriptor.
#include <stdio.h>
FILE * popen(const char *command, const char *mode);
int pclose(FILE *stream);
if you want to keep output with absolutely no redirection, the only thing we can think of is using something like "tee" - a command which splits the output to a file but maintains its own stdout. It's fairly easy to implement that in code as well, but it might not be necessary in this case.
I want to run a dos command from my program for example "dir" command.
I am doing it like,
system("dir");
Is there any way to read the output of that command directly into a program variable?
We can always redirect the output to a file and then read that file, by doing
system("dir > command.out");
And then reading command.out file. But how can we do it directly rather than redirectling to a file and then reading?
You can't redirect it to a variable, but you can do a trick similar to how pipes are used in Unix for chaining commands. Call CreateProcess(), and pass it a STARTUPINFO instance with accordingly set handles and STARTF_USESTDHANDLES in STARTUPINFO::dwFlags. Then read the data coming from the spawned process through the set handles.
If your library has popen() POSIX function, that's what you need. You can read command output from pipe and parse it any way you like.
FILE *dir;
char direntry[80];
dir = popen("dir", "r");
while (!feof(dir)) {
fgets(direntry, sizeof(direntry), dir);
/* do something with direntry */
}
Found an alternate way or rather windows equivalent of popen. It is _popen(). This works just right for me and moreover it's easy to use.
char psBuffer[128];
FILE *pPipe;
if( (pPipe = _popen( "dir", "rt" )) != NULL)
{
while(fgets(psBuffer, 128, pPipe))
{
printf(psBuffer);
}
}
Find the details with full example here.
You can't. The programs run in different memory spaces, as they are different processes. Generally, in modern operating systems, processes don't share memory.
Also, it would be difficult to define a variable in C that can hold the output of a command such as "dir"; it's would need to dynamically grow to make room.
The best way is to use a pipe, that will make it possible to read the command's output from a stream, from which you can store it as you see fit.
Use popen() it does exactly what you want.
It creates a bidirectional pipe, forks the processes. In the child it then connects the pipe to standard in and standard out then execs the command specified as the first parameter to popen().
#include <stdio.h>
#include <string>
#include <iostream>
int main()
{
std::string output;
FILE* data = popen("cat PLOP","r");
for(char c = getc(data);c != EOF;c = getc(data))
{
output += c;
}
pclose(data);
std::cout << "Data(" << output << ")\n";
}