I have two processes written in C++, piped one after the other. One gives some information to the other's stdin, then they both go on to do something else.
The problem is that the second process hangs inside cin.getline(), even though there's no more data to be exchanged. The solution was for the first process to fclose(stdout), and that works, except when I use the process wrapped up in a script. So apparently the stdout of the script is still open after closing it by the process - which seems fair but in my case, can I close it? Thanks
Since your program doesn't terminate, you can exec your-program in the script instead of just your-program and save an open file descriptor at the writing end of the pipe (and a bunch of other things).
Alternatively, start your program in the background and exit the script.
You can also close the standard output, but if you do that before you start your program, it won't be able to use the closed file descriptor. So you have to close it while the program is running. This is not exactly trivial. I can think of starting the program in the background, closing the standard output (use exec 1>&- for that) and bringing the program back to the foreground.
Related
My running process handles stdin by using getchar(). It works fine when I run it in foreground. However if I run it in background and do echo "a">> /proc/pid/fd/0 it won't work. On my system, /proc/pid/fd/0 is as same as /proc/pts/0, so how do I send to the process's stdin so that getchar() can see it? I'm working in C++ over ssh.
When you run multiple programs in background, they still have /dev/pts/XX as their control terminal (and stdin), but they are no longer eligible to read from it -- only shell or foreground task can do that. If they do, they'll get SIGTTIN signal that stops background process:
myaut#zenbook:~$ cat &
[1] 15250
myaut#zenbook:~$
[1]+ Stopped cat
Reasoning for such behavior is simple: multiple programs reading from one source leads to race condition. I.e. when you input to shell who am i, shell will read who, background task #1 will read am and task #2 will read i.
The solution is simple -- do not use pseudo-terminals to transfer data between processes:
Use pipes -- unnamed or named (with mkfifo). They are as simple as reading from stdin. Modern shells also provide coprocesses that allow to avoid named pipes.
Use UNIX sockets in complex cases
If you still need a pseudo-terminal, create a new one for your program with screen or other terminal emulator.
My Eclipse project is a C++ with makefile project. After at the end of the makefile, there is a call to the executable as if from terminal. (./myEXE)
This program is expected to accept commands from the terminal upon runtime (it is a physics simulation using MPI, with possibility to input "stop" or "stats" or "stop" commands while the simulation runs).
The input is written (not my original code) with a select() (from library sys/types, this for example) to see if there is anything readable from the stdin. The timeout option for select() makes sure MPI has time to start. After trying to read input for a while, it will check that the MPI workloads progress, and if they don't, it will raise timeout error.
Everything works like a charm when I call the makefile from terminal. It is broken when trying to run from Eclipse (shift+F9 and so on...)
It seems the problem is that stdin is always readable, thus checks on MPI before it has the possibility to initiate simulations - select() returns after <1ms.
My two main questions are therefore:
Where is Eclipse reading stdin from?
Why is it always readable?
P.S. Since the program is called via MPI in the makefile instead of directly from Eclipse, its a bit tricky to debug it all....
When select says stdin is readable, you must make sure to change its state before you call select again, otherwise it will just return immediately. Your code doesn't change its state but instead just calls select again, causing it to loop forever.
How you should fix it depends on whether stdin being closed is a fatal condition for your program. If your program must have a working stdin to continue, then if it gets an EOF while reading stdin, it should terminate, not just keep selecting blindly. If your program can continue to run usefully even without a working stdin, then it should take it out of its select set if it closes or errors. (Or stop calling select, depending on the logic.)
But you can't just ignore the case and keep running as if nothing happened.
The situation is that I have program started through system() or CreateProcess().
Now, is it possible to do stuff as that program outputs data into console. I mean as the program outputs it. That is not wait for the end, gather data and then process it, but just in the moment that this external program calls console with data that it wants to print, and then get hold of that data, process it and output something else on the console.
The easiest way is usually to start the program with _popen(your_program, "r");. That will return a FILE * you can read from, and what it reads will be whatever the child writes to its standard output. When you read EOF on that file, it means the child process has terminated. This makes it relatively easy to read and process the output from the child in real time.
On Linux, create a named pipe:
system("mkfifo pipename")
Then open the pipe in the first program, and start the program with:
system("program > pipename")
I'm not sure how to do this on Windows.
Call AllocConsole before creating child process, or use AttachConsole(ChildPID) function (in parent process).
After that, you may use any ReadConsoleXXX or WriteConsoleXXX functions.
I'm looking at the code for a c++ program which pipes the contents of a file to more. I don't quite understand it, so I was wondering if someone could write pseudocode for a c++ program that pipes something to something else? Why is it necessary to use fork?
create pipe
fork process
if child:
connect pipe to stdin
exec more
write to pipe
You need fork() so that you can replace stdin of the child before calling, and so that you don't wait for the process before continuing.
You will find your answer precisely here
Why is it necessary to use fork?
When you run a pipeline from the shell, eg.
$ ls | more
what happens? The shell runs two processes (one for ls, one for more). Additionally, the output (STDOUT) of ls is connected to the input (STDIN) of more, by a pipe.
Note that ls and more don't need to know anything about pipes, they just write to (and read from) their STDOUT (and STDIN) respectively. Further, because they're likely to do normal blocking reads and writes, it's essential that they can run concurrently. Otherwise ls could just fill the pipe buffer and block forever before more gets a chance to consume anything.
... pipes something to something else ...
Note also that aside from the concurrency argument, if your something else is another program (like more), it must run in another process. You create this process using fork. If you just run more in the current process (using exec), it would replace your program.
In general, you can use a pipe without fork, but you'll just be communicating within your own process. This means you're either doing non-blocking operations (perhaps in a synchronous co-routine setup), or using multiple threads.
I am creating a child-parent fork() to be able to communicate with a shell(/bin/sh) from the parent through a pipe.
The problem is:
In a parent I set a select() on a child output, but it unblocks only when the process is finished! So when I run say ps it's okay. but when I run /bin/sh it does not output until shell exits. But I want to read it's output!
for(;;) {
select(PARENT_READ+1,&sh,NULL,NULL,NULL); // This unblocks only when shell exits!
if (FD_ISSET(PARENT_READ,&sh)) {
while (n = read (PARENT_READ, &buf,30)) {
buf[30]='\0';
printf("C: %s\n",buf);
};
};
}
The answer is somewhere in the field of disabling buffering of pipes?
A lot of programs change their behavior depending on whether or not they think they're talking to a terminal (tty), and the shell definitely does this this. Also, the default C streams stdout and stderr are probably unbuffered if stdout is a tty, and fully buffered otherwise - that means that they don't flush until the internal buffer is full, the program explicitly flushes them, or the program ends.
To work around this problem, you have to make your program pretend to be a terminal. To do this, you can use your system's pseudo-terminal APIs (try man 7 pty). The end result is a pair of file descriptors that sort-of work like a pipe.
Also, as an aside, when select unblocks, you should read exactly once from the triggered file descriptor. If you read more than once, which is possible with the loop you've got there, you risk blocking again on subsequent reads, unless you've got your FD in non-blocking mode.
However, I have to ask: why do you need to interact with the shell in this way? Is it possible to, say, just run a shell script, or use "/bin/sh -c your_command_here" instead? There are relatively few programs that actually need a real terminal to work correctly - the main ones are programs that prompt for a password, like ssh, su or sudo.