In my code below, I forked my process into a parent and child process. In the child process, I sent the c string argv[1] to the parent process to be printed. Then I made the child process sleep for 4 seconds before printing "This is the child process. Closing\n".
In the parent process, I want the string from the child process to be printed to stdout as soon as I receive it from the child process.
The problem arises here. Instead of immediately printing argv[1] in the parent process before the string "This is the child process. Closing\n" is printed 4 seconds later, what happens is this:
$ g++ -std=c++11 printchild.cpp -o printchild
$ ./printchild helloworld
1) 4 seconds passes
2) "This is the child process. Closing\n" is printed
3) "helloworld" is printed
Why is the output from the parent process blocked by the child process?
// printchild.cpp
#include <chrono>
#include <thread>
#include <cstdio>
#include <unistd.h>
#include <cstdlib>
int main(int argc, char * argv[]){
int pipefd[2];
pid_t cpid;
if(argc != 2){
fprintf(stderr, "Usage: %s <string>\n", argv[0]);
exit(EXIT_FAILURE);
}
if(pipe(pipefd) == -1){
perror("fork");
exit(EXIT_FAILURE);
}
cpid = fork();
if(cpid == 0){
close(pipefd[0]);
FILE *fpc = fdopen(pipefd[1],"wb");
fprintf(fpc,"%s",argv[1]);
fflush(fpc);
std::this_thread::sleep_for(std::chrono::seconds(4));
fprintf(stdout,"This is the child process. Closing\n");
return 0;
}else if(cpid == -1){
perror("Error in forking");
exit(EXIT_FAILURE);
}else{
char str[80];
close(pipefd[1]);
FILE* fp = fdopen(pipefd[0], "rb");
fgets(str,80,fp);
fprintf(stdout,"%s\n",str);
return 0;
}
}
The parent process is reading the child's message via fgets(). It will continue to read until one of three things happens:
enough bytes have been read to fill the buffer, less one for a string terminator
a newline is read
end-of-file is encountered
The child does not send enough bytes to exhaust the buffer, and it does not send a newline, so the parent's fgets() does not return until the child's end of the pipe is closed upon its exit.
You can fix this in the child by having it either terminate the message with a newline or close the stream immediately after writing.
If you close the file in the client right after flushing, it will work as expected:
fflush(fpc);
fclose(fpc);
Output:
[dbush] /tmp/x1 hello
hello
[dbush] This is the child process. Closing
Your problem is the fgets call.
It tries to fulfill your request fully, and that's the cause of the blocking.
If you're prototyping some IPC, I recommend you ditch stdio and go for the raw IO syscalls.
If you substitute, e.g.:
char buf[100];
read(pipefd[0], buf, 100);
fprintf(stdout,"%s\n",buf);
for the fgets part, you'll get your message right away, because read will give you whatever's available and return (if there's at least 1 byte in the pipe buffer at the time you make the read request), whereas fgets won't back off until it has read all the requested bytes or until it figures out it can't (EOF or an IO error).
(To split hairs, it's not fgets that's blocking (userspace code alone can't block) -- it's another momentarily unsatisfiable read syscall that fgets has issued).
Related
I have two separate programs in C++, one that writes to two named pipes in unpredictable intervals and one that should wait to read new content from the pipes whenever available. For simplicity, here my writer only writes two times to the pipes (1st: "One" "Tree" , 2nd: "Two" "Fogs").
My writer program is:
#include <unistd.h>
#include <iostream>
int main()
{
int fd1, fd2;
const char* myfifo1 = "/tmp/myfifo1";
const char* myfifo2 = "/tmp/myfifo2";
mkfifo(myfifo1, 0666);
mkfifo(myfifo2, 0666);
fd1 = open(myfifo1, O_WRONLY);
fd2 = open(myfifo2, O_WRONLY);
write(fd1, "One", sizeof("One"));
write(fd2, "Tree", sizeof("Tree"));
sleep(5);
write(fd1, "Two", sizeof("Two"));
write(fd2, "Fogs", sizeof("Fogs"));
close(fd1);
close(fd2);
unlink(myfifo1);
unlink(myfifo2);
}
My reader program is:
#include <fcntl.h>
#include <stdio.h>
#include <sys/stat.h>
#include <unistd.h>
#define MAX_BUF 1024
int main()
{
int fd1, fd2;
const char * myfifo1 = "/tmp/myfifo1";
const char * myfifo2 = "/tmp/myfifo2";
char buf1[MAX_BUF], buf2[MAX_BUF];
fd1 = open(myfifo1, O_RDONLY);
fd2 = open(myfifo2, O_RDONLY);
while (1) {
read(fd1, buf1, MAX_BUF);
read(fd2, buf2, MAX_BUF);
printf("Received: %s\n", buf1);
printf("Received: %s\n", buf2);
}
close(fd1);
close(fd2);
return 0;
}
I do not want the reader to terminate or close the connection between the reads, I need it to remain active and wait until the writer writes something new in the named pipe.
However, by simultaneously running these two programs (in different cores, first the writer and then the reader) I get:
Received: One
Received: Tree
Received: Two
Received: Fogs
Received: Two
Received: Fogs
Received: Two
Received: Fogs
Received: Two
Received: Fogs
...
...
The wanted behavior would be:
Received: One
Received: Tree
Received: Two
Received: Fogs
and then nothing (the reader should wait for another write).
I understand that I should somehow clear the buffer after I read it, but isn't it the default behavior of read? Since it finished reading "Two" (and also "Fogs"), why does it keep reading it and does not wait for new content?
What modifications should I make?
Thank you very much in advance!
Closing a pipe signals end-of-file to the other end.
read will read zero bytes (and return zero) whenever it encounters an end-of-file condition. Any subsequent read after the end-of-file is a no-op.
If you want to continue reading from the same pipe, you need to check the result of read, and if it returns zero, close the pipe and open it again.
Of course you should not unlink the pipe if you need someone else to continue using it.
One other thing to note: read (1) doesn't terminate the buffer with the 0 byte and (2) doesn't necessarily read the exact amount of bytes that was written by a single write. You should detect the end-of-message yourself, rather than relying on read and write to do it for you. It is entirely possible that read will read a half of the string without the terminating null character, then your printf will print garbage.
This question already has answers here:
How to make reading from `std::cin` timeout after a particular amount of time
(5 answers)
Closed 4 years ago.
In C++ on OSX, how do we check std::cin for data without waiting?
The expected data is not coming from a keyboard or human, but from a program that has started my executable and talks to my executable via stdin. This 'parent' program will tell me to start a long process, may interrupt, and I may periodically send data back to it via stdout, therefore the checking of stdin must not block. I'm using XCode.
So far I have attempted to test via the following, neither of which seem to work:
//Return true if there is data waiting version 1
return (std::cin.peek() != EOF);
//Return true if there is data waiting version 2
return (std::cin.rdbuf() && std::cin.rdbuf()->in_avail() >= 0);
To fill out the idea mentioned in a comment, the easiest way since C++11 is simply to start a thread that waits for input and, when it is received, add it to a shared queue container. Doing so requires use of mutex locking & unlocking. This container then can be used to check if empty and, if it has something in it, the data can just be read and popped off of the front of the queue. Something like the following:
#include <thread>
#include <string>
#include <iostream>
std::mutex qmutex;
std::queue<std::string> queue;
void listen() {
std::string sin;
while (true) {
std::cin >> sin; // Thread will wait here for stdin to arrive
qmutex.lock();
queue.push(sin);
qmutex.unlock();
}
}
void mainstuff() {
std::thread t1(listen);
while (true) {
qmutex.lock();
if (!queue.empty()) {
//There's something in the queue, read it/take a copy
queue.pop();
}
qmutex.unlock();
// Respond to input
// Continue work
}
}
I don't believe cin supports non-blocking I/O operations or timeouts.
Here's an example of the lower-level way to accomplish the task, using select() and file descriptors. This program will immediately print to stdout any text it receives from stdin, and also wake up every 5 seconds (even if no text is received from stdin) just to print a fixed message. You can modify the wakeup timeout to whatever you like (including zero for an instantaneous-poll behavior), and/or add more file descriptors for it to react to if you prefer a more event-driven approach.
#include <stdio.h>
#include <sys/select.h>
static void RegisterFileDescriptor(int fd, int & maxFD, fd_set & fdset)
{
FD_SET(fd, &fdset);
if (fd > maxFD) maxFD = fd;
}
int main()
{
const int stdinFileno = fileno(stdin);
// This call is just here so that our parent process will immediately see any
// text lines that we print to stdout. If you're only reading from stdin and never
// writing responses back to stdout, then it's not really necessary.
(void) setlinebuf(stdout);
fd_set readSet;
while(true)
{
int maxFD = -1;
FD_ZERO(&readSet);
RegisterFileDescriptor(stdinFileno, maxFD, readSet);
// If you want to react to incoming data on any other
// file descriptors as well, you could call
// RegisterFileDescriptor on them here
struct timeval timeoutDelay = {5, 0}; // let's force a wakeup every 5 seconds
if (select(maxFD+1, &readSet, NULL, NULL, &timeoutDelay) < 0)
{
perror("select");
break;
}
printf("select() returned!\n");
if (FD_ISSET(stdinFileno, &readSet))
{
char buf[512];
if (fgets(buf, sizeof(buf), stdin)) printf("Read from stdin: [%s]\n", buf);
}
}
return 0;
}
Before the below code I do:
create 1 pipe to read output from forked process
fork()
execv() a python script
Then in the parent process I do:
//set pipes to non-Blocking
File * cout_f = fdopen(cout_pipe[0], "r");
int flags = fcntl(cout_pipe[0], F_GETFL, 0);
fcntl(cout_pipe[0], F_SETFL, flags|O_NONBLOCK);
// read from pipe and send it up through a callback method
int stat;
size_t size = 0;
char buffer [ 1000 ];
do
{
while((size = fread(buffer, sizeof(char), sizeof(char)*1000, cout_f))!=0)
{
call_back_function(buffer, size);
}
}while(waitpid(child_pid, &stat, WNOHANG) != -1)
//Do 1 extra read
while((size = fread(buffer, sizeof(char), sizeof(char)*1000, cout_f))!=0)
{
call_back_function(buffer, size);
}
The problem I am facing happens when the child process prints to stdout and exits (normally) before flushing. I miss what was sent in the pipe.
Here are my questions:
Is the above code safe/correct or can it be improved?
Is there a way to read the entire pipe at the last moment when the subprocess dies, even if it doesn't flush its stdout?
You don't have to wait. Just send everything as you read it. That's what your code already does. Get rid of non-blocking mode; get rid of the while(wait(...)) condition; get rid of the final read; and just perform the first read loop until end of stream. Then call wait() to get the exit code.
If the process also produces on stderr you will need to read that in another thread, otherwise when the stderr buffer fills, it will block.
I use a function (L) to execute another program (K) through execlp().
In the K program, the result is written into stream 1:
write(1, (char *)&o, sizeof(int));
Since after execlp(), the remaining part of L won't be executed, how could I get the result written in stream 1?
Don't ask me why I need to do it this way. It's the requirement for a project.
I followed your guys advices, but now the problem is, the way the K program get the arguments are from streams(one standard stream, one other stream), I'm using pipes to write arguments into corresponding streams (it's done by the parent).
After the child exec, in the parent part, I read from stream 0 (the K program writes its result back in stream 1). But what I could get is what the parent wrote into the stream, not what the K program write back.
What's wrong? Do I need to add another pipe?
Thanks!!
The key insight, which Jonathan Leffler mentioned in his comment, is that you need to fork the program which is running L before you call execlp().
After the fork, the parent continues to execute the rest of L, and the child morphs into the program K by calling execlp(), which should never return unless there is an error.
Thus, the assertion that "the remaining part of L won't be executed" is incorrect. It will get executed, in the parent process if you write the function L correctly.
Update: Since the OP made his question more specific, I am appending to this answer.
If you want to retrieve what the child process wrote to stdout (fd 1), you need to create a new pipe before the fork, and copy the writing end of this pipe into the child's stdout.
Here is an example program, slightly modified from the pipe man page.
#include <sys/types.h>
#include <sys/wait.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
int
main(int argc, char *argv[])
{
int pipefd[2];
pid_t cpid;
char buf;
if (pipe(pipefd) == -1) {
perror("pipe");
exit(EXIT_FAILURE);
}
cpid = fork();
if (cpid == -1) {
perror("fork");
exit(EXIT_FAILURE);
}
if (cpid == 0) { /* Child writes into the pipe */
close(pipefd[0]); /* Close unused read end */
// Copy the writing end of the pipe into STDOUT for the child.
dup2(pipefd[1], STDOUT_FILENO);
// Execute your program L here, and its stdout will be captured by the pipe.
const char* message = "Child is speaking to stdout!";
write(STDOUT_FILENO, message, strlen(message));
write(STDOUT_FILENO, "\n", 1);
close(pipefd[1]);
_exit(EXIT_SUCCESS);
} else { /* Parent reads child's stdout from the pipe */
close(pipefd[1]); /* Close unused write end */
// Here the parent process is reading the child's stdout.
while (read(pipefd[0], &buf, 1) > 0)
write(STDOUT_FILENO, &buf, 1);
wait(NULL); /* Wait for child */
exit(EXIT_SUCCESS);
}
}
I am using the following code to redirect stdout to a pipe, then read all the data from the pipe to a buffer. I have 2 problems:
first problem: when i send a string (after redirection) bigger then the pipe's BUFF_SIZE, the program stops responding (deadlock or something).
second problem: when i try to read from a pipe before something was sent to stdout. I get the same response, the program stops responding - _read command stuck's ...
The issue is that i don't know the amount of data that will be sent to the pipe after the redirection.
The first problem, i don't know how to handle and i'll be glad for help. The second problem i solved by a simple workaround, right after the redirection i print space character to stdout. but i guess that this solution is not the correct one ...
#include <fcntl.h>
#include <io.h>
#include <iostream>
#define READ 0
#define WRITE 1
#define BUFF_SIZE 5
using namespace std;
int main()
{
int stdout_pipe[2];
int saved_stdout;
saved_stdout = _dup(_fileno(stdout)); // save stdout
if(_pipe(stdout_pipe,BUFF_SIZE, O_TEXT) != 0 ) // make a pipe
{
exit(1);
}
fflush( stdout );
if(_dup2(stdout_pipe[1], _fileno(stdout)) != 0 ) //redirect stdout to the pipe
{
exit(1);
}
ios::sync_with_stdio();
setvbuf( stdout, NULL, _IONBF, 0 );
//anything sent to stdout goes now to the pipe
//printf(" ");//workaround for the second problem
printf("123456");//first problem
char buffer[BUFF_SIZE] = {0};
int nOutRead = 0;
nOutRead = _read(stdout_pipe[READ], buffer, BUFF_SIZE); //second problem
buffer[nOutRead] = '\0';
// reconnect stdout
if (_dup2(saved_stdout, _fileno(stdout)) != 0 )
{
exit(1);
}
ios::sync_with_stdio();
printf("buffer: %s\n", buffer);
}
Your problem is that you are using blocking I/O calls, while both ends of the pipe are connected to the same process. If you don't know how much data there will be, this is just a deadlock situation waiting to happen.
printf is a blocking call, which means that it will not return until all data has been written to the output device (the pipe in this case), or until a write error is signalled (for example, the other end of the pipe is closed).
_read works similarly. It only returns when it has a full buffer worth of data or it knows that the end of the input has been reached (which can be signalled by closing the write-end of the pipe).
The only ways around this are
to use non-blocking I/O (which is not feasible if you don't have access to the code that calls printf), or
to ensure the reading and writing happens in different processes or threads, or
to use a temporary file for buffering, instead of the buffer of a pipe.
Pipes are unidirectional. Ie. you can either write to a pipe (x)or you can read from it.
To simulate a pipeline, try the following (the below is C, not C++):
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int main(int argc)
{
int pfds[2];
pipe(pfds);
if (!fork()) {
close(1); /* close stdout, check for errors */
dup(pfds[1]); /* make stdout same as pfds[1], dup reuses lowest fd */
close(pfds[0]); /* not needed */
execlp("ls", "ls", NULL); /* or write() in whatever way you want */
} else {
close(0); /* close stdin, check for errors please! */
dup(pfds[0]); /* make stdin same as pfds[0] */
close(pfds[1]); /* not needed on this end */
execlp("wc", "wc", "-l", NULL); /* or read() */
}
return 0;
}
[edit] By the way, your code does not overflow a buffer. Its only relation to buffer overflow is that you're reading into a statically allocated array...if you read() more than sizeof buffer elements, then you'll run into problems.
You must use non-blocking I/O if you don't want read or write to be blocked in this case.