I'm trying to multiplex an unnamed pipe with some other file descriptors.
The problem is that the pipe file descriptor always appears in the result of select. In other words the event-loop reads from pipe for infinite times. Here is a metaphor of what I want to do, and what actually happens.
#include <iostream>
#include <stdlib.h>
#include <algorithm>
#include <stdio.h>
#include <errno.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <unistd.h>
using namespace std;
int main(){
fd_set master; // master file descriptor list
fd_set read_fds; // temp file descriptor list for select()
FD_ZERO(&master); // clear the master and temp sets
FD_ZERO(&read_fds);
int fdmax; // maximum file descriptor number
int pfd[2];
if(pipe(pfd)!=0) {cout<<"Unable to create a pipe.\n",exit(1);};
FD_SET(0, &master);
FD_SET(pfd[0],&master);
fdmax=pfd[0];
if(fork()){//Parent
for (;;){
read_fds = master; // copy it
if (select(fdmax+1, &read_fds, NULL, NULL, NULL) == -1) {
perror("select");
exit(4);
}
for(int i = 0; i <= fdmax; i++) {
if (FD_ISSET(i, &read_fds)) {
int n;
char buff[200];
if (i==pfd[0]){
close(pfd[1]);
n=read(pfd[0],buff,sizeof(buff));
buff[n]=0;
cout<<"Read from pipe:"<<buff<<endl;
}else if(i==0){
n=read(0,buff,sizeof(buff));
buff[n]=0;
cout<<"Read from std:"<<buff<<endl;
}
}
}
}
}else{//Child
usleep(50000);
char buff[200]="This is a simple sample.";
close(pfd[0]);
write(pfd[1],buff,sizeof(buff));
close(pfd[1]);
exit(0);
}
}
First of all, the read() call can read less than number of bytes specified in the last argument ant id does not automatically append zero-byte terminator, so your receiving code can easily access uninitialized memory in buff[] and after it (if there is no zero byte). You need to check return value when calling read and use only so many bytes from buffer.
Then, the select call returns when any file descriptor in the readfds set won't block on subsequent read. Which includes end-of-file condition. This likely happens in your case, when the forked process closes its fd. See this SO question too.
Could it be the reason for the problem you encounter? Checking return value when calling read make this clear to you, as read returns zero if-and-only-if the fd reached end of file.
Last detail – it does not make much sense to close pfd[1] only after pfd[0] is returned in the readfds. You should close it immediately after the fork, so it remains open in the child process only if you have no use for it in the parent process.
Related
I have two separate programs in C++, one that writes to two named pipes in unpredictable intervals and one that should wait to read new content from the pipes whenever available. For simplicity, here my writer only writes two times to the pipes (1st: "One" "Tree" , 2nd: "Two" "Fogs").
My writer program is:
#include <unistd.h>
#include <iostream>
int main()
{
int fd1, fd2;
const char* myfifo1 = "/tmp/myfifo1";
const char* myfifo2 = "/tmp/myfifo2";
mkfifo(myfifo1, 0666);
mkfifo(myfifo2, 0666);
fd1 = open(myfifo1, O_WRONLY);
fd2 = open(myfifo2, O_WRONLY);
write(fd1, "One", sizeof("One"));
write(fd2, "Tree", sizeof("Tree"));
sleep(5);
write(fd1, "Two", sizeof("Two"));
write(fd2, "Fogs", sizeof("Fogs"));
close(fd1);
close(fd2);
unlink(myfifo1);
unlink(myfifo2);
}
My reader program is:
#include <fcntl.h>
#include <stdio.h>
#include <sys/stat.h>
#include <unistd.h>
#define MAX_BUF 1024
int main()
{
int fd1, fd2;
const char * myfifo1 = "/tmp/myfifo1";
const char * myfifo2 = "/tmp/myfifo2";
char buf1[MAX_BUF], buf2[MAX_BUF];
fd1 = open(myfifo1, O_RDONLY);
fd2 = open(myfifo2, O_RDONLY);
while (1) {
read(fd1, buf1, MAX_BUF);
read(fd2, buf2, MAX_BUF);
printf("Received: %s\n", buf1);
printf("Received: %s\n", buf2);
}
close(fd1);
close(fd2);
return 0;
}
I do not want the reader to terminate or close the connection between the reads, I need it to remain active and wait until the writer writes something new in the named pipe.
However, by simultaneously running these two programs (in different cores, first the writer and then the reader) I get:
Received: One
Received: Tree
Received: Two
Received: Fogs
Received: Two
Received: Fogs
Received: Two
Received: Fogs
Received: Two
Received: Fogs
...
...
The wanted behavior would be:
Received: One
Received: Tree
Received: Two
Received: Fogs
and then nothing (the reader should wait for another write).
I understand that I should somehow clear the buffer after I read it, but isn't it the default behavior of read? Since it finished reading "Two" (and also "Fogs"), why does it keep reading it and does not wait for new content?
What modifications should I make?
Thank you very much in advance!
Closing a pipe signals end-of-file to the other end.
read will read zero bytes (and return zero) whenever it encounters an end-of-file condition. Any subsequent read after the end-of-file is a no-op.
If you want to continue reading from the same pipe, you need to check the result of read, and if it returns zero, close the pipe and open it again.
Of course you should not unlink the pipe if you need someone else to continue using it.
One other thing to note: read (1) doesn't terminate the buffer with the 0 byte and (2) doesn't necessarily read the exact amount of bytes that was written by a single write. You should detect the end-of-message yourself, rather than relying on read and write to do it for you. It is entirely possible that read will read a half of the string without the terminating null character, then your printf will print garbage.
This question already has answers here:
How to make reading from `std::cin` timeout after a particular amount of time
(5 answers)
Closed 4 years ago.
In C++ on OSX, how do we check std::cin for data without waiting?
The expected data is not coming from a keyboard or human, but from a program that has started my executable and talks to my executable via stdin. This 'parent' program will tell me to start a long process, may interrupt, and I may periodically send data back to it via stdout, therefore the checking of stdin must not block. I'm using XCode.
So far I have attempted to test via the following, neither of which seem to work:
//Return true if there is data waiting version 1
return (std::cin.peek() != EOF);
//Return true if there is data waiting version 2
return (std::cin.rdbuf() && std::cin.rdbuf()->in_avail() >= 0);
To fill out the idea mentioned in a comment, the easiest way since C++11 is simply to start a thread that waits for input and, when it is received, add it to a shared queue container. Doing so requires use of mutex locking & unlocking. This container then can be used to check if empty and, if it has something in it, the data can just be read and popped off of the front of the queue. Something like the following:
#include <thread>
#include <string>
#include <iostream>
std::mutex qmutex;
std::queue<std::string> queue;
void listen() {
std::string sin;
while (true) {
std::cin >> sin; // Thread will wait here for stdin to arrive
qmutex.lock();
queue.push(sin);
qmutex.unlock();
}
}
void mainstuff() {
std::thread t1(listen);
while (true) {
qmutex.lock();
if (!queue.empty()) {
//There's something in the queue, read it/take a copy
queue.pop();
}
qmutex.unlock();
// Respond to input
// Continue work
}
}
I don't believe cin supports non-blocking I/O operations or timeouts.
Here's an example of the lower-level way to accomplish the task, using select() and file descriptors. This program will immediately print to stdout any text it receives from stdin, and also wake up every 5 seconds (even if no text is received from stdin) just to print a fixed message. You can modify the wakeup timeout to whatever you like (including zero for an instantaneous-poll behavior), and/or add more file descriptors for it to react to if you prefer a more event-driven approach.
#include <stdio.h>
#include <sys/select.h>
static void RegisterFileDescriptor(int fd, int & maxFD, fd_set & fdset)
{
FD_SET(fd, &fdset);
if (fd > maxFD) maxFD = fd;
}
int main()
{
const int stdinFileno = fileno(stdin);
// This call is just here so that our parent process will immediately see any
// text lines that we print to stdout. If you're only reading from stdin and never
// writing responses back to stdout, then it's not really necessary.
(void) setlinebuf(stdout);
fd_set readSet;
while(true)
{
int maxFD = -1;
FD_ZERO(&readSet);
RegisterFileDescriptor(stdinFileno, maxFD, readSet);
// If you want to react to incoming data on any other
// file descriptors as well, you could call
// RegisterFileDescriptor on them here
struct timeval timeoutDelay = {5, 0}; // let's force a wakeup every 5 seconds
if (select(maxFD+1, &readSet, NULL, NULL, &timeoutDelay) < 0)
{
perror("select");
break;
}
printf("select() returned!\n");
if (FD_ISSET(stdinFileno, &readSet))
{
char buf[512];
if (fgets(buf, sizeof(buf), stdin)) printf("Read from stdin: [%s]\n", buf);
}
}
return 0;
}
I use a function (L) to execute another program (K) through execlp().
In the K program, the result is written into stream 1:
write(1, (char *)&o, sizeof(int));
Since after execlp(), the remaining part of L won't be executed, how could I get the result written in stream 1?
Don't ask me why I need to do it this way. It's the requirement for a project.
I followed your guys advices, but now the problem is, the way the K program get the arguments are from streams(one standard stream, one other stream), I'm using pipes to write arguments into corresponding streams (it's done by the parent).
After the child exec, in the parent part, I read from stream 0 (the K program writes its result back in stream 1). But what I could get is what the parent wrote into the stream, not what the K program write back.
What's wrong? Do I need to add another pipe?
Thanks!!
The key insight, which Jonathan Leffler mentioned in his comment, is that you need to fork the program which is running L before you call execlp().
After the fork, the parent continues to execute the rest of L, and the child morphs into the program K by calling execlp(), which should never return unless there is an error.
Thus, the assertion that "the remaining part of L won't be executed" is incorrect. It will get executed, in the parent process if you write the function L correctly.
Update: Since the OP made his question more specific, I am appending to this answer.
If you want to retrieve what the child process wrote to stdout (fd 1), you need to create a new pipe before the fork, and copy the writing end of this pipe into the child's stdout.
Here is an example program, slightly modified from the pipe man page.
#include <sys/types.h>
#include <sys/wait.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
int
main(int argc, char *argv[])
{
int pipefd[2];
pid_t cpid;
char buf;
if (pipe(pipefd) == -1) {
perror("pipe");
exit(EXIT_FAILURE);
}
cpid = fork();
if (cpid == -1) {
perror("fork");
exit(EXIT_FAILURE);
}
if (cpid == 0) { /* Child writes into the pipe */
close(pipefd[0]); /* Close unused read end */
// Copy the writing end of the pipe into STDOUT for the child.
dup2(pipefd[1], STDOUT_FILENO);
// Execute your program L here, and its stdout will be captured by the pipe.
const char* message = "Child is speaking to stdout!";
write(STDOUT_FILENO, message, strlen(message));
write(STDOUT_FILENO, "\n", 1);
close(pipefd[1]);
_exit(EXIT_SUCCESS);
} else { /* Parent reads child's stdout from the pipe */
close(pipefd[1]); /* Close unused write end */
// Here the parent process is reading the child's stdout.
while (read(pipefd[0], &buf, 1) > 0)
write(STDOUT_FILENO, &buf, 1);
wait(NULL); /* Wait for child */
exit(EXIT_SUCCESS);
}
}
int mypipe[2];
pipe(mypipe);
int dupstdout=dup2(mypipe[1],1);
cout<<"hello";//not printed on terminal
fflush(stdout);
now how to print again on terminal or redirect mypipe[0] to stdout?
Best to save a copy of standard out and restore it later. If dup2 closes your last copy of stdout, you might not be able to get it back (e.g., no controlling terminal, chroot'd and no access to /dev nor /proc, stdout was an anonymous pipe to begin with, etc.).
int mypipe[2];
pipe(mypipe);
int savstdout=dup(1); // save original stdout
dup2(mypipe[1], 1);
printf("hello"); // not printed on terminal
fflush(stdout);
dup2(savstdout, 1); // restore original stdout
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/stat.h>
#include <fcntl.h>
int main() {
int mypipe[2];
pipe(mypipe);
int dupstdout=dup2(mypipe[1], 1);
printf("hello");//not printed on terminal
fflush(stdout);
close(dupstdout);
int fd = open("/dev/tty", O_WRONLY);
stdout = fdopen(fd, "w");
printf("hello again\n");
}
Anyway, it's better to not close stdout.
If a descriptor passed as a second argument to dup2() is already opened, dup2() closes it ignoring all errors. It's safer to use close() and dup() explicitly.
I am using the following code to redirect stdout to a pipe, then read all the data from the pipe to a buffer. I have 2 problems:
first problem: when i send a string (after redirection) bigger then the pipe's BUFF_SIZE, the program stops responding (deadlock or something).
second problem: when i try to read from a pipe before something was sent to stdout. I get the same response, the program stops responding - _read command stuck's ...
The issue is that i don't know the amount of data that will be sent to the pipe after the redirection.
The first problem, i don't know how to handle and i'll be glad for help. The second problem i solved by a simple workaround, right after the redirection i print space character to stdout. but i guess that this solution is not the correct one ...
#include <fcntl.h>
#include <io.h>
#include <iostream>
#define READ 0
#define WRITE 1
#define BUFF_SIZE 5
using namespace std;
int main()
{
int stdout_pipe[2];
int saved_stdout;
saved_stdout = _dup(_fileno(stdout)); // save stdout
if(_pipe(stdout_pipe,BUFF_SIZE, O_TEXT) != 0 ) // make a pipe
{
exit(1);
}
fflush( stdout );
if(_dup2(stdout_pipe[1], _fileno(stdout)) != 0 ) //redirect stdout to the pipe
{
exit(1);
}
ios::sync_with_stdio();
setvbuf( stdout, NULL, _IONBF, 0 );
//anything sent to stdout goes now to the pipe
//printf(" ");//workaround for the second problem
printf("123456");//first problem
char buffer[BUFF_SIZE] = {0};
int nOutRead = 0;
nOutRead = _read(stdout_pipe[READ], buffer, BUFF_SIZE); //second problem
buffer[nOutRead] = '\0';
// reconnect stdout
if (_dup2(saved_stdout, _fileno(stdout)) != 0 )
{
exit(1);
}
ios::sync_with_stdio();
printf("buffer: %s\n", buffer);
}
Your problem is that you are using blocking I/O calls, while both ends of the pipe are connected to the same process. If you don't know how much data there will be, this is just a deadlock situation waiting to happen.
printf is a blocking call, which means that it will not return until all data has been written to the output device (the pipe in this case), or until a write error is signalled (for example, the other end of the pipe is closed).
_read works similarly. It only returns when it has a full buffer worth of data or it knows that the end of the input has been reached (which can be signalled by closing the write-end of the pipe).
The only ways around this are
to use non-blocking I/O (which is not feasible if you don't have access to the code that calls printf), or
to ensure the reading and writing happens in different processes or threads, or
to use a temporary file for buffering, instead of the buffer of a pipe.
Pipes are unidirectional. Ie. you can either write to a pipe (x)or you can read from it.
To simulate a pipeline, try the following (the below is C, not C++):
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int main(int argc)
{
int pfds[2];
pipe(pfds);
if (!fork()) {
close(1); /* close stdout, check for errors */
dup(pfds[1]); /* make stdout same as pfds[1], dup reuses lowest fd */
close(pfds[0]); /* not needed */
execlp("ls", "ls", NULL); /* or write() in whatever way you want */
} else {
close(0); /* close stdin, check for errors please! */
dup(pfds[0]); /* make stdin same as pfds[0] */
close(pfds[1]); /* not needed on this end */
execlp("wc", "wc", "-l", NULL); /* or read() */
}
return 0;
}
[edit] By the way, your code does not overflow a buffer. Its only relation to buffer overflow is that you're reading into a statically allocated array...if you read() more than sizeof buffer elements, then you'll run into problems.
You must use non-blocking I/O if you don't want read or write to be blocked in this case.