I call select with a single named pipe fd in read_fds. This named pipe has no writers and has only been opened in non-blocking, read only mode. I would expect that the select returns with the named pipe fd marked as ready to read, and that trying to read from the pipe returns 0:
From the manpage on read:
When attempting to read from an empty pipe or FIFO:
If no process has the pipe open for writing, read() shall return 0 to > indicate end-of-file.
However, select just blocks indefinitely. Why is this the case?
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
#include <string.h>
#include <stdexcept>
#include <thread>
#include <iostream>
int main()
{
char buf[4096];
// Create a named pipe
auto err = mkfifo("/tmp/whatever",0666);
if(err) {
throw std::runtime_error(
std::string("Failed to create fifo ")+
strerror(errno));
}
std::thread reader_thread(
[&](){
auto fd = open("/tmp/whatever",O_RDONLY|O_NONBLOCK);
if(fd < 0) {
throw std::runtime_error("Failed to open fifo");
}
fd_set fds;
while(1) {
FD_ZERO(&fds);
FD_SET(fd,&fds);
std::cerr << "calling select" << std::endl;
auto retval = select(fd+1,&fds,nullptr,nullptr,nullptr);
if(retval < 0) {
std::runtime_error("Failed to call select");
}
if(FD_ISSET(fd,&fds)) {
auto read_bytes = read(fd,buf,4096);
std::cerr << "read " << read_bytes << std::endl;
if(read_bytes==0) {
break;
}
}
}
close(fd);
});
reader_thread.join();
return 0;
}
From the POSIX documentation fo select:
A descriptor shall be considered ready for reading when a call to an input function with O_NONBLOCK clear would not block, whether or not the function would transfer data successfully. (The function might return data, an end-of-file indication, or an error other than one indicating that it is blocked, and in each of these cases the descriptor shall be considered ready for reading.
...
If none of the selected descriptors are ready for the requested operation, the pselect() or select() function shall block until at least one of the requested operations becomes ready, until the timeout occurs, or until interrupted by a signal.
From the pipe(7) manpage (which is the underlying object of a FIFO):
If all file descriptors referring to the write end of a pipe have been closed, then an attempt to read(2) from the pipe will see end-of-file (read(2) will return 0).
Mind the usage of present perfect tense! This implies the FIFO has to be opened on both sides fiorst, the closed on the writer side (for your application) to generate an EOF condition.
So, unless the fifo is eventually closed by the writer, why should select return? The setting for the (fifo-)file itself is irrelevant for good reason: It would introduce a race-condition between opening on both sides when using the most efficient method to read more than one byte at a time. That's the normal way for e.g. a command pipe: start the reader process and lateron the writer (which is typically a completely unrelated program when using a named pipe).
If you want select to return early, use the timeout argument. But normally, one uses a seperate thread which can be terminated by a signal (see the select man-page for more information).
As a sidenote: One nice thing about Linux/POSIX is that it does not really matter whether you use a FIFO or a file or your microphone driver.
Related
How do you do nonblocking console IO on Linux/OS X in C?
I want to add an example:
#include <unistd.h>
#include <fcntl.h>
#include <stdio.h>
int main(int argc, char const *argv[])
{
char buf[20];
fcntl(0, F_SETFL, fcntl(0, F_GETFL) | O_NONBLOCK);
sleep(4);
int numRead = read(0, buf, 4);
if (numRead > 0) {
printf("You said: %s", buf);
}
}
When you run this program you have 4 seconds to provide input to standard in. If no input found, it will not block and will simply return.
2 sample executions:
Korays-MacBook-Pro:~ koraytugay$ ./a.out
fda
You said: fda
Korays-MacBook-Pro:~ koraytugay$ ./a.out
Korays-MacBook-Pro:~ koraytugay$
Like Pete Kirkham, I found cc.byexamples.com, and it worked for me. Go there for a good explanation of the problem, as well as the ncurses version.
My code needed to take an initial command from standard input or a file, then watch for a cancel command while the initial command was processed. My code is C++, but you should be able to use scanf() and the rest where I use the C++ input function getline().
The meat is a function that checks if there is any input available:
#include <unistd.h>
#include <stdio.h>
#include <sys/select.h>
// cc.byexamples.com calls this int kbhit(), to mirror the Windows console
// function of the same name. Otherwise, the code is the same.
bool inputAvailable()
{
struct timeval tv;
fd_set fds;
tv.tv_sec = 0;
tv.tv_usec = 0;
FD_ZERO(&fds);
FD_SET(STDIN_FILENO, &fds);
select(STDIN_FILENO+1, &fds, NULL, NULL, &tv);
return (FD_ISSET(0, &fds));
}
This has to be called before any stdin input function When I used std::cin before using this function, it never returned true again. For example, main() has a loop that looks like this:
int main(int argc, char* argv[])
{
std::string initialCommand;
if (argc > 1) {
// Code to get the initial command from a file
} else {
while (!inputAvailable()) {
std::cout << "Waiting for input (Ctrl-C to cancel)..." << std::endl;
sleep(1);
}
std::getline(std::cin, initialCommand);
}
// Start a thread class instance 'jobThread' to run the command
// Start a thread class instance 'inputThread' to look for further commands
return 0;
}
In the input thread, new commands were added to a queue, which was periodically processed by the jobThread. The inputThread looked a little like this:
THREAD_RETURN inputThread()
{
while( !cancelled() ) {
if (inputAvailable()) {
std::string nextCommand;
getline(std::cin, nextCommand);
commandQueue.lock();
commandQueue.add(nextCommand);
commandQueue.unlock();
} else {
sleep(1);
}
}
return 0;
}
This function probably could have been in main(), but I'm working with an existing codebase, not against it.
For my system, there was no input available until a newline was sent, which was just what I wanted. If you want to read every character when typed, you need to turn off "canonical mode" on stdin. cc.byexamples.com has some suggestions which I haven't tried, but the rest worked, so it should work.
You don't, really. The TTY (console) is a pretty limited device, and you pretty much don't do non-blocking I/O. What you do when you see something that looks like non-blocking I/O, say in a curses/ncurses application, is called raw I/O. In raw I/O, there's no interpretation of the characters, no erase processing etc. Instead, you need to write your own code that checks for data while doing other things.
In modern C programs, you can simplify this another way, by putting the console I/O into a thread or lightweight process. Then the I/O can go on in the usual blocking fashion, but the data can be inserted into a queue to be processed on another thread.
Update
Here's a curses tutorial that covers it more.
I bookmarked "Non-blocking user input in loop without ncurses" earlier this month when I thought I might need non-blocking, non-buffered console input, but I didn't, so can't vouch for whether it works or not. For my use, I didn't care that it didn't get input until the user hit enter, so just used aio to read stdin.
Here's a related question using C++ -- Cross-platform (linux/Win32) nonblocking C++ IO on stdin/stdout/stderr
Another alternative to using ncurses or threads is to use GNU Readline, specifically the part of it that allows you to register callback functions. The pattern is then:
Use select() on STDIN (among any other descriptors)
When select() tells you that STDIN is ready to read from, call readline's rl_callback_read_char()
If the user has entered a complete line, rl_callback_read_char will call your callback. Otherwise it will return immediately and your other code can continue.
Let`s see how it done in one of Linux utilites. For example, perf/builtin-top.c sources (simplified):
static void *display_thread(void *arg)
{
struct pollfd stdin_poll = { .fd = 0, .events = POLLIN };
struct termios save;
set_term_quiet_input(&save);
while (!done) {
switch (poll(&stdin_poll, 1, delay_msecs)) {
...
}
}
tcsetattr(0, TCSAFLUSH, &save);
}
So, if you want to check if any data available, you can use poll() or select() like this:
#include <sys/poll.h>
...
struct pollfd pfd = { .fd = 0, .events = POLLIN };
while (...) {
if (poll(&pfd, 1, 0)>0) {
// data available, read it
}
...
}
In this case you will receive events not on each key, but on whole line, after [RETURN] key is pressed. It's because terminal operates in canonical mode (input stream is buffered, and buffer flushes when [RETURN] pressed):
In canonical input processing mode, terminal input is processed in
lines terminated by newline ('\n'), EOF, or EOL characters. No input
can be read until an entire line has been typed by the user, and the
read function (see Input and Output Primitives) returns at most a
single line of input, no matter how many bytes are requested.
If you want to read characters immediately, you can use noncanonical mode. Use tcsetattr() to switch:
#include <termios.h>
void set_term_quiet_input()
{
struct termios tc;
tcgetattr(0, &tc);
tc.c_lflag &= ~(ICANON | ECHO);
tc.c_cc[VMIN] = 0;
tc.c_cc[VTIME] = 0;
tcsetattr(0, TCSANOW, &tc);
}
Simple programm (link to playground):
#include <stdio.h>
#include <unistd.h>
#include <sys/poll.h>
#include <termios.h>
void set_term_quiet_input()
{
struct termios tc;
tcgetattr(0, &tc);
tc.c_lflag &= ~(ICANON | ECHO);
tc.c_cc[VMIN] = 0;
tc.c_cc[VTIME] = 0;
tcsetattr(0, TCSANOW, &tc);
}
int main() {
struct pollfd pfd = { .fd = 0, .events = POLLIN };
set_term_quiet_input();
while (1) {
if (poll(&pfd, 1, 0)>0) {
int c = getchar();
printf("Key pressed: %c \n", c);
if (c=='q') break;
}
usleep(1000); // Some work
}
}
Not entirely sure what you mean by 'console IO' -- are you reading from STDIN, or is this a console application that reads from some other source?
If you're reading from STDIN, you'll need to skip fread() and use read() and write(), with poll() or select() to keep the calls from blocking. You may be able to disable input buffering, which should cause fread to return an EOF, with setbuf(), but I've never tried it.
I am trying to create a inter process communication bus using pipes.
I can send messages to the client, and i can receive messages at the client.
I also tried to create another pipe which goes the other way, child-> parent.
However I need this communication to be non blocking and I want to send messages in both directions with some undefined time gap in between.
It appears i can not send more then one message, I take it it is due to the fclose(); closing the stream, however if i remove the close, the program stops working...
How can I make this code nonblocking two way parent -> child communication through pipes?
Later I want to connect a child-binary using exec(), and connecting this binary's stdin, stdout to my parent, so that I can pass messages to the stdin of the child and retrieve answers through the "childParent" descriptor.
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#define READ_SIDE 0
#define WRITE_SIDE 1
/* Read characters from the pipe and echo them to stdout. */
void
read_from_pipe (int file)
{
FILE *stream;
int c;
stream = fdopen (file, "r");
while ((c = fgetc (stream)) != EOF)
putchar (c);
fclose (stream);
}
/* Write some random text to the pipe. */
void
write_to_pipe (int file, char * message)
{
FILE *stream;
stream = fdopen (file, "w");
fprintf (stream, message);
//fprintf (stream, "goodbye, world!\n");
//Dont forget to close
fclose (stream);
}
void end(){
//kill process and close pipes
}
int
main (void)
{
pid_t pid;
int parentToChild[2];
int childToParent[2];
/* Create the pipes. */
if (pipe (parentToChild))
{
fprintf (stderr, "Pipe failed.\n");
return EXIT_FAILURE;
}
if (pipe (childToParent))
{
fprintf (stderr, "Pipe failed.\n");
return EXIT_FAILURE;
}
/* Create the child process. */
pid = fork ();
if (pid == (pid_t) 0)
{
/* This is the child process.
Close other end first. */
close(childToParent[READ_SIDE]);
close (parentToChild[WRITE_SIDE]);
read_from_pipe (parentToChild[READ_SIDE]);
write_to_pipe(childToParent[WRITE_SIDE], "Child sent message to parent");
return EXIT_SUCCESS;
}
else if (pid < (pid_t) 0)
{
/* The fork failed. */
fprintf (stderr, "Fork failed.\n");
return EXIT_FAILURE;
}
else
{
/* This is the parent process.
Close other end first. */
close(childToParent[WRITE_SIDE]);
close (parentToChild[READ_SIDE]);
write_to_pipe (parentToChild[WRITE_SIDE],"Parent sent message to child");
read_from_pipe(childToParent[READ_SIDE]);
//Send more messages...
return EXIT_SUCCESS;
}
}
UPDATE
It appears that sometimes the communication only happens one way, depending on which process gets to read/write first I guess.
There are several issues to be considered. First, pipes have a finite
maximum length (4096 was common in the distant past); any write to a
pipe which has more data in it than that will block, as will any read
from an empty pipe. These are fundamental to the way pipes work, so
when you speak of non-blocking, you have to take them into consideration
(or use asynchronous IO).
Second, if you really need to control things at this level, you might
want to consider using Posix level IO, rather than streams. If you do
use streams (either iostream or FILE*), then you have to take into
account the buffering they use. In particular, when you want to be sure
that the data is output to the pipe, you need to flush (fflush on
the FILE*). With regards to sending more than one message, and the
code not working without the fclose, replacing the fclose with
fflush should be all you need to fix this.
And while I'm at it: your use of fprintf for output is extremely
dangerous. If you're getting a message from an external source, and
(possibly) don't know what it contains, you should use fputs to output
it, or possibly something like fprintf( fd, "%s\n", message ) (if you
want to append a new line). As you've done it, if the message contains
a '%', strange things will happen. More generally, you should prefer
the much safer iostream, but for simple things like this, it doesn't
matter much. And for what you're actually doing, Posix level IO is
probably just as appropriate, or more so. With Posix level IO, you
don't get any formatting or buffering: but since you're not using the
formatting capabilities, and the buffering it part of your problem...
I have two functions writer() and reader(). I am writing message into the pipe from writer() function and reading it from the reader() function. The problem I am facing is that message is being written in the pipe, but it's not getting read. Maybe there is problem in opening pipe for reading. The code is:
#include<iostream>
#include<stdio.h>
#include<unistd.h>
#include<sys/types.h>
#include<sys/stat.h>
#include<fcntl.h>
using namespace std;
//edit
int fifo = mkfifo("/tmp/mypipe", S_IWUSR | S_IRUSR | S_IRGRP | S_IROTH);
void writer()
{
char message[] = "this is a message";
int fd;
fd = open("pipe" , O_WRONLY);
write(fd , message , sizeof(message));
cout<<"message wrote: "<<message<<"\n"; // gives output
} // message wrote: this is a message
void reader()
{
char buffer[100];
int fd;
fd = open("pipe" , O_RDONLY);
read(fd , buffer , 100);
cout<<"message is: "<<buffer<<"\n"; // gives output
} // message is:
int main()
{
writer();
reader();
return 0;
}
I've debugged it, and I think the problem is, fifo is not being created correctly. I don't know how to resolve this. Need more help.
Thank you for any help.
My guess is that you have not created the pipe in a correct way. Take a look at the mkfifo man page. For the umask value take a look at the umask man page.
something like mkfifo("/tmp/pipe", 0666), before you open /tmp/pipe in the reader/writer.
Also take a look at the fifo man page:
The kernel maintains exactly one pipe object for each FIFO special
file that is opened by at least one process. The FIFO must be opened
on both ends (reading and writing) before data can be passed.
Normally, opening the FIFO blocks until the other end is opened also.
So your problem now is, that the open(..., O_WRONLY) blocks until the reader opens the file.
To try it out, let just the reader run and then use echo "test" > /tmp/pipe.
Update:
Or use threads, i just tried it out.
int main() {
mkfifo(fifo_name.c_str(), 0666);
std::thread w(writer);
std::thread r(reader);
w.join();
r.join();
unlink(fifo_name.c_str());
return 0;
}
you also have to #include <thread>, add this compiler flag: -std=c++0x and add the following library to the linker: -lpthread.
Be sure to check returns from function calls, since they can tell you what the problem is.
Include errno.h:
#include <errno.h>
#include <string.h>
And check errno when you get an error return from your write or read open attempt:
fd = open("pipe" , O_WRONLY);
if (fd < 0)
{
cout << "writer open failed: " << errno << "(" << strerror(errno) << ")\n";
/* exit */
}
As another answer states, you're not using mkfifo(), so you're making a typical file (which would also work but may fail if you don't provide O_CREAT and a mode argument).
It is about how named pipe work in posix. You can only write in it, if there is already somebody, who is reading from it. If there is none, your write() operation will be blocked until somebody don't read.
The simplest solution were, if
You used nonblocking I/O
You implemented the reader and the writer in different processes (threads) and called the reader before the writer.
I am trying to guess how much data is in pipe, and I don't want to use while(read) because it is blocking until EOF.
Is there any way to do that?
I real I want something like this:
i = pipe1.size();
pipe1.read(i);
I say again, I don't want to use while (read) because it is blocking until EOF.
The amount of data coming from a pipe could be infinite, just a like a stream, there's no concept of size in a pipe. if you don't want it to block if there's nothing to read you should set the O_NONBLOCK flag when calling pipe2():
pipe2(pipefd, O_NONBLOCK);
This way when you call read() if there's no data it would fail and set errno to EWOULDBLOCK
if (read(fd, ...) < 0) {
if (errno == EWOULDBLOCK) {
//no data
}
//other errors
}
From the man page:
O_NONBLOCK: Set the O_NONBLOCK file status flag on the two new open
file descriptions. Using this flag saves extra calls to fcntl(2) to
achieve the same result.
You could also use select() on a blocking pipe to timeout.
This could help you, however it is unix specific:
#include <iostream>
#include <sys/types.h>
#include <sys/ioctl.h>
#include <sys/socket.h>
#include <errno.h>
int pipe_fd; /* your pipe's file descriptor */
......
int nbytes = 0;
//see how much data is waiting in buffer
if ( ioctl(pipe_fd, FIONREAD, &nbytes) < 0 )
{
std::cout << "error occured: " << errno;
}
else
{
std::cout << nbytes << " bytes waiting in buffer";
}
I'm trying to use a socketpair to have a parent process provide input to a child process that execs a different program (e.g., grep) and then read the resulting output. The program hangs in the while loop that reads the output from the program that the child execs.. The child dupes stdin and stdout on to its end of the socketpair and the parent and the child both close their unused end of the pair.
Interestingly, if the child execs a program that I wrote (OK, I ripped it off from Stevens Advanced Programming in the Unix Environment) everything works as expected. However, if the child execs grep (or some other standard program) the parent invariably hangs in trying to read the output. I can't tell if the input is not reaching grep or if the grep cannot determine the end of the input or if the output is somehow being lost.
Here's the code:
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/wait.h>
#include <unistd.h>
#include <signal.h>
#include <cstdio>
#include <cerrno>
#include <iostream>
using namespace std;
void
sigpipe_handler(int sig, siginfo_t *siginfo, void * context) {
cout << "caught SIGPIPE\n";
pid_t pid;
if (errno == EPIPE) {
throw "SIGPIPE caught";
}
}
int main(int argc, char** argv) {
struct sigaction sa;
memset(&sa, '\0', sizeof(struct sigaction));
sa.sa_sigaction = sigpipe_handler;
sa.sa_flags = SA_SIGINFO | SA_RESTART;
sigaction(SIGPIPE, &sa, NULL);
int sp[2];
socketpair(PF_UNIX, SOCK_STREAM, AF_UNIX, sp);
pid_t childPid = fork();
if (childPid == 0) {
close(sp[0]);
if (dup2(sp[1], STDIN_FILENO) != STDIN_FILENO) throw "dup2 error to stdin";
if (dup2(sp[1], STDOUT_FILENO) != STDOUT_FILENO) throw "dup2 error to stdout";
execl("/bin/grep", "grep", "-n", "namespace", (char*)NULL);
} else {
close(sp[1]);
char line[80];
int n;
try {
while (fgets(line, 80, stdin) != NULL) {
n = strlen(line);
if (write(sp[0], line, n) != n) {
throw "write error to pipe";
}
if ((n=read(sp[0], line, 80)) < 0) { // hangs here
throw "read error from pipe";
}
if (n ==0) {
throw "child closed pipe";
break;
}
line[n] = 0;
if (fputs(line, stdout) == EOF) {
throw "puts error";
}
if (ferror(stdin)) {
throw "fgets error on stdin";
}
exit(0);
}
} catch (const char* e) {
cout << e << endl;
}
int status;
waitpid(childPid, &status, 0);
}
}
Your code hangs as grep's output may be less than 80 bytes and you are issuing a blocking read on sp[0]. The proper way of doing this is by marking both sockets as non-blocking and selecting() over both of them.
You also forgot to close(sp[0]) before you wait(), which will leave your child process waiting for input.
You cannot achieve deadlock-free bidirectional communication with a subprocess using UNIX pipes or socketpairs, because you don't have control over buffering in the subprocess.
It just so happens that cat can be trusted to read one line and immediately print it, regardless of whether its standard output is a tty, a pipe or a socket. This is not the case with grep (and actually most programs using stdio), which will buffer output in-process (in the stdio buffers) and defer the write() call until either the buffer is full or the stdio stream is closed (typically because grep is about to exit after having seen EOF on input).
You can trick line-oriented programs (including grep) into not buffering by using a pseudo-tty instead; take a look at libexpect(3). But in the general case, you would have to re-run a different subprocess for each message, which allows to use EOF to signal the end of each message and cause whatever buffers in the command (or pipeline of commands) to be flushed.
See more info about this problem in the perlipc man page (it's for bi-directional pipes in Perl but the buffering considerations apply regardless of the language used for the main program).
It works fine with cat, so the problem is with grep. May be grep output behave differently when connected to something else than a terminal. Or it is not detecting the pattern for some reason.