all, I am designing a Key-Value server, and when I wrote the client, and I found a really strange thing,see the simplified code:
while(1)
{
printf("->:");
read(STDIN_FILENO, buf, sizeof(buf));
write(client_sock, buf, sizeof(buf));
int m = read(client_sock, buf, sizeof(buf));
buf[m] = '\0';
printf("%s", buf);
}
when I run the program, it first ask for input, so I input something, but nothing happen!
(the server runs well, and it well echo something, when I use other client)
then I change the code only one line:
printf("\n->:");
then it runs well! why? why "\n" can change the output? I guess it maybe the read() , but I can't explain it
printf(3) is part of the C standard IO library, which performs internal buffering to provide performance improvements.
There are three types of buffering: none, line, and block.
Which buffering is applied is determined in part by whether the descriptor being written to is 2 or not, and if it is connected to a terminal. (See isatty(3).)
If the printing is done to stderr (2) then no buffering is done.
If the printing is done to any other descriptor, then the behavior changes if it is a terminal or not: if output is a terminal, then the output is line buffered. If the output is not a terminal (file, pipe, socket, etc.) then the output is block buffered.
When line buffered, it waits for the \n before printing anything. (Or if you write enough to overflow the internal buffers before sending a \n.)
What I'd recommend instead is the following:
printf("->:");
fflush(stdout);
read(STDIN_FILENO, buf, sizeof(buf));
/* ... */
printf("%s\n", buf);
It's a small change; you won't get a pointless empty line at program start, and the prompt should show up .. promptly.
You can use the setvbuf(3) function to change the buffering for your stream once, at start up, and never need to flush it again, if you would rather.
int err = setvbuf(stdout, NULL, _IONBF, 0);
/* check err */
Standard output is line-buffered by default. If you don't write a complete line, the output will be held in the buffer until you do. You can use fflush to flush the stream or setbuf to change the buffering mode.
Related
I mean to associate a file descriptor with a file pointer and use that for writing.
I put together program io.cc below:
int main() {
ssize_t nbytes;
const int fd = 3;
char c[100] = "Testing\n";
nbytes = write(fd, (void *) c, strlen(c)); // Line #1
FILE * fp = fdopen(fd, "a");
fprintf(fp, "Writing to file descriptor %d\n", fd);
cout << "Testing alternate writing to stdout and to another fd" << endl;
fprintf(fp, "Writing again to file descriptor %d\n", fd);
close(fd); // Line #2
return 0;
}
I can alternately comment lines 1 and/or 2, compile/run
./io 3> io_redirect.txt
and check the contents of io_redirect.txt.
Whenever line 1 is not commented, it produces in io_redirect.txt the expected line Testing\n.
If line 2 is commented, I get the expected lines
Writing to file descriptor 3
Writing again to file descriptor 3
in io_redirect.txt.
But if it is not commented, those lines do not show up in io_redirect.txt.
Why is that?
What is the correct way of using fdopen?
NOTE.
This seems to be the right approach for a (partial) answer to Smart-write to arbitrary file descriptor from C/C++
I say "partial" since I would be able to use C-style fprintf.
I still would like to also use C++-style stream<<.
EDIT:
I was forgetting about fclose(fp).
That "closes" part of the question.
Why is that?
The opened stream ("stream" is an opened FILE*) is block buffered, so nothing gets written to the destination before the file is flushed. Exiting from an application closes all open streams, which flushes the stream.
Because you close the underlying file descriptor before flushing the stream, the behavior of your program is undefined. I would really recommend you to read posix 2.5.1 Interaction of File Descriptors and Standard I/O Streams (which is written in a horrible language, nonetheless), from which:
... if two or more handles are used, and any one of them is a stream, the application shall ensure that their actions are coordinated as described below. If this is not done, the result is undefined.
...
For the first handle, the first applicable condition below applies. ...
...
If it is a stream which is open for writing or appending (but not also open for reading), the application shall either perform an fflush(), or the stream shall be closed.
A "handle" is a file descriptor or a stream. An "active handle" is the last handle that you did something with.
The fp stream is the active handle that is open for appending to file descriptor 3. Because fp is an active handle and is not flushed and you switch the active handle to fd with close(fd), the behavior of your program is undefined.
What is my guess and most probably happens is that your C standard library implementation calls fflush(fp) after main returns, because fd is closed, some internal write(3, ...) call returns an error and nothing is written to the output.
What is the correct way of using fdopen?
The usage you presented is the correct way of using fdopen.
I have a c++ code that is trying to run losetup /dev/loop* and parse what is spitting back at the terminal.
The code is as so
char cmd[32] = {0, };
sprintf(cmd, "losetup /dev/loop%d", i);
FILE *fp;
fp = popen(cmd,"r");
char buf[1024];
read(fileno(fp), buf, 123);
printf("After read: %s\n", buf);
Two different things happen.
When the loop device is set up, the output is like so:
After read: /dev/loop0: [0802] ...Remaining ommitted
which is what I want and expect.
However, when the loop device is not properly set up, the output is like so:
loop: can't get info on device /dev/loop1: No such device or address
After read:
As you can see, "buf" is NULL, and the stdout that should be inside "buf" just printed itself in the terminal. I need to read the output not only when the loop device is set up but also when it isn't. So, can someone explain how to fix this so I can store stdout of both cases?
PS. I've tried "dup2" with "pipe", "fgets", and "getline", and they have all failed similarly.
Pipes only go from the standard input to the standard output. Error messages such as
loop: can't get info on device /dev/loop1: No such device or address
go to the standard error. You can work around this by redirecting the standard error in the command which you pass to popen, e.g.,
sprintf(cmd, "losetup /dev/loop%d 2>&1", i);
because it is a shell command (and shells allow you to do redirection).
I am learning about Kernel buffers and understand that when we write to a file, it doesn't necessarily go immediately to the file even after flushing; it gets put in a kernel buffer which at some point later will be flushed. Apparently we can call fsync() to flush the kernel buffers, so I experimented with this as follows:
/* This works fine, nRet is 0 */
FILE* file = fopen("MyFile", "w");
fprintf(file, "1234");
fflush(file);
int nRet = fsync(fileno(file));
/* This doesn't work, nRet is -1 and errno is 22 (Invalid argument)*/
fprintf(stdout, "Output to standard out");
fflush(stdout);
int p = fileno(stdout);
nRet = fsync(p);
I am fully aware that we don't strictly need to flush the kernel buffers, but this is purely a learning experience.
I can't find anything that says that fsync() doesn't work on standard output; surely it must do anyway as we might want to provide a super fast update to a console, in which case this would be needed if we hadn't opened the output to the console with O_SYNC...
Can someone explain what is happening please? Thanks in advance.
The documentation says:
EINVAL
fd is bound to a special file which does not support
synchronization.
The console device is not a normal file and does not have kernel buffers; any writes are handled immediately by the console driver.
If this were not the case, code like this would not work:
printf("Please enter something: ");
scanf(...);
I am trying to get the output of Tcl Interpreter as described in answer of this question Tcl C API: redirect stdout of embedded Tcl interp to a file without affecting the whole program. Instead of writing the data to file I need to get it using pipe. I changed Tcl_OpenFileChannel to Tcl_MakeFileChannel and passed write-end of pipe to it. Then I called Tcl_Eval with some puts. No data came at read-end of the pipe.
#include <sys/wait.h>
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <fcntl.h>
#include <tcl.h>
#include <iostream>
int main() {
int pfd[2];
if (pipe(pfd) == -1) { perror("pipe"); exit(EXIT_FAILURE); }
/*
int saved_flags = fcntl(pfd[0], F_GETFL);
fcntl(pfd[0], F_SETFL, saved_flags | O_NONBLOCK);
*/
Tcl_Interp *interp = Tcl_CreateInterp();
Tcl_Channel chan;
int rc;
int fd;
/* Get the channel bound to stdout.
* Initialize the standard channels as a byproduct
* if this wasn't already done. */
chan = Tcl_GetChannel(interp, "stdout", NULL);
if (chan == NULL) {
return TCL_ERROR;
}
/* Duplicate the descriptor used for stdout. */
fd = dup(1);
if (fd == -1) {
perror("Failed to duplicate stdout");
return TCL_ERROR;
}
/* Close stdout channel.
* As a byproduct, this closes the FD 1, we've just cloned. */
rc = Tcl_UnregisterChannel(interp, chan);
if (rc != TCL_OK)
return rc;
/* Duplicate our saved stdout descriptor back.
* dup() semantics are such that if it doesn't fail,
* we get FD 1 back. */
rc = dup(fd);
if (rc == -1) {
perror("Failed to reopen stdout");
return TCL_ERROR;
}
/* Get rid of the cloned FD. */
rc = close(fd);
if (rc == -1) {
perror("Failed to close the cloned FD");
return TCL_ERROR;
}
chan = Tcl_MakeFileChannel((void*)pfd[1], TCL_WRITABLE | TCL_READABLE);
if (chan == NULL)
return TCL_ERROR;
/* Since stdout channel does not exist in the interp,
* this call will make our file channel the new stdout. */
Tcl_RegisterChannel(interp, chan);
rc = Tcl_Eval(interp, "puts test");
if (rc != TCL_OK) {
fputs("Failed to eval", stderr);
return 2;
}
char buf;
while (read(pfd[0], &buf, 1) > 0) {
std::cout << buf;
}
}
I've no time at the moment to tinker with the code (might do that later) but I think this approach is flawed as I see two problems with it:
If stdout is connected to something which is not an interactive console (a call to isatty(2) is usually employed by the runtime to check for that), full buffering could be (and I think will be) engaged, so unless your call to puts in the embedded interpreter outputs so many bytes as to fill up or overflow the Tcl's channel buffer (8KiB, ISTR) and then the downstream system's buffer (see the next point), which, I think, won't be less than 4KiB (the size of a single memory page on a typical HW platform), nothing will come up at the read side.
You could test this by changing your Tcl script to flush stdout, like this:
puts one
flush stdout
puts two
You should then be able to read the four bytes output by the first puts from the pipe's read end.
A pipe is two FDs connected via a buffer (of a defined but system-dependent size). As soon as the write side (your Tcl interp) fills up that buffer, the write call which will hit the "buffer full" condition will block the writing process unless something reads from the read end to free up space in the buffer. Since the reader is the same process, such a condition has a perfect chance to deadlock since as soon as the Tcl interp is stuck trying to write to stdout, the whole process is stuck.
Now the question is: could this be made working?
The first problem might be partially fixed by turning off buffering for that channel on the Tcl side. This (supposedly) won't affect buffering provided for the pipe by the system.
The second problem is harder, and I can only think of two possibilities to fix it:
Create a pipe then fork(2) a child process ensuring its standard output stream is connected to the pipe's write end. Then embed the Tcl interpreter in that process and do nothing to the stdout stream in it as it will be implicitly connected to the child process standard output stream attached, in turn, to the pipe. You then read in your parent process from the pipe until the write side is closed.
This approach is more robust than using threads (see the next point) but it has one potential downside: if you need to somehow affect the embedded Tcl interpreter in some ways which are not known up front before the program is run (say, in response to the user's actions), you will have to set up some sort of IPC between the parent and the child processes.
Use threading and embed the Tcl interp into a separate thread: then ensure that reads from the pipe happen in another (let's call it "controlling") thread.
This approach might superficially look simpler than forking a process but then you get all the hassles related to proper synchronization common for threading. For instance, a Tcl interpreter must not be accessed directly from threads other than the one in which the interp was created. This implies not only concurrent access (which is kind of obvious by itself) but any access at all, including synchronized, because of possible TLS issues. (I'm not exactly sure this holds true, but I have a feeling this is a big can of worms.)
So, having said all that, I wonder why you seem to systematically reject suggestions to implement a custom "channel driver" for your interp and just use it to provide the implementation for the stdout channel in your interp? This would create a super-simple single-thread fully-synchronized implementation. What's wrong with this approach, really?
Also observe that if you decided to use a pipe in hope it will serve as a sort of "anonymous file", then this is wrong: a pipe assumes both sides work in parallel. And in your code you first make the Tcl interp write everything it has to write and then try to read this. This is asking for trouble, as I've described, but if this was invented just to not mess with a file, then you're just doing it wrong, and on a POSIX system the course of actions could be:
Use mkstemp() to create and open a temporary file.
Immediately delete it using the name mkstemp() returned in place of the template you passed it.
Since the file still has an open FD for it (returned by mkstemp()), it will disappear from the file system but will not be unlinked, and might be written to and read from.
Make this FD an interp's stdout. Let the interp write everything it has to.
After the interp is finished, seek() the FD back to the beginning of the file and read from it.
Close the FD when done — the space it occupied on the underlying filesystem will be reclamied.
I'm having issues in mixing freopen of stdin together with ncurses library.
What I do is to reopen the stdin to a file, parse it through flex+bison, then I should revert it back and start ncurses.
Code for the swap is the following:
void switchStdin(const char *newStream) {
fflush(stdin);
fgetpos(stdin, &pos);
fd = dup(fileno(stdin));
freopen(newStream, "r", stdin);
}
void revertStdin() {
fflush(stdin);
dup2(fd, fileno(stdin));
close(fd);
clearerr(stdin);
fsetpos(stdin, &pos);
}
In my lexer this is what I do:
void parse() {
Utils::switchStdin("./filename.ext");
yyparse();
Utils::revertStdin();
}
and in the main file I just do something like
parse();
initscr();
keypad(stdscr, TRUE);
noecho();
cbreak();
...
Now, if I don't comment the revertStdin it just halts when launching, if instead I comment it, parsing does work but the ncurses gui is unresponsive and blinks from time to time.
Since both parts do work when independently used I guess there is some caveat in mixing them in a sequential manner that I'm missing. Does anyone have a clue?
There's more than one problem, but the important one is that revertStdin does not succeed in opening a stream:
switchStdin unnecessarily calls the non-portable fflush(stdin).
it saves the file descriptor for stdin in fd, and the file-position in pos.
the file position isn't needed (since terminals don't do that)
revertStdin unnecessarily calls the non-portable fflush(stdin).
it attempts to overwrite the file descriptor of stdin using the saved fd. That won't work.
on some systems you cannot assign a new value to stdin; it happens that the GNU C library allows this. If you're using Linux, you could solve the problem using fdopen, e.g.,
stdin = fdopen(fd, "r");
if not (more portably) you would use newterm rather than initscr. Unlike initscr, newterm lets you specify the streams for input and output. In that case, you would still use fdopen, but not attempt to assign to stdin. Use your own variable and pass that to newterm.