I'm reading from a named pipe on Linux using std::ifstream. If the writing end of the file is closed, I can not continue reading from the pipe through the stream. For some reason I have to clear(), close() and open() the stream again to continue reading. Is this expected? How can I avoid the close() open() on a pipe when writers close() and open() the pipe at will?
Background: I believe the close() open() I have to do is causing the writer to sometimes receive SIGPIPE which I would like to avoid.
More details - I am using this code to read a stream
// read single line
stream_("/tmp/delme", std::ios::in|std::ios::binary);
std::getline(stream_, output_filename_);
std::cout << "got filename: " << output_filename_ << std::endl;
#if 0
// this fixes the problem
stream_.clear();
stream_.close();
stream_.open("/tmp/delme", std::ios::in|std::ios::binary);
// now the read blocks until data is available
#endif
// read more binary data
const int hsize = 4096+4;
std::array<char, hsize> b;
stream_.read(&b[0], hsize);
std::string tmp(std::begin(b), std::begin(b)+hsize);
std::cout << "got header: " << tmp << std::endl;
/tmp/delme is my pipe. I do echo "foo" > /tmp/delme and I get the foo in output_filename_ but the stream does not block there, (it should, there is no more data), it proceeds to read garbage. If I enable the code within the ifdef it works. Why?
Thanks,
Sebastian
Since you use std::getLine(), maybe you need to use an extra "\n" to signal the end of a line:
echo -e "foo\n" > /tmp/delme
instead of just
echo "foo" > /tmp/delme
This should at least get rid of the garbage reading.
Related
I've written a simple C++ program for tutorial purposes.
My goal is to loop it infinitely.
#include <iostream>
#include <string>
int main()
{
std::cout << "text";
for(;;) {
std::string string_object{};
std::getline(std::cin, string_object);
std::cout << string_object;
}
return 0;
}
After compilation I run it like this:
./bin 0>&1
What I expected to happen is that the "text" that is output to stdout, and it will now become also stdin for the program and it will loop forever. Why doesn't it happen?
First, you need to output newlines when printing to std::cout, otherwise std::getline() won't have any complete line to read.
Improved version:
#include <iostream>
#include <string>
int main()
{
std::cout << "stars" << std::endl;
for(;;) {
std::string string_object;
std::getline(std::cin, string_object);
std::cout << string_object << std::endl;
}
return 0;
}
Now try this:
./bin >file <file
you don't see any output, because it's going to the file. But if you stop the program and look at the file, behold, it's full of
stars
stars
stars
stars
:-)
Also, the reason that the feedback loop cannot start when you try
./bin 0>&1
is, that you end up with both stdin and stdout connected to /dev/tty
(meaning that you can see the output).
But a TTY device cannot ever close the loop, because it actually consists of two separate channels, one passing the output to the terminal, one passing the terminal input to the process.
If you use a regular file for in- and output, the loop can be closed. Every byte written to the file will be read from it as well, if the stdin of the process is connected to it. That's as long as no other process reads from the file simultaneously, because each byte in a stream can be only read once.
Since you're using gcc, I'm going to assume you have pipe available.
#include <cstring>
#include <iostream>
#include <unistd.h>
int main() {
char buffer[1024];
std::strcpy(buffer, "test");
int fd[2];
::pipe(fd);
::dup2(fd[1], STDOUT_FILENO);
::close(fd[1]);
::dup2(fd[0], STDIN_FILENO);
::close(fd[0]);
::write(STDOUT_FILENO, buffer, 4);
while(true) {
auto const read_bytes = ::read(STDIN_FILENO, buffer, 1024);
::write(STDOUT_FILENO, buffer, read_bytes);
#if 0
std::cerr.write(buffer, read_bytes);
std::cerr << "\n\tGot " << read_bytes << " bytes" << std::endl;
#endif
sleep(2);
}
return 0;
}
The #if 0 section can be enabled to get debugging. I couldn't get it to work with std::cout and std::cin directly, but somebody who knows more about the low-level stream code could probably tweak this.
Debug output:
$ ./io_loop
test
Got 4 bytes
test
Got 4 bytes
test
Got 4 bytes
test
Got 4 bytes
^C
Because the stdout and stdin don't create a loop. They may point to the same tty, but a tty is actually two separate channels, one for input and one for output, and they don't loop back into one another.
You can try creating a loop by running your program with its stdin connected to the read end of a pipe, and with its stdout to its write end. That will work with cat:
mkfifo fifo
{ echo text; strace cat; } <>fifo >fifo
...
read(0, "text\n", 131072) = 5
write(1, "text\n", 5) = 5
read(0, "text\n", 131072) = 5
write(1, "text\n", 5) = 5
...
But not with your program. That's because your program is trying to read lines, but its writes are not terminated by a newline. Fixing that and also printing the read line to stderr (so we don't have to use strace to demonstrate that anything happens in your program), we get:
#include <iostream>
#include <string>
int main()
{
std::cout << "text" << std::endl;
for(;;) {
std::string string_object{};
std::getline(std::cin, string_object);
std::cerr << string_object << std::endl;
std::cout << string_object << std::endl;
}
}
g++ foo.cc -o foo
mkfifo fifo; ./foo <>fifo >fifo
text
text
text
...
Note: the <>fifo way of opening a named pipe (fifo) was used in order to open both its read and its write end at once and so avoid blocking. Instead of reopening the fifo from its path, the stdout could simply be dup'ed from the stdin (prog <>fifo >&0) or the fifo could be first opened as a different file descriptor, and then the stdin and stdout could be opened without blocking, the first in read-only mode and the second in write-only mode (prog 3<>fifo <fifo >fifo 3>&-).
They will all work the same with the example at hand. On Linux, :|prog >/dev/fd/0 (and echo text | strace cat >/dev/fd/0) would also work -- without having to create a named pipe with mkfifo.
I'm trying to call a process with a string to its stdin, with Boost-1.64.0.
The current code is :
bp::opstream inStream ;
bp::ipstream outStream;
bp::ipstream errStream;
bp::child child(
command, // the command line
bp::shell,
bp::std_out > outStream,
bp::std_err > errStream,
bp::std_in < inStream);
// read the outStream/errStream in threads
child.wait();
The problem is that the child executable is waiting for its stdin EOF. Here child.wait() is hanging indefinitely…
I tried to used asio::buffer, std_in.close(),… But no luck.
The only hack I found was to delete() the inStream… And that's not really reliable.
How am I supposed to "notify" the child process and close its stdin with the new boost::process library ?
Thanks !
I tried to used asio::buffer, std_in.close()
This works. Of course it only works if you pass it to a launch function (bp::child constructor, bp::system, etc).
If you need to pass data, and then close it, simply close the associated filedescriptor. I do something like this:
boost::asio::async_write(input, bp::buffer(_stdin_data), [&input](auto ec, auto bytes_written){
if (ec) {
logger.log(LOG_WARNING) << "Standard input rejected: " << ec.message() << " after " << bytes_written << " bytes written";
}
may_fail([&] { input.close(); });
});
Where input is
bp::async_pipe input(ios);
Also, check that the process is not actually stuck sending the output! If you fail to consume the output it would be buffering and waiting if the buffer is full.
Closing the pipe by calling inStream.close(); when you're done writing to it. You can also close it while launching with bp::std_in.close().
The asio solution of course also works and avoids the danger of deadlocks.
At the command line I used mkfifo to create a pipe and confirmed that the file existed. ls -al gives:
prw-r--r-- 1 user group 0 Mar 30 11:52 my_pipe
I then use this code to attempt to read text line by line from that pipe:
ifstream pipe("/path/to/my_pipe");
string message;
while(getline(pipe, message) && message != "EXIT")
{
cout << boolalpha << pipe.good() << endl << pipe.eof() << endl << pipe.bad() << endl;
}
cout << boolalpha << pipe.good() << endl << pipe.eof() << endl << pipe.bad() << endl;
Back to the command line I do echo "banana" > my_pipe and then the program exists with the following output:
true
false
false
false
true
false
This would indicate that the stream was in a good state after reading banana but that in the second call to getline I hit EOF and exited.
Why did I hit EOF and what can I do to keep the stream reading and in a good state until I read my special EXIT message?
EDIT:
Conclusions:
Use fstream instead of ifstream.
There apparently is a bug in libc++ as of this writing when it comes to fstream and pipes.
EOF occurs on a pipe when it's closed by all the processes that have it open for writing. In your example, the only process that opens it for writing is the echo command; when it exits, , it closes the pipe, which results in EOF in the reading process.
If you want the pipe to stay open between each client writing to it, the way to do it is to open the pipe in read-write mode. That way, the reading process will also be a writing process, and as long as it has the pipe open it will never be closed by all the writers.
fstream pipe("/path/to/my_pipe");
Alternatively, you can keep the pipe open on the sending side by executing multiple commands within a single redirected processes:
( echo banana ; echo EXIT ) > my_pipe
I am trying to use an ifstream to open a named pipe that will eventually have data written to it.
std::cout << "Opening " << name << std::endl;
std::ifstream manual_shutdown_file(name.c_str());
std::cout << "Opened " << name << std::endl;
When I run the program, it blocks in the ifstream constructor. I see "Opening name" printed to the console, but the opened statement does not appear.
I know that I am connecting to the pipe, because if I execute
$ echo foo > name
from a shell, then the constructor returns and the Opened statement is printed. Is there no way to open a pipe until it has data in it, even if I do not want to immediately try reading it?
Calling open on the read end of a pipe will block until the write end is opened.
You can use the O_NONBLOCK flag to open the file descriptor for the pipe, but there is no standard way to then use the fd with std::ifstream, see here.
Guessing at your requirement, I'd say a small class that opens the fd and presents a polling signal interface would suit, something like:
namespace blah
{
class signal_t
{
private:
int fd;
// note: define sensible copy/move semantics
signal_t(const signal_t&) = delete;
signal_t& operator=(const signal_t&) = delete;
public:
signal_t(const char* named_pipe); // open fd, set O_NONBLOCK
void notify() const; // write 1 byte to fd as signal
bool poll() const; // attempt to read from fd, return true if signalled.
~signal_t(); // close fd
};
}
Since opening an input pipe via ifstream blocks until there is someone writing to it, you could always just let the ifstream block. Then to unblock it from another thread create your own ofstream to that same pipe then immediately close the ofstream. This will unblock the ifstream and have it marked with eof. This is much easier and less error prone than messing with the platform specific controls of the file handles.
You actually can open a std::ifstream on a named pipe without blocking for a writer, but you must set the flags as though you were also going to write to the stream.
Try std::ifstream pipe_stream(filename, std::ifstream::in | std::ifstream::out), or stream.open(filename, std::ifstream::in | std::ifstream::out).
The essence of my problem is that I can't write to a file in a loop with sleep(). If I have the following code:
ofstream file
file.open("file.name");
for(;;) {
file << "HELLO\n";
}
This code works perfectly and prints HELLO repeatedly into "file.name". However, I want to do something like this (I'm recording data from a real-time application):
for(;;) {
file << "HELLO\n";
sleep(1);
}
This doesn't seem to print anything into my file. Any ideas?
You need to flush the output. The output stream is buffering your data into memory but not writing it out to disk. You should either use std::endl (which prints a newline and flushes) instead of the string literal '\n', or explicitly flush the stream with std::flush:
for(;;) {
file << "HELLO" << endl;
}
// or
for(;;) {
file << "HELLO\n" << flush;
}
The magic word you are looking for is "flush".
c++ std::ofstream flush() but not close()
before the sleep, flush the file so that it isn't pending in a buffer waiting for there to be enough of a change to bother writing out.
It's probably just a buffering issue. Because you are now writing much slower, the output buffer wont fill up so fast so you may not 'see' the written data. Try adding a flush() before the sleep.
file.flush()