I am reading C++ Primer, 5th Edition. When talking about flushing streams it says:
An output stream might be tied to another stream. In this case, the buffer of
the tied stream is flushed whenever the tied stream is read or written. By
default, cin and cerr are both tied to cout. Hence, reading cin or writing to
cerr flushes the buffer in cout.
I tried to understand this point with an example:
int main() {
std::ofstream out("data.txt");
std::ifstream in("data.txt");
//in.tie(&out);// commenting this will print nothing
out << "Hello there!";
std::string str;
while (in >> str)
std::cout << str << " ";
out.close();
in.close();
}
As you can see above the input and output file stream objects in and out use the same file so the output one opens the file "data.txt" and writes some line into it BUT doesn't save it and the input stream tries to read the content of this un-saved file.
If I tie the input stream object in to out then I get the content correctly. Does this mean in forces out to be flushed?
If I comment out the line in.tie(&out) then I don't get the content using in?
Please explain to me how it works here. Thank you in advance.
The thing about buffered I/O is that you can never really be sure when the content of the buffer will be flushed to the external sequence. The size of the internal buffer of iostream objects is implementation-defined, and they can decide to flush the buffer (or not flush the buffer) at any point before the stream is closed or destructed. Only when the stream is closed (via an explicit call to close() or when its lifetime ends) is a stream required to write to the external sequence. This means that an output operation may write to external file right away, or it could wait until there is more content streamed into the buffer.
You can flush a stream explicitly by calling the flush() method, but when you are flipping between input and output like with std::cin and std::cout, it becomes tedious to do std::cout.flush() before every input operation (if there was a write before it). This is where "tying" streams comes in. When you tie an output stream, the stream that called tie() (in your case in, and in general std::cin) calls flush() on the tied stream before every I/O operation that it performs. This allows the two streams to remain synchronized.
So to answer your questions directly:
If I tie the input stream object in with out then I get the content correctly. Does this mean in forces out to be flushed?
Yes.
If I comment out the line in.tie(&out) then I don't get the content using in?
It depends on how the buffer is managed by the stream. The stream that performs output could perform a write immediately, or it could wait for its buffer to be filled further before it decides to write, or it could wait until its lifetime ends. This decision varies between iostream implementations.
Related
According to the definition of endl, it is used to insert a new-line character and flush the stream. And I remember that if a new line is inserted, then the buffer will be flushed automatically. If so, why do endl still need to flushes the stream after inserting a new line.
if a new line is inserted, then the buffer will be flushed automatically
Not necessarily, and not for all streams. It's common for std::cout (and the standard output stream it sends data to) to be line-buffered, but not universal. Some implementations, for example, only line-buffer stdout if output is going to a terminal, and fully-buffer it otherwise (if output is redirected to a pipe, for example).
And that's just for standard output. Other streams like file I/O are almost always fully-buffered and the buffer will only be flushed when it either fills up or its flushed explicitly.
In case of buffered stream it said in a book that it wait until the buffer is full to write back to the monitor. For example:
cout << "hi";
What do they mean by "the buffer is full".
cerr << "hi";
It is said in my book that everything sent to cerr is written to the standard error device immediately, what does it mean?
char *ch;
cin>> ch; // I typed "hello world";
In this example ch will be assigned to "hello" and "world" will be ignored does it mean that it still in the buffer and it will affect the results of future statements?
Your book doesn't seem very helpful.
1) The output streams send their bytes to a std::streambuf, which may
contain a buffer; the std::filebuf (derived from streambuf) used by
and std::ofstream will generally be buffered. That means that when
you output a character, it isn't necessarily output immediately; it will
be written to a buffer, and output to the OS only when the buffer is
full, or you explicitly request it in some way, generally by calling
flush() on the stream (directly, or indirectly, by using std::endl).
This can vary, however; output to std::cout is synchronized with
stdout, and most implementations will more or less follow the rules of
stdout for std::cout, changing the buffering strategy if the output
is going to an interactive device.
At any rate, if you're unsure, and you want to be sure that the output
really does leave your program, just add a call to flush.
2) Your book is wrong here.
One of the buffering strategies is unitbuf; this is a flag in the
std::ostream which you can set or reset (std::ios_base::set() and
std::ios_base::unset()—std::ios_base is a base class of
std::ostream, so you can call these functions on an std::ostream
object). When unitbuf is set, std::ostream adds a call to flush()
to the end of every output function, so when you write:
std::cerr << "hello, world";
the stream will be flushed after all of the characters in the string
are output, provided unitbuf is set. On start-up, unitbuf is set
for std::cerr; by default, it is not set on any other file. But you
are free to set or unset it as you wish. I would recommend against
unsetting it on std::cerr, but if std::cout is outputting to an
interactive device, it makes a lot of sense to set it there.
Note that all that is in question here is the buffer in the streambuf.
Typically, the OS also buffers. All flushing the buffer does is
transfer the characters to the OS; this fact means that you cannot use
ofstream directly when transactional integrity is required.
3) When you input to a string or a character buffer using >>, the
std::istream first skips leading white space, and then inputs up to
but not including the next white space. In the formal terms of the
standard, it "extracts" the characters from the stream, so that they
will not be seen again (unless you seek, if the stream supports it).
The next input will pickup where ever the previous left off. Whether
the following characters are in a buffer, or still on disk, is really
irrelevant.
Note that the buffering of input is somewhat complex, in that it occurs
at several different levels, and at the OS level, it takes different
forms depending on the device. Typically, the OS will buffer a file by
sectors, often reading several sectors in advance. The OS will always
return as many characters as were demanded, unless it encounters end of
file. Most OSs will buffer a keyboard by line: not returning from a
read request until a complete line has been entered, and never returning
characters beyond the end of the current line in a read request.
In the same manner as std::ostream uses a streambuf for output,
std::istream uses one to get each individual character. In the case
of std::cin, it will normally be a filebuf; when the istream
requests a character, the filebuf will return one from its buffer if
it has one; if it doesn't, it will attempt to refill the buffer,
requesting e.g. 512 (or whatever its buffer size is) characters from the
OS. Which will respond according to its buffering policy for the
device, as described above.
At any rate, if std::cin is connected to the keyboard, and you've
typed "hello world", all of the characters you've typed will be read
by the stream eventually. (But if you're using >>, there'll be a lot
of whitespace that you won't see.)
streams in C++ are buffer to increase efficiency, that is file and console IO is very slow in comparison to memory operations.
To combat this C++ streams have a buffer (a bank of memory) that contains everything to write to the file or output, when it is full then it flushed to the file. The inverse is true for input, it fetches more when it the buffer is depleted.
This is very import for streams because the following
std::cout << 1 << "hello" << ' ' << "world\n";
Would be 4 writes to a file which is inefficient.
However in the case of std::cout, cin, and cerr then these type actually have buffering turned off by default to ensure that it can be used in conjunction with std::printf and std::puts etc...
To re-enable it (which I recommend doing):
std::ios_base::sync_with_stdio(false);
But don't use C style console output whilst it is set false or bad things may happen.
You can check out the differences yourself with a small app.
#include <iostream>
int main() {
std::cout<<"Start";
//std::cout<<"Start"<<std::endl; // line buffered, endl will flush.
double j = 2;
for(int i = 0; i < 100000000; i++){
j = i / (i+1);
}
std::cout<<j;
return 0;
}
Try out the difference of the two "Start" statments, then change to cerr. The difference you notice is due to buffering.
The for-statement takes about 2 seconds on my rig, you might need to tweak the i < condition on yours.
1) what do they mean by "the buffer is full".
With buffered output there's a region of memory, called a buffer, where the stuff you write out is stored before it is actually written to the output. When you say cout << "hi" the string is probably only copied into that buffer and not written out. cout usually waits until that memory has been filled up before it actually starts writing things out.
The reason for this is because usually the process of starting to actually write data is slow, and so if you do that for every character you get terrible performance. A buffer is used so that the program only has to do that infrequently and you get much better performance.
2) it said in my book that everything sent to cerr is written to the standard error device immediatly, does this mean it send 'h' and then 'i'...?
It just means that no buffer is used. cerr might still send 'h' and 'i' at the same time since it already has both of them.
3)in this example ch will be assigned to "hello" and "world" will be ignored does it mean that it still in the buffer and it will affect the results of future statements ?
This doesn't really have anything to do with buffering. the operator >> for char* is defined to read until it sees whitespace, so it stops at the space between "hello" and "world". But yes, the next time you read you will get "world".
(although if that code isn't just a paraphrase of the actuall code then it has undefined behavior because you're reading the text into an undefined memory location. Instead you should do:
std::string s;
cin >> s;
)
Each call to write to the terminal is slow, so to avoid doing slow things often the data is stored in memory until either a certain amount of data has been entered or the buffer is flushed manually with fflush or std::endl. The result of this is sometimes that text might not be written to the terminal at the moment you expect it to.
Since the timing of error messages is more critical than normal output, the performance hit is ignored and the data is not buffered. However, since a string is passed in a single piece of data, it is written in one call (inside a loop somewhere).
It world would still be in the buffer, but it's quite easy to prove this yourself by trying it in a 3 line program. However, your example will fail since you are attempting to write into unallocated memory. You should be taking input into a std::string instead.
Given this:
auto f = std::ifstream{file};
if (f) {
std::stringstream stream;
stream << f.rdbuf();
return stream.str();
}
return std::string{};
I don't see why it works.
I don't know what type f is, because it says auto, but apparently you can check that for non-zero-ness.
But when the file is large, like 2 gig, the delay in running happens in
this line:
stream << f.rdbuf();
The documentation says rdbuf() gets you the pointer to the ifstream's internal buffer. So in order for it to read the entire file, the buffer would have to size the file, and load it all in in one shot. But by the time the stream << happens, rdbuf() has to already be set, or it won't be able to return a point.
I would expect the constructor to do that in this case, but it's obviously lazy loaded, because reading in the entire file on construction would be bad for every other use case, and the delay is in the stream << command.
Any thoughts? All other stack overflow references to reading in a file to a string always loop in some way.
If there's some buffer involved, which obviously there is, how big can it get? What if it is 1 byte, it would surely be slow.
Adorable c++ is very opaque, bad for programmers who have to know what's going on under the covers.
It's a function of how operator<< is defined on ostreams when the argument is a streambuf. As long as the streambuf isn't a null pointer, it extracts characters from the input sequence controlled by the streambuf and inserts them into *this until one of the following conditions are met (see operator<< overload note #9):
end-of-file occurs on the input sequence;
inserting in the output sequence fails (in which case the character to be inserted is not extracted);
an exception occurs (in which case the exception is caught).
Basically, the ostream (which stringstream inherits from) knows how to exercise a streambuf to pull all the data from the file it's associated with. It's an idiomatic, but as you note, not intuitive, way to slurp a whole file. The streambuf isn't actually buffering all the data here (as you note, reading the whole file into the buffer would be bad in the general case), it's just that it has the necessary connections to adjust the buffered window as an ostream asks for more (and more, and more) data.
if (f) works because ifstream has an overload for operator bool that is implicitly invoked when the "truthiness" of the ifstream is tested that tells you if the file is in a failure state.
To answer your first question first:
f is of the type that's assigned to it, an std::ifstream, but that's a rather silly way to write it. One would usually write std::ifstream f {...}. A stream has an overloaded operator bool () which gives you !fail().
As for the second question: What .rdbuf() returns is a streambuf object. This object doesn't contain the whole file contents when it is returned. Instead, it provides an interface to access data, and this interface is used by the stringstream stream.
auto f = std::ifstream{file};
Type of f is std::ifstream.
stream << f.rdbuf();
std::ifstream maintains a buffer which you can get by f.rdbuf() and it does not load entire file in 1 shot. The loading happens when the above commands is called, stringstream will extract data from that buffer, and ifstream will perform loading as the buffer runs out of data.
You can manually set the buffer size by using setbuf.
I know there are many buffer questions on here but I can't seem find a clear answer on this.
std::cout << "write to screen" << std::endl;
I know this code will write to the screen and flush the buffer because of the "endl", but if I wrote this:
std::cout << "write to screen";
Wouldn't the buffer be flushed regardless since the text has been outputted to the screen?
Wouldn't the buffer be flushed regardless since the text has been outputted to the screen?
Assuming that you have seen the text outputted to the screen, then yes, the buffer has been flushed.
I believe the confusion is regarding this line:
std::cout << "write to screen";
The absence of std::endl doesn't mean "don't flush the buffer". It simply means "I'm not saying when to flush the buffer".
There are multiple ways to ensure your std::ostream is flushed:
Manually with std::endl, std::flush, or a direct call to ostream::flush().
Depending on a later used input stream being bound to your ostream: std::basic_ios::tie().
Depending on the tie to C streams: std::ios_base::sync_with_stdio
This means that anything which would flush the corresponding C stream will also flush the C++ stream, like a call to fflush(), or the (maybe automatically) selected buffering strategy.
Like line-buffering.
From C11 draft:
7.21.3 Files
3 When a stream is unbuffered, characters are intended to appear from the source or at the
destination as soon as possible. Otherwise characters may be accumulated and
transmitted to or from the host environment as a block. When a stream is fully buffered,
characters are intended to be transmitted to or from the host environment as a block when
a buffer is filled. When a stream is line buffered, characters are intended to be
transmitted to or from the host environment as a block when a new-line character is
encountered. Furthermore, characters are intended to be transmitted as a block to the host
environment when a buffer is filled, when input is requested on an unbuffered stream, or
when input is requested on a line buffered stream that requires the transmission of
characters from the host environment. Support for these characteristics is
implementation-defined, and may be affected via the setbuf and setvbuf functions.
7 At program startup, three text streams are predefined and need not be opened explicitly
— standard input (for reading conventional input), standard output (for writing
conventional output), and standard error (for writing diagnostic output). As initially
opened, the standard error stream is not fully buffered; the standard input and standard
output streams are fully buffered if and only if the stream can be determined not to refer
to an interactive device.
Waiting on the internal buffer overflowing.
Now, as a general guideline: Don't manually flush your streams, doing so can significantly degrade performance. Unless, of course, it's neccessary for correctness.
std::cout << "Hello" << std::endl;
will write to the screen before executing the next line of code, while
std::cout << "Hello\n";
will print the same, but some time before your program exits normally or you use std::cin (or another instream you tie to std::cout by hand). That means that if your program is terminated abruptly or hangs in an infinite loop, you might not see the output at all.
"Wouldn't the buffer be flushed regardless since the text has been outputted to the screen?"
No! std::endl implies flushing. The underlying buffer won't be flushing (written on the screen),
until hitting a certain watermark (buffer size).
If you want to have it flushed, call cout.flush() explicitely:
std::cout << "write to screen";
std::cout.flush();
The real key to the solution is, what the underlying std::basic_streambuf interface actually implements.
There could be various implementations:
Calling flush() every time the certain watermark of the underlying buffer is hit
Calling flush() every time (not very efficient)
Calling flush() as soon a '\n' had been printed
Calling flush() as guaranteed with std::endl
The internal buffer management shouldn't be your business of concern, unless you're trying to provide your own std::basic_streambuf implementation.
I am new to C++ learning from C++ Primer book. In the first chapter, writer talks about the buffer of the iostream, cout, and endl. I could not understand well. I have two example codes here.
#include<iostream>
int v1;
int main()
{
std::cout<<"First Cout!";
std::cin>>v1;
std::cout<<"Second Cout!"<<std::endl;
std::cout<<"Third Cout!\n";
return 0;
}
I want to know the state of the cout buffer after execution of each line.
The stream contains an in-memory buffer where data is written before being flushed to the final destination (in this case, the output console), since flushing can be an expensive operation.
The buffer might be flushed automatically in some circumstances: when the stream is closed, or if there's a lot of buffered data, or if the stream is configured to flush after each line, as std::cerr is.
Sometimes you need to flush manually, for example to make sure the user sees something you've written to std::cout. That can be done in two ways:
calling the flush() member function on the stream;
streaming the std::flush manipulator into the stream.
The std::endl manipulator does two things:
inserts a new-line character (or character sequence), equivalent to << '\n'; then
flushes the stream, equivalent to << std::flush, (which is in turn calls the stream's flush() member function).
This is useful for writing short messages to the console; but should be used carefully, as it can have a considerable performance impact when producing large amounts of output.
A further complication is that one stream can be tied to another, so that one is flushed before another is accessed. In this case, cout is tied to cin, which is why you see the first output before reading from cin even though there is no explicit flush.
Finally, all the standard streams, including cout, are flushed automatically when the program ends (specifically, when the global instance of std::ios_base::Init is destroyed; but that's a detail you shouldn't need to know about.)
After first line, the output is in the buffer, so you will not see it in the terminal.
After the second line, the endl causes the buffer to be flushed so you will now see line 1 and 2 in the terminal output.
After the third line, output is in the buffer and you will not see it in the terminal until the program exits.
Edit:
When you place a cin between line 1 and 2, it causes cout to be flushed. See: std::cin
cin is tied to the standard output stream cout (see ios::tie), which indicates that cout's buffer is flushed (see ostream::flush) before each i/o operation performed on cin.
After 1st line, cout contains just the given string with no newline char.
After 2nd line, cout additionally contains a line ending and the text appears at the console window.
After 3rd line, cout contains another string including line ending which not necessarily appears at the console window.
After exiting main, the text of the 3rd line is appears.
std::endl just adds EOL (end of line) symbol to output. Usually its \n, but it can vary from one OS to another.