I have some C++ code that uses cout statements for debug purposes and for some reason I can't get all the data to print unless I do a std::cout.flush(); at the end.
I don't quite understand why this flush operation is needed.
Anyone have any insight?
To add to the other answers: your debugging statements should instead go to cerr, because:
it writes to the standard error, which means that, running the application, you can easily separate the "normal" program output from the errors/debug information via redirection;
most importantly, cerr by default is unbuffered, which means that, after each output operation, it will automatically flush itself, and in general this is desirable for errors and debug output.
(source: C++ standard, §27.3.1 ¶4-5, §27.4.2.1.2 table 83)
Are you using std::endl to terminate your lines. This should be the
usual practice, until performance issues require otherwise, but for some
reason, I see a lot of code which uses '\n' instead.
Otherwise, you can always do:
std::cout.setf( std::ios_base::unitbuf );
as one of the first things in main. This will cause a flush at the
end of every <<, which is more than you need, but for diagnostic
output to the console, is probably quite acceptable.
Is the data that isn't automatically getting flushed lacking a \n at the end? By default, standard out doesn't get delivered until a carriage return is seen.
"When you send output to a stream, it does not necessarily get printed immediately. Rather, it may wait in a buffer until some unspecified event, e.g. buffer full enough, reading from input, or exit from program. The details may vary."
http://www.cs.hmc.edu/~geoff/classes/hmc.cs070.200109/notes/io.html
It is the right behavior. You probably use std::endl that add \n and flush the buffer. http://www.cplusplus.com/reference/ostream/endl/
You need to flush the stream, if you want to see the output.
The answer of std::endl is only valid if you want a return.
Not sure how you would do this if you wanted to flush a command prompt out.
In C++ you can use endl formatter with cout operator rather then flush.
Related
Can someone please explain (preferably using plain english) how std::flush works?
What is it?
When would you flush a stream?
Why is it important?
Thank you.
Since it wasn't answered what std::flush happens to be, here is some detail on what it actually is. std::flush is a manipulator, i.e., a function with a specific signature. To start off simple, you can think of std::flush of having the signature
std::ostream& std::flush(std::ostream&);
The reality is a bit more complex, though (if you are interested, it is explained below as well).
The stream class overload output operators taking operators of this form, i.e., there is a member function taking a manipulator as argument. The output operator calls the manipulator with the object itself:
std::ostream& std::ostream::operator<< (std::ostream& (*manip)(std::ostream&)) {
(*manip)(*this);
return *this;
}
That is, when you "output" std::flush with to an std::ostream, it just calls the corresponding function, i.e., the following two statements are equivalent:
std::cout << std::flush;
std::flush(std::cout);
Now, std::flush() itself is fairly simple: All it does is to call std::ostream::flush(), i.e., you can envision its implementation to look something like this:
std::ostream& std::flush(std::ostream& out) {
out.flush();
return out;
}
The std::ostream::flush() function technically calls std::streambuf::pubsync() on the stream buffer (if any) which is associated with the stream: The stream buffer is responsible for buffering characters and sending characters to the external destination when the used buffer would overflow or when the internal representation should be synced with the external destination, i.e., when the data is to be flushed. On a sequential stream syncing with the external destination just means that any buffered characters are immediately sent. That is, using std::flush causes the stream buffer to flush its output buffer. For example, when data is written to a console flushing causes the characters to appear at this point on the console.
This may raise the question: Why aren't characters immediately written? The simple answer is that writing characters is generally fairly slow. However, the amount of time it takes to write a reasonable amount of characters is essentially identical to writing just one where. The amount of characters depends on many characteristics of the operating system, file systems, etc. but often up to something like 4k characters are written in about the same time as just one character. Thus, buffering characters up before sending them using a buffer depending on the details of the external destination can be a huge performance improvement.
The above should answer two of your three questions. The remaining question is: When would you flush a stream? The answer is: When the characters should be written to the external destination! This may be at the end of writing a file (closing a file implicitly flushes the buffer, though) or immediately before asking for user input (note that std::cout is automatically flushed when reading from std::cin as std::cout is std::istream::tie()'d to std::cin). Although there may be a few occasions where you explicitly want to flush a stream, I find them to be fairly rare.
Finally, I promised to give a full picture of what std::flush actually is: The streams are class templates capable of dealing with different character types (in practice they work with char and wchar_t; making them work with another characters is quite involved although doable if you are really determined). To be able to use std::flush with all instantiations of streams, it happens to be a function template with a signature like this:
template <typename cT, typename Traits>
std::basic_ostream<cT, Traits>& std::flush(std::basic_ostream<cT, Traits>&);
When using std::flush immediately with an instantiation of std::basic_ostream it doesn't really matter: The compiler deduces the template arguments automatically. However, in cases where this function isn't mentioned together with something facilitating the template argument deduction, the compiler will fail to deduce the template arguments.
By default, std::cout is buffered, and the actual output is only printed once the buffer is full or some other flushing situation occurs (e.g. a newline in the stream). Sometimes you want to make sure that the printing happens immediately, and you need to flush manually.
For example, suppose you want to report a progress report by printing a single dot:
for (;;)
{
perform_expensive_operation();
std::cout << '.';
std::flush(std::cout);
}
Without the flushing, you wouldn't see the output for a very long time.
Note that std::endl inserts a newline into a stream as well as causing it to flush. Since flushing is mildly expensive, std::endl shouldn't be used excessively if the flushing isn't expressly desired.
Here's a short program that you can write to observe what flush is doing
#include <iostream>
#include <unistd.h>
using namespace std;
int main() {
cout << "Line 1..." << flush;
usleep(500000);
cout << "\nLine 2" << endl;
cout << "Line 3" << endl ;
return 0;
}
Run this program: you'll notice that it prints line 1, pauses, then prints line 2 and 3. Now remove the flush call and run the program again- you'll notice that the program pauses and then prints all 3 lines at the same time. The first line is buffered before the program pauses, but because the buffer is never flushed, line 1 is not outputted until the endl call from line 2.
A stream is connected to something. In the case of standard output, it could be the console/screen or it could be redirected to a pipe or a file. There is a lot of code between your program and, for example, the hard disk where the file is stored. For example, the operating system is doing stuff with any file or the disk drive itself might be buffering data to be able to write it in fixed size blocks or just to be more efficient.
When you flush the stream, it tells the language libraries, the os and the hardware that you want to any characters that you have output so far to be forced all the way to storage. Theoretically, after a 'flush', you could kick the cord out of the wall and those characters would still be safely stored.
I should mention that the people writing the os drivers or the people designing the disk drive might are free to use 'flush' as a suggestion and they might not really write the characters out. Even when the output is closed, they might wait a while to save them. (Remember that the os does all sorts of things at once and it might be more efficient to wait a second or two to handle your bytes.)
So a flush is a sort of checkpoint.
One more example: If the output is going to the console display, a flush will make sure the characters actually get all the way out to where the user can see them. This is an important thing to do when you are expecting keyboard input. If you think you have written a question to the console and its still stuck in some internal buffer somewhere, the user doesn't know what to type in answer. So, this is a case where the flush is important.
Flush happens in the following cases:
std::cerr
std::cin
program termination
in many implementations, standard output is line-buffered, meaning "\n" flushes anyway
So it seems in most regular programs, std::endl is basically unnecessary, but it's used almost everywhere.
std::cerr
But what if I do not want to write to stderr, but rather stdout?
std::cin
But what if I just want to give the user some status update and do not want any input?
program termination
As above, what if I want to tell the user something before the program is done?
in many implementations, standard output is line-buffered, meaning "\n" flushes anyway
But what about the platforms that don't? Regardless of whether or not such a platform currently exists, I want my code to work as intended as defined by the standard, not as by "meh, will probably be fine on all implementations".
If all of the above apply (the fourth point actually always applies), you need to flush "by hand". If the flush gets handled by one of the first three, you don't.
So it seems in most regular programs, std::endl is basically unnecessary, but it's used almost everywhere.
No it's not redundant, since implementation of flushing along '\n' isn't mandated by the standard. That's just an implementation specific behavior, while the behavior of std::endl is always clearly defined.
I know there are many buffer questions on here but I can't seem find a clear answer on this.
std::cout << "write to screen" << std::endl;
I know this code will write to the screen and flush the buffer because of the "endl", but if I wrote this:
std::cout << "write to screen";
Wouldn't the buffer be flushed regardless since the text has been outputted to the screen?
Wouldn't the buffer be flushed regardless since the text has been outputted to the screen?
Assuming that you have seen the text outputted to the screen, then yes, the buffer has been flushed.
I believe the confusion is regarding this line:
std::cout << "write to screen";
The absence of std::endl doesn't mean "don't flush the buffer". It simply means "I'm not saying when to flush the buffer".
There are multiple ways to ensure your std::ostream is flushed:
Manually with std::endl, std::flush, or a direct call to ostream::flush().
Depending on a later used input stream being bound to your ostream: std::basic_ios::tie().
Depending on the tie to C streams: std::ios_base::sync_with_stdio
This means that anything which would flush the corresponding C stream will also flush the C++ stream, like a call to fflush(), or the (maybe automatically) selected buffering strategy.
Like line-buffering.
From C11 draft:
7.21.3 Files
3 When a stream is unbuffered, characters are intended to appear from the source or at the
destination as soon as possible. Otherwise characters may be accumulated and
transmitted to or from the host environment as a block. When a stream is fully buffered,
characters are intended to be transmitted to or from the host environment as a block when
a buffer is filled. When a stream is line buffered, characters are intended to be
transmitted to or from the host environment as a block when a new-line character is
encountered. Furthermore, characters are intended to be transmitted as a block to the host
environment when a buffer is filled, when input is requested on an unbuffered stream, or
when input is requested on a line buffered stream that requires the transmission of
characters from the host environment. Support for these characteristics is
implementation-defined, and may be affected via the setbuf and setvbuf functions.
7 At program startup, three text streams are predefined and need not be opened explicitly
— standard input (for reading conventional input), standard output (for writing
conventional output), and standard error (for writing diagnostic output). As initially
opened, the standard error stream is not fully buffered; the standard input and standard
output streams are fully buffered if and only if the stream can be determined not to refer
to an interactive device.
Waiting on the internal buffer overflowing.
Now, as a general guideline: Don't manually flush your streams, doing so can significantly degrade performance. Unless, of course, it's neccessary for correctness.
std::cout << "Hello" << std::endl;
will write to the screen before executing the next line of code, while
std::cout << "Hello\n";
will print the same, but some time before your program exits normally or you use std::cin (or another instream you tie to std::cout by hand). That means that if your program is terminated abruptly or hangs in an infinite loop, you might not see the output at all.
"Wouldn't the buffer be flushed regardless since the text has been outputted to the screen?"
No! std::endl implies flushing. The underlying buffer won't be flushing (written on the screen),
until hitting a certain watermark (buffer size).
If you want to have it flushed, call cout.flush() explicitely:
std::cout << "write to screen";
std::cout.flush();
The real key to the solution is, what the underlying std::basic_streambuf interface actually implements.
There could be various implementations:
Calling flush() every time the certain watermark of the underlying buffer is hit
Calling flush() every time (not very efficient)
Calling flush() as soon a '\n' had been printed
Calling flush() as guaranteed with std::endl
The internal buffer management shouldn't be your business of concern, unless you're trying to provide your own std::basic_streambuf implementation.
As the title suggests, I'm looking for a way to have cout 'flush' itself only when I explicitly tell it to. I know this can be achieved by 'buffering' the contents we want to write into a string or a stringstream, but I'm looking for the most efficient way of doing this.
Also it would be nice to also be able to eat up trailing newlines in this buffer. I've seen solutions that writes \b to cout; however I'm pretty sure this is unreliable due to flushing (correct me if I'm wrong).
Without using stringstream, your best bet it so prevent the two most common ways that lead to an automatic flush: when you are outputting something and when you explicitly call << std::endl. You can deactivate the first by setting the unitbuf flag of std::ios_base::fmtflags. This, though, won't likely stop the automatic flush when a new line is encountered. If your OS decides it needs to flush at every \n character, there's nothing you can do (on the standard C++ side).
Therefore I'd recommend manual buffering.
It's probably a long-shot that anyone can answer this without seeing all the source code and libraries, etc., but I'll try.
I have a program X written in C++ using boost-1.41. If X outputs with std::cout, then running X from another program using fp=popen("X", "r") allows X's output to be seen via fgets(buff, 1024, fp).
Now if I change X to use printf() instead of std::cout, the output of X is no longer seen. However, running X from bash produces the output as expected.
What could possibly explain this difference?! I suspect boost is involved here, but I don't know much about boost.
Note: I am happy sticking to std::cout and my problem is solved. But I'm trying to understand what the problem was with printf().
The reason is that you probably used std::endl with std::cout. That, in addition to writing a newline character, also flushes the output buffer.
To do the same with printf you can just add fflush(stdout); after the call.