Flush happens in the following cases:
std::cerr
std::cin
program termination
in many implementations, standard output is line-buffered, meaning "\n" flushes anyway
So it seems in most regular programs, std::endl is basically unnecessary, but it's used almost everywhere.
std::cerr
But what if I do not want to write to stderr, but rather stdout?
std::cin
But what if I just want to give the user some status update and do not want any input?
program termination
As above, what if I want to tell the user something before the program is done?
in many implementations, standard output is line-buffered, meaning "\n" flushes anyway
But what about the platforms that don't? Regardless of whether or not such a platform currently exists, I want my code to work as intended as defined by the standard, not as by "meh, will probably be fine on all implementations".
If all of the above apply (the fourth point actually always applies), you need to flush "by hand". If the flush gets handled by one of the first three, you don't.
So it seems in most regular programs, std::endl is basically unnecessary, but it's used almost everywhere.
No it's not redundant, since implementation of flushing along '\n' isn't mandated by the standard. That's just an implementation specific behavior, while the behavior of std::endl is always clearly defined.
Related
Can someone please explain (preferably using plain english) how std::flush works?
What is it?
When would you flush a stream?
Why is it important?
Thank you.
Since it wasn't answered what std::flush happens to be, here is some detail on what it actually is. std::flush is a manipulator, i.e., a function with a specific signature. To start off simple, you can think of std::flush of having the signature
std::ostream& std::flush(std::ostream&);
The reality is a bit more complex, though (if you are interested, it is explained below as well).
The stream class overload output operators taking operators of this form, i.e., there is a member function taking a manipulator as argument. The output operator calls the manipulator with the object itself:
std::ostream& std::ostream::operator<< (std::ostream& (*manip)(std::ostream&)) {
(*manip)(*this);
return *this;
}
That is, when you "output" std::flush with to an std::ostream, it just calls the corresponding function, i.e., the following two statements are equivalent:
std::cout << std::flush;
std::flush(std::cout);
Now, std::flush() itself is fairly simple: All it does is to call std::ostream::flush(), i.e., you can envision its implementation to look something like this:
std::ostream& std::flush(std::ostream& out) {
out.flush();
return out;
}
The std::ostream::flush() function technically calls std::streambuf::pubsync() on the stream buffer (if any) which is associated with the stream: The stream buffer is responsible for buffering characters and sending characters to the external destination when the used buffer would overflow or when the internal representation should be synced with the external destination, i.e., when the data is to be flushed. On a sequential stream syncing with the external destination just means that any buffered characters are immediately sent. That is, using std::flush causes the stream buffer to flush its output buffer. For example, when data is written to a console flushing causes the characters to appear at this point on the console.
This may raise the question: Why aren't characters immediately written? The simple answer is that writing characters is generally fairly slow. However, the amount of time it takes to write a reasonable amount of characters is essentially identical to writing just one where. The amount of characters depends on many characteristics of the operating system, file systems, etc. but often up to something like 4k characters are written in about the same time as just one character. Thus, buffering characters up before sending them using a buffer depending on the details of the external destination can be a huge performance improvement.
The above should answer two of your three questions. The remaining question is: When would you flush a stream? The answer is: When the characters should be written to the external destination! This may be at the end of writing a file (closing a file implicitly flushes the buffer, though) or immediately before asking for user input (note that std::cout is automatically flushed when reading from std::cin as std::cout is std::istream::tie()'d to std::cin). Although there may be a few occasions where you explicitly want to flush a stream, I find them to be fairly rare.
Finally, I promised to give a full picture of what std::flush actually is: The streams are class templates capable of dealing with different character types (in practice they work with char and wchar_t; making them work with another characters is quite involved although doable if you are really determined). To be able to use std::flush with all instantiations of streams, it happens to be a function template with a signature like this:
template <typename cT, typename Traits>
std::basic_ostream<cT, Traits>& std::flush(std::basic_ostream<cT, Traits>&);
When using std::flush immediately with an instantiation of std::basic_ostream it doesn't really matter: The compiler deduces the template arguments automatically. However, in cases where this function isn't mentioned together with something facilitating the template argument deduction, the compiler will fail to deduce the template arguments.
By default, std::cout is buffered, and the actual output is only printed once the buffer is full or some other flushing situation occurs (e.g. a newline in the stream). Sometimes you want to make sure that the printing happens immediately, and you need to flush manually.
For example, suppose you want to report a progress report by printing a single dot:
for (;;)
{
perform_expensive_operation();
std::cout << '.';
std::flush(std::cout);
}
Without the flushing, you wouldn't see the output for a very long time.
Note that std::endl inserts a newline into a stream as well as causing it to flush. Since flushing is mildly expensive, std::endl shouldn't be used excessively if the flushing isn't expressly desired.
Here's a short program that you can write to observe what flush is doing
#include <iostream>
#include <unistd.h>
using namespace std;
int main() {
cout << "Line 1..." << flush;
usleep(500000);
cout << "\nLine 2" << endl;
cout << "Line 3" << endl ;
return 0;
}
Run this program: you'll notice that it prints line 1, pauses, then prints line 2 and 3. Now remove the flush call and run the program again- you'll notice that the program pauses and then prints all 3 lines at the same time. The first line is buffered before the program pauses, but because the buffer is never flushed, line 1 is not outputted until the endl call from line 2.
A stream is connected to something. In the case of standard output, it could be the console/screen or it could be redirected to a pipe or a file. There is a lot of code between your program and, for example, the hard disk where the file is stored. For example, the operating system is doing stuff with any file or the disk drive itself might be buffering data to be able to write it in fixed size blocks or just to be more efficient.
When you flush the stream, it tells the language libraries, the os and the hardware that you want to any characters that you have output so far to be forced all the way to storage. Theoretically, after a 'flush', you could kick the cord out of the wall and those characters would still be safely stored.
I should mention that the people writing the os drivers or the people designing the disk drive might are free to use 'flush' as a suggestion and they might not really write the characters out. Even when the output is closed, they might wait a while to save them. (Remember that the os does all sorts of things at once and it might be more efficient to wait a second or two to handle your bytes.)
So a flush is a sort of checkpoint.
One more example: If the output is going to the console display, a flush will make sure the characters actually get all the way out to where the user can see them. This is an important thing to do when you are expecting keyboard input. If you think you have written a question to the console and its still stuck in some internal buffer somewhere, the user doesn't know what to type in answer. So, this is a case where the flush is important.
As the title suggests, I'm looking for a way to have cout 'flush' itself only when I explicitly tell it to. I know this can be achieved by 'buffering' the contents we want to write into a string or a stringstream, but I'm looking for the most efficient way of doing this.
Also it would be nice to also be able to eat up trailing newlines in this buffer. I've seen solutions that writes \b to cout; however I'm pretty sure this is unreliable due to flushing (correct me if I'm wrong).
Without using stringstream, your best bet it so prevent the two most common ways that lead to an automatic flush: when you are outputting something and when you explicitly call << std::endl. You can deactivate the first by setting the unitbuf flag of std::ios_base::fmtflags. This, though, won't likely stop the automatic flush when a new line is encountered. If your OS decides it needs to flush at every \n character, there's nothing you can do (on the standard C++ side).
Therefore I'd recommend manual buffering.
I have a large code base that uses std::cout to generate its output, and it uses std::endl all over the place to generate its newlines. This program seems to generate only linefeeds for the endl, which isn't a huge problem in itself, but for whatever reason, wasn't what I had expected.
So as a reality check, I built a simple program to pump endl's into cout, compiled it with the same compiler and examined the output from that. That program emits both CR and LF for endl.
It doesn't look like the large program plays any games with cout to change the way endl works, at least not that I can recognize, so it seems strange that it should behave differently than the small program. It seems as though the large program must be doing something to change the defaults. What am I missing here?
Both programs were compiled using MinGW gcc 4.5.2 on 32bit windows.
To add to Tomalak Geret'kal's answer: the fact that endl expands to the platform-specific endline is a popular misconception.
Inside a C/C++ application the only "official" newline (at least, as far as streams and stream-related functions are concerned) is '\n'. The translation from '\n' to the platform-specific newline is done inside the streams1 when they are opened in text mode (i.e. without the ios::bin flag).
endl, instead, is used to force a stream flush after the '\n'; this can be useful for console output, but (1) often this is not the case (cout is automatically flushed when input is requested from cin via the tied-stream mechanism) and only wastes CPU time, and (2) you often find it used also on file streams, where it is almost never useful, and results in poor file writing performance.
As discussed in the comments, as far as the standard is concerned the translation could actually happen at any level below the stream (e.g. the stream may translate the presence/absence of ios::bin into a flag for the underlying OS file management functions, and it would be responsibility of the OS to do the translation); in reality, mainstream OSes do not have particular flags for this kind of translation, mainly because their file APIs are content-agnostic (you tell them what to write, they write it without modifications).
std::endl is defined by the C++ Standard to perform the following:
Stream a '\n' character;
Stream the std::flush I/O manipulator.
If you're seeing carriage returns at the other end of the stream, then that's occurring at the file interface (and not in the stream functionality itself); in particular, on Windows, you can expect "\n" to be translated transparently to "\r\n" during text-mode output to files.
So, check that your files are both being opened in text- or binary- mode respectively. There is nothing else in C++ that could affect this behaviour.
This FAQ entry pretty much covers what I just said, too.
When a file is opened in "TEXT" Mode (the default when you create file streams).
The '\n' character is converted into a platform specific "end of line sequence" when you write it to a file. Conversely the "end of line sequence" is converted into a '\n' when you read from the file.
Note if you open a file in "BINARY" Mode this conversion does not happen.
In relation to std::endl
stream << std::endl
Is equivalent to:
stream << '\n' << std::flush;
Thus if your stream is a std::fstream (that was opened normally) then '\n' character will be converted into "end of line sequence" which on some platforms is '\r\n'
I have some C++ code that uses cout statements for debug purposes and for some reason I can't get all the data to print unless I do a std::cout.flush(); at the end.
I don't quite understand why this flush operation is needed.
Anyone have any insight?
To add to the other answers: your debugging statements should instead go to cerr, because:
it writes to the standard error, which means that, running the application, you can easily separate the "normal" program output from the errors/debug information via redirection;
most importantly, cerr by default is unbuffered, which means that, after each output operation, it will automatically flush itself, and in general this is desirable for errors and debug output.
(source: C++ standard, §27.3.1 ¶4-5, §27.4.2.1.2 table 83)
Are you using std::endl to terminate your lines. This should be the
usual practice, until performance issues require otherwise, but for some
reason, I see a lot of code which uses '\n' instead.
Otherwise, you can always do:
std::cout.setf( std::ios_base::unitbuf );
as one of the first things in main. This will cause a flush at the
end of every <<, which is more than you need, but for diagnostic
output to the console, is probably quite acceptable.
Is the data that isn't automatically getting flushed lacking a \n at the end? By default, standard out doesn't get delivered until a carriage return is seen.
"When you send output to a stream, it does not necessarily get printed immediately. Rather, it may wait in a buffer until some unspecified event, e.g. buffer full enough, reading from input, or exit from program. The details may vary."
http://www.cs.hmc.edu/~geoff/classes/hmc.cs070.200109/notes/io.html
It is the right behavior. You probably use std::endl that add \n and flush the buffer. http://www.cplusplus.com/reference/ostream/endl/
You need to flush the stream, if you want to see the output.
The answer of std::endl is only valid if you want a return.
Not sure how you would do this if you wanted to flush a command prompt out.
In C++ you can use endl formatter with cout operator rather then flush.
is there any way to automatically use correct EOL character depending on the OS used?
I was thinking of something like std::eol?
I know that it is very easy to use preprocessor directives but curious if that is already available.
What I am interested in is that I usually have some messages in my applications that I combine later into a single string and I want to have them separated with a EOL. I know that I could use std::stringstream << endl but it seems to be an overkill sometimes instead of a regular append.
std::endl is defined to do nothing besides write '\n' to the stream and flush it (§27.6.2.7). Flushing is defined to do nothing for a stringstream, so you're left with a pretty way of saying mystringstream << '\n'. The standard library implementation on your OS converts \n appropriately, so that's not your concern.
Thus endl is already the ultimate in performance and portability, and the only other thing you could want is << '\n' if you are trying to efficiently write to a file (not a stringstream). Well, << '\n' does also eliminate the pointless virtual call to stringbuf::flush. Unless profiling shows that empty function call to be taking time, don't think about it.
If you want to write a line separator to a stream:
std::cout << '\n';
or
std::cout << "\n";
or
std::cout << "whatever you were going to say anyway\n";
If the stream is text mode and the OS uses anything other than LF as a separator, it will be converted.
If you want to write a line separator and flush the stream:
std::cout << std::endl;
If you have binary-mode output for whatever reason, and you want to write a platform-specific line break, then I think you might have to do it indirectly (write '\n' to a text stream and then examine it in binary mode to see what you get). Possibly there's some way to directly get the line break sequence from the implementation, that I'm not aware of. It's not a great idea, anyway: if you're writing or reading a file in binary mode then it should be in a format which defines line breaks independently of the OS, or which doesn't have lines at all. That's what binary mode is for.
Just open a file in text mode
FILE *fp = fopen( "your_file.txt", "w+t" );
and then
fprintf( fp, "some string and integer %d\n", i );
fclose(fp);
and the OS will take care of the EOL accordingly to its standards.
Well, the STL has std::endl, which you can use as
std::cout << "Hi five!" << std::endl;
Note that besides adding an endline, std::endl also flushes the buffer, which may have undesirable performance consequences.
Files, even text files, are often transferred between machines, so "os-specific new line character" is an oxymoron.
It is though true that operating systems have a say on that matter, particularly one operating systems aka Windows, although many windows programs will read \n-spaced files correctly, even though the winapi multiline edit control would not. I suggest you consider twice what is the right for you: it's not necessarily what your OS recommends. If your files are ever to be stored on removable media, do not use OS standard. Use global standard, 0xA.