I'm testing C++ code for stream buffering. As far as my understanding goes for buffering idea, the following code should print out "Before loop " and "After loop" at the same time, even though there is delay in form of for loop. Problem is that it prints them with that loop delay. Can someone explain the reason to me? I'm passing cout as an argument.
void testBuffer(ostream& os){
os << "Before loop - ";
for(int i = 0; i < 2000000000; i++){
// waste time
}
os << "After loop " << endl;
}
Buffers are not infinite, and in the case of non-file streams probably not even all that large.
Just because you did not write std::flush doesn't mean there definitely won't be an immediate response from the stream. If the buffer is full, it's still going to flush. It's just that you're not forcing an early flush.
Furthermore you may conceivably see std::cout behaving like std::cerr (which basically disables buffering) in debug modes. I don't know whether any implementation does this.
Moral of the story:
if you need the output immediately, flush;
if you need it later, write it later;
if and only if you don't care either way, do it the way you've done it.
Related
Just started C++ learning by C++ Primer 5th ed.
The very first example of the book on page 6 is as followed
#include <iostream>
int main()
{
std::cout << "Enter two numbers:" << std::endl;
int v1 = 0, v2 = 0;
std::cin >> v1 >> v2;
std::cout << "The sum of " << v1 << " and " << v2
<< " is " << v1 + v2 << std::endl;
return 0;
}
The manipulator endl insert newline and flush the buffer.
soon followed by the code snipe in the page 7, the author emphasized that
Programmers often add print statements during debugging. Such
statements should always flush the stream. Otherwise, if the program
crashes, output may be left in the buffer, leading to incorrect
inference about where the program crashed
From the code example and the emphasized warning I feel it is important to do the flush when writing to a Stream
Here is the part I don't understand, how a bout reading from a stream case, such as std::cin, is there any necessary to do the flush then?
Appended Question：
#include <iostream>
int main()
{
int sum = 0, val = 1;
while (val <= 5) {
sum += val;
++val;
std::cout << sum << std::endl; //test
}
}
When I changed the line marked with test to std::cout << sum << '\n';, there is no visual difference in the console. Why is that, isn't it supposed to print as follows if there is no flush for each loop?
1
1 3
1 3 6
1 3 6 10
1 3 6 10 15
Thanks
Even if #Klitos Kyriacou is right when he says you should create a new post for a new question, it seems to me that both your questions arise from the same misunderstanding.
Programmers often add print statements during debugging. Such statements should always flush the stream. Otherwise, if the program crashes, output may be left in the buffer, leading to incorrect inference about where the program crashed
This quote does not mean that you need to flush every buffer in your program to create any output on the console.
By flushing the buffers you can make sure that the output is printed before the next line of code is executed.
If you don't flush the buffers and your program finishes, the buffers will be flushed anyway.
So, the reason you see the same output on the console with std::endl and \n is, that exactly the same text is printed to the console. In the former case, the output might be there slightly earlier, as the buffers are flushed early. In the latter case, the buffers are flushed later, but they will be flushed at some time.
What the quote talks about is the case when your program does not exit gracefully, e.g. when your program crashes or is interrupted by the OS. In these cases, your output might not be written to the console when you did not explicitly flush the buffers.
What the quote wants you to know is: Whenever you want to debug a program that crashes, you should explicitly flush the buffers to make sure your debug output is printed to the console before your program is interrupted.
Note that this might not be true for all implementations.
From http://en.cppreference.com/w/cpp/io/manip/endl
In many implementations, standard output is line-buffered, and writing '\n' causes a flush anyway, unless std::ios::sync_with_stdio(false) was executed.
Quote from the same book page 26:
Buffer A region of storage used to hold data. IO facilities often store input (or out-put) in a buffer and read or write the buffer
independently from actions in the program. Output buffers can be
explicitly flushed to force the buffer to be written. Be default,
reading cin flushes cout; cout is also flushed when the program ends
normally.
This question already has answers here:
endl and flushing the buffer
(5 answers)
Closed 6 years ago.
std::cout << "Enter two numbers:";
std::cout << std:endl;
This code snippet is followed by two paragraphs and a warning note, among which I understood the first para, but neither the second one nor the note. The text is as follows -
"The first output operator prints a message to the user. That message
is a string literal, which is a sequence of characters enclosed in
double quotation marks. The text between the quotation marks is
printed to the standard output.
The second operator prints endl,
which is a special value called a manipulator. Writing endl has
the effect of ending the current line and flushing the buffer
associated with that device. Flushing the buffer ensures that all the
output the program has generated so far is actually written to the
output stream, rather than sitting in memory waiting to be written.
Warning Programmers often add print statements during debugging. Such statement should always flush the stream. Otherwise, if the
program crashes, output may be left in the buffer, leading to
incorrect inferences about where the program crashed."
So I didn't understand of the part of endl, nor the following warning. Can anyone please explain this to me as explicitly as possible and please try to keep it simple.
Imagine you have some code that crashes somewhere, and you don't know where. So you insert some print statements to narrow the problem down:
std::cout << "Before everything\n";
f1();
std::cout << "f1 done, now running f2\n";
f2();
std::cout << "all done\n";
Assuming that the program crashes during the evaluation of either f1() or f2(), you may not see any output, or you may see partial output that is misleading -- e.g. you could see only "Before everything", even though the crash happened in f2(). That's because the output data may be waiting in a buffer and hasn't actually been written to the output device.
The Primer's recommendation is therefore to flush each output, which you can conveniently achieve with endl:
std::cout << "Before everything" << std::endl;
f1();
std::cout << "f1 done, now running f2" << std::endl;
f2();
std::cout << "all done" << std::endl;
An alternative is to write debug output to std::cerr instead, which is not buffered by default (though you can always change the buffering of any ostream object later).
A more realistic use case is when you want to print a progress bar in a loop. Usually, a newline (\n) causes line-based output to be printed anyway, but if you want to print a single character for progress, you may not see it printed at all until after all the work is done unless you flush:
for (int i = 0; i != N; ++i)
{
if (i % 1000 == 0)
{
std::cout << '#'; // progress marger
std::cout.flush();
}
do_work();
}
std::cout << '\n';
Well, simply:
std::cout << "Hello world!";
will print "Hello world!" and will remain in the same line. Now if you want to go to a new line, you should use:
std::cout << "\n";
or
std::cout << std::endl;
Now before I explain the difference, you have to know 1 more simple thing: When you issue a print command with the std::cout stream, things are not printed immediately. They are stored in a buffer, and at some point this buffer is flushed, either when the buffer is full, or when you force it to flush.
The first kind, \n, will not flush, but the second kind std::endl, will go to a new line + flush.
Operating systems do buffered IO. That is, when your program outputs something, they dont necessarily put it immediately where it should go (i.e. disk, or the terminal), they might decide to keep the data in an internal memory buffer for some while before performing the actual IO operation on the device.
They do this to optmize performance, because doing the IO in chunks is better than doing it immediately as soon as there are a few bytes to write.
Flushing a buffer means asking the OS to perform immediately the IO operation without any more waiting. A programmer would do this this when (s)he knows that waiting for more data doesn't make sense.
The second note says that endl not only prints a newline, but also hints the cout to flush its buffer.
The 3rd note warns that debugging errors, if buffered and not flushed immediately, might not be seen if the program crashes while the error messages are still in the buffer (not flushed yet).
I was wondering what is the most efficient performant way to output a new line to console. Please explain why one technique is more efficient. Efficient in terms of performance.
For example:
cout << endl;
cout << "\n";
puts("");
printf("\n");
The motivation for this question is that I find my self writing loops with outputs and I need to output a new line after all iterations of the loop. I'm trying to find out what's the most efficient way to do this assuming nothing else matters. This assumption that nothing else matters is probably wrong.
putchar('\n') is the most simple and probably fastest. cout and printf with string "\n" work with null terminated string and this is slower because you process 2 bytes (0A 00). By the way, carriage return is \r = 13 (0x0D). \n code is Line Feed (LF).
You don't specify whether you are demanding that the update to the screen is immediate or deferred until the next flush. Therefore:
if you're using iostream io:
cout.put('\n');
if you're using stdio io:
std::putchar('\n');
The answer to this question is really "it depends".
In isolation - if all you're measuring is the performance of writing a '\n' character to the standard output device, not tweaking the device, not changing what buffering occurs - then it will be hard to beat options like
putchar('\n');
fputchar('\n', stdout);
std::cout.put('\n');
The problem is that this doesn't achieve much - all it does (assuming the output is to a screen or visible application window) is move the cursor down the screen, and move previous output up. Not exactly a entertaining or otherwise valuable experience for a user of your program. So you won't do this in isolation.
But what comes into play to affect performance (however you measure that) if we don't output newlines in isolation? Let's see;
Output of stdout (or std::cout) is buffered by default. For the output to be visible, options include turning off buffering or for the code to periodically flush the buffer. It is also possible to use stderr (or std::cerr) since that is not buffered by default - assuming stderr is also directed to the console, and output to it has the same performance characteristics as stdout.
stdout and std::cout are formally synchronised by default (e.g. look up std::ios_base::sync_with_stdio) to allow mixing of output to stdout and std::cout (same goes for stderr and std::cerr)
If your code outputs more than a set of newline characters, there is the processing (accessing or reading data that the output is based on, by whatever means) to produce those other outputs, the handling of those by output functions, etc.
There are different measures of performance, and therefore different means of improving efficiency based on each one. For example, there might be CPU cycles, total time for output to appear on the console, memory usage, etc etc
The console might be a physical screen, it might be a window created by the application (e.g. hosted in X, windows). Performance will be affected by choice of hardware, implementation of windowing/GUI subsystems, the operating system, etc etc.
The above is just a selection, but there are numerous factors that determine what might be considered more or less performance.
On Ubuntu 15.10, g++ v5.2.1 (and an older vxWorks, and OSE)
It is easy to demonstrate that
std::cout << std::endl;
puts a new line char into the output buffer, and then flushes the buffer to the device.
But
std::cout << "\n";
puts a new line char into the output buffer, and does not output to the device. Some future action will be needed to trigger the output of the newline char in the buffer to the device.
Two such actions are:
std::cout << std::flush; // will output the buffer'd new line char
std::cout << std::endl; // will output 2 new line chars
There are also several other actions that can trigger the flush of the std::cout buffering.
#include <unistd.h> // for Linux
void msDelay (int ms) { usleep(ms * 1000); }
int main(int, char**)
{
std::cout << "with endl and no delay " << std::endl;
std::cout << "with newline and 3 sec delay " << std::flush << "\n";
msDelay(3000);
std::cout << std::endl << " 2 newlines";
return(0);
}
And, per comment by someone who knows (sorry, I don't know how to copy his name here), there are exceptions for some environments.
It's actually OS/Compiler implementation dependent.
The most efficient, least side effect guaranteed way to output a '\n' newline character is to use std::ostream::write() (and for some systems requires std::ostream was opened in std::ios_base::binary mode):
static const char newline = '\n';
std::cout.write(&newline,sizeof(newline));
I would suggest to use:
std::cout << '\n'; /* Use std::ios_base::sync_with_stdio(false) if applicable */
or
fputc('\n', stdout);
And turn the optimization on and let the compiler decide what is best way to do this trivial job.
Well if you want to change the line I'd like to add the simplest and the most common way which is using (endl), which has the added perk of flushing the stream, unlike cout << '\n'; on its own.
Example:
cout << "So i want a new line" << endl;
cout << "Here is your new line";
Output:
So i want a new line
Here is your new line
This can be done for as much new lines you want. Allow me to show an example using 2 new lines, it'll definitely clear all of your doubts,
Example:
cout << "This is the first line" << endl;
cout << "This is the second line" << endl;
cout << "This is the third line";
Output:
This is the first line
This is the second line
This is the third line
The last line will just have a semicolon to close since no newline is needed. (endl) is also chain-able if needed, as an example, cout << endl << endl; would be a valid sequence.
I am learning C++ and I found something that I can't understand:
Output buffers can be explicitly flushed to force the buffer to be
written. By default, reading cin flushes cout; cout is also flushed
when the program ends normally.
So flushing the buffer (for example an output buffer): does this clear the buffer by deleting everything in it or does it clear the buffer by outputting everything in it? Or does flushing the buffer mean something completely different?
Consider writing to a file. This is an expensive operation. If in your code you write one byte at a time, then each write of a byte is going to be very costly. So a common way to improve performance is to store the data that you are writing in a temporary buffer. Only when there is a lot of data is the buffer written to the file. By postponing the writes, and writing a large block in one go, performance is improved.
With this in mind, flushing the buffer is the act of transferring the data from the buffer to the file.
Does this clear the buffer by deleting everything in it or does it clear the buffer by outputting everything in it?
The latter.
You've quoted the answer:
Output buffers can be explicitly flushed to force the buffer to be written.
That is, you may need to "flush" the output to cause it to be written to the underlying stream (which may be a file, or in the examples listed, a terminal).
Generally, stdout/cout is line-buffered: the output doesn't get sent to the OS until you write a newline or explicitly flush the buffer. The advantage is that something like std::cout << "Mouse moved (" << p.x << ", " << p.y << ")" << endl causes only one write to the underlying "file" instead of six, which is much better for performance. The disadvantage is that a code like:
for (int i = 0; i < 5; i++) {
std::cout << ".";
sleep(1); // or something similar
}
std::cout << "\n";
will output ..... at once (for exact sleep implementation, see this question). In such cases, you will want an additional << std::flush to ensure that the output gets displayed.
Reading cin flushes cout so you don't need an explicit flush to do this:
std::string colour;
std::cout << "Enter your favourite colour: ";
std::cin >> colour;
Clear the buffer by outputting everything.
I am trying to print results in 2 nested for cycles using std::cout. However, the results are not printed to the console immediately, but with delay (after both for cycles or the program have been finished).
I do not consider such behavior normal, under Windows printing works OK. The program does not use threads.
Where could be the problem? (Ubuntu 10.10 + NetBeans 6.9).
std::cout is an stream, and it is buffered. You can flush it by several ways:
std::cout.flush();
std::cout << std::flush;
std::cout << std::endl; // same as: std::cout << "\n" << std::flush`
johny:
I am flushing the buffer before the cycle using std::endl. The problem arises when printing dot representing % of processed data inside the cycle.
If you flush the buffer before the cycle, that does not affect the output in the cycle. You have to flush in or after the cycle, to see the output.
If you don't flush your output, your output is not guaranteed to be visible outside your program. The fact that it is not printing in your terminal is just a consequence of the default behavior in linux to do line buffering when the output is a tty. If you run your program on linux with its output piped to another thing, like
./your_program | cat
Then the default buffer will be MUCH larger, most likely it will be at least 4096 bytes. So nothing will get displayed until the big buffer gets full. but really, the behaviour is OS specific unless you flush std::cout yourself.
To flush std::cout, use :
std::cout << std::flush;
also, using
std::cout << std::endl;
is a shortcut to
std::cout << '\n' << std::flush;