As the title suggests, I'm looking for a way to have cout 'flush' itself only when I explicitly tell it to. I know this can be achieved by 'buffering' the contents we want to write into a string or a stringstream, but I'm looking for the most efficient way of doing this.
Also it would be nice to also be able to eat up trailing newlines in this buffer. I've seen solutions that writes \b to cout; however I'm pretty sure this is unreliable due to flushing (correct me if I'm wrong).
Without using stringstream, your best bet it so prevent the two most common ways that lead to an automatic flush: when you are outputting something and when you explicitly call << std::endl. You can deactivate the first by setting the unitbuf flag of std::ios_base::fmtflags. This, though, won't likely stop the automatic flush when a new line is encountered. If your OS decides it needs to flush at every \n character, there's nothing you can do (on the standard C++ side).
Therefore I'd recommend manual buffering.
Related
Can someone please explain (preferably using plain english) how std::flush works?
What is it?
When would you flush a stream?
Why is it important?
Thank you.
Since it wasn't answered what std::flush happens to be, here is some detail on what it actually is. std::flush is a manipulator, i.e., a function with a specific signature. To start off simple, you can think of std::flush of having the signature
std::ostream& std::flush(std::ostream&);
The reality is a bit more complex, though (if you are interested, it is explained below as well).
The stream class overload output operators taking operators of this form, i.e., there is a member function taking a manipulator as argument. The output operator calls the manipulator with the object itself:
std::ostream& std::ostream::operator<< (std::ostream& (*manip)(std::ostream&)) {
(*manip)(*this);
return *this;
}
That is, when you "output" std::flush with to an std::ostream, it just calls the corresponding function, i.e., the following two statements are equivalent:
std::cout << std::flush;
std::flush(std::cout);
Now, std::flush() itself is fairly simple: All it does is to call std::ostream::flush(), i.e., you can envision its implementation to look something like this:
std::ostream& std::flush(std::ostream& out) {
out.flush();
return out;
}
The std::ostream::flush() function technically calls std::streambuf::pubsync() on the stream buffer (if any) which is associated with the stream: The stream buffer is responsible for buffering characters and sending characters to the external destination when the used buffer would overflow or when the internal representation should be synced with the external destination, i.e., when the data is to be flushed. On a sequential stream syncing with the external destination just means that any buffered characters are immediately sent. That is, using std::flush causes the stream buffer to flush its output buffer. For example, when data is written to a console flushing causes the characters to appear at this point on the console.
This may raise the question: Why aren't characters immediately written? The simple answer is that writing characters is generally fairly slow. However, the amount of time it takes to write a reasonable amount of characters is essentially identical to writing just one where. The amount of characters depends on many characteristics of the operating system, file systems, etc. but often up to something like 4k characters are written in about the same time as just one character. Thus, buffering characters up before sending them using a buffer depending on the details of the external destination can be a huge performance improvement.
The above should answer two of your three questions. The remaining question is: When would you flush a stream? The answer is: When the characters should be written to the external destination! This may be at the end of writing a file (closing a file implicitly flushes the buffer, though) or immediately before asking for user input (note that std::cout is automatically flushed when reading from std::cin as std::cout is std::istream::tie()'d to std::cin). Although there may be a few occasions where you explicitly want to flush a stream, I find them to be fairly rare.
Finally, I promised to give a full picture of what std::flush actually is: The streams are class templates capable of dealing with different character types (in practice they work with char and wchar_t; making them work with another characters is quite involved although doable if you are really determined). To be able to use std::flush with all instantiations of streams, it happens to be a function template with a signature like this:
template <typename cT, typename Traits>
std::basic_ostream<cT, Traits>& std::flush(std::basic_ostream<cT, Traits>&);
When using std::flush immediately with an instantiation of std::basic_ostream it doesn't really matter: The compiler deduces the template arguments automatically. However, in cases where this function isn't mentioned together with something facilitating the template argument deduction, the compiler will fail to deduce the template arguments.
By default, std::cout is buffered, and the actual output is only printed once the buffer is full or some other flushing situation occurs (e.g. a newline in the stream). Sometimes you want to make sure that the printing happens immediately, and you need to flush manually.
For example, suppose you want to report a progress report by printing a single dot:
for (;;)
{
perform_expensive_operation();
std::cout << '.';
std::flush(std::cout);
}
Without the flushing, you wouldn't see the output for a very long time.
Note that std::endl inserts a newline into a stream as well as causing it to flush. Since flushing is mildly expensive, std::endl shouldn't be used excessively if the flushing isn't expressly desired.
Here's a short program that you can write to observe what flush is doing
#include <iostream>
#include <unistd.h>
using namespace std;
int main() {
cout << "Line 1..." << flush;
usleep(500000);
cout << "\nLine 2" << endl;
cout << "Line 3" << endl ;
return 0;
}
Run this program: you'll notice that it prints line 1, pauses, then prints line 2 and 3. Now remove the flush call and run the program again- you'll notice that the program pauses and then prints all 3 lines at the same time. The first line is buffered before the program pauses, but because the buffer is never flushed, line 1 is not outputted until the endl call from line 2.
A stream is connected to something. In the case of standard output, it could be the console/screen or it could be redirected to a pipe or a file. There is a lot of code between your program and, for example, the hard disk where the file is stored. For example, the operating system is doing stuff with any file or the disk drive itself might be buffering data to be able to write it in fixed size blocks or just to be more efficient.
When you flush the stream, it tells the language libraries, the os and the hardware that you want to any characters that you have output so far to be forced all the way to storage. Theoretically, after a 'flush', you could kick the cord out of the wall and those characters would still be safely stored.
I should mention that the people writing the os drivers or the people designing the disk drive might are free to use 'flush' as a suggestion and they might not really write the characters out. Even when the output is closed, they might wait a while to save them. (Remember that the os does all sorts of things at once and it might be more efficient to wait a second or two to handle your bytes.)
So a flush is a sort of checkpoint.
One more example: If the output is going to the console display, a flush will make sure the characters actually get all the way out to where the user can see them. This is an important thing to do when you are expecting keyboard input. If you think you have written a question to the console and its still stuck in some internal buffer somewhere, the user doesn't know what to type in answer. So, this is a case where the flush is important.
Flush happens in the following cases:
std::cerr
std::cin
program termination
in many implementations, standard output is line-buffered, meaning "\n" flushes anyway
So it seems in most regular programs, std::endl is basically unnecessary, but it's used almost everywhere.
std::cerr
But what if I do not want to write to stderr, but rather stdout?
std::cin
But what if I just want to give the user some status update and do not want any input?
program termination
As above, what if I want to tell the user something before the program is done?
in many implementations, standard output is line-buffered, meaning "\n" flushes anyway
But what about the platforms that don't? Regardless of whether or not such a platform currently exists, I want my code to work as intended as defined by the standard, not as by "meh, will probably be fine on all implementations".
If all of the above apply (the fourth point actually always applies), you need to flush "by hand". If the flush gets handled by one of the first three, you don't.
So it seems in most regular programs, std::endl is basically unnecessary, but it's used almost everywhere.
No it's not redundant, since implementation of flushing along '\n' isn't mandated by the standard. That's just an implementation specific behavior, while the behavior of std::endl is always clearly defined.
Here my situation, the users of my application are asked to enter their password to start it. To get the password I simply use:
char c;
std::string password;
while ... // until the end of entry
{
c = fgetc(stdin);
password += c;
}
Once the password is checked I destroy the variable so it can't be retrieved using a core image of my program. For instance if someone use "gcore" command and then search for the password in the core generated it will not find it.
But in my case, I can still retrieve the password value because it seems that it is still in stdin buffer.
So my question is how can I clear stdin buffer in order to make the values typed by user not available in memory anymore ?
For information I already tried: fflush, __fpurge, fwrite (from the beginning position of stdin stream)... and nothing seems to work.
Thank you in advance.
My answer is: don't use stand I/O streams - use raw file I/O instead:
read(STDIN_FILENO, ...)
You'll have to do your own line buffering but you can guarantee that nothing in the libraries is keeping a buffer of your input.
Have you checked this? How do I flush the cin buffer?
Try:
cin.clear();
Since you're using C++ why not just go with the std::string class and use cin>>password directly. this will eliminate the need for a separate variable to hold each character as it's typed. If, after that, you're still worried about the contents of stdin being available just use:
fseek(stdin,0,SEEK_END);
at the end of the read. In my opinion this is much easier to use than coding the old (and in my opinion, less secure) C methods and allows you to use the C++ libraries better.
General rule of thumb when coding C++: Don't use C unless you absolutely have to.
I'm completely ignorant of C++, but what I'd do in C is either what DoxyLover suggested or, if you want to stick to using the standard library... use it without buffering, googling "c++ cin unbuffered" does give a few results after all
I am trying to read some characters that satisfy certain condition from stdin with iostream library while leave those not satisfying the condition in stdin so that those skipped characters can be read later. Is it possible?
For example, I want characters in a-c only and the input stream is abdddcxa.
First read in all characters in a-c - abca; after this input finished, start read the remaining characters dddx. (This two inputs can't happen simultaneously. They might be in two different functions).
Wouldn't it be simpler to read everything, then split the input into the two parts you need and finally send each part to the function that needs to process it?
Keeping the data in the stdin buffer is akin to using globals, it makes your program harder to understand and leaves the risk of other code (or the user) changing what is in the buffer while you process it.
On the other hand, dividing your program into "the part that reads the data", "the part that parses the data and divides the workload" and the "part that does the work" makes for a better structured program which is easy to understand and test.
You can probably use regex to do the actual split.
What you're asking for is the putback method (for more details see: http://www.cplusplus.com/reference/istream/istream/putback/). You would have to read everything, filter the part that you don't want to keep out, and put it back into the stream. So for instance:
cin >> myString;
// Do stuff to fill putbackBuf[] with characters in reverse order to be put back
pPutbackBuf = &putbackBuf[0];
do{
cin.putback(*(pPutbackBuf++));
while(*pPutbackBuf);
Another solution (which is not exactly what you're asking for) would be to split the input into two strings and then feed the "non-inputted" string into a stringstream and pass that to whatever function needs to do something with the rest of the characters.
What you want to do is not possible in general; ungetc and putback exist, but they're not guaranteed to work for more than one character. They don't actually change stdin; they just push back on an input buffer.
What you could do instead is to explicitly keep a buffer of your own, by reading the input into a string and processing that string. Streams don't let you safely rewind in many cases, though.
No, random access is not possible for streams (except for fstream an stringstream). You will have to read in the whole line/input and process the resulting string (which you could, however, do using iostreams/std::stringstream if you think it is the best tool for that -- I don't think that but iostreams gurus may differ).
I have some C++ code that uses cout statements for debug purposes and for some reason I can't get all the data to print unless I do a std::cout.flush(); at the end.
I don't quite understand why this flush operation is needed.
Anyone have any insight?
To add to the other answers: your debugging statements should instead go to cerr, because:
it writes to the standard error, which means that, running the application, you can easily separate the "normal" program output from the errors/debug information via redirection;
most importantly, cerr by default is unbuffered, which means that, after each output operation, it will automatically flush itself, and in general this is desirable for errors and debug output.
(source: C++ standard, §27.3.1 ¶4-5, §27.4.2.1.2 table 83)
Are you using std::endl to terminate your lines. This should be the
usual practice, until performance issues require otherwise, but for some
reason, I see a lot of code which uses '\n' instead.
Otherwise, you can always do:
std::cout.setf( std::ios_base::unitbuf );
as one of the first things in main. This will cause a flush at the
end of every <<, which is more than you need, but for diagnostic
output to the console, is probably quite acceptable.
Is the data that isn't automatically getting flushed lacking a \n at the end? By default, standard out doesn't get delivered until a carriage return is seen.
"When you send output to a stream, it does not necessarily get printed immediately. Rather, it may wait in a buffer until some unspecified event, e.g. buffer full enough, reading from input, or exit from program. The details may vary."
http://www.cs.hmc.edu/~geoff/classes/hmc.cs070.200109/notes/io.html
It is the right behavior. You probably use std::endl that add \n and flush the buffer. http://www.cplusplus.com/reference/ostream/endl/
You need to flush the stream, if you want to see the output.
The answer of std::endl is only valid if you want a return.
Not sure how you would do this if you wanted to flush a command prompt out.
In C++ you can use endl formatter with cout operator rather then flush.