This question already has answers here:
How can I compose output streams, so output goes multiple places at once?
(3 answers)
Closed 4 months ago.
I have to send the same string (e.g. a log message) to multiple streams.
Which of the following solutions is the most efficient?
Rebuild the same string for each stream and send it to the stream itself.
outstr1 << "abc" << 123 << 1.23 << "def" << endl;
outstr2 << "abc" << 123 << 1.23 << "def" << endl;
outstr3 << "abc" << 123 << 1.23 << "def" << endl;
Build the string once with string's operators, and send it to all the streams.
std::string str = "abc" + std::to_string(123) + std::to_string(1.23) + "def";
outstr1 << str;
outstr2 << str;
outstr3 << str;
Build the string once with a stream, and send it to all the streams:
std::stringstream sstm;
sstm << "abc" << 123 << 1.23 << "def" << endl;
std::string str = sstm.str();
outstr1 << str;
outstr2 << str;
outstr3 << str;
Some or all of these output streams could be on a RAM disk.
Any other ways to do the same thing?
I would use a tee output stream. Do something like (pseudocode):
allstreams = tee(outstr1, outstr2, outstr3);
allstreams << "abc" << 123 << 1.23 << "def" << endl;
There doesn't seem to be anything in the standard c++ library to do this, but Boost has one.
See also the answers to How can I compose output streams, so output goes multiple places at once?
Although it is unlikely that you would see much difference either way1, option #3 sounds the most plausible: unlike the first option, it does not convert ints to strings multiple times; unlike the second option, it does not allocate and delete multiple string objects for its intermediate results2. It also looks cleanest from the readability point of view: no code is duplicated, and output looks like an output, not like a concatenation.
1 Insert a mandatory disclaimer about optimization before profiling being evil here.
2 Small String Optimization may help on systems where it is supported (thanks, Prætorian), but the constructor and destructor calls for the intermediate objects are not going away.
The "proper" way to do something like this is to have a stream buffer writing to multiple destinations and use this stream buffer via a std::ostream. This way the code looks as if it writing just once but the characters are sent multiple times. Searching for "teebuf Dietmar" will find a few variations on the same theme.
To also comment on your question: Which one of the three alternatives is the fastest depends on the exact expressions you have:
needs to evaluate the involved expressions and perform the conversions three times. Depending on what you actually do this may still be fairly fast.
actually creates and destroys multiple streams and does multiple allocations for std::string. I'd expect this to be slowest.
still creates a stream (which should actually be a std::ostringstream) and allocates some memory. Out of your options I'd expect it to be fastest.
Using a teebuf is probably fastest, at least, when it does some buffering but uses only fixed suze arrays, both for the buffer and the array of stream buffer pointers. Note, that you'll need to override sync() to deal with the buffer in a timely manner, though.
To determine the actual performance you'll need to measure!
Related
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 6 years ago.
Improve this question
I am trying to add data into a text file. However, i see for some reason it produces garbage data. I also notice, it will input the correct data once, but then it follow with garbage data.
void TextFileLogger::log(std::string msg){
using namespace std;
//ofstream output_file("students.data", ios::binary);
std::ofstream logFile;
// creating, opening and writing/appending data to a file
char filename[] = "log.txt";
logFile.open(filename, ios::binary | ios::app |ios::out);
if (logFile.fail())
{
std::cout << "The " << filename << " file could not be created/opened!" << std::endl;
// 0-normal, non zero - some errors
}
else
{
if (!logFile.write((char*)&msg, sizeof(msg)))
{
cout << "Could not write file" << endl;
}
else
{
streamsize bytesWritten = logFile.tellp();
if (bytesWritten != sizeof(msg))
{
cout << "Could not write expected number of bytes" << endl;
}
else
{
logFile << msg << std::endl;
cout << "file written OK" << endl;
}
}
}
}
That one is fun!
(char*)&msg does not do what you expect: std::string is mainly a pointer to a dynamically-allocated buffer which contains the actual data. When you take the std::string's address and try to read what's inside, you get a view of its innards, not its data. Using a C++ static_cast here would have spared you the trouble by telling you that the conversion makes no sense. sizeof(msg) similarly returns the size of the std::string, not the length of its data.
So, your solution is: use msg.data() and msg.size(), it's exactly what they're designed for.
But... why would it (sometimes) output your string, and a bunch of garbage? Well, std::strings typically use SSO (Small String Optimization). The std::string actually contains a small buffer, to store short enough strings without dynamic allocation. When you inspect the whole std::string object, you see this buffer pass by.
You are writing the contents of the whole std::string object, with all the member variables that it contains internally.
You either want:
logFile << msg;
or if you really want to use write():
logFile.write( msg.c_str(), msg.length());
And, I wonder: Why do create/open the file in binary mode, when you write strings afterwards?
And finally, you write the data twice, the second time in your last else clause.
The problem is with this line:
if (!logFile.write((char*)&msg, sizeof(msg)))
It should be this:
if (!logFile.write(msg.c_str(), msg.length()))
Since you are passing a std::string into the function, you should take advantage of the functions it provides (c_str() and length()) instead of trying to cast it to a char* (this always gets messy, plus you are casting away the const, which is also typically bad).
This:
if (!logFile.write((char*)&msg, sizeof(msg)))
is wrong in so many ways. msg is not an array of char, it's a std::string - lying to the compiler by using a cast is always a bad thing to do. And the size of a string is not the size of the characters it contains. Why the heck are you not using the obvios:
logfile << msg << std::endl;
Replace sizeof(msg) with msg.size(), sizeof() is not doing what you think!
Also (char*)&msg does not do whatever you think, use msg.data() instead.
logFile.write((char*)&msg, sizeof(msg));
should be rewritten to:
logFile.write(msg.data(), msg.size());
or, even better, because ofstream overrides operator<< for std::string:
logfile << msg;
This question already has answers here:
endl and flushing the buffer
(5 answers)
Closed 6 years ago.
std::cout << "Enter two numbers:";
std::cout << std:endl;
This code snippet is followed by two paragraphs and a warning note, among which I understood the first para, but neither the second one nor the note. The text is as follows -
"The first output operator prints a message to the user. That message
is a string literal, which is a sequence of characters enclosed in
double quotation marks. The text between the quotation marks is
printed to the standard output.
The second operator prints endl,
which is a special value called a manipulator. Writing endl has
the effect of ending the current line and flushing the buffer
associated with that device. Flushing the buffer ensures that all the
output the program has generated so far is actually written to the
output stream, rather than sitting in memory waiting to be written.
Warning Programmers often add print statements during debugging. Such statement should always flush the stream. Otherwise, if the
program crashes, output may be left in the buffer, leading to
incorrect inferences about where the program crashed."
So I didn't understand of the part of endl, nor the following warning. Can anyone please explain this to me as explicitly as possible and please try to keep it simple.
Imagine you have some code that crashes somewhere, and you don't know where. So you insert some print statements to narrow the problem down:
std::cout << "Before everything\n";
f1();
std::cout << "f1 done, now running f2\n";
f2();
std::cout << "all done\n";
Assuming that the program crashes during the evaluation of either f1() or f2(), you may not see any output, or you may see partial output that is misleading -- e.g. you could see only "Before everything", even though the crash happened in f2(). That's because the output data may be waiting in a buffer and hasn't actually been written to the output device.
The Primer's recommendation is therefore to flush each output, which you can conveniently achieve with endl:
std::cout << "Before everything" << std::endl;
f1();
std::cout << "f1 done, now running f2" << std::endl;
f2();
std::cout << "all done" << std::endl;
An alternative is to write debug output to std::cerr instead, which is not buffered by default (though you can always change the buffering of any ostream object later).
A more realistic use case is when you want to print a progress bar in a loop. Usually, a newline (\n) causes line-based output to be printed anyway, but if you want to print a single character for progress, you may not see it printed at all until after all the work is done unless you flush:
for (int i = 0; i != N; ++i)
{
if (i % 1000 == 0)
{
std::cout << '#'; // progress marger
std::cout.flush();
}
do_work();
}
std::cout << '\n';
Well, simply:
std::cout << "Hello world!";
will print "Hello world!" and will remain in the same line. Now if you want to go to a new line, you should use:
std::cout << "\n";
or
std::cout << std::endl;
Now before I explain the difference, you have to know 1 more simple thing: When you issue a print command with the std::cout stream, things are not printed immediately. They are stored in a buffer, and at some point this buffer is flushed, either when the buffer is full, or when you force it to flush.
The first kind, \n, will not flush, but the second kind std::endl, will go to a new line + flush.
Operating systems do buffered IO. That is, when your program outputs something, they dont necessarily put it immediately where it should go (i.e. disk, or the terminal), they might decide to keep the data in an internal memory buffer for some while before performing the actual IO operation on the device.
They do this to optmize performance, because doing the IO in chunks is better than doing it immediately as soon as there are a few bytes to write.
Flushing a buffer means asking the OS to perform immediately the IO operation without any more waiting. A programmer would do this this when (s)he knows that waiting for more data doesn't make sense.
The second note says that endl not only prints a newline, but also hints the cout to flush its buffer.
The 3rd note warns that debugging errors, if buffered and not flushed immediately, might not be seen if the program crashes while the error messages are still in the buffer (not flushed yet).
I was wondering what is the most efficient performant way to output a new line to console. Please explain why one technique is more efficient. Efficient in terms of performance.
For example:
cout << endl;
cout << "\n";
puts("");
printf("\n");
The motivation for this question is that I find my self writing loops with outputs and I need to output a new line after all iterations of the loop. I'm trying to find out what's the most efficient way to do this assuming nothing else matters. This assumption that nothing else matters is probably wrong.
putchar('\n') is the most simple and probably fastest. cout and printf with string "\n" work with null terminated string and this is slower because you process 2 bytes (0A 00). By the way, carriage return is \r = 13 (0x0D). \n code is Line Feed (LF).
You don't specify whether you are demanding that the update to the screen is immediate or deferred until the next flush. Therefore:
if you're using iostream io:
cout.put('\n');
if you're using stdio io:
std::putchar('\n');
The answer to this question is really "it depends".
In isolation - if all you're measuring is the performance of writing a '\n' character to the standard output device, not tweaking the device, not changing what buffering occurs - then it will be hard to beat options like
putchar('\n');
fputchar('\n', stdout);
std::cout.put('\n');
The problem is that this doesn't achieve much - all it does (assuming the output is to a screen or visible application window) is move the cursor down the screen, and move previous output up. Not exactly a entertaining or otherwise valuable experience for a user of your program. So you won't do this in isolation.
But what comes into play to affect performance (however you measure that) if we don't output newlines in isolation? Let's see;
Output of stdout (or std::cout) is buffered by default. For the output to be visible, options include turning off buffering or for the code to periodically flush the buffer. It is also possible to use stderr (or std::cerr) since that is not buffered by default - assuming stderr is also directed to the console, and output to it has the same performance characteristics as stdout.
stdout and std::cout are formally synchronised by default (e.g. look up std::ios_base::sync_with_stdio) to allow mixing of output to stdout and std::cout (same goes for stderr and std::cerr)
If your code outputs more than a set of newline characters, there is the processing (accessing or reading data that the output is based on, by whatever means) to produce those other outputs, the handling of those by output functions, etc.
There are different measures of performance, and therefore different means of improving efficiency based on each one. For example, there might be CPU cycles, total time for output to appear on the console, memory usage, etc etc
The console might be a physical screen, it might be a window created by the application (e.g. hosted in X, windows). Performance will be affected by choice of hardware, implementation of windowing/GUI subsystems, the operating system, etc etc.
The above is just a selection, but there are numerous factors that determine what might be considered more or less performance.
On Ubuntu 15.10, g++ v5.2.1 (and an older vxWorks, and OSE)
It is easy to demonstrate that
std::cout << std::endl;
puts a new line char into the output buffer, and then flushes the buffer to the device.
But
std::cout << "\n";
puts a new line char into the output buffer, and does not output to the device. Some future action will be needed to trigger the output of the newline char in the buffer to the device.
Two such actions are:
std::cout << std::flush; // will output the buffer'd new line char
std::cout << std::endl; // will output 2 new line chars
There are also several other actions that can trigger the flush of the std::cout buffering.
#include <unistd.h> // for Linux
void msDelay (int ms) { usleep(ms * 1000); }
int main(int, char**)
{
std::cout << "with endl and no delay " << std::endl;
std::cout << "with newline and 3 sec delay " << std::flush << "\n";
msDelay(3000);
std::cout << std::endl << " 2 newlines";
return(0);
}
And, per comment by someone who knows (sorry, I don't know how to copy his name here), there are exceptions for some environments.
It's actually OS/Compiler implementation dependent.
The most efficient, least side effect guaranteed way to output a '\n' newline character is to use std::ostream::write() (and for some systems requires std::ostream was opened in std::ios_base::binary mode):
static const char newline = '\n';
std::cout.write(&newline,sizeof(newline));
I would suggest to use:
std::cout << '\n'; /* Use std::ios_base::sync_with_stdio(false) if applicable */
or
fputc('\n', stdout);
And turn the optimization on and let the compiler decide what is best way to do this trivial job.
Well if you want to change the line I'd like to add the simplest and the most common way which is using (endl), which has the added perk of flushing the stream, unlike cout << '\n'; on its own.
Example:
cout << "So i want a new line" << endl;
cout << "Here is your new line";
Output:
So i want a new line
Here is your new line
This can be done for as much new lines you want. Allow me to show an example using 2 new lines, it'll definitely clear all of your doubts,
Example:
cout << "This is the first line" << endl;
cout << "This is the second line" << endl;
cout << "This is the third line";
Output:
This is the first line
This is the second line
This is the third line
The last line will just have a semicolon to close since no newline is needed. (endl) is also chain-able if needed, as an example, cout << endl << endl; would be a valid sequence.
Currently I only know of two methods to insert values into a C++ string or C string.
The first method I know of is to use std::sprintf() and a C-string buffer (char array).
The second method is to use something like "value of i: " + to_string(value) + "\n".
However, the first one needs the creation of a buffer, which leads to more code if you just want to pass a string to a function. The second one produces long lines of code, where a string gets interrupted every time a value is inserted, which makes the code harder to read.
From Python I know the format() function, which is used like this:
"Value of i: {}\n".format(i)
The braces are replaced by the value in format, and further .format()'s can be appended.
I really like Python's approach on this, because the string stays readable, and no extra buffer needs to be created. Is there any similar way of doing this in C++?
Idiomatic way of formatting data in C++ is with output streams (std::ostream reference). If you want the formatted output to end up in a std::string, use an output string stream:
ostringstream res;
res << "Value of i: " << i << "\n";
Use str() member function to harvest the resultant string:
std::string s = res.str();
This matches the approach of formatting data for output:
cout << "Value of i: " << i << "\n";
We're seeing a strange scenario that basically boils down to the following:
std::string something = "someval";
std::stringstream s;
s << something;
std::cout << s.str();
is not equal to:
std::string something = "someval";
std::stringstream s;
s << something.c_str();
std::cout << s.str();
Taking that a step farther - the output is not gibberish in either case. What is happening is the output from case 1 appears to be mapped to another (valid) string in the system whereas the output from case 2 is what is expected.
We see this behavior by simply changing:
s << something;
To:
s << something.c_str();
I know this sounds crazy (or it does to me), and I haven't been able to replicate it out of the larger system - so sorry for no "working" example. But does anyone know how this kind of thing can happen? Can we be stepping on memory somewhere or doing something to a stringtable in some location or anything else like that?
It is different if the string contains nul characters, '\0'.
The .c_str() version will compute the length up to the nul, while the std::string output will know its length and output all its characters.