Overload handling of std::endl? - c++

I want to define a class MyStream so that:
MyStream myStream;
myStream << 1 << 2 << 3 << std::endl << 5 << 6 << std::endl << 7 << 8 << std::endl;
gives output
[blah]123
[blah]56
[blah]78
Basically, I want a "[blah]" inserted at the front, then inserted after every non terminating std::endl?
The difficulty here is NOT the logic management, but detecting and overloading the handling of std::endl. Is there an elegant way to do this?
Thanks!
EDIT: I don't need advice on logic management. I need to know how to detect/overload printing of std::endl.

What you need to do is write your own stream buffer: When the stream buffer is flushed you output you prefix characters and the content of the stream.
The following works because std::endl causes the following.
Add '\n' to the stream.
Calls flush() on the stream
This calls pubsync() on the stream buffer.
This calls the virtual method sync()
Override this virtual method to do the work you want.
#include <iostream>
#include <sstream>
class MyStream: public std::ostream
{
// Write a stream buffer that prefixes each line with Plop
class MyStreamBuf: public std::stringbuf
{
std::ostream& output;
public:
MyStreamBuf(std::ostream& str)
:output(str)
{}
~MyStreamBuf() {
if (pbase() != pptr()) {
putOutput();
}
}
// When we sync the stream with the output.
// 1) Output Plop then the buffer
// 2) Reset the buffer
// 3) flush the actual output stream we are using.
virtual int sync() {
putOutput();
return 0;
}
void putOutput() {
// Called by destructor.
// destructor can not call virtual methods.
output << "[blah]" << str();
str("");
output.flush();
}
};
// My Stream just uses a version of my special buffer
MyStreamBuf buffer;
public:
MyStream(std::ostream& str)
:std::ostream(&buffer)
,buffer(str)
{
}
};
int main()
{
MyStream myStream(std::cout);
myStream << 1 << 2 << 3 << std::endl << 5 << 6 << std::endl << 7 << 8 << std::endl;
}
> ./a.out
[blah]123
[blah]56
[blah]78
>

Your overloaded operators of the MyStream class have to set a previous-printed-token-was-endl flag.
Then, if the next object is printed, the [blah] can be inserted in front of it.
std::endl is a function taking and returning a reference to std::ostream. To detect it was shifted into your stream, you have to overload the operator<< between your type and such a function:
MyStream& operator<<( std::ostream&(*f)(std::ostream&) )
{
std::cout << f;
if( f == std::endl )
{
_lastTokenWasEndl = true;
}
return *this;
}

Agreed with Neil on principle.
You want to change the behavior of the buffer, because that is the only way to extend iostreams. endl does this:
flush(__os.put(__os.widen('\n')));
widen returns a single character, so you can't put your string in there. put calls putc which is not a virtual function and only occasionally hooks to overflow. You can intercept at flush, which calls the buffer's sync. You would need to intercept and change all newline characters as they are overflowed or manually synced and convert them to your string.
Designing an override buffer class is troublesome because basic_streambuf expects direct access to its buffer memory. This prevents you from easily passing I/O requests to a preexisting basic_streambuf. You need to go out on a limb and suppose you know the stream buffer class, and derive from it. (cin and cout are not guaranteed to use basic_filebuf, far as I can tell.) Then, just add virtual overflow and sync. (See ยง27.5.2.4.5/3 and 27.5.2.4.2/7.) Performing the substitution may require additional space so be careful to allocate that ahead of time.
- OR -
Just declare a new endl in your own namespace, or better, a manipulator which isn't called endl at all!

I use function pointers. It sounds terrifying to people who aren't used to C, but it's a lot more efficient in most cases. Here's an example:
#include <iostream>
class Foo
{
public:
Foo& operator<<(const char* str) { std::cout << str; return *this; }
// If your compiler allows it, you can omit the "fun" from *fun below. It'll make it an anonymous parameter, though...
Foo& operator<<(std::ostream& (*fun)(std::ostream&)) { std::cout << std::endl; }
} foo;
int main(int argc,char **argv)
{
foo << "This is a test!" << std::endl;
return 0;
}
If you really want to you can check for the address of endl to confirm that you aren't getting some OTHER void/void function, but I don't think it's worth it in most cases. I hope that helps.

Instead of attempting to modify the behavior of std::endl, you should probably create a filtering streambuf to do the job. James Kanze has an example showing how to insert a timestamp at the beginning of each output line. It should require only minor modification to change that to whatever prefix you want on each line.

I had the same question, and I thought that Potatoswatter's second answer had merit: "Just declare a new endl in your own namespace, or better, a manipulator which isn't called endl at all!"
So I found out how to write a custom manipulator which is not hard at all:
#include <sstream>
#include <iostream>
class log_t : public std::ostringstream
{
public:
};
std::ostream& custom_endl(std::ostream& out)
{
log_t *log = dynamic_cast<log_t*>(&out);
if (log)
{
std::cout << "custom endl succeeded.\n";
}
out << std::endl;
return out;
}
std::ostream& custom_flush(std::ostream& out)
{
log_t *log = dynamic_cast<log_t*>(&out);
if (log)
{
std::cout << "custom flush succeeded.\n";
}
out << std::flush;
return out;
}
int main(int argc, char **argv)
{
log_t log;
log << "custom endl test" << custom_endl;
log << "custom flush test" << custom_flush;
std::cout << "Contents of log:\n" << log.str() << std::endl;
}
Here's the output:
custom endl succeeded.
custom flush succeeded.
Contents of log:
custom endl test
custom flush test
Here I've created two custom manipulators, one that handles endl and one that handles flush. You can add whatever processing you want to these two functions, since you have a pointer to the log_t object.

You can't change std::endl - as it's name suggests it is a part of the C++ Standard Library and its behaviour is fixed. You need to change the behaviour of the stream itself, when it receives an end of line . Personally, I would not have thought this worth the effort, but if you want to venture into this area I strongly recommend reading the book Standard C++ IOStreams & Locales.

Related

Tell `endl` not to flush

My program prints a large number of short lines to cout.
As a slightly contrived example, my lines look a little like this:
cout<<"The variable's value is: "<<variable<<endl;
I'd like the program to run fast and I do believe that endl is killing me because it initiates a buffer flush on cout every time it is used.
Now, some folks on the internet have said that I could do this instead:
cout<<"The variable's value is: "<<variable<<"\n";
But this does not seem like a good solution because endl abstracts the particular system-specific ways an end line might be specified, where as \n does not. This also seems like a poor solution because, should I need buffering in the future, I would then have to modify the whole code base.
Therefore, I ask, is there a way to disable the buffer-flushing aspect of endl?
EDIT
Further digging seems to indicate that both endl and \n respect the various ways an OS might choose to end it's lines. It also seems that the output stream detects if it's in a potentially interactive situation and buffers and flushes accordingly. Therefore: the problem may be solved by manually telling the output stream to perform aggressive buffering... if I can figure out how to do that.
endl abstracts the particular system-specific ways an end line might be specified, where as \n does not".
std::endl is defined to output '\n' followed by a flush. The correct abstraction of the system-specific newline thingy is just '\n'.
To prevent flushes, one just doesn't use std::endl. In addition, the standard output may be line-buffered if it is or may be connected to an interactive device, in this case the newline character will flush the stream. If that's an issue, use an ofstream connected to a named file. I think on Unix-like systems line buffering only happens when the standard output is a terminal.
endl flushes. If you don't want that behaviour, don't use endl. If you want to change your code easily, use your own manipulator:
inline std::ostream& myendl( std::ostream& os ){
os.put(os.widen('\n'));
return os;
}
That way you can easily change the behaviour of your myendl at one place.
According to http://en.cppreference.com/w/cpp/io/manip/endl
endl:: Inserts a endline character into the output sequence os and flushes it as if by calling os.put(os.widen('\n')) followed by os.flush().
So it appears you want to just write os.put(os.widen('\n')), which should, from this definition be safe and portable and correct, as well as meeting your primary needs.
There is std::nounitbuf which is documented to have some effect in this matter.
However, I didn't notice any difference. To bypass all of the ostream's ideas of when or when not to flush I tried this:
std::ostringstream oss;
// std::cout << std::nounitbuf;
for( int i = 0; i < 1000000; i++ ){
// std::cout << "Test " << "file" << '\n';
oss << "Test " << "file" << '\n';
}
std::cout << oss.str();
This improved execution time from ~33 sec to ~25csec.
IF your output goes to an xterm, your execution speed is severly limited by xterm's work to do scrolling etc. If you use a pipeline to filter out unnecessary lines you'll see a dramatic increase in speed, e.g.
./program | grep -v "The variable"
If flushing is the problem, you can implement a stream buffer that overrides the sync() member function to only flush to the external device if you specify so. It also obligates creating your own manipulators flush_on_endl and noflush_on_endl if you intend to change these preferences throughout the program.
#include <iostream>
static int disable() {
static int index(std::ios_base::xalloc());
return index;
}
class save_buffer
{
public:
save_buffer(std::ios& other)
: str(other), buf(other.rdbuf())
{ }
~save_buffer() { str.rdbuf(buf); }
private:
std::ios& str;
std::streambuf* buf;
};
class syncing_buffer_optional : public std::streambuf, private save_buffer
{
public:
syncing_buffer_optional(std::ostream& other)
: save_buffer(other),
buf(other.rdbuf()),
disable_sync(other.iword(disable()))
{ }
std::streambuf::int_type overflow(std::streambuf::int_type c)
{
buf->sputc(c);
return 0;
}
int sync()
{
return disable_sync? 0: buf->pubsync();
}
private:
std::streambuf* buf;
bool disable_sync;
};
std::ostream& flush_on_endl(std::ostream& os)
{
os.iword(disable()) = false;
return os;
}
std::ostream& noflush_on_endl(std::ostream& os)
{
os.iword(disable()) = true;
return os;
}
std::ostream& endl(std::ostream& os)
{
syncing_buffer_optional eb(os);
os.rdbuf(&eb);
return os << std::endl;
}
int main()
{
std::cout << noflush_on_endl << endl;
}

Trying to overload cout << operator but its not working

I snipped the irrelevant parts out of my class here. I don't know what I'm doing wrong, just trying to be able to cout << the object.
#include <iostream>
class Snipped
{
public:
friend std::ostream& operator<<(std::ostream& os, const Snipped& s);
protected:
private:
};
std::ostream& operator<<(std::ostream& os, const Snipped& s)
{
os << "test";
return os;
}
int main(int argc, char* argv[])
{
Snipped* s = new Snipped();
std::cout << s << std::endl << s;
delete s;
return 0;
}
Expected output:
test
test
Actual output:
0x12ae20
0x12ae20 (random memory location?)
std::cout << s << std::endl << s;
You are calling << with an address, you need to call it with object of type Snipped.
The line of code above won't call your overloaded operator function because the parameters of the overloaded function do not match.
You need to call:
std::cout << *s << std::endl << *s;
This ensures your << overloaded operator function is called because the parameters match to it.
Try
std::cout << *s << std::endl;
By the way,
std::cout << s << std::endl;
is not really a random memory location.
It is the actual memory address on the heap, in this case.
You can actually use that address to check the identity of an object.
That's useful while debugging, or in actual code.
For example, if you look at assignment operators, you will often see:
class Foo
{
Foo& operator=( const Foo& foo )
{
// use the identity principle
if ( &foo==this )
return *this; // so I don't waste CPU cycles copying to myself
// ...really do copy here
return *this;
}
};
Although just dereferencing the pointer (i.e. to use '*s' instead of 's') there is a bigger fish to fry! Unless there is a good reason for putting an object on the heap you shouldn't do so:
int main()
{
Snipped s;
std::cout << s << '\n' << s;
}
I found the use of new and more so delete quite rare in the programs I write. Aside from simpler code this conveniently also often yields faster programs. If you really need to allocate something on the heap, use some sort of a smart pointer to make sure the object is automatically released:
int main()
{
std::unique_ptr<Snipped> s(new Snipped);
std::cout << *s << '\n' << *s;
}
As a side note, don't use std::endl either unless you really intend to flush the stream: I found inappropriate use of std::endl to be the root cause of massive performance problems more than once. Sure most of the time it doesn't matter but in even more cases you don't care about the flush. If you don't like using '\n' or "\n" you can instead use a custom manipulator:
std::ostream& nl(std::ostream& out) { return out << '\n'; }
With this you can use nl instead of std::endl and don't suffer from always flushing the stream.

is it possible to pass iostream into a function?

What I mean:
Everyone knows this method of redirecting stream to output:
cout << "sometext"
but is it possible to pass that stream to a function like this:
my_function() << "sometext";
Yes*:
#include <iostream>
#include <ostream>
std::ostream & my_function() { return std::cout; }
// ...
my_function() << "Hello world.\n";
*) Nothing you said in words is entirely correct, and you may well struggle later integrate this into your project, but this answer shows how to make your code do what you want.
Everyone knows this method of redirecting stream to output:
That's not what that does. The stream is called cout; that's the iostream object. The << operator does not redirect anything. The std::ostream objects all have overloaded operator<< functions. Those functions are invoked when you use << with a stream on the left-hand side and some type that has an overload for it on the right.
<< "sometext" is not a "stream" that can be "redirected". It isn't even a valid expression in C++. The << operator is binary. it takes two parameters.
my_function() << "sometext"; can only work if it returns a std::ostream class or something derived from it. Or something that has an overloaded operator<< defined for it and const char*.
cout << "sometext"
This is not "redirecting stream to output" it is invoking the operator << function on the cout object with the string literal "sometext"
if my_function() is returning a ostream which has operator << overloaded then my_function() << "sometext" will compile else it will give an error.
If you are looking for a way to overload << for your own function unrelated to streams, here is how you can do it:
struct MyStruct {
void DoSomething(const string& s);
};
MyStruct &operator<<(MyStruct &x, const string& s) {
x.DoSomething(s);
return x;
}
MyStruct& my_function() {
return MyStruct;
}
int main() {
my_function() << "Hello, world!";
}
In this example, DoSomething will be called on the instance of MyStruct returned from my_function, and "Hello, world!" will be passed to it as an argument.
If I understand what you are asking about, the closest equivalent of a shell redirect for a function that uses std::cout for output is probably to switch temporarily std::cout's internal stream buffer for a different one.
Of course, this is inherently not thread safe and won't cope if the function itself expects std::cout and stdout to be the same underlying thing.
#include <iostream>
#include <sstream>
int main()
{
std::stringbuf redir( std::ios_base::out );
std::streambuf* save = std::cout.rdbuf( &redir );
my_function(); // cout output ends up in redir
std::cout.rdbuf( save ); // restore original cout
}

std::stringstream as parameter

I'm somewhat new to the C++ language. I'm writing a utility class for logging to file. It works beautifully except that now I would like to enhance it by making it more convenient to use (e.g. pass stringstreams to a log function).
This is what I've been trying and it hasn't worked.
definition:
void LogStream( std::stringstream i_Log ){
m_FileHandle << i_Log << std::endl;
}
call:
m_LogObject->LogStream( "MKLBSearchEngine::Search( " << x << ", " << i_Filter << " ) - No Results Found" );
There are several problems with your solution. The first is that you're
passing stringstream by value, and it doesn't support copy. You need
by reference. The second is that at the call site, the return value of
the operator<< overloads is ostream&, not stringstream, and since
stringstream isn't a base class of ostream& (it's the other way
round), you can't initialize the stringstream (or the stringstream&)
with it. And finally, there's no operator<< which takes a
stringstream as the right hand parameter, so the statement in the
LogStream function can't work. Finally, this is going to be somewhat
awkward for the user anyway. A log of operator<< are non-members,
with an ostream& non-const reference as first argument, so you can't
call them with a temporary as the left argument. (In your example call,
of course, you forgot to create the std::ostringstream anyway; it
won't compiler because there is no overload of << which takes a char
const[] or a char const* as its left hand operand.)
There are work-arounds for almost all of these problems. Something
like:
void LogStream( std::ostream& text )
{
std::ostringstream& s = dynamic_cast<std::ostringstream&>(text);
m_FileHandle << s.str() << std::endl;
}
handles all of the problems except the last; the last has to be handled
by the client, something like:
m_LogObject->LogStream( std::ostringstream().flush() << "..." << x );
(The call to std::ostream::flush() returns a non-const reference to
the stream, which can be used to initialize further std::ostream&.
And while you can't initialize a non-const reference with a temporary,
you can call a non-const member function on it.)
The awkwardness of this for the client code makes me generally prefer a
more complex solution. I define a special LogStreamer class,
something like:
class LogStreamer
{
boost::shared_ptr< std::ostream > m_collector;
std::ostream* m_dest;
public:
LogStreamer( std::ostream& dest )
, m_collector( new std::ostringstream )
, m_dest( &dest )
{
}
~LogStreamer()
{
if ( m_collector.unique() ) {
*m_dest << m_collector->str() << std::endl;
}
}
template <typename T>
LogStreamer& operator<<( T const& value )
{
*m_collector << value;
return *this;
}
};
and
LogStreamer LogStream() { return LogStreamer( m_FileHandle ); }
The client code can then write:
m_LogObject->LogStream() << "..." << x;
In my own code: the log object is always a singleton, the call is
through a macro, which passes __FILE__ and __LINE__ to the LogStream()
function, and the final target ostream is a special streambuf with a
special function, called by LogStream(), which takes a filename and a
line number, outputs them, along with the time stamp, at the start of
the next line output, and indents all other lines. A filtering
streambuf with something like:
class LogFilter : public std::streambuf
{
std::streambuf* m_finalDest;
std::string m_currentHeader;
bool m_isAtStartOfLine;
protected:
virtual int overflow( int ch )
{
if ( m_isAtStartOfLine ) {
m_finalDest->sputn( m_currentHeader.data(), m_currentHeader.size() );
m_currentHeader = " ";
}
m_isAtStartOfLine = (ch == '\n');
return m_finalDest->sputc( ch );
}
virtual int sync()
{
return m_finalDest->sync();
}
public:
LogFilter( std::streambuf* dest )
: m_finalDest( dest )
, m_currentHeader( "" )
, m_isAtStartOfLine( true )
{
}
void startEntry( char const* filename, int lineNumber )
{
std::ostringstream header;
header << now() << ": " << filename << " (" << lineNumber << "): ";
m_currentHeader = header.str();
}
};
(The function now(), of course, returns a std::string with the
timestamp. Or a struct tm, and you've written a << for tm.)
You have a problem with your design. You don't want to accept a stream as a parameter, either accept a string, or make your class behave as a stream (or both).
If you make your object behave as a stream, then you do the following:
m_LogObject << "what to log" << etc;
To do that, simply override the << operator.
Your call should look like
m_LogObject->LogStream( stringstream() << "MKLBSearchEngine::Search( " << x
<< ", " << i_Filter << " ) - No Results Found" );
since you need to create your stringstream object that you will be passing to the function.
This call implies that you already have a desired output stream so i'd also recommend you changing your class design to use operator<< for logging unless it is already overloaded.
Your function call won't work, as "MKLBSearchEngine::Search( " is of type const char* and that has no overload for the << operator. It won't work with std::string("MKLBSearchEngine::Search( ") either, as also std::string doesn't have such an operator. What you can do is call it with std::stringstream("MKLBSearchEngine::Search( "), which converts the first argument to a stream, such that the following operators work on this stream. But as others pointed out, you will have to make the function argument a const reference, as streams are not copyable (even then it would be quite inefficient). Also just writing a std::stringstream into the file won't do what you want (if it works anyway), instead you have to take its contents (the underlying std::string). So all in all your code should look like:
void LogStream( const std::stringstream &i_Log ){ m_FileHandle << i_Log.str() << std::endl; }
...
m_LogObject->LogStream( std::stringstream("MKLBSearchEngine::Search( ") << x << ", " << i_Filter << " ) - No Results Found" );
But you could also just use a LogString(const std::string &) and let the user of this function call stream.str() himself.
You can't pass stream objects by value (since they are not copyable), so you need to pass by (and store) references:
void LogStream(std::stringstream& i_Log){
m_FileHandle << i_Log << std::endl;
}
This probably won't do what you're expecting though (it will probably print an address of i_Log, for rather obscure reasons).
If your intention is to take stuff OUT of the stringstream, this might do what you want:
i_Log.get( *m_FileHandle.rdbuf() );
You are passing std::stringstream instance by value. You want to avoid copying and pass it by reference (or pointer). For example:
void LogStream ( std::stringstream & i_Log ){ m_FileHandle << i_Log << std::endl; }
Read more about C++ references.

How does QDebug() << stuff; add a newline automatically?

I'm trying to implement my own qDebug() style debug-output stream, this is basically what I have so far:
struct debug
{
#if defined(DEBUG)
template<typename T>
std::ostream& operator<<(T const& a) const
{
std::cout << a;
return std::cout;
}
#else
template<typename T>
debug const& operator<<(T const&) const
{
return *this;
}
/* must handle manipulators (endl) separately:
* manipulators are functions that take a stream& as argument and return a
* stream&
*/
debug const& operator<<(std::ostream& (*manip)(std::ostream&)) const
{
// do nothing with the manipulator
return *this;
}
#endif
};
Typical usage:
debug() << "stuff" << "more stuff" << std::endl;
But I'd like not to have to add std::endl;
My question is basically, how can I tell when the return type of operator<< isn't going to be used by another operator<< (and so append endl)?
The only way I can think of to achieve anything like this would be to create a list of things to print with associated with each temporary object created by qDebug(), then to print everything, along with trailing newline (and I could do clever things like inserting spaces) in ~debug(), but obviously this is not ideal since I don't have a guarantee that the temporary object is going to be destroyed until the end of the scope (or do I?).
Something like this will do:
struct debug {
debug() {
}
~debug() {
std::cerr << m_SS.str() << std::endl;
}
public:
// accepts just about anything
template<class T>
debug &operator<<(const T &x) {
m_SS << x;
return *this;
}
private:
std::ostringstream m_SS;
};
Which should let you do things like this:
debug() << "hello world";
I've used a pattern like this combined with a lock to provide a stream like logging system which can guarantee that log entries are written atomically.
NOTE: untested code, but should work :-)
Qt uses a method similar to #Evan. See a version of qdebug.h for the implementation details, but they stream everything to an underlying text stream, and then flush the stream and an end-line on destruction of the temporary QDebug object returned by qDebug().
When you write that this is the typical usage:
debug() << "stuff" << "more stuff" << std::endl;
are you definitely planning to construct a debug object each time you use it? If so, you should be able to get the behavior you want by having the debug destructor add the newline:
~debug()
{
*this << std::endl;
... the rest of your destructor ...
}
That does mean you cannot do something like this:
// this won't output "line1" and "line2" on separate lines
debug d;
d << "line1";
d << "line2";
The stream insertion (<<) and extraction (>>) are supposed to be non-members.
My question is basically, how can I
tell when the return type of
operator<< isn't going to be used by
another operator<< (and so append
endl)?
You cannot. Create a member function to specially append this or append an endl once those chained calls are done with. Document your class well so that the clients know how to use it. That's your best bet.