Custom flush implementation - c++

I'm trying to follow the logic of this question to create a custom streambuf in Rcpp. Someone contributed the basic behaviour that allows us to write things like
Rcout << "some text" ;
where we implemented xsputn and overflow to redirect to Rprintf function.
std::streamsize Rcpp::Rstreambuf::xsputn(const char *s, std::streamsize num ) {
Rprintf( "%.*s", num, s );
return num;
}
int Rcpp::Rstreambuf::overflow(int c ) {
if (c != EOF) {
Rprintf( "%.1s", &c );
}
return c;
}
I would like to implement flushing too, i.e. support this syntax:
Rcout << "some text" << std::flush ;
Which method do I need to implement so that the flush manipulator works on my custom stream ?

It is sync() function (like in filebuf):
protected:
virtual int sync()
Base version of base_streambuf<>::sync() does nothing, one must overwrite it to make some synchronization with underlying stream.

Related

Limit filesize of a filestream?

My application currently logs in a very simple way:
void Log::create( const std::string& path, bool append )
{
if(append)
m_log.open(path.c_str(),std::ios_base::out | std::ios_base::app);
else
m_log.open(path.c_str(),std::ios_base::out | std::ios_base::trunc);
}
std::ofstream& Log::get()
{
return m_log;
}
void Log::write( const std::string& what )
{
get() << "[" << TimeOfDay::getDate() << "] ";
get() << what << std::endl;
}
void Log::write( const std::string& where, const std::string& what )
{
get() << "[" << TimeOfDay::getDate() << "] ";
get() << "[" << where << "] " << what << std::endl;
}
std::ofstream& Log::write()
{
get() << "[" << TimeOfDay::getDate() << "] ";
return get();
}
std::ofstream Log::m_log;
This application runs on a server. Now, if the log exceeds a certain file size, I want to stop logging.
Is there a way to do this without boost or other libraries?
Thanks
You can create a filtering stream buffer which is set up to write a file but stops writing if it has written more than a specified amount of data. Something like this:
class limitbuf
: public std::streambuf {
std::streambuf* sbuf;
size_t size;
size_t limit;
char buffer[1024];
public:
limitbuf(std::streambuf* sbuf, size_t limit)
: sbuf(sbuf), limit(limit), size(0)
{
this->setp(buffer, buffer + 1023);
}
int overflow(int c) {
if (c != std::char_traits<char>::eof()) {
this->pptr() = std::char_traits<char>::to_char_type(c);
this->pbump(1);
}
return this->sync() == 0
? std::char_traits<char>::not_eof(c)
: std::char_traits<char>::eof();
}
int sync() {
if (this->size < limit) {
this->size += this->sbuf->sputn(this->pbase(),
std::min(
size_t(this->pptr() - this->pbase()),
this->limit - this->size)
);
this->sbuf->pubsync();
}
this->setp(this->pbase(), this->epptr());
return 0;
}
};
Just install this stream buffer as a filter to your log file and it should be limiting to some suitable size:
std::ofstream out("some.log", 16384);
limitbuf sbuf(out.rdbuf());
std::ostream log(&sbuf);
The basic idea of this stream buffer is fairly simple: data is buffer internally and written upon a buffer overflow or upon a flush:
When the buffer set up with setp() overflows the stream calls overflow(c) with the next character to be written (or, potentially, with std::char_traits<char>::eof()). Since te stream buffer was told about a buffer one character smaller than actually available, the overflowing character is added to the buffer and the overall buffer is flushed.
When the buffer is flushed (e.g. by using std::endl on the std::ostream writing to this buffer) the function sync() ends up being called. Its just is to write the characters currently buffered. The code simply sees if there is still space for anything to be written and writes the character if there is space available. The size member maintains how many characters are written and limit is set up to indicate how much data is to be written.
If the stream buffer should do more than just limit the output, it may be necessary to modify the logic of what is happening if there is no more space. For example, if there are left-over characters which can't be written the stream buffer could decide to open a new file (and possibly move other files).

Using various format specifiers of c in c++

As in c we can use various format specifiers like
%nd where n is a number, to print the number with a total of atleast n space covered
%0nd same as above, except pre-padding with 0's " %05d ",3 => 00003
%.nf to set precision of n after decimal
etc ....
So is there any way to use these with std::cout ?
I got some negative feedback in a recent course (c++ for c programmers) in coursera, for using printf instead of cout because i wanted to some formatting :(
For %nd %0nd, C++ equivalents are std::setw() and std::setfill().
#include <iostream> // std::cout, std::endl
#include <iomanip> // std::setfill, std::setw
int main () {
std::cout << std::setfill ('x') << std::setw (10);
std::cout << 77 << std::endl;
return 0;
}
Output: xxxxxxxx77
%.nf can be replaced by std::setprecision and std::fixed,
#include <iostream> // std::cout, std::fixed, std::scientific
int main () {
double a = 3.1415926534;
double b = 2006.0;
double c = 1.0e-10;
std::cout.precision(5);
std::cout << "fixed:\n" << std::fixed;
std::cout << a << '\n' << b << '\n' << c << '\n';
return 0;
}
Output:
fixed:
3.14159
2006.00000
0.00000
C++ streams don't use format-specifiers like C's printf()-type functions; they use manipulators.
For example:
#include <iostream>
#include <iomanip>
int main()
{
std::cout << std::fixed << std::setprecision(6) << 42.0 << std::endl;
}
Output:
42.000000
See it run!
The usual solution in C++ is to defined manipulators which state
what your trying to format, rather than to hack physical values
directly at the point of output. (One possible exception is the
width, where std::setw may be useful directly.) Thus, for
example, when actually outputting something, you won't specify
zero padding, or fixed, with 2 decimals, but something like:
std::cout << temperature(2) << theTemporature;
where temperature would be something like:
class temperature
{
int myMinWidth;
public:
temperature( int minWidth )
: myMinWidth( minWidth )
{
}
friend std::ostream& operator<<( std::ostream& dest, temperature const& manip )
{
dest.setf( std::ios_base::fixed, std::ios_base::floatfield );
dest.precision( 2 );
dest.width( myMinWidth );
return dest;
}
};
For the list of format modifications available, see the
specification of std::ios_base, and the fields of
std::ios_base::fmtflags.
If you're doing a lot of output, you might want to modify this
to restore the original format flags at the end of the full
expression. (All of the format information except the width is
sticky, so forcing fixed format here leaves you with fixed
format for the rest of the program, which isn't necessarily what
you want.) I use the following as base class for all of my
manipulators:
class StateSavingManip
{
public:
void operator()( std::ios& stream ) const;
protected:
StateSavingManip() : myStream( nullptr ) {}
~StateSavingManip();
private:
virtual void setState( std::ios& stream ) const = 0;
private:
mutable std::ios* myStream;
mutable std::ios::fmtflags mySavedFlags;
mutable int mySavedPrec;
mutable char mySavedFill;
};
implementation:
namespace {
int getXAlloc() ;
int ourXAlloc = getXAlloc() + 1 ;
int
getXAlloc()
{
if ( ourXAlloc == 0 ) {
ourXAlloc = std::ios::xalloc() + 1 ;
assert( ourXAlloc != 0 ) ;
}
return ourXAlloc - 1 ;
}
}
StateSavingManip::~StateSavingManip()
{
if ( myStream != nullptr ) {
myStream->flags( mySavedFlags ) ;
myStream->precision( mySavedPrec ) ;
myStream->fill( mySavedFill ) ;
myStream->pword( getXAlloc() ) = NULL ;
}
}
void
StateSavingManip::operator()(
std::ios& stream ) const
{
void*& backptr = stream.pword( getXAlloc() ) ;
if ( backptr == nullptr ) {
backptr = const_cast< StateSavingManip* >( this ) ;
myStream = &stream ;
mySavedFlags = stream.flags() ;
mySavedPrec = stream.precision() ;
mySavedFill = stream.fill() ;
}
setState( stream ) ;
}
Note the use of the pword field to ensure that only the first
temporary manipulator restores the format; the destructors will
be called in the reverse order of construction, but the order of
construction will typically not be specified if you have more
than one such manipulator in an expression.
Finally: not everything is possible using this technique: if you
want to systematically append a degree sign to the temperature,
there's no way of doing so. In this case, you need to define
a class Temperature, and overload the << operator for it; this
allows everything imaginable (much more than you could ever
achieve with printf style formatting).
C++ stream manipulators (iomanip) were specifically designed to support all of the standard c format specifiers operations, just with an entirely different interface. E.g. setfill and setw for the width and fill part of %02d.
Of course if you really need format strings (e.g. because it makes i18n easier for you), then you should have a look at boost::format, and if you have C++11 then you can easily write small variadic template wrapper around it, to make the format call look more like printf.
Whatever you do, please try to not use printf. It is not typesafe, and not extendible for output operations on user defined types.
There are stream manipulators, if you need them. But I think you want to know this thing: cout is smarter than printf().
Say you have:
int x = 34;
cout<<x;
Now the compiler will call:
ostream& operator<<(ostream& stream, int arg);
for you. And this function will print the output in console( since in your case stream is cout). The standard library supplies overloads for this << operator for all fundamental datatype available.

how to get char pointer from ostringstream without a copy in c++

I have a ostringstream variable which contains some data.
I want to get set a char * pointer to the data inside the ostringstream.
If I do the following:
std::ostringstream ofs;
.....
const char *stam = (ofs.str()).c_str();
There is a copy of the content of the string in ofs.
I want to get a pointer to that content without a copy.
Is there a way to do so?
This actually answers the question... took a while but I wanted to do it for the same reasons (efficiency vs portability is fine for my situation):
class mybuf : public std::stringbuf {
public:
// expose the terribly named end/begin pointers
char *eback() {
return std::streambuf::eback();
}
char *pptr() {
return std::streambuf::pptr();
}
};
class myos : public std::ostringstream {
mybuf d_buf;
public:
myos() {
// replace buffer
std::basic_ostream<char>::rdbuf(&d_buf);
}
char *ptr();
};
char *myos::ptr() {
// assert contiguous
assert ( tellp() == (d_buf.pptr()-d_buf.eback()) );
return d_buf.eback();
}
int main() {
myos os;
os << "hello";
std::cout << "size: " << os.tellp() << std::endl;
std::string dat(os.ptr(),os.tellp());
std::cout << "data: " << dat << std::endl;
}
This points to, yet again, the deeper, underlying problem with the standard library - a confusion between contracts and "safety". When writing a messaging service, I need a library with efficient contracts... not safety. Other times, when writing a UI, I want strong safety - and cares less about efficiency.
Although you can't get a pointer to the character buffer in the ostringstream, you can get access to its characters without copying them if you switch to using stringstream. A stringstream allows input and output (reading from and writing to the stream), whereas ostringstream allows only output (writing to the stream). Example:
std::stringstream ss;
ss << "This is a test.";
// Read stringstream from index 0. Use different values to look at any character index.
ss.seekg(0);
char ch;
while (ss.get(ch)) { // loop getting single characters
std::cout << ch;
}
ss.clear(); // Clear eof bit in case you want to read more from ss
This site has a pretty good overview of stringstreams and what you can do with them.

c++ custom output stream with indentation

I'm having some trouble trying to implement a custom stream class to generate nicely indented code in an output file. I've searched online extensively but there doesn't seem to be a consensus on the best way to achieve this. Some people talk about deriving the stream, others talk about deriving the buffer, yet others suggest the use of locales/facets etc.
Essentially, I'm finding myself writing a lot of code like this:
ofstream myFile();
myFile.open("test.php");
myFile << "<html>" << endl <<
"\t<head>" << endl <<
"\t\t<title>Hello world</title>" << endl <<
"\t</head>" << endl <<
"</html>" << endl;
When the tabs start to add up it looks horrible, and it seems like it would be nice to have something like this:
ind_ofstream myFile();
myFile.open("test.php");
myFile << "<html>" << ind_inc << ind_endl <<
"<head>" << ind_inc << ind_endl <<
"<title>Hello world</title>" << ind_dec << ind_endl <<
"</head>" << ind_dec << ind_endl <<
"</html>" << ind_endl;
i.e. create a derived stream class which would keep track of its current indent depth, then some manipulators to increase/decrease the indent depth, and a manipulator to write a newline followed by however many tabs.
So here's my shot at implementing the class & manipulators:
ind_ofstream.h
class ind_ofstream : public ofstream
{
public:
ind_ofstream();
void incInd();
void decInd();
size_t getInd();
private:
size_t _ind;
};
ind_ofstream& inc_ind(ind_ofstream& is);
ind_ofstream& dec_ind(ind_ofstream& is);
ind_ofstream& endl_ind(ind_ofstream& is);
ind_ofstream.cpp
ind_ofstream::ind_ofstream() : ofstream() {_ind = 0;}
void ind_ofstream::incInd() {_ind++;}
void ind_ofstream::decInd() {if(_ind > 0 ) _ind--;}
size_t ind_ofstream::getInd() {return _ind;}
ind_ofstream& inc_ind(ind_ofstream& is)
{
is.incInd();
return is;
}
ind_ofstream& dec_ind(ind_ofstream& is)
{
is.decInd();
return is;
}
ind_ofstream& endl_ind(ind_ofstream& is)
{
size_t i = is.getInd();
is << endl;
while(i-- > 0) is << "\t";
return is;
}
This builds, but doesn't generate the expected output; any attempt to use the custom manipulators results in them being cast to a boolean for some reason and "1" written to the file. Do I need to overload the << operator for my new class? (I haven't been able to find a way of doing this that builds)
Thanks!
p.s.
1) I've omitted the #includes, using namespace etc from my code snippets to save space.
2) I'm aiming to be able to use an interface similar to the one in my second code snippet. If after reading the whole post, you think that's a bad idea, please explain why and provide an alternative.
The iostreams support adding custom data to them, so you don't need to write a full derived class just to add an indentation level that will be operated on by manipulators. This is a little-known feature of iostreams, but comes in handy here.
You would write your manipulators like this:
/* Helper function to get a storage index in a stream */
int get_indent_index() {
/* ios_base::xalloc allocates indices for custom-storage locations. These indices are valid for all streams */
static int index = ios_base::xalloc();
return index;
}
ios_base& inc_ind(ios_base& stream) {
/* The iword(index) function gives a reference to the index-th custom storage location as a integer */
stream.iword(get_indent_index())++;
return stream;
}
ios_base& dec_ind(ios_base& stream) {
/* The iword(index) function gives a reference to the index-th custom storage location as a integer */
stream.iword(get_indent_index())--;
return stream;
}
template<class charT, class traits>
basic_ostream<charT, traits>& endl_ind(basic_ostream<charT, traits>& stream) {
int indent = stream.iword(get_indent_index());
stream.put(stream.widen('\n');
while (indent) {
stream.put(stream.widen('\t');
indent--;
}
stream.flush();
return stream;
}
I have combined Bart van Ingen Schenau's solution with a facet, to allow pushing and popping of indentation levels to existing output streams. The code is available on github: https://github.com/spacemoose/ostream_indenter, and there's a more thorough demo/test in the repository, but basically it allows you to do the following:
/// This probably has to be called once for every program:
// http://stackoverflow.com/questions/26387054/how-can-i-use-stdimbue-to-set-the-locale-for-stdwcout
std::ios_base::sync_with_stdio(false);
std::cout << "I want to push indentation levels:\n" << indent_manip::push
<< "To arbitrary depths\n" << indent_manip::push
<< "and pop them\n" << indent_manip::pop
<< "back down\n" << indent_manip::pop
<< "like this.\n" << indent_manip::pop;
To produce:
I want to push indentation levels:
To arbitrary depths
and pop them
back down
like this.
I had to do a kind of nasty trick, so I'm interested in hearing feedback on the codes utility.

How to easily indent output to ofstream?

Is there an easy way to indent the output going to an ofstream object? I have a C++ character array that is null terminate and includes newlines. I'd like to output this to the stream but indent each line with two spaces. Is there an easy way to do this with the stream manipulators like you can change the base for integer output with special directives to the stream or do I have to manually process the array and insert the extra spaces manually at each line break detected?
Seems like the string::right() manipulator is close:
http://www.cplusplus.com/reference/iostream/manipulators/right/
Thanks.
-William
This is the perfect situation to use a facet.
A custom version of the codecvt facet can be imbued onto a stream.
So your usage would look like this:
int main()
{
/* Imbue std::cout before it is used */
std::ios::sync_with_stdio(false);
std::cout.imbue(std::locale(std::locale::classic(), new IndentFacet()));
std::cout << "Line 1\nLine 2\nLine 3\n";
/* You must imbue a file stream before it is opened. */
std::ofstream data;
data.imbue(indentLocale);
data.open("PLOP");
data << "Loki\nUses Locale\nTo do something silly\n";
}
The definition of the facet is slightly complex.
But the whole point is that somebody using the facet does not need to know anything about the formatting. The formatting is applied independent of how the stream is being used.
#include <locale>
#include <algorithm>
#include <iostream>
#include <fstream>
class IndentFacet: public std::codecvt<char,char,std::mbstate_t>
{
public:
explicit IndentFacet(size_t ref = 0): std::codecvt<char,char,std::mbstate_t>(ref) {}
typedef std::codecvt_base::result result;
typedef std::codecvt<char,char,std::mbstate_t> parent;
typedef parent::intern_type intern_type;
typedef parent::extern_type extern_type;
typedef parent::state_type state_type;
int& state(state_type& s) const {return *reinterpret_cast<int*>(&s);}
protected:
virtual result do_out(state_type& tabNeeded,
const intern_type* rStart, const intern_type* rEnd, const intern_type*& rNewStart,
extern_type* wStart, extern_type* wEnd, extern_type*& wNewStart) const
{
result res = std::codecvt_base::noconv;
for(;(rStart < rEnd) && (wStart < wEnd);++rStart,++wStart)
{
// 0 indicates that the last character seen was a newline.
// thus we will print a tab before it. Ignore it the next
// character is also a newline
if ((state(tabNeeded) == 0) && (*rStart != '\n'))
{
res = std::codecvt_base::ok;
state(tabNeeded) = 1;
*wStart = '\t';
++wStart;
if (wStart == wEnd)
{
res = std::codecvt_base::partial;
break;
}
}
// Copy the next character.
*wStart = *rStart;
// If the character copied was a '\n' mark that state
if (*rStart == '\n')
{
state(tabNeeded) = 0;
}
}
if (rStart != rEnd)
{
res = std::codecvt_base::partial;
}
rNewStart = rStart;
wNewStart = wStart;
return res;
}
// Override so the do_out() virtual function is called.
virtual bool do_always_noconv() const throw()
{
return false; // Sometime we add extra tabs
}
};
See: Tom's notes below
Well this is not the answer I'm looking for, but in case there is no such answer, here is a way to do this manually:
void
indentedOutput(ostream &outStream, const char *message, bool &newline)
{
while (char cur = *message) {
if (newline) {
outStream << " ";
newline = false;
}
outStream << cur;
if (cur == '\n') {
newline = true;
}
++message;
}
}
A way to add such feature would be to write a filtering streambuf (i.e. a streambuf which forwards the IO operation to another streambuf but manipulate the data transfered) which add the indentation as part of its filter operation. I gave an example of writing a streambuf here and boost provides a library to help in that.
If your case, the overflow() member would simply test for '\n' and then add the indent just after if needed (exactly what you have done in your indentedOuput function, excepted that newline would be a member of the streambuf). You could probably have a setting to increase or decrease the indent size (perhaps accessible via a manipulator, the manipulator would have to do a dynamic_cast to ensure that the streambuf associated to the stream is of the correct type; there is a mechanism to add user data to stream -- basic_ios::xalloc, iword and pword -- but here we want to act on the streambuf).
I've had good success with Martin's codecvt facet based suggestion, but I had problems using it on std::cout on OSX, since by default this stream uses a basic_streambuf based streambuf which ignores the imbued facet. The following line switches std::cout and friends to use a basic_filebuf based streambuf, which will use the imbued facet.
std::ios::sync_with_stdio(false);
With the associated side effect that the iostream standard stream objects may operate independently of the standard C streams.
Another note is since this facet does not have a static std::locale::id, which meant that calling std::has_facet<IndentFacet> on the locale always returned true. Adding a std::local::id meant that the facet was not used, since basic_filebuf looks for the base class template.
There is no simple way, but a lot has been written about the complex
ways to achieve this. Read this article for a good explanation of
the topic. Here is another article, unfortunately in German. But
its source code should help you.
For example you could write a function which logs a recursive structure. For each level of recursion the indentation is increased:
std::ostream& operator<<(std::ostream& stream, Parameter* rp)
{
stream << "Parameter: " << std::endl;
// Get current indent
int w = format::get_indent(stream);
stream << "Name: " << rp->getName();
// ... log other attributes as well
if ( rp->hasParameters() )
{
stream << "subparameter (" << rp->getNumParameters() << "):\n";
// Change indent for sub-levels in the hierarchy
stream << format::indent(w+4);
// write sub parameters
stream << rp->getParameters();
}
// Now reset indent
stream << format::indent(w);
return stream;
}
I have generalized Loki Astarti's solution to work with arbitrary indentation levels. The solution has a nice, easy to use interface, but the actual implementation is a little fishy. It can be found on github:https://github.com/spacemoose/ostream_indenter
There's a more involved demo in the github repo, but given:
#include "indent_facet.hpp"
/// This probably has to be called once for every program:
// http://stackoverflow.com/questions/26387054/how-can-i-use-stdimbue-to-set-the-locale-for-stdwcout
std::ios_base::sync_with_stdio(false);
// This is the demo code:
std::cout << "I want to push indentation levels:\n" << indent_manip::push
<< "To arbitrary depths\n" << indent_manip::push
<< "and pop them\n" << indent_manip::pop
<< "back down\n" << indent_manip::pop
<< "like this.\n" << indent_manip::pop;
}
It produces the following output:
I want to push indentation levels:
To arbitrary depths
and pop them
back down
like this.
I would appreciate any feedback as to the utility of the code.
Simple whitespace manipulator
struct Whitespace
{
Whitespace(int n)
: n(n)
{
}
int n;
};
std::ostream& operator<<(std::ostream& stream, const Whitespace &ws)
{
for(int i = 0; i < ws.n; i++)
{
stream << " ";
}
return stream;
}