I am working on an application the will need to sequentially encode/decode a series of bytes, and put them to a stream once they have been processed.
My plan was to subclass ostream and provide a constructor MyEncodeStream(istream) using istream.
I must admit though I am at a loss as to what to do next, which methods should be overridden?
What is the standard facility for doing this.
Please provide example code for a this simple case:
A char is pulled from the input buffer
The char is manipulated by char manipulationFunc(char in)
The char is put to a buffer
MyEncodeStream stops blocking so the char can be read
Does ostream provide a read function that should be overridden which operator << calls, or is operator<< the function that should be overridden?
I figured it out, I was confused. What I wanted to do was to subclass an input stream with an input stream as an argument for the constructor, that way when the custom input stream is read it can read from the standard input mechanism. I was looking up subclassing output streams, which is why google wasn't turning up many results.
Related
I need some guidance or pointers understanding how to implement a custom ostream. My requirements are:
A class with a '<<' operator for several data types.
The intention is to send output to database. Each "line" should go to a separate record.
Each record most important field would be the text (or blob), but some other fields such as time, etc. can be mostly deduced automatically
buffering is important, as I don't want to go to database for every record.
First, does it worth deriving from ostream? What do I get by deriving from ostream? What if my class simply implements few operator<< methods (including some custom data types). Which functionality do I get from ostream?
Assuming what I want is a class derived from ostream, I need some guidance understanding the relationship between the ostream and the streambuf classes. Which one do I need to implement? Looking at some samples, it appears that I don't need to derive from ostream at all, and just give the ostream constructor a custom streambuf. Is that true? is that the canonical approach?
Which virtual functions at the custom streambuf do i need to implement? I've seen some samples (including this site: here and here, and few more), some override the sync method, and other override the overflow method. Which one should I override? Also, looking at the stringbuf and filebuf sources (Visual Studio or GCC) both those buffer classes implement many methods of the streambuf.
If a custom class derived from streambuf is required, would there be any benefit deriving from stringbuf (or any other class) instead of directly from streambuf?
As for "lines". I would like at least when my users of the class using the 'endl' manipulator to be a new line (i.e. record in database). Maybe - depends on effort - every '\n' character should be considered as a new record as well. Who do my custom ostream and/or streambuf get notified for each?
A custom destination for ostream means implementing your own ostreambuf. If you want your streambuf to actually buffer (i.e. don't connect to the database after each character), the easiest way to do that is by creating a class inheriting from std::stringbuf. The only function that you'll need to override is the sync() method, which is being called whenever the stream is flushed.
class MyBuf : public std::stringbuf
{
public:
virtual int sync() {
// add this->str() to database here
// (optionally clear buffer afterwards)
}
};
You can then create a std::ostream using your buffer:
MyBuf buff;
std::ostream stream(&buf)
Most people advised against redirecting the stream to a database, but they ignored my description that the database basically has a single blob field where all text is going to.
In rare cases, I might send data to a different field. This can be facilitated with custom attributes understood by my stream. For example:
MyStream << "Some text " << process_id(1234) << "more text" << std::flush
The code above will create a record in the database with:
blob: 'Some text more text'
process_id: 1234
process_id() is a method returning a structure ProcessID. Then, in the implementation of my ostream, I have an operator<<(ProcessID const& pid), which stores the process ID until it gets written. Works great!
The simplest way is to inherit std::streambuf and override just two methods:
std::streamsize xsputn(const char_type* s, std::streamsize n) – to append a given buffer with size provided to your internal buffer, std::string for example;
int_type overflow(int_type c) – to append a single char to your internal buffer.
Your streambuf can be constructed from whatever you want (DB connection for example). After append something into the internal buffer you may try to split it into lines and push something into DB (or just buffer an SQL statements to execute later).
To use it: just attach your streambuf to any std::ostream using constructor.
Simple! I've done something like this to output strings to syslog – everything works fine with any custom operator<< for user defined classes.
my2c - I think you are tackling this the wrong way. A stream may sound like a nice idea, but you'll need a way to indicate the end of the row too (and then what if someone forgets?) I would suggest something along the lines of how the java PreparedStatements and batches work, as in provide a set of methods which accept the types and a column index, then a "batch" method which explicitly makes it clear that you are indeed batching that row and then an execute to push the batch in.
Any stream based operation will rely on type (typically) to indicate which column to fill - but what if you have two ints? IMO, as a user, it doesn't feel like a natural way of inserting records into a database...
To add a new source or destination of character input/output to the iostreams mechanism, you should create a new streambuf class. The task of the stream buffer classes is to communicate with the 'external device' that will store the characters and to provide buffering facilities.
The problem with using iostreams to communicate with your database is that a database table does not match with the concept of a character sequence. A bit like pushing a round peg in a square hole. A streambuf only operates on characters. That is the only thing ever presented to it. This means the streambuf has to parse the character stream presented to it to find the field and record separators.
If you decide to go this route, I predict you will end up writing a CSV-to-SQL converter in your streambuf, just to get it working.
You will probably be better of with just adding a few operator<< overloads to your class(es). You could look at the Qt framework for ideas here. They also have the possibility to use operator<< to add items to a collections and such.
I have an abstract class in my project, its derivatives is used for input/output to different locations.
It has virtual methods read and write.
virtual unsigned read(void *buf, unsigned len) = 0;
virtual void write(const void *buf, unsigned len) = 0;
I need a kind of an adapter between std streams (std::istream and std::ostream) and this class to redirect input/output to these methods.
So, for example, if
mystream << "some output";
is called, it will call the write method.
I guess i should overload std::istream and std::ostream or std::streambuf, but not sure which methods.
What is the better way to implement this?
There are lots of simple but not flexible ways of doing it. Most of these solution will not leverage istream or ostream. For instance, overloading the << operator is one way. The drawback is that you will have to implement this operator for all the usual types, and for all the standard manipulators, and so on. It may become a great burden.
This is sad because the whole thing about istream and ostream is only to parse and format not to do input or output. The I/O responsibility is given to the streambuf. And your task calls for a custom implementation of streambuf which uses your read and write methods.
The discussion is too long for such a small format as a stackoverflow answer but you can find good pointers in the following references.
References
http://www.mr-edd.co.uk/blog/beginners_guide_streambuf
http://en.cppreference.com/w/cpp/io/basic_streambuf
http://www.amazon.com/Standard-Library-Tutorial-Reference-2nd/dp/0321623215/ref=sr_1_1?ie=UTF8&qid=1442329302&sr=8-1&keywords=josuttis , this book contains a chapter dedicated to streambuf implementation.
Note
As advised, using boost.iostreams maybe a good fit, but I don't know it enough.
You might want to take a look at the boost iostreams library. It provides a framework that makes it easier to define iostreams with custom sources and sinks (input and output devices).
I am a fan of std::stringstream, and perhaps your class could make use of it.
std::stringstream ss;
ss << "some output";
from which something would invoke write like:
write(ss.str().c_str(), ss.str().size());
You will have to figure out how to connect the two, but this has the advantage of providing all stream io.
On the other hand, directly implementing operator << and >> is not too difficult.
I guess i should overload std::istream and std::ostream or
std::streambuf, but not sure which methods.
Start with the method you need first, then add functionality as you identify the requirements.
I've read about custom streams for C++ but it seems that generally people inherit from std::streambuf, std::istream, and std::ostream. By inspecting those type's declarations it becomes clear that these are meant for characters:
typedef basic_streambuf<char> streambuf;
The docs confirm this:
Input stream objects can read and interpret input from sequences of
characters.
Obviously, that makes sense. I'm wondering what would be the correct way of implementing a stream for other types. I do not want to allow text but other forms of binary input/output (I have specific formats). The obvious step seems to be to inherit the basic variants of the above (basic_streambuf, basic_istream, and basic_ostream) and use whatever type I see fit as the template parameter. I failed to find confirmation that this would be the right procedure. So, is it?
Edit for clarification: I have a class called Segment. These streams will send/receive segments and only segments over a WiFi connection as these are used in the communication protocol. Sending anything else would break the protocol. This means that the stream cannot support other types.
This is not an answer to your question in terms of inheriting from std::basic_* with non-char types. But following the comments and given your application, I am questioning the need to reimplement the whole standard stream machinery for your Segment type, when you can simply define a class with a stream operator:
class SegmentStream
{
public:
SegmentStream& operator<< ( const Segment& s );
}
Better yet you could clarify your code by defining methods send and recv instead of >>, <<.
Or perhaps you could explain why this would not be sufficient and why you specifically want to use standard streams?
Why do tellg() and tellp() always come together? And how can we separate them from each other?
They don't.
One indicates where the "get" cursor is in an input stream; the other indicates where the "put" cursor is in an output stream.
They are similar but not intrinsically linked, so the premise of your question is fundamentally flawed.
For starters, they're members of different classes: tellg is
a member of std::istream, and tellp is a member of
std::ostream. The reason they have different names is because
there are classes (e.g. std::iostream) which derive from both
std::istream and std::ostream, and the functions may report
different values: an actual std::streambuf type may maintain
the read and write positions in sync (as does std::filebuf) or
keep them separate (as does std::stringbuf).
One is for an input stream, the other for an output stream.
http://en.cppreference.com/w/cpp/io/basic_istream/tellg
input stream position
http://en.cppreference.com/w/cpp/io/basic_ostream/tellp
output stream position
I'd like to write a simple ostream which wraps an argument ostream and changes the stream in some way before passing it on to the argument stream. The transformation is something simple like changing a letter or erasing a word
What would a simple class inheriting from ostream look like? What methods should I override?
std::ostream is not the best place to implement filtering. It doesn't have the appropriate virtual functions to let you do this.
You probably want to write a class derived from std::streambuf containing a wrapped std::ostream (or a wrapped std::streambuf) and then create a std::ostream using this std::streambuf.
std::streambuf has a virtual function overflow which you can override and use to alter the bytes before passing them to the wrapped output class.
Consider using Boost.Iostreams
I've always thought that writing specialised streams is the wrong approach to almost any problem. The output stream is typically an end-point in your program - any data processing should be done long before you get to the stream itself. Similarly for input streams - putting the intelligence needed to (say) parse input in the stream is putting it in the wrong place. Just my 2 cents, of course.