I often use the Boost Interprocess library, which provides an I/O stream class, bufferstream, which is basically similar to std::stringstream, except it operates over an arbitrary user-provided memory buffer.
I've used this stream class extensively, and I've also used it's source code to learn a lot about implementing/extending C++ I/O streams in general.
However, one piece of implementation code has struck me as very weird, and I'd like to see if anyone knows exactly why the Boost designer implemented it this way.
Specifically, it's a function in the streambuffer class boost::interprocess::basic_bufferbuf. This class inherits from std::basic_streambuf, and overrides all the virtual protected methods to provide specific behavior.
The source code can be browsed online here.
Specifically, the function basic_bufferbuf::seekpos() is implemented as follows:
virtual pos_type seekpos(pos_type pos, std::ios_base::openmode mode
= std::ios_base::in | std::ios_base::out)
{ return seekoff(pos - pos_type(off_type(0)), std::ios_base::beg, mode); }
Generally, stream buffer implementations which override std::basic_streambuf::seekpos() just call std::basic_streambuf::seekoff with the starting position set to std::ios_base::beg, so that's normal.
But why does the implementor write pos - pos_type(off_type(0)) here? Why subtract zero from the starting position? What possible use is that?
I suspect it must have something to do with unusual off_type or pos_type types, but I can't imagine what. Generally, pos_type and off_type are just signed integral types.
So why subtract zero here?
Related
I want to use std::istream to read data from a given class which only provides 2 methods:
// Returns a byte from the stream (consuming it)
uint8_t getChar(OwnIOStream stream);
// Makes the passed pointer point to the data in the stream
bool getCharBlockPtr(OwnIOStream stream, uint8_t** buffer, uin32_t maxSize, uint32_t* size);
I first thought of inheriting from stream_buf and implement the underflow method using the getChar() method. However I would like to use the getCharBlockPtr() instead to avoid copies of data (I assume calling underflow for each read byte will decrease the performance). The problem is that I need to know in advance the quantity of bytes I want to read each time. This is why I was thinking whether it would be possible to override the read method of istream.
Using boost is not an option.
Deriving from streambuf is a perfectly valid option.
Whether you need to do much (if any) more than override underflow depends. If OwnIOStream is already buffering data, then you might get adequate performance from just doing that. I'd start by doing that, and seeing how things go.
The next obvious step would be to use getCharBlockPtr. A streambuf has a setg that you use to set the pointers to the beginning, current position, and end of the buffer. At least if I'm interpreting things correctly, you'd call getCharBlockPtr and get the beginning and end. You'd then call setg to set the beginning and current positions to the beginning, and the end to the end. From there, the streambuf should be able to read data directly from the buffer. When it runs out, it'll call underflow, and you'll need to get more data again.
One note: it doesn't look like OwnIOStream supports a putback area, which streams normally do expect to support. Getting this to work correctly (especially putting back a character when at the beginning of a buffer) may be somewhat non-trivial to support correctly.
The general rule is that most classes from the standard C++ library are not meant to be derived. The exception here is the streambuf class which is intended to be the low level interface between an IO capable object and a C++ stream.
So the standard way here would be to build a custom streambuf calling getChar or better getCharBlockPtr that would allow buffering. Jerry's answer explains how to do it.
But it looks like you have special requirement here:
you know at a higher level the number of bytes you want to read
you are in a performance critical operation and want to avoid buffer copies if possible - but this last point should be reviewed after profiling...
Then, asking for the full power of a C++ stream may not be the way to go. Building a custom class that would only implement the methods of a istream that you need to call, as well as few extractors would still give a nice code at high level and could give a more optimized code at low level.
But you and only you can know:
whether your higher level code requires a C++ stream - then a custom class is not an option and you will have to use a custom streambuf
whether you could directly give the expected size to the streambuf from the calling code as a hint. More or less:
class OwnStreamBuf: public std::streambuf {
...
void sizehint(size_t hint) { /* size for next read access */
this->hint = hint
}
int_type underflow() {
...
cr = getCharBlockPtr(stream, buffer, hint, size); /* use hint as maxSize */
...
}
}
...
OwnStreamBuf buf(...);
class OwnIStream: public istream ...
OwnIStream is(buf)
...
buf.sizehint(n)
is >> special_obj;
...
whether you only need a handful of extractors and a custom class will be simpler to build from scratch.
I have a function which works with a std::ostream. I need to support using a C file handle (FILE*). Should I be creating my own subclass of std::ostream which delegates to a FILE*?
As Ben Voigt points out, you want to subclass streambuf. There are pages on the University of Southern California's website which have the documentation, header, and source for a GNU implementation of a streambuf subclass (stdiobuf) that wraps a FILE*. It has some dependencies on the library it is a part of (GroovX), but those should be easily to remove (I would begin by removing all references to GVX_TRACE).
Interestingly, it also provides a minimalistic subclass (stdiostream) of std::iostream, in spite of what Ben Voigt said. But this does not seem to be necessary, as the rdbuf ("read buffer"/set the stream buffer) method which the stdiostream class uses to connect the stdiobuf class to a stream object is publicly accessible.
You can find more about subclassing streambuf here (look particularly at the bottom of the page, which discussing the virtual functions). The implementation linked above overrides sync, underflow (to support input) and overflow (to support output).
Further notes about the linked implementation:
The init method uses the setg and setp methods to set the pointers for the input and output sequences.
The line const int num = pptr()-pbase(); is calculating the number of characters to flush by subtracting the base output pointer from the current output pointer ("put pointer").
The variable unhelpfully named om is the mode parameter.
The variable named fd is the file descriptor.
No, ostream is not meant to be derived from. The way the iostreams library allows customization is by supplying a streambuf pointer when creating an ostream. streambuf has a lot of virtual functions so you can change its behavior.
You need to derive either directly from streambuf or from the existing filebuf subclass. You probably only need to provide the overflow function, the defaults for all the others should work ok.
How can I override a C++ standard-library class function? In my application, I use ofstream objects in many different places of code. And now I want to open files in a different permission mode in Linux, Ubuntu. But open function of ofstream has no parameter to specify the permission mode of the file it creats.
Now I want to override open() function of ofstream class so it will get a parameter to specify the permissions for user access.
For starters, to clarify your terminology, the STL usually refers to the subset of the C++ standard library containing the containers, iterators, and algorithms. The streams classes are part of the C++ standard library, but are usually not bundled together with the STL. Some purists will insist that there is no such thing as the STL in the C++ standard library (since the STL is, technically speaking, a third-party library that was adopted into the standard), but most C++ programmers will know what you mean.
As for your question, there is no way within the standard to specify permission modes with ofstream. If you want to create your own custom stream class that is like ofstream but which supports permissions, your best bet would be to do the following:
Create a subclass of basic_streambuf that allows you to open, write, and possibly read files while specifying Unix permissions. The streams classes are designed so that the details of communicating with physical devices like disk, networks, and the console are all handled by the basic_streambuf class and its derived classes. If you want to make your own stream class, implementing a stream buffer would be an excellent first step.
Define your own class that subclasses basic_ostream and installs your custom basic_streambuf. By default, the basic_ostream supports all of the standard output routines by implementing them in terms of the underlying basic_streambuf object. Once you have your own stream buffer, building a basic_ostream class that uses that streambuf will cause all of the standard stream operations on that class (such as <<) to start making the appropriate calls to your streambuf.
If you'd like more details on this, an excellent reference is Standard C++ IOStreams and Locales. As a shameless plug, I have used the techniques from this book to build a stream class that wraps a socket connection. While a lot of the code in my stream won't be particularly useful, the basic structure should help you get started.
Hope this helps!
This is not answering your question directly as I wouldn't advise overriding ofstream::open.
Instead couldn't you use the first suggestion in this post? Open the file as you normally would to get the correct permissions, and then construct an ofstream from the file descriptor.
#include <iostream>
#include <fstream>
class gstream: public std::ofstream
{
void open(const std::string& filename, ios_base::openmode mode,int stuff)
{
//put stuff here
}
};
int main() {
gstream test;
//io stuff
return 0;
}
seems to work here.
Another option would be to create a wrapper class that contains an 'ofstream' object and has the interface you want, and passes the work onto its 'oftstream' member. It would look like this.
I have a function which works with a std::ostream. I need to support using a C file handle (FILE*). Should I be creating my own subclass of std::ostream which delegates to a FILE*?
As Ben Voigt points out, you want to subclass streambuf. There are pages on the University of Southern California's website which have the documentation, header, and source for a GNU implementation of a streambuf subclass (stdiobuf) that wraps a FILE*. It has some dependencies on the library it is a part of (GroovX), but those should be easily to remove (I would begin by removing all references to GVX_TRACE).
Interestingly, it also provides a minimalistic subclass (stdiostream) of std::iostream, in spite of what Ben Voigt said. But this does not seem to be necessary, as the rdbuf ("read buffer"/set the stream buffer) method which the stdiostream class uses to connect the stdiobuf class to a stream object is publicly accessible.
You can find more about subclassing streambuf here (look particularly at the bottom of the page, which discussing the virtual functions). The implementation linked above overrides sync, underflow (to support input) and overflow (to support output).
Further notes about the linked implementation:
The init method uses the setg and setp methods to set the pointers for the input and output sequences.
The line const int num = pptr()-pbase(); is calculating the number of characters to flush by subtracting the base output pointer from the current output pointer ("put pointer").
The variable unhelpfully named om is the mode parameter.
The variable named fd is the file descriptor.
No, ostream is not meant to be derived from. The way the iostreams library allows customization is by supplying a streambuf pointer when creating an ostream. streambuf has a lot of virtual functions so you can change its behavior.
You need to derive either directly from streambuf or from the existing filebuf subclass. You probably only need to provide the overflow function, the defaults for all the others should work ok.
Some days ago, I decided that it would be fun to write a streambuf subclass that would use mmap and read-ahead.
I looked at how my STL (SGI) implemented filebuf and realized that basic_filebuf contains a FILE*. So inheriting from basic_filebuf is out of the question.
So I inherited from basic_streambuf. Then i wanted to bind my mmapbuf to a fstream.
I thought the only thing that I would have to do would be to copy the implicit interface of filebuf... but that was a clear mistake. In the SGI, basic_fstream owns a basic_filebuf. No matter if I call basic_filestream.std::::ios::rdbuf( streambuf* ), the filestream completely ignores it and uses its own filebuf.
So now I'm a bit confused... sure, I can create my own mmfstream, that would be the exact copy/paste of the fstream but that sounds really not DRY-oriented.
What I can't understand, is: why does fstream is so tightly coupled with filebuf, so that it is not possible to use anything else than a filebuf? The whole point of separating streams and bufs is that one can use a stream with a different buffer.
Solutions:
=> filestream should rely on the implicit interface of filebuf. That is, fstream should be templated by a streambuf class. That would allow everyone to provide its own streambuf subclass to a fstream as long as it implements filebuf's implicit interface. Problem: we cannot add a template parameter to fstream since it would break template selectors while using fstream as template template parameter.
=> filebuf should be a pure virtual class without any additional attributes. So that one can inherit from it without carrying all its FILE* garbage.
Your ideas on the subject ?
In the IO streams' design, most of the actual streams' functionality (as opposed to the stream buffers' functionality) is implemented in std::basic_istream, std::basic_ostream, and their base classes. The string and file stream classes are more or less just convenience wrappers which make sure a stream with the right type of buffer is instantiated.
If you want to extend the streams, you almost always want to provide your own stream buffer class, and you almost never need to provide your own stream class. .
Once you have your own stream buffer type, you can then make it the buffer for any stream object you happen to have around. Or you derive your own classes from std::basic_istream, std::basic_ostream, and std::basic_iostream which instantiates your stream buffer and pass it to their base classes.
The latter is more convenient for users, but requires you to write some boiler-plate code for the buffer's instantiation (namely constructors for the stream class).
To answer your question: File streams and file buffer are coupled so tightly because the former only exists to ease the creation of the latter. Using a file stream makes it easy to set it all up.
Using your own stream class to wrap construction of your own stream buffer shouldn't be a problem, since you shouldn't be passing around file streams anyway, but only (references) to the base classes.
Check out mapped_file in the Boost.Iostreams library. I've never used used it myself, but it seems like it might already do what you need.
EDIT: Oops, reread your questions and I see you're doing this for fun. Perhaps you can draw inspiration from Boost.Iostreams?
fstream in itself is not a big class. It inherits from basic_stream to provide support for all the << and >> operations, contains a specialized steambuf that have to be initialized, and the corresponding constructors to pass the parameters to the streambuf constructor.
In a sense, what you wrote about your templated solution is OK. But basic_stream can also be derived into a tcp_stream for example. In that case, the constructors of fstream are a bit useless. Thus you need to provide a new tcpstream class, inheriting from basic_stream with the correct parameters for the constructors to be able to create the tcp_stream. In the end, you wouldn't use anything from fstream. Creating this new tcpstream is a matter of writing 3 or 4 functions only.
In the end, you would derive from the fstream class without any real reason to. This would add more coupling in the class hierarchy, unneeded coupling.
The whole point of std::fstream is that it is a _F_ile based std::stream. If you want an ordinary std::stream backed by your mmstreambuf, then you should create a mmstreambuf and pass it to std::stream::stream(std::streambuf*)