Taking ownership of streambuf/stringbuf data - c++

Question

I'd like an interface for writing to an automatically resizing array. One way to do this is with a generic std::ostream *.
Then consider if ostringstream is the target:
void WritePNG(ostream *out, const uint8_t *pixels);
void *WritePNGToMemory(uint8_t *pixels)
{
ostringstream out;
WritePng(&out, pixels);
uint8_t *copy = new uint8_t[out.tellp()];
memcpy(copy, out.str().c_str(), out.tellp()];
return copy;
}
But I want to avoid the memcpy(). Is there a way to take ownership of the array in the underlying stringbuf class and return that?
I get the feeling this can't be done using standard library, since the stream buffer might not even be a contiguous array.

Answer 1

If you're willing to use the old, deprecated <strstream> interface, this is fairly easy - just create a std::strstreambuf pointing at your storage, and it will work by magic. std::ostrstream even has a constructor to do this for you:
#include <iostream>
#include <strstream>
int main()
{
char copy[32] = "";
std::ostrstream(copy, sizeof copy) << "Hello, world!"
<< std::ends << std::flush;
std::cout << copy << '\n';
}
With the more modern <sstream> interface, you need to access the string stream's buffer, and call pubsetbuf() to make it to point at your storage:
#include <iostream>
#include <sstream>
int main()
{
char copy[32] = "";
{
std::ostringstream out{};
out.rdbuf()->pubsetbuf(copy, sizeof copy);
out << "Hello, world!" << std::ends << std::flush;
}
std::cout << copy << '\n';
}
Obviously, in both cases, you'll need a way to know in advance how much memory to allocate for copy, because you can't wait until tellp() is ready for you...

Answer 2

Here's the solution I ended up using. The idea is the same as the one proposed by HostileFork - only needing to implement overflow(). But as already hinted, it has much better throughput by buffering. It also optionally supports random access (seekp(), tellp()).
class MemoryOutputStreamBuffer : public streambuf
{
public:
MemoryOutputStreamBuffer(vector<uint8_t> &b) : buffer(b)
{
}
int_type overflow(int_type c)
{
size_t size = this->size(); // can be > oldCapacity due to seeking past end
size_t oldCapacity = buffer.size();
size_t newCapacity = max(oldCapacity + 100, size * 2);
buffer.resize(newCapacity);
char *b = (char *)&buffer[0];
setp(b, &b[newCapacity]);
pbump(size);
if (c != EOF)
{
buffer[size] = c;
pbump(1);
}
return c;
}
#ifdef ALLOW_MEM_OUT_STREAM_RANDOM_ACCESS
streampos MemoryOutputStreamBuffer::seekpos(streampos pos,
ios_base::openmode which)
{
setp(pbase(), epptr());
pbump(pos);
// GCC's streambuf doesn't allow put pointer to go out of bounds or else xsputn() will have integer overflow
// Microsoft's does allow out of bounds, so manually calling overflow() isn't needed
if (pptr() > epptr())
overflow(EOF);
return pos;
}
// redundant, but necessary for tellp() to work
// https://stackoverflow.com/questions/29132458/why-does-the-standard-have-both-seekpos-and-seekoff
streampos MemoryOutputStreamBuffer::seekoff(streamoff offset,
ios_base::seekdir way,
ios_base::openmode which)
{
streampos pos;
switch (way)
{
case ios_base::beg:
pos = offset;
break;
case ios_base::cur:
pos = (pptr() - pbase()) + offset;
break;
case ios_base::end:
pos = (epptr() - pbase()) + offset;
break;
}
return seekpos(pos, which);
}
#endif
size_t size()
{
return pptr() - pbase();
}
private:
std::vector<uint8_t> &buffer;
};
They say a good programmer is a lazy one, so here's an alternate implementation I came up with that needs even less custom code. However, there's a risk for memory leaks because it hijacks the buffer inside MyStringBuffer, but doesn't free MyStringBuffer. In practice, it doesn't leak for GCC's streambuf, which I confirmed using AddressSanitizer.
class MyStringBuffer : public stringbuf
{
public:
uint8_t &operator[](size_t index)
{
uint8_t *b = (uint8_t *)pbase();
return b[index];
}
size_t size()
{
return pptr() - pbase();
}
};
// caller is responsible for freeing out
void Test(uint8_t *&_out, size_t &size)
{
uint8_t dummy[sizeof(MyStringBuffer)];
new (dummy) MyStringBuffer; // construct MyStringBuffer using existing memory
MyStringBuffer &buf = *(MyStringBuffer *)dummy;
ostream out(&buf);
out << "hello world";
_out = &buf[0];
size = buf.size();
}

Answer 3

IIRC the whole reason stringstream exists (vs strstream) was to sort out the fuzzy questions of memory ownership that would come up by giving direct buffer access. e.g. I think that change was to specifically prevent what you are asking to do.
One way or another I think you'd have to do it yourself, by overriding the stream buffer. To answer a similar question I suggested something for input streams that wound up getting quite a few upvotes. But honestly I didn't know what I was talking about then, nor now when I suggest the following:
Hacking up this link from the web for doing an "uppercasing stream buffer" to one that just echoes and gives you a reference to its buffer might give:
#include <iostream>
#include <streambuf>
class outbuf : public std::streambuf {
std::string data;
protected:
virtual int_type overflow (int_type c) {
if (c != EOF)
data.push_back(c);
return c;
}
public:
std::string& get_contents() { return data; }
};
int main() {
outbuf ob;
std::ostream out(&ob);
out << "some stuff";
std::string& data = ob.get_contents();
std::cout << data;
return 0;
}
I'm sure it's broken in all kinds of ways. But the uppercase-buffer-authors seemed to think that overriding the overflow() method alone would let them uppercase all output to the stream, so I guess one could argue that it's enough to see all output if writing to one's own buffer.
But even so, going one character at a time seems suboptimal...and who knows what overhead you get from inheriting from streambuf in the first place. Consult your nearest C++ iostream expert for what the actual right way is. But hopefully it's proof that something of the sort is possible.