I am using the library libvtemm, which has a function write_contents. It takes an internal buffer and outputs it to a Glib::RefPtr<Gio::OutputStream> object. I have been trying to find a way to convert the contents of the Gio::OutputStream into a std::string or something similar so that I can play with and move around the data inside to other data structures.
Does anyone know how to either construct a Gio::OutputStream to something like a std::ostream or convert its contents into a std::string?
I see there is a Gio::MemoryOutputStream, would something like this be useful in grabbing the data to a std::ostream?
For those of you who are looking for an answer, here is what I have come up with to read the console buffer into a std::string.
// Create a mock stream just for this example
Glib::RefPtr<Gio::MemoryOutputStream> bufStream =
Gio::MemoryOutputStream::create(NULL, 0, &realloc, &free);
// Create the stringstream to use as an ostream
std::stringstream ss;
// Get the stream size so we know how much to allocate
gsize streamSize = bufStream->get_data_size();
char *charBuf = new char[streamSize+1];
// Copy over the data from the buffer to the charBuf
memcpy(charBuf, bufStream->get_data(), streamSize);
// Add the null terminator to the "string"
charBuf[streamSize] = '\0';
// Create a string from it
ss << charBuf;
Hope this helps someone in the future who comes across a similar problem.
Related
Here is some sample code
std::stringstream ss;
boost::archive::binary_oarchive oa(ss);
oa << newMat;
int size = ss.tellp();
void* matBin = malloc(size);
memcpy(matBin,ss.str().c_str(), size);
Basically I am attempting to store some matrices a binary data into a database. This code should turn the matrix into binary data (don't worry about the details of the boost archive), and I am attempting to store the data into a buffer. However, I am not sure if ss.tellp() is the proper way to determine the size of binary data.
Can anyone confirm if this a proper way to determine the size of ss?
Thanks!
Like this
int size = ss.str().size();
but I see no reason why tellp wouldn't work as well.
I know a way to read from a stream and use it like below:
strstream s; // It can be another standard stream type
// ...
while (!s.eof())
{
char buf[MAX];
s.read(buf, sizeof (buf));
int count = s.gcount();
THIRD_PARTY_FUNCTION(buf, count);
// ...
}
but this code has an abusing point, It first copies data from the stream to buf and then passes buf to THIRD_PARTY_FUNCTION.
Is there any way to reform the code to something like below(I mean below code avoids an extra copy) ?
strstream s; // It can be another standard stream type
// ...
while (!s.eof())
{
char *buf = A_POINTER_TO_DATA_OF_STREAM(s);
int count = AVAIABLE_DATA_SIZE_OF_STREAM(s);
// Maybe it needs s.seekg(...) here
THIRD_PARTY_FUNCTION(buf, count);
// ...
}
Something like this might work for you.
char buffer[2000];
std::istream& s = getStreamReference();
s.rdbuf()->pubsetbuf(buffer, 2000);
while(s)
{
THIRD_PARTY_FUNCTION(buffer, s.rdbuf()->in_avail());
s.ignore(s.rdbuf()->in_avail());
// Not sure this may go into an infinite loop.
// Its late here so I have not tested it.
}
Note sure I care about the cost of copying a 2K buffer.
The profiling would have to show that this is a real hotspot that is causing a significant degrade in performance before I would look at making this kind of optimization. Readability is going to be my most important factor here 99% of the time.
You can convert a std::stringstream to a c-style string by first calling its member method str to get an std::string and then call the member function c_str of that to convert it to a c-style null-terminated char[].
Apparently boost::asio::async_read doesn't like strings, as the only overload of boost::asio::buffer allows me to create const_buffers, so I'm stuck with reading everything into a streambuf.
Now I want to copy the contents of the streambuf into a string, but it apparently only supports writing to char* (sgetn()), creating an istream with the streambuf and using getline().
Is there any other way to create a string with the streambufs contents without excessive copying?
I don't know whether it counts as "excessive copying", but you can use a stringstream:
std::ostringstream ss;
ss << someStreamBuf;
std::string s = ss.str();
Like, to read everything from stdin into a string, do
std::ostringstream ss;
ss << std::cin.rdbuf();
std::string s = ss.str();
Alternatively, you may also use a istreambuf_iterator. You will have to measure whether this or the above way is faster - i don't know.
std::string s((istreambuf_iterator<char>(someStreamBuf)),
istreambuf_iterator<char>());
Note that someStreamBuf above is meant to represent a streambuf*, so take its address as appropriate. Also note the additional parentheses around the first argument in the last example, so that it doesn't interpret it as a function declaration returning a string and taking an iterator and another function pointer ("most vexing parse").
It's really buried in the docs...
Given boost::asio::streambuf b, with size_t buf_size ...
boost::asio::streambuf::const_buffers_type bufs = b.data();
std::string str(boost::asio::buffers_begin(bufs),
boost::asio::buffers_begin(bufs) + buf_size);
Another possibility with boost::asio::streambuf is to use boost::asio::buffer_cast<const char*>() in conjunction with boost::asio::streambuf::data() and boost::asio::streambuf::consume() like this:
const char* header=boost::asio::buffer_cast<const char*>(readbuffer.data());
//Do stuff with header, maybe construct a std::string with std::string(header,header+length)
readbuffer.consume(length);
This won't work with normal streambufs and might be considered dirty, but it seems to be the fastest way of doing it.
For boost::asio::streambuf you may find a solution like this:
boost::asio::streambuf buf;
/*put data into buf*/
std::istream is(&buf);
std::string line;
std::getline(is, line);
Print out the string :
std::cout << line << std::endl;
You may find here: http://www.boost.org/doc/libs/1_49_0/doc/html/boost_asio/reference/async_read_until/overload3.html
One can also obtain the characters from asio::streambuf using std::basic_streambuf::sgetn:
asio::streambuf in;
// ...
char cbuf[in.size()+1]; int rc = in.sgetn (cbuf, sizeof cbuf); cbuf[rc] = 0;
std::string str (cbuf, rc);
The reason you can only create const_buffer from std::string is because std::string explicitly doesn't support direct pointer-based writing in its contract. You could do something evil like resize your string to a certain size, then const_cast the constness from c_str() and treat it like a raw char* buffer, but that's very naughty and will get you in trouble someday.
I use std::vector for my buffers because as long as the vector doesn't resize (or you are careful to deal with resizing), you can do direct pointer writing just fine. If I need some of the data as a std::string, I have to copy it out, but the way I deal with my read buffers, anything that needs to last beyond the read callback needs to be copied out regardless.
I didn't see an existing answer for reading exactly n chars into a std::stringstream, so here is how that can be done:
std::stringstream ss;
boost::asio::streambuf sb;
const auto len = 10;
std::copy_n(boost::asio::buffers_begin(sb.data()), len,
std::ostream_iterator<decltype(ss)::char_type>(ss));
Compiler explorer
A simpler answer would be to convert it in std::string and manipulate it some what like this
std::string buffer_to_string(const boost::asio::streambuf &buffer)
{
using boost::asio::buffers_begin;
auto bufs = buffer.data();
std::string result(buffers_begin(bufs), buffers_begin(bufs) + buffer.size());
return result;
}
Giving a very concise code for the task.
I mostly don't like answers that say "You don't want X, you want Y instead and here's how to do Y" but in this instance I'm pretty sure I know what tstenner wanted.
In Boost 1.66, the dynamic string buffer type was added so async_read can directly resize and write to a string buffer.
I tested the first answer and got a compiler error when compiling using "g++ -std=c++11"
What worked for me was:
#include <string>
#include <boost/asio.hpp>
#include <sstream>
//other code ...
boost::asio::streambuf response;
//more code
std::ostringstream sline;
sline << &response; //need '&' or you a compiler error
std::string line = sline.str();
This compiled and ran.
I think it's more like:
streambuf.commit( number_of_bytes_read );
istream istr( &streambuf );
string s;
istr >> s;
I haven't looked into the basic_streambuf code, but I believe that should be just one copy into the string.
Im trying to create a binary file in the following way:
string buf;
...
buf += filename.length();
buf += filename;
etc. So first i give the length in binary format, but how do i convert this into a 4 byte char array, or 2 byte etc? basically i want to achieve the same functionality as this would:
int len = filename.length();
fwrite(&len, sizeof(len), 1, fp);
Which works fine, but having it in one string might be easier to process.
Edit: i dont want to use streams, nor vector, im trying to find out if its possible with strings.
Streams are the way to go. Not strings.
Use a vector for holding the data, or write it straight to the file (via streams)
simply use std:vector<unsigned char> and use a istream or ostream iterator to read/write data to/from the vector. For instance to read from a file you can do:
vector<unsigned char> binary_buffer;
ifstream in_file("my_binary_file.bin", ios_base::binary | ios_base::in);
istream_iterator<unsigned char> end_of_file;
istream_iterator<unsigned char> in_file_iter(in_file);
while (in_file_iter != end_of_file)
{
binary_buffer.push_back(*in_file_iter++);
}
Output would be even simpler:
ofstream out_file("another_binary_file.bin", ios_base::binary | ios_base::out);
ostream_iterator<unsigned char> binary_output(out_file);
copy(binary_buffer.begin(), binary_buffer.end(), binary_output);
Yes, it is possible to do this, because this is C++, and everything is possible in C++.
First, here is how you do it, then I'll answer the question of why you might need to do it:
std::ifstream input( "my.png", std::ios::binary );
std::vector<unsigned char> buffer(std::istreambuf_iterator<char>(input), {});
int buffSize = buffer.size();
std::string myString(buffer.begin(), buffer.end());
In my case, I was using a framework where the HTTP Client only supported a string for post message body, but I needed to post raw binary of a file to the service I was using. I could either mess around with the internals of the library (bad) or introduce another unnecessary dependency for a simple file post (also not so good).
But since strings can hold this data, I was able to read the file, copy it to a string and then upload.
This is an unfortunate situation to be in, either way, but in some cases it is helpful to be able to do. Usually you would want to do something different. This isn't the clearest for someone else to read, and you will have some performance penalties from the copy, but it works; and it helps to understand the internals of why it works. std::string saves the data contiguously internally in binary format, like vector does, and can be initialized from iterators in the vector.
unsigned char is one byte long, so it is like a byte value. The string copies the bytes into itself, so you end up with the exact data.
Hello! My problem can be described the following way:
I have some data which actually is an array and could be represented as char* data with some size
I also have some legacy code (function) that takes some abstract std::istream object as a param and uses that stream to retrieve data to operate.
So, my question is the following - what would be the easy way to map my data to some std::istream object so that I can pass it to my function? I thought about creating a std::stringstream object from my data, but that means copying and (as I assume) isn't the best solution.
Any ideas how this could be done so that my std::istream operates on the data directly?
Thank you.
If you're looking at actually creating your own stream, I'd look at the Boost.Iostreams library. It makes it easy to create your own stream objects.
Definitely not the easiest way but just in case someone wanted to understand how std streams work inside, this seems to be a very nice introduction about how you can roll your own:
http://www.mr-edd.co.uk/blog/beginners_guide_streambuf
Use string stream:
#include <sstream>
int main()
{
char[] data = "PLOP PLOP PLOP";
int size = 13; // PS I know this is not the same as strlen(data);
std::stringstream stream(std::string(data, size));
// use stream as an istream;
}
If you want to be real effecient you can muck with the stream buffer directly. I have not tried this and do not have a compiler to test with, but the folowing should work:
#include <sstream>
int main()
{
char[] data = "PLOP PLOP PLOP";
int size = 13; // PS I know this is not the same as strlen(data);
std::stringstream stream;
stream.rdbuf()->pubsetbuf(data, size);
// use stream as an istream;
}