Hello! My problem can be described the following way:
I have some data which actually is an array and could be represented as char* data with some size
I also have some legacy code (function) that takes some abstract std::istream object as a param and uses that stream to retrieve data to operate.
So, my question is the following - what would be the easy way to map my data to some std::istream object so that I can pass it to my function? I thought about creating a std::stringstream object from my data, but that means copying and (as I assume) isn't the best solution.
Any ideas how this could be done so that my std::istream operates on the data directly?
Thank you.
If you're looking at actually creating your own stream, I'd look at the Boost.Iostreams library. It makes it easy to create your own stream objects.
Definitely not the easiest way but just in case someone wanted to understand how std streams work inside, this seems to be a very nice introduction about how you can roll your own:
http://www.mr-edd.co.uk/blog/beginners_guide_streambuf
Use string stream:
#include <sstream>
int main()
{
char[] data = "PLOP PLOP PLOP";
int size = 13; // PS I know this is not the same as strlen(data);
std::stringstream stream(std::string(data, size));
// use stream as an istream;
}
If you want to be real effecient you can muck with the stream buffer directly. I have not tried this and do not have a compiler to test with, but the folowing should work:
#include <sstream>
int main()
{
char[] data = "PLOP PLOP PLOP";
int size = 13; // PS I know this is not the same as strlen(data);
std::stringstream stream;
stream.rdbuf()->pubsetbuf(data, size);
// use stream as an istream;
}
Related
I am trying to find the most efficient way of converting an std::ostream to a std::vector<uint8_t>. I could obviously convert to string first but I am trying to avoid extra data copies. Is there a nice way to do this? I have been looking at the rdbuf of ostream and think that it might be doable using that. Not sure how to proceed though. Any advice?
Assuming you mistake ostream with istream (because you're reading input into the vector) you're probaply looking for istream_iterator. You can pass them to the constructor of vector:
std::istringstream str("5 16 32 8");
std::vector<uint8_t> numbers{ std::istream_iterator<int>(str), std::istream_iterator<int>() };
Note that you have to pass int as argument to the iterator, because operator<< is not defined for uint8_t.
Assuming you mean istream (input) and not ostream (output), then istream has a read() method for reading input into a user-provided buffer. So, simply pre-allocate your vector to the desired size and then read() directly into its data buffer, eg:
std::vector<uint8_t> buffer(size);
stream.read(reinterpret_cast<char*>(buffer.data()), size);
If you would like to read file contents without parsing:
#include <vector>
#include <iostream>
#include <cstdint>
#include <iterator>
std::vector<uint8_t> read_data(std::istream& s) {
return std::vector<uint8_t>(std::istreambuf_iterator<char>{s},
std::istreambuf_iterator<char>{});
}
int main() {
auto v = read_data(std::cin);
}
The drawback is that you cannot specify the maximum size to read. If that is a requirement (a public API that you want to protect from unfriendly clients), use istream::read with a fixed size, as Remy Lebeau advises.
If the file is large (e.g. megabytes or more), you may like to map directly into memory to avoid large data copies and memory reallocations.
I have some data in a buffer pointed to by a const char* pointer. The data is just an ASCII string. I know its size. I would like to be able to read it in the same way data is read from streams. I'm looking for a solution that would allow me to write code like this:
// for example, data points to a string "42 3.14 blah"
MemoryStreamWrapper in(data, data_size);
int x;
float y;
std::string w;
in >> x >> y >> w;
Important condition: the data must not be copied or altered in any way (otherwise I'd just use a string stream. To my best knowledge, it isn't possible to create a string stream from a const char pointer without copying the data.)
The way to do this is to create a suitable stream buffer. This can, e.g., be done like this:
#include <streambuf>
#include <istream>
struct membuf: std::streambuf {
membuf(char const* base, size_t size) {
char* p(const_cast<char*>(base));
this->setg(p, p, p + size);
}
};
struct imemstream: virtual membuf, std::istream {
imemstream(char const* base, size_t size)
: membuf(base, size)
, std::istream(static_cast<std::streambuf*>(this)) {
}
};
The only somewhat awkward thing is the const_cast<char*>() in the stream buffer: the stream buffer won't change the data but the interface still requires char* to be used, mainly to make it easier to change the buffer in "normal" stream buffers. With this, you can use imemstream as a normal input stream:
imemstream in(data, size);
in >> value;
The only way would be to subclass std::istream (which also requires subclassing std::streambuf) to create your own stream class that reads from constant memory.
It's not as easy as it sounds because the the C++ standard library stream classes are pretty messy and badly designed. I don't think it's worth it unless you need it to scale a lot.
Apparently boost::asio::async_read doesn't like strings, as the only overload of boost::asio::buffer allows me to create const_buffers, so I'm stuck with reading everything into a streambuf.
Now I want to copy the contents of the streambuf into a string, but it apparently only supports writing to char* (sgetn()), creating an istream with the streambuf and using getline().
Is there any other way to create a string with the streambufs contents without excessive copying?
I don't know whether it counts as "excessive copying", but you can use a stringstream:
std::ostringstream ss;
ss << someStreamBuf;
std::string s = ss.str();
Like, to read everything from stdin into a string, do
std::ostringstream ss;
ss << std::cin.rdbuf();
std::string s = ss.str();
Alternatively, you may also use a istreambuf_iterator. You will have to measure whether this or the above way is faster - i don't know.
std::string s((istreambuf_iterator<char>(someStreamBuf)),
istreambuf_iterator<char>());
Note that someStreamBuf above is meant to represent a streambuf*, so take its address as appropriate. Also note the additional parentheses around the first argument in the last example, so that it doesn't interpret it as a function declaration returning a string and taking an iterator and another function pointer ("most vexing parse").
It's really buried in the docs...
Given boost::asio::streambuf b, with size_t buf_size ...
boost::asio::streambuf::const_buffers_type bufs = b.data();
std::string str(boost::asio::buffers_begin(bufs),
boost::asio::buffers_begin(bufs) + buf_size);
Another possibility with boost::asio::streambuf is to use boost::asio::buffer_cast<const char*>() in conjunction with boost::asio::streambuf::data() and boost::asio::streambuf::consume() like this:
const char* header=boost::asio::buffer_cast<const char*>(readbuffer.data());
//Do stuff with header, maybe construct a std::string with std::string(header,header+length)
readbuffer.consume(length);
This won't work with normal streambufs and might be considered dirty, but it seems to be the fastest way of doing it.
For boost::asio::streambuf you may find a solution like this:
boost::asio::streambuf buf;
/*put data into buf*/
std::istream is(&buf);
std::string line;
std::getline(is, line);
Print out the string :
std::cout << line << std::endl;
You may find here: http://www.boost.org/doc/libs/1_49_0/doc/html/boost_asio/reference/async_read_until/overload3.html
One can also obtain the characters from asio::streambuf using std::basic_streambuf::sgetn:
asio::streambuf in;
// ...
char cbuf[in.size()+1]; int rc = in.sgetn (cbuf, sizeof cbuf); cbuf[rc] = 0;
std::string str (cbuf, rc);
The reason you can only create const_buffer from std::string is because std::string explicitly doesn't support direct pointer-based writing in its contract. You could do something evil like resize your string to a certain size, then const_cast the constness from c_str() and treat it like a raw char* buffer, but that's very naughty and will get you in trouble someday.
I use std::vector for my buffers because as long as the vector doesn't resize (or you are careful to deal with resizing), you can do direct pointer writing just fine. If I need some of the data as a std::string, I have to copy it out, but the way I deal with my read buffers, anything that needs to last beyond the read callback needs to be copied out regardless.
I didn't see an existing answer for reading exactly n chars into a std::stringstream, so here is how that can be done:
std::stringstream ss;
boost::asio::streambuf sb;
const auto len = 10;
std::copy_n(boost::asio::buffers_begin(sb.data()), len,
std::ostream_iterator<decltype(ss)::char_type>(ss));
Compiler explorer
A simpler answer would be to convert it in std::string and manipulate it some what like this
std::string buffer_to_string(const boost::asio::streambuf &buffer)
{
using boost::asio::buffers_begin;
auto bufs = buffer.data();
std::string result(buffers_begin(bufs), buffers_begin(bufs) + buffer.size());
return result;
}
Giving a very concise code for the task.
I mostly don't like answers that say "You don't want X, you want Y instead and here's how to do Y" but in this instance I'm pretty sure I know what tstenner wanted.
In Boost 1.66, the dynamic string buffer type was added so async_read can directly resize and write to a string buffer.
I tested the first answer and got a compiler error when compiling using "g++ -std=c++11"
What worked for me was:
#include <string>
#include <boost/asio.hpp>
#include <sstream>
//other code ...
boost::asio::streambuf response;
//more code
std::ostringstream sline;
sline << &response; //need '&' or you a compiler error
std::string line = sline.str();
This compiled and ran.
I think it's more like:
streambuf.commit( number_of_bytes_read );
istream istr( &streambuf );
string s;
istr >> s;
I haven't looked into the basic_streambuf code, but I believe that should be just one copy into the string.
Every one know stringstream.str() need a string variable type to store the content of stringstream.str() into it .
I want to store the content of stringstream.str() into char variable or char array or pointer.
Is it possible to do that?
Please, write a simple example with your answer.
Why not just
std::string s = stringstream.str();
const char* p = s.c_str();
?
Edit: Note that you cannot freely give the p outside your function: its lifetime is bound to the lifetime of s, so you may want to copy it.
Edit 2: as #David suggests, copy above means copying of the content, not the pointer itself. There are several ways for that. You can either do it manually (legacy way "inherited" from C) -- this is done with the functions like std::strcpy. This way is quite complicated, since it involves manual resources management, which is usually discouraged, since it leads to a more complicated and error-prone code. Or you can use the smart pointers or containers: it can be either std::vector<char> or std::unique_ptr/std::shared_ptr.
I personally would go for the second way. See the discussion to this and #Oli's answer, it can be useful.
If you want to get the data into a char buffer, why not put it there immediately anyway? Here is a stream class which takes an array, determines its size, fills it with null characters (primarily to make sure the resulting string is null terminated), and then sets up an std::ostream to write to this buffer directly.
#include <iostream>
#include <algorithm>
struct membuf: public std::streambuf {
template <size_t Size> membuf(char (&array)[Size]) {
this->setp(array, array + Size - 1);
std::fill_n(array, Size, 0);
}
};
struct omemstream: virtual membuf, std::ostream {
template <size_t Size> omemstream(char (&array)[Size]):
membuf(array),
std::ostream(this)
{
}
};
int main() {
char array[20];
omemstream out(array);
out << "hello, world";
std::cout << "the buffer contains '" << array << "'\n";
}
Obviously, this stream buffer and stream would probably live in a suitable namespace and would be implemented in some header (there isn't much point in putting anything of it into a C++ file because all the function are templates needing to instantiated). You could also use the [deprecated] class std::ostrstream to do something similar but it is so easy to create a custom stream that it may not worth bothering.
You can do this if you want an actual copy of the string (vital if the stringstream object is going to go out of scope at some point):
const char *p = new char[ss.str().size()+1];
strcpy(p, ss.str().c_str());
...
delete [] p;
As discussed in comments below, you should be wary of doing it like this (manual memory management is error-prone, and very non-idiomatic C++). Why do you want a raw char array?
I figured it out. Using namespace std and replacing tstingstreamwith stringstream. Next step is:
stringstream strstream;
strstream.imbue(std::locale("C"));
string str = strstream.str();
const char *sql= str .c_str();
Now you can execute sql statement.
sqlite3_exec(db, sql, callback, (void*)data, &zErrMsg);
Maybe it helps to somebody.
I am using the library libvtemm, which has a function write_contents. It takes an internal buffer and outputs it to a Glib::RefPtr<Gio::OutputStream> object. I have been trying to find a way to convert the contents of the Gio::OutputStream into a std::string or something similar so that I can play with and move around the data inside to other data structures.
Does anyone know how to either construct a Gio::OutputStream to something like a std::ostream or convert its contents into a std::string?
I see there is a Gio::MemoryOutputStream, would something like this be useful in grabbing the data to a std::ostream?
For those of you who are looking for an answer, here is what I have come up with to read the console buffer into a std::string.
// Create a mock stream just for this example
Glib::RefPtr<Gio::MemoryOutputStream> bufStream =
Gio::MemoryOutputStream::create(NULL, 0, &realloc, &free);
// Create the stringstream to use as an ostream
std::stringstream ss;
// Get the stream size so we know how much to allocate
gsize streamSize = bufStream->get_data_size();
char *charBuf = new char[streamSize+1];
// Copy over the data from the buffer to the charBuf
memcpy(charBuf, bufStream->get_data(), streamSize);
// Add the null terminator to the "string"
charBuf[streamSize] = '\0';
// Create a string from it
ss << charBuf;
Hope this helps someone in the future who comes across a similar problem.