Apparently boost::asio::async_read doesn't like strings, as the only overload of boost::asio::buffer allows me to create const_buffers, so I'm stuck with reading everything into a streambuf.
Now I want to copy the contents of the streambuf into a string, but it apparently only supports writing to char* (sgetn()), creating an istream with the streambuf and using getline().
Is there any other way to create a string with the streambufs contents without excessive copying?
I don't know whether it counts as "excessive copying", but you can use a stringstream:
std::ostringstream ss;
ss << someStreamBuf;
std::string s = ss.str();
Like, to read everything from stdin into a string, do
std::ostringstream ss;
ss << std::cin.rdbuf();
std::string s = ss.str();
Alternatively, you may also use a istreambuf_iterator. You will have to measure whether this or the above way is faster - i don't know.
std::string s((istreambuf_iterator<char>(someStreamBuf)),
istreambuf_iterator<char>());
Note that someStreamBuf above is meant to represent a streambuf*, so take its address as appropriate. Also note the additional parentheses around the first argument in the last example, so that it doesn't interpret it as a function declaration returning a string and taking an iterator and another function pointer ("most vexing parse").
It's really buried in the docs...
Given boost::asio::streambuf b, with size_t buf_size ...
boost::asio::streambuf::const_buffers_type bufs = b.data();
std::string str(boost::asio::buffers_begin(bufs),
boost::asio::buffers_begin(bufs) + buf_size);
Another possibility with boost::asio::streambuf is to use boost::asio::buffer_cast<const char*>() in conjunction with boost::asio::streambuf::data() and boost::asio::streambuf::consume() like this:
const char* header=boost::asio::buffer_cast<const char*>(readbuffer.data());
//Do stuff with header, maybe construct a std::string with std::string(header,header+length)
readbuffer.consume(length);
This won't work with normal streambufs and might be considered dirty, but it seems to be the fastest way of doing it.
For boost::asio::streambuf you may find a solution like this:
boost::asio::streambuf buf;
/*put data into buf*/
std::istream is(&buf);
std::string line;
std::getline(is, line);
Print out the string :
std::cout << line << std::endl;
You may find here: http://www.boost.org/doc/libs/1_49_0/doc/html/boost_asio/reference/async_read_until/overload3.html
One can also obtain the characters from asio::streambuf using std::basic_streambuf::sgetn:
asio::streambuf in;
// ...
char cbuf[in.size()+1]; int rc = in.sgetn (cbuf, sizeof cbuf); cbuf[rc] = 0;
std::string str (cbuf, rc);
The reason you can only create const_buffer from std::string is because std::string explicitly doesn't support direct pointer-based writing in its contract. You could do something evil like resize your string to a certain size, then const_cast the constness from c_str() and treat it like a raw char* buffer, but that's very naughty and will get you in trouble someday.
I use std::vector for my buffers because as long as the vector doesn't resize (or you are careful to deal with resizing), you can do direct pointer writing just fine. If I need some of the data as a std::string, I have to copy it out, but the way I deal with my read buffers, anything that needs to last beyond the read callback needs to be copied out regardless.
I didn't see an existing answer for reading exactly n chars into a std::stringstream, so here is how that can be done:
std::stringstream ss;
boost::asio::streambuf sb;
const auto len = 10;
std::copy_n(boost::asio::buffers_begin(sb.data()), len,
std::ostream_iterator<decltype(ss)::char_type>(ss));
Compiler explorer
A simpler answer would be to convert it in std::string and manipulate it some what like this
std::string buffer_to_string(const boost::asio::streambuf &buffer)
{
using boost::asio::buffers_begin;
auto bufs = buffer.data();
std::string result(buffers_begin(bufs), buffers_begin(bufs) + buffer.size());
return result;
}
Giving a very concise code for the task.
I mostly don't like answers that say "You don't want X, you want Y instead and here's how to do Y" but in this instance I'm pretty sure I know what tstenner wanted.
In Boost 1.66, the dynamic string buffer type was added so async_read can directly resize and write to a string buffer.
I tested the first answer and got a compiler error when compiling using "g++ -std=c++11"
What worked for me was:
#include <string>
#include <boost/asio.hpp>
#include <sstream>
//other code ...
boost::asio::streambuf response;
//more code
std::ostringstream sline;
sline << &response; //need '&' or you a compiler error
std::string line = sline.str();
This compiled and ran.
I think it's more like:
streambuf.commit( number_of_bytes_read );
istream istr( &streambuf );
string s;
istr >> s;
I haven't looked into the basic_streambuf code, but I believe that should be just one copy into the string.
Related
This question already has answers here:
How do I read an entire file into a std::string in C++?
(23 answers)
Closed 6 years ago.
I need to read a whole file into memory and place it in a C++ std::string.
If I were to read it into a char[], the answer would be very simple:
std::ifstream t;
int length;
t.open("file.txt"); // open input file
t.seekg(0, std::ios::end); // go to the end
length = t.tellg(); // report location (this is the length)
t.seekg(0, std::ios::beg); // go back to the beginning
buffer = new char[length]; // allocate memory for a buffer of appropriate dimension
t.read(buffer, length); // read the whole file into the buffer
t.close(); // close file handle
// ... Do stuff with buffer here ...
Now, I want to do the exact same thing, but using a std::string instead of a char[]. I want to avoid loops, i.e. I don't want to:
std::ifstream t;
t.open("file.txt");
std::string buffer;
std::string line;
while(t){
std::getline(t, line);
// ... Append line to buffer and go on
}
t.close()
Any ideas?
There are a couple of possibilities. One I like uses a stringstream as a go-between:
std::ifstream t("file.txt");
std::stringstream buffer;
buffer << t.rdbuf();
Now the contents of "file.txt" are available in a string as buffer.str().
Another possibility (though I certainly don't like it as well) is much more like your original:
std::ifstream t("file.txt");
t.seekg(0, std::ios::end);
size_t size = t.tellg();
std::string buffer(size, ' ');
t.seekg(0);
t.read(&buffer[0], size);
Officially, this isn't required to work under the C++98 or 03 standard (string isn't required to store data contiguously) but in fact it works with all known implementations, and C++11 and later do require contiguous storage, so it's guaranteed to work with them.
As to why I don't like the latter as well: first, because it's longer and harder to read. Second, because it requires that you initialize the contents of the string with data you don't care about, then immediately write over that data (yes, the time to initialize is usually trivial compared to the reading, so it probably doesn't matter, but to me it still feels kind of wrong). Third, in a text file, position X in the file doesn't necessarily mean you'll have read X characters to reach that point -- it's not required to take into account things like line-end translations. On real systems that do such translations (e.g., Windows) the translated form is shorter than what's in the file (i.e., "\r\n" in the file becomes "\n" in the translated string) so all you've done is reserved a little extra space you never use. Again, doesn't really cause a major problem but feels a little wrong anyway.
Update: Turns out that this method, while following STL idioms well, is actually surprisingly inefficient! Don't do this with large files. (See: http://insanecoding.blogspot.com/2011/11/how-to-read-in-file-in-c.html)
You can make a streambuf iterator out of the file and initialize the string with it:
#include <string>
#include <fstream>
#include <streambuf>
std::ifstream t("file.txt");
std::string str((std::istreambuf_iterator<char>(t)),
std::istreambuf_iterator<char>());
Not sure where you're getting the t.open("file.txt", "r") syntax from. As far as I know that's not a method that std::ifstream has. It looks like you've confused it with C's fopen.
Edit: Also note the extra parentheses around the first argument to the string constructor. These are essential. They prevent the problem known as the "most vexing parse", which in this case won't actually give you a compile error like it usually does, but will give you interesting (read: wrong) results.
Following KeithB's point in the comments, here's a way to do it that allocates all the memory up front (rather than relying on the string class's automatic reallocation):
#include <string>
#include <fstream>
#include <streambuf>
std::ifstream t("file.txt");
std::string str;
t.seekg(0, std::ios::end);
str.reserve(t.tellg());
t.seekg(0, std::ios::beg);
str.assign((std::istreambuf_iterator<char>(t)),
std::istreambuf_iterator<char>());
I think best way is to use string stream. simple and quick !!!
#include <fstream>
#include <iostream>
#include <sstream> //std::stringstream
int main() {
std::ifstream inFile;
inFile.open("inFileName"); //open the input file
std::stringstream strStream;
strStream << inFile.rdbuf(); //read the file
std::string str = strStream.str(); //str holds the content of the file
std::cout << str << "\n"; //you can do anything with the string!!!
}
You may not find this in any book or site, but I found out that it works pretty well:
#include <fstream>
// ...
std::string file_content;
std::getline(std::ifstream("filename.txt"), file_content, '\0');
Try one of these two methods:
string get_file_string(){
std::ifstream ifs("path_to_file");
return string((std::istreambuf_iterator<char>(ifs)),
(std::istreambuf_iterator<char>()));
}
string get_file_string2(){
ifstream inFile;
inFile.open("path_to_file");//open the input file
stringstream strStream;
strStream << inFile.rdbuf();//read the file
return strStream.str();//str holds the content of the file
}
I figured out another way that works with most istreams, including std::cin!
std::string readFile()
{
stringstream str;
ifstream stream("Hello_World.txt");
if(stream.is_open())
{
while(stream.peek() != EOF)
{
str << (char) stream.get();
}
stream.close();
return str.str();
}
}
If you happen to use glibmm you can try Glib::file_get_contents.
#include <iostream>
#include <glibmm.h>
int main() {
auto filename = "my-file.txt";
try {
std::string contents = Glib::file_get_contents(filename);
std::cout << "File data:\n" << contents << std::endl;
catch (const Glib::FileError& e) {
std::cout << "Oops, an error occurred:\n" << e.what() << std::endl;
}
return 0;
}
I could do it like this:
void readfile(const std::string &filepath,std::string &buffer){
std::ifstream fin(filepath.c_str());
getline(fin, buffer, char(-1));
fin.close();
}
If this is something to be frowned upon, please let me know why
I don't think you can do this without an explicit or implicit loop, without reading into a char array (or some other container) first and ten constructing the string. If you don't need the other capabilities of a string, it could be done with vector<char> the same way you are currently using a char *.
I was trying out the below snippet but it is not giving the desired output:
#include<iostream>
#include<sstream>
using namespace std;
void MyPrint(ostream& stream)
{
cout<<stream.rdbuf()<< endl;
}
int main()
{
stringstream ss;
ss<<"hello there";
MyPrint(ss); //Prints fine
ostringstream oss;
oss<<"hello there";
MyPrint(oss); //Does not print anything
getchar();
}
I am aware that the only possible differences between stringstream and ostringstream is that the later forces the direction and is a bit faster than stringstream.
Am I missing out on anything?
PS: A similar question was posted earlier but didn't get any answers.
std::stringstream and std::ostringstream pass different
flags to the std::stringbuf. In particular, the
std::stringbuf of an std::ostringstream does not support
reading. And std::cout << stream.rdbuf() is a read operation
on the streambuf.
The way to extract characters from an std::ostringstream is by
using the std::ostringstream::str() function.
stringstream shouldn't be thought of as a bidirectional implementation of ostringstream and istringstream. It is implemented as a derrived class of both ostringstream and istringstream, which is why it implements both input and output capabilities.
Choosing which one to use depends on what it is used for. If you only need to write data to it on a stream without being able to access the data via a stream, then an ostringstream is all you would need. However, if you want to implement bidirectional on something you provide to an API but restrict it, you can cast it:
stringstream ss; // My bidirectional stream
ostringstream *p_os = &ss; // Now an output stream to be passed to something only allowed to write to it.
int bytes = collectSomeData(p_oss);
This question already has answers here:
How do I read an entire file into a std::string in C++?
(23 answers)
Closed 6 years ago.
I need to read a whole file into memory and place it in a C++ std::string.
If I were to read it into a char[], the answer would be very simple:
std::ifstream t;
int length;
t.open("file.txt"); // open input file
t.seekg(0, std::ios::end); // go to the end
length = t.tellg(); // report location (this is the length)
t.seekg(0, std::ios::beg); // go back to the beginning
buffer = new char[length]; // allocate memory for a buffer of appropriate dimension
t.read(buffer, length); // read the whole file into the buffer
t.close(); // close file handle
// ... Do stuff with buffer here ...
Now, I want to do the exact same thing, but using a std::string instead of a char[]. I want to avoid loops, i.e. I don't want to:
std::ifstream t;
t.open("file.txt");
std::string buffer;
std::string line;
while(t){
std::getline(t, line);
// ... Append line to buffer and go on
}
t.close()
Any ideas?
There are a couple of possibilities. One I like uses a stringstream as a go-between:
std::ifstream t("file.txt");
std::stringstream buffer;
buffer << t.rdbuf();
Now the contents of "file.txt" are available in a string as buffer.str().
Another possibility (though I certainly don't like it as well) is much more like your original:
std::ifstream t("file.txt");
t.seekg(0, std::ios::end);
size_t size = t.tellg();
std::string buffer(size, ' ');
t.seekg(0);
t.read(&buffer[0], size);
Officially, this isn't required to work under the C++98 or 03 standard (string isn't required to store data contiguously) but in fact it works with all known implementations, and C++11 and later do require contiguous storage, so it's guaranteed to work with them.
As to why I don't like the latter as well: first, because it's longer and harder to read. Second, because it requires that you initialize the contents of the string with data you don't care about, then immediately write over that data (yes, the time to initialize is usually trivial compared to the reading, so it probably doesn't matter, but to me it still feels kind of wrong). Third, in a text file, position X in the file doesn't necessarily mean you'll have read X characters to reach that point -- it's not required to take into account things like line-end translations. On real systems that do such translations (e.g., Windows) the translated form is shorter than what's in the file (i.e., "\r\n" in the file becomes "\n" in the translated string) so all you've done is reserved a little extra space you never use. Again, doesn't really cause a major problem but feels a little wrong anyway.
Update: Turns out that this method, while following STL idioms well, is actually surprisingly inefficient! Don't do this with large files. (See: http://insanecoding.blogspot.com/2011/11/how-to-read-in-file-in-c.html)
You can make a streambuf iterator out of the file and initialize the string with it:
#include <string>
#include <fstream>
#include <streambuf>
std::ifstream t("file.txt");
std::string str((std::istreambuf_iterator<char>(t)),
std::istreambuf_iterator<char>());
Not sure where you're getting the t.open("file.txt", "r") syntax from. As far as I know that's not a method that std::ifstream has. It looks like you've confused it with C's fopen.
Edit: Also note the extra parentheses around the first argument to the string constructor. These are essential. They prevent the problem known as the "most vexing parse", which in this case won't actually give you a compile error like it usually does, but will give you interesting (read: wrong) results.
Following KeithB's point in the comments, here's a way to do it that allocates all the memory up front (rather than relying on the string class's automatic reallocation):
#include <string>
#include <fstream>
#include <streambuf>
std::ifstream t("file.txt");
std::string str;
t.seekg(0, std::ios::end);
str.reserve(t.tellg());
t.seekg(0, std::ios::beg);
str.assign((std::istreambuf_iterator<char>(t)),
std::istreambuf_iterator<char>());
I think best way is to use string stream. simple and quick !!!
#include <fstream>
#include <iostream>
#include <sstream> //std::stringstream
int main() {
std::ifstream inFile;
inFile.open("inFileName"); //open the input file
std::stringstream strStream;
strStream << inFile.rdbuf(); //read the file
std::string str = strStream.str(); //str holds the content of the file
std::cout << str << "\n"; //you can do anything with the string!!!
}
You may not find this in any book or site, but I found out that it works pretty well:
#include <fstream>
// ...
std::string file_content;
std::getline(std::ifstream("filename.txt"), file_content, '\0');
Try one of these two methods:
string get_file_string(){
std::ifstream ifs("path_to_file");
return string((std::istreambuf_iterator<char>(ifs)),
(std::istreambuf_iterator<char>()));
}
string get_file_string2(){
ifstream inFile;
inFile.open("path_to_file");//open the input file
stringstream strStream;
strStream << inFile.rdbuf();//read the file
return strStream.str();//str holds the content of the file
}
I figured out another way that works with most istreams, including std::cin!
std::string readFile()
{
stringstream str;
ifstream stream("Hello_World.txt");
if(stream.is_open())
{
while(stream.peek() != EOF)
{
str << (char) stream.get();
}
stream.close();
return str.str();
}
}
If you happen to use glibmm you can try Glib::file_get_contents.
#include <iostream>
#include <glibmm.h>
int main() {
auto filename = "my-file.txt";
try {
std::string contents = Glib::file_get_contents(filename);
std::cout << "File data:\n" << contents << std::endl;
catch (const Glib::FileError& e) {
std::cout << "Oops, an error occurred:\n" << e.what() << std::endl;
}
return 0;
}
I could do it like this:
void readfile(const std::string &filepath,std::string &buffer){
std::ifstream fin(filepath.c_str());
getline(fin, buffer, char(-1));
fin.close();
}
If this is something to be frowned upon, please let me know why
I don't think you can do this without an explicit or implicit loop, without reading into a char array (or some other container) first and ten constructing the string. If you don't need the other capabilities of a string, it could be done with vector<char> the same way you are currently using a char *.
I'm trying to read an entire stream (multiple lines) into a string.
I'm using this code, and it works, but it's offending my sense of style... Surely there's an easier way? Maybe using stringstreams?
void Obj::loadFromStream(std::istream & stream)
{
std::string s;
std::streampos p = stream.tellg(); // remember where we are
stream.seekg(0, std::ios_base::end); // go to the end
std::streamoff sz = stream.tellg() - p; // work out the size
stream.seekg(p); // restore the position
s.resize(sz); // resize the string
stream.read(&s[0], sz); // and finally, read in the data.
Actually, a const reference to a string would do as well, and that may make things easier...
const std::string &s(... a miracle occurs here...)
How about
std::istreambuf_iterator<char> eos;
std::string s(std::istreambuf_iterator<char>(stream), eos);
(could be a one-liner if not for MVP)
post-2011 edit, this approach is now spelled
std::string s(std::istreambuf_iterator<char>(stream), {});
I'm late to the party, but here is a fairly efficient solution:
std::string gulp(std::istream &in)
{
std::string ret;
char buffer[4096];
while (in.read(buffer, sizeof(buffer)))
ret.append(buffer, sizeof(buffer));
ret.append(buffer, in.gcount());
return ret;
}
I did some benchmarking, and it turns out that the std::istreambuf_iterator technique (used by the accepted answer) is actually much slower. On gcc 4.4.5 with -O3, it's about a 4.5x difference on my machine, and the gap becomes wider with lower optimization settings.
You could do
std::string s;
std::ostringstream os;
os<<stream.rdbuf();
s=os.str();
but I don't know if it's more efficient.
Alternative version:
std::string s;
std::ostringstream os;
stream>>os.rdbuf();
s=os.str();
You can try using something from algorithms. I have to get ready for work but here's a very quick stab at things (there's got to be a better way):
copy( istreambuf_iterator<char>(stream), istreambuf_iterator<char>(), back_inserter(s) );
Well, if you are looking for a simple and 'readable' way to do it. I would recomend add/use some high level framework on your project. For that I's always use Poco and Boost on all my projects. In this case, with Poco:
string text;
FileStream fstream(TEXT_FILE_PATH);
StreamCopier::copyToString(fstream, text);
Perhaps this 1 line C++11 solution:
std::vector<char> s{std::istreambuf_iterator<char>{in},{}};
What about to use getline with delimiter? The next code helps me to read whole std::cin into string on ubuntu with g++-10.
#include <iostream>
#include <string>
using namespace std;
int main() {
string s;
getline(cin, s, {}); //the whole stream into variable s
return 0;
}
What is the most optimal way to get a string or char* pointer into an istream.
I want to do the following
std::string a = "abc..";
//I know this can be done, not sure if this is most efficient
// and not sure about char* pointers
std::istringstream istr (a);
...
foo (istr);
void foo(std::istream& is) {
}
If you want to construct an istringstream from it, a char* up to the null character, or all the stuff from an std::string:
istringstream str(ptr); // char*
istringstream str(other_str); // std::string
If you talk about wanting a raw pointer into the buffer of an istream, you can't do it. Streams get their data on-demand if they need them either from files, terminals or else, optionally buffering their stuff (well, not exactly right. You can use a strstream, which accepts a raw pointer and reads/writes directly from that. But it's a deprecated class - don't use it. I'm lucky i've never done so). If all you want is something you can use somewhat like a pointer, you can use streambuf iterators. They are not really pointers though, so you can't subtract end from begin and other stuffs:
std::istreambuf_iterator<char> begin(one_istream), end;
while(begin != end)
std::cout << *begin++;
If you talk about getting a string out of what was written into a stringstream, you can use ostringstream::str:
ostringstream o;
o << "This is a number: " << 42;
std::string str = o.str(); // str == "This is a number: 42"
Otherwise, you can only generally read stuff from an istream. You need an ostream, then you can do
stream.write(ptr, N);
stream.write(ptr.c_str(), ptr.c_str() + ptr.size());
to write exactly N characters from the bytes that str points to. You can write it into the stream using << too. It will write everything up to the null character, or everything from an std::string, but will respect formatting flags, like the field width:
stream << ptr; // char*
stream << other_str; // everything from std::string
This will work:
std::istringstream is("abc...");
And since istringstream is a istream, you will be able to use your is object as an istream.