pubsetbuf member of std::stringbuf is not working at all in Visual Studio 2010!
The code:
char *FileData = ... ;
unsigned long long FileDataLen = ... ;
std::stringstream *SS = new std::stringstream(std::stringstream::in | std::stringstream::out);
SS->rdbuf()->pubsetbuf( FileData, (std::streamsize)FileDataLen );
pubsetbuf does nothing in Visual Studio!!!
Workaround #1:
std::stringstream *SS = new std::stringstream( std::string(FileData, (size_type)FileDataLen ) ),std::stringstream::in | std::stringstream::out);
Workaround #2:
SS->rdbuf()->sputn(FileData, (streamsize)FileDataLen);
But both of these workarounds produce unnecessary memory copying.
I definitely need a working pubsetbuf member of std::stringbuf.
putsetbuf only makes sense for fstream (technically, for std::basic_filebuf), where the buffer and the stream are two different things.
For stringstream (technically, std::basic_stringbuf) they are one and the same std::string.
If you need a stream that works on a string that's external to it, consider std::strstream: or boost::iostreams::array_sink
basic_ios.clear()
If you need to change rdbuf, call this first or it'll fail to work.
std::ifstream file("file1.txt"); // file1.txt contains "Welcome!"
std::stringstream ss;
ss << file.rdbuf();
std::cout << ss.str() << std::endl;
Outputs "Welcome!".
Let's try again with a new file.
// Empty it
file.close();
ss.str("");
// New file
file.open("file2.txt"); // file2.txt contains "Goodbye!"
ss << file.rdbuf();
std::cout << ss.str() << std::endl;
Outputs nothing.
ss.clear();
ss << file.rdbuf();
std::cout << ss.str() << std::endl;
file.close();
Outputs "Goodbye!"
I see the same thing. I'm working on scenarios that definitely can't afford making unnecessary data copies. This seems to be somehow intentional as per comments in basic_streambuf class:
virtual basic_streambuf *__CLR_OR_THIS_CALL setbuf(_Elem *, streamsize)
{ // offer buffer to external agent (do nothing)
return (this);
}
I recently encountered the same issue of setbuf not being implemented in Visual Studio 2017.
After some searching on Stack Overflow, I found a solution for an output stream that doesn't use copying of the buffer which I modified for an input stream. Here they are for reference.
Output Stream
Source: Setting the internal buffer used by a standard stream (pubsetbuf)
#include <streambuf>
template <typename char_type>
struct ostreambuf : public std::basic_streambuf<char_type, std::char_traits<char_type> >
{
ostreambuf(char_type* buffer, std::streamsize bufferLength)
{
// set the "put" pointer the start of the buffer and record it's length.
setp(buffer, buffer + bufferLength);
}
};
Input Stream
Source: Internal buffer used by standard input stream (pubsetbuf)
#include <streambuf>
template <typename char_type>
struct istreambuf : public basic_streambuf<char_type, char_traits<char_type>>
{
istreambuf(char_type* buffer, streamsize buffer_length)
{
// Set the "get" pointer to the start of the buffer, the next item, and record its length.
this->setg(buffer, buffer, buffer + buffer_length);
}
};
int main()
{
ifstream infile(FILENAME, ifstream::binary);
// Read entire file into buffer.
infile.seekg(0, ios::end);
streampos length = infile.tellg();
infile.seekg(0, ios::beg);
vector<char> buffer(length);
//char* buffer = new char[length];
infile.read(&buffer[0], length);
infile.close();
// Create buffer and point local_stream to it.
istreambuf<char> istream_buffer(&buffer[0], length);
istream local_stream(&istream_buffer);
string str1;
while (local_stream >> str1)
{
. . .
}
}
Related
I need to use string instead of char to write several characters at once.
I want the first cycle to take data from the file and the second cycle to go through the string cycle to \0
since in the future I want to receive 2 or 4 characters at a time.
Can I implement this to get .get working through string?
fstream fs("file.txt", fstream::in | fstream::out | ios::binary);
for (string i; fs.get(i);) {
cout << i;
}
istream::get with a c-string reads up to n characters, or a delimiter, default newline (very similar to istream::getline, but it leaves the delimiter in the stream, while getline consumes it).
To read fixed length blocks regardless there is istream::read, and istream::gcount says how much was actually read. Unfortunately neither have an overload for std::string specifically, the main downside being having to size (and thus initialize) a string first.
Putting them together you can get something like:
std::string buffer;
std::fstream is("file.txt", std::ios::in | std::ios::binary);
while (is)
{
buffer.resize(128); // Whatever size you want
is.read(buffer.data(), buffer.size()); // Read into buffer, note *does not null terminate* // C++17
//is.read(&buffer[0], buffer.size()); // Older C++
buffer.resize(is.gcount()); // Actual amount read. Might be less than requested, or even zero at the end or a read failure.
std::cout << "Read " << buffer.size() << " characters." << std::endl;
std::cout << buffer << std::endl;
}
For getline specifically, there is std::getline which handles std::string for you:
std::string buffer;
std::fstream is("file.txt", std::ios::in | std::ios::binary);
while (std::getline(is, buffer))
{
std::cout << "Line: " << buffer << std::endl;
}
Note that both get and getline can use some other delimiter, so it doesn't have to be "lines".
I'm trying to set the internal buffer of an input stream, but my implementation in C++17 doesn't implement pubsetbuf() for istringstream.
I've tried some other techniques, but they are slow or copy the original buffer. I'm looking for a fast method that doesn't do any copying.
It's closely related to this question about an output stream:
Setting the internal buffer used by a standard stream (pubsetbuf)
I've followed it closely, but the buffer for the input stream remains uninitialised/empty.
// Modified template from the other question about an output stream.
// This is for an input stream, but I can't get it to work.
template <typename char_type>
struct istreambuf : public std::basic_streambuf<char_type, std::char_traits<char_type> >
{
istreambuf(char_type* buffer, std::streamsize buffer_length)
{
// Set the "put" pointer to the start of the buffer and record its length.
this->setp(buffer, buffer + buffer_length);
}
};
int main()
{
ifstream infile(FILENAME, std::ifstream::binary);
if (!infile.is_open())
{
cerr << endl << "Failed to open file " << FILENAME << endl;
return 0;
}
// Works, but slow.
//istringstream local_stream;
//local_stream << infile.rdbuf();
// Works, but creates a copy.
//istringstream local_stream(&buffer[0]);
// Works, but creates a copy.
//local_stream.str(&buffer[0]);
// Read entire file into buffer.
infile.seekg(0, std::ios::end);
streampos length = infile.tellg();
infile.seekg(0, std::ios::beg);
vector<char> buffer(length);
//char* buffer = new char[length];
infile.read(&buffer[0], length);
// Doesn't work, but should point to original.
// It returns "this" (does nothing).
//local_stream.rdbuf()->pubsetbuf(&buffer[0], length);
// Works, but deprecated in C++98.
//std::istrstream local_stream(&buffer[0]);
//local_stream.rdbuf()->pubsetbuf(&buffer[0], length);
// I followed the example in the other question about an output stream,
// but I modified for an input stream. I can't get it to work. Any ideas?
istreambuf<char> istream_buffer(&buffer[0], length);
istream local_stream(&istream_buffer);
string str1, str2;
while (local_stream >> str1 && local_stream >> str2)
{
. . .
}
}
I've solved it! Spot the difference.
template <typename char_type>
struct istreambuf : public std::basic_streambuf<char_type, std::char_traits<char_type> >
{
istreambuf(char_type* buffer, std::streamsize buffer_length)
{
// Set the "put" pointer to the start of the buffer and record its length.
//this->setp(buffer, buffer + buffer_length);
// Set the "get" pointer to the start of the buffer, the next item, and record its length.
this->setg(buffer, buffer, buffer + buffer_length);
}
};
I needed to set the "get" pointer, not the "put" pointer. It works well now.
how can I convert istream to string, when my istream also includes newline characters and I don't want to escape whitespaces?
Thank you.
If you mean how to copy the whole std::istream into a std::string then there are many ways.
Here is one:
int main()
{
// here is your istream
std::ifstream ifs("test.txt");
// copy it to your string
std::string s;
for(char c; ifs.get(c); s += c) {}
// display
std::cout << s << '\n';
}
You can just allocate a string large enough for your whole file and read it at once:
ifstream fd(filename); // open your stream (here a file stream)
if (!fd)
exit(1);
fd.seekg(0, ios_base::end); // go to end of file
size_t filesize = fd.tellg(); // dtermine size to allocate
fd.seekg(0, ios_base::beg); // go to the begin of your file
string s; // create a new string
s.resize(filesize+1); // reserve enough space to read
fd.read(&s[0], filesize); // read all the file at one
size_t bytes_read = fd.gcount(); // it could be than less bytes are read
s.resize(bytes_read); // adapt size
You can use a istreambuf_iterator like
#include <iostream>
#include <string>
#include <fstream>
int main()
{
std::ifstream ifile("test.txt"); // open
std::string str(std::istreambuf_iterator<char>(ifile), {}); // initialize
std::cout << str; // display
}
I am under the impression fstream objects in c++ can be used to both read and write, using the same stream.
I have successfully been able to first write to a stream and then read from it. If I however try to write to it again the file is not affected.
Here is a code example that successfully compiles on windows using MinGw:
int main()
{
std::string path="file.txt";
std::fstream fs(path.c_str());
int buffSize=100;
int bytesRead=0;
char* buffer=new char[buffSize];
fs.write("hello", 5);
fs.seekp(0, std::ios::beg);
fs.read(buffer, buffSize);
bytesRead=fs.gcount();
for(int i=0;i<bytesRead;i++) {std::cout << buffer[i];}
std::cout << "\n";
fs.clear();
fs.seekp(1, std::ios::beg);
fs.write("E", 1);
std::cout << "fail: " << fs.fail() << "\n";
delete[] buffer;
}
The initial content of "file.txt" was only:
AAAAAAA
And the program outputs:
helloAA
fail: 0
Looking at the file in a text editor after running the program shows that the final content is:
helloAA
The final writing of the "E" has not taken effect, why is this and how do I fix it?
EDIT:
I tried using fs.clear() before writing again as user 0x499602D2 suggested. Also added a line printing out whether the failbit or badbit has been set or not and updated the program output. The final file content stays the same however, the problem remains.
(more verbose answer from what I posted in comments on the question)
You need to call flush() on output stream objects (derived from ostream) in order for the data to actually be written on the output stream. More information on flush() is available on this c++ reference page.
This work in GCC 4.9.0 and VS2013.
Notes:
seekg is for move the read pointer
seekp is for move the write pointer
In the sample code in line fs.seekp(0, std::ios::beg); need to be seekg. There is no problem because the read pointer has not been move (there is no read until there).
Code:
#include <algorithm>
#include <iostream>
#include <fstream>
using namespace std;
int main(int argc, char* argv[]) {
std::string path = "H:\\save.txt";
int buffSize = 100;
int bytesRead = 0;
char* buffer = new char[buffSize];
std::fstream fs(path.c_str());
fs.write("hello", 5);
fs.flush(); // flushing to disk file
fs.seekg(0, std::ios_base::beg); // moving the read pointer
fs.read(buffer, buffSize);
bytesRead = fs.gcount();
for (int i = 0; i < bytesRead; i++) {
std::cout << buffer[i];
}
std::cout << "\n";
fs.clear();
fs.seekp(1, std::ios::beg);
fs.write("E", 1);
fs.flush(); // flushing to disk file
std::cout << "fail: " << fs.fail() << "\n";
delete[] buffer;
return 0;
}
string data="";
string Newdata="New Data";
std::fstream output_file(fileName, ios::in| ios::out);
output_file >> data; //read Data
output_file.seekg( 0, ios::beg );//set point to zero
output_file<<Newdata<<"\n"; //write new Data
output_file.close();
once you read a file using fstream, tellg < read pointer > and tellp < write pointer > points to -1.
to be able to write again using fstream, just call fstream.clear() and it will reset read and write pointer to where it was before reading.
none of the solution posted above work but fstream.clear() works.
Is there anyway I can transfer data from an fstream (a file) to a stringstream (a stream in the memory)?
Currently, I'm using a buffer, but this requires double the memory, because you need to copy the data to a buffer, then copy the buffer to the stringstream, and until you delete the buffer, the data is duplicated in the memory.
std::fstream fWrite(fName,std::ios::binary | std::ios::in | std::ios::out);
fWrite.seekg(0,std::ios::end); //Seek to the end
int fLen = fWrite.tellg(); //Get length of file
fWrite.seekg(0,std::ios::beg); //Seek back to beginning
char* fileBuffer = new char[fLen];
fWrite.read(fileBuffer,fLen);
Write(fileBuffer,fLen); //This writes the buffer to the stringstream
delete fileBuffer;`
Does anyone know how I can write a whole file to a stringstream without using an inbetween buffer?
ifstream f(fName);
stringstream s;
if (f) {
s << f.rdbuf();
f.close();
}
// need to include <algorithm> and <iterator>, and of course <fstream> and <sstream>
ifstream fin("input.txt");
ostringstream sout;
copy(istreambuf_iterator<char>(fin),
istreambuf_iterator<char>(),
ostreambuf_iterator<char>(sout));
In the documentation for ostream, there are several overloads for operator<<. One of them takes a streambuf* and reads all of the streambuffer's contents.
Here is a sample use (compiled and tested):
#include <exception>
#include <iostream>
#include <fstream>
#include <sstream>
int main ( int, char ** )
try
{
// Will hold file contents.
std::stringstream contents;
// Open the file for the shortest time possible.
{ std::ifstream file("/path/to/file", std::ios::binary);
// Make sure we have something to read.
if ( !file.is_open() ) {
throw (std::exception("Could not open file."));
}
// Copy contents "as efficiently as possible".
contents << file.rdbuf();
}
// Do something "useful" with the file contents.
std::cout << contents.rdbuf();
}
catch ( const std::exception& error )
{
std::cerr << error.what() << std::endl;
return (EXIT_FAILURE);
}
The only way using the C++ standard library is to use a ostrstream instead of stringstream.
You can construct a ostrstream object with your own char buffer, and it will take ownership of the buffer then (so no more copying is needed).
Note however, that the strstream header is deprecated (though its still part of C++03, and most likely, it will always be available on most standard library implementations), and you will get into big troubles if you forget to null-terminate the data supplied to the ostrstream.This also applies to the stream operators, e.g: ostrstreamobject << some_data << std::ends; (std::ends nullterminates the data).
If you're using Poco, this is simply:
#include <Poco/StreamCopier.h>
ifstream ifs(filename);
string output;
Poco::StreamCopier::copyToString(ifs, output);