Is there anyway I can transfer data from an fstream (a file) to a stringstream (a stream in the memory)?
Currently, I'm using a buffer, but this requires double the memory, because you need to copy the data to a buffer, then copy the buffer to the stringstream, and until you delete the buffer, the data is duplicated in the memory.
std::fstream fWrite(fName,std::ios::binary | std::ios::in | std::ios::out);
fWrite.seekg(0,std::ios::end); //Seek to the end
int fLen = fWrite.tellg(); //Get length of file
fWrite.seekg(0,std::ios::beg); //Seek back to beginning
char* fileBuffer = new char[fLen];
fWrite.read(fileBuffer,fLen);
Write(fileBuffer,fLen); //This writes the buffer to the stringstream
delete fileBuffer;`
Does anyone know how I can write a whole file to a stringstream without using an inbetween buffer?
ifstream f(fName);
stringstream s;
if (f) {
s << f.rdbuf();
f.close();
}
// need to include <algorithm> and <iterator>, and of course <fstream> and <sstream>
ifstream fin("input.txt");
ostringstream sout;
copy(istreambuf_iterator<char>(fin),
istreambuf_iterator<char>(),
ostreambuf_iterator<char>(sout));
In the documentation for ostream, there are several overloads for operator<<. One of them takes a streambuf* and reads all of the streambuffer's contents.
Here is a sample use (compiled and tested):
#include <exception>
#include <iostream>
#include <fstream>
#include <sstream>
int main ( int, char ** )
try
{
// Will hold file contents.
std::stringstream contents;
// Open the file for the shortest time possible.
{ std::ifstream file("/path/to/file", std::ios::binary);
// Make sure we have something to read.
if ( !file.is_open() ) {
throw (std::exception("Could not open file."));
}
// Copy contents "as efficiently as possible".
contents << file.rdbuf();
}
// Do something "useful" with the file contents.
std::cout << contents.rdbuf();
}
catch ( const std::exception& error )
{
std::cerr << error.what() << std::endl;
return (EXIT_FAILURE);
}
The only way using the C++ standard library is to use a ostrstream instead of stringstream.
You can construct a ostrstream object with your own char buffer, and it will take ownership of the buffer then (so no more copying is needed).
Note however, that the strstream header is deprecated (though its still part of C++03, and most likely, it will always be available on most standard library implementations), and you will get into big troubles if you forget to null-terminate the data supplied to the ostrstream.This also applies to the stream operators, e.g: ostrstreamobject << some_data << std::ends; (std::ends nullterminates the data).
If you're using Poco, this is simply:
#include <Poco/StreamCopier.h>
ifstream ifs(filename);
string output;
Poco::StreamCopier::copyToString(ifs, output);
Related
how can I convert istream to string, when my istream also includes newline characters and I don't want to escape whitespaces?
Thank you.
If you mean how to copy the whole std::istream into a std::string then there are many ways.
Here is one:
int main()
{
// here is your istream
std::ifstream ifs("test.txt");
// copy it to your string
std::string s;
for(char c; ifs.get(c); s += c) {}
// display
std::cout << s << '\n';
}
You can just allocate a string large enough for your whole file and read it at once:
ifstream fd(filename); // open your stream (here a file stream)
if (!fd)
exit(1);
fd.seekg(0, ios_base::end); // go to end of file
size_t filesize = fd.tellg(); // dtermine size to allocate
fd.seekg(0, ios_base::beg); // go to the begin of your file
string s; // create a new string
s.resize(filesize+1); // reserve enough space to read
fd.read(&s[0], filesize); // read all the file at one
size_t bytes_read = fd.gcount(); // it could be than less bytes are read
s.resize(bytes_read); // adapt size
You can use a istreambuf_iterator like
#include <iostream>
#include <string>
#include <fstream>
int main()
{
std::ifstream ifile("test.txt"); // open
std::string str(std::istreambuf_iterator<char>(ifile), {}); // initialize
std::cout << str; // display
}
I am under the impression fstream objects in c++ can be used to both read and write, using the same stream.
I have successfully been able to first write to a stream and then read from it. If I however try to write to it again the file is not affected.
Here is a code example that successfully compiles on windows using MinGw:
int main()
{
std::string path="file.txt";
std::fstream fs(path.c_str());
int buffSize=100;
int bytesRead=0;
char* buffer=new char[buffSize];
fs.write("hello", 5);
fs.seekp(0, std::ios::beg);
fs.read(buffer, buffSize);
bytesRead=fs.gcount();
for(int i=0;i<bytesRead;i++) {std::cout << buffer[i];}
std::cout << "\n";
fs.clear();
fs.seekp(1, std::ios::beg);
fs.write("E", 1);
std::cout << "fail: " << fs.fail() << "\n";
delete[] buffer;
}
The initial content of "file.txt" was only:
AAAAAAA
And the program outputs:
helloAA
fail: 0
Looking at the file in a text editor after running the program shows that the final content is:
helloAA
The final writing of the "E" has not taken effect, why is this and how do I fix it?
EDIT:
I tried using fs.clear() before writing again as user 0x499602D2 suggested. Also added a line printing out whether the failbit or badbit has been set or not and updated the program output. The final file content stays the same however, the problem remains.
(more verbose answer from what I posted in comments on the question)
You need to call flush() on output stream objects (derived from ostream) in order for the data to actually be written on the output stream. More information on flush() is available on this c++ reference page.
This work in GCC 4.9.0 and VS2013.
Notes:
seekg is for move the read pointer
seekp is for move the write pointer
In the sample code in line fs.seekp(0, std::ios::beg); need to be seekg. There is no problem because the read pointer has not been move (there is no read until there).
Code:
#include <algorithm>
#include <iostream>
#include <fstream>
using namespace std;
int main(int argc, char* argv[]) {
std::string path = "H:\\save.txt";
int buffSize = 100;
int bytesRead = 0;
char* buffer = new char[buffSize];
std::fstream fs(path.c_str());
fs.write("hello", 5);
fs.flush(); // flushing to disk file
fs.seekg(0, std::ios_base::beg); // moving the read pointer
fs.read(buffer, buffSize);
bytesRead = fs.gcount();
for (int i = 0; i < bytesRead; i++) {
std::cout << buffer[i];
}
std::cout << "\n";
fs.clear();
fs.seekp(1, std::ios::beg);
fs.write("E", 1);
fs.flush(); // flushing to disk file
std::cout << "fail: " << fs.fail() << "\n";
delete[] buffer;
return 0;
}
string data="";
string Newdata="New Data";
std::fstream output_file(fileName, ios::in| ios::out);
output_file >> data; //read Data
output_file.seekg( 0, ios::beg );//set point to zero
output_file<<Newdata<<"\n"; //write new Data
output_file.close();
once you read a file using fstream, tellg < read pointer > and tellp < write pointer > points to -1.
to be able to write again using fstream, just call fstream.clear() and it will reset read and write pointer to where it was before reading.
none of the solution posted above work but fstream.clear() works.
I need to read a jpg file to a string. I want to upload this file to our server, I just find out that the API requires a string as the data of this pic. I followed the suggestions in a former question I've asked Upload pics to a server using c++ .
int main() {
ifstream fin("cloud.jpg");
ofstream fout("test.jpg");//for testing purpose, to see if the string is a right copy
ostringstream ostrm;
unsigned char tmp;
int count = 0;
while ( fin >> tmp ) {
++count;//for testing purpose
ostrm << tmp;
}
string data( ostrm.str() );
cout << count << endl;//ouput 60! Definitely not the right size
fout << string;//only 60 bytes
return 0;
}
Why it stops at 60? It's a strange character at 60, and what should I do to read the jpg to a string?
UPDATE
Almost there, but after using the suggested method, when I rewrite the string to the output file, it distorted. Find out that I should also specify that the ofstream is in binary mode by ofstream::binary. Done!
By the way what's the difference between ifstream::binary & ios::binary, is there any abbreviation for ofstream::binary?
Open the file in binary mode, otherwise it will have funny behavior, and it will handle certain non-text characters in inappropriate ways, at least on Windows.
ifstream fin("cloud.jpg", ios::binary);
Also, instead of a while loop, you can just read the whole file in one shot:
ostrm << fin.rdbuf();
You shouldn't read the file to a string because it is legal for a jpg to contain values that are 0. However in a string, the value 0 has a special meaning (it's the end of string indicator aka \0). You should instead read the file into a vector. You can do this easily like so:
#include <algorithm>
#include <iostream>
#include <fstream>
#include <vector>
int main(int argc, char* argv[])
{
std::ifstream ifs("C:\\Users\\Borgleader\\Documents\\Rapptz.h");
if(!ifs)
{
return -1;
}
std::vector<char> data = std::vector<char>(std::istreambuf_iterator<char>(ifs), std::istreambuf_iterator<char>());
//If you really need it in a string you can initialize it the same way as the vector
std::string data2 = std::string(std::istreambuf_iterator<char>(ifs), std::istreambuf_iterator<char>());
std::for_each(data.begin(), data.end(), [](char c) { std::cout << c; });
std::cin.get();
return 0;
}
Try opening the file in binary mode:
ifstream fin("cloud.jpg", std::ios::binary);
At a guess, you were probably trying to read the file on Windows and the 61st character was probably 0x26 -- a control-Z, which (on Windows) will be treated as marking the end of the file.
As far as how to best do the reading, you end up with a choice between simplicity and speed, as demonstrated in a previous answer.
pubsetbuf member of std::stringbuf is not working at all in Visual Studio 2010!
The code:
char *FileData = ... ;
unsigned long long FileDataLen = ... ;
std::stringstream *SS = new std::stringstream(std::stringstream::in | std::stringstream::out);
SS->rdbuf()->pubsetbuf( FileData, (std::streamsize)FileDataLen );
pubsetbuf does nothing in Visual Studio!!!
Workaround #1:
std::stringstream *SS = new std::stringstream( std::string(FileData, (size_type)FileDataLen ) ),std::stringstream::in | std::stringstream::out);
Workaround #2:
SS->rdbuf()->sputn(FileData, (streamsize)FileDataLen);
But both of these workarounds produce unnecessary memory copying.
I definitely need a working pubsetbuf member of std::stringbuf.
putsetbuf only makes sense for fstream (technically, for std::basic_filebuf), where the buffer and the stream are two different things.
For stringstream (technically, std::basic_stringbuf) they are one and the same std::string.
If you need a stream that works on a string that's external to it, consider std::strstream: or boost::iostreams::array_sink
basic_ios.clear()
If you need to change rdbuf, call this first or it'll fail to work.
std::ifstream file("file1.txt"); // file1.txt contains "Welcome!"
std::stringstream ss;
ss << file.rdbuf();
std::cout << ss.str() << std::endl;
Outputs "Welcome!".
Let's try again with a new file.
// Empty it
file.close();
ss.str("");
// New file
file.open("file2.txt"); // file2.txt contains "Goodbye!"
ss << file.rdbuf();
std::cout << ss.str() << std::endl;
Outputs nothing.
ss.clear();
ss << file.rdbuf();
std::cout << ss.str() << std::endl;
file.close();
Outputs "Goodbye!"
I see the same thing. I'm working on scenarios that definitely can't afford making unnecessary data copies. This seems to be somehow intentional as per comments in basic_streambuf class:
virtual basic_streambuf *__CLR_OR_THIS_CALL setbuf(_Elem *, streamsize)
{ // offer buffer to external agent (do nothing)
return (this);
}
I recently encountered the same issue of setbuf not being implemented in Visual Studio 2017.
After some searching on Stack Overflow, I found a solution for an output stream that doesn't use copying of the buffer which I modified for an input stream. Here they are for reference.
Output Stream
Source: Setting the internal buffer used by a standard stream (pubsetbuf)
#include <streambuf>
template <typename char_type>
struct ostreambuf : public std::basic_streambuf<char_type, std::char_traits<char_type> >
{
ostreambuf(char_type* buffer, std::streamsize bufferLength)
{
// set the "put" pointer the start of the buffer and record it's length.
setp(buffer, buffer + bufferLength);
}
};
Input Stream
Source: Internal buffer used by standard input stream (pubsetbuf)
#include <streambuf>
template <typename char_type>
struct istreambuf : public basic_streambuf<char_type, char_traits<char_type>>
{
istreambuf(char_type* buffer, streamsize buffer_length)
{
// Set the "get" pointer to the start of the buffer, the next item, and record its length.
this->setg(buffer, buffer, buffer + buffer_length);
}
};
int main()
{
ifstream infile(FILENAME, ifstream::binary);
// Read entire file into buffer.
infile.seekg(0, ios::end);
streampos length = infile.tellg();
infile.seekg(0, ios::beg);
vector<char> buffer(length);
//char* buffer = new char[length];
infile.read(&buffer[0], length);
infile.close();
// Create buffer and point local_stream to it.
istreambuf<char> istream_buffer(&buffer[0], length);
istream local_stream(&istream_buffer);
string str1;
while (local_stream >> str1)
{
. . .
}
}
I'm trying to write code to read a binary file into a buffer, then write the buffer to another file. I have the following code, but the buffer only stores a couple of ASCII characters from the first line in the file and nothing else.
int length;
char * buffer;
ifstream is;
is.open ("C:\\Final.gif", ios::binary );
// get length of file:
is.seekg (0, ios::end);
length = is.tellg();
is.seekg (0, ios::beg);
// allocate memory:
buffer = new char [length];
// read data as a block:
is.read (buffer,length);
is.close();
FILE *pFile;
pFile = fopen ("C:\\myfile.gif", "w");
fwrite (buffer , 1 , sizeof(buffer) , pFile );
If you want to do this the C++ way, do it like this:
#include <fstream>
#include <iterator>
#include <algorithm>
int main()
{
std::ifstream input( "C:\\Final.gif", std::ios::binary );
std::ofstream output( "C:\\myfile.gif", std::ios::binary );
std::copy(
std::istreambuf_iterator<char>(input),
std::istreambuf_iterator<char>( ),
std::ostreambuf_iterator<char>(output));
}
If you need that data in a buffer to modify it or something, do this:
#include <fstream>
#include <iterator>
#include <vector>
int main()
{
std::ifstream input( "C:\\Final.gif", std::ios::binary );
// copies all data into buffer
std::vector<unsigned char> buffer(std::istreambuf_iterator<char>(input), {});
}
Here is a short example, the C++ way using rdbuf. I got this from the web. I can't find my original source on this:
#include <fstream>
#include <iostream>
int main ()
{
std::ifstream f1 ("C:\\me.txt",std::fstream::binary);
std::ofstream f2 ("C:\\me2.doc",std::fstream::trunc|std::fstream::binary);
f2<<f1.rdbuf();
return 0;
}
sizeof(buffer) == sizeof(char*)
Use length instead.
Also, better to use fopen with "wb"....
sizeof(buffer) is the size of a pointer on your last line NOT the actual size of the buffer.
You need to use "length" that you already established instead
You should pass length into fwrite instead of sizeof(buffer).
Here is implementation of standard C++ 14 using vectors and tuples to Read and Write Text,Binary and Hex files.
Snippet code :
try {
if (file_type == BINARY_FILE) {
/*Open the stream in binary mode.*/
std::ifstream bin_file(file_name, std::ios::binary);
if (bin_file.good()) {
/*Read Binary data using streambuffer iterators.*/
std::vector<uint8_t> v_buf((std::istreambuf_iterator<char>(bin_file)), (std::istreambuf_iterator<char>()));
vec_buf = v_buf;
bin_file.close();
}
else {
throw std::exception();
}
}
else if (file_type == ASCII_FILE) {
/*Open the stream in default mode.*/
std::ifstream ascii_file(file_name);
string ascii_data;
if (ascii_file.good()) {
/*Read ASCII data using getline*/
while (getline(ascii_file, ascii_data))
str_buf += ascii_data + "\n";
ascii_file.close();
}
else {
throw std::exception();
}
}
else if (file_type == HEX_FILE) {
/*Open the stream in default mode.*/
std::ifstream hex_file(file_name);
if (hex_file.good()) {
/*Read Hex data using streambuffer iterators.*/
std::vector<char> h_buf((std::istreambuf_iterator<char>(hex_file)), (std::istreambuf_iterator<char>()));
string hex_str_buf(h_buf.begin(), h_buf.end());
hex_buf = hex_str_buf;
hex_file.close();
}
else {
throw std::exception();
}
}
}
Full Source code can be found here
There is a much simpler way. This does not care if it is binary or text file.
Use noskipws.
char buf[SZ];
ifstream f("file");
int i;
for(i=0; f >> noskipws >> buffer[i]; i++);
ofstream f2("writeto");
for(int j=0; j < i; j++) f2 << noskipws << buffer[j];
Or you can just use string instead of the buffer.
string s; char c;
ifstream f("image.jpg");
while(f >> noskipws >> c) s += c;
ofstream f2("copy.jpg");
f2 << s;
normally stream skips white space characters like space or new line, tab and all other control characters.
But noskipws makes all the characters transferred.
So this will not only copy a text file but also a binary file.
And stream uses buffer internally, I assume the speed won't be slow.
It can be done with simple commands in the following snippet.
Copies the whole file of any size. No size constraint!
Just use this. Tested And Working!!
#include<iostream>
#include<fstream>
using namespace std;
int main()
{
ifstream infile;
infile.open("source.pdf",ios::binary|ios::in);
ofstream outfile;
outfile.open("temppdf.pdf",ios::binary|ios::out);
int buffer[2];
while(infile.read((char *)&buffer,sizeof(buffer)))
{
outfile.write((char *)&buffer,sizeof(buffer));
}
infile.close();
outfile.close();
return 0;
}
Having a smaller buffer size would be helpful in copying tiny files. Even "char buffer[2]"
would do the job.