how can I convert istream to string, when my istream also includes newline characters and I don't want to escape whitespaces?
Thank you.
If you mean how to copy the whole std::istream into a std::string then there are many ways.
Here is one:
int main()
{
// here is your istream
std::ifstream ifs("test.txt");
// copy it to your string
std::string s;
for(char c; ifs.get(c); s += c) {}
// display
std::cout << s << '\n';
}
You can just allocate a string large enough for your whole file and read it at once:
ifstream fd(filename); // open your stream (here a file stream)
if (!fd)
exit(1);
fd.seekg(0, ios_base::end); // go to end of file
size_t filesize = fd.tellg(); // dtermine size to allocate
fd.seekg(0, ios_base::beg); // go to the begin of your file
string s; // create a new string
s.resize(filesize+1); // reserve enough space to read
fd.read(&s[0], filesize); // read all the file at one
size_t bytes_read = fd.gcount(); // it could be than less bytes are read
s.resize(bytes_read); // adapt size
You can use a istreambuf_iterator like
#include <iostream>
#include <string>
#include <fstream>
int main()
{
std::ifstream ifile("test.txt"); // open
std::string str(std::istreambuf_iterator<char>(ifile), {}); // initialize
std::cout << str; // display
}
Related
The contents of file.txt are:
5 3
6 4
7 1
10 5
11 6
12 3
12 4
Where 5 3 is a coordinate pair.
How do I process this data line by line in C++?
I am able to get the first line, but how do I get the next line of the file?
ifstream myfile;
myfile.open ("file.txt");
First, make an ifstream:
#include <fstream>
std::ifstream infile("thefile.txt");
The two standard methods are:
Assume that every line consists of two numbers and read token by token:
int a, b;
while (infile >> a >> b)
{
// process pair (a,b)
}
Line-based parsing, using string streams:
#include <sstream>
#include <string>
std::string line;
while (std::getline(infile, line))
{
std::istringstream iss(line);
int a, b;
if (!(iss >> a >> b)) { break; } // error
// process pair (a,b)
}
You shouldn't mix (1) and (2), since the token-based parsing doesn't gobble up newlines, so you may end up with spurious empty lines if you use getline() after token-based extraction got you to the end of a line already.
Use ifstream to read data from a file:
std::ifstream input( "filename.ext" );
If you really need to read line by line, then do this:
for( std::string line; getline( input, line ); )
{
...for each line in input...
}
But you probably just need to extract coordinate pairs:
int x, y;
input >> x >> y;
Update:
In your code you use ofstream myfile;, however the o in ofstream stands for output. If you want to read from the file (input) use ifstream. If you want to both read and write use fstream.
Reading a file line by line in C++ can be done in some different ways.
[Fast] Loop with std::getline()
The simplest approach is to open an std::ifstream and loop using std::getline() calls. The code is clean and easy to understand.
#include <fstream>
std::ifstream file(FILENAME);
if (file.is_open()) {
std::string line;
while (std::getline(file, line)) {
// using printf() in all tests for consistency
printf("%s", line.c_str());
}
file.close();
}
[Fast] Use Boost's file_description_source
Another possibility is to use the Boost library, but the code gets a bit more verbose. The performance is quite similar to the code above (Loop with std::getline()).
#include <boost/iostreams/device/file_descriptor.hpp>
#include <boost/iostreams/stream.hpp>
#include <fcntl.h>
namespace io = boost::iostreams;
void readLineByLineBoost() {
int fdr = open(FILENAME, O_RDONLY);
if (fdr >= 0) {
io::file_descriptor_source fdDevice(fdr, io::file_descriptor_flags::close_handle);
io::stream <io::file_descriptor_source> in(fdDevice);
if (fdDevice.is_open()) {
std::string line;
while (std::getline(in, line)) {
// using printf() in all tests for consistency
printf("%s", line.c_str());
}
fdDevice.close();
}
}
}
[Fastest] Use C code
If performance is critical for your software, you may consider using the C language. This code can be 4-5 times faster than the C++ versions above, see benchmark below
FILE* fp = fopen(FILENAME, "r");
if (fp == NULL)
exit(EXIT_FAILURE);
char* line = NULL;
size_t len = 0;
while ((getline(&line, &len, fp)) != -1) {
// using printf() in all tests for consistency
printf("%s", line);
}
fclose(fp);
if (line)
free(line);
Benchmark -- Which one is faster?
I have done some performance benchmarks with the code above and the results are interesting. I have tested the code with ASCII files that contain 100,000 lines, 1,000,000 lines and 10,000,000 lines of text. Each line of text contains 10 words in average. The program is compiled with -O3 optimization and its output is forwarded to /dev/null in order to remove the logging time variable from the measurement. Last, but not least, each piece of code logs each line with the printf() function for consistency.
The results show the time (in ms) that each piece of code took to read the files.
The performance difference between the two C++ approaches is minimal and shouldn't make any difference in practice. The performance of the C code is what makes the benchmark impressive and can be a game changer in terms of speed.
10K lines 100K lines 1000K lines
Loop with std::getline() 105ms 894ms 9773ms
Boost code 106ms 968ms 9561ms
C code 23ms 243ms 2397ms
Since your coordinates belong together as pairs, why not write a struct for them?
struct CoordinatePair
{
int x;
int y;
};
Then you can write an overloaded extraction operator for istreams:
std::istream& operator>>(std::istream& is, CoordinatePair& coordinates)
{
is >> coordinates.x >> coordinates.y;
return is;
}
And then you can read a file of coordinates straight into a vector like this:
#include <fstream>
#include <iterator>
#include <vector>
int main()
{
char filename[] = "coordinates.txt";
std::vector<CoordinatePair> v;
std::ifstream ifs(filename);
if (ifs) {
std::copy(std::istream_iterator<CoordinatePair>(ifs),
std::istream_iterator<CoordinatePair>(),
std::back_inserter(v));
}
else {
std::cerr << "Couldn't open " << filename << " for reading\n";
}
// Now you can work with the contents of v
}
Expanding on the accepted answer, if the input is:
1,NYC
2,ABQ
...
you will still be able to apply the same logic, like this:
#include <fstream>
std::ifstream infile("thefile.txt");
if (infile.is_open()) {
int number;
std::string str;
char c;
while (infile >> number >> c >> str && c == ',')
std::cout << number << " " << str << "\n";
}
infile.close();
Although there is no need to close the file manually but it is good idea to do so if the scope of the file variable is bigger:
ifstream infile(szFilePath);
for (string line = ""; getline(infile, line); )
{
//do something with the line
}
if(infile.is_open())
infile.close();
This answer is for visual studio 2017 and if you want to read from text file which location is relative to your compiled console application.
first put your textfile (test.txt in this case) into your solution folder. After compiling keep text file in same folder with applicationName.exe
C:\Users\"username"\source\repos\"solutionName"\"solutionName"
#include <iostream>
#include <fstream>
using namespace std;
int main()
{
ifstream inFile;
// open the file stream
inFile.open(".\\test.txt");
// check if opening a file failed
if (inFile.fail()) {
cerr << "Error opeing a file" << endl;
inFile.close();
exit(1);
}
string line;
while (getline(inFile, line))
{
cout << line << endl;
}
// close the file stream
inFile.close();
}
This is a general solution to loading data into a C++ program, and uses the readline function. This could be modified for CSV files, but the delimiter is a space here.
int n = 5, p = 2;
int X[n][p];
ifstream myfile;
myfile.open("data.txt");
string line;
string temp = "";
int a = 0; // row index
while (getline(myfile, line)) { //while there is a line
int b = 0; // column index
for (int i = 0; i < line.size(); i++) { // for each character in rowstring
if (!isblank(line[i])) { // if it is not blank, do this
string d(1, line[i]); // convert character to string
temp.append(d); // append the two strings
} else {
X[a][b] = stod(temp); // convert string to double
temp = ""; // reset the capture
b++; // increment b cause we have a new number
}
}
X[a][b] = stod(temp);
temp = "";
a++; // onto next row
}
I am trying to count the characters in my program. Initially my variable "words" was a char and the file read just fine. When trying to determine the length of the variable, it wouldn't work with .length(). Can you explain how I can make my "words" variable as a string so that the words.length() executes correctly?
error on line words = readFile.get(); is:
no match for ‘operator!=’ in ‘words != -0x00000000000000001’
#include <iostream>
#include <cmath>
#include <fstream>
#include <cstdlib>
#include <string>
#include <stdio.h>
#include <math.h>
using namespace std;
int main() {
//buff array to hold char words in the input text file
string words;
//char words;
//read file
ifstream readFile("TextFile1.txt");
//notify user if the file didn't transfer into the system
if (!readFile)
cout <<"I am sorry but we could not process your file."<<endl;
//read and output the file
while (readFile)
{
words = readFile.get();
if(words!= EOF)
cout <<words;
}
cout << "The size of the file is: " << words.length() << " bytes. \n";
return 0;
}
char c;
while (readFile.get(c))
{
words.insert(c);
}
Of course, if you were solely doing this to count the number of characters (and were intent on using std::istream::get) you'd probably be better off just doing this:
int NumChars = 0;
while (readFile.get())
{
NumChars++;
}
Oh, and by the way, you might want to close the file after you're done with it.
You should read some reference.. try cppreference.com and look for std::instream::get
I'm not sure what do you want, but if you wanna just count words, you can do something like this:
std::ifstream InFile(/*filename*/);
if(!InFile)
// file not found
std::string s;
int numWords = 0;
while(InFile >> s)
numWords++;
std::cout << numWords;
Or if you want to get to know how many characters are in file, change std::string s to char s and use std::ifstream::get instead:
std::ifstream InFile(/*filename*/);
if(!InFile)
// file not found
char s;
int numCharacters = 0;
while(InFile.get(s)) //this will read one character after another until EOF
numCharacters++;
std::cout << numCharacters;
The second approach is easier:
If file uses ASCII, numCharacters == fileSize;
Otherwise if it uses UNICODE, numCharacters == fileSize / 2;
get() returns an int, to do what you're doing, you must check that int before appending to "words" instead of checking words against EOF, e.g.:
...
//read and output the file
while (readFile)
{
const int w = readFile.get();
if (w!= EOF) {
words += w;
cout <<words;
}
}
...
pubsetbuf member of std::stringbuf is not working at all in Visual Studio 2010!
The code:
char *FileData = ... ;
unsigned long long FileDataLen = ... ;
std::stringstream *SS = new std::stringstream(std::stringstream::in | std::stringstream::out);
SS->rdbuf()->pubsetbuf( FileData, (std::streamsize)FileDataLen );
pubsetbuf does nothing in Visual Studio!!!
Workaround #1:
std::stringstream *SS = new std::stringstream( std::string(FileData, (size_type)FileDataLen ) ),std::stringstream::in | std::stringstream::out);
Workaround #2:
SS->rdbuf()->sputn(FileData, (streamsize)FileDataLen);
But both of these workarounds produce unnecessary memory copying.
I definitely need a working pubsetbuf member of std::stringbuf.
putsetbuf only makes sense for fstream (technically, for std::basic_filebuf), where the buffer and the stream are two different things.
For stringstream (technically, std::basic_stringbuf) they are one and the same std::string.
If you need a stream that works on a string that's external to it, consider std::strstream: or boost::iostreams::array_sink
basic_ios.clear()
If you need to change rdbuf, call this first or it'll fail to work.
std::ifstream file("file1.txt"); // file1.txt contains "Welcome!"
std::stringstream ss;
ss << file.rdbuf();
std::cout << ss.str() << std::endl;
Outputs "Welcome!".
Let's try again with a new file.
// Empty it
file.close();
ss.str("");
// New file
file.open("file2.txt"); // file2.txt contains "Goodbye!"
ss << file.rdbuf();
std::cout << ss.str() << std::endl;
Outputs nothing.
ss.clear();
ss << file.rdbuf();
std::cout << ss.str() << std::endl;
file.close();
Outputs "Goodbye!"
I see the same thing. I'm working on scenarios that definitely can't afford making unnecessary data copies. This seems to be somehow intentional as per comments in basic_streambuf class:
virtual basic_streambuf *__CLR_OR_THIS_CALL setbuf(_Elem *, streamsize)
{ // offer buffer to external agent (do nothing)
return (this);
}
I recently encountered the same issue of setbuf not being implemented in Visual Studio 2017.
After some searching on Stack Overflow, I found a solution for an output stream that doesn't use copying of the buffer which I modified for an input stream. Here they are for reference.
Output Stream
Source: Setting the internal buffer used by a standard stream (pubsetbuf)
#include <streambuf>
template <typename char_type>
struct ostreambuf : public std::basic_streambuf<char_type, std::char_traits<char_type> >
{
ostreambuf(char_type* buffer, std::streamsize bufferLength)
{
// set the "put" pointer the start of the buffer and record it's length.
setp(buffer, buffer + bufferLength);
}
};
Input Stream
Source: Internal buffer used by standard input stream (pubsetbuf)
#include <streambuf>
template <typename char_type>
struct istreambuf : public basic_streambuf<char_type, char_traits<char_type>>
{
istreambuf(char_type* buffer, streamsize buffer_length)
{
// Set the "get" pointer to the start of the buffer, the next item, and record its length.
this->setg(buffer, buffer, buffer + buffer_length);
}
};
int main()
{
ifstream infile(FILENAME, ifstream::binary);
// Read entire file into buffer.
infile.seekg(0, ios::end);
streampos length = infile.tellg();
infile.seekg(0, ios::beg);
vector<char> buffer(length);
//char* buffer = new char[length];
infile.read(&buffer[0], length);
infile.close();
// Create buffer and point local_stream to it.
istreambuf<char> istream_buffer(&buffer[0], length);
istream local_stream(&istream_buffer);
string str1;
while (local_stream >> str1)
{
. . .
}
}
I'm trying to write code to read a binary file into a buffer, then write the buffer to another file. I have the following code, but the buffer only stores a couple of ASCII characters from the first line in the file and nothing else.
int length;
char * buffer;
ifstream is;
is.open ("C:\\Final.gif", ios::binary );
// get length of file:
is.seekg (0, ios::end);
length = is.tellg();
is.seekg (0, ios::beg);
// allocate memory:
buffer = new char [length];
// read data as a block:
is.read (buffer,length);
is.close();
FILE *pFile;
pFile = fopen ("C:\\myfile.gif", "w");
fwrite (buffer , 1 , sizeof(buffer) , pFile );
If you want to do this the C++ way, do it like this:
#include <fstream>
#include <iterator>
#include <algorithm>
int main()
{
std::ifstream input( "C:\\Final.gif", std::ios::binary );
std::ofstream output( "C:\\myfile.gif", std::ios::binary );
std::copy(
std::istreambuf_iterator<char>(input),
std::istreambuf_iterator<char>( ),
std::ostreambuf_iterator<char>(output));
}
If you need that data in a buffer to modify it or something, do this:
#include <fstream>
#include <iterator>
#include <vector>
int main()
{
std::ifstream input( "C:\\Final.gif", std::ios::binary );
// copies all data into buffer
std::vector<unsigned char> buffer(std::istreambuf_iterator<char>(input), {});
}
Here is a short example, the C++ way using rdbuf. I got this from the web. I can't find my original source on this:
#include <fstream>
#include <iostream>
int main ()
{
std::ifstream f1 ("C:\\me.txt",std::fstream::binary);
std::ofstream f2 ("C:\\me2.doc",std::fstream::trunc|std::fstream::binary);
f2<<f1.rdbuf();
return 0;
}
sizeof(buffer) == sizeof(char*)
Use length instead.
Also, better to use fopen with "wb"....
sizeof(buffer) is the size of a pointer on your last line NOT the actual size of the buffer.
You need to use "length" that you already established instead
You should pass length into fwrite instead of sizeof(buffer).
Here is implementation of standard C++ 14 using vectors and tuples to Read and Write Text,Binary and Hex files.
Snippet code :
try {
if (file_type == BINARY_FILE) {
/*Open the stream in binary mode.*/
std::ifstream bin_file(file_name, std::ios::binary);
if (bin_file.good()) {
/*Read Binary data using streambuffer iterators.*/
std::vector<uint8_t> v_buf((std::istreambuf_iterator<char>(bin_file)), (std::istreambuf_iterator<char>()));
vec_buf = v_buf;
bin_file.close();
}
else {
throw std::exception();
}
}
else if (file_type == ASCII_FILE) {
/*Open the stream in default mode.*/
std::ifstream ascii_file(file_name);
string ascii_data;
if (ascii_file.good()) {
/*Read ASCII data using getline*/
while (getline(ascii_file, ascii_data))
str_buf += ascii_data + "\n";
ascii_file.close();
}
else {
throw std::exception();
}
}
else if (file_type == HEX_FILE) {
/*Open the stream in default mode.*/
std::ifstream hex_file(file_name);
if (hex_file.good()) {
/*Read Hex data using streambuffer iterators.*/
std::vector<char> h_buf((std::istreambuf_iterator<char>(hex_file)), (std::istreambuf_iterator<char>()));
string hex_str_buf(h_buf.begin(), h_buf.end());
hex_buf = hex_str_buf;
hex_file.close();
}
else {
throw std::exception();
}
}
}
Full Source code can be found here
There is a much simpler way. This does not care if it is binary or text file.
Use noskipws.
char buf[SZ];
ifstream f("file");
int i;
for(i=0; f >> noskipws >> buffer[i]; i++);
ofstream f2("writeto");
for(int j=0; j < i; j++) f2 << noskipws << buffer[j];
Or you can just use string instead of the buffer.
string s; char c;
ifstream f("image.jpg");
while(f >> noskipws >> c) s += c;
ofstream f2("copy.jpg");
f2 << s;
normally stream skips white space characters like space or new line, tab and all other control characters.
But noskipws makes all the characters transferred.
So this will not only copy a text file but also a binary file.
And stream uses buffer internally, I assume the speed won't be slow.
It can be done with simple commands in the following snippet.
Copies the whole file of any size. No size constraint!
Just use this. Tested And Working!!
#include<iostream>
#include<fstream>
using namespace std;
int main()
{
ifstream infile;
infile.open("source.pdf",ios::binary|ios::in);
ofstream outfile;
outfile.open("temppdf.pdf",ios::binary|ios::out);
int buffer[2];
while(infile.read((char *)&buffer,sizeof(buffer)))
{
outfile.write((char *)&buffer,sizeof(buffer));
}
infile.close();
outfile.close();
return 0;
}
Having a smaller buffer size would be helpful in copying tiny files. Even "char buffer[2]"
would do the job.
Is there anyway I can transfer data from an fstream (a file) to a stringstream (a stream in the memory)?
Currently, I'm using a buffer, but this requires double the memory, because you need to copy the data to a buffer, then copy the buffer to the stringstream, and until you delete the buffer, the data is duplicated in the memory.
std::fstream fWrite(fName,std::ios::binary | std::ios::in | std::ios::out);
fWrite.seekg(0,std::ios::end); //Seek to the end
int fLen = fWrite.tellg(); //Get length of file
fWrite.seekg(0,std::ios::beg); //Seek back to beginning
char* fileBuffer = new char[fLen];
fWrite.read(fileBuffer,fLen);
Write(fileBuffer,fLen); //This writes the buffer to the stringstream
delete fileBuffer;`
Does anyone know how I can write a whole file to a stringstream without using an inbetween buffer?
ifstream f(fName);
stringstream s;
if (f) {
s << f.rdbuf();
f.close();
}
// need to include <algorithm> and <iterator>, and of course <fstream> and <sstream>
ifstream fin("input.txt");
ostringstream sout;
copy(istreambuf_iterator<char>(fin),
istreambuf_iterator<char>(),
ostreambuf_iterator<char>(sout));
In the documentation for ostream, there are several overloads for operator<<. One of them takes a streambuf* and reads all of the streambuffer's contents.
Here is a sample use (compiled and tested):
#include <exception>
#include <iostream>
#include <fstream>
#include <sstream>
int main ( int, char ** )
try
{
// Will hold file contents.
std::stringstream contents;
// Open the file for the shortest time possible.
{ std::ifstream file("/path/to/file", std::ios::binary);
// Make sure we have something to read.
if ( !file.is_open() ) {
throw (std::exception("Could not open file."));
}
// Copy contents "as efficiently as possible".
contents << file.rdbuf();
}
// Do something "useful" with the file contents.
std::cout << contents.rdbuf();
}
catch ( const std::exception& error )
{
std::cerr << error.what() << std::endl;
return (EXIT_FAILURE);
}
The only way using the C++ standard library is to use a ostrstream instead of stringstream.
You can construct a ostrstream object with your own char buffer, and it will take ownership of the buffer then (so no more copying is needed).
Note however, that the strstream header is deprecated (though its still part of C++03, and most likely, it will always be available on most standard library implementations), and you will get into big troubles if you forget to null-terminate the data supplied to the ostrstream.This also applies to the stream operators, e.g: ostrstreamobject << some_data << std::ends; (std::ends nullterminates the data).
If you're using Poco, this is simply:
#include <Poco/StreamCopier.h>
ifstream ifs(filename);
string output;
Poco::StreamCopier::copyToString(ifs, output);