I'm trying to to read ~36KB and it would take ~20 seconds to finish this loop:
ifstream input_file;
input_file.open("text.txt");
if( !(input_file.is_open()) )
{
cout<<"File not found";
exit(1);
}
std::string line;
stringstream line_stream; //to use << operator to get words from lines
int lineNum=1;
while( getline(input_file,line) ) //Read file line by line until file ends
{
line_stream.clear(); //clear stream
line_stream << line; //read line
while(line_stream >> word) //Read the line word by word until the line ends
{
//insert word into a linked list...
}
lineNum++;
}
input_file.close();
Any help would be appreciated.
stringstream::clear() does not clear all context inside it. It only resets the error and EOF flags, see http://en.cppreference.com/w/cpp/io/basic_ios/clear.
The result is your line_stream accumulates all previous lines and the inner loop will run words over all the accumulated lines again and again.
So the total time you spend is about O(n^2) compared to O(n) of what you expect it to be.
Instead of using the same object across each line, you could define the new line_stream instance inside the while loop to have a brand new and also empty one. Like this:
fstream input_file;
input_file.open("text.txt");
if( !(input_file.is_open()) )
{
cout<<"File not found";
exit(1);
}
std::string line;
int lineNum=1;
while( getline(input_file,line) ) //Read file line by line until file ends
{
stringstream line_stream; // new instance, empty line.
line_stream << line; //read line
while(line_stream >> word) //Read the line word by word until the line ends
{
//insert word into a linked list...
}
lineNum++;
}
input_file.close();
You could attempt the following:
std::ifstream file("text.txt");
std::string str;
while (std::getline(file, str))
{
cout << str; //call function to to retrieve words of str in memory not in file
}
I ran your code in 11ms, but with the mentioned option in 8ms. May be it works for you.
Try compiling with build flag -O2 or -O3.
I was surprised to see that a simple for-loop to read a 1GB file took 4.7 seconds, whereas another higher level language (Dart) did it in 3.x seconds.
After enabling this flag, runtime dropped to 2.1 seconds.
Related
I want to read a file with std::getline. but reads first line only
string FileReader::readLine() {
string line;
string read;
ifstream ReadFile;
ReadFile.open("input.txt");
if (ReadFile.is_open()) {
getline(ReadFile, line);
//ReadFile.close();
}
return line;
}
this is my method. I call this method several time but always reads first line how can i do to read next lines?
You need to change your program flow.
Don't return a string. Use the line within the loop to do whatever it is you want. Ensuring that you either don't leave the method or return to it.
You can't keep coming back to a function like this, as it will keep reading from the beginning.
void FileReader::readLine() {
string line;
string read;
ifstream ReadFile;
ReadFile.open("input.txt");
if (ReadFile.is_open()) {
while(getline(ReadFile, line))
{
//do what you want with that line, but return program flow here.
}
ReadFile.close();
}
}
I am taking contents from one file and transferring them to another. In particular, I am looking for lines that contain the substring "LIBOR/Swap". However, this is irrelevant and I've commented them out. So, this particular code at the moment really takes every line in one .csv file (file) and transfers them to another .csv file (temp).
The issue is that the transferring skips lines. See attached photos to see the lines that it's skipping in the transfer of data.
original file
new file
As you can see, with the first USD LIBOR/Swap line, it skips the 1 Day rate. This happens elsewhere throughout the code.
Any reason as to why? Thanks.
ifstream file;
ofstream temp;
file.open("YC Rate Levels.csv");
temp.open("Temp.csv");
if (file.is_open())
{
string line;
string test = "LIBOR/Swap";
while (getline(file, line))
{
getline(file, line);
//if(line.find(test) != string::npos)
//{
temp << line << endl;
//}
}
temp.close();
file.close();
} else cout << "File did not open.";
std::cin.clear();
std::cin.ignore(32767, '\n');
std::cin.get();
}
You are calling getline(file, line) twice - once inside the while() header and then again inside the body. Remove the second one.
I am trying to read text file line by line and then read each column as vector but when i am tryin to cout first column it shows zeros i.e. not reading the file correctly.
int main(void)
{
ifstream myfile ("data1.txt");
string line;
if (myfile.is_open())
{
int ln=1;
while ( getline (myfile,line) )
{
if(ln==1){ln++; continue;}
istringstream iss(line);
string word;
vector<double> column;
int w=1;
while(iss >> word)
{
//double dw=atof(Form("%s",word));
column.push_back(atof(Form("%s",word)));
cout<<column[0];
w++;
}
ln++;
cout<<"\n";
}
myfile.close();
}
//else
else cout<<"Unable to open file";
cout<<"\n";
return ;
}enter code here
push_back appends an element as last element of the vector while columns[0] always refers to the first element of the vector.
Is the first element 0
Is there another problem?
(Please explain what is Form, give an example of input file and output in the command line)
First of all learn how to indent and consistently use some scheme for inserting blank lines that makes sense. When you do that you can read your own code and figure out if it is doing what you think it is.
Second. Save Form("%s",word) in a string ( for now call it form_string) add this line cout<<"form returns "<<form_string<<endl; 99.99% probably it will print zeros.
Finally change: cout<<column[0]; to cout<<column[0]<<" "; or cout<<*(column.rbegin())<<" ";. The latter prints out all the values that you read, the former prints out the first value you read over and over.
I am fairly new to C++, and I'm trying write a code to run some analysis on large data files. I've managed to write code that produces a text file where only one word/number is displayed per line (there are millions of lines). However, the first ~3000 or so lines contain useless stuff not needed for my analysis.
The only problem is, the actual data starts at a different line number depending on the input file.
Is there any way to write a quick code which would search the text document and remove all lines up until the point the keyword "<event>" is found?
Update:
I got it to work! May be a little more complicated than what was suggested but it still works.
Thanks for the help!
#include <iostream>
#include <fstream>
#include <cstdio>
#include <cstring>
using namespace std;
int main()
{
int counter = 0;
ifstream FileSearch("OutputVector.txt"); // search OutputVector input file.
while(!FileSearch.eof())
{
counter++;
string temp;
FileSearch >> temp;
if(temp == "<event>")
{
break; //While loop adding +1 to counter each time <event> is not found.
}
}
std::ofstream outFile("./final.txt"); //Create output file "final.txt."
std::string line;
std::ifstream inFile("OutputVector.txt"); //open input file OutputVector again.
int count = 0;
while(getline(inFile, line)){
if(count > counter-2){
outFile << line << std::endl;
}
count++; //while loop counts from counter-2 until the end and writes them to the new file.
}
outFile.close();
inFile.close(); //close the files.
remove("OutputVector.txt"); //Delete uneeded OutputVector File.
}
Basic skeleton:
std::ifstream stream("file name goes here")
std::string line;
// optional: define line number here
while (std::getline (stream, line))
{
// optional: increment line number here
if (line.find("<event>") != line.npos)
{ // Deity of choice help you if <event> naturally occurs in junk lines.
// Extra smarts may be required here.
doStuffWithRestOfFile(stream);
break;
}
}
Not enough information on how you wish to modify the source file to answer that sub-question. Once you get the reader going, ask a new question if you haven't figured it out.
Edit: Short version
std::ifstream stream("file name goes here")
std::string line;
// optional: define line number here
while (std::getline (stream, line) && (line.find("<event>") == line.npos))
{
// optional: increment line number here
}
doStuffWithRestOfFile(stream);
If you want to override the file with the new version ( without the beginning ) you can either read all the file to memory and override it, or write to a second file while reading the first one, and move/rename it after
to read all line until you find <event> :
std::ifstream input_file( filePath );
std::string line;
int current_line = 0;
do
{
std::getline( input_file, line );
++current_line;
}
while( line.find("<event>") == line.npos );
// use input_line to process the rest of the file
Keep in mind that if "<event>" is a the first line, then after the do while, current_line will contain 1, not 0
I'm trying to write a simple program that will print the contents of a text file one line at a time. However, whenever I run the program i just get a blank screen. I'm certain the file I am trying to read contains text over several lines. Any help as to why this isn't working would be super helpfull.
bool show() {
string line;
ifstream myfile;
myfile.open("tasks.txt", ios::app);
while (!myfile.eof()) {
getline (myfile, line);
cout << line << endl;
}
myfile.close();
return true;
}
The problem might be that you are using ios::app with ifstream (input stream), which makes no sense.
According to this,
ios::app: All output operations are performed at the end of the file, appending the content to the current content of the file. This flag can only be used in streams open for output-only operations.
Try this:
std::string line;
ifstream myfile ("tasks.txt");
if (myfile.is_open())
{
while ( getline (myfile,line) )
{
std::cout << line << std::endl;
}
myfile.close();
}
Did you check return value of myfile.isopen()? Perhaps the file isn't there or you don't have read permission.
Oh yes, I missed that - the append flag. Should be ios::in