I have a very big .txt file (9 MB). In it the words are stored like this :
да 2337093
е 1504540
не 1480296
се 1212312
Every line in the .txt file consists of a string followed by a single space and a number.
I want to get only the words and store them in a string array. I see that a regex will be an overkill here, but fail to think of a another way as i'm not familiar with streams in c++.
Similar to below sample
#include <bits/stdc++.h>
using namespace std;
int main() {
vector<string> strings;
ifstream file("path_to_file");
string line;
while (getline(file, line))
strings.push_back(line.substr(0, line.find(" ")));
// Do whatever you want with 'strings' vector
}
You should read file line by line, and for each line use string's substr() method to parse a line based on space location, and you can use find() method to find the location of delimiter. take the word part which is before space and ignore rest.
You can look here for an example.
Related
I am reading a file of 5 doubles in a line through a for loop and storing them into a vector of structs and sometimes there is less then 5 doubles on the line. The file looks something like
111.111 222.222 333.333 444.444 555.555
666.666
777.777 (whitespace) 888.888 999.999
struct temp_struct{
double d1,d2,d3,d4,d5
};
vector<temp_struct> data;
for(int i=0;i<3;i++)
File>>data.at(i).d1>>data.at(i).d2>>data.at(i).d3>>data.at(i).d4>>data.at(i).d5;
Is there a way to test for the empty space after 666.666 and inbetween 777.777 and 888.888.
Because right now the way I am reading it in, after 666.666 it will go to the next line of the file and read 777.777 and 888.888 999.999 so that the struct looks like
data.at(1).d1=666.666 data.at(1).d2=777.777 data.at(1).d3=888.888 and data.at(1).d4=999.999 where I would rather have
data.at(1).d1=666.666, break because of end of line
data.at(2).d1=777.777, data.at(2).d2=0, data.at(3).d3=888.8888
I am programming in VS2010 C++
The usual way to parse line-based text is to use std::getline() in combination with an std::istringstream, e.g.:
for (std::string line; std::getline(in, line); ) {
std::istringstream lin(line);
// parse the line using `line`
}
There are other approaches, too, though. For example, it is possible to set up a stream to not consider a newline whitespace: this way, trying to read numbers when a newline is encountered the attempt to read the value would fail! The way to change what is considered whitespace is to install a custom std::locale object with a custom version of std::ctype<char>.
string numbers;
string fileName = "text.txt";
ifstream inputFile;
inputFile.open(fileName.c_str(),ios_base::in);
inputFile >> numbers;
inputFile.close();
cout << numbers;
And my text.txt file is:
1 2 3 4 5
basically a set of integers separated by tabs.
The problem is the program only reads the first integer in the text.txt file and ignores the rest for some reason. If I remove the tabs between the integers it works fine, but with tabs between them, it won't work. What causes this? As far as I know it should ignore any white space characters or am I mistaken? If so is there a better way to get each of these numbers from the text file?
When reading formatted strings the input operator starts with ignoring leading whitespace. Then it reads non-whitespace characters up to the first space and stops. The non-whitespace characters get stored in the std::string. If there are only whitespace characters before the stream reaches end of file (or some error for that matter), reading fails. Thus, your program reads one "word" (in this case a number) and stops reading.
Unfortunately, you only said what you are doing and what the problems are with your approach (where you problem description failed to cover the case where reading the input fails in the first place). Here are a few things you might want to try:
If you want to read multiple words, you can do so, e.g., by reading all words:
std::vector<std::string> words;
std::copy(std::istream_iterator<std::string>(inputFile),
std::istream_iterator<std::string>(),
std::back_inserter(words));
This will read all words from inputFile and store them as a sequence of std::strings in the vector words. Since you file contains numbers you might want to replace std::string by int to read numbers in a readily accessible form.
If you want to read a line rather than a word you can use std::getline() instead:
if (std::getline(inputFile, line)) { ... }
If you want to read multiple lines, you'd put this operation into a loop: There is, unfortunately, no read-made approach to read a sequence of lines as there is for words.
If you want to read the entire file, not just the first line, into a file, you can also use std::getline() but you'd need to know about one character value which doesn't occur in your file, e.g., the null value:
if (std::getline(inputFile, text, char()) { ... }
This approach considers a "line" a sequence of characters up to a null character. You can use any other character value as well. If you can't be sure about the character values, you can read an entire file using std::string's constructor taking iterators:
std::string text((std::istreambuf_iterator<char>(inputFile)),
std::istreambuf_iterator<char>());
Note, that the extra pair of parenthesis around the first parameter is, unfortunately, necessary (if you are using C++ 2011 you can avoid them by using braces, instead of parenthesis).
Use getline to do the reading.
string numbers;
if (inputFile.is_open())//checking if open
{
getline (inputFile,numbers); //fetches entire line into string numbers
inputFile.close();
}
Your program does behave exactly as in your description : inputFile >> numbers; just extract the first integer in the input file, so if you suppress the tab, inputFile>> will extract the number 12345, not 5 five numbers [1,2,3,4,5].
a better method :
vector< int > numbers;
string fileName = "text.txt";
ifstream inputFile;
inputFile.open(fileName.c_str(),ios_base::in);
char c;
while (inputFile.good()) // loop while extraction from file is possible
{
c = inputFile.get(); // get character from file
if ( inputFile.good() and c!= '\t' and c!=' ' ) // not sure of tab and space encoding in C++
{
numbers.push_back( (int) c);
}
}
inputFile.close();
I can read the number of lines easy, using:
ifstream in(file);
string content;
while(getline(in, content))
{
// do stuff
}
Or I can read the number of words and characters easy using something like:
ifstream in(file)
string content;
int numOfCharacters = 0;
int numOfWords = 0;
while(in >> content)
{
++numOfWords;
numOfCharacters += content.size();
}
But I dont want to read the file twice. How can I read the file once, and find out the number of lines, words and characters?
PS: I would welcome a Boost sugestion, if there is a easy way.
Thank you.
Read the line and for each line count the words. See stringstream for the second part.
(I'm not giving more information, that looks too much like an homework).
This could be done with a trivial boost.spirit.qi parser.
Sticking with the iostreams solution: you could create a strstream out of each line read via getline(), and do the word/char counting operations on it, accumulating across all the lines.
I am able to read my text file. Now i would like to parse the string line by line.
I am using header file and cpp file..
can anyone help me with parsing tutorial.
Where can find a good tutorial for parsing?
You can try http://www.cppreference.com/wiki/ and look at examples of using stringstreams.
I don't see what this has to do with header files, but here's how you parse a stream line by line:
void read_line(std::istream& is)
{
// read the lisn from is, for example: reading whitespace-delimited words:
std::string word;
while(is >> word)
process_word(word);
if( !is.eof() ) // some other error?
throw "Dude, you need better error handling!";
}
void read_file(std::istream& is)
{
for(;;)
{
std::string line;
if( !std::getline(is,line) )
break;
std::istringstream iss(line);
read_line(iss);
}
if( !is.eof() ) // some other error?
throw "Dude, you need better error handling!";
}
Try this:
#include <iostream>
#include <vector>
#include <fstream>
#include <string>
using namespace std;
int main()
{
ifstream fs("myFile.txt");
string input;
vector<string> sets;
while( getline(fs, input) )
sets.push_back(input);
}
First you need to know if the lines contain fixed length fields or are the fields variable length. Fixed length fields are usually padded with some character such as spaces or zeros. Variable length fields are usually terminated by a character such as a comma or tab.
Variable Length Fields
Use the std::string::find or std::string::find_first to find the ending character; also account for the end of the string as the last field may not contain the terminating character. Use this position to determine the length of the field (ending field position - starting field position). Finally, use std::string::substr to extract the field's content.
Fixed Length Fields
Use the std::string::substr method to extract the text. The starting and ending positions can be calculated using the accumulated lengths of the previous fields, if any, and the size of the current field.
Converting Field Text
The contents of the field may not be a string and will need to be converted to an internal data type. For example, a number. Use std::istringstream to convert the text of the field to an internal data type.
once again I ask for help. I haven't coded anything for sometime!
Now I have a text file filled with random gibberish. I already have a basic idea on how I will count the number of occurrences per word.
What really stumps me is how I will determine what line the word is in. Gut instinct tells me to look for the newline character at the end of each line. However I have to do this while going through the text file the first time right? Since if I do it afterwords it will do no good.
I already am getting the words via the following code:
vector<string> words;
string currentWord;
while(!inputFile.eof())
{
inputFile >> currentWord;
words.push_back(currentWord);
}
This is for a text file with no set structure. Using the above code gives me a nice little(big) vector of words, but it doesn't give me the line they occur in.
Would I have to get the entire line, then process it into words to make this possible?
Use a std::map<std::string, int> to count the word occurrences -- the int is the number of times it exists.
If you need like by line input, use std::getline(std::istream&, std::string&), like this:
std::vector<std::string> lines;
std::ifstream file(...) //Fill in accordingly.
std::string currentLine;
while(std::getline(file, currentLine))
lines.push_back(currentLine);
You can split a line apart by putting it into an std::istringstream first and then using operator>>. (Alternately, you could cobble up some sort of splitter using std::find and other algorithmic primitaves)
EDIT: This is the same thing as in #dash-tom-bang's answer, but modified to be correct with respect to error handing:
vector<string> words;
int currentLine = 1; // or 0, however you wish to count...
string line;
while (getline(inputFile, line))
{
istringstream inputString(line);
string word;
while (inputString >> word)
words.push_back(pair(word, currentLine));
}
Short and sweet.
vector< map< string, size_t > > line_word_counts;
string line, word;
while ( getline( cin, line ) ) {
line_word_counts.push_back();
map< string, size_t > &word_counts = line_word_counts.back();
istringstream line_is( line );
while ( is >> word ) ++ word_counts[ word ];
}
cout << "'Hello' appears on line 5 " << line_word_counts[5-1]["Hello"]
<< " times\n";
You're going to have to abandon reading into strings, because operator >>(istream&, string&) discards white space and the contents of the white space (== '\n' or != '\n', that is the question...) is what will give you line numbers.
This is where OOP can save the day. You need to write a class to act as a "front end" for reading from the file. Its job will be to buffer data from the file, and return words one at a time to the caller.
Internally, the class needs to read data from the file a block (say, 4096 bytes) at a time. Then a string GetWord() (yes, returning by value here is good) method will:
First, read any white space characters, taking care to increment the object's lineNumber member every time it hits a \n.
Then read non-whitespace characters, putting them into the string object you'll be returning.
If it runs out of stuff to read, read the next block and continue.
If the you hit the end of file, the string you have is the whole word (which may be empty) and should be returned.
If the function returns an empty string, that tells the caller that the end of file has been reached. (Files usually end with whitespace characters, so reading whitespace characters cannot imply that there will be a word later on.)
Then you can call this method at the same place in your code as your cin >> line and the rest of the code doesn't need to know the details of your block buffering.
An alternative approach is to read things a line at a time, but all the read functions that would work for you require you to create a fixed-size buffer to read into beforehand, and if the line is longer than that buffer, you have to deal with it somehow. It could get more complicated than the class I described.