Reading alphabetical characters only from file - c++ - c++

Question

I am to read words from a text file. Word is defined as a consecutive sequence of letters. So for example in the following string:
"It’s a ver5y good #” idea of a line. You know it?"
the words are:
it s a ver y good idea of line you know
('it' and 'a' are doubled)
I was wondering, if there's any clever function that reads words until it finds a non-alphabetical character? Or the only way to do it is to read char by char and use push_back until we find non-alphabetical one?

Answer 1

When you read a string from a stream, the stream reads a contiguous run of non-white-space characters as the string. It then ignores any white-space characters. The next non-white-space character is the beginning of the next string it'll read. This is pretty much the behavior you want, with one more exception: you want everything other than letters to be treated like white-space.
Fortunately, the stream doesn't hard-code its idea of what's "white space". It uses a locale to tell it what's white space. A locale, in turn, is composed of pieces that deal with individual aspects ("facets") of localization. The facet that deal specifically with classifying characters is a ctype facet. So, if we write a ctype facet that classifies everything other than a letter as white space, we can read "words" from the stream quite easily.
Here's some code to do exactly that:
struct alpha_only: std::ctype<char> {
alpha_only(): std::ctype<char>(get_table()) {}
static std::ctype_base::mask const* get_table() {
static std::vector<std::ctype_base::mask>
rc(std::ctype<char>::table_size,std::ctype_base::space);
std::fill(&rc['a'], &rc['z'], std::ctype_base::lower);
std::fill(&rc['A'], &rc['Z'], std::ctype_base::upper);
return &rc[0];
}
};
The char specialization of a ctype facet is (always) table driven. All we really have to do is create a table that classifies characters properly. In this case, that means alphabetical characters are classified as upper- or lower-case, and everything else is classified as white-space. We do that by filling the table with ctype_base::space, then for the alphabetical characters basically saying: "oops, no that's not white-space, that's upper- or lower-case.
Technically, the way I've done that is slightly incorrect--it assumes that upper-case and lower-case letters are contiguous. This is true of any sane character set, but not of EBCDIC. If we wanted to be technically correct, instead of the two "std::fill" calls, we could write a loop something like this:
auto max = std::numeric_limits<unsigned char>::max();
for (int i=0; i<max; i++)
if (islower(i))
table[i] = std::ctype_base::lower;
else if (isupper(i))
table[i] = std::ctype_base::upper;
else
table[i] = std::ctype_base::space;
Either way, the conclusion is fairly simple: upper case is upper case, lower case is lower case, everything else is "white space".
Once we've written that, we need to tell the stream to use that locale; then we can read our words really easily:
int main() {
std::istringstream infile("It’s a ver5y good #” idea of a line. You know it?");
// Tell the stream to use our character classifier:
infile.imbue(std::locale(std::locale(), new alpha_only));
std::string word;
while (infile >> word)
std::cout << word << "\n";
}
[I've put a new-line between each "word" so you can easily see what it's reading as a word.]
Result:
It
s
a
ver
y
good
idea
of
a
line
You
know
it
Based on your result in the question, you apparently also only want each word to appear once in the output. To do that, you'd typically insert each word in a set as its read, and only write it to the output if insertion in the set was successful.
std::unordered_set<std::string> words;
std::string word;
while (infile >> word)
if (words.insert(word).second)
std::cout << word << "\n";
The insert for set and unordered_set returns a pair<iterator, bool>, where the bool indicates whether insertion was successful. If it was previously present, that will fail and return false, so based on that we decide whether to write the word out or not.
With this modification, it still appears in the output twice--the first instance has the i capitalized, and the second doesn't. To filter that out, you'll need to convert each string entirely to lower-case (or entirely to upper-case) before inserting it into the set.