c++ splitting string on non alphabetic characters - c++

Question

I am reading in a file line by line which I want to split on non alphabetic characters and if possible remove all non alphabetic characters at same time so I wouldn't have to do it latter.
I would like to use isalpha, but cant figure out how to use that with str.find() or similar functions, as those usually take single delimiter as a a string.
while(getline(fileToOpen,str))
{
unsigned int pos= 0;
string token;
//transform(str.begin(),str.end(),str.begin(),::tolower);
while (pos = str.find_first_not_of("abcdefghijklmnopqrstuvwxyzQWERTYUIOPASDFGHJKLZXCVBNM"))
{
token = str.substr(0, pos);
//transform(str.begin(),str.end(),str.begin(),::tolower);
Node<t>* ptr=search(token,root);
if (ptr!=NULL)
{
ptr->count++;
cout<<token<<" already in tree.Count "<<ptr->count<<"\n";
}
else
{
insert(token,root);
cout<<token<<" added to tree.\n";
}
ptr=NULL;
str.erase(0, pos);
}
}
My latest attempt which doesn't work... All of examples I could find were based on str.find("single delimiter")
Which is no good to me.
Found a way to use isalpha
template<typename t>
void Tree<t>::readFromFile(string filename)
{
string str;
ifstream fileToOpen(filename.c_str());
if (fileToOpen.is_open())
{
while(getline(fileToOpen,str))
{
unsigned int pos= 0;
string token;
//transform(str.begin(),str.end(),str.begin(),::tolower);
while (pos = find_if(str.begin(),str.end(),aZCheck)!=str.end()!=string::npos)
{
token = str.substr(0, pos);
transform(token.begin(),token.end(),token.begin(),::tolower);
Node<t>* ptr=search(token,root);
if (ptr!=NULL)
{
ptr->count++;
// cout<<token<<" already in tree.Count "<<ptr->count<<"\n";
}
else
{
insert(token,root);
cout<<token<<" added to tree.\n";
}
ptr=NULL;
str.erase(0, pos);
}
}
fileToOpen.close();
}
else
cout<<"Unable to open file!\n";
}
template<typename t>
inline bool Tree<t>::aZCheck(char c)
{
return !isalpha(c);
}
But issue still persists, string is getting split into single characters instead of words, and is whitespace considered valid by isalpha?

Answer 1

#include <algorithm>
#include <cctype>
...
template<typename t>
void Tree<t>::readFromFile(std::string filename)
{
std::string str;
std::ifstream fileToOpen(filename.c_str());
if (fileToOpen.is_open())
{
for (std::string::iterator pos, prev; std::getline(fileToOpen, str); )
{
for (pos = std::find_if(str.begin(), str.end(), isalpha); pos != str.end();
pos = std::find_if(prev, str.end(), isalpha))
{
prev = std::find_if_not(pos, str.end(), isalpha);
std::string token(pos, prev);
std::transform(token.begin(), token.end(), token.begin(), ::tolower);
Node<t>* ptr = search(token, root);
if (ptr != NULL)
{
ptr->count++;
// cout<< token << " already in tree.Count "<<ptr->count<<"\n";
}
else
{
insert(token, root);
cout << token << " added to tree.\n";
}
}
}
fileToOpen.close();
}
else
cout<<"Unable to open file!\n";
}
Online demo
Also since you say you want to save time, it would benefit you if your insert function does something extra. i.e. insert the value if it is not found in the tree, and set the counter at the position to 1. If the value is in the tree, simply increment the counter. This will save you from doing 2 iterations seeing as your tree might be potentially unbalanced

Answer 2

Try this test case. Two problems.
1 - Pos is 0 when a delimiter is found at the string start after truncation (or start)
This causes it to break out of the while. Use npos as a conditional check instead.
2 - You have to advance the postion past the delimiter when you erase, otherwise
it finds the same one over and over.
int pos= 0;
string token;
string str = "Thisis(asdfasdfasdf)and!this)))";
while ((pos=str.find_first_not_of("abcdefghijklmnopqrstuvwxyzQWERTYUIOPASDFGHJKLZXCVBNM"))!= string::npos )
{
if ( pos != 0 )
{
// Found a token
token = str.substr(0, pos);
cout << "Found: " << token << endl;
}
else
{
// Found another delimiter
// Just move on to next one
}
str.erase(0, pos+1); // Always remove pos+1 to get rid of delimiter
}
// Cover the last (or only) token
if ( str.length() > 0 )
{
token = str;
cout << "Found: " << token << endl;
}
Outputs >>
Found: Thisis
Found: asdfasdfasdf
Found: and
Found: this
Press any key to continue . . .