I've been looking for ways to count the number of words in a string, but specifically for strings that may contain typos (i.e. "_This_is_a___test" as opposed to "This_is_a_test"). Most of the pages I've looked at only handle single spaces.
This is actually my first time programming in C++, and I don't have much other programming experience to speak of (2 years of college in C and Java). Although what I have is functional, I'm also aware it's complex, and I'm wondering if there is a more efficient way to achieve the same results?
This is what I have currently. Before I run the string through numWords(), I run it through a trim function that removes leading whitespace, then check that there are still characters remaining.
int numWords(string str) {
int count = 1;
for (int i = 0; i < str.size(); i++) {
if (str[i] == ' ' || str[i] == '\t' || str[i] == '\n') {
bool repeat = true;
int j = 1;
while (j < (str.size() - i) && repeat) {
if (str[i + j] != ' ' && str[i + j] != '\t' && str[i + j] != '\n') {
repeat = false;
i = i + j;
count++;
}
else
j++;
}
}
}
return count;
}
Also, I wrote mine to take a string argument, but most of the examples I've seen used (char* str) instead, which I wasn't sure how to use with my input string.
You don't need all those stringstreams to count word boundary
#include <string>
#include <cctype>
int numWords(std::string str)
{
bool space = true; // not in word
int count = 0;
for(auto c:str){
if(std::isspace(c))space=true;
else{
if(space)++count;
space=false;
}
}
return count;
}
One solution is to utilize std::istringstream to count the number of words and to skip over spaces automatically.
#include <sstream>
#include <string>
#include <iostream>
int numWords(std::string str)
{
int count = 0;
std::istringstream strm(str);
std::string word;
while (strm >> word)
++count;
return count;
}
int main()
{
std::cout << numWords(" This is a test ");
}
Output:
4
Albeit as mentioned std::istringstream is more "heavier" in terms of performance than writing your own loop.
Sam's comment made me write a function that does not allocate strings for words. But just creates string_views on the input string.
#include <cassert>
#include <cctype>
#include <vector>
#include <string_view>
#include <iostream>
std::vector<std::string_view> get_words(const std::string& input)
{
std::vector<std::string_view> words;
// the first word begins at an alpha character
auto begin_of_word = std::find_if(input.begin(), input.end(), [](const char c) { return std::isalpha(c); });
auto end_of_word = input.begin();
auto end_of_input = input.end();
// parse the whole string
while (end_of_word != end_of_input)
{
// as long as you see text characters move end_of_word one back
while ((end_of_word != end_of_input) && std::isalpha(*end_of_word)) end_of_word++;
// create a string view from begin of word to end of word.
// no new string memory will be allocated
// std::vector will do some dynamic memory allocation to store string_view (metadata of word positions)
words.emplace_back(begin_of_word, end_of_word);
// then skip all non readable characters.
while ((end_of_word != end_of_input) && !std::isalpha(*end_of_word) ) end_of_word++;
// and if we haven't reached the end then we are at the beginning of a new word.
if ( end_of_word != input.end()) begin_of_word = end_of_word;
}
return words;
}
int main()
{
std::string input{ "This, this is a test!" };
auto words = get_words(input);
for (const auto& word : words)
{
std::cout << word << "\n";
}
return 0;
}
You can use standard function std::distance with std::istringstream the following way
#include <iostream>
#include <sstream>
#include <string>
#include <iterator>
int main()
{
std::string s( " This is a test" );
std::istringstream iss( s );
auto count = std::distance( std::istream_iterator<std::string>( iss ),
std::istream_iterator<std::string>() );
std::cout << count << '\n';
}
The program output is
4
If you want you can place the call of std::distance in a separate function like
#include <iostream>
#include <sstream>
#include <string>
#include <iterator>
size_t numWords( const std::string &s )
{
std::istringstream iss( s );
return std::distance( std::istream_iterator<std::string>( iss ),
std::istream_iterator<std::string>() );
}
int main()
{
std::string s( " This is a test" );
std::cout << numWords( s ) << '\n';
}
If separators can include other characters apart from white space characters as for example punctuations then you should use methods of the class std::string or std::string_view find_first_of and find_first_not_of.
Here is a demonstration program.
#include <iostream>
#include <string>
#include <string_view>
size_t numWords( const std::string_view s, std::string_view delim = " \t" )
{
size_t count = 0;
for ( std::string_view::size_type pos = 0;
( pos = s.find_first_not_of( delim, pos ) ) != std::string_view::npos;
pos = s.find_first_of( delim, pos ) )
{
++count;
}
return count;
}
int main()
{
std::string s( "Is it a test ? Yes ! Now we will run it ..." );
std::cout << numWords( s, " \t!?.," ) << '\n';
}
The program output is
10
you can do it easily with regex
int numWords(std::string str)
{
std::regex re("\\S+"); // or `[^ \t\n]+` to exactly match the question
return std::distance(
std::sregex_iterator(str.begin(), str.end(), re),
std::sregex_iterator()
);
}
I want to split the strings on each line of my text file into an array, similar to the split() function in python. my desired syntax is a loop that enters every split-string into the next index of an array,
so for example if my string:
"ab,cd,ef,gh,ij"
, every time I encounter a comma then I would:
datafile >> arr1[i]
and my array would end up:
arr1 = [ab,cd,ef,gh,ij]
a mock code without reading a text file is provided below
#include <iostream>
#include <fstream>
#include <stdio.h>
#include <string.h>
#include <string>
using namespace std;
int main(){
char str[] = "ab,cd,ef,gh,ij"; //" ex str in place of file contents/fstream sFile;"
const int NUM = 5;
string sArr[NUM];//empty array
char *token = strtok(str, ",");
for (int i=0; i < NUM; i++)
while((token!=NULL)){
("%s\n", token) >> sArr[i];
token = strtok(NULL, ",");
}
cout >> sArr;
return 0;
}
In C++ you can read a file line by line and directly get a std::string.
You will found below an example I made with a split() proposal as you requested, and a main() example of reading a file:
Example
data file:
ab,cd,ef,gh
ij,kl,mn
c++ code:
#include <fstream>
#include <iostream>
#include <vector>
std::vector<std::string> split(const std::string & s, char c);
int main()
{
std::string file_path("data.txt"); // I assumed you have that kind of file
std::ifstream in_s(file_path);
std::vector <std::vector<std::string>> content;
if(in_s)
{
std::string line;
std::vector <std::string> vec;
while(getline(in_s, line))
{
for(const std::string & str : split(line, ','))
vec.push_back(str);
content.push_back(vec);
vec.clear();
}
in_s.close();
}
else
std::cout << "Could not open: " + file_path << std::endl;
for(const std::vector<std::string> & str_vec : content)
{
for(unsigned int i = 0; i < str_vec.size(); ++i)
std::cout << str_vec[i] << ((i == str_vec.size()-1) ? ("") : (" : "));
std::cout << std::endl;
}
return 0;
}
std::vector<std::string> split(const std::string & s, char c)
{
std::vector<std::string> splitted;
std::string word;
for(char ch : s)
{
if((ch == c) && (!word.empty()))
{
splitted.push_back(word);
word.clear();
}
else
word += ch;
}
if(!word.empty())
splitted.push_back(word);
return splitted;
}
output:
ab : cd : ef : gh
ij : kl : mn
I hope it will help.
So, a few things to fix. Firstly, arrays and NUM are kind of limiting - you have to fix up NUM whenever you change the input string, so C++ provides std::vector which can resize itself to however many strings it finds. Secondly, you want to call strtok until it returns nullptr once, and you can do that with one loop. With both your for and NUM you call strtok too many times - even after it has returned nullptr. Next, to put the token into a std::string, you would assign using my_string = token; rather than ("%s\n", token) >> my_string - which is a broken mix of printf() formatting and C++ streaming notation. Lastly, to print the elements you've extracted, you can use another loop. All these changes are illustrated below.
char str[] = "ab,cd,ef,gh,ij";
std::vector<std::string> strings;
char* token = strtok(str, ",");
while ((token != nullptr))
{
strings.push_back(token);
token = strtok(NULL, ",");
}
for (const auto& s : strings)
cout >> s >> '\n';
Your code is overly complicated and wrong.
You probably want this:
#include <iostream>
#include <string>
#include <string.h>
using namespace std;
int main() {
char str[] = "ab,cd,ef,gh,ij"; //" ex str in place of file contents/fstream sFile;"
const int NUM = 5;
string sArr[NUM];//empty array
char *token = strtok(str, ",");
int max = 0;
while ((token != NULL)) {
sArr[max++] = token;
token = strtok(NULL, ",");
}
for (int i = 0; i < max; i++)
cout << sArr[i] << "\n";
return 0;
}
This code is still poor and no bound checking is done.
But anyway, you should rather do it the C++ way as suggested in the other answers.
Use boost::split
#include <boost/algorithm/string.hpp>
[...]
std::vector<std::string> strings;
std::string val("ab,cd,ef,gh,ij");
boost::split(strings, val, boost::is_any_of(","));
You could do something like this
std::string str = "ab,cd,ef,gh,ij";
std::vector<std::string> TokenList;
std::string::size_type lastPos = 0;
std::string::size_type pos = str.find_first_of(',', lastPos);
while(pos != std::string::npos)
{
std::string temp(str, lastPos, pos - lastPos);
TokenList.push_back(temp);
lastPos = pos + 1;
pos = str.find_first_of(',', lastPos);
}
if(lastPos != str.size())
{
std::string temp(str, lastPos, str.size());
TokenList.push_back(temp);
}
for(int i = 0; i < TokenList.size(); i++)
std::cout << TokenList.at(i) << std::endl;
I want to split a string by any occurrence of and.
First of all I have to make it clear that I do not intend to use any regex as a delimiter.
I run the following code:
#include <iostream>
#include <regex>
#include <boost/algorithm/string.hpp>
int main()
{
std::vector<std::string> results;
std::string text=
"Alexievich, Svetlana and Lindahl,Tomas and Campbell,William";
boost::split(
results,
text,
boost::is_any_of(" and "),
boost::token_compress_off
);
for(auto result:results)
{
std::cout<<result<<"\n";
}
return 0;
}
and the results are different from what I expect:
Alexievich,
Svetl
Li
hl,Tom
s
C
mpbell,Willi
m
It seems every character in the delimiter acts separately while I need to have the whole and as a delimiter.
Please do not link to this boost example unless you are sure that it will work for my case.
<algorithm> contains search - right tool for this task.
vector<string> results;
const string text{ "Alexievich, Svetlana and Lindahl,Tomas and Campbell,William" };
const string delim{ " and " };
for (auto p = cbegin(text); p != cend(text); ) {
const auto n = search(p, cend(text), cbegin(delim), cend(delim));
results.emplace_back(p, n);
p = n;
if (cend(text) != n) // we found delim, skip over it.
p += delim.length();
}
The old-fashioned way:
#include <iostream>
#include <string>
#include <vector>
int main()
{
std::vector<std::string> results;
std::string text=
"Alexievich, Svetlana and Lindahl,Tomas and Campbell,William";
size_t pos = 0;
for (;;) {
size_t next = text.find("and", pos);
results.push_back(text.substr(pos, next - pos));
if (next == std::string::npos) break;
pos = next + 3;
}
for(auto result:results)
{
std::cout<<result<<"\n";
}
return 0;
}
Packaging into a reusable function is left as an exercise for the reader.
This question already has answers here:
How do I iterate over the words of a string?
(84 answers)
Closed 4 years ago.
If I have a std::string containing a comma-separated list of numbers, what's the simplest way to parse out the numbers and put them in an integer array?
I don't want to generalise this out into parsing anything else. Just a simple string of comma separated integer numbers such as "1,1,1,1,2,1,1,1,0".
Input one number at a time, and check whether the following character is ,. If so, discard it.
#include <vector>
#include <string>
#include <sstream>
#include <iostream>
int main()
{
std::string str = "1,2,3,4,5,6";
std::vector<int> vect;
std::stringstream ss(str);
for (int i; ss >> i;) {
vect.push_back(i);
if (ss.peek() == ',')
ss.ignore();
}
for (std::size_t i = 0; i < vect.size(); i++)
std::cout << vect[i] << std::endl;
}
Something less verbose, std and takes anything separated by a comma.
stringstream ss( "1,1,1,1, or something else ,1,1,1,0" );
vector<string> result;
while( ss.good() )
{
string substr;
getline( ss, substr, ',' );
result.push_back( substr );
}
Yet another, rather different, approach: use a special locale that treats commas as white space:
#include <locale>
#include <vector>
struct csv_reader: std::ctype<char> {
csv_reader(): std::ctype<char>(get_table()) {}
static std::ctype_base::mask const* get_table() {
static std::vector<std::ctype_base::mask> rc(table_size, std::ctype_base::mask());
rc[','] = std::ctype_base::space;
rc['\n'] = std::ctype_base::space;
rc[' '] = std::ctype_base::space;
return &rc[0];
}
};
To use this, you imbue() a stream with a locale that includes this facet. Once you've done that, you can read numbers as if the commas weren't there at all. Just for example, we'll read comma-delimited numbers from input, and write then out one-per line on standard output:
#include <algorithm>
#include <iterator>
#include <iostream>
int main() {
std::cin.imbue(std::locale(std::locale(), new csv_reader()));
std::copy(std::istream_iterator<int>(std::cin),
std::istream_iterator<int>(),
std::ostream_iterator<int>(std::cout, "\n"));
return 0;
}
The C++ String Toolkit Library (Strtk) has the following solution to your problem:
#include <string>
#include <deque>
#include <vector>
#include "strtk.hpp"
int main()
{
std::string int_string = "1,2,3,4,5,6,7,8,9,10,11,12,13,14,15";
std::vector<int> int_list;
strtk::parse(int_string,",",int_list);
std::string double_string = "123.456|789.012|345.678|901.234|567.890";
std::deque<double> double_list;
strtk::parse(double_string,"|",double_list);
return 0;
}
More examples can be found Here
Alternative solution using generic algorithms and Boost.Tokenizer:
struct ToInt
{
int operator()(string const &str) { return atoi(str.c_str()); }
};
string values = "1,2,3,4,5,9,8,7,6";
vector<int> ints;
tokenizer<> tok(values);
transform(tok.begin(), tok.end(), back_inserter(ints), ToInt());
Lots of pretty terrible answers here so I'll add mine (including test program):
#include <string>
#include <iostream>
#include <cstddef>
template<typename StringFunction>
void splitString(const std::string &str, char delimiter, StringFunction f) {
std::size_t from = 0;
for (std::size_t i = 0; i < str.size(); ++i) {
if (str[i] == delimiter) {
f(str, from, i);
from = i + 1;
}
}
if (from <= str.size())
f(str, from, str.size());
}
int main(int argc, char* argv[]) {
if (argc != 2)
return 1;
splitString(argv[1], ',', [](const std::string &s, std::size_t from, std::size_t to) {
std::cout << "`" << s.substr(from, to - from) << "`\n";
});
return 0;
}
Nice properties:
No dependencies (e.g. boost)
Not an insane one-liner
Easy to understand (I hope)
Handles spaces perfectly fine
Doesn't allocate splits if you don't want to, e.g. you can process them with a lambda as shown.
Doesn't add characters one at a time - should be fast.
If using C++17 you could change it to use a std::stringview and then it won't do any allocations and should be extremely fast.
Some design choices you may wish to change:
Empty entries are not ignored.
An empty string will call f() once.
Example inputs and outputs:
"" -> {""}
"," -> {"", ""}
"1," -> {"1", ""}
"1" -> {"1"}
" " -> {" "}
"1, 2," -> {"1", " 2", ""}
" ,, " -> {" ", "", " "}
You could also use the following function.
void tokenize(const string& str, vector<string>& tokens, const string& delimiters = ",")
{
// Skip delimiters at beginning.
string::size_type lastPos = str.find_first_not_of(delimiters, 0);
// Find first non-delimiter.
string::size_type pos = str.find_first_of(delimiters, lastPos);
while (string::npos != pos || string::npos != lastPos) {
// Found a token, add it to the vector.
tokens.push_back(str.substr(lastPos, pos - lastPos));
// Skip delimiters.
lastPos = str.find_first_not_of(delimiters, pos);
// Find next non-delimiter.
pos = str.find_first_of(delimiters, lastPos);
}
}
std::string input="1,1,1,1,2,1,1,1,0";
std::vector<long> output;
for(std::string::size_type p0=0,p1=input.find(',');
p1!=std::string::npos || p0!=std::string::npos;
(p0=(p1==std::string::npos)?p1:++p1),p1=input.find(',',p0) )
output.push_back( strtol(input.c_str()+p0,NULL,0) );
It would be a good idea to check for conversion errors in strtol(), of course. Maybe the code may benefit from some other error checks as well.
I'm surprised no one has proposed a solution using std::regex yet:
#include <string>
#include <algorithm>
#include <vector>
#include <regex>
void parse_csint( const std::string& str, std::vector<int>& result ) {
typedef std::regex_iterator<std::string::const_iterator> re_iterator;
typedef re_iterator::value_type re_iterated;
std::regex re("(\\d+)");
re_iterator rit( str.begin(), str.end(), re );
re_iterator rend;
std::transform( rit, rend, std::back_inserter(result),
[]( const re_iterated& it ){ return std::stoi(it[1]); } );
}
This function inserts all integers at the back of the input vector. You can tweak the regular expression to include negative integers, or floating point numbers, etc.
#include <sstream>
#include <vector>
const char *input = "1,1,1,1,2,1,1,1,0";
int main() {
std::stringstream ss(input);
std::vector<int> output;
int i;
while (ss >> i) {
output.push_back(i);
ss.ignore(1);
}
}
Bad input (for instance consecutive separators) will mess this up, but you did say simple.
string exp = "token1 token2 token3";
char delimiter = ' ';
vector<string> str;
string acc = "";
for(int i = 0; i < exp.size(); i++)
{
if(exp[i] == delimiter)
{
str.push_back(acc);
acc = "";
}
else
acc += exp[i];
}
bool GetList (const std::string& src, std::vector<int>& res)
{
using boost::lexical_cast;
using boost::bad_lexical_cast;
bool success = true;
typedef boost::tokenizer<boost::char_separator<char> > tokenizer;
boost::char_separator<char> sepa(",");
tokenizer tokens(src, sepa);
for (tokenizer::iterator tok_iter = tokens.begin();
tok_iter != tokens.end(); ++tok_iter) {
try {
res.push_back(lexical_cast<int>(*tok_iter));
}
catch (bad_lexical_cast &) {
success = false;
}
}
return success;
}
I cannot yet comment (getting started on the site) but added a more generic version of Jerry Coffin's fantastic ctype's derived class to his post.
Thanks Jerry for the super idea.
(Because it must be peer-reviewed, adding it here too temporarily)
struct SeparatorReader: std::ctype<char>
{
template<typename T>
SeparatorReader(const T &seps): std::ctype<char>(get_table(seps), true) {}
template<typename T>
std::ctype_base::mask const *get_table(const T &seps) {
auto &&rc = new std::ctype_base::mask[std::ctype<char>::table_size]();
for(auto &&sep: seps)
rc[static_cast<unsigned char>(sep)] = std::ctype_base::space;
return &rc[0];
}
};
This is the simplest way, which I used a lot. It works for any one-character delimiter.
#include<bits/stdc++.h>
using namespace std;
int main() {
string str;
cin >> str;
int temp;
vector<int> result;
char ch;
stringstream ss(str);
do
{
ss>>temp;
result.push_back(temp);
}while(ss>>ch);
for(int i=0 ; i < result.size() ; i++)
cout<<result[i]<<endl;
return 0;
}
simple structure, easily adaptable, easy maintenance.
std::string stringIn = "my,csv,,is 10233478,separated,by commas";
std::vector<std::string> commaSeparated(1);
int commaCounter = 0;
for (int i=0; i<stringIn.size(); i++) {
if (stringIn[i] == ",") {
commaSeparated.push_back("");
commaCounter++;
} else {
commaSeparated.at(commaCounter) += stringIn[i];
}
}
in the end you will have a vector of strings with every element in the sentence separated by spaces. empty strings are saved as separate items.
Simple Copy/Paste function, based on the boost tokenizer.
void strToIntArray(std::string string, int* array, int array_len) {
boost::tokenizer<> tok(string);
int i = 0;
for(boost::tokenizer<>::iterator beg=tok.begin(); beg!=tok.end();++beg){
if(i < array_len)
array[i] = atoi(beg->c_str());
i++;
}
void ExplodeString( const std::string& string, const char separator, std::list<int>& result ) {
if( string.size() ) {
std::string::const_iterator last = string.begin();
for( std::string::const_iterator i=string.begin(); i!=string.end(); ++i ) {
if( *i == separator ) {
const std::string str(last,i);
int id = atoi(str.c_str());
result.push_back(id);
last = i;
++ last;
}
}
if( last != string.end() ) result.push_back( atoi(&*last) );
}
}
#include <sstream>
#include <vector>
#include <algorithm>
#include <iterator>
const char *input = ",,29870,1,abc,2,1,1,1,0";
int main()
{
std::stringstream ss(input);
std::vector<int> output;
int i;
while ( !ss.eof() )
{
int c = ss.peek() ;
if ( c < '0' || c > '9' )
{
ss.ignore(1);
continue;
}
if (ss >> i)
{
output.push_back(i);
}
}
std::copy(output.begin(), output.end(), std::ostream_iterator<int> (std::cout, " ") );
return 0;
}
I have an array of characters that I pass to this function that I read in from a file. They look like this:
7270706PRRío Cañas Abajo 1185 0.238885 18.037675 -66.46701029126 2.6966
I want to extract the chars signifying the name of a place from the 9th position of this char array up until the stream of spaces (so in this case "Canas Abajo"). To do this, I tried, calling with a = 9 and stop = 40:
string getPlaceName(const char (&in) [110], int a, int stop){
char ext[48];
int p =0; int sig = 0;
string toret;
for(a, p; a < stop; a+=1, p+=1)
ext[p] = in[a];
char * v = strtok(ext, " ");
while (v) {
if(sig == 0)
{string q = (v); toret = q; sig= 1;}
else
{string l = (v); toret = toret +" " +l;}
v = strtok(NULL, " ");
}
cout<<"|"<<toret<<"|"<<endl;
return toret;
}
This works for files where the last letter of the string I'm trying to extract is at the end of the line, yet in the example string given, it is returning a jumble of random characters after the name and implies there is no proper null termination. This happens with and without the string q = (v) and string l = (v). What is wrong with my attempt at combining the string and returning?
Tokenize the string, use strtok(). You can also use std::string::find() method. You can also use std::copy() algorithm. stream iterators are used to copy the stream to the vector as whitespace separated strings.
Use string::substr() to extract a substring from - to position.
char myString[] = "7270706PRRío Cañas Abajo 1185 0.238885 18.037675 -66.46701029126 2.6966";
char *p = strtok(myString, " ");
while (p) {
printf ("Token: %s\n", p);
p = strtok(NULL, " ");
}
OR
#include <string>
#include <vector>
#include <iostream>
#include <istream>
#include <ostream>
#include <iterator>
#include <sstream>
#include <algorithm>
int main()
{
std::string str = "7270706PRRío Cañas Abajo 1185 0.238885 18.037675 -66.46701029126 2.6966";
std::stringstream strstr(str);
std::istream_iterator<std::string> it(strstr);
std::istream_iterator<std::string> end;
std::vector<std::string> results(it, end);
std::ostream_iterator<std::string> oit(std::cout);
std::copy(results.begin(), results.end(), oit);
}
I've used a method for this before:
std::vector<std::string> Split(const std::string& text, const char separator)
{
std::vector<std::string> tokens;
int start = 0;
int end = 0;
while ((end = text.find(separator, start)) != std::string::npos)
{
tokens.push_back(text.substr(start, end - start));
start = end + 1;
}
tokens.push_back(text.substr(start));
return tokens;
}
Use this to get a list of tokens and then select the 1st and 2nd index.