String Tokenization with " ", breaking at "." - c++

Question

Encountering an issue where I am trying to tokenize a string that came from a getline from console. It's breaking at " " as well as "." when I only want it to break at " ".
Here is the method that I've created for this problem:
queue<string> getStringList(string list, string delimiter) {
queue<string> values;
size_t pos = 0;
string token;
while ((pos = list.find(delimiter) != string::npos))
{
token = list.substr(0, pos);
std::cout << token << std::endl;
values.push(token);
list.erase(0, pos + delimiter.length());
}
std::cout << list << endl;
return values;
}
Sample Input:
cid arrival_time dir_original dir_target
0 1.1 ^ ^
1 2.0 ^ ^
2 3.3 ^ <
3 3.5 v v
4 4.2 v >
5 4.4 ^ ^
6 5.7 > ^
7 5.9 < ^
This how the data is breaking up from the line previous.

Answer 1

while ((pos = list.find(delimiter) != string::npos))
You have the parentheses in the wrong place, meaning what you have is the equivalent of:
while ((pos = (list.find(delimiter) != string::npos) ))
// (compare first-----------------------)
// (then assign----------------------------------)
In terms of operator precedence, assignment is way down the list, long after comparisons.
So pos will always be either zero (if you don't find the space) or one (if you do find it). It will never be equal to the position of that space in the string, except for the pathological case where the space is the second character.
That's why everything works for your one-character fields, but not for the three-character 1.1 field. It correctly finds the space at position 3 but, because pos has been set to one, it uses substr in a way that deletes the ..
That line should be:
while ((pos = list.find(delimiter)) != string::npos)
// (assign first--------------)
// (then compare-------------------------------)
With a full program and the fix mentioned above:
#include <iostream>
#include <queue>
using namespace std;
queue<string> getStringList(string list, string delimiter) {
queue<string> values;
size_t pos = 0;
string token;
while ((pos = list.find(delimiter)) != string::npos) {
token = list.substr(0, pos);
std::cout << token << std::endl;
values.push(token);
list.erase(0, pos + delimiter.length());
}
values.push(list);
std::cout << list << endl;
return values;
}
int main () {
string x = "0 1.1 ^ ^";
string delim = " ";
auto y = getStringList(x, delim);
return 0;
}
You can see the output is now correct:
0
1.1
^
^
And, as per a comment from Remy Lebeau, you print the final element but do not add it to the list. Hence I've added the line that appends that last token to the list of tokens.
One other thing to watch out for. While this works well with single-space separators, multi-space ones may cause issues. If that becomes an issue, you may need a more "bullet-proof" tokenising routine.