Use of strtok to obtain certain strings from an input string - c++

vector<string> CategoryWithoutHashTags;
string tester = "#hello junk #world something #cool";
char *pch;
char *str;
str = new char [tester.size()+1];
strcpy(str, tester.c_str());
pch = strtok(str,"#");
while(pch!=NULL)
{
CategoryWithoutHashTags.push_back(pch);
pch=strtok(NULL,"#");
}
cout<<CategoryWithoutHashTags[0]<<endl;
I want to write a program which involves storing all the hash tags words in a vector of strings. The above program stores "hello junk" in the first index rather than "hello". What changes can i make to the program to make it do so?

If you are set on using strtok, you should at the very least use its re-entrant version strtok_r. Then, you should change the code to split at spaces, not at hash marks. This would give you the tokens. Finally, in the loop you would need to look for the first character to be the hash mark, adding the item to the list if it's there, and disregarding the item when the hash mark is not there.
An even better approach would be using a string stream: put your string into it, read tokens one by one, and discard ones with no hash mark.
Here is how you can do it with very little code using C++11's lambdas:
stringstream iss("#hello junk #world something #cool");
istream_iterator<string> eos;
istream_iterator<string> iit(iss);
vector<string> res;
back_insert_iterator< vector<string> > back_ins(res);
copy_if(iit, eos, back_ins, [](const string s) { return s[0] == '#'; } );
Demo on ideone.

Related

Converting strings to char arrays using cstring

I have a small assignment which partly requires me to take inputs from a file in the form of strings and place them into char arrays so I can check if the string contains any '*' character at the end of it.
I have been able to extract the strings from the files successfully, however i have failed to find a way in which to place them in char arrays so i can process them.
I would be very grateful if someone would let me know how to place a string into char arrays using cstring library. Please keep in mind that the strings are taken from a file and not as user input.
some of the ways i tried is the following:
//Try 1
char CstringArray[] = LineFromFile;
//Try 2
char CstringArray[100] = LineFromFile;
//Try 3
ifstream Test("Test.txt");
Test>>CstringArray;
//Try 4
ifstream Test("Test.txt");
Test>>CstringArray[0];
Thank you very much
Since this is an assignment, your professor will probably not be happy with you using all of C++'s functionality, particularly if you don't understand it, but since it's a one liner I figured I'd tell you how I'd print all strings ending in an asterisks. Given that you have successfully opened the file to ifstream Test you can do:
copy_if(istream_iterator<string>(Test), istream_iterator<string>(), ostream_iterator<string>(cout, " "), [](const auto& i) { return !empty(i) && i.back() == '*'; })
EDIT:
I'm using an istream_iterator to read in each string in Test and istream_iterator, I'm operating on these values immediately, but if you needed to start by saving all the strings to a vector<string> you could also do this: vector<string> CstringArray{ istream_iterator<string>(Test), istream_iterator<string>() }
I'm using an ostream_iterator to directly stream out my selected strings rather than storing them
I'm using copy_if to iterate over all the strings that are streamed in, selecting only those that meet a given criteria
I'm using the lambda: [](const auto& i) { return !empty(i) && i.back() == '*'; } to conditionally select non-empty strings which end with an asterisks character

Can I use 2 or more delimiters in C++ function getline? [duplicate]

This question already has answers here:
How can I read and parse CSV files in C++?
(39 answers)
Closed 4 years ago.
I would like to know how can I use 2 or more delimiters in the getline functon, that's my problem:
The program reads a text file... each line is goning to be like:
New Your, Paris, 100
CityA, CityB, 200
I am using getline(file, line), but I got the whole line, when I want to to get CityA, then CityB and then the number; and if I use ',' delimiter, I won't know when is the next line, so I'm trying to figure out some solution..
Though, how could I use comma and \n as a delimiter?
By the way,I'm manipulating string type,not char, so strtok is not possible :/
some scratch:
string line;
ifstream file("text.txt");
if(file.is_open())
while(!file.eof()){
getline(file, line);
// here I need to get each string before comma and \n
}
You can read a line using std::getline, then pass the line to a std::stringstream and read the comma separated values off it
string line;
ifstream file("text.txt");
if(file.is_open()){
while(getline(file, line)){ // get a whole line
std::stringstream ss(line);
while(getline(ss, line, ',')){
// You now have separate entites here
}
}
No, std::getline() only accepts a single character, to override the default delimiter. std::getline() does not have an option for multiple alternate delimiters.
The correct way to parse this kind of input is to use the default std::getline() to read the entire line into a std::string, then construct a std::istringstream, and then parse it further, into comma-separate values.
However, if you are truly parsing comma-separated values, you should be using a proper CSV parser.
Often, it is more intuitive and efficient to parse character input in a hierarchical, tree-like manner, where you start by splitting the string into its major blocks, then go on to process each of the blocks, splitting them up into smaller parts, and so on.
An alternative to this is to tokenize like strtok does -- from the beginning of input, handling one token at a time until the end of input is encountered. This may be preferred when parsing simple inputs, because its is straightforward to implement. This style can also be used when parsing inputs with nested structure, but this requires maintaining some kind of context information, which might grow too complex to maintain inside a single function or limited region of code.
Someone relying on the C++ std library usually ends up using a std::stringstream, along with std::getline to tokenize string input. But, this only gives you one delimiter. They would never consider using strtok, because it is a non-reentrant piece of junk from the C runtime library. So, they end up using streams, and with only one delimiter, one is obligated to use a hierarchical parsing style.
But zneak brought up std::string::find_first_of, which takes a set of characters and returns the position nearest to the beginning of the string containing a character from the set. And there are other member functions: find_last_of, find_first_not_of, and more, which seem to exist for the sole purpose of parsing strings. But std::string stops short of providing useful tokenizing functions.
Another option is the <regex> library, which can do anything you want, but it is new and you will need to get used to its syntax.
But, with very little effort, you can leverage existing functions in std::string to perform tokenizing tasks, and without resorting to streams. Here is a simple example. get_to() is the tokenizing function and tokenize demonstrates how it is used.
The code in this example will be slower than strtok, because it constantly erases characters from the beginning of the string being parsed, and also copies and returns substrings. This makes the code easy to understand, but it does not mean more efficient tokenizing is impossible. It wouldn't even be that much more complicated than this -- you would just keep track of your current position, use this as the start argument in std::string member functions, and never alter the source string. And even better techniques exist, no doubt.
To understand the example's code, start at the bottom, where main() is and where you can see how the functions are used. The top of this code is dominated by basic utility functions and dumb comments.
#include <iostream>
#include <string>
#include <utility>
namespace string_parsing {
// in-place trim whitespace off ends of a std::string
inline void trim(std::string &str) {
auto space_is_it = [] (char c) {
// A few asks:
// * Suppress criticism WRT localization concerns
// * Avoid jumping to conclusions! And seeing monsters everywhere!
// Things like...ah! Believing "thoughts" that assumptions were made
// regarding character encoding.
// * If an obvious, portable alternative exists within the C++ Standard Library,
// you will see it in 2.0, so no new defect tickets, please.
// * Go ahead and ignore the rumor that using lambdas just to get
// local function definitions is "cheap" or "dumb" or "ignorant."
// That's the latest round of FUD from...*mumble*.
return c > '\0' && c <= ' ';
};
for(auto rit = str.rbegin(); rit != str.rend(); ++rit) {
if(!space_is_it(*rit)) {
if(rit != str.rbegin()) {
str.erase(&*rit - &*str.begin() + 1);
}
for(auto fit=str.begin(); fit != str.end(); ++fit) {
if(!space_is_it(*fit)) {
if(fit != str.begin()) {
str.erase(str.begin(), fit);
}
return;
} } } }
str.clear();
}
// get_to(string, <delimiter set> [, delimiter])
// The input+output argument "string" is searched for the first occurance of one
// from a set of delimiters. All characters to the left of, and the delimiter itself
// are deleted in-place, and the substring which was to the left of the delimiter is
// returned, with whitespace trimmed.
// <delimiter set> is forwarded to std::string::find_first_of, so its type may match
// whatever this function's overloads accept, but this is usually expressed
// as a string literal: ", \n" matches commas, spaces and linefeeds.
// The optional output argument "found_delimiter" receives the delimiter character just found.
template <typename D>
inline std::string get_to(std::string& str, D&& delimiters, char& found_delimiter) {
const auto pos = str.find_first_of(std::forward<D>(delimiters));
if(pos == std::string::npos) {
// When none of the delimiters are present,
// clear the string and return its last value.
// This effectively makes the end of a string an
// implied delimiter.
// This behavior is convenient for parsers which
// consume chunks of a string, looping until
// the string is empty.
// Without this feature, it would be possible to
// continue looping forever, when an iteration
// leaves the string unchanged, usually caused by
// a syntax error in the source string.
// So the implied end-of-string delimiter takes
// away the caller's burden of anticipating and
// handling the range of possible errors.
found_delimiter = '\0';
std::string result;
std::swap(result, str);
trim(result);
return result;
}
found_delimiter = str[pos];
auto left = str.substr(0, pos);
trim(left);
str.erase(0, pos + 1);
return left;
}
template <typename D>
inline std::string get_to(std::string& str, D&& delimiters) {
char discarded_delimiter;
return get_to(str, std::forward<D>(delimiters), discarded_delimiter);
}
inline std::string pad_right(const std::string& str,
std::string::size_type min_length,
char pad_char=' ')
{
if(str.length() >= min_length ) return str;
return str + std::string(min_length - str.length(), pad_char);
}
inline void tokenize(std::string source) {
std::cout << source << "\n\n";
bool quote_opened = false;
while(!source.empty()) {
// If we just encountered an open-quote, only include the quote character
// in the delimiter set, so that a quoted token may contain any of the
// other delimiters.
const char* delimiter_set = quote_opened ? "'" : ",'{}";
char delimiter;
auto token = get_to(source, delimiter_set, delimiter);
quote_opened = delimiter == '\'' && !quote_opened;
std::cout << " " << pad_right('[' + token + ']', 16)
<< " " << delimiter << '\n';
}
std::cout << '\n';
}
}
int main() {
string_parsing::tokenize("{1.5, null, 88, 'hi, {there}!'}");
}
This outputs:
{1.5, null, 88, 'hi, {there}!'}
[] {
[1.5] ,
[null] ,
[88] ,
[] '
[hi, {there}!] '
[] }
I don't think that's how you should attack the problem (even if you could do it); instead:
Use what you have to read in each line
Then split up that line by the commas to get the pieces that you want.
If strtok will do the job for #2, you can always convert your string into a char array.

Manipulating specific position in string according to one another

I have a string which has different city names. The format like this :
string cities = "-Paris-Berlin-Cologne-"
And also there is another string which contains some voting results for these cities.
string vote = "-31.2-42.5-40-"
I need to define a function which takes input from user for ex : Berlin
Then function will find and change the result of vote in "string vote"
I tried with counting "-" separators but I couldn't succeed.
Any help would appreciate.
Thank you.
Split both strings at the separators and put the results in a map
std::vector<std::string> split(std::string const& s, std::string const& sep);
std::vector<std::string> const c = split(cities, "-");
std::vector<std::string> const v = split(votes, "-");
// now put them all in a map
assert(c.size() == v.size());
std::map<std::string, float> city2vote;
for (std::vector<std::string>::size_type i=0; i != c.size(); ++i)
{
city2vote[c[i]] = atof(v[i]);
}
// update vote for Berlin
city2vote["Berlin"] = 42.0;
The split function is strait forward, but Boost also provides an implementation if you can use libraries. I used atof for simplicity. In real life, a stringstream would be a much better choice.
Would using a regex search to find the positions in the string of the hyphens, then changing the values between those positions accordingly be an acceptable solution?
Edit - On second thoughts, you needn't overcomplicate things by using regex, you could just use a for loop to scan through the string
First I think you should map each delimited string from cities to vote. You can do it like this:
std::map<std::string, std:string> city_to_vote;
std::istringstream i1(cities), i2(vote);
for (std::string str1, str2; std::getline(i1, str1, '-') &&
std::getline(i2, str2, '-'); )
{
if (!str1.empty() && !str2.empty())
{
city_to_vote.emplace(std::make_pair(str1, str2));
}
}
Then once you receive string input from the user, check if it is inside the map, access the value that it is mapped to, and use a replace algorithm to augment the vote string.

Wordlist transfer for an anagram program

I'm almost finished with my program, but there's one last bug that I'm having problems ferreting out. The program is supposed to check about 10 scrambled words against a wordlist to see what the scrambled words are anagrams of. To do this, I alphabetized each word in the wordlist (apple would become aelpp), set that as the key of a map, and made the corresponding entry the original, unalphabetized word.
The program is messing up when it comes to the entries in the map. When the entry is six characters or less, the program tags a random character on the end of the string. I've narrowed down what can be causing the problem to a single loop:
while(myFile){
myFile.getline(str, 30);
int h=0;
for (; str[h] != 0; h++)//setting the initial version of str
{
strInit[h]=str[h]; //strInit is what becomes the entry into the map.
}
strInit[h+1]='\0'; //I didn't know if the for loop would include the null char
cout<<strInit; //Personal error-checking; not necessary for the program
}
And if it's necessary, here's the entire program:
Program
Prevent issues, use normal functions:
getline(str, 30);
strncpy(strInit, str, 30);
Prevent more issues, use standard strings:
std::string strInit, str;
while (std::getline(myFile, str)) {
strInit = str;
// do stuff
}
Best not to use raw C arrays at all! Here's a version, using modern C++:
#include <string>
std::string str;
while (std::getline(myFile, str))
{
// do something useful with str
// Example: mymap[str] = f(str);
std::cout << str; //Personal error-checking; not necessary for the program
}

Split a wstring by specified separator

I have a std::wstring variable that contains a text and I need to split it by separator. How could I do this? I wouldn't use boost that generate some warnings. Thank you
EDIT 1
this is an example text:
hi how are you?
and this is the code:
typedef boost::tokenizer<boost::char_separator<wchar_t>, std::wstring::const_iterator, std::wstring> Tok;
boost::char_separator<wchar_t> sep;
Tok tok(this->m_inputText, sep);
for(Tok::iterator tok_iter = tok.begin(); tok_iter != tok.end(); ++tok_iter)
{
cout << *tok_iter;
}
the results are:
hi
how
are
you
?
I don't understand why the last character is always splitted in another token...
In your code, question mark appears on a separate line because that's how boost::tokenizer works by default.
If your desired output is four tokens ("hi", "how", "are", and "you?"), you could
a) change char_separator you're using to
boost::char_separator<wchar_t> sep(L" ", L"");
b) use boost::split which, I think, is the most direct answer to "split a wstring by specified character"
#include <string>
#include <iostream>
#include <vector>
#include <boost/algorithm/string.hpp>
int main()
{
std::wstring m_inputText = L"hi how are you?";
std::vector<std::wstring> tok;
split(tok, m_inputText, boost::is_any_of(L" "));
for(std::vector<std::wstring>::iterator tok_iter = tok.begin();
tok_iter != tok.end(); ++tok_iter)
{
std::wcout << *tok_iter << '\n';
}
}
test run: https://ideone.com/jOeH9
You're default constructing boost::char_separator. The documentation says:
The function std::isspace() is used to identify dropped delimiters and std::ispunct() is used to identify kept delimiters. In addition, empty tokens are dropped.
Since std::ispunct(L'?') is true, it is treated as a "kept" delimiter, and reported as a separate token.
Hi you can use wcstok function
You said you don't want boost so...
This is maybe a wierd approach to use in C++ but I used it one in a MUD where i needed a lot of tokenization in C.
take this block of memory assigned to the char * chars:
char chars[] = "I like to fiddle with memory";
If you need to tokenize on a space character:
create array of char* called splitvalues big enough to store all tokens
while not increment pointer chars and compare value to '\0'
if not already set set address of splitvalues[counter] to current memory address - 1
if value is ' ' write 0 there
increment counter
when you finish you have the original string destroyed so do not use it, instead you have the array of strings pointing to the tokens. the count of tokens is the counter variable (upperbound of the array).
the approach is this:
iterate the string and on first occurence update token start pointer
convert the char you need to split on to zeroes that mean string termination in C
count how many times you did this
PS. Not sure if you can use a similar approach in a unicode environment tough.