I have an std::ostringstream.
I would like to iterate for each line of this std::ostringstream.
I use boost::tokenizer :
std::ostringstream HtmlStream;
.............
typedef boost::tokenizer<boost::char_separator<char> > line_tokenizer;
line_tokenizer tok(HtmlStream.str(), boost::char_separator<char>("\n\r"));
for (line_tokenizer::const_iterator i = tok.begin(), end = tok.end(); i != end; ++i)
{
std::string str = *i;
}
On the line
for (line_tokenizer::const_iterator i = tok.begin(), end = tok.end(); i != end;
I have an assert error with "string iterator incompatible".
I have read about this error, on google and on StackOverflow too, but i have diffuclty to find my error.
Anyone could help me please ?
Thanks a lot,
Best regards,
Nixeus
I like to make it non-copying for efficiency/error reporting:
See it Live on Coliru
#include <boost/algorithm/string/split.hpp>
#include <boost/algorithm/string/classification.hpp>
#include <iostream>
#include <vector>
int main()
{
auto const& s = "hello\r\nworld";
std::vector<boost::iterator_range<char const*>> lines;
boost::split(lines, s, boost::is_any_of("\r\n"), boost::token_compress_on);
for (auto const& range : lines)
{
std::cout << "at " << (range.begin() - s) << ": '" << range << "'\n";
};
}
Prints:
at 0: 'hello'
at 7: 'world'
This is more efficient than most of the alternatives shown. Of course, if you need more parsing capabilities, consider Boost Spirit:
See it Live on Coliru
#include <boost/spirit/include/qi.hpp>
int main()
{
std::string const s = "hello\r\nworld";
std::vector<std::string> lines;
{
using namespace boost::spirit::qi;
auto f(std::begin(s)),
l(std::end(s));
bool ok = parse(f, l, *(char_-eol) % eol, lines);
}
for (auto const& range : lines)
{
std::cout << "'" << range << "'\n";
};
}
Related
#include <iostream>
#include <optional>
#include <string>
#include <boost/tokenizer.hpp>
int main() {
std::string a("http://website/some-path/,file1,file2");
char *ptr = (char *)a.c_str();
boost::char_separator<char> delim(",");
std::vector<std::string> pths{};
boost::tokenizer<boost::char_separator<char>> tokens(
std::string(ptr), delim);
std::optional<std::string> pref = std::nullopt;
for (const auto& tok : tokens) {
if (!pref) {
pref = tok;
std::cerr << "prfix is set: " << tok << std::endl;
continue;
}
pths.push_back(*pref + tok);
}
for(auto &t : pths) {
std::cout << t << std::endl;
}
}
My output:
prfix is set: �site/some-path/
�site/some-path/file1
�site/some-path/file2
The question is, what is wrong with the above? If I work with std::regex, it is fine.
EDIT: the scenario with *ptr is the one I actually had: the original string was passed to a function as char *, hence the above. This is to answer the comment by #273K.
A lot can be simplified, at once removing the problems:
Live On Coliru
#include <boost/tokenizer.hpp>
#include <iostream>
#include <optional>
#include <string>
auto generate(std::string const& a) {
boost::tokenizer tokens(a, boost::char_separator<char>{","});
std::optional<std::string> prefix;
std::vector<std::string> result;
for (const auto& tok : tokens) {
if (!prefix)
prefix = tok;
else
result.push_back(*prefix + tok);
}
return result;
}
int main() {
for (auto& t : generate("http://website/some-path/,file1,file2"))
std::cout << t << std::endl;
}
Prints
http://website/some-path/file1
http://website/some-path/file2
I need the positions of characters in a string.
The String contains:
"username":"secret", "password":"also secret", "id":"secret too", "token":"secret"
and I need the positions of the quotation marks from the token that are bold: "token":"secret".
I have experimented with the code from http://www.cplusplus.com/reference/string/string/find
but everything didn't work. Can anyone help me?
Here is what i have tried but it only gives out a 0:
#include <iostream>
#include <string>
int main() {
std::string buffer("\"username\":\"secret\", \"password\":\"also secret\", \"id\":\"secret too\", \"token\":\"secret\"");
size_t found = buffer.find('"');
if (found == std::string::npos)std::cout << "something went wrong\n";
if (found != std::string::npos)
std::cout << "first " << '"' << " found at: " << found << '\n';
for (int j = 0; j <= 17; ++j) {
found = buffer.find('"');
found + 1, 6;
if (found != std::string::npos)
std::cout << "second " << '"' << " found at : " << found << '\n';
}
return 0;
There are so many possible solutions. So, it is hard to answer.
What basically needs to be done, is to iterate through the string, position by position, then check if the character is the searched one, and then do something with the result.
A first simple implementation could be:
#include <iostream>
#include <string>
const std::string buffer("\"username\":\"secret\", \"password\":\"also secret\", \"id\":\"secret too\", \"token\":\"secret\"");
int main() {
for (size_t position{}, counter{}; position < buffer.length(); ++position) {
if (buffer[position] == '\"') {
++counter;
std::cout << "Character \" number " << counter << " found at position " << position << '\n';
}
}
return 0;
}
But then, your question was about the usage of std::string.find(). In your implementation, you start always the search at the beginning of the std::string. And because of that, you will always find the same " at position 0.
Solution: After you have found the first match, use the resulting pos (incremented by one) as the second parameter to the std::string.find() function. Then you will start the search after the first found " and hence find the next one. And all this can be done in a normal for-loop.
See below the next easy example:
#include <iostream>
#include <string>
const std::string buffer("\"username\":\"secret\", \"password\":\"also secret\", \"id\":\"secret too\", \"token\":\"secret\"");
int main() {
for (size_t position{}, counter{}; std::string::npos != (position = buffer.find("\"", position)); ++position, ++counter) {
std::cout << "Character \" number " << counter << " found at position " << position << '\n';
}
return 0;
}
There are more solutions, depending on what you really want to do. You coud extract all keywords and data with a simple regex.
Something like this:
#include <iostream>
#include <string>
#include <regex>
#include <vector>
const std::regex re{ R"(\"([ a-zA-Z0-9]+)\")" };
const std::string buffer("\"username\":\"secret\", \"password\":\"also secret\", \"id\":\"secret too\", \"token\":\"secret\"");
int main() {
std::vector part(std::sregex_token_iterator(buffer.begin(), buffer.end(), re, 1), {});
std::cout << part[7] << '\n';
return 0;
}
Or, you can split everything into tokens and values. Like this:
#include <iostream>
#include <string>
#include <regex>
#include <vector>
#include <map>
#include <iomanip>
const std::regex re1{ "," };
const std::regex re2{ R"(\"([^\"]+)\")" };
const std::string buffer("\"username\":\"secret\", \"password\":\"also secret\", \"id\":\"secret too\", \"token\":\"secret\"");
int main() {
std::vector<std::string> block(std::sregex_token_iterator(buffer.begin(), buffer.end(), re1, -1), {});
std::map<std::string, std::string> entry{};
for (const auto& b : block) {
std::vector blockPart(std::sregex_token_iterator(b.begin(), b.end(), re2, 1), {});
entry[blockPart[0]] = blockPart[1];
}
for (const auto& [token, value] : entry)
std::cout << std::setw(20) << token << " --> " << value << '\n';
return 0;
}
But if you have a complex given format, like JSON, there are so many special cases that the only meaningful approach is to use an existing library.
The below code is to parse a "key=val;.." string into std::map and it fails to compile with the error:
Error C2146 : syntax error: missing '>' before identifier 'value_type'
Error C2039 : 'value_type': is not a member of 'std::pair,std::allocator,std::basic_string,std::allocator>>' c:\git\risk-engine-core_tcp\stage\boost-1.66.0-barclays-1\include\boost\spirit\home\support\container.hpp
It does not like the last parameter, "contents" (std::map), passed as a container.
Boost version is 1.66
namespace qi = boost::spirit::qi;
std::map<std::string,std::string> contents;
std::string::iterator first = str.begin();
std::string::iterator last = str.end();
const bool result = qi::phrase_parse(first,last,
*( *(qi::char_-"=") >> qi::lit("=") >> *(qi::char_-";") >> -qi::lit(";") ),
ascii::space, contents);
Looking at the boost docs and stack overflow, I do not see any issue with the above code.
Did you include
#include <boost/fusion/adapted/std_pair.hpp>
Here's a working example with some improvement suggestions:
Live On Coliru
#include <boost/fusion/adapted/std_pair.hpp>
#include <boost/spirit/include/qi.hpp>
#include <map>
#include <iomanip> // std::quoted
namespace qi = boost::spirit::qi;
int main() {
std::string str("key = value");
std::string::const_iterator first = str.begin();
std::string::const_iterator last = str.end();
std::map<std::string, std::string> contents;
bool const result = qi::phrase_parse(first,last,
*( *~qi::char_('=') >> '=' >> *~qi::char_(';') >> -qi::lit(';') ),
qi::ascii::space, contents);
if (result) {
std::cout << "Parsed " << contents.size() << " elements\n";
for (auto& [k,v] : contents) {
std::cout << "\t" << std::quoted(k) << ": " << std::quoted(v) << "\n";
}
} else {
std::cout << "Parse failed\n";
}
if (first != last)
std::cout << "Remaining input unparsed: " << std::quoted(std::string(first, last)) << "\n";
}
Prints
Parsed 1 elements
"key": "value"
How can use boost.spirit x3 to parse into structs like:
struct person{
std::string name;
std::vector<std::string> friends;
}
Coming from boost.spirit v2 I would use a grammar but since X3 doesnt support grammars I have no idea how to do this clean.
EDIT: It would be nice if someone could help me writing a parser parsing a list of strings and returns a person with the first string is the name and the res of the strings are in the friends vector.
Parsing with x3 is much simpler than it was with v2, so you shouldn't have too much trouble moving over. Grammars being gone is a good thing!
Here's how you can parse into a vector of strings:
//#define BOOST_SPIRIT_X3_DEBUG
#include <fstream>
#include <iostream>
#include <string>
#include <type_traits>
#include <vector>
#include <boost/fusion/include/adapt_struct.hpp>
#include <boost/fusion/include/io.hpp>
#include <boost/spirit/home/x3.hpp>
#include <boost/spirit/home/x3/support/ast/variant.hpp>
namespace x3 = boost::spirit::x3;
struct person
{
std::string name;
std::vector<std::string> friends;
};
BOOST_FUSION_ADAPT_STRUCT(
person,
(std::string, name)
(std::vector<std::string>, friends)
);
auto const name = x3::rule<struct name_class, std::string> { "name" }
= x3::raw[x3::lexeme[x3::alpha >> *x3::alnum]];
auto const root = x3::rule<struct person_class, person> { "person" }
= name >> *name;
int main(int, char**)
{
std::string const input = "bob john ellie";
auto it = input.begin();
auto end = input.end();
person p;
if (phrase_parse(it, end, root >> x3::eoi, x3::space, p))
{
std::cout << "parse succeeded" << std::endl;
std::cout << p.name << " has " << p.friends.size() << " friends." << std::endl;
}
else
{
std::cout << "parse failed" << std::endl;
if (it != end)
std::cout << "remaining: " << std::string(it, end) << std::endl;
}
return 0;
}
As you can see on Coliru, the output is :
parse succeeded
bob has 2 friends.
So having a string like remixsettings_bits=1; wysiwyg=1,2,3,abc; remixclosed_tabs=0; remixgroup_closed_tabs=786432; remixlang=0; remixchk=5; remixsid=35d4f9907281708019490d07728c27ca5c10e5de7a869c322222225e3219e; audio_vol=100
I wonder how to parse tham into map name <-> value using boost::spirit and than be capable to write it back using boost::spirit?
Update:
So what I have done:
#include <iostream>
#include <sstream>
#include <string>
#include <map>
//...
std::map<std::string, std::string> user_control::parse_cookie( std::string cookie_data )
{
std::map<std::string, std::string> parsed_cookie;
std::string token, token2;
std::istringstream iss(cookie_data);
while ( getline(iss, token, ' ') )
{
std::string name, val;
std::istringstream iss2(token);
int num = 0 ;
while ( getline(iss2, token2, '=') )
{
if ( num == 0)
{
name = token2;
num++;
}
else
{
val = token2;
std::string::iterator it = val.end() - 1;
if (*it == ';')
val.erase(it);
}
}
std::cout << "name: " << name << " value: " << val << std::endl;
parsed_cookie.insert(std::pair<std::string, std::string>(name, val));
}
return parsed_cookie;
}
but I really wonder how to port my code into boost::spirit code.
This should do the trick, parsing pairs and printing the results using Karma, although we should probably both go read Hartmut's article!
#include <boost/spirit/include/qi.hpp> // Parsing
#include <boost/spirit/include/karma.hpp> // Generation
#include <boost/fusion/adapted/std_pair.hpp> // Make std::pair a fusion vector
int main( int argc, char**argv)
{
using namespace boost::spirit;
std::string str = "keyA=value1; keyB=value2;keyC=value3;";
std::map<std::string,std::string> contents;
std::string::iterator first = str.begin();
std::string::iterator last = str.end();
const bool result = qi::phrase_parse(first,last,
*( *(qi::char_-"=") >> qi::lit("=") >> *(qi::char_-";") >> -qi::lit(";") ),
ascii::space, contents);
assert(result && first==last);
std::cout << karma::format(*(karma::string << '=' <<
karma::string << karma::eol), contents);
}
Have you seen this parser article and this generator article? AFAICT, they explain exactly what you're trying to do.