I am trying to parse the following messages with Spirit Qi:
"A/B AND C/D", "A/B", "A/B AND C/D AND E/F"
I am able to parse "A/B" but cannot get the correct results for the other strings.
I tried to following code:
qi::rule<It, AstNodeVector()> entries;
qi::rule<It, AstNodeVector()> lists;
qi::rule<It, std::string()> element;
this->entries= *(this->lists % " AND ");
this->lists= this->element >> '/' >> this->element;
this->element = qi::char_("A-Z");
What is wrong with my grammar?
It seems you're not skipping whitespace. Maybe that's a conceptual problem (see Boost spirit skipper issues).
Regardless, it does parse:
Live On Coliru
#include <boost/spirit/include/qi.hpp>
#include <iomanip>
namespace qi = boost::spirit::qi;
using AstNodeVector = std::vector<std::string>;
template <typename It>
struct P : qi::grammar<It, AstNodeVector()> {
P() : P::base_type(entries) {
entries = *(lists % " AND ");
lists = element >> '/' >> element;
element = qi::char_("A-Z");
}
private:
qi::rule<It, AstNodeVector()> entries;
qi::rule<It, AstNodeVector()> lists;
qi::rule<It, std::string()> element;
};
int main() {
using It = std::string::const_iterator;
P<It> const p {};
for (std::string const input: {
"A/B AND C/D",
"A/B",
"A/B AND C/D AND E/F",
})
{
It f = begin(input), l = end(input);
AstNodeVector results;
if (phrase_parse(f, l, p, qi::space, results)) {
std::cout << "Success: " << std::quoted(input) << "\n";
for (auto& el : results) {
std::cout << " -- " << std::quoted(el) << "\n";
}
} else {
std::cout << "FAIL: " << std::quoted(input) << "\n";
}
if (f != l) {
std::cout << "Remaining input: " << std::quoted(std::string(f,l)) << "\n";
}
}
}
Prints
Success: "A/B AND C/D"
-- "A"
-- "B"
-- "C"
-- "D"
Success: "A/B"
-- "A"
-- "B"
Success: "A/B AND C/D AND E/F"
-- "A"
-- "B"
-- "C"
-- "D"
-- "E"
-- "F"
Perhaps you should have included self-contained code, or elaborate on what exactly is the problem.
Related
Assume I already have a struct that looks like this:
struct LETTER
{
double one;
char[12] two;
double three;
char[12] four;
};
And my inputs are comma separated, for example:
"32,CATSANDDOGS,42,WHAT"
"43,BATANDZEBRAS,23,PARROT"
I've been trying to adapt this example (Spirit Qi : rule for char [5]) to to roll through BOOST_FUSION_ADAPT_STRUCT but haven't had any luck. I tried using std::array as shown here (http://www.boost.org/doc/libs/1_64_0/libs/spirit/example/qi/boost_array.cpp) but I haven't been able to make it work in a struct. Is what I'm trying to do even possible? What am I doing wrong here? I would think this would be the most obvious use case.
Is what I'm trying to do even possible?
I'm going to assume you want to write idiomatic C++ code (which is obviously the target domain for Spirit Qi), so you can use std::string:
Live On Coliru
#include <boost/fusion/adapted.hpp>
#include <boost/fusion/include/io.hpp>
#include <boost/spirit/include/qi.hpp>
#include <iostream>
namespace qi = boost::spirit::qi;
struct Letter {
double one;
std::string two;
double three;
std::string four;
};
BOOST_FUSION_ADAPT_STRUCT(Letter, one, two, three, four)
template <typename Iterator> struct LETTERParser : qi::grammar<Iterator, Letter()> {
LETTERParser() : LETTERParser::base_type(start) {
using namespace qi;
_11c = repeat(11) [char_];
start = skip(space) [ "LETTER" >> double_ >> _11c >> double_ >> _11c ];
}
private:
qi::rule<Iterator, Letter()> start;
qi::rule<Iterator, std::string()> _11c;
};
int main() {
const std::string input("LETTER 42 12345678901 +Inf abcdefghijk ");
using It = std::string::const_iterator;
LETTERParser<It> parser;
Letter example;
It f = input.begin(), l = input.end();
if (phrase_parse(f, l, parser, qi::ascii::space, example)) {
std::cout << "parsed: " << boost::fusion::as_vector(example) << "\n";
std::cout << " example.one: " << example.one << "\n";
std::cout << " example.two: '" << example.two << "'\n";
std::cout << " example.three: " << example.three << "\n";
std::cout << " example.four: '" << example.four << "'\n";
} else {
std::cout << "couldn't parse '" << input << "'\n";
}
if (f != l)
std::cout << "Remaining unparsed input: '" << std::string(f,l) << "'\n";
}
Prints
parsed: (42 12345678901 inf abcdefghijk)
example.one: 42
example.two: '12345678901'
example.three: inf
example.four: 'abcdefghijk'
I am trying to parse nested lists of numbers with Boost.Spirit. This is what I have so far:
//define a test string
std::string string = "[[\t1 , 2 ], [3, 4, 65.4]]";
auto it = string.begin();
//parse the string
std::vector<std::vector<double>> vector;
auto listRule = "[" >> (qi::double_ % ",") >> "]";
auto list2Rule = "[" >> (listRule % ",") >> "]";
bool match = qi::phrase_parse(it, string.end(), list2Rule, ascii::space, vector);
//check if we have a match
std::cout << "matched: " << std::boolalpha << match << '\n';
if (it != string.end())
std::cout << "unmatched part: " << std::string{it, string.end()} << '\n';
//print the result
std::cout << "result\n";
for (const auto& v : vector) {
std::cout << "[";
for (double i : v)
std::cout << i << ",";
std::cout << "]\n";
}
The above works wonderfully and prints:
matched: true
result
[1,2,]
[3,4,65.4,]
The problem I am facing is that it does not accept empty lists. For example, by changing the string like so:
std::string string = "[[\t1 , 2 ], [3, 4, 65.4], []]";
Then I have no match (that is match == false and it == string.begin()). The vector still gets populated, apparently, but the last empty list is missing. Can anyone provide an explanation on why this is the case, and how to fix it?
You're using auto for proto expression templates in the Qi domain - that's undefined behaviour 99.9% of the time:
undefined behaviour somewhere in boost::spirit::qi::phrase_parse
Now, while you fix that, also make the list body optional:
Live On Coliru
#include <boost/spirit/include/qi.hpp>
namespace qi = boost::spirit::qi;
int main() {
using It = std::string::const_iterator;
using Skipper = qi::space_type;
for(std::string const input : { "[[\t1 , 2 ], [3, 4, 65.4]]", "[[\t1 , 2 ], [3, 4, 65.4], []]", "[]" })
{
std::cout << " ---- '" << input << "' ----\n";
auto it = input.begin();
//parse the string
using doubles = std::vector<double>;
using vectors = std::vector<doubles>;
qi::rule<It, doubles(), Skipper> doubles_ = "[" >> -(qi::double_ % ",") >> "]";
qi::rule<It, vectors(), Skipper> vectors_ = "[" >> -(doubles_ % ",") >> "]";
vectors data;
bool match = qi::phrase_parse(it, input.end(), vectors_, qi::space, data);
//check if we have a match
std::cout << "matched: " << std::boolalpha << match << '\n';
if (it != input.end())
std::cout << "unmatched part: " << std::string{it, input.end()} << '\n';
//print the result
std::cout << "result\n";
for (const auto& v : data) {
std::cout << "[";
for (double i : v)
std::cout << i << ",";
std::cout << "]\n";
}
}
}
Prints
---- '[[ 1 , 2 ], [3, 4, 65.4]]' ----
matched: true
result
[1,2,]
[3,4,65.4,]
---- '[[ 1 , 2 ], [3, 4, 65.4], []]' ----
matched: true
result
[1,2,]
[3,4,65.4,]
[]
---- '[]' ----
matched: true
result
In my boost::spirit grammar I have the following snippet;
implicit_method_declaration = (-(qi::token(ABSTRACT)) >> ...)
The type of -(qi::token(ABSTRACT) is boost::optional<boost::iterator_range<std::string::iterator>> however I'm only using this construct to check if the abstract keyword, is actually present, that is, I'd rather have -(qi::token(ABSTRACT) have the type bool with the value boost::optional<...> operator bool() const.
How would I go about achieving this?
I think you're looking for qi::matches[]:
implicit_method_declaration =
qi::matches[qi::token(ABSTRACT)] >> ...;
An alternative would be to use qi::attr() with alternatives:
implicit_method_declaration =
(
qi::token(ABSTRACT) >> qi::attr(true)
| qi::attr(false)
) >> ...;
A quick demo again: http://coliru.stacked-crooked.com/a/ed8bbad53e8c1943
#include <boost/spirit/include/qi.hpp>
namespace qi = boost::spirit::qi;
template <typename It, typename Skipper = qi::space_type>
struct parser : qi::grammar<It, bool(), Skipper>
{
parser() : parser::base_type(implicit_method_declaration)
{
using namespace qi;
implicit_method_declaration = matches["abstract"];
BOOST_SPIRIT_DEBUG_NODES((implicit_method_declaration));
}
private:
qi::rule<It, bool(), Skipper> implicit_method_declaration;
};
bool doParse(const std::string& input)
{
typedef std::string::const_iterator It;
auto f(begin(input)), l(end(input));
parser<It, qi::space_type> p;
bool data;
try
{
bool ok = qi::phrase_parse(f,l,p,qi::space,data);
if (ok)
{
std::cout << "parse success\n";
std::cout << "data: " << data << "\n";
}
else std::cerr << "parse failed: '" << std::string(f,l) << "'\n";
if (f!=l) std::cerr << "trailing unparsed: '" << std::string(f,l) << "'\n";
return ok;
} catch(const qi::expectation_failure<It>& e)
{
std::string frag(e.first, e.last);
std::cerr << e.what() << "'" << frag << "'\n";
}
return false;
}
int main()
{
doParse("abstract");
doParse("static final");
}
Output
parse success
data: 1
parse success
data: 0
trailing unparsed: 'static final'
Why the parser using the rules below returns an empty container? There're 3 rules. One is for parsing a string of characters except double quote, the second parses a pair (e.g. "col1" : 2) and the third parses the vector of such pairs. The ouput of the program below in MSVS2012 is
parse success
result: '' : 0
result: '' : 0
result: '' : 0
.
namespace parsers
{
spirit::qi::rule< iterator, column_name_t() > quoted_string =
spirit::qi::lexeme["\"" >> +~spirit::qi::char_("\"") >> "\""];
spirit::qi::rule< iterator, column_and_aggregate(), spirit::qi::space_type > agg_pair =
quoted_string//::boost::bind( &apply_col_and_aggr_visitor, spirit::qi::_val, spirit::qi::_1 )]
> ':'
// A rule validation technic is used below.
> spirit::int_[spirit::qi::_pass = (spirit::qi::_1 >=AVG && spirit::qi::_1<=SUM)];//::boost::bind( &apply_col_and_aggr_visitor, spirit::qi::_val, spirit::qi::_1 )];
spirit::qi::rule< iterator, column_and_aggregate_container(), spirit::qi::space_type > aggregates_parser =
'{'
> agg_pair/*[phoenix::push_back(spirit::qi::_val, spirit::qi::_1)]*/ % ',' // N.B.!!! list-redux technic
> '}';
}
using namespace parsers;
using namespace boost::spirit;
bool doParse(const std::string& input)
{
typedef std::string::const_iterator It;
auto f(begin(input)), l(end(input));
//parser<It, qi::space_type> p;
column_and_aggregate_container data;
typedef BOOST_TYPEOF(qi::space) skipper_type;
try
{
bool ok = qi::phrase_parse(f,l,aggregates_parser,qi::space,data);
if (ok)
{
std::cout << "parse success\n";
for (auto& pair : data)
std::cout << "result: '" << pair.first << "' : " << (int) pair.second << "\n";
}
else std::cerr << "parse failed: '" << std::string(f,l) << "'\n";
if (f!=l) std::cerr << "trailing unparsed: '" << std::string(f,l) << "'\n";
return ok;
}
catch(const qi::expectation_failure<It>& e)
{
std::string frag(e.first, e.last);
std::cerr << e.what() << "'" << frag << "'\n";
}
return false;
}
int main()
{
//bool ok = doParse("{ 'column 1' : 1, 'column 2' : 0, 'column 3' : 1 }");
doParse("{ \"column 1\" : 1, \"column 2\" : 0, \"column 3\" : 1 }");
//return ok? 0 : 255;
}
template <typename it, typename skipper = qi::space_type>
struct quoted_string_parser
{
quoted_string_parser()
{
using namespace qi;
quoted_string %= lexeme['"' >> *~char_('"') >> '"'];
BOOST_SPIRIT_DEBUG_NODE(quoted_string);
}
qi::rule<it, std::string(), skipper> quoted_string;
};
template <typename it, typename skipper = qi::space_type>
struct aggregates_parser : qi::grammar<it, column_and_aggregate_container(), skipper>
{
aggregates_parser() : aggregates_parser::base_type(aggregates_parser_)
{
using namespace qi;
agg_pair %= quoted_string_parser<it,skipper> > ':' > int_[_pass = (qi::_1 >= AVG && qi::_1 <= SUM)];
aggregates_parser_ = '{' > agg_pair % ',' > '}';
BOOST_SPIRIT_DEBUG_NODE(aggregates_parser_);
}
private:
qi::rule<it, sql_faggregate(), skipper> faggr;
qi::rule<it, column_and_aggregate(), skipper> agg_pair;
qi::rule<it, column_and_aggregate_container(), skipper> aggregates_parser_;
};
Like I said in the answer right where I suggested this semantic action for validation:
When a semantic action is present, automatic attribute propagation does not normally occur. Using %= forces automatic attribute propagation (we want this because the semantic action doesn't assign the attribute value(s), it just validates them).
Again, a fully working demonstration, incorporating your rules:
#include <boost/fusion/adapted/std_pair.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
namespace qi = boost::spirit::qi;
namespace phx = boost::phoenix;
typedef std::string column_name_t;
enum sql_faggregate
{
AVG,
// ....
SUM,
};
typedef std::pair<column_name_t, sql_faggregate> column_and_aggregate;
typedef std::vector<column_and_aggregate> column_and_aggregate_container;
template <typename It, typename Skipper = qi::space_type>
struct parser : qi::grammar<It, column_and_aggregate_container(), Skipper>
{
parser() : parser::base_type(aggregates_parser)
{
using namespace qi;
quoted_string = lexeme['"' >> +~char_('"') >> '"'];
agg_pair %= quoted_string > ':' // A rule validation technic is used below.
> int_[_pass = (_1 >=AVG && _1<=SUM)];
aggregates_parser = '{' > agg_pair % ',' > '}';
BOOST_SPIRIT_DEBUG_NODE(aggregates_parser);
}
private:
qi::rule<It, std::string(), qi::space_type> quoted_string;
qi::rule<It, sql_faggregate(), qi::space_type> faggr;
qi::rule<It, column_and_aggregate(), qi::space_type> agg_pair;
qi::rule<It, column_and_aggregate_container(), qi::space_type> aggregates_parser;
};
bool doParse(const std::string& input)
{
typedef std::string::const_iterator It;
auto f(begin(input)), l(end(input));
parser<It, qi::space_type> p;
column_and_aggregate_container data;
try
{
bool ok = qi::phrase_parse(f,l,p,qi::space,data);
if (ok)
{
std::cout << "parse success\n";
for (auto& pair : data)
std::cout << "result: '" << pair.first << "' : " << (int) pair.second << "\n";
}
else std::cerr << "parse failed: '" << std::string(f,l) << "'\n";
if (f!=l) std::cerr << "trailing unparsed: '" << std::string(f,l) << "'\n";
return ok;
} catch(const qi::expectation_failure<It>& e)
{
std::string frag(e.first, e.last);
std::cerr << e.what() << "'" << frag << "'\n";
}
return false;
}
int main()
{
bool ok = doParse("{ \"column 1\" : 1, \"column 2\" : 0, \"column 3\" : 1 }");
return ok? 0 : 255;
}
Prints
parse success
result: 'column 1' : 1
result: 'column 2' : 0
result: 'column 3' : 1
as expected
I'm trying to get into boost spirit 2, but the following code does not work as expected:
template<typename Iterator>
struct my_grammar : qi::grammar<Iterator, std::string()>
{
my_grammar() : my_grammar::base_type(time_literal)
{
using ascii::char_;
using ascii::digit;
time_literal = digit >> -digit >> char_(':') >> digit >> digit >> -(char_(':') >> digit >> digit);
}
qi::rule<Iterator, std::string()> time_literal;
};
void main()
{
my_grammar<std::string::iterator> g;
std::string input("01:02:03");
std::string::iterator begin = input.begin();
std::string::iterator iter = begin;
std::string::iterator end = input.end();
std::string result;
bool matched = phrase_parse(iter, end, g, ascii::space, result);
std::cout << (matched ? "matched "+std::string(begin, iter) : "no match") << std::endl;
if (iter != end)
std::cout << "remaining: " << std::string(iter, end) << std::endl;
else
std::cout << "result: " << result << std::endl;
std::cout << std::endl;
}
This prints:
matched: 01:02:03
result: 01:02:
But I expected to see:
matched: 01:02:03
result: 01:02:03
So where did those last two digits go and how can I get them back?