Synthesized attributes of partial matches from alternatives - c++

When I remove x3::eps in the below rule, the string result from the first partial match is still in the second match, resulting in a string with duplicated content.
If I add another case in between I still only get 1 duplicate instead of two.
Why do I need to use x3::eps, or, what am I misunderstanding in the evaluation of rules and synthesized attributes?
Should I use lookahead instead?
#include <boost/config/warning_disable.hpp>
#include <boost/spirit/home/x3.hpp>
#include <boost/spirit/include/qi_char_class.hpp>
#include <boost/fusion/include/adapt_struct.hpp>
#include <iostream>
#include <string>
#include <vector>
namespace x3 = boost::spirit::x3;
namespace ascii = x3::ascii;
struct AstChannel {
std::string label;
bool complement;
};
x3::rule<class AstLabel, std::string> const astLabel = "astLabel";
auto const astLabel_def = ascii::lower >> *(ascii::alnum);
BOOST_SPIRIT_DEFINE(astLabel)
x3::rule<class AstChannel, AstChannel> const astChannel = "astChannel";
auto const astChannel_def = astLabel >> '!' >> x3::attr(true)
| astLabel >> x3::eps >> x3::attr(false) ;
BOOST_SPIRIT_DEFINE(astChannel)
BOOST_FUSION_ADAPT_STRUCT(
AstChannel,
(std::string, label)
(bool, complement)
)
int main() {
std::string str("hello");
auto iter = str.begin();
auto end = str.end();
AstChannel channel;
bool r = phrase_parse(iter, end, astChannel, ascii::space, channel);
if (r) {
std::cout << channel.label << ',' << channel.complement << std::endl;
}
return 0;
}

The real answer is: force atomic attribute propagation of container attributes (e.g. with x3::hold or semantic actions).
The better answer is: use x3::matches:
Live On Coliru
#include <boost/spirit/home/x3.hpp>
#include <boost/fusion/include/adapt_struct.hpp>
#include <iostream>
using namespace std::string_literals;
struct AstChannel {
std::string label;
bool complement;
};
BOOST_FUSION_ADAPT_STRUCT(AstChannel, label, complement)
namespace Grammar {
namespace x3 = boost::spirit::x3;
namespace ascii = x3::ascii;
auto const label = x3::rule<struct labelRule, std::string> {"label" }
= x3::lexeme[ascii::lower >> *(ascii::alnum)];
auto const channel = label >> x3::matches['!'];
auto const entry = x3::skip(ascii::space) [ channel ];
}
int main() {
auto const str = "hello"s;
AstChannel channel;
if (parse(str.begin(), str.end(), Grammar::entry, channel)) {
std::cout << channel.label << ',' << std::boolalpha << channel.complement << "\n";
}
}
Prints
hello,false

Related

Cleanest way to handle both quoted and unquoted strings in Spirit.X3

Buon giorno,
I have to parse something such as:
foo: 123
"bar": 456
The quotes should be removed if they are here. I tried:
((+x3::alnum) | ('"' >> (+x3::alnum) >> '"'))
But the parser actions for this are of type variant<string, string> ; is there a way to make it so that the parser understands that those two are equivalent, and for my action to only get a single std::string as argument in its call?
edit: minimal repro (live on godbolt: https://gcc.godbolt.org/z/GcE8Pj4r5) :
#include <boost/spirit/home/x3.hpp>
using namespace boost::spirit;
// action handlers
struct handlers {
void create_member(const std::string& str) { }
};
// rules
static const x3::rule<struct id_obj_mem> obj_mem = "obj_mem";
#define EVENT(e) ([](auto& ctx) { x3::get<handlers>(ctx).e(x3::_attr(ctx)); })
static const auto obj_mem_def = ((
((+x3::alnum) | ('"' >> (+x3::alnum) >> '"'))
>> ':' >> x3::lit("123"))[EVENT(create_member)] % ',');
BOOST_SPIRIT_DEFINE(obj_mem)
// execution
int main()
{
handlers r;
std::string str = "foo: 123";
auto first = str.begin();
auto last = str.end();
bool res = phrase_parse(
first,
last,
boost::spirit::x3::with<handlers>(r)[obj_mem_def],
boost::spirit::x3::ascii::space);
}
I too consider this a kind of defect. X3 is definitely less "friendly" in terms of the synthesized attribute types. I guess it's just a tacit side-effect of being more core-language oriented, where attribute assignment is effectively done via default "visitor" actions.
Although I understand the value of keeping the magic to a minimum, and staying close to "pure C++", I vastly prefer the Qi way of synthesizing attributes here. I believe it has proven a hard problem to fix, as this problem has been coming/going in some iterations of X3.
I've long decided to basically fix it myself with variations of this idiom:
template <typename T> struct as_type {
auto operator()(auto p) const { return x3::rule<struct Tag, T>{} = p; }
};
static constexpr as_type<std::string> as_string{};
Now I'd write that as:
auto quoted = '"' >> +x3::alnum >> '"';
auto name = as_string(+x3::alnum | quoted);
auto prop = (name >> ':' >> "123")[EVENT(create_member)] % ',';
That will compile no problem:
Live On Coliru
#include <boost/spirit/home/x3.hpp>
#include <iomanip>
#include <iostream>
namespace x3 = boost::spirit::x3;
struct handlers {
void create_member(std::string const& str) {
std::cerr << __FUNCTION__ << " " << std::quoted(str) << "\n";
}
};
namespace Parser {
#define EVENT(e) ([](auto& ctx) { get<handlers>(ctx).e(_attr(ctx)); })
template <typename T> struct as_type {
auto operator()(auto p) const { return x3::rule<struct Tag, T>{} = p; }
};
static constexpr as_type<std::string> as_string{};
auto quoted = '"' >> +x3::alnum >> '"';
auto name = as_string(+x3::alnum | quoted);
auto prop = (name >> ':' >> "123")[EVENT(create_member)] % ',';
auto grammar = x3::skip(x3::space)[prop];
} // namespace Parser
int main() {
handlers r;
std::string const str = "foo: 123";
auto first = str.begin(), last = str.end();
bool res = parse(first, last, x3::with<handlers>(r)[Parser::grammar]);
return res ? 1 : 0;
}
Prints
create_member "foo"
Interesting Links
Spirit X3, How to get attribute type to match rule type?
Combining rules at runtime and returning rules
spirit x3 cannot propagate attributes of type optional<vector>
etc.

Using boost-spirit to parse string to struct of QStrings

Today, I tried to write a very simple parser using boost-spirit.
Now, I stumbled over an obstacles, that I really don't know how to resolve.
Basically, I wanted the semantic action to return a QString instead of the Stl std::string. And I also wanted to do this conversion in a very simple manner right in the semantic action, as there will be more an more QString objects to come by. The final goal is to pack all these QStrings in a common struct.
(QString is the Qt-Framework equivalent of an std::string).
This is what I got so far.
#include <QString>
#include <vector>
#include <iostream>
#include <boost/config/warning_disable.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix_core.hpp>
#include <boost/spirit/include/phoenix_operator.hpp>
#include <boost/spirit/include/phoenix_fusion.hpp>
#include <boost/spirit/include/phoenix_stl.hpp>
#include <boost/phoenix/object/construct.hpp>
class QStringHelper {
public:
QStringHelper() {}
QStringHelper(const std::string& string) : mString(QString::fromStdString(string)){
}
// This operator might help?
QString operator()() const {
return mString;
}
QString mString;
};
template <typename Iterator, typename Skipper>
struct grammar : boost::spirit::qi::grammar<Iterator, void(), Skipper>
{
QString color;
std::string stdColor;
QStringHelper helper;
grammar() : grammar::base_type(listOfProperties)
{
namespace fusion = boost::fusion;
namespace phoenix = boost::phoenix;
namespace qi = boost::spirit::qi;
namespace ascii = boost::spirit::ascii;
using qi::lit;
using qi::lexeme;
using ascii::char_;
using ascii::string;
using boost::phoenix::ref;
using namespace qi::labels;
listOfProperties = (colorProperty % lit(";"));
// 1. Works, but is not what I desire!
// colorProperty = lit("color") >> lit(":") >> qi::as_string[lexeme[+(char_ - ';')]][boost::phoenix::ref(stdColor) =_1];
// 2. Cannot compile, as QString has not constructor like QString(const std::string&)
// colorProperty = lit("color") >> lit(":") >> qi::as_string[lexeme[+(char_ - ';')]][boost::phoenix::ref(color) = boost::phoenix::construct<QString>(_1)];
// 3. Does compile, but is not exactly what I desire!
colorProperty = lit("color") >> lit(":") >> qi::as_string[lexeme[+(char_ - ';')]][boost::phoenix::ref(helper) = boost::phoenix::construct<QStringHelper>(_1)];
// 4. My desired solution, that I really don't know what someMagicExpression might be?
// colorProperty = lit("color") >> lit(":") >> qi::as_string[lexeme[+(char_ - ';')]][boost::phoenix::ref(color) = someMagicExpression(_1)];
// Many more complex properties will go by!
}
boost::spirit::qi::rule<Iterator, void(), Skipper> listOfProperties;
boost::spirit::qi::rule<Iterator, void(), Skipper> colorProperty;
};
int main(int argc, char**args) {
try {
std::string stringToParse = "color:#ff00ff";
boost::spirit::qi::space_type space;
grammar<std::string::const_iterator, decltype(space)> parser;
auto b = stringToParse.begin();
auto e = stringToParse.end();
bool ok = boost::spirit::qi::phrase_parse(b, e, parser, space);
if (b != e) {
return EXIT_SUCCESS;
}
else {
std::cout << "Color: " << parser.color.toStdString();
}
return EXIT_SUCCESS;
}
catch (const boost::spirit::qi::expectation_failure<std::string::iterator>&) {
return EXIT_FAILURE;
}
}

boost spirit x3 variant and std::pair

I tried to run some simple parser that will parse [ 1, 11, 3, 6-4]. Basically, integer list with range notation.
I want to put everything into AST without semantic action. So I use x3::variant. My code 'seems' very similar to the expression example. However, it can't compile under g++ 6.2. It indeed compile ok with clang++ 6.0 but yield wrong result.
The boost version is 1.63.
It seems that I have some 'move' or initialization issue.
#include <iostream>
#include <list>
#include <vector>
#include <utility>
#include <boost/spirit/home/x3.hpp>
#include <boost/spirit/home/x3/support/ast/variant.hpp>
#include <boost/fusion/include/adapt_struct.hpp>
#include <boost/fusion/adapted/std_pair.hpp>
#include <boost/fusion/include/io.hpp>
namespace ns
{
namespace ast
{
namespace x3 = boost::spirit::x3;
// forward definition
class uintObj;
struct varVec;
// define type
using uintPair_t = std::pair<unsigned int, unsigned int>;
using uintVec_t = std::vector<uintObj>;
// general token value:
class uintObj : public x3::variant <
unsigned int,
uintPair_t
>
{
public:
using base_type::base_type;
using base_type::operator=;
};
struct varVec
{
uintVec_t valVector;
};
}
}
BOOST_FUSION_ADAPT_STRUCT(
ns::ast::varVec,
valVector
)
namespace ns
{
namespace parser
{
// namespace x3 = boost::spirit::x3;
// using namespace x3;
using namespace boost::spirit::x3;
// definition of the range pair:
rule<class uintPair, ast::uintPair_t> const uintPair = "uintPair";
auto const uintPair_def =
uint_
>> '-'
>> uint_
;
rule<class uintObj, ast::uintObj> const uintObj = "uintObj";
auto const uintObj_def =
uint_
| uintPair
;
// define rule definition : rule<ID, attrib>
// more terse definition :
// struct varVec_class;
// using varVec_rule_t = x3::rule<varVec_class, ast::varVec>;
// varVec_rule_t const varVec = "varVec";
// varVec is the rule, "varVec" is the string name of the rule.
rule<class varVec, ast::varVec> const varVec = "varVec";
auto const varVec_def =
'['
>> uintObj % ','
>> ']'
;
BOOST_SPIRIT_DEFINE(
varVec,
uintObj,
uintPair
);
}
}
int main()
{
std::string input ("[1, 11, 3, 6-4]\n");
std::string::const_iterator begin = input.begin();
std::string::const_iterator end = input.end();
ns::ast::varVec result; // ast tree
using ns::parser::varVec; // grammar
using boost::spirit::x3::ascii::space;
bool success = phrase_parse(begin, end, varVec, space, result);
if (success && begin == end)
std::cout << "good" << std::endl;
else
std::cout << "bad" << std::endl;
return 0;
}
Swap the alternative order for the uintObj_def
auto const uintObj_def =
uintPair
| uint_
;
The formulation you have now will always match on a uint_ because the uintPair begins with a valid uint_.
mjcaisse's answer calls out the main problem I think you had. There were a few missing pieces, so I decided to make a simplified version that shows parsing results:
Live On Wandbox
#include <iostream>
#include <iomanip>
//#include <boost/fusion/adapted.hpp>
//#include <boost/fusion/include/io.hpp>
#include <boost/spirit/home/x3.hpp>
#include <boost/spirit/home/x3/support/ast/variant.hpp>
namespace x3 = boost::spirit::x3;
namespace ns { namespace ast {
// forward definition
struct uintObj;
//struct varVec;
// define type
using uintPair_t = std::pair<unsigned int, unsigned int>;
using uintVec_t = std::vector<uintObj>;
// general token value:
struct uintObj : x3::variant<unsigned int, uintPair_t> {
using base_type::base_type;
using base_type::operator=;
friend std::ostream& operator<<(std::ostream& os, uintObj const& This) {
struct {
std::ostream& os;
void operator()(unsigned int v) const { os << v; }
void operator()(uintPair_t v) const { os << v.first << "-" << v.second; }
} vis { os };
boost::apply_visitor(vis, This);
return os;
}
};
using varVec = uintVec_t;
} }
namespace ns { namespace parser {
using namespace boost::spirit::x3;
template <typename T> auto as = [](auto p) { return rule<struct _, T> {} = p; };
auto const uintPair = as<ast::uintPair_t> ( uint_ >> '-' >> uint_ );
auto const uintObj = as<ast::uintObj> ( uintPair | uint_ );
auto const varVec = as<ast::varVec> ( '[' >> uintObj % ',' >> ']' );
} }
int main() {
using namespace ns;
std::string const input("[1, 11, 3, 6-4]\n");
auto begin = input.begin(), end = input.end();
ast::varVec result; // ast tree
bool success = phrase_parse(begin, end, parser::varVec, x3::ascii::space, result);
if (success) {
std::cout << "good\n";
for (auto& r : result)
std::cout << r << "\n";
}
else
std::cout << "bad\n";
if (begin != end)
std::cout << "Remaining unparsed: " << std::quoted(std::string(begin, end)) << std::endl;
}
Prints
good
1
11
3
6-4

Segmentation fault for nested boost::variant

The following program has been reduced from the original. I get a segmentation fault when it runs. If I remove line 24 with ArithmeticUnaryExpression then the program no longer crashes. How do I get rid of the segmentation fault?
#include <boost/config/warning_disable.hpp>
#include <boost/spirit/home/x3.hpp>
#include <boost/spirit/home/x3/support/ast/variant.hpp>
#include <boost/spirit/include/qi_expect.hpp>
#include <boost/spirit/home/x3/directive/expect.hpp>
#include <iostream>
#include <string>
namespace wctl_parser {
namespace x3 = boost::spirit::x3;
namespace ascii = x3::ascii;
namespace qi = boost::spirit::qi;
using x3::ulong_;
using x3::lexeme;
//--- Ast structures
struct ArithmeticUnaryExpression;
using AtomicProp = std::string;
using ArithmeticExpression = x3::variant<
x3::forward_ast<ArithmeticUnaryExpression>,
unsigned long
>;
struct ArithmeticUnaryExpression {
std::string op;
ArithmeticExpression operand;
};
using Expression = x3::variant<
ArithmeticExpression
>;
template <typename T> auto rule = [](const char* name = typeid(T).name()) {
struct _{};
return x3::rule<_, T> {name};
};
template <typename T> auto as = [](auto p) { return rule<T>() = p; };
//--- Rules
x3::rule<struct aTrivRule, ArithmeticExpression> aTriv("aTriv");
x3::rule<struct exprRule, Expression> expr("expression");
auto const aTriv_def = rule<ArithmeticExpression>("aTriv")
= ulong_
// | '(' > expr > ')'
;
auto const primitive = rule<Expression>("primitive")
= aTriv
;
auto const expr_def
= primitive
;
BOOST_SPIRIT_DEFINE(aTriv)
BOOST_SPIRIT_DEFINE(expr)
auto const entry = x3::skip(ascii::space) [expr];
} //End namespace
int main() {
std::string str("prop");
namespace x3 = boost::spirit::x3;
wctl_parser::Expression root;
auto iter = str.begin();
auto end = str.end();
bool r = false;
r = parse(iter, end, wctl_parser::entry, root);
if (r) {
std::cout << "Parses OK:" << std::endl << str << std::endl;
if (iter != end) std::cout << "Partial match" << std::endl;
std::cout << std::endl << "----------------------------\n";
}
else {
std::cout << "!! Parsing failed:" << std::endl << str << std::endl << std::endl << "----------------------------\n";
}
return 0;
}
Your variant
using ArithmeticExpression = x3::variant<
x3::forward_ast<ArithmeticUnaryExpression>,
unsigned long
>;
will default-construct to the first element type. The first element type contains ArithmeticExpression which is also default constructed. Can you see the problem already?
Just make sure the default constructed state doesn't lead to infinite recursion:
using ArithmeticExpression = x3::variant<
unsigned long,
x3::forward_ast<ArithmeticUnaryExpression>
>;

boost::spirit append a vector

I have a problem with inserting data into a vector using phoenix::insert.
The code should parse input such as "(move x y z - loc r - robot item)" into a struct Predicate with name "move" and 3 variables of type loc, 1 variable of type robot and 1 variable with default type object. All those symbols are just strings not really relevant to the problem (I believe). The problem is using phoenix::insert in the definition of the rule for predicate.
Here is the code I have:
#include <boost/config/warning_disable.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <boost/spirit/include/phoenix_core.hpp>
#include <boost/spirit/include/phoenix_operator.hpp>
#include <boost/spirit/include/phoenix_fusion.hpp>
#include <boost/spirit/include/phoenix_stl.hpp>
#include <boost/spirit/include/phoenix_object.hpp>
#include <boost/spirit/home/phoenix/container.hpp>
#include <boost/fusion/include/adapt_struct.hpp>
namespace client {
namespace fusion = boost::fusion;
namespace phoenix = boost::phoenix;
namespace qi = boost::spirit::qi;
namespace ascii = boost::spirit::ascii;
struct Variable {
std::string name;
std::string type;
};
struct Predicate {
std::string name;
std::vector<Variable> vars;
};
struct TermList {
std::vector<Variable> vars;
TermList() = default;
TermList(std::vector<std::string> names, std::string type)
{
for (auto& n : names)
{
Variable t;
t.name = n;
t.type = type;
vars.push_back(t);
}
}
TermList& operator=(const TermList& rhs) = default;
TermList(const TermList& from) = default;
TermList(TermList&& from) = default;
};
}
BOOST_FUSION_ADAPT_STRUCT(
client::Variable,
(std::string, name)
(std::string, type)
)
BOOST_FUSION_ADAPT_STRUCT(
client::Predicate,
(std::string, name)
(std::vector<client::Variable>, vars)
)
BOOST_FUSION_ADAPT_STRUCT(
client::TermList,
(std::vector<client::Variable>, vars)
)
namespace client {
template <typename Iterator, typename Skipper = ascii::space_type>
struct strips_domain_grammar
: qi::grammar<Iterator, Predicate(),
qi::locals<std::vector<Variable>>, Skipper>
{
strips_domain_grammar()
: strips_domain_grammar::base_type(predicate, "predicate")
{
using qi::eps;
using qi::lit;
using qi::lexeme;
using qi::raw;
using qi::on_error;
using qi::fail;
using phoenix::at_c;
using phoenix::push_back;
using phoenix::insert;
using phoenix::begin;
using phoenix::end;
using phoenix::construct;
using phoenix::val;
using ascii::char_;
using ascii::string;
using ascii::alpha;
using ascii::alnum;
using namespace qi::labels;
// identifier such as move or ?from
identifier %= raw[lexeme[((alpha | char_('_') | char_('?'))
>> *(alnum | char_('_') | char_('-')))]];
// x | x y | x - type | x y z - type
term_list =
+(identifier [push_back(_a, _1)])
>>
(
('-' >
identifier [qi::_val = phoenix::construct<TermList>(qi::_a, qi::_1)])
|
eps [qi::_val = phoenix::construct<TermList>(qi::_a, "object")]
)
;
// (move x y z - loc r - robot item) // item is detault type - object
predicate =
char_('(')
> identifier [at_c<0>(_val) = _1]
> +(term_list [insert(at_c<1>(_val), end(at_c<1>(_val)), // <- ERROR
begin(at_c<0>(_1)), end(at_c<0>(_1)))])
> ')'
;
predicate.name("predicate");
term_list.name("term list");
identifier.name("id");
// on_error is called only when an expectation fails (> instead of >>)
on_error<fail>
(
predicate
, std::cout
<< val("Error! Expecting ")
<< _4 // what failed?
<< val(" here: \"")
<< construct<std::string>(_3, _2) // iterators to error-pos, end
<< val("\"")
<< std::endl
);
}
qi::rule<Iterator, std::string(), Skipper> identifier;
qi::rule<Iterator, TermList(),
qi::locals<std::vector<std::string>>, Skipper> term_list;
qi::rule<Iterator, Predicate(),
qi::locals<std::vector<Variable>>, Skipper> predicate;
};
} // namespace client
int main(int argc, const char** argv)
{
typedef std::string::const_iterator iterator_type;
typedef client::strips_domain_grammar<iterator_type> domain_grammar;
domain_grammar g;
std::string str;
while (std::getline(std::cin, str))
{
if (str.empty() || str[0] == 'q' || str[0] == 'Q')
break;
using boost::spirit::ascii::space;
client::Predicate predicate;
std::string::const_iterator iter = str.begin();
std::string::const_iterator end = str.end();
bool r = phrase_parse(iter, end, g, space, predicate);
if (r && iter == end)
{
std::cout << "-------------------------\n";
std::cout << "Parsing succeeded\n";
std::cout << "got: " << predicate.name;
std::cout << "\n-------------------------\n";
}
else
{
std::cout << "-------------------------\n";
std::cout << "Parsing failed\n";
std::cout << "-------------------------\n";
}
}
}
but the code leads to the following error (clang3.3 with libc++ and c++11; mac os x 10.8):
boost/spirit/home/phoenix/stl/container/container.hpp:416:16: error: void function 'operator()' should not return a value [-Wreturn-type]
return c.insert(arg1, arg2, arg3);
As mentioned above, I believe the error is the result of using phoenix::insert in an action in the predicate rule.
I "fixed" the problem by editing the boost header and removing the return statement, but given my limited understanding of this library I would like to avoid that...
Can someone please explain the problem or suggest a different solution?