My grammar has various entries which start with a generic name.
After I determined the type I would like to use the expectation operator in order to create parsing errors.
rule1=name >> (type1 > something);
rule2=name >> (type2 > something);
I already figured that I cannot mix the two operators > and >> -- that's why the parenthesis. My guess is that the parenthesis causes a tuple to be created.
How do I access the elements of the tuple in the semantic action?
The following is certainly wrong but should clarify what I want to accomplish.
rule1=(name >> (type1 > something))[qi::_val = boost::phoenix::bind(
create,
qi::_1,
std::get<0>(qi::_2),
std::get<1>(qi::_2))];
thanks
Directly addressing the question:
using px::at_c;
rule1 = (name >> (type1 > something)) [_val = px::bind(create, _1, at_c<0>(_2), at_c<1>(_2))];
However, I'd use this little trick with qi::eps to avoid the complexity:
rule2 = (name >> type1 >> (eps > something)) [_val = px::bind(create, _1, _2, _3)];
Finally, look at boost::phoenix::function<>:
px::function<decltype(&create)> create_(create); // or just decltype(create) if it's a function object
rule3 = (name >> type1 >> (eps > something)) [_val = create_(_1, _2, _3)];
That way you can even have readable code!
DEMO
Just to prove that all three have the same behaviour¹
Live On Coliru
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <boost/fusion/include/at_c.hpp>
namespace qi = boost::spirit::qi;
namespace px = boost::phoenix;
static int create(char n, char t, char s) {
assert(n=='n' && t=='t' && s=='s');
return 42;
}
int main() {
using It = std::string::const_iterator;
// fake rules just for demo
qi::rule<It, char()>
name = qi::char_("n"),
type1 = qi::char_("t"),
something = qi::char_("s");
//using boost::fusion::at_c;
qi::rule<It, int(), qi::space_type> rule1, rule2, rule3;
{
using namespace qi;
using px::at_c;
rule1 = (name >> (type1 > something)) [_val = px::bind(create, _1, at_c<0>(_2), at_c<1>(_2))];
rule2 = (name >> type1 >> (eps > something)) [_val = px::bind(create, _1, _2, _3)];
px::function<decltype(&create)> create_(create); // or just decltype(create) if it's a function object
rule3 = (name >> type1 >> (eps > something)) [_val = create_(_1, _2, _3)];
}
for(auto& parser : { rule1, rule2, rule3 }) {
for(std::string const input : { "n t s", "n t !" }) {
std::cout << "Input: '" << input << "'\n";
auto f = input.begin(), l = input.end();
int data;
try {
bool ok = qi::phrase_parse(f, l, parser, qi::space, data);
if (ok) {
std::cout << "Parsing result: " << data << '\n';
} else {
std::cout << "Parsing failed\n";
}
} catch(qi::expectation_failure<It> const& e) {
std::cout << "Expectation failure: " << e.what() << " at '" << std::string(e.first, e.last) << "'\n";
}
if (f!=l) {
std::cout << "Remaining unparsed: '" << std::string(f,l) << "'\n";
}
std::cout << "-------------------------------------------\n";
}
}
}
Which prints 3x the same output:
Input: 'n t s'
Parsing result: 42
-------------------------------------------
Input: 'n t !'
Expectation failure: boost::spirit::qi::expectation_failure at '!'
Remaining unparsed: 'n t !'
-------------------------------------------
Input: 'n t s'
Parsing result: 42
-------------------------------------------
Input: 'n t !'
Expectation failure: boost::spirit::qi::expectation_failure at '!'
Remaining unparsed: 'n t !'
-------------------------------------------
Input: 'n t s'
Parsing result: 42
-------------------------------------------
Input: 'n t !'
Expectation failure: boost::spirit::qi::expectation_failure at '!'
Remaining unparsed: 'n t !'
-------------------------------------------
¹ PS let this serve as an example of how to create a SSCCE code example in your questions
Related
I tried to use qi::uint_parser<int>(). But it is the same like qi::uint_. They all match integers range from 0 to std::numeric_limits<unsigned int>::max().
Is qi::uint_parser<int>() designed to be like this? What parser shall I use to match an integer range from 0 to std::numeric_limits<int>::max()? Thanks.
Simplest demo, attaching a semantic action to do the range check:
uint_ [ _pass = (_1>=0 && _1<=std::numeric_limits<int>::max()) ];
Live On Coliru
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
template <typename It>
struct MyInt : boost::spirit::qi::grammar<It, int()> {
MyInt() : MyInt::base_type(start) {
using namespace boost::spirit::qi;
start %= uint_ [ _pass = (_1>=0 && _1<=std::numeric_limits<int>::max()) ];
}
private:
boost::spirit::qi::rule<It, int()> start;
};
template <typename Int>
void test(Int value, char const* logical) {
MyInt<std::string::const_iterator> p;
std::string const input = std::to_string(value);
std::cout << " ---------------- Testing '" << input << "' (" << logical << ")\n";
auto f = input.begin(), l = input.end();
int parsed;
if (parse(f, l, p, parsed)) {
std::cout << "Parse success: " << parsed << "\n";
} else {
std::cout << "Parse failed\n";
}
if (f!=l) {
std::cout << "Remaining unparsed: '" << std::string(f,l) << "'\n";
}
}
int main() {
unsigned maxint = std::numeric_limits<int>::max();
MyInt<std::string::const_iterator> p;
test(maxint , "maxint");
test(maxint-1, "maxint-1");
test(maxint+1, "maxint+1");
test(0 , "0");
test(-1 , "-1");
}
Prints
---------------- Testing '2147483647' (maxint)
Parse success: 2147483647
---------------- Testing '2147483646' (maxint-1)
Parse success: 2147483646
---------------- Testing '2147483648' (maxint+1)
Parse failed
Remaining unparsed: '2147483648'
---------------- Testing '0' (0)
Parse success: 0
---------------- Testing '-1' (-1)
Parse failed
Remaining unparsed: '-1'
I'm trying to understand how boost spirit assign_to_* customization points work.
Here is an exemple I am using:
I have this parser in a rule in a grammar:
int_ >> lit(':') >> char_
And I want the result to be put in this struct:
struct IntAndChar{
int i;
char c;
};
(This is just an exemple to use the customization point so I won't use the BOOST_FUSION_ADAPT_STRUCT or semantic actions.)
I thought I could just define a specialization of assign_to_attribute_from_value but I only get the int this way and the second element is dropped.
Can someone give me a hint to understand how it works?
You don't want to assign to the attribute¹. Instead you wish to transform boost::fusion::vector2<int, char> into IntAndChar.
Therefore, let's start off telling spirit our type is not container-like:
template<>
struct is_container<IntAndChar, void> : mpl::false_ { };
Next, tell it how it can transform e between raw and cooked forms of our attributes:
template<>
struct transform_attribute<IntAndChar, fusion::vector2<int, char>, qi::domain, void> {
using Transformed = fusion::vector2<int, char>;
using Exposed = IntAndChar;
using type = Transformed;
static Transformed pre(Exposed&) { return Transformed(); }
static void post(Exposed& val, Transformed const& attr) {
val.i = fusion::at_c<0>(attr);
val.c = fusion::at_c<1>(attr);
}
static void fail(Exposed&) {}
};
That's it! There is one catch though. It won't work unless you trigger a transformation. The docs say:
It is invoked by Qi rule, semantic action and attr_cast, [...]
1. Using qi::rule (not very helpful)
So here's a solution using rule:
Live On Coliru
int main() {
using It = std::string::const_iterator;
qi::rule<It, boost::fusion::vector2<int, char>(), qi::space_type> rule = qi::int_ >> ':' >> qi::char_;
//qi::rule<It, IntAndChar(), qi::space_type> rule = qi::attr_cast(qi::int_ >> ':' >> qi::char_);
for (std::string const input : { "123:a", "-4 : \r\nq" }) {
It f = input.begin(), l = input.end();
IntAndChar data;
bool ok = qi::phrase_parse(f, l, rule, qi::space, data);
if (ok) std::cout << "Parse success: " << data.i << ", " << data.c << "\n";
else std::cout << "Parse failure ('" << input << "')\n";
if (f != l) std::cout << "Remaining unparsed input: '" << std::string(f, l) << "'\n";
}
}
Prints:
Parse success: 123, a
Parse success: -4, q
Of course this approach requires you to spell out boost::fusion::vector2<int, char> which is tedious and error-prone.
2. Using qi::attr_cast
You can use qi::attr_cast to trigger the transform:
qi::rule<It, IntAndChar(), qi::space_type> rule = qi::attr_cast<IntAndChar, boost::fusion::vector2<int, char> >(qi::int_ >> ':' >> qi::char_);
// using deduction:
qi::rule<It, IntAndChar(), qi::space_type> rule = qi::attr_cast<IntAndChar>(qi::int_ >> ':' >> qi::char_);
// using even more deduction:
qi::rule<It, IntAndChar(), qi::space_type> rule = qi::attr_cast(qi::int_ >> ':' >> qi::char_);
CAVEAT That should work. However, due to very subtle behaviour (bugs?) you need to deep-copy the Proto expression tree there, in order for it to work without Undefined Behaviour:
qi::rule<It, IntAndChar(), qi::space_type> rule = qi::attr_cast(qi::copy(qi::int_ >> ':' >> qi::char_));
Bringing it all together, we can even do without the qi::rule:
Live On Coliru
int main() {
using It = std::string::const_iterator;
for (std::string const input : { "123:a", "-4 : \r\nq" }) {
It f = input.begin(), l = input.end();
IntAndChar data;
bool ok = qi::phrase_parse(f, l, qi::attr_cast(qi::copy(qi::int_ >> ':' >> qi::char_)), qi::space, data);
if (ok) std::cout << "Parse success: " << data.i << ", " << data.c << "\n";
else std::cout << "Parse failure ('" << input << "')\n";
if (f != l) std::cout << "Remaining unparsed input: '" << std::string(f, l) << "'\n";
}
}
Prints
Parse success: 123, a
Parse success: -4, q
¹ (unless you want to treat IntAndChar as a container, which is a different story)
I am currently implementing a parser which succeeds on the "strongest" match for spirit::qi. There are meaningful applications for such a thing. E.g matching references to either simple refs (eg "willy") or namespace qualified refs (eg. "willy::anton"). That's not my actual real world case but it is almost self-explanatory, I guess. At least it helped me to track down the issue.
I found a solution for that. It works perfectly, when the skipper parser is not involved (i.e. there is nothing to skip). It does not work as expected if there are areas which need skipping.
I believe, I tracked down the problem. It seems like under certain conditions spaces are actually not skipped allthough they should be.
Below is find a self-contained very working example. It loops over some rules and some input to provide enough information. If you run it with BOOST_SPIRIT_DEBUG enabled, you get in particular the output:
<qualifier>
<try> :: anton</try>
<fail/>
</qualifier>
I think, this one should not have failed. Am I right guessing so? Does anyone know a way to get around that? Or is it just my poor understanding of qi semantics? Thank you very much for your time. :)
My environment: MSVC 2015 latest, target win32 console
#define BOOST_SPIRIT_DEBUG
#include <io.h>
#include<map>
#include <boost/spirit/include/qi.hpp>
typedef std::string::const_iterator iterator_type;
namespace qi = boost::spirit::qi;
using map_type = std::map<std::string, qi::rule<iterator_type, std::string()>&>;
namespace maxence { namespace parser {
template <typename Iterator>
struct ident : qi::grammar<Iterator, std::string()>
{
ident();
qi::rule<Iterator, std::string()>
id, id_raw;
qi::rule<Iterator, std::string()>
not_used,
qualifier,
qualified_id, simple_id,
id_reference, id_reference_final;
map_type rules = {
{ "E1", id },
{ "E2", id_raw}
};
};
template <typename Iterator>
// we actually don't need the start rule (see below)
ident<Iterator>::ident() : ident::base_type(not_used)
{
id_reference = (!simple_id >> qualified_id) | (!qualified_id >> simple_id);
id_reference_final = id_reference;
///////////////////////////////////////////////////
// standard simple id (not followed by
// delimiter "::")
simple_id = (qi::alpha | '_') >> *(qi::alnum | '_') >> !qi::lit("::");
///////////////////////////////////////////////////
// this is qualifier <- "::" simple_id
// I repeat the simple_id pattern here to make sure
// this demo has no "early match" issues
qualifier = qi::string("::") > (qi::alpha | '_') >> *(qi::alnum | '_');
///////////////////////////////////////////////////
// this is: qualified_id <- simple_id qualifier*
qualified_id = (qi::alpha | '_') >> *(qi::alnum | '_') >> +(qualifier) >> !qi::lit("::");
id = id_reference_final;
id_raw = qi::raw[id_reference_final];
BOOST_SPIRIT_DEBUG_NODES(
(id)
(id_raw)
(qualifier)
(qualified_id)
(simple_id)
(id_reference)
(id_reference_final)
)
}
}}
int main()
{
maxence::parser::ident<iterator_type> ident;
using ss_map_type = std::map<std::string, std::string>;
ss_map_type parser_input =
{
{ "Simple id (behaves ok)", "willy" },
{ "Qualified id (behaves ok)", "willy::anton" },
{ "Skipper involved (unexpected)", "willy :: anton" }
};
for (ss_map_type::const_iterator input = parser_input.begin(); input != parser_input.end(); input++) {
for (map_type::const_iterator example = ident.rules.begin(); example != ident.rules.end(); example++) {
std::string to_parse = input->second;
std::string result;
std::string parser_name = (example->second).name();
std::cout << "--------------------------------------------" << std::endl;
std::cout << "Description: " << input->first << std::endl;
std::cout << "Parser [" << parser_name << "] parsing [" << to_parse << "]" << std::endl;
auto b(to_parse.begin()), e(to_parse.end());
// --- test for parser success
bool success = qi::phrase_parse(b, e, (example)->second, qi::space, result);
if (success) std::cout << "Parser succeeded. Result: " << result << std::endl;
else std::cout << " Parser failed. " << std::endl;
//--- test for EOI
if (b == e) {
std::cout << "EOI reached.";
if (success) std::cout << " The sun is shining brightly. :)";
} else {
std::cout << "Failure: EOI not reached. Remaining: [";
while (b != e) std::cout << *b++; std::cout << "]";
}
std::cout << std::endl << "--------------------------------------------" << std::endl;
}
}
return 0;
}
I'm trying to parse either an additive expression of the form "A+C", or "A" alone. After a few tests I realized that the problem is apparently my use of the optional parser, so to exemplify:
qi::rule<string::iterator, string()> Test;
Test =
(
qi::string("A")[qi::_val= qi::_1]
>> -(
qi::string("B")[qi::_val += qi::_1]
>> qi::string("C")[qi::_val += qi::_1]
)
)
;
string s1, s2;
s1 = "AB";
bool a= qi::parse(s1.begin(), s1.end(), Test, s2);
The idea is to parse 'A' or "ABC", but if the s1 value is "AB" without 'C', the value of a is true. I believe that although I put parenthesis after the operator '-' and then I use the ">>" operator, the 'C' part is considered optional, and not the B>>C as a whole. Any ideas?
Container attributes are not backtracked.
That's a performance choice. You need to explicitly control propagation using e.g. qi::hold:
Live On Coliru
#include <boost/spirit/include/qi.hpp>
namespace qi = boost::spirit::qi;
int main() {
using It = std::string::const_iterator;
qi::rule<It, std::string()> Test;
Test =
(
qi::char_('A')
>> -qi::hold [
qi::char_('B')
>> qi::char_('C')
]
)
;
for (std::string const input : { "A", "AB", "ABC" })
{
std::cout << "-------------------------\nTesting '" << input << "'\n";
It f = input.begin(), l = input.end();
std::string parsed;
bool ok = qi::parse(f, l, Test, parsed);
if (ok)
std::cout << "Parsed success: " << parsed << "\n";
else
std::cout << "Parsed failed\n";
if (f != l)
std::cout << "Remaining unparsed: '" << std::string(f,l) << "'\n";
}
}
Prints:
-------------------------
Testing 'A'
Parsed success: A
-------------------------
Testing 'AB'
Parsed success: A
Remaining unparsed: 'B'
-------------------------
Testing 'ABC'
Parsed success: ABC
Note I have made a number of simplifications.
See also:
Boost Spirit: "Semantic actions are evil"?
boost::spirit::qi duplicate parsing on the output
Understanding Boost.spirit's string parser
I'm pretty new to boost::spirit. I would like to parse a string of comma separated objects into an std::vector (similarly as in the tutorials). The string could be of different types (known at compile time): integers, like "1,2,3", strings "Apple, Orange, Banana", etc. etc.
I would like to have a unified interface for all types.
If I parse a single element I can use the auto_ expression.
Is it possible to have a similar interface with vectors?
Can I define a rule that, given a template parameter, can actually parse this vector?
Here is a simple sample code (which does not compile due to the last call to phrase_parse):
#include <boost/config/warning_disable.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix_core.hpp>
#include <boost/spirit/include/phoenix_operator.hpp>
#include <boost/spirit/include/phoenix_stl.hpp>
#include <iostream>
#include <vector>
#include <boost/spirit/include/qi_auto.hpp>
namespace qi = boost::spirit::qi;
namespace ascii = boost::spirit::ascii;
namespace phoenix = boost::phoenix;
using qi::auto_;
using qi::phrase_parse;
using ascii::space;
using phoenix::push_back;
int main()
{
std::string line1 = "3";
std::string line2 = "1, 2, 3";
int v;
std::vector<int> vector;
typedef std::string::iterator stringIterator;
stringIterator first = line1.begin();
stringIterator last = line1.end();
namespace qi = boost::spirit::qi;
namespace ascii = boost::spirit::ascii;
bool r1 = qi::phrase_parse( first,
last,
qi::auto_,
ascii::space,
v );
first = line2.begin();
last = line2.end();
//The following call is wrong!
bool r2 = qi::phrase_parse( first,
last,
// Begin grammar
(
qi::auto_[push_back(phoenix::ref(vector), qi::_1)]
>> *(',' >> qi::auto_[push_back(phoenix::ref(vector),qi::_1)])
),
// End grammar
ascii::space,
vector);
return 0;
}
UPDATE
I found a solution, in the case the size of the vector is known before parsing. On the other hand I cannot use the syntax *( ',' >> qi::auto_ ).
#include <boost/spirit/include/qi.hpp>
namespace qi = boost::spirit::qi;
int main()
{
std::string s = "1, 2, 3";
std::vector<int> vector;
//This works
qi::phrase_parse(s.begin(), s.end(), qi::auto_ >> ',' >> qi::auto_ >> ',' >> qi::auto_ , qi::blank, vector);
//This does not compile
qi::phrase_parse(s.begin(), s.end(), qi::auto_ >> *( ',' >> qi::auto_ ) , qi::blank, vector);
for(int i = 0; i < vector.size() ; i++)
std::cout << i << ": " << vector[i] << std::endl;
return 0;
}
Moreover using auto_, I cannot parse a a string. Is it possible to define e template function, where the grammar can be deduced by the template parameter?
template< typename T >
void MyParse(std::string& line, std::vector<T> vec)
{
qi::phrase_parse( line.begin(),
line.end(),
/*
How do I define a grammar based on T
such as:
double_ >> *( ',' >> double_ ) for T = double
+qi::alnum >> *( ',' >> +qi::alnum ) for T = std::string
*/,
qi::blank,
vec);
}
auto_ has support for container attributes out of the box:
Live On Coliru
std::istringstream iss("1 2 3 4 5; 6 7 8 9;");
iss.unsetf(std::ios::skipws);
std::vector<int> i;
std::vector<double> d;
if (iss >> qi::phrase_match(qi::auto_ >> ";" >> qi::auto_, qi::space, i, d))
{
for (auto e:i) std::cout << "int: " << e << "\n";
for (auto e:d) std::cout << "double: " << e << "\n";
}
Prints
int: 1
int: 2
int: 3
int: 4
int: 5
double: 6
double: 7
double: 8
double: 9
So you could basically write your template function by using ',' as the skipper. I'd prefer the operator% variant though.
Simple Take
template<typename Container>
void MyParse(std::string const& line, Container& container)
{
auto f(line.begin()), l(line.end());
bool ok = qi::phrase_parse(
f, l,
qi::auto_ % ',', qi::blank, container);
if (!ok || (f!=l))
throw "parser error: '" + std::string(f,l) + "'"; // FIXME
}
Variant 2
template<typename Container>
void MyParse(std::string const& line, Container& container)
{
auto f(line.begin()), l(line.end());
bool ok = qi::phrase_parse(
f, l,
qi::auto_, qi::blank | ',', container);
if (!ok || (f!=l))
throw "parser error: '" + std::string(f,l) + "'"; // FIXME
}
Solving the string case (and others):
If the element type is not 'deducible' by Spirit (anything could be parsed into a string), just take an optional parser/grammar that knows how to parse the element type?
template<typename Container, typename ElementParser = qi::auto_type>
void MyParse(std::string const& line, Container& container, ElementParser const& elementParser = ElementParser())
{
auto f(line.begin()), l(line.end());
bool ok = qi::phrase_parse(
f, l,
elementParser % ",", qi::blank, container);
if (!ok || (f!=l))
throw "parser error: '" + std::string(f,l) + "'"; // FIXME
}
Now, it parses strings just fine:
std::vector<int> i;
std::set<std::string> s;
MyParse("1,22,33,44,15", i);
MyParse("1,22,33,44,15", s, *~qi::char_(","));
for(auto e:i) std::cout << "i: " << e << "\n";
for(auto e:s) std::cout << "s: " << e << "\n";
Prints
i: 1
i: 22
i: 33
i: 44
i: 15
s: 1
s: 15
s: 22
s: 33
s: 44
Full Listing
Live On Coliru
#include <boost/spirit/include/qi.hpp>
#include <iostream>
namespace qi = boost::spirit::qi;
template<typename Container, typename ElementParser = qi::auto_type>
void MyParse(std::string const& line, Container& container, ElementParser const& elementParser = ElementParser())
{
auto f(line.begin()), l(line.end());
bool ok = qi::phrase_parse(
f, l,
elementParser % ",", qi::blank, container);
if (!ok || (f!=l))
throw "parser error: '" + std::string(f,l) + "'"; // FIXME
}
#include <set>
int main()
{
std::vector<int> i;
std::set<std::string> s;
MyParse("1,22,33,44,15", i);
MyParse("1,22,33,44,15", s, *~qi::char_(","));
for(auto e:i) std::cout << "i: " << e << "\n";
for(auto e:s) std::cout << "s: " << e << "\n";
}