I'm getting myself familiarized with boost spirit v3. The question I want to ask is how to state the fact that you don't want to use skip parser in any way.
Consider a simple example of parsing comma-separated sequence of integers:
#include <iostream>
#include <string>
#include <vector>
#include <boost/spirit/home/x3.hpp>
int main()
{
using namespace boost::spirit::x3;
const std::string input{"2,4,5"};
const auto parser = int_ % ',';
std::vector<int> numbers;
auto start = input.cbegin();
auto r = phrase_parse(start, input.end(), parser, space, numbers);
if(r && start == input.cend())
{
// success
for(const auto &item: numbers)
std::cout << item << std::endl;
return 0;
}
std::cerr << "Input was not parsed successfully" << std::endl;
return 1;
}
This works totally fine. However, I would like to forbid having spaces in between (i.e. "2, 4,5" should not be parsed well).
I tried using eps as a skip parser in phrase_parse, but as you can guess, the program ended up in the infinite loop because eps matches to an empty string.
Solution I found is to use no_skip directive (https://www.boost.org/doc/libs/1_75_0/libs/spirit/doc/html/spirit/qi/reference/directive/no_skip.html). So the parser now becomes:
const auto parser = no_skip[int_ % ','];
This works fine, but I don't find it to be an elegant solution (especially providing "space" parser in phrase_parse when I want no whitespace skips). Are there no skip parsers that would simply do nothing? Am I missing something?
Thanks for Your time. Looking forward to any replies.
You can use either no_skip[] or lexeme[]. They're almost identical, except for pre-skip (Boost Spirit lexeme vs no_skip).
Are there no skip parsers that would simply do nothing? Am I missing something?
A wild guess, but you might be missing the parse API that doesn't accept a skipper in the first place
Live On Coliru
#include <iostream>
#include <iomanip>
#include <boost/spirit/home/x3.hpp>
namespace x3 = boost::spirit::x3;
int main() {
std::string const input{ "2,4,5" };
auto f = begin(input), l = end(input);
const auto parser = x3::int_ % ',';
std::vector<int> numbers;
auto r = parse(f, l, parser, numbers);
if (r) {
// success
for (const auto& item : numbers)
std::cout << item << std::endl;
} else {
std::cerr << "Input was not parsed successfully" << std::endl;
return 1;
}
if (f!=l) {
std::cout << "Remaining input " << std::quoted(std::string(f,l)) << "\n";
return 2;
}
}
Prints
2
4
5
Related
I am still trying to wrap my head around Boost::Spirit.
I want to parse two words into a variable. When I can do that, into a struct.
The single word compiles, the Variable doesn't. Why?
#include <boost/spirit/include/qi.hpp>
#include <boost/tuple/tuple.hpp>
#include <string>
#include <iostream>
using namespace boost::spirit;
/*
class Syntax : public qi::parser{
};
*/
int main()
{
//get user input
std::string input;
std::getline(std::cin, input);
auto it = input.begin();
bool result;
//define grammar for a single word
auto word_grammar = +qi::alnum - qi::space;
std::string singleWord;
result = qi::parse(
it, input.end(),
word_grammar,
singleWord
);
if(!result){
std::cout << "Failed to parse a word" << '\n';
return -1;
}
std::cout << "\"" << singleWord << "\"" << '\n';
//Now parse two words into a variable
std::cout << "Variable:\n";
typedef boost::tuple<std::string, std::string> Variable;
Variable variable;
auto variable_grammar = word_grammar >> word_grammar;
result = qi::parse(
it, input.end(),
variable_grammar,
variable
);
if(!result){
std::cout << "Failed to parse a variable" << '\n';
return -1;
}
std::cout << "\"" << variable.get<0>() << "\" \"" << variable.get<1>() << "\"" << '\n';
//now parse a list of variables
std::cout << "List of Variables:\n";
std::list<Variable> variables;
result = qi::parse(
it, input.end(),
variable_grammar % +qi::space,
variable
);
if(!result){
std::cout << "Failed to parse a list of variables" << '\n';
return -1;
}
for(auto var : variables)
std::cout << "DataType: " << var.get<0>() << ", VariableName: " << var.get<1>() << '\n';
}
In the end I want to parse something like this:
int a
float b
string name
Templates are nice, but when problems occur the error messages are just not human readable (thus no point in posting them here).
I am using the gcc
Sorry to take so long. I've been building a new web server in a hurry and had much to learn.
Here is what it looks like in X3. I think it is easier to deal with than qi. And then, I've used it a lot more. But then qi is much more mature, richer. That said, x3 is meant to be adaptable, hackable. So you can make it do just about anything you want.
So, live on coliru
#include <string>
#include <iostream>
#include <vector>
#include <boost/spirit/home/x3.hpp>
#include <boost/tuple/tuple.hpp>
//as pointed out, for the error 'The parser expects tuple-like attribute type'
#include <boost/fusion/adapted/boost_tuple.hpp>
//our declarations
using Variable = boost::tuple<std::string, std::string>;
using Vector = std::vector<Variable>;
namespace parsers {
using namespace boost::spirit::x3;
auto const word = lexeme[+char_("a-zA-Z")];
//note, using 'space' as the stock skipper
auto const tuple = word >> word;
}
std::ostream& operator << (std::ostream& os, /*const*/ Variable& obj) {
return os << obj.get<0>() << ' ' << obj.get<1>();
}
std::ostream& operator << (std::ostream& os, /*const*/ Vector& obj) {
for (auto& item : obj)
os << item << " : ";
return os;
}
template<typename P, typename A>
bool test_parse(std::string in, P parser, A& attr) {
auto begin(in.begin());
bool r = phrase_parse(begin, in.end(), parser, boost::spirit::x3::space, attr);
std::cout << "result:\n " << attr << std::endl;
return r;
}
int main()
{
//not recomended but this is testing stuff
using namespace boost::spirit::x3;
using namespace parsers;
std::string input("first second third forth");
//parse one word
std::string singleWord;
test_parse(input, word, singleWord);
//parse two words into a variable
Variable variable;
test_parse(input, tuple, variable);
//parse two sets of two words
Vector vector;
test_parse(input, *tuple, vector);
}
You may like this form of testing. You can concentrate on testing parsers without a lot of extra code. It makes it easier down the road to keep your basic parsers in their own namespace. Oh yea, x3 compiles much faster than qi!
The single word compiles, the Variable doesn't. Why?
There are missing two #includes:
#include <boost/fusion/adapted/boost_tuple.hpp>
#include <boost/spirit/include/qi_list.hpp>
I'm working on a project for my univertitiy studies. My goal is to read double numbers from a large file (2,6 GB) into a double vector.
I am working with the boost spirit x3 library with mmap. I have found some code in the net: https://github.com/sehe/bench_float_parsing which i am using.
Before pushing these double values into the vector i would like to do some arithmetic operations on these. So here i'm stuck. How can i do some artihmetic operations to double values before pushing them?
template <typename source_it>
size_t x3_phrase_parse<data::float_vector, source_it>::parse(source_it f, source_it l, data::float_vector& data) const {
using namespace x3;
bool ok = phrase_parse(f, l, *double_ % eol, space, data);
if (ok)
std::cout << "parse success\n";
else
std::cerr << "parse failed: '" << std::string(f, l) << "'\n";
if (f != l) std::cerr << "trailing unparsed: '" << std::string(f, l) << "'\n";
std::cout << "data.size(): " << data.size() << "\n";
return data.size();
}
I am sorry to not exactly answer your question. But boost spirit is not the appropriate tool. Spirit is a parser generator (as a subset is does of course also lexical analysis) . So, one level to high in the Chomsky hiearchy of languages. You do not need a parser but regular expressions: std:regex
A double can easily be found with a regular expression. In the attached code, I created a simple pattern for a doubles. And a regex can be used to search for it.
So, we will read from an istream (what can be a file, a stringstream, console input or whatever). We will read line by line, until the whole input is consumed.
For each line, we will check, if the input matches the expected pattern, being 1 double.
Then we read this double, do some calculations and then push it into the vector.
Please see the following very simple code.
#include <iostream>
#include <fstream>
#include <sstream>
#include <string>
#include <regex>
std::istringstream input{R"(0.0
1.5
2.0
3.0
4.0
-5.0
)"};
using VectorDouble = std::vector<double>;
const std::regex reDouble{R"(([-+]?[0-9]*\.?[0-9]*))"};
std::istream& get(std::istream& is, VectorDouble& dd)
{
// Reset vector to empty before reading
dd.clear();
//Read all data from istream
std::string line{};
while (getline(is, line)) {
// Search for 2 doubles
std::smatch sm;
if (std::regex_search(line, sm, reDouble)) {
// Convert found strings to double
double d1{std::stod(sm[1])};
// Do some calculations
d1 = d1 + 10.0;
// Push back into vector
dd.emplace_back(d1);
}
else
std::cerr << "Error found in line: " << line << "\n";
}
return is;
}
int main()
{
// Define vector and fill it
VectorDouble dd{};
(void)get(input, dd);
// Some debug output
for (double& d : dd) {
std::cout << d << "\n";
}
return 0;
}
Why not use semantic actions to perform the arithmetic operations?
In the following code:
#include <iostream>
#include <sstream>
#include <string>
#include <cstdio>
#include <vector>
using VectorDouble = std::vector<double>;
void show( VectorDouble const& dd)
{
std::cout<<"vector result=\n";
for (double const& d : dd) {
std::cout << d << "\n";
}
}
auto arith_ops=[](double&x){ x+=10.0;};
std::string input_err_yes{R"(0.0
1.5
2.0xxx
not double
4.0
-5.0
)"};
std::string input_err_not{R"(0.0
1.5
2.0
3.0
4.0
-5.0
)"};
void stod_error_recov(std::string const&input)
//Use this for graceful error recovery in case input has syntax errors.
{
std::cout<<__func__<<":\n";
VectorDouble dd;
std::istringstream is(input);
std::string line{};
while (getline(is, line) ) {
try {
std::size_t eod;
double d1(std::stod(line,&eod));
arith_ops(d1);
dd.emplace_back(d1);
auto const eol=line.size();
if(eod!=eol) {
std::cerr << "Warning: trailing chars after double in line: "<< line << "\n";
}
}
catch (const std::invalid_argument&) {
if(!is.eof())
std::cerr << "Error: found in line: " << line << "\n";
}
}
show(dd);
}
void stod_error_break(std::string const&input)
//Use this if input is sure to have correct syntax.
{
std::cout<<__func__<<":\n";
VectorDouble dd;
char const*d=input.data();
while(true) {
try {
std::size_t eod;
double d1(std::stod(d,&eod));
d+=eod;
arith_ops(d1);
dd.emplace_back(d1);
}
catch (const std::invalid_argument&) {
//Either syntax error
//Or end of input.
break;
}
}
show(dd);
}
#include <boost/spirit/home/x3.hpp>
void x3_error_break(std::string const&input)
//boost::spirit::x3 method.
{
std::cout<<__func__<<":\n";
VectorDouble dd;
auto f=input.begin();
auto l=input.end();
using namespace boost::spirit::x3;
auto arith_action=[](auto&ctx)
{ arith_ops(_attr(ctx));
};
phrase_parse(f, l, double_[arith_action] % eol, blank, dd);
show(dd);
}
int main()
{
//stod_error_recov(input_err_yes);
//stod_error_break(input_err_not);
x3_error_break(input_err_not);
return 0;
}
the stod_* functions, unlike that of Armin's, don't need regex because
std:stod does the parsing and, because it doesn't
use regex it probably runs a bit faster.
There are 2 stod_* functions shown with in-source comments
indicated which should be used.
For completeness, a 3ird function using boost::spirit::x3 is
shown. IMHO, it's readability is better than the others; however,
it would probably take more time to compile.
#include <boost/spirit/include/qi.hpp>
#include <string>
#include <vector>
#include <iterator>
#include <algorithm>
#include <iostream>
using namespace boost::spirit;
int main()
{
std::string s;
std::getline(std::cin, s);
auto specialtxt = *(qi::char_('-', '.', '_'));
auto txt = no_skip[*(qi::char_("a-zA-Z0-9_.\\:$\'-"))];
auto anytxt = *(qi::char_("a-zA-Z0-9_.\\:${}[]+/()-"));
qi::rule <std::string::iterator, void(),ascii::space_type> rule2 = txt ('=') >> ('[') >> (']');
auto begin = s.begin();
auto end = s.end();
if (qi::phrase_parse(begin, end, rule2, ascii::space))
{
std::cout << "MATCH" << std::endl;
}
else
{
std::cout << "NO MATCH" << std::endl;
}
}
this code works fine in debug mode
parser fails in release mode
rule is to just parse text=[]; any thing else than this should fail it works fine in debug mode but not in release mode it shows result no match for any string.
if i enter string like
abc=[];
this passes in debug as expected but fails in release
You can't use auto with Spirit v2:
Assigning parsers to auto variables
You have Undefined Behaviour
DEMO
I tried to make (more) sense of the rest of the code. There were various instances that would never work:
txt('=') is an invalid Qi expression. I assumed you wanted txt >> ('=') instead
qi::char_("a-zA-Z0-9_.\\:$\\-{}[]+/()") doesn't do what you think because $-{ is actually the character "range" \x24-\x7b... Escape the - (or put it at the very end/start of the set like in the other char_ call).
qi::char_('-','.','_') can't work. Did you mean qi::char_("-._")?
specialtxt and anytxt were unused...
prefer const_iterator
prefer namespace aliases above using namespace to prevent hard-to-detect errors
Live On Coliru
#include <boost/spirit/include/qi.hpp>
#include <iostream>
namespace qi = boost::spirit::qi;
int main() {
std::string const s = "abc=[];";
auto specialtxt = qi::copy(*(qi::char_("-._")));
auto anytxt = qi::copy(*(qi::char_("a-zA-Z0-9_.\\:$\\-{}[]+/()")));
(void) specialtxt;
(void) anytxt;
auto txt = qi::copy(qi::no_skip[*(qi::char_("a-zA-Z0-9_.\\:$\'-"))]);
qi::rule<std::string::const_iterator, qi::space_type> rule2 = txt >> '=' >> '[' >> ']';
auto begin = s.begin();
auto end = s.end();
if (qi::phrase_parse(begin, end, rule2, qi::space)) {
std::cout << "MATCH" << std::endl;
} else {
std::cout << "NO MATCH" << std::endl;
}
if (begin != end) {
std::cout << "Trailing unparsed: '" << std::string(begin, end) << "'\n";
}
}
Printing
MATCH
Trailing unparsed: ';'
I am trying to use the Boost Spirit X3 directive repeat with a repetition factor that is variable. The basic idea is that of a header + payload, where the header specifies the size of the payload. A simple example “3 1 2 3” is interpreted as header = 3, data= {1, 2, 3} (3 integers).
I could only find examples from the spirit qi documentation. It uses boost phoenix reference to wrap the variable factor: http://www.boost.org/doc/libs/1_50_0/libs/spirit/doc/html/spirit/qi/reference/directive/repeat.html
std::string str;
int n;
test_parser_attr("\x0bHello World",
char_[phx::ref(n) = _1] >> repeat(phx::ref(n))[char_], str);
std::cout << n << ',' << str << std::endl; // will print "11,Hello World"
I wrote the following simple example for spirit x3 without luck:
#include <boost/spirit/home/x3.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <string>
#include <iostream>
namespace x3 = boost::spirit::x3;
using x3::uint_;
using x3::int_;
using x3::phrase_parse;
using x3::repeat;
using x3::space;
using std::string;
using std::cout;
using std::endl;
int main( int argc, char **argv )
{
string data("3 1 2 3");
string::iterator begin = data.begin();
string::iterator end = data.end();
unsigned int n = 0;
auto f = [&n]( auto &ctx ) { n = x3::_attr(ctx); };
bool r = phrase_parse( begin, end, uint_[f] >> repeat(boost::phoenix::ref(n))[int_], space );
if ( r && begin == end )
cout << "Parse success!" << endl;
else
cout << "Parse failed, remaining: " << string(begin,end) << endl;
return 0;
}
Compiling the code above with boost 1.59.0 and clang++ (flags: -std=c++14) gives the following:
boost_1_59_0/boost/spirit/home/x3/directive/repeat.hpp:72:47: error: no matching constructor for
initialization of 'proto_child0' (aka 'boost::reference_wrapper<unsigned int>')
typename RepeatCountLimit::type i{};
If I hardcode repeat(3) instead of repeat(boost::phoenix::ref(n)) it works properly, but it is not a possible solution since it should support a variable repetition factor.
Compilation with repeat(n) completes successfully, but it fails parsing with the following output:
“Parse failed, remaining: 1 2 3"
Looking at the source code for boost/spirit/home/x3/directive/repeat.hpp:72 it calls the empty constructor for template type RepeatCountLimit::type variable i and then assign during the for loop, iterating over min and max. However since the type is a reference it should be initialized in the constructor, so compilation fails. Looking at the equivalent source code from the previous library version boost/spirit/home/qi/directive/repeat.hpp:162 it is assigned directly:
typename LoopIter::type i = iter.start();
I am not sure what I am doing wrong here, or if x3 currently does not support variable repetition factors. I would appreciate some help solving this issue. Thank you.
From what I gather, reading the source and the mailing list, Phoenix is not integrated into X3 at all: the reason being that c++14 makes most of it obsolete.
I agree that this leaves a few spots where Qi used to have elegant solutions, e.g. eps(DEFERRED_CONDITION), lazy(*RULE_PTR) (the Nabialek trick), and indeed, this case.
Spirit X3 is still in development, so we might see this added¹
For now, Spirit X3 has one generalized facility for stateful context. This essentially replaces locals<>, in some cases inherited arguments, and can be /made to/ validate the number of elements in this particular case as well:
x3::with²
Here's how you could use it:
with<_n>(std::ref(n))
[ omit[uint_[number] ] >>
*(eps [more] >> int_) >> eps [done] ]
Here, _n is a tag type that identifies the context element for retrieval with get<_n>(cxtx).
Note, currently we have to use a reference-wrapper to an lvalue n because with<_n>(0u) would result in constant element inside the context. I suppose this, too, is a QoI that may be lifted as X# matures
Now, for the semantic actions:
unsigned n;
struct _n{};
auto number = [](auto &ctx) { get<_n>(ctx).get() = _attr(ctx); };
This stores the parsed unsigned number into the context. (In fact, due to the ref(n) binding it's not actually part of the context for now, as mentioned)
auto more = [](auto &ctx) { _pass(ctx) = get<_n>(ctx) > _val(ctx).size(); };
Here we check that we're actually not "full" - i.e. more integers are allowed
auto done = [](auto &ctx) { _pass(ctx) = get<_n>(ctx) == _val(ctx).size(); };
Here we check that we're "full" - i.e. no more integers are allowed.
Putting it all together:
Live On Coliru
#include <string>
#include <iostream>
#include <iomanip>
#include <boost/spirit/home/x3.hpp>
int main() {
for (std::string const input : {
"3 1 2 3", // correct
"4 1 2 3", // too few
"2 1 2 3", // too many
//
" 3 1 2 3 ",
})
{
std::cout << "\nParsing " << std::left << std::setw(20) << ("'" + input + "':");
std::vector<int> v;
bool ok;
{
using namespace boost::spirit::x3;
unsigned n;
struct _n{};
auto number = [](auto &ctx) { get<_n>(ctx).get() = _attr(ctx); };
auto more = [](auto &ctx) { _pass(ctx) = get<_n>(ctx) > _val(ctx).size(); };
auto done = [](auto &ctx) { _pass(ctx) = get<_n>(ctx) == _val(ctx).size(); };
auto r = rule<struct _r, std::vector<int> > {}
%= with<_n>(std::ref(n))
[ omit[uint_[number] ] >> *(eps [more] >> int_) >> eps [done] ];
ok = phrase_parse(input.begin(), input.end(), r >> eoi, space, v);
}
if (ok) {
std::copy(v.begin(), v.end(), std::ostream_iterator<int>(std::cout << v.size() << " elements: ", " "));
} else {
std::cout << "Parse failed";
}
}
}
Which prints:
Parsing '3 1 2 3': 3 elements: 1 2 3
Parsing '4 1 2 3': Parse failed
Parsing '2 1 2 3': Parse failed
Parsing ' 3 1 2 3 ': 3 elements: 1 2 3
¹ lend your support/voice at the [spirit-general] mailing list :)
² can't find a suitable documentation link, but it's used in some of the samples
I am trying to use Boost Spirit X3 with semantic actions while parsing the structure to an AST. If I use a rule without separate definition and instantiation it works just fine, for example:
#include <vector>
#include <string>
#include <iostream>
#include <boost/fusion/include/adapt_struct.hpp>
#include <boost/spirit/home/x3.hpp>
namespace ast
{
struct ast_struct
{
int number;
std::vector<int> numbers;
};
}
BOOST_FUSION_ADAPT_STRUCT(
ast::ast_struct,
(int, number)
(std::vector<int>, numbers)
)
namespace x3 = boost::spirit::x3;
using namespace std;
void parse( const std::string &data )
{
string::const_iterator begin = data.begin();
string::const_iterator end = data.end();
unsigned n(0);
auto f = [&n]( auto &ctx )
{
n = x3::_attr(ctx);
};
ast::ast_struct ast;
bool r = x3::parse( begin, end,
x3::int_[f] >> +( x3::omit[+x3::blank] >> x3::int_ ), ast );
if ( r && begin == end )
{
cout << "n: " << n << ", ";
std::copy(ast.numbers.begin(), ast.numbers.end(),
std::ostream_iterator<int>(std::cout << ast.numbers.size() << " elements: ", " "));
cout << endl;
}
else
cout << "Parse failed" << endl;
}
int main()
{
parse( "3 1 2 3" );
parse( "4 1 2 3 4" );
return 0;
}
Running the code above (compiled with flags -std=c++14) outputs the expected result:
n: 3, 3 elements: 1 2 3
n: 4, 4 elements: 1 2 3 4
Now I am trying to have my Spirit X3 parser organized more or less the same way as the calc 9 example from Boost Spirit X3, but it does not work:
ast.hxx: defines the abstract syntax tree.
grammar.hxx: user interface exposing the parser methods.
grammar.cxx: instantiates the rules.
grammar_def.hxx: parser grammar definition.
config.hxx: parser configuration.
main.cxx: parser usage example.
ast.hxx:
#ifndef AST_HXX
#define AST_HXX
#include <vector>
#include <boost/fusion/include/adapt_struct.hpp>
namespace ast
{
struct ast_struct
{
int number;
std::vector<int> numbers;
};
}
BOOST_FUSION_ADAPT_STRUCT(
ast::ast_struct,
(int, number)
(std::vector<int>, numbers)
)
#endif
grammar.hxx:
#ifndef GRAMMAR_HXX
#define GRAMMAR_HXX
#include "ast.hxx"
#include <boost/spirit/home/x3.hpp>
namespace parser
{
namespace x3 = boost::spirit::x3;
using my_rule_type = x3::rule<class my_rule_class, ast::ast_struct>;
BOOST_SPIRIT_DECLARE( my_rule_type );
const my_rule_type &get_my_rule();
}
#endif
grammar.cxx:
#include "grammar_def.hxx"
#include "config.hxx"
namespace parser
{
BOOST_SPIRIT_INSTANTIATE( my_rule_type, iterator_type, context_type )
}
grammar_def.hxx:
#ifndef GRAMMAR_DEF_HXX
#define GRAMMAR_DEF_HXX
#include <iostream>
#include <boost/spirit/home/x3.hpp>
#include "grammar.hxx"
#include "ast.hxx"
namespace parser
{
namespace x3 = boost::spirit::x3;
const my_rule_type my_rule( "my_rule" );
unsigned n;
auto f = []( auto &ctx )
{
n = x3::_attr(ctx);
};
auto my_rule_def = x3::int_[f] >> +( x3::omit[+x3::blank] >> x3::int_ );
BOOST_SPIRIT_DEFINE( my_rule )
const my_rule_type &get_my_rule()
{
return my_rule;
}
}
#endif
config.hxx:
#ifndef CONFIG_HXX
#define CONFIG_HXX
#include <string>
#include <boost/spirit/home/x3.hpp>
namespace parser
{
namespace x3 = boost::spirit::x3;
using iterator_type = std::string::const_iterator;
using context_type = x3::unused_type;
}
#endif
main.cxx:
#include "ast.hxx"
#include "grammar.hxx"
#include "config.hxx"
#include <iostream>
#include <boost/spirit/home/x3.hpp>
#include <string>
namespace x3 = boost::spirit::x3;
using namespace std;
void parse( const std::string &data )
{
parser::iterator_type begin = data.begin();
parser::iterator_type end = data.end();
ast::ast_struct ast;
cout << "Parsing [" << string(begin,end) << "]" << endl;
bool r = x3::parse( begin, end, parser::get_my_rule(), ast );
if ( r && begin == end )
{
std::copy(ast.numbers.begin(), ast.numbers.end(),
std::ostream_iterator<int>(std::cout << ast.numbers.size() << " elements: ", " "));
cout << endl;
}
else
cout << "Parse failed" << endl;
}
int main()
{
parse( "3 1 2 3" );
parse( "4 1 2 3 4" );
return 0;
}
Compiling main.cxx and grammar.cxx (flags: -std=c++14) and running the code above prints:
Parsing [3 1 2 3]
0 elements:
Parsing [4 1 2 3 4]
0 elements:
I apologize for the long source code, I tried to make it as small as possible.
Please notice I have some usage for the unsigned n global variable, it will be used with a custom repeat directive (see question here and one of the solutions here). In order to keep the question focused I removed the repeat part from this question, so even though I could remove the semantic action in this example, it is not a possible solution.
I would appreciate some help to get this issue uncovered, it is not clear to me why the code above does not work. Thank you in advance.
I must admit actually reconstructing your sample was a bit too much work for me (call me lazy...).
However, I know the answer and a trick to make your life simpler.
The Answer
Semantic actions on a rule definition inhibit automatic attribute propagation. From the Qi docs (the same goes for X3, but I always lose the link to the docs):
r = p; Rule definition
This is equivalent to r %= p (see below) if there are no semantic actions attached anywhere in p.
r %= p; Auto-rule definition
The attribute of p should be compatible with the synthesized attribute of r. When p is successful, its attribute is automatically propagated to r's synthesized attribute.
The Trick
You can inject state (your n reference, in this case) using the x3::with<> directive. That way you don't have the namespace global (n) and can make the parser reentrant, threadsafe etc.
Here's my "simplist" take on things, in a single file:
namespace parsing {
x3::rule<struct parser, ast::ast_struct> parser {"parser"};
struct state_tag { };
auto record_number = [](auto &ctx) {
unsigned& n = x3::get<state_tag>(ctx);
n = x3::_attr(ctx);
};
auto parser_def = x3::rule<struct parser_def, ast::ast_struct> {}
%= x3::int_[record_number] >> +(x3::omit[+x3::blank] >> x3::int_);
BOOST_SPIRIT_DEFINE(parser)
}
Tip: run the demo with = instead of the %= to see the difference in behaviour!
Note that get<state_tag>(ctx) returns a reference_wrapper<unsigned> just because we use the parser as follows:
void parse(const std::string &data) {
using namespace std;
ast::ast_struct ast;
unsigned n;
auto parser = x3::with<parsing::state_tag>(ref(n)) [parsing::parser] >> x3::eoi;
if (x3::parse(data.begin(), data.end(), parser, ast)) {
cout << "n: " << n << ", ";
copy(ast.numbers.begin(), ast.numbers.end(), ostream_iterator<int>(cout << ast.numbers.size() << " elements: ", " "));
cout << "\n";
} else
cout << "Parse failed\n";
}
Live Demo
Live On Coliru
#include <boost/fusion/include/adapt_struct.hpp>
#include <boost/spirit/home/x3.hpp>
#include <iostream>
namespace ast {
struct ast_struct {
int number;
std::vector<int> numbers;
};
}
BOOST_FUSION_ADAPT_STRUCT(ast::ast_struct, number, numbers)
namespace x3 = boost::spirit::x3;
namespace parsing {
x3::rule<struct parser, ast::ast_struct> parser {"parser"};
struct state_tag { };
auto record_number = [](auto &ctx) {
unsigned& n = x3::get<state_tag>(ctx); // note: returns reference_wrapper<T>
n = x3::_attr(ctx);
};
auto parser_def = x3::rule<struct parser_def, ast::ast_struct> {}
%= x3::int_[record_number] >> +(x3::omit[+x3::blank] >> x3::int_);
BOOST_SPIRIT_DEFINE(parser)
}
void parse(const std::string &data) {
using namespace std;
ast::ast_struct ast;
unsigned n = 0;
auto parser = x3::with<parsing::state_tag>(ref(n)) [parsing::parser] >> x3::eoi;
if (x3::parse(data.begin(), data.end(), parser, ast)) {
cout << "n: " << n << ", ";
copy(ast.numbers.begin(), ast.numbers.end(), ostream_iterator<int>(cout << ast.numbers.size() << " elements: ", " "));
cout << "\n";
} else
cout << "Parse failed\n";
}
int main() {
parse("3 1 2 3");
parse("4 1 2 3 4");
}
Prints
n: 3, 3 elements: 1 2 3
n: 4, 4 elements: 1 2 3 4