The below code is to parse a "key=val;.." string into std::map and it fails to compile with the error:
Error C2146 : syntax error: missing '>' before identifier 'value_type'
Error C2039 : 'value_type': is not a member of 'std::pair,std::allocator,std::basic_string,std::allocator>>' c:\git\risk-engine-core_tcp\stage\boost-1.66.0-barclays-1\include\boost\spirit\home\support\container.hpp
It does not like the last parameter, "contents" (std::map), passed as a container.
Boost version is 1.66
namespace qi = boost::spirit::qi;
std::map<std::string,std::string> contents;
std::string::iterator first = str.begin();
std::string::iterator last = str.end();
const bool result = qi::phrase_parse(first,last,
*( *(qi::char_-"=") >> qi::lit("=") >> *(qi::char_-";") >> -qi::lit(";") ),
ascii::space, contents);
Looking at the boost docs and stack overflow, I do not see any issue with the above code.
Did you include
#include <boost/fusion/adapted/std_pair.hpp>
Here's a working example with some improvement suggestions:
Live On Coliru
#include <boost/fusion/adapted/std_pair.hpp>
#include <boost/spirit/include/qi.hpp>
#include <map>
#include <iomanip> // std::quoted
namespace qi = boost::spirit::qi;
int main() {
std::string str("key = value");
std::string::const_iterator first = str.begin();
std::string::const_iterator last = str.end();
std::map<std::string, std::string> contents;
bool const result = qi::phrase_parse(first,last,
*( *~qi::char_('=') >> '=' >> *~qi::char_(';') >> -qi::lit(';') ),
qi::ascii::space, contents);
if (result) {
std::cout << "Parsed " << contents.size() << " elements\n";
for (auto& [k,v] : contents) {
std::cout << "\t" << std::quoted(k) << ": " << std::quoted(v) << "\n";
}
} else {
std::cout << "Parse failed\n";
}
if (first != last)
std::cout << "Remaining input unparsed: " << std::quoted(std::string(first, last)) << "\n";
}
Prints
Parsed 1 elements
"key": "value"
Related
I was toying with Boost.Spirit X3 calculator example when I encountered an error I couldn't get my head around.
I minimized the program to reduce complexity still throwing the same error.
Say I want to parse an input as a list of statements (strings) followed by a delimiter (';').
This is my structure:
namespace client { namespace ast
{
struct program
{
std::list<std::string> stmts;
};
}}
BOOST_FUSION_ADAPT_STRUCT(client::ast::program,
(std::list<std::string>, stmts)
)
The grammar is as follows:
namespace client
{
namespace grammar
{
x3::rule<class program, ast::program> const program("program");
auto const program_def =
*((*char_) > ';')
;
BOOST_SPIRIT_DEFINE(
program
);
auto calculator = program;
}
using grammar::calculator;
}
Invoked
int
main()
{
std::cout <<"///////////////////////////////////////////\n\n";
std::cout << "Expression parser...\n\n";
std::cout << //////////////////////////////////////////////////\n\n";
std::cout << "Type an expression...or [q or Q] to quit\n\n";
typedef std::string::const_iterator iterator_type;
typedef client::ast::program ast_program;
std::string str;
while (std::getline(std::cin, str))
{
if (str.empty() || str[0] == 'q' || str[0] == 'Q')
break;
auto& calc = client::calculator; // Our grammar
ast_program program; // Our program (AST)
iterator_type iter = str.begin();
iterator_type end = str.end();
boost::spirit::x3::ascii::space_type space;
bool r = phrase_parse(iter, end, calc, space, program);
if (r && iter == end)
{
std::cout << "-------------------------\n";
std::cout << "Parsing succeeded\n";
std::cout<< '\n';
std::cout << "-------------------------\n";
}
else
{
std::cout << "-------------------------\n";
std::cout << "Parsing failed\n";
std::cout << "-------------------------\n";
}
}
std::cout << "Bye... :-) \n\n";
return 0;
}
Error I get is
/opt/boost_1_66_0/boost/spirit/home/x3/support/traits/container_traits.hpp: In instantiation of ‘struct boost::spirit::x3::traits::container_value<client::ast::program, void>’:
.
.
.
/opt/boost_1_66_0/boost/spirit/home/x3/support/traits/container_traits.hpp:76:12: error: no type named ‘value_type’ in ‘struct client::ast::program’
struct container_value
/opt/boost_1_66_0/boost/spirit/home/x3/operator/detail/sequence.hpp:497:72: error: no type named ‘type’ in ‘struct boost::spirit::x3::traits::container_value<client::ast::program, void>’
, typename traits::is_substitute<attribute_type, value_type>::type());
^~~~~~
Things I tried:
Following Getting boost::spirit::qi to use stl containers
Even though it uses Qi I nonetheless tried:
namespace boost{namespace spirit{ namespace traits{
template<>
struct container_value<client::ast::program>
//also with struct container<client::ast::program, void>
{
typedef std::list<std::string> type;
};
}}}
You see I'm kinda in the dark, so expectably to no avail.
parser2.cpp:41:8: error: ‘container_value’ is not a class template
struct container_value<client::ast::program>
^~~~~~~~~~~~~~~
In the same SO question I author says "There is one known limitation though, when you try to use a struct that has a single element that is also a container compilation fails unless you add qi::eps >> ... to your rule."
I did try adding a dummy eps also without success.
Please, help me decipher what that error means.
Yup. This looks like another limitation with automatic propagation of attributes when single-element sequences are involved.
I'd probably bite the bullet and change the rule definition from what it is (and what you'd expect to work) to:
x3::rule<class program_, std::vector<std::string> >
That removes the root of the confusion.
Other notes:
you had char_ which also eats ';' so the grammar would never succeed because no ';' would follow a "statement".
your statements aren't lexeme, so whitespace is discarded (is this what you meant? See Boost spirit skipper issues)
your statement could be empty, which meant parsing would ALWAYS fail at the end of input (where it would always read an empty state, and then discover that the expected ';' was missing). Fix it by requiring at least 1 character before accepting a statement.
With some simplifications/style changes:
Live On Coliru
#include <boost/fusion/adapted.hpp>
#include <boost/spirit/home/x3.hpp>
#include <list>
namespace x3 = boost::spirit::x3;
namespace ast {
using statement = std::string;
struct program {
std::list<statement> stmts;
};
}
BOOST_FUSION_ADAPT_STRUCT(ast::program, stmts)
namespace grammar {
auto statement
= x3::rule<class statement_, ast::statement> {"statement"}
= +~x3::char_(';');
auto program
= x3::rule<class program_, std::list<ast::statement> > {"program"}
= *(statement >> ';');
}
#include <iostream>
#include <iomanip>
int main() {
std::cout << "Type an expression...or [q or Q] to quit\n\n";
using It = std::string::const_iterator;
for (std::string str; std::getline(std::cin, str);) {
if (str.empty() || str[0] == 'q' || str[0] == 'Q')
break;
auto &parser = grammar::program;
ast::program program; // Our program (AST)
It iter = str.begin(), end = str.end();
if (phrase_parse(iter, end, parser, x3::space, program)) {
std::cout << "Parsing succeeded\n";
for (auto& s : program.stmts) {
std::cout << "Statement: " << std::quoted(s, '\'') << "\n";
}
}
else
std::cout << "Parsing failed\n";
if (iter != end)
std::cout << "Remaining unparsed: " << std::quoted(std::string(iter, end), '\'') << "\n";
}
}
Which for input "a;b;c;d;" prints:
Parsing succeeded
Statement: 'a'
Statement: 'b'
Statement: 'c'
Statement: 'd'
I am trying to get the current line of the file I am parsing using boost spirit. I created a grammar class and my structures to parse my commands into. I would also like to keep track of which line the command was found on and parse that into my structures as well. I have wrapped my istream file iterator in a multi_pass iterator and then wrapped that in a boost::spirit::classic::position_iterator2. In my rules of my grammar how would I get the current position of the iterator or is this not possible?
Update: It is similar to that problem but I just need to be able to keep a count of all the lines processed. I don't need to do all of the extra buffering that was done in the solution.
Update: It is similar to that problem but I just need to be able to keep a count of all the lines processed. I don't need to do all of the extra buffering that was done in the solution.
Keeping a count of all lines processed is not nearly the same as "getting the current line".
Simple Take
If this is what you need, just check it after the parse:
Live On Wandbox
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/support_line_pos_iterator.hpp>
#include <fstream>
#include <set>
namespace qi = boost::spirit::qi;
int main() {
using It = boost::spirit::istream_iterator;
std::ifstream ifs("main.cpp");
boost::spirit::line_pos_iterator<It> f(It(ifs >> std::noskipws)), l;
std::set<std::string> words;
if (qi::phrase_parse(f, l, *qi::lexeme[+qi::graph], qi::space, words)) {
std::cout << "Parsed " << words.size() << " words";
if (!words.empty())
std::cout << " (from '" << *words.begin() << "' to '" << *words.rbegin() << "')";
std::cout << "\nLast line processed: " << boost::spirit::get_line(f) << "\n";
}
}
Prints
Parsed 50 words (from '"' to '}')
Last line processed: 22
Slightly More Complex Take
If you say "no, wait, I really did want to get the current line /while parsing/". The real full monty is here:
boost::spirit access position iterator from semantic actions
Here's the completely trimmed down version using iter_pos:
Live On Wandbox
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <boost/spirit/include/support_line_pos_iterator.hpp>
#include <boost/spirit/repository/include/qi_iter_pos.hpp>
#include <boost/fusion/adapted/std_pair.hpp>
#include <fstream>
#include <map>
namespace qi = boost::spirit::qi;
namespace qr = boost::spirit::repository::qi;
using LineNum = size_t;
struct line_number_f {
template <typename It> LineNum operator()(It it) const { return get_line(it); }
};
static boost::phoenix::function<line_number_f> line_number_;
int main() {
using Underlying = boost::spirit::istream_iterator;
using It = boost::spirit::line_pos_iterator<Underlying>;
qi::rule<It, LineNum()> line_no = qr::iter_pos [ qi::_val = line_number_(qi::_1) ];
std::ifstream ifs("main.cpp");
It f(Underlying{ifs >> std::noskipws}), l;
std::multimap<LineNum, std::string> words;
if (qi::phrase_parse(f, l, +(line_no >> qi::lexeme[+qi::graph]), qi::space, words)) {
std::cout << "Parsed " << words.size() << " words.\n";
if (!words.empty()) {
auto& first = *words.begin();
std::cout << "First word: '" << first.second << "' (in line " << first.first << ")\n";
auto& last = *words.rbegin();
std::cout << "Last word: '" << last.second << "' (in line " << last.first << ")\n";
}
std::cout << "Line 20 contains:\n";
auto p = words.equal_range(20);
for (auto it = p.first; it != p.second; ++it)
std::cout << " - '" << it->second << "'\n";
}
}
Printing:
Parsed 166 words.
First word: '#include' (in line 1)
Last word: '}' (in line 46)
Line 20 contains:
- 'int'
- 'main()'
- '{'
I'm trying to parse input which has either a plus or minus character, followed by an X or Y character, followed by an unsigned integer.
(char_('+') | char_('-')) >> char_("xyXY") >> uint_
According to my reading of the docs, the synthesised attribute for this would be tuple<vector<char>,unsigned int> because the alternative parser (char | char) would be of type char, the char >> char("xyXY") would be vector<char>, and the vector<char> >> uint_ would be a tuple of the types, so tuple<vector<char>,unsigned int>. This fails to compile
qi\detail\assign_to.hpp(152) : error C2440: 'static_cast' : cannot convert from 'const char' to 'boost::tuples::tuple<T0,T1>'
Code:
#include <iostream>
#include <string>
#include <vector>
#include <boost/fusion/include/tuple.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/tuple/tuple.hpp>
using namespace boost::spirit::qi;
int main()
{
std::string input("-Y 512");
typedef std::string::const_iterator Iterator;
Iterator first = input.begin();
Iterator last = input.end();
boost::tuple<std::vector<char>,unsigned int> output;
bool result = phrase_parse(first,last,(char_('+') | char_('-')) >> char_("xyXY") >> uint_,ascii::space,output);
if(result && first == last)
std::cout << "sign=" << boost::get<0>(output)[0] << ", xy=" << boost::get<0>(output)[1] << ", size=" << boost::get<1>(output) << '\n';
else
std::cerr << "Parse error\n";
}
I then tried tuple<char,char,unsigned int> as the attribute type:
#include <iostream>
#include <string>
#include <vector>
#include <boost/fusion/include/tuple.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/tuple/tuple.hpp>
using namespace boost::spirit::qi;
int main()
{
std::string input("-Y 512");
typedef std::string::const_iterator Iterator;
Iterator first = input.begin();
Iterator last = input.end();
boost::tuple<char,char,unsigned int> output;
bool result = phrase_parse(first,last,(char_('+') | char_('-')) >> char_("xyXY") >> uint_,ascii::space,output);
if(result && first == last)
std::cout << "sign=" << boost::get<0>(output) << ", xy=" << boost::get<1>(output) << ", size=" << boost::get<2>(output) << '\n';
else
std::cerr << "Parse error\n";
}
This compiles but the output is incorrect. The first token of the input is parsed correctly, but the subsequent tokens aren't:
sign=-, xy= , size=0
I also tried as_string[]:
#include <iostream>
#include <string>
#include <vector>
#include <boost/fusion/include/tuple.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/tuple/tuple.hpp>
using namespace boost::spirit::qi;
int main()
{
std::string input("-Y 512");
typedef std::string::const_iterator Iterator;
Iterator first = input.begin();
Iterator last = input.end();
boost::tuple<std::string,unsigned int> output;
bool result = phrase_parse(first,last,as_string[(char_('+') | char_('-')) >> char_("xyXY")] >> uint_,ascii::space,output);
if(result && first == last)
std::cout << "sign=" << boost::get<0>(output)[0] << ", xy=" << boost::get<0>(output)[1] << ", size=" << boost::get<1>(output) << '\n';
else
std::cerr << "Parse error\n";
}
This improved things as the x/y token got parsed, but not the third integer token:
sign=-, xy=Y, size=0
Please show me where I'm going wrong.
I'm using Spirit version 2.5.2 (from Boost 1.58.0) and Microsoft Visual Studio 2008.
Spirit library docs recommend to use Fusion tuple. I think I saw somewhere (can't find it now) that Boost tuple may not be fully compatible with Spirit library.
Here is your fixed example:
#include <iostream>
#include <string>
#include <vector>
#include <boost/fusion/include/tuple.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/fusion/sequence.hpp>
namespace qi = boost::spirit::qi;
int main()
{
std::string input("-Y 512");
typedef std::string::const_iterator Iterator;
Iterator first = input.begin();
Iterator last = input.end();
boost::fusion::tuple<char, char, unsigned int> output;
bool result = qi::phrase_parse(first, last, (qi::char_('+') | qi::char_('-')) >> qi::char_("xyXY") >> qi::uint_, qi::ascii::space, output);
if (result && first == last)
std::cout << "sign=" << boost::fusion::get<0>(output) << ", xy=" << boost::fusion::get<1>(output) << ", size=" << boost::fusion::get<2>(output) << '\n';
else
std::cerr << "Parse error\n";
return 0;
}
Output:
sign=-, xy=Y, size=512
Update: Actually I found here that it's possible to use boost::tuple but different header needs to be included: #include <boost/fusion/include/boost_tuple.hpp>.
#include <boost/spirit/include/qi.hpp>
#include <string>
#include <vector>
#include <iterator>
#include <algorithm>
#include <iostream>
using namespace boost::spirit;
int main()
{
std::string s;
std::getline(std::cin, s);
auto specialtxt = *(qi::char_('-', '.', '_'));
auto txt = no_skip[*(qi::char_("a-zA-Z0-9_.\\:$\'-"))];
auto anytxt = *(qi::char_("a-zA-Z0-9_.\\:${}[]+/()-"));
qi::rule <std::string::iterator, void(),ascii::space_type> rule2 = txt ('=') >> ('[') >> (']');
auto begin = s.begin();
auto end = s.end();
if (qi::phrase_parse(begin, end, rule2, ascii::space))
{
std::cout << "MATCH" << std::endl;
}
else
{
std::cout << "NO MATCH" << std::endl;
}
}
this code works fine in debug mode
parser fails in release mode
rule is to just parse text=[]; any thing else than this should fail it works fine in debug mode but not in release mode it shows result no match for any string.
if i enter string like
abc=[];
this passes in debug as expected but fails in release
You can't use auto with Spirit v2:
Assigning parsers to auto variables
You have Undefined Behaviour
DEMO
I tried to make (more) sense of the rest of the code. There were various instances that would never work:
txt('=') is an invalid Qi expression. I assumed you wanted txt >> ('=') instead
qi::char_("a-zA-Z0-9_.\\:$\\-{}[]+/()") doesn't do what you think because $-{ is actually the character "range" \x24-\x7b... Escape the - (or put it at the very end/start of the set like in the other char_ call).
qi::char_('-','.','_') can't work. Did you mean qi::char_("-._")?
specialtxt and anytxt were unused...
prefer const_iterator
prefer namespace aliases above using namespace to prevent hard-to-detect errors
Live On Coliru
#include <boost/spirit/include/qi.hpp>
#include <iostream>
namespace qi = boost::spirit::qi;
int main() {
std::string const s = "abc=[];";
auto specialtxt = qi::copy(*(qi::char_("-._")));
auto anytxt = qi::copy(*(qi::char_("a-zA-Z0-9_.\\:$\\-{}[]+/()")));
(void) specialtxt;
(void) anytxt;
auto txt = qi::copy(qi::no_skip[*(qi::char_("a-zA-Z0-9_.\\:$\'-"))]);
qi::rule<std::string::const_iterator, qi::space_type> rule2 = txt >> '=' >> '[' >> ']';
auto begin = s.begin();
auto end = s.end();
if (qi::phrase_parse(begin, end, rule2, qi::space)) {
std::cout << "MATCH" << std::endl;
} else {
std::cout << "NO MATCH" << std::endl;
}
if (begin != end) {
std::cout << "Trailing unparsed: '" << std::string(begin, end) << "'\n";
}
}
Printing
MATCH
Trailing unparsed: ';'
I have the following code:
#include <boost/fusion/include/adapt_struct.hpp>
#include <boost/fusion/include/io.hpp>
#include <boost/spirit/include/qi.hpp>
#include <iostream>
#include <string>
struct function
{
std::string ret_type;
std::string name;
};
BOOST_FUSION_ADAPT_STRUCT(
::function,
(std::string, ret_type)
(std::string, name)
)
template <typename Iterator>
struct function_parser : boost::spirit::qi::grammar<Iterator, function(), boost::spirit::qi::ascii::space_type>
{
function_parser() : function_parser::base_type(start)
{
using boost::spirit::qi::ascii::char_;
using boost::spirit::qi::int_;
start %= +char_ >> +char_;
}
boost::spirit::qi::rule<Iterator, function(), boost::spirit::qi::ascii::space_type> start;
};
int main()
{
std::string input_data("void foo");
function fn;
auto itr = input_data.begin();
auto end = input_data.end();
function_parser<decltype(itr)> g;
bool res = boost::spirit::qi::phrase_parse(itr, end, g, boost::spirit::ascii::space, fn);
if (res && itr == end)
{
std::cout << boost::fusion::tuple_open('[');
std::cout << boost::fusion::tuple_close(']');
std::cout << boost::fusion::tuple_delimiter(", ");
std::cout << "Parsing succeeded\n";
std::cout << "got: " << boost::fusion::as_vector(fn) << std::endl;
}
else
{
std::cout << "Parsing failed \n";
}
}
Output
Parsing failed
What am I doing wrong? How can I fix it?
+char_
eats all input! Now, the next
+char_
requires at least a single character, which isn't there (the first kleen plus ate it) so the parse fails.
I suggest instead:
using namespace boost::spirit::qi;
start = lexeme[+graph] >> lexeme[+graph];
The documentation should be able to tell you what that does (I hope. No time to elaborate)