I'm working on a parser combinator library, and I'd really like my parser to simply be some callable object:
typedef std::function<parse_result(parse_stream)> parser;
Which makes the parser combinators nice, eg:
parser operator &(parser a, parser b) { return both(a,b); }
but I'd like two features:
1) I'd like string literals to get promoted to a parser automatically so you can do things like:
parser option = "<" & regexp("[^+>]+");
2) I'd like a parser to have a name that I can use for error formatting. In the case of the "both" parser above, I could print that I expected a.name() and b.name() for example.
The two options I've tried so far are
a parser class that's callable, this lets me build from strings and std::function instances but a general callable has to be converted to a std::function first and from there to a parser, and C++ won't do two implicit conversions
Inheriting from std::function so I can implicitly convert the functions, but this seems to have a lot of gotchas in terms of only converting callables into a parser.
Does anyone have any thoughts on how to structure this?
You don't want a raw typedef of std function; your parser is more than a any std function.
struct parser: std::function<parse_result(parse_stream)>{
using base = std::function<parse_result(parse_stream)>;
using base::base;
};
this should permit
parser p = []( parse_stream str ) { return parse_result(7); };
as we use inheriting constructors to expose the raw std::function ctors in parser.
While you can override:
parser operator&(parser a, parser b) { return both(a,b); }
with the typedef version by putting & in the namespace of parse_result or parse_stream, I'd advise against it; there has been chat in the standarizatoin to restrict that kind of template-argument ADL. With a bare parser type, the spot to put such operator overloads is clear.
In addition some types cannot be overloaded outside of the class, like &=. With a struct you can do it in there.
None of this fixes
parser option = "<" & regexp("[^+>]+");
as the problem here is that the right hand side has no idea what the left hand side is doing (unless regexp is a function returing a parser).
First do this:
struct parser: std::function<parse_result(parse_stream)>{
using base = std::function<parse_result(parse_stream)>;
parser( char const* str ):
base( [str=std::string(str)](parse_stream stream)->parse_result { /* do work */ } )
{}
parser( char c ):
base( [c](parse_stream str)->parse_result { /* do work */ } )
{}
using base::base;
};
then you can add
namespace parsing {
// parser definition goes here
inline namespace literals {
inline parser operator""_p( char const* str ) { return str; }
}
}
and using namespace parsing::literals means "hello"_p is a parser that attempts to parse the string "hello".
Related
I have the following C++ snippet:
template_par<std::string> a("name", "Joh Node");
template_par<int> b("age", 23);
std::string result = templater<SomeTemplateClass>().templatize(a, b).get();
Which tries to implement a template engine for various purposes. The important parts of the templater class are:
template<class T>
class templater
{
std::map<std::string, std::string> kps;
public:
template <typename T1>
templater& templatize(template_par<T1> b)
{
kps.insert(make_pair(b.getKey(), to_string(b.getValue())));
return *this;
}
template<typename T1, typename... Args1>
templater& templatize(T1 first, Args1... args)
{
kps.insert(make_pair(first.getKey(), to_string(first.getValue())));
return templatize(args...);
}
}
ie. a template function with variable arguments .... template_par<T> are just template parameter classes for basic stuff. Right now as it is, this works, does the job nicely.
However, I would like to be able to shorten somehow the way I call the templatize method not only for aesthetics but also for a challenge... I think it would look much nicer something like:
std::string result = templater<SomeTemplateClass>().templatize(
"name" -> "Joh Node",
"age" -> 42
);
However this approach is not feasible due to the operator -> being a somewhat rigid piece of C++ ... (std::string result = templater<SomeTemplateClass>().templatize is not the important part here, that can be hided in a friendly construct, I am more worried about the variable number of parameters)
Any good beautification ideas for the challenge above?
Take a look at Boost.Assign, in particular the map assignment part of it, which you could co-opt here:
std::string result = templater<SomeTemplateClass>()
("Name", "Joh Node")
("Age", 42).templatize();
I would avoid getting much more creative than that, it makes the code cryptic. That said, if you want to experiment wildly, you might like my named operators, which would allow syntax such as:
std::string result = templater<SomeTemplateClass>().templatize(
"name" <is> "Joh Node",
"age" <is> 42
);
Here, is can be any valid C++ identifier. So conventional operators are unfortunately out, but pretty much everything flies. Even, if you really want to push it, <_>.
I want to change a local variable value in semantic action, like following:
#define BOOST_SPIRIT_USE_PHOENIX_V3
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <string>
namespace qi = boost::spirit::qi;
namespace spirit = boost::spirit;
namespace ascii = boost::spirit::ascii;
using boost::phoenix::ref;
using boost::phoenix::bind;
void dummy(const std::vector<char>& v, int& var)
{
var = 7;
}
template <typename Iterator>
struct x_grammar : public qi::grammar<Iterator, std::string(), ascii::space_type>
{
public:
x_grammar() : x_grammar::base_type(start_rule, "x_grammar")
{
using namespace qi;
int local_var = 0;
start_rule = (+(char_ - ";"))[bind(dummy, _1, ref(local_var))];
//repeat(ref(local_var))[some_rule];
}
private:
qi::rule<Iterator, std::string(), ascii::space_type> start_rule;
};
int main()
{
typedef std::string::const_iterator iter;
std::string storage("string;aaa");
iter it_begin(storage.begin());
iter it_end(storage.end());
std::string read_data;
using boost::spirit::ascii::space;
x_grammar<iter> g;
try {
bool r = qi::phrase_parse(it_begin, it_end, g, space, read_data);
std::cout << "Pass!\n";
} catch (const qi::expectation_failure<iter>& x) {
std::cout << "Error!\n";
}
}
I am getting some annoying compile errors using GCC 4.6.1 with boost 1.55.
I can't help but note that if compiler errors annoy you, then perhaps you should write valid code :/
Instructive Hat On...
While that's of course a flippant remark, it's also somewhat enlightening.
I've told you twice now that the whole idea of using constructor local variables in your grammar is fundamentally broken:
Boost spirit semantic action not invoked
Boost spirit using local variables
What you want is
inherited attributes
qi::locals
maayyyyybe, maaaayyybe grammar member variables; with the caveat that they make your rules non-re-entrant.
The important thing here to really get inside your head is
Boost Spirit generates parser from expression templates. Expression templates are 90% static information (type only), and get "compiled" (.compile()) into "invokable" (.parse()) form.
Most importantly, while you can write control flow in your semantic actions, none of this actually executed at the definition site. It's "compiled" into a lazy actor that can later be invoked.
The generated parse will conditionally invoke the lazy actor when the corresponding parse expression matches
Constructive Hat On...
It looks like you just want to transform attributes using a function.
Here's what you can do:
transform as part of the semantic action, placing the result into the regular attribute (maintaining 'functional' semantics for parser composition):
qi::rule<Iterator, exposed(), Skipper> myrule;
myrule = int_ [ _val = phx::bind(custom_xform, _1) ];
Where custom_xform is any old-school calleable (including polymorphic ones):
exposed custom_xform(int i) { return make_new_exposed(i); }
// or
struct custom_xfrom_t {
template <typename> struct result { typedef exposed type; };
template <typename Int>
exposed operator()(Int i) const {
return make_new_exposed(i);
}
};
static const custom_xform_t custom_xform;
You can add some syntactic sugar [1]
qi::rule<Iterator, exposed(), Skipper> myrule;
myrule = int_ [ _val = custom_xform(_1) ];
This requires custom_xform is defined as a lazy actor:
phx::function<custom_xform_t> custom_xform; // `custom_xform_t` again the (polymorphic) functor
You may note this wouldn't work for a regular function. You could wrap it in a calleable object, or use the BOOST_PHOENIX_ADAPT_FUNCTION macro to do just that for you
If you have some more involved transformations that you want to apply more often, consider using the Spirit Customization Points:
Customization of Spirit's Attribute Handling, specifically:
Transform an Attribute to a Different Type
Store a Parsed Attribute Value
These work most smoothly if you choose specific types for your attributes (e.g. Ast::Multiplicity or Ast::VelocityRanking, instead of int or double
[1] using BOOST_SPIRIT_USE_PHOENIX_V3
The code compiles with C++03. However, when using GCC 4.6's C++11 support, the code fails to compile. Here are the relevant excerpts from the error:
/usr/local/include/boost/spirit/home/support/action_dispatch.hpp: In static
member function 'static void boost::spirit::traits::action_dispatch<
Component>::caller(F&&, A&& ...) [with F =
const std::_Bind<with Boost.Phoenix actors>]'
...
main.cpp:25:9: instantiated from 'x_grammar<Iterator>::x_grammar() [...]
/usr/local/include/boost/spirit/home/support/action_dispatch.hpp:142:13: error:
no matching function for call to 'boost::spirit::traits::
action_dispatch<...>::do_call(const std::_Bind<with Boost.Phoenix actors>)'
Despite the using boost::phoenix::bind directive, the unqualified call to bind() is resolving to std::bind() rather than boost::phoenix::bind(), but the arguments are resolving to Boost.Phoenix actors. The Boost.Spirit documentation specifically warns about not mixing placeholders from different libraries:
You have to make sure not to mix placeholders with a library they don't belong to and not to use different libraries while writing a semantic action.
Hence, the compilation problem can be resolved by being explicit when defining the semantic action. Use either:
std::bind(dummy, std::placeholders::_1, std::ref(local_var))
or:
boost::phoenix::bind(dummy, _1, ref(local_var))
While that resolves the compiler error, it is worth noting that the ref(local_var) object will maintain a dangling reference, as its lifetime extends beyond that of local_var. Here is a working example where local_var's lifetime is extend to beyond the scope of the constructor by making it static.
Say we have the following source code:
#include <iostream>
#include <string>
#include <iterator>
#include <boost/spirit/include/karma.hpp>
namespace karma = boost::spirit::karma;
template <typename OutputIterator> struct grammar : karma::grammar<OutputIterator, std::nullptr_t()> {
grammar() : grammar::base_type(query) {
query = "yeah";
}
karma::rule<OutputIterator, std::nullptr_t()> query;
};
int main(void) {
typedef std::back_insert_iterator<std::string> iterator_type;
std::string generated;
iterator_type output_it(generated);
//keys_and_values<sink_type> g;
grammar<iterator_type> g;
bool result = karma::generate(output_it, g, nullptr);
std::cout << result << ":" << generated << std::endl;
return 0;
}
This fails to compile because karma lacks some traits for std::nullptr_t (those are boost::spirit::traits::extract_c_string and boost::spirit::traits::char traits). More specifically, it fails because karma is not able to find a generator for an attribute of type std::nullptr_t.
I see several ways to cope with that:
Replace std::nullptr_t by karma::unused_type in the grammar definition : It works on this example but may introduce ambiguity in a more complex grammar.
Defining the traits specialization : In my opinion, this is dirty and not generic. Plus, it exposes my specialization of a standard type for everyone, leading to potential conflicts.
Specializing an attribute transform : Same problem of specializing a standard type just for me.
Write a custom generator : The best candidate so far, but it makes a serious load of highly templated code lines compared to the task complexity.
Put a intermediate rule with a karma::unused_type attribute. A quick fix that works but have no sense.
Question : How can I tell the karma::rule to generate a simple literal and not care about having or not a generator for its attribute ?
You seem to have stumbled on the inverse of the infamous single-element fusion sequence conundrum[1] :(
I noticed, because the error emanates from the code trying to verify that the input string matches the attribute (lit.hpp):
// fail if attribute isn't matched by immediate literal
typedef typename attribute<Context>::type attribute_type;
typedef typename spirit::result_of::extract_from<attribute_type, Attribute>::type
extracted_string_type;
using spirit::traits::get_c_string;
if (!detail::string_compare(
get_c_string(
traits::extract_from<attribute_type>(attr, context))
, get_c_string(str_), char_encoding(), Tag()))
{
return false;
}
However, that makes no sense at all, since the docs state:
lit, like string, also emits a string of characters. The main difference is that lit does not consumes [sic] an attribute. A plain string like "hello" or a std::basic_string is equivalent to a lit
So I just... on a whim thought to coerce things a little, by using the same workaround that works for single-element fusion sequences on the Qi side:
query = karma::eps << "yeah";
And, voilĂ : it works: Live On Coliru
[1] See
How do I use a class with only one attribute in a AST with Boost Spirit?
Spirit Qi attribute propagation issue with single-member struct
Compiler error when adapting struct with BOOST_FUSION_ADAPT_STRUCT
Adapt class containing a string member as synthesized attribute
Etc. This is a sad flaw that will probably need to be worked around for SpiritV2.
Possible answer : After posting this, I found a solution which satisfies me for the moment. That is : introduce a intermediate rule.
template <typename OutputIterator> struct grammar : karma::grammar<OutputIterator, std::nullptr_t()> {
grammar() : grammar::base_type(query) {
query = null_rule;
null_rule = "null";
}
karma::rule<OutputIterator, std::nullptr_t()> query;
karma::rule<OutputIterator, karma::unused_type()> null_rule;
};
I'm still interested in any comment, reproach or other solution.
Given the following actions
struct Data {
double d;
void operator()( double dd,
boost::spirit::qi::unused_type,
boost::spirit::qi::unused_type )
{ d = dd; }
};
struct Printer {
void operator()( double dd,
boost::spirit::qi::unused_type,
boost::spirit::qi::unused_type ) const
{ std::cout << dd; }
};
the code
void foo( const std::string &s ) {
Printer p;
boost::spirit::qi::parse( s.begin(), s.end(),
boost::spirit::qi::double_[ p ] );
}
does compile while
double foo( const std::string &s ) {
Data d;
boost::spirit::qi::parse( s.begin(), s.end(),
boost::spirit::qi::double_[ d ] );
return d.d;
}
does not.
Looking at the examples in http://www.boost.org/doc/libs/1_53_0/libs/spirit/doc/html/spirit/qi/tutorials/semantic_actions.html, one sees that the function objects use a operator() declared const. The error message C3848 of MSVC suggests something similar.
Is constness required here? The documentation in http://www.boost.org/doc/libs/1_53_0/libs/spirit/doc/html/spirit/qi/reference/action.html only says the signature void( Attrib&, Context, bool& ) is required.
Remark: I must admit I don't really understand the sentence
The function or function object is expected to return the value to
generate output from by assigning it to the first parameter, attr.
in this context.
Q.2: Remark: I must admit I don't really understand the sentence
A. You could look at boost spirit semantic action parameters for indepth explanation of it. Here is the short version:
void action_f(std::string& attribute,
qi::unused_type const& context,
bool& flag)
{
boost::fusion::at_c<0>(context.attributes) = "hello world"; // return the attribute value
flag = true; // signal parse success
}
Q.1: Is constness required here? The documentation in http://www.boost.org/doc/libs/1_53_0/libs/spirit/doc/html/spirit/qi/reference/action.html only says the signature void( Attrib&, Context, bool& ) is required.
A. The const-ness may not be explicitely required by the library, however the requirement is implicitely introduced by the C++ language due the way you are using the parser expression1:
The expression template
boost::spirit::qi::double_[ d ]
yields a temporary, which when passed to the qi::parse API can only get bound to a const reference2,3. This is where the 'const' on the whole parser expresion is introduced, and it extends equally to members of the subexpression(s), such as those that store the semantic action, d.
Therefore the instance of d at the time of deferred invocation will be logically const and therefore the operator() will not be selected, whereas operator() const will.
1. Upon more thought, it doesn't really depend on how you use it. The logic of my explanation is sound, but because spirit even supports inline parser expressions, call()-ing parsers is necessarily a const member operation on that parser, and as such all other operations will be in const context anyway. And indeed, in boost::spirit::traits::action_dispatch::operator() you'll see the calleables being passed as F const&, e.g. reflecting this.
2. Spirit V2 does not support move semantics for rules - nor, indeed, does it require them
3. The lifetime of the temporary will be extends to the end of the containing full expression, as per standardese
I've recently been doing a huge refactoring where I was changing a lot of my code to return booleans instead of an explicit return code. To aid this refactoring I decided to lean on the compiler where possible by getting it to tell me the places where my code needed to be changed. I did this by introducing the following class (see here for the lowdown on how this works):
///
/// Typesafe boolean class
///
class TypesafeBool
{
private:
bool m_bValue;
struct Bool_ {
int m_nValue;
};
typedef int Bool_::* bool_;
inline bool_ True() const { return &Bool_::m_nValue; }
inline bool_ False() const { return 0; }
public:
TypesafeBool( const bool bValue ) : m_bValue( bValue ){}
operator bool_() const { return m_bValue ? True() : False(); }
};
Now, instead of using a normal bool type as the return type, I used this class which meant that I couldn't compile something like this any more:
TypesafeBool SomeFunction();
long result = SomeFunction(); // error
Great: it has made the refactoring manageable on a huge codebase by letting the compiler do a lot of the hard work for me. So now I've finished my refactoring and I'd quite like to keep this class hanging around and carry on using it since it affords us an extra level of safety that the built-in bool type doesn't.
There is however one "problem" which is preventing me from doing this. At the moment we make heavy use of the ternary operator in our code, and the problem is that it is not compatible with this new class without explicit casts:
TypesafeBool result = ( 1 == 2 ? SomeFunction() : false ); // error: different types used
TypesafeBool result = ( 1 == 2 ? SomeFunction() : (TypesafeBool)false );
If I could "solve" this issue so that I could use my class in a seamless manner I would probably carry on using it throughout the codebase. Does anyone know of a solution to this issue? Or is it just impossible to do what I want?
In the context of the conditional operator, the type of the expression is the common type of the last two operands. The complete rules to determine this common type are a bit complex, but your case happens to be trivial: if one of the two possible return values is a class type, the other value must have the same class and the common type is obviously also that class.
That means that if one of the operands is a TypesafeBool, then the other must be as well.
Now the problem you're really trying to solve has been solved before. The trick is not providing a class; instead use a typedef. See for instance safe bool.
class CCastableToBool
{
public:
// ...
operator bool() const
{
//...
{
return true;
}
//...
return false;
}
private:
// ...
};
but beware, in C++ it is considered really dangerous to have a class that can be casted to bool. You are warned :-)
you can read this there, SafeBool
You should explicitely call TypesafeBool::True() in all your ternary tests.
TypesafeBool result = ( 1 == 2 ? SomeFunction().True() : false );
I don't know about a seamless manner, the ternary operator has some restrictions on its use...
However, why don't you define two constants ?
TypesafeBool const True = TypesafeBool(true);
TypesafeBool const False = TypesafeBool(false);
And then:
TypesafeBool result = ( 1 == 2 ? SomeFunction() : False );
Of course, it's a bit unorthodox since I play on the capitalization to avoid reusing a reserved word :)
Is it a possibility to make the constructor of TypesafeBool explicit? Of course, now the usage has to be
TypesafeBool result( 1 == 2 ? b : false );
Could you use an assignment operator that takes in a bool as the external argument, as well as one that takes a TypesafeBool? It might be something to try out...
Nice try, but if your code base is large, you are probably better off using a static checker such as PC-Lint to look for implicit bool<->int conversions instead.