Decode char UTF8 escapes in Boost Spirit - c++

Question

question asked: Spirit-general list
Hello all,
I'm not sure if my subject is correct, but the testcode will probably show
what I want to achieve.
I'm trying to parse things like:
'%40' to '#'
'%3C' to '<'
I have a minimal testcase below. I don't understand why
this doesn't work. It's probably me making a mistake but I don't see it.
Using:
Compiler: gcc 4.6
Boost: current trunk
I use the following compile line:
g++ -o main -L/usr/src/boost-trunk/stage/lib -I/usr/src/boost-trunk -g -Werror -Wall -std=c++0x -DBOOST_SPIRIT_USE_PHOENIX_V3 main.cpp
#include <iostream>
#include <string>
#define BOOST_SPIRIT_UNICODE
#include <boost/cstdint.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/phoenix/phoenix.hpp>
typedef boost::uint32_t uchar; // Unicode codepoint
namespace qi = boost::spirit::qi;
int main(int argc, char **argv) {
// Input
std::string input = "%3C";
std::string::const_iterator begin = input.begin();
std::string::const_iterator end = input.end();
using qi::xdigit;
using qi::_1;
using qi::_2;
using qi::_val;
qi::rule<std::string::const_iterator, uchar()> pchar =
('%' > xdigit > xdigit) [_val = (_1 << 4) + _2];
std::string result;
bool r = qi::parse(begin, end, pchar, result);
if (r && begin == end) {
std::cout << "Output: " << result << std::endl;
std::cout << "Expected: < (LESS-THAN SIGN)" << std::endl;
} else {
std::cerr << "Error" << std::endl;
return 1;
}
return 0;
}
Regards,
Matthijs Möhlmann

Answer 1

qi::xdigit does not do what you think it does: it returns the raw character (i.e. '0', not 0x00).
You could leverage qi::uint_parser to your advantage, making your parse much simpler as a bonus:
typedef qi::uint_parser<uchar, 16, 2, 2> xuchar;
no need to rely on phoenix (making it work on older versions of Boost)
get both characters in one go (otherwise, you might have needed to add copious casting to prevent integer sign extensions)
Here is a fixed up sample:
#include <iostream>
#include <string>
#define BOOST_SPIRIT_UNICODE
#include <boost/cstdint.hpp>
#include <boost/spirit/include/qi.hpp>
typedef boost::uint32_t uchar; // Unicode codepoint
namespace qi = boost::spirit::qi;
typedef qi::uint_parser<uchar, 16, 2, 2> xuchar;
const static xuchar xuchar_ = xuchar();
int main(int argc, char **argv) {
// Input
std::string input = "%3C";
std::string::const_iterator begin = input.begin();
std::string::const_iterator end = input.end();
qi::rule<std::string::const_iterator, uchar()> pchar = '%' > xuchar_;
uchar result;
bool r = qi::parse(begin, end, pchar, result);
if (r && begin == end) {
std::cout << "Output: " << result << std::endl;
std::cout << "Expected: < (LESS-THAN SIGN)" << std::endl;
} else {
std::cerr << "Error" << std::endl;
return 1;
}
return 0;
}
Output:
Output: 60
Expected: < (LESS-THAN SIGN)
'<' is indeed ASCII 60