how to use the PEGTL parser with a separate lexer? - c++

Question

I have already a lexer and want to use my own token types as the input for a PEGTL parser. As an example, here is a modified version of the sum example, which reads from stdin:
#include <cstdlib>
#include <iostream>
#include <sstream>
#include <string>
#include <tao/pegtl.hpp>
using namespace TAO_PEGTL_NAMESPACE;
namespace sum
{
struct num : seq< plus< digit > > {};
struct int_list
: list< num, one< ',' > >
{};
struct grammar
: seq< int_list, eof >
{};
template< typename Rule >
struct action
{};
template<>
struct action< num >
{
template< typename ActionInput >
static void apply( const ActionInput& in, int& sum )
{
sum += atoi(in.string().c_str());
}
};
} // namespace sum
struct Token {
typedef enum { COMMA, NUM, END_OF_FILE } Type;
Token(Type type, int num = 0) : type(type), num(num) {}
Type type;
int num;
};
int main()
{
// this works, can be called like this:
// echo -n "1,2,3" | ./a.out
int d = 0.0;
if( parse< sum::grammar, sum::action >( istream_input(std::cin, 16, "stdin"), d )) {
std::cout << "parsing OK; sum = " << d << std::endl;
}
else {
std::cout << "parsing failed" << std::endl;
}
// how can I do this when I have already the tokens in a vector?
std::vector<Token> tokens;
tokens.push_back(Token(Token::Type::NUM, 1));
tokens.push_back(Token(Token::Type::COMMA));
tokens.push_back(Token(Token::Type::NUM, 2));
tokens.push_back(Token(Token::Type::COMMA));
tokens.push_back(Token(Token::Type::NUM, 3));
tokens.push_back(Token(Token::Type::END_OF_FILE));
}
It can be compiled like this: g++ -std=c++17 -I PEGTL/include parser.cc. The files in PEGTL/include are from the PEGTL repository.
How do I need to modify the parser so that the vector of tokens are used instead of std::cin? I think I have to write my own ParseInput struct, but I couldn't define all required methods and types for it. And how can I use then the token objects in the parser for this sum example?