Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 4 years ago.
Improve this question
I am trying to build a simple compiler and I am in the stage to test the Bison parser I created recently over some sample .decaf files, the parser works well with all keywords and the grammar's terminal and non-terminal tokens/types and rest of the grammar rules and actions, but there is only one problem that my parser does not recognize the New keyword/operator, when ever a statement includes a New keyword it results into an error in the output!
Defining New as a terminal token
%token T_New
CFG grammar rule and action for Expr that also includes rule and action for T_New
Expr : LValue '=' Expr { $$=new AssignExpr($1,new Operator(#2,"="),$3); }
| '(' Expr ')' { $$=$2; }
| Expr '+' Expr { $$=new ArithmeticExpr($1,new Operator(#2,"+"),$3); }
| Expr '-' Expr { $$=new ArithmeticExpr($1,new Operator(#2,"-"),$3); }
| Expr '*' Expr { $$=new ArithmeticExpr($1,new Operator(#2,"*"),$3); }
| Expr '/' Expr { $$=new ArithmeticExpr($1,new Operator(#2,"/"),$3); }
| Expr '%' Expr { $$=new ArithmeticExpr($1,new Operator(#2,"%"),$3); }
| '-' Expr %prec T_UnaryMinus { $$=new ArithmeticExpr(new Operator(#1,"-"),$2); }
| Expr T_And Expr { $$=new LogicalExpr($1,new Operator(#2,"&&"),$3); }
| Expr T_Or Expr { $$=new LogicalExpr($1,new Operator(#2,"||"),$3); }
| Expr '<' Expr { $$=new RelationalExpr($1,new Operator(#2,"<"),$3); }
| Expr T_LessEqual Expr { $$=new RelationalExpr($1,new Operator(#2,"<="),$3); }
| Expr '>' Expr { $$=new RelationalExpr($1,new Operator(#2,">"),$3); }
| Expr T_GreaterEqual Expr { $$=new RelationalExpr($1,new Operator(#2,">="),$3); }
| Expr T_Equal Expr { $$=new EqualityExpr($1,new Operator(#2,"=="),$3); }
| Expr T_NotEqual Expr { $$=new EqualityExpr($1,new Operator(#2,"!="),$3); }
| '!' Expr { $$=new LogicalExpr(new Operator(#1, "!"), $2); }
| T_ReadInteger '(' ')' { $$=new ReadIntegerExpr(#1); }
| T_ReadLine '(' ')' { $$=new ReadLineExpr(#1); }
| T_New Identifier { $$=new NewExpr(#2,new NamedType($2)); }
| T_NewArray '(' Expr ',' Type ')' { $$=new NewArrayExpr(#1,$3,$5); }
| LValue { $$=$1; }
| T_This { $$=new This(#1); }
| Call { $$=$1; }
| Constant { $$=$1; }
;
for example I have this sample file interface.decaf for testing and it has a main function as below:
void main() {
Colorable s;
Color green;
green = New(Color);
green.SetRGB(0, 0, 255);
s = New(Rectangle);
s.SetColor(green);
}
But when I run my parser over this sample file in the terminal I get this error:
*** Error line 33.
green = New(Color);
*** syntax error
I tried with other sample files and noticed that any file that has a statement that mentions 'New' keyword returns the same error.
I got some hint from this question that probably New keyword is mixed up between C and C++ and that's why its not recognized by bison. but I am still not able to figure out how to fix this ! Can anyone help please ?
Your grammar has a rule
| T_New Identifier { ...
matching a New keyword followed immediately by an identifier. However, your examples all have parenthesis around the identifier:
green = New(Color)
s = new(Rectangle)
thus the syntax error you are seeing -- the input has a ( where the grammar expects an identifier...
Related
I am learning Flex/Bison and we are currently on the part about semantics, previously have dealt with lexical and syntax errors. I have googled extensively and haven't been able to find a solution to my error. I am having trouble trying to understand why I need to declare '$4' when I thought it to be automatically done.
When I try to makefile I get this error:
flex scanner.l
mv lex.yy.c scanner.c
bison -d -v parser.y
paser.y:114.71-72: error: $4 of 'case' has no declared type
114 | case WHEN INT_LITERAL ARROW statement_ {case_statements.push_back($4);};
Here is the pseudo code I am trying to follow:
statement:
CASE expression IS cases OTHERS ARROW statement_ ENDCASE
{If the attribute of cases, is a number then
return it as the attribute otherwise return the
attribute of the OTHERS clause};
cases:
cases case
{if the attribute of cases is a number then return it as the
attribute otherwise return the attribute of case} |
%empty
{Set the attribute to the sentinel NAN} ;
case:
WHEN INT_LITERAL ARROW statement_
{$-2 contains the value of the expression after CASE.
It must be compared with the attribute of INT_LITERAL.
If they match the attribute of this production
should become the attribute of statement_
If they don't match, the attribute should be set to the
sentinel value NAN} ;
parser.y:
%{
#include <iostream>
#include <string>
#include <vector>
#include <map>
#include <math.h>
using namespace std;
#include "values.h"
#include "listing.h"
#include "symbols.h"
#include <stdlib.h>
#include <stdio.h>
int yylex();
void yyerror(const char* message);
Symbols<int> symbols;
//----------------------------------------------------------------------------------------------
vector<int> case_statements; //<<<<<<<<<<<<Is this wrong?
//---------------------------------------------------------------------------------------------
int result;
double *params;
%}
%define parse.error verbose
%union
{
CharPtr iden;
Operators oper;
int value;
}
%token <iden> IDENTIFIER
%token <value> INT_LITERAL REAL_LITERAL BOOL_LITERAL CASE TRUE FALSE
%token <oper> ADDOP MULOP RELOP OROP NOTOP REMOP EXPOP
%token ANDOP
%token BEGIN_ BOOLEAN END ENDREDUCE FUNCTION INTEGER IS REDUCE RETURNS
%token THEN WHEN ARROW
%token ELSE ENDCASE ENDIF IF OTHERS REAL
%type <value> body statement_ statement reductions expression relation term
factor case cases exponent unary primary
%type <oper> operator
%%
function:
function_header optional_variable body {result = $3;} ;
function_header:
FUNCTION IDENTIFIER optional_parameter RETURNS type ';' |
FUNCTION IDENTIFIER RETURNS type ';' |
error ';' ;
optional_variable:
optional_variable variable |
error ';' |
%empty ;
variable:
IDENTIFIER ':' type IS statement_ ;
parameters:
parameter optional_parameter;
optional_parameter:
optional_parameter ',' parameter |
%empty ;
parameter:
IDENTIFIER ':' type {symbols.insert($1, params[0]);} ;
type:
INTEGER |
REAL |
BOOLEAN ;
body:
BEGIN_ statement_ END ';' {$$ = $2;} ;
statement_:
statement ';' |
error ';' {$$ = 0;} ;
statement:
expression |
REDUCE operator reductions ENDREDUCE {$$ = $3;} |
IF expression THEN statement_ ELSE statement_ ENDIF {
if ($2 == true) {
$$ = $4;
}
else {
$$ = $6;
}
} ; |
CASE expression IS cases OTHERS ARROW statement_ ENDCASE {$$ = $<value>4 == $1 ? $4 : $7;} ;
cases:
cases case {$$ = $<value>1 == $1 ? $1 : $2;} |
%empty {$$ = NAN;} ;
//-----------------------------------------------------------------------------------------------------------
case:
case WHEN INT_LITERAL ARROW statement_ {case_statements.push_back($4);} ; //<<<<<<<<<How do I declare $4?
//-------------------------------------------------------------------------------------------------------------
operator:
ADDOP |
RELOP |
EXPOP |
MULOP ;
reductions:
reductions statement_ {$$ = evaluateReduction($<oper>0, $1, $2);} |
{$$ = $<oper>0 == ADD ? 0 : 1;} %empty ;
expression:
expression OROP relation {$$ = $1 || $3;} |
relation ;
expression:
expression ANDOP relation {$$ = $1 && $3;} |
relation ;
relation:
relation RELOP term {$$ = evaluateRelational($1, $2, $3);} |
term ;
term:
term ADDOP factor {$$ = evaluateArithmetic($1, $2, $3);} |
factor ;
factor:
factor MULOP primary {$$ = evaluateArithmetic($1, $2, $3);} |
factor REMOP exponent {$$ = $1 % $3;} |
exponent ;
exponent:
unary |
unary EXPOP exponent {$$ = pow($1, $3);} ;
unary:
NOTOP primary {$$ = $2;} |
primary;
primary:
'(' expression ')' {$$ = $2;} |
INT_LITERAL |
REAL_LITERAL |
BOOL_LITERAL |
IDENTIFIER {if (!symbols.find($1, $$)) appendError(UNDECLARED, $1);} ;
%%
void yyerror(const char* message)
{
appendError(SYNTAX, message);
}
int main(int argc, char *argv[])
{
params = new double[argc - 1]
for (int i = 1; i < argc; i++)
{
params[i - 1] = atof(argv[i]);
}
firstLine();
yyparse();
if (lastLine() == 0)
cout << "Result = " << result << endl;
return 0;
}
You need to assign a value/type to statement_:
statement_:
statement ';' {$$ = $1;}|
error ';' {$$ = MISMATCH;} ;
I am writing a compiler in C++ (using Visual Studio) for a small scripting language and I use this C parsing library.
So, I followed instructions from the documentation and I ended up on this peace of code:
int main()
{
mpc_parser_t* Int = mpc_new("int");
mpc_parser_t* Char = mpc_new("char");
mpc_parser_t* String = mpc_new("string");
mpc_parser_t* Id = mpc_new("id");
mpc_parser_t* Type = mpc_new("type");
mpc_parser_t* Formal = mpc_new("formal");
mpc_parser_t* Header = mpc_new("header");
mpc_parser_t* FuncDecl = mpc_new("funcdecl");
mpc_parser_t* VarDef = mpc_new("vardef");
mpc_parser_t* Expr = mpc_new("expr");
mpc_parser_t* Call = mpc_new("call");
mpc_parser_t* Atom = mpc_new("atom");
mpc_parser_t* Simple = mpc_new("simple");
mpc_parser_t* SimpleList = mpc_new("simplelist");
mpc_parser_t* Stmt = mpc_new("stmt");
mpc_parser_t* FuncDef = mpc_new("funcdef");
mpc_parser_t* Program = mpc_new("program");
/* Define them with the following Language */
mpca_lang(MPCA_LANG_DEFAULT,
" \
int : /-?[0-9]+/ ; \
char : /'[a-zA-Z0-9!##$%^&*()\\_+-,.\\/<>?;'|\"`~]'/ ; \
string : /\"(\\\\.|[^\"])*\"/ ; \
id : /[a-zA-Z][a-zA-Z0-9_-]*/ ; \
type : \"int\" | \"bool\" | \"char\" | <type> '[' ']' | \"list\" '[' <type> ']' ; \
formal : (\"ref\")? <type> <id> (',' <id>)* ; \
header : <type>? <id> '(' (<formal> (';' <formal>)*)? ')' ; \
funcdecl : \"decl\" <header> ; \
vardef : <type> <id> (',' <id>)* ; \
expr : <atom> | <int> | <char> | '(' <expr> ')' \
| ('+' | '-') <expr> | <expr> ('+' | '-' | '*' | '/' | \"mod\") <expr> \
| <expr> ('=' | \"<>\" | '<' | '>' | \"<=\" | \">=\") <expr> \
| \"true\" | \"false\" | \"not\" <expr> | <expr> (\"and\" | \"or\") <expr> \
| \"new\" <type> '[' <expr> ']' | \"nil\" | \"nil?\" '(' <expr> ')' \
| <expr> '#' <expr> | \"head\" '(' <expr> ')' | \"tail\" '(' <expr> ')' ; \
call : <id> '(' (<expr> (',' <expr>)*)? ')' ; \
atom : <id> | <string> | <atom> '[' <expr> ']' | <call> ; \
simple : \"skip\" | <atom> \":=\" <expr> | <call> ; \
simplelist : <simple> (',' <simple>)* ; \
stmt : <simple> | \"exit\" | \"return\" <expr> \
| \"if\" <expr> ':' <stmt>+ (\"elif\" <expr> ':' <stmt>+)* \
(\"else\" ':' <stmt>+)? \"end\" \
| \"for\" <simplelist> ';' <expr> ';' <simplelist> ':' <stmt>+ \"end\" ; \
funcdef : \"def\" <header> ':' (<funcdef> | <funcdecl> | <vardef>)* <stmt>+ \"end\" ; \
program : /^/ <funcdef> /$/ ; \
",
Int, Char, String, Id, Type, Formal, Header, FuncDecl, VarDef, Expr,
Call, Atom, Simple, SimpleList, Stmt, FuncDef, Program);
mpc_result_t r;
char* input = "def hey () : return 1 end";
if(mpc_parse("input", input, Program, &r))
{
mpc_ast_print((mpc_ast_t*)r.output);
mpc_ast_delete((mpc_ast_t*)r.output);
}
else
{
mpc_err_print(r.error);
mpc_err_delete(r.error);
}
PAUSE("Press any key to continue . . .");
/* Undefine and Delete our Parsers */
mpc_cleanup(17, Int, Char, String, Id, Type, Formal, Header, FuncDecl, VarDef, Expr,
Call, Atom, Simple, SimpleList, Stmt, FuncDef, Program);
return 0;
}
The problem is that I run into an huge loop in mpc_parse. That loop never actually reaches the end. After some time I get this exception:
Unhandled exception at 0x00CBBC9C in TonyCC.exe: 0xC0000005: Access violation reading location 0x0000000C.
I don't know why. I suspect there is something wrong with my grammar but I cannot figure out what.
If someone has used this library before, do you have any idea what the problem might be?
Note: I know it is difficult to read the grammar from C code so here is an image of the grammar:
Is there a way to specify that a Bison rule should NOT match if the lookahead token is a given value?
I currently have the following Bison grammar (simplified):
var_decl:
type ident
{
$$ = new NVariableDeclaration(*$1, *$2);
} |
type ident ASSIGN_EQUAL expr
{
$$ = new NVariableDeclaration(*$1, *$2, $4);
} |
type CURVED_OPEN STAR ident CURVED_CLOSE CURVED_OPEN func_decl_args CURVED_CLOSE
{
$$ = new NVariableDeclaration(*(new NFunctionPointerType(*$1, *$7)) /* TODO: free this memory */, *$4);
} |
type CURVED_OPEN STAR ident CURVED_CLOSE CURVED_OPEN func_decl_args CURVED_CLOSE ASSIGN_EQUAL expr
{
$$ = new NVariableDeclaration(*(new NFunctionPointerType(*$1, *$7)) /* TODO: free this memory */, *$4, $10);
} ;
...
deref:
STAR ident
{
$$ = new NDereferenceOperator(*$<ident>2);
} |
...
type:
ident
{
$$ = new NType($<type>1->name, 0, false);
delete $1;
} |
... ;
...
expr:
deref
{
$$ = $1;
} |
...
ident
{
$<ident>$ = $1;
} |
...
ident CURVED_OPEN call_args CURVED_CLOSE
{
$$ = new NMethodCall(*$1, *$3);
delete $3;
} |
...
CURVED_OPEN expr CURVED_CLOSE
{
$$ = $2;
} ;
...
call_args:
/* empty */
{
$$ = new ExpressionList();
} |
expr
{
$$ = new ExpressionList();
$$->push_back($1);
} |
call_args COMMA expr
{
$1->push_back($3);
} ;
The problem is that when parsing:
void (*ident)(char* some_arg);
It's seeing void (*ident) and deducing that it must be a function call instead of a function declaration. Is there a way I can tell Bison that it should favour looking ahead to match var_decl instead of reducing *ident and void into derefs and exprs?
any identifier can be a type
That's exactly the problem. LALR(1) grammars for C-like languages (or languages with C-like syntax for types) need to differentiate types and other identifiers at the token level. That is, you need IDENT and TYPEIDENT be two different tokens. (You will have to feed data about identifiers from the compiler back to the tokenizer). It's the most standard way to disambiguate the otherwise ambiguous grammar.
Update See, for instance, this ANSI C grammar for Yacc.
I was teaching myself Bison and headed over to wikipedia for the same and copy-pasted the entire code from the example that was put there [ http://en.wikipedia.org/wiki/GNU_Bison ]. It compiled and works perfect. Then, I OOPed it by adding in a bit of C++. Here's is my new Parser.y file:
%{
#include "TypeParser.h"
#include "ParserParam.h"
#include "addition.h"
%}
%define api.pure
%left '+' TOKEN_PLUS
%left '*' TOKEN_MULTIPLY
%left '-' TOKEN_SUBTRACT
%left '/' TOKEN_DIVIDE
%left '^' TOKEN_EXP
%token TOKEN_LPAREN
%token TOKEN_RPAREN
%token TOKEN_PLUS
%token TOKEN_MULTIPLY
%token <value> TOKEN_NUMBER
%type <expression> expr
%%
input:
expr { ((SParserParam*)data)->expression = $1; }
;
expr:
expr TOKEN_PLUS expr { $$ = new Addition($1, $2); }
| expr TOKEN_MULTIPLY expr { $$ = new Multiplication($1, $2); }
| expr TOKEN_SUBTRACT expr { $$ = new Addition($1, $2); }
| expr TOKEN_DIVIDE expr { $$ = new Multiplication($1, $2); }
| expr TOKEN_EXP expr { $$ = new Addition($1, $2); }
| TOKEN_LPAREN expr TOKEN_RPAREN { $$ = $2; }
| TOKEN_NUMBER { $$ = new Value($1); }
;
%%
But then I keep getting the following errors:
Parser.y:33.52-53: $2 of `expr' has no declared type
Parser.y:34.62-63: $2 of `expr' has no declared type
Parser.y:35.56-57: $2 of `expr' has no declared type
Parser.y:36.60-61: $2 of `expr' has no declared type
Parser.y:37.52-53: $2 of `expr' has no declared type
How do I resolve it? I mean, what have I changed that is causing this? I haven't changed anything from the wikipedia code, the %type% declaration is still there [The union has the same members, with type changed from SExpression to Expression.]. All classes i.e. Addition, Expression, Multiplication are defined and declared. I don't think that is what is causing the problem here, but just saying.
And why exactly does it have a problem only with $2. Even $1 is of type expr, then why do I not get any errors for $1?
Any help is appreciated...
In the rule expr TOKEN_PLUS expr $1 is the first expression, $2 is TOKEN_PLUS, and $3 is the second expression. See the bison manual.
So the semantic action needs to change from your { $$ = new Addition($1, $2); } to { $$ = new Addition($1, $3); }.
I am completely out of ideas. I spend every free minute this day on this, but I am completely out of ideas.
This is my Ocamlyacc grammar:
input: /* empty */ { }
| input stmt { }
stmt:
extern { print_endline "Got an extern import" }
| func { print_endline "Got function definition" }
| call { print_endline "Got function call" }
extern:
EXTERN proto { Extern $2 }
func:
DEF proto expr { Function ($2, $3) }
proto:
IDENTIFIER LPAREN id_list RPAREN { print_endline "Got prototype definition"; Prototype ($1, $3) }
id_list:
/* empty */ { [] }
| IDENTIFIER { [$1] }
| id_list COMMA IDENTIFIER { $3 :: $1 }
expr_list:
/* empty */ { [] }
| expr { [$1] }
| expr_list COMMA expr { $3 :: $1 }
expr:
call { $1 }
| expr OP expr { Binary ($2, $1, $3) }
| IDENTIFIER { Variable $1 }
| NUMBER { Number $1 }
| LPAREN expr RPAREN { $2 }
call:
IDENTIFIER LPAREN expr_list RPAREN { Call ($1, $3) }
When I start parsing def foo(a,b) a+b it should tell me it got a function and a prototype declaration, according to debug messages. But instead, I only get the message on parsing the proto rule.
Further debug messages show that the parser comes as far as to the a of the expression a+b and then stops. No error message, nothing else. It just stops as if the entire text hat been parsed completely without meeting any of the rules in stmt.
There are no shift/reduce error or similar. The AST types are also not the problem. I have no idea any more, maybe someone else can help. Surely it is something obvious but I cannot see it.
EDIT: Lexer by popular demand:
{
open Parser
}
rule token = parse
| [' ' '\t' '\n'] { token lexbuf }
| "def" { DEF }
| "extern" { EXTERN }
| "if" { IF }
| "then" { THEN }
| "else" { ELSE }
| ['+' '-' '*' '/'] as c { OP c }
| ['A'-'Z' 'a'-'z'] ['A'-'Z' 'a'-'z' '0'-'9' '_']* as id { IDENTIFIER id }
| ['0'-'9']*'.'['0'-'9']+ as num { NUMBER (float_of_string num) }
| '(' { LPAREN }
| ')' { RPAREN }
| ',' { COMMA }
| '#' { comment lexbuf }
| _ { raise Parsing.Parse_error }
| eof { raise End_of_file }
and comment = parse
| '\n' { token lexbuf }
| _ { comment lexbuf }
First point: I hated you a bit for not giving a compilable source code. I had to reinvent the AST types, the %token declarations etc. to test your code.
The problem is a delicate interplay between the
| eof { raise End_of_file }
lexing rule, and your grammar.
Raising Enf_of_file on EOF in the lexer is a good idea if your grammar never naturally encounters the end of the file. For example, grammars that are naturally \n-terminated or ;;-terminated will stop parsing at this point, and never get to the EOF token.
But your grammar isn't one of those. When the parser gets to DEF proto expr ., it asks for the next token to see if it weren't, by chance, and OP, so it calls the lexer, which finds EOF, and blows.
Here is my fix:
In lex.mll:
| eof { EOF }
In parse.mly:
%token EOF
%start stmt_eof
%type <Types.stmt> stmt_eof
[...]
stmt_eof: stmt EOF { $1 }
Finally, you should seriously consider Menhir as a replacement for ocamlyacc. It does everything ocamlyacc does, only better, with clearer grammar files (eg. you wouldn't have to reinvent the foo_list nonterminal each time), better error messages, debugging features...