When I run my bison code in Linux Mint I get these warnings:
warning: 64 shift/reduce conflicts [-Wconflicts-sr]
I cannot find the ambiguity. I've tried to make my implementation hanlde both INT and FLOAT datatypes, but I believe that is the source of the warnings.
My Bison code is below:
%{
/**
* Definition section
*/
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <cstdio>
#include <iostream>
using namespace std;
/**
* Declare stuff from Flex that Bison needs to know about:
*/
extern int yylex();
extern int yyparse();
extern FILE* yyin;
extern int line_num;
void yyerror(const char* s);
%}
/** Bison fundamentally works by asking flex to get the next token,
* which it returns as an object of type "yystype". Initially (by default),
* yystype is merely a typedef of "int", but for non-trivial projects, tokens
* could be of any arbitrary data type. So, to deal with that, the idea is to
* override yystype's default typedef to a C union instead. Unions can hold all
* of the types of tokens that Flex could return, and this means we can return
* ints or floats or string cleanly. Bison implements this mechanism with the
* %union directive:
*/
%union {
int ival;
float fval;
}
/**
* Define the "Terminal Symbol" toke types. We use (in CAPS by convention),
* and associate each with a field of the %union:
*/
%token<ival> INT
%token<fval> FLOAT
%token PLUS MINUS MULTIPLY DIVIDE
%token NEWLINE QUIT
%type<ival> expression
%type<fval> mixed_expression
%start calculation
%%
calculation:
| calculation line
;
line: NEWLINE
| mixed_expression NEWLINE {printf("\tResult: %f\n", $1);}
| expression NEWLINE {printf("\tResult: %i\n", $1);}
| QUIT NEWLINE {printf("bye!\n"); exit(0);}
;
mixed_expression: FLOAT {$$ = $1;}
| mixed_expression PLUS mixed_expression {$$ = $1 + $3;}
| mixed_expression MINUS mixed_expression {$$ = $1 - $3;}
| mixed_expression MULTIPLY mixed_expression {$$ = $1 * $3;}
| mixed_expression DIVIDE mixed_expression {$$ = $1 / $3;}
| expression PLUS mixed_expression {$$ = $1 + $3;}
| expression MINUS mixed_expression {$$ = $1 - $3;}
| expression MULTIPLY mixed_expression {$$ = $1 * $3;}
| expression DIVIDE mixed_expression {$$ = $1 / $3;}
| mixed_expression PLUS expression {$$ = $1 + $3;}
| mixed_expression MINUS expression {$$ = $1 - $3;}
| mixed_expression MULTIPLY expression {$$ = $1 * $3;}
| mixed_expression DIVIDE expression {$$ = $1 / $3;}
| expression DIVIDE expression {$$ = $1 / (float)$3;}
;
expression: INT {$$ = $1;}
| expression PLUS expression {$$ = $1 + $3;}
| expression MINUS expression {$$ = $1 - $3;}
| expression MULTIPLY expression {$$ = $1 * $3;}
;
%%
int main(int, char**) {
// Open a file handle to a particular file:
FILE *myfile = fopen("numbers.txt", "r");
// Make sure it is valid:
if(!myfile) {
cout << "I can't open numbers.txt!" << endl;
return -1;
}
// Set Flex to read from it instead of defaulting to STDIN:
yyin = myfile;
// Parse through the input:
yyparse();
}
void yyerror(const char *s) {
cout << "AAAHHHH, parse error on line " << line_num << "! Message: " << s << endl;
// might as well halt now:
exit(-1);
}
I believe that the issue may be with my implementation of expression and mixed_expression.
The datatype of a computation is semantic, not syntactic; intuitively, one wouldn't expect a grammar to deal with it. Under some very limited circumstances it's possible — your attempt is workable, at least until you decide to implement variables — but it's almost never a good idea.
But that's not what's producing the shift-reduce conflicts. The problem is that your grammar is ambiguous and you have not told Bison how to resolve the ambiguities. The grammar is ambiguous because 4-2*3 could be parsed either as expression(4-2) TIMES expression(3) or as expression(4) MINUS expression(2*3), which obviously have different values. The usual way to resolve this particular ambiguity is through precedence declarations, which you will find deployed in pretty well all of the useful example parsers.
Related
I have the following code below and I am receiving the following error
parser.y:111.47-48: error: $$ for the midrule at $5 of ‘statement’ has no declared type
111 | REDUCE operator reductions ENDREDUCE {$$ = $3;} ';'|
I know it's generated because I didn't declare a type for something in the statement, I need help understanding line 111. Also the REAL_LITERAL is a float, that I should add a float to the union and create token like this %token <f_value>REAL_LITERAL.
include <iostream>
#include <string>
#include <vector>
#include <map>
using namespace std;
#include "math.h"
#include "values.h"
#include "listing.h"
#include "symbols.h"
int yylex();
void yyerror(const char* message);
Symbols<int> symbols;
int result;
double *params;
%}
%define parse.error verbose
%union
{
CharPtr iden;
Operators oper;
int value;
}
%token <iden> IDENTIFIER
%token <value>INT_LITERAL REAL_LITERAL BOOL_LITERAL CASE TRUE FALSE
%token ARROW
%token <oper> ADDOP MULOP RELOP OROP NOTOP REMOP EXPOP
%token ANDOP
%token BEGIN_ BOOLEAN END ENDREDUCE FUNCTION INTEGER IS REDUCE RETURNS
%token THEN WHEN
%token ELSE ENDCASE ENDIF IF OTHERS REAL
%type <value> body statement_ statement reductions expression binary relation term
factor primary
%type <oper> operator
%left OROP
%left ANDOP
%left RELOP
%left ADDOP
%left MULOP REMOP
%right EXPOP
%left NOTOP
%%
function:
function_header optional_variable body {result = $3;} ;
function_header:
FUNCTION IDENTIFIER parameters RETURNS type ';' |
FUNCTION IDENTIFIER RETURNS type ';' |
FUNCTION IDENTIFIER optional_parameters RETURNS type ';' |
error ';' ;
optional_variable:
optional_variable variable |
error ';' ;
;
variable:
IDENTIFIER ':' type IS statement_ {symbols.insert($1, $5);} ;
variables:
variable variables |
;
type:
INTEGER |
BOOLEAN ;
optional_parameters:
parameters |
;
parameters:
parameter ',' parameters |
parameter ;
parameter:
IDENTIFIER ':' type ;
type:
INTEGER |
REAL |
BOOLEAN ;
body:
BEGIN_ statement_ END ';' {$$ = $2;} ;
statement_:
statement ';' |
error ';' {$$ = 0;} ;
statement:
expression |
REDUCE operator reductions ENDREDUCE {$$ = $3;} ';'|
IF expression THEN statement_ ELSE statement_ ENDIF
{
if ($2 == true) {
$$ = $4;
}
else {
$$ = $6;
}
}';' /*|
CASE expression IS cases OTHERS ARROW statement_ ENDCASE
{$$ = $<value>4 == $1 ? $4 : $7;} ;
cases:
cases case
{$$ = $<value>1 == $1 ? $1 : $2;} |
%empty {$$ = NAN;};
case:
case WHEN INT_LITERAL ARROW statement_ |
;
*/
operator:
ADDOP |
RELOP |
EXPOP |
MULOP ;
reductions:
reductions statement_ {$$ = evaluateReduction($<oper>0, $1, $2);} |
{$$ = $<oper>0 == ADD ? 0 : 1;} ;
expression:
expression OROP binary {$$ = $1 || $3;} |
binary;
binary:
binary ANDOP relation {$$ = $1 && $3;} |
relation ;
relation:
relation RELOP term {$$ = evaluateRelational($1, $2, $3);} |
term ;
term:
term ADDOP factor {$$ = evaluateArithmetic($1, $2, $3);} |
factor ;
factor:
factor MULOP primary {$$ = evaluateArithmetic($1, $2, $3);} |
primary ;
primary:
'(' expression ')' {$$ = $2;} |
INT_LITERAL |
IDENTIFIER {if (!symbols.find($1, $$)) appendError(UNDECLARED, $1);} ;
%%
void yyerror(const char* message)
{
appendError(SYNTAX, message);
}
int main(int argc, char *argv[])
{
firstLine();
yyparse();
if (lastLine() == 0)
cout << "Result = " << result << endl;
return 0;
}
The basic problem is that in-rule actions do not get a default type in bison (unlike yacc). So in your action
REDUCE operator reductions ENDREDUCE {$$ = $3;} ';'
there's no %type for $$ so you need to specify it explicitly -- perhaps something like { $<value>$ = $3; }. That's equivalent to what yacc would do here, as it gives in-rule actions the same type as the lhs, even though there's not really anything connecting them.
The bigger issue is that this really makes no sense -- an in-rule action like this does NOT set the value for the symbol being reduced. That can only happen in the end-of-rule action. So this is just copying a value to a temp and then throwing it away, never doing anything with it. The implicit end-rule action just does { $$ = $1; } which makes no sense as statement and REDUCE have different types.
I am learning Flex/Bison and we are currently on the part about semantics, previously have dealt with lexical and syntax errors. I have googled extensively and haven't been able to find a solution to my error. I am having trouble trying to understand why I need to declare '$4' when I thought it to be automatically done.
When I try to makefile I get this error:
flex scanner.l
mv lex.yy.c scanner.c
bison -d -v parser.y
paser.y:114.71-72: error: $4 of 'case' has no declared type
114 | case WHEN INT_LITERAL ARROW statement_ {case_statements.push_back($4);};
Here is the pseudo code I am trying to follow:
statement:
CASE expression IS cases OTHERS ARROW statement_ ENDCASE
{If the attribute of cases, is a number then
return it as the attribute otherwise return the
attribute of the OTHERS clause};
cases:
cases case
{if the attribute of cases is a number then return it as the
attribute otherwise return the attribute of case} |
%empty
{Set the attribute to the sentinel NAN} ;
case:
WHEN INT_LITERAL ARROW statement_
{$-2 contains the value of the expression after CASE.
It must be compared with the attribute of INT_LITERAL.
If they match the attribute of this production
should become the attribute of statement_
If they don't match, the attribute should be set to the
sentinel value NAN} ;
parser.y:
%{
#include <iostream>
#include <string>
#include <vector>
#include <map>
#include <math.h>
using namespace std;
#include "values.h"
#include "listing.h"
#include "symbols.h"
#include <stdlib.h>
#include <stdio.h>
int yylex();
void yyerror(const char* message);
Symbols<int> symbols;
//----------------------------------------------------------------------------------------------
vector<int> case_statements; //<<<<<<<<<<<<Is this wrong?
//---------------------------------------------------------------------------------------------
int result;
double *params;
%}
%define parse.error verbose
%union
{
CharPtr iden;
Operators oper;
int value;
}
%token <iden> IDENTIFIER
%token <value> INT_LITERAL REAL_LITERAL BOOL_LITERAL CASE TRUE FALSE
%token <oper> ADDOP MULOP RELOP OROP NOTOP REMOP EXPOP
%token ANDOP
%token BEGIN_ BOOLEAN END ENDREDUCE FUNCTION INTEGER IS REDUCE RETURNS
%token THEN WHEN ARROW
%token ELSE ENDCASE ENDIF IF OTHERS REAL
%type <value> body statement_ statement reductions expression relation term
factor case cases exponent unary primary
%type <oper> operator
%%
function:
function_header optional_variable body {result = $3;} ;
function_header:
FUNCTION IDENTIFIER optional_parameter RETURNS type ';' |
FUNCTION IDENTIFIER RETURNS type ';' |
error ';' ;
optional_variable:
optional_variable variable |
error ';' |
%empty ;
variable:
IDENTIFIER ':' type IS statement_ ;
parameters:
parameter optional_parameter;
optional_parameter:
optional_parameter ',' parameter |
%empty ;
parameter:
IDENTIFIER ':' type {symbols.insert($1, params[0]);} ;
type:
INTEGER |
REAL |
BOOLEAN ;
body:
BEGIN_ statement_ END ';' {$$ = $2;} ;
statement_:
statement ';' |
error ';' {$$ = 0;} ;
statement:
expression |
REDUCE operator reductions ENDREDUCE {$$ = $3;} |
IF expression THEN statement_ ELSE statement_ ENDIF {
if ($2 == true) {
$$ = $4;
}
else {
$$ = $6;
}
} ; |
CASE expression IS cases OTHERS ARROW statement_ ENDCASE {$$ = $<value>4 == $1 ? $4 : $7;} ;
cases:
cases case {$$ = $<value>1 == $1 ? $1 : $2;} |
%empty {$$ = NAN;} ;
//-----------------------------------------------------------------------------------------------------------
case:
case WHEN INT_LITERAL ARROW statement_ {case_statements.push_back($4);} ; //<<<<<<<<<How do I declare $4?
//-------------------------------------------------------------------------------------------------------------
operator:
ADDOP |
RELOP |
EXPOP |
MULOP ;
reductions:
reductions statement_ {$$ = evaluateReduction($<oper>0, $1, $2);} |
{$$ = $<oper>0 == ADD ? 0 : 1;} %empty ;
expression:
expression OROP relation {$$ = $1 || $3;} |
relation ;
expression:
expression ANDOP relation {$$ = $1 && $3;} |
relation ;
relation:
relation RELOP term {$$ = evaluateRelational($1, $2, $3);} |
term ;
term:
term ADDOP factor {$$ = evaluateArithmetic($1, $2, $3);} |
factor ;
factor:
factor MULOP primary {$$ = evaluateArithmetic($1, $2, $3);} |
factor REMOP exponent {$$ = $1 % $3;} |
exponent ;
exponent:
unary |
unary EXPOP exponent {$$ = pow($1, $3);} ;
unary:
NOTOP primary {$$ = $2;} |
primary;
primary:
'(' expression ')' {$$ = $2;} |
INT_LITERAL |
REAL_LITERAL |
BOOL_LITERAL |
IDENTIFIER {if (!symbols.find($1, $$)) appendError(UNDECLARED, $1);} ;
%%
void yyerror(const char* message)
{
appendError(SYNTAX, message);
}
int main(int argc, char *argv[])
{
params = new double[argc - 1]
for (int i = 1; i < argc; i++)
{
params[i - 1] = atof(argv[i]);
}
firstLine();
yyparse();
if (lastLine() == 0)
cout << "Result = " << result << endl;
return 0;
}
You need to assign a value/type to statement_:
statement_:
statement ';' {$$ = $1;}|
error ';' {$$ = MISMATCH;} ;
I'm trying to pinpoint these error messages, but even when I move code around the error shows up at the same line.
Also why am I still getting "calc.y: In function 'int yyparse()':" and "calc.l: In function 'int yylex()':" even though I have it in my calc.y.
Error Messages
flex calc.l
bison -d calc.y
g++ -o calc calc.tab.c lex.yy.c
calc.y: In function 'int yyparse()':
calc.y:35: error: cannot convert 'double' to 'std::string*' in assignment
calc.tab.c:1490: warning: deprecated conversion from string constant to 'char*'
calc.tab.c:1633: warning: deprecated conversion from string constant to 'char*'
calc.l: In function 'int yylex()':
calc.l:17: error: 'yyerror' was not declared in this scope
calc.l
%{
#include <cstdlib>
#include <string>
#include <string.h>
#include "calc.tab.h"
%}
%%
"print" {return print;}
"exit" {return exit_command;}
[0-9]+ {yylval.num = atof(yytext); return number;}
[_[:alpha:]][_[:alnum:]]* {yylval.index = new std::string(yytext); return identifier; }
[ \t] ;
[\n] {return(CR);}
[-()+*/;=] {return yytext[0];}
. {ECHO; yyerror ("unexpected character");}
%%
int yywrap (void) {return 1;}
calc.y
%{
void yyerror (char *s);
int yylex(void);
#include <stdio.h> /* C declarations used in actions */
#include <stdlib.h>
#include <string>
#include <string.h>
#include <map>
static std::map<std::string, double> vars;
%}
%union {double num; std::string *index;} /* Yacc definitions */
%start line
%token print
%token CR
%token exit_command
%token <num> number
%token <index> identifier
%type <num> line exp term
%type <index> assignment
%%
/* descriptions of expected inputs corresponding actions */
line : assignment CR {;}
| exit_command CR {exit(EXIT_SUCCESS);}
| print exp CR {printf("Printing %f\n", $2);}
| line assignment CR {;}
| line print exp CR {printf("Printing %f\n", $3);}
| line exit_command CR {exit(EXIT_SUCCESS);}
;
assignment : identifier '=' exp {$$ = vars[*$1] = $3; delete $1; }
;
exp : term {$$ = $1;}
| exp '+' term {$$ = $1 + $3;}
| exp '-' term {$$ = $1 - $3;}
| exp '*' term {$$ = $1 * $3;}
| exp '/' term {$$ = $1 / $3;}
| '(' exp ')' {$$ = $2;}
;
term : number {$$ = $1;}
| identifier {$$ = vars[*$1]; delete $1; }
;
%%
int main (void) {
return yyparse();
}
extern int yyparse();
void yyerror (char *s)
{
extern int yylineno; // defined and maintained in lex
extern char *yytext; // defined and maintained in lex
fprintf (stderr, "%s\n", s);
}
Here's one place where the double to std::string* error will be produced:
assignment : identifier '=' exp {$$ = vars[*$1] = $3; delete $1; }
The type of assignment is <index>, which is std::string*. The type of exp is <num>, which is double. So in the action above, $$ is a std::string*, and $3 is a double (as is vars[*$1]).
My best guess is that you didn't intend the semantic type of assignment to be <index>.
As for the fact that yyerror isn't defined in calc.l, that is clearly true. yyerror is defined in calc.y, and the declaration is also in calc.y. The C programs produced by (f)lex and yacc/bison are different programs, and they are compiled independently, so the fact that yyerror is declared in the yacc/bison-generated program does not make it's declaration visible to the (f)lex-generated program. If you need to use it from a (f)lex action, you'll need to declare it in both files.
These warnings:
calc.tab.c:1490: warning: deprecated conversion from string constant to 'char*'
suggest that you should find a newer version of bison. If that is impossible, you can just ignore them; g++ is complaining that the bison template is not C++-clean, because it is assigning a string literal (which is a const char*) to a char* variable without a cast. Newer bison versions avoid the warning.
I am trying to do a simple if condition from an input file.
i will have something like
if(color = black)
No matter what i do i keep getting 1 shift / reduce
I am very new to lex and yacc
Do YACC grammars often have shift-reduce conflicts? and should i not worry about them?
My lex file will return every character in the file correctly so i wont show you the lex file
However, here is my yacc file:
%{
#include <ctype.h>
#include <stdio.h>
%}
|IF LPAREN COLOR EQ BLACK RPAREN {$$ = $1; printf("WONT COMPILE\n");}
;
in the yacc file i tried this but that is where i am getting the shift/ reduce
IF LPAREN COLOR EQ BLACK RPAREN {$$ = $1; printf("If statement\n");}
SOLVED
I originally wrote a long answer about the “dangling else” ambiguity, but then I took a closer look at your grammar. Let’s cut it down to size a bit and demonstrate where the problems are:
%token IF COLOR BLACK
%%
statement
: statement command
| /*nothing*/
;
command
: IF {$$ = $1; printf ("IF\n");}
| ELSE {$$ = $1; printf("ELSE\n");}
| EQ {$$ = $1; printf("EQ\n");}
| THEN {$$ = $1; printf("THEN\n");}
| LPAREN {$$ = $1; printf("LPAREN\n");}
| RPAREN {$$ = $1; printf("RPAREN\n");}
| COLOR EQ BLACK {$$ = $3; printf("color is black\n");}
| IF LPAREN COLOR EQ BLACK RPAREN {$$ = $1; printf("WONT COMPILE\n");}
;
Just how do you expect the statement if(color = black) to be parsed? Notice that the “color = black” can reduce to a command via COLOR EQ BLACK or can be “shifted” onto the stack to become part of the longer parse IF LPAREN COLOR EQ BLACK RPAREN.
That explains the specific warning you’re getting. Now, on to the rest of your grammar:
You don’t want to be writing your grammar so incomplete statements are meaningful. Notice that the single symbol “=” is a complete valid command and therefore a complete valid statement—is that really what you want?
You’re going to want to rewrite this from scratch. Start simple:
%token NUMBER COMMAND IF THEN ELSE COLOR BLACK
%%
statement
: COMMAND NUMBER
| IF cond THEN statement
| /* nothing */
;
cond
: '(' COLOR '=' BLACK ')'
;
Not tested; but this should be enough to get you started. If you need to do something when you encounter a token, you can (for example) replace IF cond THEN COMMAND with if cond then command and add rules like
if : IF { printf("%s\n", "IF"); }
;
then: THEN { printf("%s\n", "THEN"); }
;
Start simple, add slowly, and refactor when rules get too hairy or repetitive. And work through a tutorial before you jump into a large project. The GNU Bison manual has a good tutorial, as does Kernighan & Pike’s The Unix Programming Environment.
I was teaching myself Bison and headed over to wikipedia for the same and copy-pasted the entire code from the example that was put there [ http://en.wikipedia.org/wiki/GNU_Bison ]. It compiled and works perfect. Then, I OOPed it by adding in a bit of C++. Here's is my new Parser.y file:
%{
#include "TypeParser.h"
#include "ParserParam.h"
#include "addition.h"
%}
%define api.pure
%left '+' TOKEN_PLUS
%left '*' TOKEN_MULTIPLY
%left '-' TOKEN_SUBTRACT
%left '/' TOKEN_DIVIDE
%left '^' TOKEN_EXP
%token TOKEN_LPAREN
%token TOKEN_RPAREN
%token TOKEN_PLUS
%token TOKEN_MULTIPLY
%token <value> TOKEN_NUMBER
%type <expression> expr
%%
input:
expr { ((SParserParam*)data)->expression = $1; }
;
expr:
expr TOKEN_PLUS expr { $$ = new Addition($1, $2); }
| expr TOKEN_MULTIPLY expr { $$ = new Multiplication($1, $2); }
| expr TOKEN_SUBTRACT expr { $$ = new Addition($1, $2); }
| expr TOKEN_DIVIDE expr { $$ = new Multiplication($1, $2); }
| expr TOKEN_EXP expr { $$ = new Addition($1, $2); }
| TOKEN_LPAREN expr TOKEN_RPAREN { $$ = $2; }
| TOKEN_NUMBER { $$ = new Value($1); }
;
%%
But then I keep getting the following errors:
Parser.y:33.52-53: $2 of `expr' has no declared type
Parser.y:34.62-63: $2 of `expr' has no declared type
Parser.y:35.56-57: $2 of `expr' has no declared type
Parser.y:36.60-61: $2 of `expr' has no declared type
Parser.y:37.52-53: $2 of `expr' has no declared type
How do I resolve it? I mean, what have I changed that is causing this? I haven't changed anything from the wikipedia code, the %type% declaration is still there [The union has the same members, with type changed from SExpression to Expression.]. All classes i.e. Addition, Expression, Multiplication are defined and declared. I don't think that is what is causing the problem here, but just saying.
And why exactly does it have a problem only with $2. Even $1 is of type expr, then why do I not get any errors for $1?
Any help is appreciated...
In the rule expr TOKEN_PLUS expr $1 is the first expression, $2 is TOKEN_PLUS, and $3 is the second expression. See the bison manual.
So the semantic action needs to change from your { $$ = new Addition($1, $2); } to { $$ = new Addition($1, $3); }.