I am writing a compiler with Yacc and having trouble figuring out how to write productions to match a function. In my language, functions are defined like this:
function foo(a, b, c);
I created lex patterns to match the word function to FUNC, and any C style name to NAME.
Ideally, I would want something like this:
FUNC NAME OBRACKET NAME (COMMA NAME)* CBRACKET
Which would allow some unknown number of pairs of COMMA NAME in between NAME and CBRACKET.
Additionally, how would I know how many it found?
You might try something like this:
funcdecl: FUNC NAME OBRACKET arglist CBRACKET SEMI
;
arglist: nonemptyarglist
|
;
nonemptyarglist: nonemptyarglist COMMA NAME
| NAME
;
I'd suggest using the grammar to build a syntax tree for your language and then doing whatever you need to the syntax tree after parsing has finished. Bison and yacc have features that make this rather simple; look up %union and %type in the info page.
After a little experimentation, I found this to work quite well:
int argCount;
int args[128];
arglist: nonemptyarglist
|
;
nonemptyarglist: nonemptyarglist COMMA singleArgList
| singleArgList
{
};
singleArgList:
REGISTER
{
args[argCount++] = $1;
};
Related
I am trying to write Grammar for java specification
for example:-
COMPILATION_UNIT: PACKAGE_DEC? IMPORT_DECS? TYPE_DECS?
but it doesn't work
I have the following error:
invalid character: `?'
for each question mark I use in my file.y
I know that Bison has special characters and it should handle it
Please help
Bison does not allow a ? meaning that the prior token is optional, you have to write out the grammar with the optional elements:
package_decl_opt: %empty
| SOME_TOKEN
;
package: package)_dec_opt TOKEN_PACKAGE TOKEN_IDENTIFIER
;
would allow both of the following:
SOME_TOKEN TOKEN_PACKAGE TOKEN_IDENTIFIER
TOKEN_PACKAGE TOKEN_IDENTIFIER
As you have seen, bison does not implement the ? regular expression optionality operator. Nor does it implement + or * repetition operators. That's because the right-hand sides of productions in contex-free grammars are not regular expressions.
Yacc/bison context-free grammars do allow the | alternation operator, but as an abbreviation:
a : b | c
Is exactly the same as writing
a : b
a : c
and semantic actions only apply to the alternative in which they are specified, so that
a : b | c { /* C action; */ }
Is equivalent to:
a : b { /* Implicit default action*/ }
a : c { /* C action; */ }
It is tempting to create X_opt non-terminals to capture the semantics of X?:
X_opt: X | %empty { $$ = default_value; }
In many simple cases that will work fine, but there are also many grammars in which that introduces an unnecessary shift-reduce conflict. Consider, for example:
label: IDENT ':'
label_opt: label | %empty
statement: label_opt expr
Since expr can start with an identifier, there is no way to know if an IDENT token starts a label or if it starts an expr following an empty label_opt. But LR(1) requires that the empty label_opt be reduced before the IDENT is consumed. So the above grammar is LR(2) and cannot be correctly parsed by an LR(1) parser.
That problem does not occur without the use of the label_opt shortcut:
label: IDENT ':'
statement: label expr
| expr
Since the parser now does not have decide between label and expr before the ':' is encountered.
So I have a grammar that includes the empty string. The grammar is something like this:
S->ε
S->expression ;; S
I'm getting the error "No more states to discard" when I run my parser so I believe I'm not representing the empty string correctly. So how would I go about representing it, specifically in the lexer .mll file?
I know I need to make a rule for it so I think I have that down. This is what I think it should look like for the parser .mly file, excluding the stuff for expression.
s:
| EMPTY_STRING { [] }
| expression SEMICOLON s { $1::$3 }
You're thinking of epsilon as a token, but it's not a token. It's a 0-length sequence of tokens. Since there are no tokens there, it's not something your scanner needs to know about. Just the parser needs to know about it.
Here's a grammar something like what I think you want:
%token X
%token SEMICOLON
%token EOF
%start main
%type <char list> main
%%
main :
s EOF { $1 }
s :
| epsilon { $1 }
| X SEMICOLON s { 'x' :: $3 }
epsilon :
{ [] }
Note that epsilon is a non-terminal (not a token). Its definition is an empty sequence of symbols.
I would like to define an expression in C++ with macros and I am having quite a bit of trouble.
The expression is :
MATCH string WITH other_string
where string and other_string do not require " "
For example: MATCH r1 WITH string1 is the result i desire.
The purpose of this macro would be to check if r1 string matches with r2.
(I already have the code for the matching)
UPDATE
I would like to call MATCH hello WITH hi
in my main function
int main(){
MATCH hello WITH hi
}
and call my function from this macro to compare them. **Both hello and hi are unquoted arguments and must be treated as variable names
It is always dubious to use macros to make your code look like a different language. It is probably better to consider using a separate parser for your "meta-language" that generates the C++ code for you.
In this case, since C++ syntax requires some way to indicate the end of a statement (close braces or semi-colon) you are in kind of a jam.
Consider your example:
int main () { MATCH hello WITH hi }
Since hi is the last token before the end of main, there is no chance to fix-up the syntax to match C++ requirements.
You can't do what you want, so you have to do something different
If you really intend to embed this syntax into your C++ code, you need sentinel tokens to allow you to fix-up the syntax. My proposed syntax is:
int main () {
BEGIN_MATCHING
MATCH hello WITH hi
MATCH hello WITH hi
END_MATCHING
};
If this syntax is acceptable, then you can use the following macros.
#define BEGIN_MATCHING ((void)0
#define MATCH ); my_function(
#define WITH ,
#define END_MATCHING );
This will cause the code in the proposed syntax example to expand to:
int main () {
((void)0
); my_function( hello , hi
); my_function( hello , hi
);
}
Live Demo
Simply stringify your arguments with #, something like:
#define MATCH_WITH(str1, str2) MATCH #str1 WITH #str2
That way:
MATCH_WITH(testing, testing)
becomes:
MATCH "testing" WITH "testing"
I have been tasked with a project that involves me taking a Grammar (in BNF form) and creating a lexical scanner (using lex) and a parser (using bison). I've never worked with any of these programs and I think a good reference would be to see how these items are created from a grammar. I am looking for a grammar and it's associated .l and .ypp files, preferably in C++. I've been able to find sample files or sample grammars, but not both of them. I've spent some time searching and I could not find anything. I figure I'd post here in hopes that someone has something for me, but I will continue searching in the meantime.
I am currently reading Tom Niemann's
http://epaperpress.com/lexandyacc/download/LexAndYaccTutorial.pdf which seems to be pretty well written and understandable.
Thanks
Edit: I am still searching, I am starting to think that what I am looking for does not exist. Google usually never fails me!
Edit 2: Maybe if I provide some of the grammar, you folks could show me what the appropriate .l and .ypp files would look like. This is just a snippet of the grammar, I just need a little 'taste' of how this works and I think I can take it from there.
Grammar:
Program ::= Compound
Statements ::= Compound | Assignment | ...
Assignment ::= Var ASSIGN Expression
Expression ::= Var | Operator Expression Expression | Number
Compound := START Statements END
Number ::= NUMBER
Descriptions:
Assignment is the equal sign ":="
Var is an identifier that begins with a lower case letter and is followed by lower case letters or digits
START is the "start" keyword
END is the "end keyword
Operator is "+", "-", "*", "/"
Number is decimal digits which could potentially be negative (minus sign in front)
Most of this is fairly straightforward. One part, however, is decidedly problematic. You've defined a number to (potentially) include a leading -, and that's a problem.
The problem is pretty simple. Given an input like 321-123, it's essentially impossible for the lexer (which won't normally keep track of current state) to guess at whether that's supposed to be two tokens (321 and -123 or three 321, -, 123). In this case, the - is almost certainly intended to be separate from the 123, but if the input were 321 + -123 you'd apparently want -123 as a single token instead.
To deal with that, you probably want to change your grammar so the leading - isn't part of the number. Instead, you always want to treat the - as an operator, and the number itself is composed solely of the digits. Then it's up to the parser to sort out expressions where the - is unary vs. binary.
Taking that into account, the lexer file would look something like this:
%{
#include "y.tab.h"
%}
%option noyywrap case-insensitive
%%
:= { return ASSIGN; }
start { return START; }
end { return END; }
[+/*] { return OPERATOR; }
- { return MINUS; }
[0-9]+ { return NUMBER; }
[a-z][a-z0-9]* { return VAR; }
[ \r\n] { ; }
%%
void yyerror(char const *s) { fputs(s, stderr); }
The matching yacc file would look something like this:
%token ASSIGN START END OPERATOR MINUS NUMBER VAR
%left '-' '+' '*' '/'
%%
program : compound
statement : compound
| assignment
;
assignment : VAR ASSIGN expression
;
statements :
| statements statement
;
expression : VAR
| expression OPERATOR expression
| expression MINUS expression
| value
;
value: NUMBER
| MINUS NUMBER
;
compound : START statements END
%%
int main() {
yyparse();
return 0;
}
Note: I've tested these only extremely minimally--enough to verify input I believe is grammatical, such as: start a:=1 b:=2 end and start a:=1+3*3 b:=a+4 c:=b*3 end is accepted (no error message printed out) and input I believe is un-grammatical, such as: 9:=13 and a=13 do both print out syntax error messages. Since this doesn't attempt to do any more with the expressions than recognize those which are or are not grammatical, that's about the best we can do though.
Say I have this C program.
#include <stdio.h>
int main(void)
{
int monday = 1;
int tuesday = 2;
if(monday == tuesday) { fprintf("I should quit my day job"); }
return 1;
}
What would the tokens be?
What does bison provide me, as a programmer? Certianly, bison does not generate machine code with just a parser grammar? So how do I interface with bison? I am not expecting a full answer here, just a pointer to good websites and books.
Bison implements a generalized LR parser. See http://www.gnu.org/software/bison/manual/bison.html for fairly extensive documentation, with examples. You don't get back a parse tree per se; instead, you write "actions" that activate on each reduction. Of course, if your actions simply build a parse tree, that will do the trick, if you want to obtain a parse tree. Modern bison also has a lot of extra tweaking you can insert.
The tokens, line by line:
#include <stdio.h>
The above line is not a C statement, but used by the C preprocessor.
int main(void)
Five tokens: The keyword int, the identifier main, the symbol (, the keyword void and the symbol ).
{
One token: The symbol {.
int monday = 1;
Five tokens: The keyword int, the identifier monday, the symbol =, the integer number 1, and the symbol ;.
... And so on
It should also be noted that = and == are two separate tokens, and that the string is one single token.