I'm working through Problems with reentrant Flex and Bison. It compiles and runs just fine on my machine. What I want to do though is make use of C++ STL. Anytime I try to include a CPP header, it says it can't be found. There are only a handful of questions about this on Goog. Does anyone have a working example of this sort of setup, or a solution I might implement?
Any help would be greatly appreciated.
Thanks!
EDIT So for one reason or another, I have to add the include path of any headers in the build settings. Must be due to the custom makefile of this person's example. It's above my pay-grade. Anyway, I can now use STL libraries inside of main.
WHAT I REALLY WANT TO DO IS USE FLEX/BISON WITH CPP, AND IF I TRY TO INCLUDE STL HEADERS ANYWHERE BUT MAIN, I GET ERROR "HEADER NOT FOUND".
I can include C-headers just fine, though.
Here's answer from the author of another answer in the linked topic.
I have adapted that my example to work with C++.
The key points are:
I am using recent Flex / Bison: brew install flex and brew install bison. Not sure if the same will work with default OSX/Xcode's flex/bison.
Generated flex/bison files should have C++ extensions (lexer.[hpp|mm], parser.[hpp|mm]) for Xcode to pick up the C++ code.
There is a Xcode's Build Phase that runs Make.
All the relevant files follow below but I recommend you to check out the example project.
main.mm's code is
#include "parser.hpp"
#include "lexer.hpp"
extern YY_BUFFER_STATE yy_scan_string(const char * str);
extern void yy_delete_buffer(YY_BUFFER_STATE buffer);
ParserConsumer *parserConsumer = [ParserConsumer new];
char input[] = "RAINBOW UNICORN 1234 UNICORN";
YY_BUFFER_STATE state = yy_scan_string(input);
yyparse(parserConsumer);
yy_delete_buffer(state);
Lexer.lm:
%{
#include "ParserConsumer.h"
#include "parser.hpp"
#include <iostream>
#include <cstdio>
int yylex(void);
void yyerror(id <ParserConsumer> consumer, const char *msg);
%}
%option header-file = "./Parser/Generated Code/lexer.hpp"
%option outfile = "./Parser/Generated Code/lexer.mm"
%option noyywrap
NUMBER [0-9]+
STRING [A-Z]+
SPACE \x20
%%
{NUMBER} {
yylval.numericValue = (int)strtoul(yytext, NULL, 10);
std::cout << "Lexer says: Hello from C++\n";
printf("[Lexer, number] %s\n", yytext);
return Token_Number;
}
{STRING} {
yylval.stringValue = strdup(yytext);
printf("[Lexer, string] %s\n", yytext);
return Token_String;
}
{SPACE} {
// Do nothing
}
<<EOF>> {
printf("<<EOF>>\n");
return 0;
}
%%
void yyerror (id <ParserConsumer> consumer, const char *msg) {
printf("%s\n", msg);
abort();
}
Parser.ym:
%{
#include <iostream>
#include <cstdio>
#include "ParserConsumer.h"
#include "parser.hpp"
#include "lexer.hpp"
int yylex();
void yyerror(id <ParserConsumer> consumer, const char *msg);
%}
%output "Parser/Generated Code/parser.mm"
%defines "Parser/Generated Code/parser.hpp"
//%define api.pure full
%define parse.error verbose
%parse-param { id <ParserConsumer> consumer }
%union {
char *stringValue;
int numericValue;
}
%token <stringValue> Token_String
%token <numericValue> Token_Number
%%
/* http://www.tldp.org/HOWTO/Lex-YACC-HOWTO-6.html 6.2 Recursion: 'right is wrong' */
tokens: /* empty */
| tokens token
token:
Token_String {
std::cout << "Parser says: Hello from C++\n";
printf("[Parser, string] %s\n", $1);
[consumer parserDidParseString:$1];
free($1);
}
| Token_Number {
printf("[Parser, number]\n");
[consumer parserDidParseNumber:$1];
}
%%
Makefile:
generate-parser: clean flex bison
clean:
rm -rf './Parser/Generated Code'
mkdir -p './Parser/Generated Code'
flex:
# brew install flex
/usr/local/bin/flex ./Parser/Lexer.lm
bison:
# brew install bison
/usr/local/bin/bison -d ./Parser/Parser.ym
Related
I use chdir() to switch the directory, and then use execvp() to execute "java Main". I'm sure there is Main.class, but something went wrong. I want to know why.
#include <cstdio>
#include <unistd.h>
using namespace std;
int main(){
char buf[80];
getcwd(buf,sizeof(buf));
printf("current working directory: %s\n", buf);
chdir("/home/keane/Judge/temp");
getcwd(buf,sizeof(buf));
printf("current working directory: %s\n", buf);
char *array[3];
array[0] = "java";
array[1] = "Main";
array[2] = NULL;
execvp("java", array);
return 0;
}
the error is could not find the main class , and I can run java Main in that directory.
What drives me crazy is that I can't use system("java Main"), and the error is that Error: Could not find or load main class Main, and it's just like this on my computer
update:
#include <unistd.h>
#include <cstdlib>
int main(){
chdir("/home/keane/Judge/temp");
system("pwd");
system("ls");
system("java Main");
return 0;
}
the output on console is:
/home/keane/Judge/temp
1.out 3.out 5.out Main.class stdout_spj.txt
2.out 4.out ce.txt Main.java
Error: Could not find or load the main class Main
my final solution is to reboot the computer and add -cp . to the java command.
althought I don't why is necessary.
thanks everyone!
This works as intended on my system, maybe you need to add -cp . to your java call.
EDIT: to elaborate: -cp (for classpath) tells java where to look for user provided .class files. This does not necessarily include the current working directory by default.
The execution of execvp() is non-blocking and takes ownership of the caller, that means that when it starts if the program ends too quickly you will never be able to see the result, to solve this I use fork(). The wait is just to avoid using sleep as I used at the begining. Its all in c.
#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <sys/wait.h>
int main(int argc, char** argv){
char buf[80];
getcwd(buf,sizeof(buf));
printf("current working directory: %s\n", buf);
chdir("/home/");
getcwd(buf,sizeof(buf));
printf("current working directory: %s\n", buf);
char *array[3] = {"java", "Main", NULL};
if(fork() == 0) {
if(execvp("java", array) < 0) {
fprintf(stderr, "Error spawning command: %s\n", strerror(errno));
}
} else {
printf("Command spawned\n");
wait(NULL); // Wait to the forked process to end (avoid using sleep)
}
return 0;
}
I have written Flex stuff in a previous class but none of my previously working code is solving the issue I am having.
I have searched around StackOverflow for a solution but none of them have solved it.
I have:
Checked I have no errant spaces in the %{ ... %} area
Tried using #include <'iostream>
Tried %option noyywrap
Here is my code (I removed all the tokens and such because there is a lot of them):
%{
...
int numLines = 0;
void printTokenInfo(char* tokenType, char* lexeme);
void handleComments(char* text);
%}
WSPACE [ \t\r]+
NEWLINE \n
DIGIT [0-9]
LETTER [a-zA-Z]
IDENT ({LETTER}|_)({LETTER}|{DIGIT}|_)*
INTCONST {DIGIT}+
CHARCONST "'"{LETTER}+"'"
%%
...
%%
// User-written code goes here
void printTokenInfo(char* tokenType, char* lexeme)
{
printf("A");
printf("TOKEN: %s LEXEME: %s\n", tokenType, lexeme);
}
void handleComments(char* text)
{
printf("%s\n", text);
}
int yywrap() { return 1; }
int main()
{
do {
yylex();
} while (!feof(yyin));
return 0;
}
Here is how I am compiling and running it:
flex FILENAME.l
g++ lex.yy.c -o lexer
lexer < INPUT.txt
And the instructor provided us with input files but none of them have worked. They all fail with 'premature EOF' or 'bad character'
Any ideas?
Well, I think I finally discovered the answer... Try running it with the complete path rather than just the name of your compiled lexer. I discovered the 'actual' path by running it under gdb (Which admittedly should have been my first instinct).
gdb lexer
(gdb) run < INPUT.txt
Originally, I was trying to run it with:
lexer < INPUT.txt
But I discovered by running it with gdb that this worked:
/nethome/users/mjc7w6/Classes/lexer < INPUT.txt
Edit: Someone chimed in on my Facebook with a further improvement. If the above solution fixes it for you, you might need to edit your ~/.bashrc with the following:
export PATH=/nethome/users/mjc7w6/Classes:$PATH
Or however you find that path to be set-up.
I have a problem in compiling my code.
It works when main() is in the same file as yacc parser but its not working when I put main() in another file.
This is my Flex file: (lex1.ll)
%{
#include <stdio.h>
#include "yac1.tab.hh"
extern "C"
{
int yylex(void);
}
int line_num = 1;
%}
alpha [A-Za-z]
digit [0-9]
%%
"DELETE ALL" return DELALL;
[ \t] ;
INSERT return INSERT;
DELETE return DELETE;
FIND return FIND;
[0-9]+\.[0-9]+ { yylval.fval = atof(yytext); return FLOAT; }
[0-9]+ { yylval.ival = atoi(yytext); return INT; }
[a-zA-Z0-9_]+ { yylval.sval = strdup(yytext);return STRING; }
\n { ++line_num; return ENDL; }
. ;
%%
This is my Bison file: (yac1.yy)
%{
#include <stdio.h>
#include <stdlib.h>
int intval;
void yyerror(const char *s)
{
fprintf(stderr, "error: %s\n", s);
}
extern "C"
{
int yyparse(void);
int yylex(void);
int yywrap()
{
return 1;
}
}
%}
%token INSERT DELETE DELALL FIND ENDL
%union {
int ival;
float fval;
char *sval;
}
%token <ival> INT
%token <fval> FLOAT
%token <sval> STRING
%%
S:T {printf("INPUT ACCEPTED....\n");exit(0);};
T: INSERT val {printf("hey insert FOUND\n");}
| DELETE val
| DELALL ENDL
| FIND val
;
val : INT ENDL {printf("hey %d\n",$1);intval=$1; }
|
FLOAT ENDL
|
STRING ENDL {printf("hey %s\n",$1);}
;
%%
/*
It works if I uncomment this block of code
int main()
{
while(1){
printf("Enter the string");
yyparse();
}
}
*/
This is my main program: (testlex.cc)
#include <stdio.h>
#include "lexheader.h"
#include "yac1.tab.hh"
extern int intval;
main()
{
/*
char * line = "INSERT 54\n";
YY_BUFFER_STATE bp = yy_scan_string( line );
yy_switch_to_buffer(bp);
yyparse();
yy_delete_buffer(bp);
printf ("hello %d",intval);
*/
printf("Enter the query:");
//while(1)
printf ("%d\n",yyparse());
}
And this is my Makefile
parser: lex1.ll yac1.yy testlex.cc
bison -d yac1.yy
flex --header-file="lexheader.h" lex1.ll
g++ -o parser yac1.tab.cc lex.yy.c testlex.cc -lfl
clean:
rm -rf *.o parser
When I compile I get this error.
bison -d yac1.yy
flex --header-file="lexheader.h" lex1.ll
g++ -o parser yac1.tab.cc lex.yy.c testlex.cc -lfl
testlex.cc: In function ‘int main()’:
testlex.cc:21:24: error: ‘yyparse’ was not declared in this scope
make: *** [parser] Error 1
PS: It is necessary for me to compile with g++.With gcc I have a working code.
Any help in this regard is highly appreciated.
Thanks.
Did you read the documentation of GNU Bison ? It has a chapter about C++ parsers with a complete example (quite similar to yours).
You could explicitly declare yyparse as suggested by this answer, but making a real C++ parser is perhaps better.
Your Makefile is not very good. You could have something like
LEX= flex
YACC= bison
LIBES= -lfl
CXXFLAGS= -Wall
parser: lex1.o yac1.tab.o lex.yy.o testlex.o
$(LINKER.cc) -o $# $^ $(LIBES)
lex1.cc: lex1.ll
$(LEX) --header-file="lexheader.h" $< -o $#
yac1.tag.cc: yac1.yy
$(YACC) -d $<
Take also time to read the documentation of GNU make. You might want to use remake as remake -x to debug your Makefile.
You need to declare the yyparse function in the testlex.cc file:
int yyparse();
This is what is known as a function prototype, and tells the compiler that the function exists and can be called.
After looking a little closer at your source I now know the reason why the existing prototype didn't work: It's because you declared it as extern "C" but compiled the file as a C++ source. The extern "C" told the compiler that the yyparse function was an old C style function but then you continued to compile the source with a C++ compiler. This caused a name mismatch.
i would like to use the code generated by lex in another code that i have , but all the examples that i have seen is embedding the main function inside the lex file not the opposite.
is it possible to use(include) the c generated file from lex into other code that to have something like this (not necessarily the same) ?
#include<something>
int main(){
Lexer l = Lexer("some string or input file");
while (l.has_next()){
Token * token = l.get_next_token();
//somecode
}
//where token is just a simple object to hold the token type and lexeme
return 0;
}
This is what I would start with:
Note: this is an example of using a C interface
To use the C++ interface add %option c++ See below
Test.lex
IdentPart1 [A-Za-z_]
Identifier {IdentPart1}[A-Za-z_0-9]*
WHITESPACE [ \t\r\n]
%option noyywrap
%%
{Identifier} {return 257;}
{WHITESPACE} {/* Ignore */}
. {return 258;}
%%
// This is the bit you want.
// It is best just to put this at the bottom of the lex file
// By default functions are extern. So you can create a header file with
// these as extern then included that header file in your code (See Lexer.h)
void* setUpBuffer(char const* text)
{
YY_BUFFER_STATE buffer = yy_scan_string(text);
yy_switch_to_buffer(buffer);
return buffer;
}
void tearDownBuffer(void* buffer)
{
yy_delete_buffer((YY_BUFFER_STATE)buffer);
}
Lexer.h
#ifndef LOKI_A_LEXER_H
#define LOKI_A_LEXER_H
#include <string>
extern int yylex();
extern char* yytext;
extern int yyleng;
// Here is the interface to the lexer you set up above
extern void* setUpBuffer(char const* text);
extern void tearDownBuffer(void* buffer);
class Lexer
{
std::string token;
std::string text;
void* buffer;
public:
Lexer(std::string const& t)
: text(t)
{
// Use the interface to set up the buffer
buffer = setUpBuffer(text.c_str());
}
~Lexer()
{
// Tear down your interface
tearDownBuffer(buffer);
}
// Don't use RAW pointers
// This is only a quick and dirty example.
bool nextToken()
{
int val = yylex();
if (val != 0)
{
token = std::string(yytext, yyleng);
}
return val;
}
std::string const& theToken() const {return token;}
};
#endif
main.cpp
#include "Lexer.h"
#include <iostream>
int main()
{
Lexer l("some string or input file");
// Did not like your hasToken() interface.
// Just call nextToken() until it fails.
while (l.nextToken())
{
std::cout << l.theToken() << "\n";
delete token;
}
//where token is just a simple object to hold the token type and lexeme
return 0;
}
Build
> flext test.lex
> g++ main.cpp lex.yy.c
> ./a.out
some
string
or
input
file
>
Alternatively you can use the C++ interface to flex (its experimental)
test.lext
%option c++
IdentPart1 [A-Za-z_]
Identifier {IdentPart1}[A-Za-z_0-9]*
WHITESPACE [ \t\r\n]
%%
{Identifier} {return 257;}
{WHITESPACE} {/* Ignore */}
. {return 258;}
%%
// Note this needs to be here
// If you define no yywrap() in the options it gets added to the header file
// which leads to multiple definitions if you are not careful.
int yyFlexLexer::yywrap() { return 1;}
main.cpp
#include "MyLexer.h"
#include <iostream>
#include <sstream>
int main()
{
std::istringstream data("some string or input file");
yyFlexLexer l(&data, &std::cout);
while (l.yylex())
{
std::cout << std::string(l.YYText(), l.YYLeng()) << "\n";
}
//where token is just a simple object to hold the token type and lexeme
return 0;
}
build
> flex --header-file=MyLexer.h test.lex
> g++ main.cpp lex.yy.cc
> ./a.out
some
string
or
input
file
>
Sure. I'm not sure about the generated class; we use the C generated
parsers, and call them from C++. Or you can insert any sort of wrapper
code you want in the lex file, and call anything there from outside of
the generated file.
The keywords are %option reentrant or %option c++.
As an example here's the ncr2a scanner:
/** ncr2a_lex.l: Replace all NCRs by corresponding printable ASCII characters. */
%%
&#(1([01][0-9]|2[0-6])|3[2-9]|[4-9][0-9]); { /* accept 32..126 */
/** `+2` skips '&#', `atoi()` ignores ';' at the end */
fputc(atoi(yytext + 2), yyout); /* non-recursive version */
}
The scanner code can be left unchanged.
Here the program that uses it:
/** ncr2a.c */
#include "ncr2a_lex.h"
typedef struct {
int i,j; /** put here whatever you need to keep extra state */
} State;
int main () {
yyscan_t scanner;
State my_custom_data = {0,0};
yylex_init(&scanner);
yyset_extra(&my_custom_data, scanner);
yylex(scanner);
yylex_destroy(scanner);
return 0;
}
To build ncr2a executable:
flex -R -oncr2a_lex.c --header-file=ncr2a_lex.h ncr2a_lex.l
cc -c -o ncr2a_lex.o ncr2a_lex.c
cc -o ncr2a ncr2a_lex.o ncr2a.c -lfl
Example
$ echo 'three colons :::' | ./ncr2a
three colons :::
This example uses stdin/stdout as input/output and it calls yylex() once.
To read from a file:
yyin = fopen("input.txt", "r" );
#Loki Astari's answer shows how to read from a string (buffer = yy_scan_string(text, scanner); yy_switch_to_buffer(buffer, scanner))
.
To call yylex() once for each token add return inside rule definitions that yield full token in the *.l file.
I'm getting started with lua. I have a very simple project but I can't get it to run. I keep getting the same error: fatal error LNK1107: file broken or damaged: cannot read at 0x2C3C file: lua.h line:1
help would be greatly appreciated
thanks in advance
c++ code
#include <iostream>
extern "C"
{
#include "lua.h"
#include "lualib.h"
#include "lauxlib.h"
#include "luaconf.h"
}
int main()
{
int s=0;
lua_State *L = lua_open();
// load the libs
luaL_openlibs(L);
//run a Lua scrip here
luaL_dofile(L,"foo.lua");
printf("\nI am done with Lua in C++.\n");
lua_close(L);
return 0;
}
lua code
io.write("Please enter your name: ")
name = io.read() -- read input from user
print ("Hi " .. name .. ", enjoy hacking with Lua");
It seems you're trying to link a header file into your program.