When taking in a expression like (10+5*15) and following orders of operations.
How would one best solve a problem like this? What kind of data structure is best?
Thanks.
I'd go with Dijkstra's Shunting yard algorithm to create the AST.
Try parsing the expression using recursive descent. This would give you a parse tree respecting order of operations.
The usual data structure for this task is a stack. When you're doing things like compiling, creating an abstract syntax tree is useful, but for simple evaluation it's usually overkill.
Think about it for a second - what is an operator? Pretty much every operator (+, -, *, /) are all binary operators. Parenthesis are depth constructors; you move one level deeper with parenthesis.
In fact, constructing the tree of data you need to solve this problem is going to be your biggest hurdle.
It's in Java, but this seems to convert from infix to postfix, and then evaluates using a stack-based approach. It puts numbers onto the stack, reaches operators, and then pops the two numbers from the stack to evaluate them with the operator (x + / -).
http://enel.ucalgary.ca/People/Norman/enel315_winter1999/lab_solutions/lab5sol/exF/Calculator.java
The conversion is as follows:
Scan the Infix string from left to
right.
Initialise an empty stack.
If the scannned character is an operand, add it to the Postfix string. If the scanned character is an operator and if the stack is empty
Push the character to stack.
If the scanned character is an Operand and the stack is not empty, compare the precedence of the character with the element on top of the stack (topStack). If topStack has higher precedence over the scanned character Pop the stack else Push the scanned character to stack. Repeat this step as long as stack is not empty and topStack has precedence over the character.
Repeat this step till all the characters are scanned. (After all characters are scanned, we have to add any character that the stack may have to the Postfix string.)
If stack is not empty add topStack to
Postfix string and Pop the stack.
Repeat this step as long as stack is
not empty.
Return the Postfix string.
Evaluate the Postfix string.
If you need to simply compute the result of the expression that is available as a string then I'd go with no data structure at all and just functions like:
//
// expression ::= addendum [ { "-" | "+" } addendum ]
// addendum ::= factor [ { "*" | "/" } factor ]
// factor ::= { number | sub-expression | "-" factor }
// sub-expression ::= "(" expression ")"
// number ::= digit [ digit ]
// digit ::= { "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9" }
//
int calcExpression(const char *& p);
int calcDigit(const char *& p);
int calcNumber(const char *& p);
int calcFactor(const char *& p);
int calcAddendum(const char *& p);
where each function just accepts a const char * by reference that reads from it (incrementing the pointer) and returning as value the numeric value of the result, throwing instead an exception in case of problems.
This approach doesn't need any data structure because uses the C++ stack for intermediate results. As an example...
int calcDigit(const char *& p)
{
if (*p >= '0' && *p <= '9')
return *p++ - '0';
throw std::runtime_error("Digit expected");
}
int calcNumber(const char *& p)
{
int acc = calcDigit(p);
while (*p >= '0' && *p <= '9')
acc = acc * 10 + calcDigit(p);
return acc;
}
If you need instead to write a compiler that transforms a string (for example including variables or function calls) into code or bytecode then probably the best solution is to start either using a generic n-way tree or a tree with specific structures for the different AST node types.
Related
I'm doing an assignment for school right now that focuses on building a tokenizer and parser for simple sets of instructions. This instruction set uses EBNF grammar which we have to output to the user. As a simple example, imagine this instruction set code looks like this:
set 0, 2 * (4 + 20)
halt
And the EBNF grammar is given like this:
<Program> -> <Statement> { <Statement> }
<Statement> -> <Set> | 'halt'
<Set> -> set ('write'|<Expr>), ('read'|<Expr>)
<Expr> -> <Term> {(+|-) <Term>}
<Term> -> <Factor> {(*|/|%) <Factor>}
<Factor> -> <Number> | 'D['<Expr>']' | '('<Expr>')'
<Number> -> 0 | (1...9){0...9}
In this case, <...> is what I output to the user, {...} means choose 0 or more, (...) means choose 1 or more and '...' is a string literal.
So in this case, when running the program, the correct output should look like this:
Program
Statement
Set // set
Expr // 0
Term // 0 is a term
Factor // 0 is a factor
Number // 0 is finally a number
Expr // go to the next string, "2*(4+2)"
Term // start with the number 2
Factor // 2 is a factor
Number // 2 is a number
Factor // after 2, we have a * so after that is a factor
Expr // the factor leads to a (...) which leads to an expr
Term // the expr leads to a term 4
Factor // 4 is a factor
Number // 4 is returned as a number
Term // we have a + so the next string (20) is a term
Factor // 20 is a factor
Number // return 20 as a number
Statement // halt is a statement, end
I've already built the tokenizer portion of the assignment, so running the above instruction set produces a std::vector<std::string> that looks like this (where each new string is separated by a new line):
set
0
2*(4+20)
halt
Now the parser is where I am stuck. I first push each string into a std::queue, examine the string, parse it, and then pop it from the queue. I already have it so that a set statement pushes the string "Set" in another std::vector<std::string> so I can print it out later.
I also have a function, parse_expression(std::string& str) to parse my expressions. The real trouble I have is with how I could correctly parse the 2 * (4 + 20) part of it.
My teacher told me to go character by character through the string, check whether it is a number (which I know how to do), or if it matches a '+', '-', '/', '*', '%' character, or if the next character is a 'D' then I should parse another expression.
I'm sort of confused on how I should attempt this though. If I go character by character, I can get the first number, go until I hit a non number character, and then just push out Term followed by Factor followed by Number. But how can I then go back to that original character and keep pushing onwards towards the next ones. Almost like, how do I go back to the original level I was at so that I can correctly determine whether or not I need to create another Expr, or whether or not I should just be doing another term.
I realize this is a long, confusing question, but I would appreciate any push in the right direction, whether it be something glaringly obvious that I'm doing wrong, or whether it be how a parser actually works.
I've tried to make a loop to get the first number after y=
but sometimes the value of term1coffs is: 65(
and sometimes the value is: 65 without the bracket ?!
std::string fx = cwin.get_string("Enter a polynomial formula that has a form like this: y=1(x^3)+2(x^2)+0.5(x^1)+1");
std::string j;
//j is a condition to end for loop
int n=2;
std::string term1coffs;
//to get the number before the bracket y= ?? ( X^.....
for (j=fx.substr(n,1);j=="(";n+0)
{
n=n+1;
}
term1coffs=fx.substr(2,n);
double term1coff= atof(term1coffs.c_str());
Traditional parsers do something like this (written in pseudocode):
do
{
get a character
if (character is operator)
{
store character as operator
convert operand to double
if conversion succeeded
store operand value
else
store operand as variable name
operand = "";
}
else
{
add current character to operand
}
} while(there is more to do)
If you want to later use the parsed results to perform calculations, you probably want to produce tree-structure, based on order of precedence, taking into account parenthesis. The basic principle for this is the Shunting-Yard Algorithm
This is a piece of code I found in my textbook for using recursion to evaluate prefix expressions. I'm having trouble understanding this code and the process in which it goes through.
char *a; int i;
int eval()
{ int x = 0;
while (a[i] == ' ') i++;
if (a[i] == '+')
{ i++; return eval() + eval(); }
if (a[i] == '*')
{ i++; return eval() * eval(); }
while ((a[i] >= '0') && (a[i] <= '9'))
x = 10*x + (a[i++] - '0');
return x;
}
I guess I'm confused primarily with the return statements and how it eventually leads to solving a prefix expression. Thanks in advance!
The best way to understand recursive examples is to work through an example :
char* a = "+11 4"
first off, i is initialized to 0 because there is no default initializer. i is also global, so updates to it will affect all calls of eval().
i = 0, a[i] = '+'
there are no leading spaces, so the first while loop condition fails. The first if statement succeeds, i is incremented to 1 and eval() + eval() is executed. We'll evaluate these one at a time, and then come back after we have our results.
i = 1, a[1] = '1'
Again, no leading spaces, so the first while loop fails. The first and second if statements fail. In the last while loop, '1' is between 0 and 9(based on ascii value), so x becomes 0 + a[1] - '0', or 0 + 1 = 1. Important here is that i is incremented after a[i] is read, then i is incremented. The next iteration of the while loop adds to x. Here x = 10 * 1 + a[2] - '0', or 10 + 1 = 11. With the correct value of x, we can exit eval() and return the result of the first operand, again here 11.
i = 2, a[2] = '4'
As in the previous step, the only statement executed in this call of eval() is the last while loop. x = 0 + a[2] - '0', or 0 + 4 = 4. So we return 4.
At this point the control flow returns back to the original call to eval(), and now we have both values for the operands. We simply perform the addition to get 11 + 4 = 15, then return the result.
Every time eval() is called, it computes the value of the immediate next expression starting at position i, and returns that value.
Within eval:
The first while loop is just to ignore all the spaces.
Then there are 3 cases:
(a) Evaluate expressions starting with a + (i.e. An expression of the form A+B which is "+ A B" in prefix
(b) Evaluate expressions starting with a * (i.e. A*B = "* A B")
(c) Evaluate integer values (i.e. Any consecutive sequence of digits)
The while loop at the end takes care of case (c).
The code for case (a) is similar to that for case (b). Think about case (a):
If we encounter a + sign, it means we need to add the next two "things" we find in the sequence. The "things" might be numbers, or may themselves be expressions to be evaluated (such as X+Y or X*Y).
In order to get what these "things" are, the function eval() is called with an updated value of i. Each call to eval() will fetch the value of the immediate next expression, and update position i.
Thus, 2 successive calls to eval() obtain the values of the 2 following expressions.
We then apply the + operator to the 2 values, and return the result.
It will help to work through an example such as "+ * 2 3 * 4 5", which is prefix notation for (2*3)+(4*5).
So this piece of code can only eat +, *, spaces and numbers. It is supposed to eat one command which can be one of:
- + <op1> <op2>
- * <op1> <op2>
<number>
It gets a pointer to a string, and a reading position which is incremented as the program goes along that string.
char *a; int i;
int eval()
{ int x = 0;
while (a[i] == ' ') i++; // it eats all spaces
if (a[i] == '+')
/* if the program encounters '+', two operands are expected next.
The reading position i already points just before the place
from which you have to start reading the next operand
(which is what first eval() call will do).
After the first eval() is finished,
the reading position is moved to the begin of the second operand,
which will be read during the second eval() call. */
{ i++; return eval() + eval(); }
if (a[i] == '*') // exactly the same, but for '*' operation.
{ i++; return eval() * eval(); }
while ((a[i] >= '0') && (a[i] <= '9')) // here it eats all digit until something else is encountered.
x = 10*x + (a[i++] - '0'); // every time the new digit is read, it multiplies the previously obtained number by 10 and adds the new digit.
return x;
// base case: returning the number. Note that the reading position already moved past it.
}
The example you are given uses a couple of global variables. They persist outside of the function's scope and must be initialized before calling the function.
i should be initialized to 0 so that you start at the beginning of the string, and the prefix expression is the string in a.
the operator is your prefix and so should be your first non-blank character, if you start with a number (string of numbers) you are done, that is the result.
example: a = " + 15 450"
eval() finds '+' at i = 1
calls eval()
which finds '1' at i = 3 and then '5'
calculates x = 1 x 10 + 5
returns 15
calls eval()
which finds '4' at i = 6 and then '5' and then '0'
calclulates x = ((4 x 10) + 5) x 10) + 0
returns 450
calculates the '+' operator of 15 and 450
returns 465
The returns are either a value found or the result of an operator and the succeeding results found. So recursively, the function successively looks through the input string and performs the operations until either the string ends or an invalid character is found.
Rather than breaking up code into chunks and so on, i'll try and just explain the concept it as simple as possible.
The eval function always skips spaces so that it points to either a number character ('0'->'9'), an addition ('+') or a multiply ('*') at the current place in the expression string.
If it encounters a number, it proceeds to continue to eat the number digits, until it reaches a non-number digit returning the total result in integer format.
If it encounters operator ('+' and '*') it requires two integers, so eval calls itself twice to get the next two numbers from the expression string and returns that result as an integer.
One hair in the soup may be evaluation order, cf. https://www.securecoding.cert.org/confluence/display/seccode/EXP10-C.+Do+not+depend+on+the+order+of+evaluation+of+subexpressions+or+the+order+in+which+side+effects+take+place.
It is not specified which eval in "eval() + eval()" is, well, evaluated first. That's ok for commutative operators but will fail for - or /, because eval() as a side effect advances the global position counter so that the (in time) second eval gets the (in space) second expression. But that may well be the (in space) first eval.
I think the fix is easy; assign to a temp and compute with that:
if (a[i] == '-')
{ i++; int tmp = eval(); return tmp - eval(); }
There are many algorithms to convert infix to postfix all over the web. But my question is how to make that to support functions? For example sin(x+y)*z.
I will appreciate a code.
If you are looking for an algorithm that gives you the conversion infix to postfix including function call support, you can use the below pseudocode(which looks like python code). I have written this for my case but not yet tested thouroughly. If you find any bugs please let me know.
I have also written a Java implementation for the same.
Also, there are few things to note about this implementation:
This algorithm assumes a stream of tokens in infix. It does not parse a expression string. So each token can be identified as an operand, operator, function call etc.
There are 7 different kinds of tokens:
Operands X, Y etc
Left Paranthesis - (
Right Paranthesis - )
Operators - +, *
Function call starts - sin(
Function call ends - sin( x )
Comma - ,
Function call starts are denoted by [ character in the algorithm and function call ends are denoted by ]. Please note that function call termination is a different token than Right Paranthesis ) although they may be represented by the same character in the string expression.
Every operator is a binary operator with precedence and associativity as their usual meaning.
Comma , is a special binary operator with precedence of NEGATIVE INFINITY and associativity as LEFT (same as + and *). Comma operator is used to separate the arguments of a function call. So for a function call:
f(a,b,c)
first comma separates a and b
second comma separates a,b and c
So the postfix for the above will be
ab,c,f
You can view Comma operator as a add to list function which adds the second argument to the list specified by the first argument or if both are single values it creates a list of two values.
Algorithm
infix_to_postfix(infix):
postfix = []
infix.add(')')
stack = []
stack.push('(')
for each token in infix:
if token is operand:
postfix.add(token)
if token is '[':
stack.push(token)
else if token is operator:
if stack is empty OR
stack[top] is '(' or stack[top] is '[':
stack.push(token)
else if (operator)token['precedence'] > stack[top]['precedence'] OR
( (operator)token['precedence'] == stack[top]['precedence'] AND
(operator)token['associativity') == 'RIGHT' ):
stack.push(token)
else
postfix.add(stack.pop())
stack.push(token)
else if token is '(':
stack.push(token)
else if token is ')':
while topToken = stack.pop() NOT '(':
postfix.add(topToken)
else if token is ']':
while True:
topToken = stack.pop()
postfix.add(topToken)
if topToken is '[':
break
else if token is ',':
while topToken = stack.peek() NOT '[':
postfix.add(topToken)
stack.pop()
stack.push(token)
Thats quite easy: It work with functions too, the regular operators you use (like +,-,*) are functions too. Your problem is, that what you consider "function" (like sin) is not in infix, but they are in prefix.
To come back to your problem: Just convert these prefix functions into postfix (you should find prefix to postfix on the web too - my assumption is that you dont know the "prefix" term) beforehand.
EDIT: Basicaly it is nothing more that first convert the arguments and output them in sequence and append the name of the function afterwards.
Although #mickeymoon algorithm seems to work, I still had to make some adjustments(didn't work for me) so I think it can be helpful for somebody another implementation(Java like implementation). Based on https://en.wikipedia.org/wiki/Shunting-yard_algorithm
Stack<Token> stack = new Stack<>();
List<Token> result = new ArrayList<>();
//https://en.wikipedia.org/wiki/Shunting-yard_algorithm
// with small adjustment for expressions in functions. Wiki example works only for constants as arguments
for (Token token : tokens) {
if (isNumber(token) || isIdentifier(token)) {
result.add(token);
continue;
}
if (isFunction(token)) {
stack.push(token);
continue;
}
// if OP(open parentheses) then put to stack
if (isOP(token)) {
stack.push(token);
continue;
}
// CP(close parentheses) pop stack to result until OP
if (isCP(token)) {
Token cur = stack.pop();
while (!isOP(cur)) {
if (!isComma(cur)) {
result.add(cur);
}
cur = stack.pop();
}
continue;
}
if (isBinaryOperation(token)) {
if (!stack.empty()) {
Token cur = stack.peek();
while ((!isBinaryOperation(cur)
|| (isBinaryOperation(cur) && hasHigherPriority(cur, token))
|| (hasEqualPriority(cur, token) && isLeftAssociative(token)))
&& !isOP(cur)
) {
// no need in commas in resulting list if we now how many parameters the function need
if (!isComma(cur)) {
result.add(cur);
}
stack.pop();
if (!stack.empty()) {
cur = stack.peek();
}
}
}
stack.push(token);
continue;
}
if (isComma(token)) {
Token cur = stack.peek();
while (!(isOP(cur) || isComma(cur))) {
result.add(cur);
stack.pop();
if (!stack.empty()) {
cur = stack.peek();// don't pop if priority is less
}
}
stack.push(token);
}
}
while (!stack.empty()) {
Token pop = stack.pop();
if (!isComma(pop)) {
result.add(pop);
}
}
return result;
I tested it with various complex expressions including function composition and complex arguments(doesn't work with example from Wiki algorithm). A couple of examples(e is just a variable, min,max, rand - functions):
Input: (3.4+2^(5-e))/(1+5/5)
Output: 3.4 2 5 e - ^ + 1 5 / + /
Input: 2+rand(1.4+2, 3+4)
Output: 2 1.4 2 + 3 4 + rand +
Input: max(4+4,min(1*10,2+(3-e)))
Output: 4 4 + 1 10 * 2 3 e - + min max
I also tested it with complex function with three arguments(where each argument is an expression by itself) and it words fine.
Here is the github for my java function that takes the list of tokens and returns the list of tokens in postfix notation. And here is the function that takes the output from first function and calculates the value of the expression
The code you'll have to work out yourself. Using your specific case as an example might help get you started; the postfix form of sin(x + y) * z would be:
x y + sin z *
Note that in this one example some operations operation on two values (+ and *), and others one (sin)
binary operators like + can be considered as +(x,y)
Similarly Consider sin, cos, etc functions as unary operators. So, sin(x+y)*z can be written as x y + sin z *. You need to give these unary functions special treatment.
How can I convert the char in the array into an integer?
Ignore lines 5-100 it is just my stack.
http://ideone.com/KQytD
Scroll down output #2 worked properly but output #3 did not. Some how when I pushed the value back into the stack and when I popped it it had the +'43' because of the ASCII and I cannot seem to get it into a regular integer value so I can do these operations easily.
line 116 puts input into char postfix. NOTE: input must be in postfix notation line 117 puts the single integer value into final after it has run through the function.
convertPostfixToEvaluation works as such: I scroll through each index of postfix until I read in '=' then I output the total/sum. The first if statement pushed the operands (0-9) into a stack. The second if statement if it reads in an operator then it attempts to do the operation as such in lines 134-158. After the if statements I increase the index value by 1 so it can scan the entire array.
The issue lies within the switch where I try adding,subtracting,multiply, or dividing more than 3 operands. so the 3rd one i believe is still has the value (+43 because of the ASCII).
My outputs(on the bottom of my program) show what the awkwardness is.
The cut to the chase issue. Issue converting char to int the second time around.
There are many things very likely wrong with this code.
Look up the function isdigit. This should eliminate the huge if statement.
You may want to use a string lookup instead of the other complex if statement:
const std::string my_operators = "+-/*";
if (my_operators.find(postfix[i]) != std::string::npos)
{
// Enter here if the character is a valid symbol.
}
If you "parse" character by character, you will have to build your number:
int number = 0;
// After detecting the character is a number:
number = number * 10 + (postfix[i] - '0');
The expression "postfix[i] - '0'" will return the distance between the number character and the character for zero. The C and C++ languages guarantee the following relationship:
'0' < '1' < '2' < '3' < '4' < '5' < '6' < '7' < '8' < '9'
The languages also state that those numbers are contiguous.
Suggestion: use std::string instead of an array of characters. The std::string contains some helpful functions for searching, skipping characters, and obtaining a substring.