I want to be able to specify mathematical rules in an external location to my application and have my application process them at run time and perform actions (again described externally). What is the best way to do this?
For example, I might want to execute function MyFunction1() when the following evaluates to true:
(a < b) & MyFunction2() & (myWord == "test").
Thanks in advance for your help.
(If it is of any relevance, I wish to use C++, C or C++/CLI)
I'd consider not reinventing the wheel --- use an embedded scripting engine. This means you'd be using a standard form for describing the actions and logic. There are several great options out there that will probably be fine for your needs.
Good options include:
Javascript though google v8. (I don't love this from an embedding point of view,
but javascript is easy to work with, and many people already know it)
Lua. Its fast and portable. Syntax is maybe not as nice as Javascript, but embedding is
easy.
Python. Clean syntax, lots of libraries. Not much fun to embed though.
I'd consider using SWIG to help generate the bindings ... I know it works for python and lua, not sure about v8.
I would look at the command design pattern to handle calling external mathematical predicates, and the Factory design pattern to run externally defined code.
If your mathematical expression language is that simple then uou could define a grammar for it, e.g.:
expr = bin-op-expr | rel-expr | func-expr | var-expr | "(" expr ")"
bin-op = "&" | "|" | "!"
bin-op-expr = expr bin-op expr
rel-op = "<" | ">" | "==" | "!=" | "<=" | ">="
rel-expr = expr rel-op expr
func-args = "(" ")"
func-expr = func-name func-args
var-expr = name
and then translate that into a grammar for a parser. E.g. you could use Boost.Spirit which provides a DSL to allow you to express a grammar within your C++ code.
If that calculation happens at an inner loop, you want high performance, you cannot go with scripting languages. Based on how "deployable" and how much platform independent you would like that to be:
1) You could express the equations in C++ and let g++ compile it for you at run-time, and you could link to the resulting shared object. But this method is very much platform dependent at every step! The necessary system calls, the compiler to use, the flags, loading a shared object (or a DLL)... That would be super-fast in the end though, especially if you compile the innermost loop with the equation. The equation would be inlined and all.
2) You could use java in the same way. You can get a nice java compiler in java (from Eclipse I think, but you can embed it easily). With this solution, the result would be slightly slower (depending on how much template magic you want), I would expect, by a factor of 2 for most purposes. But this solution is extremely portable. Once you get it working, there's no reason it shouldn't work anywhere, you don't need anything external to your program. Another down side to this is having to write your equations in Java syntax, which is ugly for complex math. The first solution is much better in that respect, since operator overloading greatly helps math equations.
3) I don't know much about C#, but there could be a solution similar to (2). If there is, I know that there's operator overloading in C#, so your equations would be more pleasant to write and look at.
Related
I'm parsing spice netlists, for which I already have a parser. Since I actually use spectre (cadence, integrated electronics), I want to support both simulator languages (they differ, unfortunately). I could use a switch (e.g. commandline) and use the correct parser from start. However, spectre allows simulator lang=spectre statements, which I would also want to support (and vice versa, of course). How can this be done with boost::spirit?
My grammar looks roughly like this:
line = component_parser |
command_parser |
comment_parser |
subcircuit_parser |
subcircuit_instance_parser;
main = -line % qi::eol >> qi::eoi;
This toplevel structure is fine for both languages, so i need to change the subparsers. A first idea for me would be to have the toplevel parser hold instances (or objects) to the respective parser and to switch on finding the simulator lang statement (with a semantic action). Is this a good approach? If not, how else would one do this?
You can use qi::lazy (https://www.boost.org/doc/libs/1_68_0/libs/spirit/doc/html/spirit/qi/reference/auxiliary/lazy.html).
There's an idiomatic pattern related to that, known as The Nabialek Trick.
I have several answers up on this site that show these various techniques.
https://stackoverflow.com/search?q=user%3A85371+qi%3A%3Alazy
I want to write a program that takes an string like x^2+1 and understand it.
I want to ask the user to enter her/his function and I want to be able to process and understand it. Any Ideas?
char s[100];
s <- "x*I+2"
x=5;
I=2;
res=calc(s);
I think it could be done by something like string analyses but I think Its so hard for me.
I have another Idea and that is using tcc in main program and doing a realtime compile and run and delete a seprated program (or maybe function) that has the string s in it.
and I will create a temp file every time and ask tcc to compile it and run it by exec or similar syntax.
/*tmp.cpp:*/
#include <math.h>
void main(/*input args*/){
return x*I+2;
}
the tmp.cpp will created dynamically.
thanks in advance.
I am not sure what do you expect. It's too complex to give the code as answer, but the general idea is not very complex. It's not out of reach to code, even for a normal hobbyist programmer.
You need to define grammar, tokenize string, recognize operators, constants and variables.
Probably put expression into a tree. Make up a method for substituting the variables... and you can evaluate!
You need to have some kind of a parser. The easiest way to have math operations parsable is to have them written in RPN. You can, however, write your own parser using parser libraries, like Spirit from boost or Yacc
I use with success , function parser
from www it looks like it supports also std::complex, but I never used it
As luck would have it, I recently wrote one!
Look for {,include/}lib/MathExpression/Term. It handles complex numbers but you can easily adapt it for plain old floats.
The licence is GPL 2.
The theory in brief, when you have an expression like
X*(X+2)
Your highest level parser can parse expressions of the form A + B + C... In this case A is the whole expression.
You recurse to parse an operator of higher precedence, A * B * C... In this case A is X and B is (X+2)
Keep recursing until you're parsing either basic tokens such as X or hit an opening parenthesis, in which case push some kind of stack to track where your are and recurse into the parentheses with the top-level low-precedence parser.
I recommend you use RAII and throw exceptions when there are parse errors.
use a Recursive descent parser
Sample: it's in german, but a small and powerfull solution
look here
here is exactly what You are searching for. Change the function read_varname to detect a variable like 'x' or 'I'.
As the topic indicates, my program needs to read several function expressions and plug-in different variables many times. Parsing the whole expression again every time I need to plug-in a new value is definitely way too ugly, so I need a way to store parsed expression.
The expression may look like 2x + sin(tan(5x)) + x^2. Oh, and the very important point -- I'm using C++.
Currently I have three ideas on it, but all not very elegant:
Storing the S-expression as a tree; evaluate it by recurring. It may
be the old-school way to handle this, but it's ugly, and I would
have to handle with different number of parameters (like + vs. sin).
Composing anonymous functions with boost::lambda. It may work nice,
but personally I don't like boost.
Writing a small python/lisp script, use its native lambda
expression and call it with IPC... Well, this is crazy.
So, any ideas?
UPDATE:
I did not try to implement support for parenthesis and functions with only one parameter, like sin().
I tried the second way first; but I did not use boost::lambda, but a feature of gcc which could be used to create (fake) anonymous functions I found from here. The resulting code has 340 lines, and not working correctly because of scoping and a subtle issue with stack.
Using lambda could not make it better; and I don't know if it could handle with scoping correctly. So sorry for not testing boost::lambda.
Storing the parsed string as S-expressions would definitely work, but the implementation would be even longer -- maybe ~500 lines? My project is not that kind of gigantic projects with tens of thousands lines of code, so devoting so much energy on maintaining that kind of twisted code which would not be used very often seems not a nice idea.
So finally I tried the third method -- it's awesome! The Python script has only 50 lines, pretty neat and easy to read. But, on the other hand, it would also make python a prerequisite of my program. It's not that bad on *nix machines, but on windows... I guess it would be very painful for the non-programmers to install Python. So is lisp.
However, my final solution is opening bc as a subprocess. Maybe it's a bad choice for most situations, however, it fits me well.
On the other hand, for projects work only under *nix or already have python as a prerequisite, personally I recommend the third way if the expression is simple enough to be parsed with hand-written parser. If it's very complicated, like Hurkyl said, you could consider creating a mini-language.
Why not use a scripting language designed for exactly this kind of purpose? There are several such languages floating around, but my experience is with lua.
I use lua to do this kind of thing "all the time". The code to embed and parse an expression like that is very small. It would look something like this (untested):
std::string my_expression = "2*x + math.sin( math.tan( x ) ) + x * x";
//Initialise lua and load the basic math library.
lua_State * L = lua_open();
lua_openmath(L);
//Create your function and load it into lua
std::string fn = "function myfunction(x) return "+my_expression+"end";
luaL_dostring( L, fn.c_str(), fn.size() );
//Use your function
for(int i=0; i<10; ++i)
{
// add the function to the stack
lua_getfield(L, LUA_GLOBALSINDEX, "myfunction");
// add the argument to the stack
lua_pushnumber(L, i);
// Make the call, using one argument and expecting one result.
// stack looks like this : FN ARG
lua_pcall(L,1,1)
// stack looks like this now : RESULT
// so get the result and print it
double result = lua_getnumber(L,-1);
std::cout<<i<<" : "<<result<<std::endl;
// The result is still on the stack, so clean it up.
lua_pop(L,1);
}
I am pretty new to scipting languages (Perl in particular), and most of the code I write is an unconscious effort to convert C code to Perl.
Reading about Perl, one of the things that is often mentioned as the biggest difference is that Perl is a dynamic language. So, it can do stuff at runtime that the other languages (static ones) can only do at compiletime, and so be better at it because it can have access to realtime information.
All that is okay, but what specific features should I, with some experience in C and C++, keep in mind while writing code in Perl to use all the dynamic programming features that it has, to produce some awesome code?
This question is more than enough to fill a book. In fact, that's precisely what happened!
Mark Jason Dominus' excellent Higher-Order Perl is available online for free.
Here is a quote from it's preface that really grabbed me by the throat when I first read the book:
Around 1993 I started reading books
about Lisp, and I discovered something
important: Perl is much more like Lisp
than it is like C. If you pick up a
good book about Lisp, there will be a
section that describes Lisp’s good
features. For example, the book
Paradigms of Artificial Intelligence
Programming, by Peter Norvig, includes
a section titled What Makes Lisp
Different? that describes seven
features of Lisp. Perl shares six of
these features; C shares none of them.
These are big, important features,
features like first-class functions,
dynamic access to the symbol table,
and automatic storage management.
A list of C habits not to carry over into Perl 5:
Don't declare your variables at the top of the program/function. Declare them as they are needed.
Don't assign empty lists to arrays and hashes when declaring them (they are empty already and need to be initialized).
Don't use if (!(complex logical statement)) {}, that is what unless is for.
Don't use goto to break deeply nested loops, next, last, and redo all take a loop label as an argument.
Don't use global variables (this is a general rule even for C, but I have found a lot of C people like to use global variables).
Don't create a function where a closure will do (callbacks in particular). See perldoc perlsub and perldoc perlref for more information.
Don't use in/out returns, return multiple values instead.
Things to do in Perl 5:
Always use the strict and warnings pragmas.
Read the documentation (perldoc perl and perldoc -f function_name).
Use hashes the way you used structs in C.
Use the features that solve your problem with the best combination of maintainability, developer time, testability, and flexibility. Talking about any technique, style, or library outside of the context of a particular application isn't very useful.
Your goal shouldn't be to find problems for your solutions. Learn a bit more Perl than you plan on using immediately (and keep learning). One day you'll come across a problem and think "I remember something that might help with this".
You might want to see some of these book, however:
Higher-Order Perl
Mastering Perl
Effective Perl Programming
I recommend that you slowly and gradually introduce new concepts into your coding. Perl is designed so that you don't have to know a lot to get started, but you can improve your code as you learn more. Trying to grasp lots of new features all at once usually gets you in trouble in other ways.
I think the biggest hurdle will not be the dynamic aspect but the 'batteries included' aspect.
I think the most powerful aspects of perl are
hashes : they allow you to easily express very effective datastructures
regular expressions : they're really well integrated.
the use of the default variables like $_
the libraries and the CPAN for whatever is not installed standard
Something I noticed with C converts is the over use of for loops. Many can be removed using grep and map
Another motto of perl is "there is more than one way to do it". In order to climb the learning curve you have to tell yourself often : "There has got to be a better way of doing this, I cannot be the first one wanting to do ...". Then you can typically turn to google and the CPAN with its riduculous number of libraries.
The learning curve of perl is not steep, but it is very long... take your time and enjoy the ride.
Two points.
First, In general, I think you should be asking yourself 2 slightly different questions:
1) Which dynamic programming features of Perl can be used in which situations/to solve which problems?
2) What are the trade-offs, pitfalls and downsides of each feature.
Then the answer to your question becomes extremely obvious: you should be using the features that solve your problem better (performance or code maintainability wise) than a comparable non-DP solution, and which incurs less than the maximum accaptable level of downsides.
As an example, to quote from FM's comment, string form of eval has some fairly nasty downsides; but it MIGHT in certain cases be an extremely elegant solution which is orders of magnitude better than any alternate DP or SP approach.
Second, please be aware that a lot of "dynamic programming" features of Perl are actually packaged for you into extremely useful modules that you might not even recognize as being of the DP nature.
I'll have to think of a set of good examples, but one that immediately springs to mind is Text template modules, many of which are implemented using the above-mentioned string form of eval; or Try::Tiny exception mechanism which uses block form of eval.
Another example is aspect programming which can be achieved via Moose (I can't find the relevant StackOverflow link now - if someone has it please edit in the link) - which underneath uses access to symbol table featrue of DP.
Most of the other comments are complete here and I won't repeat them. I will focus on my personal bias about excessive or not enough use of language idioms in the language you are writing code in. As a quip goes, it is possible to write C in any language. It is also possible to write unreadable code in any language.
I was trained in C and C++ in college and picked up Perl later. Perl is fabulous for quick solutions and some really long life solutions. I built a company on Perl and Oracle solving logistics solutions for the DoD with about 100 active programmers. I also have some experience in managing the habits of other Perl programmers new and old. (I was the founder / ceo and not in technical management directly however...)
I can only comment on my transition to a Perl programmer and what I saw at my company. Many of our engineers shared my background of primarily being C / C++ programers by training and Perl programmers by choice.
The first issue I have seen (and had myself) is writing code that is so idiomatic that it is unreadable, unmaintainable, and unusable after a short period of time. Perl, and C++ share the ability to write terse code that is entertaining to understand at the moment but you will forget, not be around, and others won't get it.
We hired (and fired) many programmers over the 5 years I had the company. A common Interview Question was the following: Write a short Perl program that will print all the odd numbers between 1 and 50 inclusive separated by a space between each number and terminated with a CR. Do not use comments. They could do this on their own time of a few minutes and could do it on a computer to proof the output.
After they wrote the script and explained it, we would then ask them to modify it to print only the evens, (in front of the interviewer), then have a pattern of results based on every single digit even, every odd, except every seventh and 11th as an example. Another potential mod would be every even in this range, odd in that range, and no primes, etc. The purpose was to see if their original small script withstood being modified, debugged, and discussed by others and whether they thought in advance that the spec may change.
While the test did not say 'in a single line' many took the challenge to make it a single terse line and with the cost of readability. Others made a full module that just took too long given the simple spec. Our company needed to delver solid code very quickly; so that is why we used Perl. We needed programmers that thought the same way.
The following submitted code snippets all do exactly the same thing:
1) Too C like, but very easy to modify. Because of the C style 3 argument for loop it takes more bug prone modifications to get alternate cycles. Easy to debug, and a common submission. Any programmer in almost any language would understand this. Nothing particularly wrong with this, but not killer:
for($i=1; $i<=50; $i+=2) {
printf("%d ", $i);
}
print "\n";
2) Very Perl like, easy to get evens, easy (with a subroutine) to get other cycles or patterns, easy to understand:
print join(' ',(grep { $_ % 2 } (1..50))), "\n"; #original
print join(' ',(grep { !($_ % 2) } (1..50))), "\n"; #even
print join(' ',(grep { suba($_) } (1..50))), "\n"; #other pattern
3) Too idiomatic, getting a little weird, why does it get spaces between the results? Interviewee made mistake in getting evens. Harder to debug or read:
print "#{[grep{$_%2}(1..50)]}\n"; #original
print "#{[grep{$_%2+1}(1..50)]}\n"; #even - WRONG!!!
print "#{[grep{~$_%2}(1..50)]}\n"; #second try for even
4) Clever! But also too idiomatic. Have to think about what happens to the annon hash created from a range operator list and why that creates odds and evens. Impossible to modify to another pattern:
print "$_ " for (sort {$a<=>$b} keys %{{1..50}}), "\n"; #orig
print "$_ " for (sort {$a<=>$b} keys %{{2..50}}), "\n"; #even
print "$_ " for (sort {$a<=>$b} values %{{1..50}}), "\n"; #even alt
5) Kinda C like again but a solid framework. Easy to modify beyond even/odd. Very readable:
for (1..50) {
print "$_ " if ($_%2);
} #odd
print "\n";
for (1..50) {
print "$_ " unless ($_%2);
} #even
print "\n";
6) Perhaps my favorite answer. Very Perl like yet readable (to me anyway) and step-wise in formation and right to left in flow. The list is on the right and can be changed, the processing is immediately to the left, formatting again to the left, final operation of 'print' on the far left.
print map { "$_ " } grep { $_ & 1 } 1..50; #original
print "\n";
print map { "$_ " } grep { !($_ & 1) } 1..50; #even
print "\n";
print map { "$_ " } grep { suba($_) } 1..50; #other
print "\n";
7) This is my least favorite credible answer. Neither C nor Perl, impossible to modify without gutting the loop, mostly showing the applicant knew Perl array syntax. He wanted to have a case statement really badly...
for (1..50) {
if ($_ & 1) {
$odd[++$#odd]="$_ ";
next;
} else {
push #even, "$_ ";
}
}
print #odd, "\n";
print #even;
Interviewees with answers 5, 6, 2 and 1 got jobs and did well. Answers 7,3,4 did not get hired.
Your question was about using dynamic constructs like eval or others that you cannot do in a purely compiled language such as C. This last example is "dynamic" with the eval in the regex but truly poor style:
$t='D ' x 25;
$i=-1;
$t=~s/D/$i+=2/eg;
print "$t\n"; # don't let the door hit you on the way out...
Many will tell you "don't write C in Perl." I think this is only partially true. The error and mistake is to rigidly write new Perl code in C style even when there are so many more expressive forms in Perl. Use those. And yes, don't write NEW Perl code in C style because C syntax and idiom is all you know. (bad dog -- no biscuit)
Don't write dynamic code in Perl just because you can. There are certain algorithms that you will run across that you will say 'I don't quite know how I would write THAT in C' and many of these use eval. You can write a Perl regex to parse many things (XML, HTML, etc) using recursion or eval in the regex, but you should not do that. Use a parser just like you would in C. There are certain algorithms though that eval is a gift. Larry Wall's file name fixer rename would take a lot more C code to replicate, no? There are many other examples.
Don't rigidly avoid C stye either. The C 3 argument form of a for loop may be the perfect fit to certain algorithms. Also, remember why you are using Perl: assumably for high programmer productivity. If I have a completely debugged piece of C code that does exactly what I want and I need that in Perl, I just rewrite the silly thing C style in Perl! That is one of the strengths of the language (but also its weakness for larger or team projects where individual coding styles may vary and make the overall code difficult to follow.)
By far the killer verbal response to this interview question (from the applicant who wrote answer 6) was: This single line of code fits the spec and can easily be modified. However, there are many other ways to write this. The right way depends on the style of the surrounding code, how it will be called, performance considerations, and if the output format may change. Dude! When can you start?? (He ended up in management BTW.)
I think that attitude also applies to your question.
At least IME, the "dynamic" nature isn't really that big of a deal. I think the biggest difference you need to take into account is that in C or C++, you're mostly accustomed to there being only a fairly minor advantage to using library code. What's in the library is already written and debugged, so it's convenient, but if push comes to shove you can generally do pretty much the same thing on your own. For efficiency, it's mostly a question of whether your ability to write something a bit more specialized outweighs the library author's ability to spend more time on polishing each routine. There's little enough difference, however, that unless a library routine really does what you want, you may be better off writing your own.
With Perl, that's no longer true. Much of what's in the (huge, compared to C) library is actually written in C. Attempting to write anything very similar at all on your own (unless you write a C module, of course) will almost inevitably come out quite a bit slower. As such, if you can find a library routine that does even sort of close to what you want, you're probably better off using it. Using pre-written library code is much more important than in C or C++.
Good programming practices arent specific to individual languages. They are valid across all languages. In the long run, you may find it best not to rely on tricks possible in dynamic languages (for example, functions that can return either integer or text values) as it makes the code harder to maintain and quickly understand. So ultimately, to answer your question, I dont think you should be looking for features specific to dynamicly typed languages unless you have some compelling reason that you need them. Keep things simple and easy to maintain - that will be far more valuable in the long run.
There are many things you can do only with dynamic language but the coolest one is eval. See here for more detail.
With eval, you can execute string as if it was a pre-written command. You can also access variable by name at runtime.
For example,
$Double = "print (\$Ord1 * 2);";
$Opd1 = 8;
eval $Double; # Prints 8*2 =>16.
$Opd1 = 7;
eval $Double; # Prints 7*2 =>14.
The variable $Double is a string but we can execute it as it is a regular statement. This cannot be done in C/C++.
The cool thing is that a string can be manipulated at run time; therefore, we can create a command at runtime.
# string concatenation of operand and operator is done before eval (calculate) and then print.
$Cmd = "print (eval (\"(\$Ord1 \".\$Opr.\" \$Ord2)\"));";
$Opr = "*";
$Ord1 = "5";
$Ord1 = "2";
eval $Cmd; # Prints 5*2 => 10.
$Ord1 = 3;
eval $Cmd; # Prints 5*3 => 15.
$Opr = "+";
eval $Cmd; # Prints 5+3 => 8.
eval is very powerful so (as in Spiderman) power comes with responsibility. Use it wisely.
Hope this helps.
We would like to have user defined formulas in our c++ program.
e.g. The value v = x + ( y - (z - 2)) / 2. Later in the program the user would define x,y and z -> the program should return the result of the calculation. Somewhen later the formula may get changed, so the next time the program should parse the formula and add the new values. Any ideas / hints how to do something like this ? So far I just came to the solution to write a parser to calculate these formulas - maybe any ideas about that ?
If it will be used frequently and if it will be extended in the future, I would almost recommend adding either Python or Lua into your code. Lua is a very lightweight scripting language which you can hook into and provide new functions, operators etc. If you want to do more robust and complicated things, use Python instead.
You can represent your formula as a tree of operations and sub-expressions. You may want to define types or constants for Operation types and Variables.
You can then easily enough write a method that recurses through the tree, applying the appropriate operations to whatever values you pass in.
Building your own parser for this should be a straight-forward operation:
) convert the equation from infix to postfix notation (a typical compsci assignment) (I'd use a stack)
) wait to get the values you want
) pop the stack of infix items, dropping the value for the variable in where needed
) display results
Using Spirit (for example) to parse (and the 'semantic actions' it provides to construct an expression tree that you can then manipulate, e.g., evaluate) seems like quite a simple solution. You can find a grammar for arithmetic expressions there for example, if needed... (it's quite simple to come up with your own).
Note: Spirit is very simple to learn, and quite adapted for such tasks.
There's generally two ways of doing it, with three possible implementations:
as you've touched on yourself, a library to evaluate formulas
compiling the formula into code
The second option here is usually done either by compiling something that can be loaded in as a kind of plugin, or it can be compiled into a separate program that is then invoked and produces the necessary output.
For C++ I would guess that a library for evaluation would probably exist somewhere so that's where I would start.
If you want to write your own, search for "formal automata" and/or "finite state machine grammar"
In general what you will do is parse the string, pushing characters on a stack as you go. Then start popping the characters off and perform tasks based on what is popped. It's easier to code if you force equations to reverse-polish notation.
To make your life easier, I think getting this kind of input is best done through a GUI where users are restricted in what they can type in.
If you plan on doing it from the command line (that is the impression I get from your post), then you should probably define a strict set of allowable inputs (e.g. only single letter variables, no whitespace, and only certain mathematical symbols: ()+-*/ etc.).
Then, you will need to:
Read in the input char array
Parse it in order to build up a list of variables and actions
Carry out those actions - in BOMDAS order
With ANTLR you can create a parser/compiler that will interpret the user input, then execute the calculations using the Visitor pattern. A good example is here, but it is in C#. You should be able to adapt it quickly to your needs and remain using C++ as your development platform.