expression evaluator in scala (with maybe placeholders?) - regex
I am reading something like this from my configuration file :
metric1.critical = "<2000 || >20000"
metric1.okay = "=1"
metric1.warning = "<=3000"
metric2.okay = ">0.9 && < 1.1 "
metric3.warning ="( >0.9 && <1.5) || (<500 &&>200)"
and I have a
metric1.value = //have some value
My aim is to basically evaluate
if(metric1.value<2000 || metric1.value > 20000)
metric1.setAlert("critical");
else if(metric1.value=1)
metric.setAlert("okay");
//and so on
I am not really good with regex so I am going to try not to use it. I am coding in Scala and wanted to know if any existing library can help with this. Maybe i need to put placeholders to fill in the blanks and then evaluate the expression? But how do I evaluate the expression most efficiently and with less overhead?
EDIT:
In java how we have expression evaluator Libraries i was hoping i could find something similar for my code . Maybe I can add placeholders in the config file like "?" these to substitute my metric1.value (read variables) and then use an evaluator?
OR
Can someone suggest a good regex for this?
Thanks in advance!
This sounds like you want to define your own syntax using a parser combinator library.
There is a parser combinator built into the scala class library. Since the scala library has been modularized, it is now a separate project that lives at https://github.com/scala/scala-parser-combinators.
Update: everybody looking for a parser combinator library that is conceptually similar to scala-parser-combinators should take a look at fastparse. It is very fast, and does not use macros. So it can serve as a drop-in replacement for scala-parser-combinators.
There are some examples on how to use it in Programming in Scala, Chapter 33, "Combinator Parsing".
Here is a little grammar, ast and evaluator to get you started. This is missing a lot of things such as whitespace handling, operator priority etc. You should also not use strings for encoding the different comparison operators. But I think with this and the chapter from Programming in Scala you should be able to come up with something that suits your needs.
import scala.util.parsing.combinator.{JavaTokenParsers, PackratParsers}
sealed abstract class AST
sealed abstract class BooleanExpression extends AST
case class BooleanOperation(op: String, lhs: BooleanExpression, rhs:BooleanExpression) extends BooleanExpression
case class Comparison(op:String, rhs:Constant) extends BooleanExpression
case class Constant(value: Double) extends AST
object ConditionParser extends JavaTokenParsers with PackratParsers {
val booleanOperator : PackratParser[String] = literal("||") | literal("&&")
val comparisonOperator : PackratParser[String] = literal("<=") | literal(">=") | literal("==") | literal("!=") | literal("<") | literal(">")
val constant : PackratParser[Constant] = floatingPointNumber.^^ { x => Constant(x.toDouble) }
val comparison : PackratParser[Comparison] = (comparisonOperator ~ constant) ^^ { case op ~ rhs => Comparison(op, rhs) }
lazy val p1 : PackratParser[BooleanExpression] = booleanOperation | comparison
val booleanOperation = (p1 ~ booleanOperator ~ p1) ^^ { case lhs ~ op ~ rhs => BooleanOperation(op, lhs, rhs) }
}
object Evaluator {
def evaluate(expression:BooleanExpression, value:Double) : Boolean = expression match {
case Comparison("<=", Constant(c)) => value <= c
case Comparison(">=", Constant(c)) => value >= c
case Comparison("==", Constant(c)) => value == c
case Comparison("!=", Constant(c)) => value != c
case Comparison("<", Constant(c)) => value < c
case Comparison(">", Constant(c)) => value > c
case BooleanOperation("||", a, b) => evaluate(a, value) || evaluate(b, value)
case BooleanOperation("&&", a, b) => evaluate(a, value) && evaluate(b, value)
}
}
object Test extends App {
def parse(text:String) : BooleanExpression = ConditionParser.parseAll(ConditionParser.p1, text).get
val texts = Seq(
"<2000",
"<2000||>20000",
"==1",
"<=3000",
">0.9&&<1.1")
val xs = Seq(0.0, 1.0, 100000.0)
for {
text <- texts
expression = parse(text)
x <- xs
result = Evaluator.evaluate(expression, x)
} {
println(s"$text $expression $x $result")
}
}
Scala has built in Interpreter library which you can use. The library provides functionalities similar to eval() in many other languages. You can pass Scala code snippet as String to the .interpret method and it will evaluate it.
import scala.tools.nsc.{ Interpreter, Settings }
val settings = new Settings
settings.usejavacp.value = true
val in = new Interpreter(settings)
val lowerCritical = "<2000" // set the value from config
val value = 200
in.interpret(s"$value $lowerCritical") //> res0: Boolean = true
val value1 = 20000 //> value1 : Int = 20000
in.interpret(s"$value1 $lowerCritical") //> res1: Boolean = false
You want to use an actual parser for this.
Most answers are suggesting Scala's parser combinators, and that's a perfectly valid choice, if a bit out-of-date.
I'd suggest Parboiled2, an other parser combinator implementation that has the distinct advantage of being written as Scala macros - without getting too technical, it means your parser is generated at compile time rather than runtime, which can yield significant performance improvements. Some benchmarks have Parboiled2 up to 200 times as fast as Scala's parser combinator.
And since parser combinators are now in a separate dependency (as of 2.11, I believe), there really is no good reason to prefer them to Parboiled2.
I recently faced the same problem and I ended up writing my own expression evaluation library scalexpr. It is a simple library but it can validate / evaluate expressions that are similar to the ones in the question. You can do things like:
val ctx = Map("id" -> 10L, "name" -> "sensor1")
val parser = ExpressionParser()
val expr = parser.parseBooleanExpression(""" id == 10L || name == "sensor1" """).get
println(expr3.resolve(ctx3)) // prints true
If you don't want to use the library, I recommend the fastparse parser... It is much faster than parser combinators, a little bit slower than parboiled, but much easier to use than both.
Related
Is there an alternative to an if statement in Lua?
I would like to know if there is a alternative to using a normal if statement in lua. For example, in java there are switch statements but I cant seem to find that in Lua
Lua lacks a C-style switch statement. A simple version of a switch statement can be implemented using a table to map the case value to an action. This is very efficient in Lua since tables are hashed by key value which avoids repetitive if then ... elseif ... end statements. action = { [1] = function (x) print(1) end, [2] = function (x) z = 5 end, ["nop"] = function (x) print(math.random()) end, ["my name"] = function (x) print("fred") end, }
The frequently used pattern local var; if condition then var = x else var = y end can be shortened using an and-or "ternary" substitute if x is truthy: local var = condition and x or y
if test == nil or test == false then return 0xBADEAFFE else return test end Can be shorten up to... return test or 0xBADEAFFEE This works even where you dont can do: if ... then ... else ... end Like in a function... print(test or 0xBADEAFFE) -- Output: 3135156222 ...or fallback to a default if an argument is ommited... function check(test) local test = test or 0xBADEAFFE return test end print(check()) -- Returns: 3135156222
How to transfer constraints of string type to z3 solver expr in C++?
For example, I got a constraint: "(number < 10) && (name == "hello")"; I can do the following now: context c; expr number= c.int_const(number); expr name = c->string_val(name.c_str()); expr constrain = ***procedure***("(number < 10) && (name == \"hello\")"); How can I implement this procedure()? there is an incomplete and unproofed answer in Use a C++ string in z3::expr?, and I still can not figure out how to implement it? I am very eagered and appreciateed for your help! Thanks!
Try: #include <z3++.h> using namespace z3; using namespace std; int main () { context c; expr number = c.int_const("number"); expr name = c.constant(c.str_symbol("name"), c.string_sort()); expr hello = c.string_val("hello"); expr constraint = number < 10 && name == hello; solver s(c); s.add(constraint); cout << s.check() << "\n"; cout << s.get_model() << "\n"; return 0; }; Assuming you put the above in a file called a.cpp, you compile it like this: $ g++ -std=c++11 a.cpp -l z3 And when run, it produces: sat (define-fun number () Int 9) (define-fun name () String "hello") Using higher-level APIs As you no doubt noticed, programming z3 in C/C++ is very verbose and terribly error prone. Unless you've some other reason to use C/C++, I'd recommend using a higher-level API, such as Python or Haskell, which simplifies programming in z3 to a great extent. Python For instance, you'd code your problem in Python like this: from z3 import * number = Int('number') name = String('name') s = Solver() s.add(number < 10, name == "hello") print(s.check()) print(s.model()) producing: sat [number = 9, name = "hello"] Haskell And in Haskell, it would look like: import Data.SBV ex :: IO SatResult ex = sat $ do number <- sInteger "number" name <- sString "name" constrain $ number .< 10 .&& name .== literal "hello" producing: *Main> ex Satisfiable. Model: number = 9 :: Integer name = "hello" :: String Summary Long story short, programming z3 in C/C++—while entirely possible—is something that's best avoided if you can use a higher-level interface. If you must stick to C/C++, be sure to study the API: https://z3prover.github.io/api/html/namespacez3.html
Scala functional programming concepts instead of multiple for loops
I am trying to learn functional programming in Scala. Right now I'm using the OOP way of having for loops to do a job. I have two lists userCurrentRole and entitlements over which I'm doing a double for loop: for { curr <- userCurrentRole { ent <- entitlements } { if (ent.userEmail.split("#")(0) == curr.head) { if (ent.roleName != curr(1)) { grantOrRevoke += 1 grantList += SomeCaseClass(curr.head, ent.roleName) } } } } Is it possible to convert this double for loop into a logic that uses map or filter or both or any functional programming features of scala, but without a for loop? EDIT 1: Added a list addition inside the double if..
The good news is: you are already using functional style! Since the for is not a loop per se, but a "for comprehension", which desugars into flatMap and map calls. It's only easier to read / write. However, the thing you should avoid is mutable variables, like the grantOrRevoke thing you have there. val revocations = for { curr <- userCurrentRole { ent <- entitlements if ent.userEmail.split("#")(0) == curr.head if ent.roleName != curr(1) } yield { 1 } revocations.size // same as revocations.sum Note that the ifs inside the for block (usually) desugar to withFilter calls, which is often preferable to filter calls, since the latter builds up a new collection whereas the former avoids that.
You can write it like this: val grantOrRevoke = userCurrentRole .map(curr => entitlements .filter(ent => ent.userEmail.split("#")(0) == curr.head && ent.roleName != curr(1)) .size) .sum
Well, you are already using some higher order functions, only that you don't notice it, because you believe those are for loops, but they aren't loops. They are just sugar syntax for calls to map & flatMap. But in your case, also to foreach and that plus mutability, is want doesn't make it functional. I would recommend you to take a look to the scaladoc, you will find that collections have a lot of useful methods. For example, in this case, we may use count & sum. val grantOrRevoke = userCurrentRole.iterator.map { // Maybe it would be better to have a list of tuples instead of a list of lists. case List(username, userRole) => entitlements.count { ent => (ent.userEmail.split("#", 2)(0) == username) && (ent.roleName == userRole) } }.sum
Scala memory issue on List vs. Vector
I wrote a solution to project Euler problem #59 in Scala and I do not understand why switching between Vector and List adds what I think is a memory leak. Here is a working, brute force solution using Vectors. val code = scala.io.Source.fromFile("e59.txt").getLines() .flatMap(l => l.split(',')).map(_.toInt).toVector val commonWords = scala.io.Source.fromFile("common_words.txt").getLines().toVector def decode(k: Int)(code: Vector[Int])(pswd: Vector[Int]): Vector[Int] = { code.grouped(k).flatMap(cs => cs.toVector.zip(pswd).map(t => t._1 ^ t._2)).toVector } def scoreText(text: Vector[Int]): Int = { if (text.contains((c: Int) => (c < 0 || c > 128))) -1 else { val words = text.map(_.toChar).mkString.toLowerCase.split(' ') words.length - words.diff(commonWords).length } } lazy val psswds = for { a <- (97 to 122); b <- (97 to 122); c <- (97 to 122) } yield Vector(a, b, c) val ans = psswds.toStream.map(decode(3)(code)) .map(text => (text, scoreText(text))) .maxBy(_._2)._1.sum println(ans) I store original code (a collection of ordered ints), each password and some common English words as Vectors. However, if I replace Vector with List, my program slows down with each checked password and eventually crashes: val code = scala.io.Source.fromFile("e59.txt").getLines() .flatMap(l => l.split(',')).map(_.toInt).toList val commonWords = scala.io.Source.fromFile("common_words.txt").getLines().toList def decode(k: Int)(code: List[Int])(pswd: List[Int]): List[Int] = { println(pswd) code.grouped(k).flatMap(cs => cs.toList.zip(pswd).map(t => t._1 ^ t._2)).toList } def scoreText(text: List[Int]): Int = { if (text.contains((c: Int) => (c < 0 || c > 128))) -1 else { val words = text.map(_.toChar).mkString.toLowerCase.split(' ') words.length - words.diff(commonWords).length } } lazy val psswds = for { a <- (97 to 122); b <- (97 to 122); c <- (97 to 122) } yield List(a, b, c) val ans = psswds.toStream.map(decode(3)(code)) .map(text => (text, scoreText(text))) .maxBy(_._2)._1.sum println(ans) Error: java.lang.OutOfMemoryError: GC overhead limit exceeded at java.lang.String.valueOf(String.java:2861) at java.lang.Character.toString(Character.java:4439) at java.lang.String.valueOf(String.java:2847) at scala.collection.mutable.StringBuilder.append(StringBuilder.scala:200) at scala.collection.TraversableOnce$$anonfun$addString$1.apply(TraversableOnce.scala:349) at scala.collection.immutable.List.foreach(List.scala:381) at scala.collection.TraversableOnce$class.addString(TraversableOnce.scala:342) at scala.collection.AbstractTraversable.addString(Traversable.scala:104) at scala.collection.TraversableOnce$class.mkString(TraversableOnce.scala:308) at scala.collection.AbstractTraversable.mkString(Traversable.scala:104) at scala.collection.TraversableOnce$class.mkString(TraversableOnce.scala:310) at scala.collection.AbstractTraversable.mkString(Traversable.scala:104) at scala.collection.TraversableOnce$class.mkString(TraversableOnce.scala:312) at scala.collection.AbstractTraversable.mkString(Traversable.scala:104) at Main$$anon$1.Main$$anon$$scoreText(e59_list.scala:14) at Main$$anon$1$$anonfun$5.apply(e59_list.scala:26) at Main$$anon$1$$anonfun$5.apply(e59_list.scala:26) at scala.collection.immutable.Stream$$anonfun$map$1.apply(Stream.scala:418) at scala.collection.immutable.Stream$$anonfun$map$1.apply(Stream.scala:418) at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1222) at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1212) at scala.collection.immutable.Stream.foreach(Stream.scala:595) at scala.collection.TraversableOnce$class.maxBy(TraversableOnce.scala:227) at scala.collection.AbstractTraversable.maxBy(Traversable.scala:104) at Main$$anon$1.<init>(e59_list.scala:27) at Main$.main(e59_list.scala:1) at Main.main(e59_list.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at scala.reflect.internal.util.ScalaClassLoader$$anonfun$run$1.apply(ScalaClassLoader.scala:70) Files used: common_words.txt a able about across after all almost also am among an and any are as at be because been but by can cannot could dear did do does either else ever every for from get got had has have he her hers him his how however i if in into is it its just least let like likely may me might most must my neither no nor not of off often on only or other our own rather said say says she should since so some than that the their them then there these they this tis to too twas us wants was we were what when where which while who whom why will with would yet you your e59.txt 79,59,12,2,79,35,8,28,20,2,3,68,8,9,68,45,0,12,9,67,68,4,7,5,23,27,1,21,79,85,78,79,85,71,38,10,71,27,12,2,79,6,2,8,13,9,1,13,9,8,68,19,7,1,71,56,11,21,11,68,6,3,22,2,14,0,30,79,1,31,6,23,19,10,0,73,79,44,2,79,19,6,28,68,16,6,16,15,79,35,8,11,72,71,14,10,3,79,12,2,79,19,6,28,68,32,0,0,73,79,86,71,39,1,71,24,5,20,79,13,9,79,16,15,10,68,5,10,3,14,1,10,14,1,3,71,24,13,19,7,68,32,0,0,73,79,87,71,39,1,71,12,22,2,14,16,2,11,68,2,25,1,21,22,16,15,6,10,0,79,16,15,10,22,2,79,13,20,65,68,41,0,16,15,6,10,0,79,1,31,6,23,19,28,68,19,7,5,19,79,12,2,79,0,14,11,10,64,27,68,10,14,15,2,65,68,83,79,40,14,9,1,71,6,16,20,10,8,1,79,19,6,28,68,14,1,68,15,6,9,75,79,5,9,11,68,19,7,13,20,79,8,14,9,1,71,8,13,17,10,23,71,3,13,0,7,16,71,27,11,71,10,18,2,29,29,8,1,1,73,79,81,71,59,12,2,79,8,14,8,12,19,79,23,15,6,10,2,28,68,19,7,22,8,26,3,15,79,16,15,10,68,3,14,22,12,1,1,20,28,72,71,14,10,3,79,16,15,10,68,3,14,22,12,1,1,20,28,68,4,14,10,71,1,1,17,10,22,71,10,28,19,6,10,0,26,13,20,7,68,14,27,74,71,89,68,32,0,0,71,28,1,9,27,68,45,0,12,9,79,16,15,10,68,37,14,20,19,6,23,19,79,83,71,27,11,71,27,1,11,3,68,2,25,1,21,22,11,9,10,68,6,13,11,18,27,68,19,7,1,71,3,13,0,7,16,71,28,11,71,27,12,6,27,68,2,25,1,21,22,11,9,10,68,10,6,3,15,27,68,5,10,8,14,10,18,2,79,6,2,12,5,18,28,1,71,0,2,71,7,13,20,79,16,2,28,16,14,2,11,9,22,74,71,87,68,45,0,12,9,79,12,14,2,23,2,3,2,71,24,5,20,79,10,8,27,68,19,7,1,71,3,13,0,7,16,92,79,12,2,79,19,6,28,68,8,1,8,30,79,5,71,24,13,19,1,1,20,28,68,19,0,68,19,7,1,71,3,13,0,7,16,73,79,93,71,59,12,2,79,11,9,10,68,16,7,11,71,6,23,71,27,12,2,79,16,21,26,1,71,3,13,0,7,16,75,79,19,15,0,68,0,6,18,2,28,68,11,6,3,15,27,68,19,0,68,2,25,1,21,22,11,9,10,72,71,24,5,20,79,3,8,6,10,0,79,16,8,79,7,8,2,1,71,6,10,19,0,68,19,7,1,71,24,11,21,3,0,73,79,85,87,79,38,18,27,68,6,3,16,15,0,17,0,7,68,19,7,1,71,24,11,21,3,0,71,24,5,20,79,9,6,11,1,71,27,12,21,0,17,0,7,68,15,6,9,75,79,16,15,10,68,16,0,22,11,11,68,3,6,0,9,72,16,71,29,1,4,0,3,9,6,30,2,79,12,14,2,68,16,7,1,9,79,12,2,79,7,6,2,1,73,79,85,86,79,33,17,10,10,71,6,10,71,7,13,20,79,11,16,1,68,11,14,10,3,79,5,9,11,68,6,2,11,9,8,68,15,6,23,71,0,19,9,79,20,2,0,20,11,10,72,71,7,1,71,24,5,20,79,10,8,27,68,6,12,7,2,31,16,2,11,74,71,94,86,71,45,17,19,79,16,8,79,5,11,3,68,16,7,11,71,13,1,11,6,1,17,10,0,71,7,13,10,79,5,9,11,68,6,12,7,2,31,16,2,11,68,15,6,9,75,79,12,2,79,3,6,25,1,71,27,12,2,79,22,14,8,12,19,79,16,8,79,6,2,12,11,10,10,68,4,7,13,11,11,22,2,1,68,8,9,68,32,0,0,73,79,85,84,79,48,15,10,29,71,14,22,2,79,22,2,13,11,21,1,69,71,59,12,14,28,68,14,28,68,9,0,16,71,14,68,23,7,29,20,6,7,6,3,68,5,6,22,19,7,68,21,10,23,18,3,16,14,1,3,71,9,22,8,2,68,15,26,9,6,1,68,23,14,23,20,6,11,9,79,11,21,79,20,11,14,10,75,79,16,15,6,23,71,29,1,5,6,22,19,7,68,4,0,9,2,28,68,1,29,11,10,79,35,8,11,74,86,91,68,52,0,68,19,7,1,71,56,11,21,11,68,5,10,7,6,2,1,71,7,17,10,14,10,71,14,10,3,79,8,14,25,1,3,79,12,2,29,1,71,0,10,71,10,5,21,27,12,71,14,9,8,1,3,71,26,23,73,79,44,2,79,19,6,28,68,1,26,8,11,79,11,1,79,17,9,9,5,14,3,13,9,8,68,11,0,18,2,79,5,9,11,68,1,14,13,19,7,2,18,3,10,2,28,23,73,79,37,9,11,68,16,10,68,15,14,18,2,79,23,2,10,10,71,7,13,20,79,3,11,0,22,30,67,68,19,7,1,71,8,8,8,29,29,71,0,2,71,27,12,2,79,11,9,3,29,71,60,11,9,79,11,1,79,16,15,10,68,33,14,16,15,10,22,73
Large amount of Lists create more load on GC comparing to the same Vectors. But your problem is not about right choice of collections, but about wrong use of Stream. Scala's streams can be very memory inefficient if used improperly. In your case, I assume, you were trying to use Stream to avoid eager computation of the transformed passwds collection, but you actually made the things worse (as Stream not only memoized your elements, it created extra overhead with Stream wrappers of these elements). What you had to do is just to replace toStream with view. It will create collection wrapper which makes nearly all transformations lazy (basically what you tried to achieve). val ans = psswds.view.map(decode(3)(code)) .map(text => (text, scoreText(text))) .maxBy(_._2)._1.sum After this tiny fix you program runs fine even with -Xmx5m (I checked). There are also many other things to optimize in your program (try to avoid creating excessive collections), but I'll leave it to you.
Writing a simple MSBuild condition parser
I'm trying to write an MSBuild condition parser. The conditions are described here. So I've come up with a grammar that appears to work fine: S -> !S S -> S == S S -> S != S S -> S && S S -> S || S S -> Fn( str ) S -> str == str S -> str != str S -> n < n S -> n <= n S -> n > n S -> n >= n This seems to fit my needs and I've come up with a set of C++ classes that define this simple language. ie I can create the classes such that they fit above and then I can call "run" on the base statement and I get a boolean value out of the other end. Using this language the following: (!Exists( "C:\\config.sys" ) && 14 < 17) || (AString == AString2 && HasTrailingSlash( "C:" )) becomes defined as (And forgive the huge line of code! ;)): Statement baseStatement( new StatementOrStatement( new StatementAndStatement( new NotStatement( new ExistsFunctionStatement( "C:\\Config.sys" ) ), new NumberLessThanNumberStatement( 14.0, 17.0 ) ), new StatementAndStatement( new StringEqualStringStatement( "AString", "AString2" ), new HasTrailingSlashFunctionStatement( "C:\\" ) ) ) ); I can simply run the above statement as follows: const bool result = baseStatement.Run(); So thus far all is fine. It seems I can represent the whole "language" using my grammar and i can build the grammar into a valid full statement. However now I have the next problem. I need to actually parse the string. I have no idea of where to start on this. Can anyone help me on how I write the actual string parser to parse the original statement into the set of C++ classes above? I'm rather lost on this front. I'm doing this purely for my own learning reasons so as far as possible I don't want to use someone else's parser library. Thanks in advance!
There are quite a few relevant questions already What's the best way to write a parser by hand? Resources for writing a recursive descent parser by hand How to write a recursive descent parser from scratch? https://stackoverflow.com/questions/2405623/looking-for-a-tutorial-on-recursive-descent-parsing