How to encode a constraint on the format of String values - regex

As I frequently observe and how I often implement a name attribute, is to simply model it as String.
What now, if the name has to follow a certain syntax, i.e. format? In Java I probably would define a constructor with a check on its arguments, something like:
public Name(str: String) {
if (str == null) throw new IllegalArgumentException("Str must not be null.");
if (!str.matches("name format expressed as regex")) throw new IllegalArgumentException("Str must match 'regex' but was " + str);
this.str = str;
}
In Scala I came up with the following solution:
import StdDef.Str
import StdDef.Bol
import StdDef.?
import scala.util.parsing.combinator.RegexParsers
final case class Name private (pfx: ?[Str] = None, sfx: Str) {
override def toString = pfx.mkString + sfx
}
object Name extends RegexParsers {
implicit def apply(str: Str): Name = parseAll(syntax, str) match {
case Success(res, _) => Name(res._1, res._2)
case rej: NoSuccess => error(rej.toString)
}
lazy val syntax = (prefix ?) ~! suffix
lazy val prefix = (("x" | "X") ~! hyph) ^^ { case a ~ b => a + b }
lazy val suffix = alpha ~! (alpha | digit | hyph *) ^^ { case a ~ b => a + b.mkString }
lazy val alpha: Parser[Str] = """\p{Alpha}""".r
lazy val digit: Parser[Str] = """\p{Digit}""".r
lazy val hyph: Parser[Str] = "-"
override lazy val skipWhitespace = false
}
My intents here are:
Compose a Name from its natural representation, i.e. a String value
Check whether its natural representation forms a valid Name at construction time.
Disallow any other construction than through the factory method apply:(str:Str)Str.
Make the construction from its natural representation implicit, e.g. val a: Name = "ISBN 978-0-9815316-4-9".
Decompose a Name into its parts according to its syntactical elements.
Have errors being thrown with messages, such as:
===
--
^
[1.3] error: string matching regex `\p{Alpha}' expected but end of source found
I would like to know what solutions you come up with.
After giving the topic some more thoughts, I am currently taking the following approach.
Token.scala:
abstract class Token {
val value: Str
}
object Token {
def apply[A <: Token](ctor: Str => A, syntax: Regex) = (value: Str) => value match {
case syntax() => ctor(value)
case _ => error("Value must match '" + syntax + "' but was '" + value + "'.")
}
}
Tokens.scala:
final case class Group private (val value: Str) extends Token
final case class Name private (val value: Str) extends Token
trait Tokens {
import foo.{ bar => outer }
val Group = Token(outer.Group, """(?i)[a-z0-9-]++""".r)
val Name = Token(outer.Name, """(?i)(?:x-)?+[a-z0-9-]++""".r)
}

Given that you'd be comfortable using a regex in Java, it seems like overkill to then try and solve the same problem with a parser in Scala.
Stick with what you know here, but add a Scala twist to clean up the solution a bit. Regexes in Scala also define extractors, allowing them to be used in a pattern match:
//triple-quote to make escaping easier, the .r makes it a regex
//Note how the value breaks normal naming conventions and starts in uppercase
//This is to avoid backticks when pattern matching
val TestRegex = """xxyyzz""".r
class Name(str: String) {
str match {
case Null => throw new IllegalArgumentException("Str must not be null")
case TestRegex => //do nothing
case _ => throw new IllegalArgumentException(
"Str must match 'regex' but was " + str)
}
}
disclaimer: I didn't actually test this code, it may contain typos

Related

How to use match with regular expressions in Scala

I am starting to learn Scala and want to use regular expressions to match a character from a string so I can populate a mutable map of characters and their value (String values, numbers etc) and then print the result.
I have looked at several answers on SO and gone over the Scala Docs but can't seem to get this right. I have a short Lexer class that currently looks like this:
class Lexer {
private val tokens: mutable.Map[String, Any] = collection.mutable.Map()
private def checkCharacter(char: Character): Unit = {
val Operator = "[-+*/^%=()]".r
val Digit = "[\\d]".r
val Other = "[^\\d][^-+*/^%=()]".r
char.toString match {
case Operator(c) => tokens(c) = "Operator"
case Digit(c) => tokens(c) = Integer.parseInt(c)
case Other(c) => tokens(c) = "Other" // Temp value, write function for this
}
}
def lex(input: String): Unit = {
val inputArray = input.toArray
for (s <- inputArray)
checkCharacter(s)
for((key, value) <- tokens)
println(key + ": " + value)
}
}
I'm pretty confused by the sort of strange method syntax, Operator(c), that I have seen being used to handle the value to match and am also unsure if this is the correct way to use regex in Scala. I think what I want this code to do is clear, I'd really appreciate some help understanding this. If more info is needed I will supply what I can
This official doc has lot's of examples: https://www.scala-lang.org/api/2.12.1/scala/util/matching/Regex.html. What might be confusing is the type of the regular expression and its use in pattern matching...
You can construct a regex from any string by using .r:
scala> val regex = "(something)".r
regex: scala.util.matching.Regex = (something)
Your regex becomes an object that has a few useful methods to be able to find matching groups like findAllIn.
In Scala it's idiomatic to use pattern matching for safe extraction of values, thus Regex class also has unapplySeq method to support pattern matching. This makes it an extractor object. You can use it directly (not common):
scala> regex.unapplySeq("something")
res1: Option[List[String]] = Some(List(something))
or you can let Scala compiler call it for you when you do pattern matching:
scala> "something" match {
| case regex(x) => x
| case _ => ???
| }
res2: String = something
You might ask why exactly this return type on unapply/unapplySeq. The doc explains it very well:
The return type of an unapply should be chosen as follows:
If it is just a test, return a Boolean. For instance case even().
If it returns a single sub-value of type T, return an Option[T].
If you want to return several sub-values T1,...,Tn, group them in an optional tuple Option[(T1,...,Tn)].
Sometimes, the number of values to extract isn’t fixed and we would
like to return an arbitrary number of values, depending on the input.
For this use case, you can define extractors with an unapplySeq method
which returns an Option[Seq[T]]. Common examples of these patterns
include deconstructing a List using case List(x, y, z) => and
decomposing a String using a regular expression Regex, such as case
r(name, remainingFields # _*) =>
In short your regex might match one or more groups, thus you need to return a list/seq. It has to be wrapped in an Option to comply with extractor contract.
The way you are using regex is correct, I would just map your function over the input array to avoid creating mutable maps. Perhaps something like this:
class Lexer {
private def getCharacterType(char: Character): Any = {
val Operator = "([-+*/^%=()])".r
val Digit = "([\\d])".r
//val Other = "[^\\d][^-+*/^%=()]".r
char.toString match {
case Operator(c) => "Operator"
case Digit(c) => Integer.parseInt(c)
case _ => "Other" // Temp value, write function for this
}
}
def lex(input: String): Unit = {
val inputArray = input.toArray
val tokens = inputArray.map(x => x -> getCharacterType(x))
for((key, value) <- tokens)
println(key + ": " + value)
}
}
scala> val l = new Lexer()
l: Lexer = Lexer#60f662bd
scala> l.lex("a-1")
a: Other
-: Operator
1: 1

Extract part of String using regex and assign it to variable scala

I have
val str = "abc:def"
val pattern = "(.*):.*".r
val prefix = str match {
case pattern(prefix) => prefix
case _ => ""
}
I want to avoid MatchError so if it matches I will return the match otherwise I will return "". This compiles and do extract "prefix = "abc". However I get a warning saying
Suspicious shadowing by a Variable Pattern
IDE is suggesting putting `` around pattern(`prefix`).
Is there a better way of doing this? Why is the IDE suggesting that solution.
The IDE shows the warning because you have 2 variables with the same name, I would suggest renaming the inner variable to something different:
val prefix = str match {
case pattern(pref) => pref
case _ => ""
}
The back ticks suggestion is not what you want in this case, you would use that if you wanted to match against the value of a variable (the IDE thinks you want that because prefix is the name of an existing variable).
You can rewrite it like below, using any other name for the prefix extracted from the pattern, because that value is then returned by the match and assigned to prefix
val str = "abc:def"
val pattern = "(.*):.*".r
val prefix = str match {
case pattern(aaa) => aaa
case _ => ""
}
Alternatively,
scala> val prefix = pattern.unapplySeq(str).map(_.head).getOrElse("")
prefix: String = abc
scala> val prefix = pattern.findPrefixMatchOf(str).map(_.group(1)).getOrElse("")
prefix: String = abc
Avoids inventing names.

Scala Regex Parser throws weird error

I have a simple RegexParser that matches {key}={value} repeating for several times:
object CommandOptionsParser extends RegexParsers {
private val key: Parser[String] = "[^= ]+".r
private val value: Parser[String] = "[^ ]*".r
val pair: Parser[Option[(String, Option[String])]] =
(key ~ ("=".r ~> value).?).? ^^ {
case None => None
case Some(k ~ v) => Some(k.trim -> v.map(_.trim))
}
val pairs: Parser[Map[String, Option[String]]] = phrase(repsep(pair, whiteSpace)) ^^ {
case v =>
Map(v.flatten: _*)
}
def apply(input: String): Map[String, Option[String]] = parseAll(pairs, input) match {
case Success(plan, _) => plan
case x => sys.error(x.toString)
}
}
However the matching of value seems to fail on more than 1 capturing groups (despite that the regex doesn't limit it). when I try to match against "token=abc again=abc", I have the following error:
[1.11] failure: string matching regex `\z' expected but `a' found
token=abc again=abc'
^
Why RegexParser has such strange behaviour?
The fix for your unexpected behavior is quite easy, just change the value of skipWhitespace:
object CommandOptionsParser extends RegexParsers {
override val skipWhitespace = false
From description of RegexParsers:
The parsing methods call the method skipWhitespace (defaults to
true) and, if true, skip any whitespace before each parser is
called.
So, what happened, your first pair was matched, then whiteSpace was skipped and then, as repsep couldn't find another whitespace separator, it just assumed that parsing is over, hence that "\z" expected.
Also, I can't help but note that the whole Parser approach for such simple task seems overcomplicated, simple regexps would suffice.
UPD: Also your parsers can be a bit simpler:
val pair: Parser[Option[(String, Option[String])]] =
(key ~ ("=" ~> value).?).? ^^ (_.map {case (k ~ v) => k.trim -> v.map(_.trim)})
val pairs: Parser[Map[String, Option[String]]] = phrase(repsep(pair, whiteSpace)) ^^
{ l => Map(l.flatten: _*)}

NullPointerException while working with stateful PartialFunction and collectFirst

Consider this (very ugly code):
object ExternalReferences2 {
import java.util.regex._
implicit def symbol2string(sym: Symbol) = sym.name
object Mapping {
def fromXml(mapping: scala.xml.NodeSeq) = {
new Mapping(mapping \ 'vendor text,
mapping \ 'match text,
mapping \ 'format text)
}
}
case class Mapping(vendor: String,
matches: String,
format: String) extends PartialFunction[String, String] {
private val pattern = Pattern.compile(matches)
private var _currentMatcher: Matcher = null
private def currentMatcher =
{ println("Getting matcher: " + _currentMatcher); _currentMatcher }
private def currentMatcher_=(matcher: Matcher) =
{ println("Setting matcher: " + matcher); _currentMatcher = matcher }
def isDefinedAt(entity: String) =
{ currentMatcher = pattern.matcher(entity); currentMatcher.matches }
def apply(entity: String) = apply
def apply = {
val range = 0 until currentMatcher.groupCount()
val groups = range
map (currentMatcher.group(_))
filterNot (_ == null)
map (_.replace('.', '/'))
format.format(groups: _*)
}
}
val config =
<external-links>
<mapping>
<vendor>OpenJDK</vendor>
<match>{ """^(javax?|sunw?|com.sun|org\.(ietf\.jgss|omg|w3c\.dom|xml\.sax))(\.[^.]+)+$""" }</match>
<format>{ "http://download.oracle.com/javase/7/docs/api/%s.html" }</format>
</mapping>
</external-links>
def getLinkNew(entity: String) =
(config \ 'mapping)
collectFirst({ case m => Mapping.fromXml(m)})
map(_.apply)
def getLinkOld(entity: String) =
(config \ 'mapping).view
map(m => Mapping.fromXml(m))
find(_.isDefinedAt(entity))
map(_.apply)
}
I tried to improve the getLinkOld method by using collectFirst as shown in getLinkNew, but I always get a NullPointerException because _currentMatcher is still set to null
scala> ExternalReferences2.getLinkNew("java.util.Date")
Getting matcher: null
java.lang.NullPointerException
at ExternalReferences2$Mapping.apply(<console>:32)
at ExternalReferences2$$anonfun$getLinkNew$2.apply(<console>:58)
at ExternalReferences2$$anonfun$getLinkNew$2.apply(<console>:58)
at scala.Option.map(Option.scala:131)
at ExternalReferences2$.getLinkNew(<console>:58)
at .<init>(<console>:13)
at .<clinit>(<console>)
at .<init>(<console>:11)
at .<clinit>(<console>)
while it works perfectly with getLinkOld.
What is the problem here?
Your matcher is created as a side-effect in isDefined. Passing side effecting functions to routine such as map is usually a recipe for disaster, but this is not even what happens here. Your code requires isDefined to have been called just before apply is, with the same argument. That makes your code very fragile, and that is what you should change.
Clients of PartialFunction do not have to do follow that protocol in general. Imagine for instance
if (f.isDefinedAt(x) && f.isDefinedAt(y)) {fx = f(x); fy = f(y)}.
And here the code that calls apply is not even yours, but the collection classes', so you do not control what happens.
Your specific problem in getLinkNew is that isDefined is simply never called.The PartialFunction argument of collectFirst is {case m => ...}. The isDefined that is called is the isDefined of this function. As m is an irrefutable pattern, it is allways true, and collectFirst will always return the first element if there is one. That the partial function returns another partial function (the Mapping) which happens not to be defined at m, is irrelevant.
Edit - Possible workaround
A very light change would be to check whether a matcher is available and create it if it is not. Better, keep the entity string that has been used to create it too, so that you can check it is the proper one. This should make the side effect benign as long as there is no multithreading. But the way, do not use null, use Option, so the compiler will not let you ignore the possibility that it may be None.
var _currentMatcher : Option[(String, Matcher)] = None
def currentMatcher(entity: String) : Matcher = _currentMatcher match{
case Some(e,m) if e == entity => m
case _ => {
_currentMatcher = (entity, pattern.matcher(entity))
_currentmatcher._2
}
}
Edit again. Stupid me
Sorry, the so called workaround indeed makes the class safer, but it does not make the collectFirst solution work. Again, the case m => partial function is always defined (note: entity does not even appears in your getLinkNew code, which should be worrying). The problem is that one would need a PartialFunction of a NodeSeq (not of entity, which will be known to the function, but not passed as argument). isDefined will be called, then apply. The pattern and the matcher depends on the NodeSeq, so they cannnot be created beforehand, but only in isDefined and/or apply. In the same spirit, you can cache what is computed in isDefined to reuse in Apply. This is definitely not pretty
def linkFor(entity: String) = new PartialFunction[NodeSeq, String] {
var _matcher : Option[String, Matcher] = None
def matcher(regexp: String) = _matcher match {
case Some(r, m) where r == regexp => m
case None => {
val pattern = Pattern.compile(regexp)
_matcher = (regexp, pattern.matcher(entity))
_matcher._2
}
}
def isDefined(mapping: NodeSeq) = {
matcher(mapping \ "match" text).matches
}
def apply(mapping: NodeSeq) = {
// call matcher(...), it is likely to reuse previous matcher, build result
}
}
You use that with (config \ mapping).collectFirst(linkFor(entity))

How to pattern match using regular expression in Scala?

I would like to be able to find a match between the first letter of a word, and one of the letters in a group such as "ABC". In pseudocode, this might look something like:
case Process(word) =>
word.firstLetter match {
case([a-c][A-C]) =>
case _ =>
}
}
But how do I grab the first letter in Scala instead of Java? How do I express the regular expression properly? Is it possible to do this within a case class?
You can do this because regular expressions define extractors but you need to define the regex pattern first. I don't have access to a Scala REPL to test this but something like this should work.
val Pattern = "([a-cA-C])".r
word.firstLetter match {
case Pattern(c) => c bound to capture group here
case _ =>
}
Since version 2.10, one can use Scala's string interpolation feature:
implicit class RegexOps(sc: StringContext) {
def r = new util.matching.Regex(sc.parts.mkString, sc.parts.tail.map(_ => "x"): _*)
}
scala> "123" match { case r"\d+" => true case _ => false }
res34: Boolean = true
Even better one can bind regular expression groups:
scala> "123" match { case r"(\d+)$d" => d.toInt case _ => 0 }
res36: Int = 123
scala> "10+15" match { case r"(\d\d)${first}\+(\d\d)${second}" => first.toInt+second.toInt case _ => 0 }
res38: Int = 25
It is also possible to set more detailed binding mechanisms:
scala> object Doubler { def unapply(s: String) = Some(s.toInt*2) }
defined module Doubler
scala> "10" match { case r"(\d\d)${Doubler(d)}" => d case _ => 0 }
res40: Int = 20
scala> object isPositive { def unapply(s: String) = s.toInt >= 0 }
defined module isPositive
scala> "10" match { case r"(\d\d)${d # isPositive()}" => d.toInt case _ => 0 }
res56: Int = 10
An impressive example on what's possible with Dynamic is shown in the blog post Introduction to Type Dynamic:
object T {
class RegexpExtractor(params: List[String]) {
def unapplySeq(str: String) =
params.headOption flatMap (_.r unapplySeq str)
}
class StartsWithExtractor(params: List[String]) {
def unapply(str: String) =
params.headOption filter (str startsWith _) map (_ => str)
}
class MapExtractor(keys: List[String]) {
def unapplySeq[T](map: Map[String, T]) =
Some(keys.map(map get _))
}
import scala.language.dynamics
class ExtractorParams(params: List[String]) extends Dynamic {
val Map = new MapExtractor(params)
val StartsWith = new StartsWithExtractor(params)
val Regexp = new RegexpExtractor(params)
def selectDynamic(name: String) =
new ExtractorParams(params :+ name)
}
object p extends ExtractorParams(Nil)
Map("firstName" -> "John", "lastName" -> "Doe") match {
case p.firstName.lastName.Map(
Some(p.Jo.StartsWith(fn)),
Some(p.`.*(\\w)$`.Regexp(lastChar))) =>
println(s"Match! $fn ...$lastChar")
case _ => println("nope")
}
}
As delnan pointed out, the match keyword in Scala has nothing to do with regexes. To find out whether a string matches a regex, you can use the String.matches method. To find out whether a string starts with an a, b or c in lower or upper case, the regex would look like this:
word.matches("[a-cA-C].*")
You can read this regex as "one of the characters a, b, c, A, B or C followed by anything" (. means "any character" and * means "zero or more times", so ".*" is any string).
To expand a little on Andrew's answer: The fact that regular expressions define extractors can be used to decompose the substrings matched by the regex very nicely using Scala's pattern matching, e.g.:
val Process = """([a-cA-C])([^\s]+)""".r // define first, rest is non-space
for (p <- Process findAllIn "aha bah Cah dah") p match {
case Process("b", _) => println("first: 'a', some rest")
case Process(_, rest) => println("some first, rest: " + rest)
// etc.
}
String.matches is the way to do pattern matching in the regex sense.
But as a handy aside, word.firstLetter in real Scala code looks like:
word(0)
Scala treats Strings as a sequence of Char's, so if for some reason you wanted to explicitly get the first character of the String and match it, you could use something like this:
"Cat"(0).toString.matches("[a-cA-C]")
res10: Boolean = true
I'm not proposing this as the general way to do regex pattern matching, but it's in line with your proposed approach to first find the first character of a String and then match it against a regex.
EDIT:
To be clear, the way I would do this is, as others have said:
"Cat".matches("^[a-cA-C].*")
res14: Boolean = true
Just wanted to show an example as close as possible to your initial pseudocode. Cheers!
First we should know that regular expression can separately be used. Here is an example:
import scala.util.matching.Regex
val pattern = "Scala".r // <=> val pattern = new Regex("Scala")
val str = "Scala is very cool"
val result = pattern findFirstIn str
result match {
case Some(v) => println(v)
case _ =>
} // output: Scala
Second we should notice that combining regular expression with pattern matching would be very powerful. Here is a simple example.
val date = """(\d\d\d\d)-(\d\d)-(\d\d)""".r
"2014-11-20" match {
case date(year, month, day) => "hello"
} // output: hello
In fact, regular expression itself is already very powerful; the only thing we need to do is to make it more powerful by Scala. Here are more examples in Scala Document: http://www.scala-lang.org/files/archive/api/current/index.html#scala.util.matching.Regex
Note that the approach from #AndrewMyers's answer matches the entire string to the regular expression, with the effect of anchoring the regular expression at both ends of the string using ^ and $. Example:
scala> val MY_RE = "(foo|bar).*".r
MY_RE: scala.util.matching.Regex = (foo|bar).*
scala> val result = "foo123" match { case MY_RE(m) => m; case _ => "No match" }
result: String = foo
scala> val result = "baz123" match { case MY_RE(m) => m; case _ => "No match" }
result: String = No match
scala> val result = "abcfoo123" match { case MY_RE(m) => m; case _ => "No match" }
result: String = No match
And with no .* at the end:
scala> val MY_RE2 = "(foo|bar)".r
MY_RE2: scala.util.matching.Regex = (foo|bar)
scala> val result = "foo123" match { case MY_RE2(m) => m; case _ => "No match" }
result: String = No match