I need to write a program which contains the function
doesExist: [A](p:A => Boolean) (xs: List[A]) Boolean.
e.g. doesExist ((x:Int) => x==2) (List(5,1,2,3)) => true
It should just return true or false depending on whether the element is found in the list.
I guess that the gist of it is to not use the built in function of lists... So here's an alternative using foldLeft:
def doesExist[A](p: A => Boolean)(xs: List[A]): Boolean = {
xs.foldLeft(false)((acc, elem) => acc || p(elem))
}
It could be a good exercise to practice pattern-matching:
def doesExist[A](p:A => Boolean)(xs: List[A]): Boolean = xs match {
case Nil => false
case head :: tail => p(head) || doesExist(p)(tail)
}
Simply:
def doesExist[A](p:A => Boolean)(xs: List[A]) = xs.exists(p)
If you cannot use exists:
def doesExist[A](p:A => Boolean)(xs: List[A]) =
xs.fold(false)((alreadyFound, a) => alreadyFound || p(a))
Here's how the Scala 2.11.8 standard library does it.
From LinearSeqOptimized.scala:
override /*IterableLike*/
def exists(p: A => Boolean): Boolean = {
var these = this // <--- Look, a var!
while (!these.isEmpty) {
if (p(these.head)) return true // <--- Look, a return!
these = these.tail
}
false
}
In other words, Scala uses a while loop, which mutates a var that points to the next object, and breaks out of the loop when it reaches the desired object.
I found out, when comparing answers to a similar question, that while with some sort of mutating iteration variable is the most efficient way to write this in Scala. Pure-functional approaches like recursion, Streams, etc. are slower—as much as 8 times slower (and still without scanning beyond the first 'true' element). I gather that the best approach is to use some kind of mutable iterator when you need to do a linear search and hide it inside a function that presents a pure-functional interface to the rest of the program. The Scala standard library does that in many places for you, so you get the best possible performance with a pure-functional interface.
Related
I am just trying to figure out how immutable things like a List are working, and how I can add things to it?
I am very sorry for asking such dumb questions, but why is here my list always empty when printing it out?
var end = false
val list = List()
while (!end) {
val input = scala.io.StdIn.readLine("input:")
if (input == "stop" ) end = true
else input :: list
}
println(list)
}
Sorry for my inconvenience and this rather stupid question!
I am just trying to figure out how immutable things like a List are working, and how I can add things to it?
You can't. That's what immutable means, after all. If Latin is not your cup of tea, the English translation of immutable is unchangeable. It should be clear now, why you can't change something that is unchangeable.
I am very sorry for asking such dumb questions, but why is here my list always empty when printing it out?
You create an empty list, and you never change it (because it cannot be changed anyway). So, of course it is empty.
What can you can do, however, is create a new list which is almost exactly like the old list, except with a new item prepended to the front. That's what you are doing here:
input :: list
However, you don't assign this new list anywhere, you don't return it, you completely ignore it.
If you want to actually use your list in any way, you need to remember it somehow. The most obvious solution would be to assign it to a variable:
var end = false
var list: List[String] = List() // note: `var` instead of `val`
while (!end) {
val input = scala.io.StdIn.readLine("input:")
if (input == "stop" ) end = true
else list = input :: list // note: assign to `list`
}
println(list)
However, that's not very idiomatic. After all, we have now taken an immutable list and assigned it to a mutable variable … IOW, we have just moved the mutability around.
Instead, we could use a recursive solution:
def buildListFromInput(list: List[String] = List()): List[String] = {
val input = scala.io.StdIn.readLine("input:")
if (input == "stop") list else buildListFromInput(input :: list)
}
println(buildListFromInput())
This solution is not only recursive, the recursive call is also in tail position (IOW, the method is tail-recursive), which means that it will be just as efficient as a while loop (in fact, it will be compiled into a while loop, or more precisely, into a GOTO). The Scala Language Specification guarantees that all implementations of Scala must eliminate direct tail-recursion.
The reason
println(list)
is only printing out an empty list is because the bit
input :: list
isn't actually mutating the list itself. It is simply, in this case, very temporarily, creating a list containing the input at the front.
Try
println(input :: list)
or
val newList = input :: list
println(newList)
and you'll see what I mean.
Try rewriting the code in more functional way. Every operation on Immutable data structures return new instance with change. So :: operator creates new List with input on front. You might want to try rewrite this code as tail recursive function as follows.
#tailrec
def scanInput(continue: Boolean,acc: List[String]): List[String] = {
val input = scala.io.StdIn.readLine("input:")
if(!continue) acc
else scanInput(input != "stop", input :: acc)
}
Above code has no mutating state and it suits more Scala functional style.
In scala List is immutable.
Then how can I add items to the list?
When you add an item to list a new List instance is crated with a item as its head and its tail now contains the previous list.
If you have list of "1,2,3" called intList internally it is represented as
List(3, List(2, List(1, Nil) ) )
If you add an element 4 to this intList
List(4, intList )
Lets call this newList
Note intList still contains List(3, List(2, List(1, Nil) ) ).
If you want the intList to refer the newList You will have to do
intList = intList.add(4)
How can I fix my code
Change list from val to var. Then you can assign resulting List to list variable
list = input :: list
Source: Online course on Scala called Functional Programming Principles in Scala
Thank you for all your help, I appreciate your help so much from all of you!
I should have taken a closer look at recursion since it seems to be really important like in Scala!
But trough your help I am getting an better idea of how it works!
I just tried to figure out how your solutions are working and created my own:
val list = List()
def scanInput(acc: List[String]): List[String] = {
val input = scala.io.StdIn.readLine("input:")
input match {
case "stop" => acc
case _ => scanInput(input :: acc)
}
}
println(scanInput(list))
I'm trying to write the get method for a key, value pair implemented using a list. I want to use the Option type as I heard its good for this but I'm new to Scala and I'm not really sure how to use it in this case...
This is as far as I got, only the method header.
def get(key : String): Option[Any] = {}
My guess is you are looking for something like this:
class KeyValueStore(pairs: List[(String, Any)]) {
def get(key: String): Option[Any] = pairs.collectFirst {
case (k, v) if k == key => v
}
}
This uses the collectFirst method for sequences. If you want a more "do it yourself" approach, this should work:
def get(key: String): Option[Any] = {
def search(xs: List[(String, Any)]): Option[Any] = {
xs match {
case List() => None //end of list and key not found. We return None
case (k, v) :: rest if k == key => Some(v) // found our key. Returning some value
case _ :: rest => search(rest) // not found until nou. Carrying on with the rest of the list
}
search(pairs)
}
}
You can turn a List of Pairs into a Map:
class Store[K, V](values: List[(K, V)]) {
val map = values.toMap
def get(key: K): Option[V] = map get key
}
Although #Marius' collectFirst version is probably the most elegant (and maybe a little bit faster as it only uses one closure), I find it more intuitive to use find for your problem :
def get[A, B](key: A, pairs: List[(A, B)]): Option[B] = pairs.find(_._1 == key).map(_._2)
In case you were wondering (or need high performance), you will need either #Marius' recursive or the following imperative version which may look more familiar (although less idiomatic):
def get[A, B](key: A, pairs: List[(A, B)]): Option[B] = {
var l = pairs
var found: Option[B] = None
while (l.nonEmpty && found.isEmpty) {
val (k, v) = l.head
if (k == key) {
found = Some(v)
} else {
l = l.tail
}
}
found
}
What you must understand is that Option[B] is a class that may either be instantiated to None (which replaces and improves the null reference used in other languages) or Some(value: B). Some is a case class, which allows, among other neat features, to instantiate it without the new keyword (thanks to some compiler magic, Google Scala case class for more info). You can think of Option as a List which may contain either 0 or 1 element: most operations that can be done on sequences can also be applied to Options (such as map in the find version).
I've struggled to find a way to stay true to functional style in for expressions when I need to collect multiple parameters of an object into a List.
As an example, say I have a Notification object, which has both a fromId (the user id the notification is from) and an objectOwnerId (the id of the user who created the original object). These can differ in facebook style notifications ("X also commented on Y's post").
I can collect the userIds with a for expression like so
val userIds = for { notification <- notifications } yield notification.fromId
however say I want to collect both the fromIds and the objectOwnerIds into a single list, is there any way to do this in a single for expression without the user of vars?
I've done something like this in the past:
var ids = List()
for {
notification <- notifications
ids = ids ++ List(notification.fromId, notification.objectOwnerId)
}
ids = ids.distinct
but it feels like there must be a better way. The use of a var, and the need to call distinct after I complete the collection are both ugly. I could avoid the distinct with some conditionals, but I'm trying to learn the proper functional methods to do things.
Thanks in advance for any help!
For such cases, there is foldLeft:
(notifications foldLeft Set.empty[Id]) { (set, notification) =>
set ++ Seq(notification.fromId, notification.ownerId)
}
or in short form:
(Set.empty[Id] /: notifications) { (set, notification) =>
set ++ Seq(notification.fromId, notification.ownerId)
}
A set doesn't hold duplicates. After the fold you can convert the set to another collection if you want.
val userIds = for {
notification <- notifications
id <- List(notification.fromId, notification.objectOwnerId)
} yield id
Apply distinct afterwards if required. If the id can only be duplicated on a single notification, you can apply distinct on the second generator instead.
Sure, instead of just yielding the fromId, yield a tuple
val idPairs:List[(String, String)] = for(notification <- notifications) yield(notification.fromId, notification.objectOwnerId)
Well, here is my answer to the following:
How to map from [Y(x1, x2), Y(x3, x4)] to [x1,x2,x3,x4]?
Use flatMap (see Collection.Traversable, but note it's actually first defined higher up).
case class Y(a: Int, b: Int)
var in = List(Y(1,2), Y(3,4))
var out = in.flatMap(x => List(x.a, x.b))
> defined class Y
> in: List[Y] = List(Y(1,2), Y(3,4))
> out: List[Int] = List(1, 2, 3, 4)
Also, since for..yield is filter, map and flatMap in one (but also see "sugar for flatMap?" that points out that this isn't as efficient as it could be: there is an extra map):
var out = for { i <- in; x <- Seq(i.a, i.b) } yield x
I would likely pick one of the other answers, however, as this does not directly address the final problem being solved.
Happy coding.
You can also use Stream to transform the pairs into a stream of individual items:
def toStream(xs: Iterable[Y]): Stream[Int] = {
xs match {
case Y(a, b) :: t => a #:: b #:: toStream(t)
case _ => Stream.empty
}
}
But like pst said, this doesn't solve your final problem of getting the distinct values, but once you have the stream it's trivial:
val result = toStream(ys).toList.removeDuplicates
Or a slight modification to the earlier suggestions to use flatten - add a function that turns a Y into a List:
def yToList(y: Y) = List(y.a, y.b)
Then you can do:
val ys = List(Y(1, 2), Y(3, 4))
(ys map yToList flatten).removeDuplicates
I agree with Dave's solution but another approach is to fold over the list, producing your map of id to User object as you go. The function to apply In the fold will query the db for both users and add them to the map being accumulated.
What about simple map? AFAIK for yield gets converted to series of flatMap and map anyway. Your problem could be solved simply as follows:
notifications.map(n => (n.fromId, n.objectOwnerId)).distinct
What is the best way to remove the first occurrence of an object from a list in Scala?
Coming from Java, I'm accustomed to having a List.remove(Object o) method that removes the first occurrence of an element from a list. Now that I'm working in Scala, I would expect the method to return a new immutable List instead of mutating a given list. I might also expect the remove() method to take a predicate instead of an object. Taken together, I would expect to find a method like this:
/**
* Removes the first element of the given list that matches the given
* predicate, if any. To remove a specific object <code>x</code> from
* the list, use <code>(_ == x)</code> as the predicate.
*
* #param toRemove
* a predicate indicating which element to remove
* #return a new list with the selected object removed, or the same
* list if no objects satisfy the given predicate
*/
def removeFirst(toRemove: E => Boolean): List[E]
Of course, I can implement this method myself several different ways, but none of them jump out at me as being obviously the best. I would rather not convert my list to a Java list (or even to a Scala mutable list) and back again, although that would certainly work. I could use List.indexWhere(p: (A) ⇒ Boolean):
def removeFirst[E](list: List[E], toRemove: (E) => Boolean): List[E] = {
val i = list.indexWhere(toRemove)
if (i == -1)
list
else
list.slice(0, i) ++ list.slice(i+1, list.size)
}
However, using indices with linked lists is usually not the most efficient way to go.
I can write a more efficient method like this:
def removeFirst[T](list: List[T], toRemove: (T) => Boolean): List[T] = {
def search(toProcess: List[T], processed: List[T]): List[T] =
toProcess match {
case Nil => list
case head :: tail =>
if (toRemove(head))
processed.reverse ++ tail
else
search(tail, head :: processed)
}
search(list, Nil)
}
Still, that's not exactly succinct. It seems strange that there's not an existing method that would let me do this efficiently and succinctly. So, am I missing something, or is my last solution really as good as it gets?
You can clean up the code a bit with span.
scala> def removeFirst[T](list: List[T])(pred: (T) => Boolean): List[T] = {
| val (before, atAndAfter) = list span (x => !pred(x))
| before ::: atAndAfter.drop(1)
| }
removeFirst: [T](list: List[T])(pred: T => Boolean)List[T]
scala> removeFirst(List(1, 2, 3, 4, 3, 4)) { _ == 3 }
res1: List[Int] = List(1, 2, 4, 3, 4)
The Scala Collections API overview is a great place to learn about some of the lesser known methods.
This is a case where a little bit of mutability goes a long way:
def withoutFirst[A](xs: List[A])(p: A => Boolean) = {
var found = false
xs.filter(x => found || !p(x) || { found=true; false })
}
This is easily generalized to dropping the first n items matching the predicate. (i<1 || { i = i-1; false })
You can also write the filter yourself, though at this point you're almost certainly better off using span since this version will overflow the stack if the list is long:
def withoutFirst[A](xs: List[A])(p: A => Boolean): List[A] = xs match {
case x :: rest => if (p(x)) rest else x :: withoutFirst(rest)(p)
case _ => Nil
}
and anything else is more complicated than span without any clear benefits.
I have a list l:List[T1] and currently im doing the following:
myfun : T1 -> Option[T2]
val x: Option[T2] = l.map{ myfun(l) }.flatten.find(_=>true)
The myfun function returns None or Some, flatten throws away all the None's and find returns the first element of the list if any.
This seems a bit hacky to me. Im thinking that there might exist some for-comprehension or similar that will do this a bit less wasteful or more clever.
For example: I dont need any subsequent answers if myfun returns any Some during the map of the list l.
How about:
l.toStream flatMap (myfun andThen (_.toList)) headOption
Stream is lazy, so it won't map everything in advance, but it won't remap things either. Instead of flattening things, convert Option to List so that flatMap can be used.
In addition to using toStream to make the search lazy, we can use Stream::collectFirst:
List(1, 2, 3, 4, 5, 6, 7, 8).toStream.map(myfun).collectFirst { case Some(d) => d }
// Option[String] = Some(hello)
// given def function(i: Int): Option[String] = if (i == 5) Some("hello") else None
This:
Transforms the List into a Stream in order to stop the search early.
Transforms elements using myFun as Option[T]s.
Collects the first mapped element which is not None and extract it.
Starting Scala 2.13, with the deprecation of Streams in favor of LazyLists, this would become:
List(1, 2, 3, 4, 5, 6, 7, 8).to(LazyList).map(function).collectFirst { case Some(d) => d }
Well, this is almost, but not quite
val x = (l flatMap myfun).headOption
But you are returning a Option rather than a List from myfun, so this may not work. If so (I've no REPL to hand) then try instead:
val x = (l flatMap(myfun(_).toList)).headOption
Well, the for-comprehension equivalent is pretty easy
(for(x<-l, y<-myfun(x)) yield y).headOption
which, if you actually do the the translation works out the same as what oxbow_lakes gave. Assuming reasonable laziness of List.flatmap, this is both a clean and efficient solution.
As of 2017, the previous answers seem to be outdated. I ran some benchmarks (list of 10 million Ints, first match roughly in the middle, Scala 2.12.3, Java 1.8.0, 1.8 GHz Intel Core i5). Unless otherwise noted, list and map have the following types:
list: scala.collection.immutable.List
map: A => Option[B]
Simply call map on the list: ~1000 ms
list.map(map).find(_.isDefined).flatten
First call toStream on the list: ~1200 ms
list.toStream.map(map).find(_.isDefined).flatten
Call toStream.flatMap on the list: ~450 ms
list.toStream.flatMap(map(_).toList).headOption
Call flatMap on the list: ~100 ms
list.flatMap(map(_).toList).headOption
First call iterator on the list: ~35 ms
list.iterator.map(map).find(_.isDefined).flatten
Recursive function find(): ~25 ms
def find[A,B](list: scala.collection.immutable.List[A], map: A => Option[B]) : Option[B] = {
list match {
case Nil => None
case head::tail => map(head) match {
case None => find(tail, map)
case result # Some(_) => result
}
}
}
Iterative function find(): ~25 ms
def find[A,B](list: scala.collection.immutable.List[A], map: A => Option[B]) : Option[B] = {
for (elem <- list) {
val result = map(elem)
if (result.isDefined) return result
}
return None
}
You can further speed up things by using Java instead of Scala collections and a less functional style.
Loop over indices in java.util.ArrayList: ~15 ms
def find[A,B](list: java.util.ArrayList[A], map: A => Option[B]) : Option[B] = {
var i = 0
while (i < list.size()) {
val result = map(list.get(i))
if (result.isDefined) return result
i += 1
}
return None
}
Loop over indices in java.util.ArrayList with function returning null instead of None: ~10 ms
def find[A,B](list: java.util.ArrayList[A], map: A => B) : Option[B] = {
var i = 0
while (i < list.size()) {
val result = map(list.get(i))
if (result != null) return Some(result)
i += 1
}
return None
}
(Of course, one would usually declare the parameter type as java.util.List, not java.util.ArrayList. I chose the latter here because it's the class I used for the benchmarks. Other implementations of java.util.List will show different performance - most will be worse.)