Suppose I have a list List[() => Int] and need to invoke all the functions to get the list of results.
def invoke(fs: List[() => Int]): List[Int] = fs map (_())
What if fs has duplicates ? I can probably memoize the results but I need to invoke these functions concurrently. It looks like I need to do some preprocessing to make sure each function is invoked only once.
What would you suggest ?
Generically, there's no way of knowing whether two functions are equal. Even if you were looking at two copies of the same function instance, they might invoke some side effect (e.g. generating a random number) so it would be incorrect in some sense to elide the second call. In the cases where the function provably doesn't have side effects, the JVM can probably figure that out for itself. So I honestly think you're solving the wrong problem here.
But if you really want to memoize, I'd use scalaz Memo. The different kinds of Memo document what thread safety guarantees they offer.
def execute(fs: List[() => Int]) = {
val m = Memo.mutableHashMapMemo({f: (() => Int) => f()})
fs map m
}
May be using lazy:
def foo1(): Int = { println("f1"); 1 }
def foo2(): Int = { println("f2"); 2 }
def toLazy(f: () => Int): () => Int = {
lazy val res = f()
() => res
}
val f1 = toLazy(foo1)
val f2 = toLazy(foo2)
val flist = List(f1, f2, f1)
println("invoking...")
val res = invoke(flist)
println(res)
// invoking...
// f1
// f2
// List(1, 2, 1)
If f1 requests fail and you handle Exception to allow invocation chain to proceed, lazy for f1 will be initialized by the first successful response for f1.
Related
val list1 : List[String] = List(....)
val list2 : List[String] = List(....)
val filtered = list1.filter(list2.toSet.contains)
When using the code above, will toSet be called for each item in list1 (in which case I should add another val set2 = list2.toSet, or does the Scala compiler optimizes this in which case the code above will only run toSet once?
There are no optimizations needed, toSet can only run once:
eta-expansion rules for converting list2.toSet.contains to a lambda. In this specific case the translation is
list1.filter({
val x1 = list2.toSet
{ (y1: String) => x1.contains(y1) }
})
the argument to a method is evaluated before calling it (except for call-by-name parameters, but this isn't one), and to evaluate this block val x1 = list2.toSet is executed first and then { (y1: String) => x1.contains(y1) } is returned.
But of course you can be more explicit if you want.
To make sure it is called once, you can just do:
val list1 : List[String] = List(....)
val list2 : List[String] = List(....)
val s: Set[String] = list2.toSet
val filtered = list1.filter(s.contains)
Having said that, you can use intersect method:
list1.intersect(list2)
This method is located on a trait called StrictOptimizedSeqOps, and it has comment in its implementation: // Avoid multiple map lookups So I guess this is optimized.
fun count_wcs p =
let
val count = 0
in
g (fn () => count + 1) (fn y => 1) p
end
I'm doing homework and we're not supposed to use mutations, this doesn't reassign the value to anything but it doesn't feel right. Please, don't say what is the right way to do this, because I'm supposed to figure this out.
datatype pattern = Wildcard
| Variable of string
| UnitP
| ConstP of int
| TupleP of pattern list
| ConstructorP of string * pattern
fun g f1 f2 p =
let
val r = g f1 f2
in
case p of
Wildcard => f1 ()
| Variable x => f2 x
| TupleP ps => List.foldl (fn (p,i) => (r p) + i) 0 ps
| ConstructorP(_,p) => r p
| _ => 0
end
This function g has to receive a type unit -> int function as the first argument. I checked the count after I called the function and got 0, so it is ok to write it like this, right? But still, it does feel sloppy.
Added the context (function g and the datatype used). The function count_wcs is supposed to count the number a Wildcard pattern appears in a pattern.
This does not count as a mutation, but it does resemble what you might do if you had them and is probably not going to work, depending on what it is you're doing. Mutations require references and they're made with ref and are de-referenced with !. So just stick away from those. :-)
You're doing something that will be of little benefit:
let
val count = 0
in
...
end
will bind count to 0, but will never cause count to have any other value; I presume that you want to eventually have that count increases. If it had been a reference, val count = ref 0, you could increment it by doing count := !count + 1, but since it's not, you have to make count a variable of some function for it to change.
For example:
fun count_4s ([], count) = count
| count_4s (x::xs, count) = count_4s (xs, if x = 4 then count + 1 else count)
In each call count is constant, but in each subsequent, recursive call it is possibly incremented.
This function g has to receive a type unit -> int function as first argument.
...
g (fn () => count + 1) (fn y => 1) p
...
[...] But still it does feel sloppy.
Assuming that g's first arguments have no side-effects and do not throw exceptions and do not loop forever, all they can do is return the same thing on every call. This makes them rather boring. Typically functions that either take as input or return () : unit do something else, like read or write from a source external to the program.
I wouldn't call it sloppy. Just a bit weird, without knowing the context.
It's very easy to identify mutation in standard ML - if you don't use ref variables and assign them new values with :=, it's not mutation.
And you don't, so you aren't mutating anything.
Also, your function can be simplified to
fun count_wildcards p = g (fn () => 1) (fn _ => 1) p
which makes it even clearer that nothing gets mutated.
(It's not at all clear what it's supposed to accomplish, so it's impossible to tell whether your solution is correct.)
This is a combination of a stylistic question, and my attempts to broaden my Scala understanding.
I've got a list containing Future's, I want to compute the values of the futures, transform into Option's, and flatten the list using a for comprehension:
import scala.util.Try
import scala.concurrent._
import ExecutionContext.Implicits.global
val l= List(Future.successful(1),Future.failed(new IllegalArgumentException))
implicit def try2Traversible[A](xo: Try[A]): Iterable[A] = xo.toOption.toList
val res = for{f <- l; v <- f.value} yield v
scala> res: List[scala.util.Try[Int]] = List(Success(1), Failure(java.lang.IllegalArgumentException))
res.flatten
res16: List[Int] = List(1)
What I want to do is get the flatten stage into the for comprehension, anyone got any suggestions?
Doing this is incorrect:
for{f <- l; v <- f.value} yield v
It appears to work in your case only because the futures are already fulfiled, which is why their value member is defined.
However in the general case they might not yet be fulfilled when you execute the for comprehension, and thus value will return None
(despite the fact that at some point they will eventually be fulfilled).
By example, try this in the REPL:
val f1 = Future{
Thread.sleep(3000) // Just a test to illustrate, never do this!
1
}
val f2 = Future{
Thread.sleep(3000) // Just a test to illustrate, never do this!
throw new IllegalArgumentException
}
val l = List( f1, f2 )
for{f <- l; v <- f.value} yield v
The result is an empty list, because none of the futures in l is fulfilled yet. Then wait a bit (at most 3 seconds) and reexecute the for comprehension (the last line), and you will get a non empty list because the futures have finally been fulfilled.
To fix this, you will have to either block (that is, wait for all the futures to be fulfiled) using scala.concurrent.Await, or stay in the asynchronous world by using something like Future.map or Future.flatMap.
By example, if you want to block, you could do:
Await.result( Future.sequence( l ), duration.Duration.Inf )
Await.result waits for the result of the future, allowing to go from the asynchronous world to the synchronous world. The result of the above is a List[Int]
The problem now is that you lose the failure cases (the result is not List[Try[Int]] as you wanted), and will actually rethrow the first exception.
To fix this, you can use this helper method that I posted in another answer: https://stackoverflow.com/a/15776974/1632462
Using it, you can do:
Await.result( Future.sequence( l map mapValue ), duration.Duration.Inf )
This will wait until all the futures are fulfiled (either with a correct value, or with an error) and return the expected List[Try[Int]]
The idea is to traverse to Try object as if it were an Option (i.e. a 0 or 1 element collection) within the for-comprehension itself.
For this traversal to work there has to be a conversion from the Try type to the Option type.
This should work:
implicit def try2option[A](xo: Try[A]) = xo.toOption
val res = for (f <- l; t <- f.value; x <- t) yield x
You should keep a Future around your final result to retain the asynchronous nature of the computation.
The nice way to do this (and obtain a Future[List[Int]]) would be (probably what you tried):
for {
f <- l // Extract individual future
v <- f // Extract value from future
} yield v
Unfortunately this translates to:
l.flatMap(f => f.map(v => v))
Which does not work, because Future does not inherit GenTraversableOnce (and probably shouldn't), but List needs this trait for its flatMap.
However, we can do this manually:
val res = l.foldRight(Future.successful(List.empty[Int])) {
case (x,xs) => xs.flatMap(vxs => x.map(vx => vx :: vxs))
}
We can use Future.sequence to do that:
Future.sequence(l)
This will return a Future[List[Int]] which only completes when all futures are completed and will contain all values of the futures that completed successfully.
I have a list l:List[T1] and currently im doing the following:
myfun : T1 -> Option[T2]
val x: Option[T2] = l.map{ myfun(l) }.flatten.find(_=>true)
The myfun function returns None or Some, flatten throws away all the None's and find returns the first element of the list if any.
This seems a bit hacky to me. Im thinking that there might exist some for-comprehension or similar that will do this a bit less wasteful or more clever.
For example: I dont need any subsequent answers if myfun returns any Some during the map of the list l.
How about:
l.toStream flatMap (myfun andThen (_.toList)) headOption
Stream is lazy, so it won't map everything in advance, but it won't remap things either. Instead of flattening things, convert Option to List so that flatMap can be used.
In addition to using toStream to make the search lazy, we can use Stream::collectFirst:
List(1, 2, 3, 4, 5, 6, 7, 8).toStream.map(myfun).collectFirst { case Some(d) => d }
// Option[String] = Some(hello)
// given def function(i: Int): Option[String] = if (i == 5) Some("hello") else None
This:
Transforms the List into a Stream in order to stop the search early.
Transforms elements using myFun as Option[T]s.
Collects the first mapped element which is not None and extract it.
Starting Scala 2.13, with the deprecation of Streams in favor of LazyLists, this would become:
List(1, 2, 3, 4, 5, 6, 7, 8).to(LazyList).map(function).collectFirst { case Some(d) => d }
Well, this is almost, but not quite
val x = (l flatMap myfun).headOption
But you are returning a Option rather than a List from myfun, so this may not work. If so (I've no REPL to hand) then try instead:
val x = (l flatMap(myfun(_).toList)).headOption
Well, the for-comprehension equivalent is pretty easy
(for(x<-l, y<-myfun(x)) yield y).headOption
which, if you actually do the the translation works out the same as what oxbow_lakes gave. Assuming reasonable laziness of List.flatmap, this is both a clean and efficient solution.
As of 2017, the previous answers seem to be outdated. I ran some benchmarks (list of 10 million Ints, first match roughly in the middle, Scala 2.12.3, Java 1.8.0, 1.8 GHz Intel Core i5). Unless otherwise noted, list and map have the following types:
list: scala.collection.immutable.List
map: A => Option[B]
Simply call map on the list: ~1000 ms
list.map(map).find(_.isDefined).flatten
First call toStream on the list: ~1200 ms
list.toStream.map(map).find(_.isDefined).flatten
Call toStream.flatMap on the list: ~450 ms
list.toStream.flatMap(map(_).toList).headOption
Call flatMap on the list: ~100 ms
list.flatMap(map(_).toList).headOption
First call iterator on the list: ~35 ms
list.iterator.map(map).find(_.isDefined).flatten
Recursive function find(): ~25 ms
def find[A,B](list: scala.collection.immutable.List[A], map: A => Option[B]) : Option[B] = {
list match {
case Nil => None
case head::tail => map(head) match {
case None => find(tail, map)
case result # Some(_) => result
}
}
}
Iterative function find(): ~25 ms
def find[A,B](list: scala.collection.immutable.List[A], map: A => Option[B]) : Option[B] = {
for (elem <- list) {
val result = map(elem)
if (result.isDefined) return result
}
return None
}
You can further speed up things by using Java instead of Scala collections and a less functional style.
Loop over indices in java.util.ArrayList: ~15 ms
def find[A,B](list: java.util.ArrayList[A], map: A => Option[B]) : Option[B] = {
var i = 0
while (i < list.size()) {
val result = map(list.get(i))
if (result.isDefined) return result
i += 1
}
return None
}
Loop over indices in java.util.ArrayList with function returning null instead of None: ~10 ms
def find[A,B](list: java.util.ArrayList[A], map: A => B) : Option[B] = {
var i = 0
while (i < list.size()) {
val result = map(list.get(i))
if (result != null) return Some(result)
i += 1
}
return None
}
(Of course, one would usually declare the parameter type as java.util.List, not java.util.ArrayList. I chose the latter here because it's the class I used for the benchmarks. Other implementations of java.util.List will show different performance - most will be worse.)
Can you create a list of functions and then execute them sequentially, perhaps passing them into do notation?
I'm currently doing this by mapping over a list of data and am wondering if I can call somehow pass the result as a series of sequential calls?
Something like this?
sequence [putStrLn "Hello", putStrLn "World"]
If these are functions, ie pure, then you can use ($) or "apply":
execute functions argument = map ($argument) functions
-- execute [id,(1+),(1-)] 12 => [12,13,-11]
There's no guarantee that this happens sequentially of course, but you'll get a list of the return values.
If these are actions, ie impure, then what you want is called sequence_:
sequence_ [putStr "Hello", putStr " world", putStrLn "!"]
sequence_ is pretty easy to write yourself:
sequence_ [] = return ()
sequence_ (action:actions) = action >> sequence_ actions
There is also a sequence (without the underscore) that runs a bunch of actions and returns their results:
main = do
ss <- sequence [readFile "foo.txt", readFile "bar.txt", readFile "baz.txt"]
-- or ss <- mapM readFile ["foo.txt", "bar.txt", "baz.txt"]
good answers so far, but if you also want each function to act not on the original data but on the result of the previous function, look at the foldding functions, such as foldl, foldl1, and foldr:
fns = [(1-), (+2), (abs), (+1)]
helperFunction a f = f a
test1 n = foldl helperFunction n fns
and you may need the monadic version, foldM and foldM_ :
import Control.Monad
import Data.Char
helperFunction a f = f a
prnt = \s-> do putStrLn s; return s
actions = [return, prnt, return.reverse, prnt, return.(map toUpper), prnt, return.reverse, prnt]
test2 str = foldM_ helperFunction str actions