Can't get flatten of Try into a for comprehension - list

Question

This is a combination of a stylistic question, and my attempts to broaden my Scala understanding.
I've got a list containing Future's, I want to compute the values of the futures, transform into Option's, and flatten the list using a for comprehension:
import scala.util.Try
import scala.concurrent._
import ExecutionContext.Implicits.global
val l= List(Future.successful(1),Future.failed(new IllegalArgumentException))
implicit def try2Traversible[A](xo: Try[A]): Iterable[A] = xo.toOption.toList
val res = for{f <- l; v <- f.value} yield v
scala> res: List[scala.util.Try[Int]] = List(Success(1), Failure(java.lang.IllegalArgumentException))
res.flatten
res16: List[Int] = List(1)
What I want to do is get the flatten stage into the for comprehension, anyone got any suggestions?

Answer 1

Doing this is incorrect:
for{f <- l; v <- f.value} yield v
It appears to work in your case only because the futures are already fulfiled, which is why their value member is defined.
However in the general case they might not yet be fulfilled when you execute the for comprehension, and thus value will return None
(despite the fact that at some point they will eventually be fulfilled).
By example, try this in the REPL:
val f1 = Future{
Thread.sleep(3000) // Just a test to illustrate, never do this!
1
}
val f2 = Future{
Thread.sleep(3000) // Just a test to illustrate, never do this!
throw new IllegalArgumentException
}
val l = List( f1, f2 )
for{f <- l; v <- f.value} yield v
The result is an empty list, because none of the futures in l is fulfilled yet. Then wait a bit (at most 3 seconds) and reexecute the for comprehension (the last line), and you will get a non empty list because the futures have finally been fulfilled.
To fix this, you will have to either block (that is, wait for all the futures to be fulfiled) using scala.concurrent.Await, or stay in the asynchronous world by using something like Future.map or Future.flatMap.
By example, if you want to block, you could do:
Await.result( Future.sequence( l ), duration.Duration.Inf )
Await.result waits for the result of the future, allowing to go from the asynchronous world to the synchronous world. The result of the above is a List[Int]
The problem now is that you lose the failure cases (the result is not List[Try[Int]] as you wanted), and will actually rethrow the first exception.
To fix this, you can use this helper method that I posted in another answer: https://stackoverflow.com/a/15776974/1632462
Using it, you can do:
Await.result( Future.sequence( l map mapValue ), duration.Duration.Inf )
This will wait until all the futures are fulfiled (either with a correct value, or with an error) and return the expected List[Try[Int]]

Answer 2

The idea is to traverse to Try object as if it were an Option (i.e. a 0 or 1 element collection) within the for-comprehension itself.
For this traversal to work there has to be a conversion from the Try type to the Option type.
This should work:
implicit def try2option[A](xo: Try[A]) = xo.toOption
val res = for (f <- l; t <- f.value; x <- t) yield x

Answer 3

You should keep a Future around your final result to retain the asynchronous nature of the computation.
The nice way to do this (and obtain a Future[List[Int]]) would be (probably what you tried):
for {
f <- l // Extract individual future
v <- f // Extract value from future
} yield v
Unfortunately this translates to:
l.flatMap(f => f.map(v => v))
Which does not work, because Future does not inherit GenTraversableOnce (and probably shouldn't), but List needs this trait for its flatMap.
However, we can do this manually:
val res = l.foldRight(Future.successful(List.empty[Int])) {
case (x,xs) => xs.flatMap(vxs => x.map(vx => vx :: vxs))
}
We can use Future.sequence to do that:
Future.sequence(l)
This will return a Future[List[Int]] which only completes when all futures are completed and will contain all values of the futures that completed successfully.