Using Thread.sleep() inside an foreach in scala - web-services

Question

I've a list of URLs inside a List.
I want to get the data by calling WS.url(currurl).get(). However, I want add a delay between each request. Can I add Thread.sleep() ? or is there another way of doing this?
one.foreach {
currurl => {
import play.api.libs.ws.WS
println("using " + currurl)
val p = WS.url(currurl).get()
p.onComplete {
case Success(s) => {
//do something
}
case Failure(f) => {
println("failed")
}
}
}
}

Answer 1

Sure, you can call Thread.sleep inside your foreach function, and it will do what you expect.
That will tie up a thread, though. If this is just some utility that you need to run sometimes, then who cares, but if it's part of some server you are trying to write and you might tie up many threads, then you probably want to do better. One way you could do better is to use Akka (it looks like you are using Play, so you are already using Akka) to implement the delay -- write an actor that uses scheduler.schedule to arrange to receive a message periodically, and then handle one request each time the message is read. Note that Akka's scheduler itself ties up a thread, but it can then send periodic messages to an arbitrary number of actors.

Answer 2

You can do it with scalaz-stream
import org.joda.time.format.DateTimeFormat
import scala.concurrent.duration._
import scalaz.stream._
import scalaz.stream.io._
import scalaz.concurrent.Task
type URL = String
type Fetched = String
val format = DateTimeFormat.mediumTime()
val urls: Seq[URL] =
"http://google.com" :: "http://amazon.com" :: "http://yahoo.com" :: Nil
val fetchUrl = channel[URL, Fetched] {
url => Task.delay(s"Fetched " +
s"url:$url " +
s"at: ${format.print(System.currentTimeMillis())}")
}
val P = Process
val process =
(P.awakeEvery(1.second) zipWith P.emitAll(urls))((b, url) => url).
through(fetchUrl)
val fetched = process.runLog.run
fetched.foreach(println)
Output:
Fetched url:http://google.com at: 1:04:25 PM
Fetched url:http://amazon.com at: 1:04:26 PM
Fetched url:http://yahoo.com at: 1:04:27 PM