How to parallelize this groovy code? - concurrency

I'm trying to write a reusable component in Groovy to easily shoot off emails from some of our Java applications. I would like to pass it a List, where Email is just a POJO(POGO?) with some email info. I'd like it to be multithreaded, at least running all the email logic in a second thread, or make one thread per email.
I am really foggy on multithreading in Java so that probably doesn't help! I've attempted a few different ways, but here is what I have right now:
void sendEmails(List<Email> emails) {
def threads = []
def sendEm = emails.each{ email ->
def th = new Thread({
Random rand = new Random()
def wait = (long)(rand.nextDouble() * 1000)
println "in closure"
this.sleep wait
sendEmail(email)
})
println "putting thread in list"
threads << th
}
threads.each { it.run() }
threads.each { it.join() }
}
I was hoping the sleep would randomly slow some threads down so the console output wouldn't be sequential. Instead, I see this:
putting thread in list
putting thread in list
putting thread in list
putting thread in list
putting thread in list
putting thread in list
putting thread in list
putting thread in list
putting thread in list
putting thread in list
in closure
sending email1
in closure
sending email2
in closure
sending email3
in closure
sending email4
in closure
sending email5
in closure
sending email6
in closure
sending email7
in closure
sending email8
in closure
sending email9
in closure
sending email10
sendEmail basically does what you'd expect, including the println statement, and the client that calls this follows,
void doSomething() {
Mailman emailer = MailmanFactory.getExchangeEmailer()
def to = ["one","two"]
def from = "noreply"
def li = []
def email
(1..10).each {
email = new Email(to,null,from,"email"+it,"hello")
li << email
}
emailer.sendEmails li
}

To get you example above running concurrently you have to replace the line
threads.each { it.run() }
with
threads.each { it.start() }
as run() doesn't start a new thread and thus your code was running sequentially.
There's also a Groovy extension available called GPars. It supports several concurrency techniques like Fork/Join or the Actor model. Using GPars, your code could be simplified to this:
def sendEmails(emails) {
GParsPool.withPool {
emails.eachParallel { email ->
def wait = (long) new Random().nextDouble() * 1000
println "in closure"
this.sleep wait
sendEmail(email)
}
}
}

A couple of Java versions back (1.5) they introduced some new concurrency stuff that makes Java threading (even more) simple. Google for java ThreadExecutor, and you'll find some pages such as:
http://www.deitel.com/articles/java_tutorials/20051126/JavaMultithreading_Tutorial_Part4.html
Whether Groovy makes it even simpler, I can't say, but you might want apply the "new" Java techniques to your Java example first before making the comparison.

Related

how to avoid sending messages to actors not created yet?

I hope it is ok to ask this. I am using akka and have two actors, where one is initiated/created fast and the other much slower. The rapidly created one asks the other for something (ask-pattern), and the message is sent to dead letters since the other is not initiated yet. What is the preferred way of making an actor waiting with sending it´s message? I am not so eager to make an actor sleep or something without knowing there is no other way.
I would use the functionality become()/unbecome() Akka provides for Actors. I am assuming in the following code that the slowActor gets created by the fastActor. The trick here is that the fastActor will have two behaviors: one for when the slowActor is getting initiated and the other for when it's ready to do some work. When slowActor is ready, it will send a message to the fastActor to advertise that is able to receive messages. fastActor will be watching slowActor and if it gets terminated, it will change its behavior again. What to do next would be up to your solution.
Here is a mock code as a guide (I have not compiled the code and it might contain some errors):
case object Ready
case object DoWork
case object WorkDone
class FastActor extends Actor with ActorLogging {
val slowActor = context.actorOf(SlowActor.props)
context.watch(slowActor)
def receive = slowActorNotReadyBehavior
def slowActorNotReadyBehavior = {
case DoWork => log.warning("Slow actor in not ready, I am sorry...")
case Ready => context.become(slowActorReadyBehavior)
}
def slowActorReadyBehavior = {
case DoWork => (slowActor ? DoWork).pipeTo(self)
case Terminated(ref) =>
log.error("Slow actor terminated")
context.unbecome()
//... do something with slowActor
}
}
class SlowActor extends Actor {
override def preStart = {
context.parent ! Ready
}
def receive = {
case DoWork =>
//do something
sender ! WorkDone
}
}

Does akka-typed create actors when I use behavior combinator like And?

In my system I want an actor A to send the same messages to actors B,C,D.
Instead of creating the three actors, I was thinking of just combining their behaviors with an And behavior contaminator, and then passing that behavior to A.
If I do this, how many actors will get created? Will I get just one actor with three behaviors in it, or three actors with separate behaviors?
Here is my real code using the non-And approach, for concreteness (see how ReplyGenerator gets passed the references to other actors):
object Foobar {
def foobar(): Behavior[Request] =
ContextAware[Request] {
context =>
val foo1 = context.spawn(Props(Foo1.behavior()), "foo1")
val foo2 = context.spawn(Props(Foo2.behavior()), "foo2")
val foo3 = context.spawn(Props(Foo3.behavior()), "foo3")
val generator = context.spawn(Props(ReplyGenerator.behavior(List(foo1, foo2, foo3))),
"generator")
Static {
case request: Request =>
generator ! request
}
}
}
and here is the ReplyGenerator behavior that sends the same message to all subscribers:
object ReplyGenerator {
def behavior(subscribers: List[ActorRef[Reply]]): Behavior[Request] = {
Static {
case request: Request =>
subscribers.foreach(_ ! Reply.empty)
}
}
Considering that I want the actors foo 1,2,3 to run in parallel, can And combinator be used here instead?
Thank you.
If you mean execution parallelism then you’ll have to create separate Actors (by separate spawn calls) as you do in the example code—using And will only create a single Actor that runs the contained behaviors one after the other.

Creating a lazy actor router in Akka with timeout

I have spent the last two days learning Actors, and I want to create an expiring cache. Now we use a tenant model so I want each tenant to be represented by an actor. I would like these actors to be created when required, and timeout after a period of being idle.
To solve this I have mocked up the following as I am unaware of a provided solution, and am looking for any critique or validation of the approach.
//A simple Message carrying just the name of the actor
case class Message(name:String)
//Actor that will expire after a timeout period and stop itself
class ExpireActor extends Actor {
val id = Random.nextInt(1000)
context.setReceiveTimeout(100 milliseconds)
def receive ={
case Message(_) => println("Message: " + id + " " + System.currentTimeMillis())
case ReceiveTimeout => {
println("Timeout: " + id + " " + System.currentTimeMillis())
self ! PoisonPill
}
}
}
//Router for creating actors on demand
case class LazyRouter() extends RouterConfig {
def routerDispatcher: String = Dispatchers.DefaultDispatcherId
def supervisorStrategy: SupervisorStrategy = SupervisorStrategy.defaultStrategy
def createRoute(routeeProvider: RouteeProvider): Route = {
{
case (sender, Message(name)) ⇒
routeeProvider.context
.child(name)
.map(a => List(Destination(sender, a)))
.getOrElse{
synchronized {
routeeProvider.context
.child(name) //Dont want to call sync until I have to, so need to check existence again
.map(a => List(Destination(sender, a)))
.getOrElse{
val ref = routeeProvider.context.actorOf(Props[ExpireActor], name)
routeeProvider.registerRoutees(List(ref))
List(Destination(sender, ref))
}
}
}
}
}
}
I'm not sure I'm fully on board with your approach. Actors are very lightweight. When they are not doing anything they are not costing you anything CPU wise. Why not just pre-create all of the cache actors (for each possible tenant) before hand so the router does not have to have that scary synchronized block around whether or not it needs to create the routee. Then, instead of stopping the actors when they are idle for a specific amount of time, just clear out their internal state (which I'm assuming is the cached data) if you want to free up that memory. This will greatly simplify your code and make it more reliable (and probably faster to boot).

Call multiple webservices from play 2

I am a play2.0-Scala-beginner and have to call several Webservices to generate a HTML page.
After reading the The Play WS API page and a very interesting article from Sadek Drobi I am still unsure what's the best way to accomplish this.
The article shows some code snippets which I don't fully understand as a Play beginner.
Figure 2 on page 4:
val response: Either[Response,Response] =
WS.url("http://someservice.com/post/123/comments").focusOnOk
val responseOrUndesired: Either[Result,Response] = response.left.map {
case Status(4,0,4) => NotFound
case Status(4,0,3) => NotAuthorized
case _ => InternalServerError
}
val comments: Either[Result,List[Comment]] =
responseOrUndesired.right.map(r => r.json.as[List[Comment]])
// in the controller
comment.fold(identity, cs => Ok(html.showComments(cs)))
What does the last line with the fold do? Should comment be comments? Haven't I group the last statement in an Async block?
Figure 4 shows how to combine several IO calls with a single for-expression:
for {
profile <- profilePromise
events <- attachedEventsPromise
articles <- topArticlesPromise
} yield Json.obj(
"profile" -> profile,
"events" -> events,
"articles" -> articles )
}
// in the controller
def showInfo(...) = Action { rq =>
Async {
actorInfo(...).map(info => Ok(info))
}
}
How can I use this snippet? (I am a bit confused by the extra-} after the for-expression.)
Should I write something like this?
var actorInfo = for { // Model
profile <- profilePromise
events <- attachedEventsPromise
articles <- topArticlesPromise
} yield Json.obj(
"profile" -> profile,
"events" -> events,
"articles" -> articles )
def showInfo = Action { rq => // Controller
Async {
actorInfo.map(info => Ok(info))
}
}
What's the best way to combine the snippets from figure 2 and 4 (error handling + composition of IO non-blocking calls)? (f.ex. I want to produce a Error 404 status code if any of the called webservice produce an Error 404).
Maybe someone knows a complete example of calling webservices in the play framework (cannot find an example in the play Sample applications or anywhere else).
I have to say that the article is wrong in the example you show in Figure 2. The method focusOnOk does not exist in Play 2.0. I assume the author of the article used a pre-release version of Play 2 then.
Regarding comment, yes it should be comments. The fold in the statement is operating on an Either. It takes 2 functions as parameters. The first is a function to apply if it is a left value. The second is a function to apply if it is a right value. A more detailed explanation can be found here: http://daily-scala.blogspot.com/2009/11/either.html
So what the line does is. If I have a left value (which meant I got an undesired response), apply the built-in identity function which just gives you back the value. If it has a right value (which means I got an OK response), make a new result that shows the comments somehow.
Regarding Async, it's not actually asynchronous. focusOnOk is a blocking function (a remnant from the old Java days of Play 1.x). But remember, that's not valid Play 2 code.
As for Figure 4, the trailing } is actually because it's a partial alternative of what's in Figure 3. Instead of the numerous promise flatMaps. You can do a for comprehension instead. Also, I think it should be userInfo(...).map instead of actorInfo(...).map.
The Play documentation you linked to actually already shows you a full example.
def feedTitle(feedUrl: String) = Action {
Async {
WS.url(feedUrl).get().map { response =>
Ok("Feed title: " + (response.json \ "title").as[String])
}
}
}
will get whatever is at feedUrl, and you map it to do something with the response which has a status field you can check to see if it was a 404 or something else.
To that end, the Figure 3 and 4 of your linked article should give you a starting point. So you'd have something like,
def getInfo(...) : Promise[String] = {
val profilePromise = WS.url(...).get()
val attachedEventsPromise = WS.url(...).get()
val topArticlesPromise = WS.url(...).get()
for {
profile <- profilePromise
events <- attachedEventsPromise
articles <- topArticlesPromise
} yield {
// or return whatever you want
// remember to change String to something else in the return type
profile.name
}
}
def showInfo(...) = Action { rq =>
Async {
getInfo(...).map { info =>
// convert your info to a Result
Ok(info)
}
}
}

Unit testing scala actors

Anyone know of a good way to unit test Scala actors? In the general sense I have an actor that receives a message and will send out other messages in response. This is done on multiple threads, and an actor that is not correct may either send the wrong messages or no message at all. I need a simple way of creating a mockup actor that send and receives messages to the actor being tested. Any experiences in this area?
Because of the dynamic nature of actor-style message passing, mocking actors is usually no trouble at all. Just create an actor which receives the desired message and you're home free. You will of course need to ensure that this mock actor is the one to which messages are passed, but that shouldn't be a problem as long as the actor you are attempting to test is reentrant.
I think the complexity depends on a couple factors...
How stateful is the actor?
If it behaves like a idempotent function, only asynchronous, then it should be a simple matter of mocking up an actor that sends a message and then checks that it receives the expected messages back. You probably want to use a react/receiveWithin on the mock actor in case there is response within a reasonable period of time you can fail rather than hanging.
However if the messages aren't independent of one another, then you should test it with various sequences of messages and expected results.
How many actors will the actor being tested interact with?
If an actor is expected to interact with many others, and it is stateful, then it should be tested with several actors sending/receiving messages. Since you probably have no guarantee of the order in which the messages will arrive, you should be sure to either permute the orders in which the actors send the messages or introduce random pauses in the actors generating messages and run the test many times.
I'm not aware of any prebuilt frameworks for testing actors, but you could possibly look to Erlang for inspiration.
http://svn.process-one.net/contribs/trunk/eunit/doc/overview-summary.html
I have been wondering about how to test Actors myself.
Here is what I came up with, does anybody see problems with this approach?
Rather than send messages directly, what if your actor delegated message sending to a function?
Then your tests can swap out the function with one that tracks the number of times called and/or the arguments with which the method was called:
class MyActor extends Actor {
var sendMessage:(Actor, ContactMsg) => Unit = {
(contactActor, msg) => {
Log.trace("real sendMessage called")
contactActor ! msg
}
}
var reactImpl:PartialFunction(Any, Unit) = {
case INCOMING(otherActor1, otherActor2, args) => {
/* logic to test */
if(args){
sendMessage(otherActor1, OUTGOING_1("foo"))
} else {
sendMessage(otherActor2, OUTGOING_2("bar"))
}
}
}
final def act = loop {
react {
reactImpl
}
}
Your test case might contain code like:
// setup the test
var myActor = new MyActor
var target1 = new MyActor
var target2 = new MyActor
var sendMessageCalls:List[(Actor, String)] = Nil
/*
* Create a fake implementation of sendMessage
* that tracks the arguments it was called with
* in the sendMessageCalls list:
*/
myActor.sendMessage = (actor, message) => {
Log.trace("fake sendMessage called")
message match {
case OUTGOING_1(payload) => {
sendMessageCalls = (actor, payload) :: sendMessageCalls
}
case _ => { fail("Unexpected Message sent:"+message) }
}
}
// run the test
myActor.start
myActor.reactImpl(Incoming(target1, target2, true))
// assert the results
assertEquals(1, sendMessageCalls.size)
val(sentActor, sentPayload) = sendMessageCalls(0)
assertSame(target1, sentActor)
assertEquals("foo", sentPayload)
// .. etc.
My attempt at unit testing an actor (it works). I'm using Specs as a framework.
object ControllerSpec extends Specification {
"ChatController" should{
"add a listener and respond SendFriends" in{
var res = false
val a = actor{}
val mos = {ChatController !? AddListener(a)}
mos match{
case SendFriends => res = true
case _ => res = false
}
res must beTrue
}
How this works is by sending a synchronous call to the singleton ChatController. ChatController responds by use of reply(). The response is sent as a return of the called function, which gets stored into mos. Then a match is applied to mos getting the case class that was sent from ChatController. If the result is what is expected (SendFriends) set res to true. The res must beTrue assertion determines the success or failure of test.
My actor singleton that I'm testing
import ldc.socialirc.model._
import scala.collection.mutable.{HashMap, HashSet}
import scala.actors.Actor
import scala.actors.Actor._
import net.liftweb.util.Helpers._
//Message types
case class AddListener(listener: Actor)
case class RemoveListener(listener: Actor)
case class SendFriends
//Data Types
case class Authority(usr: Actor, role: String)
case class Channel(channelName: String, password: String, creator: String, motd: String, users: HashSet[Authority])
object ChatController extends Actor {
// The Channel List - Shows what actors are in each Chan
val chanList = new HashMap[String, Channel]
// The Actor List - Shows what channels its in
val actorList = new HashMap[Actor, HashSet[String]]
def notifyListeners = {
}
def act = {
loop {
react {
case AddListener(listener: Actor)=>
actorList += listener -> new HashSet[String]
reply(SendFriends)
}
}
}
start //Dont forget to start
}
Though its not complete it does return the Sendfriends case class as expected.
Suite for unit testing of Actors has recently been added to Akka. You can find some information and code snippets in this blogpost.