Akka-stream and delegating processing to an actor - akka

I have the following case, where I'm trying to delegate processing to an actor. What I want to happen is that whenever my flow processes a message, it sends it to the actor, and the actor will uppercase it and write it to the stream as a response.
So I should be able to connect to port 8000, type in "hello", have the flow send it to the actor, and have the actor publish it back to the stream so it's echoed back to me uppercased. The actor itself is pretty basic, from the ActorPublisher example in the docs.
I know this code doesn't work, I cleaned up my experiments to get it to compile. Right now, it's just two separate streams. I tried to experiment with merging the sources or the sinks, to no avail.
object Sample {
def main(args: Array[String]): Unit = {
implicit val system = ActorSystem("sample")
implicit val materializer = ActorMaterializer()
val connections: Source[IncomingConnection,
Future[ServerBinding]] = Tcp().bind("localhost", 8000)
val filter = Source.actorPublisher[ByteString](Props[Filter])
val filterRef = Flow[ByteString]
.to(Sink.ignore)
.runWith(filter)
connections runForeach { conn =>
val echo = Flow[ByteString] .map {
// would like to send 'p' to the actor,
// and have it publish to the stream
case p:ByteString => filterRef ! p
}
}
}
}
// this actor is supposed to simply uppercase all
// input and write it to the stream
class Filter extends ActorPublisher[ByteString] with Actor
{
var buf = Vector.empty[ByteString]
val delay = 0
def receive = {
case p: ByteString =>
if (buf.isEmpty && totalDemand > 0)
onNext(p)
else {
buf :+= ByteString(p.utf8String.toUpperCase)
deliverBuf()
}
case Request(_) =>
deliverBuf()
case Cancel =>
context.stop(self)
}
#tailrec final def deliverBuf(): Unit =
if (totalDemand > 0) {
if (totalDemand <= Int.MaxValue) {
val (use, keep) = buf.splitAt(totalDemand.toInt)
buf = keep
use foreach onNext
} else {
val (use, keep) = buf.splitAt(Int.MaxValue)
buf = keep
use foreach onNext
deliverBuf()
}
}
}

I've had this problem too before, solved it in a bit of roundabout way, hopefully you're ok with this way. Essentially, it involves creating a sink that immediately forwards the messages it gets to the src actor.
Of course, you can use a direct flow (commented it out), but I guess that's not the point of this exercise :)
object Sample {
def main(args: Array[String]): Unit = {
implicit val system = ActorSystem("sample")
implicit val materializer = ActorMaterializer()
val connections: Source[IncomingConnection,
Future[ServerBinding]] = Tcp().bind("localhost", 8000)
def filterProps = Props[Filter]
connections runForeach { conn =>
val actorRef = system.actorOf(filterProps)
val snk = Sink.foreach[ByteString]{s => actorRef ! s}
val src = Source.fromPublisher(ActorPublisher[ByteString](actorRef))
conn.handleWith(Flow.fromSinkAndSource(snk, src))
// conn.handleWith(Flow[ByteString].map(s => ByteString(s.utf8String.toUpperCase())))
}
}
}
// this actor is supposed to simply uppercase all
// input and write it to the stream
class Filter extends ActorPublisher[ByteString]
{
import akka.stream.actor.ActorPublisherMessage._
var buf = mutable.Queue.empty[String]
val delay = 0
def receive = {
case p: ByteString =>
buf += p.utf8String.toUpperCase
deliverBuf()
case Request(n) =>
deliverBuf()
case Cancel =>
context.stop(self)
}
def deliverBuf(): Unit = {
while (totalDemand > 0 && buf.nonEmpty) {
val s = ByteString(buf.dequeue() + "\n")
onNext(s)
}
}

Related

akka actor current receive function name

i have a simple actor with 2 behavior
package com.hello
import akka.actor.{Actor, ActorLogging}
case object Ping
class Hello extends Actor with ActorLogging {
import context._
def receive: Receive = behaviorFoo
self ! Ping
def behaviorFoo: Receive = {
case _ =>
log.info(this.receive.getClass.getSimpleName)
become(behaviorBar)
self ! Ping
}
def behaviorBar: Receive = {
case _ =>
log.info(this.receive.getClass.getSimpleName)
}
}
this thing do those:
ping self
logging current receive function
change behavior to behaviorBar
ping self
logging current receive function
in both cases it logs "$anonfun$behaviorFoo$1"
why it is not "$anonfun$behaviorBar$1" in second log?
if i change code to
self ! Ping
def receive: Receive = {
case _ =>
log.info(this.receive.getClass.getSimpleName)
become(behaviorFoo)
self ! Ping
}
def behaviorFoo: Receive = {
case _ =>
log.info(this.receive.getClass.getSimpleName)
become(behaviorBar)
self ! Ping
}
def behaviorBar: Receive = {
case _ =>
log.info(this.receive.getClass.getSimpleName)
}
it logs 3 times "$anonfun$receive$1"
is exist any way to get current behavior (Receive) function name?
or i need to hardwrite like log.info("behaviorFoo") any time?
update:
for logging issues, i added
trait MyLogging extends ActorLogging {
this: Actor ⇒
private[this] val actorClassName = this.getClass.getSimpleName
private[this] var receiveName: String = {
val receiveClassName = s"${this.receive.getClass.getSimpleName}"
val left = receiveClassName.substring(0, receiveClassName.lastIndexOf("$"))
left.substring(left.lastIndexOf("$") + 1)
}
def become(behavior: Actor.Receive): Unit = {
val behaviorClassName = behavior.getClass.getSimpleName
val left = behaviorClassName.substring(0, behaviorClassName.lastIndexOf("$"))
receiveName = left.substring(left.lastIndexOf("$") + 1)
context.become(behavior)
}
def info(message: Any): Unit = log.info(s"$actorClassName : $receiveName got $message")
}
then, my actor code become
class Hello extends Actor with MyLogging {
def receive: Receive = behaviorFoo
self ! Ping
def behaviorFoo: Receive = {
case any =>
info(any)
become(behaviorBar)
self ! Ping
}
def behaviorBar: Receive = {
case any => info(any)
}
}
now logs looks like
... Hello : behaviorFoo got Ping
... Hello : behaviorBar got Ping
Maybe this is why those are called anonymous functions. The behaviorFoo, behaviorBar, ... names have no influence on those anonymous functions. To illustrate this point, consider this:
// this is a val!
val behaviorFoo: Receive = {
case _ =>
log.info(this.receive.getClass.getSimpleName)
become(behaviorBar)
self ! Ping
}
// this returns the same anonymous function
def behaviorFoo2 = behaviorFoo
With the above, you should see that the name under which you store the anonymous function has nothing to do with the anonymous function itself...
Now, if you realize what those anonymous functions are (they are partial functions, aliased with type Actor.Receive) you could do something like below:
// not an anonymous function anymore
class BehaviourFoo extends Actor.Receive {
val anonymousFun: Actor.Receive = {
case _ =>
log.info(this.receive.getClass.getSimpleName)
become(behaviorBar)
self ! Ping
}
override def isDefinedAt(x: Any) = anonymousFun.isDefinedAt(x)
override def apply(x: Any) = anonymousFun(x)
}
// again, the name behaviorFoo doesn't matter
def behaviorFoo: Receive = new BehaviourFoo
It is certainly NOT worth the hassle, but it should help you understand what is happening.

Akka Kafka stream supervison strategy not working

I am running an Akka Streams Kafka application and I want to incorporate the supervision strategy on the stream consumer such that if the broker goes down, and the stream consumer dies after a stop timeout, the supervisor can restart the consumer.
Here is my complete code:
UserEventStream:
import akka.actor.{Actor, PoisonPill, Props}
import akka.kafka.{ConsumerSettings, Subscriptions}
import akka.kafka.scaladsl.Consumer
import akka.stream.scaladsl.Sink
import akka.util.Timeout
import org.apache.kafka.clients.consumer.ConsumerConfig
import org.apache.kafka.common.serialization.{ByteArrayDeserializer, StringDeserializer}
import scala.concurrent.duration._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.{Failure, Success}
import akka.pattern.ask
import akka.stream.ActorMaterializer
class UserEventStream extends Actor {
val settings = Settings(context.system).KafkaConsumers
implicit val timeout: Timeout = Timeout(10 seconds)
implicit val materializer = ActorMaterializer()
override def preStart(): Unit = {
super.preStart()
println("Starting UserEventStream....s")
}
override def receive = {
case "start" =>
val consumerConfig = settings.KafkaConsumerInfo
println(s"ConsumerConfig with $consumerConfig")
startStreamConsumer(consumerConfig("UserEventMessage" + ".c" + 1))
}
def startStreamConsumer(config: Map[String, String]) = {
println(s"startStreamConsumer with config $config")
val consumerSource = createConsumerSource(config)
val consumerSink = createConsumerSink()
val messageProcessor = context.actorOf(Props[MessageProcessor], "messageprocessor")
println("START: The UserEventStream processing")
val future =
consumerSource
.mapAsync(parallelism = 50) { message =>
val m = s"${message.record.value()}"
messageProcessor ? m
}
.runWith(consumerSink)
future.onComplete {
case Failure(ex) =>
println("FAILURE : The UserEventStream processing, stopping the actor.")
self ! PoisonPill
case Success(ex) =>
}
}
def createConsumerSource(config: Map[String, String]) = {
val kafkaMBAddress = config("bootstrap-servers")
val groupID = config("groupId")
val topicSubscription = config("subscription-topic").split(',').toList
println(s"Subscriptiontopics $topicSubscription")
val consumerSettings = ConsumerSettings(context.system, new ByteArrayDeserializer, new StringDeserializer)
.withBootstrapServers(kafkaMBAddress)
.withGroupId(groupID)
.withProperty(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest")
.withProperty(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "true")
Consumer.committableSource(consumerSettings, Subscriptions.topics(topicSubscription: _*))
}
def createConsumerSink() = {
Sink.foreach(println)
}
}
StreamProcessorSupervisor (this is the supervisor class of the UserEventStream class):
import akka.actor.{Actor, Props}
import akka.pattern.{Backoff, BackoffSupervisor}
import akka.stream.ActorMaterializer
import stream.StreamProcessorSupervisor.StartClient
import scala.concurrent.duration._
object StreamProcessorSupervisor {
final case object StartSimulator
final case class StartClient(id: String)
def props(implicit materializer: ActorMaterializer) =
Props(classOf[StreamProcessorSupervisor], materializer)
}
class StreamProcessorSupervisor(implicit materializer: ActorMaterializer) extends Actor {
override def preStart(): Unit = {
self ! StartClient(self.path.name)
}
def receive: Receive = {
case StartClient(id) =>
println(s"startCLient with id $id")
val childProps = Props(classOf[UserEventStream])
val supervisor = BackoffSupervisor.props(
Backoff.onFailure(
childProps,
childName = "usereventstream",
minBackoff = 1.second,
maxBackoff = 1.minutes,
randomFactor = 0.2
)
)
context.actorOf(supervisor, name = s"$id-backoff-supervisor")
val userEventStrean = context.actorOf(Props(classOf[UserEventStream]),"usereventstream")
userEventStrean ! "start"
}
}
App (the main application class):
import akka.actor.{ActorSystem, Props}
import akka.stream.ActorMaterializer
object App extends App {
implicit val system = ActorSystem("stream-test")
implicit val materializer = ActorMaterializer()
system.actorOf(StreamProcessorSupervisor.props,"StreamProcessorSupervisor")
}
application.conf:
kafka {
consumer {
num-consumers = "1"
c1 {
bootstrap-servers = "localhost:9092"
bootstrap-servers = ${?KAFKA_CONSUMER_ENDPOINT1}
groupId = "localakkagroup1"
subscription-topic = "test"
subscription-topic = ${?SUBSCRIPTION_TOPIC1}
message-type = "UserEventMessage"
poll-interval = 50ms
poll-timeout = 50ms
stop-timeout = 30s
close-timeout = 20s
commit-timeout = 15s
wakeup-timeout = 10s
max-wakeups = 10
use-dispatcher = "akka.kafka.default-dispatcher"
kafka-clients {
enable.auto.commit = true
}
}
}
}
After running the application, I purposely killed the Kafka broker and then found that after 30 seconds, the actor is stopping itself by sending a poison pill. But strangely it doesn't restart as mentioned in the BackoffSupervisor strategy.
What could be the issue here?
There are two instances of UserEventStream in your code: one is the child actor that the BackoffSupervisor internally creates with the Props that you pass to it, and the other is the val userEventStrean that is a child of StreamProcessorSupervisor. You're sending the "start" message to the latter, when you should be sending that message to the former.
You don't need val userEventStrean, because the BackoffSupervisor creates the child actor. Messages sent to the BackoffSupervisor are forwarded to the child, so to send a "start" message to the child, send it to the BackoffSupervisor:
class StreamProcessorSupervisor(implicit materializer: ActorMaterializer) extends Actor {
override def preStart(): Unit = {
self ! StartClient(self.path.name)
}
def receive: Receive = {
case StartClient(id) =>
println(s"startCLient with id $id")
val childProps = Props[UserEventStream]
val supervisorProps = BackoffSupervisor.props(...)
val supervisor = context.actorOf(supervisorProps, name = s"$id-backoff-supervisor")
supervisor ! "start"
}
}
The other issue is that when an actor receives a PoisonPill, that's not the same thing as that actor throwing an exception. Therefore, Backoff.onFailure won't be triggered when UserEventStream sends itself a PoisonPill. A PoisonPill stops the actor, so use Backoff.onStop instead:
val supervisorProps = BackoffSupervisor.props(
Backoff.onStop( // <--- use onStop
childProps,
...
)
)
val supervisor = context.actorOf(supervisorProps, name = s"$id-backoff-supervisor")
supervisor ! "start"

call akka scheduler.scheduleOnce in actor did not work

I want to use akka system's scheduler to do some thing interval
system.scheduler.scheduleOnce(interval.milliseconds, dayActor, DaySwitch)
I works fine, dayActor will receive DaySwitch message.
in dayActor
def receive = {
case DaySwitch =>
log.info("begin day switch")
context.system.scheduler.scheduleOnce(1.day, self, DaySwitch)
after one day, the dayActor didn't receive DaySwitch message.
How to fix it. Thanks!
I suppose that you have troubles with starting of scheduler or in place which serve interaction with receive event (please provide the whole code).
case object PullCounter
case class PullResult(counter: Int)
case object PullFailed
class PullActor extends Actor {
val period = 2.seconds
var timerCancellable: Option[Cancellable] = None
def scheduleTimer() = {
timerCancellable = Some(
context.system.scheduler.scheduleOnce(
period, context.self, PullCounter
)
)
}
override def preStart() = scheduleTimer()
// so we don't call preStart and schedule a new message
// see http://doc.akka.io/docs/akka/2.2.4/scala/howto.html
override def postRestart(reason: Throwable) = {}
def receive = LoggingReceive {
case PullCounter =>
val fReq = Database.fakeRequest()
fReq.map(counter => PullResult(counter)) pipeTo self
fReq.onFailure{ case _ => self ! PullFailed }
case PullFailed =>
scheduleTimer()
case r: PullResult =>
if(r.counter >= 5) {
context.system.shutdown()
} else {
scheduleTimer()
}
}
}
I suppose, that example like above can help you (at least, it helped me with the similar situation). I took it from this

message goes to dead letter instead of sender (akka router) [scala]

Actually I`m having trouble with getting my actor (router) system to work correctly.
My Setup:
I`m trying to use an akka router within an play controller. For dependency injection I use scaldi.
scaldi module:
class UserDAOModule extends Module {
binding to new ExampleRouter
binding toProvider new UserDAOWorker
}
akka router:
class UserDAORouter(implicit inj:Injector) extends Actor with AkkaInjectable {
val userDAOProps = injectActorProps[UserDAOWorker]
var router = {
val routees = Vector.fill(5) {
val r = context.actorOf(userDAOProps)
context watch r
ActorRefRoutee(r)
}
Router(RoundRobinRoutingLogic(), routees)
}
override def receive: Receive = {
case mm: MongoDBMessage =>
router.route(mm, sender)
case Terminated(a) =>
router = router.removeRoutee(a)
val r = context.actorOf(userDAOProps)
context watch r
router = router.addRoutee(r)
}
}
worker:
class UserDAOWorker(implicit inj:Injector) extends Actor with Injectable {
val db = inject[DefaultDB]
val collection:JSONCollection = db("users")
val currentSender = sender
override def receive: Receive = {
case InsertUser(user) => insertUser(user)
}
def insertUser(user:User) = {
collection.save(user).onComplete {
case Failure(e) => currentSender ! new UserDAOReturnMessage(Some(e), None)
case Success(lastError) => currentSender ! new UserDAOReturnMessage(None, lastError)
}
}
}
When I send a message (insertUser message) to the router, it is routed correctly and the worker receives the message, but when the worker sends a message back to the sender it cant be delivered, so it is send to dead letter office. I cant figure out how to fix this. Is there someone able to help me?
Thanks in advance
I guess the problem is that the currentSender is initialized with null (i.e. ActorRef.noSender) in the constructor on actor creation. 'sender' is only valid in context of receiving a message in receive(). Sending a message to ActorRef.noSender is an equivalent of sending to dead letter queue.
Something like this should work:
class UserDAOWorker(implicit inj:Injector) extends Actor with Injectable {
val db = inject[DefaultDB]
val collection:JSONCollection = db("users")
override def receive: Receive = {
case InsertUser(user) => {
insertUser(sender, user)
}
}
def insertUser(currentSender : ActorRef, user:User) = {
collection.save(user).onComplete {
case Failure(e) => currentSender ! new UserDAOReturnMessage(Some(e), None)
case Success(lastError) => currentSender ! new UserDAOReturnMessage(None, lastError)
}
}
}

Testing actor system

i have a next code
class MySystem(outerResourse: OuterResourse) extends Actor {
val firstActor = context.actorOf(Props(new FirstActor(outerResourse)), "first")
val secondActor = context.actorOf(Props(new SecondActor(firstActor)), "second")
secondActor ! Go
def receive = {
case x: AnotherMessage => printl(s"another message: $x")
case x => println(x)
}
}
class FirstActor(outerResourse: OuterResourse) extends Actor {
def receive = {
case Test =>
context.parent ! AnotherMessage
sender ! "ok"
}
}
class SecondActor(firstActor: ActorRef) extends Actor {
def receive = {
case Go => firstActor ! Test
case "ok" => println("ok")
}
}
Here OuterResourse is a any resourse - file, internet connection...
i would like to check a behavior, but i in embarrassment, i do not know how to check that second actor will be got a "ok", and mySystem actor will be got a AnotherMessage
class MyTest(_system: ActorSystem) extends TestKit(_system)
with ImplicitSender with FunSpecLike with Matchers {
def this() = this(ActorSystem("myTest"))
val outerResourse = new OuterResourse()
val mySystem = system.actorOf(Props(new MySytem(outerResourse)))
describe("Actors") {
it("should get AnotherMessage message and ok message") {
???
}
}
}
Hot to check, that a secondActor got a "ok" message?
You could set custom OutputStream for println usnig Console.withOut. It could be mock and "wait" for OK message from first actor. But it is not very nice....
// edit: please read documentation http://doc.akka.io/docs/akka/snapshot/scala/testing.html
Akka provide they owm testing "framework"