I am looking to build a system that is able to process a stream of requests that needs a long processing time say 5 min each. My goal is to speed up request processing with minimal resource footprint which at times can be a burst of messages.
I can use something like a service bus to queue the request and have multiple process (a.k.a Actors in akka) that can subscribe for a message and start processing. Also can have a watchdog that looks at the queue length in the service bus and create more actors/ actor systems or stop a few.
if I want to do the same in the Actor system like Akka.net how can this be done. Say something like this:
I may want to spin up/stop new Remote Actor systems based on my request queue length
Send the message to any one of the available actor who can start processing without having to check who has the bandwidth to process on the sender side.
Messages should not be lost, and if the actor fails, it should be passed to next available actor.
can this be done with the Akka.net or this is not a valid use case for the actor system. Can some one please share some thoughts or point me to resources where I can get more details.
I may want to spin up/stop new Remote Actor systems based on my request queue length
This is not supported out of the box by Akka.Cluster. You would have to build something custom for it.
However Akka .NET has pool routers which are able to resize automatically according to configurable parameters. You may be able to build something around them.
Send the message to any one of the available actor who can start processing without having to check who has the bandwidth to process on the sender side.
If you look at Akka .NET Routers, there are various strategies that can be used to assign work. SmallestMailbox is probably the closest to what you're after.
Messages should not be lost, and if the actor fails, it should be passed to next available actor.
Akka .NET supports At Least Once Delivery. Read more about it in the docs or at the Petabridge blog.
While you may achieve some of your goals with Akka cluster, I wouldn't advise that. From your requirements it clearly states that your concerns are oriented about:
Reliable message delivery (where service buses and message queues are better option). There are a lot of solutions here, depending on your needs i.e. MassTransit, NServiceBus or queues (RabbitMQ).
Scaling workers (which is infrastructure problem and it's not solved by actor frameworks themselves). From what you've said, you don't even even need a cluster.
You could use akka for building a message processing logic, like workers. But as I said, you don't need it if your goal is to replace existing service bus.
Related
I have to implement a process which polls messages from a queue and forwards the data to a http endpoint. Normally I think the best approach for this would be to use Akka streams (with akka http), because it handles backpressure etc. for you. The problem is that I have to create multiple of these pipelines on demand during runtime (based on a http call) with different configurations. Another requirement is that it should be possible to dynamically stop one of those pipelines during runtime and start it with another configuration.
I'm currently not sure if it is really possible to dynamically spin up new parallel akka streams pipelines during runtime (and also remove and restart some of them). For me it currently sounds better to use akka actors for this (based on a queue consumer actor and a akka http actor and I could create a pair dynamically during runtime), but to use them I also have to implement backpressure etc. manually.
Do you think it would be possible to create this via akka streams or would it be a hassle?
Thanks in advance!
You cant reroute the stream once its meterialized and running. But since you configure streams programmatically it is very handy to dynamically define new streams and run them. Everything depends on certain use case but I'd go for single coordinator actor which receives queue messages and dispatch them to the given stream. The same coordinator would receive configuration change requests so it configure new streams, removes old ones etc.
Dont go for manually wiring actors. Its hard to follow and maintain.
I have two actor systems that communicate via akka remoting.
When I take a look into the JVM heap I am seeing (too) many instances of akka.dispatch.Envelope containing SelectChildName messages from akka.remote.RemoteActorRefProvider$RemoteDeadLetterActorRef.
The retained heap of these messages is pretty large and causes memory problems.
What is the purpose of these SelectChildName messages? Is there a way to avoid them?
FYI This seems to relate with Disassociation errors that occur between the two actor systems.
Thanks,
Michail
SelectChildName messages are used by Akka Remoting to resolve a remote actor. If you see a lot of them, there is a chance you are interacting directly with an ActorSelection, instead of an ActorRef.
Every time you send a message to an ActorSelection, for example (these are taken from the docs)
val selection = context.actorSelection("akka.tcp://actorSystemName#10.0.0.1:2552/user/actorName")
selection ! "Pretty awesome feature"
the - possibly remote - actor is resolved, and that involves exchanging of SelectChildName messages by the underlying Akka infrastructure.
If that's the case, try and use directly ActorRefs. You can obtain one from an ActorSelection by using the resolveOne method.
Citing the docs again:
It is always preferable to communicate with other Actors using
their ActorRef instead of relying upon ActorSelection. Exceptions are
sending messages using the At-Least-Once Delivery facility
initiating first contact with a remote system
I'm looking for a framework/platform to (easily) support cross microservices communication. I was guided to look into Akka with Kafka. Unfortunately I was unable to find specifically this set-up and use case, but also didn't find any specific messages that can say it will not work.
Based on articles/messages here and on other sites I've compiled set of expectation from Akka + Kafka with a few open points.
Could you please review/correct the points below?
Platform for Microservices “cluster” implementation.
Microservices “cluster” is system implemented with set of
Microservices [application]
a. Microservices interact with each other by means of AKKA messaging API. (e.g. tell, ask)
b. There can be multiple instances of Microservice
i. Q: Is mailbox shared between instances (of one microservice=Actor)?
AKKA can be connected to Kafka and use it as transport (Message
Broker), i.e. Kafka API is encapsulated into AKKA API
a. Q: What are the values from Kafka? E.g. performance, throughput, reliability. Why Akka remoting is not enough?
Messaging
a. First, some initialization is needed:
i. e.g. create Actors System, supervisor
ii. There should be some shared pre-configuration between all Microservices (Actors in the cluster). E.g. IP/URI, message types/cases, etc.
b. Worker-Actors can be added/instantiated in run time:
i. Find supervisor by name/URI
ii. Add Work-Actor as child
c. There is ability to broadcast message to all children (e.g. specified by name and wildcards) and wait for responses from all children. Hower it seems it is not "native" for Akka API and code doesn't look as clear enough (samples at stackoverlow: 1, 2)
d. There is no ability to subscribe to certain type of events i.e. define filtering conditions during subscription, in case of filtering is needed – it can be done only inside each actor in onReceive method. So, in case of broadcasting each Actor will receive message and than need to decide whether it is applicable for him or not.
e. Q: what are message tracing/debugging tools/capabilities available in Akka?
Actors execution can be “chained”: E.g. A->B->C, in this case after processing C and B control will be passed back (in case of “ask” method)
AKKA Cluster
a. Provides APIs for app lifecycle mgmt.: start all/start one/stop/scale etc.
Q: how I can support/implement graceful shutdown?
b. It has built-in monitoring capabilities (check app availability,
health etc.)
c. Q: What about Infrastructure Monitoring? Do I need to care about it?
d. Q: Is ConductR is a must or just nice to have?
Service Discovery (e.g. Eureka) is not needed, since we use reactive communications (async. Messages via Message Broker)
a. Q: What about affinity/stickiness? Do I need to care about it? What Akka can offer in this area?
b. Q: How balancing is done?
In addition, if AKKA can really work on underlying Kafka - is there good example? I found only samples with data streaming, but I need just event/message processing.
I'm learning Akka, and I'm struggling to find a good pattern to share a single, limited resource among the whole actor hierarchy.
My use case is that I have a HTTP REST endpoint to which I'm only allowed 10 simultaneous connections at any time. Different actors at different levels of the hierarchy need to be able to make HTTP REST calls. I'm using non-blocking I/O to make the HTTP requests (AsyncHttpClient).
The obvious solution is to have a single actor in charge of this REST resource, and have any actors who want to access it send a message to it and expect a reply at a later stage, however:
Having a single actor in charge of this resource feels a bit fragile to me
How should any "client" actor know how to reach this resource manager actor? Is it best to create it at a well known location like /user/rest-manager and use an actor selection, or is it better to try to pass its ActorRef to every actor that needs it (but meaning it will need to be passed down in a lot of actors that don't use it, just so they can in turn pass it down)
In addition, how to deal with "blocking" the client actors when 10 connections are already in progress, especially since I'm using non-blocking I/O? Is it best practice to re-send a message to self (perhaps after some time) as a wait pattern?
I also thought of a token-based approach where the resource manager actor could reply with "access tokens" to client actors that needs to access the resource until exhaustion. However it means that client actors are supposed to "return" the token once they're done which doesn't sound ideal, and I will also need cater for actors dying without returning the token (with some sort of expiration timeout I guess).
What are the patterns / best practices to deal with that situation?
Updated: To indicate I'm using non-blocking I/O
My suggestions would be:
Use the Error Kernel pattern, as the REST endpoint, as you said, is a fragile code (I/O operations can generate any kind of errors). In other words, Master/Worker actor hierarchy, where Workers do the job, while Master does any supervision
Connection limit could be handled by Akka Routing feature, where number of Routees is, in your case, 10. This also drops into Master/Worker category
Addressing - either way sounds good
Connection timeout - to be handled by a client code, as it's always done in a majority of network libs.
I'm new to akka and intend to use it in my new project as a data replication mechanism.
In this scenario, there is a master server and a replicate data server. The replicate data should contain the same data as the master. Each time a data change occurred in the master, it sends an update message to the replicate server. Here the master server is the Sender, and the Replicate server is the Receiver.
But after digging the docs I'm still not sure how to satisfy the following use cases:
When the receiver crashes, the sender should pile up messages to send, none messages should be lost. It should be able to reconnect to the receiver later and continue with last successful message.
when the sender crashes, it should restart and no messages between restart is lost.
Messages are dealt with the same order they were sent.
So my question is, how to config akka to create a sender and a receiver that could do this?
I'm not sure actor with a DurableMessageBox could solve this. If it could, how can i simulate the above situations for testing?
Update:
After reading the docs Victor pointed at, I now got the point that what I wanted was once-and-only-once pattern, which is extremely costly.
In the akka docs it says
Actual transports may provide stronger semantics, but at-most-once is the semantics you should expect. The alternatives would be once-and-only-once, which is extremely costly, or at-least-once which essentially requires idempotency of message processing, which is a user-level concern.
So inorder to achieve Guaranteed Delivery, I may need to turn to some other MQ solution (for example Kafka), or try to implement once-and-only-once with DurableMessageBox, and see if the complexity with it could be relieved with my specific use case.
You'd need to write your own remoting that utilizes the durable subscriber pattern, as Akka message send guarantees are less strict than what you are going for: http://doc.akka.io/docs/akka/2.0/general/message-send-semantics.html
Cheers,
√