I have a specific use-case for an Akka implementation.
I have a set of agents who send heartbeats to Akka. Akka takes this heartbeat, and assigns actors to send them to my meta-data server (a separate server). This part is done.
Now my meta-data server also needs to send action information to the agents. However, since these agents may be behind firewalls, Akka cannot communicate to them directly so it needs to send the action as a response to the Heartbeat. Thus when the meta-data server sends an action Akka stores it in a DurableMessageQueue (separate one for each agentID) and keeps the mapping of agent-ID to DurableMessageQueue in a HashMap. Then whenever the heartbeat comes, before responding it checks this queue and piggybacks the action in the response.
The issue with this is that the HashMap will be in a single JVM and therefor I cannot scale this. Am I missing something or is there a better way to do it?
I have Akka running behind Mina server running which received and sends messages.
Related
I have a dynamic peer to peer network where nodes communicates with grpc. Each node has its own server and client. One grpc method is defined for login of a new Node. I use a sinchronous message to communicate the login to all others, where I create a new channel with each other servers, send a single message and wait a response.
rpc enter(LogIn) returns (Response);
If I have one node in my network (node 1) and then two or more nodes enter at the same time, for example node 2 and node 3, they'll call both the grpc method "enter" on node1's server. With this type of method, which is the behavior of node1's server? It's able to manage both this request? so with a method like this, grpc queues messages that arrive concurrently or will it handle only one request?
thanks
gRPC supports concurrent execution of multiple RPCs. When an RPC (or RPC event) arrives, it is queued on the serverBuilder.executor() specified when building the server. If you don't specify, a default executor is used that uses as many threads as necessary. Nothing in gRPC will behave differently if the RPCs are the same RPC method or different RPC methods.
I am looking to build a system that is able to process a stream of requests that needs a long processing time say 5 min each. My goal is to speed up request processing with minimal resource footprint which at times can be a burst of messages.
I can use something like a service bus to queue the request and have multiple process (a.k.a Actors in akka) that can subscribe for a message and start processing. Also can have a watchdog that looks at the queue length in the service bus and create more actors/ actor systems or stop a few.
if I want to do the same in the Actor system like Akka.net how can this be done. Say something like this:
I may want to spin up/stop new Remote Actor systems based on my request queue length
Send the message to any one of the available actor who can start processing without having to check who has the bandwidth to process on the sender side.
Messages should not be lost, and if the actor fails, it should be passed to next available actor.
can this be done with the Akka.net or this is not a valid use case for the actor system. Can some one please share some thoughts or point me to resources where I can get more details.
I may want to spin up/stop new Remote Actor systems based on my request queue length
This is not supported out of the box by Akka.Cluster. You would have to build something custom for it.
However Akka .NET has pool routers which are able to resize automatically according to configurable parameters. You may be able to build something around them.
Send the message to any one of the available actor who can start processing without having to check who has the bandwidth to process on the sender side.
If you look at Akka .NET Routers, there are various strategies that can be used to assign work. SmallestMailbox is probably the closest to what you're after.
Messages should not be lost, and if the actor fails, it should be passed to next available actor.
Akka .NET supports At Least Once Delivery. Read more about it in the docs or at the Petabridge blog.
While you may achieve some of your goals with Akka cluster, I wouldn't advise that. From your requirements it clearly states that your concerns are oriented about:
Reliable message delivery (where service buses and message queues are better option). There are a lot of solutions here, depending on your needs i.e. MassTransit, NServiceBus or queues (RabbitMQ).
Scaling workers (which is infrastructure problem and it's not solved by actor frameworks themselves). From what you've said, you don't even even need a cluster.
You could use akka for building a message processing logic, like workers. But as I said, you don't need it if your goal is to replace existing service bus.
All,
I'm dealing with what seems like a simple case, but one that poses some design challenges:
There's a local actor system with a client, which reaches out to a
remote system that runs the bulk of the business logic.
The remote system will have a fixed IP address, port, etc. -
therefore, one can use the context.actorSelection(uri) strategy to
get a hold of the ActorRef for the current incarnation of the actor
(or a group of routees behind a router).
The remote system, being a server, shouldnt be in the business of
knowing the location of the client.
Given this, it's pretty straightforward to propagate messages from the client to the server, process them, and send a message back to the client. Even if there are several steps, one can propagate the responses through the hierarchy until one reaches the top-level remote actor that the client called, which will know who the sender was on the client side.
Let's say on the server side, we have a Master actor that has a router of Worker actors. You can have the worker respond directly to the client, since the message received from the Client by the Master can be sent to the Worker via the router as "router.tell(message, sender)" instead of "router ! message." Of course, you can also propagate responses from the Worker to the Master and then to the Client.
However, let's say the Worker throws an exception. If its parent (the Master) is its supervisor and it handles the Workers' failures, the Master can do the usual Restart/Resume/Stop. But let's say that we also want to notify the Client of the failure, e.g. for UI updating purposes. Even if we handle the failure of the Worker via the Master's SupervisorStrategy, we won't know who the original caller was (the Client) that had the processing request payload at the time when the Master intercepted the Worker's failure.
Here's a diagram
Client (local) -> Master (remote) -> Router (remote) -> Worker (remote)
Worker throws an exception, Master handles it. Now the Master can restart the Worker, but it doesn't know which Client to notify, in case there are several, their IP addresses change, etc.
If there's one Client and the Client has a host/port/etc. known to the server, then one could use context.actorSelection(uri) to look up the client and send it a message. However, with the server not being in the business of knowing where the Client is coming from (#3), this shouldn't be a requirement.
One obvious solution to this is to propagate messages from the Client to the Worker with the Client's ActorRef in the payload, in which case the Master would know about whom to send the failure notification to. It seems ugly, though. Is there a better way?
I suppose the Client can have the Workers on DeathWatch, but the Client shouldn't really have to know the details of the actor DAG on the server. So, I guess I'm coming back to the issue of whether the message sent from the Client should contain not just the originally intended payload, but also the ActorRef of the Client.
Also, this brings another point. Akka's "let it crash" philosophy suggests that the actor's supervisor should handle the actor's failures. However, if we have a Client, a Master (with a router) and a Worker, if the Worker fails, the Master can restart it - but it would have to tell the Client that something went wrong. In such a case, the Master would have to correlate the messages from the Client to the Workers to let the Client know about the failure. Another approach is to send the ActorRef of the Client along with the payload to the Worker, which would allow the Client to use the standard try/catch approach to intercept a failure, send a message to the client before failing, and then throw an exception that would be handled by the Master. However, this seems against Akka's general philosophy. Would Akka Persistence help in this case, since Processors track message IDs?
Thanks in advance for your help!
Best,
Marek
Quick answer, use this:
def preRestart(reason: Throwable, message: Option[Any]): Unit
More elaborate answer that gives no easy answers (as I struggle with this myself):
There are several ideas on how you can achieve what you need.
The question you asked should the worker answer the client or the master. Well that depends.
Let's assume that client sends you some work W1 and you pass it to the worker. The worker fails. Now the question is, if that work was important? If so, the master should still hold the reference to the W1 as it will probably retry the attempt in some near future. Maybe it was some data that should be persisted and the connection to database was lost for a second?
It the work was not important you may just set a timeout on the client that the operation was unsuccesfull and you're done. This way you will lost the exception details. But maybe it does not matter? You only want to check the logs afterwards and you just give a '500 Server Error' response.
This is not as easy to answer as it seems at first.
One possibility is to side-step most of this complexity by changing your approach. This may or may not be feasible for your use case.
For example, if the Exception can be anticipated and it is not of a sort that requires a restart of the worker actor, then don't let the master supervisor handle it. Simply build an appropriate response for that Exception (possible the Exception itself, or something containing the Exception), and send that to the client as a normal response message. You could send a scala Try message, for example, or create whatever messages make sense.
Of course, there are non-expected Exceptions, but in this case, the actor dealing with the UI can simply time-out and return a general error. Since the exception is unexpected, you probably wouldn't be able to do better than a general UI error anyway (e.g. a 500 error if the UI is HTTP-based), even if the exception was propagated to that layer. One downside of course is that the timeout will take longer to report the problem to the UI than if the error was propagated explicitly.
Lastly, I don't think there is anything wrong at all with sending ActorRef's as part of the payload, to handle this case from within the master actor as you suggested. I believe ActorRef was designed explicitly with the intent of sending them between actors (including remote actors). From the ScalaDoc of ActorRef:
Immutable and serializable handle to an actor, which may or may not reside on the local host or inside the same ActorSystem.
...
ActorRefs can be freely shared among actors by message passing.
I have an AMQP application that has a persistent RabbitMQ queue in the client side and a RabbitMQ queue in the server side. The client always writes in the local persistent queue and those messages are transmited to server using shovel plugin.
Producer -> Local Queue --------- SHOVEL ---------- Remote queue -> Consumer
Whether the server is not present the app stills works and shovel does the send when possible. In the other hand the server doesn't require to know the location of the clients becaiuse it consmes always from local queues. I would like to migrate this topology to AKKA using the FilePersistent Mailbox. Is it even possible? Is there something like Federation or Shovel plugin in Akka core libraries.
PS: What I want to achieve is replacing completetly AMQP to get rid of RabbitMQ. It works fine but is another piece of software to install, configure and mantain. I would like to provide all this functionality from my application using just libraries and not another server like RabbitMQ.
Just to clarify a little more what I'm looking to achieve is something like this:
Actor1 -> DurableMailBox1 ----Shovel? Federation?---- DurableMailbox2 <- Actor2
[EDIT]
It looks like there's no way to communicate directly mailbox to mailbox. The possible topologies that can be implemented with AKKA are these:
Remote Actor1 -> [DurableMailBox1 <- Actor2]
Where the arrow can be secured in order to ensure message delivery but is not possible to copy messages from one Mailbox to other Mailbox automatically.
Take a look at Akka Remoting and the Reliable Proxy Pattern.
Sending via a ReliableProxy makes the message send exactly as reliable
as if the represented target were to live within the same JVM,
provided that the remote actor system does not terminate. In effect,
both ends (i.e. JVM and actor system) must be considered as one when
evaluating the reliability of this communication channel. The benefit
is that the network in-between is taken out of that equation.
See this enhancement to the ReliableProxy that mitigates the problem with the remote actor system terminating.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
Under what conditions would one favor apps talking via a message queue instead of via web services (I just mean XML or JSON or YAML or whatever over HTTP here, not any particular type)?
I have to talk between two apps on a local network. One will be a web app and have to request commands on another app (running on different hardware). The requests are things like creating users, moving files around, and creating directories. Under what conditions would I prefer XML Web Services (or straight TCP or something) to using a Message queue?
The web app is Ruby on Rails, but I think the question is broader than that.
When you use a web service you have a client and a server:
If the server fails the client must take responsibility to handle the error.
When the server is working again the client is responsible of resending it.
If the server gives a response to the call and the client fails the operation is lost.
You don't have contention, that is: if million of clients call a web service on one server in a second, most probably your server will go down.
You can expect an immediate response from the server, but you can handle asynchronous calls too.
When you use a message queue like RabbitMQ, Beanstalkd, ActiveMQ, IBM MQ Series, Tuxedo you expect different and more fault tolerant results:
If the server fails, the queue persist the message (optionally, even if the machine shutdown).
When the server is working again, it receives the pending message.
If the server gives a response to the call and the client fails, if the client didn't acknowledge the response the message is persisted.
You have contention, you can decide how many requests are handled by the server (call it worker instead).
You don't expect an immediate synchronous response, but you can implement/simulate synchronous calls.
Message Queues has a lot more features but this is some rule of thumb to decide if you want to handle error conditions yourself or leave them to the message queue.
There's been a fair amount of recent research in considering how REST HTTP calls could replace the message queue concept.
If you introduce the concept of a process and a task as a resource, the need for middle messaging layer starts to evaporate.
Ex:
POST /task/name
- Returns a 202 accepted status immediately
- Returns a resource url for the created task: /task/name/X
- Returns a resource url for the started process: /process/Y
GET /process/Y
- Returns status of ongoing process
A task can have multiple steps for initialization, and a process can return status when polled or POST to a callback URL when complete.
This is dead simple, and becomes quite powerful when you realize that you can now subscribe to an rss/atom feed of all running processes and tasks without any middle layer. Any queuing system is going to require some sort of web front end anyway, and this concept has it built in without another layer of custom code.
Your resources exist until you delete them, which means you can view historical information long after the process and task complete.
You have built in service discovery, even for a task that has multiple steps, without any extra complicated protocols.
GET /task/name
- returns form with required fields
POST (URL provided form's "action" attribute)
Your service discovery is an HTML form - a universal and human readable format.
The entire flow can be used programmatically or by a human, using universally accepted tools. It's a client driven, and therefore RESTful. Every tool created for the web can drive your business processes. You still have alternate message channels by POSTing asynchronously to a separate array of log servers.
After you consider it for a while, you sit back and start to realize that REST may just eliminate the need for a messaging queue and an ESB altogether.
http://www.infoq.com/presentations/BPM-with-REST
Message queues are ideal for requests which may take a long time to process. Requests are queued and can be processed offline without blocking the client. If the client needs to be notified of completion, you can provide a way for the client to periodically check the status of the request.
Message queues also allow you to scale better across time. It improves your ability to handle bursts of heavy activity, because the actual processing can be distributed across time.
Note that message queues and web services are orthogonal concepts, i.e. they are not mutually exclusive. E.g. you can have a XML based web service which acts as an interface to a message queue. I think the distinction your looking for is Message Queues versus Request/Response, the latter is when the request is processed synchronously.
Message queues are asynchronous and can retry a number of times if delivery fails. Use a message queue if the requester doesn't need to wait for a response.
The phrase "web services" make me think of synchronous calls to a distributed component over HTTP. Use web services if the requester needs a response back.
I think in general, you'd want a web service for a blocking task (this tasks needs to be completed before we execute more code), and a message queue for a non-blocking task (could take quite a while, but we don't need to wait for it).