I understand that Akka actors should not block in order to stay reactive to messages, but how do I structure my service where I want to monitor a process running for an indefinite period of time?
For example, we are using the Amazon Kinesis Connector library. You create a connector with a given configuration, which inherits from Runnable, and then call the Run() method. The connector simply runs indefinitely, pulling data from Kinesis, and writing it to Amazon S3. In fact, if the runnable returns, then that is an error, and it needs to be restarted.
Approach (1) would be to simply create a child actor for each Kinesis Connector running, and if the Run() method returns, you throw an exception, the Supervising Actor notices the exception and restarts the child actor. One connector per child actor per thread.
Approach (2) would be for the child actor to wrap the Kinesis Connector in a Future, and if the future returns, the actor would restart the Connector in another Future. Conceivably a single actor could manage multiple Connectors, but does this mean each Future is executing in a separate thread?
Which approach would be most in line with the philosophy of Akka, or is there some other approach people recommend? In general, I want to catch any problems with any Connector, and restart it. In total there would not be more than a half dozen Connectors running in parallel.
I would take approach 1. It should be noted though that actors do not have a dedicated thread by default but they share a thread pool (the so called dispatcher, see: http://doc.akka.io/docs/akka/2.3.6/scala/dispatchers.html). This means that blocking is inherently dangerous because it exhausts the threads of the pool not letting other non-blocked actors to run (since the blocked actors do not put the thread back into the pool). Therefore you should separate blocking calls into a fixed size pool of dedicated actors, and you should assign these actors a PinnedDispatcher. This latter step ensures that these actors do not interfere with each other (they each have a dedicated thread) and ensures that these actors do not interfere with the rest of the system (all of the other actors will run on another dispatchers, usually on default-dispatcher). Be sure though to limit the number of actors running on the PinnedDispatcher since the number of used threads will grow with the number of actors on that dispatcher.
Of your two options, I'd say 1 is the more appropriate. No.2 suffers from the fact that, in order to exit from the future monad's world you need to call an Await somewhere, and there you need to specify a max duration which, in your case, does not make sense.
Maybe you could look into other options before going for it, tough. A few keywords that may inspire you are streams and distributed channels.
Related
Does gRPC server/client have any concept of thread pool for connections? That it is possible to reuse threads, pre-allocate threads, queue request on thread limit reached, etc.
If no, how does it work, is it just allocating/destroying a thread when it need, without any limitation and/or reuse? If yes, is it possible to configure it?
It depends whether you are using sync or async API.
For a sync client, your RPC calls blocks the calling thread, so it is not really relevant. For a sync server, there is an internal threadpool handling all the incoming requests, you can use a grpc::ResourceQuota on a ServerBuilder to limit the max number of threads used by the threadpool.
For async client and server, gRPC uses CompletionQueue as a way for users to define their own threading model. A common way of building clients and servers is to use a user-provided threadpool to run CompletionQueue::Next in each thread. Then once it gets some tag from the Next call, you can cast it to a user-defined type and run some methods to proceed the state. In this case, the user has full control of threads being used.
Note gRPC does create some internal threads, but they should not be used for the majority of the rpc work.
I'm learning Akka, and I'm struggling to find a good pattern to share a single, limited resource among the whole actor hierarchy.
My use case is that I have a HTTP REST endpoint to which I'm only allowed 10 simultaneous connections at any time. Different actors at different levels of the hierarchy need to be able to make HTTP REST calls. I'm using non-blocking I/O to make the HTTP requests (AsyncHttpClient).
The obvious solution is to have a single actor in charge of this REST resource, and have any actors who want to access it send a message to it and expect a reply at a later stage, however:
Having a single actor in charge of this resource feels a bit fragile to me
How should any "client" actor know how to reach this resource manager actor? Is it best to create it at a well known location like /user/rest-manager and use an actor selection, or is it better to try to pass its ActorRef to every actor that needs it (but meaning it will need to be passed down in a lot of actors that don't use it, just so they can in turn pass it down)
In addition, how to deal with "blocking" the client actors when 10 connections are already in progress, especially since I'm using non-blocking I/O? Is it best practice to re-send a message to self (perhaps after some time) as a wait pattern?
I also thought of a token-based approach where the resource manager actor could reply with "access tokens" to client actors that needs to access the resource until exhaustion. However it means that client actors are supposed to "return" the token once they're done which doesn't sound ideal, and I will also need cater for actors dying without returning the token (with some sort of expiration timeout I guess).
What are the patterns / best practices to deal with that situation?
Updated: To indicate I'm using non-blocking I/O
My suggestions would be:
Use the Error Kernel pattern, as the REST endpoint, as you said, is a fragile code (I/O operations can generate any kind of errors). In other words, Master/Worker actor hierarchy, where Workers do the job, while Master does any supervision
Connection limit could be handled by Akka Routing feature, where number of Routees is, in your case, 10. This also drops into Master/Worker category
Addressing - either way sounds good
Connection timeout - to be handled by a client code, as it's always done in a majority of network libs.
After reading through the ZMQ manual about the load balancing broker, I thought that it would be great to implement in my own code. So I did, adding some additional touches to make it more responsive. One performance enhancement I was looking to add was the ability to dispatch to multiple long-running work jobs concurrently. I think I'm right about this, I could be wrong though, so consider the following with respect to just the lbbroker code that's in the manual:
Two workers (clients) simultaneously request work, each with long running jobs given to them (by a manager, or manager). In the current code, It's good because it's not round-robin-ing the work to different recipients, it's selecting FCFS. But there's also a problem in that a reply is first needed from the first worker who gets through before work can be dispensed to the second worker.
Basically, I want to dole worker out as fast as there are workers ready to receive it, FCFS style and concurrently as well. At the same time, I don't want to lose the model that I have where manager A gets through to worker B, and worker B's reply gets back to manager A. Keeping this, which is facilitated by the request-reply pattern, while at the same time allowing worker B to receive the only manager's second work job while A may still be processing it's job is very desired.
How can I most easily go about achieving this? Preferably by modifying my current lbbroker implementation, which isn't too different from lbbroker in the manual.
Thanks in advance.
As it turns out, my difficulties stemmed from an unsufficiently specific understanding of the load balancing broker example; it is not a broker that has REP sockets in order that it must receive between each work request/worker request. So the asynchronous issue does not exist at all.
Basically, a Router has an identity message and in forwarding that along in a consistent manner, you can avoid the issue entirely, and the router is free to connect other manager worker pairs while N concurrent workers work.
I am wanting to write some web services using WCF.
I would like to have a "thread pool" in my web service.
For example, I have nearly 6gb of data I need to manipulate.
I would like the client to call an operation on the webservice and have a new task or thread created. The client is able to call a ListRunningTasks(); and have the webservice return a list of tasks. The client should be able to forcefully kill a task if it is taking too long e.g. KillTask(int taskID); or something. I have previously done some threading, but not inside WCF or a service that doesn't have state. Is this possible? If so, how would one go about implementing such a thing? Any reading, links or suggestions would be great.
Thanks, Mike.
One possible solution:
Implement explicit queues for your outstanding tasks taking into consideration that they take that long (20-30mins as you wrote).
Build a custom component to manage those queues e.g. you might even want capabilities to persist them, resume work when you restart the service etc.
Have explicitly created worker threads that pickup work from those queues.
Implement a WCF service to make your queue manager available to external systems.
Thread pools are more designed to process a high volume of short-running tasks.
You should consider using Windows Workflow Foundation to create such services. A state machine workflow can be exposed as a service in such a way that when method A is called, it will start the workflow (task), after which methods can be called to stop, suspend, or query the running task. WF will handle the state transitions, preventing illegal state changes, and making sure that new tasks are only spun up as necessary.
Note that WF will handle the threading issues for you in an almost transparent manner.
I want to know the technical reasons why the lift webframework has high performance and scalability? I know it uses scala, which has an actor library, but according to the install instructions it default configuration is with jetty. So does it use the actor library to scale?
Now is the scalability built right out of the box. Just add additional servers and nodes and it will automatically scale, is that how it works? Can it handle 500000+ concurrent connections with supporting servers.
I am trying to create a web services framework for the enterprise level, that can beat what is out there and is easy to scale, configurable, and maintainable. My definition of scaling is just adding more servers and you should be able to accommodate the extra load.
Thanks
Lift's approach to scalability is within a single machine. Scaling across machines is a larger, tougher topic. The short answer there is: Scala and Lift don't do anything to either help or hinder horizontal scaling.
As far as actors within a single machine, Lift achieves better scalability because a single instance can handle more concurrent requests than most other servers. To explain, I first have to point out the flaws in the classic thread-per-request handling model. Bear with me, this is going to require some explanation.
A typical framework uses a thread to service a page request. When the client connects, the framework assigns a thread out of a pool. That thread then does three things: it reads the request from a socket; it does some computation (potentially involving I/O to the database); and it sends a response out on the socket. At pretty much every step, the thread will end up blocking for some time. When reading the request, it can block while waiting for the network. When doing the computation, it can block on disk or network I/O. It can also block while waiting for the database. Finally, while sending the response, it can block if the client receives data slowly and TCP windows get filled up. Overall, the thread might spend 30 - 90% of it's time blocked. It spends 100% of its time, however, on that one request.
A JVM can only support so many threads before it really slows down. Thread scheduling, contention for shared-memory entities (like connection pools and monitors), and native OS limits all impose restrictions on how many threads a JVM can create.
Well, if the JVM is limited in its maximum number of threads, and the number of threads determines how many concurrent requests a server can handle, then the number of concurrent requests will be determined by the number of threads.
(There are other issues that can impose lower limits---GC thrashing, for example. Threads are a fundamental limiting factor, but not the only one!)
Lift decouples thread from requests. In Lift, a request does not tie up a thread. Rather, a thread does an action (like reading the request), then sends a message to an actor. Actors are an important part of the story, because they are scheduled via "lightweight" threads. A pool of threads gets used to process messages within actors. It's important to avoid blocking operations inside of actors, so these threads get returned to the pool rapidly. (Note that this pool isn't visible to the application, it's part of Scala's support for actors.) A request that's currently blocked on database or disk I/O, for example, doesn't keep a request-handling thread occupied. The request handling thread is available, almost immediately, to receive more connections.
This method for decoupling requests from threads allows a Lift server to have many more concurrent requests than a thread-per-request server. (I'd also like to point out that the Grizzly library supports a similar approach without actors.) More concurrent requests means that a single Lift server can support more users than a regular Java EE server.
at mtnyguard
"Scala and Lift don't do anything to either help or hinder horizontal scaling"
Ain't quite right. Lift is highly statefull framework. For example if a user requests a form, then he can only post the request to the same machine where the form came from, because the form processeing action is saved in the server state.
And this is actualy a thing which hinders scalability in a way, because this behaviour is inconistent to the shared nothing architecture.
No doubt that lift is highly performant but perfomance and scalability are two different things. So if you want to scale horizontaly with lift you have to define sticky sessions on the loadbalancer which will redirect a user during a session to the same machine.
Jetty maybe the point of entry, but the actor ends up servicing the request, I suggest having a look at the twitter-esque example, 'skitter' to see how you would be able to create a very scalable service. IIRC, this is one of the things that made the twitter people take notice.
I really like #dre's reply as he correctly states the statefulness of lift being a potential problem for horizontal scalability.
The problem -
Instead of me describing the whole thing again, check out the discussion (Not the content) on this post. http://javasmith.blogspot.com/2010/02/automagically-cluster-web-sessions-in.html
Solution would be as #dre said sticky session configuration on load balancer on the front and adding more instances. But since request handling in lift is done in thread + actor combination you can expect one instance handle more requests than normal frameworks. This would give an edge over having sticky sessions in other frameworks. i.e. Individual instance's capacity to process more may help you to scale
you have Akka lift integration which would be another advantage in this.