I am wanting to write some web services using WCF.
I would like to have a "thread pool" in my web service.
For example, I have nearly 6gb of data I need to manipulate.
I would like the client to call an operation on the webservice and have a new task or thread created. The client is able to call a ListRunningTasks(); and have the webservice return a list of tasks. The client should be able to forcefully kill a task if it is taking too long e.g. KillTask(int taskID); or something. I have previously done some threading, but not inside WCF or a service that doesn't have state. Is this possible? If so, how would one go about implementing such a thing? Any reading, links or suggestions would be great.
Thanks, Mike.
One possible solution:
Implement explicit queues for your outstanding tasks taking into consideration that they take that long (20-30mins as you wrote).
Build a custom component to manage those queues e.g. you might even want capabilities to persist them, resume work when you restart the service etc.
Have explicitly created worker threads that pickup work from those queues.
Implement a WCF service to make your queue manager available to external systems.
Thread pools are more designed to process a high volume of short-running tasks.
You should consider using Windows Workflow Foundation to create such services. A state machine workflow can be exposed as a service in such a way that when method A is called, it will start the workflow (task), after which methods can be called to stop, suspend, or query the running task. WF will handle the state transitions, preventing illegal state changes, and making sure that new tasks are only spun up as necessary.
Note that WF will handle the threading issues for you in an almost transparent manner.
Related
I have a quite heavy batch process (a python script called "run_simulation.py") on which I have very little control, it can be launched by a single user through a web api but it read and writes from disk so it wouldn't handle parallel requests.
Now, I'd like to have one docker container instanciated per request so that all requests can be handled in parallel, what would be the way to do this ? Is this even doable with Docker ? What would be the module responsible to instanciate the container and pass the http request to it ?
Generally you don’t do this. There are two good reasons for that: if you unconditionally launch a container per request it becomes very easy to swamp your system with these background jobs to the point where none can progress; and the setup that would allow you to launch more Docker containers would also give you unlimited root-level access to the host, which you don’t want in a process that accepts network requests.
A better approach is to set up a job queue system. RabbitMQ is popular and open-source, but by no means the only option. When you receive a request that needs background work, you add a job to the queue and return immediately. Meanwhile, you have some number of worker processes which accept jobs from the queue and do the work.
This gives you a couple of benefits. You control how much work can be done in parallel (by controlling the number of worker containers). If you need to do more work by setting up a second server (or even more), they can all connect back to the same queue server, without requiring a complex multi-host container setup. If your workers crash (or get OOM-killed) their jobs will be returned to the queue and can be picked up and retried by other workers. If you decide Docker doesn’t work for you, or that you need a different orchestrator (Nomad, Kubernetes) you can run this exact same setup without making any code changes, just changing the deployment configuration.
I was thinking of creating a web service that does a long running process. What would be the best way to design these to work with a load balancer? I can't think of any way of doing it besides writing a custom queue.
That is exactly what you should do. You typically want your web service calls to be a quick request/response. So make a call to the web service, have the web service queue the work then have worker processes pick up the messages from the queue and process them.
This is the way to go, queuing the long running processes allows your system to scale, allows you to add recovery logic if a process fails, allows you to scale quickly by adding additional workers to process the queue, and best of all does not tie up the client waiting for a response.
REDIS (http://redis.io/) has been my choice over the past few years, if you are using Azure or AWS they have messaging services as well.
You can also use websockets to notify the client when processes are completed to keep the UI state in the loop.
After reading through the ZMQ manual about the load balancing broker, I thought that it would be great to implement in my own code. So I did, adding some additional touches to make it more responsive. One performance enhancement I was looking to add was the ability to dispatch to multiple long-running work jobs concurrently. I think I'm right about this, I could be wrong though, so consider the following with respect to just the lbbroker code that's in the manual:
Two workers (clients) simultaneously request work, each with long running jobs given to them (by a manager, or manager). In the current code, It's good because it's not round-robin-ing the work to different recipients, it's selecting FCFS. But there's also a problem in that a reply is first needed from the first worker who gets through before work can be dispensed to the second worker.
Basically, I want to dole worker out as fast as there are workers ready to receive it, FCFS style and concurrently as well. At the same time, I don't want to lose the model that I have where manager A gets through to worker B, and worker B's reply gets back to manager A. Keeping this, which is facilitated by the request-reply pattern, while at the same time allowing worker B to receive the only manager's second work job while A may still be processing it's job is very desired.
How can I most easily go about achieving this? Preferably by modifying my current lbbroker implementation, which isn't too different from lbbroker in the manual.
Thanks in advance.
As it turns out, my difficulties stemmed from an unsufficiently specific understanding of the load balancing broker example; it is not a broker that has REP sockets in order that it must receive between each work request/worker request. So the asynchronous issue does not exist at all.
Basically, a Router has an identity message and in forwarding that along in a consistent manner, you can avoid the issue entirely, and the router is free to connect other manager worker pairs while N concurrent workers work.
I understand that Akka actors should not block in order to stay reactive to messages, but how do I structure my service where I want to monitor a process running for an indefinite period of time?
For example, we are using the Amazon Kinesis Connector library. You create a connector with a given configuration, which inherits from Runnable, and then call the Run() method. The connector simply runs indefinitely, pulling data from Kinesis, and writing it to Amazon S3. In fact, if the runnable returns, then that is an error, and it needs to be restarted.
Approach (1) would be to simply create a child actor for each Kinesis Connector running, and if the Run() method returns, you throw an exception, the Supervising Actor notices the exception and restarts the child actor. One connector per child actor per thread.
Approach (2) would be for the child actor to wrap the Kinesis Connector in a Future, and if the future returns, the actor would restart the Connector in another Future. Conceivably a single actor could manage multiple Connectors, but does this mean each Future is executing in a separate thread?
Which approach would be most in line with the philosophy of Akka, or is there some other approach people recommend? In general, I want to catch any problems with any Connector, and restart it. In total there would not be more than a half dozen Connectors running in parallel.
I would take approach 1. It should be noted though that actors do not have a dedicated thread by default but they share a thread pool (the so called dispatcher, see: http://doc.akka.io/docs/akka/2.3.6/scala/dispatchers.html). This means that blocking is inherently dangerous because it exhausts the threads of the pool not letting other non-blocked actors to run (since the blocked actors do not put the thread back into the pool). Therefore you should separate blocking calls into a fixed size pool of dedicated actors, and you should assign these actors a PinnedDispatcher. This latter step ensures that these actors do not interfere with each other (they each have a dedicated thread) and ensures that these actors do not interfere with the rest of the system (all of the other actors will run on another dispatchers, usually on default-dispatcher). Be sure though to limit the number of actors running on the PinnedDispatcher since the number of used threads will grow with the number of actors on that dispatcher.
Of your two options, I'd say 1 is the more appropriate. No.2 suffers from the fact that, in order to exit from the future monad's world you need to call an Await somewhere, and there you need to specify a max duration which, in your case, does not make sense.
Maybe you could look into other options before going for it, tough. A few keywords that may inspire you are streams and distributed channels.
What would be a more standard way of notifying a web service consumer of a data change?
Having the consumer periodically calling the web service to pull change notification.
Consumer setting up a call back web service that can be invoked to forward notification about the change.
Other?
Both of these are options. There is also something called "comet" which is like setting up a stream between between the consumer and producer - messages can then be passed back and forth between the two. Wikipedia is probably the best place to start investigating to see if it will work for you project: http://en.wikipedia.org/wiki/Comet_(programming)
Depends on the scenario. If you're working in a closed environment with only a few consumers of your service, you could switch to a COMET style service which allows a callback from the service to the client. More about that here:
Wikipedia - COMET
From what I've read, that method doesn't scale well in larger environments so I'd be careful.
The more traditional method is your first option of polling the service for changes. As long as your service performs well and you have the appropriate hardware to serve up the requests, it's probably your best bet for a public facing web service.
In case you weren't aware of it, and in case it helps: WCF can work with a Duplex contract that in effect creates a callback service contract on the client. It's fairly transparent.