I am preparing to rewrite my Play1 application with Play2 and I need to implement scheduled tasks that run exactly once a day at some specific time.
In my old app I implemented it as follows:
the task is scheduled using Play1 jobs and the app runs on multiple nodes
at the specified time all healthy nodes start the task and I use lock record in the database to ensure only one of them proceeds with execution and all others exit without doing anything.
How do I implement similar functionality with Akka?
You can just use the Scheduler to either execute a runnable or to send a message to an actor:
system.scheduler().scheduleOnce(Duration.create(24, TimeUnit.HOURS),
taskActor, "doTask", system.dispatcher(), null);
Or
system.scheduler().scheduleOnce(Duration.create(24, TimeUnit.HOURS),
new Runnable() {
#Override
public void run() {
doTask()
}
}, system.dispatcher());
I would prefer the method including an actor though.
You can read up on how to create an actor to receive the doTask message here.
Related
I'm trying to create a simple Camunda BPM workflow with a parallel gateway and compensating actions like this:
All the Service Tasks are configured as external tasks that are executed by a C# program. This program calls the fetchAndLock method on the API to get a list of tasks to execute and then executes these tasks in parallel in the background. I'm experiencing some problems with this approach:
The Lock in the fetchAndLock method doesn't seem to do anything and the workflow engine does not seem to wait until all the fetched tasks are handled whenever one of the tasks is completed with a bpmnError'. Instead it immediately plans the execution of the compensating actions for the tasks it already received a complete` call and deletes the instances of all the other planned tasks without waiting for their results.
This results in the following problems:
The C# program continues to execute the unfinished tasks and when they complete it tries to call the complete method on the API, but that fails with a 404 error because the Camunda engine already deleted these task instances.
The compensating actions for these tasks are never called, which leaves the business process in an invalid state.
What am I doing wrong? I am not very familiar with BPMN so maybe there is a better way to design this process. Or is this a major bug in Camunda?
I can assume that after the parallel gateway there are two errors that trigger the event subprocess twice. You can try using the terminate event in the event subprocess
I have a task queue which users can push tasks onto, only one task can run at a time enforced by the concurrency setting for the queue. In some cases (e.g. long running task) they may wish to cancel a running task in order to free up the queue to process the next task.
To achieve this I have been running the task queue as a Flask application, should a user wish to cancel a task I call the delete_task method of the python client library for a given queue & task.
However, I am seeing that the underlying task continues to be processed even after the task has been deleted. Have been trying to find documentation of how Cloud Tasks handles a task being deleted, but haven't found anything concrete.
Hoping that i'd be able to listen for a signal of some sort in order to gracefully shut down the process if a deletion is received. Or that the underlying process would be killed if the parent task is deleted.
Has anyone worked with the Cloud Tasks API before? Is it correct to assume that a deleted task will cleanup any processes that are running?
I don't see how a worker would be able to find out that the task it is working on has been deleted.
In the eyes of the worker, a task is an incoming Http request. I don't know how the Queue could tell that specific process to stop. I'm fairly certain that "deleting" a task just removes it from the Queue only.
You'd have to build a custom 'cancel' function that would be able to reach out to this worker.
Or this worker would have to periodically check with the Queue to see if its task still exists.
https://cloud.google.com/tasks/docs/reference/rest/v2/projects.locations.queues.tasks/get
https://googleapis.dev/python/cloudtasks/latest/gapic/v2/api.html#google.cloud.tasks_v2.CloudTasksClient.get_task
I'm not actually sure what the Queue will return if you try to call 'get task' a deleted task since i don't see a 'status' property for task. Maybe it will return an error like 'task does not exist'
Most of the beginner's Akka examples seem to advocate calling the actor system's stop() and shutdown() methods like so:
object Main extends App {
// create the ActorSystem
val system = ActorSystem("HelloSystem")
// put your actors to work here ...
// shut down the ActorSystem when the work is finished
system.stop
system.shutdown
}
However what if your Akka app is meant to be a running service, that should (conceivably) live forever? Meaning it starts, the actor system is created, and actors simply idle until work (perhaps coming in from connected clients, etc.) needs to be done?
Is it OK to just initialize/start the actor system and leave it be (that is, omit invoking stop and shutdown altogether? Why/why not?
Yes, it is ok. This is a problem similar to AkkaHTTP implementation. In AkkaHTTP, you start actors which open a socket and wait for requests.
One possible issue comes to my mind: if you need some short-living actors (inside your long-running service) to process a single request, you should stop them after they are no longer needed (to free resources), especially if the actors are stateful.
I wrote a blog post about that issue: https://mikulskibartosz.name/always-stop-unused-akka-actors-a2ceeb1ed41
The maximum amount of time the pollForActivityTask method stays open polling for requests is 60 seconds. I am currently scheduling a cron job every minute to call my activity worker file so that my activity worker machine is constantly polling for jobs.
Is this the correct way to have continuous queue coverage?
The way that the Java Flow SDK does it and the way that you create an ActivityWorker, give it a tasklist, domain, activity implementations, and a few other settings. You set both the setPollThreadCount and setTaskExecutorSize. The polling threads long poll and then hand over work to the executor threads to avoid blocking further polling. You call start on the ActivityWorker to boot it up and when wanting to shutdown the workers, you can call one of the shutdown methods (usually best to call shutdownAndAwaitTermination).
Essentially your workers are long lived and need to deal with a few factors:
New versions of Activities
Various tasklists
Scaling independently on tasklist, activity implementations, workflow workers, host sizes, etc.
Handle error cases and deal with polling
Handle shutdowns (in case of deployments and new versions)
I ended using a solution where I had another script file that is called by a cron job every minute. This file checks whether an activity worker is already running in the background (if so, I assume a workflow execution is already being processed on the current server).
If no activity worker is there, then the previous long poll has completed and we launch the activity worker script again. If there is an activity worker already present, then the previous poll found a workflow execution and started processing so we refrain from launching another activity worker.
1) I am currently working on a web application that exposes a REST api and uses Django and Celery to handle request and solve them. For a request in order to get solved, there have to be submitted a set of celery tasks to an amqp queue, so that they get executed on workers (situated on other machines). Each task is very CPU intensive and takes very long (hours) to finish.
I have configured Celery to use also amqp as results-backend, and I am using RabbitMQ as Celery's broker.
Each task returns a result that needs to be stored afterwards in a DB, but not by the workers directly. Only the "central node" - the machine running django-celery and publishing tasks in the RabbitMQ queue - has access to this storage DB, so the results from the workers have to return somehow on this machine.
The question is how can I process the results of the tasks execution afterwards? So after a worker finishes, the result from it gets stored in the configured results-backend (amqp), but now I don't know what would be the best way to get the results from there and process them.
All I could find in the documentation is that you can either check on the results's status from time to time with:
result.state
which means that basically I need a dedicated piece of code that runs periodically this command, and therefore keeps busy a whole thread/process only with this, or to block everything with:
result.get()
until a task finishes, which is not what I wish.
The only solution I can think of is to have on the "central node" an extra thread that runs periodically a function that basically checks on the async_results returned by each task at its submission, and to take action if the task has a finished status.
Does anyone have any other suggestion?
Also, since the backend-results' processing takes place on the "central node", what I aim is to minimize the impact of this operation on this machine.
What would be the best way to do that?
2) How do people usually solve the problem of dealing with the results returned from the workers and put in the backend-results? (assuming that a backend-results has been configured)
I'm not sure if I fully understand your question, but take into account each task has a task id. If tasks are being sent by users you can store the ids and then check for the results using json as follows:
#urls.py
from djcelery.views import is_task_successful
urlpatterns += patterns('',
url(r'(?P<task_id>[\w\d\-\.]+)/done/?$', is_task_successful,
name='celery-is_task_successful'),
)
Other related concept is that of signals each finished task emits a signal. A finnished task will emit a task_success signal. More can be found on real time proc.