When using the celerybeat in django, I would like to schedule a task to start at a specific time, and then run every 5 minutess. I was able to get the task to run every 5 minutes, using crontab(minute='*/5') and this will run after celerybeat is up but i want to run it for example at 8:30, how to do this?
First you set up your task to run every 5 minutes and you have already done that.
The second step is to wrap the body of your task into a conditional that checks if at the current time it should run or not.
Something like this
from django.utils import timezone
import datetime
#app.task
def my_task():
now = timezone.now().time()
start_time = datetime.time(8, 30, 0)
end_time = datetime.time(17, 30, 0)
if now >= start_time and now < end_time:
# your task ...
Related
I have a celery beat cron schedule set for day 3 of every week.
In django shell I can access this schedule:
>>> schedule = PeriodicTask.objects.get(name="Bulk Newsletter Send")
which results in:
>>> <PeriodicTask: Bulk Newsletter Send: * * * * 3 (m/h/dM/MY/d) UTC>
My question is, how can I calculate the time difference in days between datetime.now() and the next PeriodicTask?
This is the solution that I am going to run with:
Derived from Celery docs celery.schedules
periodic_task = PeriodicTask.objects.get(name="my_task")
time_to_go = periodic_task.schedule.is_due(datetime.now())
time_to_go.next = time to next run in seconds
Is there any way to make periodic_task to run on call only, I see Pingit() starts as soon as i run my django-app python manage.py runserver
#periodic_task(run_every=crontab(minute="*/1"),options={"task_id":task_name})
def Pingit():
print('Every Minute Im Called')
I Would like to make it run the periodic task only if i call it by Pingit.
You may be better to use a #task for this and get it to re-queue itself after it executes, for example:
#app.task
def pingit(count=0):
if count < 60 * 24 * 7: # 7 days in minutes
print('Every Minute Im Called')
# Queue a new task to run in 1 minute
pingit.apply_async(kwargs={'count': count + 1}, countdown=60)
# Start the task manually
pingit.apply_async()
If you need to add positional arguments to the function, you can specify those with args. For example, to pass a name argument:
#app.task
def pingit(name, count=0):
if count < 60 * 24 * 7: # 7 days in minutes
print('Every Minute Im Called')
# Queue a new task to run in 1 minute
pingit.apply_async(args=[name], kwargs={'count': count + 1}, countdown=60)
# Start the task manually
pingit.apply_async(args=['MyName'])
*Using celery 3.1.25 because django-celery-beat 1.0.1 has an issue with scheduling periodic tasks.
Recently I encountered an issue with celerybeat whereby periodic tasks with an interval of a day or longer appear to be 'forgotten' by the scheduler. If I change the interval to every 5 seconds the task executes normally (every 5 seconds) and the last_run_at attribute gets updated. This means celerybeat is responding to the scheduler to a certain degree, but if I reset the last_run_at i.e. PeriodicTask.objects.update(last_run_at=None), none of the tasks with an interval of every day run anymore.
Celerybeat crashed at one point and that may have corrupted something so I created a new virtualenv and database to see if the problem persists. I'd like to know if there is a way to retrieve the next run time so that I don't have to wait a day to know whether or not my periodic task has been executed.
I have also tried using inspect <active/scheduled/reserved> but all returned empty. Is this normal for periodic tasks using djcelery's database scheduler?
Here's the function that schedules the tasks:
def schedule_data_collection(request, project):
if (request.method == 'POST'):
interval = request.POST.get('interval')
target_project = Project.objects.get(url_path=project)
interval_schedule = dict(every=json.loads(interval), period='days')
schedule, created = IntervalSchedule.objects.get_or_create(
every=interval_schedule['every'],
period=interval_schedule['period'],
)
task_name = '{} data collection'.format(target_project.name)
try:
task = PeriodicTask.objects.get(name=task_name)
except PeriodicTask.DoesNotExist:
task = PeriodicTask.objects.create(
interval=schedule,
name=task_name,
task='myapp.tasks.collect_tool_data',
args=json.dumps([target_project.url_path])
)
else:
if task.interval != schedule:
task.interval = schedule
if task.enabled is False:
task.enabled = True
task.save()
return HttpResponse(task.interval)
else:
return HttpResponseForbidden()
You can see your scheduler by going into shell and looking at app.conf.CELERYBEAT_SCEDULE.
celery -A myApp shell
print(app.conf.CELERYBEAT_SCHEDULE)
This should show you all your Periodic Tasks.
How do I schedule a task to run when I start celery beat then again in 1 hours and so.
Currently I have schedule in settings.py:
CELERYBEAT_SCHEDULE = {
'update_database': {
'task': 'myapp.tasks.update_database',
'schedule': timedelta(seconds=60),
},
}
I saw a post from 1 year here on stackoverflow asking the same question:
How to run celery schedule instantly?
However this does not work for me, because my celery worker get 3-4 requests for the same task, when I run django server
I'm starting my worker and beat like this:
celery -A dashboard_web worker -B --loglevel=INFO --concurrency=10
Crontab schedule
You could try to use a crontab schedule instead which will run every hour and start 1 min after initialization of the scheduler. Warning: you might want to do it a couple of minutes later in case it takes longer to start, otherwise you might need to wait the full hour.
from celery.schedules import crontab
from datetime import datetime
CELERYBEAT_SCHEDULE = {
'update_database': {
'task': 'myapp.tasks.update_database',
'schedule': crontab(minute=(datetime.now().minute + 1) % 60),
},
}
Reference: http://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html#crontab-schedules
Ready method of MyAppConfig
In order to ensure that your task is run right away, you could use the same method as before to create the periodic task without adding 1 to the minute. Then, you call your task in the ready method of MyAppConfig which is called whenever your app is ready.
#myapp/apps.py
class MyAppConfig(AppConfig):
name = "myapp"
def ready(self):
from .tasks import update_database
update_database.delay()
Please note that you could also create the periodic task directly in the ready method if you were to use django_celery_beat.
Edit: Didn't see that the second method was already covered in the link you mentioned. I'll leave it here in case it is useful for someone else arriving here.
Try setting the configuration parameter CELERY_ALWAYS_EAGER = True
Something like this
app.conf.CELERY_ALWAYS_EAGER = True
I'm trying to setup a Periodic Task that should expire after some time. I'm using Django 1.5.1, celery 3.0.19 and django-celery 3.0.17 (everything from pip).
This is the excerpt code to create the task:
from django.utils import timezone
from datetime import timedelta, datetime
from djcelery.models import PeriodicTask, IntervalSchedule
interval = IntervalSchedule.objects.get(pk=1) # Added through fixture - 3sec interval
expiration = timezone.now() + timedelta(seconds=10)
task = PeriodicTask(name='fill_%d' % profile.id,
task='fill_album',
args=[instance.id],
interval=interval,
expires=expiration) task.save()
And I'm running celery with ./manage.py celeryd -B
The task is being created just fine, and beat is running it every 3 seconds, but after 10 seconds it doesn't expire. At first I thought it was some timezone issue between django and celery, so I let it running for 3 hours (my difference to UTC) but it still wouldn't expire.
During my tests I've actually managed to make it expire once (and the logger kept repeating it was expired, every 3 seconds) but I haven't been able to reproduce it since.
Can anyone shed some light on what I'm doing wrong?
Thanks!
I'm having the same problem and I think celery beat is not honoring the expires. If you set a breakpoint in your task take a look at the current_task.request object and see if expires has a value (or just print current_task.request from within the task.)
For me, if I manually run the task, current_task.request.expires has a value, but if celery beat schedules it, it is None.
I'm using celery 3.1.11
I filed a bug: https://github.com/celery/celery/issues/2283
You can try use last_run_at as:
task = PeriodicTask(name='fill_%d' % profile.id,
task='fill_album',
args=[instance.id],
interval=interval,
expires=expiration,
last_run_at=expiration)
task.save()