Shutting Off CELERY.BACKEND_CLEANUP on Amazon SQS - django

Question

I'm using Django with Celery. I need to turn off the celery.backend_cleanup that runs every day at 4 UTC. I've been looking at the documentation and can't find how to disable it. Below is my last try:
celery.py
from __future__ import absolute_import, unicode_literals
from django.conf import settings
from celery import Celery
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE",
"settings")
app = Celery('app')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()
app.conf.beat_schedule = {
'backend_cleanup': {
'task': 'celery.backend_cleanup',
'schedule': None,
'result_expires': None
},
}
I don't want this to run. How can I stop it?
UPDATE:
I also tried adding this to settings.py
CELERYBEAT_SCHEDULE = {
'backend_cleanup': {
'task': 'celery.backend_cleanup',
'schedule': 0,
'result_expires': 0
},
}
I know deleting task in db is an option, but if later on beat has to be restarted it creates the backend_cleanup again and it starts running it. I may not be the person maintaining this in the future, so I need this configured in the code not manually deleted from database.

Answer 1

Here are a few approaches you can try:
You could use a crontab definition that runs once far in the future, e.g.:
app.conf.beat_schedule = {
# Disable cleanup task by scheduling to run every ~1000 years
'backend_cleanup': {
'task': 'celery.backend_cleanup',
'schedule': timedelta(days=365*1000),
'relative': True,
},
}
You can try setting app.backend.supports_autoexpire = True, because this attribute is checked before the default backend_cleanup task is added (src).
You can create a subclass your backend class and set supports_autoexpire = True

Answer 2

Simply set the result_expires setting equal to None, but define it as a global setting in your settings.py, and not a setting bound to a particular task (eg. backend_cleanup in your example). You could also pass it into your app declaration directly.
The docs:
Default: Expire after 1 day.
Time (in seconds, or a timedelta object) for when after stored task tombstones will be deleted.
A built-in periodic task will delete the results after this time (celery.backend_cleanup), assuming that celery beat is enabled. The task runs daily at 4am.
A value of None or 0 means results will never expire (depending on backend specifications).
Note that this only works with the AMQP, database, cache, Couchbase, and Redis backends, and not on SQS!
The docs on SQS:
Warning
Don’t use the amqp result backend with SQS.
It will create one queue for every task, and the queues will not be collected. This could cost you money that would be better spent contributing an AWS result store backend back to Celery :)

Answer 3

You may be using django-celery-beat. if so, you'll have to delete the scheduled task from your database. You can do this through the django admin panel.