I am new to Celery and Django. I got the task id using
task_id = task.request.id
but not able to get the task state.
Any suggestion to get the task state? Any help would be appreciated. Thanks!
I got the status of task as below
task.py
from celery.result import AsyncResult
result = task_name.AsyncResult(task_name.request.id)
taskStatus = result.state
celeryconfig.py
CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'
CELERY_IGNORE_RESULT = False
CELERY_TRACK_STARTED = True
Related
I trying to run tasks with celery-beat. When I launch it by minute or hour schedule, tasks will start correctly, but if I trying to run daily task, it display in django admin panel, but not run in time.
It must to work in the following way: regular django code starts a 'start_primaries' task in Party class:
def setup_task(self):
schedule, created = IntervalSchedule.objects.get_or_create(every=7, period=IntervalSchedule.DAYS)
self.task = PeriodicTask.objects.create(
name=self.title + ', id ' + str(self.pk),
task='start_primaries',
interval=schedule,
args=json.dumps([self.id]),
start_time=timezone.now()
)
self.save()
Is it possible that there are some settings that limit the duration of the task's life? At the moment I have the following among the Django settings:
CELERY_TASK_TRACK_STARTED = True
CELERY_TASK_TIME_LIMIT = 950400
CELERY_BROKER_URL = 'redis://redis:6379/0'
CELERY_RESULT_BACKEND = 'django-db'
CELERY_BEAT_SCHEDULER = 'django_celery_beat.schedulers:DatabaseScheduler'
CELERY_TASK_RESULT_EXPIRES = None
CELERY_BROKER_TRANSPORT_OPTIONS = {'visibility_timeout': 950400}
Finally,I found the anwser in github.
First, You should set 'last_run_at' to start_time - interval, so that beat will check when it was last "executed", so the next time it executes will be at start_time.
Second, update your start_time setting, and let the scheduler get the updated info.
Good luck.
https://github.com/celery/django-celery-beat/issues/259
In my views.py I am using celery to run a shared task present in tasks.py.
Here is how I call from views.py
task = task_addnums.delay()
task_id = task.id
tasks.py looks as
from celery import shared_task
from celery.result import AsyncResult
#shared_task
def task_addnums():
# print self.request.id
# do something
return True
Now, as we can see we already have task_id from task.id in views.py . But, Let's say If I want to fetch task id from the shared_task itself how can I ? The goal is to get task id from the task_addnums itself so I can use that to pass into some other function.
I tried using self.request.id considering the first param is self . But it didn't worked.
Solved.
This answer is a gem Getting task_id inside a Celery task
You can do function_name.request.id to get task id.
current_task from celery will get the current task.Code like this:
from celery import shared_task, current_task
#shared_task
def task_addnums():
print(current_task.request)
# do something
return True
This is a model for all assignment taken by any user. How can I delete a particular instance of this when the user hasn't submitted his assignment in two days. After submission user data is saved in subassignment model.
class UserAssignment(models.Model):
user = models.ForeignKey(settings.AUTH_USER_MODEL)
assignment = models.ForeignKey(Assignment)
time_taken = models.DateTimeField(auto_now_add=True)
submitted = models.DateTimeField(null=True, blank = True)
class SubAssignment(models.Model):
user = models.ForeignKey(settings.AUTH_USER_MODEL)
assignment = models.ForeignKey(Assignment)
time_submitted =models.DateTimeField(blank = True, null = True)
score = models.IntegerField(default=0)
pip install django-celery==3.2.2
add 'djcelery' to INSTALLED_APPS
add task to check all UserAssignment everyday in 1 am:
tasks.py:
from celery import task
from .models import *
#task
def check_user_assignment():
for user_assignment in UserAssignment.objects.all():
# check all every day,if need to delete,then delete it
pass
add this task to
settings.py
BROKER_URL = 'amqp://root:root#localhost:5672/'
CELERY_TIMEZONE = 'Asia/Shanghai'
CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler'
CELERYBEAT_SCHEDULE = {
'check_user_assignment_everyday': {
'task': 'user.tasks.check_user_assignment',
'schedule': crontab(minute=0, hour=1),
'args': (),
},
}
run celery :
input in terminal
python manage.py celery beat -l info
python manage.py celery worker -E -l info
You have to use timedelta:
from datetime import datetime, timedelta
# Some of your code
# Let's create 2 day threshold
threshold = datetime.now()-timedelta(days=2)
missed_assignment = UserAssignment.objects.filter(user=user,
time_taken__gte=threshold)
missed_assignment.delete()
missed_subassignment = SubAssignment.objects.filter(user=user,
time_submitted__gte=threshold)
missed_subassignment.delete()
Note that time_submitted__gte means "time passed is more than or at threshold".
If you do not actually run code, you may use rq worker -https://github.com/rq/django-rq or just a cron tab that periodically checks.
After an update it appears celery has stopped to work like it should.
I have periodic tasks per day and from 6-22.
All of the tasks running from 6-22 run every 5 minutes for no reason.
I changed the task from running every hour to running from 6-22. The every hour function wasn't working either.
I tried:
minute=0 hour=6-22, minute=0, hour='*/3,8-17' and completely written like below.
The last one i copied from the docs because i thought maybe this would work.
#periodic_task(
run_every=(crontab(minute=0, hour='6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22')),
queue='feed',
name="feed_update",
ignore_result=True
)
def feed_update():
"""
checks for feed updates
"""
feed_update_for_all_users()
logger.info("Feed Update complete")
settings.py
#CELERY STUFF
CELERY_IMPORTS = ('reviews.tasks',)
CELERY_TIMEZONE = 'Europe/Berlin'
BROKER_URL = 'redis://127.0.0.1:6379'
BROKER_TRANSPORT_OPTIONS = {'visibility_timeout': 7776000}
CELERY_RESULT_BACKEND = 'redis://127.0.0.1:6379'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_ENABLE_UTC = False
According to the Docs these are all valid contrab vars. Why isn't it working properly?
After reading through every Bug Report in Celery.
Celery doesn't accept any other timestamp!!!
Using CELERY_ENABLE_UTC = False, pretty much wrecks the Programm.
Once you enable it, the problem will be solved.
However if you have any eta functions you have to adjust the time before sending it to celery and of course if you have exact timeframes adjust these as well.
I'm struggling with Django, Celery, djcelery & PeriodicTasks.
I've created a task to pull a report for Adsense to generate a live stat report. Here is my task:
import datetime
import httplib2
import logging
from apiclient.discovery import build
from celery.task import PeriodicTask
from django.contrib.auth.models import User
from oauth2client.django_orm import Storage
from .models import Credential, Revenue
logger = logging.getLogger(__name__)
class GetReportTask(PeriodicTask):
run_every = datetime.timedelta(minutes=2)
def run(self, *args, **kwargs):
scraper = Scraper()
scraper.get_report()
class Scraper(object):
TODAY = datetime.date.today()
YESTERDAY = TODAY - datetime.timedelta(days=1)
def get_report(self, start_date=YESTERDAY, end_date=TODAY):
logger.info('Scraping Adsense report from {0} to {1}.'.format(
start_date, end_date))
user = User.objects.get(pk=1)
storage = Storage(Credential, 'id', user, 'credential')
credential = storage.get()
if not credential is None and credential.invalid is False:
http = httplib2.Http()
http = credential.authorize(http)
service = build('adsense', 'v1.2', http=http)
reports = service.reports()
report = reports.generate(
startDate=start_date.strftime('%Y-%m-%d'),
endDate=end_date.strftime('%Y-%m-%d'),
dimension='DATE',
metric='EARNINGS',
)
data = report.execute()
for row in data['rows']:
date = row[0]
revenue = row[1]
try:
record = Revenue.objects.get(date=date)
except Revenue.DoesNotExist:
record = Revenue()
record.date = date
record.revenue = revenue
record.save()
else:
logger.error('Invalid Adsense Credentials')
I'm using Celery & RabbitMQ. Here are my settings:
# Celery/RabbitMQ
BROKER_HOST = "localhost"
BROKER_PORT = 5672
BROKER_USER = "myuser"
BROKER_PASSWORD = "****"
BROKER_VHOST = "myvhost"
CELERYD_CONCURRENCY = 1
CELERYD_NODES = "w1"
CELERY_RESULT_BACKEND = "amqp"
CELERY_TIMEZONE = 'America/Denver'
CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler'
import djcelery
djcelery.setup_loader()
On first glance everything seems to work, but after turning on the logger and watching it run I have found that it is running the task at least four times in a row - sometimes more. It also seems to be running every minute instead of every two minutes. I've tried changing the run_every to use a crontab but I get the same results.
I'm starting celerybeat using supervisor. Here is the command I use:
python manage.py celeryd -B -E -c 1
Any ideas as to why its not working as expected?
Oh, and one more thing, after the day changes, it continues to use the date range it first ran with. So as days progress it continues to get stats for the day the task started running - unless I run the task manually at some point then it changes to the date I last ran it manually. Can someone tell me why this happens?
Consider creating a separate queue with one worker process and fixed rate for this type of tasks and just add the tasks in this new queue instead of running them in directly from celerybeat. I hope that could help you to figure out what is wrong with your code, is it problem with celerybeat or your tasks are running longer than expected.
#task(queue='create_report', rate_limit='0.5/m')
def create_report():
scraper = Scraper()
scraper.get_report()
class GetReportTask(PeriodicTask):
run_every = datetime.timedelta(minutes=2)
def run(self, *args, **kwargs):
create_report.delay()
in settings.py
CELERY_ROUTES = {
'myapp.tasks.create_report': {'queue': 'create_report'},
}
start additional celery worker with that would handle tasks in your queue
celery worker -c 1 -Q create_report -n create_report.local
Problem 2. Your YESTERDAY and TODAY variables are set at class level, so within one thread they are set only once.