I have an Django project, that has some functionality to run as cron job several times i.e(every half an hour I need this functionality to be executed).
Till now the job is scheduling but not executing the function. Here I am attaching the code below:
from __future__ import unicode_literals
from django.apps import AppConfig
from projectApp.views import function_to_exec
from django_redis import get_redis_connection
rc = get_redis_connection('default')
from rq_scheduler import Scheduler
scheduler = Scheduler(connection=rc)
def ready():
for job in scheduler.get_jobs():
job.delete()
scheduler.schedule(datetime.utcnow(), function_to_exec, interval=60, queue_name='high')
# scheduler.cron("15 * * * *", func=get_dfp_report, queue_name='high')
ready();
The above code is in my application's apps.py
and the views.py code is like this :
#job('high')
def function_to_exec():
# some logic here
And in my django-scheduler the status is
The status is always in queued state.
Can anyone share the some reference for this to achieve.
Thanks in advance.
Have you started the rqscheduler from the command line to make sure that the jobs are executed?
The scheduler can be started with
rqscheduler
Use -v if you need verbose output
rqscheduler -v
Documentation
Related
I am trying to execute a task with celery in Django.I want to execute the task at 12:30 pm everyday for which I have written this in my tasks.py
#periodic_task(run_every=crontab(minute=30, hour=12), name="elast")
def elast():
do something
This is not working but if I want to schedule it at every 30 seconds I write this code
#periodic_task(run_every=(timedelta(seconds=30)), name="elast")
def elast():
do something
This works.I wanted to know that what is wrong with the first piece of code?Any help would be appreciated.
As per latest celery 4.3 version , to execute the task at 12:30 pm below code will be useful
celery.py
from celery.schedules import crontab
app.conf.beat_schedule = {
# Executes every day at 12:30 pm.
'run-every-afternoon': {
'task': 'tasks.elast',
'schedule': crontab(hour=12, minute=30),
'args': (),
},
}
tasks.py
import celery
#celery.task
def elast():
do something
to start celery beat scheduler
celery -A proj worker -B
for older version around celery 2.0
from celery.task.schedules import crontab
from celery.decorators import periodic_task
#periodic_task(run_every=crontab(hour=12, minute=30))
def elast():
print("code execution started.")
please check timezone setting.
New userguide
Old userguide
Check out the documentation, especially the parts specific for Django users. Also note that using #periodic_task decorator is deprecated and should be replaced with beat_schedule configuration (see the code).
i work on a celery beat task within a django project which creates Database entries periodically. I know so beacuse when i set the task up like this :
celery.py:
from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
from celery.schedules import crontab
app = Celery("clock-backend", broker=os.environ.get("RABBITMQ_URL"))
app.config_from_object("django.conf:settings", namespace="CELERY")
app.conf.beat_schedule = {
'create_reports_monthly': {
'task': 'project_celery.tasks.create_reports_monthly',
'schedule': 10.0,
},
}
app.autodiscover_tasks()
And start my project it really creates an object every 10 seconds.
But what i really want to do is to set it up to run every first day of a month.
To do so i would change "schedule": crontab(0, 0, day_of_month="1").
Here comes my actual problem : How do i test that this really works ?
And by testing i mean actual (unit)tests.
What I've tried is to work with a package called freezegun.
A test with this looks like this :
def test_start_of_month_report_creation(self, user_object, contract_object, report_object):
# set time to the last day of January
with freeze_time("2019-01-31 23:59:59") as frozen_time:
# let one second pass
frozen_time.tick()
# give the celery task some time
time.sleep(20)
# Test Logic to check whether the object was created
# Example: assert MyModel.objects.count() > 0
But this did not work. I suspect that the celery beat does not use the time set via freezgun/python but the real "hardware" clock.
I've also tried setting the Hardwareclock like here but this did not work in my setup.
I'm thankful for any comments, remarks or help on this topic since i'd really like to implement a test for this.
Unit tests cannot test third-party libraries.
You can set the system log, to keep track.
You can check if your task is already on model PeriodicTask. This model defines a single periodic task to be run. It must be associated with a schedule, which defines how often the task should run.
Suppose I have 1000 user_ids in a table and I would run every hour to get from Google API info and update 3 fields in that table. How would the impact be and how can it be done efficiently?
I've seen this variant:
m = Module.objects.get(user_id=1).update(field_one=100, field_two=200, field_three=300)
And this one:
m = Module.objects.get(user_id=1)
m.field_one = 100
m.field_two = 200
m.field_three = 300
m.save()
Also how can it be done so that it will run every hour and grab that information? Never done something like this.
Use Redis, Celery to setup asynchronous task queue every hour. Look here https://realpython.com/blog/python/asynchronous-tasks-with-django-and-celery/ for more info on how to setup asych task queue system for django.
Here is the code for tasks.py
from celery.task import periodic_task
from celery.schedules import crontab
#periodic_task(run_every=crontab(minute=0, hour='*/1'))
def get_data_from_google_api():
data_from_google =ping_google_api() # ping google api to get data
return Module.objects.get(user_id=1).update(field_one= data_from_google['field_one'], field_two= data_from_google['field_two'], field_three= data_from_google['field_three'])
Look here for more info :
https://www.caktusgroup.com/blog/2014/06/23/scheduling-tasks-celery/
How to run a Django celery task every 6am and 6pm daily?
Fof this purpose you need to run background queries with periodic taks.
Here is most popular in django task-queue-libs
For example, if you decide use celery, you can write simple periodic task:
from celery.schedules import crontab
from celery.task import periodic_task
#periodic_task(
name='UPDATE_USER',
run_every=crontab(
minute='1',
hour='1,4,7,10,13,16,19,22'))
def update_user():
#get some value from api
Module.objects.filter(user_id=1).update(
field_one=value, field_two=value, field_three=value)
All settings for django you can look in celery docs
I configured django-celery in my application. This is my task:
from celery.decorators import task
import simplejson as json
import requests
#task
def call_api(sid):
try:
results = requests.put(
'http://localhost:8000/api/v1/sids/'+str(sid)+"/",
data={'active': '1'}
)
json_response = json.loads(results.text)
except Exception, e:
print e
logger.info('Finished call_api')
When I add in my view:
call_api.apply_async(
(instance.service.id,),
eta=instance.date
)
celeryd shows me:
Got task from broker: my_app.tasks.call_api[755d50fd-0f0f-4861-9a18-7f4e4563290a]
Task my_app.tasks.call_api[755d50fd-0f0f-4861-9a18-7f4e4563290a] succeeded in 0.00513911247253s: None
so should be good, but nothing happen... There is no call to for example:
http://localhost:8000/api/v1/sids/1/
What am I doing wrong?
Are you running celery as a separate process?
For example in Ubuntu run using the command
sudo python manage.py celeryd
Till you run celery (or django celery) as a separate process, the jobs will be stored in the database (or queue or the persistent mechanism you have configured - generally in settings.py).
I would like to compute some information in my Django application on regular basis.
I need to select and insert data each second and want to use Django ORM.
How can I do this?
In a shell script, set the DJANGO_SETTINGS_MODULE variable and call a python script
export DJANGO_SETTINGS_MODULE=yourapp.settings
python compute_some_info.py
In compute_some_info.py, set up django and import your modules (look at how the manage.py script sets up to run Django)
#!/usr/bin/env python
import sys
try:
import settings # Assumed to be in the same directory.
except ImportError:
sys.stderr.write("Error: Can't find the file 'settings.py'")
sys.exit(1)
sys.path = sys.path + ['/yourapphome']
from yourapp.models import YourModel
YourModel.compute_some_info()
Then call your shell script in a cron job.
Alternatively -- you can just keep running and sleeping (better if it's every second) -- you would still want to be outside of the webserver and in your own process that is set up this way.
One way to do it would be to create a custom command, and invoke python manage.py your_custom_command from cron or windows scheduler.
http://docs.djangoproject.com/en/dev/howto/custom-management-commands/
For example, create myapp/management/commands/myapp_task.py which reads:
from django.core.management.base import NoArgsCommand
class Command(NoArgsCommand):
def handle_noargs(self, **options):
print 'Doing task...'
# invoke the functions you need to run on your project here
print 'Done'
Then you can run it from cron like this:
export DJANGO_SETTINGS_MODULE=myproject.settings; export PYTHONPATH=/path/to/project_parent; python manage.py myapp_task