I have been trying a long for creating a periodic task in Django but there are lot of version constraints and not a clear explanation.
I recommend Celery. What is Celery?
Celery supports scheduling tasks. Check this doc
First of all, you want to create a management command following this guide.
https://docs.djangoproject.com/en/2.1/howto/custom-management-commands/
Say we want to run the closepoll command in the example every 5 minutes.
You'll then need to create a script to run this command.
Linux / MacOS:
#!/bin/bash -e
cd path/to/your/django/project
source venv/bin/activate # if you use venv
python manage.py closepoll # maybe you want to >> /path/to/log so you can log the results
store the file as run_closepoll.sh, run chmod +x run_closepoll.sh in command line
Now we can use crontab to run our command
run crontab -e in your command line
add this line:
*/5 * * * * /path/to/run_closepoll.sh
Now the command will run every 5 minutes.
If you're not familiar with crontab, you can use this website
https://crontab-generator.org/
Windows:
Same content as the above example, but remove the first line and save as run_closepoll.bat
In your start menu, search for Task Scheduler, follow the instructions on the GUI, it should be pretty simple from there.
for more info about the task scheduler, see here: https://learn.microsoft.com/en-us/windows/desktop/taskschd/using-the-task-scheduler
This blog explains clearly
https://medium.com/#yehandjoe/celery-4-periodic-task-in-django-9f6b5a8c21c7
Thanks!!!
I'm using django-cron and It works as expected. The only caveat is that you have to set a Cron job in the Linux system to run the command python manage.py runcrons.
Related
I want to run shell srcipt in EC2 Instance when i want to do.
so I thought 3 ways how can i do this problem.
To run shell script in EC2 from Lambda at a specific time using EventBridge.
https://aws.amazon.com/ko/blogs/compute/scheduling-ssh-jobs-using-aws-lambda/
To run SSM Run Command at a specific time using EventBridge
https://medium.com/the-cloud-architect/creating-your-own-chaos-monkey-with-aws-systems-manager-automation-6ad2b06acf20
To run shell script with cron by installing cron tab package on EC2
https://docs.aws.amazon.com/opsworks/latest/userguide/workingcookbook-extend-cron.html
Which method is the best in terms of performance or maintenance?
In my opinion, depending on the complexity of what you want to run, crontab is easy and lightweight. I am not entirely positive, but I'm pretty sure crontab is installed on EC2 by default.
To view the current scheduled cron entries, you can run the following: crontab -l
To edit the cron jobs, run the following: crontab -e
Note: It will use the default EDITOR which is typically either vi or vim.
You can find out more about the syntax for crontab here.
I wrote a bash script to create a scheduled task and run a django app. I am using Git Bash as my terminal, and I have been able to manually run the commands in the snippet posted below with success. However, when I run bash script with these same commands, the scheduled task and the django app are never run. Why is there a discrepancy in behavior and how can I correct this in my bash script?
#!/usr/bin/env bash
// Create scheduled task
echo schtasks //create //tn my-task //tr '"python app/manage.py loaddata /resources/output.json"' //sc daily //st 09:30 //ri 60 //et 16:00
// Run app
echo python app/manage.py runserver
echo "TERMINATED"
$SHELL
For a bash script to run from a CMD session (triggered by the Windows scheduler), you would need:
a script named git-xxx (replace xxx by the name of your chosing)
that script in your Windows PATH (as well as Git itself)
a schedule task running git xxx (note the space)
That would run git-xxx in a Git bash session.
The other option would be, still from a CMD (or a scheduled task) to run:
bash -c "/c/path/to/your/script"
In both instances, make sure bash is not the one from WSL if you are on Windows 10, and have activated that feature.
I have a machine learning project and I have to get data from a website every 15 minutes. And I cannot use my own computer so I will use Google cloud. I am trying to use Google Compute Engine and I have a script for getting data (here is the link: https://github.com/BurkayKirnik/Automatic-Crypto-Currency-Data-Getter/blob/master/code.py). This script gets data every 15 mins and writes it down to csv files. I can run this code by opening an SSH terminal and executing it from there but it stops working when I close the terminal. I tried to run it by executing it in startup script but it doesn't work this way too. How can I run this and save the csv files? BTW I have to install an API to run the code and I am doing it in startup script. There is no problem in this part.
Instances running in Google Cloud Platform can be configured with the same tools available in the operating system that they are running. If your instance is a Linux instance, the best method would be to use a cronjob to execute your script repeatedly at your chosen interval.
Once you have accessed the instance via SSH, you can open the crontab configuration file by running the following command:
$ crontab -e
The above command will provide access to your personal crontab configuration (for the user you are logged in as). If you want to run the script as root you can use this instead:
$ sudo crontab -e
You can now edit the crontab configuration and add an entry that tells cron to execute your script at your required interval (in your case every 15 minutes).
Therefore, your crontab entry should look something like this:
*/15 * * * * /path/to/you/script.sh
Notice the first entry is for minutes, so by using the */15, you are telling the cron daemon to execute the script once every 15 minutes.
Once you have edited the crontab configuration file, it is a good idea to restart the cron daemon to ensure the change you made will take place. To do this you can run:
$ sudo service cron restart
If you would like to check the status to ensure the cron service is running you can run:
$ sudo service cron status
You script will now execute every 15 minutes.
In terms of storing the CSV files, you could either program your script to store them on the instance, or an alternative would be to use Google Cloud Storage bucket. File can be copied to buckets easily by making use of the gsutil (part of Cloud SDK) command as described here. It's also possible to mount buckets as a file system as described here.
I want to run my development django server at startup so I defined following cron job:
#reboot screen -d -m django-admin.py runserver 192.168.0.28:8000
But it didn't work.
What is really interesting, when I copy/paste directly to terminal and execute it works just fine.
I even tried something like this:
#reboot cd /home/ubuntu && /usr/bin/screen -d -m /usr/bin/python /usr/local/bin/django-admin.py runserver 192.168.0.28:8000 &> /home/ubuntu/cron.err
To be sure I'm not using some undefined commands in wrong location and examined contents of cron.err file but it's empty.
And (of course) when I fire this directly from the console it works immediately.
Please help.
Does it work if you try and run it from cron at a specific time? Eg:
50 12 2 8 * /usr/bin/screen -dmS set_from_cron
I want my custom made Django command to be executed every minute. However it seems like python /path/to/project/myapp/manage.py mycommand doesn't seem to work while at the directory python manage.py mycommand works perfectly.
How can I achieve this ? I use /etc/crontab with:
****** root python /path/to/project/myapp/manage.py mycommand
I think the problem is that cron is going to run your scripts in a "bare" environment, so your DJANGO_SETTINGS_MODULE is likely undefined. You may want to wrap this up in a shell script that first defines DJANGO_SETTINGS_MODULE
Something like this:
#!/bin/bash
export DJANGO_SETTINGS_MODULE=myproject.settings
./manage.py mycommand
Make it executable (chmod +x) and then set up cron to run the script instead.
Edit
I also wanted to say that you can "modularize" this concept a little bit and make it such that your script accepts the manage commands as arguments.
#!/bin/bash
export DJANGO_SETTINGS_MODULE=myproject.settings
./manage.py ${*}
Now, your cron job can simply pass "mycommand" or any other manage.py command you want to run from a cron job.
cd /path/to/project/myapp && python manage.py mycommand
By chaining your commands like this, python will not be executed unless cd correctly changes the directory.
If you want your Django life a lot more simple, use django-command-extensions within your project:
http://code.google.com/p/django-command-extensions/
You'll find a command named "runscript" so you simply add the command to your crontab line:
****** root python /path/to/project/myapp/manage.py runscript mycommand
And such a script will execute with the Django context environment.
This is what i have done recently in one of my projects,(i maintain venvs for every project i work, so i am assumning you have venvs)
***** /path/to/venvs/bin/python /path/to/app/manage.py command_name
This worked perfectly for me.
How to Schedule Django custom Commands on AWS EC-2 Instance?
Step -1
First, you need to write a .cron file
Step-2
Write your script in .cron file.
MyScript.cron
* * * * * /home/ubuntu/kuzo1/venv/bin/python3 /home/ubuntu/Myproject/manage.py transfer_funds >> /home/ubuntu/Myproject/cron.log 2>&1
Where * * * * * means that the script will be run at every minute. you can change according to need (https://crontab.guru/#*_*_*_*_*). Where /home/ubuntu/kuzo1/venv/bin/python3 is python virtual environment path. Where /home/ubuntu/kuzo1/manage.py transfer_funds is Django custom command path & /home/ubuntu/kuzo1/cron.log 2>&1 is a log file where you can check your running cron log
Step-3
Run this script
$ crontab MyScript.cron
Step-4
Some useful command
1. $ crontab -l (Check current running cron job)
2. $ crontab -r (Remove cron job)
The runscript extension wasn't well documented. Unlike the django command this one can go anywhere in your project and requires a scripts folder. The .py file requires a run() function.
If its a standalone script, you need to do this:
from django.conf import settings
from django.core.management import setup_environ
setup_environ(settings)
#your code here which uses django code, like django model
If its a django command, its easier: https://coderwall.com/p/k5p6ag
In (management/commands/exporter.py)
from django.core.management.base import BaseCommand, CommandError
class Command(BaseCommand):
args = ''
help = 'Export data to remote server'
def handle(self, *args, **options):
# do something here
And then, in the command line:
$ python manage.py exporter
Now, it's easy to add a new cron task to a Linux system, using crontab:
$ crontab -e
or $ sudo crontab -e if you need root privileges
In the crontab file, for example for run this command every 15 minutes, something like this:
# m h dom mon dow command
*/15 * * * * python /var/www/myapp/manage.py exporter