I'm having trouble getting my cron jobs to execute.
Setup:
Django - 1.9
Elastic beanstalk - 64bit Amazon Linux 2016.03 v2.1.3 running Python 3.4
I've tried doing this a couple of ways so far:
Using a cron.yaml file:
Didn't touch anything else - just added a cron.yaml file to my project root folder
version: 1
cron: - name: "test"
url: "http://website.com/workers/test"
schedule: "*/10 * * * *"
Using a container command and a separate cron.txt file:
Added this line in my .ebextensions/development.config file
05_some_cron:
command: "cat .ebextensions/crontab.txt > /etc/cron.d/crontab && chmod 644 /etc/cron.d/crontab"
leader_only: true
and in .ebextensions/crontab.txt
*/10 * * * * source /opt/python/run/venv/bin/activate && python mysite/manage.py test
The app deploys successfully in both cases.
Manually (in a browser) going to http://website.com/workers/test has
the intended outcome (in the first case).
Adding source /opt/python/run/venv/bin/activate && python mysite/manage.py test as a management command runs the correct script once on deploying.
The logs do not show any GETS on that url.
What am I doing wrong? Am I missing some step of the process or some setup step on EBS?
Also what is the best ways to run cron jobs for django applications hosted on EBS? - django apps can run management commands either from the command line as in attempt 2 or by extending a GET or POST url as in attempt 1.
2018 Update using Django 2.0.6 + AWS ElasticBeanstalk + Cronjob
I found I needed to export the AWS environment variables using source /opt/python/current/env to prevent manage.py from throwing the error "The SECRET_KEY setting must not be empty".
This is because I had placed my Django Secret key in the os.environ for beanstalk which it seems is not accessible by shell/cron by default.
See https://forums.aws.amazon.com/thread.jspa?threadID=108465
My final cron job .txt script was as follows. (process_emails is my own django management function, myjob.log is where the output of the function call is logged).
*/15 * * * * source /opt/python/run/venv/bin/activate && cd /opt/python/current/app/ && source /opt/python/current/env && python manage.py process_emails >> /var/log/myjob.log 2>&1
This works for me in Django 1.10.6 + AWS ElasticBeanstalk + Cronjob
* * * * * source /opt/python/run/venv/bin/activate && cd /opt/python/current/app/ && python manage.py do_your_thing > $HOME/cron-sqs.log 2>&1
do_your_thing is a management command
Related
I am trying to set a cron job on my project to send a email after 15 minutes.
This is in django.config
04_cronjb:
command: "cat .ebextensions/cron-linux.config > /etc/cron.d/crontab && chmod 644 /etc/cron.d/crontab"
leader_only: true
This is my cron-linux.config file
files:
"/etc/cron.d/mycron":
mode: "000644"
owner: root
group: root
content: |
* * * * * source /opt/python/current/env && python /opt/python/current/app/manage.py cronjb
It all deploys successfully but i am not receiving any email.
Cronjb script is working i have tested it. So the error is one these two files.
Is there some mistake in it?
Your 04_cronjb copies entire content of cron-linux.config into crontab. This is sadly incorrect.
Instead you should do as shown here. This includes putting all the bash commands you want to execute in a custom script called, e.g., myscript.sh and then adding myscript.sh to cron only.
I'm a newbie developer and I am messing around with Symfony and AWS. I have a Symfony app deployed via elastic beanstalk and I am trying to add a cron job every minute.
Basically I want the cron job to execute a Symfony command I created : update-pings, that is supposed to update some fields in my RDS DB.
I'm using a 02-crons.config file in my ./ebextensions directory that looks like this (greatly inspired from AWS documentation https://aws.amazon.com/fr/premiumsupport/knowledge-center/cron-job-elastic-beanstalk/ ) :
files:
"/etc/cron.d/mycron":
mode: "000644"
owner: root
group: root
content: |
* * * * * root php /var/app/current/bin/console app:update-pings
commands:
remove_old_cron:
command: "rm -f /etc/cron.d/mycron.bak"
Well, first of all, this is not working, the fields in the DB are not updating. So i connected in SSH to my EC2 instance to see what's happening.
The mycron file in /etc/cron.d/was correctly created with * * * * * root php /var/app/current/bin/console app:update-pingsinside of it.
Now, I tried replacing the content of this file by a simple * * * * * root echo test >> /tmp/cron_temp, this is working perfectly, a line with 'test' is added in my cron_temp file every minute
I tried running the initial command php /var/app/current/bin/console app:update-pingsmanually on the instance, works perfectly as ec2-user
When I switch to root user using sudo su -, the command still works. But when I try to run it with sudo, it fails, not sure if that's any relevant :
In EnvVarProcessor.php line 171:
Environment variable not found: "DATABASE_DBNAME".
I believe there is something I'm missing with linux users/rights, but I'm a total newbie with that and cannot find what's going on
Any ideas ? Thanks a lot in advance ! :)
Edit 1 : I'm know logging the sterr of the commande into a file and I get the same environment variable error as when I try to run the command manually with sudo
In EnvVarProcessor.php line 171:
Environment variable not found: "DATABASE_DBNAME".
The command works fine without sudo though
I have written few django management commands that I want to run hourly, daily and weekly basis. I'm using Elastic Beanstalk and created a worker instance where the code is deployed. Can someone help me how to run the django management command with crontab using elastic beanstalk. Thanks
Here is my management command:
python manage.py command_name
Please help me write the container_command in .ebextensions/django.config file for crontab that will schedule the command on hourly basis. Thanks
Any help?
in .ebextensions create a file crontab.txt
# Set the cron to run with utf8 encoding
PYTHONIOENCODING=utf8
0 1 * * * root source /opt/python/current/env && nice /opt/python/current/app/manage.py command_name
# this file needs a blank space as the last line otherwise it will fail
This is a crontab file that will run command_name at 1 am every day. You'll need to get you path right which depends on your file structure.
In one of your config files include:
02_cron_job:
command: "cp .ebextensions/crontab.txt /etc/cron.d/my_cron_jobs && chmod 644 /etc/cron.d/my_cron_jobs"
leader_only: true
This is going to copy your cron file into the right place on the ec2 instance so the job runs.
I have been working on configuring a cron job while deploying an environment on Elastic Beanstalk. The purpose of the job is to run every 30 minutes and execute a Node.js script that processes SMS schedules in our system; when a schedule is ready, an SMS message is sent via the Twilio api.
Unfortunately, Node.js environments don't have the file /opt/elasticbeanstalk/support/envvars containing the environment variables defined (Maybe I am missing something?).
To work around this issue, I am loading the environment variables from /opt/elasticbeanstalk/bin/get-config in Python and then executing my Node.js script.
Everything is working as expected so hopefully this can help someone in the same situation; however, I am wondering if there is a better way to accomplish this... Open to suggestions.
In my .ebextensions folder I have the config file for the cron:
files:
"/etc/cron.d/process-sms-schedules":
mode: "000644"
owner: root
group: root
content: |
*/30 * * * * root /usr/local/bin/process-sms-schedules.sh
"/usr/local/bin/process-sms-schedules.sh":
mode: "000755"
owner: root
group: root
content: |
#!/bin/bash
# Execute script to process schedules
python /var/app/current/process-sms-schedules.py > /var/log/process-sms-schedules.log
exit 0
commands:
remove_old_cron:
command: "rm -f /etc/cron.d/*.bak"
Here is the Python script that gets executed as part of the cron job:
#!/usr/bin/env python
import os
import subprocess
from subprocess import Popen, PIPE, call
import simplejson as json
envData = json.loads(Popen(['/opt/elasticbeanstalk/bin/get-config', 'environment'], stdout = PIPE).communicate()[0])
for k, v in envData.iteritems():
os.environ[k] = v
call(["babel-node", "/var/app/current/process-sms-schedules.js"])
Thanks in advance for any feedback.
References
Cron Job Elastic Beanstalk
How to set environment variable in Amazon Elastic Beanstalk (Python)
I had this issue while trying to execute a PHP file inside an Elastic Beanstalk environment. In particular I was trying to execute wp-cron.php file.
Basically you should write a cron job like this:
/etc/cron.d/wp-cronjob :
PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/aws/bin:/home/ec2-user/.local/bin:/home/ec2-user/bin
*/5 * * * * ec2-user . /opt/elasticbeanstalk/support/envvars; /usr/bin/php /var/www/html/wp-cron.php > /dev/null 2>&1
Explained:
Every 5 minutes executes commands as user "ec2-user": '*/5 * * * * ec2-user'
Loads elasticbeanstalk environment variables: '. /opt/elasticbeanstalk/support/envvars;'.
Do not print any output: '> /dev/null 2>&1'.
Also:
.ebextensions/wp-cronjob.txt
# Load paths
PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/aws/bin:/home/ec2-user/.local/bin:/home/ec2-user/bin
# Every 5 minutes executes commands as user "ec2-user": '*/5 * * * * ec2-user'.
# Loads elasticbeanstalk environment variables: '. /opt/elasticbeanstalk/support/envvars;'.
# Do not print any output: '> /dev/null 2>&1'.
*/5 * * * * ec2-user . /opt/elasticbeanstalk/support/envvars; /usr/bin/php /var/www/html/wp-cron.php > /dev/null 2>&1
.ebextensions/cronjob.config
container_commands:
add_wp_cronjob:
command: "cat .ebextensions/wp-cronjob.txt > /etc/cron.d/wp-cronjob && chmod 644 /etc/cron.d/wp-cronjob"
leader_only: true
commands:
remove_old_cron:
command: "rm -f /etc/cron.d/*.bak"
Maybe is not the same for a Node.js environment but I am pretty sure they are similar.
I have a web app running on elastic beanstalk. For some reason I was able to install the composer files in order to run my laravel app. The problem is that no other config file works. I have put newrelic.config into the .ebextensions/ directory, but that file got ignored.
I recently tried to create a cron job using this, AWS Elastic Beanstalk, running a cronjob, but it is not working.
Example of a .config file:
container_commands:
01_some_cron_job:
command: "cat .ebextensions/some_cron_job.txt > /etc/cron.d/some_cron_job && chmod 644 /etc/cron.d/some_cron_job"
leader_only: true
When I ssh into the ec2 instance, there is no such directory as some_cron_job.
The source gets committed to beanstalk, but beanstalk is not running the commands.
How can I make beanstalk acknowledge the .config files. Fixing this cronjob will also fix installing new relic, because both configs are being ignored and I do not know why.
Try putting it in commands section. It is more of a server command than a container command.
commands:
01_some_cron_job:
command: "cat .ebextensions/some_cron_job.txt > /etc/cron.d/some_cron_job && chmod 644 /etc/cron.d/some_cron_job"
leader_only: true
I had similar issues as well using container_commands and files, however, I deferred to the files event and it worked like a charm. My specific setup is as below.
.ebextensions/cron.config
files:
"/etc/cron.d/mycronstuff":
mode: "000644"
owner: root
group: root
content: |
# Run daily job every 8 hours
0 */8 * * * root curl -i XXXXXXXXXX
# Run nightly job at 2AM (8AM UTC)
0 8 * * * root curl -i XXXXXXXXX
Update 2019:
You must use a cron.yaml file on your project directory.
inside the file you can mention:
version: 1
cron:
- name: "task1"
url: "/scheduled"
schedule: "* * * * *"