I am running an amazon web services web server tier elastic beanstalk application, and I would like to run a script every 3 hours. I've googled around quite a bit, and the answer for this seems to be to create a ".ebextensions" folder, with a .config file that looks something like this:
commands:
01_cronjobs:
leader_only: true
command: "cat .ebextensions/etss.txt > /etc/cron.d/crontab_etss && chmod 644 /etc/cron.d/crontab_etss"
Unfortunately, when I do this, the application fails to deploy with the error:
[Instance: xxx] Command failed on instance. Return code: 1 Output: cat: .ebextensions/etss.txt: No such file or directory. command 01_cronjobs in .ebextensions/config.config failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
I have an etss.txt file in my .ebextensions folder that looks like this:
0 */3 * * * /usr/bin/php download_etss_tar.php > /dev/null
#blank newline here
I also have tried putting it in the root and referencing it there. Why can't the cron find the file that I know exists in the zip? What am I doing wrong here?
Furthermore, I also tried using the full file name, to no avail:
Command failed on instance. Return code: 1 Output: cat: /var/www/html/.ebextensions/etss.txt: No such file or directory. command 01_cronjobs in .ebextensions/etss.config failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
Related
In Build Step, I've added Send files or execute command over SSh -> SSH Publishers -> Exec command, I'm trying to run aws command to copy file from ec2 to s3. The same command runs fine when I execute it over the terminal, but via jenkins it simply returns:
bash: aws: command not found
The command is
cd ~/.local/bin/ && aws s3 cp /home/ec2-user/lambda_test/lambda_function.zip s3://temp-airflow-us/lambda_function.zip
Based on the comments.
The solution was to use the following command:
cd ~/.local/bin/ && ./aws s3 cp /home/ec2-user/lambda_test/lambda_function.zip s3://temp-airflow-us/lambda_function.zip
since aws is not available in PATH env variable.
command not found indicates that the aws utility is not on $PATH for the jenkins user.
To confirm, sudo su -l jenkins and then issue the command which aws - this will most likely return no results.
You have two options:
use the full path (likely /usr/local/bin/aws)
add /usr/local/bin to the jenkins user's $PATH
I need my Makefile to work in both Linux and Windows so the accepted answer is not an option for me.
I diagnosed the problem by adding the following to the top of my build script:
whoami
which aws
env|grep PATH
This returned:
root
which: no aws in (/sbin:/bin:/usr/sbin:/usr/bin)
PATH=/sbin:/bin:/usr/sbin:/usr/bin
Bizarrely, the path does not include /usr/local/bin, even though the interactive shell on the Jenkins host includes it. The fix is simple enough, create a symlink on the Jenkins host:
ln -s /usr/local/bin/aws /bin/aws
Now the aws command can be found by scripts running in Jenkins (in /bin).
I have already deployed my Django project to EB on AWS and it is working fine.
I tried deploying it now with an Immutable Deployment Policy and I am getting this error:
Command failed on instance. Return code: 2 Output: python: can't open file '/opt/python/current/app/manage.py': [Errno pt/python/current/app/manage.py': [Errno 2] No such file or directory.
And that happens during the command: source /opt/python/current/env && python /opt/python/current/app/manage.py migrate
For some reason, I can't include that command on the first deployment.
Does anyone know how to use the immutable deployment method with Django?
The following:
source /opt/python/current/env && python /opt/python/current/app/manage.py migrate
is useful when you ssh into the EB instance and want to manually run your code with environmental variables.
When you deploy your EB, container_commands in your .ebextensions would be better in my view.
For example, you could have the following file .ebextensions/10_run_migrate.config
container_commands:
10_run_migrate:
command: 'python ./manage.py migrate'
Please note that this is an example only. Thus the file may require adjustment with Django specific commands or settings.
I have written few django management commands that I want to run hourly, daily and weekly basis. I'm using Elastic Beanstalk and created a worker instance where the code is deployed. Can someone help me how to run the django management command with crontab using elastic beanstalk. Thanks
Here is my management command:
python manage.py command_name
Please help me write the container_command in .ebextensions/django.config file for crontab that will schedule the command on hourly basis. Thanks
Any help?
in .ebextensions create a file crontab.txt
# Set the cron to run with utf8 encoding
PYTHONIOENCODING=utf8
0 1 * * * root source /opt/python/current/env && nice /opt/python/current/app/manage.py command_name
# this file needs a blank space as the last line otherwise it will fail
This is a crontab file that will run command_name at 1 am every day. You'll need to get you path right which depends on your file structure.
In one of your config files include:
02_cron_job:
command: "cp .ebextensions/crontab.txt /etc/cron.d/my_cron_jobs && chmod 644 /etc/cron.d/my_cron_jobs"
leader_only: true
This is going to copy your cron file into the right place on the ec2 instance so the job runs.
I have a web app running on elastic beanstalk. For some reason I was able to install the composer files in order to run my laravel app. The problem is that no other config file works. I have put newrelic.config into the .ebextensions/ directory, but that file got ignored.
I recently tried to create a cron job using this, AWS Elastic Beanstalk, running a cronjob, but it is not working.
Example of a .config file:
container_commands:
01_some_cron_job:
command: "cat .ebextensions/some_cron_job.txt > /etc/cron.d/some_cron_job && chmod 644 /etc/cron.d/some_cron_job"
leader_only: true
When I ssh into the ec2 instance, there is no such directory as some_cron_job.
The source gets committed to beanstalk, but beanstalk is not running the commands.
How can I make beanstalk acknowledge the .config files. Fixing this cronjob will also fix installing new relic, because both configs are being ignored and I do not know why.
Try putting it in commands section. It is more of a server command than a container command.
commands:
01_some_cron_job:
command: "cat .ebextensions/some_cron_job.txt > /etc/cron.d/some_cron_job && chmod 644 /etc/cron.d/some_cron_job"
leader_only: true
I had similar issues as well using container_commands and files, however, I deferred to the files event and it worked like a charm. My specific setup is as below.
.ebextensions/cron.config
files:
"/etc/cron.d/mycronstuff":
mode: "000644"
owner: root
group: root
content: |
# Run daily job every 8 hours
0 */8 * * * root curl -i XXXXXXXXXX
# Run nightly job at 2AM (8AM UTC)
0 8 * * * root curl -i XXXXXXXXX
Update 2019:
You must use a cron.yaml file on your project directory.
inside the file you can mention:
version: 1
cron:
- name: "task1"
url: "/scheduled"
schedule: "* * * * *"
I am having a problem with getting a simple cron task set up on Elastic Beanstalk. I have found some of the other questions on here useful, but i still can't seem to get the cron to execute. I am unsure if it is an AWS issue, or if the script itself is not executing. The script is set up inside YII as a Console Command. I am not finding any PHP errors, and the ec2 instance is loaded without errors. Here is what i have done so far:
I have created a folder on the root of my application called .ebextensions.
Within that folder i have created a configuration file with the contents
# Installing dos2unix in case files are edited on windows PC
packages:
yum:
dos2unix: []
container_commands:
01-command:
command: dos2unix -k cron_setup.sh
02-command:
command: chmod 700 cron_setup.sh
03-command:
command: "cat .ebextensions/cron_task.txt > /etc/cron.d/cron_task && chmod 644 /etc/cron.d/cron_task"
# leader_only prevents problems when EB auto-scales
leader_only: true
the file cron_task.txt exists inside the .ebextensions folder with the contents
# The newline at the end of this file is extremely important. Cron won't run without it.
* * * * * /bin/php /var/www/html/crons.php test > /dev/null
Crons.php is a file at the root of the application that includes the Yii framework
defined('YII_DEBUG') or define('YII_DEBUG',true);
// including Yii
require_once(dirname(__FILE__).$yii.'/yii.php');
// we'll use a separate config file
$configFile=dirname(__FILE__).'/protected/config/cron.php';
// creating and running console application
Yii::createConsoleApplication($configFile)->run();
the config/cron.php file is a setup file for the framework, includes database connection and model inclusions, etc
and the cron script being referenced in the cron_task.txt file is a console command that looks like this
class TestCommand extends CConsoleCommand {
public function run($args) {
$message = new Crontasks();
$message->timestamp = date("Y-m-d H:i:s");
$message->message = "test";
$message->insert();
}
}
here i am just trying to get a record into the database to prove the cron was executed successfully. And i can't seem to get a record added.
The problem is, i don't know where this is failing. I am not getting any instance errors. And i took a snapshot log and cant seem to find any relevant errors in there either. Should php errors be logged here? OR do i have to set it up myself to log errors? The problem, i am also having trouble getting into ec2 via SSH. I am getting a permission denied (public key) error!! Even though i have set up the security group/key pair and using the correct public DNS for the instance!
If anyone can see anything obvious is what im doing wrong here, please let me know! Otherwise could you give any advice on where to look for any errors that might be preventing this cron task to execute? Many thanks!!