We're deploying our app on AWS using Elasticbeanstalk. We have a main service that is an API. We also have a cron job defined using crob tabs:
files:
"/etc/cron.d/jobexecutor":
mode: "000644"
owner: root
group: root
content: |
* * * * * root /usr/local/bin/run.sh >> /var/log/jobexecutor.log 2>&1
"/usr/local/bin/run.sh":
mode: "000755"
owner: root
group: root
content: |
#!/bin/bash
export DATABASE_HOST=$(/opt/elasticbeanstalk/bin/get-config environment -k DATABASE_HOST)
# export other env vars
/usr/bin/node /var/app/current/lib/workers/job-executor.js
And now the whole deployments fail because cron job script cannot find certain module when executing (Awillix npm module). This error started occurring only recently, it worked just fine before. I understand that it does not find an error since npm install is not done. Is this like 'race condition' error? If npm install of main service is executed before the cron job it would work. Otherwise, it will not?
What to do to prevent this? Is there a way to tell cron tab to not execute before the main service (API) is started (that's when we're sure all dependencies are installed)
Thanks in advance
Related
I am trying to set a cron job on my project to send a email after 15 minutes.
This is in django.config
04_cronjb:
command: "cat .ebextensions/cron-linux.config > /etc/cron.d/crontab && chmod 644 /etc/cron.d/crontab"
leader_only: true
This is my cron-linux.config file
files:
"/etc/cron.d/mycron":
mode: "000644"
owner: root
group: root
content: |
* * * * * source /opt/python/current/env && python /opt/python/current/app/manage.py cronjb
It all deploys successfully but i am not receiving any email.
Cronjb script is working i have tested it. So the error is one these two files.
Is there some mistake in it?
Your 04_cronjb copies entire content of cron-linux.config into crontab. This is sadly incorrect.
Instead you should do as shown here. This includes putting all the bash commands you want to execute in a custom script called, e.g., myscript.sh and then adding myscript.sh to cron only.
I recently was able to get my Laravel app deployed using codepipeline on Elastic Beanstalk but ran into a problem. I noticed that my routes where failing because of php.conf Nginx configuration. I had to add a few lines of code to EB's nginx php.conf file to get it to work.
My problem now was that after every deployment, the instance of the application I modified the php.conf file was destroyed and recreated fresh. I wanted a way to dynamically update the file after every successful deployment. I had a version of the file I wanted versioned with my application and so wanted to create a symlink to that file after deployment.
After loads of research, I stumbled on appDeploy Hooks on Elastic Beanstalk that runs post scripts after deployment so did this
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/91_post_deploy_script.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
sudo mkdir /var/testing1
sudo ln -sfn /var/www/html/php.conf.example /etc/nginx/conf.d/elasticbeanstalk/php.conf
sudo mkdir /var/testing
sudo nginx -s reload
And this for some reason does not work. The symlink is not created so my routes are still not working..
I even added some mkdir so am sure the commands in that script runs, none of those commands ran because none of those directories where created.
Please note that if I ssh into the ec2 instance and run the commands there it works. That bash script also exists in the post directory and if I manually run in on the server it works too.
Any pointers to how I could fix this would be helpful. Maybe I am doing something wrong too.
Now I have gotten my scripts to run by following this. However, the script is not running. I am getting an error
2020/06/28 08:22:13.653339 [INFO] Following platform hooks will be executed in order: [01_myconf.config]
2020/06/28 08:22:13.653344 [INFO] Running platform hook: .platform/hooks/postdeploy/01_myconf.config
2020/06/28 08:22:13.653516 [ERROR] An error occurred during execution of command [app-deploy] - [RunPostDeployHooks]. Stop running the command. Error: Command .platform/hooks/postdeploy/01_myconf.config failed with error fork/exec .platform/hooks/postdeploy/01_myconf.config: permission denied
I tried to follow this forum post here to make my file executable by adding to my container command a new command like so:
01_chmod1:
command: "chmod +x .platform/hooks/postdeploy/91_post_deploy_script.sh"
I am still running into the same issue. Permission denied
Sadly, the hooks you are describing (i.e. /opt/elasticbeanstalk/hooks/appdeploy) are for Amazon Linux 1.
Since you are using Amazon Linux 2, as clarified in the comments, the hooks you are trying to use do not apply. Thus they are not being executed.
In Amazon Linux 2, there are new hooks as described here and they are:
prebuild – Files here run after the Elastic Beanstalk platform engine downloads and extracts the application source bundle, and before it sets up and configures the application and web server.
predeploy – Files here run after the Elastic Beanstalk platform engine sets up and configures the application and web server, and before it deploys them to their final runtime location.
postdeploy – Files here run after the Elastic Beanstalk platform engine deploys the application and proxy server.
The use of these new hooks is different than in Amazon Linux 1. Thus you have to either move back to Amazon Linux 1 or migrate your application to Amazon Linux 2.
General migration steps from Amazon Linux 1 to Amazon Linux 2 in EB are described here
Create a folder called .platform in your project root folder and create a file with name 00_myconf.config inside the .platform folder.
.platform/
00_myconf.config
Open 00_myconf.config and add the scripts
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/91_post_deploy_script.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
sudo mkdir /var/testing1
sudo ln -sfn /var/www/html/php.conf.example /etc/nginx/conf.d/elasticbeanstalk/php.conf
sudo mkdir /var/testing
sudo nginx -s reload
Commit your changes or reupload the project. This .platform folder will be considered in each new instance creation and your application will deploy properly in all the new instances Amazon Elastic beanstalk creates.
If you access the documentation here and scroll to the section with the title "Application example with extensions" you can see an example of the folder structure of your .platform folder so it adds your custom configuration to NGINX conf on every deploy.
You can either replace the entire nginx.conf file with your file or add additional configuration files to the conf.d directory
Replace conf file with your file on app deploy:
.platform/nginx/nginx.conf
Add configuration files to nginx.conf:
.platform/nginx/conf.d/custom.conf
I've done this before a long time ago, but now it's not working... :)
I am trying to use EBExtensions in an ElasticBeanstalk application. I created a vanilla Elastic Beanstalk environment with no configuration beyond the defaults. I gave it an application version that had a directory structure like the following:
.ebextensions
40testextension.config
app.js
other files
The important part is that I have a folder called .ebextensions at the root of my deployable artifact, which is where I believe it should be located.
The 40testextension.config file inside that file has the following contents:
files:
"/home/ec2-user/myfile" :
mode: "000755"
owner: root
group: root
content: |
# This is my file
# with content
I uploaded that version when creating the environment, and the environment created successfully. But when I look for that file, it is not present. Furthermore, when do a recursive grep for that ebextension file name in the logs at /var/log, I only get one result:
./eb-activity.log: inflating: /tmp/deployment/application/.ebextensions/40testextension.config
Having looked at the logs, it seems that the file is present when the artifact gets pulled down to the host, but the ebextension never gives any indication of running.
What am I missing here? I've done this in the distant past and things have worked very nicely, but this time I can't seem to get the thing to be executed by the Beanstalk deploy lifecycle.
try to run it with -x Print commands and their arguments as they are executed to debug and try to change the mode to 000777.
files:
"/home/ec2-user/myfile" :
mode: "000777"
owner: root
group: root
content: |
#!/usr/bin/env bash
set -xe
I have a web app running on elastic beanstalk. For some reason I was able to install the composer files in order to run my laravel app. The problem is that no other config file works. I have put newrelic.config into the .ebextensions/ directory, but that file got ignored.
I recently tried to create a cron job using this, AWS Elastic Beanstalk, running a cronjob, but it is not working.
Example of a .config file:
container_commands:
01_some_cron_job:
command: "cat .ebextensions/some_cron_job.txt > /etc/cron.d/some_cron_job && chmod 644 /etc/cron.d/some_cron_job"
leader_only: true
When I ssh into the ec2 instance, there is no such directory as some_cron_job.
The source gets committed to beanstalk, but beanstalk is not running the commands.
How can I make beanstalk acknowledge the .config files. Fixing this cronjob will also fix installing new relic, because both configs are being ignored and I do not know why.
Try putting it in commands section. It is more of a server command than a container command.
commands:
01_some_cron_job:
command: "cat .ebextensions/some_cron_job.txt > /etc/cron.d/some_cron_job && chmod 644 /etc/cron.d/some_cron_job"
leader_only: true
I had similar issues as well using container_commands and files, however, I deferred to the files event and it worked like a charm. My specific setup is as below.
.ebextensions/cron.config
files:
"/etc/cron.d/mycronstuff":
mode: "000644"
owner: root
group: root
content: |
# Run daily job every 8 hours
0 */8 * * * root curl -i XXXXXXXXXX
# Run nightly job at 2AM (8AM UTC)
0 8 * * * root curl -i XXXXXXXXX
Update 2019:
You must use a cron.yaml file on your project directory.
inside the file you can mention:
version: 1
cron:
- name: "task1"
url: "/scheduled"
schedule: "* * * * *"
I have been trying to implement celery with django and elastic beanstalk with SQS, but I still don't know how should I start the workers in background, it seems that I need to create an AMI outside of the EB. Am I even following the right path ? Is there a better way to have parallel tasks?
Update:
I found an alternative solution for this that is simpler and more stable. See my answer in this question: How do you run a worker with AWS Elastic Beanstalk?
I just needed to figure this out for a project I am working on. It took some tinkering, but in the end the solution is quite easy to implement. You can add three files "dynamically" to the server using the files: directive in a ebextension hook. The three files are:
A script that starts the daeomon (located in /etc/init.d/)
A config file, configuring the daemon starting script, located in /etc/default/
A shell script that copies the env vars from your app to the environment of celeryd and starts the service (post deployment)
The start script can be the default from the repository, so it is sourced directly from github.
The config has to be adopted to your project. You need to add your own app's name in to the CELERY_APP setting and you can pass additional arguments to the worker through the CELERYD_OPTS setting (for instance, the concurrency value could be set there).
Then you also need to pass your environment variables for your project to the worker daemon, as it needs the same environment variables as the main app. An example are the AWS secret keys that the celery worker needs to have to be able to connect to SQS and possibly S3. You can do that by simply appending the env vars from the current app to the configuration file:
cat /opt/python/current/env | tee -a /etc/default/celeryd
Finally the celery worker should be started. This step needs to happen after the codebase has been deployed to the server, so it needs to be activated "post" deployment. You can do that by using the undocumented post-deploy hooks. Any shell file in /opt/elasticbeanstalk/hooks/appdeploy/post/ will be executed by elasticbeanstalk post deployment. So you can add a service celeryd restart command into a script file in that folder. For convenience, I placed both the copying of environment variables and the start command in one file.
Note that you can not use the services: directive directly to start the daemon, as this will try to start the celeryd worker before the codebase is deployed to the server, so that won't work (hence the "post" deploy script).
Ok, all that put together, the only thing needed is to create a file ./ebextensions/celery.config in the main directory of your codebase with the following content (adopted to your codebase of course):
files:
"/etc/init.d/celeryd":
mode: "000755"
owner: root
group: root
source: https://raw2.github.com/celery/celery/22ae169f570f77ae70eab03346f3d25236a62cf5/extra/generic-init.d/celeryd
"/etc/default/celeryd":
mode: "000755"
owner: root
group: root
content: |
CELERYD_NODES="worker1"
CELERY_BIN="/opt/python/run/venv/bin/celery"
CELERY_APP="yourappname"
CELERYD_CHDIR="/opt/python/current/app"
CELERYD_OPTS="--time-limit=30000"
CELERYD_LOG_FILE="/var/log/celery/%N.log"
CELERYD_PID_FILE="/var/run/celery/%N.pid"
CELERYD_USER="ec2-user"
CELERYD_GROUP="ec2-user"
CELERY_CREATE_DIRS=1
"/opt/elasticbeanstalk/hooks/appdeploy/post/myapp_restart_celeryd.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
# Copy env vars to celeryd and restart service
su -c "cat /opt/python/current/env | tee -a /etc/default/celeryd" $EB_CONFIG_APP_USER
su -c "service celeryd restart" $EB_CONFIG_APP_USER
services:
sysvinit:
celeryd:
enabled: true
ensureRunning: false
Hope this helps.