I run a django app on eb. My goal is to create a cron script in /etc/cron.d at deployment time that runs django jobs. I tried to create it using a an eb configuration file, but so far I've been unable to create any file anywhere (manually checked by sshing into the host).
Platform: Python 3.7 running on 64bit Amazon Linux 2/3.3.9
The content of cron.config looks like this:
files:
"/etc/cron.d/cron_job":
mode: "000644"
owner: root
group: root
content: |
* * * * * root echo "Cron running at "`date` > /home/ec2-user/cron_job.log
What I've tried is following. In all cases, app deploys but no file created.
Placing cron.config in .ebextensions folder.
Placing cron.config in .platform/hook/postdeploy folder. Log says "The dir .platform/hooks/postdeploy/ does not exist in the application. Skipping this step...".
Specify file location as /tmp/cron_job.
Specify file location as /home/ec2-user/cron_job.
Would appreciate any help.
There is nothing wrong with your cron.config and it creates /etc/cron.d/cron_job. It works as expected as I run it on my own EB application. Whatever is happening is due to some other factors/code which is not shown in your question.
Related
I have created a Django API application that is deployed on AWS Beanstalk Amazon Linux 2 (Python 3.8). In a part of my application, the user should be able to upload a file through API.
Nginx, on default, is returning CORS error to the user when the uploaded file is bigger than 3 MB and logging 'user tries to upload huge file' in the Nginx log.
The only solution which works for me is creating the bellow configuration file and reloading the Nginx:
/etc/nginx/conf.d/proxy.conf:
client_max_body_size 50M;
and then:
sudo service nginx reload
I have done this procedure manually by connecting to the EC2 which is the host of my beanstalk application via SSH. I want to automate this procedure to be done at every deployment and every instance.
I have created a file called nginx_max_upload.config file in .ebextensions folder which is located in the root of my project:
nginx_max_upload.config:
files:
/etc/nginx/conf.d/proxy.conf:
mode: "000755"
owner: root
group: root
content: |
client_max_body_size 50M
commands:
reload_nginx:
command: "sudo service nginx reload"
ignoreErrors: true
My problem is, the above code didn't create the file I want in the specified directory. When I changed the directory to another directory like /usr/local/bin/proxy.conf, the file would be successfully created but it can't create the config file in the Nginx configuration folder.
I guess the problem might be from the permissions but I don't know how to grant the needed permission to the deployment agent.
Also, I have tried these two solutions but none of them works:
I have tried to create the config file in another folder and then, move it via mv command to the right directory but it didn't work.
Also, I have tried to put the creation code in the predeply hook and put manual echo commands in my code. I saw all of my echo commands output in the beanstalk logs but it didn't do anything (nor creating the file in the nginx configuration neither moving it from somewhere else to the configuration folder).
Since you are using Amazon Linux 2 (AL2), your configuration files are incorrect. They used to work in AL1, but for AL2, they are in different place and have different format as shown in the docs.
Thus could have the following .platform/nginx/conf.d/myconfig.conf (not in .ebextensions) with content:
client_max_body_size 50M;
I have been looking to find an easy way to view debug statements on Beanstalk as I develop. I thought I could simply log to a file on Beanstalk.
In my application.properties file I set
logging.file.path=/var/log
And that did not produce any results even on my local machine. I am not sure if it's a permission issue or what, but locally I set the path to my home directory and then I saw the file, spring.log, appear.
With Beanstalk I tried /var/log, var/log/tomcat, /home/webapp/, ./, ~, and various other values. Nothing worked.
I even tried what was suggested here with no luck: https://medium.com/vividcode/logging-practice-for-elastic-beanstalk-java-apps-308ed7c4d63f
If logging to file is not a good idea, what are the alternatives? I have Googled a lot about this issue and all answers are not very clear.
Yes, this is permission issues. Your app runs under webapp user, while /var/log is own by root. Thus you can't write to it.
The proper way of adding your log files to be recognized by EB is through config files.
Specifically, assuming Amazon Linux 2, you can create .ebextensions/mylogfiles.config with the content of:
files:
"/opt/elasticbeanstalk/config/private/logtasks/bundle/myapplogs.conf":
mode: "000644"
owner: root
group: root
content: |
/var/app/current/log/*.log
Obviously, /var/app/current/log/*.log would point to location where your app stores its log files. /var/app/current is the home folder of your app.
I recently was able to get my Laravel app deployed using codepipeline on Elastic Beanstalk but ran into a problem. I noticed that my routes where failing because of php.conf Nginx configuration. I had to add a few lines of code to EB's nginx php.conf file to get it to work.
My problem now was that after every deployment, the instance of the application I modified the php.conf file was destroyed and recreated fresh. I wanted a way to dynamically update the file after every successful deployment. I had a version of the file I wanted versioned with my application and so wanted to create a symlink to that file after deployment.
After loads of research, I stumbled on appDeploy Hooks on Elastic Beanstalk that runs post scripts after deployment so did this
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/91_post_deploy_script.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
sudo mkdir /var/testing1
sudo ln -sfn /var/www/html/php.conf.example /etc/nginx/conf.d/elasticbeanstalk/php.conf
sudo mkdir /var/testing
sudo nginx -s reload
And this for some reason does not work. The symlink is not created so my routes are still not working..
I even added some mkdir so am sure the commands in that script runs, none of those commands ran because none of those directories where created.
Please note that if I ssh into the ec2 instance and run the commands there it works. That bash script also exists in the post directory and if I manually run in on the server it works too.
Any pointers to how I could fix this would be helpful. Maybe I am doing something wrong too.
Now I have gotten my scripts to run by following this. However, the script is not running. I am getting an error
2020/06/28 08:22:13.653339 [INFO] Following platform hooks will be executed in order: [01_myconf.config]
2020/06/28 08:22:13.653344 [INFO] Running platform hook: .platform/hooks/postdeploy/01_myconf.config
2020/06/28 08:22:13.653516 [ERROR] An error occurred during execution of command [app-deploy] - [RunPostDeployHooks]. Stop running the command. Error: Command .platform/hooks/postdeploy/01_myconf.config failed with error fork/exec .platform/hooks/postdeploy/01_myconf.config: permission denied
I tried to follow this forum post here to make my file executable by adding to my container command a new command like so:
01_chmod1:
command: "chmod +x .platform/hooks/postdeploy/91_post_deploy_script.sh"
I am still running into the same issue. Permission denied
Sadly, the hooks you are describing (i.e. /opt/elasticbeanstalk/hooks/appdeploy) are for Amazon Linux 1.
Since you are using Amazon Linux 2, as clarified in the comments, the hooks you are trying to use do not apply. Thus they are not being executed.
In Amazon Linux 2, there are new hooks as described here and they are:
prebuild – Files here run after the Elastic Beanstalk platform engine downloads and extracts the application source bundle, and before it sets up and configures the application and web server.
predeploy – Files here run after the Elastic Beanstalk platform engine sets up and configures the application and web server, and before it deploys them to their final runtime location.
postdeploy – Files here run after the Elastic Beanstalk platform engine deploys the application and proxy server.
The use of these new hooks is different than in Amazon Linux 1. Thus you have to either move back to Amazon Linux 1 or migrate your application to Amazon Linux 2.
General migration steps from Amazon Linux 1 to Amazon Linux 2 in EB are described here
Create a folder called .platform in your project root folder and create a file with name 00_myconf.config inside the .platform folder.
.platform/
00_myconf.config
Open 00_myconf.config and add the scripts
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/91_post_deploy_script.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
sudo mkdir /var/testing1
sudo ln -sfn /var/www/html/php.conf.example /etc/nginx/conf.d/elasticbeanstalk/php.conf
sudo mkdir /var/testing
sudo nginx -s reload
Commit your changes or reupload the project. This .platform folder will be considered in each new instance creation and your application will deploy properly in all the new instances Amazon Elastic beanstalk creates.
If you access the documentation here and scroll to the section with the title "Application example with extensions" you can see an example of the folder structure of your .platform folder so it adds your custom configuration to NGINX conf on every deploy.
You can either replace the entire nginx.conf file with your file or add additional configuration files to the conf.d directory
Replace conf file with your file on app deploy:
.platform/nginx/nginx.conf
Add configuration files to nginx.conf:
.platform/nginx/conf.d/custom.conf
My app currently uses a folder called "Documents" that is located in the root of the app. This is where it stores supporting docs, temporary files, uploaded files etc. I'm trying to move my app from Azure to Beanstalk and I don't know how to give permissions to this folder and sub-folders. I think it's supposed to be done using .ebextensions but I don't know how to format the config file. Can someone suggest how this config file should look? This is an ASP.NET app running on Windows/IIS.
Unfortunately, you cannot use .ebextensions to set permissions to files/folders within your deployment directory.
If you look at the event hooks for an elastic beanstalk deployment:
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-windows-ec2.html#windows-container-commands
You'll find that commands run before the ec2 app and web server are set up, and
container_commands run after the ec2 app and web server are setup, but before your application version is deployed.
The solution is to use a wpp.targets file to set the necessary ACLs.
The following SO post is most useful
Can Web Deploy's setAcl provider be used on a sub-directory?
Given below is the sample .ebextensions config file to create a directory/file and modify the permissions and add some content to the file
====== .ebextensions/custom_directory.config ======
commands:
create_directory:
command: mkdir C:\inetpub\AspNetCoreWebApps\backgroundtasks\mydirectory
command: cacls C:\inetpub\AspNetCoreWebApps\backgroundtasks\mydirectory /t /e /g username:W
files:
"C:/inetpub/AspNetCoreWebApps/backgroundtasks/mydirectory/mytestfile.txt":
content: |
This is my Sample file created from ebextensions
ebextensions go into the root of the application source code through a directory called .ebextensions. For more information on how to use ebextensions, please go through the documentation here
Place a file 01_fix_permissions.config inside .ebextensions folder.
files:
"/opt/elasticbeanstalk/hooks/appdeploy/pre/49_change_permissions.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
sudo chown -R ec2-user:ec2-user tmp/
Following that you can set your folder permissions as you want.
See this answer on Serverfault.
There are platform hooks that you can use to run scripts at various points during deployment that can get you around the shortcomings of the .ebextension Commands and Platform Commands that Napoli describes.
There seems to be some debate on whether or not this setup is officially supported, but judging by comments made on the AWS github, it seems to be not explicitly prohibited.
I can see where Napoli's answer could be the more standard MS way of doing things, but wpp.targets looks like hot trash IMO.
The general scheme of that answer is to use Commands/Platform commands to copy a script file into the appropriate platform hook directory (/opt/elasticbeanstalk/hooks or C:\Program Files\Amazon\ElasticBeanstalk\hooks\ ) to run at your desired stage of deployment.
I think its worth noting that differences exist between platforms and versions such as Amazon Linux 1 and Linux 2.
I hope this helps someone. It took me a day to gather that info and what's on this page and pick what I liked best.
Edit 11/4 - I would like to note that I saw some inconsistencies with the File .ebextension directive when trying to place scripts drirectly into the platform hook dir's during repeated deployments. Specifically the File directive failed to correctly move the backup copies named .bak/.bak1/etc. I would suggest using a Container Command to copy with overwriting from another directory into the desired hook directory to overcome this issue.
I've done this before a long time ago, but now it's not working... :)
I am trying to use EBExtensions in an ElasticBeanstalk application. I created a vanilla Elastic Beanstalk environment with no configuration beyond the defaults. I gave it an application version that had a directory structure like the following:
.ebextensions
40testextension.config
app.js
other files
The important part is that I have a folder called .ebextensions at the root of my deployable artifact, which is where I believe it should be located.
The 40testextension.config file inside that file has the following contents:
files:
"/home/ec2-user/myfile" :
mode: "000755"
owner: root
group: root
content: |
# This is my file
# with content
I uploaded that version when creating the environment, and the environment created successfully. But when I look for that file, it is not present. Furthermore, when do a recursive grep for that ebextension file name in the logs at /var/log, I only get one result:
./eb-activity.log: inflating: /tmp/deployment/application/.ebextensions/40testextension.config
Having looked at the logs, it seems that the file is present when the artifact gets pulled down to the host, but the ebextension never gives any indication of running.
What am I missing here? I've done this in the distant past and things have worked very nicely, but this time I can't seem to get the thing to be executed by the Beanstalk deploy lifecycle.
try to run it with -x Print commands and their arguments as they are executed to debug and try to change the mode to 000777.
files:
"/home/ec2-user/myfile" :
mode: "000777"
owner: root
group: root
content: |
#!/usr/bin/env bash
set -xe