I have been looking to find an easy way to view debug statements on Beanstalk as I develop. I thought I could simply log to a file on Beanstalk.
In my application.properties file I set
logging.file.path=/var/log
And that did not produce any results even on my local machine. I am not sure if it's a permission issue or what, but locally I set the path to my home directory and then I saw the file, spring.log, appear.
With Beanstalk I tried /var/log, var/log/tomcat, /home/webapp/, ./, ~, and various other values. Nothing worked.
I even tried what was suggested here with no luck: https://medium.com/vividcode/logging-practice-for-elastic-beanstalk-java-apps-308ed7c4d63f
If logging to file is not a good idea, what are the alternatives? I have Googled a lot about this issue and all answers are not very clear.
Yes, this is permission issues. Your app runs under webapp user, while /var/log is own by root. Thus you can't write to it.
The proper way of adding your log files to be recognized by EB is through config files.
Specifically, assuming Amazon Linux 2, you can create .ebextensions/mylogfiles.config with the content of:
files:
"/opt/elasticbeanstalk/config/private/logtasks/bundle/myapplogs.conf":
mode: "000644"
owner: root
group: root
content: |
/var/app/current/log/*.log
Obviously, /var/app/current/log/*.log would point to location where your app stores its log files. /var/app/current is the home folder of your app.
Related
I have created a Django API application that is deployed on AWS Beanstalk Amazon Linux 2 (Python 3.8). In a part of my application, the user should be able to upload a file through API.
Nginx, on default, is returning CORS error to the user when the uploaded file is bigger than 3 MB and logging 'user tries to upload huge file' in the Nginx log.
The only solution which works for me is creating the bellow configuration file and reloading the Nginx:
/etc/nginx/conf.d/proxy.conf:
client_max_body_size 50M;
and then:
sudo service nginx reload
I have done this procedure manually by connecting to the EC2 which is the host of my beanstalk application via SSH. I want to automate this procedure to be done at every deployment and every instance.
I have created a file called nginx_max_upload.config file in .ebextensions folder which is located in the root of my project:
nginx_max_upload.config:
files:
/etc/nginx/conf.d/proxy.conf:
mode: "000755"
owner: root
group: root
content: |
client_max_body_size 50M
commands:
reload_nginx:
command: "sudo service nginx reload"
ignoreErrors: true
My problem is, the above code didn't create the file I want in the specified directory. When I changed the directory to another directory like /usr/local/bin/proxy.conf, the file would be successfully created but it can't create the config file in the Nginx configuration folder.
I guess the problem might be from the permissions but I don't know how to grant the needed permission to the deployment agent.
Also, I have tried these two solutions but none of them works:
I have tried to create the config file in another folder and then, move it via mv command to the right directory but it didn't work.
Also, I have tried to put the creation code in the predeply hook and put manual echo commands in my code. I saw all of my echo commands output in the beanstalk logs but it didn't do anything (nor creating the file in the nginx configuration neither moving it from somewhere else to the configuration folder).
Since you are using Amazon Linux 2 (AL2), your configuration files are incorrect. They used to work in AL1, but for AL2, they are in different place and have different format as shown in the docs.
Thus could have the following .platform/nginx/conf.d/myconfig.conf (not in .ebextensions) with content:
client_max_body_size 50M;
I had this working before on AWS Linux AMI but no luck with AWS Linux 2.
I need to access my environment properties from the Nginx configuration file during the EB application deployment. It's a Single instance Node Server.
I did it like this with the AWS Linux AMI and it worked without a problem:
.ebextensions/00_options.config
option_settings:
aws:elasticbeanstalk:application:environment:
DOMAIN: socket.example.com
MASTER_DOMAIN: https://example.com
etc..
.ebextensions/10_proxy.config
... some configs ...
files:
/etc/nginx/conf.d/proxy.conf:
mode: "000644"
owner: root
group: root
content: |
upstream nodejs {
server 127.0.0.1:8081;
keepalive 256;
}
map $http_origin $cors_header {
hostnames;
default "";
`{"Fn::GetOptionSetting": {"Namespace": "aws:elasticbeanstalk:application:environment", "OptionName": "MASTER_DOMAIN"}}` "$http_origin";
}
server {
listen 80;
listen 8080;
server_name `{"Fn::GetOptionSetting": {"Namespace": "aws:elasticbeanstalk:application:environment", "OptionName": "DOMAIN"}}`;
location ~ /.well-known {
allow all;
root /usr/share/nginx/html;
}
location / {
return 301 https://$host$request_uri;
}
}
etc..
.... some more configs ....
I'm not including most of the configs above because they're not relevant.
So when I did this before, everything worked as expected. The config file inserted my properties and created the file in the /etc/nginx/conf.d/proxy.conf folder.
Now with AWS Linux 2 the specs have changed and we have to add our Nginx configuration files in the .platform/nginx/conf.d folder located in our application bundle root folder.
Here the reference ( see Reverse proxy configuration)
So I created a proxy.conf file in the location mentioned above with the content that was previously inserted in /etc/nginx/conf.d/proxy.conf.
.platform/nginx/conf.d/proxy.conf
upstream nodejs {
server 127.0.0.1:8081;
keepalive 256;
}
map $http_origin $cors_header {
hostnames;
default "";
`{"Fn::GetOptionSetting": {"Namespace": "aws:elasticbeanstalk:application:environment", "OptionName": "MASTER_DOMAIN"}}` "$http_origin";
}
etc...
And then the problems began..
This first trial throwed unexpected "{" in /var/proxy/staging/nginx/conf.d/proxy.conf:11 at me.
And after that I tried a lot of things. Tried it with ${MASTER_DOMAIN} and messed around with the new EB AWS Linux 2 hooks (see link above Platform hooks). All for no avail it seems like you can't access the properties from the Nginx configs. I've read an article or a documentation from Nginx mentioning something similar today but I can't find it anymore (did a lot of googling).
I also tried to create a config file like I did with the working version which purpose was to save a temp file somewhere with the included properties and then include this file in the needed .platform/nginx/conf.d/proxy.conf file because I started to think that there is no way to include them directly with the new specs.
.ebextensions/10_proxy.config
... some configs ....
files:
/var/proxy/staging/custom_folder/proxy.conf:
mode: "000644"
owner: root
group: root
content: |
etc...
.platform/nginx/conf.d/proxy.conf
include custom_folder/proxy.conf;
With this idea in mind I did a lot of nonsense, I created hooks for creating (mkdir) directories in which I tried to temporarily save the file which leaded to new permission errors. I wasn't able to give the proper permissions to prebuild, postdeploy files but this is another issue.
And a lot more of trying and failing...
But then I've read (also from the link above):
"If you configure your proxy to send traffic to multiple application processes, you can configure several environment properties, and use their values in both proxy configuration and your application code."
And hope came back.. Does this mean I actually CAN directly add environmental variables into the Nginx configs located in the .platform directory? ... I don't know.. Do you?
I could continue to describe all the things I tried all night long so I will stop here. I hope you get the issue. If not ask me and I will do my best to make all this understandable.
Also my mind isn't very clear anymore after 14 hours of battling this issue. I need a break.
If you did it to the end thank you for your time and help would be greatly appreciated.
Summary
One way to do it is to create a shell script in .platform/hooks/postdeploy.
Here is a simplified example, assuming you have an Elastic Beanstalk environment property called MASTER_DOMAIN:
#!/bin/bash
# write nginx config file
cat > /etc/nginx/conf.d/elasticbeanstalk/test.conf << LIMIT_STRING
location /test/ {
default_type text/html;
return 200 "nginx variable: \$host, and EB env property: $MASTER_DOMAIN";
}
LIMIT_STRING
# restart nginx service so the config takes effect
systemctl restart nginx.service
The location block from this example can be replaced by the nginx content from .ebextensions/10_proxy.config in the original post. No need for the Fn::GetOptionSetting stuff though.
I think you also need a duplicate script in .platform/confighooks/postdeploy.
Details below.
(sorry for the wall of text)
Environment variables in nginx
Actually, as discussed in here and here, it is not possible (out-of-the-box) to use os environment variables inside the http, server, or location blocks in nginx config files. There are some workarounds, such as using lua, perl, or templates, but let's not get into those. This part has nothing to do with AWS.
In the OP's original configuration for Amazon Linux AMI (AL1), using the files section in .ebextensions/10_proxy.config, they were actually using a shell script to write the nginx config file during deployment. The shell script expanded the environment variables, but the resulting proxy.conf for nginx did not actually access any environment variables.
That's why it worked on AL1.
Platform hooks
Now, for Amazon Linux 2 (AL2), we can do something similar using shell scripts in the .platform/hooks and .platform/confighooks folders.
These .platform hook scripts are executed as the root user, and they have access to the Elastic Beanstalk (EB) environment properties. The EB environment properties can be accessed just like normal OS environment variables, so there is no need to use the Fn::GetOptionSetting stuff.
Basically, we need to create a shell script that writes a file with the content from your original .ebextensions/10_proxy.config. However, there are two questions we need to consider:
Should we use a prebuild, predeploy, or postdeploy hook?
What is the proper destination directory for our nginx proxy.conf file?
File locations
To answer these questions, we have to refer to the AWS documentation for Extending Elastic Beanstalk Linux platforms, and specifically the Instance deployment workflow section.
... The current working directory (cwd) for platform hooks is the application's root directory. For prebuild and predeploy files it's the application staging directory, and for postdeploy files it's the current application directory. If one of the files fails (exits with a non-zero exit code), the deployment aborts and fails.
This is interesting, but leaves some questions, e.g. where is the "application staging directory" located? We can fill in the blanks by inspecting one of our deployment log files. Based on our eb-engine.log, here's what happens with the platform hooks and nginx config files during app deployment (skipping a lot of details):
the source bundle is downloaded from S3 and extracted to /var/app/staging/
platform hooks in .platform/hooks/prebuild/ are executed
proxy server configuration is copied from /var/app/staging/.platform/nginx/ to /var/proxy/staging/nginx
platform hooks in .platform/hooks/predeploy/ are executed
proxy server is started, configuration is copied from /var/proxy/staging/nginx/ to /etc/nginx
platform hooks in .platform/hooks/postdeploy/ are executed
Note, after deployment the app is located in /var/app/current.
Based on the above, there are several options:
Create a shell script in .platform/hooks/postdeploy that writes to /etc/nginx/conf.d/proxy.conf.
The nginx service is already running, at this stage, so we need to restart for the configuration to take effect.
Below is a minimal test example. In this example we write to the elasticbeanstalk subdirectory, because we just want to add a location inside the default server block. We can then visit the /test/ page in a browser, to check that the configuration works.
We use some bash io redirection (<<, >) to write the nginx config file.
Note that we need to escape any nginx variables, e.g. $host becomes \$host, otherwise the shell will interpret them as environment variables.
Also note that the shell scripts need to have execution permission, as explained under More about platform hooks in the docs.
#!/bin/bash
cat > /etc/nginx/conf.d/elasticbeanstalk/test.conf << LIMIT_STRING
location /test/ {
default_type text/html
return 200 "nginx variable: \$host, and EB env property: $MASTER_DOMAIN";
}
LIMIT_STRING
systemctl restart nginx.service
Alternatively, we could create a shell script in .platform/hooks/predeploy that writes to /var/proxy/staging/nginx/conf.d/proxy.conf.
There is no need to restart the nginx service in this case, because this hook is executed before the server configuration is applied.
BEWARE:
Not sure if this is a bug or a design feature, but our newly created proxy.conf disappears after a configuration deployment (as opposed to an application deployment), unless we put a duplicate script in the .platform/confighooks/postdeploy directory. Not very DRY...
EDIT: AWS support confirmed that we need duplicate scripts in hooks and confighooks in this case. The application example in the docs also shows some duplicates (at least duplicate filenames) in hooks and confighooks.
EDIT:
Instead of duplicating scripts, we can also write a confighook that calls a hook, e.g. .platform/confighooks/predeploy/01_my_confighook.sh could look like this:
#!/bin/bash
source "/var/app/current/.platform/hooks/predeploy/01_my_hook.sh"
Disclaimer: This was tested on a freshly created single instance EB environment with "Python 3.7 running on 64bit Amazon Linux 2/3.1.5" using all default configuration and the default AWS Python sample application (only extended with our custom hooks).
My app currently uses a folder called "Documents" that is located in the root of the app. This is where it stores supporting docs, temporary files, uploaded files etc. I'm trying to move my app from Azure to Beanstalk and I don't know how to give permissions to this folder and sub-folders. I think it's supposed to be done using .ebextensions but I don't know how to format the config file. Can someone suggest how this config file should look? This is an ASP.NET app running on Windows/IIS.
Unfortunately, you cannot use .ebextensions to set permissions to files/folders within your deployment directory.
If you look at the event hooks for an elastic beanstalk deployment:
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-windows-ec2.html#windows-container-commands
You'll find that commands run before the ec2 app and web server are set up, and
container_commands run after the ec2 app and web server are setup, but before your application version is deployed.
The solution is to use a wpp.targets file to set the necessary ACLs.
The following SO post is most useful
Can Web Deploy's setAcl provider be used on a sub-directory?
Given below is the sample .ebextensions config file to create a directory/file and modify the permissions and add some content to the file
====== .ebextensions/custom_directory.config ======
commands:
create_directory:
command: mkdir C:\inetpub\AspNetCoreWebApps\backgroundtasks\mydirectory
command: cacls C:\inetpub\AspNetCoreWebApps\backgroundtasks\mydirectory /t /e /g username:W
files:
"C:/inetpub/AspNetCoreWebApps/backgroundtasks/mydirectory/mytestfile.txt":
content: |
This is my Sample file created from ebextensions
ebextensions go into the root of the application source code through a directory called .ebextensions. For more information on how to use ebextensions, please go through the documentation here
Place a file 01_fix_permissions.config inside .ebextensions folder.
files:
"/opt/elasticbeanstalk/hooks/appdeploy/pre/49_change_permissions.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
sudo chown -R ec2-user:ec2-user tmp/
Following that you can set your folder permissions as you want.
See this answer on Serverfault.
There are platform hooks that you can use to run scripts at various points during deployment that can get you around the shortcomings of the .ebextension Commands and Platform Commands that Napoli describes.
There seems to be some debate on whether or not this setup is officially supported, but judging by comments made on the AWS github, it seems to be not explicitly prohibited.
I can see where Napoli's answer could be the more standard MS way of doing things, but wpp.targets looks like hot trash IMO.
The general scheme of that answer is to use Commands/Platform commands to copy a script file into the appropriate platform hook directory (/opt/elasticbeanstalk/hooks or C:\Program Files\Amazon\ElasticBeanstalk\hooks\ ) to run at your desired stage of deployment.
I think its worth noting that differences exist between platforms and versions such as Amazon Linux 1 and Linux 2.
I hope this helps someone. It took me a day to gather that info and what's on this page and pick what I liked best.
Edit 11/4 - I would like to note that I saw some inconsistencies with the File .ebextension directive when trying to place scripts drirectly into the platform hook dir's during repeated deployments. Specifically the File directive failed to correctly move the backup copies named .bak/.bak1/etc. I would suggest using a Container Command to copy with overwriting from another directory into the desired hook directory to overcome this issue.
I am using .git to deploy eb (PHP), and in the root folder, I've created folder .ebextensions and a file options.config with content
config_options:
aws:elasticbeanstalk:container:php:phpini:
document_root: /laravel/public
However, in the /var/log/eb-activity.log and I find the document_root is never being set
++ export PHP_DOCUMENT_ROOT=
++ PHP_DOCUMENT_ROOT=
In the S3 package I find the .ebextensions is bundled, so what can I do to debug this?
The docs state using option_settings rather than config_options (node.js, python and ruby examples use it, however the PHP specific example is for config_options)
option_settings:
- namespace: 'aws:elasticbeanstalk:container:php:phpini':
option_name: document_root
value: /laravel/public
or shorthand YAML as you are using
option_settings:
aws:elasticbeanstalk:container:php:phpini:
document_root: /laravel/public
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/ebextensions-optionsettings.html
Also a simple thing but is the .ebextensions folder in the source root and the options.config in the .ebextensions folder? http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/ebextensions.html
This is an old issue (the link) but as it is said in this link:
Having said that, we were concerned when a number of our customers
began reporting issues with setting the Document Root environment
configuration. Many of these environments were inexplicably "going
red" and failing to update correctly under certain conditions. After
some digging, we discovered that while the configuration settings were
being written correctly during a configuration update, they weren't
being written correctly when a new instance was launched (via Auto
Scaling, the AWS Toolkit for Eclipse, or otherwise).
I think your problem is not the configuration, but may be the AMI you are using, which AMI is it? The updated AMI version is:
Amazon Linux 2012.03
Could you please verify your AMI version? Maybe you are using an old custom one?
EDIT
Also in this link, it says there is a document root configuration in the AWS EB PHP console:
To access the configuration options for your PHP environment
1) Open the Elastic Beanstalk console.
2) Navigate to the management console for your environment.
3) Choose Configuration.
4) In the Software Configuration section, choose Edit.
PHP Settings
Document root – The folder that contains your site's default page. If
your welcome page is not at the root of your source bundle, specify
the folder that contains it relative to the root path. For example,
/public if the welcome page is in a folder named public.
Maybe you can configure it from the dashboard?
So I am attempting to setup CodeDeploy for my application and I keep getting an error during the BeforeInstall part of the deployment. Below is the error.
Error Code UnknownError
Script Name
Message No such file or directory - /opt/codedeploy-agent/deployment-root/06100f1b-5495-42d9-bd01-f33d59fb5deb/d-NL5K1THE8/deployment-archive/appspec.yml
Log Tail
I assumed this meant the YAML file was in the wrong place. However it is in the root directory of my revision. I have tried using a simple AppSpec file like so instead of a more complex one.
## YAML Template.
---
version: 0.0
os: linux
files:
- source: /
destination: /home/ubuntu/www
More or less since this is a first deployment I want it to add all files in the revision to the public directory on the web server.
I am tearing my hair out over this and I feel it is a simple issue. I have the IAM policies and roles correct and I have CodeDeploy setup and running on my instance I am trying to deploy to.
It seems to think you had a successful deploy at some point.
Go into /opt/codedeploy-agent/deployment-root/deployment-instructions/ and delete all the files in there. Then it won't look for this last deploy.
I just had this SAME problem and I figured it out! Make sure your AppSpec file has the right EXTENSION! I was using yaml and not yml, now everything works perfectly.
I made it work like this:
I had a couple of failed deployments for various reasons.
The thing is that CD keeps in the EC2 instance and in the path /opt/codedeploy-agent/deployment-root/ a folder named by the ID of the failed deployment [a very long alphanumeric sting] .
Delete this folder and create a new deployment [from the aws UI console] and redeploy the application. This way the appspec.yml file that is in the wrong place will be deleted.
It should now succeed.
Extra Notice:
CD does not rewrite files [that have not been created by it's specific deployment]
CodeDeploy does not deploy in a folder that there is already code[files] as it does not want to interfere with different CD deployments and/or other CI/CD tools [like Jenkins].
It only deploys in a path that has already deploy code with the specific deployment.
You can empty the folder where your deployment want to happen and redeploy your code via CD.
When you login to the host, do you see the appspec.yml file in the directory there? If not are you positive it has been checked in with the rest of your deployed code?
Just encountered this issue too. In my case, the revision zip file extracts into a directory when deployed. Because of that /opt/codedeploy-agent/deployment-root/xxx/xxx/deployment-archive contains the parent directory of my revision files (instead of the actual revision files).
The key is to compress your revision without the parent directory. In mac terminal,
cd your-app-directory-containing-appspec
zip -r app.zip .