As far as I know the only way to set environment variables to elastic beanstalk is either:
The AWS Online Console.
The eb setenv command.
I have a .env file in my project that contains all of my environment variables over 100 variables and I'm looking for a way to push them all to the eb instance at the same time.
First off, you should note that Elastic Beanstalk has a limit of fifty (50) environment variables per environment.
Now, if you've got a bunch of environment variables set in a .env file, an easy way to set them en-mass is to just dump them all into the eb setenv command like this...
eb setenv `cat .env | sed '/^#/ d' | sed '/^$/ d'`
This will set them all in one shot and save you spending all night on the aws console page. The sed command remove commented lines and new lines.
Explain Shell
From the beginning, personally I had a deploy.sh script which was doing npm install, npm build, etc.
So, in a .ebextensions/example.config file, I put the following:
option_settings:
aws:elasticbeanstalk:environment:process:default:
HealthCheckPath: /health
# This must be at the end of the file
aws:elasticbeanstalk:application:environment:
# Will be populated by the deployment script
And in my script I have something like this:
sed -r 's/^([_[:alnum:]]+)=/ \1: /g' .env >> dist/.ebextensions/example.config
This sed command replaces ^[_\w]+= with $1: so that all my environment variables are passed in a config file. Thanks to this, I don't have to:
Use eb setenv
Wait for the update to process
Use eb deploy
Related
So this has been a long-running problem for me and I'd love to fix it - I also think it will help a lot of others. I'd love to run Django commands after ssh'ing on my Elastic Beanstalk EC2 instance. E. g.
python manage.py dumpdata
The reason why this is not possible are the missing environment variables. They are present when the server boots up but are unset as soon as the server is running (EB will create a virtual env within the EC2 and delete the variables from there).
I've recently figured out that there is a prebuilt script to retrieve the env variables on the EC2 instances:
/opt/elasticbeanstalk/bin/get-config environment
This will return a stringified object like this:
{"AWS_STATIC_ASSETS_SECRET_ACCESS_KEY":"xxx-xxx-xxx","DJANGO_KEY":"xxx-xxx-xxx","DJANGO_SETTINGS_MODULE":"xx.xx.xx","PYTHONPATH":"/var/app/venv/staging-LQM1lest/bin","RDS_DB_NAME":"xxxxxxx":"xxxxxx","RDS_PASSWORD":"xxxxxx"}
This is where I'm stuck currently. I think need would need a script, that takes this object parses it and sets the key / values as environment variables. I would need to be able to run this script from the ec2 instance.
Or a command to execute from the .ebextensions that would get the variables and sets them.
Am I absolutely unsure how to proceed at this point? Am I overlooking something obvious here? Is there someone who has written a script for this already? Is this even the right approach?
I'd love your help!
Your env variables are stored in /opt/elasticbeanstalk/deployment/env
Thus to export them, you can do the following (must be root to access the file):
export $(cat /opt/elasticbeanstalk/deployment/env | xargs)
Once you execute the command you can confirm the presence of your env variables using:
env
To use this in your .extentions, you can try:
container_commands:
10_dumpdata:
command: |
export $(cat /opt/elasticbeanstalk/deployment/env | xargs)
source $PYTHONPATH/activate
python ./manage.py dumpdata
AWS EB (Elastic Beanstalk) CLI not running in git bash (Windows 10). I have successfully installed the AWS EB CLI from AWS documentation at https://github.com/aws/aws-elastic-beanstalk-cli-setup/blob/master/README.md . At the end I have set the environment variables as mentioned in the doc. So "eb" command is working from Windows Power shell. But when I am trying to access the "eb" command from GIT Bash / IntelliJ bash prompt, it is not working.
Working fine with windows power shell:
PS C:\> eb --version
EB CLI 3.19.2 (Python 3.7.3)
Environment variable set as below under "User Variable" -> "Path":
Environment variable set windows
While trying to access the "eb" from Git Bash the error is as below:
$ eb
bash: eb: command not found
$ echo $PATH
.....
......
/c/Users/xxxxxx/.ebcli-virtual-env/executables:
Restarted the system and commandline interfaces multiple time.
Can someone please let me know if there are some issue with environment variable set, or need to configure something additional in bash environment?
After so many trial and error with different solution available in internet along with AWS doc suggestion, finally I can use "eb" from Git bash of windows 10. The problem fixed after I put the below location in my environment variable path:
C:\Users\XXXX\AppData\Roaming\Python\Python37\Scripts
The issue for me was a username with a space. The path would then look like this: C:\Users\fname lastname.ebcli-virtual-env\executables. The problem came about with the .bat files created by the AWS script did not wrap the path in double quotes. Windows then interprets it as multiple parameters.
I had to go edit eb.bat and path_exporter.bat and wrap the directives like this: (in eb.bat) CALL "C:\Users\fname lastname.ebcli-virtual-env\Scripts\activate.bat"
#start CALL "C:\Users\fname lastname.ebcli-virtual-env\Scripts\eb.exe" %args%
The EB cli seems to work properly now.
I'm using elastic beanstalk to deploy a Django app. I'd like to SSH on the EC2 instance to execute some shell commands but the environment variables don't seem to be there. I specified them via the AWS GUI (configuration -> environment properties) and they seem to work during the boot-up of my app.
I tried activating and deactivating the virtual env via:
source /var/app/venv/*/bin/activate
Is there some environment (or script I can run) to access an environment with all the properties set? Otherwise, I'm hardly able to run any command like python3 manage.py ... since there is no settings module configured (I know how to specify it manually but my app needs around 7 variables to work).
During deployment, the environment properties are readily available to your .platform hook scripts.
After deployment, e.g. when using eb ssh, you need to load the environment properties manually.
One option is to use the EB get-config tool. The environment properties can be accessed either individually (using the -k option), or as a JSON or YAML object with key-value pairs.
For example, one way to export all environment properties would be:
export $(/opt/elasticbeanstalk/bin/get-config --output YAML environment |
sed -r 's/: /=/' | xargs)
Here the get-config part returns all environment properties as YAML, the sed part replaces the ': ' in the YAML output with '=', and the xargs part fixes quoted numbers.
Note this does not require sudo.
Alternatively, you could refer to this AWS knowledge center post:
Important: On Amazon Linux 2, all environment properties are centralized into a single file called /opt/elasticbeanstalk/deployment/env. You must use this file during Elastic Beanstalk's application deployment process only. ...
The post describes how to make a copy of the env file during deployment, using .platform hooks, and how to set permissions so you can access the file later.
You can also perform similar steps manually, using SSH. Once you have the copy set up, with the proper permissions, you can source it.
Beware:
Note: Environment properties with spaces or special characters are interpreted by the Bash shell and can result in a different value.
Try running the command /opt/elasticbeanstalk/bin/get-config environment after you ssh into the EC2 instance.
If you are trying to access the environment variables in eb script elastic beanstalk
Use this
$(/opt/elasticbeanstalk/bin/get-config environment -k ENVURL)
{ "Ref" : "AWSEBEnvironmentName" }
$(/opt/elasticbeanstalk/bin/get-config environment -k ENVURL)
I have a YAML script that sets up an Ubuntu server. Under the UserData portion of it, I have an HTML doc that contains information. I want the AWS Region and the Public DNS name of the server to be displayed on the web page once it is created.
I have variables in lines 8, 9 which are supposed to find the EC2 Availability zone and parse through it to find the specific region. Line 11 has the variable for the public DNS. Initially I tried the "sed" command to replace the values (%AWS_REGION% and %DNS_HOSTNAME%) on the HTML page with the variables. When I checked the page after running the script, nothing was replaced. (i.e. "AWS region: %AWS_REGION%" was displayed.)
THEN I tried the code below, I replaced %AWS_REGION% with $EC2_REGION in hopes that the variable would just get substituted in, but when I ran the script, it was blank (i.e. After "AWS region:" there was nothing, where last time %AWS_REGION% was there.)
UserData:
'Fn::Base64': |
#!/bin/bash -x
# set timezone
timedatectl set-timezone America/New_York
# get region
EC2_AVAIL_ZONE=curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone
EC2_REGION="`echo \"$EC2_AVAIL_ZONE\" | sed 's/[a-z]$//'`"
# get DNS
EC2_DNS=`curl http://169.254.169.254/latest/meta-data/public-hostname`
# install and setup apache
apt-get update
apt-get install -y nginx
cd /var/www/html
echo "<title>Jonah Ryder</title> <h1>Jonah Ryder</h1> <p>AWS region: $EC2_REGION</p> <p>Public hostname: %DNS_HOSTNAME%</p>" > index.html
sed 's/%AWS_REGION%/EC2_REGION/g' index.html
sed 's/%DNS_HOSTNAME%/EC2_DNS/g' index.html
service nginx start
I want the HTML page to take the variables and display them. I don't know where my mistake is.
To use the EC2_REGION environment variable after you have set its value, you need to use $EC2_REGION, not EC2_REGION. This is how you read env variables in general.
Also, it's worth echoing $EC2_AVAIL_ZONE and $EC2_REGION to stdout after you set them so that you can debug this later, if needed, using the EC2 instance console log. For example:
EC2_AVAIL_ZONE=curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone
EC2_REGION="`echo \"$EC2_AVAIL_ZONE\" | sed 's/[a-z]$//'`"
EC2_DNS=`curl http://169.254.169.254/latest/meta-data/public-hostname`
echo "AZ: $EC2_AVAIL_ZONE"
echo "Region: $EC2_REGION"
echo "DNS: $EC2_DNS"
I´m running an EC2 instance through AWS Elastic Beanstalk. Unfortunately it has the incorrect timezone - it´s 2 hours earlier than it should be, because timezone is set to UTC. What I need is GMT+1.
Is there a way to set up the .ebextensions configuration, in order to force the EC2 instance to use the right timezone?
Yes, you can.
Just create a file /.ebextensions/00-set-timezone.config with following content
commands:
set_time_zone:
command: ln -f -s /usr/share/zoneinfo/Australia/Sydney /etc/localtime
This is assuming your are using default Amazon Linux AMI image. If you use some other Linux distribution, just change the command to whatever it requires to set timezone in that Linux.
This is a response from the aws Support Business and this works!
---- Original message ----
How can I change the timezone of an enviroment or rather to the instances of the enviroment in Elastic Beasntalk to UTC/GMT -3 hours (Buenos Aires, Argentina)?
I´m currently using Amazon Linux 2016.03. Thanks in advance for your help.
Regards.
---------- Response ----------
Hello,Thank you for contacting AWS support regarding modifying your Elastic Beanstalk instances time zone to use UTC/GMT -3 hours (Buenos Aires, Argentina), please see below on steps on how to perform this modification.
The below example shows how to modify timezone for Elastic Beanstalk environment using .ebextensions for Amazon Linux OS:
Create .ebextensions folder in the root of your application
Create a .config file for example 00-set-timezone.config file and add the below content in yaml formatting.
container_commands:
01changePHP:
command: sed -i '/PHP_DATE_TIMEZONE/ s/UTC/America\/Argentina\/Buenos_Aires/' /etc/php.d/environment.ini
01achangePHP:
command: sed -i '/aws.php_date_timezone/ s/UTC/America\/Argentina\/Buenos_Aires/' /etc/php.d/environment.ini
02change_AWS_PHP:
command: sed -i '/PHP_DATE_TIMEZONE/ s/UTC/America\/Argentina\/Buenos_Aires/' /etc/httpd/conf.d/aws_env.conf
03php_ini_set:
command: sed -i '/date.timezone/ s/UTC/America\/Argentina\/Buenos_Aires/' /etc/php.ini
commands:
01remove_local:
command: "rm -rf /etc/localtime"
02link_Buenos_Aires:
command: "ln -s /usr/share/zoneinfo/America/Argentina/Buenos_Aires /etc/localtime"
03restart_http:
command: sudo service httpd restart
Deploy application to Elastic Beanstalk including the .ebextensions and the timezone will change as per the above.
I hope that helps
Regards!
If you are running windows in your eb environment...
.
create a folder named .ebextensions in the root of your project..
inside that folder create a file named timezone.config
in that file add the following :
commands:
set_time_zone:
command: tzutil /s "Central Standard Time"
set the time zone as needed
screenshot
I'm using custom .ini file in php.d folder along with regular recommendations from http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time.html#change_time_zone:
The sed command inserts (rewrites) only the first line of /etc/sysconfig/clock, since the second line (UTC=true) should be left alone, per the above AWS documentation.
# .ebextensions/02-timezone.config
files:
/etc/php.d/webapp.ini:
mode: "000644"
owner: root
group: root
content: |
date.timezone="Europe/Amsterdam"
commands:
01_set_ams_timezone:
command:
- sed -i '1 s/UTC/Europe\/Amsterdam/g' /etc/sysconfig/clock
- ln -sf /usr/share/zoneinfo/Europe/Amsterdam /etc/localtime
Changing the time zone of EC2 with Elastic Beanstalk is simple:
Create a .ebextensions folder in the root
Add a file with filename end with .config (timezone.config)
Inside the file
container_commands:
time_zone:
command: ln -f -s /usr/share/zoneinfo/America/Argentina/Buenos_Aires /etc/localtime
Then you have done.
Note that the container_commands is different from commands, from the document it states:
commands run before the application and web server are set up and
the application version file is extracted.
That's the reason of your time zone command doesn't work because the server hasn't started yet.
container_commands run after the application and web server have been
set up and the application version file has been extracted, but before
the application version is deployed.
If you are runing a java/Tomcat container, just put the JVM Option on the configuration.
-Duser.timezone=America/Sao_Paulo
Possibles values: timezones
Moving to AWS Linux 2 was challenging. It took me a while to work out how to do this easily in .ebextensions.
I wrote the simple solution in another stackoverflow question .. but for anyone needing instant gratification .. add the following commands into the file .ebextensions/xxyyzz.config:
container_commands:
01_set_bne:
command: "sudo timedatectl set-timezone Australia/Brisbane"
command: "sudo systemctl restart crond.service"
These workarounds only fixes the timezone for applications. But when you have any system services like a cron run it looks at the /etc/sysconfig/clock and that is always UTC. If you tail the cron logs or aws-sqsd logs would will notice timestamps are still 2hrs behind - in my case. And a change to the clock setting would need a reboot into order to take effect - which is not an option to consider should you have autoscaling in place or should you want to use ebextensions to change the system clock's config.
Amazon is aware of this issue and I dont think they have resolved it yet.
If your EB application is using the Java/Tomcat container, you can add the JVM timezone Option to the Procfile configuration. Example:
web: java -Duser.timezone=Europe/Berlin -jar application.jar
Make sure to add all configuration options before the -jar option, otherwise they are ignored.
in the .ebextensions added below for PHP
container_commands:
00_changePHP:
command: sed -i '/;date.timezone =/c\date.timezone = \"Australia/Sydney\"' /etc/php.ini
01_changePHP:
command: sed -i '/date.timezone = UTC/c\date.timezone = \"Australia/Sydney\"' /etc/php.d/aws.ini
02_set_tz_AEST:
command: "sudo timedatectl set-timezone Australia/Sydney"
command: "sudo systemctl restart crond.service"
commands:
01remove_local:
command: "rm -rf /etc/localtime"
02change_clock:
command: sed -i 's/\"UTC\"/\"Australia\/Sydney\"/g' /etc/sysconfig/clock
03link_Australia_Sydney:
command: "ln -f -s /usr/share/zoneinfo/Australia/Sydney /etc/localtime"
cwd: /etc
Connect AMI(amazon linux instance) via putty or ssh and execute the commands below;
sudo rm /etc/localtime
sudo ln -sf /usr/share/zoneinfo/Europe/Istanbul /etc/localtime
sudo reboot
Explanation of the procedure above is simply;
remove localtime,
update the timezone,
reboot
Please notify that I've changed my timezone to Turkey's localtime, you can find your timezone by listing zoneinfo directory with the command below;
ls /usr/share/zoneinfo
or just check timezone abbrevetaions via wikipedia;
http://en.wikipedia.org/wiki/Category:Tz_database
You can also check out the related Amazon AWS documentation;
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time.html
Note: I'm not sure that if this is the best practice or not (probably not), however I've applied the procedure I've written above and it's working for me.