When I ssh into my EC2 Instance and run the following commands my SpringServer.jar file executes and I can access my Spring application by going to myawsaccount:8080/times. when I specify the following commands in User Data I cant access my application at myawsaccount:8080/times and im not sure why. Any help would be appreciated.
Commands
#!/bin/bash --> only in user script
sudo su
wget --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u141-b15/336fa29ff2bb4ef291e347e091f7f4a7/jdk-8u141-linux-x64.rpm
yum install -y jdk-8u141-linux-x64.rpm
wget https://myawsaccount.s3-eu-west-1.amazonaws.com/SpringServer-1-0.0.1-SNAPSHOT.jar
java -jar SpringServer-1-0.0.1-SNAPSHOT.jar
To troubleshoot UserData issues, the best thing to do is to login to an instance,
and inspect one of UserData log files.
Most impotently /var/log/cloud-init-output.log:
The cloud-init output log file (/var/log/cloud-init-output.log) captures console output so it is easy to debug your scripts following a launch if the instance does not behave the way you intended.
Also your UserData script will be located in /var/lib/cloud/instances/<instance-id>/. Thus, once you are in the instance you can manually try to run it and fix/debug while in the instance.
Setting environment variables using export doesn't work in user data as it only sets them for the current shell session. You can fix this by copying them to your profile configuration:
#!/bin/bash
...
echo 'export JAVA_HOME=/opt/jdk1.8.0_141' >> /etc/profile
echo 'export JRE_HOME=/opt/jdk1.8.0_141/jre' >> /etc/profile
echo 'export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin' >> /etc/profile
...
This way, the environment variables will be available in every session.
I need to update /etc/hosts for all instances in my EMR cluster (EMR AMI 4.3).
The whole script is nothing more than:
#!/bin/bash
echo -e 'ip1 uri1' >> /etc/hosts
echo -e 'ip2 uri2' >> /etc/hosts
...
This script needs to run as sudo or it fails.
From here: https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-bootstrap.html#bootstrapUses
Bootstrap actions execute as the Hadoop user by default. You can execute a bootstrap action with root privileges by using sudo.
Great news... but I can't figure out how to do this, and I can't find an example.
I've tried a bunch of things... including...
running as Hadoop and adding 'sudo' to each of the 'echo' statements in the script
using a shell script to copy and chmod the above ('echo' statements with no 'sudo') and running local copy using run-if bootstrap that calls 1=1 sudo bash /home/hadoop/myDir/myScript.sh
hard coding the whole script as a one-liner into a run-if bootstrap action
I consistently get:
On the master instance (i-xxx), bootstrap action 2 returned a non-zero return code
If i check the logs for the "Setup hadoop debugging" step, there's nothing there.
From here: https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-overview.html#emr-overview-cluster-lifecycle
summary emr setup (in order):
provisions ec2 instances
runs bootstrap actions
installs native applications... like hadoop, spark, etc.
So it seems like there's some risk that since I'm mucking around as user Hadoop before hadoop is installed, I could be messing something up there, but I can't imagine what.
I think it must be that my script isn't running as 'sudo' and it's failing to update /etc/hosts.
My question... how can I use bootstrap actions (or something else) on EMR to run a simple shell script as sudo? ...specifically to update /etc/hosts?
I've not had problems using sudo from within a shell script run as an EMR bootstrap action, so it should work. You can test that it works with a simple script that simply does "sudo ls /root".
Your script is trying to append to /etc/hosts by redirecting stdout with:
sudo echo -e 'ip1 uri1' >> /etc/hosts
The problem here is that while the echo is run with sudo, the redirection (>>) is not. It's run by the underlying hadoop user, who does not have permission to write to /etc/hosts. The fix is:
sudo sh -c 'echo -e "ip1 uri1" >> /etc/hosts'
This runs the entire command, including the stdout redirection, in a shell with sudo.
When running the following command on kube-master (CoreOS):
export KUBERNETES_PROVIDER=aws; wget -q -O - https://get.k8s.io | bash
I get following error:
Can't find aws in PATH, please fix and retry.
I have already set PATH. Can anyopne tell which 'aws' it is searching for? Is it the aws directory in kubernetes repo directory i.e. kubernetes/cluster/aws?
Follow the AWS CLI installation guide and then ensure your PATH is set correctly.
Yes, you are right.
If you set "aws" as KUBERNETES_PROVIDER, Kubernetes will use scripts that reside in kubernetes/cluster/aws. If no KUBERNETES_PROVIDER is set, I believe the default it to rely on gcloud CLI tool.
If you are using Ubuntu OS. run the below command. it will resolve your issue.
apt-get install awscli
I'm getting the following error updating composer in an AWS Elastic Beanstalk deployment.
The HOME or COMPOSER_HOME environment variable must be set for composer to run correctly
The command causing the error actually sets the environment variable on the same line. It's part of a .ebextensions command
commands:
01_update_composer:
command: export COMPOSER_HOME=/var/home && composer.phar self-update
How can I fix this? Is there something else that could cause that error?
I have tried setting the environment in this configuration env: but that made no difference. I've also added printenv && composer.phar self-update and can confirm the variable is set.
Replace with this:
command: export COMPOSER_HOME=/root && /usr/bin/composer.phar self-update 1.0.0-alpha11
There is a composer issue. issue
I´m running an EC2 instance through AWS Elastic Beanstalk. Unfortunately it has the incorrect timezone - it´s 2 hours earlier than it should be, because timezone is set to UTC. What I need is GMT+1.
Is there a way to set up the .ebextensions configuration, in order to force the EC2 instance to use the right timezone?
Yes, you can.
Just create a file /.ebextensions/00-set-timezone.config with following content
commands:
set_time_zone:
command: ln -f -s /usr/share/zoneinfo/Australia/Sydney /etc/localtime
This is assuming your are using default Amazon Linux AMI image. If you use some other Linux distribution, just change the command to whatever it requires to set timezone in that Linux.
This is a response from the aws Support Business and this works!
---- Original message ----
How can I change the timezone of an enviroment or rather to the instances of the enviroment in Elastic Beasntalk to UTC/GMT -3 hours (Buenos Aires, Argentina)?
I´m currently using Amazon Linux 2016.03. Thanks in advance for your help.
Regards.
---------- Response ----------
Hello,Thank you for contacting AWS support regarding modifying your Elastic Beanstalk instances time zone to use UTC/GMT -3 hours (Buenos Aires, Argentina), please see below on steps on how to perform this modification.
The below example shows how to modify timezone for Elastic Beanstalk environment using .ebextensions for Amazon Linux OS:
Create .ebextensions folder in the root of your application
Create a .config file for example 00-set-timezone.config file and add the below content in yaml formatting.
container_commands:
01changePHP:
command: sed -i '/PHP_DATE_TIMEZONE/ s/UTC/America\/Argentina\/Buenos_Aires/' /etc/php.d/environment.ini
01achangePHP:
command: sed -i '/aws.php_date_timezone/ s/UTC/America\/Argentina\/Buenos_Aires/' /etc/php.d/environment.ini
02change_AWS_PHP:
command: sed -i '/PHP_DATE_TIMEZONE/ s/UTC/America\/Argentina\/Buenos_Aires/' /etc/httpd/conf.d/aws_env.conf
03php_ini_set:
command: sed -i '/date.timezone/ s/UTC/America\/Argentina\/Buenos_Aires/' /etc/php.ini
commands:
01remove_local:
command: "rm -rf /etc/localtime"
02link_Buenos_Aires:
command: "ln -s /usr/share/zoneinfo/America/Argentina/Buenos_Aires /etc/localtime"
03restart_http:
command: sudo service httpd restart
Deploy application to Elastic Beanstalk including the .ebextensions and the timezone will change as per the above.
I hope that helps
Regards!
If you are running windows in your eb environment...
.
create a folder named .ebextensions in the root of your project..
inside that folder create a file named timezone.config
in that file add the following :
commands:
set_time_zone:
command: tzutil /s "Central Standard Time"
set the time zone as needed
screenshot
I'm using custom .ini file in php.d folder along with regular recommendations from http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time.html#change_time_zone:
The sed command inserts (rewrites) only the first line of /etc/sysconfig/clock, since the second line (UTC=true) should be left alone, per the above AWS documentation.
# .ebextensions/02-timezone.config
files:
/etc/php.d/webapp.ini:
mode: "000644"
owner: root
group: root
content: |
date.timezone="Europe/Amsterdam"
commands:
01_set_ams_timezone:
command:
- sed -i '1 s/UTC/Europe\/Amsterdam/g' /etc/sysconfig/clock
- ln -sf /usr/share/zoneinfo/Europe/Amsterdam /etc/localtime
Changing the time zone of EC2 with Elastic Beanstalk is simple:
Create a .ebextensions folder in the root
Add a file with filename end with .config (timezone.config)
Inside the file
container_commands:
time_zone:
command: ln -f -s /usr/share/zoneinfo/America/Argentina/Buenos_Aires /etc/localtime
Then you have done.
Note that the container_commands is different from commands, from the document it states:
commands run before the application and web server are set up and
the application version file is extracted.
That's the reason of your time zone command doesn't work because the server hasn't started yet.
container_commands run after the application and web server have been
set up and the application version file has been extracted, but before
the application version is deployed.
If you are runing a java/Tomcat container, just put the JVM Option on the configuration.
-Duser.timezone=America/Sao_Paulo
Possibles values: timezones
Moving to AWS Linux 2 was challenging. It took me a while to work out how to do this easily in .ebextensions.
I wrote the simple solution in another stackoverflow question .. but for anyone needing instant gratification .. add the following commands into the file .ebextensions/xxyyzz.config:
container_commands:
01_set_bne:
command: "sudo timedatectl set-timezone Australia/Brisbane"
command: "sudo systemctl restart crond.service"
These workarounds only fixes the timezone for applications. But when you have any system services like a cron run it looks at the /etc/sysconfig/clock and that is always UTC. If you tail the cron logs or aws-sqsd logs would will notice timestamps are still 2hrs behind - in my case. And a change to the clock setting would need a reboot into order to take effect - which is not an option to consider should you have autoscaling in place or should you want to use ebextensions to change the system clock's config.
Amazon is aware of this issue and I dont think they have resolved it yet.
If your EB application is using the Java/Tomcat container, you can add the JVM timezone Option to the Procfile configuration. Example:
web: java -Duser.timezone=Europe/Berlin -jar application.jar
Make sure to add all configuration options before the -jar option, otherwise they are ignored.
in the .ebextensions added below for PHP
container_commands:
00_changePHP:
command: sed -i '/;date.timezone =/c\date.timezone = \"Australia/Sydney\"' /etc/php.ini
01_changePHP:
command: sed -i '/date.timezone = UTC/c\date.timezone = \"Australia/Sydney\"' /etc/php.d/aws.ini
02_set_tz_AEST:
command: "sudo timedatectl set-timezone Australia/Sydney"
command: "sudo systemctl restart crond.service"
commands:
01remove_local:
command: "rm -rf /etc/localtime"
02change_clock:
command: sed -i 's/\"UTC\"/\"Australia\/Sydney\"/g' /etc/sysconfig/clock
03link_Australia_Sydney:
command: "ln -f -s /usr/share/zoneinfo/Australia/Sydney /etc/localtime"
cwd: /etc
Connect AMI(amazon linux instance) via putty or ssh and execute the commands below;
sudo rm /etc/localtime
sudo ln -sf /usr/share/zoneinfo/Europe/Istanbul /etc/localtime
sudo reboot
Explanation of the procedure above is simply;
remove localtime,
update the timezone,
reboot
Please notify that I've changed my timezone to Turkey's localtime, you can find your timezone by listing zoneinfo directory with the command below;
ls /usr/share/zoneinfo
or just check timezone abbrevetaions via wikipedia;
http://en.wikipedia.org/wiki/Category:Tz_database
You can also check out the related Amazon AWS documentation;
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time.html
Note: I'm not sure that if this is the best practice or not (probably not), however I've applied the procedure I've written above and it's working for me.