The command that I used to create a new (ubuntu) instance is this:
aws ec2 run-instances --image-id ami-XXXXXXXX --count 1 --instance-type
t2.micro --key-name abcdef --query 'Instances[0].InstanceId' --user-data file:///Users/<username>/<somedir>/UserData.sh --subnet-id subnet-XXXXXX --associate-public-ip-address
The UserData.sh contains this 3 lines including the newline:
#!/bin/bash
mkdir ~/latest
However, I do not see the "latest" dir when the system gets created and comes up. What am I doing wrong?
Disclaimer: I have already checked this: how to pass in the user-data when launching AWS instances using CLI. as well as other forums.
Also, is there any way to know if there is any warning messages etc which can give me some insights into what I am doing wrong? Is there any permission necessary at a AWS level ?
mkdir ~/latest
Whose home directory? Specify absolute path like mkdir /home/myuser/latest. Don't use C-Shell style notation.
Related
I am trying to provide bash command for bootstrapping my ec2 instance during creation time in the following way
aws ec2 run-instances --image-id ami-0000025f7c02a13b2 --count 1 --instance-type t2.micro --user-data '#!/bin/bash\nyum install git -y'
I can spin up the ec2 but I cannot get the bash script to work. In the logs I see the following
/bin/bash\nyum: bad interpreter: No such file or directory
which makes me feel like the string is formatted wrong.
Try adding a $ in front of your user data string.
aws ec2 run-instances --image-id ami-0000025f7c02a13b2 --count 1 --instance-type t2.micro --user-data $'#!/bin/bash\nyum install git -y'
If you intend to load a long script, it would be better to load the script from a file like this:
aws ec2 run-instances --image-id ami-abcd1234 --count 1 --instance-type m3.medium \
--key-name my-key-pair --subnet-id subnet-abcd1234 --security-group-ids sg-abcd1234 \
--user-data file://my_script.txt
and you file should be like this:
#!/bin/bash
yum update -y
service httpd start
chkconfig httpd on
See more details about loading data from a file while working with aws cli from this link
I have an AWS Educate stater account. I want to create an instances via the AWS command line.
Command:
aws2 ec2 run-instances --image-id ami-0bba96c31d87e65d9 --count 1 --instance-type t2.micro --key-name awskey --security-group-ids sg-1830914d --subnet-id subnet-a5d2def9 --region us-east-1 --placement AvailabilityZone=us-east-1d
This is the error I get:
An error occurred (Unsupported) when calling the RunInstances operation: The requested configuration is currently not supported. Please check the documentation for supported configurations.`
I am new to AWS, your help would highly be appreciated.
Thanks
There are few things you should check. First you should check that you have enough rights for your user to work with the ec2 section.
The next thing is that you should use the last version of the aws cli to prevent some problems.
Here is an example command how to run an instance from an AMI.
aws ec2 run-instances --image-id ami-xxxxxxxx --count 1 --instance-type t2.micro --key-name MyKeyPair --security-group-ids sg-903004f8 --subnet-id subnet-6e7f829e
So your line looks pretty fine.
Actual version of the CLI is 1.16. You use version 2 which is still in development. This could be the reason why you get some messages that the functions aren't supported at this moment. Perhaps you name is only "aws2".
https://pypi.org/project/awscli/#history
TL;DR: when launching an instance of an AMI created with CLI aws ec2 create-image the previous applied user-data is gone, whereas launching off an AMI created in the console has all user-data modifications.
Scenario:
I want to automate creation of a custom AMI for our use which itself is based on a regularly updated base AMI. Whenever I get a notification I take the new AMI ID and then run a script, which I'll excerpt from.
I spin up an EC2 instance to which I add user-data of some form. Create files, add packages, etc. This step is straight-forward and works.
# base_ami_id is set elsewhere
ec2_id=$(aws ec2 run-instances \
--image-id ${base_ami_id} \
--count 1 \
--instance-type t2.micro \
--key-name ${key_name} \
--security-group-ids ${security_group} \
--subnet-id ${subnet} \
--user-data file://user-data.sh \
--iam-instance-profile Name=${iamprof} \
--output text --query 'Instances[*].InstanceId' \
)
echo "Instance ID is ${ec2_id}"
echo "Waiting for instance ${ec2_id} to run"
aws ec2 wait instance-running --instance-ids ${ec2_id}
NOTE: At this point, I can ssh into the created instance and verify that cloud-init applied all my user-data correctly. All is well.
Taking the returned Instance ID, I create an AMI image, as per
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/creating-an-ami-ebs.html
echo Create image from instance ${ec2_id}
image_id=$(aws ec2 create-image --name ${image_name} --instance-id ${ec2_id} --output text --query 'ImageId')
echo "image ID is ${image_id}"
echo "Waiting for image ${image_id} to be available"
aws ec2 wait image-available --image-ids ${image_id}
echo "Image ${image_id} (${image_name}) available"
Watching in the console, after a while I see my new AMI.
To test, I launch an instance off the AMI created by this step - and am surprised to find that my modifications are NOT in the instance! It's as if I had launched off the original AMI. Which makes no sense: as described above, the user-data was there when I did a test login. And as seen in the shell excerpt above, I used the return $[ec2_id} which I got from the aws ec2 run-instance stage, as basis for the AMI creation and not , inadvertently, some other ID.
Making this even more confusing, I use the console and test by doing a Create Image from exactly that running instance, the one with Instance ID ${ec2_id} as above, which showed that all my user-data was there.
Then I launch an instance off that AMI - and won't you know it has all my modifications! Everything is there.
I've checked and triple-checked and I just don't see where/what I'm doing wrong! I thought maybe there's some extra command line options in aws ec2 create-image which is used in the console equivalent when making the API call. If there is, I can't see it.
What am I missing?!
It's like the AMI created from the console, off the same instance ID and the one from the CLI are different, but I've compared the ID numbers, they're definitely the same. You would think that using the right instance ID implies that the underlying snapshots and/or volumes would be the same, because --instance-id is the only value I can provide to create-image, right?
EDIT:
Following #Michael-sqlbot advice, I looked into the CloudTrail logs. Sadly that made this even more frustrating.
EDIT of EDIT:
I have removed the CloudTrail logs, as they turned out not to be pertinent to the issue and its solution and would quite possibly only confuse things.
I found the issue and how to fix it, and it may help others running into the same issue:
It turns out that using
aws ec2 wait instance-running
is NOT sufficient to ensure that all user-data is complete and has finished.
You may want to use
aws ec2 wait instance-status-ok
either in addition or instead. Even then you may want to be paranoid and add a simple sleep of several minutes to be certain!
I have been using the following AWS CLi command t launch EC2 Instances which works well but i would like to add these instances to my Load-balancer at the same time if possible??
aws ec2 run-instances --image-id ami-xxxxxxxx --count 1 --instance-type t1.micro --key-name MyKeyPair --security-group-ids sg-xxxxxxxx --subnet-id subnet-xxxxxxxx
To Manually Register i could use
elb-register-instances-with-lb MyLoadBalancer --instances i-xxxxxxx
but i want to add this to the launch process like autoscaling does?
Take a look at "Launching Instances with User Data". You can pass in an arbitrary shell script to be run after the instance launches. This includes AWS CLI commands, like elb-register-instances-with-lb.
On aws ec2 run-instances, you can include user data inline or as a file. You can also include user data when autoscaling.
From the run-instances documentation:
[-d user_data | -f filename]
-d, --user-data user_data
The base64-encoded MIME user data for the instances.
Type: String
Default: None
Required: No
Example: -d s3-bucket:my-logs
-f, --user-data-file filename
The file name of the user data for the instances.
Type: String
Default: None
Required: No
Example: -f user-data.txt
My base64 encoded user-data is ignored while running aws ec2 run-instances command.
Here is my user data:
$ cat user-data.sh
#!/bin/bash
cat >> /var/tmp/user-data-testing <<EOF
this is test line added at $(date)
EOF
here is base64 blob of above script:
IyEvYmluL2Jhc2gKY2F0ID4+IC92YXIvdG1wL3VzZXItZGF0YS10ZXN0aW5nIDw8RU9GCnRoaXMgaXMgdGVzdCBsaW5lIGFkZGVkIGF0ICQoZGF0ZSkKRU9GCg==
Now, My below command does read the user-data fine:
aws ec2 run-instances --image-id ami-8635a9b6 --instance-type t1.micro --placement AvailabilityZone=us-west-2a --security-groups quicklaunch-1 --key-name devops --user-data file://user-data.sh
I do see that file /var/tmp/user-data-testing is created.
However, when I try to pass-in user-data as a base64 encoded blob as below, then it gets ignored:
aws ec2 run-instances --image-id ami-8635a9b6 --instance-type t1.micro --placement AvailabilityZone=us-west-2a --security-groups quicklaunch-1 --key-name devops --user-data IyEvYmluL2Jhc2gKY2F0ID4+IC92YXIvdG1wL3VzZXItZGF0YS10ZXN0aW5nIDw8RU9GCnRoaXMgaXMgdGVzdCBsaW5lIGFkZGVkIGF0ICQoZGF0ZSkKRU9GCg==
Now, I do not see the file /var/tmp/user-data-testing created.
Also, I know that my base64 blob is healthy as I can decode it fine:
$ base64 --decode <<< IyEvYmluL2Jhc2gKY2F0ID4+IC92YXIvdG1wL3VzZXItZGF0YS10ZXN0aW5nIDw8RU9GCnRoaXMgaXMgdGVzdCBsaW5lIGFkZGVkIGF0ICQoZGF0ZSkKRU9GCg==
#!/bin/bash
cat >> /var/tmp/user-data-testing <<EOF
this is test line added at $(date)
EOF
However, I do see that instance metadata has my user data in base64 format:
$ curl -L http://169.254.169.254/latest/user-data/
IyEvYmluL2Jhc2gKY2F0ID4+IC92YXIvdG1wL3VzZXItZGF0YS10ZXN0aW5nIDw8RU9GCnRoaXMgaXMgdGVzdCBsaW5lIGFkZGVkIGF0ICQoZGF0ZSkKRU9GCg==
So, what am I doing wrong in using base64 user-data blob?
My instance meta-data is aware of it but seems like it is not really being executed (or decoded and executed) at the time of instance launch.
UPDATE:
If I pass the same base64 blob via AWS Console while launching the instance, It works. So seems like something is wrong in the way I am using it along with AWS-CLI.
UPDATE:
I just tried the same base64 blob with my ruby code as below and it worked as well:
ec2 = Aws::EC2.new
resp = ec2.run_instances(
min_count: 1,
max_count: 1,
image_id: 'ami-8635a9b6',
instance_type: 't1.micro',
placement: {
availability_zone: 'us-west-2a'
},
security_groups: ['quicklaunch-1'],
key_name: 'devops',
user_data: 'IyEvYmluL2Jhc2gKY2F0ID4+IC92YXIvdG1wL3VzZXItZGF0YS10ZXN0aW5nIDw8RU9GCnRoaXMgaXMgdGVzdCBsaW5lIGFkZGVkIGF0ICQoZGF0ZSkKRU9GCg=='
)
So, then WTF is wrong my implementation of AWS-CLI ?
It seems like awscli does the base64 encoding for you, so you should pass unencoded text to --user-data.
Apparently the documentation is not very clear on this. Check this link.
This syntax should then be:
aws ec2 run-instances --image-id ami-8635a9b6 --user-data "echo TEST"
or
aws ec2 run-instances --image-id ami-8635a9b6 --user-data file://path/to/file
Had the same issue, very frustrating to track down the problem, finally got it working.
did not base64 encode did put script in file.
placing seems to be important worked for me only when
--user-data file://path
is placed at the end
This format worked obviously change the some data to yours
aws ec2 run-instances --image-id amisomthing --count 1 --instance-type t1.micro --key-name keysomthing --security-group-ids somegroup --subnet-id somesubnetid --associate-public-ip-address --user-data file://someuserdata
According to the docs http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html, the base64 is only for API calls and not the CLI
AWS documentation - User data and the AWS CLI
Example: Specify user data at launch
To specify user data when you launch your instance, use the run-instances >command with the --user-data parameter. With run-instances, the AWS CLI >performs base64 encoding of the user data for you.
aws ec2 run-instances --image-id ami-abcd1234 --count 1 --instance-type m3.medium \
--key-name my-key-pair --subnet-id subnet-abcd1234 --security-group-ids sg-abcd1234 \
--user-data echo user data
The following example shows how to specify a script using a text file. Be sure to use the file:// prefix to specify the file.
aws ec2 run-instances --image-id ami-abcd1234 --count 1 --instance-type m3.medium \
--key-name my-key-pair --subnet-id subnet-abcd1234 --security-group-ids sg-abcd1234 \
--user-data file://my_script.txt
Luis already mentioned this earlier and should be awarded the correct answer.