I have setup a few of the amazon AWS CLI tools (EC2, Auto Scaling, MOnitoring and ELB). The tools are setup correctly and work perfectly. My environment vars are all set, the relevant ones to this Q being:
export EC2_REGION=eu-west-1
export EC2_URL=https://ec2.$EC2_REGION.amazonaws.com
export AWS_ELB_URL=https://elasticloadbalancing.$EC2_REGION.amazonaws.com
When I run ec2-describe-instance-status i-XXXXXXXX for ANY of my instances, I get:
Client.InvalidInstanceID.NotFound: The instance ID 'i-XXXXXXXX' does not exist
I KNOW the instance ID exists, I copied it out of the AWS web console, and it is in the eu-west-1 region, and my env vars are set to this region.
For the life of me I can't figure out why it will not find my instances. Is there anything glaringly obvious that I am doing incorrectly?
UPDATE: recreating x509 cert/pk solved this... for some reason.
I had the same problem. It was because I wasn't defining a region for my commands. I assumed it would list all instances across all regions but it defaults to us-west-1 and I don't have any instances there.
To describe my machines in Ireland I use the following:
ec2-describe-instances --region eu-west-1
NB: I'm defining my AWS access key and secret elsewhere.
To avoid this problem going forward, I've now set my region via an environment variable on my linux and windows machines: EC2_URL=https://ec2.eu-west-1.amazonaws.com
so that I don't have to be explicit on the command line.
Update May 2014 You can also set the region by adding the following lines to the ~/.aws/config file in your home folder (not tested on Windows). This is my preferred method now, especially on my VM's and containers:
[default]
region = eu-west-1
For more information see the offical docs here.
Update May 2021
Since I work across so many regions now I use Implicit and ephemeral environment variables to define my region for that command and NOT have a default in my .aws/config which can be dangerous. This also makes bash scripting easier as I can define it for the whole script/utility. It's a tiny bit more typing but far safer, more flexible and transparent e.g.:
AWS_DEFAULT_REGION=eu-central-1 aws ec2 describe-instances
# or for a script/utility
AWS_DEFAULT_REGION=us-east-1 ./tagInstances.sh
In my case, I had two sets of credentials in ~/.aws/credentials. To specify the credentials tag, use
aws ec2 describe-instances --instance-id <your-instance-id> --profile <your-profile-name> --region <your-region>
Weird issue - as usual when encountering something weird in software development, one should first question the assumptions:
I KNOW the instance ID exists, I copied it out of the AWS web console,
and it is in the eu-west-1 region, and my env vars are set to this
region.
So the instance ID stems from a different environment than the one you want to use it in - I would try to derive the instance ID via the same environment instead, i.e.:
ec2-describe-instances
I venture the guess that the list won't return the instances you are expecting. This would indicate that you are either using AWS credentials that belong to another account or that these credentials do not have the required Amazon EC2 read permissions assigned via IAM policies for example.
I had a similar issue and I write here the solution for anybody who can find it helpful.
I was stucked with this error during some hours.
Client.InvalidInstanceID.NotFound: The instance ID 'i-XXXXXXXX' does not exist
Finally I found what was happening: I had my instance in a different region than the default region (US East (Northern Virginia)) and I had to update this information. By default the commands look only for instances in the default region!
It is explained in the docs, section (Optional): Set the region http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/SettingUp_CommandLine.html
It's very simple problem. If you are getting this error
Client.InvalidInstanceID.NotFound: The instance ID 'i-XXXXXXXX' does not exist
Then follow the steps.
Check in which country region your instance is image here
Now enter root#Indian:~# aws configure
Enter
AWS Access Key ID [****************D7M2]:
AWS Secret Access Key [****************2h3r]:
Default region name [us-east-1]:
When asked for default region, Change the region to which the instances is residing. Eg: us-east-2. Then press Enter.
Acutally these are the list of available Region names "RegionNames"
"Regions": [
{
"RegionName": "ap-south-1",
"Endpoint": "ec2.ap-south-1.amazonaws.com"
},
{
"RegionName": "eu-west-2",
"Endpoint": "ec2.eu-west-2.amazonaws.com"
},
{
"RegionName": "eu-west-1",
"Endpoint": "ec2.eu-west-1.amazonaws.com"
},
{
"RegionName": "ap-northeast-2",
"Endpoint": "ec2.ap-northeast-2.amazonaws.com"
},
{
"RegionName": "ap-northeast-1",
"Endpoint": "ec2.ap-northeast-1.amazonaws.com"
},
{
"RegionName": "sa-east-1",
"Endpoint": "ec2.sa-east-1.amazonaws.com"
},
{
"RegionName": "ca-central-1",
"Endpoint": "ec2.ca-central-1.amazonaws.com"
},
{
"RegionName": "ap-southeast-1",
"Endpoint": "ec2.ap-southeast-1.amazonaws.com"
},
{
"RegionName": "ap-southeast-2",
"Endpoint": "ec2.ap-southeast-2.amazonaws.com"
},
{
"RegionName": "eu-central-1",
"Endpoint": "ec2.eu-central-1.amazonaws.com"
},
{
"RegionName": "us-east-1",
"Endpoint": "ec2.us-east-1.amazonaws.com"
},
{
"RegionName": "us-east-2",
"Endpoint": "ec2.us-east-2.amazonaws.com"
},
{
"RegionName": "us-west-1",
"Endpoint": "ec2.us-west-1.amazonaws.com"
},
{
"RegionName": "us-west-2",
"Endpoint": "ec2.us-west-2.amazonaws.com"
}
]
}
Default output format [None]:
Leave the output format blank and press Enter. Now You are Done
Now in the console just type
root#Indian-3543:~# aws ec2 describe-instance --instance-id i-06343434322t
MAKE HAPPY BE HAPPY
I got this fixed by changing EC2_URL from 'https://ec2.ap-southeast-1.amazonaws.com' to 'ec2.ap-southeast-1.amazonaws.com'
Related
I'm trying to create an AMI by packer in a AWS codebuild project.
This AMI will be used to launch template
and the launch template will be used to ASG.
and when the ASG get an instance by this launch template, it should work with an existing target group for ALB.
for clarification, my expectation is...
generate AMI in a code build project by packer
create launch template with the #1 AMI
use the #2 launch template to ASG
ASG launch a new instance
existing target group do health check #4 instance.
In the step 5, my existing target group failed to do health check well for the new instance because it had different vpc.
(existing target group is using a custom VPC and the #4 instance had default vpc)
So, I backed to #1 to set the same VPC during the AMI generation.
But the codebuild project failed when it called the packer template in it.
it returned below
==> amazon-ebs: Prevalidating AMI Name...
amazon-ebs: Found Image ID: ami-12345678
==> amazon-ebs: Creating temporary keypair: packer_6242d99f-6cdb-72db-3299-12345678
==> amazon-ebs: Launching a source AWS instance...
==> amazon-ebs: Error launching source instance: UnauthorizedOperation: You are not authorized to perform this operation.
Before this update, there were no vpc and subnet related settings in the packer template, and they worked.
I added some vpc related permissions for this code build project but no lucks yet.
Below is my builders configuration on the packer-template.json
"builders": [
{
"type": "amazon-ebs",
"region": "{{user `aws_region`}}",
"instance_type": "t2.micro",
"ssh_username": "ubuntu",
"associate_public_ip_address": true,
"subnet_id": "subnet-12345678",
"vpc_id": "vpc-12345678",
"iam_instance_profile": "blah-profile-12345678",
"security_group_id": "sg-12345678",
"ami_name": "{{user `new_ami_name`}}",
"ami_description": "AMI from Packer {{isotime \"20060102-030405\"}}",
"source_ami_filter": {
"filters": {
"virtualization-type": "hvm",
"name": "{{user `source_ami_name`}}",
"root-device-type": "ebs"
},
"owners": ["************"],
"most_recent": true
},
"tags": {
"Name": "{{user `new_ami_name`}}"
}
}
],
Added on this step (not exist before)
subnet_id
vpc_id
iam_instance_profile
security_group_id
Q1. Is this correct configuration to use VPC on here?
Q1-1. If yes, which permissions are required to allow this task?
Q1-2. If not, could you let me know the correct format of this?
Q2. Or... Is this correct way to get some instances which are able to communicate with my existing target groups...?
Thanks in advance. Your any kind of mentions will be helpful to me.
I got some helps from a local community.
And now I see I wrote too much wide and not good question without enough informations. There were several issues.
I should have used CloudTrail instead of CloudWatch to know which role and actions are making problems. My codebuild project had not ec2.RunInstances permission.
After I saw this on CloudTrail, I updated the role policy for the codebuild project and it was passed. But there was another issue.
After launch the instance by packer, it failed to connect with ssh. I got some answers from Stack overflow by searching about packer's timeout issue by ssh. and update the security group to allow ssh for packer.
Will remove this question if it is required.
Thanks for my local community and the previous answers & questioners on Stack overflow.
I am writing a small python function to know the aws region of our AWS EIPs. When I use the ipinfo services such as ipinfo.io, I do get these details:
~ ➤ curl ipinfo.io/18.138.84.13/json
{
"ip": "18.138.84.13",
"hostname": "ec2-18-138-84-13.ap-southeast-1.compute.amazonaws.com",
"city": "Singapore",
"region": "Singapore",
"country": "SG",
"loc": "1.2897,103.8501",
"org": "AS16509 Amazon.com, Inc.",
"postal": "048508",
"timezone": "Asia/Singapore",
"readme": "https://ipinfo.io/missingauth"
}%
~ ➤
From this information, I get all the details except the region_name. Of course, I can trim the hostname URL to grep the region name i.e. ap-southeast-1 but that is not optimal.
Is there any way in either boto3 or aws cli where I can hit the aws API with params like region=Singapore and in response I get ap-southeast-1.
Or if there is any aws api I can hit directly by giving input as EIP and it return me details including region_name. I need to use region_name further in my script for automating the job.
In AWS, IP addresses are assigned dynamically, they are not bounded by region, The eip pool will automatically you IP address which they have available. I would suggest to ignore the region base on ip address.
I have solved it by this:
hostname.rsplit('.', 4)[1]
This hostname is in the response of ipinfo.io/18.138.84.13/json
While I try to create a new secrets for RDS using AWS CLI, I couldn't find the way where I could associate my secrets to RDS DB on creation itself.I have gone through the AWS CLI(https://docs.aws.amazon.com/cli/latest/reference/secretsmanager/create-secret.html) and wasn't able to find a way for its association.
Do I need to do this step manually by logging into the console or could this be automated in some way?
Could anyone please help.
Thanks.
The association is only based on the form of the secret-string. For the RDS the forms are listed here.
For instance, for mysql the form of the secret-string is as follows:
{
"engine": "mysql",
"host": "<required: instance host name/resolvable DNS name>",
"username": "<required: username>",
"password": "<required: password>",
"dbname": "<optional: database name. If not specified, defaults to None>",
"port": "<optional: TCP port number. If not specified, defaults to 3306>"
}
Thus, to create the secret for mysql using CLI:
Create file called mydb.json (example):
{
"username": "admin",
"password": "asdf435325gfdg",
"engine": "mysql",
"host": "database-2.cba4sasaubqv.us-east-1.rds.amazonaws.com",
"port": 3306,
"dbInstanceIdentifier": "database-2"
}
Execute:
aws secretsmanager create-secret --name mysql-info --secret-string file://mydb.json
The more confusing CLI part begins when you want to enable an automatic secret rotations. I will just leave a link for that (it also has CLI info) if you are interested in this as well:
Enabling Rotation for an Amazon RDS Database Secret
I am going through the AWS well-architected labs guides and am on the automated deployment of the VPC (https://www.wellarchitectedlabs.com/Security/200_Automated_Deployment_of_VPC/Lab_Guide.html).
The instructions are pretty straight forward and I upload the template provided (tried both the one linked directly in the instructions and the one from their git repo I found here https://github.com/awslabs/aws-well-architected-labs/blob/master/Security/200_Automated_Deployment_of_VPC/Code/vpc-alb-app-db.yaml). But when it runs, I keep getting failure messages. These errors saying unable to spin up things in us-west-1c:
And this for the VPC:
I am logged in with an IAM user that had admin access and tried cleaning up the YAML as there was some duplicate fields I found, but nothing worked. Is it just that my account can't create anything in us-west-1c because it is overloaded or something, or is it some other issue?
The answer is in the picture: CloudFormation could not create ALB1Subnet3 because the availability zone us-west-1c doesn't exist. When anything in a new stack fails, the entire stack is rolled back.
As described here, you can get the list of availability zones for a region with the CLI:
aws ec2 describe-availability-zones --region us-west-1
You'll see that this region only has two availability zones:
{
"AvailabilityZones": [
{
"State": "available",
"OptInStatus": "opt-in-not-required",
"Messages": [],
"RegionName": "us-west-1",
"ZoneName": "us-west-1b",
"ZoneId": "usw1-az3",
"GroupName": "us-west-1",
"NetworkBorderGroup": "us-west-1"
},
{
"State": "available",
"OptInStatus": "opt-in-not-required",
"Messages": [],
"RegionName": "us-west-1",
"ZoneName": "us-west-1c",
"ZoneId": "usw1-az1",
"GroupName": "us-west-1",
"NetworkBorderGroup": "us-west-1"
}
]
}
Update
It's been pointed out that us-west-1c appears in my availability zone listing. This happens because my availability zones are not the same as the OP's availability zones. To quote the docs:
To ensure that resources are distributed across the Availability Zones for a Region, we independently map Availability Zones to names for each AWS account. For example, the Availability Zone us-east-1a for your AWS account might not be the same location as us-east-1a for another AWS account.
So for the OP, the error message was correct: there is no us-west-1c. And my explanation -- that us-west-1 only has two zones -- was also correct.
However, to take this to the bitter end (and head off any more comments), if you look at the AWS Global Infrastructure map (about halfway down the page here), and mouse over the us-west-1 region, you'll see that it has three AZs. However, any given account may not have access to all of the AZs in a region.
I am starting to play with AWS.
I have created an EC2 instance using the AWS management console.
I would like to be able to create new, similar instances using the CLI so I've been looking at get-launch-template-data (which states "Retrieves the configuration data of the specified instance. You can use this data to create a launch template.") and expected the output of that to be valid input to create-launch-template.
I've viewed the AWS CLI documentation, and looked on StackOverflow but the only related issues I've found has been these ones:
Unable to create launchtemplate using awscli and
Amazon Launch Template - Updated AMI
I've been running:
aws ec2 get-launch-template-data --instance-id "i-xxx" --query "LaunchTemplateData" > MyLaunchData
aws ec2 create-launch-template --launch-template-name xxx --launch-template-data file://MyLaunchData
The error I get is:
An error occurred (InvalidInterfaceType.Malformed) when calling the CreateLaunchTemplate operation: '%s' is not a valid value for interface type. Enter a valid value and try again.
What I think is the relevant part of MyLaunchData is:
"NetworkInterfaces": [
{
"AssociatePublicIpAddress": true,
"DeleteOnTermination": true,
"Description": "",
"DeviceIndex": 0,
"Groups": [
"sg-xxx"
],
"InterfaceType": "interface",
"Ipv6Addresses": [],
"PrivateIpAddresses": [
{
"Primary": true,
"PrivateIpAddress": "xxx"
}
],
"SubnetId": "subnet-xxx"
}
],
Can someone point me in the right direction please?
(I've obviously replaced what I think is my data with xxx for privacy)
Many thanks
The InterfaceType is not allowed:
'%s' is not a valid value for interface type
I know that you are using the output from get-launch-template-data but there is no "InterfaceType": "interface".
AWS EC2 documentation
InterfaceType
Indicates the type of network interface. To create an Elastic Fabric Adapter (EFA),
specify efa. For more information, see Elastic Fabric Adapter in the Amazon Elastic
Compute Cloud User Guide.
Required: No
Type: String
Allowed Values: efa
To add to vo1umen's answer, to get around this issue you obviously need to remove InterfaceType from the NetworkInterfaces array. One way of doing this is to use jq:
jq -c 'del(.NetworkInterfaces[0]["InterfaceType"])' <<< $TemplateData
The -c is to keep the JSON flat formatted, you may not need to do that when storing in a file.