I would like to --query across multiple levels of the aws ec2 describe-instances API. For example, I would like to combine:
aws ec2 describe-instances --query 'Reservations[*].Instances[*].[ImageId, InstanceType, KeyName, State.Name, PublicIpAddress, NetworkInterfaces.Groups.GroupName]' --output json
and fields such as OwnerId which are a level above Instances.
You can include the top-level OwnerId by specifying it prior to Instances[*]:
aws ec2 describe-instances --query 'Reservations[*].[OwnerId,Instances[*].[ImageId, InstanceType, KeyName, State.Name, PublicIpAddress, NetworkInterfaces.Groups.GroupName]]' --output json
However, the fact that there is a one-to-many relationship to the instances means it will be returned at a higher level:
[
[
"123456789012",
[
[
"ami-48d38c2b",
"t2.micro",
"class",
"running",
"54.2.33.44",
null
]
]
],
[
"123456789012",
[
[
"ami-f806349b",
"t2.small",
"class",
"running",
"54.1.22.33",
null
]
]
]
]
By the way, the OwnerId is merely the ID of the AWS Account that owns the resource, which in almost every case would be the same value.
Related
I have a script that fetches list of instances having tag x having abc value. The count of ec2 instances returned are in hundreds, now for each instance I need to fetch 2 tag values. Not all instances will have both the tags, it could be 1 or both or none. For now I am issuing 2 calls to get the value of each tag (this is a bash shell)
market=`aws ec2 describe-tags --filters "Name=resource-id,Values=$id" "Name=key,Values=market" --query Tags[].Value --region $aws_region --output text`
service=`aws ec2 describe-tags --filters "Name=resource-id,Values=$id" "Name=key,Values=service" --query Tags[].Value --region $aws_region --output text`
Is there any way to fetch the values of both tags in a single call?
I have 4 instances like this:
i-020f43a6253e1dd25 tags:market=1
i-0a5c4b42fe3e75c15 tags:service=1
i-027ca3de0fe11f1d3 tags:market=4,service=4
i-0e77b17601f9b2fd2 tags:none
Server side filtering using --filters returns 4 matching records
% aws ec2 describe-tags --filters "Name=key,Values=market,service"
{
"Tags": [
{
"Key": "market",
"ResourceId": "i-020f43a6253e1dd25",
"ResourceType": "instance",
"Value": "1"
},
{
"Key": "market",
"ResourceId": "i-027ca3de0fe11f1d3",
"ResourceType": "instance",
"Value": "4"
},
{
"Key": "service",
"ResourceId": "i-027ca3de0fe11f1d3",
"ResourceType": "instance",
"Value": "4"
},
{
"Key": "service",
"ResourceId": "i-0a5c4b42fe3e75c15",
"ResourceType": "instance",
"Value": "1"
}
]
}
I want to export the list displayed in the Volume section of the EC2 Console into CSV (or at least text) and I am trying to use the CLI for that.
I have trouble getting the tags information right using Filter Expression:
me#home:/$ aws ec2 describe-volumes --query 'Volumes[*].Tags[?Key=='Name'].Value'
[
[],
[],
[],
//...
[]
]
Here I expected to have a list with sometimes the value of the tag Name for the columes that have it and [] for those who don't.
Just to be sure, I checked and I do have Tags with that key (not always, though):
me#home:/$ aws ec2 describe-volumes --query 'Volumes[*].Tags[*].Key'
[
[
"kubernetes.io/created-for/pvc/name",
"Team",
"env",
"kubernetes.io/created-for/pvc/namespace",
"Name",
"kubernetes.io/cluster/DW",
"Software",
"Env",
"kubernetes.io/created-for/pv/name"
],
[
"env",
"Software",
"eks:nodegroup-name",
"Team",
"eks:cluster-name"
],
[
"Name",
"Software",
"Team",
"kubernetes.io/created-for/pvc/name",
"kubernetes.io/cluster/DW",
"kubernetes.io/created-for/pv/name",
"env",
"Env",
"kubernetes.io/created-for/pvc/namespace"
],
//...
]
So, how can I get that information right ?
One thing is that you are breaking the shell quotes with the ones inside your JMESPath expression.
You should be using a literal expression, instead
aws ec2 describe-volumes --query 'Volumes[*].Tags[?Key==`Name`].Value'
or using double quotes for the shell query option value
aws ec2 describe-volumes --query "Volumes[*].Tags[?Key=='Name'].Value"
I need to list EBS VolumeID and the instance that it's attached to using the aws cli. This is the line I used:
aws ec2 describe-volumes --output text --query 'Volumes[*].{VolumeID:VolumeId, Instance:InstanceId}' | head -5
None vol-07210e47
None vol-743d1234
None vol-933d12d3
None vol-493c1309
None vol-1e3b145e
For some reason the instance IDs are showing as none. When the unfiltered output of the command shows that they're there:
aws ec2 describe-volumes | head -25
{
"Volumes": [
{
"AvailabilityZone": "us-east-1d",
"Attachments": [
{
"AttachTime": "2013-09-05T15:17:39.000Z",
"InstanceId": "i-c28e20ae",
"VolumeId": "vol-07210e47",
"State": "attached",
"DeleteOnTermination": false,
"Device": "/dev/sda1"
}
],
What am I doing wrong?
You're not querying into Attachments. This worked for me:
aws ec2 describe-volumes --output text --query 'Volumes[*].Attachments[].{VolumeID:VolumeId,InstanceID:InstanceId}'
This is a good link:
https://docs.aws.amazon.com/cli/latest/userguide/controlling-output.html
I want to run an ansible role, for all the ip address in the vpc which are running.
How to get all the ip address of running instance in given vpc
Things: I have tired:
aws ec2 describe-instances --filters "Name=vpc-id,
Values="vpc-******"" --query
"Reservations[].Instances[].PrivateIpAddresses[*]" --output text
This is returning null
The name of the parameter is PrivateIpAddress not PrivateIpAddresses as you can see from Json object
[
[
{
"Monitoring": {
"State": "disabled"
},
"PublicDnsName": "xxxx",
"RootDeviceType": "ebs",
"State": {
"Code": 16,
"Name": "running"
},
"EbsOptimized": false,
"LaunchTime": "xxx",
"PublicIpAddress": "xxx",
"PrivateIpAddress": "xxxxx",
"ProductCodes": [
....
so if you run your command as
aws ec2 describe-instances --filters "Name=vpc-id, Values="vpc-cda7c6a8"" --query "Reservations[*].Instances[*].PrivateIpAddress" --output text
you will have your expected result
it's PrivateIPAddress, not Addresses
aws ec2 describe-instances --instance-ids --query Reservations[].Instances[].PrivateIpAddress
Hope this helps
In aws cloudformation how to add "NetworkInterfaces" in Autoscaling launchconfig group as I want to configure every instance launched and I need "NetworkInterfaces" to be there same AWS::EC2::Instance?
The solution I'm currently using is to ensure that every Instance is launched with an IAM Instance Profile that allows contains the Policy
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ec2:CreateTags",
"ec2:DescribeSubnets",
"ec2:AttachNetworkInterface",
"ec2:CreateNetworkInterface",
"ec2:ModifyNetworkInterfaceAttribute"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
My Cloud Formation Template for creating the AutoScalingGroup and LaunchConfiguration receives Parameters for Subnets and Security Groups to attach to each additional ENI.
"SecondaryNICSubnetIds":{
"Type" : "CommaDelimitedList",
"Description" : "Ensure that the spread of Availability Zones for these Subnets matches the SubnetIds used to create Instances, as when creating a Secondary ENI it must exist in the same AZ as the Instance"
},
"SecondaryNICSecurityGroupIds":{
"Type" : "CommaDelimitedList",
"Description" : "Security Groups to associate to the Secondary ENI"
},
Then the LaunchConfiguration contains a UserData property that
ensures awscli is installed
sets the region to use by parse the instance metadata identity document
gets the instance id from the instance metadata
gets the availability zone for the instance from the instance metadata
from the Subnets that have been passed in find the first that matches my instances AZ by using the awscli describe-subnets call
Creates a Network Interface in my chosen Subnet, and add Security Groups to it using the awscli create-network-interface call
Tag my ENI using the awscli create-tags call
Attach ENI to my Instance using the awscli attach-network-interface call
Modify Attachment to Delete the ENI on Instance Termination using the modify-network-interface-attribute call
"UserData": {
"Fn::Base64" : {
"Fn::Join": [ "\n",
[
"#!/bin/bash -xe",
"sudo apt-get install -y awscli",
"export AWS_DEFAULT_REGION=$(curl -sS http://169.254.169.254/latest/dynamic/instance-identity/document | python -c 'import sys, json; print(json.load(sys.stdin)[\"region\"])')",
"INSTANCE_ID=$(curl -sS http://169.254.169.254/latest/meta-data/instance-id)",
"AZ=$(curl -sS http://169.254.169.254/latest/meta-data/placement/availability-zone)",
"echo Availability Zone: ${AZ}",
{"Fn::Sub":[
"SUBNET_ID=$(aws ec2 describe-subnets --subnet-ids ${SubnetNetIds} --filters Name=availabilityZone,Values=${!AZ} --query 'Subnets[0].SubnetId' --output text)",
{"SubnetNetIds": {"Fn::Join": [" ", {"Ref": "SecondaryNICSubnetIds"} ] }}
]},
"echo Subnet Id: ${SUBNET_ID}",
{"Fn::Sub":[
"ENI_ID=$(aws ec2 create-network-interface --subnet ${!SUBNET_ID} --description 'Secondary ENI' --groups ${SecurityGroups} --query 'NetworkInterface.NetworkInterfaceId' --output text)",
{"SecurityGroups": {"Fn::Join": [" ", {"Ref": "SecondaryNICSecurityGroupIds"}]} }
]},
"echo ENI ID: ${ENI_ID}",
"aws ec2 create-tags --resources ${!ENI_ID} --tags Key=Some,Value=Tag",
"ATTACHMENT_ID=$(aws ec2 attach-network-interface --network-interface-id ${ENI_ID} --instance-id ${INSTANCE_ID} --device-index 1 --output text)",
"echo Attachment ID: ${ATTACHMENT_ID}",
"echo Delete On Termination: $(aws ec2 modify-network-interface-attribute --network-interface-id ${ENI_ID} --attachment AttachmentId=${ATTACHMENT_ID},DeleteOnTermination=true --output text)"
]
]
}
}
If you didn't want to pass the Subnets into the Cloud Formation Template you could attempt to look them up by adding tags to --query in the awscli describe-subnets call, if your infrastructure allows you to identify them this way.