I want to export the list displayed in the Volume section of the EC2 Console into CSV (or at least text) and I am trying to use the CLI for that.
I have trouble getting the tags information right using Filter Expression:
me#home:/$ aws ec2 describe-volumes --query 'Volumes[*].Tags[?Key=='Name'].Value'
[
[],
[],
[],
//...
[]
]
Here I expected to have a list with sometimes the value of the tag Name for the columes that have it and [] for those who don't.
Just to be sure, I checked and I do have Tags with that key (not always, though):
me#home:/$ aws ec2 describe-volumes --query 'Volumes[*].Tags[*].Key'
[
[
"kubernetes.io/created-for/pvc/name",
"Team",
"env",
"kubernetes.io/created-for/pvc/namespace",
"Name",
"kubernetes.io/cluster/DW",
"Software",
"Env",
"kubernetes.io/created-for/pv/name"
],
[
"env",
"Software",
"eks:nodegroup-name",
"Team",
"eks:cluster-name"
],
[
"Name",
"Software",
"Team",
"kubernetes.io/created-for/pvc/name",
"kubernetes.io/cluster/DW",
"kubernetes.io/created-for/pv/name",
"env",
"Env",
"kubernetes.io/created-for/pvc/namespace"
],
//...
]
So, how can I get that information right ?
One thing is that you are breaking the shell quotes with the ones inside your JMESPath expression.
You should be using a literal expression, instead
aws ec2 describe-volumes --query 'Volumes[*].Tags[?Key==`Name`].Value'
or using double quotes for the shell query option value
aws ec2 describe-volumes --query "Volumes[*].Tags[?Key=='Name'].Value"
Related
by using the below command i can able to get the details of my autoscaling group.
aws autoscaling describe-auto-scaling-groups --auto-scaling-group-name my-ASG --region=eu-west-1
but I need the value of a particular tag value only instead of the whole data in windows terminal. can someone help me
You can get all the tags for your ASG like the following.
aws autoscaling describe-tags --filters Name=auto-scaling-group,Values=my-asg-autoscaling --query 'Tags[].{Key: Key, Value: Value}'
Here is the result(dummy values)
[
{
"Key": "Env",
"Value": "qa"
},
{
"Key": "Function",
"Value": "as"
},
{
"Key": "Name",
"Value": "my-asg-autoscaling"
},
{
"Key": "Project",
"Value": "test"
},
{
"Key": "VPC",
"Value": "nonprod"
},
{
"Key": "monitored",
"Value": "non-prod"
}
]
If instead you want to get a particular tag e.g Function in the example, you can get it with the following query.
aws autoscaling describe-tags --filters Name=auto-scaling-group,Values=my-asg-autoscaling --query 'Tags[?Key==`Function`].Value[]'
I have a script that fetches list of instances having tag x having abc value. The count of ec2 instances returned are in hundreds, now for each instance I need to fetch 2 tag values. Not all instances will have both the tags, it could be 1 or both or none. For now I am issuing 2 calls to get the value of each tag (this is a bash shell)
market=`aws ec2 describe-tags --filters "Name=resource-id,Values=$id" "Name=key,Values=market" --query Tags[].Value --region $aws_region --output text`
service=`aws ec2 describe-tags --filters "Name=resource-id,Values=$id" "Name=key,Values=service" --query Tags[].Value --region $aws_region --output text`
Is there any way to fetch the values of both tags in a single call?
I have 4 instances like this:
i-020f43a6253e1dd25 tags:market=1
i-0a5c4b42fe3e75c15 tags:service=1
i-027ca3de0fe11f1d3 tags:market=4,service=4
i-0e77b17601f9b2fd2 tags:none
Server side filtering using --filters returns 4 matching records
% aws ec2 describe-tags --filters "Name=key,Values=market,service"
{
"Tags": [
{
"Key": "market",
"ResourceId": "i-020f43a6253e1dd25",
"ResourceType": "instance",
"Value": "1"
},
{
"Key": "market",
"ResourceId": "i-027ca3de0fe11f1d3",
"ResourceType": "instance",
"Value": "4"
},
{
"Key": "service",
"ResourceId": "i-027ca3de0fe11f1d3",
"ResourceType": "instance",
"Value": "4"
},
{
"Key": "service",
"ResourceId": "i-0a5c4b42fe3e75c15",
"ResourceType": "instance",
"Value": "1"
}
]
}
I would like to --query across multiple levels of the aws ec2 describe-instances API. For example, I would like to combine:
aws ec2 describe-instances --query 'Reservations[*].Instances[*].[ImageId, InstanceType, KeyName, State.Name, PublicIpAddress, NetworkInterfaces.Groups.GroupName]' --output json
and fields such as OwnerId which are a level above Instances.
You can include the top-level OwnerId by specifying it prior to Instances[*]:
aws ec2 describe-instances --query 'Reservations[*].[OwnerId,Instances[*].[ImageId, InstanceType, KeyName, State.Name, PublicIpAddress, NetworkInterfaces.Groups.GroupName]]' --output json
However, the fact that there is a one-to-many relationship to the instances means it will be returned at a higher level:
[
[
"123456789012",
[
[
"ami-48d38c2b",
"t2.micro",
"class",
"running",
"54.2.33.44",
null
]
]
],
[
"123456789012",
[
[
"ami-f806349b",
"t2.small",
"class",
"running",
"54.1.22.33",
null
]
]
]
]
By the way, the OwnerId is merely the ID of the AWS Account that owns the resource, which in almost every case would be the same value.
I need to list EBS VolumeID and the instance that it's attached to using the aws cli. This is the line I used:
aws ec2 describe-volumes --output text --query 'Volumes[*].{VolumeID:VolumeId, Instance:InstanceId}' | head -5
None vol-07210e47
None vol-743d1234
None vol-933d12d3
None vol-493c1309
None vol-1e3b145e
For some reason the instance IDs are showing as none. When the unfiltered output of the command shows that they're there:
aws ec2 describe-volumes | head -25
{
"Volumes": [
{
"AvailabilityZone": "us-east-1d",
"Attachments": [
{
"AttachTime": "2013-09-05T15:17:39.000Z",
"InstanceId": "i-c28e20ae",
"VolumeId": "vol-07210e47",
"State": "attached",
"DeleteOnTermination": false,
"Device": "/dev/sda1"
}
],
What am I doing wrong?
You're not querying into Attachments. This worked for me:
aws ec2 describe-volumes --output text --query 'Volumes[*].Attachments[].{VolumeID:VolumeId,InstanceID:InstanceId}'
This is a good link:
https://docs.aws.amazon.com/cli/latest/userguide/controlling-output.html
I want to assign one of my reserved Elastic IP's(ec2 classic ip) to Autoscaling group in VPC. Using AWS Cli I moved ip to vpc:
$ aws ec2 move-address-to-vpc --public-ip 23.23.23.23
And saw in aws concole, that this IP passed to VPC.
And Assigned in tags of AutoscalingGroup in Cloudformation template in Resources:
"Process": {
"Type" : "AWS::AutoScaling::AutoScalingGroup",
"Properties": {
"LaunchConfigurationName": {"Ref": "PreprocessorLC"},
"LoadBalancerNames": [{"Ref": "ProcessELB"}],
"VPCZoneIdentifier" : [{ "Fn::Join" : [",", [ { "Ref" : "PublicSubnet1"}, { "Ref" : "PublicSubnet2"} ]]}],
"AvailabilityZones": {"Ref": "AZs"},
"MinSize" : "1",
"MaxSize" : "1",
"HealthCheckGracePeriod": 300,
"Tags" : [
{"Key": "Name", "Value": {"Fn::Join": ["", [{"Ref": "Env"}, "-Process"]]}, "PropagateAtLaunch": true},
{"Key": "WorkersScalingGroup", "Value": {"Fn::Join": ["", ["Offering-", {"Ref": "Env"}, "-Process-Worker"]]}, "PropagateAtLaunch": true},
{"Key": "EIP", "Value": {"Ref": "ProcessIP"}, "PropagateAtLaunch": true},
{"Key": "Environment", "Value": {"Ref": "Env"}, "PropagateAtLaunch": true}
]
}
}
And added value of "ProcessIP" in Parameters:
"ProcessIP":{
"Description": "DEV: 23.23.23.23",
"Type": "String",
"Default": "23.23.23.23",
"AllowedValues": ["23.23.23.23"]
}
And it doesn't worked. Still get random IP.
If someone can tell where I'm wrong or what should to add for make it work?
Thanks!
In my case, I needed to keep a bank of unassigned EIPs and randomly assign them to the EC2 when they boot. That way I always know my servers will be using a specific list of IPs that I can whitelist in other places.
If you create several EIPs named "prod-pool" you can then use this script.
apt install -y jq awscli
ALLOCATION_ID=`aws ec2 describe-addresses --filters="Name=tag:Name,Values=prod-pool" | jq -r '.Addresses[] | "\(.InstanceId) \(.AllocationId)"' | grep null | awk '{print $2}' | xargs shuf -n1 -e`
if [ ! -z $ALLOCATION_ID ]; then
aws ec2 associate-address --instance-id $INSTANCE_ID --allocation-id $ALLOCATION_ID --allow-reassociation
fi
You can attached this policy to your IAM user
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowEIPAttachment",
"Effect": "Allow",
"Resource": [
"*"
],
"Action": [
"ec2:AssociateAddress",
"ec2:DisassociateAddress"
]
}
]
}
Here is simple bash script:
#!/bin/sh
# Region in Which instance is running
EC2_REGION='us-east-1'
AWS_ACCESS_KEY='xxxxxxxxxxxx'
AWS_SECRET_ACCESS_KEY='xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
#Instance ID captured through Instance meta data
InstanceID=`/usr/bin/curl -s http://169.254.169.254/latest/meta-data/instance-id`
#Elastic IP captured through the EIP instance tag
Elastic_IP=`/opt/aws/apitools/ec2/bin/ec2-describe-tags -O $AWS_ACCESS_KEY -W $AWS_SECRET_ACCESS_KEY --filter resource-id=$InstanceID --filter key='EIP' | cut -f5`
Allocate_ID=`/opt/aws/apitools/ec2/bin/ec2-describe-tags -O $AWS_ACCESS_KEY -W $AWS_SECRET_ACCESS_KEY --filter resource-id=$InstanceID --filter key="AllocationID" | cut -f5`
#Assigning Elastic IP to Instance
aws ec2 associate-address --instance-id $InstanceID --allocation-id $Allocate_ID
You need to explicitly associate the Elastic IP address with your desired EC2 instance. You can do this in a userdata script at launch time, or externally through other scripting or Configuration Management.
PropagateAtLaunch simply propagates tags from the Auto Scaling Group to any instances that are launched as a result of Auto Scaling actions. I'm not aware of any magic that would cause a tagged Elastic IP address to be associated with a launched instance.
See more discussion and examples of launch time scripting with EIPs here.
I created a AWS Lambda function which will automatically bind an Elastic IP address from a pool to instance of an autoscaling group. This alleviates the need to grab an EIP address in the bootscript of the instances. For a complete description check out
https://binx.io/blog/2019/09/02/how-to-dynamically-bind-elastic-ip-addresses-to-an-auto-scaling-group/