I am using the aws cli to give me some output which I require in .csv format.
The below shows the commands I've put together:
aws ec2 describe-instances --output text --query 'Reservations[*].Instances[*].[VpcId, SubnetId, InstanceId, InstanceType, ImageId, State.Name, LaunchTime, Placement.AvailabilityZone, Placement.Tenancy, PrivateIpAddress, PrivateDnsName, PublicDnsName,[Tags[?Key==`Name`].Value],[Tags[?Key==`PowerData`].Value] ]' --output text | sed -E 's/\s+/,/g'
For some reason, the final two pieces of info, "Name" and "PowerData" end up on separate lines. This screws up my formatting when I open it up in a spreadsheet.
Anyone have any ideas on what might be causing that or how I may resolve it please?
The reason is TAGs return array so you need to print the only value on 0 index to display the TAGs on a single line.
You can verify this
aws ec2 describe-instances --output json --query 'Reservations[*].Instances[*].Tags[]'
output
[
{
"Key": "Name",
"Value": "demo"
},
{
"Key": "PowerData",
"Value": "demo"
}
]
so you need pipe expression
Tags[?Key==`Name`].Value| [0]
With the addition of filters, we could pass the result of one expression to another, operating on the result of a projection (or any expression).
Expression:
foo.*.bar | [0]
jmespath-pipe-expressions
So try this
aws ec2 describe-instances --output text --query 'Reservations[*].Instances[*].[VpcId, SubnetId, InstanceId, InstanceType, ImageId, State.Name, LaunchTime, Placement.AvailabilityZone, Placement.Tenancy, PrivateIpAddress, PrivateDnsName, PublicDnsName,Tags[?Key==`Name`].Value| [0],Tags[?Key==`PowerData`].Value | [0] ]' --output text | sed -E 's/\s+/,/g'
Related
I am trying to show all instances along with attached security groups, block device name and their deleteontermination status. I need to show this data in table format but I am continuously getting the below error.
Kindly help me to understand what I am missing here.
PR-MacBook-Pro:~ pr$ aws ec2 describe-instances --output table --query 'Reservations[*].Instances[*].[InstanceId,SecurityGroups[].GroupName,Placement.AvailabilityZone,BlockDeviceMappings[].DeviceName, BlockDeviceMappings[].Ebs.DeleteOnTermination]'
Row should have 1 elements, instead it has 2
PR-MacBook-Pro:~ pr$
I came across this while searching in google.
After I did a bit of research I found that you have to use | join(`, `, #)
So the command will be
aws ec2 describe-instances --output table --query 'Reservations[*].Instances[*].[InstanceId,SecurityGroups[].GroupName | join(`, `, #),Placement.AvailabilityZone,BlockDeviceMappings[].DeviceName | join(`, `, #), BlockDeviceMappings[].Ebs.DeleteOnTermination | join(`, `, to_array(to_string(#)))]'
modify your command as below to make it work
aws ec2 describe-instances --output table --region "us-east-1" --query 'Reservations[*].Instances[*].[InstanceId,SecurityGroups[].GroupName |[0],Placement.AvailabilityZone,BlockDeviceMappings[].DeviceName |[0], BlockDeviceMappings[].Ebs.DeleteOnTermination|[0] ]'
the response was in the format of
[
[
"i-xxxxxxxx",
[
"xxxxxx"
],
"us-east-1a",
[
"/dev/xvda"
],
[
true
]
]
]
so you need parse the array element using | [0]
I've got the following from describe-instances:
{
"Reservations": [
{
"Instances": [
{
"PublicDnsName": "ec2-xxxxx.amazonaws.com",
"Tags": [
{
"Key": "Name",
"Value": "yyyyy"
},
{
"Key": "budget_cluster",
"Value": "zzzzz"
},
{
"Key": "poc",
"Value": "aaaaaaa"
}
]
}
]
}
]
}
For each instance, I would like to extract the PublicDnsName and the value of the "budget_cluster" tag key. How to do this either with ec2 describe-instances or with jq ?
Modifying Frédéric's answer:
aws ec2 describe-instances --output text --query \
'Reservations[].Instances[].[PublicDnsName, Tags[?Key==`budget_cluster`].Value | [0]]'
Would produce:
ec2-xxxxx.amazonaws.com zzzzz
ec2-bbbbb.amazonaws.com yyyyy
I've changed the output to text, which removes as much formatting as possible and selected the individual tag value with | [0] since there will only ever be one per instance anyway. Finally, I removed the [] at the end so that the resulting list isn't flattened. That way in text output each entry will be on its own line.
You can also make this more robust by only selecting instances that actually have that tag. You could do so with further modifications to the --query parameter, but it is better in this case to use the --filters parameter since it does service-side filtering. Specifically you want the tag-key filter: --filters "Name=tag-key,Values=budget_cluster"
aws ec2 describe-instances --output text \
--filters "Name=tag-key,Values=budget_cluster" --query \
'Reservations[].Instances[?Tags[?Key==`budget_cluster`]].[PublicDnsName, Tags[?Key==`budget_cluster`].Value | [0]]'
Would still produce:
ec2-xxxxx.amazonaws.com zzzzz
ec2-bbbbb.amazonaws.com yyyyy
But over the wire you would only be getting the instances you care about, thus saving money on bandwidth.
Using jq 1.5 or later, the simplest approach is to use from_entries.
After a minimal fix of the illustrative input, the following invocation:
$ jq '.Reservations[]
| .Instances[]
| [.PublicDnsName, (.Tags|from_entries|.budget_cluster)]' input.json
produces:
[
"ec2-xxxxx.amazonaws.com",
"zzzzz"
]
If you do not have jq 1.5 or later, the following should work:
$ jq1.4 '.Reservations[]
| .Instances[]
| [.PublicDnsName, (.Tags[]|select(.Key=="budget_cluster") | .Value)]' input.json
The answer of #peak is great and I keep learning from him on the jq part but again you can achieve quite a lot from the AWS CLI
aws ec2 describe-instances --query \
'Reservations[].Instances[].{PublicDnsName:PublicDnsName, Budget:Tags[?Key==`budget_cluster`].Value}'
would produce
[
{
"PublicDnsName": "ec2-xxxxx.amazonaws.com",
"Budget": [
"zzzz",
]
}
]
If you do not want to make a new strict JSon out of it, just take the values
aws ec2 describe-instances --query \
'Reservations[].Instances[].[PublicDnsName, Tags[?Key==`budget_cluster`].Value][]'
will produce
[
"ec2-xxxxx.amazonaws.com",
[
"zzzz"
]
]
I am trying to get the info using of NetworkInterface using following command.
[root#ip-172-29-45-82 ~]# aws ec2 describe-instances --instance-ids i-dd6f6f53 --query Reservations[*].{VpcId:Instances[*].NetworkInterfaces[*].VpcId}
[
{
"VpcId": [
[
"vpc-38fb075d"
]
]
}
]
I don't want extra parentheses and angular brackets.
It should be something like as follows:
["VpcId":"vpc-38fb075d"] or [{"VpcId":"vpc-38fb075d"}] or {"VpcId":"vpc-38fb075d"}
Is there any way to achieve above output from the above command.
Thanks in advance.
Result can be further refined by adding this to the end of your command
--output text
And the output will be unstructured text:
"VpcId": "vpc-38fb075d"
I solved it.
I used following command.
aws ec2 describe-instances --instance-ids i-dd6f6f53 --query Reservations[0].{VpcId:Instances[0].NetworkInterfaces[0].VpcId}
I just changed * to 0 and its working.
Now the output is
{
"VpcId": "vpc-38fb075d"
}
You could run - given that you can't have ENIs in different VPCs:
aws ec2 describe-instances --query 'Reservations[].Instances[].VpcId' --output text --instance-ids i-dd6f6f53
You can also try nesting your JSON objects - for example, trying to get the name tag from the parent bracket Tags:
aws ec2 describe-instances --query 'Reservations[].Instances[].[InstanceId,PrivateIpAddress, [Tags[?Key==`Name`].Value][0][0]]' --output text
It is easier to develop the query using the JSON output, then cleaning things up with --output text.
I want to run an ec2 describe-instances command and get output in a table format as follows (where name is the Value of the Tag with Key 'Name'):
----------------------------------------------------------
| DescribeInstances |
+-------------+----------------+--------------+----------+
| instance_id | ip_address | name | state |
+-------------+----------------+--------------+----------+
| i-g93g494d | 99.99.99.01 | name1 | running |
| i-a93f754c | 99.99.99.02 | name2 | running |
+-------------+----------------+--------------+----------+
I can run the following command:
aws ec2 describe-instances --instance-ids i-g93g494d i-a93f754c --query "Reservations[*].Instances[*].{name: Tags[?Key=='Name'].Value, instance_id: InstanceId, ip_address: PrivateIpAddress, state: State.Name}" --output json
and obtain output:
[
[
{
"instance_id": "i-g93g494d",
"state": "running",
"ip_address": "99.99.99.01",
"name": [
"name1"
]
}
],
[
{
"instance_id": "i-a93f754c",
"state": "running",
"ip_address": "99.99.99.02",
"name": [
"name2"
]
}
]
]
However, when I run the same command with --output table rather than --output json I get an error.
command:
aws ec2 describe-instances --instance-ids i-g93g494d i-a93f754c --query "Reservations[*].Instances[*].{name: Tags[?Key=='Name'].Value, instance_id: InstanceId, ip_address: PrivateIpAddress, state: State.Name}" --output json
output:
list index out of range
I would like the table output to look like the above example but I'm having difficulty solving this. I'd very much appreciate any help anyone can offer on this.
You need to use pipe expression to filter the Tag results and get the first value such as:
aws ec2 describe-instances --instance-ids i-g93g494d i-a93f754c --query "Reservations[*].Instances[*].{name: Tags[?Key=='Name'] | [0].Value, instance_id: InstanceId, ip_address: PrivateIpAddress, state: State.Name}" --output table
There is a nice related blog post here: Get a list of instance with id, name and type
Here's a python program that can be used to generate listings from the descibe-instances command:
import json
import sys
with open( sys.argv[1] ) as f:
aws = json.load( f )
Reservations = aws['Reservations']
for Reservation in Reservations:
Instances = Reservation['Instances']
for Instance in Instances:
Tags = Instance['Tags']
for Tag in Tags:
if Tag['Key'] == "Name":
print Tag['Value'],Instance['PrivateIpAddress'],Instance['InstanceId'],Instance['State']['Name']
BTW, I absolutely LOVE Volkan Paksoy's answer. That's going into my bag o' tricks. The above python may have the advantage of allowing you to be more expressive with your search criteria or somehow combining results. In short, it's python, and you don't have to figure out how to use the aws syntax.
Here's how to invoke the python script above:
python parse.py <( aws ec2 describe-instances )
Of course, your mileage may vary. For instance, you may not have your region defaulted so, you may need to add an extra parameter as follows:
python parse.py <( aws ec2 --region us-east-2 describe-instances )
And you can than manipulate the output a bit to select only running instances and put in a well formatted column list:
python parse.py <( aws ec2 --region us-east-2 describe-instances ) | column -t | sort -k1,1 | cat -n | grep running
One last note, as I stated, I am a fan of the one liner provided by Volkan. I was just wondering how if the column ordering can be manipulated. I found that aws is sorting the columns alphabetically. To make the ordering obvious, I put numbers in front of the attribute names. However, these attribute names weren't interpreted properly as identifiers, so I had to quote them with double quotes. Here's what I wound up with, please note the subtle quoting changes ( and not so subtle quoting around 'Name' ):
aws ec2 describe-instances --query 'Reservations[*].Instances[*].{"01-name": Tags[?Key=='"'Name'"'] | [0].Value, "03-instance_id": InstanceId, "02-ip_address": PrivateIpAddress, "04-state" : State.Name}' --output table
I using the CLI tools and I want to list only the instance ID and related tags of an instance. The command that I am using is ec2-describe instances. I have tried certain filters but nothing is working out.
Can anybody help ??
error :
./Instance_Audit.sh: line 24: $: command not found
./Instance_Audit.sh: line 25: syntax error near unexpected token `do'
./Instance_Audit.sh: line 25: ` do echo $ID ; echo $(aws ec2 describe-instances --query "Reservations[].Instances[?InstanceId==\`$ID\`].Tags[]") done'
Using the aws cli, I can produce a few different alternatives of the tag output:
aws ec2 describe-instances --output text --query 'Reservations[*].Instances[*].[InstanceId,Tags[]]'
Result: i-xxxxx [{u'Value': 'key1value', u'Key': 'key1name'}, {u'Value': 'key2value', u'Key': 'key2name'}]
aws ec2 describe-instances --output text --query 'Reservations[*].Instances[*].[InstanceId,Tags[].*]'
Result: i-xxxxx [['key1value', 'key1name'], ['key2value', 'key2name']]
aws ec2 describe-instances --output text --query 'Reservations[*].Instances[*].[InstanceId,Tags[].Key,Tags[].Value]'
Result: i-xxxx ['key1name', 'key2name'] ['key1value' 'key2value']
Is any of those formats what you're looking for?
The AWS CLI command
aws ec2 describe-instances --region us-west-2 --query 'Reservations[].Instances[].[InstanceId,Tags[]]' --output text
produces these results:
i-789c55c3
Name Instance1
aws:cloudformation:logical-id Instance1
aws:cloudformation:stack-id arn:aws:cloudformation:us-west-2:012345678901:stack/test10-test10-foobarfoobar/a6e545a0-af52-11e4-a0be-50d5020578e0
aws:cloudformation:stack-name test10-test10-foobarfoobar
i-77c108cc
Name Instance2
aws:cloudformation:logical-id Instance2
aws:cloudformation:stack-id arn:aws:cloudformation:us-west-2:012345678901:stack/test10-test10-foobarfoobar/a6e545a0-af52-11e4-a0be-50d5020578e0
aws:cloudformation:stack-name test10-test10-foobarfoobar
If that is not in the format you are looking for, can you provide an example of they format you are expecting?
If you want to only display the instance id and the tags associated to that instance, you can use something like :
$ for ID in $(aws ec2 describe-instances --region eu-west-1 --query "Reservations[].Instances[].InstanceId" --output text); do echo $ID ; echo $(aws ec2 describe-instances --region eu-west-1 --query "Reservations[].Instances[?InstanceId==\`$ID\`].Tags[]") done
i-60****2a
[ [ { "Value": "SB", "Key": "Name" } ] ]
i-1a****56
[ [ { "Value": "Course and Schedule on Apache 2.4", "Key": "Name" } ] ]
i-88eff3cb
[ [ { "Value": "secret.com", "Key": "Name" } ] ]
i-e7****a5
[ [ { "Value": "2014.03 + Docker", "Key": "Name" } ] ]
I could not find a way to do that with only one AWS CLI call. Should someone come up with a shorter solution, I would love to hear from you.
If you want to filter to certain tag key/value only, you can edit the aws ec2 describe-instances to add a --filter option.
For example :
aws ec2 describe-instances --region eu-west-1 --filter "Name=tag:Name,Values=SB"
--Seb
simple answer:
aws ec2 describe-instances \
--query 'Reservations[*].Instances[*].[InstanceId,Tags[*]]'
not so simple answer .. the same principle could be applied to both instances and vpc's
# how-to get all the tags per instance-id with aws ec2 for prd env
while read -r i ; do \
aws ec2 describe-tags --filters "Name=resource-id,Values=$i" \
--profile prd; done < <(aws ec2 describe-instances \
--query 'Reservations[*].Instances[*].InstanceId' --profile prd)
# how-to get all the tags per virtual-private-cloud-id
while read -r v; do aws ec2 describe-tags \
--filters "Name=resource-id,Values=$v" ; \
done < <(aws ec2 describe-vpcs \
--query 'Vpcs[*].VpcId'|perl -ne 's/\t/\n/g;print')