List all AWS Elasticache snapshots taken after a specified date - amazon-web-services

I am trying to write a query in AWS CLI which will provide with the elasticache snapshots names older than a specific creation date.
I tried with a JMESPath query like:
aws elasticache describe-snapshots \
--region ap-southeast-1 \
--snapshot-source "manual" \
--query 'Snapshots[*].NodeSnapshots[?SnapshotCreateTime >`2022-10-01`] | [?not_null(node)]'
But, this is giving me an empty result.
Snippet of aws elasticache describe-snapshots:
{
"Snapshots": [{
"SnapshotName": "snapshot-name",
"ReplicationGroupId": "rep-id",
"ReplicationGroupDescription": "redis cluster",
"CacheClusterId": null,
"SnapshotStatus": "available",
"SnapshotSource": "automated",
"CacheNodeType": "cache.r6g.large",
"Engine": "redis",
"EngineVersion": "6.0.5",
"NumCacheNodes": null,
"PreferredAvailabilityZone": null,
"CacheClusterCreateTime": null,
"PreferredMaintenanceWindow": "sun:20:00-sun:20:00",
"TopicArn": null,
"Port": "6379",
"CacheParameterGroupName": "default.redis6.x.cluster.on",
"CacheSubnetGroupName": "redis-group",
"VpcId": "vpc-01bcajghfghj",
"AutoMinorVersionUpgrade": "true",
"SnapshotRetentionLimit": "18",
"SnapshotWindow": "20:00-21:00",
"NumNodeGroups": "1",
"AutomaticFailover": "enabled",
"NodeSnapshots": [{
"CacheClusterId": "redis-cluster-01",
"NodeGroupId": "001",
"CacheNodeId": "001",
"NodeGroupConfiguration": null,
"CacheSize": "20 GB",
"CacheNodeCreateTime": "1632909889675",
"SnapshotCreateTime": "1667246439000"
}],
"KmsKeyId": "kms-id.."
}]
}

If we take as an example the JSON given in the documentation:
{
"Snapshots": [
{
"SnapshotName": "automatic.my-cluster2-002-2019-12-05-06-38",
"NodeSnapshots": [
{
"CacheNodeId": "0001",
"SnapshotCreateTime": "2019-12-05T06:38:23Z"
}
]
},
{
"SnapshotName": "myreplica-backup",
"NodeSnapshots": [
{
"CacheNodeId": "0001",
"SnapshotCreateTime": "2019-11-26T00:25:01Z"
}
]
},
{
"SnapshotName": "my-cluster",
"NodeSnapshots": [
{
"CacheNodeId": "0001",
"SnapshotCreateTime": "2019-11-26T03:08:33Z"
}
]
}
]
}
Then, you can realise that you need to filter on SnapshotCreateTime, nested under the array NodeSnapshots.
So, what you need, here, is a double filter:
One to filter by date:
[?SnapshotCreateTime > `2022-10-01`]
Then, one to exclude all snapshots where the NodeSnapshots array have been emptied by the previous filter:
[?NodeSnapshots[?SnapshotCreateTime > `2022-10-01`]]
And, so, if you only care about the name of the snapshot, you could do with the query:
Snapshots[?NodeSnapshots[?SnapshotCreateTime > `2022-10-01`]].SnapshotName
So, your command ends up being:
aws elasticache describe-snapshots \
--region ap-southeast-1 \
--snapshot-source "manual" \
--query 'Snapshots[?
NodeSnapshots[?SnapshotCreateTime > `2022-10-01`]
].SnapshotName'
Now, with what you are showing as an output in your question, your issue is also coming from the fact that your SnapshotCreateTime is in epoch format in liliseconds, so you just have to convert 2022-10-01 in the right format.
If you are on Linux, you can do this within your command, with date:
aws elasticache describe-snapshots \
--region ap-southeast-1 \
--snapshot-source "manual" \
--query "Snapshots[?
NodeSnapshots[?
SnapshotCreateTime > \`$(date --date='2022-10-01' +'%s')000\`
]
].SnapshotName"

Related

AWS CLI: try to write query for subnets

When listing route tables, I want to show the subnet id of all those who has more than one route, which I couldn't get it right.
$ aws ec2 describe-route-tables --region us-west-2 --query 'RouteTables[*]'
[
{
"Associations": [
{
"Main": false,
"RouteTableAssociationId": "rtbassoc-0c6d59285cc28b997",
"RouteTableId": "rtb-0d56ac20552c39cb4",
"SubnetId": "subnet-029daed7c320c9cc0",
"AssociationState": {
"State": "associated"
}
}
],
"PropagatingVgws": [],
"RouteTableId": "rtb-0d56ac20552c39cb4",
"Routes": [
{
"DestinationCidrBlock": "10.96.110.0/23",
"GatewayId": "local",
"Origin": "CreateRouteTable",
"State": "active"
},
{
"DestinationCidrBlock": "0.0.0.0/0",
"GatewayId": "igw-02fae1eda618a542d",
"Origin": "CreateRoute",
"State": "active"
}
],
I try something below but it fails.
$ aws ec2 describe-route-tables --region us-west-2 --query 'RouteTables[?Routes[].Size>1].Associations[*].SubnetId'
Bad value for --query RouteTables[?Routes[].Size > 1].Associations[*].SubnetId: invalid token: Parse error at column 29, token "1" (NUMBER), for expression:
"RouteTables[?Routes[].Size > 1].Associations[*].SubnetId"
^
I think the following should do what you are after:
RouteTables[?length(Routes[*]) > `1`].Associations[*].SubnetId

How to parse aws cli output using jq

aws elbv2 describe-target-group-attributes \
--target-group-arn arn:aws:elasticloadbalancing:us-west-2:123456789012:targetgroup/my-targets/73e2d6bc24d8a067
provides
{
"Attributes": [
{
"Value": "false",
"Key": "stickiness.enabled"
},
{
"Value": "300",
"Key": "deregistration_delay.timeout_seconds"
},
{
"Value": "lb_cookie",
"Key": "stickiness.type"
},
{
"Value": "86400",
"Key": "stickiness.lb_cookie.duration_seconds"
},
{
"Value": "0",
"Key": "slow_start.duration_seconds"
}
]
}
I would like to fetch deregistration_delay.timeout_seconds from the output
I tried which works for this case when deregistration_delay.timeout_seconds appears on the second position.
aws elbv2 describe-target-group-attributes \
--target-group-arn arn:aws:elasticloadbalancing:us-west-2:123456789012:targetgroup/my-targets/73e2d6bc24d8a067
| jq -r '.Attributes[1].Value'
but for some target groups the deregistration_delay.timeout_seconds is placed at a different number.
How can I use jq to fetch deregistration_delay.timeout_seconds
You can actually use JMESPATH in the AWS CLI without needing to use jq:
aws elbv2 describe-target-group-attributes \
--target-group-arn arn:aws:elasticloadbalancing:us-west-2:123456789012:targetgroup/my-targets/73e2d6bc24d8a067 \
--query "Attributes[?Key=='deregistration_delay.timeout_seconds']|[0].Value" \
--output text
JMESPATH was created by James Saryerwinnie, one of the authors of the AWS CLI. The tutorial is well worth reading.

Which AWS resources can be attached / related to a specific VPC? (to verify it's safe to delete it)

I'm looking for a way to understand if we are making use of a specific VPC
The easy way is to review resources 1-by-1 like:
EC2 Machines
RDS
Client-VPN-Endpoint
Other resources - What else do I need to check?
And check manually.
is there another way to determine what is relying on a specific VPC before I'll delete it?
You can do it in two ways: AWS CLI or AWS console.
AWS CLI
You can use AWS CLI to list all ENIs associated with the VPC and prettify the output using the --query parameter to get a resource list with the desired fields (AZ, instance-id, etc.).
`aws ec2 describe-network-interfaces --filters Name=vpc-id,Values=<vpc-id> --query 'NetworkInterfaces[*].[AvailabilityZone, OwnerId, Attachment.InstanceId, PrivateIpAddresses[*].Association.PublicIp]'
`aws ec2 describe-network-interfaces --filters Name=vpc-id,Values=<vpc-id> --query 'NetworkInterfaces[*].[RequesterId,Description]'
A sample of the raw output (only one instance on the VPC):
"NetworkInterfaces": [
{
"Association": {
"IpOwnerId": "amazon",
"PublicDnsName": "ec2-54-196-57-169.compute-1.amazonaws.com",
"PublicIp": "54.196.57.169"
},
"Attachment": {
"AttachTime": "2020-08-24T10:59:16+00:00",
"AttachmentId": "eni-attach-047e562690aabbffd",
"DeleteOnTermination": true,
"DeviceIndex": 0,
"InstanceId": "i-0fe495a6c17bd0f82",
"InstanceOwnerId": "570398916848",
"Status": "attached"
},
"AvailabilityZone": "us-east-1d",
"Description": "",
"Groups": [
{
"GroupName": "launch-wizard-1",
"GroupId": "sg-0aa7d8257bb487e1b"
}
],
"InterfaceType": "interface",
"Ipv6Addresses": [],
"MacAddress": "0e:58:38:33:9a:31",
"NetworkInterfaceId": "eni-0b20855178d276783",
"OwnerId": "570398916848",
"PrivateDnsName": "ip-172-31-34-30.ec2.internal",
"PrivateIpAddress": "172.31.34.30",
"PrivateIpAddresses": [
{
"Association": {
"IpOwnerId": "amazon",
"PublicDnsName": "ec2-54-196-57-169.compute-1.amazonaws.com",
"PublicIp": "54.196.57.169"
},
"Primary": true,
"PrivateDnsName": "ip-172-31-34-30.ec2.internal",
"PrivateIpAddress": "172.31.34.30"
}
],
"RequesterManaged": false,
"SourceDestCheck": true,
"Status": "in-use",
"SubnetId": "subnet-e2bc5fbd",
"TagSet": [],
"VpcId": "vpc-6ad2e110"
}
]
And now filtered:
For the first --query
[
"us-east-1d",
"57039816848",
"i-0fe495a6c17bd0f82",
[
"44.196.57.169"
]
]
And for the second --query (another VPC):
[
"amazon-elasticache",
"ElastiCache alon-001"
],
[
"amazon-elasticache",
"ElastiCache alon-002"
],
[
"975289786086",
"arn:aws:ecs:us-east-2:57039916848:attachment/22a90802-fae7-4afb-9a7e-43e6f4be8ca4"
],
[
"074689309192",
"Interface for NAT Gateway nat-069344579d8bda20"
],
[
"amazon-elb",
"ELB app/EC2Co-EcsEl-YX74WCWEGOK/0b6d7bc60b540b1"
],
[
"amazon-elb",
"ELB app/EC2Co-EcsEl-YX74WCWGGOK/0b6bd7c60b540b1"
],
[
"amazon-elasticache",
"ElastiCache alon-003"
]
AWS Console
You can do the same using the AWS console.
Under EC2->Network Interfaces, search for the desired vpc-id in the search bar.

Trying to get the route table ID in AWS cli using the tag name as a filter

How can I get the AWS route table ID using a tag name as the filter?
The tag name I want to look for is - eksctl-live-cluster/PublicRouteTable
In the below example, the end result is that I would want to get the command to return the id of "rtb-0b6d5359a281c6fd9"
Using the below command I can get all the info for all the route tables in my VPC. I have tried adding tags and names in the query part unsuccessfully and played around with --filter. I just want to get the ID for one table that uses the name "eksctl-live-cluster/PublicRouteTable".
aws ec2 describe-route-tables --filters "Name=vpc-id,Values=vpc-0a75516801dc9a130" --query "RouteTables[]"
The name I want to use in the search is - eksctl-live-cluster/PublicRouteTable
Here is the output of all the route tables when i use the first command -
[
{
"Associations": [
{
"AssociationState": {
"State": "associated"
},
"RouteTableAssociationId": "rtbassoc-07ef991c747ba58a5",
"Main": true,
"RouteTableId": "rtb-0ad0dde171cc946c9"
}
],
"RouteTableId": "rtb-0ad0dde171cc946c9",
"VpcId": "vpc-0a75516801dc9a130",
"PropagatingVgws": [],
"Tags": [],
"Routes": [
{
"GatewayId": "local",
"DestinationCidrBlock": "10.170.0.0/16",
"State": "active",
"Origin": "CreateRouteTable"
}
],
"OwnerId": "000000000"
},
{
"Associations": [
{
"SubnetId": "subnet-0e079eb96b85fc72c",
"AssociationState": {
"State": "associated"
},
"RouteTableAssociationId": "rtbassoc-062f19d9175f4f596",
"Main": false,
"RouteTableId": "rtb-0b6d5359a281c6fd9"
},
{
"SubnetId": "subnet-0b1fae931da8c9d8f",
"AssociationState": {
"State": "associated"
},
"RouteTableAssociationId": "rtbassoc-0a22d395d0b6196ac",
"Main": false,
"RouteTableId": "rtb-0b6d5359a281c6fd9"
}
],
"RouteTableId": "rtb-0b6d5359a281c6fd9",
"VpcId": "vpc-0a75516801dc9a130",
"PropagatingVgws": [],
"Tags": [
{
"Value": "live",
"Key": "eksctl.cluster.k8s.io/v1alpha1/cluster-name"
},
{
"Value": "live",
"Key": "alpha.eksctl.io/cluster-name"
},
{
"Value": "0.29.2",
"Key": "alpha.eksctl.io/eksctl-version"
},
{
"Value": "PublicRouteTable",
"Key": "aws:cloudformation:logical-id"
},
{
"Value": "eksctl-live-cluster",
"Key": "aws:cloudformation:stack-name"
},
{
"Value": "eksctl-live-cluster/PublicRouteTable",
"Key": "Name"
},
{
"Value": "arn:aws:cloudformation:us-east-1:000000000:stack/eksctl-live-cluster/ef543610-3981-11eb-abcc-0af655d000e7",
"Key": "aws:cloudformation:stack-id"
}
],
"Routes": [
{
"GatewayId": "local",
"DestinationCidrBlock": "10.170.0.0/16",
"State": "active",
"Origin": "CreateRouteTable"
},
{
"GatewayId": "igw-072414b2b1d313970",
"DestinationCidrBlock": "0.0.0.0/0",
"State": "active",
"Origin": "CreateRoute"
}
],
"OwnerId": "996762160"
}
]
This should return what you're looking for:
aws ec2 describe-route-tables --filters 'Name=tag:Name,Values=eksctl-live-cluster/PublicRouteTable' --query 'RouteTables[].Associations[].RouteTableId'
In general you can filter with tags using the tag:<tag name> construct. I'm not sure what a / value will do.
tag :key- The key/value combination of a tag assigned to the resource. Use the tag key in the filter name and the tag value as the filter value. For example, to find all resources that have a tag with the key Owner and the value TeamA , specify tag:Owner for the filter name and TeamA for the filter value.
If you want to further filter it by VPC id, you can add on to the filter like this:
aws ec2 describe-route-tables --filters 'Name=tag:Name,Values=eksctl-live-cluster/PublicRouteTable' Name=vpc-id,Values=<VPC ID> --query 'RouteTables[].Associations[].RouteTableId'
This line:
aws ec2 --profile prod --region eu-west-1 describe-route-tables --filters Name=tag:Name,Values=private-route-table-eu-west-1b --query 'RouteTables[].Associations[].RouteTableId'
Returned the following because my route table is associated with two subnets:
[
"rtb-04d4b860",
"rtb-04d4b860"
]
If you need a unique output you could pipe this all through jq :
|jq -r .[] |sort |uniq
References
describe-route-tables

AWS CLI to find out list of active reserved instances

I am trying to find out list of reserved instances that are active.
aws ec2 describe-reserved-instances --filters "Name=instance-state-State,Values=active" --query 'Reservations[].Instances[].Tags[?Key==`Name`].Value[]'
The above command does not work, and I think the Name field is not right.
Any help?
Thanks
Here's another take on it. Put the search for
State == `active`
into the query statement. The output includes labels as well. By the way, changing from single-quotes around the query to double-quotes (on a mac) will require you to escape the backticks with backslashes.
aws ec2 describe-reserved-instances --query 'ReservedInstances[?State == `active`].{Count: InstanceCount, Type: InstanceType}' --output json
Output:
[
{
"Count": 50,
"Type": "t2.medium"
},
{
"Count": 3,
"Type": "m4.2xlarge"
},
{
"Count": 17,
"Type": "m3.large"
},
{
"Count": 3,
"Type": "m3.2xlarge"
},
{
"Count": 2,
"Type": "m3.2xlarge"
},
{
"Count": 3,
"Type": "m3.xlarge"
},
{
"Count": 6,
"Type": "m4.4xlarge"
}
]
aws ec2 describe-reserved-instances --filter Name=state,Values=active
--query 'ReservedInstances[*][InstanceType,InstanceCount]'
Output:
[
[
"m3.medium",
4
],
[
"c4.large",
5
]
]