When listing route tables, I want to show the subnet id of all those who has more than one route, which I couldn't get it right.
$ aws ec2 describe-route-tables --region us-west-2 --query 'RouteTables[*]'
[
{
"Associations": [
{
"Main": false,
"RouteTableAssociationId": "rtbassoc-0c6d59285cc28b997",
"RouteTableId": "rtb-0d56ac20552c39cb4",
"SubnetId": "subnet-029daed7c320c9cc0",
"AssociationState": {
"State": "associated"
}
}
],
"PropagatingVgws": [],
"RouteTableId": "rtb-0d56ac20552c39cb4",
"Routes": [
{
"DestinationCidrBlock": "10.96.110.0/23",
"GatewayId": "local",
"Origin": "CreateRouteTable",
"State": "active"
},
{
"DestinationCidrBlock": "0.0.0.0/0",
"GatewayId": "igw-02fae1eda618a542d",
"Origin": "CreateRoute",
"State": "active"
}
],
I try something below but it fails.
$ aws ec2 describe-route-tables --region us-west-2 --query 'RouteTables[?Routes[].Size>1].Associations[*].SubnetId'
Bad value for --query RouteTables[?Routes[].Size > 1].Associations[*].SubnetId: invalid token: Parse error at column 29, token "1" (NUMBER), for expression:
"RouteTables[?Routes[].Size > 1].Associations[*].SubnetId"
^
I think the following should do what you are after:
RouteTables[?length(Routes[*]) > `1`].Associations[*].SubnetId
Related
I am trying to write a query in AWS CLI which will provide with the elasticache snapshots names older than a specific creation date.
I tried with a JMESPath query like:
aws elasticache describe-snapshots \
--region ap-southeast-1 \
--snapshot-source "manual" \
--query 'Snapshots[*].NodeSnapshots[?SnapshotCreateTime >`2022-10-01`] | [?not_null(node)]'
But, this is giving me an empty result.
Snippet of aws elasticache describe-snapshots:
{
"Snapshots": [{
"SnapshotName": "snapshot-name",
"ReplicationGroupId": "rep-id",
"ReplicationGroupDescription": "redis cluster",
"CacheClusterId": null,
"SnapshotStatus": "available",
"SnapshotSource": "automated",
"CacheNodeType": "cache.r6g.large",
"Engine": "redis",
"EngineVersion": "6.0.5",
"NumCacheNodes": null,
"PreferredAvailabilityZone": null,
"CacheClusterCreateTime": null,
"PreferredMaintenanceWindow": "sun:20:00-sun:20:00",
"TopicArn": null,
"Port": "6379",
"CacheParameterGroupName": "default.redis6.x.cluster.on",
"CacheSubnetGroupName": "redis-group",
"VpcId": "vpc-01bcajghfghj",
"AutoMinorVersionUpgrade": "true",
"SnapshotRetentionLimit": "18",
"SnapshotWindow": "20:00-21:00",
"NumNodeGroups": "1",
"AutomaticFailover": "enabled",
"NodeSnapshots": [{
"CacheClusterId": "redis-cluster-01",
"NodeGroupId": "001",
"CacheNodeId": "001",
"NodeGroupConfiguration": null,
"CacheSize": "20 GB",
"CacheNodeCreateTime": "1632909889675",
"SnapshotCreateTime": "1667246439000"
}],
"KmsKeyId": "kms-id.."
}]
}
If we take as an example the JSON given in the documentation:
{
"Snapshots": [
{
"SnapshotName": "automatic.my-cluster2-002-2019-12-05-06-38",
"NodeSnapshots": [
{
"CacheNodeId": "0001",
"SnapshotCreateTime": "2019-12-05T06:38:23Z"
}
]
},
{
"SnapshotName": "myreplica-backup",
"NodeSnapshots": [
{
"CacheNodeId": "0001",
"SnapshotCreateTime": "2019-11-26T00:25:01Z"
}
]
},
{
"SnapshotName": "my-cluster",
"NodeSnapshots": [
{
"CacheNodeId": "0001",
"SnapshotCreateTime": "2019-11-26T03:08:33Z"
}
]
}
]
}
Then, you can realise that you need to filter on SnapshotCreateTime, nested under the array NodeSnapshots.
So, what you need, here, is a double filter:
One to filter by date:
[?SnapshotCreateTime > `2022-10-01`]
Then, one to exclude all snapshots where the NodeSnapshots array have been emptied by the previous filter:
[?NodeSnapshots[?SnapshotCreateTime > `2022-10-01`]]
And, so, if you only care about the name of the snapshot, you could do with the query:
Snapshots[?NodeSnapshots[?SnapshotCreateTime > `2022-10-01`]].SnapshotName
So, your command ends up being:
aws elasticache describe-snapshots \
--region ap-southeast-1 \
--snapshot-source "manual" \
--query 'Snapshots[?
NodeSnapshots[?SnapshotCreateTime > `2022-10-01`]
].SnapshotName'
Now, with what you are showing as an output in your question, your issue is also coming from the fact that your SnapshotCreateTime is in epoch format in liliseconds, so you just have to convert 2022-10-01 in the right format.
If you are on Linux, you can do this within your command, with date:
aws elasticache describe-snapshots \
--region ap-southeast-1 \
--snapshot-source "manual" \
--query "Snapshots[?
NodeSnapshots[?
SnapshotCreateTime > \`$(date --date='2022-10-01' +'%s')000\`
]
].SnapshotName"
I'm looking for a way to understand if we are making use of a specific VPC
The easy way is to review resources 1-by-1 like:
EC2 Machines
RDS
Client-VPN-Endpoint
Other resources - What else do I need to check?
And check manually.
is there another way to determine what is relying on a specific VPC before I'll delete it?
You can do it in two ways: AWS CLI or AWS console.
AWS CLI
You can use AWS CLI to list all ENIs associated with the VPC and prettify the output using the --query parameter to get a resource list with the desired fields (AZ, instance-id, etc.).
`aws ec2 describe-network-interfaces --filters Name=vpc-id,Values=<vpc-id> --query 'NetworkInterfaces[*].[AvailabilityZone, OwnerId, Attachment.InstanceId, PrivateIpAddresses[*].Association.PublicIp]'
`aws ec2 describe-network-interfaces --filters Name=vpc-id,Values=<vpc-id> --query 'NetworkInterfaces[*].[RequesterId,Description]'
A sample of the raw output (only one instance on the VPC):
"NetworkInterfaces": [
{
"Association": {
"IpOwnerId": "amazon",
"PublicDnsName": "ec2-54-196-57-169.compute-1.amazonaws.com",
"PublicIp": "54.196.57.169"
},
"Attachment": {
"AttachTime": "2020-08-24T10:59:16+00:00",
"AttachmentId": "eni-attach-047e562690aabbffd",
"DeleteOnTermination": true,
"DeviceIndex": 0,
"InstanceId": "i-0fe495a6c17bd0f82",
"InstanceOwnerId": "570398916848",
"Status": "attached"
},
"AvailabilityZone": "us-east-1d",
"Description": "",
"Groups": [
{
"GroupName": "launch-wizard-1",
"GroupId": "sg-0aa7d8257bb487e1b"
}
],
"InterfaceType": "interface",
"Ipv6Addresses": [],
"MacAddress": "0e:58:38:33:9a:31",
"NetworkInterfaceId": "eni-0b20855178d276783",
"OwnerId": "570398916848",
"PrivateDnsName": "ip-172-31-34-30.ec2.internal",
"PrivateIpAddress": "172.31.34.30",
"PrivateIpAddresses": [
{
"Association": {
"IpOwnerId": "amazon",
"PublicDnsName": "ec2-54-196-57-169.compute-1.amazonaws.com",
"PublicIp": "54.196.57.169"
},
"Primary": true,
"PrivateDnsName": "ip-172-31-34-30.ec2.internal",
"PrivateIpAddress": "172.31.34.30"
}
],
"RequesterManaged": false,
"SourceDestCheck": true,
"Status": "in-use",
"SubnetId": "subnet-e2bc5fbd",
"TagSet": [],
"VpcId": "vpc-6ad2e110"
}
]
And now filtered:
For the first --query
[
"us-east-1d",
"57039816848",
"i-0fe495a6c17bd0f82",
[
"44.196.57.169"
]
]
And for the second --query (another VPC):
[
"amazon-elasticache",
"ElastiCache alon-001"
],
[
"amazon-elasticache",
"ElastiCache alon-002"
],
[
"975289786086",
"arn:aws:ecs:us-east-2:57039916848:attachment/22a90802-fae7-4afb-9a7e-43e6f4be8ca4"
],
[
"074689309192",
"Interface for NAT Gateway nat-069344579d8bda20"
],
[
"amazon-elb",
"ELB app/EC2Co-EcsEl-YX74WCWEGOK/0b6d7bc60b540b1"
],
[
"amazon-elb",
"ELB app/EC2Co-EcsEl-YX74WCWGGOK/0b6bd7c60b540b1"
],
[
"amazon-elasticache",
"ElastiCache alon-003"
]
AWS Console
You can do the same using the AWS console.
Under EC2->Network Interfaces, search for the desired vpc-id in the search bar.
How can I get the AWS route table ID using a tag name as the filter?
The tag name I want to look for is - eksctl-live-cluster/PublicRouteTable
In the below example, the end result is that I would want to get the command to return the id of "rtb-0b6d5359a281c6fd9"
Using the below command I can get all the info for all the route tables in my VPC. I have tried adding tags and names in the query part unsuccessfully and played around with --filter. I just want to get the ID for one table that uses the name "eksctl-live-cluster/PublicRouteTable".
aws ec2 describe-route-tables --filters "Name=vpc-id,Values=vpc-0a75516801dc9a130" --query "RouteTables[]"
The name I want to use in the search is - eksctl-live-cluster/PublicRouteTable
Here is the output of all the route tables when i use the first command -
[
{
"Associations": [
{
"AssociationState": {
"State": "associated"
},
"RouteTableAssociationId": "rtbassoc-07ef991c747ba58a5",
"Main": true,
"RouteTableId": "rtb-0ad0dde171cc946c9"
}
],
"RouteTableId": "rtb-0ad0dde171cc946c9",
"VpcId": "vpc-0a75516801dc9a130",
"PropagatingVgws": [],
"Tags": [],
"Routes": [
{
"GatewayId": "local",
"DestinationCidrBlock": "10.170.0.0/16",
"State": "active",
"Origin": "CreateRouteTable"
}
],
"OwnerId": "000000000"
},
{
"Associations": [
{
"SubnetId": "subnet-0e079eb96b85fc72c",
"AssociationState": {
"State": "associated"
},
"RouteTableAssociationId": "rtbassoc-062f19d9175f4f596",
"Main": false,
"RouteTableId": "rtb-0b6d5359a281c6fd9"
},
{
"SubnetId": "subnet-0b1fae931da8c9d8f",
"AssociationState": {
"State": "associated"
},
"RouteTableAssociationId": "rtbassoc-0a22d395d0b6196ac",
"Main": false,
"RouteTableId": "rtb-0b6d5359a281c6fd9"
}
],
"RouteTableId": "rtb-0b6d5359a281c6fd9",
"VpcId": "vpc-0a75516801dc9a130",
"PropagatingVgws": [],
"Tags": [
{
"Value": "live",
"Key": "eksctl.cluster.k8s.io/v1alpha1/cluster-name"
},
{
"Value": "live",
"Key": "alpha.eksctl.io/cluster-name"
},
{
"Value": "0.29.2",
"Key": "alpha.eksctl.io/eksctl-version"
},
{
"Value": "PublicRouteTable",
"Key": "aws:cloudformation:logical-id"
},
{
"Value": "eksctl-live-cluster",
"Key": "aws:cloudformation:stack-name"
},
{
"Value": "eksctl-live-cluster/PublicRouteTable",
"Key": "Name"
},
{
"Value": "arn:aws:cloudformation:us-east-1:000000000:stack/eksctl-live-cluster/ef543610-3981-11eb-abcc-0af655d000e7",
"Key": "aws:cloudformation:stack-id"
}
],
"Routes": [
{
"GatewayId": "local",
"DestinationCidrBlock": "10.170.0.0/16",
"State": "active",
"Origin": "CreateRouteTable"
},
{
"GatewayId": "igw-072414b2b1d313970",
"DestinationCidrBlock": "0.0.0.0/0",
"State": "active",
"Origin": "CreateRoute"
}
],
"OwnerId": "996762160"
}
]
This should return what you're looking for:
aws ec2 describe-route-tables --filters 'Name=tag:Name,Values=eksctl-live-cluster/PublicRouteTable' --query 'RouteTables[].Associations[].RouteTableId'
In general you can filter with tags using the tag:<tag name> construct. I'm not sure what a / value will do.
tag :key- The key/value combination of a tag assigned to the resource. Use the tag key in the filter name and the tag value as the filter value. For example, to find all resources that have a tag with the key Owner and the value TeamA , specify tag:Owner for the filter name and TeamA for the filter value.
If you want to further filter it by VPC id, you can add on to the filter like this:
aws ec2 describe-route-tables --filters 'Name=tag:Name,Values=eksctl-live-cluster/PublicRouteTable' Name=vpc-id,Values=<VPC ID> --query 'RouteTables[].Associations[].RouteTableId'
This line:
aws ec2 --profile prod --region eu-west-1 describe-route-tables --filters Name=tag:Name,Values=private-route-table-eu-west-1b --query 'RouteTables[].Associations[].RouteTableId'
Returned the following because my route table is associated with two subnets:
[
"rtb-04d4b860",
"rtb-04d4b860"
]
If you need a unique output you could pipe this all through jq :
|jq -r .[] |sort |uniq
References
describe-route-tables
I am trying to get the Availability Zone ID out of either the AWS CLI or from boto3. However, despite the documentation showing it, the command only returns the AZ, not the id for the AZ. Am I missing a step or is this just bad documentation, etc?
aws ec2 describe-subnets --region us-east-1
{
"VpcId": "vpc-054c741523f481755",
"CidrBlock": "10.150.3.32/27",
"MapPublicIpOnLaunch": false,
"State": "available",
"Ipv6CidrBlockAssociationSet": [],
"AssignIpv6AddressOnCreation": false,
"SubnetId": "subnet-0a36ed4643fb511d1",
"AvailabilityZone": "us-east-1a",
"DefaultForAz": false,
"AvailableIpAddressCount": 27,
"Tags": [
{
"Key": "aws:cloudformation:stack-id",
"Value": "arn:aws:cloudformation:us-east-1:186940489315:stack/dantooine-a-elastic-subnets/dc3f7500-7b39-11ea-a67d-0e763951b664"
},
{
"Key": "aws:cloudformation:stack-name",
"Value": "dantooine-a-elastic-subnets"
},
{
"Key": "Name",
"Value": "dantooine-a-elastic-subnets-endpointSubnet"
},
{
"Key": "aws:cloudformation:logical-id",
"Value": "endpointSubnet"
}
]
}
The documentation shows:
{
"Subnets": [
{
"AvailabilityZone": "us-east-2c",
"AvailabilityZoneId": "use2-az3",
"AvailableIpAddressCount": 251,
"CidrBlock": "10.0.2.0/24",
"DefaultForAz": false,
"MapPublicIpOnLaunch": false,
"State": "available",
"SubnetId": "subnet-0bb1c79de3EXAMPLE",
"VpcId": "vpc-0ee975135dEXAMPLE",
"OwnerId": "111122223333",
"AssignIpv6AddressOnCreation": false,
"Ipv6CidrBlockAssociationSet": [],
"SubnetArn": "arn:aws:ec2:us-east-2:111122223333:subnet/subnet-0bb1c79de3EXAMPLE"
},
If you wish to view the Availability Zone IDs, use:
aws ec2 describe-availability-zones --region us-east-1
It will output:
{
"AvailabilityZones": [
{
"State": "available",
"OptInStatus": "opt-in-not-required",
"Messages": [],
"RegionName": "us-east-1",
"ZoneName": "us-east-1a",
"ZoneId": "use1-az1",
"GroupName": "us-east-1",
"NetworkBorderGroup": "us-east-1"
},
...
You can then map this information to any subnets you have created.
This works fine for me with both the awscli and boto3. For example:
import boto3
client = boto3.client('ec2')
subnets = client.describe_subnets()
for subnet in subnets['Subnets']:
print(subnet['AvailabilityZone'], subnet['AvailabilityZoneId'])
Output is:
us-east-1b use1-az2
us-east-1e use1-az3
us-east-1d use1-az6
...
I think your installation of awscli and boto3 may be out of date.
Here is an example for boto3 in Python:
import json
import boto3
def lambda_handler(event, context):
ec2 = boto3.client('ec2', region_name="us-east-1")
azs = ec2.describe_availability_zones()["AvailabilityZones"]
for az in azs:
print (az['ZoneName'], az['ZoneId'])
This is the output:
us-east-1a use1-az4
us-east-1b use1-az6
us-east-1c use1-az1
us-east-1d use1-az2
us-east-1e use1-az3
us-east-1f use1-az5
I followed the tutorial to create a VPC link to my private elb balancer.
https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-api-with-vpclink-cli.html
But it failed, and got an error message "statusMessage": "NLB ARN is malformed".
I do find the ELB with same ARN by elbv2 cli, so the ARN must be a legal one...
I can't find document to solve the problem.
anyone can help me? thank you.
what i did is as following.
$ aws elbv2 describe-load-balancers --load-balancer-arns arn:aws:elasticloadbalancing:ap-northeast-1:846239845603:loadbalancer/app/v2-api-balancer/db49ab0ecaef1de8
{
"LoadBalancers": [
{
"Scheme": "internal",
"SecurityGroups": [
"sg-9282b8f4"
],
"LoadBalancerArn": "arn:aws:elasticloadbalancing:ap-northeast-1:846239845603:loadbalancer/app/v2-api-balancer/db49ab0ecaef1de8",
"State": {
"Code": "active"
},
"CreatedTime": "2017-10-18T04:27:28.780Z",
"VpcId": "vpc-dbe3f2be",
"DNSName": "internal-v2-api-balancer-988454399.ap-northeast-1.elb.amazonaws.com",
"AvailabilityZones": [
{
"SubnetId": "subnet-7642062e",
"ZoneName": "ap-northeast-1c"
},
{
"SubnetId": "subnet-c454fa8d",
"ZoneName": "ap-northeast-1b"
}
],
"IpAddressType": "ipv4",
"Type": "application",
"LoadBalancerName": "v2-api-balancer",
"CanonicalHostedZoneId": "Z14GRHDCWA56QT"
}
]
}
$ aws apigateway create-vpc-link \
--name my-test-vpc-link-1 \
--target-arns "arn:aws:elasticloadbalancing:ap-northeast-1:846239845603:loadbalancer/app/v2-api-balancer/db49ab0ecaef1de8"
{
"name": "my-test-vpc-link-1",
"targetArns": [
"arn:aws:elasticloadbalancing:ap-northeast-1:846239845603:loadbalancer/app/v2-api-balancer/db49ab0ecaef1de8"
],
"id": "7eexgn",
"status": "PENDING"
}
$ aws apigateway get-vpc-link --vpc-link-id 7eexgn
{
"id": "7eexgn",
"targetArns": [
"arn:aws:elasticloadbalancing:ap-northeast-1:846239845603:loadbalancer/app/v2-api-balancer/db49ab0ecaef1de8"
],
"status": "FAILED",
"name": "my-test-vpc-link-1",
"statusMessage": "NLB ARN is malformed"
}
VPC Links must be to a network LB. Looks like you are trying to use an application LB.
https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-nlb-for-vpclink-using-console.html