I want to use below template generated from Cloud-former tool in my another AWS account but it gives me error each time.
It is simple template with VPC, Subnet, Routetables and IGW.
URL: https://s3.amazonaws.com/elasticbeanstalk-us-east-1-459239532405/cloudformer.template
Error: The following resource(s) failed to create: [rtb50d7b237, subnet3237ac6a, gw1, subnet47f0bd31]. . Rollback requested by user.
What am i doing wrong?
Just be sure you're in us-east-1
Each account has it's own set of AZ. You can look for yours
aws ec2 describe-availability-zones --region us-east-1
result for me:
{
"AvailabilityZones": [
{
"State": "available",
"RegionName": "us-east-1",
"Messages": [],
"ZoneName": "us-east-1a"
},
{
"State": "available",
"RegionName": "us-east-1",
"Messages": [],
"ZoneName": "us-east-1b"
},
{
"State": "available",
"RegionName": "us-east-1",
"Messages": [],
"ZoneName": "us-east-1d"
},
{
"State": "available",
"RegionName": "us-east-1",
"Messages": [],
"ZoneName": "us-east-1e"
}
]
}
I would try to change us-east-1c subnets to another zone in my template
Related
When listing route tables, I want to show the subnet id of all those who has more than one route, which I couldn't get it right.
$ aws ec2 describe-route-tables --region us-west-2 --query 'RouteTables[*]'
[
{
"Associations": [
{
"Main": false,
"RouteTableAssociationId": "rtbassoc-0c6d59285cc28b997",
"RouteTableId": "rtb-0d56ac20552c39cb4",
"SubnetId": "subnet-029daed7c320c9cc0",
"AssociationState": {
"State": "associated"
}
}
],
"PropagatingVgws": [],
"RouteTableId": "rtb-0d56ac20552c39cb4",
"Routes": [
{
"DestinationCidrBlock": "10.96.110.0/23",
"GatewayId": "local",
"Origin": "CreateRouteTable",
"State": "active"
},
{
"DestinationCidrBlock": "0.0.0.0/0",
"GatewayId": "igw-02fae1eda618a542d",
"Origin": "CreateRoute",
"State": "active"
}
],
I try something below but it fails.
$ aws ec2 describe-route-tables --region us-west-2 --query 'RouteTables[?Routes[].Size>1].Associations[*].SubnetId'
Bad value for --query RouteTables[?Routes[].Size > 1].Associations[*].SubnetId: invalid token: Parse error at column 29, token "1" (NUMBER), for expression:
"RouteTables[?Routes[].Size > 1].Associations[*].SubnetId"
^
I think the following should do what you are after:
RouteTables[?length(Routes[*]) > `1`].Associations[*].SubnetId
I've a lambda in VPC to access Amazon DocDB, but failed to access any resource in VPC. I've read the official guide for days still didn't fix this issue.
I checked all vpc configurations according to Official Guide but got no luck.
VPC is assigned when creating lambda.
Could anyone give me some help on the lambda configurations ? :)
def access_mongodb(event, context):
url = event.get('url')
if url:
db = event.get('db')
coll = event.get('collection')
query = event.get('query')
limit = int(event.get('limit'))
try:
with Mongo(url=url, db=db) as conn:
logger.info('Lambda Start query with Mongo')
for row in conn[coll].find(query).limit(limit):
logger.info(f'got row => {json.dumps(row, default=str)}')
except Exception as e:
logger.error(f'Got exception {e}')
else:
logger.info('Lambda End with out Mongo')
Errors:
Got exception No servers found yet, Timeout: 2.0s, Topology Description: <TopologyDescription id: 62b5186720247fb7d69a0765, topology_type: Single, servers: [<ServerDescription ('docdb-test.xxxx-southeast-1.docdb.amazonaws.com', 27017) server_type: Unknown, rtt: None>]>
Configurations:
aws lambda get-function-configuration --function-name hello_py3
{
"FunctionName": "hello_py3",
"FunctionArn": "arn:aws:lambda:ap-southeast-1:592017647781:function:hello_py3",
"Runtime": "python3.9",
"Role": "arn:aws:iam::592017647781:role/service-role/hello_py3-role-xh39m23g",
"Handler": "lambda_function.lambda_handler",
"CodeSize": 5701329,
"Description": "",
"Timeout": 10,
"MemorySize": 128,
"LastModified": "2022-06-24T01:26:48.000+0000",
"CodeSha256": "VLwda8fP2DM62/y4Ouy9/U3KpzvfSRWoH7ocCwl1G6g=",
"Version": "$LATEST",
"VpcConfig": {
"SubnetIds": [
"subnet-08dacd9b6970624aa",
"subnet-09f80e8227735f6cf",
"subnet-028392620db2f9753"
],
"SecurityGroupIds": [
"sg-0002ee69773ca6f9d"
],
"VpcId": "vpc-0eee2636f691ad96b"
},
"TracingConfig": {
"Mode": "PassThrough"
},
"RevisionId": "55af10eb-f777-4ba9-aea5-05a010ce7637",
"State": "Active",
"LastUpdateStatus": "Successful",
"PackageType": "Zip",
"Architectures": [
"x86_64"
],
"EphemeralStorage": {
"Size": 512
}
}
aws iam list-attached-role-policies --role-name hello_py3-role-xh39m23g
{
"AttachedPolicies": [
{
"PolicyName": "AWSLambdaVPCAccessExecutionRole-2400d95b-c83c-4fce-8e12-b1a8c5c4b503",
"PolicyArn": "arn:aws:iam::592017647781:policy/service-role/AWSLambdaVPCAccessExecutionRole-2400d95b-c83c-4fce-8e12-b1a8c5c4b503"
},
{
"PolicyName": "AWSLambdaBasicExecutionRole-a8dac45b-b9f1-4eab-8170-2c9b9f9358ce",
"PolicyArn": "arn:aws:iam::592017647781:policy/service-role/AWSLambdaBasicExecutionRole-a8dac45b-b9f1-4eab-8170-2c9b9f9358ce"
}
]
}
aws ec2 describe-vpcs --vpc-ids vpc-0eee2636f691ad96b
{
"Vpcs": [
{
"CidrBlock": "172.31.0.0/16",
"DhcpOptionsId": "dopt-0b9edd5b6deafa0db",
"State": "available",
"VpcId": "vpc-0eee2636f691ad96b",
"OwnerId": "592017647781",
"InstanceTenancy": "default",
"CidrBlockAssociationSet": [
{
"AssociationId": "vpc-cidr-assoc-0200675b36f061104",
"CidrBlock": "172.31.0.0/16",
"CidrBlockState": {
"State": "associated"
}
}
],
"IsDefault": true
}
]
}
aws ec2 describe-security-groups --group-ids sg-0002ee69773ca6f9d
{
"SecurityGroups": [
{
"Description": "default VPC security group",
"GroupName": "default",
"IpPermissions": [
{
"FromPort": 80,
"IpProtocol": "tcp",
"IpRanges": [
{
"CidrIp": "0.0.0.0/0"
}
],
"Ipv6Ranges": [],
"PrefixListIds": [],
"ToPort": 80,
"UserIdGroupPairs": []
},
{
"IpProtocol": "-1",
"IpRanges": [],
"Ipv6Ranges": [],
"PrefixListIds": [],
"UserIdGroupPairs": [
{
"GroupId": "sg-0047473f289f0ffd3",
"UserId": "592017647781"
},
{
"GroupId": "sg-031e0901b061eb92d",
"UserId": "592017647781"
},
{
"GroupId": "sg-03f39f48c7887e46b",
"UserId": "592017647781"
},
{
"GroupId": "sg-07d8dbe45e3e81e44",
"UserId": "592017647781"
}
]
}
],
"OwnerId": "592017647781",
"GroupId": "sg-0002ee69773ca6f9d",
"IpPermissionsEgress": [
{
"IpProtocol": "-1",
"IpRanges": [
{
"CidrIp": "0.0.0.0/0"
}
],
"Ipv6Ranges": [],
"PrefixListIds": [],
"UserIdGroupPairs": []
}
],
"VpcId": "vpc-0eee2636f691ad96b"
}
]
}
UPDATE:
I finally figure it out by applying ReachabilityAnalyzer, and it was proved to be my fault on confusing configuration items. This is a very helpful tool, guys have same issue can try to use this tool to help themselves out.
Thanks John for help.
You appear to be using a single Security Group for both the AWS Lambda function and the DocDB database. I think your Security Group is missing Outbound permissions, which be restricting traffic from the Lambda function.
The typical security setup would be:
A security group on the AWS Lambda function (Lambda-SG) that permits all Outbound access
A security group on the DocDB (DB-SG) that permits Inbound access from Lambda-SG on port 27017
Could you please check the connection from the lambda subnet to documentdb subnet and there sg & nacl just to confirm that lambda can connect to the documentdb using port 27017.
Thanks,
Chinmoy Layek
I want to terminate my AWS Elastic Beanstalk environment, but I would like to keep a snapshot of the RDS database instance associated with the environment.
I have successfully created a snapshot of the concerned RDS database, but when I go to terminate the Elastic Beanstalk environment, it says that doing so will:
Terminate database-in-question with snapshot.
Does this mean that the snapshot I created will be deleted as well as the database instance? If so, how can I avoid the snapshot being deleted?
Elastic Beanstalk uses CloudFormation in the background to provision your environment. CloudFormation works so that it has a stack that contains all the resources that it has created, and once you delete your Beanstalk environment, said stack is removed. Thus, it will remove all the resources which are part of your stack. You can go to CloudFormation in console, and check it out.
If you created a DB Snapshot manually, it will be outside of the CloudFormation stack, and thus it won't be removed.
However, as part of Beanstalk environment setup, there will be some automated backups of your DB. These are removed, when you terminate your environment.
Here's my test environment stack:
$ aws cloudformation describe-stack-resources --stack-name awseb-e-jjqgv3nwgp-stack --profile=personal
{
"StackResources": [
[...],
{
"StackName": "awseb-e-jjqgv3nwgp-stack",
"StackId": "arn:aws:cloudformation:eu-central-1:[my_account]:stack/awseb-e-jjqgv3nwgp-stack/f21c2e00-ea6d-11eb-9f61-02ad9e7e97f6",
"LogicalResourceId": "AWSEBRDSDBSecurityGroup",
"PhysicalResourceId": "awseb-e-jjqgv3nwgp-stack-AWSEBRDSDBSecurityGroup-1SDABJ60VF7G8",
"ResourceType": "AWS::EC2::SecurityGroup",
"Timestamp": "2021-07-21T21:52:52.931000+00:00",
"ResourceStatus": "CREATE_COMPLETE",
"DriftInformation": {
"StackResourceDriftStatus": "NOT_CHECKED"
}
},
{
"StackName": "awseb-e-jjqgv3nwgp-stack",
"StackId": "arn:aws:cloudformation:eu-central-1:[my_account]:stack/awseb-e-jjqgv3nwgp-stack/f21c2e00-ea6d-11eb-9f61-02ad9e7e97f6",
"LogicalResourceId": "AWSEBRDSDatabase",
"PhysicalResourceId": "aa1v9kyuepq8x1c",
"ResourceType": "AWS::RDS::DBInstance",
"Timestamp": "2021-07-21T21:59:24.817000+00:00",
"ResourceStatus": "CREATE_COMPLETE",
"DriftInformation": {
"StackResourceDriftStatus": "NOT_CHECKED"
}
},
[...]
]
}
Here are my RDS snapshots:
$ aws rds describe-db-snapshots --profile=personal
{
"DBSnapshots": [
{
"DBSnapshotIdentifier": "foobar-snapshot-test",
"DBInstanceIdentifier": "aa1v9kyuepq8x1c",
"SnapshotCreateTime": "2021-07-21T22:09:03.752000+00:00",
"Engine": "mysql",
"AllocatedStorage": 5,
"Status": "available",
"Port": 3306,
"AvailabilityZone": "eu-central-1a",
"VpcId": "vpc-128d5178",
"InstanceCreateTime": "2021-07-21T21:56:51.205000+00:00",
"MasterUsername": "foo",
"EngineVersion": "8.0.23",
"LicenseModel": "general-public-license",
"SnapshotType": "manual",
"OptionGroupName": "default:mysql-8-0",
"PercentProgress": 100,
"StorageType": "standard",
"Encrypted": false,
"DBSnapshotArn": "arn:aws:rds:eu-central-1:[my_account]:snapshot:foobar-snapshot-test",
"IAMDatabaseAuthenticationEnabled": false,
"ProcessorFeatures": [],
"DbiResourceId": "db-PFQFQBRFBELDWUQTONIPRJRVCU",
"TagList": []
},
{
"DBSnapshotIdentifier": "rds:aa1v9kyuepq8x1c-2021-07-21-21-57",
"DBInstanceIdentifier": "aa1v9kyuepq8x1c",
"SnapshotCreateTime": "2021-07-21T21:57:05.277000+00:00",
"Engine": "mysql",
"AllocatedStorage": 5,
"Status": "available",
"Port": 3306,
"AvailabilityZone": "eu-central-1a",
"VpcId": "vpc-128d5178",
"InstanceCreateTime": "2021-07-21T21:56:51.205000+00:00",
"MasterUsername": "foo",
"EngineVersion": "8.0.23",
"LicenseModel": "general-public-license",
"SnapshotType": "automated",
"OptionGroupName": "default:mysql-8-0",
"PercentProgress": 100,
"StorageType": "standard",
"Encrypted": false,
"DBSnapshotArn": "arn:aws:rds:eu-central-1:[my_account]:snapshot:rds:aa1v9kyuepq8x1c-2021-07-21-21-57",
"IAMDatabaseAuthenticationEnabled": false,
"ProcessorFeatures": [],
"DbiResourceId": "db-PFQFQBRFBELDWUQTONIPRJRVCU",
"TagList": []
}
]
}
So, we have one automated snapshot, and one manual.
After deleting the application, the CloudFormation stack was removed, and what's left is the manual snapshot, but the automated one has been deleted:
$ aws rds describe-db-snapshots --profile=personal
{
"DBSnapshots": [
{
"DBSnapshotIdentifier": "foobar-snapshot-test",
"DBInstanceIdentifier": "aa1v9kyuepq8x1c",
"SnapshotCreateTime": "2021-07-21T22:09:03.752000+00:00",
"Engine": "mysql",
"AllocatedStorage": 5,
"Status": "available",
"Port": 3306,
"AvailabilityZone": "eu-central-1a",
"VpcId": "vpc-128d5178",
"InstanceCreateTime": "2021-07-21T21:56:51.205000+00:00",
"MasterUsername": "foo",
"EngineVersion": "8.0.23",
"LicenseModel": "general-public-license",
"SnapshotType": "manual",
"OptionGroupName": "default:mysql-8-0",
"PercentProgress": 100,
"StorageType": "standard",
"Encrypted": false,
"DBSnapshotArn": "arn:aws:rds:eu-central-1:[my_account]:snapshot:foobar-snapshot-test",
"IAMDatabaseAuthenticationEnabled": false,
"ProcessorFeatures": [],
"DbiResourceId": "db-PFQFQBRFBELDWUQTONIPRJRVCU",
"TagList": []
}
]
}
You can always double check your RDS snapshots to be sure you have a manual one there, before terminating anything on Beanstalk.
I'm looking for a way to understand if we are making use of a specific VPC
The easy way is to review resources 1-by-1 like:
EC2 Machines
RDS
Client-VPN-Endpoint
Other resources - What else do I need to check?
And check manually.
is there another way to determine what is relying on a specific VPC before I'll delete it?
You can do it in two ways: AWS CLI or AWS console.
AWS CLI
You can use AWS CLI to list all ENIs associated with the VPC and prettify the output using the --query parameter to get a resource list with the desired fields (AZ, instance-id, etc.).
`aws ec2 describe-network-interfaces --filters Name=vpc-id,Values=<vpc-id> --query 'NetworkInterfaces[*].[AvailabilityZone, OwnerId, Attachment.InstanceId, PrivateIpAddresses[*].Association.PublicIp]'
`aws ec2 describe-network-interfaces --filters Name=vpc-id,Values=<vpc-id> --query 'NetworkInterfaces[*].[RequesterId,Description]'
A sample of the raw output (only one instance on the VPC):
"NetworkInterfaces": [
{
"Association": {
"IpOwnerId": "amazon",
"PublicDnsName": "ec2-54-196-57-169.compute-1.amazonaws.com",
"PublicIp": "54.196.57.169"
},
"Attachment": {
"AttachTime": "2020-08-24T10:59:16+00:00",
"AttachmentId": "eni-attach-047e562690aabbffd",
"DeleteOnTermination": true,
"DeviceIndex": 0,
"InstanceId": "i-0fe495a6c17bd0f82",
"InstanceOwnerId": "570398916848",
"Status": "attached"
},
"AvailabilityZone": "us-east-1d",
"Description": "",
"Groups": [
{
"GroupName": "launch-wizard-1",
"GroupId": "sg-0aa7d8257bb487e1b"
}
],
"InterfaceType": "interface",
"Ipv6Addresses": [],
"MacAddress": "0e:58:38:33:9a:31",
"NetworkInterfaceId": "eni-0b20855178d276783",
"OwnerId": "570398916848",
"PrivateDnsName": "ip-172-31-34-30.ec2.internal",
"PrivateIpAddress": "172.31.34.30",
"PrivateIpAddresses": [
{
"Association": {
"IpOwnerId": "amazon",
"PublicDnsName": "ec2-54-196-57-169.compute-1.amazonaws.com",
"PublicIp": "54.196.57.169"
},
"Primary": true,
"PrivateDnsName": "ip-172-31-34-30.ec2.internal",
"PrivateIpAddress": "172.31.34.30"
}
],
"RequesterManaged": false,
"SourceDestCheck": true,
"Status": "in-use",
"SubnetId": "subnet-e2bc5fbd",
"TagSet": [],
"VpcId": "vpc-6ad2e110"
}
]
And now filtered:
For the first --query
[
"us-east-1d",
"57039816848",
"i-0fe495a6c17bd0f82",
[
"44.196.57.169"
]
]
And for the second --query (another VPC):
[
"amazon-elasticache",
"ElastiCache alon-001"
],
[
"amazon-elasticache",
"ElastiCache alon-002"
],
[
"975289786086",
"arn:aws:ecs:us-east-2:57039916848:attachment/22a90802-fae7-4afb-9a7e-43e6f4be8ca4"
],
[
"074689309192",
"Interface for NAT Gateway nat-069344579d8bda20"
],
[
"amazon-elb",
"ELB app/EC2Co-EcsEl-YX74WCWEGOK/0b6d7bc60b540b1"
],
[
"amazon-elb",
"ELB app/EC2Co-EcsEl-YX74WCWGGOK/0b6bd7c60b540b1"
],
[
"amazon-elasticache",
"ElastiCache alon-003"
]
AWS Console
You can do the same using the AWS console.
Under EC2->Network Interfaces, search for the desired vpc-id in the search bar.
I am trying to get the Availability Zone ID out of either the AWS CLI or from boto3. However, despite the documentation showing it, the command only returns the AZ, not the id for the AZ. Am I missing a step or is this just bad documentation, etc?
aws ec2 describe-subnets --region us-east-1
{
"VpcId": "vpc-054c741523f481755",
"CidrBlock": "10.150.3.32/27",
"MapPublicIpOnLaunch": false,
"State": "available",
"Ipv6CidrBlockAssociationSet": [],
"AssignIpv6AddressOnCreation": false,
"SubnetId": "subnet-0a36ed4643fb511d1",
"AvailabilityZone": "us-east-1a",
"DefaultForAz": false,
"AvailableIpAddressCount": 27,
"Tags": [
{
"Key": "aws:cloudformation:stack-id",
"Value": "arn:aws:cloudformation:us-east-1:186940489315:stack/dantooine-a-elastic-subnets/dc3f7500-7b39-11ea-a67d-0e763951b664"
},
{
"Key": "aws:cloudformation:stack-name",
"Value": "dantooine-a-elastic-subnets"
},
{
"Key": "Name",
"Value": "dantooine-a-elastic-subnets-endpointSubnet"
},
{
"Key": "aws:cloudformation:logical-id",
"Value": "endpointSubnet"
}
]
}
The documentation shows:
{
"Subnets": [
{
"AvailabilityZone": "us-east-2c",
"AvailabilityZoneId": "use2-az3",
"AvailableIpAddressCount": 251,
"CidrBlock": "10.0.2.0/24",
"DefaultForAz": false,
"MapPublicIpOnLaunch": false,
"State": "available",
"SubnetId": "subnet-0bb1c79de3EXAMPLE",
"VpcId": "vpc-0ee975135dEXAMPLE",
"OwnerId": "111122223333",
"AssignIpv6AddressOnCreation": false,
"Ipv6CidrBlockAssociationSet": [],
"SubnetArn": "arn:aws:ec2:us-east-2:111122223333:subnet/subnet-0bb1c79de3EXAMPLE"
},
If you wish to view the Availability Zone IDs, use:
aws ec2 describe-availability-zones --region us-east-1
It will output:
{
"AvailabilityZones": [
{
"State": "available",
"OptInStatus": "opt-in-not-required",
"Messages": [],
"RegionName": "us-east-1",
"ZoneName": "us-east-1a",
"ZoneId": "use1-az1",
"GroupName": "us-east-1",
"NetworkBorderGroup": "us-east-1"
},
...
You can then map this information to any subnets you have created.
This works fine for me with both the awscli and boto3. For example:
import boto3
client = boto3.client('ec2')
subnets = client.describe_subnets()
for subnet in subnets['Subnets']:
print(subnet['AvailabilityZone'], subnet['AvailabilityZoneId'])
Output is:
us-east-1b use1-az2
us-east-1e use1-az3
us-east-1d use1-az6
...
I think your installation of awscli and boto3 may be out of date.
Here is an example for boto3 in Python:
import json
import boto3
def lambda_handler(event, context):
ec2 = boto3.client('ec2', region_name="us-east-1")
azs = ec2.describe_availability_zones()["AvailabilityZones"]
for az in azs:
print (az['ZoneName'], az['ZoneId'])
This is the output:
us-east-1a use1-az4
us-east-1b use1-az6
us-east-1c use1-az1
us-east-1d use1-az2
us-east-1e use1-az3
us-east-1f use1-az5