AWS Lambda in VPC sometimes doesn't have internet access - amazon-web-services

I have Lambda which was deployed to VPC.
This deploymens has next configs:
VPC (192.168.0.0/16)
Public Subnet A (192.168.32.0/20) has NAT Gateway and Route 0.0.0.0/0 to Internet Gateway
Private Subnet A (192.168.48.0/20) has Route 0.0.0.0/0 to NAT Gateway
Private Subnet B (192.168.64.0/20)
Lambda has own Securiy Group and references to "Private Subnet A" and "Private Subnet B"
I have strange problem: time to time Lambda doesn't have Internet Access. 3rd party service works normal.
One more strange thing that Lambda gets IP's like 127.0.0.1, 169.254.76.13, 169.254.79.1 instead of IP's from Subnets (192.168.48.0/20 and 192.168.64.0/20).
Error:
Error: connect ETIMEDOUT x.x.x.x:443
at Object._errnoException (util.js:1022:11)
at _exceptionWithHostPort (util.js:1044:20)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1198:14)
Deployment schema:
Here full CloudFormation template:
---
AWSTemplateFormatVersion: '2010-09-09'
Description: 'Base Infrastructure'
Metadata:
'AWS::CloudFormation::Interface':
ParameterGroups:
- Label:
default: 'VPC Parameters'
Parameters:
- VpcId
- InternetGatewayId
Parameters:
VpcId:
Type: String
InternetGatewayId:
Type: String
Resources:
SubnetAPublic:
Type: 'AWS::EC2::Subnet'
Properties:
AvailabilityZone: !Select [0, !GetAZs '']
CidrBlock: !Sub '192.168.32.0/20'
MapPublicIpOnLaunch: true
VpcId: !Sub '${VpcId}'
SubnetAPrivate:
Type: 'AWS::EC2::Subnet'
Properties:
AvailabilityZone: !Select [0, !GetAZs '']
CidrBlock: !Sub '192.168.48.0/20'
VpcId: !Sub '${VpcId}'
SubnetBPrivate:
Type: 'AWS::EC2::Subnet'
Properties:
AvailabilityZone: !Select [1, !GetAZs '']
CidrBlock: !Sub '192.168.64.0/20'
VpcId: !Sub '${VpcId}'
RouteTablePublic:
Type: 'AWS::EC2::RouteTable'
Properties:
VpcId: !Sub '${VpcId}'
RouteTablePrivate:
Type: 'AWS::EC2::RouteTable'
Properties:
VpcId: !Sub '${VpcId}'
RouteTableBPrivate:
Type: 'AWS::EC2::RouteTable'
Properties:
VpcId: !Sub '${VpcId}'
RouteTableAssociationAPublic:
Type: 'AWS::EC2::SubnetRouteTableAssociation'
Properties:
SubnetId: !Ref SubnetAPublic
RouteTableId: !Ref RouteTablePublic
RouteTableAssociationAPrivate:
Type: 'AWS::EC2::SubnetRouteTableAssociation'
Properties:
SubnetId: !Ref SubnetAPrivate
RouteTableId: !Ref RouteTablePrivate
RouteTableAssociationBPrivate:
Type: 'AWS::EC2::SubnetRouteTableAssociation'
Properties:
SubnetId: !Ref SubnetBPrivate
RouteTableId: !Ref RouteTableBPrivate
EIP:
Type: 'AWS::EC2::EIP'
Properties:
Domain: vpc
NatGateway:
Type: 'AWS::EC2::NatGateway'
Properties:
AllocationId: !GetAtt 'EIP.AllocationId'
SubnetId: !Ref SubnetAPublic
RouteTablePublicInternetRoute:
Type: AWS::EC2::Route
Properties:
RouteTableId: !Ref RouteTablePrivate
DestinationCidrBlock: '0.0.0.0/0'
NatGatewayId: !Ref NatGateway
RouteTablePublicInternetRoute2:
Type: AWS::EC2::Route
Properties:
RouteTableId: !Ref RouteTablePublic
DestinationCidrBlock: '0.0.0.0/0'
GatewayId: !Sub '${InternetGatewayId}'
ServerlessSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: SecurityGroup for Serverless Functions
VpcId: !Sub '${VpcId}'
ServerlessSecurityGroupIngress:
Type: AWS::EC2::SecurityGroupIngress
Properties:
GroupId: !Ref ServerlessSecurityGroup
IpProtocol: -1
SourceSecurityGroupId: !Ref ServerlessSecurityGroup
Any ideas what I do wrong?
P.S.: I found similar issues AWS VPC Lambda Function keeps losing internet access and Why AWS lambda functions In a VPC sometimes timeout and sometimes work fine?
UPD
Added Route and it works now
RouteTableBPrivateInternetRoute:
Type: AWS::EC2::Route
Properties:
RouteTableId: !Ref RouteTableBPrivate
DestinationCidrBlock: '0.0.0.0/0'
NatGatewayId: !Ref NatGateway

For an AWS Lambda function running inside a VPC to be able to access resources outside the VPC (such as the Internet), it must be in a private subnet with a NAT gateway. In your instance Private Subnet A is the only subnet with the appropriate configuration to allow a Lambda function to access the Internet. So you need to edit your Lambda function's configuration to only run in that subnet.

Have you selected your public subnet as one of the subnets your lambda will run in?
A lambda can only access the internet when it is running in private subnets, so go to your lambda page on the AWS Console, deselect the public subnet and save and the issue shouldn't re-occur.

Related

Lambda HTTP request to presigned S3 URL timing out

I have a Custom Lambda resource that inits my DB and then is supposed make the call to the presigned S3 url when done. It's initing the DB correctly but is timing out when making the call to S3. My guess is I did something wrong in my CloudFormation template that's causing that due to my limited networking knowledge. Would appreciate any help. Thank you in advance!
Trimmed down YAML:
AWSTemplateFormatVersion: 2010-09-09
Transform: "AWS::Serverless-2016-10-31"
Resources:
InternetGateway:
Type: "AWS::EC2::InternetGateway"
Properties:
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-InternetGateway
VPC:
Type: "AWS::EC2::VPC"
Properties:
CidrBlock: 10.0.0.0/16
EnableDnsSupport: "true"
EnableDnsHostnames: "true"
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-VPC
VPCGatewayAttachment:
Type: "AWS::EC2::VPCGatewayAttachment"
Properties:
VpcId: !Ref VPC
InternetGatewayId: !Ref InternetGateway
NATGateway:
Type: AWS::EC2::NatGateway
Properties:
AllocationId: !GetAtt ElasticIPAddress.AllocationId
SubnetId: !Ref PublicSubnet1
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-NATGateway
ElasticIPAddress:
Type: AWS::EC2::EIP
Properties:
Domain: VPC
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-EIP
PublicRouteTable:
Type: "AWS::EC2::RouteTable"
Properties:
VpcId: !Ref VPC
Tags:
- Key: Name
Value: Public
PublicRoute1:
Type: "AWS::EC2::Route"
Properties:
RouteTableId: !Ref PublicRouteTable
GatewayId: !Ref InternetGateway
DestinationCidrBlock: 0.0.0.0/0
DependsOn:
- InternetGateway
PublicSubnet1:
Type: "AWS::EC2::Subnet"
Properties:
VpcId: !Ref VPC
CidrBlock: 10.0.0.0/24
AvailabilityZone: !Select [0, !GetAZs ]
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-PublicSubnet1
CreateRDSDatabaseLambdaSG:
Type: "AWS::EC2::SecurityGroup"
Properties:
VpcId: !Ref VPC
GroupDescription: Allow Lambda to access RDS in same VPC
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-CreateRDSDatabaseLambdaSG
LambdaRDSCFNInit:
Type: AWS::Serverless::Function
DependsOn:
- InternetGateway
- VPC
- VPCGatewayAttachment
- NATGateway
- ElasticIPAddress
- PublicRouteTable
- PublicRoute1
- PublicSubnet1
- CreateRDSDatabaseLambdaSG
Properties:
CodeUri: CreateRDSDatabase/
Description: "Lambda function which will execute when this CFN template is created, updated or deleted"
Handler: app.createRDSDatabase
Runtime: nodejs12.x
Timeout: 300
VpcConfig:
SecurityGroupIds:
- !Ref CreateRDSDatabaseLambdaSG
SubnetIds:
- !Ref PublicSubnet1
Environment:
Variables:
RDS_ENDPOINT: !GetAtt RDSCluster.Endpoint.Address
RDS_DB_NAME: !Ref RDSDBName
RDS_USERNAME: !Ref RDSUserName
RDS_PASSWORD: !Ref RDSPassword
LambdaRDSCFNTrigger:
Type: Custom::ProvisionRDS
DependsOn: LambdaRDSCFNInit
Version: 1.0
Properties:
ServiceToken: !GetAtt LambdaRDSCFNInit.Arn
You are placing your lambda in PublicSubnet1. Thus, your lambda will not have internet connectivity despite your NAT or Internet gateway. You need to place your function in a private subnet, and configure your private subnet to use the NAT gateway. From docs:
To access private resources, connect your function to private subnets. If your function needs internet access, use network address translation (NAT). Connecting a function to a public subnet doesn't give it internet access or a public IP address.
Alternatively, use S3 VPC gateway endpoint and associate it with route tables of your public subnet. This way your function will access s3 using the gateway, rather then internet. No need for NAT nor private subnet in this case.

Use Qa, Dev and Prod as an environement in Cloudformation

I have created this nested stack. I want to implement the same stack with {prod, dev, qa} environment. Like I want to up the same stack but it doesn't have any name conflicts with each other. I want to deploy the same stack in three different environment, What changes do I have to make to achieve it
Root:
---
AWSTemplateFormatVersion: 2010-09-09
Parameters:
bucketname:
Type: String
Description: Path to the bucket
Default: webserver
bucketpath:
Type: String
Description: Path to the bucket
Default: /env #/mysql
Env:
Type: String
Description: Select the appropriate environment
AllowedValues:
- dev
- test
- uat
- prod
Cidr:
Type: String
Description: Cidr for vpc
Publicsubnet1:
Type: String
Description: public subnet 1
Publicsubnet2:
Type: String
Description: public subnet 2
Privatesubnet1:
Type: String
Description: Private subnet 1
Privatesubnet2:
Type: String
Description: Private subnet 2
Resources:
Vpcstack:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: !Sub "https://${bucketname}.s3.us-east-2.amazonaws.com${bucketpath}/vpc.yml"
Parameters:
Env: Ref: Env
Cidr: !Ref Cidr
Publicsubnet1: !Ref Publicsubnet1
Publicsubnet2: !Ref Publicsubnet2
Privatesubnet1: !Ref Privatesubnet1
Privatesubnet2: !Ref Privatesubnet2
Vpc:
---
AWSTemplateFormatVersion: 2010-09-09
Parameters:
Cidr:
Type: String
Description: Cidr for vpc
Publicsubnet1:
Type: String
Description: public subnet 1
Publicsubnet2:
Type: String
Description: public subnet 2
Privatesubnet1:
Type: String
Description: Private subnet 1
Privatesubnet2:
Type: String
Description: Private subnet 2
Env:
Type: String
Description: Select the appropriate environment
Resources:
VPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: !Ref Cidr
EnableDnsSupport: true
EnableDnsHostnames: true
InstanceTenancy: default
InternetGateway:
Type: AWS::EC2::InternetGateway
VPCGatewayAttachment:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
VpcId: !Ref VPC
InternetGatewayId: !Ref InternetGateway
SubnetA:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: us-east-2a
VpcId: !Ref VPC
CidrBlock: !Ref Publicsubnet1
MapPublicIpOnLaunch: true
SubnetB:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: us-east-2b
VpcId: !Ref VPC
CidrBlock: !Ref Publicsubnet2
MapPublicIpOnLaunch: true
SubnetC:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: us-east-2a
VpcId: !Ref VPC
CidrBlock: !Ref Privatesubnet1
MapPublicIpOnLaunch: false
SubnetD:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: us-east-2b
VpcId: !Ref VPC
CidrBlock: !Ref Privatesubnet2
MapPublicIpOnLaunch: false
RouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref VPC
RouteTable2:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref VPC
InternetRoute:
Type: AWS::EC2::Route
DependsOn: VPCGatewayAttachment
Properties:
DestinationCidrBlock: 0.0.0.0/0
GatewayId: !Ref InternetGateway
RouteTableId: !Ref RouteTable
SubnetARouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref RouteTable
SubnetId: !Ref SubnetA
SubnetBRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref RouteTable
SubnetId: !Ref SubnetB
SubnetCRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref RouteTable2
SubnetId: !Ref SubnetC
SubnetDRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref RouteTable2
SubnetId: !Ref SubnetD
SecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: "Internet Group"
GroupDescription: "SSH traffic in, all traffic out."
VpcId: !Ref VPC
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: "22"
ToPort: "22"
CidrIp: 0.0.0.0/0
SecurityGroupEgress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
NAT:
Type: AWS::EC2::NatGateway
Properties:
AllocationId:
Fn::GetAtt:
- EIP
- AllocationId
SubnetId:
Ref: SubnetA
Tags:
- Key: Name
Value: !Sub "nat-${Env}"
EIP:
DependsOn: VPCGatewayAttachment
Type: AWS::EC2::EIP
Properties:
Domain: VPC
Route:
Type: AWS::EC2::Route
Properties:
RouteTableId:
Ref: RouteTable2
DestinationCidrBlock: 0.0.0.0/0
NatGatewayId:
Ref: NAT
Outputs:
VpcID:
Description: VPC id
Value: !Ref VPC
Export:
Name: "VpcID"
SubnetA:
Description: public subnet
Value: !Ref SubnetA
Export:
Name: "SubnetA"
SubnetB:
Description: public subnet 2
Value: !Ref SubnetB
Export:
Name: "SubnetB"
SubnetC:
Description: priavte subnet
Value: !Ref SubnetC
Export:
Name: "SubnetC"
SubnetD:
Description: private subnet 2
Value: !Ref SubnetD
Export:
Name: "SubnetD"
You can specify different names for your top-level stacks by adding the environment to the top-level stack name. You do this at the time of stack creation, via the console or programmatically.
Then when each top-level environment-specific stack runs it will create the necessary nested stacks without name conflicts. You won't be able to control the stack names for the nested stacks, but you can get the name using outputs.
See the following:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-stack.html
You can add output values from a nested stack within the containing
template. You use the GetAtt function with the nested stack's logical
name and the name of the output value in the nested stack in the
format Outputs.NestedStackOutputName.
CloudFormation nested stack name
If you need to use different resource values for the different environments, then you can use mappings to specify the settings that correspond to the selected environment. Here is an example of mappings:
Mappings:
EnvTypeMap:
prod:
vpc: vpc-a6842gb0
subnet: subnet-hjk23553
dev:
vpc: vpc-b7742gb0
subnet: subnet-abc23553
qa:
vpc: vpc-c2542gb0
subnet: subnet-uio23553
Then to reference one of these mapping values you would do so like this:
VpcId:
Fn::FindInMap:
- EnvTypeMap
- Ref: Env
- vpc
CF stack is identified by a stack-name. All you have to do is to specify this stack-name when you are deploying the CF template.
aws cloudformation deploy --stack-name <value> --template-file <value> ...
If you specify the name of an existing stack, that stack will be updated. If you specify a new name, you will create a new stack from a given template.
You can create as many stacks as you want from a singe template by choosing new stack name each time. You do not need to worry about naming conflicts because each resource's name in a given stack is uniquely identified based on the stack name which will be different.
aws cloudformation deploy --stack-name dev --template-file the-same-template.yaml ...
aws cloudformation deploy --stack-name test --template-file the-same-template.yaml ...
aws cloudformation deploy --stack-name uat --template-file the-same-template.yaml ...
aws cloudformation deploy --stack-name prod --template-file the-same-template.yaml ...
This will create 4 separate stacks (dev, test, uat, prod).
Note that since you are hardcoding IP address ranges, resources in these stacks will not be able to communicate between each other because of overlapping networks (which probably is what you want anyway) but if for some reason you need these to communicate, you will need to create Parameters for CIDR blocks (VPC, subnets) as well.

Can't Connect to MariaDB RDS instance in VPC

I built a VPC with a few AWS resources inside of it. I can get Internet access if I'm in the VPC and resources inside the VPC can communicate with each other. For instance, I have a Lambda function that can communicate to the Internet and also reach an RDS instance inside the VPC. However, the issue comes in when I am trying to connect to the RDS instance from my local machine.
I've tried updating the VPCSecurityGroup to allow all incoming traffic, but still won't work. The only thing that seems to work is if I switch all of the Route Tables to use an IGW as opposed to a NAT, but I'd prefer if that wasn't the case. Also, I'm not even sure I'd be allowed to do that because I'm pretty sure lambda functions have to exist in private subnets.
vpc.yml
AWSTemplateFormatVersion: 2010-09-09
Description: VPC Stack
Resources:
Vpc:
Type: 'AWS::EC2::VPC'
Properties:
CidrBlock: 10.0.0.0/16
EnableDnsSupport: true
EnableDnsHostnames: true
InstanceTenancy: default
InternetGateway:
Type: 'AWS::EC2::InternetGateway'
VpcGatewayAttachment:
Type: 'AWS::EC2::VPCGatewayAttachment'
Properties:
VpcId: !Ref Vpc
InternetGatewayId: !Ref InternetGateway
ElasticIP:
Type: 'AWS::EC2::EIP'
Properties:
Domain: vpc
NatGateway:
Type: 'AWS::EC2::NatGateway'
DependsOn:
- VpcGatewayAttachment
Properties:
AllocationId: !GetAtt
- ElasticIP
- AllocationId
SubnetId: !Ref SubnetAPublic
SubnetAPublic:
Type: 'AWS::EC2::Subnet'
Properties:
AvailabilityZone: !Select
- '0'
- !GetAZs ''
CidrBlock: 10.0.0.0/19
MapPublicIpOnLaunch: true
VpcId: !Ref Vpc
SubnetBPublic:
Type: 'AWS::EC2::Subnet'
Properties:
AvailabilityZone: !Select
- '1'
- !GetAZs ''
CidrBlock: 10.0.32.0/19
MapPublicIpOnLaunch: true
VpcId: !Ref Vpc
SubnetAPrivate:
Type: 'AWS::EC2::Subnet'
Properties:
AvailabilityZone: !Select
- '0'
- !GetAZs ''
CidrBlock: 10.0.64.0/19
VpcId: !Ref Vpc
SubnetBPrivate:
Type: 'AWS::EC2::Subnet'
Properties:
AvailabilityZone: !Select
- '1'
- !GetAZs ''
CidrBlock: 10.0.96.0/19
VpcId: !Ref Vpc
RouteTableAPublic:
Type: 'AWS::EC2::RouteTable'
Properties:
VpcId: !Ref Vpc
RouteTableBPublic:
Type: 'AWS::EC2::RouteTable'
Properties:
VpcId: !Ref Vpc
RouteTableAPrivate:
Type: 'AWS::EC2::RouteTable'
Properties:
VpcId: !Ref Vpc
RouteTableBPrivate:
Type: 'AWS::EC2::RouteTable'
Properties:
VpcId: !Ref Vpc
RouteTableAssociationAPublic:
Type: 'AWS::EC2::SubnetRouteTableAssociation'
Properties:
SubnetId: !Ref SubnetAPublic
RouteTableId: !Ref RouteTableAPublic
RouteTableAssociationBPublic:
Type: 'AWS::EC2::SubnetRouteTableAssociation'
Properties:
SubnetId: !Ref SubnetBPublic
RouteTableId: !Ref RouteTableBPublic
RouteTableAssociationAPrivate:
Type: 'AWS::EC2::SubnetRouteTableAssociation'
Properties:
SubnetId: !Ref SubnetAPrivate
RouteTableId: !Ref RouteTableAPrivate
RouteTableAssociationBPrivate:
Type: 'AWS::EC2::SubnetRouteTableAssociation'
Properties:
SubnetId: !Ref SubnetBPrivate
RouteTableId: !Ref RouteTableBPrivate
RouteTableAPrivateInternetRoute:
Type: 'AWS::EC2::Route'
DependsOn:
- VpcGatewayAttachment
Properties:
RouteTableId: !Ref RouteTableAPrivate
DestinationCidrBlock: 0.0.0.0/0
NatGatewayId: !Ref NatGateway
RouteTableBPrivateInternetRoute:
Type: 'AWS::EC2::Route'
DependsOn:
- VpcGatewayAttachment
Properties:
RouteTableId: !Ref RouteTableBPrivate
DestinationCidrBlock: 0.0.0.0/0
NatGatewayId: !Ref NatGateway
RouteTableAPublicInternetRoute:
Type: 'AWS::EC2::Route'
Properties:
RouteTableId: !Ref RouteTableAPublic
DestinationCidrBlock: 0.0.0.0/0
GatewayId: !Ref InternetGateway
RouteTableBPublicInternetRoute:
Type: 'AWS::EC2::Route'
Properties:
RouteTableId: !Ref RouteTableBPublic
DestinationCidrBlock: 0.0.0.0/0
GatewayId: !Ref InternetGateway
NetworkAclPublic:
Type: 'AWS::EC2::NetworkAcl'
Properties:
VpcId: !Ref Vpc
NetworkAclPrivate:
Type: 'AWS::EC2::NetworkAcl'
Properties:
VpcId: !Ref Vpc
SubnetNetworkAclAssociationAPublic:
Type: 'AWS::EC2::SubnetNetworkAclAssociation'
Properties:
SubnetId: !Ref SubnetAPublic
NetworkAclId: !Ref NetworkAclPublic
SubnetNetworkAclAssociationBPublic:
Type: 'AWS::EC2::SubnetNetworkAclAssociation'
Properties:
SubnetId: !Ref SubnetBPublic
NetworkAclId: !Ref NetworkAclPublic
SubnetNetworkAclAssociationAPrivate:
Type: 'AWS::EC2::SubnetNetworkAclAssociation'
Properties:
SubnetId: !Ref SubnetAPrivate
NetworkAclId: !Ref NetworkAclPrivate
SubnetNetworkAclAssociationBPrivate:
Type: 'AWS::EC2::SubnetNetworkAclAssociation'
Properties:
SubnetId: !Ref SubnetBPrivate
NetworkAclId: !Ref NetworkAclPrivate
NetworkAclEntryInPublicAllowAll:
Type: 'AWS::EC2::NetworkAclEntry'
Properties:
NetworkAclId: !Ref NetworkAclPublic
RuleNumber: 99
Protocol: -1
RuleAction: allow
Egress: false
CidrBlock: 0.0.0.0/0
NetworkAclEntryOutPublicAllowAll:
Type: 'AWS::EC2::NetworkAclEntry'
Properties:
NetworkAclId: !Ref NetworkAclPublic
RuleNumber: 99
Protocol: -1
RuleAction: allow
Egress: true
CidrBlock: 0.0.0.0/0
NetworkAclEntryInPrivateAllowVpc:
Type: 'AWS::EC2::NetworkAclEntry'
Properties:
NetworkAclId: !Ref NetworkAclPrivate
RuleNumber: 99
Protocol: -1
RuleAction: allow
Egress: false
CidrBlock: 0.0.0.0/0
NetworkAclEntryOutPrivateAllowVpc:
Type: 'AWS::EC2::NetworkAclEntry'
Properties:
NetworkAclId: !Ref NetworkAclPrivate
RuleNumber: 99
Protocol: -1
RuleAction: allow
Egress: true
CidrBlock: 0.0.0.0/0
LambdaSecurityGroup:
Type: 'AWS::EC2::SecurityGroup'
Properties:
GroupDescription: Lambdas security group
SecurityGroupEgress:
- CidrIp: 0.0.0.0/0
IpProtocol: '-1'
SecurityGroupIngress:
- CidrIp: 0.0.0.0/0
IpProtocol: '-1'
VpcId: !Ref Vpc
Outputs:
VpcId:
Description: VPC ID
Value: !Ref Vpc
Export:
Name: !Sub "Portal-VpcId"
SubnetAPrivate:
Description: Subnet A Private
Value: !Ref SubnetAPrivate
Export:
Name: !Sub "SubnetAPrivate"
SubnetBPrivate:
Description: Subnet B Private
Value: !Ref SubnetBPrivate
Export:
Name: !Sub "SubnetBPrivate"
SubnetAPublic:
Description: Subnet A Public
Value: !Ref SubnetAPublic
Export:
Name: !Sub "SubnetAPublic"
SubnetBPublic:
Description: Subnet B Public
Value: !Ref SubnetBPublic
Export:
Name: !Sub "SubnetBPublic"
LambdaSecurityGroup:
Description: Access to Lambda functions
Value: !Ref LambdaSecurityGroup
Export:
Name: !Sub "LambdaSecurityGroup"
rds.yml
DBSubnetGroup:
Type: 'AWS::RDS::DBSubnetGroup'
Properties:
DBSubnetGroupDescription: Subnets available for the RDS DB Instance
SubnetIds:
- !Ref SubnetAPublic
- !Ref SubnetBPublic
VPCSecurityGroup:
Type: 'AWS::EC2::SecurityGroup'
Properties:
GroupDescription: Security group for RDS DB Instance.
VpcId: !Ref VpcId
SecurityGroupIngress:
-
IpProtocol: "tcp"
FromPort: "3306"
ToPort: "3306"
CidrIp: "[my IP]"
-
IpProtocol: "tcp"
FromPort: "3306"
ToPort: "3306"
CidrIp: "10.0.64.0/19"
-
IpProtocol: "tcp"
FromPort: "3306"
ToPort: "3306"
CidrIp: "10.0.96.0/19"
DBInstance:
Type: 'AWS::RDS::DBInstance'
Properties:
DBName: !Join
- ''
- - portal
- !Ref Environment
AllocatedStorage: !Ref DBAllocatedStorage
DBInstanceClass: !Ref DBClass
Engine: MariaDB
EngineVersion: '10.1.23'
MasterUsername: !Ref DBUsername
MasterUserPassword: !Ref DBPassword
DBSubnetGroupName: !Ref DBSubnetGroup
StorageEncrypted: true
PubliclyAccessible: true
VPCSecurityGroups:
- !Ref VPCSecurityGroup
DatabaseDnsRecord:
Type: AWS::Route53::RecordSet
Properties:
HostedZoneName: !Join
- ''
- - !Ref HostedZoneName
- .
Name: !Join
- ''
- - portal
- !Ref Environment
- 'db'
- .
- !Ref HostedZoneName
- .
Type: CNAME
TTL: '60'
ResourceRecords:
- !GetAtt
- DBInstance
- Endpoint.Address
DependsOn: DBInstance
Your problem is this
The only thing that seems to work is if I switch all of the Route
Tables to use an IGW as opposed to a NAT
Your instances are located in private subnets which are not accessible from the public Internet (your home PC). You have three (or more) solutions:
1) Move your instances to a public subnet. NOT RECOMMENDED.
2) Convert your private subnet into a public subnet (switch from NAT to IGW). NOT RECOMMENDED.
3) Create a VPN from your home network to a new EC2 instance located in your public subnet that routes your traffic to the instances in the private subnet. RECOMMENDED.
OpenVPN is a very cool solution. You can build this yourself or just launch an OpenVPN instance from Amazon Marketplace for free (I think the free is limited to 2 users). OpenVPN Access Server
There will be EC2 instance charges while the OpenVPN access server is running. What I do is shutdown that instance when I don't need it and start it back up when I do using the AWS CLI commands stored in batch files.

Creating a publicly accessible RDS instance in AWS Cloudformation

I'm totally throwing my hands up with this one. I've been trying to create a publicly accessible RDS instance using CloudFormation. I want to be able to connect to my instance via a mysql client. When I deploy this stack it says that the instance is publicly accessible in the RDS console, but I can't connect to via the endpoint provided in the RDS console. I'm guessing that I messed up/missed something with the VPC pieces. He's my stack.yaml file:
Resources:
Vpc:
Type: AWS::EC2::VPC
Properties:
CidrBlock: 10.0.0.0/16
EnableDnsHostnames: true
Tags:
- Key: Name
Value: 'VPC created by cf'
InternetGateway:
Type: AWS::EC2::InternetGateway
Properties:
Tags:
- Key: Name
Value: Created By CF
VPCGatewayAttachment:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
VpcId: !Ref Vpc
InternetGatewayId: !Ref InternetGateway
DataSourceSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Open database for access
VpcId: !Ref Vpc
DSSGIngressRule:
Type: AWS::EC2::SecurityGroupIngress
Properties:
FromPort: "3306"
ToPort: "3306"
GroupId: !Ref DataSourceSecurityGroup
IpProtocol: tcp
SourceSecurityGroupId: !Ref DataSourceSecurityGroup
PublicSubnet1:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: us-east-1a
CidrBlock: 10.0.0.0/20
MapPublicIpOnLaunch: true
VpcId: !Ref Vpc
PublicSubnet2:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: us-east-1b
CidrBlock: 10.0.16.0/20
MapPublicIpOnLaunch: true
VpcId: !Ref Vpc
RouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref Vpc
Tags:
- Key: Name
Value: 'RouteTable created by CF'
RouteTable1Association:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PublicSubnet1
RouteTableId: !Ref RouteTable
RouteTable2Association:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PublicSubnet2
RouteTableId: !Ref RouteTable
InternetRouteRule:
Type: AWS::EC2::Route
DependsOn: VPCGatewayAttachment
Properties:
RouteTableId: !Ref RouteTable
DestinationCidrBlock: 0.0.0.0/0
GatewayId: !Ref InternetGateway
DataSourceSubtNetGroup:
Type: AWS::RDS::DBSubnetGroup
Properties:
DBSubnetGroupDescription: Created by CF
SubnetIds:
- !Ref PublicSubnet1
- !Ref PublicSubnet2
DataSource:
Type: AWS::RDS::DBInstance
Properties:
AllocatedStorage: '5'
DBInstanceClass: db.m1.small
DBName: MyDb
DBSubnetGroupName: !Ref DataSourceSubtNetGroup
Engine: MySQL
MasterUsername: AdminUser
MasterUserPassword: AdminPassword
PubliclyAccessible: true
VPCSecurityGroups:
- !Ref DataSourceSecurityGroup
DeletionPolicy: Snapshot
Thanks
Your DataSourceSecurityGroup security group is currently configured as:
Permit inbounded connections on Port 3306 from Security Group DataSourceSecurityGroup
That is, it will allow inbound connections from any Amazon EC2 instance that is itself a member of the DataSourceSecurityGroup security group.
If you wanted to allow access from anywhere on the Internet, then change your template to permit inbound access from 0.0.0.0/0:
DSSGIngressRule:
Type: AWS::EC2::SecurityGroupIngress
Properties:
FromPort: "3306"
ToPort: "3306"
GroupId: !Ref DataSourceSecurityGroup
IpProtocol: tcp
CidrIp: 0.0.0.0/0
I made this change, tested your template and it worked fine.
For future reference: You can debug this type of thing by creating the stack and then examining the Security Group in the management console.

AWS CloudFormation Errors with Internet Gateway

CloudFormation beginner here. I've been researching and working on developing a CloudFormation template that will eventually be used as the starting point for a development environment for my team.
I've been picking at bits and pieces through some courses and examples online and have been relatively successful in my small attempt... Until tonight.
I am now trying to attach an Internet Gateway to my VPC and it is causing the Stack creation job to fail and rollback. The Internet Gateway will not attach and for the life of me I just cannot determine why.
My full template is here. The plan is to create a VPC with 2 public and 2 private subnets. There will be an Internet Gateway attached to the 2 public subnets. This is where the failure comes in. If I comment out the Internet Gateway creation, the template is successful. Thanks in advance for your help.
AWSTemplateFormatVersion: '2010-09-09'
Resources:
DevVPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: 10.0.0.0/16
EnableDnsSupport: 'true'
EnableDnsHostnames: 'true'
InstanceTenancy: default
Tags:
- Key: Name
Value: dev-vpc
DevRoute53HostedZone:
Type: "AWS::Route53::HostedZone"
Properties:
HostedZoneConfig:
Comment: "aws hosted dev environment"
Name: "mydomain.oregon-dev.local"
VPCs:
-
VPCId: !Ref DevVPC
VPCRegion: "us-west-2"
DevPublicSubnetA:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref DevVPC
CidrBlock: 10.0.8.0/25
AvailabilityZone: "us-west-2a"
Tags:
- Key: Name
Value: DevPublicSubnetA
DevPublicSubnetB:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref DevVPC
CidrBlock: 10.0.8.128/25
AvailabilityZone: "us-west-2b"
Tags:
- Key: Name
Value: DevPublicSubnetB
DevPrivateSubnetA:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref DevVPC
CidrBlock: 10.0.9.0/25
AvailabilityZone: "us-west-2a"
Tags:
- Key: Name
Value: DevPrivateSubnetA
DevPrivateSubnetB:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref DevVPC
CidrBlock: 10.0.9.128/25
AvailabilityZone: "us-west-2b"
Tags:
- Key: Name
Value: DevPrivateSubnetB
RouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId:
Ref: DevVPC
Tags:
- Key: Name
Value: DevRouteTable
DevRoute:
Type: AWS::EC2::Route
DependsOn: NonProdNATGateway
Properties:
RouteTableId:
Ref: RouteTable
DestinationCidrBlock: 0.0.0.0/0
GatewayId:
Ref: NonProdNATGateway
NonProdNATEIP:
Type: AWS::EC2::EIP
Properties:
Domain: vpc
NonProdNATGateway:
Type: AWS::EC2::NatGateway
Properties:
AllocationId: !GetAtt NonProdNATEIP.AllocationId
SubnetId: !Ref DevPublicSubnetA
SubnetId: !Ref DevPublicSubnetB
DependsOn:
- NonProdNATEIP
- DevPublicSubnetA
- DevPublicSubnetB
NonProdGWVPCAttachment:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
InternetGatewayId: !Ref NonProdNATGateway
VpcId: !Ref DevVPC
DependsOn:
- NonProdNATGateway
Route:
Type: AWS::EC2::Route
Properties:
RouteTableId:
Ref: RouteTable
DestinationCidrBlock: 0.0.0.0/0
NatGatewayId:
Ref: NonProdNATGateway
PrivateRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref RouteTable
SubnetId: !Ref DevPrivateSubnetA
SubnetId: !Ref DevPrivateSubnetB
PublicRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref RouteTable
SubnetId: !Ref DevPublicSubnetA
SubnetId: !Ref DevPublicSubnetB
Mappings:
R53EnvironmentMapping:
dev:
oregonawslocal: mydomain.oregon-dev.local
Outputs:
DevPublicSubnetA:
Description: ID for dev subnet A
Value: !Ref DevPublicSubnetA
Export:
Name: DevPublicSubnetA
DevPublicSubnetB:
Description: ID for dev subnet B
Value: !Ref DevPublicSubnetB
Export:
Name: DevPublicSubnetB
DevPrivateSubnetA:
Description: ID for dev subnet A
Value: !Ref DevPrivateSubnetA
Export:
Name: DevPrivateSubnetA
DevPrivateSubnetB:
Description: ID for dev subnet B
Value: !Ref DevPrivateSubnetB
Export:
Name: DevPrivateSubnetB
DevRoute53OregonAWSLocalHostedZone:
Description: Hosted zone ID for hosted zone
Value: !Ref DevRoute53HostedZone
Export:
Name: DevRoute53OregonAWSLocalHostedZone
DevRoute53OregonAWSLocalHostedZoneName:
Description: Hosted zone name for hosted zone
Value: !FindInMap [R53EnvironmentMapping, dev, oregonawslocal]
Export:
Name: DevRoute53OregonAWSLocalHostedZoneName
As Michael - sqlbot mentioned in a comment, one issue is that you're referencing an AWS::EC2::NATGateway resource in the AWS::EC2::VPCGatewayAttachment resource's InternetGatewayId property, which requires an AWS::EC2::InternetGateway resource.
NAT Gateways and Internet Gateways are two different types of AWS resources - a NAT Gateway provides outbound-only Internet access to a private Subnet, while an Internet Gateway provides two-way Internet access to a public Subnet.
Another issue is that you need two separate sets of AWS::EC2::RouteTable and AWS::EC2::Route Resources, one set for your public Subnet and another for your private Subnet. The public Route should have GatewayId referencing the Internet Gateway, and the private Route should have NatGatewayId referencing the NAT Gateway.
Finally, you have some invalid duplicate SubnetId properties in several resources (NatGateway, SubnetRouteTableAssociation)- each of these Resources only points accepts a single Subnet ID.
Since you're a CloudFormation beginner, I strongly recommend leveraging AWS Quick Start's Amazon VPC Architecture template to get started quickly with a reference VPC architecture. This AWS-supported template creates a single VPC containing both public and private subnets within each specified Availability Zone (you provide 2-4 Availability Zones as Parameters). You can later customize this template to better fit your specific needs if necessary, or use it as a reference for configuring your own template's resources.