CloudFormation beginner here. I've been researching and working on developing a CloudFormation template that will eventually be used as the starting point for a development environment for my team.
I've been picking at bits and pieces through some courses and examples online and have been relatively successful in my small attempt... Until tonight.
I am now trying to attach an Internet Gateway to my VPC and it is causing the Stack creation job to fail and rollback. The Internet Gateway will not attach and for the life of me I just cannot determine why.
My full template is here. The plan is to create a VPC with 2 public and 2 private subnets. There will be an Internet Gateway attached to the 2 public subnets. This is where the failure comes in. If I comment out the Internet Gateway creation, the template is successful. Thanks in advance for your help.
AWSTemplateFormatVersion: '2010-09-09'
Resources:
DevVPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: 10.0.0.0/16
EnableDnsSupport: 'true'
EnableDnsHostnames: 'true'
InstanceTenancy: default
Tags:
- Key: Name
Value: dev-vpc
DevRoute53HostedZone:
Type: "AWS::Route53::HostedZone"
Properties:
HostedZoneConfig:
Comment: "aws hosted dev environment"
Name: "mydomain.oregon-dev.local"
VPCs:
-
VPCId: !Ref DevVPC
VPCRegion: "us-west-2"
DevPublicSubnetA:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref DevVPC
CidrBlock: 10.0.8.0/25
AvailabilityZone: "us-west-2a"
Tags:
- Key: Name
Value: DevPublicSubnetA
DevPublicSubnetB:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref DevVPC
CidrBlock: 10.0.8.128/25
AvailabilityZone: "us-west-2b"
Tags:
- Key: Name
Value: DevPublicSubnetB
DevPrivateSubnetA:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref DevVPC
CidrBlock: 10.0.9.0/25
AvailabilityZone: "us-west-2a"
Tags:
- Key: Name
Value: DevPrivateSubnetA
DevPrivateSubnetB:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref DevVPC
CidrBlock: 10.0.9.128/25
AvailabilityZone: "us-west-2b"
Tags:
- Key: Name
Value: DevPrivateSubnetB
RouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId:
Ref: DevVPC
Tags:
- Key: Name
Value: DevRouteTable
DevRoute:
Type: AWS::EC2::Route
DependsOn: NonProdNATGateway
Properties:
RouteTableId:
Ref: RouteTable
DestinationCidrBlock: 0.0.0.0/0
GatewayId:
Ref: NonProdNATGateway
NonProdNATEIP:
Type: AWS::EC2::EIP
Properties:
Domain: vpc
NonProdNATGateway:
Type: AWS::EC2::NatGateway
Properties:
AllocationId: !GetAtt NonProdNATEIP.AllocationId
SubnetId: !Ref DevPublicSubnetA
SubnetId: !Ref DevPublicSubnetB
DependsOn:
- NonProdNATEIP
- DevPublicSubnetA
- DevPublicSubnetB
NonProdGWVPCAttachment:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
InternetGatewayId: !Ref NonProdNATGateway
VpcId: !Ref DevVPC
DependsOn:
- NonProdNATGateway
Route:
Type: AWS::EC2::Route
Properties:
RouteTableId:
Ref: RouteTable
DestinationCidrBlock: 0.0.0.0/0
NatGatewayId:
Ref: NonProdNATGateway
PrivateRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref RouteTable
SubnetId: !Ref DevPrivateSubnetA
SubnetId: !Ref DevPrivateSubnetB
PublicRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref RouteTable
SubnetId: !Ref DevPublicSubnetA
SubnetId: !Ref DevPublicSubnetB
Mappings:
R53EnvironmentMapping:
dev:
oregonawslocal: mydomain.oregon-dev.local
Outputs:
DevPublicSubnetA:
Description: ID for dev subnet A
Value: !Ref DevPublicSubnetA
Export:
Name: DevPublicSubnetA
DevPublicSubnetB:
Description: ID for dev subnet B
Value: !Ref DevPublicSubnetB
Export:
Name: DevPublicSubnetB
DevPrivateSubnetA:
Description: ID for dev subnet A
Value: !Ref DevPrivateSubnetA
Export:
Name: DevPrivateSubnetA
DevPrivateSubnetB:
Description: ID for dev subnet B
Value: !Ref DevPrivateSubnetB
Export:
Name: DevPrivateSubnetB
DevRoute53OregonAWSLocalHostedZone:
Description: Hosted zone ID for hosted zone
Value: !Ref DevRoute53HostedZone
Export:
Name: DevRoute53OregonAWSLocalHostedZone
DevRoute53OregonAWSLocalHostedZoneName:
Description: Hosted zone name for hosted zone
Value: !FindInMap [R53EnvironmentMapping, dev, oregonawslocal]
Export:
Name: DevRoute53OregonAWSLocalHostedZoneName
As Michael - sqlbot mentioned in a comment, one issue is that you're referencing an AWS::EC2::NATGateway resource in the AWS::EC2::VPCGatewayAttachment resource's InternetGatewayId property, which requires an AWS::EC2::InternetGateway resource.
NAT Gateways and Internet Gateways are two different types of AWS resources - a NAT Gateway provides outbound-only Internet access to a private Subnet, while an Internet Gateway provides two-way Internet access to a public Subnet.
Another issue is that you need two separate sets of AWS::EC2::RouteTable and AWS::EC2::Route Resources, one set for your public Subnet and another for your private Subnet. The public Route should have GatewayId referencing the Internet Gateway, and the private Route should have NatGatewayId referencing the NAT Gateway.
Finally, you have some invalid duplicate SubnetId properties in several resources (NatGateway, SubnetRouteTableAssociation)- each of these Resources only points accepts a single Subnet ID.
Since you're a CloudFormation beginner, I strongly recommend leveraging AWS Quick Start's Amazon VPC Architecture template to get started quickly with a reference VPC architecture. This AWS-supported template creates a single VPC containing both public and private subnets within each specified Availability Zone (you provide 2-4 Availability Zones as Parameters). You can later customize this template to better fit your specific needs if necessary, or use it as a reference for configuring your own template's resources.
Related
I've created a VPC and EKS cluster using CloudFormation, and when I try to create an AWS managed nodegroup through CloudFormation, it fails to create with the error message: Nodegroup test-ng failed to stabilize: [{Code: NodeCreationFailure,Message: Unhealthy nodes in the kubernetes cluster. I haven't been able to pinpoint the exact issue, but my setup is based on the AWS docs. For reference, I want a VPC with 3 public and 3 private subnets, with the managed nodegroup deployed to the private subnets. Here are the templates I've used to deploy everything:
VPC Template:
---
AWSTemplateFormatVersion: '2010-09-09'
Description: 'EKS VPC - Private and Public Subnets'
Parameters:
VpcName:
Type: String
Default: EKS-VPC
Description: The name of the VPC
VpcBlock:
Type: String
Default: 10.0.0.0/16
Description: The CIDR range for the VPC. This should be a valid private (RFC 1918) CIDR range.
PrivateSubnet01Block:
Type: String
Default: 10.0.0.0/19
Description: CidrBlock for private subnet 01 within the VPC
PrivateSubnet02Block:
Type: String
Default: 10.0.32.0/19
Description: CidrBlock for private subnet 02 within the VPC
PrivateSubnet03Block:
Type: String
Default: 10.0.64.0/19
Description: CidrBlock for private subnet 03 within the VPC
PublicSubnet01Block:
Type: String
Default: 10.0.128.0/20
Description: CidrBlock for public subnet 01 within the VPC
PublicSubnet02Block:
Type: String
Default: 10.0.144.0/20
Description: CidrBlock for public subnet 02 within the VPC
PublicSubnet03Block:
Type: String
Default: 10.0.160.0/20
Description: CidrBlock for public subnet 02 within the VPC
Metadata:
AWS::CloudFormation::Interface:
ParameterGroups:
-
Label:
default: "Main"
Parameters:
- VpcName
-
Label:
default: "Network Configuration"
Parameters:
- VpcBlock
- PublicSubnet01Block
- PublicSubnet02Block
- PublicSubnet03Block
- PrivateSubnet01Block
- PrivateSubnet02Block
- PrivateSubnet03Block
Resources:
VPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: !Ref VpcBlock
EnableDnsSupport: true
EnableDnsHostnames: true
Tags:
- Key: Name
Value: !Ref VpcName
InternetGateway:
Type: "AWS::EC2::InternetGateway"
VPCGatewayAttachment:
Type: "AWS::EC2::VPCGatewayAttachment"
Properties:
InternetGatewayId: !Ref InternetGateway
VpcId: !Ref VPC
PublicRouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref VPC
Tags:
- Key: Name
Value: Public Subnets RT
- Key: Network
Value: Public
PrivateRouteTable01:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref VPC
Tags:
- Key: Name
Value: Private Subnet 01 RT
- Key: Network
Value: Private
PrivateRouteTable02:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref VPC
Tags:
- Key: Name
Value: Private Subnet 02 RT
- Key: Network
Value: Private
PrivateRouteTable03:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref VPC
Tags:
- Key: Name
Value: Private Subnet 03 RT
- Key: Network
Value: Private
PublicRoute:
DependsOn: VPCGatewayAttachment
Type: AWS::EC2::Route
Properties:
RouteTableId: !Ref PublicRouteTable
DestinationCidrBlock: 0.0.0.0/0
GatewayId: !Ref InternetGateway
PrivateRoute01:
DependsOn:
- VPCGatewayAttachment
- NatGateway01
Type: AWS::EC2::Route
Properties:
RouteTableId: !Ref PrivateRouteTable01
DestinationCidrBlock: 0.0.0.0/0
NatGatewayId: !Ref NatGateway01
PrivateRoute02:
DependsOn:
- VPCGatewayAttachment
- NatGateway02
Type: AWS::EC2::Route
Properties:
RouteTableId: !Ref PrivateRouteTable02
DestinationCidrBlock: 0.0.0.0/0
NatGatewayId: !Ref NatGateway02
PrivateRoute03:
DependsOn:
- VPCGatewayAttachment
- NatGateway03
Type: AWS::EC2::Route
Properties:
RouteTableId: !Ref PrivateRouteTable03
DestinationCidrBlock: 0.0.0.0/0
NatGatewayId: !Ref NatGateway03
NatGateway01:
DependsOn:
- NatGatewayEIP1
- PublicSubnet01
- VPCGatewayAttachment
Type: AWS::EC2::NatGateway
Properties:
AllocationId: !GetAtt 'NatGatewayEIP1.AllocationId'
SubnetId: !Ref PublicSubnet01
Tags:
- Key: Name
Value: !Sub '${VpcName}-NatGateway01'
NatGateway02:
DependsOn:
- NatGatewayEIP2
- PublicSubnet02
- VPCGatewayAttachment
Type: AWS::EC2::NatGateway
Properties:
AllocationId: !GetAtt 'NatGatewayEIP2.AllocationId'
SubnetId: !Ref PublicSubnet02
Tags:
- Key: Name
Value: !Sub '${VpcName}-NatGateway02'
NatGateway03:
DependsOn:
- NatGatewayEIP3
- PublicSubnet03
- VPCGatewayAttachment
Type: AWS::EC2::NatGateway
Properties:
AllocationId: !GetAtt 'NatGatewayEIP3.AllocationId'
SubnetId: !Ref PublicSubnet03
Tags:
- Key: Name
Value: !Sub '${VpcName}-NatGateway03'
NatGatewayEIP1:
DependsOn:
- VPCGatewayAttachment
Type: 'AWS::EC2::EIP'
Properties:
Domain: vpc
NatGatewayEIP2:
DependsOn:
- VPCGatewayAttachment
Type: 'AWS::EC2::EIP'
Properties:
Domain: vpc
NatGatewayEIP3:
DependsOn:
- VPCGatewayAttachment
Type: 'AWS::EC2::EIP'
Properties:
Domain: vpc
PublicSubnet01:
Type: AWS::EC2::Subnet
Metadata:
Comment: Public Subnet 01
Properties:
MapPublicIpOnLaunch: true
AvailabilityZone:
Fn::Select:
- '0'
- Fn::GetAZs:
Ref: AWS::Region
CidrBlock:
Ref: PublicSubnet01Block
VpcId:
Ref: VPC
Tags:
- Key: Name
Value: !Sub "${VpcName}-PublicSubnet01"
- Key: kubernetes.io/role/elb
Value: 1
PublicSubnet02:
Type: AWS::EC2::Subnet
Metadata:
Comment: Public Subnet 02
Properties:
MapPublicIpOnLaunch: true
AvailabilityZone:
Fn::Select:
- '1'
- Fn::GetAZs:
Ref: AWS::Region
CidrBlock:
Ref: PublicSubnet02Block
VpcId:
Ref: VPC
Tags:
- Key: Name
Value: !Sub "${VpcName}-PublicSubnet02"
- Key: kubernetes.io/role/elb
Value: 1
PublicSubnet03:
Type: AWS::EC2::Subnet
Metadata:
Comment: Public Subnet 03
Properties:
MapPublicIpOnLaunch: true
AvailabilityZone:
Fn::Select:
- '2'
- Fn::GetAZs:
Ref: AWS::Region
CidrBlock:
Ref: PublicSubnet03Block
VpcId:
Ref: VPC
Tags:
- Key: Name
Value: !Sub "${VpcName}-PublicSubnet03"
- Key: kubernetes.io/role/elb
Value: 1
PrivateSubnet01:
Type: AWS::EC2::Subnet
Metadata:
Comment: Private Subnet 01
Properties:
AvailabilityZone:
Fn::Select:
- '0'
- Fn::GetAZs:
Ref: AWS::Region
CidrBlock:
Ref: PrivateSubnet01Block
VpcId:
Ref: VPC
Tags:
- Key: Name
Value: !Sub "${VpcName}-PrivateSubnet01"
- Key: kubernetes.io/role/internal-elb
Value: 1
PrivateSubnet02:
Type: AWS::EC2::Subnet
Metadata:
Comment: Private Subnet 02
Properties:
AvailabilityZone:
Fn::Select:
- '1'
- Fn::GetAZs:
Ref: AWS::Region
CidrBlock:
Ref: PrivateSubnet02Block
VpcId:
Ref: VPC
Tags:
- Key: Name
Value: !Sub "${VpcName}-PrivateSubnet02"
- Key: kubernetes.io/role/internal-elb
Value: 1
PrivateSubnet03:
Type: AWS::EC2::Subnet
Metadata:
Comment: Private Subnet 03
Properties:
AvailabilityZone:
Fn::Select:
- '2'
- Fn::GetAZs:
Ref: AWS::Region
CidrBlock:
Ref: PrivateSubnet03Block
VpcId:
Ref: VPC
Tags:
- Key: Name
Value: !Sub "${VpcName}-PrivateSubnet03"
- Key: kubernetes.io/role/internal-elb
Value: 1
PublicSubnet01RouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PublicSubnet01
RouteTableId: !Ref PublicRouteTable
PublicSubnet02RouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PublicSubnet02
RouteTableId: !Ref PublicRouteTable
PublicSubnet02RouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PublicSubnet03
RouteTableId: !Ref PublicRouteTable
PrivateSubnet01RouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PrivateSubnet01
RouteTableId: !Ref PrivateRouteTable01
PrivateSubnet02RouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PrivateSubnet02
RouteTableId: !Ref PrivateRouteTable02
PrivateSubnet03RouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PrivateSubnet03
RouteTableId: !Ref PrivateRouteTable03
ControlPlaneSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Cluster communication with worker nodes
VpcId: !Ref VPC
WorkerNodeSshSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: SG for ssh access to worker nodes in managed nodegroup
VpcId: !Ref VPC
Outputs:
PublicSubnetIds:
Description: Public Subnets IDs in the VPC
Value: !Join [ ", ", [ !Ref PublicSubnet01, !Ref PublicSubnet02, !Ref PublicSubnet03 ] ]
PrivateSubnetIds:
Description: Private Subnets IDs in the VPC
Value: !Join [ ", ", [ !Ref PrivateSubnet01, !Ref PrivateSubnet02, !Ref PrivateSubnet03 ] ]
ControlPlaneSecurityGroups:
Description: Security group for the cluster control plane communication with worker nodes
Value: !Join [ ",", [ !Ref ControlPlaneSecurityGroup ] ]
WorkerNodeSshSecurityGroup:
Description: SG for ssh access to worker nodes in managed nodegroup
Value: !Ref WorkerNodeSshSecurityGroup
VpcId:
Description: The VPC Id
Value: !Ref VPC
IAM Roles Template:
Mappings:
ServicePrincipals:
aws-cn:
ec2: ec2.amazonaws.com.cn
aws-us-gov:
ec2: ec2.amazonaws.com
aws:
ec2: ec2.amazonaws.com
Resources:
eksClusterRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- eks.amazonaws.com
Action:
- sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AmazonEKSClusterPolicy
NodeInstanceRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- !FindInMap [ServicePrincipals, !Ref "AWS::Partition", ec2]
Action:
- "sts:AssumeRole"
ManagedPolicyArns:
- !Sub "arn:${AWS::Partition}:iam::aws:policy/AmazonEKSWorkerNodePolicy"
- !Sub "arn:${AWS::Partition}:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
Path: /
Outputs:
eksClusterRoleArn:
Description: The role that Amazon EKS will use to create AWS resources for Kubernetes clusters
Value: !GetAtt eksClusterRole.Arn
NodeInstanceRole:
Description: The node instance role
Value: !GetAtt NodeInstanceRole.Arn
EKS Cluster Template:
---
AWSTemplateFormatVersion: '2010-09-09'
Description: 'EKS Production-Grade Cluster'
Parameters:
KubernetesVersion:
Type: String
Description: The EKS supported Kubernetes version for your cluster
Default: 1.19
AllowedValues:
- 1.19
- 1.18
- 1.17
ClusterName:
Type: String
Description: The name of the cluster
ControlPlaneClusterRoleArn:
Type: String
Description: The eksClusterRole Arn to use for the eks cluster (control plane)
ControlPlaneSubnetIds:
Type: List<AWS::EC2::Subnet::Id>
Description: Register your public and private subnets with your EKS managed control plane
SecurityGroupIds:
Type: List<AWS::EC2::SecurityGroup::Id>
Description: The security group(s) for the cross-account elastic network interfaces that Amazon EKS creates to use to allow communication between your nodes and the Kubernetes control plane.
Resources:
EKSManagedControlPlane:
Type: AWS::EKS::Cluster
Properties:
KubernetesNetworkConfig:
ServiceIpv4Cidr: 172.16.0.0/12
Name: !Ref ClusterName
ResourcesVpcConfig:
SecurityGroupIds: !Ref SecurityGroupIds
SubnetIds: !Ref ControlPlaneSubnetIds
RoleArn: !Ref ControlPlaneClusterRoleArn
Version: !Ref KubernetesVersion
Managed Nodegroup Template:
---
AWSTemplateFormatVersion: '2010-09-09'
Description: 'EKS Production-Grade Nodegroup'
Parameters:
KubernetesVersion:
Type: String
Description: The EKS supported Kubernetes version for your cluster
Default: 1.19
AllowedValues:
- 1.19
- 1.18
- 1.17
ClusterName:
Type: String
Description: The name of the cluster
NodeInstanceRoleArn:
Type: String
Description: The NodeInstanceRole Arn to use for the eks nodegroup (managed data plane)
DataPlanePrivateSubnetIds:
Type: List<AWS::EC2::Subnet::Id>
Description: Private subnets for your Amazon EKS data plane nodes
WorkerNodeGroupName:
Type: String
Description: The name of the node group for the worker nodes in the data plane. Right now we are only supporting 1 node group per cluster.
WorkerNodesInstanceType:
Type: String
Description: The instance type for the worker nodes in the data plane. Right now we are only supporting 1 instance type for all worker nodes.
Default: t3.medium
AllowedValues:
- t3.small
- t3.medium
- t3.large
- m5.large
- m5.xlarge
- c5.large
- c5.xlarge
WorkerNodesEc2SshKey:
Type: AWS::EC2::KeyPair::KeyName
Description: The Amazon EC2 SSH key that provides access for SSH communication with the nodes in the managed node group
SourceSecurityGroupsForWorkerNodes:
Type: List<AWS::EC2::SecurityGroup::Id>
Description: The security groups that are allowed SSH access (port 22) to the nodes. If you specify an Amazon EC2 SSH key but do not specify a source security group when you create a managed node group, then port 22 on the nodes is opened to the internet.
Resources:
EKSManagedDataPlane:
Type: AWS::EKS::Nodegroup
Properties:
AmiType: AL2_x86_64
CapacityType: ON_DEMAND
ClusterName: !Ref ClusterName
ForceUpdateEnabled: false
InstanceTypes:
- !Ref WorkerNodesInstanceType
NodegroupName: !Ref WorkerNodeGroupName
NodeRole: !Ref NodeInstanceRoleArn
RemoteAccess:
Ec2SshKey: !Ref WorkerNodesEc2SshKey
SourceSecurityGroups: !Ref SourceSecurityGroupsForWorkerNodes
ScalingConfig:
DesiredSize: 3
MaxSize: 4
MinSize: 3
Subnets: !Ref DataPlanePrivateSubnetIds
Version: !Ref KubernetesVersion
For the EKS Cluster template, I use the eksClusterRole ARN for the EKS cluster parameter, and I pass in all 6 subnet ids (public and private) from the VPC template output when creating the cluster. The SG id I pass in from the VPC ControlPlaneSecurityGroups output field as well.
For the Managed Nodegroup template I only pass it the private subnet ids from the VPC output, the ssh security group id from the VPC output, and the NodeInstanceRole ARN. I have ensured that the cluster name matches what I have given for the EKS Cluster template when creating that.
I plan on configuring the VPC CNI plugin to use IAM Roles for Service accounts once the cluster and managed nodegroup are setup, which is why I have left the policy for the CNI off of that IAM role.
I have a Custom Lambda resource that inits my DB and then is supposed make the call to the presigned S3 url when done. It's initing the DB correctly but is timing out when making the call to S3. My guess is I did something wrong in my CloudFormation template that's causing that due to my limited networking knowledge. Would appreciate any help. Thank you in advance!
Trimmed down YAML:
AWSTemplateFormatVersion: 2010-09-09
Transform: "AWS::Serverless-2016-10-31"
Resources:
InternetGateway:
Type: "AWS::EC2::InternetGateway"
Properties:
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-InternetGateway
VPC:
Type: "AWS::EC2::VPC"
Properties:
CidrBlock: 10.0.0.0/16
EnableDnsSupport: "true"
EnableDnsHostnames: "true"
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-VPC
VPCGatewayAttachment:
Type: "AWS::EC2::VPCGatewayAttachment"
Properties:
VpcId: !Ref VPC
InternetGatewayId: !Ref InternetGateway
NATGateway:
Type: AWS::EC2::NatGateway
Properties:
AllocationId: !GetAtt ElasticIPAddress.AllocationId
SubnetId: !Ref PublicSubnet1
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-NATGateway
ElasticIPAddress:
Type: AWS::EC2::EIP
Properties:
Domain: VPC
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-EIP
PublicRouteTable:
Type: "AWS::EC2::RouteTable"
Properties:
VpcId: !Ref VPC
Tags:
- Key: Name
Value: Public
PublicRoute1:
Type: "AWS::EC2::Route"
Properties:
RouteTableId: !Ref PublicRouteTable
GatewayId: !Ref InternetGateway
DestinationCidrBlock: 0.0.0.0/0
DependsOn:
- InternetGateway
PublicSubnet1:
Type: "AWS::EC2::Subnet"
Properties:
VpcId: !Ref VPC
CidrBlock: 10.0.0.0/24
AvailabilityZone: !Select [0, !GetAZs ]
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-PublicSubnet1
CreateRDSDatabaseLambdaSG:
Type: "AWS::EC2::SecurityGroup"
Properties:
VpcId: !Ref VPC
GroupDescription: Allow Lambda to access RDS in same VPC
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-CreateRDSDatabaseLambdaSG
LambdaRDSCFNInit:
Type: AWS::Serverless::Function
DependsOn:
- InternetGateway
- VPC
- VPCGatewayAttachment
- NATGateway
- ElasticIPAddress
- PublicRouteTable
- PublicRoute1
- PublicSubnet1
- CreateRDSDatabaseLambdaSG
Properties:
CodeUri: CreateRDSDatabase/
Description: "Lambda function which will execute when this CFN template is created, updated or deleted"
Handler: app.createRDSDatabase
Runtime: nodejs12.x
Timeout: 300
VpcConfig:
SecurityGroupIds:
- !Ref CreateRDSDatabaseLambdaSG
SubnetIds:
- !Ref PublicSubnet1
Environment:
Variables:
RDS_ENDPOINT: !GetAtt RDSCluster.Endpoint.Address
RDS_DB_NAME: !Ref RDSDBName
RDS_USERNAME: !Ref RDSUserName
RDS_PASSWORD: !Ref RDSPassword
LambdaRDSCFNTrigger:
Type: Custom::ProvisionRDS
DependsOn: LambdaRDSCFNInit
Version: 1.0
Properties:
ServiceToken: !GetAtt LambdaRDSCFNInit.Arn
You are placing your lambda in PublicSubnet1. Thus, your lambda will not have internet connectivity despite your NAT or Internet gateway. You need to place your function in a private subnet, and configure your private subnet to use the NAT gateway. From docs:
To access private resources, connect your function to private subnets. If your function needs internet access, use network address translation (NAT). Connecting a function to a public subnet doesn't give it internet access or a public IP address.
Alternatively, use S3 VPC gateway endpoint and associate it with route tables of your public subnet. This way your function will access s3 using the gateway, rather then internet. No need for NAT nor private subnet in this case.
I have created this nested stack. I want to implement the same stack with {prod, dev, qa} environment. Like I want to up the same stack but it doesn't have any name conflicts with each other. I want to deploy the same stack in three different environment, What changes do I have to make to achieve it
Root:
---
AWSTemplateFormatVersion: 2010-09-09
Parameters:
bucketname:
Type: String
Description: Path to the bucket
Default: webserver
bucketpath:
Type: String
Description: Path to the bucket
Default: /env #/mysql
Env:
Type: String
Description: Select the appropriate environment
AllowedValues:
- dev
- test
- uat
- prod
Cidr:
Type: String
Description: Cidr for vpc
Publicsubnet1:
Type: String
Description: public subnet 1
Publicsubnet2:
Type: String
Description: public subnet 2
Privatesubnet1:
Type: String
Description: Private subnet 1
Privatesubnet2:
Type: String
Description: Private subnet 2
Resources:
Vpcstack:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: !Sub "https://${bucketname}.s3.us-east-2.amazonaws.com${bucketpath}/vpc.yml"
Parameters:
Env: Ref: Env
Cidr: !Ref Cidr
Publicsubnet1: !Ref Publicsubnet1
Publicsubnet2: !Ref Publicsubnet2
Privatesubnet1: !Ref Privatesubnet1
Privatesubnet2: !Ref Privatesubnet2
Vpc:
---
AWSTemplateFormatVersion: 2010-09-09
Parameters:
Cidr:
Type: String
Description: Cidr for vpc
Publicsubnet1:
Type: String
Description: public subnet 1
Publicsubnet2:
Type: String
Description: public subnet 2
Privatesubnet1:
Type: String
Description: Private subnet 1
Privatesubnet2:
Type: String
Description: Private subnet 2
Env:
Type: String
Description: Select the appropriate environment
Resources:
VPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: !Ref Cidr
EnableDnsSupport: true
EnableDnsHostnames: true
InstanceTenancy: default
InternetGateway:
Type: AWS::EC2::InternetGateway
VPCGatewayAttachment:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
VpcId: !Ref VPC
InternetGatewayId: !Ref InternetGateway
SubnetA:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: us-east-2a
VpcId: !Ref VPC
CidrBlock: !Ref Publicsubnet1
MapPublicIpOnLaunch: true
SubnetB:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: us-east-2b
VpcId: !Ref VPC
CidrBlock: !Ref Publicsubnet2
MapPublicIpOnLaunch: true
SubnetC:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: us-east-2a
VpcId: !Ref VPC
CidrBlock: !Ref Privatesubnet1
MapPublicIpOnLaunch: false
SubnetD:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: us-east-2b
VpcId: !Ref VPC
CidrBlock: !Ref Privatesubnet2
MapPublicIpOnLaunch: false
RouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref VPC
RouteTable2:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref VPC
InternetRoute:
Type: AWS::EC2::Route
DependsOn: VPCGatewayAttachment
Properties:
DestinationCidrBlock: 0.0.0.0/0
GatewayId: !Ref InternetGateway
RouteTableId: !Ref RouteTable
SubnetARouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref RouteTable
SubnetId: !Ref SubnetA
SubnetBRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref RouteTable
SubnetId: !Ref SubnetB
SubnetCRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref RouteTable2
SubnetId: !Ref SubnetC
SubnetDRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref RouteTable2
SubnetId: !Ref SubnetD
SecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: "Internet Group"
GroupDescription: "SSH traffic in, all traffic out."
VpcId: !Ref VPC
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: "22"
ToPort: "22"
CidrIp: 0.0.0.0/0
SecurityGroupEgress:
- IpProtocol: -1
CidrIp: 0.0.0.0/0
NAT:
Type: AWS::EC2::NatGateway
Properties:
AllocationId:
Fn::GetAtt:
- EIP
- AllocationId
SubnetId:
Ref: SubnetA
Tags:
- Key: Name
Value: !Sub "nat-${Env}"
EIP:
DependsOn: VPCGatewayAttachment
Type: AWS::EC2::EIP
Properties:
Domain: VPC
Route:
Type: AWS::EC2::Route
Properties:
RouteTableId:
Ref: RouteTable2
DestinationCidrBlock: 0.0.0.0/0
NatGatewayId:
Ref: NAT
Outputs:
VpcID:
Description: VPC id
Value: !Ref VPC
Export:
Name: "VpcID"
SubnetA:
Description: public subnet
Value: !Ref SubnetA
Export:
Name: "SubnetA"
SubnetB:
Description: public subnet 2
Value: !Ref SubnetB
Export:
Name: "SubnetB"
SubnetC:
Description: priavte subnet
Value: !Ref SubnetC
Export:
Name: "SubnetC"
SubnetD:
Description: private subnet 2
Value: !Ref SubnetD
Export:
Name: "SubnetD"
You can specify different names for your top-level stacks by adding the environment to the top-level stack name. You do this at the time of stack creation, via the console or programmatically.
Then when each top-level environment-specific stack runs it will create the necessary nested stacks without name conflicts. You won't be able to control the stack names for the nested stacks, but you can get the name using outputs.
See the following:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-stack.html
You can add output values from a nested stack within the containing
template. You use the GetAtt function with the nested stack's logical
name and the name of the output value in the nested stack in the
format Outputs.NestedStackOutputName.
CloudFormation nested stack name
If you need to use different resource values for the different environments, then you can use mappings to specify the settings that correspond to the selected environment. Here is an example of mappings:
Mappings:
EnvTypeMap:
prod:
vpc: vpc-a6842gb0
subnet: subnet-hjk23553
dev:
vpc: vpc-b7742gb0
subnet: subnet-abc23553
qa:
vpc: vpc-c2542gb0
subnet: subnet-uio23553
Then to reference one of these mapping values you would do so like this:
VpcId:
Fn::FindInMap:
- EnvTypeMap
- Ref: Env
- vpc
CF stack is identified by a stack-name. All you have to do is to specify this stack-name when you are deploying the CF template.
aws cloudformation deploy --stack-name <value> --template-file <value> ...
If you specify the name of an existing stack, that stack will be updated. If you specify a new name, you will create a new stack from a given template.
You can create as many stacks as you want from a singe template by choosing new stack name each time. You do not need to worry about naming conflicts because each resource's name in a given stack is uniquely identified based on the stack name which will be different.
aws cloudformation deploy --stack-name dev --template-file the-same-template.yaml ...
aws cloudformation deploy --stack-name test --template-file the-same-template.yaml ...
aws cloudformation deploy --stack-name uat --template-file the-same-template.yaml ...
aws cloudformation deploy --stack-name prod --template-file the-same-template.yaml ...
This will create 4 separate stacks (dev, test, uat, prod).
Note that since you are hardcoding IP address ranges, resources in these stacks will not be able to communicate between each other because of overlapping networks (which probably is what you want anyway) but if for some reason you need these to communicate, you will need to create Parameters for CIDR blocks (VPC, subnets) as well.
I have a CloudFormation template that creates a custom VPC.
The template creates the following resources - a VPC, an Internet Gateway, attaches the IGW to the VPC, and creates a Public Subnet.
I want to add a route (destination 0.0.0.0/0, target IGW) to the Route Table that gets created as part of the VPC.
I have read through the cloudformation documentation for routes, route tables to figure out how to do this, but to no avail.
I can use the Fn::Ref function to refer to resources or parameters that are explicitly created as part of the template, but how do I refer to resources that get created inherently with the VPC?
Any insights on how to re-use the existing route table, NACL and Security Group are much appreciated.
Thanks,
Good job so far - you have your internet gateway, route table, and a public subnet. Now you need to create the route and attach the route table to the subnet if you haven't already done so. If you're using YAML it might look something like this:
InternetGateway:
Type: AWS::EC2::InternetGateway
Properties:
Tags:
- Key: Name
Value: !Ref EnvironmentName
InternetGatewayAttachment:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
InternetGatewayId: !Ref InternetGateway
VpcId: !Ref VPC
PublicSubnet1:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VPC
AvailabilityZone: !Select [ 0, !GetAZs '' ]
CidrBlock: !Ref PublicSubnet1CIDR
MapPublicIpOnLaunch: true
Tags:
- Key: Name
Value: !Sub ${EnvironmentName} Public Subnet (AZ1)
PublicRouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref VPC
Tags:
- Key: Name
Value: !Sub ${EnvironmentName} Public Routes
DefaultPublicRoute:
Type: AWS::EC2::Route
DependsOn: InternetGatewayAttachment
Properties:
RouteTableId: !Ref PublicRouteTable
DestinationCidrBlock: 0.0.0.0/0
GatewayId: !Ref InternetGateway
PublicSubnet1RouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId: !Ref PublicRouteTable
SubnetId: !Ref PublicSubnet1
Don't use the default route table (see https://serverfault.com/questions/588904/aws-vpc-default-route-table-in-cloudformation)
You can get default security group as per https://serverfault.com/questions/544439/aws-cloudformation-vpc-default-security-group
And finally you can also get the DefaultNetworkAcl in the same as DefaultSecurityGroup above. See also https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-vpc.html)
I have Lambda which was deployed to VPC.
This deploymens has next configs:
VPC (192.168.0.0/16)
Public Subnet A (192.168.32.0/20) has NAT Gateway and Route 0.0.0.0/0 to Internet Gateway
Private Subnet A (192.168.48.0/20) has Route 0.0.0.0/0 to NAT Gateway
Private Subnet B (192.168.64.0/20)
Lambda has own Securiy Group and references to "Private Subnet A" and "Private Subnet B"
I have strange problem: time to time Lambda doesn't have Internet Access. 3rd party service works normal.
One more strange thing that Lambda gets IP's like 127.0.0.1, 169.254.76.13, 169.254.79.1 instead of IP's from Subnets (192.168.48.0/20 and 192.168.64.0/20).
Error:
Error: connect ETIMEDOUT x.x.x.x:443
at Object._errnoException (util.js:1022:11)
at _exceptionWithHostPort (util.js:1044:20)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1198:14)
Deployment schema:
Here full CloudFormation template:
---
AWSTemplateFormatVersion: '2010-09-09'
Description: 'Base Infrastructure'
Metadata:
'AWS::CloudFormation::Interface':
ParameterGroups:
- Label:
default: 'VPC Parameters'
Parameters:
- VpcId
- InternetGatewayId
Parameters:
VpcId:
Type: String
InternetGatewayId:
Type: String
Resources:
SubnetAPublic:
Type: 'AWS::EC2::Subnet'
Properties:
AvailabilityZone: !Select [0, !GetAZs '']
CidrBlock: !Sub '192.168.32.0/20'
MapPublicIpOnLaunch: true
VpcId: !Sub '${VpcId}'
SubnetAPrivate:
Type: 'AWS::EC2::Subnet'
Properties:
AvailabilityZone: !Select [0, !GetAZs '']
CidrBlock: !Sub '192.168.48.0/20'
VpcId: !Sub '${VpcId}'
SubnetBPrivate:
Type: 'AWS::EC2::Subnet'
Properties:
AvailabilityZone: !Select [1, !GetAZs '']
CidrBlock: !Sub '192.168.64.0/20'
VpcId: !Sub '${VpcId}'
RouteTablePublic:
Type: 'AWS::EC2::RouteTable'
Properties:
VpcId: !Sub '${VpcId}'
RouteTablePrivate:
Type: 'AWS::EC2::RouteTable'
Properties:
VpcId: !Sub '${VpcId}'
RouteTableBPrivate:
Type: 'AWS::EC2::RouteTable'
Properties:
VpcId: !Sub '${VpcId}'
RouteTableAssociationAPublic:
Type: 'AWS::EC2::SubnetRouteTableAssociation'
Properties:
SubnetId: !Ref SubnetAPublic
RouteTableId: !Ref RouteTablePublic
RouteTableAssociationAPrivate:
Type: 'AWS::EC2::SubnetRouteTableAssociation'
Properties:
SubnetId: !Ref SubnetAPrivate
RouteTableId: !Ref RouteTablePrivate
RouteTableAssociationBPrivate:
Type: 'AWS::EC2::SubnetRouteTableAssociation'
Properties:
SubnetId: !Ref SubnetBPrivate
RouteTableId: !Ref RouteTableBPrivate
EIP:
Type: 'AWS::EC2::EIP'
Properties:
Domain: vpc
NatGateway:
Type: 'AWS::EC2::NatGateway'
Properties:
AllocationId: !GetAtt 'EIP.AllocationId'
SubnetId: !Ref SubnetAPublic
RouteTablePublicInternetRoute:
Type: AWS::EC2::Route
Properties:
RouteTableId: !Ref RouteTablePrivate
DestinationCidrBlock: '0.0.0.0/0'
NatGatewayId: !Ref NatGateway
RouteTablePublicInternetRoute2:
Type: AWS::EC2::Route
Properties:
RouteTableId: !Ref RouteTablePublic
DestinationCidrBlock: '0.0.0.0/0'
GatewayId: !Sub '${InternetGatewayId}'
ServerlessSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: SecurityGroup for Serverless Functions
VpcId: !Sub '${VpcId}'
ServerlessSecurityGroupIngress:
Type: AWS::EC2::SecurityGroupIngress
Properties:
GroupId: !Ref ServerlessSecurityGroup
IpProtocol: -1
SourceSecurityGroupId: !Ref ServerlessSecurityGroup
Any ideas what I do wrong?
P.S.: I found similar issues AWS VPC Lambda Function keeps losing internet access and Why AWS lambda functions In a VPC sometimes timeout and sometimes work fine?
UPD
Added Route and it works now
RouteTableBPrivateInternetRoute:
Type: AWS::EC2::Route
Properties:
RouteTableId: !Ref RouteTableBPrivate
DestinationCidrBlock: '0.0.0.0/0'
NatGatewayId: !Ref NatGateway
For an AWS Lambda function running inside a VPC to be able to access resources outside the VPC (such as the Internet), it must be in a private subnet with a NAT gateway. In your instance Private Subnet A is the only subnet with the appropriate configuration to allow a Lambda function to access the Internet. So you need to edit your Lambda function's configuration to only run in that subnet.
Have you selected your public subnet as one of the subnets your lambda will run in?
A lambda can only access the internet when it is running in private subnets, so go to your lambda page on the AWS Console, deselect the public subnet and save and the issue shouldn't re-occur.