Configure Multiple NAT gateways - amazon-web-services

I have two public & private subnets across two AZs. I want to create a NAT in both public subnets. How do I route my private route table to use both NATs? Case three in this accepted answer. I know it's possible but I'm unsure on how to implement it. Should the destination cidr_block be the public subnet's cidr block? I also tried 0.0.0.0/0 and errored out.
resource "aws_route" "r" {
count = length(aws_nat_gateway.this)
route_table_id = aws_vpc.vpc.default_route_table_id
destination_cidr_block = "aws_subnet.public_subnets.${count.index}.cidr_block"
nat_gateway_id = "aws_nat_gateway.this.${count.index}.id"
depends_on = [aws_vpc.vpc,aws_subnet.public_subnets,aws_nat_gateway.this]
}
When I tried 0.0.0.0/0
error creating Route in Route Table (rtb-059eee10e310ade77) with destination (0.0.0.0/0): RouteAlreadyExists: The route identified by 0.0.0.0/0 already exists.

Short answer: you need to have a separate route table for each private subnet.
Long answer: I would start by defining a map that contains configuration for each availability zone. I avoid count in modern Terraform configs because it's too easy to cause incompatibilities down the road (especially if you're using it to index an array).
locals {
subnets = {
"1a" = { "public_cidr": "172.31.0.0/24", private_cidr: "172.31.1.0/24" }
"1b" = { "public_cidr": "172.31.2.0/24", private_cidr: "172.31.3.0/24" }
}
}
Now, you can use for_each to create public and private subnets for each of these availability zones:
resource "aws_subnet" "public" {
for_each = local.subnets
vpc_id = aws_vpc.example.id
cidr_block = each.value["public_cidr"]
tags = {
Name = "public-${each.key}"
}
}
resource "aws_subnet" "private" {
for_each = local.subnets
vpc_id = aws_vpc.example.id
cidr_block = each.value["private_cidr"]
tags = {
Name = "private-${each.key}"
}
}
You can associate the public subnets with the default route table (which must have a route to the Internet Gateway). Personally, I prefer creating my own public route table, rather than updating the default.
Next, you'll need to create a NAT Gateway in each public subnet:
resource "aws_nat_gateway" "example" {
depends_on = [aws_internet_gateway.example]
for_each = local.subnets
allocation_id = aws_eip.nat_ip[each.key].id
subnet_id = aws_subnet.public[each.key].id
tags = {
Name = "nat-${each.key}"
}
}
And now the important part: each private subnet needs its own route table, which references the NAT Gateway specific to its availability zone.
resource "aws_route_table" "private" {
for_each = local.subnets
vpc_id = aws_vpc.example.id
route {
cidr_block = "0.0.0.0/0"
nat_gateway_id = aws_nat_gateway.example[each.key].id
}
tags = {
Name = "private-${each.key}"
}
}
resource "aws_route_table_association" "private" {
for_each = local.subnets
subnet_id = aws_subnet.private[each.key].id
route_table_id = aws_route_table.private[each.key].id
}

Related

Dynamic routes in Terraform

My simplified setup is something like this:
2 AZs and each consists of 2 subnets (Subnet1-AZ-A, Subnet2-AZ-A, Subnet1-AZ-B, Subnet2-AZ-B)
I want to create routing table that will fetch CIDR blocks from Subnet1 (from both AZs) and create routes:
CIDR_BLOCK of Subnet-1-AZ-A -> VPC_ENDPOINT in Subnet-2-AZ-A
CIDR_BLOCK of Subnet 1-AZ-B -> VPC_ENDPOINT in Subnet-2-AZ-B
I don't want to hardcode values, because number of AZs (and subnets) may change.
Here's code that I'm using:
Subnet-1A
resource "aws_subnet" "Subnet1" {
count = length(var.az)
vpc_id = aws_vpc.app-vpc.id
cidr_block = cidrsubnet(var.vpc_cidr, 8, 4+count.index)
availability_zone = var.az[count.index]
}
VPC_Endpoints:
resource "aws_vpc_endpoint" "endpoint-spoke" {
count = length(var.az)
service_name = var.glbe_service_name
subnet_ids = [aws_subnet.endpoint-subnet[count.index].id]
vpc_endpoint_type = var.glbe_endpoint_type
vpc_id = aws_vpc.app-vpc.id
}
Route-Table
resource "aws_route_table" "IGWRT" {
vpc_id = aws_vpc.app-vpc.id
dynamic "route" {
for_each = aws_subnet.Subnet1[*].cidr_block
content {
cidr_block = route.value
vpc_endpoint_id = aws_vpc_endpoint.endpoint-spoke[*].id
}
}
}
The code in Route-Table is not working. I know aws_subnet.Subnet1[*].cidr_block will be a list that contains Subnet1 CIDR blocks for example: [10.0.0.0/20, 10.0.1.0/24], but how I can create dynamic routes where:
10.0.0.0/0 will be pointing to VPC_Endpoint in respective AZ
10.0.0.1/0 will be pointing to VPC_Endpoint in respective AZ
Update the dynamic block in the aws_route_table resource:
resource "aws_route_table" "IGWRT" {
vpc_id = aws_vpc.app-vpc.id
dynamic "route" {
for_each = aws_subnet.Subnet1
content {
cidr_block = route.value.cidr_block
vpc_endpoint_id = aws_vpc_endpoint.endpoint-spoke[route.key].id
}
}
}
However, I strongly recommend you to use a for_each instead of the count, if the elements of the list change (the variable az), it will affect all of the resources as their index will be updated.

AWS Route Table - Terraform | An argument named "gateway_id" is not expected here

I have an internet gateway and two subnets (public and private).
While i can attach my internet gateway to the public one, i can't on the private one. I add this line in the code:
gateway_id = aws_internet_gateway.wally_jeropa_igw.id
but i have this error:
An argument named "gateway_id" is not expected here.
This is my IGW
resource "aws_internet_gateway" "wally_jeropa_igw" {
vpc_id = aws_vpc.wally_jeropa.id
tags = {
Name = "Testn IGW"
}
}
Public subnet
resource "aws_route_table" "wally_jeropa_vpc_public_route_table" {
vpc_id = aws_vpc.wally_jeropa.id
gateway_id = aws_internet_gateway.wally_jeropa_igw.id
route {
cidr_block = "0.0.0.0/0"
}
tags = {
Name = "Public Subnet Route Table."
}
}
Private subnet
resource "aws_route_table" "wally_jeropa_vpc_private_route_table" {
vpc_id = aws_vpc.wally_jeropa.id
gateway_id = aws_internet_gateway.wally_jeropa_igw.id
tags = {
Name = "Private Subnet Route Table."
}
}
The gateway_id attribute must be inside the route block [1]
References:
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route_table#route

How to have AWS VPC subnets associate with one Route Table in Terraform?

I have a Terraform file that is supposed to create an AWS VPC remotely. In doing so, it would create 3 Public Subnets and 3 Private Subnets. There would then be one Route Table for each group of public and private subnets. My current configuration has it so that a route table is created for EACH subnet. I'm also having a similar issue with NAT Gateways and Public Subnets; I only want one NAT Gateway for all Public Subnets.
These are the pertinent blocks from my main.tf file:
//Creating Public Subnets
resource "aws_subnet" "publicsubnets" {
provider = aws.east
count = "${length(var.public_subnets)}"
vpc_id = aws_vpc.Main.id
cidr_block = "${var.public_subnets[count.index]}" //CIDR block of public subnets
availability_zone = "${var.availability_zones[count.index]}"
tags = {
Name = "publicsubnets"
}
}
//Creating Private Subnets
resource "aws_subnet" "privatesubnets" {
provider = aws.east
count = "${length(var.private_subnets)}"
vpc_id = aws_vpc.Main.id
cidr_block = "${var.private_subnets[count.index]}" //CIDR block of private subnets
availability_zone = "${var.availability_zones[count.index]}"
tags = {
Name = "privatesubnets"
}
}
//Creating Route Table for Public Subnets
resource "aws_route_table" "PublicRT" {
provider = aws.east
count = "${length(var.public_subnets)}"
vpc_id = aws_vpc.Main.id
route {
cidr_block = "IP_Addresses" //Traffic from Public Subnet reaches Internet via Internet Gateway
gateway_id = aws_internet_gateway.IGW.id
}
tags = {
Name = "publicRT"
}
}
//Creating Route Table for Private Subnet
resource "aws_route_table" "PrivateRT" {
provider = aws.east
count = "${length(var.private_subnets)}"
vpc_id = aws_vpc.Main.id
route {
cidr_block = "IP_Addresses" //Traffic from Private Subnet reaches Internet via NAT Gateway
nat_gateway_id = aws_nat_gateway.NATgw[count.index].id
}
tags = {
Name = "privateRT"
}
}
// Associate Public Subnets with Route Table
resource "aws_route_table_association" "PublicRTassociation" {
provider = aws.east
count = "${length(var.public_subnets)}"
subnet_id = aws_subnet.publicsubnets[count.index].id
route_table_id = aws_route_table.PublicRT[count.index].id
}
//Associate Private Subnets with Route Table
resource "aws_route_table_association" "PrivateRTassociation" {
provider = aws.east
count = "${length(var.private_subnets)}"
subnet_id = aws_subnet.privatesubnets[count.index].id
route_table_id = aws_route_table.PrivateRT[count.index].id
}
//Associate Elastic IP's to Public Subnets
resource "aws_eip" "nateIP" {
provider = aws.east
count = "${length(var.public_subnets)}"
vpc = true
tags = {
Name = "nateIP"
}
}
//Associate Public Subnets to NAT Gateways
resource "aws_nat_gateway" "NATgw" {
provider = aws.east
count = "${length(var.public_subnets)}"
allocation_id = aws_eip.nateIP[count.index].id
subnet_id = aws_subnet.publicsubnets[count.index].id
tags = {
Name = "NATgw"
}
}
I have provider and variables declared in separate files.
Note: I have changed some names for security/privacy reasons.
What should I do?
You are creating NATs in a loop. If you want to use single NAT gateway, you need to create only one NAT gateway resource (without a loop).
Something like below:
resource "aws_eip" "nateIP" {
provider = aws.east
vpc = true
tags = {
Name = "nateIP"
}
}
resource "aws_nat_gateway" "NATgw" {
provider = aws.east
allocation_id = aws_eip.nateIP.id
subnet_id = aws_subnet.publicsubnets[0].id
tags = {
Name = "NATgw"
}
}
Then, in your route table, you'd set NAT like below:
resource "aws_route_table" "PrivateRT" {
provider = aws.east
count = "${length(var.private_subnets)}"
vpc_id = aws_vpc.Main.id
route {
cidr_block = "IP_Addresses" //Traffic from Private Subnet reaches Internet via NAT Gateway
nat_gateway_id = aws_nat_gateway.NATgw.id
}
tags = {
Name = "privateRT"
}
}
However, having a single NAT has its downside. Your infrastructure would be reylying only on one availablity zone for NAT. In case there are issues with that Availability zone, your application may have issues.

Terraform - Multi Loops or Maps

I am creating few public subnets, private subnets, igw, nat, route_tables, and route_table entry in AWS using terraform. Below is the number of resources I am creating.
terraform.tfvars
vpc_cidr = "10.0.0.0/16"
public_subnet_count = 6
public_subnets_cidr = ["10.0.1.0/24","10.0.2.0/24","10.0.3.0/24","10.0.4.0/24","10.0.5.0/24", "10.0.6.0/24"]
availability_zones = ["us-east-2a", "us-east-2b","us-east-2c","us-east-2d"]
resources.tf
resource "aws_vpc" "vpc" {
cidr_block = var.vpc_cidr
enable_dns_hostnames = true
enable_dns_support = true
tags = {
Name = "${var.environment}-vpc"
Environment = var.environment
}
}
resource "aws_subnet" "public_subnet" {
vpc_id = "${aws_vpc.vpc.id}"
count = var.public_subnet_count
cidr_block = "${element(var.public_subnets_cidr, count.index)}"
availability_zone = length(var.availability_zones) > 1 ? var.availability_zones[count.index % length(var.availability_zones)] : var.availability_zones[0]
map_public_ip_on_launch = false
tags = {
Name = "${var.environment}-${element(var.availability_zones, count.index)}-public-subnet"
Environment = "${var.environment}"
}
}
resource "aws_internet_gateway" "ig" {
count = 1
vpc_id = aws_vpc.vpc.id
tags = {
Name = "${var.environment}-igw"
Environment = "${var.environment}"
}
}
resource "aws_route_table" "public" {
count = length(var.availability_zones)
vpc_id = aws_vpc.vpc.id
tags = {
Name = "${var.environment}-public-route-table"
Environment = "${var.environment}"
}
}
resource "aws_route" "public_internet_gateway" {
count = length(aws_route_table.public.*.id)
route_table_id = element(aws_route_table.public.*.id, count.index)
destination_cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.ig[0].id
}
resource "aws_route_table_association" "public" {
count = length(var.availability_zones)
subnet_id = element(aws_subnet.public_subnet.*.id, count.index)
route_table_id = element(aws_route_table.public.*.id, count.index)
}
Query in aws_route_table_association.public section.
How do I attach private subnet of a particular az to a route table id. For example if i use 6 private subnets in tfvars, it creates atleast 2 private subnets in one az (ex: us-east-2a). How do i loop and attach 2 subnet from us-east-2a to one route table created for us-east-2a. Kind of map between multiple subnets in one az to route table in that az.
This is to attach each az nat gateway to that az subnet for routing.
You don't need to create a route table per AZ, the route tables are linked to subnets. So, you can just create a single route table and then link all your public subnets to it:
resource "aws_route_table" "public" {
vpc_id = aws_vpc.vpc.id
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.ig.id
}
}
resource "aws_route_table_association" "public" {
count = var.public_subnet_count
subnet_id = aws_subnet.public_subnet[count.index].id
route_table_id = aws_route_table.public.id
}
I've included the single route into the route_table to reduce the amount of code, so you can delete the aws_route resource if you use this.
[Edit]
Sorry I just noticed your mention of private subnets, but attaching route tables follows the same principle, and you don't need a NAT gateway per AZ either (that's expensive!) so you could route all private subnets through a single NAT gateway with one route table.
If you really want 4 NAT gateways, then you could do something like:
resource "aws_route_table" "private" {
count = length(var.availability_zones)
vpc_id = aws_vpc.vpc.id
route {
cidr_block = "0.0.0.0/0"
nat_gateway_id = aws_nat_gateway.nat[count.index].id
}
}
resource "aws_route_table_association" "private" {
count = var.private_subnet_count
subnet_id = aws_subnet.private[count.index].id
route_table_id = aws_route_table.private[
count.index % length(var.availability_zones)
].id
}
Since you already use the modulus (%) operator to distribute the subnets, I think you'll get the same result doing it this way.

Terraform project to create VPC/IGW, VPN, and private subnet with NATGW - Connectivity Analyzer says no route from NATGW to IGW

https://github.com/phillhocking/aws-network-vpn/tree/1000
I have been trying to figure this out for quite some time, and I really am struggling at understanding why these components can't talk to each other. The VPC has an IGW which is necessary for an EIP for the NATGW, however, nothing on this subnet can get out to the public internet. Everything works fine over the VPN link, but the Connectivity Analyzer indicates there's no connectivity between the NATGW and the IGW as there is not a route - how would I route just the NATGW traffic to the IGW without something like a 0.0.0.0/0 route which already is assigned to the NATGW for this subnet?
Connectivity Analyzer - Route Table has no route to IGW
I know I am just missing something bonehead simple. Look at the repo for how the whole thing goes together, but here's the VPC module:
data "aws_availability_zones" "available" {
state = "available"
}
resource "aws_vpc" "main" {
cidr_block = var.cidr_block
enable_dns_hostnames = true
enable_dns_support = true
tags = {
Name = var.vpc_name
}
}
resource "aws_subnet" "dev" {
count = var.subnet_count
# This line is necessary to ensure that we pick availabiltiy zones that can launch any size ec2 instance
availability_zone = data.aws_availability_zones.available.names[0]
vpc_id = aws_vpc.main.id
cidr_block = cidrsubnet(var.cidr_block, 6, count.index * 2 + 1)
tags = {
Name = "dev-subnet-${count.index}"
}
}
resource "aws_network_acl" "dev" {
vpc_id = aws_vpc.main.id
subnet_ids = aws_subnet.dev[*].id
ingress {
protocol = -1
rule_no = 1000
action = "allow"
#cidr_block = var.prem_network_address_space
cidr_block = "0.0.0.0/0"
from_port = 0
to_port = 0
}
egress {
protocol = -1
rule_no = 100
action = "allow"
cidr_block = "0.0.0.0/0"
from_port = 0
to_port = 0
}
tags = {
Name = "dev-acl"
}
}
# Gateways
resource "aws_internet_gateway" "gw" {
vpc_id = aws_vpc.main.id
tags = {
Name = "${var.vpc_name}-internet-gateway"
}
}
resource "aws_eip" "nat-gw" {
vpc = true
tags = {
Name = "nat-elastic-ip"
}
depends_on = [aws_internet_gateway.gw]
}
resource "aws_nat_gateway" "gw" {
allocation_id = aws_eip.nat-gw.id
subnet_id = aws_subnet.dev[0].id
tags = {
Name = "${var.vpc_name}-nat-gateway-dev"
}
depends_on = [aws_internet_gateway.gw]
}
# VPC Route Table
resource "aws_default_route_table" "default" {
default_route_table_id = aws_vpc.main.main_route_table_id
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.gw.id
}
tags = {
Name = "${var.vpc_name}-public"
}
depends_on = [aws_internet_gateway.gw]
}
# dev Subnet Route Table
resource "aws_route_table" "dev" {
vpc_id = aws_vpc.main.id
tags = {
Name = "dev-route-table"
}
}
resource "aws_route_table_association" "dev_routes" {
subnet_id = aws_subnet.dev[0].id
route_table_id = aws_route_table.dev.id
depends_on = [aws_nat_gateway.gw]
}
resource "aws_route" "dev_nat" {
route_table_id = aws_route_table.dev.id
destination_cidr_block = "0.0.0.0/0"
nat_gateway_id = aws_nat_gateway.gw.id
depends_on = [aws_nat_gateway.gw]
You are placing NAT in aws_subnet.dev[0], then you create aws_route_table.dev that you attach to the subnet. What's more the aws_route_table.dev has a route aws_route.dev_nat which points to the NAT.
So basically you are doing some circular routing - all traffic in aws_subnet.dev[0] is direct to NAT in the same subnet, which in turn is directed again to the same NAT.
As you pointed in comment, NAT should be in public subnet, while subnet directing traffic to NAT should be private subnets.