I am looking to launch a subnet in the 'a' or 'b' availability zone of any region I specify - is there a way I can allow terraform to do this?
Something along the lines of this:
variable "az" {
default = {
a = 1
b = 2
}
}
resource "aws_subnet" "example" {
vpc_id = aws_vpc.vpc.id
availability_zone = var.az.(places subnets in either of the specified AZs)
}
The accepted answer is a fine answer for the question as posed. I'm writing this answer to show an alternative that doesn't directly solve the problem as posed but would achieve an equivalent result in any region where you have access to both the a and b AZs:
data "aws_availability_zones" "all" {
}
locals {
sorted_availability_zones = sort(data.aws_availability_zones.all.names)
selected_availability_zones = toset([
local.sorted_availability_zones[0],
local.sorted_availability_zones[1],
])
}
resource "aws_subnet" "example" {
for_each = local.selected_availability_zones
vpc_id = aws_vpc.vpc.id
availability_zone = each.value
}
The above uses the aws_availability_zones data source to find all of the zones your account has access to in the current region, then sorts them lexically so that they will be in a predictable order, and finally takes the first two items from that sorted list. Assuming you have access to the a and b zones this will always pick those two, because (per the current AZ naming scheme) they will always sort first in the list.
You could simply take the current region name, and append the a or b characters to it, like so:
data "aws_region" "current" {}
locals {
az1 = "${data.aws_region.current.name}a"
az2 = "${data.aws_region.current.name}b"
}
Related
Trying to create GCP subnetwork for 2 regions using terraform module. use case will be we may create either of the region subnet one time or both regions subnets along with secondary ranges (not all the time and can be more than one secondary range within same subnet). Like,
one region (from 2 regions) subnet at a time without secondary range.
both regions subnet at a time without secondary range.
one region (from 2 regions) subnet at a time with one or more secondary range.
both regions subnet at a time with one or more secondary ranges.
below code hasrestriction to use count and for_each on same resource. planning to add another resource block for central region as well in same main.tf
**main.tf file**
resource "google_compute_subnetwork" "subnet_east4" {
count = "${var.npe_subnet_east4_enable ? 1 : 0}"
name = "${var.npe_subnet_name_east4}"
ip_cidr_range = "${var.npe_cidr_range_east4}"
network = "${var.npe_vpc_name_east4}"
region = "${var.npe_region_east4}"
private_ip_google_access = "true"
for_each = var.npe_secondary_subnets_east4
secondary_ip_range {
range_name = each.value["npe_east4_subnet_pod_range_name"]
ip_cidr_range = each.value["npe_east4_subnet_pod_ip_cidr_range"]
}
secondary_ip_range {
range_name = each.value["npe_east4_subnet_service_range_name"]
ip_cidr_range = each.value["npe_east4_subnet_service_ip_cidr_range"]
}
**variable.tf file**
variable "npe_secondary_subnets_east4" {
type = map(object({
npe_east4_subnet_pod_range_name = string
npe_east4_subnet_pod_ip_cidr_range = string
npe_east4_subnet_service_range_name = string
npe_east4_subnet_service_ip_cidr_range = string
}))
description = "Secondary subnets"
}
Please suggest some alternate logic here
count is not necessary, you can use a conditional expression in your for_each:
for_each = var.npe_subnet_east4_enable ? var.npe_secondary_subnets_east4 : {}
I'm trying to deploy an EKS cluster and everything seems to be fine except for one!
The facade module looks like this:
module "eks" {
source = "../../../../infrastructure_modules/eks"
## EKS ##
create_eks = var.create_eks
cluster_version = var.cluster_version
cluster_name = local.cluster_name
vpc_id = data.aws_vpc.this.id
subnets = data.aws_subnet_ids.this.ids
# note: either pass worker_groups or node_groups
# this is for (EKSCTL API) unmanaged node group
worker_groups = var.worker_groups
# this is for (EKS API) managed node group
node_groups = var.node_groups
## Common tag metadata ##
env = var.env
app_name = var.app_name
tags = local.eks_tags
region = var.region
}
The VPC id is retrieved through the following block :
data "aws_vpc" "this" {
tags = {
Name = "tagName"
}
}
Which then is used to retrieve the subnet_IDs as following:
data "aws_subnet_ids" "this" {
vpc_id = data.aws_vpc.this.id
}
Nevertheless, deploying this results in error stating:
Error: error creating EKS Cluster (data-layer-eks):
UnsupportedAvailabilityZoneException: Cannot create cluster
'data-layer-eks' because us-east-1e, the targeted availability zone,
does not currently have sufficient capacity to support the cluster.
Which is a well known error, and anybody can come across this for even EC2s.
I could solve this by simply hardcoding the subnet value, but that's really undesirable and hardly maintainable.
So the question is, how can I filter out subnet_IDs based on availability zones that have sufficient capacity?
First you need to collect the subnets with all of their attributes:
data "aws_subnets" "this" {
filter {
name = "vpc-id"
values = [data.aws_vpc.this.id]
}
}
data "aws_subnet" "this" {
for_each = toset(data.aws_subnets.this.ids)
id = each.value
}
data.aws_subnet.this is now a map(object) with all of the subnets and their attributes. You can now filter by availability zone accordingly:
subnets = [for subnet in data.aws_subnet.this : subnet.id if subnet.availability_zone != "us-east-1e"]
You can also filter by truthy conditionals if that condition is easier for you:
subnets = [for subnet in data.aws_subnet.this : subnet.id if contains(["us-east-1a", "us-east-1b", "us-east-1c", "us-east-1d"], subnet.availability_zone)]
It depends on your personal use case.
I'm wondering if anyone can help me with the following I have a base resource to create aws subnets
resource aws_subnet subnet {
vpc_id = var.vpc_id
cidr_block = var.cidr_block
}
output subnetId {
value = aws_subnet.subnet.id
}
module private_subnet {
source = "linktoresourcedetailedabove"
for_each = var.privateSubnet
vpd.id = var.vpc_id
cidr_block = each.value.cidr_block
}
I have a module which calls using a for_each loop based on a var based in, my question is this resource might be called 10 times and I want to store each id and then access this from another module but I seem to be hitting issues here, I tried updating aws_subnet.subnet.id to aws_subnet.subnet.*.id but am still not having anyluck and can't seem to find anything out there that can help me.
If your private_subnet modules has output
output subnetId {
value = aws_subnet.subnet.id
}
then once you create your private_subnet modules, you can get the list of all subnetId creates as:
values(module.private_subnet)[*].subnetId
I am new to terraform. I want to create single public subnet and three private subnet per availability zone in particular region in AWS using terraform. I am able to create one private and public subnet per availability zone by referring the following link https://medium.com/#maneetkum/create-subnet-per-availability-zone-in-aws-through-terraform-ea81d1ec1883. However I need to split the one private subet created into another 2. Is that possible in terraform?
data "aws_availability_zones" "available" {}resource "aws_vpc" "myVpc" {
cidr_block = "10.20.0.0/16"
enable_dns_hostnames = true
tags {
Name = "myVpc"
}
}
resource "aws_subnet" "public_subnet" {
count = "${length(data.aws_availability_zones.available.names)}"
vpc_id = "${aws_vpc.myVpc.id}"
cidr_block = "10.20.${10+count.index}.0/24"
availability_zone = "${data.aws_availability_zones.available.names[count.index]}"
map_public_ip_on_launch = true
tags {
Name = "PublicSubnet"
}
}
resource "aws_subnet" "private_subnet" {
count = "${length(data.aws_availability_zones.available.names)}"
vpc_id = "${aws_vpc.myVpc.id}"
cidr_block = "10.20.${20+count.index}.0/24"
availability_zone= "${data.aws_availability_zones.available.names[count.index]}"
map_public_ip_on_launch = false
tags {
Name = "PrivateSubnet"
}
}
The above code is used to create the one private and public subnet per availability zone.
It is possible to create multiple subnets, and automatically place them into an Availability Zone, without duplicating code. Let's keep things DRY. To avoid duplicating the code, use the magic of Terraform meta-arguments and built-in functions. Specifically, use "count" and "cidrsubnet". The "count", will generate as many copies of your subnet as you want.
If you want to provide each subnet with unique values, such as the subnet's tag Name, you can give each subnet a unique, and mnemonic name, by creating a data dictionary with the names that you want assigned to each of the subnets. Then assign them as you create the subnets, by using "count.index". IF that is too much work, you could also just embed the count.index in the name as well.
Different regions have a different number of Availability Zones. To make sure that you are assigning your subnets to Availability Zones that actually exit, you should generate the list of the Availability Zones dynamically. This way, you know that all of the Availability Zones, that are in the list, are actually available in the region that you are working.
What happens if you have more subnets then the region has Availability Zones? Use modulo arithmetic to wrap your working index. Rather than using the index.count directly, do a modulo on the index.count, using the length of the list. This will wrap the index around, so your working index never overflows the length of the list of Availability Zones.
But, the real magic is "cidrsubnet" command. The example below will take the size of the base CIDR block that is passed (which happens to be a /16), add the 2nd parameter (4), and generate a /20 CIDR block. The third parameter indexes through the available CIDR block, thus ensuring that each subnet gets a different sub-CIDR block.
Note: The related cidrsubnets command is quite a bit different. So, be careful, and don't get the two functions confused.
resource "aws_subnet" "area_subnets" {
count = 4 # creates four subnets
vpc_id = var.area_vpc_id
map_public_ip_on_launch = var.map_public_ip_on_launch
cidr_block = cidrsubnet(var.area_subnet_cidr, 4, count.index)
availability_zone_id = data.aws_availability_zones.available.zone_ids[count.index % length(data.aws_availability_zones.available.zone_ids)]
tags = tomap({ "Name" = "${var.subnet_names[count.index]}" })
}
variable "subnet_names" {
type = list(string)
default = [
"Primary NAT Gateway Subnet",
"Secondary NAT Gateway Subnet",
"Channel A Subnet",
"Channel B Subnet"
]
}
variable "map_public_ip_on_launch" {
type = bool
default = true
}
variable "area_vpc_id"
documentation = "The Terraform ID of the containing VPC"
type = string
default = "vpc-abcdefghijklmno"
}
variable "area_subnet_cidr"
documentation = "The base CIDR that you are working with"
type = string
default = "10.0.0.0/16"
}
data "aws_availability_zones" "available" {
state = "available"
filter { # Only fetch Availability Zones (no Local Zones)
name = "opt-in-status"
values = ["opt-in-not-required"]
}
}
You can simply duplicate the private_subnet resource element to create the two new subnets in each AZ:
...
resource "aws_subnet" "private_subnet" {
count = "${length(data.aws_availability_zones.available.names)}"
vpc_id = "${aws_vpc.myVpc.id}"
cidr_block = "10.20.${20+count.index}.0/24"
availability_zone= "${data.aws_availability_zones.available.names[count.index]}"
map_public_ip_on_launch = false
tags {
Name = "PrivateSubnet"
}
}
resource "aws_subnet" "private_subnet_2" {
count = "${length(data.aws_availability_zones.available.names)}"
vpc_id = "${aws_vpc.myVpc.id}"
cidr_block = "10.30.${20+count.index}.0/24"
availability_zone= "${data.aws_availability_zones.available.names[count.index]}"
map_public_ip_on_launch = false
tags {
Name = "PrivateSubnet2"
}
}
You will need to modify the CIDR blocks for each subnet to make sure they don't overlap with each other.
The VPC I'm working on has 3 logical tiers: Web, App and DB. For each tier there is one subnet in each availability zone. Total of 6 subnets in the region I'm using.
I'm trying to create EC2 instances using a module and the count parameter but I don't know how to tell terraform to use the two subnets of the App tier. An additional constraint I have is to use static IP addresses (or a way to have a deterministic private name)
I'm playing around with the resource
resource "aws_instance" "app_server" {
...
count = "${var.app_servers_count}"
# Not all at the same time, though!
availability_zone = ...
subnet_id = ...
private_ip = ...
}
Things I've tried/thought so far:
Use data "aws_subnet" "all_app_subnets" {...}, filter by name, get all the subnets that match and use them as a list. But aws_subnet cannot return a list;
Use data "aws_availability_zones" {...} to find all the zones. But I still have the problem of assigning the correct subnet;
Use data "aws_subnet_ids" {...} which looks like the best option. But apparently it doesn't have a filter option to match the networks namel
Pass the subnets IDs as list of strings to the module. But I don't want to hard code the IDs, it's not automation;
Hard code the subnets as data "aws_subnet" "app_subnet_1" {...}, data "aws_subnet" "app_subnet_2" {...} but then I have to use separate sets of variables for each subnet which I don't like;
Get information for each subnet like in the point above but then create a map to access it as a list. But it's not possibile to use interpolation in variables definition;
Not using modules and hard-code each instance for each environment. Mmmm... really?
I really ran out of ideas. It seems that nobody has to deploy instances in specific subnetworks and keep a good degree of abstration. I see only examples where subnetworks are not specified or where people just use default values for everything. Is this really something so unusual?
Thanks in advance to everyone.
It is possible to evenly distribute instances across multiple zones using modulo.
variable "zone" {
description = "for single zone deployment"
default = "europe-west4-b"
}
variable "zones" {
description = "for multi zone deployment"
default = ["europe-west4-b", "europe-west4-c"]
}
resource "google_compute_instance" "default" {
count = "${var.role.count}"
...
zone = "${var.zone != "" ? var.zone: var.zones[ count.index % length(var.zones) ]}"
...
}
This distribution mechanism allow to distribute nodes evenly across zones.
E.g. zones = [A,B] - instance-1 will be in A, instance-2 will in B, instance-3 will be in A again.
By adding zone C to zones will shift instance-3 to C.
The count index in the resource will throw an error if you have more instances than subnets. Use the element interpolation from Terraform
element(list, index) - Returns a single element from a list at the given index. If the index is greater than the number of elements, this function will wrap using a standard mod algorithm. This function only works on flat lists.
subnet_id = "${element(data.aws_subnet_ids.app_tier_ids.ids, count.index)}"
At the end I figured out how to do it, using data "aws_subnet_ids" {...} and more importantly understanding that terraform creates lists out of resources when using count:
variable "target_vpc" {}
variable "app_server_count" {}
variable "app_server_ip_start" {}
# Discover VPC
data "aws_vpc" "target_vpc" {
filter = {
name = "tag:Name"
values = [var.target_vpc]
}
}
# Discover subnet IDs. This requires the subnetworks to be tagged with Tier = "AppTier"
data "aws_subnet_ids" "app_tier_ids" {
vpc_id = data.aws_vpc.target_vpc.id
tags {
Tier = "AppTier"
}
}
# Discover subnets and create a list, one for each found ID
data "aws_subnet" "app_tier" {
count = length(data.aws_subnet_ids.app_tier_ids.ids)
id = data.aws_subnet_ids.app_tier_ids.ids[count.index]
}
resource "aws_instance" "app_server" {
...
# Create N instances
count = var.app_server_count
# Use the "count.index" subnet
subnet_id = data.aws_subnet_ids.app_tier_ids.ids[count.index]
# Create an IP address using the CIDR of the subnet
private_ip = cidrhost(element(data.aws_subnet.app_tier.*.cidr_block, count.index), var.app_server_ip_start + count.index)
...
}
I get Terraform to loop through the subnets in an availability zone by using the aws_subnet_ids data source and filtering by a tag representing the tier (in my case public/private).
This then looks something like this:
variable "vpc" {}
variable "ami" {}
variable "subnet_tier" {}
variable "instance_count" {}
data "aws_vpc" "selected" {
tags {
Name = "${var.vpc}"
}
}
data "aws_subnet_ids" "selected" {
vpc_id = "${data.aws_vpc.selected.id}"
tags {
Tier = "${var.subnet_tier}"
}
}
resource "aws_instance" "instance" {
count = "${var.instance_count}"
ami = "${var.ami}"
subnet_id = "${data.aws_subnet_ids.selected.ids[count.index]}"
instance_type = "${var.instance_type}"
}
This returns a consistent sort order but not necessarily starting with AZ A in your account. I suspect that the AWS API returns the subnets in AZ order but ordered by their own internal id as the AZs are shuffled by account (presumably to stop AZ A being flooded as humans are predictably bad at putting everything in the first place they can use).
You would have to tie yourself in some horrible knots if for some odd reason you particularly care about instances being placed in AZ A first but this minimal example should at least get instances being round-robined through the AZs you have subnets in by relying on Terraform's looping back through arrays when exceeding the array length.