I am trying to create multiple packet mirror resources, using for_each
However in GCP packet mirror policy is restricted to only 5 subnets per policy
Now I am stumped how I can create multiple packet mirror policies referencing lets say from variable mirror_vpc_subnets below
variable "mirror_vpc_subnets" {
description = "Mirror VPC Subnets list to be mirrored."
type = list(string)
default = []
}
Now the objective is to get terraform to loop the huge list in my tfvars below and loop, but cherry picking the 1st 5 then assigning it to the 1st packet mirror resource I want to create called lets say packetmirror1
Then it looks again starting from the next list with appworkstream6-subnet and creates me packetmirror2
Then it looks again starting from the next list with appworkstream11-subnet and creates me packetmirror3
Hope this makes sense...
TFVARS here
mirror_vpc_subnets = [
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream1-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream2-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream3-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream4-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream5-subnet"
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream6-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream7-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream8-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream9-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream10-subnet"
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream11-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream12-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream13-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream14-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream15-subnet"
]
Please advise how this resource can be created in a loop also incrementing the packetmirror name on that creation loop
resource "google_compute_packet_mirroring" "main" {
name = var.packet_mirror_policy_name
project = var.gcp_project_id
region = var.region
network {
url = var.collector_mirror_network_selflink
}
collector_ilb {
url = var.forwarding_rule
}
mirrored_resources {
tags = var.mirrored_tags
dynamic "subnetworks" {
for_each = var.mirror_vpc_subnets
content {
url = subnetworks.value
}
}
dynamic "instances" {
for_each = var.mirror_vpc_instances
content {
url = instances.value
}
}
}
Related
We have a long(ish) list of ip addresses to trust, for services published by CloudFlare, and rather than asking each team that publishes a service from an account in aws to implement this in their security groups / acls etc. I thought a prefix list would be perfect. I would like to set this up in a central account, that is then shared to all the child accounts across the aws Organisation. Ideally, I would like to avoid declaring more resources than necessary, so I am using for_each loops and a dynamic entry. So far so good.
However, prefix lists are not a global object, meaning they need to be created per region, and Terraform requires that this is set on the Provider level.
Is there a way to have a single resource declaration work with a single local map to dynamically manage all these moving parts?
It seems the fewest steps I can do, is to have a resource declaration per region, and then a local map per provider, which is already making 6 "blocks" for only 3 regions, + 2 lists for ipv4 and ipv6...
Here's the code example:
locals {
# The two ip lists to add to appropriate prefix lists
cloudflare-ips = {
prefixlist_cloudflare_ipv4 = {
ips = [
"10.0.0.0/32",
"173.245.48.0/20",
...
],
type = "IPv4",
},
prefixlist_cloudflare_ipv6 = {
ips = [
"2400:cb00::/32",
"2606:4700::/32",
...
],
type = "IPv6",
}
}
}
# Generate multipel predix lists from the map of ipv4 and ipv6 addresses
resource "aws_ec2_managed_prefix_list" "cloudflare-ipv4" {
for_each = local.cloudflare-ips
name = "cloudflare_${each.value.type}"
# Here is where I have to add a provider for the region, and this cannot be done within any type of loop ?
provider = aws.ap-southeast-1
address_family = each.value.type
max_entries = 50
dynamic "entry" {
for_each = tolist( each.value.ips )
content {
cidr = entry.value
description = "CloudFlare ${entry.key}"
}
}
}
I've tried including a list of strings with region / provider names and calling these directly from inside the loop, but even though it can evaluate a string (e.g. provider = "aws.ap-southeast-1") it does not accept reading from the local map like so:
provider = each.value.region
with error:
│ The provider argument requires a provider type name, optionally followed by a period and then a configuration alias.
I guess the next step would be to make this into a module ? Any other suggestions... ?
this cannot be done within any type of loop
That's correct. You can't have dynamic provider, thus you can't use any loops and variables. Its value must be hardcoded.
I am working with a GCP Cloud Composer resource and added in a dynamic block to create an attribute for the resource to set allowed_ip_ranges which can be used as an IP filter for accessing the Apache Airflow Web UI.
I was able to get the allowed ranges setup and can update them in place to new values also.
If I attempt to pass in a blank list I am expecting the IP address(es) to be removed as attributes for the resource but Terraform seems to think that no changes are needed.
There is probably something wrong in my code but I am not sure what exactly I would need to do. Does it involve adding in a conditional expression to the for_each loop in the dynamic block?
Child module main.tf
web_server_network_access_control {
dynamic "allowed_ip_range" {
for_each = var.allowed_ip_range
content {
value = allowed_ip_range.value["value"]
description = allowed_ip_range.value["description"]
}
}
}
Child module variables.tf
variable "allowed_ip_range" {
description = "The IP ranges which are allowed to access the Apache Airflow Web Server UI."
type = list(map(string))
default = []
}
Parent module terraform.tfvars
allowed_ip_range = [
{
value = "11.0.0.2/32"
description = "Test dynamic block 1"
},
]
You can set the default value in your variables.tf file:
variable "allowed_ip_range" {
description = "The IP ranges which are allowed to access the Apache Airflow Web Server UI"
type = list(map(string))
default = [
{
value = "0.0.0.0/0"
description = "Allows access from all IPv4 addresses (default value)"
},
{
value = "::0/0"
description = "Allows access from all IPv6 addresses (default value)"
},
]
}
And when you will delete your variable from terraform.tfvars, you will have the default values
I'm currently trying to use TF 0.12 to create AWS Organizations accounts. Right now I have a map of accounts with applicable info, here is an example where "Services" is the account name:
accountMap = {
...
Services = {
OU = ["Development", "Production"]
},
...
}
OU refers to the org units the account should be a part of. I'm currently already using for_each to loop through this map of account names, but I'm stuck on how to use the OUs as a suffix, so the org account name would become "Services-Development" and "Services-Production". I have tried similar to the following:
resource "aws_organizations_account" "main" {
for_each = var.ouMap
name = "${each.key}-${var.accountMap["${each.value[*]}"]}"
...
}
However, "name" requires a string and I get an error since I am providing a list of the OUs, but I may want one account to belong to several OUs or just a single OU. So, how can I either convert the list to a string one at a time, while in the same for_each iteration (but for my differing OUs)?
I'm open to other suggestions on best practice to map AWS Org accounts to multiple OUs as I'm still rather new to Terraform.
A local value can be computed using nested for loop in terraform v0.12.
The local value can later be used in resources. This example treats a null resource.
accountMap = {
Services = {
OU = ["Development", "Production"]
}
}
locals {
organization_account = flatten(
[for k, v in var.accountMap: [for v2 in v.OU: "${k}-${v2}"]]
)
}
resource "null_resource" "foo" {
count = length(local.organization_account)
provisioner "local-exec" {
command = "echo ${local.organization_account[count.index]}"
}
}
output "organization_account" {
value = local.organization_account
}
When I was just starting to use Terraform, I more or less naively declared resources individually, like this:
resource "aws_cloudwatch_log_group" "image1_log" {
name = "${var.image1}-log-group"
tags = module.tagging.tags
}
resource "aws_cloudwatch_log_group" "image2_log" {
name = "${var.image2}-log-group"
tags = module.tagging.tags
}
resource "aws_cloudwatch_log_stream" "image1_stream" {
name = "${var.image1}-log-stream"
log_group_name = aws_cloudwatch_log_group.image1_log.name
}
resource "aws_cloudwatch_log_stream" "image2_stream" {
name = "${var.image2}-log-stream"
log_group_name = aws_cloudwatch_log_group.image2_log.name
}
Then, 10-20 different log groups later, I realized this wasn't going to work well as infrastructure grew. I decided to define a variable list:
variable "image_names" {
type = list(string)
default = [
"image1",
"image2"
]
}
Then I replaced the resources using indices:
resource "aws_cloudwatch_log_group" "service-log-groups" {
name = "${element(var.image_names, count.index)}-log-group"
count = length(var.image_names)
tags = module.tagging.tags
}
resource "aws_cloudwatch_log_stream" "service-log-streams" {
name = "${element(var.image_names, count.index)}-log-stream"
log_group_name = aws_cloudwatch_log_group.service-log-groups[count.index].name
count = length(var.image_names)
}
The problem here is that when I run terraform apply, I get 4 resources to add, 4 resources to destroy. I tested this with an old log group, and saw that all my logs were wiped (obviously, since the log was destroyed).
The names and other attributes of the log groups/streams are identical- I'm simply refactoring the infrastructure code to be more maintainable. How can I maintain my existing log groups without deleting them yet still refactor my code to use lists?
You'll need to move the existing resources within the Terraform state.
Try running terraform show to get the strings under which the resources are stored, this will be something like [module.xyz.]aws_cloudwatch_log_group.image1_log ...
You can move it with terraform state mv [module.xyz.]aws_cloudwatch_log_group.image1_log '[module.xyz.]aws_cloudwatch_log_group.service-log-groups[0]'.
You can choose which index to assign to each resource by changing [0] accordingly.
Delete the old resource definition for each moved resource, as Terraform would otherwise try to create a new group/stream.
Try it with the first import and check with terraform plan if the resource was moved correctly...
Also check if you need to choose some index for the image_names list jsut to be sure, but I think that won't be necessary.
I have 3 different version of an AMI, for 3 different nodes in a cluster.
data "aws_ami" "node1"
{
# Use the most recent AMI that matches the pattern below in 'values'.
most_recent = true
filter {
name = "name"
values = ["AMI_node1*"]
}
filter {
name = "tag:version"
values = ["${var.node1_version}"]
}
}
data "aws_ami" "node2"
{
# Use the most recent AMI that matches the pattern below in 'values'.
most_recent = true
filter {
name = "name"
values = ["AMI_node2*"]
}
filter {
name = "tag:version"
values = ["${var.node2_version}"]
}
}
data "aws_ami" "node3"
{
...
}
I would like to create 3 different Launch Configuration and Auto Scaling Group using each of the AMIs respectively.
resource "aws_launch_configuration" "node"
{
count = "${local.node_instance_count}"
# Name-prefix must be used otherwise terraform fails to perform updates to existing launch configurations due to
# a name conflict: LCs are immutable and the LC cannot be destroyed without destroying attached ASGs as well, which
# terraform will not do. Using name-prefix lets a new LC be created and swapped into the ASG.
name_prefix = "${var.environment_name}-node${count.index + 1}-"
image_id = "${data.aws_ami.node[count.index].image_id}"
instance_type = "${var.default_ec2_instance_type}"
...
}
However, I am not able use aws_ami.node1, aws_ami.node2, aws_ami.node3 using the cound.index the way I have shown above. I get the following error:
Error reading config for aws_launch_configuration[node]: parse error at 1:39: expected "}" but found "."
Is there another way I can do this in Terraform?
Indexing data sources isn't something that's doable; at the moment.
You're likely better off simply dropping the data sources you've defined and codifying the image IDs into a Terraform map variable.
variable "node_image_ids" {
type = "map"
default = {
"node1" = "1234434"
"node2" = "1233334"
"node3" = "1222434"
}
}
Then, consume it:
image_id = "${lookup(var.node_image_ids, concat("node", count.index), "some_default_image_id")}"
The downside of this is that you'll need to manually update the image id when images are upgraded.