Terraform skip null for_each - amazon-web-services

I am setting tags to AMI. I have ubuntu and EKS AMI, where EKS AMI needs to be set with k8s version, containerd etc which are not required for other OS. I have declared all variables that are EKS specific to default to null in an assumption that they will skipped while I run for_each but it errors out. Here is the code and error.
locals {
tags = {
docker_version = var.docker_version
kubernetes = var.k8s_version
cni_plugin_version = var.cni_plugin_version
containerd_version = var.containerd_version
target = var.target
source_ami_id = var.source_ami_id
Release = var.ami_release-version
Description = var.ami_description
Name = var.ami_name
Creator = var.creator
}
data "aws_ami" "ami_image" {
most_recent = true
owners = ["xxxxxxxx"]
name_regex = var.ami_regex
}
output "ami_id" {
value = data.aws_ami.ami_image.id
}
output "ami_arn" {
value = data.aws_ami.ami_image.arn
}
output "ami_name" {
value = data.aws_ami.ami_image.name
}
resource "aws_ec2_tag" "ami-taggging" {
resource_id = data.aws_ami.ami_image.id
for_each = local.tags
key = each.key
value = each.value
}
Error when values are null:
Error: Missing required argument
on main.tf line 41, in resource "aws_ec2_tag" "ami-tagging":
41: value = each.value
Is there a way to skip or gracefully move to next record if the value is null.

Assuming that default_tags (not shown in your question) is similar to your local.tag, you could do the following:
resource "aws_ec2_tag" "ami-taggging" {
resource_id = data.aws_ami.ami_image.id
for_each = {for k,v in local.tags: k => v if v != null}
key = each.key
value = each.value
}
The solution uses for expression. Basically, it takes your original local.tags and creates new temporary map that will be used in the for_each. The new map will have same keys and values (k=>v) except those that don't meet the if condition.

Related

WAF for all ALBs in an account using terraform

This is my waf.tf terraform file:
resource "aws_wafv2_web_acl" "waf_acl-dev" {
name = "waf_log4j_Protections-dev"
description = "WAFv2 for dev"
scope = "REGIONAL"
default_action {
allow {}
}
rule {
name = "AWSManagedRulesKnownBadInputsRule"
priority = 1
override_action {
count {}
}
statement {
managed_rule_group_statement {
name = "AWSManagedRulesKnownBadInputsRuleSet"
vendor_name = "AWS"
# Excluding all these leaves only Log4JRCE
excluded_rule {
name = "Host_localhost_HEADER"
}
excluded_rule {
name = "PROPFIND_METHOD"
}
excluded_rule {
name = "ExploitablePaths_URIPATH"
}
}
}
visibility_config {
cloudwatch_metrics_enabled = true
metric_name = "AWSManagedRulesKnownBadInputsRule"
sampled_requests_enabled = true
}
}
visibility_config {
cloudwatch_metrics_enabled = true
metric_name = "AWSManagedKnownBadInputsRule"
sampled_requests_enabled = true
}
}
variable "lb_arn" {
type = string
default = ""
}
data "aws_lb" "all_alb" {
tags = {
Environment = "Dev"
}
# arn = var.lb_arn
# for_each = data.aws_lb.all_alb
# id = each.value
}
output "all_alb" {
value = data.aws_lb.all_alb
}
resource "aws_wafv2_web_acl_association" "waf_acl-association-dev" {
for_each = data.aws_lb.all_alb.arn
resource_arn = each.value
web_acl_arn = aws_wafv2_web_acl.waf_acl-dev.arn
}
My objective is to create a WAF with rules (that portion works fine), and attach multiple ALBs to the WAF using tags on each ALB. Im running it inside a teamcity loop that loops into multiple AWS accounts (That is out of scope). Each account can have one or multiple ALBs with the tag provided. When I run this code I get the following error:
Error: Search returned 2 results, please revise so only one is returned
17:39:18 │
17:39:18 │ with data.aws_lb.all_alb,
17:39:18 │ on xa-waf-inow.tf line 49, in data "aws_lb" "all_alb":
17:39:18 │ 49: data "aws_lb" "all_alb" {
I also tried a few other alternatives like putting [0] at the end of line 49 but Im still getting hit with some syntax error or the other. Can someone please help?Thanks
Edit:
I also tried for_each in data block:
data "aws_lb" "all_alb" {
for_each = {
tags = {
Environment = "Dev"
}
}
arn = each.key
}
output "all_alb" {
value = data.aws_lb.all_alb
}
resource "aws_wafv2_web_acl_association" "waf_acl-association-dev" {
# for_each = data.aws_lb.all_alb.arn
resource_arn = data.aws_lb.all_alb[each.key]
web_acl_arn = aws_wafv2_web_acl.waf_acl-dev.arn
}
But got this error:
Error: Reference to "each" in context without for_each
18:24:01 │
18:24:01 │ on xa-waf-inow.tf line 65, in resource "aws_wafv2_web_acl_association" "waf_acl-association-dev":
18:24:01 │ 65: resource_arn = data.aws_lb.all_alb[each.key]
18:24:01 │
18:24:01 │ The "each" object can be used only in "module" or "resource" blocks, and
18:24:01 │ only when the "for_each" argument is set.
I was able to get this to work for myself. My issue was a mixture of lists and sets. I believe I can change my variable to a set, and not have to deal with any sets here, but I know the below works as is.
VARIABLES.TF
variable "list_of_alb" {
type = list(string)
}
MAIN.TF
list_of_alb = [
"${terraform.workspace}-unique-1",
"${terraform.workspace}-unique-2"
]
DATA.TF - the problem child
data "aws_lb" "main" {
for_each = toset( var.list_of_alb )
name = each.value
}
WAF.TF
resource "aws_wafv2_web_acl_association" "main" {
for_each = toset(var.list_of_alb)
resource_arn = data.aws_lb.main[each.value].arn
web_acl_arn = aws_wafv2_web_acl.main.arn
}
The aws_lb data source must return only one result. You can't change that, as this is how it was destined.
If you want to return multiple ALBs you have two choices:
Use for_each with the data source. This way your data source will run for each id of your alb. This means that you have to provide the alb ids as an input variable.
Or, create your own custom data source. Since this is fully custom code that you have to write, it can overcome any limitations of TF's build in data sources.

Terraform lunch EC2 condition based on count

Trying to lunch 1 or 2 instances based on a condition ( "single-target" )
ERROR im getting :
132: target_id = var.ec2-2
var.ec2-2 is empty tuple
Inappropriate value for attribute "target_id": string required.
the use of that variable :
resource "aws_lb_target_group_attachment" "b" {
target_id = var.ec2-2
}
outputs :
output "ec2-2_ot" {
value = aws_instance.ec2V2[*].id
description = "the value of the network module ec2-2 id"
}
variable :
variable "single-target" {
type=bool
default=false
}
main.tf :
module "network_module" {
ec2-2 = module.compute_module.ec2-2_ot
}
ec2 launch :
resource "aws_instance" "ec2V2" {
count = var.single-target ? 0 : 1
ami = var.ec2_ami
instance_type = var.ec2_type
associate_public_ip_address = true
key_name = var.key_pair
vpc_security_group_ids = [ var.vpc-sg ]
subnet_id = var.subnet-2
user_data = file("PATH/user-data.sh")
tags = {
Name = var.ec2-2_name
}
}
As the count has been already set to the resource you are trying to create, the returning result list would need to be accessed via Splat Expression mentioned in this terraform documentation.
If you need to choose specific index from the list of the result , then you can use the element function to do so
So in your case, if you need to output EC2 id it would be as follows.
output "thisisoutput" {
value = aws_instance.ec2V2[*].arn
}

Error - Missing resource instance key : Because google_compute_instance has "count" set, its attributes must be accessed on specific instances

I am trying to pass mat_ip of google compute instances created in module "microservice-instance" to another module "database". Since I am creating more than one instance, I am getting following error for output variable in module "microservice-instance".
Error: Missing resource instance key
on modules/microservice-instance/ms-outputs.tf line 3, in output "nat_ip": 3: value = google_compute_instance.apps.network_interface[*].access_config[0].nat_ip
Because google_compute_instance.apps has "count" set, its attributes must be accessed on specific instances.
For example, to correlate with indices of a referring resource, use:
google_compute_instance.apps[count.index]
I have looked at following and using the same way of accessing attribute but its not working. Here is code -
main.tf
provider "google" {
credentials = "${file("../../service-account.json")}"
project = var.project
region =var.region
}
# Include modules
module "microservice-instance" {
count = var.appserver_count
source = "./modules/microservice-instance"
appserver_count = var.appserver_count
}
module "database" {
count = var.no_of_db_instances
source = "./modules/database"
nat_ip = module.microservice-instance.nat_ip
no_of_db_instances = var.no_of_db_instances
}
./modules/microservice-instance/microservice-instance.tf
resource "google_compute_instance" "apps" {
count = var.appserver_count
name = "apps-${count.index + 1}"
# name = "apps-${random_id.app_name_suffix.hex}"
machine_type = "f1-micro"
boot_disk {
initialize_params {
image = "ubuntu-os-cloud/ubuntu-1804-lts"
}
}
network_interface {
network = "default"
access_config {
// Ephemeral IP
}
}
}
./modules/microservice-instance/ms-outputs.tf
output "nat_ip" {
value = google_compute_instance.apps.network_interface[*].access_config[0].nat_ip
}
./modules/database/database.tf
resource "random_id" "db_name_suffix" {
byte_length = 4
}
resource "google_sql_database_instance" "postgres" {
name = "postgres-instance-${random_id.db_name_suffix.hex}"
database_version = "POSTGRES_11"
settings {
tier = "db-f1-micro"
ip_configuration {
dynamic "authorized_networks" {
for_each = var.nat_ip
# iterator = ip
content {
# value = ni.0.access_config.0.nat_ip
value = each.key
}
}
}
}
}
You are creating var.appserver_count number of google_compute_instance.apps resources. So you will have:
google_compute_instance.apps[0]
google_compute_instance.apps[1]
...
google_compute_instance.apps[var.appserver_count - 1]
Therefore, in your output, instead of:
output "nat_ip" {
value = google_compute_instance.apps.network_interface[*].access_config[0].nat_ip
}
you have to reference individual apps resources or all of them using [*], for example:
output "nat_ip" {
value = google_compute_instance.apps[*].network_interface[*].access_config[0].nat_ip
}

Terraform fail due output for_each values

When I trying to dynamically allocate subnet ids by running this tf file:
data "aws_availability_zones" "available" {}
resource "aws_vpc" "wpl_vpc" {
cidr_block = var.vpc_cidr
enable_dns_hostnames = true
tags = {
Name = "WPL-VPC"
CreatedBy = var.created_by
}
}
resource "aws_subnet" "wpl_public_subnet" {
for_each = { for index, az_name in data.aws_availability_zones.available.names : index => az_name }
vpc_id = aws_vpc.wpl_vpc.id
cidr_block = cidrsubnet(var.vpc_cidr, 8, each.key + 10)
availability_zone = data.aws_availability_zones.available.names[each.key]
map_public_ip_on_launch = true
tags = {
Name = "WPL-PublicSubnet"
CreatedBy = var.created_by
}
}
output "wpl_public_subnet_ids" {
value = [aws_subnet.wpl_public_subnet.*.id]
}
I get an error:
Error: Unsupported attribute
on ../modules/vpc/outputs.tf line 5, in output "wpl_public_subnet_ids":
5: value = [aws_subnet.wpl_public_subnet.*.id]
This object does not have an attribute named "id".
However, everything works like charm, if I replace the output with this one:
aws_subnet.wpl_public_subnet.0.id
How could I output all subnet ids values?
Since you are using for_each, you will have a map, not a list. Thus, you have to get values first, before getting the ids. Also you don't need extra square brackets:
output "wpl_public_subnet_ids" {
value = values(aws_subnet.wpl_public_subnet)[*].id
}

Incorrect attribute value type error in terraform when passing ec2 instance ids

I am using below code to create 3 ec2 instances and register them with a load balancer,
variables
instance_type = "t3.large"
root_block_volume_type = "standard"
root_block_volume_size = "50"
instance_count = "3"
ec2 creation
resource "aws_instance" "ec2" {
count = "${var.instance_count}"
ami = "${var.ami_id}"
instance_type = "${var.instance_type}"
key_name = "${var.key_pair_name}"
subnet_id = "${var.private_subnet_id}"
iam_instance_profile = "${aws_iam_instance_profile.iam_instance_profile.name}"
/*
* CAUTION: changing value of below fields will cause the EC2 instance to be terminated and
* re-created. Think before running the "apply" command.
*/
associate_public_ip_address = false
tags = {
Environment = "${var.env}"
Project = "${var.project}"
Provisioner="different-box"
Name = "${local.name}-1"
}
root_block_device {
volume_type = "${var.root_block_volume_type}"
volume_size = "${var.root_block_volume_size}"
}
}
resource "aws_network_interface_sg_attachment" "sg_attachment" {
count = "${var.instance_count}"
security_group_id = "${aws_security_group.ec2_sg.id}"
network_interface_id = "${aws_instance.ec2[count.index].primary_network_interface_id}"
}
Registering with load balancer
resource "aws_alb_target_group_attachment" "alb_target_group_attachment" {
count = length("${var.ec2_instance_ids}")
target_group_arn = "${aws_alb_target_group.alb_target_group.arn}"
target_id = "${var.ec2_instance_ids[count.index]}"
port = "${var.alb_target_group_port}"
}
But however when I pass ec2 instance ids as below,
module "alb_engine-ui" {
source = "./modules/load-balancer"
env = "${lower(var.env)}"
project = "engine-ui"
vpc_id = "${data.aws_vpc.main.id}"
public_subnet1_id = "${var.public_subnet1_id}"
public_subnet2_id = "${var.public_subnet2_id}"
health_check_target_group_path = "/"
certificate_arn = "${var.certificate_arn}"
alb_target_group_port = "2016"
ec2_instance_ids = ["${aws_instance.ec2[*].id}"]
}
variable "ec2_instance_ids" {
description = "the ec2 instance ids to be used for alb target group"
type = "list"
}
I'm getting the below error,
Error: Incorrect attribute value type
on provisioners/different-box/modules/load-balancer/resources.tf line 109, in resource "aws_alb_target_group_attachment" "alb_target_group_attachment":
109: target_id = "${var.ec2_instance_ids[count.index]}"
Inappropriate value for attribute "target_id": string required.
Is there a way to avoid this error and pass ec2 ids to the list?
"${aws_instance.ec2[*].id}" is already a list of IDs so if you wrap it in square brackets it makes it a list of lists.
Removing the square brackets instead leaves it as a list of strings so when you index it you get a single string which is what the target_id parameter wants.