Terraform Module with Count Can't Have Duplicate Elements - digital-ocean

Tricky one to explain in a title, or even form a question around it, so I'll start with some code (simplified for, err, simplicity):
resource "digitalocean_domain" "this_domain" {
name = "${var.domain}"
ip_address = "${var.main_ip}"
}
resource "digitalocean_record" "this_a_record" {
count = "${length(var.a_records)}"
domain = "${var.domain}"
type = "A"
name = "${element(keys(var.a_records), count.index)}"
value = "${lookup(var.a_records, element(keys(var.a_records), count.index))}"
}
Given the above being part of a module called dns, I can call it like this:
module "example_com_dns" {
source = "./modules/dns"
domain = "example.com"
main_ip = "1.2.3.4"
a_records = {
"#" = "5.6.7.8"
"self" = "9.10.11.12"
"www" = "5.6.7.8"
}
}
Running this works as expected. I get the A records I expect; #, self, www, all pointing to the correct IPs.
However, it can't handle duplicate names. For example, putting in multiple # records results in only one of them being written, I'm guessing because each iteration for the name simply overwrites the previous # record.
Is there a way to have multiple duplicate names? i.e. In the example above, have something like:
....
"#" = "5.6.7.8"
"#" = "20.21.22.23"
"#" = "30.31.32.33"
"self" = "9.10.11.12"
"www" = 5.6.7.8"
...

In that case instead of using keys to get map keys you should put the A records into a list like:
a_records = [
["#" , "5.6.7.8"],
["#" , "7.6.7.8"],
["#" , "9.9.9.9"],
["self", "7.10.11.12"],
["self", "11.11.111.11"],
["self", "12.12.12.12"],
["self", "13.13.13.13"],
["www" , "14.14.14.14"]
]
and assign name and value as
name = "${element(var.a_records[count.index],0)}"
value = "${element(var.a_records[count.index],1)}"
and you will get all the duplicate A records that you want
The following is a simple working example of how to do it. You can cut and paste the code into a functional terraform session and it will run without errors and create duplicate A records.
You will need to extend the example to fit your needs of external modules, etc.
variable "a_records" { default = [
["#" , "5.6.7.8"],
["#" , "7.6.7.8"],
["#" , "9.9.9.9"],
["self", "7.10.11.12"],
["self", "11.11.111.11"],
["self", "12.12.12.12"],
["self", "13.13.13.13"],
["www" , "14.14.14.14"],
["www" , "14.14.14.14"]
]
}
variable domain { default = "example20180731.com" }
variable main_ip { default = "1.1.1.1" }
resource "digitalocean_domain" "this_domain" {
name = "${var.domain}"
ip_address = "${var.main_ip}"
}
resource "digitalocean_record" "this_a_record" {
depends_on = ["digitalocean_domain.this_domain"]
count = "${length(var.a_records)}"
domain = "${var.domain}"
type = "A"
name = "${element(var.a_records[count.index],0)}"
value = "${element(var.a_records[count.index],1)}"
}
This is another example. It shows how to do it using external data supplied through a module. It's also code which can be copied and pasted into a functional terraform session and will work, as is.
create-dup-A-records.tf
module "dns" {
source = "./modules/dns"
}
variable domain { default = "example20180731.com" }
variable main_ip { default = "1.1.1.1" }
resource "digitalocean_domain" "this_domain" {
name = "${var.domain}"
ip_address = "${var.main_ip}"
}
resource "digitalocean_record" "this_a_record" {
depends_on = ["digitalocean_domain.this_domain"]
count = "${length(module.dns.a_records_in_dns_module)}"
domain = "${var.domain}"
type = "A"
name = "${element(module.dns.a_records_in_dns_module[count.index],0)}"
value = "${element(module.dns.a_records_in_dns_module[count.index],1)}"
}
./modules/dns/dns.tf
variable "a_records" { default = [
["#" , "5.6.7.8"],
["#" , "7.6.7.8"],
["#" , "9.9.9.9"],
["self", "7.10.11.12"],
["self", "11.11.111.11"],
["self", "12.12.12.12"],
["self", "13.13.13.13"],
["www" , "14.14.14.14"],
["www" , "14.14.14.14"]
]
}
output "a_records_in_dns_module" {
value = "${var.a_records}"
}

Turns out it's a limitation of Terraform:
https://github.com/hashicorp/terraform/issues/18573#event-1765829698
#don is mostly there in his answer, but you can't have that work with passed in variables. You need to use the split hack to pass it is as a string, then make it a list, as described in the above linked issue:
Call from main.tf:
module "example_com_dns" {
source = "./modules/dns"
domain = "example.com"
main_ip = "${lookup(var.example_com_dns, "at01")}"
a_records = [
"# 5.6.7.8",
"# 7.6.7.8",
]
}
dns.tf module:
resource "digitalocean_record" "this_a_record" {
depends_on = ["digitalocean_domain.this_domain"]
count = "${length(var.a_records)}"
domain = "${var.domain}"
type = "A"
name = "${element(split(" ", var.a_records[count.index]),0)}"
value = "${element(split(" ", var.a_records[count.index]),1)}"
}
For whoever else is trying to do this from a Google search; my advice is don't. 0.12 will be out soon, and will fix all these kinds of insanity.

Related

Terraform: How to add multiple option_settings to option, in option group?

I'm trying to create an option group that requires an option with option settings that add multiple values.
See the following, scrubbed for sensitivity:
option {
option_name = "VALID_OPTION_NAME"
option_settings = [
{
name = "foobar1"
value = "foobar1"
},
{
name = "foobar2"
value = "foobar2"
},
{
name = "foobar3"
value = "foobar3"
},
{
name = "foobar4"
value = "foobar4"
},
{
name = "foobar5"
value = "foobar5"
}
]
}
terraform validate gives the following error:
[0m on rds.tf line 112, in resource "aws_db_option_group" "rds-option-group":
112: [4moption_settings[0m = [
[0m
An argument named "option_settings" is not expected here. Did you mean to
define a block of type "option_settings"?
I've tried numerous variations of this syntax to no avail. AWS in the GUI has the option by default to include multiple option settings, so there should be a way to do it in Terraform as well.
The Option Group docs for Terraform unfortunately don't include an example where one option has multiple settings.
Among other things, I also checked out this thread which didn't help me, I believe because I'm not using that module.
Any recommendations?
Answer, for any future viewers:
option {
option_name = "xx"
option_settings {
name = "xx"
value = "xx"
}
option_settings {
name = "xx"
value = "xx"
}
option_settings {
name = "xx"
value = "xx"
}
option_settings {
name = "xx"
value = "xx"
}
}

Terraform - Copy AWS SSM Parameters

longtime lurker first time poster
Looking for some guidance from you all. I'm trying to replicate the aws command to essentially get the parameters (ssm get-parameters-by-path) then loop through the parameters and get them
then loop through and put them into a new parameter (ssm put-parameter)
I understand there's a for loop expression in TF but for the life of me I can't put together how I would achieve this.
so thanks to the wonderful breakdown below, I've gotten closer! But have this one issue. Code below:
provider "aws" {
region = "us-east-1"
}
data "aws_ssm_parameters_by_path" "parameters" {
path = "/${var.old_env}"
recursive = true
}
output "old_params_by_path" {
value = data.aws_ssm_parameters_by_path.parameters
sensitive = true
}
locals {
names = toset(data.aws_ssm_parameters_by_path.parameters.names)
}
data "aws_ssm_parameter" "old_param_name" {
for_each = local.names
name = each.key
}
output "old_params_names" {
value = data.aws_ssm_parameter.old_param_name
sensitive = true
}
resource "aws_ssm_parameter" "new_params" {
for_each = local.names
name = replace(data.aws_ssm_parameter.old_param_name[each.key].name, var.old_env, var.new_env)
type = data.aws_ssm_parameter.old_param_name[each.key].type
value = data.aws_ssm_parameter.old_param_name[each.key].value
}
I have another file like how the helpful poster mentioned and created the initial dataset. But what's interesting is that after you create the set after the second set, it overwrites the first set! The idea is that I would be able to tell terraform, I have this current set of SSM parameters and I want you to copy that info (values, type) and create a brand new set of parameters (and not destroy anything that's already there).
Any and all help would be appreciated!
I understand, It's not easy at the beginning. I will try to elaborate step-by-step on how I achieve that.
Anyway, it's nice to include any code, that you tried before, even if doesn't work.
So, firstly I create some example parameters:
# create_parameters.tf
resource "aws_ssm_parameter" "p" {
count = 3
name = "/test/${count.index}/p${count.index}"
type = "String"
value = "test-${count.index}"
}
Then I try to view them:
# example.tf
data "aws_ssm_parameters_by_path" "parameters" {
path = "/test/"
recursive = true
}
output "params_by_path" {
value = data.aws_ssm_parameters_by_path.parameters
sensitive = true
}
As an output I received:
terraform output params_by_path
{
"arns" = tolist([
"arn:aws:ssm:eu-central-1:999999999999:parameter/test/0/p0",
"arn:aws:ssm:eu-central-1:999999999999:parameter/test/1/p1",
"arn:aws:ssm:eu-central-1:999999999999:parameter/test/2/p2",
])
"id" = "/test/"
"names" = tolist([
"/test/0/p0",
"/test/1/p1",
"/test/2/p2",
])
"path" = "/test/"
"recursive" = true
"types" = tolist([
"String",
"String",
"String",
])
"values" = tolist([
"test-0",
"test-1",
"test-2",
])
"with_decryption" = true
}
aws_ssm_parameters_by_path is unusable without additional processing, so we need to use another data source, to get a suitable object for a copy of provided parameters. n the documentation I found aws_ssm_parameter. However, to use it, I need the full name of the parameter.
List of the parameter names I retrieved in the previous stage, so now only needed is to iterate through them:
# example.tf
locals {
names = toset(data.aws_ssm_parameters_by_path.parameters.names)
}
data "aws_ssm_parameter" "param" {
for_each = local.names
name = each.key
}
output "params" {
value = data.aws_ssm_parameter.param
sensitive = true
}
And as a result, I get:
terraform output params
{
"/test/0/p0" = {
"arn" = "arn:aws:ssm:eu-central-1:999999999999:parameter/test/0/p0"
"id" = "/test/0/p0"
"name" = "/test/0/p0"
"type" = "String"
"value" = "test-0"
"version" = 1
"with_decryption" = true
}
"/test/1/p1" = {
"arn" = "arn:aws:ssm:eu-central-1:999999999999:parameter/test/1/p1"
"id" = "/test/1/p1"
"name" = "/test/1/p1"
"type" = "String"
"value" = "test-1"
"version" = 1
"with_decryption" = true
}
"/test/2/p2" = {
"arn" = "arn:aws:ssm:eu-central-1:999999999999:parameter/test/2/p2"
"id" = "/test/2/p2"
"name" = "/test/2/p2"
"type" = "String"
"value" = "test-2"
"version" = 1
"with_decryption" = true
}
}
Each parameter object has been retrieved, so now it is possible to create new parameters - which can be done like this:
# example.tf
resource "aws_ssm_parameter" "new_param" {
for_each = local.names
name = "/new_path${data.aws_ssm_parameter.param[each.key].name}"
type = data.aws_ssm_parameter.param[each.key].type
value = data.aws_ssm_parameter.param[each.key].value
}

Terraform variable referencing locals not working

I need to pass the database host name (that is dynamically generated) as an environmental variable into my task definition. I thought I could set locals and have the variable map refer to a local but it seems to not work, as I receive this error: “error="failed to check table existence: dial tcp: lookup local.grafana-db-address on 10.0.0.2:53: no such host". I am able to execute the terraform plan without issues and the code works when I hard code the database host name, but that is not optimal.
My Variables and Locals
//MySql Database Grafana Username (Stored as ENV Var in Terraform Cloud)
variable "username_grafana" {
description = "The username for the DB grafana user"
type = string
sensitive = true
}
//MySql Database Grafana Password (Stored as ENV Var in Terraform Cloud)
variable "password_grafana" {
description = "The password for the DB grafana password"
type = string
sensitive = true
}
variable "db-port" {
description = "Port for the sql db"
type = string
default = "3306"
}
locals {
gra-db-user = var.username_grafana
}
locals {
gra-db-password = var.password_grafana
}
locals {
db-address = aws_db_instance.grafana-db.address
}
locals {
grafana-db-address = "${local.db-address}.${var.db-port}"
}
variable "app_environments_vars" {
type = list(map(string))
description = "Database environment variables needed by Grafana"
default = [
{
"name" = "GF_DATABASE_TYPE",
"value" = "mysql"
},
{
"name" = "GF_DATABASE_HOST",
"value" = "local.grafana-db-address"
},
{
"name" = "GF_DATABASE_USER",
"value" = "local.gra-db-user"
},
{
"name" = "GF_DATABASE_PASSWORD",
"value" = "local.gra-db-password"
}
]
}
Task Definition Variable reference
"environment": ${jsonencode(var.app_environments_vars)},
Thank you to everyone who has helped me with this project. I am new to all of this and could not have done it without help from this community.
You can't use dynamic references in your app_environments_vars. So your default values "value" = "local.grafana-db-address" will never get resolved by TF. If will be just a literal string "local.grafana-db-address".
You have to modify your code so that all these dynamic references in app_environments_vars get populated in locals.
UPDATE
Your app_environments_vars should be local variable for it to be resolved:
locals {
app_environments_vars = [
{
"name" = "GF_DATABASE_TYPE",
"value" = "mysql"
},
{
"name" = "GF_DATABASE_HOST",
"value" = local.grafana-db-address
},
{
"name" = "GF_DATABASE_USER",
"value" = local.gra-db-user
},
{
"name" = "GF_DATABASE_PASSWORD",
"value" = local.gra-db-password
}
]
}
then you pass that local to your template for the task definition.

Terraform Variable looping to generate properties

I have to admit, this is the first time I have to ask something that I dont even myself know how to ask for it or explain, so here is my code.
It worth explains that, for specific reasons I CANNOT change the output resource, this, the metadata sent to the resource has to stay as is, otherwise it will cause a recreate and I dont want that.
currently I have a terraform code that uses static/fixed variables like this
user1_name="Ed"
user1_Age ="10"
user2_name="Mat"
user2_Age ="20"
and then those hard typed variables get used in several places, but most importanly they are passed as metadata to instances, like so
resource "google_compute_instance_template" "mytemplate" {
...
metadata = {
othervalues = var.other
user1_name = var.user1_name
user1_Age = var.user1_Age
user2_name = var.user2_name
user2_Age = var.user2_Age
}
...
}
I am not an expert on terraform, thus asking, but I know for fact this is 100% ugly and wrong, and I need to use lists or array or whatever, so I am changing my declarations to this:
users = [
{ "name" : "yo", "age" : "10", "last" : "other" },
{ "name" : "El", "age" : "20", "last" : "other" }
]
but then, how do i get around to generate the same result for that resource? The resulting resource has to still have the same metadata as shown.
Assuming of course that the order of the users will be used as the "index" of the value, first one gets user1_name and so on...
I assume I need to use a for_each loop in there but cant figure out how to get around a loop inside properties of a resource
Not sure if I make myself clear on this, probably not, but didn't found a better way to explain.
From your example it seems like your intent is for these to all ultimately appear as a single map with keys built from two parts.
Your example doesn't make clear what the relationship is between user1 and Ed, though: your first example shows that "user1's" name is Ed, but in your example of the data structure you want to create there is only one "name" and it isn't clear to me whether that name would replace "user1" or "Ed" from your first example.
Instead, I'm going to take a slightly different variable structure which still maintains both the key like "user1" and the name attribute, like this:
variable "users" {
type = map(object({
name = string
age = number
})
}
locals {
# First we'll transform the map of objects into a
# flat set of key/attribute/value objects, because
# that's easier to work with when we generate the
# flattened map below.
users_flat = flatten([
for key, user in var.users : [
for attr, value in user : {
key = key
attr = attr
value = value
}
]
])
}
resource "google_compute_instance_template" "mytemplate" {
metadata = merge(
{
othervalues = var.other
},
{
for vo in local.users_flat : "${vo.key}_${vo.attr}" => vo.value
}
)
}
local.users_flat here is an intermediate data structure that flattens the two-level heirarchy of keys and object attributes from the input. It would be shaped something like this:
[
{ key = "user1", attr = "name", value = "Ed" },
{ key = "user1", attr = "age", value = 10 },
{ key = "user2", attr = "name", value = "Mat" },
{ key = "user2", attr = "age", value = 20 },
]
The merge call in the metadata argument then merges a directly-configured mapping of "other values" with a generated mapping derived from local.users_flat, shaped like this:
{
"user1_name" = "Ed"
"user1_age" = 10
"user2_name" = "Mat"
"user2_age" = 20
}
From the perspective of the caller of the module, the users variable should be defined with the following value in order to get the above results:
users = {
user1 = {
name = "Ed"
age = 10
}
user2 = {
name = "Mat"
age = 20
}
}
metadata is not a block, but a regular attribute of type map. So you can do:
# it would be better to use map, not list for users:
variable "users"
default {
user1 = { "name" : "yo", "age" : "10", "last" : "other" },
user2 = { "name" : "El", "age" : "20", "last" : "other" }
}
}
resource "google_compute_instance_template" "mytemplate" {
for_each = var.users
metadata = each.value
#...
}

Terrraform list of objects syntax

I'm using a module that references a central module used to build a Puppet server in terraform. There is one variable in the root module that allows additional tags to be used with the ASG however I can't seem to get the syntax right. This is the information in the core repository:
variable "additional_asg_tags" {
description = "A map of additional tags to add to the puppet server ASG."
type = list(object({ key = string, value = string, propagate_at_launch = bool }))
default = []
}
I've tried everything I can think of to call this but it always errors with messages like "incorrect list element type: string required." or "This default value is not compatible with the variable's type constraint: list of object required."
I'm trying to call the above with something like;
variable "additional_asg_tags" {
description = "A map of additional tags to add to ASG."
type = list(object({ key = string, value = string, propagate_at_launch = bool }))
default = { key = "Name", value = "Puppet-nonprod", propagate_at_launch = "true"
}
}
I've removed the square braces around this as that was causing errors also but I may need to add these back in.
Can someone help please in what is the correct way to reference a list of objects with these values
The correct default value for your additional_asg_tags is a list:
variable "additional_asg_tags" {
description = "A map of additional tags to add to ASG."
type = list(object({
key = string,
value = string,
propagate_at_launch = bool
}))
default = [{
key = "Name",
value = "Puppet-nonprod",
propagate_at_launch = "true"
}]
}
You can reference individual elements as follows (some examples):
var.additional_asg_tags[0]["key"]
var.additional_asg_tags[0].value
# to get list
var.additional_asg_tags[*].propagate_at_launch