I would like to understand about the resources reference in terraform.
Example:
In my project, pubsub topic has been referred with .name as well as .id
resource "google_pubsub_topic" "topic" {
name = "my_topic"
}
resource "google_pubsub_subscription" "subscription" {
name = "my_subscription"
topic = google_pubsub_topic.topic.name
}
resource "google_cloudiot_registry" "cloudiot" {
name = "my_iot_registry"
region = "us-central1"
log_level = "ERROR"
event_notification_configs {
pubsub_topic_name = google_pubsub_topic.topic.id
}
mqtt_config = {
mqtt_enabled_state = "MQTT_ENABLED"
}
}
I could not get the information the difference in referring by .name/.id from many online forums.
For which resources using terraform we need to refer by .name and .id?
There is no such hard referring notion, it appears to be an anomaly with usage specific to this resource.
I guess it needs to be
event_notification_configs {
pubsub_topic_id = google_pubsub_topic.topic.id
}
From google_cloudiot_registry, I see id being returned by the resource which contains the resource name & the same is being passed to pubsub_topic_name part of event_notification_configs block.
If you wish to change pubsub_topic_name to pubsub_topic_id, you could create PR on the provider code base.
To conclude, if you would like to refer output of some resource/data source, you would need to fetch the attributes returned in the response & assign it to appropriate field in the next resource.
Related
This is the terraform shown in the docs:
resource "aws_sagemaker_project" "example" {
project_name = "example"
service_catalog_provisioning_details {
product_id = aws_servicecatalog_product.example.id
}
}
I created a service catalog product with id: "prod-xxxxxxxxxxxxx".
When I substitute the service catalog product id into the above template,
to get the following:
resource "aws_sagemaker_project" "example" {
project_name = "example"
service_catalog_provisioning_details {
product_id = aws_servicecatalog_product.prod-xxxxxxxxxxxxx
}
}
I run terraform plan, but the following error occurs:
A managed resource "aws_servicecatalog_product" "prod-xxxxxxxxxxxxx" has not been declared in the root module.
What do I need to do to fix this error?
Since the documentation is lacking a bit of clarity, in order to have this work as in the example, you would first have to create the Service Catalog product in Terraform as well, e.g.:
resource "aws_servicecatalog_product" "example" {
name = "example"
owner = [aws_security_group.example.id] # <---- This would need to be created first
type = aws_subnet.main.id # <---- This would need to be created first
provisioning_artifact_parameters {
template_url = "https://s3.amazonaws.com/cf-templates-ozkq9d3hgiq2-us-east-1/temp1.json"
}
tags = {
foo = "bar"
}
}
You can reference it then in the SageMaker project the same way as in the example:
resource "aws_sagemaker_project" "example" {
project_name = "example"
service_catalog_provisioning_details {
product_id = aws_servicecatalog_product.example.id
}
}
Each of the resources that gets created has a set of attributes that can be accessed as needed by other resources, data sources or outputs. In order to understand how this works, I strongly suggest reading the documentation about referencing values [1]. Since you already created the Service Catalog product, the only thing you need to do is provide the string value for the product ID:
resource "aws_sagemaker_project" "example" {
project_name = "example"
service_catalog_provisioning_details {
product_id = "prod-xxxxxxxxxxxxx"
}
}
When I can't understand what value is expected by an argument (e.g., product_id in this case), I usually read the docs and look for examples like in [2]. Note: That example is CloudFormation, but it can help you understand what type of a value is expected (e.g., string, number, bool).
You could also import the created Service Catalog product into Terraform so you can manage it with IaC [3]. You should understand all the implications of terraform import though before trying it [4].
[1] https://www.terraform.io/language/expressions/references
[2] https://docs.amazonaws.cn/en_us/AWSCloudFormation/latest/UserGuide/aws-resource-sagemaker-project.html#aws-resource-sagemaker-project--examples--SageMaker_Project_Example
[3] https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/servicecatalog_product#import
[4] https://www.terraform.io/cli/commands/import
I'm importing roles which already have been created in AWS console and unfortunately the names are strange. So in order to use those roles I am trying like this
I've two IAM roles as follows
data "aws_iam_role" "reithera-rtcov201" {
name = "exomcloudrosareitherartcov-YRX1M2GJKD6H"
}
data "aws_iam_role" "dompe-rlx0120" {
name = "exomcloudrosadomperlx0120p-1SCGY0RG5JXFF"
}
In this file I have 2 variables as follows:
sponsor = ["reithera", "dompe"]
study = ["rtcov201", "rlx0120"]
I'm trying in the following way, but terraform doesn't allow to use $.
data.aws_iam_role.${var.sponsor}-${var.study}.arn
Do you know any solution for this.
Its not possible. You can dynamically create references to resources.
Instead of two separate data sources you should create one:
variable "iam_roles"
default = ["exomcloudrosareitherartcov-YRX1M2GJKD6H", "exomcloudrosadomperlx0120p-1SCGY0RG5JXFF"]
}
and then
data "aws_iam_role" "role" {
for_each = toset(var.iam_roles)
name = each.key
}
and you can refer to them using role name:
data.aws_iam_role["exomcloudrosareitherartcov-YRX1M2GJKD6H"].arn
I want to create a resource group in AWS inspector with terraform, that has few tags with key "Name" and different values. I can do this with AWS GUI, but I Want to do it in terraform also. If I do it like in the example below, it will just override the name..
resource "aws_inspector_resource_group" "bar" {
tags = {
Name = "Master"
Name = "UF"
}
}
Could you please try the following:
resource "aws_inspector_resource_group" "bar" {
tags = {
Name = ["Master", "UF"]
}
}
If that doesn't work you could just use the aws_ec2_tag resource:
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_tag
Should do the job
I have created a WAF ACL using AWS Console. Now I need to create WAF Rule using Terraform, so I have implemented below rule.
resource "aws_wafregional_byte_match_set" "blocked_path_match_set" {
name = format("%s-%s-blocked-path", local.name, var.module)
dynamic "byte_match_tuples" {
for_each = length(var.blocked_path_prefixes) > 0 ? var.blocked_path_prefixes : []
content {
field_to_match {
type = lookup(byte_match_tuples.value, "type", null)
}
target_string = lookup(byte_match_tuples.value, "target_string", null)
positional_constraint = lookup(byte_match_tuples.value, "positional_constraint", null)
text_transformation = lookup(byte_match_tuples.value, "text_transformation", null)
}
}
}
resource "aws_wafregional_rule" "blocked_path_allowed_ipaccess" {
metric_name = format("%s%s%sBlockedPathIpaccess", var.application, var.environment, var.module)
name = format("%s%s%sBlockedPathIpaccessRule", var.application, var.environment, var.module)
predicate {
type = "ByteMatch"
data_id = aws_wafregional_byte_match_set.blocked_path_match_set.id
negated = false
}
}
But how do I map this new rule to existing "web_acl" which was created through AWS Console. As per documentation I can use "aws_wafregional_web_acl" to create new web_acl, but is there a way to associate rule created through terraform with existing waf_acl ? I have a gitlab pipeline which deploys terraform code to aws, so eventually I will pass id/arn of existing web_acl and through pipeline just add/update new rule without impacting existing rules which were created through console.
Please share your valuable feedback.
Thank you.
As per the WAF documentation you associate the rule via an AWS WAF resource, see the example below for a code snippet.
resource "aws_wafregional_web_acl" "foo" {
name = "foo"
metric_name = "foo"
default_action {
type = "ALLOW"
}
rule {
action {
type = "BLOCK"
}
priority = 1
rule_id = aws_wafregional_rule.blocked_path_allowed_ipaccess.id
}
}
However, as you said you have created the resource already in the AWS console. Terraform does support the import of an AWS resource, so you would need to go with this method if you would like to manage it via Terraform.
When I was just starting to use Terraform, I more or less naively declared resources individually, like this:
resource "aws_cloudwatch_log_group" "image1_log" {
name = "${var.image1}-log-group"
tags = module.tagging.tags
}
resource "aws_cloudwatch_log_group" "image2_log" {
name = "${var.image2}-log-group"
tags = module.tagging.tags
}
resource "aws_cloudwatch_log_stream" "image1_stream" {
name = "${var.image1}-log-stream"
log_group_name = aws_cloudwatch_log_group.image1_log.name
}
resource "aws_cloudwatch_log_stream" "image2_stream" {
name = "${var.image2}-log-stream"
log_group_name = aws_cloudwatch_log_group.image2_log.name
}
Then, 10-20 different log groups later, I realized this wasn't going to work well as infrastructure grew. I decided to define a variable list:
variable "image_names" {
type = list(string)
default = [
"image1",
"image2"
]
}
Then I replaced the resources using indices:
resource "aws_cloudwatch_log_group" "service-log-groups" {
name = "${element(var.image_names, count.index)}-log-group"
count = length(var.image_names)
tags = module.tagging.tags
}
resource "aws_cloudwatch_log_stream" "service-log-streams" {
name = "${element(var.image_names, count.index)}-log-stream"
log_group_name = aws_cloudwatch_log_group.service-log-groups[count.index].name
count = length(var.image_names)
}
The problem here is that when I run terraform apply, I get 4 resources to add, 4 resources to destroy. I tested this with an old log group, and saw that all my logs were wiped (obviously, since the log was destroyed).
The names and other attributes of the log groups/streams are identical- I'm simply refactoring the infrastructure code to be more maintainable. How can I maintain my existing log groups without deleting them yet still refactor my code to use lists?
You'll need to move the existing resources within the Terraform state.
Try running terraform show to get the strings under which the resources are stored, this will be something like [module.xyz.]aws_cloudwatch_log_group.image1_log ...
You can move it with terraform state mv [module.xyz.]aws_cloudwatch_log_group.image1_log '[module.xyz.]aws_cloudwatch_log_group.service-log-groups[0]'.
You can choose which index to assign to each resource by changing [0] accordingly.
Delete the old resource definition for each moved resource, as Terraform would otherwise try to create a new group/stream.
Try it with the first import and check with terraform plan if the resource was moved correctly...
Also check if you need to choose some index for the image_names list jsut to be sure, but I think that won't be necessary.