We have monitors and dashboards templated in Terraform that are used when creating new accounts and have found that ones using queries by "autoscaling_group" now report no data.
Looking in metrics I can see the only option for grouping by ASG is "aws_autoscaling_groupname" but can't seem to find where this is set. AWS Auto Scaling integration documentation also shows that this should be autoscaling_group.
Where can I set this?
If you're generating the Autoscaling group via the Terraform aws_autoscaling_group resource, then there's a name parameter that is distinct from the resource name.
An example that shows the difference:
resource "aws_placement_group" "prod-asg" {
name = "application123"
strategy = "cluster"
}
In this example, when generating dashboards, the ASG name you want to add to widgets would be application123, which should end up as the autoscaling_group name in Datadog.
If using Terraform to build the dashboard widgets, then the reference would be something like this:
resource "datadog_dashboard" "monitoring" {
title = "..."
widget {
type = "timeseries"
title = "..."
request {
q = "avg:aws.autoscaling.desired_capacity{name:${aws_autoscaling_group.prod-asg.name}}.as_count()"
}
}
}
Refs: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/autoscaling_group
Related
I have a service deployed on GCP compute engine. It consists of a compute engine instance template, instance group, instance group manager, and load balancer + associated forwarding rules etc.
We're forced into using compute engine rather than Cloud Run or some other serverless offering due to the need for docker-in-docker for the service in question.
The deployment is managed by terraform. I have a config that looks something like this:
data "google_compute_image" "debian_image" {
family = "debian-11"
project = "debian-cloud"
}
resource "google_compute_instance_template" "my_service_template" {
name = "my_service"
machine_type = "n1-standard-1"
disk {
source_image = data.google_compute_image.debian_image.self_link
auto_delete = true
boot = true
}
...
metadata_startup_script = data.local_file.startup_script.content
metadata = {
MY_ENV_VAR = var.whatever
}
}
resource "google_compute_region_instance_group_manager" "my_service_mig" {
version {
instance_template = google_compute_instance_template.my_service_template.id
name = "primary"
}
...
}
resource "google_compute_region_backend_service" "my_service_backend" {
...
backend {
group = google_compute_region_instance_group_manager.my_service_mig.instance_group
}
}
resource "google_compute_forwarding_rule" "my_service_frontend" {
depends_on = [
google_compute_region_instance_group_manager.my_service_mig,
]
name = "my_service_ilb"
backend_service = google_compute_region_backend_service.my_service_backend.id
...
}
I'm running into issues where Terraform is unable to perform any kind of update to this service without running into conflicts. It seems that instance templates are immutable in GCP, and doing anything like updating the startup script, adding an env var, or similar forces it to be deleted and re-created.
Terraform prints info like this in that situation:
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
~ update in-place
-/+ destroy and then create replacement
Terraform will perform the following actions:
# module.connectors_compute_engine.google_compute_instance_template.airbyte_translation_instance1 must be replaced
-/+ resource "google_compute_instance_template" "my_service_template" {
~ id = "projects/project/..." -> (known after apply)
~ metadata = { # forces replacement
+ "TEST" = "test"
# (1 unchanged element hidden)
}
The only solution I've found for getting out of this situation is to entirely delete the entire service and all associated entities from the load balancer down to the instance template and re-create them.
Is there some way to avoid this situation so that I'm able to change the instance template without having to manually update all the terraform config two times? At this point I'm even fine if it ends up creating some downtime for the service in question rather than a full rolling update or something since that's what's happening now anyway.
I was triggered by this issue as well.
However, according to:
https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_instance_template#using-with-instance-group-manager
Instance Templates cannot be updated after creation with the Google
Cloud Platform API. In order to update an Instance Template, Terraform
will destroy the existing resource and create a replacement. In order
to effectively use an Instance Template resource with an Instance
Group Manager resource, it's recommended to specify
create_before_destroy in a lifecycle block. Either omit the Instance
Template name attribute, or specify a partial name with name_prefix.
I would also test and plan with this lifecycle meta argument as well:
+ lifecycle {
+ prevent_destroy = true
+ }
}
Or more realistically in your specific case, something like:
resource "google_compute_instance_template" "my_service_template" {
version {
instance_template = google_compute_instance_template.my_service_template.id
name = "primary"
}
+ lifecycle {
+ create_before_destroy = true
+ }
}
So terraform plan with either create_before_destroy or prevent_destroy = true before terraform apply on google_compute_instance_template to see results.
Ultimately, you can remove google_compute_instance_template.my_service_template.id from state file and import it back.
Some suggested workarounds in this thread:
terraform lifecycle prevent destroy
As per subject, I have set up log based metrics for a platform in gcp i.e. firewall, audit, route etc. monitoring.
enter image description here
Now I need to setup alert policies tied to these log based metrics, which is easy enough to do manually in gcp.
enter image description here
However, I need to do it via terraform thus using this module:
https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/monitoring_alert_policy#nested_alert_strategy
I might be missing something very simple but finding it hard to understand this as the alert strategy is apparently required but yet does not seem to be supported?
I am also a bit confused on which kind of condition I should be using to match my already setup log based metric?
This is my module so far, PS. I have tried using the same filter as I did for setting up the log based metric as well as the name of the log based filter:
resource "google_monitoring_alert_policy" "alert_policy" {
display_name = var.display_name
combiner = "OR"
conditions {
display_name = var.display_name
condition_matched_log {
filter = var.filter
#duration = "600s"
#comparison = "COMPARISON_GT"
#threshold_value = 1
}
}
user_labels = {
foo = "bar"
}
}
var filter is:
resource.type="gce_route" AND (protoPayload.methodName:"compute.routes.delete" OR protoPayload.methodName:"compute.routes.insert")
Got this resolved in the end.
Turns out common issue:
https://issuetracker.google.com/issues/143436657?pli=1
Had to add this to the filter parameter in my terraform module after the metric name - AND resource.type="global"
I have the gcp-organization-admins Cloud Identity User Group to which I want to add a new user user-01#example.com as a Member using Terraform.
Getting error - Error creating GroupMembership: googleapi: got HTTP response code 404.
The requested URL /v1beta1/gcp-organization-admins#example.com/memberships?alt=json was not found on this server.
Can anyone suggest how to resolve this please.
fyi...Just as a test, I was able to create new Cloud Identity user groups and added some test users into it without any problems using Terraform module https://github.com/terraform-google-modules/terraform-google-group
#=====================
# terraform.tfvars
#=====================
org_admin_user = ["user-01#example.com"]
org_admin_group = "gcp-organization-admins#example.com"
#=========================================================
# add-member.tf (adds user to google group as a member)
#=========================================================
resource "google_cloud_identity_group_membership" "user-01" {
for_each = toset(var.org_admin_user)
provider = google-beta
group = var.org_admin_group
preferred_member_key {
id = each.key
}
roles {
name = "MEMBER"
}
}
I ran into this same problem. Turns out that Terraform wants the GCP Group "name" and not the email address.
So the group attribute of the google_cloud_identity_group_membership resource block should look something like "groups/23097432uwhwiyo" and not "gcp-organization-admins#example.com"
You can look up the group "name" with the following gcloud command:
gcloud identity groups describe "gcp-organization-admins#example.com"
This might to be the same issue as documented here: https://github.com/hashicorp/terraform-provider-google/issues/7616
A comment in that bug mentions the following:
I can work around the above issue by switching the order of the two roles in the resource, i.e.
From
roles { name = "MANAGER" }
roles { name = "MEMBER" }
to
roles { name = "MEMBER" }
roles { name = "MANAGER" }
https://github.com/hashicorp/terraform-provider-google/issues/7616#issuecomment-742779169
I want to create a resource group in AWS inspector with terraform, that has few tags with key "Name" and different values. I can do this with AWS GUI, but I Want to do it in terraform also. If I do it like in the example below, it will just override the name..
resource "aws_inspector_resource_group" "bar" {
tags = {
Name = "Master"
Name = "UF"
}
}
Could you please try the following:
resource "aws_inspector_resource_group" "bar" {
tags = {
Name = ["Master", "UF"]
}
}
If that doesn't work you could just use the aws_ec2_tag resource:
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_tag
Should do the job
just started exploring terraform to spin up droplets and volumes on digital ocean.
My question is to know the right way to do the following:
create a certain number of droplet instances using count within digitalocean_dropletresource named ubuntu16
assign a digitalocean_volume only to one or a subset of previously created droplets.
How to do it?I was assuming to use droplets_id property on digitalocean_volume resource. Something like:
resource "digitalocean_volume" "foovolume" {
...
droplet_ids = ["${digitalocean_droplet.ubuntu16.0.id}"]
}
Validating it with terraform validate I got:
Error: digitalocean_volume.foovolume: "droplet_ids": this field cannot be set
Any advice? Thanks to any inputs on it.
Regards
The way the Terraform provider for DigtialOcean is currently implemented requires that you take the opposite approach. You can specify which volumes are attached to which Droplets by defining the volume_ids of the Droplet resource. For example:
resource "digitalocean_volume" "volume" {
region = "nyc3"
count = 3
name = "volume-${count.index + 1}"
size = 100
description = "an example volume"
}
resource "digitalocean_droplet" "web" {
count = 3
image = "ubuntu-17-10-x64"
name = "web-${count.index + 1}"
region = "nyc3"
size = "1gb"
volume_ids = ["${element(digitalocean_volume.volume.*.id, count.index)}"]
}
If you look at the docs for the volume resource, you'll see that droplet_ids is a "computed" field. This means that you are unable to set the field, and that its value is computed by Terraform via the provider's API.