GCP terraform compute instance template labels - google-cloud-platform

I have a TF GCP google_compute_instance_template configured to deploy a range of individual VMs, each of which will perform a different role in a "micro-services" style application. I am adding a single label to my instance_template, costing="app". However when I go to deploy the various VM components of the app with google_compute_instance_group_manager, I was expecting to be able to add another label in the in the instance group manager configuration, specific to the VM that is being deployed, such as "component=blah". The google_compute_instance_group_manager is not talking labels as a configuration element. Does anyone know how I can use the template to add a generic label, but then add additional machine-specific labels when the VMs are created?
Here is the TF code:
// instance template
resource "google_compute_instance_template" "app" {
name = "appserver-template"
machine_type = var.app_machine_type
labels = {
costing = "app"
}
disk {
source_image = data.google_compute_image.debian_image.self_link
auto_delete = true
boot = true
disk_size_gb = 20
}
tags = ["compute", "app"]
network_interface {
subnetwork = var.subnetwork
}
// no access config
service_account {
email = var.service_account_email
// email = google_service_account.vm_sa.email
scopes = ["cloud-platform"]
}
}
// create instances --how to add instance-specific label here? eg component="admin"
resource "google_compute_instance_group_manager" "admin" {
provider = google-beta
name = "admin-igm"
base_instance_name = "${var.project_short_name}-admin"
zone = var.zone
target_size = 1
version {
name = "appserver"
instance_template = google_compute_instance_template.app.id
}
}

I got the desired outcome by creating a google_compute_instance_template for each server type in my application, which then allowed me to assign both a universal label and a component-specific label. It was more code than I hoped to have to write, but the objective is met.
resource "google_compute_instance_template" "admin" {
name = "admin-template"
machine_type = var.app_machine_type
labels = {
costing = "app",
component = "admin"
}
disk {
source_image = data.google_compute_image.debian_image.self_link
auto_delete = true
boot = true
disk_size_gb = 20
}
tags = ["compute"]
network_interface {
subnetwork = var.subnetwork
}
// no access config
service_account {
email = var.service_account_email
// email = google_service_account.vm_sa.email
scopes = ["cloud-platform"]
}
}

Related

Is it possible to get the terraform state file from the remote backend and set it for modifications

I need to get the terraform state file from the backend GCS and to use it while I update the resource is it possible to overwrite the excising terraform state file and fetch it on need.
this is my main.tf
provider "google" {
project = var.project
region = var.region
zone = var.zone
}
###################################################
########################################################
data "google_compute_default_service_account" "default" {
}
#create instance1
resource "random_id" "bucket_prefix" {
byte_length = 8
}
#create instance
resource "google_compute_instance" "vm_instance" {
name = var.instance_name
machine_type = var.machine_type
zone = var.zone
metadata_startup_script = var.script
allow_stopping_for_update = true
#metadata = {
# enable-oslogin = "TRUE"
#}
service_account {
email = data.google_compute_default_service_account.default.email
scopes = ["cloud-platform"]
}
boot_disk {
initialize_params {
image = var.image
#image = "ubuntu-2004-lts" # TensorFlow Enterprise
size = 30
}
}
# Install Flask
tags = ["http-server","allow-ssh-ingress-from-iap", "https-server"]
network_interface {
network = "default"
access_config {
}
}
guest_accelerator{
#type = "nvidia-tesla-t4" // Type of GPU attahced
type = var.type
count = var.gpu_count
#count = 2 // Num of GPU attached
}
scheduling{
on_host_maintenance = "TERMINATE"
automatic_restart = true// Need to terminate GPU on maintenance
}
}
This is my variables.tfvars:
instance_name = "test-vm-v5"
machine_type = "n1-standard-16"
region = "europe-west4"
zone = "europe-west4-a"
image = "tf28-np-pandas-nltk-scikit-py39"
#image = "debian-cloud/debian-10"
project = "my_project"
network = "default"
type = ""
gpu_count = "0"
I wanted to create multiple instances by changing the variables.tfvars and need to modify the instance on the basis of the name of vm.

GCP Terraform error creating multiple VMs from Instance Template with "additional_disks"

I have a requirement to create multiple VMs in GCP using the Instance Template module located here:
https://github.com/terraform-google-modules/terraform-google-vm/tree/master/modules/instance_template
My Instance Template code looks like this:
module "db_template" {
source = "terraform-google-modules/vm/google//modules/instance_template"
version = "7.8.0"
name_prefix = "${var.project_short_name}-db-template"
machine_type = var.app_machine_type
disk_size_gb = 20
source_image = "debian-10-buster-v20220719"
source_image_family = "debian-10"
source_image_project = "debian-cloud"
additional_disks = var.additional_disks
labels = {
costing = "db",
inventory = "gcp",
}
network = var.network
subnetwork = var.subnetwork
access_config = []
service_account = {
email = var.service_account_email
scopes = ["cloud-platform"]
}
tags = ["compute"]
}
in my tfvars I have this:
additional_disks = [
{ disk_name = "persistent-disk-1"
device_name = "persistent-disk-1"
auto_delete = true
boot = false
disk_size_gb = 50
disk_type = "pd-standard"
interface = "SCSI"
disk_labels = {}
}
]
However when my code has multiple VMs to deploy with this template, only 1 VM gets deployed--the first--and the subsequent VMs error out with this message:
Error: Error creating instance: googleapi: Error 409: The resource 'projects/<PATH>/persistent-disk-1' already exists, alreadyExists
I understand what is happening but I don't know how to fix it. The subsequent VMs cannot be created because the additional_disk name has already been taken by the first VM. I thought the whole point of using the instance template would be that there is logic built into this where you can use the same template and create multiple VMs of that type.
But it seems like I have to do some additional coding to get multiple VMs deployed with this template.
Can anyone suggest how to do this?
Ultimately got this working with various for_each constructs:
locals {
app_servers = ["inbox", "batch", "transfer", "tools", "elastic", "artemis"]
db_servers = ["inboxdb", "batchdb", "transferdb", "gatewaydb", "artemisdb"]
}
resource "google_compute_disk" "db_add_disk" {
for_each = toset(local.db_servers)
name = "${each.value}-additional-disk"
type = "pd-standard" // pd-ssd
zone = var.zone
size = 50
// interface = "SCSI"
labels = {
environment = "dev"
}
physical_block_size_bytes = 4096
}
module "db_template" {
source = "terraform-google-modules/vm/google//modules/instance_template"
version = "7.8.0"
name_prefix = "${var.project_short_name}-db-template"
machine_type = var.app_machine_type
disk_size_gb = 20
source_image = "debian-10-buster-v20220719"
source_image_family = "debian-10"
source_image_project = "debian-cloud"
labels = {
costing = "db",
inventory = "gcp",
}
network = var.network
subnetwork = var.subnetwork
access_config = []
service_account = {
email = var.service_account_email
scopes = ["cloud-platform"]
}
tags = ["compute"]
}
resource "google_compute_instance_from_template" "db_server-1" {
for_each = toset(local.db_servers)
name = "${var.project_short_name}-${each.value}-1"
zone = var.zone
source_instance_template = module.db_template.self_link
// Override fields from instance template
labels = {
costing = "db",
inventory = "gcp",
component = "${each.value}"
}
lifecycle {
ignore_changes = [attached_disk]
}
}
resource "google_compute_attached_disk" "db_add_disk" {
for_each = toset(local.db_servers)
disk = google_compute_disk.db_add_disk[each.key].id
instance = google_compute_instance_from_template.db_server-1[each.key].id
}

Add existing Service account to terraform Google compute instance

Hello I am trying to add an existing service account to a new Google compute instance I am spinning up.
How can I attach the already existing account?
Here is what my file looks like
provider "google" {
project = var.project
region = "us-central1"
zone = "us-central1-c"
}
resource "google_compute_disk" "data_disk" {
name = "data-test"
size = 120
}
resource "google_compute_disk" "other_data_disk" {
name = "other-data-test"
size = 400
}
resource "google_compute_attached_disk" "data" {
disk = google_compute_disk.data_disk.id
instance = google_compute_instance.test_vm_instance.id
}
resource "google_compute_attached_disk" "other_data" {
disk = google_compute_disk.other_data_disk.id
instance = google_compute_instance.tyler_test_vm_instance.id
}
resource "google_compute_instance" "test_vm" {
name = "worker-test"
machine_type = "n2-standard-2"
min_cpu_platform = "Intel Ice Lake"
boot_disk {
initialize_params {
image = "ubuntu-2004-focal-v20210211"
}
}
network_interface {
network = var.gcp_project_network_name
subnetwork = var.subnet_name
#access_config {
#}
}
metadata_startup_script = "${file("./startup-script.sh")}"
What's the best way to insert add an already existing service account to the new VM.
I want to be able to create and destroy the VM without worrying about deleting this service account.
Add to this resource:
resource "google_compute_instance" "test_vm" {
...
service_account {
email = REPLACE_WITH_EXISTING_SERVICE_ACCOUNT_EMAIL_ADDRESS
scopes = ["cloud-platform"]
}
...
}

GCP Compute Engine using Terraform

How to provision multiple instances in GCP Compute Engine using Terraform. I've tried using 'count' parameter in the resource block. But terraform is not provisioning more than one instance because the VM with a particular name is created once when first count is executed.
provider "google" {
version = "3.5.0"
credentials = file("battleground01-5c86f5873d44.json")
project = "battleground01"
region = "us-east1"
zone = "us-east1-b"
}
variable "node_count" {
default = "3"
}
resource "google_compute_instance" "appserver" {
count = "${var.node_count}"
name = "battleground"
machine_type = "f1-micro"
boot_disk {
initialize_params {
image = "debian-cloud/debian-9"
}
}
network_interface {
network = "default"
}
}
In order for this to work, you would have to make a slight change in the way you are naming your compute instances:
provider "google" {
version = "3.5.0"
credentials = file("battleground01-5c86f5873d44.json")
project = "battleground01"
region = "us-east1"
zone = "us-east1-b"
}
variable "node_count" {
type = number
default = 3
}
resource "google_compute_instance" "appserver" {
count = var.node_count
name = "battleground-${count.index}"
machine_type = "f1-micro"
boot_disk {
initialize_params {
image = "debian-cloud/debian-9"
}
}
network_interface {
network = "default"
}
}
As you are using the count meta-argument, the way to access the array index is by using the count.index attribute [1]. You have also set the node_count variable default value to be a string and even though it would probably get converted to a number by Terraform, make sure to use the right variable types.
[1] https://www.terraform.io/language/meta-arguments/count#the-count-object

Terraform GCP when creating instance template, Error getting relative path for source image

I have a new issue with setting up GCP instance template. I am presuming there was an update on the terraform gcp provider.
resource "google_compute_instance_template" "backend-template" {
name = "${var.platform_name}-backend-instance-template"
description = "Template used for backend instances"
instance_description = "backend Instance"
machine_type = "n1-standard-1"
metadata_startup_script = "${lookup(var.startup_scripts,"backend-server")}"
disk {
boot = "true"
source_image = "backend-packer-image"
}
metadata {
APP_SETTINGS = "${var.app_settings}"
URL_STAGING = "${var.url_staging}"
API_URL_STAGING = "${var.api_url_staging}"
URL_PRODUCTION = "${var.url_production}"
API_URL_PRODUCTION = "${var.api_url_production}"
LOGIN_URL = "${var.login_url}"
API_URL = "${var.api_url}"
vault_server_IP = "${lookup(var.static_ips, "vault-server")}"
environment = "${var.environment}"
}
network_interface {
subnetwork = "${google_compute_subnetwork.private-fe-be.self_link}"
}
lifecycle {
create_before_destroy = true
}
tags = ["no-ip", "backend-server"]
service_account {
scopes = ["cloud-platform"]
}
}
This is the current error after running the script. However, the image backend-packer-image was already created and exists on GCP
* google_compute_instance_template.backend-template: 1 error(s) occurred:
* google_compute_instance_template.backend-template: error flattening disks: Error getting relative path for source image: String was not a self link: global/images/backend-packer-image
I had the exact same problem today, I had to go look directly into the pull request to find a way to use this correctly.
So, what I came with is this:
you must first be sure to be in your project before typing this command or you won't find the image you are looking for if it's a custom one:
gcloud compute images list --uri | grep "your image name"
Like this you will have the uri of your image, you can then put it fully for the image and it will work.
Replace the image name with the URI on source_image
resource "google_compute_instance_template" "backend-template" {
name = "${var.platform_name}-backend-instance-
template"
description = "Template used for backend instances"
instance_description = "backend Instance"
machine_type = "n1-standard-1"
metadata_startup_script = "${lookup(var.startup_scripts,"backend-server")}"
disk {
boot = "true"
source_image = "https://www.googleapis.com/compute/v1/projects/<project-name>/global/images/backend-packer-image"
}
metadata {
APP_SETTINGS = "${var.app_settings}"
URL_STAGING = "${var.url_staging}"
API_URL_STAGING = "${var.api_url_staging}"
URL_PRODUCTION = "${var.url_production}"
API_URL_PRODUCTION = "${var.api_url_production}"
LOGIN_URL = "${var.login_url}"
API_URL = "${var.api_url}"
vault_server_IP = "${lookup(var.static_ips, "vault-server")}"
environment = "${var.environment}"
}
network_interface {
subnetwork = "${google_compute_subnetwork.private-fe-be.self_link}"
}
lifecycle {
create_before_destroy = true
}
tags = ["no-ip", "backend-server"]
service_account {
scopes = ["cloud-platform"]
}
}
It is also possible to tie the terraform scripts to run previous versions
provider "google"{
version = "<= 1.17"
credentials = "${var.service_account_path}"
project = "${var.gcloud_project}"
region = "${var.gcloud_region}"
}