I read this article on using Terraform with GCP:
https://cloud.google.com/community/tutorials/managing-gcp-projects-with-terraform
I almost have it working, but I ran into some issues and wanted some clarification.
I made a terraform admin project, and made a service account in that project with the roles/viewer and roles/storage.admin roles. I then made a bucket in the admin project and use that as the terraform backend storage.
terraform {
backend "gcs" {
bucket = "test-terraform-admin-1"
prefix = "terraform/state"
credentials = "service-account.json"
}
}
I then use that service account to create another project and provision resources in that project:
provider "google" {
alias = "company_a"
credentials = "./service-account.json"
region = "us-east4"
zone = "us-east4-c"
version = "~> 2.12"
}
resource "google_project" "project" {
name = var.project_name
project_id = "${random_id.id.hex}"
billing_account = "${var.billing_account}"
org_id = "${var.org_id}"
}
I thought that it would be sufficient to enable services for the project created with terraform like this:
resource "google_project_service" "container_service" {
project = "${google_project.project.project_id}"
service = "container.googleapis.com"
}
However, I got an error when terraform tried to create my gke cluster:
resource "google_container_cluster" "primary" {
project = "${google_project.project.project_id}"
name = "main-gke-cluster"
node_pool {
....
}
network = "${google_compute_network.vpc_network.self_link}"
}
It said that the container service was not enabled yet for my project, and it referenced the terraform admin project ID (not the project created with the google_project resource!). It seems that I have to enable the services on the terraform admin project in order for the service account to access those services on any projects created by the service account.
In fact, I can get it working without ever enabling the container, servicenetworking, etc. services on the create project as long as they are enabled on the terraform admin project.
Is there some parent/child relationship between the projects where services in one project are inherited by projects created from a service account in the parent project? This seems to be the case, but I cannot find any documentation about this anywhere.
Thanks for listening!
In my company, we created a folder and a service account in this folder. Then, we created a project for terraform and each terraform job use the folder level service account for creating projects and resources into this folder.
As the role and permission are inherited from folder to lower level (folders or projects) we don't have issue to create resources.
I don't if it helps your specific issue, but for us, it solved a lot and simplify the service account management.
Related
To be Specific, I have two projects A and B. I want to add an IAM role to service account from project A in Project B. I'm executing terraform script from bitbucket pipeline
Below is the resource block I tried to implement.
resource "google_project_iam_member" "role1" {
project = var.project
role = "roles/dialogflow.admin"
member = "user:cui-server-service-account#cproject.iam.gserviceaccount.com"
}
project is the variable used for Project B and cproject is a variable used for Project A which I'll pass the project names during bitbucket pipeline execution.
member = "serviceAccount:<<service account email>>"
The prefix is to be serviceAccount rather than user from your example - see how the member/members argument is described
And check that the user (or service account) who (which) runs the terraform 'apply' - has relevant IAM roles to assign what you would like.
I have a brand new GCP account that I'm the only owner of, this is a personal/clean brand new project.
I manage the infrastructure exclusively with terraform and trying to create a HTTPS load balancer to route requests to a fixed ip, which I then want to forward to a bucket where I have a webapp, I've setup this type of infrastructure in the past via the UI but not via terraform.
When attempting to create this piece of infrastructure:
resource "google_compute_global_address" "main" {
name = "main"
}
resource "google_compute_forwarding_rule" "lb" {
name = "lb"
provider = google-beta
region = "europe-west2"
project = "owlee-software"
ip_protocol = "TCP"
load_balancing_scheme = "EXTERNAL_MANAGED"
port_range = "443"
target = google_compute_global_address.main.address
network_tier = "PREMIUM"
}
this throws
│ Response: {
│ "error": "invalid_grant",
│ "error_description": "Account has been deleted"
│ }
for brand new service accounts -- I have deleted the account and created another one -- with another file, if I remove this resource everything gets instantiated fine via the service account in the GCP account. This is the only 'google beta' provider that I have in the project as well.
I'm not sure if this is because of the resource configuration being wrong or something is happening with the service account.
When I try to inspect the service account I just created and made editor on the project, I can't, it shows greyed out with the following text:
Could not find an ancestor of the selected project where you have access to view a policy report. On at least one ancestor, you need the following permissions:
cloudasset.assets.analyzeIamPolicy,
cloudasset.assets.searchAllIamPolicies,
and cloudasset.assets.searchAllResources
My single-main account has the following roles attached to it:
Cloud Asset Owner
Organization Administrator
Owner
Project IAM Admin
Security Admin
I've added all of them after the Owner didn't seem to be enough.
Any suggestions appreciated, thank you.
It's an authentication issue. For me what solved was running.
gcloud auth application-default login
Followed by the terraform apply
But you can check here the documentation on how to authenticate with terraform
I’m trying to create all new sandbox project in GCP for easy deployment and upgrade project. Using Terraform I am creating a GKE cluster. Issue is, the terraform scripts are written for the service accounts of a project named let’s say NP-H. Now, I am trying to create clusters using the same scripts in a project named let’ say NP-S.
I ran Terraform init and experiencing an
error 403: XXX.serviceaccount does not have storage.object.create access to google cloud storage objects., forbidden.
storage: object doesn’t exist.
Now, is the problem with Terraform script or service account permissions?
If it is Terraform script, what are the changes I need to make?
PS: I was able to create a buckets and upload them to cloud storage…
Two ways you can store credentials:
provider "google" {
credentials = file("service-account-file.json")
project = "PROJECT_ID"
region = "us-central1"
zone = "us-central1-c"
}
or
export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/service-account-file.json"
Make sure service account is from project ID NP-S, Menu > IAM & admin > service account, and has proper permissions: Menu > IAM & admin > IAM > ...
cat service-account-file.json
and make sure it is the email from correct project ID. You can do a quick test with owner/editor role to isolate the issue if need be as those role have most permissions.
If you're using service account impersonation, do this :
terraform {
backend "gcs" {
bucket = "<your-bucket>"
prefix = "<prefix>/terraform.tfstate"
impersonate_service_account = "<your-service-account>#<your-project-id>.iam.gserviceaccount.com"
}
}
Source : Updating remote state files with a service account
I'm pretty new to Terraform, my apologies if this question has an obvious answer I'm missing.
I am trying to create a terraform configuration file for an existing organization. I am able to provision everything I have in the main.tf outlined bellow except for the Shared folder that already exists within this organization.
Related github issues :
The folder operation violates display name uniqueness within the parent.
Generic error message when folder rename matches existing folder
Here are the steps I followed:
Manually create a Shared folder within the organization administration UI.
Manually create a Terraform admin project <redacted-project-name> at the root of the Shared folder.
Manually create a service account named terraform#<redacted-project-name> from the terraform admin project
Create, download and securely store a key for the terraform#<redacted-project-name> service account.
Enable APIs : cloudresourcemanager.googleapis.com, cloudbilling.googleapis.com, iam.googleapis.com, serviceusage.googleapis.com within the terraform admin project
Set permissions of the service account to role/owner, roles/resourcemanager.organizationAdmin, roles/resourcemanager.folderAdmin and roles/resourcemanager.projectCreator.
Create the main.tf
terraform {
required_providers {
google = {
source = "hashicorp/google"
version = "3.85.0"
}
}
}
provider "google" {
credentials = file(var.credentials_file)
region = var.region
zone = var.zone
}
data "google_organization" "org" {
organization = var.organization.id
}
resource "google_folder" "shared" {
display_name = "Shared"
parent = data.google_organization.org.name
}
resource "google_folder" "ddm" {
display_name = "Data and Digital Marketing"
parent = data.google_organization.org.name
}
resource "google_folder" "dtl" {
display_name = "DTL"
parent = google_folder.ddm.name
}
The error I receive :
Error: Error creating folder 'Shared' in 'organizations/<redacted-org-id>': Error waiting for creating folder: Error code 9, message: Folder reservation failed for parent [organizations/<redacted-org-id>], folder [] due to constraint: The folder operation violates display name uniqueness within the parent.
How do I include existing resources within the terraform config file?
For (organization) folders (such as the example above)
For the billing account
For projects, i.e. Am I supposed to declare or import the terraform admin project within the main.tf?
For service accounts, how to handle existing keys and permissions of the account that is running the terraform apply
For existing policies and enabling APIs
In order to include already-existing resources within the terraform template, use the import statement.
For Folders
In the Terraform documentation for google_folder :
# Both syntaxes are valid
$ terraform import google_folder.department1 1234567
$ terraform import google_folder.department1 folders/1234567
So for the example above,
Fetch the folder id using gcloud alpha resource-manager folders list --organization=<redacted_org_id> providing the organization id.
Save the folder id somewhere, and if not already done, declare the folder as a resource within the main.tf
resource "google_folder" "shared" {
display_name = "Shared"
parent = data.google_organization.org.name
}
Run the command : terraform import google_folder.shared folders/<redacted_folder_id>. You should get an output like google_folder.shared: Import prepared!
Make sure your infrastructure is updated via terraform plan.
No changes. Your infrastructure matches the configuration.
Terraform has compared your real infrastructure against your configuration
and found no differences, so no changes are needed.
I am planning to use terraform to deploy to GCP and I have read the instruction on how to set it up:
provider "google" {
project = "{{YOUR GCP PROJECT}}"
region = "us-central1"
zone = "us-central1-c"
}
it requires a project name in the provider configuration. But I am planning to create the project via terraform like below code:
resource "google_project" "my_project" {
name = "My Project"
project_id = "your-project-id"
org_id = "1234567"
}
how can I use terraform without a pre-created project?
Take a look on this tutorial (from Community):
Creating Google Cloud projects with Terraform
This tutorial assumes that you already have a Google Cloud account set up for your organization and that you are allowed to make organization-level changes in the account
First step,for example, is to setup your ENV variables with your Organization ID and your billing account ID which will allow you to create the projects using terraform:
export TF_VAR_org_id=YOUR_ORG_ID
export TF_VAR_billing_account=YOUR_BILLING_ACCOUNT_ID
export TF_ADMIN=${USER}-terraform-admin
export TF_CREDS=~/.config/gcloud/${USER}-terraform-admin.json