Terraform/GCP Kubernetes error when trying to create namespace - google-cloud-platform

Question

getting the below error with my terraform config.
Error: Post "https://35.224.178.141/api/v1/namespaces": x509: certificate signed by unknown authority
on main.tf line 66, in resource "kubernetes_namespace" "example":
66: resource "kubernetes_namespace" "example" {
Here is my config, all I want to do for now is create a cluster auth with it, and create a namespace.
I have searched everyone and cant see where anyone else has run into this problem.
It is most likely something stupid I am doing. I thought this would be relatively simple, but its turning out to be a pain. I dont want to have to wrap gcloud commands in my build script.
provider "google" {
project = var.project
region = var.region
zone = var.zone
credentials = "google-key.json"
}
terraform {
backend "gcs" {
bucket = "tf-state-bucket-devenv"
prefix = "terraform"
credentials = "google-key.json"
}
}
resource "google_container_cluster" "my_cluster" {
name = var.kube-clustername
location = var.zone
remove_default_node_pool = true
initial_node_count = 1
master_auth {
username = ""
password = ""
client_certificate_config {
issue_client_certificate = false
}
}
}
resource "google_container_node_pool" "primary_preemptible_nodes" {
name = var.kube-poolname
location = var.zone
cluster = google_container_cluster.my_cluster.name
node_count = var.kube-nodecount
node_config {
preemptible = var.kube-preemptible
machine_type = "n1-standard-1"
disk_size_gb = 10
disk_type = "pd-standard"
metadata = {
disable-legacy-endpoints = "true",
}
oauth_scopes = [
"https://www.googleapis.com/auth/logging.write",
"https://www.googleapis.com/auth/monitoring",
]
}
}
data "google_client_config" "provider" {}
provider "kubernetes" {
load_config_file = false
host = "https://${google_container_cluster.my_cluster.endpoint}"
cluster_ca_certificate = "{base64decode(google_container_cluster.my_cluster.master_auth.0.cluster_ca_certificate)}"
token = "{data.google_client_config.provider.access_token}"
}
resource "kubernetes_namespace" "example" {
metadata {
name = "my-first-namespace"
}
}

Answer 1

TL;DR
Change the provider definition to:
provider "kubernetes" {
load_config_file = false
host = "https://${google_container_cluster.my_cluster.endpoint}"
cluster_ca_certificate = base64decode(google_container_cluster.my_cluster.master_auth.0.cluster_ca_certificate)
token = data.google_client_config.provider.access_token
}
What changed?
The "{}" was deleted from the cluster_ca_certificate and token values
I included the explanation below.
I used your original terraform file and I received the same error as you. I modified (simplified) your terraform file and added the output definitions:
resource "google_container_cluster" "my_cluster" {
OMMITED
}
data "google_client_config" "provider" {}
provider "kubernetes" {
load_config_file = false
host = "https://${google_container_cluster.my_cluster.endpoint}"
cluster_ca_certificate = "{base64decode(google_container_cluster.my_cluster.master_auth.0.cluster_ca_certificate)}"
token = "{data.google_client_config.provider.access_token}"
}
output "cert" {
value = "{base64decode(google_container_cluster.my_cluster.master_auth.0.cluster_ca_certificate)}"
}
output "token" {
value = "{data.google_client_config.provider.access_token}"
}
Running above file showed:
$ terraform apply --auto-approve
data.google_client_config.provider: Refreshing state...
google_container_cluster.my_cluster: Creating...
google_container_cluster.my_cluster: Creation complete after 2m48s [id=projects/PROJECT-NAME/locations/europe-west3-c/clusters/gke-terraform]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Outputs:
cert = {base64decode(google_container_cluster.my_cluster.master_auth.0.cluster_ca_certificate)}
token = {data.google_client_config.provider.access_token}
As you can see the values were interpreted as strings from the provider and not "processed" to get the needed values. To fix that you will need change the provider definition to:
cluster_ca_certificate = base64decode(google_container_cluster.my_cluster.master_auth.0.cluster_ca_certificate)
token = data.google_client_config.provider.access_token
Running $ terraform apply --auto-approve once again:
data.google_client_config.provider: Refreshing state...
google_container_cluster.my_cluster: Creation complete after 3m18s [id=projects/PROJECT-NAME/locations/europe-west3-c/clusters/gke-terraform]
kubernetes_namespace.example: Creating...
kubernetes_namespace.example: Creation complete after 0s [id=my-first-namespace]
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
Outputs:
cert = -----BEGIN CERTIFICATE-----
MIIDKzCCAhOgAwIBAgIRAO2bnO3FU6HZ0T2u3XBN1jgwDQYJKoZIhvcNAQELBQAw
<--OMMITED-->
a9Ybow5tZGu+fqvFHnuCg/v7tln/C3nVuTbwa4StSzujMsPxFv4ONVl4F4UaGw0=
-----END CERTIFICATE-----
token = ya29.a0AfH6SMBx<--OMMITED-->fUvCeFg
As you can see the namespace was created. You can check it by running:
$ gcloud container clusters get-credentials CLUSTER-NAME --zone=ZONE
$ kubectl get namespace my-first-namespace
Output:
NAME STATUS AGE
my-first-namespace Active 3m14s
Additional resources:
Terraform.io: Docs: Configuration: Outputs
Terraform.io: Docs: Configuration: Variables