GCP: Output CSV file using local-exec in terraform - google-cloud-platform

Question

I am trying to create an reporting output file where I am listing all the buckets of different GCP projects.
The challenge is neither it prints out the output on the stdout nor it creates the file.
I can confirm that one bucket is there in the project for testing purposes.
Here is the code:
main.tf
data "google_client_config" "default" {}
resource "null_resource" "list_all_buckets" {
triggers = {
filename = "${path.module}/storage_output.csv"
}
provisioner "local-exec" {
command ="bash list_buckets.sh ${data.google_client_config.default.access_token} '*<gcp project name in single quote>*' >> storage_ouput.csv"
working_dir = "${path.module}"
}
}
data "local_file" "test" {
filename = "${null_resource.list_all_buckets.triggers.filename}"
}
output "result" {
value = "${data.local_file.test.content}"
}
list_buckets.sh
#!/bin/bash
readonly TOKEN="$1"
readonly PROJECT="$2"
readonly URL="https://storage.googleapis.com/storage/v1/b?project=${2}"
LIST_BUCKETS="$(curl -X GET -H "Authorization: Bearer "${TOKEN} ${URL})"
Please feel free to ask any questions.
I am expecting a storage_ouput.csv file to be created in the existing module directory.
Output Truncated
You can apply this plan to save these new output values to the Terraform
state, without changing any real infrastructure.
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
result = ""
Cleaning up project directory and file based variables
00:00
Job succeeded

Answer 1

provisioners in Terraform only run on creation or deletion time (defaulted to on create), which means that your script will be only called in the first time your run the plan. That run should create a storage_ouput.csv file (which has a typo, unlike the path in your triggers storage_output.csv).
For testing purposes only I recreated the following:
resource "null_resource" "list_all_buckets" {
triggers = {
filename = "${path.module}/storage_output.csv"
}
provisioner "local-exec" {
command = "date >> ${path.module}/storage_output.csv"
working_dir = path.module
}
}
data "local_file" "test" {
filename = null_resource.list_all_buckets.triggers.filename
}
output "result" {
value = data.local_file.test.content
}
which runs date command and writes it to storage_output.csv. When running this for the first time it creates the CSV file on the current directory, but if I run it again nothing will happen since the null_resource was already created.
To force-recreating this I would need to taint it, or I run:
terraform apply --replace null_resource.list_all_buckets
Which force recreating null_resource, thus executing the script in your provisioned.
As Hashicorp mention in their documentation:
Use provisioners as a last resort. There are better alternatives for most situations.
You may want to have a look at Google Storage's data sources like:
google_storage_bucket
Or, if you created these buckets with Terraform you can list your resources with show or state list. And if they are created in different workspaces you could pull them using terraform_remote_state