trying to do dataflow job through terraform but got error - google-cloud-platform

enter image description here
│ Error: googleapi: Error 404: (d958f8ba4ff671db): Unable to open template file: gs://sample_bucket_terraform/data_file., notFound
│
│ with google_dataflow_flex_template_job.big_data_job,
│ on main.tf line 13, in resource "google_dataflow_flex_template_job" "big_data_job":

In order to deploy flex template dataflow job, you have to build a flex template image or use an existing flex template, then refer the flex template metadata json file in your terraform
container_spec_gcs_path=gs://sample_bucket_terraform/data_file.json
refer this link for step by step process to create flex template
Note:
The data_file.json should be available in the gs://sample_bucket_terraform/ location.

Related

Error while restoring RDS DB Instance using terraform

Can someone please tell me what I'm doing wrong here. I am trying to import a newly-restored RDS instance into terraform state
I have application module that calls a core-module from company's repository
\applications\rds\main.tf
module "rds" {
source = "git::https://mysource-control/public/core-modules//aws/rds/rds?ref=v3.5.0"
}
then I have root environment specific module that calls application module
\env\us-west-2\qa\rds\main.tf
module "rds" {
source = "../../../../applications/rds"
}
On import I am getting error
C:\....\env\us-west-2\qa\rds>terraform import module.rds.module.rds qa-db-instane
Error: Invalid address │ │ on line 1: │ 1:
module.rds.module.rds │ │ A resource instance address is required
here. The module path must be followed by a resource instance
specification. ╵
For information on valid syntax, see:
https://www.terraform.io/docs/cli/state/resource-addressing.html
In the official documentation there is an example for importing RDS instances https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_instance#import
You'll notice they mention aws_db_instance.default which is the Terraform resource name, followed by whatever you've named it.
In your case if the underlying resource in your nested module (for example sake, say it's called my_db) you'll be able to go
terraform import module.rds.module.rds.aws_db_instance.my_db qa-db-instance

Terraform with GCP module

I begin to use terraform with GCP module, I am novice in Terraform.
I donwloaded and put GCP module like "Google VM" in my Terraform to use it.
I created main.tf file outside of google VM module, and I call this module like exemple provide in "exemple" folder
I copy/paste this example in my main.tf file with var.tfvars (where I put my variables)
when I do "terraform plan" I have errors like
│ Error: Reference to undeclared input variable│
│ on main.tf line 31, in module "instance_template":
│ 31: project_id = var.project_id│
│ An input variable with the name "project_id" has not been declared. This variable can be declared with a variable "project_id" {} block.
I think I have forget something....
Variables must be set in module and not in root module (tfvars) ?
thanks for your help

Terraform Modules - Recognizing child module variables from root

Versioning:
Terraform v0.15.0
on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v3.37.0
+ provider registry.terraform.io/hashicorp/random v3.1.0
I have the following folder structure:
terraform/
├── main.tf
└── terraform.tfvars
├── core
│ ├── main.tf
│ └── variables.tf
The files of concern here are:
terraform/main.tf
terraform/terraform.tfvars
core/main.tf
core/variables.tf
The high-level overview is I'm trying to use TF to create a VPC in AWS, using the terraform-aws-vpc module, so my root module is calling the core module which calls the terraform-provided vpc module to build. The reason it's a child calling a child is that once I get this vpc step working, the "core" module is going to be responsible for building other infrastructure assets in the account. The idea is to have the root module call a child module for each important piece of the AWS Account assets.
The problem I'm running into here is one of variables.
In core/, I have the following variables defined in core/variables.tf:
variable "vpc_facts" {
type = map
}
That variable is referenced for information for the vpc module in core/main.tf:
module "vpc" {
source = "terraform-aws-modules/vpc/aws"
name = var.vpc_facts.name
cidr = var.vpc_facts.cidr
}
I define these values, which I hope will be passed into the module, in terraform/terraform.tfvars:
vpc_facts = {
name = "demo-vpc"
cidr = "192.168.0.0/16"
}
I then call the core module from terraform/main.tf
module "core" {
source = "./core/"
}
I am trying to call terraform plan from my root folder of terraform/ - I have included the core as a module within terraform/main.tf
My thought process here is terraform/main.tf -> gathers terraform/terraform.tfvars -> core module -> uses terraform/terraform.tfvars as input variables for vpc module
However, that doesn't seem to be happening. I keep getting this error when running terraform plan from the root module folder:
Releasing state lock. This may take a few moments...
╷
│ Warning: Value for undeclared variable
│
│ The root module does not declare a variable named "vpc_facts" but a value was found in file "terraform.tfvars". If you meant to use
│ this value, add a "variable" block to the configuration.
│
│ To silence these warnings, use TF_VAR_... environment variables to provide certain "global" settings to all configurations in your
│ organization. To reduce the verbosity of these warnings, use the -compact-warnings option.
╵
╷
│ Error: Missing required argument
│
│ on main.tf line 57, in module "core":
│ 57: module "core" {
│
│ The argument "vpc_facts" is required, but no definition was found.
I would guess this should already be defined since I'm defining it in both tfvars and core/variables, but when I actually try to define it in the root main.tf:
module "core" {
source = "./core/"
vpc_facts = var.vpc_facts
}
I get a different error:
╷
│ Error: Reference to undeclared input variable
│
│ on main.tf line 60, in module "core":
│ 60: vpc_facts = var.vpc_facts
│
│ An input variable with the name "vpc_facts" has not been declared. This variable can be declared with a variable "vpc_facts" {}
│ block.
but it is declared, in core/variables.tf and valued in terraform/terraform.tfvars
What am I missing here? Does this mean I need to repeatedly define variables in both child modules and the root module? I would think that if a root module is calling a child module, it's a flat structure in terms of variables, and that child module can see child/variables.tf
Does this mean I need to repeatedly define variables in both child modules and the root module?
Yes, exactly what it means. All modules are self-contained, and a child module does not inherit variables from the parent. You have to explicitly define the variables in the module, and then set them in the parent module when you create the module.

Failed to load edgegrid config when importing Akamai property into Pulumi

I am trying out the new import functionality for an Akamai property. When I run the import command inside a pulumi project, I get the error message below. What is this .edgerc file and how can I create it?
Type Name Plan Info
+ pulumi:pulumi:Stack vdp-akamai-akamai-import create 1 error
= └─ akamai:index:Property example import 1 error
Diagnostics:
pulumi:pulumi:Stack (vdp-akamai-akamai-import):
error: preview failed
akamai:index:Property (example):
error: Preview failed: 1 error occurred:
* failed to load edgegrid config: unable to load config from environment or .edgerc file: loading config file: open /Users/vcui/.edgerc: no such file or directory
From the Akamai documentation, it looks like it's used as a config file that you can store your credentials in.
Documentation found here

Create a new project using outputs from Terraformer

Currently trying out a tool called Terraformer (it's a reverse Terraform) - https://github.com/GoogleCloudPlatform/terraformer.
I have a simple GCP project called test-one which only has one resource, vm_instance (google_compute_instance). I ran Terraformer and managed to get the outputs:
$ generated/google/test-one/instances/us-central1
.
├── compute_instance.tf
├── outputs.tf
├── provider.tf
└── terraform.tfstate
My question is - What should I do next if say I want to have the same exact configuration but for a new group that I'm going to name as test-two?
Should I go to each file, replace anything that has the string "test-one" to "test-two", and then perform terraform plan and terraform apply?
You need to create a terraform module which will deploy whatever enviroment you want and will take as few parameters as possible, only name (e.g: "test-two") if possible.
Converting your current state to use module is not the easiest , but is usually possible without destroying any resource when using terraform import
I would also recommend watching this video