Terraform 0.15 - Multiple Providers \ Regions and Guardduty - amazon-web-services

I’m trying to deploy AWS Guardduty using Organisations to multiple regions.
In my root config I’ve created the following provider:
# If I remove this default provider out i get prompted for a region
provider "aws" {
profile = "default"
region = var.region
}
provider "aws" {
profile = "default"
alias = "eu-west-2"
region = "eu-west-2"
}
provider "aws" {
profile = "default"
alias = "eu-west-3"
region = "eu-west-3"
}
then in my module call I have multiple calls to the module passing in my providers alias’s
module "guardduty_orgs_eu_west_2" {
source = "../../modules/aws_guardduty_organisations"
security_account_id = var.security_account_id
providers = {
aws.alternate = aws.eu-west-2
}
}
module "guardduty_orgs_eu_west_3" {
source = "../../modules/aws_guardduty_organisations"
security_account_id = var.security_account_id
providers = {
aws.alternate = aws.eu-west-3
}
}
In my module I then have the required providers block and ‘configuration_aliases’
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 3.27"
configuration_aliases = [ aws.alternate ]
}
}
}
and finally my resource
resource "aws_guardduty_organization_admin_account" "gdoaa" {
admin_account_id = var.security_account_id
provider = aws.alternate
}
However, i get an error :
" Error: error enabling GuardDuty Organization Admin Account (123456789): BadRequestException: The request failed because the account is already enabled as the GuardDuty delegated administrator for the organization."
Now, this is correct as the first module call enables the Admin Account for “eu-west-2”, but i would think passing in the 2nd provider for “eu-west-3” would enable the Admin Account for this region as per the Guardduty best practices \ docs.
Any help appreciated
cheers
Paul

/*resource "aws_guardduty_detector" "MyDetector" {
enable = true
datasources {
s3_logs {
enable = false
}
kubernetes {
audit_logs {
enable = false
}
}
}
}
*/
resource "aws_guardduty_organization_configuration" "example" {
provider = aws.securityacc
auto_enable = true
detector_id = "12345678"
}
this worked for me. hashout guardduty detector as it gets enabled already when you delegate it as a admin account.

Related

Terraform loop through multiple providers(accounts) - invokation through module

i have a use case where need help to use for_each to loop through multiple providers( AWS accounts & regions) and this is a module, the TF will be using hub and spoke model.
below is the TF Pseudo code i would like to achieve.
module.tf
---------
app_accounts = [
{ "account" : "53xxxx08", "app_vpc_id" : "vpc-0fxxxxxfec8", "role" : "xxxxxxx", "profile" : "child1"},
{ "account" : "53xxxx08", "app_vpc_id" : "vpc-0fxxxxxfec8", "role" : "xxxxxxx", "profile" : "child2"}
]
below are the provider and resource files, pleas ignore the variables and output files, as its not relevant here
provider.tf
------------
provider "aws" {
for_each = var.app_accounts
alias = "child"
profile = each.value.role
}
here is the main resouce block where i want to multiple child accounts against single master account, so i want to iterate through the loop
resource "aws_route53_vpc_association_authorization" "master" {
provider = aws.master
vpc_id = vpc_id
zone_id = zone_id
}
resource "aws_route53_zone_association" "child" {
provider = aws.child
vpc_id = vpc_id
zone_id = zone_id
}
any idea on how to achieve this, please? thanks in advance.
The typical way to achieve your goal in Terraform is to define a shared module representing the objects that should be present in a single account and then to call that module once for each account, passing a different provider configuration into each.
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
}
}
}
provider "aws" {
alias = "master"
# ...
}
provider "aws" {
alias = "example1"
profile = "example1"
}
module "example1" {
source = "./modules/account"
account = "53xxxx08"
app_vpc_id = "vpc-0fxxxxxfec8"
providers = {
aws = aws.example1
aws.master = aws.master
}
}
provider "aws" {
alias = "example2"
profile = "example2"
}
module "example2" {
source = "./modules/account"
account = "53xxxx08"
app_vpc_id = "vpc-0fxxxxxfec8"
providers = {
aws = aws.example2
aws.master = aws.master
}
}
The ./modules/account directory would then contain the resource blocks describing what should exist in each individual account. For example:
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
configuration_aliases = [ aws, aws.master ]
}
}
}
variable "account" {
type = string
}
variable "app_vpc_id" {
type = string
}
resource "aws_route53_zone" "example" {
# (omitting the provider argument will associate
# with the default provider configuration, which
# is different for each instance of this module)
# ...
}
resource "aws_route53_vpc_association_authorization" "master" {
provider = aws.master
vpc_id = var.app_vpc_id
zone_id = aws_route53_zone.example.id
}
resource "aws_route53_zone_association" "child" {
provider = aws.master
vpc_id = var.app_vpc_id
zone_id = aws_route53_zone.example.id
}
(I'm not sure if you actually intended var.app_vpc_id to be the VPC specified for those zone associations, but my goal here is only to show the general pattern, not to show a fully-working example.)
Using a shared module in this way allows to avoid repeating the definitions for each account separately, and keeps each account-specific setting specified in only one place (either in a provider "aws" block or in a module block).
There is no way to make this more dynamic within the Terraform language itself, but if you expect to be adding and removing accounts regularly and want to make it more systematic then you could use code generation for the root module to mechanically produce the provider and module block for each account, to ensure that they all remain consistent and that you can update them all together in case you need to change the interface of the shared module in a way that will affect all of the calls.

Using Terraform Provider in aws module

I am going through the terraform documentation, and it seems unclear to me. I'm quite new to Terraform so no doubt i'm misunderstanding something here:
https://developer.hashicorp.com/terraform/language/modules/develop/providers
Problem:
My terraform pipeline is returning the following warning:
│
│ on waf-cdn.tf line 9, in module "waf_cdn":
│ 9: aws = aws.useastone
│
│ Module module.waf_cdn does not declare a provider named aws.
│ If you wish to specify a provider configuration for the module, add an entry for aws in the required_providers block within the module.
My root module is calling a child waf module. I understand that i need to configure my provider within my root module. There are 2 files within my root module:
...terraform.tf...
terraform {
backend "s3" {}
required_providers {
aws = {
source = "hashicorp/aws"
version = ">= 4.33.0"
}
random = {
source = "hashicorp/random"
version = "3.1.0"
}
local = {
source = "hashicorp/local"
version = "2.1.0"
}
kubernetes = {
source = "hashicorp/kubernetes"
version = ">= 2.0.1"
}
}
}
...and providers.tf...
provider "aws" {
region = var.region
assume_role {
role_arn = "arn:aws:iam::${var.account_id}:role/${local.role_name}"
}
}
provider "aws" {
region = "us-east-1"
alias = "useastone"
assume_role {
role_arn = "arn:aws:iam::${var.account_id}:role/${local.role_name}"
}
}
provider "aws" {
region = var.region
alias = "master"
assume_role {
role_arn = replace(
"arn:aws:iam::${var.master_account_id}:role/${local.role_name}",
local.app_region,
"master"
)
}
}
When calling the child module, the SCOPE attribute of the waf needs to specify the region as us-east-1 for CLOUDFRONT as it is a global service in AWS. Therefore, i need to pass the useastone provider when calling the child waf module as seen below:
module "waf_cdn" {
source = "../modules/qa-aws-waf-common"
name = "${local.waf_prefix}-cdn"
logging_arn = aws_kinesis_firehose_delivery_stream.log_stream_cdn.arn
scope = "CLOUDFRONT"
tags = merge(module.tags.tags, { name = "${local.name_prefix}-qa-waf-cdn" })
providers = {
aws = aws.useastone
}
}
With this code i'm getting the error show above.
I'm banging my head against the documentation here so any help guys would be really appreciated.
Here's hoping, thanks!
As per the documentation you linked, here is the passage you are interested in [1]:
Additional provider configurations (those with the alias argument set) are never inherited automatically by child modules, and so must always be passed explicitly using the providers map.
Since that is the case, you need to define the provider(s) on the module level as well:
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = ">= 4.33.0"
configuration_aliases = [ aws.useastone ]
}
}
}
That would probably be an additional providers.tf file in ../modules/qa-aws-waf-common.
[1] https://developer.hashicorp.com/terraform/language/modules/develop/providers#passing-providers-explicitly

Cannot create Cloud Composer

I am getting the error below while provisioning Composer via terraform.
Error: Error waiting to create Environment: Error waiting to create Environment: Error waiting for Creating Environment: error while retrieving operation: Get "https://composer.googleapis.com/v1beta1/projects/aayush-terraform/locations/us-central1/operations/ee459492-abb0-4646-893e-09d112219d79?alt=json&prettyPrint=false": write tcp 10.227.112.165:63811->142.251.12.95:443: write: broken pipe. An initial environment was or is still being created, and clean up failed with error: Getting creation operation state failed while waiting for environment to finish creating, but environment seems to still be in 'CREATING' state. Wait for operation to finish and either manually delete environment or import "projects/aayush-terraform/locations/us-central1/environments/example-composer-env" into your state.
Below is the code snippet:
terraform {
required_providers {
google = {
source = "hashicorp/google"
version = "~>3.0"
}
}
}
variable "gcp_region" {
type = string
description = "Region to use for GCP provider"
default = "us-central1"
}
variable "gcp_project" {
type = string
description = "Project to use for this config"
default = "aayush-terraform"
}
provider "google" {
region = var.gcp_region
project = var.gcp_project
}
resource "google_service_account" "test" {
account_id = "composer-env-account"
display_name = "Test Service Account for Composer Environment"
}
resource "google_project_iam_member" "composer-worker" {
role = "roles/composer.worker"
member = "serviceAccount:${google_service_account.test.email}"
}
resource "google_compute_network" "test" {
name = "composer-test-network"
auto_create_subnetworks = false
}
resource "google_compute_subnetwork" "test" {
name = "composer-test-subnetwork"
ip_cidr_range = "10.2.0.0/16"
region = "us-central1"
network = google_compute_network.test.id
}
resource "google_composer_environment" "test" {
name = "example-composer-env"
region = "us-central1"
config {
node_count = 3
node_config {
zone = "us-central1-a"
machine_type = "n1-standard-1"
network = google_compute_network.test.id
subnetwork = google_compute_subnetwork.test.id
service_account = google_service_account.test.name
}
}
}
NOTE: Composer is getting created even after this error is being thrown and I am provisioning this composer via service account which has been given owner access.
I had the same problem and I solved it by giving the "composer.operations.get
" permission to the service account which is provisioning the Composer.
This permission is part of the Composer Administrator role.
To prevent future operations like updates or deletion through Terraform, I think it's better to use the role rather than a single permission.
Or if you want to make some least privileges work, you can first use the role, then removing permissions you think you won't need and test your terraform code.

Deploy multiple Cloudrun service with same dockerimage

There are 25+ Cloudrun services that use the same docker image(from GCR) but are configured with different variables. What is an easy and reliable method to deploy all the services with the latest container image from any kind of incoming events?
Currently using below CLI command to execute one by one manually. Is there any automated way to implement auto deployment for all the service one after another or in parallel.
gcloud run deploy SERVICE --image IMAGE_URL
Addn: Labels are been used to mark the 25 containers which have the same container images. Not required to build docker image everytime from source. The same image can be used.
In case Terraform is an option for you, you can automate all Cloud Run services deployment using either with the count or for_each meta-arguments:
count if you need the same service name with indexes
provider "google" {
project = "MY-PROJECT-ID"
}
resource "google_cloud_run_service" "default" {
count = 25
name = "MY-SERVICE-${count.index}"
location = "MY-REGION"
metadata {
annotations = {
"run.googleapis.com/client-name" = "terraform"
}
}
template {
spec {
containers {
image = "IMAGE_URL"
}
}
}
}
data "google_iam_policy" "noauth" {
binding {
role = "roles/run.invoker"
members = ["allUsers"]
}
}
resource "google_cloud_run_service_iam_policy" "noauth" {
for_each = google_cloud_run_service.default
location = each.value.location
project = each.value.project
service = each.value.name
policy_data = data.google_iam_policy.noauth.policy_data
}
where MY-PROJECT-ID and MY-REGION needs to be replaced with your project specific values.
for_each if you need different service names
provider "google" {
project = "MY-PROJECT-ID"
}
resource "google_cloud_run_service" "default" {
for_each = toset( ["Service 1", "Service 2", "Service 25"] )
name = each.key
location = "MY-REGION"
metadata {
annotations = {
"run.googleapis.com/client-name" = "terraform"
}
}
template {
spec {
containers {
image = "IMAGE_URL"
}
}
}
}
data "google_iam_policy" "noauth" {
binding {
role = "roles/run.invoker"
members = ["allUsers"]
}
}
resource "google_cloud_run_service_iam_policy" "noauth" {
for_each = google_cloud_run_service.default
location = each.value.location
project = each.value.project
service = each.value.name
policy_data = data.google_iam_policy.noauth.policy_data
}
where MY-PROJECT-ID and MY-REGION needs to be replaced with your project specific values as well.
You can refer to the official GCP Cloud Run documentation for further details on Terraform usage.

Creating endpoint in cloud run with Terraform and Google Cloud Platform

I'm research for a way to use Terraform with GCP provider to create cloud run endpoint. For starter I'm creating testing data a simple hello world. I have resource cloud run service configured and cloud endpoints resource configured with cloud endpoints depends_on cloud run. However, I'm trying to pass in the cloud run url as a service name to the cloud endpoints. File are constructed with best practice, with module > cloud run and cloud endpoints resource. However, the Terraform interpolation for passing the output of
service_name = "${google_cloud_run_service.default.status[0].url}"
Terraform throughs an Error: Invalid character. I've also tried module.folder.output.url.
I have the openapi_config.yml hardcoded in the TF config within
I'm wondering if it's possible to have to work. I research many post and some forum are outdated.
#Cloud Run
resource "google_cloud_run_service" "default" {
name = var.name
location = var.location
template {
spec {
containers {
image = "gcr.io/cloudrun/hello"
}
}
metadata {
annotations = {
"autoscaling.knative.dev/maxScale" = "1000"
"run.googleapis.com/cloudstorage" = "project_name:us-central1:${google_storage_bucket.storage-run.name}"
"run.googleapis.com/client-name" = "terraform"
}
}
}
traffic {
percent = 100
latest_revision = true
}
autogenerate_revision_name = true
}
output "url" {
value = "${google_cloud_run_service.default.status[0].url}"
}
data "google_iam_policy" "noauth" {
binding {
role = "roles/run.invoker"
members = [
"allUsers",
]
}
}
resource "google_cloud_run_service_iam_policy" "noauth" {
location = google_cloud_run_service.default.location
project = google_cloud_run_service.default.project
service = google_cloud_run_service.default.name
policy_data = data.google_iam_policy.noauth.policy_data
}
#CLOUD STORAGE
resource "google_storage_bucket" "storage-run" {
name = var.name
location = var.location
force_destroy = true
bucket_policy_only = true
}
data "template_file" "openapi_spec" {
template = file("${path.module}/openapi_spec.yml")
}
#CLOUD ENDPOINT SERVICE
resource "google_endpoints_service" "api-service" {
service_name = "api_name.endpoints.project_name.cloud.goog"
project = var.project
openapi_config = data.template_file.openapi_spec.rendered
}
ERROR: googleapi: Error 400: Service name 'CLOUD_RUN_ESP_NAME' provided in the config files doesn't match the service name 'api_name.endpoints.project_name.cloud.goog' provided in the request., badRequest
So I later discovered, that the service name must match the same as the host/cloud run esp service url without https:// in order for the cloud endpoint services to provisioner. Terraform docs states otherwise in the form of " $apiname.endpoints.$projectid.cloud.goog " terraform_cloud_endpoints and in GCP docs states that the cloud run ESP service must be the url without https:// > gateway-12345-uc.a.run.app
Getting Started with Endpoints for Cloud Run