Terraform AWS not accessing localstack - amazon-web-services

I'm having trouble getting a terraform AWS provider to talk to localstack. Whatever I try I just get the same error:
Error: error configuring Terraform AWS Provider: error validating provider credentials: error calling sts:GetCallerIdentity: InvalidClientTokenId: The security token included in the request is invalid.
status code: 403, request id: dc96c65d-84a7-4e64-947d-833195464538
This error suggest that the provider is making contact with a HTTP server but the credentials are being rejected (as per any 403). You might imagine the problem is that I'm feeding in the wrong credentials (through environment variables).
However the hostname local-aws exists in my /etc/hosts file, but blahblahblah does not. If I swap the endpoint to point to http://blahblahblah:4566 I still get the same 403. So I think the problem is that the provider isn't using my local endpoint. I can't work out why.
resource "aws_secretsmanager_secret_version" "foo" {
secret_id = aws_secretsmanager_secret.foo.id
secret_string = "bar"
}
resource "aws_secretsmanager_secret" "foo" {
name = "rabbitmq_battery_emulator"
}
provider "aws" {
region = "eu-west-2"
endpoints {
secretsmanager = "http://local-aws:4566"
}
}

Firstly check that localstack is configured to run sts. In docker-compose this was just the SERVICES environment variable:
services:
local-aws:
image: localstack/localstack
environment:
EDGE_PORT: 4566
SERVICES: secretsmanager, sts
Then make sure that you set the sts endpoint as well as the service you require:
provider "aws" {
region = "eu-west-2"
endpoints {
sts = "http://local-aws:4566"
secretsmanager = "http://local-aws:4566"
}
}

In addition to the SERVICES and sts endpoint config mentioned by #philip-couling, I also had to remove a terraform block from my main.tf:
#terraform {
# backend "s3" {
# bucket = "valid-bucket"
# key = "terraform/state/account/terraform.tfstate"
# region = "eu-west-1"
# }
# required_providers {
# local = {
# version = "~> 2.1"
# }
# }
#}

Related

How can I provide to terraform different aws credentials to s3 backend and ec2 with role_arn usage

I want to store terraform state files in s3 bucket in one aws account and deploy instance changes in another aws account with role_arn usage.
This is my configuration:
providers.tf
terraform {
backend "s3" {
bucket = "bucket"
key = "tf/terraform.tfstate"
encrypt = "false"
region = "us-east-1"
profile = "s3"
role_arn = "arn:aws:iam::1111111111111:role/s3-role"
dynamodb_table = "name"
}
}
provider "aws" {
profile = "ec2"
region = "eu-north-1"
assume_role {
role_arn = "arn:aws:iam::2222222222222:role/ec2-role"
}
}
~/.aws/credentials
[s3-def]
aws_access_key_id = aaaaaaaaaa
aws_secret_access_key = sssssssss
[ec2-def]
aws_access_key_id = aaaaaaa
aws_secret_access_key = sssss
[s3]
role_arn = arn:aws:iam::1111111111:role/s3-role
region = us-east-1
source_profile = s3-def
[ec2]
role_arn = arn:aws:iam::22222222222:role/ec2-role
region = eu-north-1
source_profile = ec2-def
And when I try terraform init -migrate-state I get:
2022-08-03T17:23:21.334+0300 [INFO] Terraform version: 1.2.5
2022-08-03T17:23:21.334+0300 [INFO] Go runtime version: go1.18.1
2022-08-03T17:23:21.334+0300 [INFO] CLI args: []string{"terraform", "init", "-migrate-state"}
2022-08-03T17:23:21.334+0300 [INFO] Loading CLI configuration from /
2022-08-03T17:23:21.335+0300 [INFO] CLI command args: []string{"init", "-migrate-state"}
Initializing the backend...
2022-08-03T17:23:21.337+0300 [WARN] backend config has changed since last init
Backend configuration changed!
Terraform has detected that the configuration specified for the backend
has changed. Terraform will now check for existing state in the backends.
2022-08-03T17:23:21.338+0300 [INFO] Attempting to use session-derived credentials
╷
│ Error: error configuring S3 Backend: no valid credential sources for S3 Backend found.
│
│ Please see https://www.terraform.io/docs/language/settings/backends/s3.html
│ for more information about providing credentials.
│
│ Error: NoCredentialProviders: no valid providers in chain. Deprecated.
I just dont understand what is this error and it even possible to provide two different set of credentials to s3 and ec2?

How to fix 403 error when applying Terraform?

I created a new EC2 instance on AWS.
I am trying to create a Terraform on the AWS server and I'm getting an error.
I don't have the AMI previously created, so I'm not sure if this is the issue.
I checked my keypair and ensured it is correct.
I also checked the API details and they are correct too. I'm using a college AWS App account where the API details are the same for all users. Not sure if that would be an issue.
This is the error I'm getting after running terraform apply:
Error: error configuring Terraform AWS Provider: error validating provider credentials: error calling sts:GetCallerIdentity: InvalidClientTokenId: The security token included in the request is invalid.
│ status code: 403, request id: be2bf9ee-3aa4-401a-bc8b-f15c8a1e63d0
│
│ with provider["registry.terraform.io/hashicorp/aws"],
│ on main.tf line 10, in provider "aws":
│ 10: provider "aws" {
My main.tf file:
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 3.27"
}
}
required_version = ">= 0.14.9"
}
provider "aws" {
profile = "default"
region = "eu-west-1"
}
resource "aws_instance" "app_server" {
ami = "ami-04505e74c0741db8d"
instance_type = "t2.micro"
key_name = "<JOEY'S_KEYPAIR>"
tags = {
Name = "joey_terraform"
}
}
Credentials:
AWS Access Key ID [****************LRMC]:
AWS Secret Access Key [****************6OO3]:
Default region name [eu-west-1]:
Default output format [None]:

Using Elasticsearch Terraform EC provider to deploy a cluster in AWS

I am looking to deploy ECE (Elastic Cloud Enterprise) in AWS with Terraform. Reading through the documentation, I'm still not clear how this model works.
In the provider below, what is the reason for the endpoint? Is terraform connecting to this endpoint with the specified username and password? And are these credentials are being provided with the ECE license?
Hence, I'm thinking that the ECE installation endpoint can't be private. But I need to provision this privately - probably won't be able to do it via Terraform. Any experience with this?
provider "ec" {
# ECE installation endpoint
endpoint = "https://my.ece-environment.corp"
# If the ECE installation has a self-signed certificate
# you must set insecure to true.
insecure = true
username = "my-username"
password = "my-password"
}
data "ec_stack" "latest" {
version_regex = "latest"
region = "us-east-1"
}
resource "ec_deployment" "example_minimal" {
# Optional name.
name = "my_example_deployment"
# Mandatory fields
region = "us-east-1"
version = data.ec_stack.latest.version
deployment_template_id = "aws-io-optimized-v2"
elasticsearch {}
}

How to create a secret in Google Cloud Secret Manager by Terraform?

This is official page: https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/secret_manager_secret
I created these files:
variables.tf
variable gcp_project {
type = string
}
main.tf
resource "google_secret_manager_secret" "my_password" {
provider = google-beta
secret_id = "my-password"
replication {
automatic = true
}
}
data "google_secret_manager_secret_version" "my_password_v1" {
provider = google-beta
project = var.gcp_project
secret = google_secret_manager_secret.my_password.secret_id
version = 1
}
outputs.tf
output my_password_version {
value = data.google_secret_manager_secret_version.my_password_v1.version
}
When apply it, got error:
Error: Error retrieving available secret manager secret versions: googleapi: Error 404: Secret Version [projects/2381824501/secrets/my-password/versions/1] not found.
So I created the secret by gcloud cli:
echo -n "my_secret_password" | gcloud secrets create "my-password" \
--data-file - \
--replication-policy "automatic"
Then apply terraform again, it said Error: project: required field is not set.
If use terraform to create a secret with a real value, how to do?
I found the following article that I consider to be useful on Managing Secret Manager with Terraform.
You have to:
Create the Setup
Create a file named versions.tf that define the version constraints.
Create a file named main.tf and configure the Google provider stanza:
This is the code for creating a Secret Manager secret named "my-secret" with an automatic replication policy:
resource "google_secret_manager_secret" "my-secret" {
provider = google-beta
secret_id = "my-secret"
replication {
automatic = true
}
depends_on = [google_project_service.secretmanager]
}
Following #marian.vladoi's answer, if you're having issues with cloud resource manager api, enable it like so:
resource "google_project_service" "cloudresourcemanager" {
service = "cloudresourcemanager.googleapis.com"
}
You can also enable the cloud resource manager api using this gcloud command in terminal:
gcloud services enable secretmanager.googleapis.com

Terraform import fails due to erroneous region although defined in the provider

I am using the following tf configuration:
variable "aws_profile" {
description = "The AWS profile to use for this account"
}
provider "aws" {
version = "~> 2"
region = "us-east-1"
profile = "${var.aws_profile}"
}
provider "aws" {
version = "~> 2"
region = "us-west-2"
alias = "us_west_2"
profile = "profile-us-west-2"
}
where
cat ~/.aws/credentials
[profile-us-west-2]
region=us-west-2
aws_access_key_id = ΧΧΧΧΧΧΧΧΧΧΧ
aws_secret_access_key = ΧΧΧΧΧΧΧΧΧΧΧΧΧ
and trying to import an existing S3 bucket
to the tf resource below
resource "aws_s3_bucket" "my_tf_bucket" {
provider = "aws.us_west_2"
bucket = "my_tf_bucket"
with the following command:
terraform import aws_s3_bucket.my_tf_bucket existing_bucket_name
which fails as follows:
Error importing AWS S3 bucket policy: AuthorizationHeaderMalformed: The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'us-west-2'
status code: 400, request id: 64242424244D21946, host id: bddw422424
Why isn't the provider alias working?
The problem was that (for some reason), terraform requires to explicitly pass the provider value when importing
terraform import --provider=aws.us_west_2 aws_s3_bucket.my_tf_bucket existing_bucket_name