How to choose from multiple providers when running terraform apply? - amazon-web-services

Question

I have some terraform code which creates resources in AWS.
For production, we have a CICD pipeline that runs against the real AWS. For local development I use localstack. So there are two providers like this:
# AWS config
provider "aws" {
region = ...
}
# Localstack config
provider "aws" {
region = ...
access_key = "mock_access_key"
secret_key = "mock_secret_key"
skip_credentials_validation = true
endpoints {
s3 = "http://localhost:4566"
...
}
}
My current solution is to comment out the provider which isn't used. This isn't ideal because we need to remember to uncomment the localstack config when developing locally and uncomment the real aws config when pushing code, forgetting this can be a big issue.
I also tried creating an alias for the localstack config alias = localstack and a couple variables:
variable provider_id {
type = map
default = {
localdev = "aws.localstack"
prod = "aws"
}
}
variable aws_account {
type = string
default = ""
}
Then the resources can lookup the provider like this:
provider = lookup(var.provider_id, var.aws_account)
Then the idea is we can pass the variable aws_account=... when running terraform apply so the resources pick the right provider. However, this doesn't work and returns this error (I presume because it should look up aws not "aws"):
╷
│ Error: Invalid provider configuration reference
│
│ on s3.tf line 2, in resource "aws_s3_bucket" "bucket":
│ 2: provider = lookup(var.provider_id, var.aws_account)
│
│ The provider argument requires a provider type name, optionally followed by a period and then a configuration alias.
╵
And even if it did work it wouldn't be ideal because we'd have to remember to add the provider lookup to every resource block.
I was wondering what is the best way to work with multiple providers like this? Ideally a simple solution such as passing a variable or parameter when running terraform apply. I don't mind if it requires a refactor of the terraform (I only learnt this tool last week).

Answer 1

The typical way to deal with that is through terraform workspaces:
In particular, organizations commonly want to create a strong separation between multiple deployments of the same infrastructure serving different development stages (e.g. staging vs. production) or different internal teams.
So you could try having two workspaces, one for localstack and the second for real aws.

Answer 2

You can achieve this via Alias workflow - but require code changes to include alias in resources sections. Please refer this Doc:
https://www.terraform.io/docs/language/modules/develop/providers.html
Though as you mentioned that one config is for prod/root account and one for local development in this case recommended and tested approach is to create two workspaces - separating workspaces will give you a lot of advantages such as:
1- code is separate so no chances of messing with it accidentally
2- debugging will be much easier
3- you are free to experiment with your code without fear