error modifying Lambda Function configuration : ValidationException with Lambda and VPC - amazon-web-services

I am building a lambda in terraform using it's AWS module and my code is as below:
module "lambda_function" {
# * Lambda module configs
source = "terraform-aws-modules/lambda/aws"
version = "3.0.0"
# * Lambda Configs
function_name = "${var.function_name}-${var.env}"
description = "My Project"
handler = local.constants.lambda.HANDLER
runtime = local.constants.lambda.VERSION
memory_size = 128
cloudwatch_logs_retention_in_days = 14
source_path = "./function/"
timeout = local.constants.lambda.TIMEOUT
create_async_event_config = true
maximum_retry_attempts = local.constants.lambda.RETRIES_ATTEMPT
layers = [
data.aws_lambda_layer_version.layer_requests.arn
]
environment_variables = {
AWS_ACCOUNT = var.env
SLACK_HOOK_CHANNEL = var.SLACK_HOOK_CHANNEL
}
tags = {
Name = "${var.function_name}-${var.env}"
}
trusted_entities = local.constants.lambda.TRUSTED_ENTITIES
}
This code works fine and the lambda get's deployed. Now i need to put the lambda in the VPC. When i add the code below in the resource block, i get the error error modifying Lambda Function (lambda_name) configuration : ValidationException: │ status code: 400, request id: de2641f6-1125-4c83-87fa-3fe32dee7b06 │ │ with module.lambda_function.aws_lambda_function.this[0], │ on .terraform/modules/lambda_function/main.tf line 22, in resource "aws_lambda_function" "this": │ 22: resource "aws_lambda_function" "this" {
The code for the vpc is:
# * VPC configurations
vpc_subnet_ids = ["10.21.0.0/26", "10.21.0.64/26", "10.21.0.128/26"]
vpc_security_group_ids = ["sg-ffffffffff"] # Using a dummy value here
attach_network_policy = true
If i use the same values in the AWS console and deploy the lambda in the VPC, it works fine.
Can someone please help ?

You have to provide valid subnet ids, not CIDR ranges. So instead of
vpc_subnet_ids = ["10.21.0.0/26", "10.21.0.64/26", "10.21.0.128/26"]
it should be
vpc_subnet_ids = ["subnet-asfid1", "subnet-asfid2", "subnet-as4id1"]

Related

Terraform AWS - Unable to update Transfer Server with incomplete error message

I am trying to update a test AWS Transfer Server because I was unable to connect to it via SFTP
Now trying to use the FTP / FTPS protocols, I have used the same layout as the example here
This is the example in the docs
resource "aws_transfer_server" "example" {
endpoint_type = "VPC"
endpoint_details {
subnet_ids = [aws_subnet.example.id]
vpc_id = aws_vpc.example.id
}
protocols = ["FTP", "FTPS"]
certificate = aws_acm_certificate.example.arn
identity_provider_type = "API_GATEWAY"
url = "${aws_api_gateway_deployment.example.invoke_url}${aws_api_gateway_resource.example.path}"
}
And here is my code
resource "aws_transfer_server" "transfer_x3" {
tags = {
Name = "${var.app}-${var.env}-transfer-x3-server"
}
endpoint_type = "VPC"
endpoint_details {
vpc_id = data.aws_vpc.vpc_global.id
subnet_ids = [data.aws_subnet.vpc_subnet_pri_commande_a.id, data.aws_subnet.vpc_subnet_pri_commande_b.id]
}
protocols = ["FTP", "FTPS"]
certificate = var.certificate_arn
identity_provider_type = "API_GATEWAY"
url = "https://${aws_api_gateway_rest_api.Api.id}.execute-api.${var.region}.amazonaws.com/latest/servers/{serverId}/users/{username}/config"
invocation_role = data.aws_iam_role.terraform-commande.arn
}
And here is the error message
╷
│ Error: error creating Transfer Server: InvalidRequestException: Bad value in IdentityProviderDetails
│
│ with aws_transfer_server.transfer_x3,
│ on transfer-x3.tf line 1, in resource "aws_transfer_server" "transfer_x3":
│ 1: resource "aws_transfer_server" "transfer_x3" {
│
╵
My guess is, it doesn't like the value in the url parameter
I have tried using the same form as one provided in the example: url = "${aws_api_gateway_deployment.ApiDeployment.invoke_url}${aws_api_gateway_resource.ApiResourceServerIdUserUsernameConfig.path}", but encountered the same error message
I have tried ordering the parameters around if it was that, but I had the same error over and over when I use the command terraform apply
The commands terraform validate and terraform plan didn't show the error message at all
What value could the url parameter need? Or is there a parameter missing in my resource declaration?
As per the documentation (CloudFormation in this case) [1], the examples say the only thing needed is the invoke URL of the API Gateway:
.
.
.
"IdentityProviderDetails": {
"InvocationRole": "Invocation-Role-ARN",
"Url": "API_GATEWAY-Invocation-URL"
},
"IdentityProviderType": "API_GATEWAY",
.
.
.
Comparing that to the attributes provided by the API Gateway stage resource in terraform, the only thing that is needed is the invoke_url attribute [2].
[1] https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-transfer-server.html#aws-resource-transfer-server--examples
[2] https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/api_gateway_stage#invoke_url

Pass gcp service account json file

We are creating confluent kafka sink connector (https://registry.terraform.io/providers/confluentinc/confluent/latest/docs/resources/confluent_connector) using terraform.
resource "confluent_connector" "gcs-sink" {
for_each = { for topic in var.topics : "${topic.name} ${topic.tasks}" => topic }
environment {
id = var.env_id
}
kafka_cluster {
id = var.cluster_id
}
config_nonsensitive = {
"name" = "${each.value.name}-gcs-connector"
"connector.class" = "GcsSink"
"topics" = "${each.value.name}"
"kafka.auth.mode" = "SERVICE_ACCOUNT"
"kafka.service.account.id" = "${var.connector_sa}"
"gcs.bucket.name" = "${var.gcs_bucket_name}"
"input.data.format" = "AVRO"
"output.data.format" = "AVRO"
"time.interval" = "HOURLY"
"flush.size" = "1000"
"tasks.max" = "${each.value.tasks}"
"topics.dir" = "avro-hourly"
"path.format" = "'process_date'=YYYY-MM-dd/'hour'=HH"
"rotate.schedule.interval.ms" = "60000"
"gcs.credentials.config" = var.gcs_sa_json
}
}
We need to pass service json key file to gcs.credentials.config, so i'm placing the json file in gcp secret manager and reading it from secret manger on runtime and storing it in variable gcs_sa_json, but I'm running into the below issue
Error: error waiting for Connector "g-gg-prod-gcs-connector" to provision: connector "display_name"="g-gg-prod-gcs-connector" provisioning status is "FAILED": Unable to validate configuration. If an update was made to the configuration, this means that the configuration was invalid, and the connector continues to operate on a previous configuration that passed validation. Errors:
│ gcs.credentials.config: Unable to retrieve credentials
│ gcs.bucket.name: Unable to retrieve credentials
│ . You might need to remove Connector manually before retrying.
│
│ with module.sink_connector.confluent_connector.gcs-sink["g-gg-prod-topic 2"],
│ on ../../terraform-modules/confluent-kafka/sink-connector/main.tf line 1, in resource "confluent_connector" "gcs-sink":
│ 1: resource "confluent_connector" "gcs-sink" {
I used the jsondecode function as well, but in vain.

Dynamically run terraform block based on input variable value

The Problem:
AWS doesn't support enhanced monitoring for t3.small instances which is what we use for smaller deployments of RDS but does on larger instance sizes for RDS. We want to disable it in Terraform when the instance class is t3.
Looking at the terraform resource docs: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_instance seems like you don't specify interval and role when you don't want to enable enhanced monitoring.
I'm trying to dynamically execute the resource block based on what the monitoring interval is set to. Thus, when its set to 0 run the block without monitoring role arn and when its set to anything other than 0 run the block where it is set.
however I'm getting an error:
╷
│ Error: Missing newline after argument
│
│ On main.tf line 68: An argument definition must end with a newline.
╵
Error: Terraform exited with code 1.
Error: Process completed with exit code 1.
I was following the following stack post: How to conditional create resource in Terraform based on a string variable
but it doesn't seem to work with the above error.
Here is my terraform code:
resource "aws_rds_cluster_instance" "cluster_instances" {
count = var.monitoring_interval != "0" ? var.cluster_instance_count : 0
identifier = "${var.service}-${var.environment}-${count.index}"
cluster_identifier = aws_rds_cluster.default.id
instance_class = var.instance_class
engine = aws_rds_cluster.default.engine
monitoring_role_arn = var.monitoring_role
engine_version = aws_rds_cluster.default.engine_version
monitoring_interval = var.monitoring_interval
db_parameter_group_name = var.regional_instance_param_group_name
copy_tags_to_snapshot = true
publicly_accessible = false
db_subnet_group_name = var.regional_subnet_group_name
}
resource "aws_rds_cluster_instance" "cluster_instances" {
count = var.monitoring_interval = "0" ? var.cluster_instance_count : 0
identifier = "${var.service}-${var.environment}-${count.index}"
cluster_identifier = aws_rds_cluster.default.id
instance_class = var.instance_class
engine = aws_rds_cluster.default.engine
engine_version = aws_rds_cluster.default.engine_version
db_parameter_group_name = var.regional_instance_param_group_name
copy_tags_to_snapshot = true
publicly_accessible = false
db_subnet_group_name = var.regional_subnet_group_name
}
Line Reference:
Thanks for your help. Probably something small I'm just missing or misunderstood about terraform conditionals.
You're missing an = in your condition. Change it to this:
var.monitoring_interval == "0" ? var.cluster_instance_count : 0

How to describe a shared VPC and define its subnets in GCP via terraform

I have created two shared VPCs for my organization, one for prod and one for non-prod usage.
For the these shared VPCs I want to create a description and define subnets, but I cannot find the right entries in terraform for these elements.
I.e. Here is how I defined the resource block to specify the host project (and create a shared VPC) :
resource "google_compute_shared_vpc_host_project" "dev-shared-shared-vpc-host" {
provider = google.as_network_admin
project = google_project.dev-shared-vpc-host.project_id
}
Now when I try to create the subnet :
resource "google_compute_subnetwork" "dev-subnetwork" {
provider = google.as_network_admin
name = var.vpc_and_subnet_info.for_dev_env.subnetwork.name
ip_cidr_range = var.vpc_and_subnet_info.for_dev_env.subnetwork.ip_cidr_range
region = var.region
secondary_ip_range {
range_name = var.vpc_and_subnet_info.for_dev_env.subnetwork.secondary_ip_range.name
ip_cidr_range = var.vpc_and_subnet_info.for_dev_env.subnetwork.secondary_ip_range.ip_cidr_range
}
network = google_compute_shared_vpc_host_project.dev-shared-shared-vpc-host.id
project = google_project.dev-shared-vpc-host.id
}
I get an error like
╷
│ Error: Error creating Subnetwork: googleapi: Error 400: Invalid value for field 'resource.network': 'projects/projects/<redacted_project_id>/global/networks/<redacted_project_id>'. The URL is malformed., invalid
│
│ with google_compute_subnetwork.dev-subnetwork,
│ on networking.tf line 5, in resource "google_compute_subnetwork" "dev-subnetwork":
│ 5: resource "google_compute_subnetwork" "dev-subnetwork" {
│
Obviously the projects/projects/.. is messing up the network parameter, but in the documentation for google_compute_shared_vpc_host_project there isn't any other output other than id. And for the input arguments there is no description. However, when I try to manually create the shared VPC, I can enter a description, and create a subnet.
Mind you, the google_compute_network that creates regular VPCs is quite well documented and the subnet that I defined above works well with it.
EDIT :
Fixing the project argument into project = google_project.dev-shared-vpc-host.project_id instead of id removes the projects/projects/... network error, but gives this error instead :
╷
│ Error: Error creating Subnetwork: googleapi: Error 404: The resource 'projects/<redacted_project_id>/global/networks/<redacted_project_id>' was not found, notFound
│
│ with google_compute_subnetwork.dev-subnetwork,
│ on networking.tf line 5, in resource "google_compute_subnetwork" "dev-subnetwork":
│ 5: resource "google_compute_subnetwork" "dev-subnetwork" {
│
╵
I seem to have misunderstood the creation of a shared VPC via google_compute_shared_vpc_host_project, this does not create a vpc perse, but only designates a project as the host project, thus sharing a vpc that must exist beforehand.
Therefore I should have created a google_compute_network beforehand, here is the HCL necessary to achieve what I wanted in the question :
resource "google_compute_network" "dev-vpc-network" {
provider = google.as_network_admin
name = var.vpc_and_subnet_info.for_dev_env.vpc.name
auto_create_subnetworks = var.vpc_and_subnet_info.for_dev_env.vpc.auto_create_subnetworks
project = google_project.dev-shared-vpc-host.project_id
description = var.vpc_and_subnet_info.for_dev_env.vpc.description
}
resource "google_compute_shared_vpc_host_project" "dev-shared-shared-vpc-host" {
provider = google.as_network_admin
project = google_project.dev-shared-vpc-host.project_id
}
resource "google_compute_subnetwork" "dev-subnetwork" {
provider = google.as_network_admin
name = var.vpc_and_subnet_info.for_dev_env.subnetwork.name
ip_cidr_range = var.vpc_and_subnet_info.for_dev_env.subnetwork.ip_cidr_range
region = var.region
secondary_ip_range {
range_name = var.vpc_and_subnet_info.for_dev_env.subnetwork.secondary_ip_range.name
ip_cidr_range = var.vpc_and_subnet_info.for_dev_env.subnetwork.secondary_ip_range.ip_cidr_range
}
network = google_compute_network.dev-vpc-network.id
project = google_project.dev-shared-vpc-host.project_id
}
Of course, in these examples I used variables declared in .tfvars to fill in the name, and other arguments needed in the resource blocks.

Unable to create new EKS with terraform

I'm having problems creating a new EKS version 1.22 in a dev environment.
I'm using the module in Terraform registry, trimming some parts since it's only for testing purposes (we just want to test the version 1.22).
I'm using a VPC that was created for testing EKS's, and 2 public subnets and 2 private subnets.
This is my main.tf:
module "eks" {
source = "terraform-aws-modules/eks/aws"
version = "18.21.0"
cluster_name = "EKSv2-update-test"
cluster_version = "1.22"
cluster_endpoint_private_access = true
cluster_endpoint_public_access = true
cluster_addons = {
coredns = {
resolve_conflicts = "OVERWRITE"
}
kube-proxy = {}
vpc-cni = {
resolve_conflicts = "OVERWRITE"
}
}
vpc_id = "vpc-xxx" # eks-vpc
subnet_ids = ["subnet-priv-1-xxx", "subnet-priv-2-xxx", "subnet-pub-1-xxx", "subnet-pub-2-xxx"]
}
Terraform apply times out after 20 min (it just hangs on module.eks.aws_eks_addon.this["coredns"]: Still creating... [20m0s elapsed])
and this is the error
│ Error: unexpected EKS Add-On (EKSv2-update-test:coredns) state returned during creation: timeout while waiting for state to become 'ACTIVE' (last state: 'DEGRADED', timeout: 20m0s)
│ [WARNING] Running terraform apply again will remove the kubernetes add-on and attempt to create it again effectively purging previous add-on configuration
│
│ with module.eks.aws_eks_addon.this["coredns"],
│ on .terraform/modules/eks/main.tf line 305, in resource "aws_eks_addon" "this":
│ 305: resource "aws_eks_addon" "this" {
The EKS gets created, but this is clearly not the way to go.
Regarding coredns, what am I missing?
Thanks
a minimum of 2 cluster nodes are required for addon coredns to meet its requirements for its replica set